2024-04-16T17:33:53.229 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T17:33:53.229 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_smanjara@teuthology --archive /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456 --description rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi} --name smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi -- /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/orig.config.yaml 2024-04-16T17:33:53.265 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:33:53.350 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456 branch: wip-shilpa-multisite-tests description: rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi} email: null first_in_suite: false job_id: '7658456' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-shilpa-multisite-tests ceph: conf: client: debug rgw: 20 debug rgw lifecycle: 20 debug rgw notification: 20 debug rgw sync: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= rgw curl low speed time: 300 rgw data log num shards: 4 rgw data sync poll interval: 5 rgw md log max shards: 4 rgw meta sync poll interval: 5 rgw sync data inject err probability: 0.1 rgw sync log trim interval: 0 rgw sync meta inject err probability: 0.1 rgw sync obj etag verify: true setgroup: ceph setuser: ceph client.0: rgw data notify interval msec: 0 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 rocksdb delete range threshold: 0 osd.0: osd_max_omap_entries_per_request: 10 osd.1: osd_max_omap_entries_per_request: 1000 osd.2: osd_max_omap_entries_per_request: 10000 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: b0c7b917267c471208541fdeb652f5e897d00086 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: b0c7b917267c471208541fdeb652f5e897d00086 rgw: compression type: random frontend: beast rgw-multisite: realm: is default: true name: test-realm zonegroups: - enabled_features: - resharding - notification_v2 endpoints: - c1.client.0 is_default: true is_master: true name: test-zonegroup zones: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zone1 - endpoints: - c2.client.0 is_default: true name: test-zone2 rgw-multisite-tests: args: - tests.py - -a - '!fails_on_rgw' workunit: branch: wip-shilpa-multisite-tests sha1: 9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b owner: scheduled_smanjara@teuthology priority: 49 repo: https://git.ceph.com/ceph-ci.git roles: - - c1.mon.a - c1.osd.0 - c1.osd.1 - c1.osd.2 - c1.client.0 - - c1.mgr.x - c1.osd.3 - c1.osd.4 - c1.osd.5 - c1.client.1 - - c2.mon.a - c2.osd.0 - c2.osd.1 - c2.osd.2 - c2.client.0 - - c2.mgr.x - c2.osd.3 - c2.osd.4 - c2.osd.5 - c2.client.1 sha1: b0c7b917267c471208541fdeb652f5e897d00086 sleep_before_teardown: 0 suite: rgw:multisite suite_branch: wip-shilpa-multisite-tests suite_path: /home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa suite_relpath: qa suite_repo: https://github.com/smanjara/ceph.git suite_sha1: 9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9vtxKrRme6/isUwEX2lhl8Ybr0T6XOkffrdu3KUaVOAXLP5Y37Tmkgxpc7kst406V1WRz+VK6xeEGL3E8XHEI= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6BfUv5py4V6x14q9+tGyc4AQrymyRKyAz6y6sP+C7i7TH6fqhkepIeAS1YXpmNgRSN28Z6rVAE8v6vXjqqcYQ= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN7+QuOyR1DYPk/GC5c9CS7zlOEbjs3gZpYoCAodd4KN3mbnad1OmBxBuc9eY060s6zeyO+Ul/4sy25LGzWxW6s= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUJ9KsHPOwr6bVTpKPWxuVw2M8nCWXz8Wb3iPZ0uBK78ORQ5r/LsER0oB/sFQ+EsO4buhZ2vscDOaYUzUJAiJM= tasks: - install: null - ceph: cluster: c1 - ceph: cluster: c2 - rgw: c1.client.0: port: 8000 c1.client.1: port: 8001 c2.client.0: port: 8000 c2.client.1: port: 8001 - rgw-multisite: null - rgw-multisite-tests: config: reconfigure_delay: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_17:28:09 tube: smithi user: smanjara verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T17:33:53.350 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa; will attempt to use it 2024-04-16T17:33:53.350 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks 2024-04-16T17:33:53.351 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T17:33:53.352 INFO:teuthology.task.internal:Checking packages... 2024-04-16T17:33:53.375 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b0c7b917267c471208541fdeb652f5e897d00086' 2024-04-16T17:33:53.375 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T17:33:53.375 INFO:teuthology.packaging:ref: None 2024-04-16T17:33:53.376 INFO:teuthology.packaging:tag: None 2024-04-16T17:33:53.376 INFO:teuthology.packaging:branch: wip-shilpa-multisite-tests 2024-04-16T17:33:53.376 INFO:teuthology.packaging:sha1: b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:33:53.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-shilpa-multisite-tests 2024-04-16T17:33:53.531 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:33:53.533 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T17:33:53.539 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T17:33:53.539 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T17:33:53.550 INFO:teuthology.task.internal:Saving configuration 2024-04-16T17:33:53.564 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T17:33:53.572 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:33:53.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.261805', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIL66Qtx/wwpu7ICKRicYEXdTazWsUjK4A290QE2WhXiE'} 2024-04-16T17:33:53.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.258949', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJdGLBeuTTpsSrW0r58OWoYCj4vh17vk/stWKy+XB6L+'} 2024-04-16T17:33:53.641 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.260354', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILiRab9vipSZC2rq2XFBY2lN9TuQk4uHPmNNL6f2j/B9'} 2024-04-16T17:33:53.664 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.257390', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHlJ++jfnn2pOdQ6UDJuCmVQ882dMN06iICch8JKPK8w'} 2024-04-16T17:33:53.664 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T17:33:53.670 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['c1.mon.a', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'] 2024-04-16T17:33:53.670 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['c1.mgr.x', 'c1.osd.3', 'c1.osd.4', 'c1.osd.5', 'c1.client.1'] 2024-04-16T17:33:53.670 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['c2.mon.a', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0'] 2024-04-16T17:33:53.670 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['c2.mgr.x', 'c2.osd.3', 'c2.osd.4', 'c2.osd.5', 'c2.client.1'] 2024-04-16T17:33:53.670 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T17:33:53.806 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3d4309d160>, signals=[15]) 2024-04-16T17:33:53.807 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T17:33:53.813 INFO:teuthology.task.internal:Opening connections... 2024-04-16T17:33:53.813 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T17:33:53.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:33:53.897 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T17:33:53.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:33:53.972 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T17:33:53.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:33:54.052 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T17:33:54.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:33:54.126 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T17:33:54.133 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-04-16T17:33:54.139 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-04-16T17:33:54.140 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Ubuntu" 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="22.04" 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_CODENAME=jammy 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:ID=ubuntu 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE=debian 2024-04-16T17:33:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-16T17:33:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-16T17:33:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-16T17:33:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-16T17:33:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:UBUNTU_CODENAME=jammy 2024-04-16T17:33:54.189 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-04-16T17:33:54.208 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-16T17:33:54.214 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-16T17:33:54.214 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Ubuntu" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="22.04" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_CODENAME=jammy 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:ID=ubuntu 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE=debian 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-16T17:33:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:UBUNTU_CODENAME=jammy 2024-04-16T17:33:54.262 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-16T17:33:54.283 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-16T17:33:54.288 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-16T17:33:54.289 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="22.04" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=jammy 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-16T17:33:54.337 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-16T17:33:54.338 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=jammy 2024-04-16T17:33:54.338 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-16T17:33:54.358 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-04-16T17:33:54.364 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-04-16T17:33:54.364 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-04-16T17:33:54.409 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-16T17:33:54.409 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Ubuntu" 2024-04-16T17:33:54.409 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="22.04" 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_CODENAME=jammy 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:ID=ubuntu 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE=debian 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-16T17:33:54.410 INFO:teuthology.orchestra.run.smithi188.stdout:UBUNTU_CODENAME=jammy 2024-04-16T17:33:54.410 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-04-16T17:33:54.433 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T17:33:54.442 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T17:33:54.512 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T17:33:54.512 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-04-16T17:33:54.514 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-16T17:33:54.517 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-04-16T17:33:54.519 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-04-16T17:33:54.524 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T17:33:54.530 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T17:33:54.530 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-04-16T17:33:54.561 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-16T17:33:54.564 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-04-16T17:33:54.566 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-04-16T17:33:54.654 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T17:33:54.756 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T17:33:54.770 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-16T17:33:54.770 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-16T17:33:54.770 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T17:33:54.770 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T17:33:54.771 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T17:33:54.771 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T17:33:54.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:33:54.776 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-04-16T17:33:54.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:33:54.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:33:54.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:33:54.778 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-04-16T17:33:54.778 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-16T17:33:54.778 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-16T17:33:54.821 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-56-generic 2024-04-16T17:33:54.821 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.15.0-56-generic 2024-04-16T17:33:54.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-04-16T17:33:54.825 INFO:teuthology.orchestra.run.smithi071.stdout:5.15.0-56-generic 2024-04-16T17:33:54.825 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.15.0-56-generic 2024-04-16T17:33:54.825 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get clean 2024-04-16T17:33:54.826 INFO:teuthology.orchestra.run.smithi188.stdout:5.15.0-56-generic 2024-04-16T17:33:54.827 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.15.0-56-generic 2024-04-16T17:33:54.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-get clean 2024-04-16T17:33:54.828 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-56-generic 2024-04-16T17:33:54.828 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.15.0-56-generic 2024-04-16T17:33:54.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-04-16T17:33:54.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-04-16T17:33:54.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-get update 2024-04-16T17:33:54.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2024-04-16T17:33:54.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-16T17:33:55.124 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:33:55.128 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:33:55.133 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:33:55.133 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-16T17:33:55.133 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-16T17:33:55.144 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:33:55.148 INFO:teuthology.orchestra.run.smithi188.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-16T17:33:55.148 INFO:teuthology.orchestra.run.smithi188.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-16T17:33:55.150 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-16T17:33:55.151 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-16T17:33:55.221 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-16T17:33:55.229 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-16T17:33:55.231 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-16T17:33:55.234 INFO:teuthology.orchestra.run.smithi188.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-16T17:33:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-16T17:33:55.234 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-16T17:33:55.346 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-16T17:33:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-16T17:33:55.370 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-16T17:33:55.392 INFO:teuthology.orchestra.run.smithi188.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-16T17:33:55.439 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-16T17:33:55.442 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-16T17:33:55.460 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-16T17:33:55.468 INFO:teuthology.orchestra.run.smithi188.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-16T17:33:55.474 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-16T17:33:55.483 INFO:teuthology.orchestra.run.smithi188.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-16T17:33:55.486 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-16T17:33:55.493 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-16T17:33:55.493 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-16T17:33:55.494 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-16T17:33:55.507 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-16T17:33:55.508 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-16T17:33:55.509 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-16T17:33:55.547 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-16T17:33:55.547 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-16T17:33:55.563 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-16T17:33:55.615 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:55.615 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-16T17:33:55.623 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-16T17:33:55.629 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:55.629 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-16T17:33:55.635 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-16T17:33:55.636 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-16T17:33:55.640 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-16T17:33:55.641 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-16T17:33:55.644 INFO:teuthology.orchestra.run.smithi188.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-16T17:33:55.645 INFO:teuthology.orchestra.run.smithi188.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-16T17:33:55.648 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-16T17:33:55.653 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-16T17:33:55.653 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-16T17:33:55.655 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-16T17:33:55.656 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-16T17:33:55.657 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-16T17:33:55.657 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-16T17:33:55.673 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-16T17:33:55.674 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-16T17:33:55.674 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-16T17:33:55.675 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-16T17:33:55.675 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-16T17:33:55.677 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-16T17:33:55.677 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-16T17:33:55.677 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-16T17:33:55.687 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-16T17:33:55.687 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-16T17:33:55.687 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-16T17:33:55.687 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-16T17:33:55.691 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:55.691 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-16T17:33:55.698 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-16T17:33:55.698 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-16T17:33:55.699 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-16T17:33:55.708 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-16T17:33:55.709 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-16T17:33:55.711 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-16T17:33:55.711 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-16T17:33:55.735 INFO:teuthology.orchestra.run.smithi188.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-16T17:33:55.736 INFO:teuthology.orchestra.run.smithi188.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-16T17:33:55.736 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-16T17:33:55.737 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-16T17:33:55.738 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-16T17:33:55.738 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-16T17:33:55.743 INFO:teuthology.orchestra.run.smithi188.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:55.743 INFO:teuthology.orchestra.run.smithi188.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-16T17:33:55.750 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-16T17:33:55.750 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-16T17:33:55.750 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-16T17:33:55.750 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-16T17:33:55.751 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-16T17:33:55.766 INFO:teuthology.orchestra.run.smithi188.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-16T17:33:55.779 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-16T17:33:55.786 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-16T17:33:55.786 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-16T17:33:55.787 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-16T17:33:55.789 INFO:teuthology.orchestra.run.smithi188.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-16T17:33:55.794 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-16T17:33:55.795 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-16T17:33:55.797 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-16T17:33:55.798 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-16T17:33:55.798 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-16T17:33:55.808 INFO:teuthology.orchestra.run.smithi188.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-16T17:33:55.808 INFO:teuthology.orchestra.run.smithi188.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-16T17:33:55.809 INFO:teuthology.orchestra.run.smithi188.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-16T17:33:55.819 INFO:teuthology.orchestra.run.smithi188.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-16T17:33:55.819 INFO:teuthology.orchestra.run.smithi188.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-16T17:33:55.838 INFO:teuthology.orchestra.run.smithi188.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-16T17:33:55.840 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-16T17:33:55.841 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-16T17:33:55.847 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:55.848 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-16T17:33:55.863 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-16T17:33:55.883 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-16T17:33:55.887 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-16T17:33:55.887 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-16T17:33:55.888 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-16T17:33:55.889 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-16T17:33:55.889 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-16T17:33:55.902 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-16T17:33:55.903 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-16T17:33:55.904 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-16T17:33:55.924 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-16T17:33:55.924 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-16T17:33:55.924 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-16T17:33:55.945 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-16T17:33:55.945 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-16T17:33:55.946 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-16T17:33:55.954 INFO:teuthology.orchestra.run.smithi188.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-16T17:33:55.987 INFO:teuthology.orchestra.run.smithi188.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-16T17:33:55.990 INFO:teuthology.orchestra.run.smithi188.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-16T17:33:56.060 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-16T17:33:56.084 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-16T17:33:56.095 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-16T17:33:56.095 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-16T17:33:56.097 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-16T17:33:56.115 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-16T17:33:56.127 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-16T17:33:56.128 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-16T17:33:56.130 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-16T17:33:56.188 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-16T17:33:56.199 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:56.199 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-16T17:33:56.216 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-16T17:33:56.224 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:56.224 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-16T17:33:56.230 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-16T17:33:56.238 INFO:teuthology.orchestra.run.smithi188.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-16T17:33:56.239 INFO:teuthology.orchestra.run.smithi188.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-16T17:33:56.245 INFO:teuthology.orchestra.run.smithi188.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-16T17:33:56.245 INFO:teuthology.orchestra.run.smithi188.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-16T17:33:56.247 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-16T17:33:56.303 INFO:teuthology.orchestra.run.smithi188.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-16T17:33:56.386 INFO:teuthology.orchestra.run.smithi188.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-16T17:33:56.404 INFO:teuthology.orchestra.run.smithi188.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-16T17:33:56.406 INFO:teuthology.orchestra.run.smithi188.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-16T17:33:56.407 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-16T17:33:56.407 INFO:teuthology.orchestra.run.smithi188.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-16T17:33:56.410 INFO:teuthology.orchestra.run.smithi188.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-16T17:33:56.411 INFO:teuthology.orchestra.run.smithi188.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-16T17:33:56.411 INFO:teuthology.orchestra.run.smithi188.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-16T17:33:56.416 INFO:teuthology.orchestra.run.smithi188.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-16T17:33:56.421 INFO:teuthology.orchestra.run.smithi188.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-16T17:33:56.421 INFO:teuthology.orchestra.run.smithi188.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-16T17:33:56.425 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-16T17:33:56.432 INFO:teuthology.orchestra.run.smithi188.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-16T17:33:56.432 INFO:teuthology.orchestra.run.smithi188.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-16T17:33:56.434 INFO:teuthology.orchestra.run.smithi188.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-16T17:33:56.451 INFO:teuthology.orchestra.run.smithi188.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-16T17:33:56.806 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-16T17:33:56.812 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-16T17:33:56.812 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-16T17:33:56.824 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-16T17:33:56.826 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-16T17:33:56.828 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-16T17:33:56.830 INFO:teuthology.orchestra.run.smithi071.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-16T17:33:56.830 INFO:teuthology.orchestra.run.smithi071.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-16T17:33:56.845 INFO:teuthology.orchestra.run.smithi071.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-16T17:33:56.847 INFO:teuthology.orchestra.run.smithi071.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-16T17:34:01.097 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 12.9 MB in 2s (6,472 kB/s) 2024-04-16T17:34:01.141 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 12.9 MB in 2s (6,852 kB/s) 2024-04-16T17:34:01.242 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 12.9 MB in 2s (6,987 kB/s) 2024-04-16T17:34:01.273 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 12.9 MB in 2s (7,022 kB/s) 2024-04-16T17:34:02.371 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2024-04-16T17:34:02.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:02.448 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2024-04-16T17:34:02.474 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-04-16T17:34:02.491 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:02.550 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-04-16T17:34:02.557 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-16T17:34:02.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:02.637 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-16T17:34:02.638 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-04-16T17:34:02.645 INFO:teuthology.orchestra.run.smithi188.stdout:Building dependency tree... 2024-04-16T17:34:02.645 INFO:teuthology.orchestra.run.smithi188.stdout:Reading state information... 2024-04-16T17:34:02.655 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:02.713 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-04-16T17:34:02.747 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-04-16T17:34:02.747 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-04-16T17:34:02.818 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:02.818 INFO:teuthology.orchestra.run.smithi188.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:02.818 INFO:teuthology.orchestra.run.smithi188.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:02.818 INFO:teuthology.orchestra.run.smithi188.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:02.819 INFO:teuthology.orchestra.run.smithi188.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-16T17:34:02.819 INFO:teuthology.orchestra.run.smithi188.stdout: python2.7-minimal 2024-04-16T17:34:02.819 INFO:teuthology.orchestra.run.smithi188.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:The following additional packages will be installed: 2024-04-16T17:34:02.820 INFO:teuthology.orchestra.run.smithi188.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:02.820 INFO:teuthology.orchestra.run.smithi188.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-16T17:34:02.820 INFO:teuthology.orchestra.run.smithi188.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:02.822 INFO:teuthology.orchestra.run.smithi188.stdout:Suggested packages: 2024-04-16T17:34:02.822 INFO:teuthology.orchestra.run.smithi188.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-16T17:34:02.822 INFO:teuthology.orchestra.run.smithi188.stdout:Recommended packages: 2024-04-16T17:34:02.822 INFO:teuthology.orchestra.run.smithi188.stdout: thermald 2024-04-16T17:34:02.834 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-16T17:34:02.835 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-16T17:34:02.865 INFO:teuthology.orchestra.run.smithi188.stdout:The following NEW packages will be installed: 2024-04-16T17:34:02.865 INFO:teuthology.orchestra.run.smithi188.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:02.865 INFO:teuthology.orchestra.run.smithi188.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-16T17:34:02.865 INFO:teuthology.orchestra.run.smithi188.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:02.867 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages will be upgraded: 2024-04-16T17:34:02.867 INFO:teuthology.orchestra.run.smithi188.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-16T17:34:02.910 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2024-04-16T17:34:02.911 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2024-04-16T17:34:02.921 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:02.921 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:02.921 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:02.922 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:02.922 INFO:teuthology.orchestra.run.smithi122.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-16T17:34:02.922 INFO:teuthology.orchestra.run.smithi122.stdout: python2.7-minimal 2024-04-16T17:34:02.922 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:02.923 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-04-16T17:34:02.923 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:02.923 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-16T17:34:02.924 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:02.924 INFO:teuthology.orchestra.run.smithi188.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:02.924 INFO:teuthology.orchestra.run.smithi188.stdout:Need to get 113 MB of archives. 2024-04-16T17:34:02.924 INFO:teuthology.orchestra.run.smithi188.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-16T17:34:02.924 INFO:teuthology.orchestra.run.smithi188.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-16T17:34:02.925 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-04-16T17:34:02.925 INFO:teuthology.orchestra.run.smithi122.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-16T17:34:02.925 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-04-16T17:34:02.925 INFO:teuthology.orchestra.run.smithi122.stdout: thermald 2024-04-16T17:34:02.977 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-04-16T17:34:02.977 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:02.977 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-16T17:34:02.977 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:02.979 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-04-16T17:34:02.979 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-16T17:34:03.003 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:03.003 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-16T17:34:03.004 INFO:teuthology.orchestra.run.smithi083.stdout: python2.7-minimal 2024-04-16T17:34:03.004 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:03.005 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-16T17:34:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-16T17:34:03.005 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:03.007 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-16T17:34:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-16T17:34:03.007 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-04-16T17:34:03.007 INFO:teuthology.orchestra.run.smithi083.stdout: thermald 2024-04-16T17:34:03.055 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-16T17:34:03.055 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:03.055 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-16T17:34:03.055 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:03.056 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-04-16T17:34:03.056 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-16T17:34:03.082 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:03.082 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:03.082 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:03.083 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:03.083 INFO:teuthology.orchestra.run.smithi071.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-16T17:34:03.083 INFO:teuthology.orchestra.run.smithi071.stdout: python2.7-minimal 2024-04-16T17:34:03.083 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:03.084 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2024-04-16T17:34:03.084 INFO:teuthology.orchestra.run.smithi071.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:03.084 INFO:teuthology.orchestra.run.smithi071.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-16T17:34:03.085 INFO:teuthology.orchestra.run.smithi071.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:03.086 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2024-04-16T17:34:03.086 INFO:teuthology.orchestra.run.smithi071.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-16T17:34:03.086 INFO:teuthology.orchestra.run.smithi071.stdout:Recommended packages: 2024-04-16T17:34:03.086 INFO:teuthology.orchestra.run.smithi071.stdout: thermald 2024-04-16T17:34:03.122 INFO:teuthology.orchestra.run.smithi083.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:03.123 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 113 MB of archives. 2024-04-16T17:34:03.123 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-16T17:34:03.123 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-16T17:34:03.134 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2024-04-16T17:34:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-16T17:34:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-16T17:34:03.134 INFO:teuthology.orchestra.run.smithi071.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-16T17:34:03.136 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages will be upgraded: 2024-04-16T17:34:03.136 INFO:teuthology.orchestra.run.smithi071.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-16T17:34:03.159 INFO:teuthology.orchestra.run.smithi122.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:03.159 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 113 MB of archives. 2024-04-16T17:34:03.159 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-16T17:34:03.159 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-16T17:34:03.214 INFO:teuthology.orchestra.run.smithi071.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:03.214 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 113 MB of archives. 2024-04-16T17:34:03.214 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-16T17:34:03.214 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-16T17:34:03.489 INFO:teuthology.orchestra.run.smithi188.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-16T17:34:03.832 INFO:teuthology.orchestra.run.smithi188.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-16T17:34:03.962 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-16T17:34:04.267 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-16T17:34:04.362 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-16T17:34:04.607 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-16T17:34:04.887 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-16T17:34:04.961 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-16T17:34:06.006 INFO:teuthology.orchestra.run.smithi188.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-16T17:34:06.006 INFO:teuthology.orchestra.run.smithi188.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-16T17:34:06.006 INFO:teuthology.orchestra.run.smithi188.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-16T17:34:06.254 INFO:teuthology.orchestra.run.smithi188.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-16T17:34:06.320 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-16T17:34:06.438 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-16T17:34:06.438 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-16T17:34:06.439 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-16T17:34:06.610 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-16T17:34:06.650 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-16T17:34:06.707 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 113 MB in 3s (32.9 MB/s) 2024-04-16T17:34:06.756 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-16T17:34:06.756 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-16T17:34:06.756 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-16T17:34:06.878 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-16T17:34:06.972 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-16T17:34:07.004 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-16T17:34:07.049 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 113 MB in 4s (31.6 MB/s) 2024-04-16T17:34:07.222 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-16T17:34:07.265 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-16T17:34:07.265 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-16T17:34:07.266 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-16T17:34:07.412 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 113 MB in 4s (29.4 MB/s) 2024-04-16T17:34:07.606 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-16T17:34:08.277 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-16T17:34:08.382 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-16T17:34:08.775 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 113 MB in 5s (21.0 MB/s) 2024-04-16T17:34:08.862 INFO:teuthology.orchestra.run.smithi188.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-16T17:34:08.867 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:08.951 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:08.952 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-16T17:34:09.215 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-16T17:34:09.220 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:09.303 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:09.633 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-16T17:34:09.638 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:09.761 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:10.955 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-16T17:34:10.959 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:11.075 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:11.369 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-16T17:34:11.391 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:11.661 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:11.904 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-16T17:34:11.926 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:12.010 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:12.156 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-16T17:34:12.176 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:12.214 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:12.412 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-16T17:34:12.416 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-16T17:34:12.434 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:12.437 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:12.474 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:12.537 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:13.024 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-16T17:34:13.044 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:13.082 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:13.176 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-16T17:34:13.198 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:13.286 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:13.746 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-16T17:34:13.767 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:13.804 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:18.577 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:18.693 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:18.753 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:18.854 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:19.026 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:19.130 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:19.187 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:19.316 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:19.417 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-16T17:34:19.439 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-16T17:34:19.483 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:19.635 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-16T17:34:19.657 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-16T17:34:19.701 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:19.714 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:19.866 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:20.199 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:20.311 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:20.533 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:20.651 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-16T17:34:20.658 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:20.673 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-16T17:34:20.714 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:20.966 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:21.095 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:21.539 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-16T17:34:21.550 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-16T17:34:21.589 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:24.754 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-16T17:34:24.791 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:24.820 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:25.514 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-16T17:34:25.551 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:25.588 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:26.754 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-16T17:34:26.791 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:26.828 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:26.895 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-16T17:34:26.932 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-16T17:34:26.970 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:27.999 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:28.116 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:28.218 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:28.319 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:28.441 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:28.563 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:28.619 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:28.689 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-16T17:34:28.740 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:28.817 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:28.866 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-16T17:34:29.011 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:29.958 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:30.035 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-16T17:34:30.081 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:30.128 INFO:teuthology.orchestra.run.smithi188.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:30.128 INFO:teuthology.orchestra.run.smithi188.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:30.128 INFO:teuthology.orchestra.run.smithi188.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:30.128 INFO:teuthology.orchestra.run.smithi188.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:30.163 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-16T17:34:30.295 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:30.330 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:30.330 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:30.330 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:30.331 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:30.414 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:30.489 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:30.519 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:31.188 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-16T17:34:31.197 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-16T17:34:31.501 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:31.558 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:31.627 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-16T17:34:31.680 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:31.797 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:32.332 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:32.876 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:32.876 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:32.876 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:32.876 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:33.027 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:33.081 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-16T17:34:33.083 INFO:teuthology.orchestra.run.smithi071.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:33.083 INFO:teuthology.orchestra.run.smithi071.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:33.083 INFO:teuthology.orchestra.run.smithi071.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:33.083 INFO:teuthology.orchestra.run.smithi071.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:33.207 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-16T17:34:33.266 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:33.333 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:33.399 INFO:teuthology.orchestra.run.smithi188.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-16T17:34:33.399 INFO:teuthology.orchestra.run.smithi188.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:33.742 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-16T17:34:33.868 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-16T17:34:33.969 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:34.024 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-16T17:34:34.024 INFO:teuthology.orchestra.run.smithi083.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:34.420 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:34.675 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:35.796 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-16T17:34:35.922 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-16T17:34:36.048 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:36.110 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-16T17:34:36.115 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-16T17:34:36.115 INFO:teuthology.orchestra.run.smithi122.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:36.227 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-16T17:34:36.796 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-16T17:34:37.251 INFO:teuthology.orchestra.run.smithi071.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-16T17:34:37.251 INFO:teuthology.orchestra.run.smithi071.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:47.816 INFO:teuthology.orchestra.run.smithi188.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-16T17:34:47.816 INFO:teuthology.orchestra.run.smithi188.stdout:Sourcing file `/etc/default/grub' 2024-04-16T17:34:47.833 INFO:teuthology.orchestra.run.smithi188.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:47.853 INFO:teuthology.orchestra.run.smithi188.stdout:Generating grub configuration file ... 2024-04-16T17:34:48.167 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-16T17:34:48.167 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub' 2024-04-16T17:34:48.187 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:48.202 INFO:teuthology.orchestra.run.smithi188.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:48.208 INFO:teuthology.orchestra.run.smithi083.stdout:Generating grub configuration file ... 2024-04-16T17:34:48.222 INFO:teuthology.orchestra.run.smithi188.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:48.480 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:48.503 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:48.584 INFO:teuthology.orchestra.run.smithi188.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:48.590 INFO:teuthology.orchestra.run.smithi188.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:48.660 INFO:teuthology.orchestra.run.smithi188.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:48.665 INFO:teuthology.orchestra.run.smithi188.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:48.787 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:48.792 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:48.834 INFO:teuthology.orchestra.run.smithi188.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:48.834 INFO:teuthology.orchestra.run.smithi188.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:48.834 INFO:teuthology.orchestra.run.smithi188.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:48.863 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:48.867 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:48.870 INFO:teuthology.orchestra.run.smithi188.stdout:done 2024-04-16T17:34:49.015 INFO:teuthology.orchestra.run.smithi083.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:49.015 INFO:teuthology.orchestra.run.smithi083.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:49.015 INFO:teuthology.orchestra.run.smithi083.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:49.063 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-04-16T17:34:49.232 DEBUG:teuthology.orchestra.run.smithi188:> dpkg -s linux-image-generic 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Package: linux-image-generic 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Status: install ok installed 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Priority: optional 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Section: kernel 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Installed-Size: 21 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: amd64 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Source: linux-meta 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:49.264 INFO:teuthology.orchestra.run.smithi188.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:49.265 INFO:teuthology.orchestra.run.smithi188.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:49.265 INFO:teuthology.orchestra.run.smithi188.stdout:Recommends: thermald 2024-04-16T17:34:49.265 INFO:teuthology.orchestra.run.smithi188.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:49.265 INFO:teuthology.orchestra.run.smithi188.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:49.265 INFO:teuthology.orchestra.run.smithi188.stdout: available. 2024-04-16T17:34:49.265 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-16T17:34:49.265 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T17:34:49.265 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T17:34:49.265 INFO:teuthology.task.kernel:Installing distro kernel on c2.mgr.x... 2024-04-16T17:34:49.265 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T17:34:49.266 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-get clean 2024-04-16T17:34:49.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-get update 2024-04-16T17:34:49.574 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:49.602 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:49.602 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-04-16T17:34:49.602 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:49.602 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:49.602 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-04-16T17:34:49.602 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-16T17:34:49.602 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T17:34:49.602 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T17:34:49.602 INFO:teuthology.task.kernel:Installing distro kernel on c1.mgr.x... 2024-04-16T17:34:49.602 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T17:34:49.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-04-16T17:34:49.603 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:34:49.603 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:34:49.613 INFO:teuthology.orchestra.run.smithi188.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-16T17:34:49.704 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:34:49.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-16T17:34:49.905 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:34:49.905 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:34:49.918 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-16T17:34:49.939 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:34:50.000 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 119 kB in 0s (321 kB/s) 2024-04-16T17:34:50.237 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-16T17:34:50.237 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub' 2024-04-16T17:34:50.248 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:50.275 INFO:teuthology.orchestra.run.smithi122.stdout:Generating grub configuration file ... 2024-04-16T17:34:50.549 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:50.570 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:50.874 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:50.879 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:50.925 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2024-04-16T17:34:50.942 DEBUG:teuthology.orchestra.run.smithi188:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:50.946 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:50.952 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:51.003 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2024-04-16T17:34:51.100 INFO:teuthology.orchestra.run.smithi122.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:51.100 INFO:teuthology.orchestra.run.smithi122.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:51.100 INFO:teuthology.orchestra.run.smithi122.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:51.133 INFO:teuthology.orchestra.run.smithi122.stdout:done 2024-04-16T17:34:51.200 INFO:teuthology.orchestra.run.smithi188.stdout:Building dependency tree... 2024-04-16T17:34:51.201 INFO:teuthology.orchestra.run.smithi188.stdout:Reading state information... 2024-04-16T17:34:51.209 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-16T17:34:51.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:51.277 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-16T17:34:51.374 INFO:teuthology.orchestra.run.smithi188.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-16T17:34:51.375 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:51.375 INFO:teuthology.orchestra.run.smithi188.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:34:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:34:51.376 INFO:teuthology.orchestra.run.smithi188.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:51.418 INFO:teuthology.orchestra.run.smithi188.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:51.420 DEBUG:teuthology.orchestra.run.smithi188:> dpkg -s linux-image-generic 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Package: linux-image-generic 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Status: install ok installed 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Priority: optional 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Section: kernel 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Installed-Size: 21 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: amd64 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Source: linux-meta 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:51.436 INFO:teuthology.orchestra.run.smithi188.stdout:Recommends: thermald 2024-04-16T17:34:51.437 INFO:teuthology.orchestra.run.smithi188.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:51.437 INFO:teuthology.orchestra.run.smithi188.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:51.437 INFO:teuthology.orchestra.run.smithi188.stdout: available. 2024-04-16T17:34:51.437 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-04-16T17:34:51.452 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-16T17:34:51.452 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-16T17:34:51.481 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.KYykxVKNcX 2024-04-16T17:34:51.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub/grub.cfg /tmp/tmp.KYykxVKNcX 2024-04-16T17:34:51.545 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.KYykxVKNcX 2024-04-16T17:34:51.574 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-04-16T17:34:51.613 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-04-16T17:34:51.613 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-04-16T17:34:51.613 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-04-16T17:34:51.613 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-04-16T17:34:51.613 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi083.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:51.614 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:34:51.615 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:34:51.615 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:51.615 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-16T17:34:51.615 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T17:34:51.615 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T17:34:51.615 INFO:teuthology.task.kernel:Installing distro kernel on c2.mon.a... 2024-04-16T17:34:51.615 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T17:34:51.615 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-04-16T17:34:51.646 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:51.648 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-04-16T17:34:51.662 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.KYykxVKNcX is 10KB 2024-04-16T17:34:51.676 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.KYykxVKNcX 2024-04-16T17:34:51.681 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-04-16T17:34:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-04-16T17:34:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-04-16T17:34:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-04-16T17:34:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:51.707 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-04-16T17:34:51.707 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-16T17:34:51.737 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:34:51.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-16T17:34:51.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:51.752 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.aup0y0L4Qu 2024-04-16T17:34:51.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub/grub.cfg /tmp/tmp.aup0y0L4Qu 2024-04-16T17:34:51.787 INFO:teuthology.orchestra.run.smithi071.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-16T17:34:51.787 INFO:teuthology.orchestra.run.smithi071.stdout:Sourcing file `/etc/default/grub' 2024-04-16T17:34:51.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-04-16T17:34:51.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.aup0y0L4Qu 2024-04-16T17:34:51.815 INFO:teuthology.orchestra.run.smithi071.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:51.820 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-16T17:34:51.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo update-grub 2024-04-16T17:34:51.841 INFO:teuthology.orchestra.run.smithi071.stdout:Generating grub configuration file ... 2024-04-16T17:34:51.910 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:34:51.918 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.aup0y0L4Qu is 10KB 2024-04-16T17:34:51.930 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.aup0y0L4Qu 2024-04-16T17:34:51.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:51.987 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:34:51.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-16T17:34:51.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:52.023 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:34:52.051 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-16T17:34:52.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo update-grub 2024-04-16T17:34:52.104 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-16T17:34:52.114 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:52.153 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:52.466 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:52.466 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:34:52.471 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:52.539 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:52.545 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:52.652 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 119 kB in 1s (169 kB/s) 2024-04-16T17:34:52.717 INFO:teuthology.orchestra.run.smithi071.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:52.717 INFO:teuthology.orchestra.run.smithi071.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:52.717 INFO:teuthology.orchestra.run.smithi071.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:52.753 INFO:teuthology.orchestra.run.smithi071.stdout:done 2024-04-16T17:34:53.043 INFO:teuthology.orchestra.run.smithi188.stderr:Sourcing file `/etc/default/grub' 2024-04-16T17:34:53.044 INFO:teuthology.orchestra.run.smithi188.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:53.046 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-04-16T17:34:53.243 DEBUG:teuthology.orchestra.run.smithi071:> dpkg -s linux-image-generic 2024-04-16T17:34:53.264 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub' 2024-04-16T17:34:53.264 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:53.266 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-16T17:34:53.267 INFO:teuthology.orchestra.run.smithi071.stdout:Package: linux-image-generic 2024-04-16T17:34:53.267 INFO:teuthology.orchestra.run.smithi071.stdout:Status: install ok installed 2024-04-16T17:34:53.267 INFO:teuthology.orchestra.run.smithi071.stdout:Priority: optional 2024-04-16T17:34:53.267 INFO:teuthology.orchestra.run.smithi071.stdout:Section: kernel 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Installed-Size: 21 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: amd64 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Source: linux-meta 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Recommends: thermald 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:53.268 INFO:teuthology.orchestra.run.smithi071.stdout: available. 2024-04-16T17:34:53.268 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-16T17:34:53.269 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T17:34:53.269 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T17:34:53.269 INFO:teuthology.task.kernel:Installing distro kernel on c1.mon.a... 2024-04-16T17:34:53.269 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T17:34:53.269 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get clean 2024-04-16T17:34:53.271 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:53.276 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:53.451 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:53.458 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:53.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2024-04-16T17:34:53.540 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:53.545 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:53.572 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-04-16T17:34:53.587 INFO:teuthology.orchestra.run.smithi188.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:53.590 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:53.593 INFO:teuthology.orchestra.run.smithi188.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:53.622 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:34:53.622 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:34:53.633 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-16T17:34:53.649 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-04-16T17:34:53.654 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:34:53.688 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:53.693 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:53.705 INFO:teuthology.orchestra.run.smithi188.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:53.705 INFO:teuthology.orchestra.run.smithi188.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:53.705 INFO:teuthology.orchestra.run.smithi188.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:53.718 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-04-16T17:34:53.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-04-16T17:34:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:53.737 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:53.856 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-04-16T17:34:53.857 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-04-16T17:34:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:53.889 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-16T17:34:53.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-04-16T17:34:54.031 INFO:teuthology.orchestra.run.smithi122.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-16T17:34:54.031 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:54.031 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:54.031 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:54.031 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:34:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:34:54.032 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:54.073 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:54.074 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:54.088 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:54.089 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-04-16T17:34:54.089 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:54.089 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:54.089 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-04-16T17:34:54.089 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-04-16T17:34:54.140 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.M83i7LEqtk 2024-04-16T17:34:54.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub/grub.cfg /tmp/tmp.M83i7LEqtk 2024-04-16T17:34:54.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.M83i7LEqtk 2024-04-16T17:34:54.343 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.M83i7LEqtk is 10KB 2024-04-16T17:34:54.357 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.M83i7LEqtk 2024-04-16T17:34:54.362 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:54.415 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:34:54.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-16T17:34:54.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:54.490 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-16T17:34:54.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo update-grub 2024-04-16T17:34:55.060 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-04-16T17:34:55.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-16T17:34:55.137 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-04-16T17:34:55.336 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2024-04-16T17:34:55.337 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2024-04-16T17:34:55.505 INFO:teuthology.orchestra.run.smithi071.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-16T17:34:55.505 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:34:55.505 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:34:55.505 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:34:55.505 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:34:55.506 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:34:55.506 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:34:55.506 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:34:55.543 INFO:teuthology.orchestra.run.smithi071.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-16T17:34:55.545 DEBUG:teuthology.orchestra.run.smithi071:> dpkg -s linux-image-generic 2024-04-16T17:34:55.560 INFO:teuthology.orchestra.run.smithi071.stdout:Package: linux-image-generic 2024-04-16T17:34:55.560 INFO:teuthology.orchestra.run.smithi071.stdout:Status: install ok installed 2024-04-16T17:34:55.560 INFO:teuthology.orchestra.run.smithi071.stdout:Priority: optional 2024-04-16T17:34:55.560 INFO:teuthology.orchestra.run.smithi071.stdout:Section: kernel 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Installed-Size: 21 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Maintainer: Ubuntu Kernel Team 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: amd64 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Source: linux-meta 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Version: 5.15.0.102.99 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Recommends: thermald 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout:Description: Generic Linux kernel image 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout: This package will always depend on the latest generic kernel image 2024-04-16T17:34:55.561 INFO:teuthology.orchestra.run.smithi071.stdout: available. 2024-04-16T17:34:55.561 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-04-16T17:34:55.606 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.LI5fIKskAt 2024-04-16T17:34:55.607 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub/grub.cfg /tmp/tmp.LI5fIKskAt 2024-04-16T17:34:55.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.LI5fIKskAt 2024-04-16T17:34:55.692 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-04-16T17:34:55.693 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:55.695 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-04-16T17:34:55.784 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.LI5fIKskAt is 10KB 2024-04-16T17:34:55.796 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.LI5fIKskAt 2024-04-16T17:34:55.801 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:55.855 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:34:55.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-16T17:34:55.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-16T17:34:55.906 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:55.911 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:55.948 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-16T17:34:55.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo update-grub 2024-04-16T17:34:56.134 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:56.140 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:56.181 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:56.186 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:56.330 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:56.330 INFO:teuthology.orchestra.run.smithi122.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:56.330 INFO:teuthology.orchestra.run.smithi122.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-04-16T17:34:56.359 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-04-16T17:34:57.346 INFO:teuthology.orchestra.run.smithi071.stderr:Sourcing file `/etc/default/grub' 2024-04-16T17:34:57.348 INFO:teuthology.orchestra.run.smithi071.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-16T17:34:57.351 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-04-16T17:34:57.544 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-16T17:34:57.551 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-16T17:34:57.780 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-16T17:34:57.785 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-16T17:34:57.826 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-16T17:34:57.831 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-16T17:34:57.939 INFO:teuthology.orchestra.run.smithi071.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-16T17:34:57.939 INFO:teuthology.orchestra.run.smithi071.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-16T17:34:57.939 INFO:teuthology.orchestra.run.smithi071.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-16T17:34:57.958 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-04-16T17:34:57.975 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-04-16T17:35:23.759 INFO:teuthology.task.kernel:Checking client c2.mgr.x for new kernel version... 2024-04-16T17:35:23.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:35:23.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:23.901 INFO:teuthology.task.kernel:Checking client c1.mgr.x for new kernel version... 2024-04-16T17:35:23.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:35:23.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:26.364 INFO:teuthology.task.kernel:Checking client c2.mon.a for new kernel version... 2024-04-16T17:35:26.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:35:26.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:27.980 INFO:teuthology.task.kernel:Checking client c1.mon.a for new kernel version... 2024-04-16T17:35:27.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:35:27.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:42.111 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-04-16T17:35:42.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-16T17:35:44.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-16T17:35:46.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-04-16T17:35:51.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:35:51.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:51.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:35:51.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:53.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:35:53.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:54.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-04-16T17:35:54.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-16T17:35:55.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:35:55.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:35:56.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-16T17:35:58.530 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-04-16T17:36:06.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:36:06.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:06.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:36:06.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:08.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:36:08.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:10.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:36:10.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:24.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-04-16T17:36:24.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-16T17:36:27.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-16T17:36:28.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-04-16T17:36:39.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:36:39.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:39.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:36:39.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:39.950 ERROR:teuthology.orchestra.connection:Error authenticating with smithi188.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2024-04-16T17:36:40.176 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-16T17:36:41.493 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:36:41.494 INFO:teuthology.task.kernel:Checking kernel version of c1.mgr.x, want "5.15.0-102-generic"... 2024-04-16T17:36:41.494 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-16T17:36:41.499 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-102-generic 2024-04-16T17:36:41.499 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-16T17:36:41.499 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T17:36:41.500 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-16T17:36:42.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:36:42.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:42.501 INFO:teuthology.task.kernel:Enabling kdb on c1.mgr.x... 2024-04-16T17:36:42.501 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-16T17:36:42.613 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-16T17:36:42.752 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-04-16T17:36:42.777 DEBUG:teuthology.parallel:result is None 2024-04-16T17:36:43.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:36:43.278 INFO:teuthology.task.kernel:Checking kernel version of c2.mon.a, want "5.15.0-102-generic"... 2024-04-16T17:36:43.278 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-04-16T17:36:43.324 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-102-generic 2024-04-16T17:36:43.324 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-16T17:36:43.324 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T17:36:43.324 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-16T17:36:43.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:36:43.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:36:44.072 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-16T17:36:44.218 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-16T17:36:44.325 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2024-04-16T17:36:44.325 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-16T17:36:44.401 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-04-16T17:36:44.426 DEBUG:teuthology.parallel:result is None 2024-04-16T17:36:44.858 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:36:44.858 INFO:teuthology.task.kernel:Checking kernel version of c1.mon.a, want "5.15.0-102-generic"... 2024-04-16T17:36:44.859 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-16T17:36:44.892 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:36:44.892 INFO:teuthology.task.kernel:Checking kernel version of c2.mgr.x, want "5.15.0-102-generic"... 2024-04-16T17:36:44.893 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-04-16T17:36:44.905 INFO:teuthology.orchestra.run.smithi071.stdout:5.15.0-102-generic 2024-04-16T17:36:44.905 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-16T17:36:44.905 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T17:36:44.905 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-16T17:36:44.941 INFO:teuthology.orchestra.run.smithi188.stdout:5.15.0-102-generic 2024-04-16T17:36:44.941 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-16T17:36:44.941 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T17:36:44.941 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-16T17:36:45.906 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2024-04-16T17:36:45.907 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-16T17:36:45.942 INFO:teuthology.task.kernel:Enabling kdb on c2.mgr.x... 2024-04-16T17:36:45.943 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-16T17:36:46.013 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2024-04-16T17:36:46.039 DEBUG:teuthology.parallel:result is None 2024-04-16T17:36:46.253 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-04-16T17:36:46.275 DEBUG:teuthology.parallel:result is None 2024-04-16T17:36:46.275 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T17:36:46.286 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T17:36:46.286 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T17:36:46.288 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T17:36:46.291 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T17:36:46.293 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T17:36:46.300 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T17:36:46.308 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T17:36:46.317 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T17:36:46.317 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T17:36:46.334 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T17:36:46.339 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T17:36:46.341 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T17:36:46.444 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T17:36:46.451 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T17:36:46.451 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T17:36:46.454 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T17:36:46.457 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T17:36:46.459 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T17:36:46.470 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.475 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.478 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.481 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.483 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.485 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.490 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.494 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T17:36:46.496 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T17:36:46.502 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T17:36:46.502 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T17:36:46.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T17:36:46.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T17:36:46.543 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T17:36:46.560 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T17:36:46.568 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T17:36:46.569 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T17:36:46.606 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T17:36:46.609 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T17:36:46.611 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T17:36:46.617 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T17:36:46.657 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T17:36:46.705 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:36:46.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T17:36:46.792 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T17:36:46.798 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T17:36:46.846 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:36:46.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T17:36:46.908 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T17:36:46.914 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T17:36:46.960 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:36:46.960 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T17:36:47.028 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T17:36:47.033 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T17:36:47.081 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:36:47.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T17:36:47.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-04-16T17:36:47.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-16T17:36:47.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-04-16T17:36:47.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-04-16T17:36:47.235 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T17:36:47.243 INFO:teuthology.task.internal:Starting timer... 2024-04-16T17:36:47.243 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T17:36:47.254 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T17:36:47.263 DEBUG:teuthology.task.selinux:Excluding smithi071: OS 'ubuntu' does not support SELinux 2024-04-16T17:36:47.263 DEBUG:teuthology.task.selinux:Excluding smithi083: OS 'ubuntu' does not support SELinux 2024-04-16T17:36:47.263 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-04-16T17:36:47.263 DEBUG:teuthology.task.selinux:Excluding smithi188: OS 'ubuntu' does not support SELinux 2024-04-16T17:36:47.263 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T17:36:47.263 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-16T17:36:47.263 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T17:36:47.263 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T17:36:47.272 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T17:36:47.287 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T17:36:47.288 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi083.front.sepia.ceph.com,smithi122.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T17:43:49.895 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-04-16T17:43:49.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:43:49.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:43:50.003 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-16T17:43:50.078 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T17:43:50.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:43:50.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:43:50.159 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-16T17:43:50.238 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-16T17:43:50.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:43:50.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:43:50.319 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-16T17:43:50.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T17:43:50.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:43:50.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T17:43:50.474 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-16T17:43:50.550 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-16T17:43:50.551 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T17:43:50.562 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T17:43:50.562 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T17:43:50.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:43:50.566 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T17:43:50.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:43:50.568 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T17:43:50.568 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:43:50.571 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T17:43:50.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:43:50.591 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-16T17:43:50.591 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Command line: ntpd -gq 2024-04-16T17:43:50.591 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: ---------------------------------------------------- 2024-04-16T17:43:50.591 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: ntp-4 is maintained by Network Time Foundation, 2024-04-16T17:43:50.591 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-16T17:43:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: corporation. Support and training for ntp-4 are 2024-04-16T17:43:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: available at https://www.nwtime.org/support 2024-04-16T17:43:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: ---------------------------------------------------- 2024-04-16T17:43:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: proto: precision = 0.059 usec (-24) 2024-04-16T17:43:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: basedate set to 2022-02-04 2024-04-16T17:43:50.592 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: gps base set to 2022-02-06 (week 2196) 2024-04-16T17:43:50.593 INFO:teuthology.orchestra.run.smithi071.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.593 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.593 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.593 INFO:teuthology.orchestra.run.smithi071.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.594 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listen and drop on 0 v6wildcard [::]:123 2024-04-16T17:43:50.595 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-16T17:43:50.595 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listen normally on 2 lo 127.0.0.1:123 2024-04-16T17:43:50.595 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listen normally on 3 enp3s0f1 172.21.15.71:123 2024-04-16T17:43:50.595 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listen normally on 4 lo [::1]:123 2024-04-16T17:43:50.595 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d765%5]:123 2024-04-16T17:43:50.595 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:50 ntpd[17181]: Listening on routing socket on fd #22 for interface updates 2024-04-16T17:43:50.597 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-16T17:43:50.597 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Command line: ntpd -gq 2024-04-16T17:43:50.597 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: ---------------------------------------------------- 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: ntp-4 is maintained by Network Time Foundation, 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: corporation. Support and training for ntp-4 are 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: available at https://www.nwtime.org/support 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: ---------------------------------------------------- 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: proto: precision = 0.061 usec (-24) 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: basedate set to 2022-02-04 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: gps base set to 2022-02-06 (week 2196) 2024-04-16T17:43:50.598 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Command line: ntpd -gq 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: ---------------------------------------------------- 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: ntp-4 is maintained by Network Time Foundation, 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: corporation. Support and training for ntp-4 are 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: available at https://www.nwtime.org/support 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: ---------------------------------------------------- 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: proto: precision = 0.060 usec (-24) 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: basedate set to 2022-02-04 2024-04-16T17:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: gps base set to 2022-02-06 (week 2196) 2024-04-16T17:43:50.600 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.600 INFO:teuthology.orchestra.run.smithi083.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.600 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.600 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.601 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.601 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.601 INFO:teuthology.orchestra.run.smithi083.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.604 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listen and drop on 0 v6wildcard [::]:123 2024-04-16T17:43:50.604 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-16T17:43:50.604 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listen normally on 2 lo 127.0.0.1:123 2024-04-16T17:43:50.604 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-04-16T17:43:50.604 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listen normally on 4 lo [::1]:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:43:50 ntpd[17209]: Listening on routing socket on fd #22 for interface updates 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listen and drop on 0 v6wildcard [::]:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listen normally on 2 lo 127.0.0.1:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listen normally on 3 enp3s0f1 172.21.15.83:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listen normally on 4 lo [::1]:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce91%5]:123 2024-04-16T17:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:50 ntpd[17211]: Listening on routing socket on fd #22 for interface updates 2024-04-16T17:43:50.618 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-16T17:43:50.618 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Command line: ntpd -gq 2024-04-16T17:43:50.618 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: ---------------------------------------------------- 2024-04-16T17:43:50.618 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: ntp-4 is maintained by Network Time Foundation, 2024-04-16T17:43:50.619 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-16T17:43:50.619 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: corporation. Support and training for ntp-4 are 2024-04-16T17:43:50.619 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: available at https://www.nwtime.org/support 2024-04-16T17:43:50.619 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: ---------------------------------------------------- 2024-04-16T17:43:50.619 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: proto: precision = 0.044 usec (-24) 2024-04-16T17:43:50.619 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: basedate set to 2022-02-04 2024-04-16T17:43:50.620 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: gps base set to 2022-02-06 (week 2196) 2024-04-16T17:43:50.620 INFO:teuthology.orchestra.run.smithi188.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.620 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-16T17:43:50.620 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.621 INFO:teuthology.orchestra.run.smithi188.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-16T17:43:50.621 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listen and drop on 0 v6wildcard [::]:123 2024-04-16T17:43:50.622 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-16T17:43:50.622 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listen normally on 2 lo 127.0.0.1:123 2024-04-16T17:43:50.622 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listen normally on 3 enp3s0f1 172.21.15.188:123 2024-04-16T17:43:50.622 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listen normally on 4 lo [::1]:123 2024-04-16T17:43:50.622 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8039%5]:123 2024-04-16T17:43:50.622 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:50 ntpd[17177]: Listening on routing socket on fd #22 for interface updates 2024-04-16T17:43:51.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:51.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:51.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:51.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:51 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:51.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:51 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:51 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:51 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:51 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:51 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:51.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:51 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:51.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:51.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:51.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:51.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:51.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:51 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:53.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:53 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:53 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:53.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:53.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:53 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:54 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:54 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:54 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:54 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:54 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:54 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:54 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:54.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:54.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:54.603 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:54 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:54.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:54 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:54 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:54 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:54.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:54 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:54.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:54 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:54.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:54 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:55.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:55 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:55 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:55 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:55 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:55 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:55 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:55 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:55.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:55.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:55 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:55.619 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:55.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:55 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:56.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:56 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:56 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:56 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:56 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:56 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:56.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:56 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:56.596 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.596 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.596 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.596 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:56.598 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:56 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:56.600 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.600 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.600 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:56.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:56.601 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:56.602 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:56 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:56.619 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:56 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:56 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:56 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:56.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:56 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:56.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:56 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:56.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:56 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:57.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:57.593 INFO:teuthology.orchestra.run.smithi071.stdout:16 Apr 17:43:57 ntpd[17181]: ntpd: time slew +0.001272 s 2024-04-16T17:43:57.593 INFO:teuthology.orchestra.run.smithi071.stdout:ntpd: time slew +0.001272s 2024-04-16T17:43:57.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-16T17:43:57.593 INFO:teuthology.orchestra.run.smithi071.stderr:16 Apr 17:43:57 ntpd[17181]: can't open /var/log/ntpstats/loopstats.20240416: Permission denied 2024-04-16T17:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.594 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:57.595 INFO:teuthology.orchestra.run.smithi083.stdout:16 Apr 17:43:57 ntpd[17211]: ntpd: time slew -0.004800 s 2024-04-16T17:43:57.595 INFO:teuthology.orchestra.run.smithi083.stdout:ntpd: time slew -0.004800s 2024-04-16T17:43:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-16T17:43:57.595 INFO:teuthology.orchestra.run.smithi083.stderr:16 Apr 17:43:57 ntpd[17211]: can't open /var/log/ntpstats/loopstats.20240416: Permission denied 2024-04-16T17:43:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:57 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:57 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:57 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:57 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:57 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:57 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:57.619 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.619 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.619 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.619 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stdout:16 Apr 17:43:57 ntpd[17177]: ntpd: time slew -0.001818 s 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stdout:ntpd: time slew -0.001818s 2024-04-16T17:43:57.620 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-16T17:43:57.621 INFO:teuthology.orchestra.run.smithi188.stderr:16 Apr 17:43:57 ntpd[17177]: can't open /var/log/ntpstats/loopstats.20240416: Permission denied 2024-04-16T17:43:57.643 INFO:teuthology.orchestra.run.smithi071.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T17:43:57.643 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================== 2024-04-16T17:43:57.643 INFO:teuthology.orchestra.run.smithi071.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.643 INFO:teuthology.orchestra.run.smithi071.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.643 INFO:teuthology.orchestra.run.smithi071.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.643 INFO:teuthology.orchestra.run.smithi071.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.649 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T17:43:57.649 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-04-16T17:43:57.649 INFO:teuthology.orchestra.run.smithi083.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.649 INFO:teuthology.orchestra.run.smithi083.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.649 INFO:teuthology.orchestra.run.smithi083.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.649 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.671 INFO:teuthology.orchestra.run.smithi188.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T17:43:57.671 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================== 2024-04-16T17:43:57.671 INFO:teuthology.orchestra.run.smithi188.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.671 INFO:teuthology.orchestra.run.smithi188.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.672 INFO:teuthology.orchestra.run.smithi188.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:57.672 INFO:teuthology.orchestra.run.smithi188.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:58.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:58.598 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:58.598 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:58.598 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:58.598 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:58 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:43:59.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:59 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:59.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:59 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:59.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:59 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:43:59.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:59 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:43:59.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:59 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:43:59.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:43:59 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:44:00.596 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240416: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240416: Permission denied 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stdout:16 Apr 17:44:00 ntpd[17209]: ntpd: time slew -0.003607 s 2024-04-16T17:44:00.597 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew -0.003607s 2024-04-16T17:44:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-16T17:44:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:16 Apr 17:44:00 ntpd[17209]: can't open /var/log/ntpstats/loopstats.20240416: Permission denied 2024-04-16T17:44:00.655 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T17:44:00.655 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-04-16T17:44:00.655 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:44:00.655 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:44:00.655 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:44:00.655 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-16T17:44:00.656 INFO:teuthology.run_tasks:Running task install... 2024-04-16T17:44:00.665 DEBUG:teuthology.task.install:project ceph 2024-04-16T17:44:00.665 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}} 2024-04-16T17:44:00.665 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'} 2024-04-16T17:44:00.665 INFO:teuthology.task.install:Using flavor: default 2024-04-16T17:44:00.670 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T17:44:00.670 INFO:teuthology.task.install:extra packages: [] 2024-04-16T17:44:00.671 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-key list | grep Ceph 2024-04-16T17:44:00.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-key list | grep Ceph 2024-04-16T17:44:00.672 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-04-16T17:44:00.672 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-key list | grep Ceph 2024-04-16T17:44:00.731 INFO:teuthology.orchestra.run.smithi071.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-16T17:44:00.736 INFO:teuthology.orchestra.run.smithi188.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-16T17:44:00.738 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-16T17:44:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-16T17:44:00.757 INFO:teuthology.orchestra.run.smithi071.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-16T17:44:00.757 INFO:teuthology.orchestra.run.smithi071.stdout:uid [ unknown] Ceph.com (release key) 2024-04-16T17:44:00.758 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-16T17:44:00.759 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:44:00.763 INFO:teuthology.orchestra.run.smithi188.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-16T17:44:00.763 INFO:teuthology.orchestra.run.smithi188.stdout:uid [ unknown] Ceph.com (release key) 2024-04-16T17:44:00.764 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-16T17:44:00.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:44:00.767 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-16T17:44:00.767 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph.com (release key) 2024-04-16T17:44:00.768 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-16T17:44:00.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:44:00.781 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-16T17:44:00.781 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-04-16T17:44:00.782 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-16T17:44:00.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:44:00.900 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default/ 2024-04-16T17:44:00.900 INFO:teuthology.task.install.deb:Package version is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:00.917 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default/ 2024-04-16T17:44:00.917 INFO:teuthology.task.install.deb:Package version is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:00.932 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default/ 2024-04-16T17:44:00.932 INFO:teuthology.task.install.deb:Package version is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:00.982 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default/ 2024-04-16T17:44:00.982 INFO:teuthology.task.install.deb:Package version is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:01.014 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:44:01.015 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-16T17:44:01.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2024-04-16T17:44:01.085 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:44:01.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-16T17:44:01.098 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:44:01.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-16T17:44:01.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-04-16T17:44:01.113 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-04-16T17:44:01.157 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:44:01.157 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-16T17:44:01.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo apt-get update 2024-04-16T17:44:01.249 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:44:01.274 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:44:01.285 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-16T17:44:01.288 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:44:01.288 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:44:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-16T17:44:01.304 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:44:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:44:01.322 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:44:01.335 INFO:teuthology.orchestra.run.smithi188.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-16T17:44:01.381 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy InRelease 2024-04-16T17:44:01.388 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-16T17:44:01.403 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-16T17:44:01.425 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-16T17:44:01.449 INFO:teuthology.orchestra.run.smithi188.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy InRelease 2024-04-16T17:44:01.455 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-16T17:44:01.462 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-16T17:44:01.482 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:44:01.485 INFO:teuthology.orchestra.run.smithi188.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-16T17:44:01.516 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-16T17:44:01.523 INFO:teuthology.orchestra.run.smithi188.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-16T17:44:01.536 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-16T17:44:01.559 INFO:teuthology.orchestra.run.smithi188.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-16T17:44:01.562 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy InRelease 2024-04-16T17:44:01.566 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,417 B] 2024-04-16T17:44:01.583 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-16T17:44:01.601 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,417 B] 2024-04-16T17:44:01.605 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-16T17:44:01.618 INFO:teuthology.orchestra.run.smithi188.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-16T17:44:01.626 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-16T17:44:01.637 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 29.3 kB in 0s (81.6 kB/s) 2024-04-16T17:44:01.655 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,417 B] 2024-04-16T17:44:01.736 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 29.3 kB in 0s (67.6 kB/s) 2024-04-16T17:44:01.779 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 140 kB in 0s (324 kB/s) 2024-04-16T17:44:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy InRelease 2024-04-16T17:44:02.596 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-16T17:44:02.605 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-16T17:44:02.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3089-gb0c7b917-1jammy cephadm=19.0.0-3089-gb0c7b917-1jammy ceph-mds=19.0.0-3089-gb0c7b917-1jammy ceph-mgr=19.0.0-3089-gb0c7b917-1jammy ceph-common=19.0.0-3089-gb0c7b917-1jammy ceph-fuse=19.0.0-3089-gb0c7b917-1jammy ceph-test=19.0.0-3089-gb0c7b917-1jammy ceph-volume=19.0.0-3089-gb0c7b917-1jammy radosgw=19.0.0-3089-gb0c7b917-1jammy python3-rados=19.0.0-3089-gb0c7b917-1jammy python3-rgw=19.0.0-3089-gb0c7b917-1jammy python3-cephfs=19.0.0-3089-gb0c7b917-1jammy python3-rbd=19.0.0-3089-gb0c7b917-1jammy libcephfs2=19.0.0-3089-gb0c7b917-1jammy libcephfs-dev=19.0.0-3089-gb0c7b917-1jammy librados2=19.0.0-3089-gb0c7b917-1jammy librbd1=19.0.0-3089-gb0c7b917-1jammy rbd-fuse=19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:02.635 INFO:teuthology.orchestra.run.smithi071.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-16T17:44:02.672 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-16T17:44:02.683 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-04-16T17:44:02.712 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-04-16T17:44:02.719 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,417 B] 2024-04-16T17:44:02.727 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3089-gb0c7b917-1jammy cephadm=19.0.0-3089-gb0c7b917-1jammy ceph-mds=19.0.0-3089-gb0c7b917-1jammy ceph-mgr=19.0.0-3089-gb0c7b917-1jammy ceph-common=19.0.0-3089-gb0c7b917-1jammy ceph-fuse=19.0.0-3089-gb0c7b917-1jammy ceph-test=19.0.0-3089-gb0c7b917-1jammy ceph-volume=19.0.0-3089-gb0c7b917-1jammy radosgw=19.0.0-3089-gb0c7b917-1jammy python3-rados=19.0.0-3089-gb0c7b917-1jammy python3-rgw=19.0.0-3089-gb0c7b917-1jammy python3-cephfs=19.0.0-3089-gb0c7b917-1jammy python3-rbd=19.0.0-3089-gb0c7b917-1jammy libcephfs2=19.0.0-3089-gb0c7b917-1jammy libcephfs-dev=19.0.0-3089-gb0c7b917-1jammy librados2=19.0.0-3089-gb0c7b917-1jammy librbd1=19.0.0-3089-gb0c7b917-1jammy rbd-fuse=19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:02.739 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2024-04-16T17:44:02.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3089-gb0c7b917-1jammy cephadm=19.0.0-3089-gb0c7b917-1jammy ceph-mds=19.0.0-3089-gb0c7b917-1jammy ceph-mgr=19.0.0-3089-gb0c7b917-1jammy ceph-common=19.0.0-3089-gb0c7b917-1jammy ceph-fuse=19.0.0-3089-gb0c7b917-1jammy ceph-test=19.0.0-3089-gb0c7b917-1jammy ceph-volume=19.0.0-3089-gb0c7b917-1jammy radosgw=19.0.0-3089-gb0c7b917-1jammy python3-rados=19.0.0-3089-gb0c7b917-1jammy python3-rgw=19.0.0-3089-gb0c7b917-1jammy python3-cephfs=19.0.0-3089-gb0c7b917-1jammy python3-rbd=19.0.0-3089-gb0c7b917-1jammy libcephfs2=19.0.0-3089-gb0c7b917-1jammy libcephfs-dev=19.0.0-3089-gb0c7b917-1jammy librados2=19.0.0-3089-gb0c7b917-1jammy librbd1=19.0.0-3089-gb0c7b917-1jammy rbd-fuse=19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:02.779 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-04-16T17:44:02.806 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 29.3 kB in 2s (18.5 kB/s) 2024-04-16T17:44:02.807 INFO:teuthology.orchestra.run.smithi188.stdout:Reading package lists... 2024-04-16T17:44:02.883 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-04-16T17:44:02.884 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-04-16T17:44:02.953 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-04-16T17:44:02.953 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-04-16T17:44:02.985 INFO:teuthology.orchestra.run.smithi188.stdout:Building dependency tree... 2024-04-16T17:44:02.986 INFO:teuthology.orchestra.run.smithi188.stdout:Reading state information... 2024-04-16T17:44:03.080 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:44:03.081 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:44:03.081 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:44:03.081 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:44:03.081 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:44:03.081 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:44:03.082 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:44:03.082 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-04-16T17:44:03.082 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-16T17:44:03.083 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-16T17:44:03.084 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet zip 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-16T17:44:03.085 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-04-16T17:44:03.086 INFO:teuthology.orchestra.run.smithi083.stdout: btrfs-tools 2024-04-16T17:44:03.126 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:44:03.126 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:44:03.126 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:44:03.126 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:44:03.127 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:44:03.127 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:44:03.127 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:44:03.129 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-04-16T17:44:03.129 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-16T17:44:03.129 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-16T17:44:03.129 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:03.130 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-16T17:44:03.131 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet zip 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-04-16T17:44:03.132 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-04-16T17:44:03.161 INFO:teuthology.orchestra.run.smithi188.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:44:03.161 INFO:teuthology.orchestra.run.smithi188.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:44:03.161 INFO:teuthology.orchestra.run.smithi188.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:44:03.161 INFO:teuthology.orchestra.run.smithi188.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:44:03.162 INFO:teuthology.orchestra.run.smithi188.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:44:03.162 INFO:teuthology.orchestra.run.smithi188.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:44:03.162 INFO:teuthology.orchestra.run.smithi188.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:44:03.163 INFO:teuthology.orchestra.run.smithi188.stdout:The following additional packages will be installed: 2024-04-16T17:44:03.163 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:03.164 INFO:teuthology.orchestra.run.smithi188.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-16T17:44:03.165 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet zip 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout:Suggested packages: 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-16T17:44:03.166 INFO:teuthology.orchestra.run.smithi188.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-16T17:44:03.167 INFO:teuthology.orchestra.run.smithi188.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-16T17:44:03.167 INFO:teuthology.orchestra.run.smithi188.stdout:Recommended packages: 2024-04-16T17:44:03.167 INFO:teuthology.orchestra.run.smithi188.stdout: btrfs-tools 2024-04-16T17:44:03.185 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-04-16T17:44:03.185 INFO:teuthology.orchestra.run.smithi083.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-16T17:44:03.185 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-16T17:44:03.185 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-16T17:44:03.186 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-16T17:44:03.186 INFO:teuthology.orchestra.run.smithi083.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-16T17:44:03.186 INFO:teuthology.orchestra.run.smithi083.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:03.187 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-16T17:44:03.188 INFO:teuthology.orchestra.run.smithi083.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-16T17:44:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-16T17:44:03.233 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-16T17:44:03.234 INFO:teuthology.orchestra.run.smithi122.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-16T17:44:03.235 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-16T17:44:03.235 INFO:teuthology.orchestra.run.smithi122.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-16T17:44:03.235 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-16T17:44:03.235 INFO:teuthology.orchestra.run.smithi122.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-16T17:44:03.251 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-16T17:44:03.251 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 181 MB of archives. 2024-04-16T17:44:03.251 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 829 MB of additional disk space will be used. 2024-04-16T17:44:03.251 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout:The following NEW packages will be installed: 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-16T17:44:03.267 INFO:teuthology.orchestra.run.smithi188.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-16T17:44:03.268 INFO:teuthology.orchestra.run.smithi188.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-16T17:44:03.268 INFO:teuthology.orchestra.run.smithi188.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-16T17:44:03.268 INFO:teuthology.orchestra.run.smithi188.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-16T17:44:03.269 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-16T17:44:03.270 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-16T17:44:03.270 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-16T17:44:03.270 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-16T17:44:03.270 INFO:teuthology.orchestra.run.smithi188.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-16T17:44:03.304 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-16T17:44:03.305 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 181 MB of archives. 2024-04-16T17:44:03.305 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 829 MB of additional disk space will be used. 2024-04-16T17:44:03.305 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-16T17:44:03.329 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3089-gb0c7b917-1jammy [3,554 kB] 2024-04-16T17:44:03.352 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-16T17:44:03.355 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-16T17:44:03.375 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-16T17:44:03.407 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-16T17:44:03.410 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-16T17:44:03.430 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-16T17:44:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-16T17:44:03.443 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-16T17:44:03.456 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-16T17:44:03.456 INFO:teuthology.orchestra.run.smithi188.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-16T17:44:03.456 INFO:teuthology.orchestra.run.smithi188.stdout:Need to get 181 MB of archives. 2024-04-16T17:44:03.456 INFO:teuthology.orchestra.run.smithi188.stdout:After this operation, 829 MB of additional disk space will be used. 2024-04-16T17:44:03.457 INFO:teuthology.orchestra.run.smithi188.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-16T17:44:03.460 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-16T17:44:03.460 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-16T17:44:03.462 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-16T17:44:03.470 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-16T17:44:03.472 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-16T17:44:03.474 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-16T17:44:03.476 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-16T17:44:03.482 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-16T17:44:03.485 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-16T17:44:03.487 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-16T17:44:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-16T17:44:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-16T17:44:03.502 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-16T17:44:03.503 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-16T17:44:03.503 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-16T17:44:03.505 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-16T17:44:03.505 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-16T17:44:03.505 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-16T17:44:03.506 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-16T17:44:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-16T17:44:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-16T17:44:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-16T17:44:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-16T17:44:03.522 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-16T17:44:03.522 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-16T17:44:03.522 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-16T17:44:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-16T17:44:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-16T17:44:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-16T17:44:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-16T17:44:03.540 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-16T17:44:03.543 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-16T17:44:03.543 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-16T17:44:03.543 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-16T17:44:03.544 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-16T17:44:03.544 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-16T17:44:03.546 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-16T17:44:03.546 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-16T17:44:03.546 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-16T17:44:03.557 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-16T17:44:03.559 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-16T17:44:03.559 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-16T17:44:03.560 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-16T17:44:03.561 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-16T17:44:03.561 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-16T17:44:03.562 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-16T17:44:03.562 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-16T17:44:03.562 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-16T17:44:03.563 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-16T17:44:03.564 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-16T17:44:03.580 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-16T17:44:03.580 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-16T17:44:03.581 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-16T17:44:03.582 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-16T17:44:03.582 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-16T17:44:03.583 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-16T17:44:03.583 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-16T17:44:03.585 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-16T17:44:03.587 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-16T17:44:03.588 INFO:teuthology.orchestra.run.smithi083.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-16T17:44:03.600 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-16T17:44:03.600 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-16T17:44:03.601 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-16T17:44:03.601 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-16T17:44:03.602 INFO:teuthology.orchestra.run.smithi083.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-16T17:44:03.602 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-16T17:44:03.603 INFO:teuthology.orchestra.run.smithi083.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-16T17:44:03.603 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-16T17:44:03.603 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-16T17:44:03.605 INFO:teuthology.orchestra.run.smithi083.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-16T17:44:03.620 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-16T17:44:03.621 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-16T17:44:03.621 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-16T17:44:03.624 INFO:teuthology.orchestra.run.smithi083.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-16T17:44:03.625 INFO:teuthology.orchestra.run.smithi083.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-16T17:44:03.627 INFO:teuthology.orchestra.run.smithi083.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-16T17:44:03.651 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-16T17:44:03.651 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-16T17:44:03.652 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-16T17:44:03.653 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-16T17:44:03.655 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-16T17:44:03.658 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-16T17:44:03.659 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-16T17:44:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-16T17:44:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-16T17:44:03.680 INFO:teuthology.orchestra.run.smithi083.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-16T17:44:03.681 INFO:teuthology.orchestra.run.smithi083.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-16T17:44:03.683 INFO:teuthology.orchestra.run.smithi083.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-16T17:44:03.683 INFO:teuthology.orchestra.run.smithi083.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-16T17:44:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-16T17:44:03.684 INFO:teuthology.orchestra.run.smithi083.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-16T17:44:03.701 INFO:teuthology.orchestra.run.smithi083.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-16T17:44:03.703 INFO:teuthology.orchestra.run.smithi083.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-16T17:44:03.703 INFO:teuthology.orchestra.run.smithi083.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-16T17:44:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-16T17:44:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-16T17:44:03.709 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-16T17:44:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3089-gb0c7b917-1jammy [3,554 kB] 2024-04-16T17:44:03.726 INFO:teuthology.orchestra.run.smithi083.stdout:Get:60 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3089-gb0c7b917-1jammy [3,224 kB] 2024-04-16T17:44:03.741 INFO:teuthology.orchestra.run.smithi083.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-16T17:44:03.742 INFO:teuthology.orchestra.run.smithi083.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-16T17:44:03.744 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-04-16T17:44:03.746 INFO:teuthology.orchestra.run.smithi188.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3089-gb0c7b917-1jammy [3,554 kB] 2024-04-16T17:44:03.755 INFO:teuthology.orchestra.run.smithi083.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-16T17:44:03.756 INFO:teuthology.orchestra.run.smithi083.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-16T17:44:03.756 INFO:teuthology.orchestra.run.smithi083.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-16T17:44:03.757 INFO:teuthology.orchestra.run.smithi083.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-16T17:44:03.757 INFO:teuthology.orchestra.run.smithi083.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-16T17:44:03.757 INFO:teuthology.orchestra.run.smithi083.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-16T17:44:03.760 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3089-gb0c7b917-1jammy cephadm=19.0.0-3089-gb0c7b917-1jammy ceph-mds=19.0.0-3089-gb0c7b917-1jammy ceph-mgr=19.0.0-3089-gb0c7b917-1jammy ceph-common=19.0.0-3089-gb0c7b917-1jammy ceph-fuse=19.0.0-3089-gb0c7b917-1jammy ceph-test=19.0.0-3089-gb0c7b917-1jammy ceph-volume=19.0.0-3089-gb0c7b917-1jammy radosgw=19.0.0-3089-gb0c7b917-1jammy python3-rados=19.0.0-3089-gb0c7b917-1jammy python3-rgw=19.0.0-3089-gb0c7b917-1jammy python3-cephfs=19.0.0-3089-gb0c7b917-1jammy python3-rbd=19.0.0-3089-gb0c7b917-1jammy libcephfs2=19.0.0-3089-gb0c7b917-1jammy libcephfs-dev=19.0.0-3089-gb0c7b917-1jammy librados2=19.0.0-3089-gb0c7b917-1jammy librbd1=19.0.0-3089-gb0c7b917-1jammy rbd-fuse=19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:44:03.761 INFO:teuthology.orchestra.run.smithi083.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-16T17:44:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-16T17:44:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-16T17:44:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-16T17:44:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-16T17:44:03.781 INFO:teuthology.orchestra.run.smithi083.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-16T17:44:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-16T17:44:03.782 INFO:teuthology.orchestra.run.smithi083.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-16T17:44:03.783 INFO:teuthology.orchestra.run.smithi083.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-16T17:44:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-16T17:44:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-16T17:44:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-16T17:44:03.807 INFO:teuthology.orchestra.run.smithi083.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-16T17:44:03.811 INFO:teuthology.orchestra.run.smithi083.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-16T17:44:03.811 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2024-04-16T17:44:03.812 INFO:teuthology.orchestra.run.smithi083.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-16T17:44:03.812 INFO:teuthology.orchestra.run.smithi083.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-16T17:44:03.817 INFO:teuthology.orchestra.run.smithi083.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-16T17:44:03.817 INFO:teuthology.orchestra.run.smithi083.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-16T17:44:03.821 INFO:teuthology.orchestra.run.smithi083.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-16T17:44:03.828 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-16T17:44:03.828 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-16T17:44:03.829 INFO:teuthology.orchestra.run.smithi083.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-16T17:44:03.829 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-16T17:44:03.830 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-16T17:44:03.830 INFO:teuthology.orchestra.run.smithi083.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-16T17:44:03.830 INFO:teuthology.orchestra.run.smithi083.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-16T17:44:03.831 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-16T17:44:03.831 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-16T17:44:03.831 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-16T17:44:03.832 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-16T17:44:03.837 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-16T17:44:03.848 INFO:teuthology.orchestra.run.smithi083.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-16T17:44:03.849 INFO:teuthology.orchestra.run.smithi083.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-16T17:44:03.851 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-16T17:44:03.856 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-16T17:44:03.857 INFO:teuthology.orchestra.run.smithi083.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-16T17:44:03.857 INFO:teuthology.orchestra.run.smithi083.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-16T17:44:03.858 INFO:teuthology.orchestra.run.smithi083.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-16T17:44:03.860 INFO:teuthology.orchestra.run.smithi083.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-16T17:44:03.860 INFO:teuthology.orchestra.run.smithi083.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-16T17:44:03.878 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-16T17:44:03.883 INFO:teuthology.orchestra.run.smithi188.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-16T17:44:03.884 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-16T17:44:03.888 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-16T17:44:03.890 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-16T17:44:03.891 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-16T17:44:03.895 INFO:teuthology.orchestra.run.smithi083.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3089-gb0c7b917-1jammy [712 kB] 2024-04-16T17:44:03.895 INFO:teuthology.orchestra.run.smithi188.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-16T17:44:03.898 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-16T17:44:03.899 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-16T17:44:03.904 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-16T17:44:03.904 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-16T17:44:03.906 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-16T17:44:03.918 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-16T17:44:03.920 INFO:teuthology.orchestra.run.smithi083.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3089-gb0c7b917-1jammy [356 kB] 2024-04-16T17:44:03.920 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-16T17:44:03.922 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-16T17:44:03.932 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-16T17:44:03.937 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-16T17:44:03.938 INFO:teuthology.orchestra.run.smithi083.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3089-gb0c7b917-1jammy [32.8 kB] 2024-04-16T17:44:03.938 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-16T17:44:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3089-gb0c7b917-1jammy [180 kB] 2024-04-16T17:44:03.941 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-16T17:44:03.942 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-16T17:44:03.943 INFO:teuthology.orchestra.run.smithi083.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3089-gb0c7b917-1jammy [66.9 kB] 2024-04-16T17:44:03.954 INFO:teuthology.orchestra.run.smithi083.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3089-gb0c7b917-1jammy [330 kB] 2024-04-16T17:44:03.977 INFO:teuthology.orchestra.run.smithi188.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-16T17:44:03.984 INFO:teuthology.orchestra.run.smithi083.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3089-gb0c7b917-1jammy [6,889 kB] 2024-04-16T17:44:03.993 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2024-04-16T17:44:03.994 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2024-04-16T17:44:04.171 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2024-04-16T17:44:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-16T17:44:04.171 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-16T17:44:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-16T17:44:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-16T17:44:04.172 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-16T17:44:04.172 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-16T17:44:04.173 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2024-04-16T17:44:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-16T17:44:04.173 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:04.175 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-16T17:44:04.176 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet zip 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout:Recommended packages: 2024-04-16T17:44:04.177 INFO:teuthology.orchestra.run.smithi071.stdout: btrfs-tools 2024-04-16T17:44:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-16T17:44:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-16T17:44:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-16T17:44:04.260 INFO:teuthology.orchestra.run.smithi083.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3089-gb0c7b917-1jammy [112 kB] 2024-04-16T17:44:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-16T17:44:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-16T17:44:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-16T17:44:04.263 INFO:teuthology.orchestra.run.smithi083.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3089-gb0c7b917-1jammy [466 kB] 2024-04-16T17:44:04.278 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2024-04-16T17:44:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-16T17:44:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-16T17:44:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-16T17:44:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-16T17:44:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-16T17:44:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-16T17:44:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-16T17:44:04.281 INFO:teuthology.orchestra.run.smithi188.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-16T17:44:04.282 INFO:teuthology.orchestra.run.smithi083.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3089-gb0c7b917-1jammy [25.8 MB] 2024-04-16T17:44:04.346 INFO:teuthology.orchestra.run.smithi071.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-16T17:44:04.346 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 181 MB of archives. 2024-04-16T17:44:04.346 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 829 MB of additional disk space will be used. 2024-04-16T17:44:04.346 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-16T17:44:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-16T17:44:04.450 INFO:teuthology.orchestra.run.smithi188.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-16T17:44:04.450 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-16T17:44:04.470 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-16T17:44:04.474 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3089-gb0c7b917-1jammy [3,554 kB] 2024-04-16T17:44:04.533 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-16T17:44:04.538 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-16T17:44:04.551 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-16T17:44:04.555 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-16T17:44:04.556 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-16T17:44:04.556 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-16T17:44:04.565 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-16T17:44:04.567 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-16T17:44:04.569 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-16T17:44:04.571 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-16T17:44:04.577 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-16T17:44:04.580 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-16T17:44:04.582 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-16T17:44:04.597 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-16T17:44:04.598 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-16T17:44:04.598 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-16T17:44:04.600 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-16T17:44:04.600 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-16T17:44:04.600 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-16T17:44:04.600 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-16T17:44:04.616 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-16T17:44:04.617 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-16T17:44:04.617 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-16T17:44:04.636 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-16T17:44:04.637 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-16T17:44:04.637 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-16T17:44:04.638 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-16T17:44:04.639 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-16T17:44:04.642 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-16T17:44:04.642 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-16T17:44:04.656 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-16T17:44:04.657 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-16T17:44:04.657 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-16T17:44:04.676 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-16T17:44:04.676 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-16T17:44:04.676 INFO:teuthology.orchestra.run.smithi071.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-16T17:44:04.677 INFO:teuthology.orchestra.run.smithi071.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-16T17:44:04.679 INFO:teuthology.orchestra.run.smithi071.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-16T17:44:04.680 INFO:teuthology.orchestra.run.smithi071.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-16T17:44:04.681 INFO:teuthology.orchestra.run.smithi071.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-16T17:44:04.682 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3089-gb0c7b917-1jammy [3,224 kB] 2024-04-16T17:44:04.695 INFO:teuthology.orchestra.run.smithi071.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-16T17:44:04.696 INFO:teuthology.orchestra.run.smithi071.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-16T17:44:04.699 INFO:teuthology.orchestra.run.smithi071.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-16T17:44:04.715 INFO:teuthology.orchestra.run.smithi071.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-16T17:44:04.716 INFO:teuthology.orchestra.run.smithi071.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-16T17:44:04.718 INFO:teuthology.orchestra.run.smithi071.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-16T17:44:04.771 INFO:teuthology.orchestra.run.smithi071.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-16T17:44:04.772 INFO:teuthology.orchestra.run.smithi071.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-16T17:44:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-16T17:44:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-16T17:44:04.775 INFO:teuthology.orchestra.run.smithi071.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-16T17:44:04.792 INFO:teuthology.orchestra.run.smithi071.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-16T17:44:04.794 INFO:teuthology.orchestra.run.smithi071.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-16T17:44:04.794 INFO:teuthology.orchestra.run.smithi071.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-16T17:44:04.809 INFO:teuthology.orchestra.run.smithi071.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-16T17:44:04.809 INFO:teuthology.orchestra.run.smithi071.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-16T17:44:04.815 INFO:teuthology.orchestra.run.smithi071.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-16T17:44:04.815 INFO:teuthology.orchestra.run.smithi071.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-16T17:44:04.816 INFO:teuthology.orchestra.run.smithi071.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-16T17:44:04.816 INFO:teuthology.orchestra.run.smithi071.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-16T17:44:04.817 INFO:teuthology.orchestra.run.smithi071.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-16T17:44:04.835 INFO:teuthology.orchestra.run.smithi071.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-16T17:44:04.838 INFO:teuthology.orchestra.run.smithi071.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-16T17:44:04.840 INFO:teuthology.orchestra.run.smithi071.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-16T17:44:04.841 INFO:teuthology.orchestra.run.smithi071.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-16T17:44:04.841 INFO:teuthology.orchestra.run.smithi071.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-16T17:44:04.854 INFO:teuthology.orchestra.run.smithi071.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-16T17:44:04.857 INFO:teuthology.orchestra.run.smithi071.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-16T17:44:04.858 INFO:teuthology.orchestra.run.smithi071.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-16T17:44:04.858 INFO:teuthology.orchestra.run.smithi188.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-16T17:44:04.858 INFO:teuthology.orchestra.run.smithi071.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-16T17:44:04.862 INFO:teuthology.orchestra.run.smithi071.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-16T17:44:04.874 INFO:teuthology.orchestra.run.smithi071.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-16T17:44:04.877 INFO:teuthology.orchestra.run.smithi071.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-16T17:44:04.877 INFO:teuthology.orchestra.run.smithi071.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-16T17:44:04.878 INFO:teuthology.orchestra.run.smithi071.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-16T17:44:04.878 INFO:teuthology.orchestra.run.smithi071.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-16T17:44:04.892 INFO:teuthology.orchestra.run.smithi188.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-16T17:44:04.894 INFO:teuthology.orchestra.run.smithi071.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-16T17:44:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-16T17:44:04.894 INFO:teuthology.orchestra.run.smithi071.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-16T17:44:04.901 INFO:teuthology.orchestra.run.smithi071.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-16T17:44:04.902 INFO:teuthology.orchestra.run.smithi071.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-16T17:44:04.902 INFO:teuthology.orchestra.run.smithi071.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-16T17:44:04.913 INFO:teuthology.orchestra.run.smithi071.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-16T17:44:04.914 INFO:teuthology.orchestra.run.smithi071.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-16T17:44:04.973 INFO:teuthology.orchestra.run.smithi188.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-16T17:44:05.024 INFO:teuthology.orchestra.run.smithi188.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-16T17:44:05.046 INFO:teuthology.orchestra.run.smithi188.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-16T17:44:05.053 INFO:teuthology.orchestra.run.smithi188.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-16T17:44:05.054 INFO:teuthology.orchestra.run.smithi188.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-16T17:44:05.065 INFO:teuthology.orchestra.run.smithi188.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-16T17:44:05.066 INFO:teuthology.orchestra.run.smithi188.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-16T17:44:05.232 INFO:teuthology.orchestra.run.smithi188.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-16T17:44:05.315 INFO:teuthology.orchestra.run.smithi188.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-16T17:44:05.343 INFO:teuthology.orchestra.run.smithi188.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-16T17:44:05.360 INFO:teuthology.orchestra.run.smithi188.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-16T17:44:05.402 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3089-gb0c7b917-1jammy [712 kB] 2024-04-16T17:44:05.405 INFO:teuthology.orchestra.run.smithi188.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-16T17:44:05.406 INFO:teuthology.orchestra.run.smithi188.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3089-gb0c7b917-1jammy [3,224 kB] 2024-04-16T17:44:05.525 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3089-gb0c7b917-1jammy [356 kB] 2024-04-16T17:44:05.561 INFO:teuthology.orchestra.run.smithi083.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3089-gb0c7b917-1jammy [5,094 kB] 2024-04-16T17:44:05.590 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3089-gb0c7b917-1jammy [32.8 kB] 2024-04-16T17:44:05.591 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3089-gb0c7b917-1jammy [180 kB] 2024-04-16T17:44:05.597 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3089-gb0c7b917-1jammy [66.9 kB] 2024-04-16T17:44:05.620 INFO:teuthology.orchestra.run.smithi188.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-16T17:44:05.624 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3089-gb0c7b917-1jammy [330 kB] 2024-04-16T17:44:05.635 INFO:teuthology.orchestra.run.smithi188.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-16T17:44:05.641 INFO:teuthology.orchestra.run.smithi188.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-16T17:44:05.649 INFO:teuthology.orchestra.run.smithi188.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-16T17:44:05.671 INFO:teuthology.orchestra.run.smithi188.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-16T17:44:05.675 INFO:teuthology.orchestra.run.smithi188.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-16T17:44:05.680 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3089-gb0c7b917-1jammy [6,889 kB] 2024-04-16T17:44:05.755 INFO:teuthology.orchestra.run.smithi083.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3089-gb0c7b917-1jammy [244 kB] 2024-04-16T17:44:05.775 INFO:teuthology.orchestra.run.smithi083.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3089-gb0c7b917-1jammy [124 kB] 2024-04-16T17:44:05.779 INFO:teuthology.orchestra.run.smithi083.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3089-gb0c7b917-1jammy [1,526 kB] 2024-04-16T17:44:05.840 INFO:teuthology.orchestra.run.smithi188.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-16T17:44:05.866 INFO:teuthology.orchestra.run.smithi083.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3089-gb0c7b917-1jammy [6,258 kB] 2024-04-16T17:44:05.996 INFO:teuthology.orchestra.run.smithi071.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3089-gb0c7b917-1jammy [3,224 kB] 2024-04-16T17:44:06.170 INFO:teuthology.orchestra.run.smithi188.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-16T17:44:06.210 INFO:teuthology.orchestra.run.smithi188.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-16T17:44:06.223 INFO:teuthology.orchestra.run.smithi188.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-16T17:44:06.237 INFO:teuthology.orchestra.run.smithi083.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3089-gb0c7b917-1jammy [22.8 MB] 2024-04-16T17:44:06.335 INFO:teuthology.orchestra.run.smithi188.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3089-gb0c7b917-1jammy [712 kB] 2024-04-16T17:44:06.434 INFO:teuthology.orchestra.run.smithi188.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-16T17:44:06.435 INFO:teuthology.orchestra.run.smithi188.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-16T17:44:06.441 INFO:teuthology.orchestra.run.smithi188.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-16T17:44:06.446 INFO:teuthology.orchestra.run.smithi188.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-16T17:44:06.461 INFO:teuthology.orchestra.run.smithi188.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-16T17:44:06.482 INFO:teuthology.orchestra.run.smithi188.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-16T17:44:06.489 INFO:teuthology.orchestra.run.smithi188.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-16T17:44:06.552 INFO:teuthology.orchestra.run.smithi188.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3089-gb0c7b917-1jammy [356 kB] 2024-04-16T17:44:06.635 INFO:teuthology.orchestra.run.smithi188.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3089-gb0c7b917-1jammy [32.8 kB] 2024-04-16T17:44:06.640 INFO:teuthology.orchestra.run.smithi188.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3089-gb0c7b917-1jammy [180 kB] 2024-04-16T17:44:06.663 INFO:teuthology.orchestra.run.smithi188.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-16T17:44:06.706 INFO:teuthology.orchestra.run.smithi188.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3089-gb0c7b917-1jammy [66.9 kB] 2024-04-16T17:44:06.713 INFO:teuthology.orchestra.run.smithi188.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3089-gb0c7b917-1jammy [330 kB] 2024-04-16T17:44:06.750 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3089-gb0c7b917-1jammy [112 kB] 2024-04-16T17:44:06.760 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3089-gb0c7b917-1jammy [466 kB] 2024-04-16T17:44:06.795 INFO:teuthology.orchestra.run.smithi188.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3089-gb0c7b917-1jammy [6,889 kB] 2024-04-16T17:44:06.833 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3089-gb0c7b917-1jammy [25.8 MB] 2024-04-16T17:44:06.841 INFO:teuthology.orchestra.run.smithi071.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3089-gb0c7b917-1jammy [712 kB] 2024-04-16T17:44:06.927 INFO:teuthology.orchestra.run.smithi188.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-16T17:44:07.030 INFO:teuthology.orchestra.run.smithi071.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3089-gb0c7b917-1jammy [356 kB] 2024-04-16T17:44:07.106 INFO:teuthology.orchestra.run.smithi071.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3089-gb0c7b917-1jammy [32.8 kB] 2024-04-16T17:44:07.106 INFO:teuthology.orchestra.run.smithi188.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-16T17:44:07.107 INFO:teuthology.orchestra.run.smithi071.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3089-gb0c7b917-1jammy [180 kB] 2024-04-16T17:44:07.119 INFO:teuthology.orchestra.run.smithi188.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-16T17:44:07.139 INFO:teuthology.orchestra.run.smithi188.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-16T17:44:07.168 INFO:teuthology.orchestra.run.smithi071.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3089-gb0c7b917-1jammy [66.9 kB] 2024-04-16T17:44:07.171 INFO:teuthology.orchestra.run.smithi071.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3089-gb0c7b917-1jammy [330 kB] 2024-04-16T17:44:07.204 INFO:teuthology.orchestra.run.smithi188.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-16T17:44:07.240 INFO:teuthology.orchestra.run.smithi071.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3089-gb0c7b917-1jammy [6,889 kB] 2024-04-16T17:44:07.493 INFO:teuthology.orchestra.run.smithi188.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-16T17:44:07.495 INFO:teuthology.orchestra.run.smithi188.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-16T17:44:07.498 INFO:teuthology.orchestra.run.smithi188.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-16T17:44:07.498 INFO:teuthology.orchestra.run.smithi188.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-16T17:44:07.501 INFO:teuthology.orchestra.run.smithi188.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-16T17:44:07.719 INFO:teuthology.orchestra.run.smithi188.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-16T17:44:07.720 INFO:teuthology.orchestra.run.smithi083.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3089-gb0c7b917-1jammy [14.1 kB] 2024-04-16T17:44:07.720 INFO:teuthology.orchestra.run.smithi083.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [870 kB] 2024-04-16T17:44:07.738 INFO:teuthology.orchestra.run.smithi083.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3089-gb0c7b917-1jammy [2,437 kB] 2024-04-16T17:44:07.809 INFO:teuthology.orchestra.run.smithi083.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3089-gb0c7b917-1jammy [791 kB] 2024-04-16T17:44:07.845 INFO:teuthology.orchestra.run.smithi083.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3089-gb0c7b917-1jammy [148 kB] 2024-04-16T17:44:07.851 INFO:teuthology.orchestra.run.smithi083.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3089-gb0c7b917-1jammy [2,304 kB] 2024-04-16T17:44:08.004 INFO:teuthology.orchestra.run.smithi083.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3089-gb0c7b917-1jammy [8,625 kB] 2024-04-16T17:44:08.150 INFO:teuthology.orchestra.run.smithi188.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-16T17:44:08.156 INFO:teuthology.orchestra.run.smithi188.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-16T17:44:08.505 INFO:teuthology.orchestra.run.smithi188.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-16T17:44:08.506 INFO:teuthology.orchestra.run.smithi188.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-16T17:44:08.542 INFO:teuthology.orchestra.run.smithi083.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3089-gb0c7b917-1jammy [14.2 kB] 2024-04-16T17:44:08.542 INFO:teuthology.orchestra.run.smithi083.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3089-gb0c7b917-1jammy [51.6 MB] 2024-04-16T17:44:08.601 INFO:teuthology.orchestra.run.smithi188.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-16T17:44:08.602 INFO:teuthology.orchestra.run.smithi188.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-16T17:44:08.603 INFO:teuthology.orchestra.run.smithi188.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-16T17:44:08.604 INFO:teuthology.orchestra.run.smithi188.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-16T17:44:08.605 INFO:teuthology.orchestra.run.smithi188.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-16T17:44:08.606 INFO:teuthology.orchestra.run.smithi188.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-16T17:44:08.779 INFO:teuthology.orchestra.run.smithi188.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-16T17:44:08.898 INFO:teuthology.orchestra.run.smithi188.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3089-gb0c7b917-1jammy [112 kB] 2024-04-16T17:44:08.945 INFO:teuthology.orchestra.run.smithi188.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3089-gb0c7b917-1jammy [466 kB] 2024-04-16T17:44:08.982 INFO:teuthology.orchestra.run.smithi071.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3089-gb0c7b917-1jammy [112 kB] 2024-04-16T17:44:08.989 INFO:teuthology.orchestra.run.smithi071.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3089-gb0c7b917-1jammy [466 kB] 2024-04-16T17:44:09.063 INFO:teuthology.orchestra.run.smithi188.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3089-gb0c7b917-1jammy [25.8 MB] 2024-04-16T17:44:09.126 INFO:teuthology.orchestra.run.smithi071.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3089-gb0c7b917-1jammy [25.8 MB] 2024-04-16T17:44:09.191 INFO:teuthology.orchestra.run.smithi188.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-16T17:44:09.242 INFO:teuthology.orchestra.run.smithi188.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-16T17:44:09.273 INFO:teuthology.orchestra.run.smithi188.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-16T17:44:09.372 INFO:teuthology.orchestra.run.smithi188.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-16T17:44:09.410 INFO:teuthology.orchestra.run.smithi188.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-16T17:44:09.420 INFO:teuthology.orchestra.run.smithi188.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-16T17:44:09.431 INFO:teuthology.orchestra.run.smithi188.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-16T17:44:09.480 INFO:teuthology.orchestra.run.smithi188.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-16T17:44:09.482 INFO:teuthology.orchestra.run.smithi188.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-16T17:44:09.503 INFO:teuthology.orchestra.run.smithi188.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-16T17:44:09.667 INFO:teuthology.orchestra.run.smithi188.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-16T17:44:09.941 INFO:teuthology.orchestra.run.smithi188.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-16T17:44:09.958 INFO:teuthology.orchestra.run.smithi188.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-16T17:44:10.111 INFO:teuthology.orchestra.run.smithi188.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-16T17:44:10.154 INFO:teuthology.orchestra.run.smithi188.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-16T17:44:10.320 INFO:teuthology.orchestra.run.smithi188.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-16T17:44:10.322 INFO:teuthology.orchestra.run.smithi188.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-16T17:44:10.325 INFO:teuthology.orchestra.run.smithi188.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-16T17:44:10.354 INFO:teuthology.orchestra.run.smithi188.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-16T17:44:10.363 INFO:teuthology.orchestra.run.smithi188.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-16T17:44:10.630 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3089-gb0c7b917-1jammy [5,094 kB] 2024-04-16T17:44:11.286 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3089-gb0c7b917-1jammy [244 kB] 2024-04-16T17:44:11.305 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3089-gb0c7b917-1jammy [124 kB] 2024-04-16T17:44:11.325 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3089-gb0c7b917-1jammy [1,526 kB] 2024-04-16T17:44:11.573 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3089-gb0c7b917-1jammy [6,258 kB] 2024-04-16T17:44:12.125 INFO:teuthology.orchestra.run.smithi083.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3089-gb0c7b917-1jammy [121 kB] 2024-04-16T17:44:12.136 INFO:teuthology.orchestra.run.smithi083.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3089-gb0c7b917-1jammy [38.9 kB] 2024-04-16T17:44:12.138 INFO:teuthology.orchestra.run.smithi083.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3089-gb0c7b917-1jammy [13.6 MB] 2024-04-16T17:44:12.443 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3089-gb0c7b917-1jammy [22.8 MB] 2024-04-16T17:44:12.865 INFO:teuthology.orchestra.run.smithi083.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [91.4 kB] 2024-04-16T17:44:13.427 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 181 MB in 10s (18.7 MB/s) 2024-04-16T17:44:13.592 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-16T17:44:13.690 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-16T17:44:13.695 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-16T17:44:13.782 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:44:13.783 INFO:teuthology.orchestra.run.smithi071.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3089-gb0c7b917-1jammy [5,094 kB] 2024-04-16T17:44:14.136 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-16T17:44:14.162 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-16T17:44:14.194 INFO:teuthology.orchestra.run.smithi071.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3089-gb0c7b917-1jammy [244 kB] 2024-04-16T17:44:14.202 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:44:14.216 INFO:teuthology.orchestra.run.smithi071.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3089-gb0c7b917-1jammy [124 kB] 2024-04-16T17:44:14.228 INFO:teuthology.orchestra.run.smithi071.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3089-gb0c7b917-1jammy [1,526 kB] 2024-04-16T17:44:14.372 INFO:teuthology.orchestra.run.smithi071.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3089-gb0c7b917-1jammy [6,258 kB] 2024-04-16T17:44:14.497 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-16T17:44:14.508 INFO:teuthology.orchestra.run.smithi188.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3089-gb0c7b917-1jammy [5,094 kB] 2024-04-16T17:44:14.523 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-16T17:44:14.533 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3089-gb0c7b917-1jammy [14.1 kB] 2024-04-16T17:44:14.534 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [870 kB] 2024-04-16T17:44:14.550 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3089-gb0c7b917-1jammy [2,437 kB] 2024-04-16T17:44:14.555 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:44:14.725 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3089-gb0c7b917-1jammy [791 kB] 2024-04-16T17:44:14.779 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3089-gb0c7b917-1jammy [148 kB] 2024-04-16T17:44:14.787 INFO:teuthology.orchestra.run.smithi122.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3089-gb0c7b917-1jammy [2,304 kB] 2024-04-16T17:44:14.923 INFO:teuthology.orchestra.run.smithi071.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3089-gb0c7b917-1jammy [22.8 MB] 2024-04-16T17:44:14.980 INFO:teuthology.orchestra.run.smithi122.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3089-gb0c7b917-1jammy [8,625 kB] 2024-04-16T17:44:15.017 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-16T17:44:15.043 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:15.093 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:15.146 INFO:teuthology.orchestra.run.smithi188.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3089-gb0c7b917-1jammy [244 kB] 2024-04-16T17:44:15.184 INFO:teuthology.orchestra.run.smithi188.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3089-gb0c7b917-1jammy [124 kB] 2024-04-16T17:44:15.190 INFO:teuthology.orchestra.run.smithi188.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3089-gb0c7b917-1jammy [1,526 kB] 2024-04-16T17:44:15.414 INFO:teuthology.orchestra.run.smithi188.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3089-gb0c7b917-1jammy [6,258 kB] 2024-04-16T17:44:15.487 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-16T17:44:15.513 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:15.554 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:15.656 INFO:teuthology.orchestra.run.smithi122.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3089-gb0c7b917-1jammy [14.2 kB] 2024-04-16T17:44:15.657 INFO:teuthology.orchestra.run.smithi122.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3089-gb0c7b917-1jammy [51.6 MB] 2024-04-16T17:44:15.865 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-16T17:44:15.892 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:15.957 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:16.299 INFO:teuthology.orchestra.run.smithi188.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3089-gb0c7b917-1jammy [22.8 MB] 2024-04-16T17:44:16.412 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-16T17:44:16.412 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-16T17:44:16.412 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:44:16.704 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librados2. 2024-04-16T17:44:16.730 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../007-librados2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:16.771 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:17.119 INFO:teuthology.orchestra.run.smithi071.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3089-gb0c7b917-1jammy [14.1 kB] 2024-04-16T17:44:17.120 INFO:teuthology.orchestra.run.smithi071.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [870 kB] 2024-04-16T17:44:17.144 INFO:teuthology.orchestra.run.smithi071.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3089-gb0c7b917-1jammy [2,437 kB] 2024-04-16T17:44:17.224 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librbd1. 2024-04-16T17:44:17.250 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../008-librbd1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:17.357 INFO:teuthology.orchestra.run.smithi071.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3089-gb0c7b917-1jammy [791 kB] 2024-04-16T17:44:17.384 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:17.428 INFO:teuthology.orchestra.run.smithi071.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3089-gb0c7b917-1jammy [148 kB] 2024-04-16T17:44:17.442 INFO:teuthology.orchestra.run.smithi071.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3089-gb0c7b917-1jammy [2,304 kB] 2024-04-16T17:44:17.645 INFO:teuthology.orchestra.run.smithi071.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3089-gb0c7b917-1jammy [8,625 kB] 2024-04-16T17:44:17.902 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs2. 2024-04-16T17:44:17.929 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:17.968 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:18.446 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rados. 2024-04-16T17:44:18.447 INFO:teuthology.orchestra.run.smithi071.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3089-gb0c7b917-1jammy [14.2 kB] 2024-04-16T17:44:18.447 INFO:teuthology.orchestra.run.smithi071.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3089-gb0c7b917-1jammy [51.6 MB] 2024-04-16T17:44:18.472 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../010-python3-rados_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:18.512 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:18.816 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-16T17:44:18.842 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:18.882 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:19.118 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cephfs. 2024-04-16T17:44:19.133 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:19.168 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:19.411 INFO:teuthology.orchestra.run.smithi188.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3089-gb0c7b917-1jammy [14.1 kB] 2024-04-16T17:44:19.412 INFO:teuthology.orchestra.run.smithi188.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [870 kB] 2024-04-16T17:44:19.458 INFO:teuthology.orchestra.run.smithi188.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3089-gb0c7b917-1jammy [2,437 kB] 2024-04-16T17:44:19.506 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-16T17:44:19.532 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:19.570 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:19.764 INFO:teuthology.orchestra.run.smithi188.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3089-gb0c7b917-1jammy [791 kB] 2024-04-16T17:44:19.866 INFO:teuthology.orchestra.run.smithi188.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3089-gb0c7b917-1jammy [148 kB] 2024-04-16T17:44:19.905 INFO:teuthology.orchestra.run.smithi188.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3089-gb0c7b917-1jammy [2,304 kB] 2024-04-16T17:44:19.907 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-16T17:44:19.933 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-16T17:44:19.974 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:44:20.174 INFO:teuthology.orchestra.run.smithi188.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3089-gb0c7b917-1jammy [8,625 kB] 2024-04-16T17:44:20.258 INFO:teuthology.orchestra.run.smithi122.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3089-gb0c7b917-1jammy [121 kB] 2024-04-16T17:44:20.260 INFO:teuthology.orchestra.run.smithi122.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3089-gb0c7b917-1jammy [38.9 kB] 2024-04-16T17:44:20.261 INFO:teuthology.orchestra.run.smithi122.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3089-gb0c7b917-1jammy [13.6 MB] 2024-04-16T17:44:20.301 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-prettytable. 2024-04-16T17:44:20.328 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-16T17:44:20.368 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-16T17:44:20.587 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rbd. 2024-04-16T17:44:20.602 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:20.637 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:20.982 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-16T17:44:21.008 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-16T17:44:21.049 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:44:21.274 INFO:teuthology.orchestra.run.smithi188.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3089-gb0c7b917-1jammy [14.2 kB] 2024-04-16T17:44:21.275 INFO:teuthology.orchestra.run.smithi188.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3089-gb0c7b917-1jammy [51.6 MB] 2024-04-16T17:44:21.334 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-16T17:44:21.344 INFO:teuthology.orchestra.run.smithi122.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [91.4 kB] 2024-04-16T17:44:21.351 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-16T17:44:21.393 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:44:21.698 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-16T17:44:21.722 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-16T17:44:21.768 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:44:21.904 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 181 MB in 18s (10.0 MB/s) 2024-04-16T17:44:22.043 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-16T17:44:22.101 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua5.1. 2024-04-16T17:44:22.136 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-16T17:44:22.144 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-16T17:44:22.147 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-16T17:44:22.167 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:44:22.471 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-any. 2024-04-16T17:44:22.478 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:44:22.485 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-16T17:44:22.500 INFO:teuthology.orchestra.run.smithi071.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3089-gb0c7b917-1jammy [121 kB] 2024-04-16T17:44:22.503 INFO:teuthology.orchestra.run.smithi071.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3089-gb0c7b917-1jammy [38.9 kB] 2024-04-16T17:44:22.504 INFO:teuthology.orchestra.run.smithi071.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3089-gb0c7b917-1jammy [13.6 MB] 2024-04-16T17:44:22.529 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-16T17:44:22.756 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package zip. 2024-04-16T17:44:22.772 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-16T17:44:22.806 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking zip (3.0-12build2) ... 2024-04-16T17:44:22.824 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-16T17:44:22.850 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-16T17:44:22.890 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:44:23.147 INFO:teuthology.orchestra.run.smithi071.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [91.4 kB] 2024-04-16T17:44:23.151 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package unzip. 2024-04-16T17:44:23.178 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-16T17:44:23.202 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-16T17:44:23.218 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:44:23.229 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-16T17:44:23.269 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:44:23.723 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-16T17:44:23.745 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 181 MB in 19s (9,564 kB/s) 2024-04-16T17:44:23.749 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:23.772 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package luarocks. 2024-04-16T17:44:23.798 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-16T17:44:23.798 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:23.838 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:44:23.908 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-16T17:44:24.008 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-16T17:44:24.014 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-16T17:44:24.106 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:44:24.180 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librgw2. 2024-04-16T17:44:24.206 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../025-librgw2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:24.218 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-16T17:44:24.245 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:24.246 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:24.285 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:24.427 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-16T17:44:24.453 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-16T17:44:24.485 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:44:24.597 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-16T17:44:24.623 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:24.655 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:24.774 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rgw. 2024-04-16T17:44:24.800 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:24.821 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-16T17:44:24.847 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-16T17:44:24.849 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:24.888 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:44:25.000 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-16T17:44:25.026 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-16T17:44:25.067 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:44:25.186 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-16T17:44:25.212 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-16T17:44:25.252 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:44:25.340 INFO:teuthology.orchestra.run.smithi188.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3089-gb0c7b917-1jammy [121 kB] 2024-04-16T17:44:25.341 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-16T17:44:25.343 INFO:teuthology.orchestra.run.smithi188.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3089-gb0c7b917-1jammy [38.9 kB] 2024-04-16T17:44:25.344 INFO:teuthology.orchestra.run.smithi188.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3089-gb0c7b917-1jammy [13.6 MB] 2024-04-16T17:44:25.368 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:25.395 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librados2. 2024-04-16T17:44:25.416 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:25.422 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-librados2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:25.461 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:25.572 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libradosstriper1. 2024-04-16T17:44:25.598 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:25.638 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:25.777 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-16T17:44:25.792 INFO:teuthology.orchestra.run.smithi188.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-shilpa-multisite-tests/b0c7b917267c471208541fdeb652f5e897d00086/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3089-gb0c7b917-1jammy [91.4 kB] 2024-04-16T17:44:25.804 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:25.836 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:25.873 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librbd1. 2024-04-16T17:44:25.888 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-librbd1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:25.923 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-common. 2024-04-16T17:44:26.018 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../029-ceph-common_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:26.050 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:26.180 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-16T17:44:26.207 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:26.247 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:26.318 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-04-16T17:44:26.345 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:26.384 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:26.428 INFO:teuthology.orchestra.run.smithi188.stdout:Fetched 181 MB in 23s (8,015 kB/s) 2024-04-16T17:44:26.567 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-16T17:44:26.591 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-16T17:44:26.593 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-16T17:44:26.625 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:44:26.662 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-04-16T17:44:26.689 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-python3-rados_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:26.691 INFO:teuthology.orchestra.run.smithi188.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-16T17:44:26.697 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-16T17:44:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:26.772 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:44:26.919 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librados2. 2024-04-16T17:44:26.946 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../007-librados2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:26.986 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:27.032 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-16T17:44:27.048 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:27.082 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:27.084 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-16T17:44:27.111 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-16T17:44:27.151 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:44:27.301 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-04-16T17:44:27.328 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:27.368 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:27.472 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-base. 2024-04-16T17:44:27.480 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librbd1. 2024-04-16T17:44:27.487 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-16T17:44:27.498 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../030-ceph-base_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:27.506 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../008-librbd1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:27.514 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-16T17:44:27.547 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:27.554 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:44:27.565 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:27.821 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-16T17:44:27.847 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:27.888 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:27.991 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-16T17:44:28.018 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:28.066 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:28.144 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-16T17:44:28.170 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-16T17:44:28.175 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libcephfs2. 2024-04-16T17:44:28.201 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:28.210 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:44:28.225 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-16T17:44:28.241 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:28.251 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-16T17:44:28.291 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:44:28.486 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-16T17:44:28.488 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cheroot. 2024-04-16T17:44:28.513 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:28.514 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-16T17:44:28.535 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rados. 2024-04-16T17:44:28.550 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../010-python3-rados_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:28.552 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:28.554 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:44:28.578 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-04-16T17:44:28.585 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:28.604 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-16T17:44:28.636 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-16T17:44:28.858 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-16T17:44:28.880 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-16T17:44:28.884 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-16T17:44:28.896 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-16T17:44:28.897 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-04-16T17:44:28.907 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-16T17:44:28.913 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:28.922 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:28.924 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:44:28.947 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:44:28.956 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:28.963 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:29.199 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cephfs. 2024-04-16T17:44:29.214 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:29.227 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-16T17:44:29.254 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-16T17:44:29.257 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:29.275 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-16T17:44:29.294 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:44:29.302 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-16T17:44:29.309 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-16T17:44:29.335 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-16T17:44:29.342 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:44:29.376 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:44:29.576 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-16T17:44:29.597 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-16T17:44:29.602 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:29.623 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-16T17:44:29.643 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:29.654 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-16T17:44:29.663 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:44:29.669 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-16T17:44:29.703 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librados2. 2024-04-16T17:44:29.704 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:44:29.729 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../007-librados2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:29.770 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:29.950 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempora. 2024-04-16T17:44:29.974 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-16T17:44:29.976 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-16T17:44:29.987 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-16T17:44:29.989 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-16T17:44:30.008 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-16T17:44:30.013 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-16T17:44:30.032 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:44:30.054 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:44:30.223 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librbd1. 2024-04-16T17:44:30.249 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../008-librbd1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:30.289 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:30.320 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-portend. 2024-04-16T17:44:30.346 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-16T17:44:30.357 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-prettytable. 2024-04-16T17:44:30.369 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua5.1. 2024-04-16T17:44:30.383 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-16T17:44:30.386 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-16T17:44:30.396 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-16T17:44:30.423 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-16T17:44:30.436 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:44:30.684 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rbd. 2024-04-16T17:44:30.710 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:30.731 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-any. 2024-04-16T17:44:30.734 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libcephfs2. 2024-04-16T17:44:30.751 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:30.757 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-16T17:44:30.760 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:30.764 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-16T17:44:30.789 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-16T17:44:30.790 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-16T17:44:30.801 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:30.839 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-16T17:44:31.017 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package zip. 2024-04-16T17:44:31.043 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-16T17:44:31.084 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking zip (3.0-12build2) ... 2024-04-16T17:44:31.086 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rados. 2024-04-16T17:44:31.104 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-16T17:44:31.113 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../010-python3-rados_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:31.130 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-16T17:44:31.153 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:31.170 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:44:31.184 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-16T17:44:31.210 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-16T17:44:31.250 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:44:31.429 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package unzip. 2024-04-16T17:44:31.448 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-16T17:44:31.455 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-16T17:44:31.474 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-16T17:44:31.496 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:44:31.523 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:44:31.663 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-natsort. 2024-04-16T17:44:31.673 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-16T17:44:31.689 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-16T17:44:31.699 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:31.737 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-16T17:44:31.739 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:31.784 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-16T17:44:31.799 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-16T17:44:31.842 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:44:31.883 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package luarocks. 2024-04-16T17:44:31.909 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-16T17:44:31.941 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:44:31.992 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cephfs. 2024-04-16T17:44:32.008 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:32.042 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:32.116 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-logutils. 2024-04-16T17:44:32.142 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-16T17:44:32.170 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua5.1. 2024-04-16T17:44:32.190 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-16T17:44:32.197 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-16T17:44:32.237 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:44:32.292 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-04-16T17:44:32.318 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-librgw2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:32.328 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-16T17:44:32.355 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:32.358 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:32.395 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:32.486 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-16T17:44:32.512 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-16T17:44:32.552 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:44:32.573 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua-any. 2024-04-16T17:44:32.589 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-16T17:44:32.632 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-16T17:44:32.723 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-16T17:44:32.749 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-16T17:44:32.789 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:44:32.842 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package zip. 2024-04-16T17:44:32.858 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-16T17:44:32.888 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-mako. 2024-04-16T17:44:32.892 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking zip (3.0-12build2) ... 2024-04-16T17:44:32.914 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-16T17:44:32.963 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:44:33.128 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-04-16T17:44:33.134 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-prettytable. 2024-04-16T17:44:33.147 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:33.160 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-16T17:44:33.186 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:33.200 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-16T17:44:33.229 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package unzip. 2024-04-16T17:44:33.256 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-16T17:44:33.283 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-16T17:44:33.295 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:44:33.310 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-16T17:44:33.342 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:44:33.470 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rbd. 2024-04-16T17:44:33.497 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:33.523 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-16T17:44:33.536 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:33.549 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-16T17:44:33.590 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:44:33.645 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-16T17:44:33.671 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-16T17:44:33.711 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:44:33.724 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package luarocks. 2024-04-16T17:44:33.750 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-16T17:44:33.791 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:44:33.877 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-04-16T17:44:33.881 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-16T17:44:33.903 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:33.908 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-16T17:44:33.947 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:44:33.960 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:34.157 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librgw2. 2024-04-16T17:44:34.183 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../025-librgw2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:34.200 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-16T17:44:34.215 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-16T17:44:34.223 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:34.231 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webob. 2024-04-16T17:44:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-16T17:44:34.258 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:44:34.297 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:44:34.389 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-04-16T17:44:34.415 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-ceph-common_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:34.455 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:34.578 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-16T17:44:34.604 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-16T17:44:34.645 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:44:34.651 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-waitress. 2024-04-16T17:44:34.677 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-16T17:44:34.720 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:44:34.792 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rgw. 2024-04-16T17:44:34.808 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:34.851 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:35.012 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempita. 2024-04-16T17:44:35.031 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package lua5.1. 2024-04-16T17:44:35.038 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-16T17:44:35.057 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-16T17:44:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:44:35.098 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:44:35.128 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-16T17:44:35.155 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-16T17:44:35.187 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:44:35.356 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-paste. 2024-04-16T17:44:35.383 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-16T17:44:35.409 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package lua-any. 2024-04-16T17:44:35.423 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:44:35.424 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-16T17:44:35.459 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-16T17:44:35.473 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libradosstriper1. 2024-04-16T17:44:35.499 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:35.539 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:35.703 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package zip. 2024-04-16T17:44:35.719 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-16T17:44:35.751 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-16T17:44:35.762 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking zip (3.0-12build2) ... 2024-04-16T17:44:35.767 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-16T17:44:35.803 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-04-16T17:44:35.810 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-ceph-base_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:35.870 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:35.968 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-common. 2024-04-16T17:44:35.995 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../029-ceph-common_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:36.034 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:36.090 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package unzip. 2024-04-16T17:44:36.117 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-16T17:44:36.121 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-16T17:44:36.147 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-16T17:44:36.157 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:44:36.187 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:44:36.458 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-16T17:44:36.484 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-16T17:44:36.491 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webtest. 2024-04-16T17:44:36.517 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-16T17:44:36.557 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-16T17:44:36.677 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package luarocks. 2024-04-16T17:44:36.704 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-16T17:44:36.744 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:44:36.835 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pecan. 2024-04-16T17:44:36.862 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-16T17:44:36.902 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:44:37.008 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:44:37.094 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package librgw2. 2024-04-16T17:44:37.120 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../025-librgw2_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:37.160 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:37.240 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-certifi. 2024-04-16T17:44:37.266 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-16T17:44:37.307 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-16T17:44:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-base. 2024-04-16T17:44:37.416 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../030-ceph-base_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:37.465 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:37.561 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cheroot. 2024-04-16T17:44:37.577 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-16T17:44:37.611 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:44:37.668 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-idna. 2024-04-16T17:44:37.695 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-16T17:44:37.696 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rgw. 2024-04-16T17:44:37.722 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:37.735 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-16T17:44:37.763 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:37.915 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-16T17:44:37.941 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-16T17:44:37.981 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:44:38.055 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-urllib3. 2024-04-16T17:44:38.069 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-16T17:44:38.081 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-16T17:44:38.095 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-16T17:44:38.107 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-16T17:44:38.121 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:44:38.133 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-16T17:44:38.135 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:44:38.174 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:44:38.276 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-16T17:44:38.292 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-16T17:44:38.326 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:44:38.450 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests. 2024-04-16T17:44:38.468 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libradosstriper1. 2024-04-16T17:44:38.476 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-16T17:44:38.495 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:38.516 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:44:38.535 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:38.605 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-16T17:44:38.631 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-16T17:44:38.671 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:44:38.803 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-16T17:44:38.829 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-16T17:44:38.861 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:44:38.897 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-common. 2024-04-16T17:44:38.924 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../029-ceph-common_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:38.963 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:38.974 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempora. 2024-04-16T17:44:39.001 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-16T17:44:39.041 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-16T17:44:39.147 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-16T17:44:39.174 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:39.213 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:39.305 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cheroot. 2024-04-16T17:44:39.332 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-16T17:44:39.344 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-portend. 2024-04-16T17:44:39.363 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:44:39.371 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-16T17:44:39.411 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-16T17:44:39.602 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-16T17:44:39.629 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:39.669 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:39.683 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-16T17:44:39.689 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-16T17:44:39.709 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-16T17:44:39.715 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-16T17:44:39.749 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:44:39.755 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-16T17:44:39.980 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-16T17:44:40.006 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-16T17:44:40.042 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-16T17:44:40.046 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:44:40.052 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-16T17:44:40.069 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-16T17:44:40.079 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-16T17:44:40.109 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:44:40.119 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:44:40.308 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-openssl. 2024-04-16T17:44:40.323 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-16T17:44:40.357 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-16T17:44:40.405 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-16T17:44:40.411 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-base. 2024-04-16T17:44:40.431 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-16T17:44:40.437 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../030-ceph-base_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:40.463 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-04-16T17:44:40.471 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:44:40.486 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:40.489 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-16T17:44:40.529 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-16T17:44:40.711 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr. 2024-04-16T17:44:40.737 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:40.741 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-tempora. 2024-04-16T17:44:40.756 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-16T17:44:40.777 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:40.799 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-16T17:44:40.799 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-04-16T17:44:40.826 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-16T17:44:40.866 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-16T17:44:41.040 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-16T17:44:41.066 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-16T17:44:41.102 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-portend. 2024-04-16T17:44:41.106 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:44:41.128 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-16T17:44:41.144 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-16T17:44:41.168 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-16T17:44:41.171 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-16T17:44:41.198 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mon. 2024-04-16T17:44:41.211 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:44:41.225 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:41.264 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:41.368 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cheroot. 2024-04-16T17:44:41.394 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-16T17:44:41.426 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:44:41.471 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-16T17:44:41.497 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-16T17:44:41.514 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-04-16T17:44:41.538 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-16T17:44:41.540 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-16T17:44:41.581 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:44:41.679 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-16T17:44:41.694 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-16T17:44:41.737 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:44:41.832 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-16T17:44:41.839 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-osd. 2024-04-16T17:44:41.858 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-16T17:44:41.861 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:41.890 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:44:41.893 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:41.909 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-16T17:44:41.936 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-16T17:44:41.976 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:44:42.040 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-16T17:44:42.066 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-16T17:44:42.107 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:44:42.238 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-16T17:44:42.244 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-natsort. 2024-04-16T17:44:42.253 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-16T17:44:42.271 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-16T17:44:42.287 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:44:42.310 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-16T17:44:42.410 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-16T17:44:42.436 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-16T17:44:42.476 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:44:42.574 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-04-16T17:44:42.601 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-16T17:44:42.640 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:44:42.647 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-logutils. 2024-04-16T17:44:42.673 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-16T17:44:42.714 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-16T17:44:42.737 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-tempora. 2024-04-16T17:44:42.752 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-16T17:44:42.787 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-16T17:44:42.963 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph. 2024-04-16T17:44:42.989 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../066-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:43.011 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-04-16T17:44:43.037 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:43.038 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-16T17:44:43.058 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-16T17:44:43.074 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-portend. 2024-04-16T17:44:43.081 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:44:43.084 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-16T17:44:43.100 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-16T17:44:43.125 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:44:43.140 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-16T17:44:43.357 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-fuse. 2024-04-16T17:44:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:43.423 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:43.452 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-16T17:44:43.470 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-mako. 2024-04-16T17:44:43.478 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-16T17:44:43.495 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-16T17:44:43.518 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-16T17:44:43.544 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:44:43.547 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-04-16T17:44:43.562 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-16T17:44:43.597 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:44:43.796 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-16T17:44:43.822 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-16T17:44:43.827 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mds. 2024-04-16T17:44:43.854 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:43.875 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-04-16T17:44:43.887 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:44:43.897 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-16T17:44:43.902 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:43.902 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-16T17:44:43.924 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-16T17:44:43.942 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:44:43.964 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:44:44.216 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-natsort. 2024-04-16T17:44:44.234 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-16T17:44:44.243 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-16T17:44:44.260 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-16T17:44:44.283 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-16T17:44:44.300 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:44:44.304 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-16T17:44:44.330 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-16T17:44:44.371 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:44:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephadm. 2024-04-16T17:44:44.558 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../069-cephadm_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:44.594 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-logutils. 2024-04-16T17:44:44.595 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-webob. 2024-04-16T17:44:44.621 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-16T17:44:44.621 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-16T17:44:44.630 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:44.661 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:44:44.661 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-16T17:44:44.673 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-16T17:44:44.689 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-16T17:44:44.732 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:44:44.980 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-16T17:44:44.983 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-16T17:44:44.993 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-04-16T17:44:45.006 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-waitress. 2024-04-16T17:44:45.007 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-16T17:44:45.009 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-16T17:44:45.009 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-16T17:44:45.032 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-16T17:44:45.043 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-16T17:44:45.047 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:44:45.049 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-16T17:44:45.075 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:44:45.300 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-mako. 2024-04-16T17:44:45.327 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-16T17:44:45.342 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-tempita. 2024-04-16T17:44:45.355 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-04-16T17:44:45.366 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:44:45.369 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-16T17:44:45.381 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-16T17:44:45.400 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:44:45.402 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-16T17:44:45.422 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:44:45.429 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:45.469 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:45.687 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-16T17:44:45.703 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-paste. 2024-04-16T17:44:45.713 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-16T17:44:45.730 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-16T17:44:45.752 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-certifi. 2024-04-16T17:44:45.753 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:44:45.769 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:44:45.778 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-16T17:44:45.819 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-16T17:44:46.056 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-16T17:44:46.083 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-16T17:44:46.122 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:44:46.124 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-idna. 2024-04-16T17:44:46.131 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-16T17:44:46.139 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-16T17:44:46.158 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-16T17:44:46.174 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-16T17:44:46.206 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:44:46.383 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-webob. 2024-04-16T17:44:46.399 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-16T17:44:46.434 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:44:46.435 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-urllib3. 2024-04-16T17:44:46.451 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-16T17:44:46.494 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:44:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-16T17:44:46.533 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-babel. 2024-04-16T17:44:46.544 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-16T17:44:46.560 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:46.583 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:44:46.599 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:46.797 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests. 2024-04-16T17:44:46.824 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-16T17:44:46.864 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:44:46.886 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-webtest. 2024-04-16T17:44:46.913 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-16T17:44:46.919 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jinja2. 2024-04-16T17:44:46.928 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-waitress. 2024-04-16T17:44:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-16T17:44:46.953 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-16T17:44:46.955 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-16T17:44:46.977 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:44:46.990 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:44:47.176 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-16T17:44:47.202 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-16T17:44:47.242 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:44:47.256 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-tempita. 2024-04-16T17:44:47.283 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-16T17:44:47.289 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-16T17:44:47.297 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pecan. 2024-04-16T17:44:47.316 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:47.323 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:44:47.324 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-16T17:44:47.355 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:47.372 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:44:47.504 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-16T17:44:47.519 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:47.562 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:47.651 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-paste. 2024-04-16T17:44:47.675 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-16T17:44:47.678 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-16T17:44:47.702 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-16T17:44:47.717 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:44:47.741 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-16T17:44:47.777 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-certifi. 2024-04-16T17:44:47.804 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-16T17:44:47.852 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-16T17:44:47.951 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-16T17:44:47.977 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:48.017 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:48.020 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-routes. 2024-04-16T17:44:48.037 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-16T17:44:48.047 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-16T17:44:48.064 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-16T17:44:48.086 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:44:48.104 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:44:48.238 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-idna. 2024-04-16T17:44:48.267 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-16T17:44:48.287 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-16T17:44:48.313 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-16T17:44:48.314 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-16T17:44:48.346 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:44:48.423 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-16T17:44:48.450 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-16T17:44:48.465 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-16T17:44:48.490 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:44:48.492 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:48.531 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:48.633 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-urllib3. 2024-04-16T17:44:48.659 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-16T17:44:48.666 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-04-16T17:44:48.692 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-16T17:44:48.699 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:44:48.732 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-16T17:44:48.768 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-webtest. 2024-04-16T17:44:48.794 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-16T17:44:48.835 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-16T17:44:49.019 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-requests. 2024-04-16T17:44:49.045 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-16T17:44:49.077 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-04-16T17:44:49.085 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:44:49.104 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:49.144 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:49.146 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pecan. 2024-04-16T17:44:49.172 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-16T17:44:49.212 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:44:49.363 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-16T17:44:49.391 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-16T17:44:49.430 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:44:49.523 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-04-16T17:44:49.539 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:49.572 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:49.578 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-certifi. 2024-04-16T17:44:49.594 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-16T17:44:49.636 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-16T17:44:49.708 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-16T17:44:49.723 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:49.758 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:49.879 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-16T17:44:49.906 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-16T17:44:49.931 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-idna. 2024-04-16T17:44:49.953 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:44:49.957 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-16T17:44:49.989 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-16T17:44:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-16T17:44:50.177 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-04-16T17:44:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:50.203 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:50.221 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:50.243 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:50.283 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-urllib3. 2024-04-16T17:44:50.310 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-16T17:44:50.358 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:44:50.507 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-16T17:44:50.508 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-joblib. 2024-04-16T17:44:50.534 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-16T17:44:50.534 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-16T17:44:50.574 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:44:50.582 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:44:50.670 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-requests. 2024-04-16T17:44:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-16T17:44:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:44:50.835 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-openssl. 2024-04-16T17:44:50.862 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-16T17:44:50.901 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-16T17:44:50.969 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-16T17:44:50.997 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-16T17:44:51.036 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:44:51.065 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-16T17:44:51.091 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-16T17:44:51.131 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:44:51.238 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr. 2024-04-16T17:44:51.265 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:51.304 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:51.330 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn. 2024-04-16T17:44:51.338 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-04-16T17:44:51.354 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-16T17:44:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:44:51.405 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:51.426 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-16T17:44:51.453 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:51.493 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:51.658 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-04-16T17:44:51.685 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:51.716 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mon. 2024-04-16T17:44:51.724 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:51.743 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:51.782 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:51.864 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-16T17:44:51.891 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:51.931 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:52.087 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-04-16T17:44:52.113 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:52.153 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:52.164 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-16T17:44:52.192 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:52.217 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-16T17:44:52.244 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-16T17:44:52.292 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:44:52.324 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-osd. 2024-04-16T17:44:52.445 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:52.494 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:52.582 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-04-16T17:44:52.595 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-openssl. 2024-04-16T17:44:52.608 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-cephadm_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:52.622 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-16T17:44:52.656 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:52.661 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-16T17:44:52.918 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-16T17:44:52.933 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-16T17:44:52.968 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-16T17:44:53.006 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr. 2024-04-16T17:44:53.033 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:53.072 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:53.287 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cachetools. 2024-04-16T17:44:53.287 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-16T17:44:53.314 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:53.314 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-16T17:44:53.353 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-16T17:44:53.354 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:53.434 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mon. 2024-04-16T17:44:53.461 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:53.493 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:53.596 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph. 2024-04-16T17:44:53.612 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../066-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:53.656 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-16T17:44:53.663 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:53.683 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-16T17:44:53.722 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:44:54.068 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-16T17:44:54.088 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-osd. 2024-04-16T17:44:54.095 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-16T17:44:54.114 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:54.134 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:44:54.154 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:54.191 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-fuse. 2024-04-16T17:44:54.217 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:54.257 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:54.455 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rsa. 2024-04-16T17:44:54.482 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-16T17:44:54.521 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-16T17:44:54.576 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-babel. 2024-04-16T17:44:54.603 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:54.643 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:54.778 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mds. 2024-04-16T17:44:54.791 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-google-auth. 2024-04-16T17:44:54.804 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:54.820 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-16T17:44:54.844 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:55.049 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-16T17:44:55.071 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-04-16T17:44:55.098 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-16T17:44:55.138 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:44:55.272 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package cephadm. 2024-04-16T17:44:55.299 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../069-cephadm_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:55.338 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:55.341 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph. 2024-04-16T17:44:55.367 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../066-ceph_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:55.407 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:55.470 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-16T17:44:55.483 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-16T17:44:55.497 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-16T17:44:55.510 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:55.544 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:44:55.549 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:55.666 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-16T17:44:55.685 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-fuse. 2024-04-16T17:44:55.693 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-16T17:44:55.712 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:55.732 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-16T17:44:55.743 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:55.870 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-16T17:44:55.873 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-websocket. 2024-04-16T17:44:55.897 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-16T17:44:55.900 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-16T17:44:55.936 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-16T17:44:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-16T17:44:56.060 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-16T17:44:56.087 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:56.127 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:56.130 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mds. 2024-04-16T17:44:56.157 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:56.197 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:56.239 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-04-16T17:44:56.255 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-16T17:44:56.259 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-16T17:44:56.286 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-16T17:44:56.298 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:44:56.325 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:44:56.608 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package cephadm. 2024-04-16T17:44:56.634 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../069-cephadm_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:56.651 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-16T17:44:56.674 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:56.678 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:56.718 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:56.965 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-16T17:44:56.977 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-16T17:44:56.981 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:57.004 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-16T17:44:57.023 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:57.035 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-16T17:44:57.090 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-babel. 2024-04-16T17:44:57.117 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:57.156 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:57.334 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-16T17:44:57.361 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-16T17:44:57.401 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:44:57.526 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jinja2. 2024-04-16T17:44:57.553 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-16T17:44:57.555 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-16T17:44:57.571 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:57.592 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:44:57.613 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:57.746 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-16T17:44:57.773 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:44:57.804 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:44:57.962 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-16T17:44:57.974 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-16T17:44:57.989 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:57.990 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-16T17:44:58.024 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:44:58.028 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:58.065 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package jq. 2024-04-16T17:44:58.092 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:44:58.131 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-16T17:44:58.382 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-16T17:44:58.384 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package socat. 2024-04-16T17:44:58.400 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-16T17:44:58.409 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-16T17:44:58.442 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:44:58.448 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-16T17:44:58.486 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-04-16T17:44:58.513 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-16T17:44:58.552 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:44:58.593 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-babel. 2024-04-16T17:44:58.609 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-16T17:44:58.651 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:44:58.709 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-routes. 2024-04-16T17:44:58.725 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-16T17:44:58.754 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package xmlstarlet. 2024-04-16T17:44:58.759 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:44:58.770 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-16T17:44:58.821 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:44:58.898 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-16T17:44:58.926 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-16T17:44:58.946 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-jinja2. 2024-04-16T17:44:58.965 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:44:58.973 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-16T17:44:59.012 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:44:59.088 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-16T17:44:59.115 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:59.154 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:59.250 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-test. 2024-04-16T17:44:59.268 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-04-16T17:44:59.278 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../097-ceph-test_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:44:59.295 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-16T17:44:59.307 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-16T17:44:59.316 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:59.334 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:44:59.334 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:44:59.374 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:44:59.678 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-16T17:44:59.705 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-16T17:44:59.761 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-16T17:45:00.064 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-routes. 2024-04-16T17:45:00.091 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-16T17:45:00.111 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-16T17:45:00.130 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:45:00.138 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:00.263 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:00.433 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-16T17:45:00.460 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:00.476 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-16T17:45:00.492 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-16T17:45:00.500 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:00.534 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:45:01.046 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-joblib. 2024-04-16T17:45:01.073 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-16T17:45:01.112 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:45:01.209 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-04-16T17:45:01.236 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-16T17:45:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:01.476 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-16T17:45:01.503 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-16T17:45:01.542 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:45:01.596 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-16T17:45:01.623 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-16T17:45:01.662 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:01.790 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-volume. 2024-04-16T17:45:01.814 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-16T17:45:01.817 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:01.841 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-16T17:45:01.856 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:01.871 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-sklearn. 2024-04-16T17:45:01.880 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:45:01.898 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-16T17:45:01.937 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:45:01.991 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-16T17:45:02.018 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-16T17:45:02.058 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:02.152 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-16T17:45:02.168 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:02.210 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:02.392 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-joblib. 2024-04-16T17:45:02.395 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-04-16T17:45:02.419 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-16T17:45:02.421 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-16T17:45:02.458 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:45:02.461 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-16T17:45:02.530 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-16T17:45:02.558 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-16T17:45:02.588 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:02.655 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-16T17:45:02.683 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:02.757 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-04-16T17:45:02.773 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-16T17:45:02.807 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:02.807 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:02.812 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-16T17:45:02.839 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-16T17:45:02.878 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:45:02.910 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-16T17:45:02.937 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-16T17:45:02.976 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:03.085 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-16T17:45:03.113 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-16T17:45:03.144 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:03.190 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-sklearn. 2024-04-16T17:45:03.216 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-16T17:45:03.256 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:45:03.364 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package nvme-cli. 2024-04-16T17:45:03.391 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-16T17:45:03.430 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:03.430 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-04-16T17:45:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-16T17:45:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-16T17:45:03.778 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cachetools. 2024-04-16T17:45:03.805 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-16T17:45:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-16T17:45:03.836 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-16T17:45:03.844 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:03.852 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pkg-config. 2024-04-16T17:45:03.875 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:03.880 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-16T17:45:03.918 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:03.923 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-16T17:45:03.951 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:04.067 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:04.114 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-16T17:45:04.129 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-16T17:45:04.155 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-16T17:45:04.183 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-16T17:45:04.213 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:04.381 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:04.565 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-16T17:45:04.581 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:04.615 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:04.633 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-16T17:45:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-16T17:45:04.691 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:04.701 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-16T17:45:04.728 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-16T17:45:04.768 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:04.952 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-16T17:45:04.979 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-16T17:45:04.986 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastescript. 2024-04-16T17:45:05.014 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-16T17:45:05.018 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:05.021 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-cachetools. 2024-04-16T17:45:05.048 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-16T17:45:05.079 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rsa. 2024-04-16T17:45:05.087 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:05.106 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-16T17:45:05.135 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:05.146 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-16T17:45:05.355 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-16T17:45:05.382 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:45:05.421 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:05.423 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-16T17:45:05.441 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-google-auth. 2024-04-16T17:45:05.451 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-16T17:45:05.468 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-16T17:45:05.482 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pluggy. 2024-04-16T17:45:05.508 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-16T17:45:05.572 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:05.666 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-04-16T17:45:05.682 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:45:05.716 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:05.794 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-16T17:45:05.803 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-16T17:45:05.821 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-16T17:45:05.826 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-16T17:45:05.860 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:05.861 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:05.884 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-psutil. 2024-04-16T17:45:05.911 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-16T17:45:05.944 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-04-16T17:45:05.950 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:05.971 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-16T17:45:06.010 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:06.148 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-websocket. 2024-04-16T17:45:06.155 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-rsa. 2024-04-16T17:45:06.175 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-16T17:45:06.182 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-16T17:45:06.214 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-16T17:45:06.221 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-16T17:45:06.253 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-py. 2024-04-16T17:45:06.269 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-16T17:45:06.303 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-16T17:45:06.331 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-04-16T17:45:06.358 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-16T17:45:06.397 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:06.534 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-16T17:45:06.542 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-google-auth. 2024-04-16T17:45:06.562 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-16T17:45:06.570 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-16T17:45:06.601 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:06.607 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pygments. 2024-04-16T17:45:06.617 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:06.634 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-16T17:45:06.673 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:06.844 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-04-16T17:45:06.872 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-ceph-test_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:06.919 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:07.070 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-16T17:45:07.097 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-16T17:45:07.137 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:07.178 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-16T17:45:07.205 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-16T17:45:07.252 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:07.407 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-16T17:45:07.415 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-websocket. 2024-04-16T17:45:07.423 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:07.441 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-16T17:45:07.465 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:07.473 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-16T17:45:07.614 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-toml. 2024-04-16T17:45:07.642 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-16T17:45:07.680 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-16T17:45:07.768 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-16T17:45:07.796 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-16T17:45:07.840 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:07.960 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-16T17:45:07.983 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pytest. 2024-04-16T17:45:07.988 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-16T17:45:08.011 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-16T17:45:08.027 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:08.050 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:08.180 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-16T17:45:08.208 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:45:08.247 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:08.453 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplejson. 2024-04-16T17:45:08.481 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-16T17:45:08.507 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package jq. 2024-04-16T17:45:08.523 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:45:08.528 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:08.557 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:08.707 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-16T17:45:08.735 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:08.773 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:08.785 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package socat. 2024-04-16T17:45:08.801 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-16T17:45:08.831 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-16T17:45:08.844 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:08.859 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-16T17:45:08.897 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:09.110 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-16T17:45:09.137 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-16T17:45:09.172 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package xmlstarlet. 2024-04-16T17:45:09.176 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:09.188 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-16T17:45:09.230 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:09.466 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-04-16T17:45:09.488 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-16T17:45:09.494 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:09.515 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:45:09.532 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:09.546 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:09.618 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-test. 2024-04-16T17:45:09.627 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package radosgw. 2024-04-16T17:45:09.634 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../097-ceph-test_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:09.655 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../116-radosgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:09.676 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:09.710 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:09.798 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package jq. 2024-04-16T17:45:09.826 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-16T17:45:09.845 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-16T17:45:09.865 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:09.873 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:09.911 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:10.118 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package socat. 2024-04-16T17:45:10.145 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-16T17:45:10.184 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:10.256 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-16T17:45:10.284 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-16T17:45:10.331 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:10.504 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package xmlstarlet. 2024-04-16T17:45:10.531 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-16T17:45:10.571 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:10.720 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-16T17:45:10.748 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-16T17:45:10.755 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package rbd-fuse. 2024-04-16T17:45:10.771 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:10.795 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:10.813 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:10.909 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-test. 2024-04-16T17:45:10.924 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../097-ceph-test_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:10.975 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:11.115 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:11.199 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-04-16T17:45:11.227 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-16T17:45:11.266 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:11.368 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:45:11.494 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:11.663 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pkg-config. 2024-04-16T17:45:11.691 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-16T17:45:11.729 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:11.754 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-16T17:45:11.914 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-volume. 2024-04-16T17:45:11.931 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:11.973 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:11.974 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-16T17:45:11.990 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-16T17:45:12.033 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:12.083 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-16T17:45:12.310 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-16T17:45:12.338 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:12.376 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:12.428 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-16T17:45:12.444 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-16T17:45:12.478 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:12.499 INFO:teuthology.orchestra.run.smithi083.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-16T17:45:12.571 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:12.696 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-16T17:45:12.724 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-16T17:45:12.763 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:12.781 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-04-16T17:45:12.808 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-16T17:45:12.847 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:13.080 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user cephadm....done 2024-04-16T17:45:13.093 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-16T17:45:13.121 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-16T17:45:13.159 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:13.168 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-04-16T17:45:13.196 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-16T17:45:13.235 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:13.236 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:45:13.263 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package ceph-volume. 2024-04-16T17:45:13.290 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-3089-gb0c7b917-1jammy_all.deb ... 2024-04-16T17:45:13.329 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:13.522 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package nvme-cli. 2024-04-16T17:45:13.546 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:45:13.546 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-04-16T17:45:13.549 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-16T17:45:13.573 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-16T17:45:13.588 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:13.612 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:13.633 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-16T17:45:13.649 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:13.699 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:13.797 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:13.923 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:45:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-04-16T17:45:13.977 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-16T17:45:14.011 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package pkg-config. 2024-04-16T17:45:14.019 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-16T17:45:14.024 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-16T17:45:14.039 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-16T17:45:14.047 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-16T17:45:14.077 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:14.085 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:14.174 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-16T17:45:14.338 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-16T17:45:14.345 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pygments. 2024-04-16T17:45:14.354 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-16T17:45:14.372 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-16T17:45:14.397 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:14.411 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:14.417 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:45:14.423 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-16T17:45:14.451 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-16T17:45:14.490 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:14.518 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-16T17:45:14.794 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package nvme-cli. 2024-04-16T17:45:14.812 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:45:14.822 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-16T17:45:14.842 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-16T17:45:14.866 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-16T17:45:14.869 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:14.870 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-16T17:45:14.893 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-16T17:45:14.908 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:14.940 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:15.170 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pastescript. 2024-04-16T17:45:15.181 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:15.197 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-16T17:45:15.228 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:15.252 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-toml. 2024-04-16T17:45:15.280 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-16T17:45:15.291 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package pkg-config. 2024-04-16T17:45:15.318 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-16T17:45:15.319 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-16T17:45:15.357 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:15.457 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-16T17:45:15.523 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pluggy. 2024-04-16T17:45:15.551 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-16T17:45:15.589 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:15.630 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-04-16T17:45:15.644 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-16T17:45:15.658 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-16T17:45:15.660 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-16T17:45:15.696 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:15.702 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:15.716 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:45:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-psutil. 2024-04-16T17:45:15.929 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-16T17:45:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:15.968 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:45:16.017 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-04-16T17:45:16.045 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-16T17:45:16.084 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:16.164 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-16T17:45:16.191 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-16T17:45:16.202 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:16.230 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:16.296 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-py. 2024-04-16T17:45:16.323 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-16T17:45:16.353 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-16T17:45:16.362 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-16T17:45:16.381 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-16T17:45:16.420 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:16.445 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:45:16.499 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pastescript. 2024-04-16T17:45:16.527 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-16T17:45:16.557 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:16.682 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pygments. 2024-04-16T17:45:16.710 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-16T17:45:16.713 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:16.749 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:16.819 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pluggy. 2024-04-16T17:45:16.835 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-16T17:45:16.877 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:16.989 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:45:17.116 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:17.175 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-04-16T17:45:17.189 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-psutil. 2024-04-16T17:45:17.202 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../116-radosgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:17.217 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-16T17:45:17.241 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:45:17.249 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:17.254 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-16T17:45:17.255 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:17.282 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-16T17:45:17.320 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:17.367 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:45:17.602 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:45:17.606 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-toml. 2024-04-16T17:45:17.608 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-py. 2024-04-16T17:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-16T17:45:17.636 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-16T17:45:17.681 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-16T17:45:17.683 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-16T17:45:17.968 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pytest. 2024-04-16T17:45:17.977 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-16T17:45:17.995 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-16T17:45:18.008 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-16T17:45:18.034 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:18.053 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-04-16T17:45:18.072 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:45:18.080 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:18.119 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:18.198 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:18.220 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pygments. 2024-04-16T17:45:18.248 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-16T17:45:18.286 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:18.371 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-simplejson. 2024-04-16T17:45:18.390 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:18.399 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-16T17:45:18.429 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:18.558 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:18.625 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:45:18.657 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-16T17:45:18.673 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-16T17:45:18.684 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-16T17:45:18.707 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:18.732 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-16T17:45:18.751 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:18.760 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-16T17:45:18.798 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:18.952 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:45:19.013 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-16T17:45:19.118 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-toml. 2024-04-16T17:45:19.146 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-16T17:45:19.185 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-16T17:45:19.194 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:19.292 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-16T17:45:19.445 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package radosgw. 2024-04-16T17:45:19.446 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-pytest. 2024-04-16T17:45:19.462 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-16T17:45:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../116-radosgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:19.487 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up zip (3.0-12build2) ... 2024-04-16T17:45:19.495 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:19.622 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:19.721 INFO:teuthology.orchestra.run.smithi122.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-16T17:45:19.753 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:19.778 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:19.941 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package python3-simplejson. 2024-04-16T17:45:19.969 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-16T17:45:20.016 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:20.282 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:45:20.352 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-16T17:45:20.370 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-04-16T17:45:20.380 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-16T17:45:20.427 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:20.533 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:45:20.543 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:45:20.598 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package rbd-fuse. 2024-04-16T17:45:20.626 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:20.643 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:20.664 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:20.744 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:45:20.820 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:45:20.969 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:21.020 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-16T17:45:21.073 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:21.197 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:45:21.211 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:45:21.231 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package radosgw. 2024-04-16T17:45:21.259 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../116-radosgw_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:21.305 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:21.337 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:21.413 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-16T17:45:21.457 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-16T17:45:21.600 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-16T17:45:21.664 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:45:21.708 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:45:21.818 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-16T17:45:21.945 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-16T17:45:22.066 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:45:22.120 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:45:22.134 INFO:teuthology.orchestra.run.smithi188.stdout:Selecting previously unselected package rbd-fuse. 2024-04-16T17:45:22.150 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-3089-gb0c7b917-1jammy_amd64.deb ... 2024-04-16T17:45:22.192 INFO:teuthology.orchestra.run.smithi188.stdout:Unpacking rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:22.345 INFO:teuthology.orchestra.run.smithi071.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-16T17:45:22.358 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:45:22.488 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-16T17:45:22.497 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:22.623 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:22.718 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:45:22.722 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-16T17:45:22.765 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-16T17:45:22.848 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-16T17:45:22.952 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:23.042 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:45:23.070 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:23.121 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-16T17:45:23.124 INFO:teuthology.orchestra.run.smithi071.stdout:Adding system user cephadm....done 2024-04-16T17:45:23.280 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:45:23.285 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:45:23.372 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-idna (3.3-1) ... 2024-04-16T17:45:23.452 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-16T17:45:23.531 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:45:23.536 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:23.690 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:45:23.775 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:23.796 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:45:23.876 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:45:23.908 INFO:teuthology.orchestra.run.smithi188.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-16T17:45:23.975 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:24.047 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:24.127 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-16T17:45:24.378 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:45:24.390 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:45:24.458 INFO:teuthology.orchestra.run.smithi188.stdout:Adding system user cephadm....done 2024-04-16T17:45:24.504 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-16T17:45:24.516 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:24.606 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-16T17:45:24.755 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:45:24.815 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:45:24.883 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-16T17:45:24.920 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:25.069 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:45:25.134 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-16T17:45:25.167 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:25.172 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:25.260 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-16T17:45:25.290 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-16T17:45:25.353 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:45:25.418 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-16T17:45:25.511 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-16T17:45:25.533 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:45:25.651 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:25.686 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:45:25.729 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-16T17:45:25.754 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-16T17:45:25.761 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-16T17:45:25.776 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:45:25.823 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:45:25.897 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-16T17:45:25.946 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:45:25.952 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:26.070 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:26.189 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:26.190 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-16T17:45:26.311 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:26.329 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-16T17:45:26.421 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-16T17:45:26.431 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:45:26.551 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-16T17:45:26.572 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:26.672 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:45:26.673 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:26.775 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:26.810 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-16T17:45:26.931 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:26.966 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:45:27.093 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:45:27.103 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-16T17:45:27.202 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-16T17:45:27.219 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:27.224 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up zip (3.0-12build2) ... 2024-04-16T17:45:27.345 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:45:27.346 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-16T17:45:27.350 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:27.460 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-16T17:45:27.471 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:45:27.614 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-16T17:45:27.714 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:45:27.720 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:45:27.873 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-16T17:45:27.971 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-16T17:45:28.011 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:45:28.086 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-16T17:45:28.120 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-16T17:45:28.124 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-ceph-argparse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:28.176 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:45:28.255 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:45:28.261 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:45:28.302 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-16T17:45:28.388 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:28.522 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:45:28.624 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:28.662 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:28.676 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-16T17:45:28.725 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:45:28.788 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-16T17:45:28.801 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-16T17:45:28.823 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-16T17:45:28.911 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-16T17:45:29.010 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:45:29.062 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-16T17:45:29.064 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:45:29.207 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-16T17:45:29.311 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:45:29.315 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:29.330 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-16T17:45:29.459 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:45:29.579 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:45:29.608 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up zip (3.0-12build2) ... 2024-04-16T17:45:29.713 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:29.734 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:29.748 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-16T17:45:29.778 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-16T17:45:29.827 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:45:29.839 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:29.842 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-16T17:45:29.928 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-16T17:45:29.993 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-16T17:45:30.112 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:45:30.149 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:30.276 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:30.369 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:30.411 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:45:30.438 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:45:30.487 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-16T17:45:30.671 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:45:30.680 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:45:30.689 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:30.740 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-16T17:45:30.792 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:30.824 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:30.890 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:45:30.937 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-16T17:45:30.989 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-16T17:45:31.063 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-16T17:45:31.125 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-idna (3.3-1) ... 2024-04-16T17:45:31.175 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-16T17:45:31.291 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up zip (3.0-12build2) ... 2024-04-16T17:45:31.322 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:31.416 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-16T17:45:31.417 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:45:31.448 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:45:31.568 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-16T17:45:31.774 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:31.819 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:45:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:45:32.110 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-16T17:45:32.220 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:45:32.343 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:32.369 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-16T17:45:32.469 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-16T17:45:32.470 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-16T17:45:32.505 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:45:32.580 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-16T17:45:32.615 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:32.754 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-16T17:45:32.840 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:45:32.872 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-16T17:45:32.875 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:33.009 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-16T17:45:33.038 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:33.075 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:33.201 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:33.260 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-16T17:45:33.268 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:45:33.386 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:33.400 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:45:33.494 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-idna (3.3-1) ... 2024-04-16T17:45:33.495 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:45:33.510 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-16T17:45:33.780 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:33.793 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:33.811 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:45:33.861 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-16T17:45:34.031 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-16T17:45:34.086 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:45:34.138 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-16T17:45:34.274 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:34.384 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:34.456 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:45:34.481 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-16T17:45:34.582 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:34.694 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:45:34.708 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:34.740 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-16T17:45:34.859 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:45:34.866 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:34.953 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-16T17:45:35.001 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:35.168 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-idna (3.3-1) ... 2024-04-16T17:45:35.196 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:35.229 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-16T17:45:35.253 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:35.314 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:35.388 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-16T17:45:35.415 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:35.480 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:45:35.494 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-16T17:45:35.639 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:45:35.674 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:35.731 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-16T17:45:35.739 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:35.776 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:35.849 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:45:35.885 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:35.991 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:45:36.101 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:36.102 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:36.116 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-16T17:45:36.143 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:36.409 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:36.419 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-16T17:45:36.544 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:45:36.586 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:36.632 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-16T17:45:36.670 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:36.712 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:36.796 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:36.836 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:45:36.847 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:36.909 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:36.973 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:37.069 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-16T17:45:37.114 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:45:37.129 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:45:37.357 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-16T17:45:37.361 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-16T17:45:37.380 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:45:37.506 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:37.512 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-16T17:45:37.617 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-16T17:45:37.634 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:37.688 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-16T17:45:37.721 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-16T17:45:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:45:37.951 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:37.998 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-16T17:45:38.085 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:38.128 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-16T17:45:38.219 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:38.282 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-16T17:45:38.370 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:38.387 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:45:38.470 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:45:38.495 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:38.513 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-16T17:45:38.542 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:38.613 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:38.715 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:38.721 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-16T17:45:38.789 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:38.847 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-16T17:45:38.871 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-16T17:45:38.899 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:45:39.106 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:39.184 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:45:39.203 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-16T17:45:39.233 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:45:39.241 INFO:teuthology.orchestra.run.smithi083.stdout:Adding group ceph....done 2024-04-16T17:45:39.479 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-16T17:45:39.485 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:45:39.550 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:39.626 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user ceph....done 2024-04-16T17:45:39.736 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:45:39.754 INFO:teuthology.orchestra.run.smithi083.stdout:Setting system user ceph properties....done 2024-04-16T17:45:39.755 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-16T17:45:39.768 INFO:teuthology.orchestra.run.smithi083.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-16T17:45:39.843 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:45:39.862 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:39.903 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-16T17:45:39.988 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:40.031 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-16T17:45:40.076 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-16T17:45:40.120 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:40.240 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-16T17:45:40.298 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:40.299 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-16T17:45:40.414 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-16T17:45:40.424 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:40.566 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-16T17:45:40.709 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-16T17:45:40.766 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:40.771 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:41.002 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-16T17:45:41.039 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:41.069 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-16T17:45:41.153 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-16T17:45:41.195 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-16T17:45:41.384 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:45:41.385 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:41.385 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:41.454 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:41.479 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-16T17:45:41.556 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:45:41.778 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:41.797 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-16T17:45:41.857 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:41.995 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:42.054 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:45:42.064 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-16T17:45:42.175 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:45:42.191 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-16T17:45:42.325 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-16T17:45:42.361 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-16T17:45:42.381 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:45:42.415 INFO:teuthology.orchestra.run.smithi188.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-16T17:45:42.451 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:42.482 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:42.578 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-16T17:45:42.600 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:42.677 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-16T17:45:42.726 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:45:42.828 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-16T17:45:42.853 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-16T17:45:42.872 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.063 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.105 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:43.107 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:43.107 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:43.189 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.238 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-16T17:45:43.298 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:43.450 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:45:43.505 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-16T17:45:43.558 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.582 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.648 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-16T17:45:43.684 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.796 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:43.797 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:43.810 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:43.835 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:43.916 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-16T17:45:44.023 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:44.023 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:44.075 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-16T17:45:44.120 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:45:44.283 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:44.393 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-16T17:45:44.456 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:45:44.495 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:44.560 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:44.560 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:44.576 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:44.621 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:44.719 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:44.719 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-16T17:45:44.747 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:44.845 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:45:44.873 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:45.029 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-16T17:45:45.035 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.114 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.155 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-16T17:45:45.161 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.241 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.329 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.367 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:45.430 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-16T17:45:45.547 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:45.548 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:45.652 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.715 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-16T17:45:45.778 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:45.904 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.030 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.077 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-16T17:45:46.112 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:46.112 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:46.118 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.173 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.244 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.274 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.345 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.408 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.446 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.470 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-16T17:45:46.564 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.588 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.698 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.772 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-16T17:45:46.816 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:46.841 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-16T17:45:46.942 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:46.958 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-16T17:45:47.082 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-04-16T17:45:47.102 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-16T17:45:47.124 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-16T17:45:47.283 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librados2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:47.435 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:47.475 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-04-16T17:45:47.585 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-16T17:45:47.634 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-04-16T17:45:47.652 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-16T17:45:47.792 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-16T17:45:47.938 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libcephfs2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.064 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libradosstriper1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.096 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.127 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-16T17:45:48.170 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-16T17:45:48.197 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.198 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-16T17:45:48.315 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.441 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.492 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librbd1 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.567 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.582 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.635 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-modules-core (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-16T17:45:48.826 DEBUG:teuthology.parallel:result is None 2024-04-16T17:45:49.009 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:49.009 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-16T17:45:49.078 INFO:teuthology.orchestra.run.smithi071.stdout:Adding group ceph....done 2024-04-16T17:45:49.194 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:49.194 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:49.421 INFO:teuthology.orchestra.run.smithi071.stdout:Adding system user ceph....done 2024-04-16T17:45:49.512 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up libcephfs-dev (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:49.598 INFO:teuthology.orchestra.run.smithi071.stdout:Setting system user ceph properties....done 2024-04-16T17:45:49.609 INFO:teuthology.orchestra.run.smithi071.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-16T17:45:49.680 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rados (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:49.684 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:49.744 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-16T17:45:49.814 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up librgw2 (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:49.974 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-16T17:45:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-16T17:45:50.186 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-16T17:45:50.629 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:50.803 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:50.850 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:50.850 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:50.938 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.220 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rbd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.326 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.346 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up rbd-fuse (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.415 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:51.415 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:51.472 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-rgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.544 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:51.544 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:51.598 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up python3-cephfs (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.749 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-common (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.890 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:51.986 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:52.238 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-16T17:45:52.253 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:52.253 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:52.384 INFO:teuthology.orchestra.run.smithi188.stdout:Adding group ceph....done 2024-04-16T17:45:52.688 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:52.717 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:52.759 INFO:teuthology.orchestra.run.smithi188.stdout:Adding system user ceph....done 2024-04-16T17:45:52.797 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:52.903 INFO:teuthology.orchestra.run.smithi188.stdout:Setting system user ceph properties....done 2024-04-16T17:45:52.915 INFO:teuthology.orchestra.run.smithi188.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-16T17:45:52.940 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:52.953 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:52.953 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:53.052 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-16T17:45:53.156 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:53.156 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:53.420 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-16T17:45:53.444 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:53.624 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:53.638 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:53.638 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:53.767 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:53.893 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:53.908 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-test (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:54.053 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:54.059 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up radosgw (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:54.087 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:54.337 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:54.338 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:54.389 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-16T17:45:54.498 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-16T17:45:54.628 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:54.628 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-16T17:45:54.797 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:54.842 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-16T17:45:54.923 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:55.074 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:55.207 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-base (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:55.294 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:55.294 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:55.533 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-16T17:45:55.567 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-16T17:45:55.776 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:55.978 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:56.079 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:56.096 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mds (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:56.214 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:56.225 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-16T17:45:56.230 DEBUG:teuthology.parallel:result is None 2024-04-16T17:45:56.404 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:56.404 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-16T17:45:56.566 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-16T17:45:56.659 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-16T17:45:56.795 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-16T17:45:56.928 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:57.146 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:57.146 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-16T17:45:57.646 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-osd (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:57.932 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:57.932 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-16T17:45:57.938 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-16T17:45:58.456 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:58.565 INFO:teuthology.orchestra.run.smithi071.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-16T17:45:58.570 DEBUG:teuthology.parallel:result is None 2024-04-16T17:45:58.591 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:58.742 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mon (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:58.953 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:58.953 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-16T17:45:59.443 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-cephadm (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:59.619 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:45:59.770 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-mgr-dashboard (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:46:00.400 INFO:teuthology.orchestra.run.smithi188.stdout:Setting up ceph-volume (19.0.0-3089-gb0c7b917-1jammy) ... 2024-04-16T17:46:00.762 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-16T17:46:00.862 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-16T17:46:01.032 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-16T17:46:01.798 INFO:teuthology.orchestra.run.smithi188.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-16T17:46:02.441 INFO:teuthology.orchestra.run.smithi188.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-16T17:46:02.446 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:02.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:46:02.596 DEBUG:teuthology.orchestra.run.smithi071:> dpkg-query -W -f '${Version}' ceph 2024-04-16T17:46:02.615 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:02.615 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:02.615 INFO:teuthology.task.install:The correct ceph version 19.0.0-3089-gb0c7b917-1jammy is installed. 2024-04-16T17:46:02.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:46:02.770 DEBUG:teuthology.orchestra.run.smithi083:> dpkg-query -W -f '${Version}' ceph 2024-04-16T17:46:02.790 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:02.790 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:02.790 INFO:teuthology.task.install:The correct ceph version 19.0.0-3089-gb0c7b917-1jammy is installed. 2024-04-16T17:46:02.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:46:03.008 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-04-16T17:46:03.027 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:03.028 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:03.028 INFO:teuthology.task.install:The correct ceph version 19.0.0-3089-gb0c7b917-1jammy is installed. 2024-04-16T17:46:03.029 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b0c7b917267c471208541fdeb652f5e897d00086 2024-04-16T17:46:03.170 DEBUG:teuthology.orchestra.run.smithi188:> dpkg-query -W -f '${Version}' ceph 2024-04-16T17:46:03.189 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:03.189 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3089-gb0c7b917-1jammy 2024-04-16T17:46:03.189 INFO:teuthology.task.install:The correct ceph version 19.0.0-3089-gb0c7b917-1jammy is installed. 2024-04-16T17:46:03.191 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T17:46:03.191 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:03.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T17:46:03.206 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:03.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T17:46:03.220 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:03.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T17:46:03.236 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:03.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T17:46:03.251 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T17:46:03.251 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:03.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T17:46:03.264 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T17:46:03.317 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:03.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T17:46:03.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T17:46:03.385 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:03.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T17:46:03.398 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T17:46:03.455 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:03.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T17:46:03.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T17:46:03.526 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T17:46:03.526 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:03.526 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T17:46:03.539 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T17:46:03.593 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:03.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T17:46:03.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T17:46:03.661 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:03.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T17:46:03.675 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T17:46:03.731 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:03.731 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T17:46:03.745 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T17:46:03.798 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T17:46:03.798 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:03.798 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T17:46:03.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T17:46:03.864 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:03.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T17:46:03.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T17:46:03.933 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:03.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T17:46:03.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T17:46:04.003 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:04.003 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T17:46:04.016 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T17:46:04.070 INFO:teuthology.run_tasks:Running task ceph... 2024-04-16T17:46:04.153 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-16T17:46:04.153 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /var/log/ceph 2024-04-16T17:46:04.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-04-16T17:46:04.158 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-04-16T17:46:04.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-04-16T17:46:04.172 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-16T17:46:04.172 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T17:46:04.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T17:46:04.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T17:46:04.216 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T17:46:04.229 INFO:tasks.ceph:Creating extra log directories... 2024-04-16T17:46:04.229 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T17:46:04.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T17:46:04.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T17:46:04.273 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T17:46:04.297 INFO:tasks.ceph:Creating ceph cluster c1... 2024-04-16T17:46:04.297 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'c1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-16T17:46:04.298 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'branch': 'wip-shilpa-multisite-tests', 'description': 'rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi}', 'email': None, 'first_in_suite': False, 'job_id': '7658456', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-shilpa-multisite-tests'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'enabled_features': ['resharding', 'notification_v2'], 'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}]}]}, 'rgw-multisite-tests': {'args': ['tests.py', '-a', '!fails_on_rgw']}, 'workunit': {'branch': 'wip-shilpa-multisite-tests', 'sha1': '9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b'}}, 'owner': 'scheduled_smanjara@teuthology', 'priority': 49, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['c1.mon.a', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'], ['c1.mgr.x', 'c1.osd.3', 'c1.osd.4', 'c1.osd.5', 'c1.client.1'], ['c2.mon.a', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0'], ['c2.mgr.x', 'c2.osd.3', 'c2.osd.4', 'c2.osd.5', 'c2.client.1']], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086', 'sleep_before_teardown': 0, 'suite': 'rgw:multisite', 'suite_branch': 'wip-shilpa-multisite-tests', 'suite_path': '/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/smanjara/ceph.git', 'suite_sha1': '9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b', 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9vtxKrRme6/isUwEX2lhl8Ybr0T6XOkffrdu3KUaVOAXLP5Y37Tmkgxpc7kst406V1WRz+VK6xeEGL3E8XHEI=', 'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6BfUv5py4V6x14q9+tGyc4AQrymyRKyAz6y6sP+C7i7TH6fqhkepIeAS1YXpmNgRSN28Z6rVAE8v6vXjqqcYQ=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN7+QuOyR1DYPk/GC5c9CS7zlOEbjs3gZpYoCAodd4KN3mbnad1OmBxBuc9eY060s6zeyO+Ul/4sy25LGzWxW6s=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUJ9KsHPOwr6bVTpKPWxuVw2M8nCWXz8Wb3iPZ0uBK78ORQ5r/LsER0oB/sFQ+EsO4buhZ2vscDOaYUzUJAiJM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}}, {'ceph': {'cluster': 'c2'}}, {'rgw': {'c1.client.0': {'port': 8000}, 'c1.client.1': {'port': 8001}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 90}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-16_17:28:09', 'tube': 'smithi', 'user': 'smanjara', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-16T17:46:04.298 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2024-04-16T17:46:04.319 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2024-04-16T17:46:04.323 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2024-04-16T17:46:04.325 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2024-04-16T17:46:04.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:04.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:04.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:04.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:04.394 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:04.394 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T17:46:04.427 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T17:46:04.427 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-04-16T17:46:04.474 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 869 Links: 1 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 17:40:33.396432332 +0000 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 17:40:33.136434654 +0000 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 17:40:33.136434654 +0000 2024-04-16T17:46:04.475 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-16T17:46:04.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T17:46:04.528 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T17:46:04.529 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T17:46:04.529 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000450695 s, 1.1 MB/s 2024-04-16T17:46:04.530 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T17:46:04.579 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 894 Links: 1 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 17:40:33.620430331 +0000 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 17:40:33.616430367 +0000 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 17:40:33.616430367 +0000 2024-04-16T17:46:04.626 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-16T17:46:04.627 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T17:46:04.679 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T17:46:04.680 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T17:46:04.680 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000304073 s, 1.7 MB/s 2024-04-16T17:46:04.681 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T17:46:04.727 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 17:40:34.416423221 +0000 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 17:40:34.136425721 +0000 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 17:40:34.136425721 +0000 2024-04-16T17:46:04.774 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-16T17:46:04.775 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T17:46:04.828 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T17:46:04.828 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T17:46:04.828 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000324381 s, 1.6 MB/s 2024-04-16T17:46:04.829 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T17:46:04.879 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-04-16T17:46:04.926 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T17:46:04.926 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:04.926 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-16T17:46:04.927 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:04.927 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 17:40:34.904418862 +0000 2024-04-16T17:46:04.927 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 17:40:34.636421256 +0000 2024-04-16T17:46:04.927 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 17:40:34.636421256 +0000 2024-04-16T17:46:04.927 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-16T17:46:04.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T17:46:04.982 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T17:46:04.982 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T17:46:04.982 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000401404 s, 1.3 MB/s 2024-04-16T17:46:04.983 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T17:46:05.031 INFO:tasks.ceph:osd dev map: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:05.031 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:05.031 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T17:46:05.037 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T17:46:05.037 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-16T17:46:05.086 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-16 17:40:34.622422919 +0000 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-16 17:40:34.370425169 +0000 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-16 17:40:34.370425169 +0000 2024-04-16T17:46:05.087 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-16T17:46:05.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T17:46:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-16T17:46:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-16T17:46:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000441441 s, 1.2 MB/s 2024-04-16T17:46:05.143 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T17:46:05.191 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-16T17:46:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T17:46:05.238 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-04-16T17:46:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:05.238 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-16 17:40:34.866420739 +0000 2024-04-16T17:46:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-16 17:40:34.862420775 +0000 2024-04-16T17:46:05.239 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-16 17:40:34.862420775 +0000 2024-04-16T17:46:05.239 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-16T17:46:05.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T17:46:05.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-16T17:46:05.292 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-16T17:46:05.293 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00041204 s, 1.2 MB/s 2024-04-16T17:46:05.294 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T17:46:05.343 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-16T17:46:05.389 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-16 17:40:35.634413880 +0000 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-16 17:40:35.362416309 +0000 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-16 17:40:35.362416309 +0000 2024-04-16T17:46:05.390 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-16T17:46:05.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T17:46:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-16T17:46:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-16T17:46:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000438431 s, 1.2 MB/s 2024-04-16T17:46:05.446 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T17:46:05.495 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-16T17:46:05.541 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T17:46:05.541 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-04-16T17:46:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-16 17:40:36.130409449 +0000 2024-04-16T17:46:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-16 17:40:35.862411842 +0000 2024-04-16T17:46:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-16 17:40:35.862411842 +0000 2024-04-16T17:46:05.542 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-16T17:46:05.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T17:46:05.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-16T17:46:05.596 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-16T17:46:05.596 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.0003513 s, 1.5 MB/s 2024-04-16T17:46:05.597 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T17:46:05.647 INFO:tasks.ceph:osd dev map: {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:05.647 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'}} 2024-04-16T17:46:05.647 INFO:tasks.ceph:Generating config... 2024-04-16T17:46:05.648 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] debug rgw lifecycle = 20 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] debug rgw notification = 20 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] debug rgw sync = 20 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw data log num shards = 4 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw data sync poll interval = 5 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw md log max shards = 4 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw meta sync poll interval = 5 2024-04-16T17:46:05.649 INFO:tasks.ceph:[client] rgw sync data inject err probability = 0.1 2024-04-16T17:46:05.650 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2024-04-16T17:46:05.650 INFO:tasks.ceph:[client] rgw sync meta inject err probability = 0.1 2024-04-16T17:46:05.650 INFO:tasks.ceph:[client] rgw sync obj etag verify = True 2024-04-16T17:46:05.650 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-16T17:46:05.650 INFO:tasks.ceph:[client] setuser = ceph 2024-04-16T17:46:05.650 INFO:tasks.ceph:[client.0] rgw data notify interval msec = 0 2024-04-16T17:46:05.650 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-16T17:46:05.650 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-16T17:46:05.650 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-16T17:46:05.650 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-16T17:46:05.650 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-16T17:46:05.650 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-16T17:46:05.650 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-16T17:46:05.650 INFO:tasks.ceph:[osd] rocksdb delete range threshold = 0 2024-04-16T17:46:05.651 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2024-04-16T17:46:05.651 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:05.651 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:05.651 INFO:tasks.ceph:Setting up c1.mon.a... 2024-04-16T17:46:05.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c1.keyring 2024-04-16T17:46:05.685 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/c1.keyring 2024-04-16T17:46:05.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c1.keyring 2024-04-16T17:46:05.782 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/c1.keyring 2024-04-16T17:46:05.837 DEBUG:tasks.ceph:Ceph mon addresses: [('c1.mon.a', '172.21.15.71')] 2024-04-16T17:46:05.837 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.71'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'client.0': {'rgw data notify interval msec': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}} 2024-04-16T17:46:05.838 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:05.838 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T17:46:05.886 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.71 --print /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:05.965 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:46:05.965 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:46:05.965 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: generated fsid 5a58eb97-56a6-45c2-947d-e09bf8e2f67b 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 0 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:fsid 5a58eb97-56a6-45c2-947d-e09bf8e2f67b 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:last_changed 2024-04-16T17:46:05.971132+0000 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-16T17:46:05.971132+0000 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:min_mon_release 17 (quincy) 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:election_strategy: 1 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-04-16T17:46:05.966 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c1.monmap (1 monitors) 2024-04-16T17:46:05.967 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T17:46:06.014 INFO:tasks.ceph:Writing /etc/ceph/c1.conf for FSID 5a58eb97-56a6-45c2-947d-e09bf8e2f67b... 2024-04-16T17:46:06.015 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2024-04-16T17:46:06.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2024-04-16T17:46:06.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2024-04-16T17:46:06.064 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2024-04-16T17:46:06.088 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-16T17:46:06.088 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-16T17:46:06.088 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:06.088 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-16T17:46:06.088 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.089 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:06.090 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = 172.21.15.71 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5a58eb97-56a6-45c2-947d-e09bf8e2f67b 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-16T17:46:06.091 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-16T17:46:06.092 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-16T17:46:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-16T17:46:06.094 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:06.095 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw = 20 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:06.096 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw notification = 20 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw sync = 20 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw crypt require ssl = False 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw curl low speed time = 300 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw data log num shards = 4 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw md log max shards = 4 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: setgroup = ceph 2024-04-16T17:46:06.097 INFO:teuthology.orchestra.run.smithi083.stdout: setuser = ceph 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-04-16T17:46:06.098 INFO:teuthology.orchestra.run.smithi083.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:06.099 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-16T17:46:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-16T17:46:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-16T17:46:06.099 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.100 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.101 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = 172.21.15.71 2024-04-16T17:46:06.102 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 5a58eb97-56a6-45c2-947d-e09bf8e2f67b 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-16T17:46:06.103 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:06.104 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.105 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.106 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-16T17:46:06.107 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw = 20 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw notification = 20 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw sync = 20 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw crypt require ssl = False 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw curl low speed time = 300 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw data log num shards = 4 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw md log max shards = 4 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:06.108 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: setgroup = ceph 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: setuser = ceph 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:06.109 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.110 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:06.111 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-04-16T17:46:06.112 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = 172.21.15.71 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 5a58eb97-56a6-45c2-947d-e09bf8e2f67b 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-04-16T17:46:06.113 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-04-16T17:46:06.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-04-16T17:46:06.115 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-04-16T17:46:06.116 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:06.117 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw = 20 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw notification = 20 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw sync = 20 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw crypt require ssl = False 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: rgw curl low speed time = 300 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw data log num shards = 4 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw md log max shards = 4 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: setgroup = ceph 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: setuser = ceph 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:06.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-04-16T17:46:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:06.120 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-04-16T17:46:06.120 INFO:teuthology.orchestra.run.smithi122.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:06.121 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:06.122 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = 172.21.15.71 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = 5a58eb97-56a6-45c2-947d-e09bf8e2f67b 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2024-04-16T17:46:06.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug trim objects = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2024-04-16T17:46:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2024-04-16T17:46:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2024-04-16T17:46:06.127 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.128 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw = 20 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw notification = 20 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw sync = 20 2024-04-16T17:46:06.129 INFO:teuthology.orchestra.run.smithi071.stdout: rgw crypt require ssl = False 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw curl low speed time = 300 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw data log num shards = 4 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw md log max shards = 4 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: setgroup = ceph 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout: setuser = ceph 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2024-04-16T17:46:06.130 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-04-16T17:46:06.131 INFO:teuthology.orchestra.run.smithi071.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:06.132 INFO:tasks.ceph:Creating admin key on c1.mon.a... 2024-04-16T17:46:06.133 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c1.keyring 2024-04-16T17:46:06.214 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-16T17:46:06.214 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:06.214 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/c1.keyring of=/dev/stdout 2024-04-16T17:46:06.219 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:06.219 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/c1.monmap of=/dev/stdout 2024-04-16T17:46:06.266 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T17:46:06.266 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:06.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/c1.keyring 2024-04-16T17:46:06.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/c1.keyring 2024-04-16T17:46:06.331 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:06.331 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:06.378 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T17:46:06.378 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:06.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/c1.keyring 2024-04-16T17:46:06.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/c1.keyring 2024-04-16T17:46:06.402 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:06.402 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:06.450 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T17:46:06.450 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:06.450 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/c1.keyring 2024-04-16T17:46:06.450 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/c1.keyring 2024-04-16T17:46:06.475 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:06.475 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:06.524 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T17:46:06.524 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:06.524 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/c1.keyring 2024-04-16T17:46:06.524 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/c1.keyring 2024-04-16T17:46:06.546 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:06.546 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:06.595 INFO:tasks.ceph:Setting up mon nodes... 2024-04-16T17:46:06.595 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-16T17:46:06.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/c1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c1-x/keyring 2024-04-16T17:46:06.638 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/c1-x/keyring 2024-04-16T17:46:06.649 INFO:tasks.ceph:Setting up mds nodes... 2024-04-16T17:46:06.649 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-16T17:46:06.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c1.client.0.keyring && sudo chmod 0644 /etc/ceph/c1.client.0.keyring 2024-04-16T17:46:06.686 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/c1.client.0.keyring 2024-04-16T17:46:06.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c1.client.1.keyring && sudo chmod 0644 /etc/ceph/c1.client.1.keyring 2024-04-16T17:46:06.732 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/c1.client.1.keyring 2024-04-16T17:46:06.743 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-16T17:46:06.743 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'}} 2024-04-16T17:46:06.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/c1-0 2024-04-16T17:46:06.757 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:06.757 INFO:tasks.ceph:role: c1.osd.0 2024-04-16T17:46:06.757 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T17:46:06.757 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T17:46:06.824 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:06.824 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:06.824 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:06.825 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:07.297 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T17:46:07.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T17:46:07.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c1-0 2024-04-16T17:46:07.443 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-0 2024-04-16T17:46:07.456 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:07.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:07.457 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/c1-1 2024-04-16T17:46:07.514 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:07.514 INFO:tasks.ceph:role: c1.osd.1 2024-04-16T17:46:07.514 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T17:46:07.515 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:07.577 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:07.578 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:07.999 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T17:46:08.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T17:46:08.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c1-1 2024-04-16T17:46:08.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-1 2024-04-16T17:46:08.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:08.120 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:08.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/c1-2 2024-04-16T17:46:08.173 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:08.173 INFO:tasks.ceph:role: c1.osd.2 2024-04-16T17:46:08.173 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T17:46:08.174 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:08.238 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:08.704 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T17:46:08.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T17:46:08.706 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c1-2 2024-04-16T17:46:08.797 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-2 2024-04-16T17:46:08.808 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:08.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:08.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:08.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:08.931+0000 7fef198e67c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-0/keyring: can't open /var/lib/ceph/osd/c1-0/keyring: (2) No such file or directory 2024-04-16T17:46:08.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:08.935+0000 7fef198e67c0 -1 created new key in keyring /var/lib/ceph/osd/c1-0/keyring 2024-04-16T17:46:08.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:08.935+0000 7fef198e67c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2024-04-16T17:46:08.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:08.935+0000 7fef198e67c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2024-04-16T17:46:08.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:08.935+0000 7fef198e67c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2024-04-16T17:46:08.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:08.935+0000 7fef198e67c0 -1 bluestore(/var/lib/ceph/osd/c1-0) _read_fsid unparsable uuid 2024-04-16T17:46:11.102 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-0 2024-04-16T17:46:11.116 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:11.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:11.199+0000 7f08c37f87c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-1/keyring: can't open /var/lib/ceph/osd/c1-1/keyring: (2) No such file or directory 2024-04-16T17:46:11.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:11.199+0000 7f08c37f87c0 -1 created new key in keyring /var/lib/ceph/osd/c1-1/keyring 2024-04-16T17:46:11.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:11.199+0000 7f08c37f87c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2024-04-16T17:46:11.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:11.199+0000 7f08c37f87c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2024-04-16T17:46:11.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:11.199+0000 7f08c37f87c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2024-04-16T17:46:11.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:11.199+0000 7f08c37f87c0 -1 bluestore(/var/lib/ceph/osd/c1-1) _read_fsid unparsable uuid 2024-04-16T17:46:13.380 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-1 2024-04-16T17:46:13.435 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:13.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:13.519+0000 7f07d2e6e7c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-2/keyring: can't open /var/lib/ceph/osd/c1-2/keyring: (2) No such file or directory 2024-04-16T17:46:13.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:13.519+0000 7f07d2e6e7c0 -1 created new key in keyring /var/lib/ceph/osd/c1-2/keyring 2024-04-16T17:46:13.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:13.519+0000 7f07d2e6e7c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2024-04-16T17:46:13.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:13.519+0000 7f07d2e6e7c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2024-04-16T17:46:13.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:13.519+0000 7f07d2e6e7c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2024-04-16T17:46:13.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:46:13.519+0000 7f07d2e6e7c0 -1 bluestore(/var/lib/ceph/osd/c1-2) _read_fsid unparsable uuid 2024-04-16T17:46:15.697 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-2 2024-04-16T17:46:15.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/c1-3 2024-04-16T17:46:15.765 INFO:tasks.ceph:roles_to_devs: {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:15.765 INFO:tasks.ceph:role: c1.osd.3 2024-04-16T17:46:15.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T17:46:15.765 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T17:46:15.828 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:15.828 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:15.828 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:15.828 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:15.828 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:15.829 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:15.829 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:15.829 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:15.829 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:15.829 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:16.272 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-16T17:46:16.274 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-16T17:46:16.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c1-3 2024-04-16T17:46:16.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-3 2024-04-16T17:46:16.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:16.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:16.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/c1-4 2024-04-16T17:46:16.427 INFO:tasks.ceph:roles_to_devs: {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:16.427 INFO:tasks.ceph:role: c1.osd.4 2024-04-16T17:46:16.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T17:46:16.427 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:16.493 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:16.963 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-16T17:46:16.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-16T17:46:16.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c1-4 2024-04-16T17:46:17.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-4 2024-04-16T17:46:17.145 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:17.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:17.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/c1-5 2024-04-16T17:46:17.201 INFO:tasks.ceph:roles_to_devs: {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:17.201 INFO:tasks.ceph:role: c1.osd.5 2024-04-16T17:46:17.202 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T17:46:17.202 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-16T17:46:17.265 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:17.265 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:17.266 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:17.266 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:17.266 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:17.722 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-16T17:46:17.724 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-16T17:46:17.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c1-5 2024-04-16T17:46:17.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-5 2024-04-16T17:46:17.867 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:17.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:17.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:17.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:17.974+0000 7f7f6163c7c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-3/keyring: can't open /var/lib/ceph/osd/c1-3/keyring: (2) No such file or directory 2024-04-16T17:46:17.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:17.982+0000 7f7f6163c7c0 -1 created new key in keyring /var/lib/ceph/osd/c1-3/keyring 2024-04-16T17:46:17.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:17.982+0000 7f7f6163c7c0 -1 bluestore(/var/lib/ceph/osd/c1-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-3/block: (2) No such file or directory 2024-04-16T17:46:17.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:17.982+0000 7f7f6163c7c0 -1 bluestore(/var/lib/ceph/osd/c1-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-3/block: (2) No such file or directory 2024-04-16T17:46:17.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:17.982+0000 7f7f6163c7c0 -1 bluestore(/var/lib/ceph/osd/c1-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-3/block: (2) No such file or directory 2024-04-16T17:46:17.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:17.982+0000 7f7f6163c7c0 -1 bluestore(/var/lib/ceph/osd/c1-3) _read_fsid unparsable uuid 2024-04-16T17:46:20.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-3 2024-04-16T17:46:20.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:20.254+0000 7f75432f07c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-4/keyring: can't open /var/lib/ceph/osd/c1-4/keyring: (2) No such file or directory 2024-04-16T17:46:20.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:20.254+0000 7f75432f07c0 -1 created new key in keyring /var/lib/ceph/osd/c1-4/keyring 2024-04-16T17:46:20.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:20.254+0000 7f75432f07c0 -1 bluestore(/var/lib/ceph/osd/c1-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-4/block: (2) No such file or directory 2024-04-16T17:46:20.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:20.254+0000 7f75432f07c0 -1 bluestore(/var/lib/ceph/osd/c1-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-4/block: (2) No such file or directory 2024-04-16T17:46:20.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:20.254+0000 7f75432f07c0 -1 bluestore(/var/lib/ceph/osd/c1-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-4/block: (2) No such file or directory 2024-04-16T17:46:20.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:20.254+0000 7f75432f07c0 -1 bluestore(/var/lib/ceph/osd/c1-4) _read_fsid unparsable uuid 2024-04-16T17:46:22.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-4 2024-04-16T17:46:22.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:22.578+0000 7f2b96bec7c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-5/keyring: can't open /var/lib/ceph/osd/c1-5/keyring: (2) No such file or directory 2024-04-16T17:46:22.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:22.578+0000 7f2b96bec7c0 -1 created new key in keyring /var/lib/ceph/osd/c1-5/keyring 2024-04-16T17:46:22.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:22.578+0000 7f2b96bec7c0 -1 bluestore(/var/lib/ceph/osd/c1-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-5/block: (2) No such file or directory 2024-04-16T17:46:22.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:22.578+0000 7f2b96bec7c0 -1 bluestore(/var/lib/ceph/osd/c1-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-5/block: (2) No such file or directory 2024-04-16T17:46:22.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:22.578+0000 7f2b96bec7c0 -1 bluestore(/var/lib/ceph/osd/c1-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-5/block: (2) No such file or directory 2024-04-16T17:46:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:46:22.578+0000 7f2b96bec7c0 -1 bluestore(/var/lib/ceph/osd/c1-5) _read_fsid unparsable uuid 2024-04-16T17:46:24.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-5 2024-04-16T17:46:24.810 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-16T17:46:24.810 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:24.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/c1-0/keyring of=/dev/stdout 2024-04-16T17:46:24.825 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:24.825 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/c1-1/keyring of=/dev/stdout 2024-04-16T17:46:24.882 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:24.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/c1-2/keyring of=/dev/stdout 2024-04-16T17:46:24.937 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:24.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/c1-x/keyring of=/dev/stdout 2024-04-16T17:46:24.950 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:24.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/c1-3/keyring of=/dev/stdout 2024-04-16T17:46:25.005 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:25.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/c1-4/keyring of=/dev/stdout 2024-04-16T17:46:25.060 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:25.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/c1-5/keyring of=/dev/stdout 2024-04-16T17:46:25.117 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:25.117 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/c1.client.0.keyring of=/dev/stdout 2024-04-16T17:46:25.122 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:25.122 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/c1.client.1.keyring of=/dev/stdout 2024-04-16T17:46:25.165 INFO:tasks.ceph:Adding keys to all mons... 2024-04-16T17:46:25.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo tee -a /etc/ceph/c1.keyring 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBguR5m3YnANxAA5UhALWtJaB19EqbF6bIb4w== 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBjuR5m4VMJDBAAzr4vpIBM2rkP6VJmmbGgTA== 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBluR5mpLQVHxAA/9XAHz93nQfopH2GmnZtBg== 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.x] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBeuR5mZlQ2JhAAZOU2n8X4U8/qPbJbMtitnw== 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.3] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBpuR5mButlOhAAVhf+KZm8Doua8+k3dKXQkg== 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.4] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBsuR5mSEReDxAAY+7iERj2jzrCcBxY3HbFAQ== 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.5] 2024-04-16T17:46:25.178 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBuuR5mSfOlIhAAuIuf3QZsITs756ZA1N5qFA== 2024-04-16T17:46:25.179 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-04-16T17:46:25.179 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBeuR5mo8lIKRAAskHfJUHh/aAksz8wypZvOQ== 2024-04-16T17:46:25.179 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2024-04-16T17:46:25.179 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBeuR5mXNHGKxAAjZIQ6p+tQIVNvkqx2KRBSA== 2024-04-16T17:46:25.179 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:46:25.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:46:25.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:46:25.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T17:46:25.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:46:25.556 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:46:25.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:46:25.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T17:46:25.802 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T17:46:25.884 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-16T17:46:25.884 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/c1-a 2024-04-16T17:46:25.935 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c1 --mkfs -i a --monmap /home/ubuntu/cephtest/c1.monmap --keyring /etc/ceph/c1.keyring 2024-04-16T17:46:26.023 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c1-a 2024-04-16T17:46:26.075 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/c1.monmap 2024-04-16T17:46:26.122 INFO:tasks.ceph:Starting mon daemons in cluster c1... 2024-04-16T17:46:26.122 INFO:tasks.ceph.c1.mon.a:Restarting daemon 2024-04-16T17:46:26.122 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster c1 -i a 2024-04-16T17:46:26.167 INFO:tasks.ceph.c1.mon.a:Started 2024-04-16T17:46:26.167 INFO:tasks.ceph:Starting mgr daemons in cluster c1... 2024-04-16T17:46:26.167 INFO:tasks.ceph.c1.mgr.x:Restarting daemon 2024-04-16T17:46:26.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster c1 -i x 2024-04-16T17:46:26.170 INFO:tasks.ceph.c1.mgr.x:Started 2024-04-16T17:46:26.170 INFO:tasks.ceph:Setting crush tunables to default 2024-04-16T17:46:26.170 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster c1 osd crush tunables default 2024-04-16T17:46:26.411 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-04-16T17:46:26.429 INFO:tasks.ceph:check_enable_crimson: False 2024-04-16T17:46:26.429 INFO:tasks.ceph:Starting osd daemons in cluster c1... 2024-04-16T17:46:26.429 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:26.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/c1-0/fsid of=/dev/stdout 2024-04-16T17:46:26.440 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:26.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/c1-1/fsid of=/dev/stdout 2024-04-16T17:46:26.495 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:26.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/c1-2/fsid of=/dev/stdout 2024-04-16T17:46:26.525 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:26.526+0000 7f41fffca180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:46:26.547 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:26.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/c1-3/fsid of=/dev/stdout 2024-04-16T17:46:26.560 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:26.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/c1-4/fsid of=/dev/stdout 2024-04-16T17:46:26.611 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:26.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/c1-5/fsid of=/dev/stdout 2024-04-16T17:46:26.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster c1 osd new ef14f41e-080b-4b2a-8e53-7c7cc39a1efe 0 2024-04-16T17:46:26.904 INFO:teuthology.orchestra.run.smithi083.stdout:0 2024-04-16T17:46:26.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster c1 osd new ad347559-0237-4958-b508-679c2d02986c 1 2024-04-16T17:46:27.034 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:27.034+0000 7f41fffca180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:46:27.115 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-16T17:46:27.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster c1 osd new 311c7137-c932-46ff-b043-8fedfac27a99 2 2024-04-16T17:46:27.215 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:27.214+0000 7f41fffca180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:46:27.332 INFO:teuthology.orchestra.run.smithi083.stdout:2 2024-04-16T17:46:27.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster c1 osd new 9de18877-f732-451f-9568-deb8b0cf2f51 3 2024-04-16T17:46:27.538 INFO:teuthology.orchestra.run.smithi083.stdout:3 2024-04-16T17:46:27.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster c1 osd new 675ac319-ba12-4f0c-930f-89812a1c1980 4 2024-04-16T17:46:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:4 2024-04-16T17:46:27.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster c1 osd new aba623b1-7cf6-457d-8765-4ce5d78f4ec0 5 2024-04-16T17:46:27.771 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:27.770+0000 7f41fffca180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:46:27.828 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:27.826+0000 7f41fffca180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:46:27.882 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:27.882+0000 7f41fffca180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:46:27.957 INFO:teuthology.orchestra.run.smithi083.stdout:5 2024-04-16T17:46:27.960 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:27.958+0000 7f41fffca180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:46:27.973 INFO:tasks.ceph.c1.osd.0:Restarting daemon 2024-04-16T17:46:27.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 0 2024-04-16T17:46:27.976 INFO:tasks.ceph.c1.osd.0:Started 2024-04-16T17:46:27.976 INFO:tasks.ceph.c1.osd.1:Restarting daemon 2024-04-16T17:46:27.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 1 2024-04-16T17:46:27.979 INFO:tasks.ceph.c1.osd.1:Started 2024-04-16T17:46:27.979 INFO:tasks.ceph.c1.osd.2:Restarting daemon 2024-04-16T17:46:27.979 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 2 2024-04-16T17:46:27.981 INFO:tasks.ceph.c1.osd.2:Started 2024-04-16T17:46:27.981 INFO:tasks.ceph.c1.osd.3:Restarting daemon 2024-04-16T17:46:27.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 3 2024-04-16T17:46:28.014 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:28.014+0000 7f41fffca180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:46:28.015 INFO:tasks.ceph.c1.osd.3:Started 2024-04-16T17:46:28.016 INFO:tasks.ceph.c1.osd.4:Restarting daemon 2024-04-16T17:46:28.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 4 2024-04-16T17:46:28.018 INFO:tasks.ceph.c1.osd.4:Started 2024-04-16T17:46:28.018 INFO:tasks.ceph.c1.osd.5:Restarting daemon 2024-04-16T17:46:28.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 5 2024-04-16T17:46:28.021 INFO:tasks.ceph.c1.osd.5:Started 2024-04-16T17:46:28.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2024-04-16T17:46:28.098 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:28.098+0000 7f41fffca180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:46:28.151 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:28.150+0000 7f41fffca180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:46:28.204 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:28.204 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"fsid":"5a58eb97-56a6-45c2-947d-e09bf8e2f67b","created":"2024-04-16T17:46:26.316212+0000","modified":"2024-04-16T17:46:27.961326+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T17:46:27.961326+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ef14f41e-080b-4b2a-8e53-7c7cc39a1efe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ad347559-0237-4958-b508-679c2d02986c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"311c7137-c932-46ff-b043-8fedfac27a99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9de18877-f732-451f-9568-deb8b0cf2f51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"675ac319-ba12-4f0c-930f-89812a1c1980","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"aba623b1-7cf6-457d-8765-4ce5d78f4ec0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:46:28.222 INFO:tasks.ceph.ceph_manager.c1:[] 2024-04-16T17:46:28.222 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-16T17:46:28.300 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:28.298+0000 7f41fffca180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:46:28.361 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:28.362+0000 7f41fffca180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:46:28.595 INFO:tasks.ceph.c1.osd.1.smithi071.stderr:2024-04-16T17:46:28.599+0000 7ff0415137c0 -1 Falling back to public interface 2024-04-16T17:46:28.619 INFO:tasks.ceph.c1.osd.0.smithi071.stderr:2024-04-16T17:46:28.623+0000 7f0ab99c27c0 -1 Falling back to public interface 2024-04-16T17:46:28.627 INFO:tasks.ceph.c1.osd.3.smithi083.stderr:2024-04-16T17:46:28.626+0000 7fd0bb2e47c0 -1 Falling back to public interface 2024-04-16T17:46:28.627 INFO:tasks.ceph.c1.osd.5.smithi083.stderr:2024-04-16T17:46:28.626+0000 7f44bb4ea7c0 -1 Falling back to public interface 2024-04-16T17:46:28.634 INFO:tasks.ceph.c1.osd.2.smithi071.stderr:2024-04-16T17:46:28.639+0000 7fbda7ef07c0 -1 Falling back to public interface 2024-04-16T17:46:28.638 INFO:tasks.ceph.c1.osd.4.smithi083.stderr:2024-04-16T17:46:28.638+0000 7f75d2c897c0 -1 Falling back to public interface 2024-04-16T17:46:28.826 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2024-04-16T17:46:29.014 INFO:teuthology.misc.health.smithi071.stdout: 2024-04-16T17:46:29.014 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":8,"fsid":"5a58eb97-56a6-45c2-947d-e09bf8e2f67b","created":"2024-04-16T17:46:26.316212+0000","modified":"2024-04-16T17:46:27.961326+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T17:46:27.961326+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ef14f41e-080b-4b2a-8e53-7c7cc39a1efe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ad347559-0237-4958-b508-679c2d02986c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"311c7137-c932-46ff-b043-8fedfac27a99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9de18877-f732-451f-9568-deb8b0cf2f51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"675ac319-ba12-4f0c-930f-89812a1c1980","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"aba623b1-7cf6-457d-8765-4ce5d78f4ec0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:46:29.030 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-04-16T17:46:29.055 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-16T17:46:29.055 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-16T17:46:29.055 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr: from numpy import show_config as show_numpy_config 2024-04-16T17:46:29.124 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:29.122+0000 7f41fffca180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:46:29.308 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:29.306+0000 7f41fffca180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:46:29.472 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:29.470+0000 7f41fffca180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:46:29.543 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:29.542+0000 7f41fffca180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:46:29.690 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:29.690+0000 7f41fffca180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:46:30.325 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:30.326+0000 7f41fffca180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:46:30.401 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:30.398+0000 7f41fffca180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:46:30.581 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:30.578+0000 7f41fffca180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:46:30.654 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:30.654+0000 7f41fffca180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:46:30.941 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:30.942+0000 7f41fffca180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:46:31.003 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:31.002+0000 7f41fffca180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:46:31.110 INFO:tasks.ceph.c1.osd.2.smithi071.stderr:2024-04-16T17:46:31.115+0000 7fbda7ef07c0 -1 osd.2 0 log_to_monitors true 2024-04-16T17:46:31.372 INFO:tasks.ceph.c1.osd.4.smithi083.stderr:2024-04-16T17:46:31.370+0000 7f75d2c897c0 -1 osd.4 0 log_to_monitors true 2024-04-16T17:46:31.496 INFO:tasks.ceph.c1.osd.1.smithi071.stderr:2024-04-16T17:46:31.499+0000 7ff0415137c0 -1 osd.1 0 log_to_monitors true 2024-04-16T17:46:31.588 INFO:tasks.ceph.c1.osd.3.smithi083.stderr:2024-04-16T17:46:31.586+0000 7fd0bb2e47c0 -1 osd.3 0 log_to_monitors true 2024-04-16T17:46:31.597 INFO:tasks.ceph.c1.osd.0.smithi071.stderr:2024-04-16T17:46:31.599+0000 7f0ab99c27c0 -1 osd.0 0 log_to_monitors true 2024-04-16T17:46:31.630 INFO:tasks.ceph.c1.osd.5.smithi083.stderr:2024-04-16T17:46:31.630+0000 7f44bb4ea7c0 -1 osd.5 0 log_to_monitors true 2024-04-16T17:46:32.039 INFO:tasks.ceph.c1.mgr.x.smithi083.stderr:2024-04-16T17:46:32.038+0000 7f41a7411640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-16T17:46:34.477 INFO:tasks.ceph.c1.osd.3.smithi083.stderr:2024-04-16T17:46:34.478+0000 7fd0b7a9f640 -1 osd.3 0 waiting for initial osdmap 2024-04-16T17:46:34.499 INFO:tasks.ceph.c1.osd.3.smithi083.stderr:2024-04-16T17:46:34.498+0000 7fd0af0c1640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:46:34.529 INFO:tasks.ceph.c1.osd.5.smithi083.stderr:2024-04-16T17:46:34.530+0000 7f44b7ca5640 -1 osd.5 0 waiting for initial osdmap 2024-04-16T17:46:34.565 INFO:tasks.ceph.c1.osd.5.smithi083.stderr:2024-04-16T17:46:34.566+0000 7f44af2c7640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:46:34.575 INFO:tasks.ceph.c1.osd.2.smithi071.stderr:2024-04-16T17:46:34.579+0000 7fbda46ab640 -1 osd.2 0 waiting for initial osdmap 2024-04-16T17:46:34.576 INFO:tasks.ceph.c1.osd.4.smithi083.stderr:2024-04-16T17:46:34.574+0000 7f75cec32640 -1 osd.4 0 waiting for initial osdmap 2024-04-16T17:46:34.582 INFO:tasks.ceph.c1.osd.0.smithi071.stderr:2024-04-16T17:46:34.587+0000 7f0ab596b640 -1 osd.0 0 waiting for initial osdmap 2024-04-16T17:46:34.585 INFO:tasks.ceph.c1.osd.1.smithi071.stderr:2024-04-16T17:46:34.587+0000 7ff03dcce640 -1 osd.1 0 waiting for initial osdmap 2024-04-16T17:46:34.594 INFO:tasks.ceph.c1.osd.2.smithi071.stderr:2024-04-16T17:46:34.599+0000 7fbd9bccd640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:46:34.606 INFO:tasks.ceph.c1.osd.4.smithi083.stderr:2024-04-16T17:46:34.606+0000 7f75c6a66640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:46:34.610 INFO:tasks.ceph.c1.osd.0.smithi071.stderr:2024-04-16T17:46:34.615+0000 7f0aad79f640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:46:34.610 INFO:tasks.ceph.c1.osd.1.smithi071.stderr:2024-04-16T17:46:34.615+0000 7ff0352f0640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:46:35.635 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2024-04-16T17:46:35.931 INFO:teuthology.misc.health.smithi071.stdout: 2024-04-16T17:46:35.931 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":11,"fsid":"5a58eb97-56a6-45c2-947d-e09bf8e2f67b","created":"2024-04-16T17:46:26.316212+0000","modified":"2024-04-16T17:46:35.484595+0000","last_up_change":"2024-04-16T17:46:35.484595+0000","last_in_change":"2024-04-16T17:46:27.961326+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ef14f41e-080b-4b2a-8e53-7c7cc39a1efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6809","nonce":2482459553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6811","nonce":2482459553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6815","nonce":2482459553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6813","nonce":2482459553}]},"public_addr":"172.21.15.71:6809/2482459553","cluster_addr":"172.21.15.71:6811/2482459553","heartbeat_back_addr":"172.21.15.71:6815/2482459553","heartbeat_front_addr":"172.21.15.71:6813/2482459553","state":["exists","up"]},{"osd":1,"uuid":"ad347559-0237-4958-b508-679c2d02986c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6801","nonce":3255159991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6803","nonce":3255159991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6807","nonce":3255159991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6805","nonce":3255159991}]},"public_addr":"172.21.15.71:6801/3255159991","cluster_addr":"172.21.15.71:6803/3255159991","heartbeat_back_addr":"172.21.15.71:6807/3255159991","heartbeat_front_addr":"172.21.15.71:6805/3255159991","state":["exists","up"]},{"osd":2,"uuid":"311c7137-c932-46ff-b043-8fedfac27a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6817","nonce":3171611431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6819","nonce":3171611431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6823","nonce":3171611431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6821","nonce":3171611431}]},"public_addr":"172.21.15.71:6817/3171611431","cluster_addr":"172.21.15.71:6819/3171611431","heartbeat_back_addr":"172.21.15.71:6823/3171611431","heartbeat_front_addr":"172.21.15.71:6821/3171611431","state":["exists","up"]},{"osd":3,"uuid":"9de18877-f732-451f-9568-deb8b0cf2f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6804","nonce":3919083014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6808","nonce":3919083014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6815","nonce":3919083014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6812","nonce":3919083014}]},"public_addr":"172.21.15.83:6804/3919083014","cluster_addr":"172.21.15.83:6808/3919083014","heartbeat_back_addr":"172.21.15.83:6815/3919083014","heartbeat_front_addr":"172.21.15.83:6812/3919083014","state":["exists","up"]},{"osd":4,"uuid":"675ac319-ba12-4f0c-930f-89812a1c1980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6817","nonce":3700970227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6819","nonce":3700970227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6823","nonce":3700970227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6821","nonce":3700970227}]},"public_addr":"172.21.15.83:6817/3700970227","cluster_addr":"172.21.15.83:6819/3700970227","heartbeat_back_addr":"172.21.15.83:6823/3700970227","heartbeat_front_addr":"172.21.15.83:6821/3700970227","state":["exists","up"]},{"osd":5,"uuid":"aba623b1-7cf6-457d-8765-4ce5d78f4ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6801","nonce":136860477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6805","nonce":136860477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6813","nonce":136860477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6809","nonce":136860477}]},"public_addr":"172.21.15.83:6801/136860477","cluster_addr":"172.21.15.83:6805/136860477","heartbeat_back_addr":"172.21.15.83:6813/136860477","heartbeat_front_addr":"172.21.15.83:6809/136860477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:46:35.948 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-04-16T17:46:35.948 INFO:tasks.ceph:Creating RBD pool 2024-04-16T17:46:35.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster c1 osd pool create rbd 8 2024-04-16T17:46:36.487 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'rbd' created 2024-04-16T17:46:36.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster c1 osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-16T17:46:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-16T17:46:37.520 INFO:tasks.ceph:Starting mds daemons in cluster c1... 2024-04-16T17:46:37.520 INFO:tasks.ceph:Waiting until c1 daemons up and pgs clean... 2024-04-16T17:46:37.521 INFO:tasks.ceph.ceph_manager.c1:waiting for mgr available 2024-04-16T17:46:37.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 mgr dump --format=json 2024-04-16T17:46:37.521 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-16T17:46:37.862 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:37.879 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":4,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3952957548},{"type":"v1","addr":"172.21.15.83:6825","nonce":3952957548}]},"active_addr":"172.21.15.83:6825/3952957548","active_change":"2024-04-16T17:46:31.029094+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3324067736}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":394468303}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":863789478}]}]} 2024-04-16T17:46:37.881 INFO:tasks.ceph.ceph_manager.c1:mgr available! 2024-04-16T17:46:37.881 INFO:tasks.ceph.ceph_manager.c1:waiting for all up 2024-04-16T17:46:37.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2024-04-16T17:46:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:38.168 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":13,"fsid":"5a58eb97-56a6-45c2-947d-e09bf8e2f67b","created":"2024-04-16T17:46:26.316212+0000","modified":"2024-04-16T17:46:37.490454+0000","last_up_change":"2024-04-16T17:46:35.484595+0000","last_in_change":"2024-04-16T17:46:27.961326+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-16T17:46:36.260452+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-16T17:46:37.085579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef14f41e-080b-4b2a-8e53-7c7cc39a1efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6809","nonce":2482459553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6811","nonce":2482459553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6815","nonce":2482459553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6813","nonce":2482459553}]},"public_addr":"172.21.15.71:6809/2482459553","cluster_addr":"172.21.15.71:6811/2482459553","heartbeat_back_addr":"172.21.15.71:6815/2482459553","heartbeat_front_addr":"172.21.15.71:6813/2482459553","state":["exists","up"]},{"osd":1,"uuid":"ad347559-0237-4958-b508-679c2d02986c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6801","nonce":3255159991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6803","nonce":3255159991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6807","nonce":3255159991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6805","nonce":3255159991}]},"public_addr":"172.21.15.71:6801/3255159991","cluster_addr":"172.21.15.71:6803/3255159991","heartbeat_back_addr":"172.21.15.71:6807/3255159991","heartbeat_front_addr":"172.21.15.71:6805/3255159991","state":["exists","up"]},{"osd":2,"uuid":"311c7137-c932-46ff-b043-8fedfac27a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6817","nonce":3171611431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6819","nonce":3171611431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6823","nonce":3171611431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6821","nonce":3171611431}]},"public_addr":"172.21.15.71:6817/3171611431","cluster_addr":"172.21.15.71:6819/3171611431","heartbeat_back_addr":"172.21.15.71:6823/3171611431","heartbeat_front_addr":"172.21.15.71:6821/3171611431","state":["exists","up"]},{"osd":3,"uuid":"9de18877-f732-451f-9568-deb8b0cf2f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6804","nonce":3919083014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6808","nonce":3919083014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6815","nonce":3919083014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6812","nonce":3919083014}]},"public_addr":"172.21.15.83:6804/3919083014","cluster_addr":"172.21.15.83:6808/3919083014","heartbeat_back_addr":"172.21.15.83:6815/3919083014","heartbeat_front_addr":"172.21.15.83:6812/3919083014","state":["exists","up"]},{"osd":4,"uuid":"675ac319-ba12-4f0c-930f-89812a1c1980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6817","nonce":3700970227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6819","nonce":3700970227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6823","nonce":3700970227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6821","nonce":3700970227}]},"public_addr":"172.21.15.83:6817/3700970227","cluster_addr":"172.21.15.83:6819/3700970227","heartbeat_back_addr":"172.21.15.83:6823/3700970227","heartbeat_front_addr":"172.21.15.83:6821/3700970227","state":["exists","up"]},{"osd":5,"uuid":"aba623b1-7cf6-457d-8765-4ce5d78f4ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6801","nonce":136860477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6805","nonce":136860477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6813","nonce":136860477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6809","nonce":136860477}]},"public_addr":"172.21.15.83:6801/136860477","cluster_addr":"172.21.15.83:6805/136860477","heartbeat_back_addr":"172.21.15.83:6813/136860477","heartbeat_front_addr":"172.21.15.83:6809/136860477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.628033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.524646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.165496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.546615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.394696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.623608+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:46:38.185 INFO:tasks.ceph.ceph_manager.c1:all up! 2024-04-16T17:46:38.185 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2024-04-16T17:46:38.472 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:38.472 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":13,"fsid":"5a58eb97-56a6-45c2-947d-e09bf8e2f67b","created":"2024-04-16T17:46:26.316212+0000","modified":"2024-04-16T17:46:37.490454+0000","last_up_change":"2024-04-16T17:46:35.484595+0000","last_in_change":"2024-04-16T17:46:27.961326+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-16T17:46:36.260452+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-16T17:46:37.085579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef14f41e-080b-4b2a-8e53-7c7cc39a1efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6809","nonce":2482459553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6811","nonce":2482459553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6815","nonce":2482459553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6813","nonce":2482459553}]},"public_addr":"172.21.15.71:6809/2482459553","cluster_addr":"172.21.15.71:6811/2482459553","heartbeat_back_addr":"172.21.15.71:6815/2482459553","heartbeat_front_addr":"172.21.15.71:6813/2482459553","state":["exists","up"]},{"osd":1,"uuid":"ad347559-0237-4958-b508-679c2d02986c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6801","nonce":3255159991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6803","nonce":3255159991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6807","nonce":3255159991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6805","nonce":3255159991}]},"public_addr":"172.21.15.71:6801/3255159991","cluster_addr":"172.21.15.71:6803/3255159991","heartbeat_back_addr":"172.21.15.71:6807/3255159991","heartbeat_front_addr":"172.21.15.71:6805/3255159991","state":["exists","up"]},{"osd":2,"uuid":"311c7137-c932-46ff-b043-8fedfac27a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6817","nonce":3171611431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6819","nonce":3171611431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6823","nonce":3171611431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6821","nonce":3171611431}]},"public_addr":"172.21.15.71:6817/3171611431","cluster_addr":"172.21.15.71:6819/3171611431","heartbeat_back_addr":"172.21.15.71:6823/3171611431","heartbeat_front_addr":"172.21.15.71:6821/3171611431","state":["exists","up"]},{"osd":3,"uuid":"9de18877-f732-451f-9568-deb8b0cf2f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6804","nonce":3919083014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6808","nonce":3919083014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6815","nonce":3919083014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6812","nonce":3919083014}]},"public_addr":"172.21.15.83:6804/3919083014","cluster_addr":"172.21.15.83:6808/3919083014","heartbeat_back_addr":"172.21.15.83:6815/3919083014","heartbeat_front_addr":"172.21.15.83:6812/3919083014","state":["exists","up"]},{"osd":4,"uuid":"675ac319-ba12-4f0c-930f-89812a1c1980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6817","nonce":3700970227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6819","nonce":3700970227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6823","nonce":3700970227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6821","nonce":3700970227}]},"public_addr":"172.21.15.83:6817/3700970227","cluster_addr":"172.21.15.83:6819/3700970227","heartbeat_back_addr":"172.21.15.83:6823/3700970227","heartbeat_front_addr":"172.21.15.83:6821/3700970227","state":["exists","up"]},{"osd":5,"uuid":"aba623b1-7cf6-457d-8765-4ce5d78f4ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6801","nonce":136860477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6805","nonce":136860477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6813","nonce":136860477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6809","nonce":136860477}]},"public_addr":"172.21.15.83:6801/136860477","cluster_addr":"172.21.15.83:6805/136860477","heartbeat_back_addr":"172.21.15.83:6813/136860477","heartbeat_front_addr":"172.21.15.83:6809/136860477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.628033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.524646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.165496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.546615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.394696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.623608+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:46:38.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.0 flush_pg_stats 2024-04-16T17:46:38.491 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.1 flush_pg_stats 2024-04-16T17:46:38.491 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.2 flush_pg_stats 2024-04-16T17:46:38.492 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.3 flush_pg_stats 2024-04-16T17:46:38.492 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.4 flush_pg_stats 2024-04-16T17:46:38.492 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.5 flush_pg_stats 2024-04-16T17:46:38.676 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-04-16T17:46:38.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2024-04-16T17:46:38.694 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:38.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.3 2024-04-16T17:46:38.717 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-04-16T17:46:38.717 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.5 2024-04-16T17:46:38.730 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:38.730 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.2 2024-04-16T17:46:38.731 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-04-16T17:46:38.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.1 2024-04-16T17:46:38.744 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:38.745 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.4 2024-04-16T17:46:39.071 INFO:teuthology.orchestra.run.smithi071.stdout:47244640258 2024-04-16T17:46:39.088 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640259 got 47244640258 for osd.3 2024-04-16T17:46:39.155 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:39.170 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:39.176 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640260 got 47244640259 for osd.1 2024-04-16T17:46:39.194 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640260 got 47244640259 for osd.5 2024-04-16T17:46:39.204 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:39.221 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640260 got 47244640259 for osd.0 2024-04-16T17:46:39.240 INFO:teuthology.orchestra.run.smithi071.stdout:47244640258 2024-04-16T17:46:39.257 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640259 got 47244640258 for osd.2 2024-04-16T17:46:39.275 INFO:teuthology.orchestra.run.smithi071.stdout:47244640258 2024-04-16T17:46:39.291 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640259 got 47244640258 for osd.4 2024-04-16T17:46:40.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.3 2024-04-16T17:46:40.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.1 2024-04-16T17:46:40.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.5 2024-04-16T17:46:40.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2024-04-16T17:46:40.258 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.2 2024-04-16T17:46:40.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.4 2024-04-16T17:46:40.459 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:40.485 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640259 got 47244640259 for osd.3 2024-04-16T17:46:40.485 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:40.504 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-04-16T17:46:40.527 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640260 got 47244640260 for osd.1 2024-04-16T17:46:40.527 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:40.591 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-04-16T17:46:40.609 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640260 got 47244640260 for osd.0 2024-04-16T17:46:40.609 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:47244640260 2024-04-16T17:46:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:40.634 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640260 got 47244640260 for osd.5 2024-04-16T17:46:40.634 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:40.636 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640259 got 47244640259 for osd.4 2024-04-16T17:46:40.636 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:40.656 INFO:teuthology.orchestra.run.smithi071.stdout:47244640259 2024-04-16T17:46:40.672 INFO:tasks.ceph.ceph_manager.c1:need seq 47244640259 got 47244640259 for osd.2 2024-04-16T17:46:40.672 DEBUG:teuthology.parallel:result is None 2024-04-16T17:46:40.672 INFO:tasks.ceph.ceph_manager.c1:waiting for clean 2024-04-16T17:46:40.672 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T17:46:40.990 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:40.990 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T17:46:41.007 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-16T17:46:39.500086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":95,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":163236,"kb_used_data":968,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":628982364,"statfs":{"total":644245094400,"available":644077940736,"internally_reserved":0,"allocated":991232,"data_stored":467240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.466639"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597396+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T17:46:38.597396+0000","last_peered":"2024-04-16T17:46:38.597396+0000","last_clean":"2024-04-16T17:46:38.597396+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T17:46:38.597396+0000","last_undegraded":"2024-04-16T17:46:38.597396+0000","last_fullsized":"2024-04-16T17:46:38.597396+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.502206+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T17:46:38.502206+0000","last_peered":"2024-04-16T17:46:38.502206+0000","last_clean":"2024-04-16T17:46:38.502206+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T17:46:38.502206+0000","last_undegraded":"2024-04-16T17:46:38.502206+0000","last_fullsized":"2024-04-16T17:46:38.502206+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597358+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T17:46:38.597358+0000","last_peered":"2024-04-16T17:46:38.597358+0000","last_clean":"2024-04-16T17:46:38.597358+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T17:46:38.597358+0000","last_undegraded":"2024-04-16T17:46:38.597358+0000","last_fullsized":"2024-04-16T17:46:38.597358+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.503368+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T17:46:38.503368+0000","last_peered":"2024-04-16T17:46:38.503368+0000","last_clean":"2024-04-16T17:46:38.503368+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T17:46:38.503368+0000","last_undegraded":"2024-04-16T17:46:38.503368+0000","last_fullsized":"2024-04-16T17:46:38.503368+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'62","reported_seq":91,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.657679+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T17:46:38.657679+0000","last_peered":"2024-04-16T17:46:38.657679+0000","last_clean":"2024-04-16T17:46:38.657679+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T17:46:38.657679+0000","last_undegraded":"2024-04-16T17:46:38.657679+0000","last_fullsized":"2024-04-16T17:46:38.657679+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:59.406447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":95,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.503172+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T17:46:38.503172+0000","last_peered":"2024-04-16T17:46:38.503172+0000","last_clean":"2024-04-16T17:46:38.503172+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T17:46:38.503172+0000","last_undegraded":"2024-04-16T17:46:38.503172+0000","last_fullsized":"2024-04-16T17:46:38.503172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597365+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T17:46:38.597365+0000","last_peered":"2024-04-16T17:46:38.597365+0000","last_clean":"2024-04-16T17:46:38.597365+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T17:46:38.597365+0000","last_undegraded":"2024-04-16T17:46:38.597365+0000","last_fullsized":"2024-04-16T17:46:38.597365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597393+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T17:46:38.597393+0000","last_peered":"2024-04-16T17:46:38.597393+0000","last_clean":"2024-04-16T17:46:38.597393+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T17:46:38.597393+0000","last_undegraded":"2024-04-16T17:46:38.597393+0000","last_fullsized":"2024-04-16T17:46:38.597393+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.503092+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T17:46:38.503092+0000","last_peered":"2024-04-16T17:46:38.503092+0000","last_clean":"2024-04-16T17:46:38.503092+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T17:46:38.503092+0000","last_undegraded":"2024-04-16T17:46:38.503092+0000","last_fullsized":"2024-04-16T17:46:38.503092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":95,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27128,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830472,"statfs":{"total":107374182400,"available":107346403328,"internally_reserved":0,"allocated":106496,"data_stored":27924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27544,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830056,"statfs":{"total":107374182400,"available":107345977344,"internally_reserved":0,"allocated":397312,"data_stored":301158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":131072,"data_stored":38486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:46:41.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T17:46:41.279 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T17:46:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-16T17:46:41.034123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":95,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":163236,"kb_used_data":968,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":628982364,"statfs":{"total":644245094400,"available":644077940736,"internally_reserved":0,"allocated":991232,"data_stored":467240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000676"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597396+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T17:46:38.597396+0000","last_peered":"2024-04-16T17:46:38.597396+0000","last_clean":"2024-04-16T17:46:38.597396+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T17:46:38.597396+0000","last_undegraded":"2024-04-16T17:46:38.597396+0000","last_fullsized":"2024-04-16T17:46:38.597396+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.502206+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T17:46:38.502206+0000","last_peered":"2024-04-16T17:46:38.502206+0000","last_clean":"2024-04-16T17:46:38.502206+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T17:46:38.502206+0000","last_undegraded":"2024-04-16T17:46:38.502206+0000","last_fullsized":"2024-04-16T17:46:38.502206+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597358+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T17:46:38.597358+0000","last_peered":"2024-04-16T17:46:38.597358+0000","last_clean":"2024-04-16T17:46:38.597358+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T17:46:38.597358+0000","last_undegraded":"2024-04-16T17:46:38.597358+0000","last_fullsized":"2024-04-16T17:46:38.597358+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.503368+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T17:46:38.503368+0000","last_peered":"2024-04-16T17:46:38.503368+0000","last_clean":"2024-04-16T17:46:38.503368+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T17:46:38.503368+0000","last_undegraded":"2024-04-16T17:46:38.503368+0000","last_fullsized":"2024-04-16T17:46:38.503368+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'62","reported_seq":91,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.657679+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T17:46:38.657679+0000","last_peered":"2024-04-16T17:46:38.657679+0000","last_clean":"2024-04-16T17:46:38.657679+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T17:46:38.657679+0000","last_undegraded":"2024-04-16T17:46:38.657679+0000","last_fullsized":"2024-04-16T17:46:38.657679+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:59.406447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":95,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.503172+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T17:46:38.503172+0000","last_peered":"2024-04-16T17:46:38.503172+0000","last_clean":"2024-04-16T17:46:38.503172+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T17:46:38.503172+0000","last_undegraded":"2024-04-16T17:46:38.503172+0000","last_fullsized":"2024-04-16T17:46:38.503172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597365+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T17:46:38.597365+0000","last_peered":"2024-04-16T17:46:38.597365+0000","last_clean":"2024-04-16T17:46:38.597365+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T17:46:38.597365+0000","last_undegraded":"2024-04-16T17:46:38.597365+0000","last_fullsized":"2024-04-16T17:46:38.597365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.597393+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T17:46:38.597393+0000","last_peered":"2024-04-16T17:46:38.597393+0000","last_clean":"2024-04-16T17:46:38.597393+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T17:46:38.597393+0000","last_undegraded":"2024-04-16T17:46:38.597393+0000","last_fullsized":"2024-04-16T17:46:38.597393+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:46:38.503092+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T17:46:38.503092+0000","last_peered":"2024-04-16T17:46:38.503092+0000","last_clean":"2024-04-16T17:46:38.503092+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T17:46:38.503092+0000","last_undegraded":"2024-04-16T17:46:38.503092+0000","last_fullsized":"2024-04-16T17:46:38.503092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":95,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27128,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830472,"statfs":{"total":107374182400,"available":107346403328,"internally_reserved":0,"allocated":106496,"data_stored":27924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27544,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830056,"statfs":{"total":107374182400,"available":107345977344,"internally_reserved":0,"allocated":397312,"data_stored":301158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":131072,"data_stored":38486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:46:41.296 INFO:tasks.ceph.ceph_manager.c1:clean! 2024-04-16T17:46:41.296 INFO:tasks.ceph:Waiting until ceph cluster c1 is healthy... 2024-04-16T17:46:41.296 INFO:tasks.ceph.ceph_manager.c1:wait_until_healthy 2024-04-16T17:46:41.297 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 health --format=json 2024-04-16T17:46:41.599 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:41.599 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T17:46:41.616 INFO:tasks.ceph.ceph_manager.c1:wait_until_healthy done 2024-04-16T17:46:41.616 INFO:teuthology.run_tasks:Running task ceph... 2024-04-16T17:46:41.625 INFO:tasks.ceph:Creating ceph cluster c2... 2024-04-16T17:46:41.625 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'c2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-16T17:46:41.626 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'branch': 'wip-shilpa-multisite-tests', 'description': 'rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi}', 'email': None, 'first_in_suite': False, 'job_id': '7658456', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-shilpa-multisite-tests'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'enabled_features': ['resharding', 'notification_v2'], 'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}]}]}, 'rgw-multisite-tests': {'args': ['tests.py', '-a', '!fails_on_rgw']}, 'workunit': {'branch': 'wip-shilpa-multisite-tests', 'sha1': '9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b'}}, 'owner': 'scheduled_smanjara@teuthology', 'priority': 49, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['c1.mon.a', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'], ['c1.mgr.x', 'c1.osd.3', 'c1.osd.4', 'c1.osd.5', 'c1.client.1'], ['c2.mon.a', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0'], ['c2.mgr.x', 'c2.osd.3', 'c2.osd.4', 'c2.osd.5', 'c2.client.1']], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086', 'sleep_before_teardown': 0, 'suite': 'rgw:multisite', 'suite_branch': 'wip-shilpa-multisite-tests', 'suite_path': '/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/smanjara/ceph.git', 'suite_sha1': '9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b', 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9vtxKrRme6/isUwEX2lhl8Ybr0T6XOkffrdu3KUaVOAXLP5Y37Tmkgxpc7kst406V1WRz+VK6xeEGL3E8XHEI=', 'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6BfUv5py4V6x14q9+tGyc4AQrymyRKyAz6y6sP+C7i7TH6fqhkepIeAS1YXpmNgRSN28Z6rVAE8v6vXjqqcYQ=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN7+QuOyR1DYPk/GC5c9CS7zlOEbjs3gZpYoCAodd4KN3mbnad1OmBxBuc9eY060s6zeyO+Ul/4sy25LGzWxW6s=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUJ9KsHPOwr6bVTpKPWxuVw2M8nCWXz8Wb3iPZ0uBK78ORQ5r/LsER0oB/sFQ+EsO4buhZ2vscDOaYUzUJAiJM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}}, {'ceph': {'cluster': 'c2', 'conf': {'client': {'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b0c7b917267c471208541fdeb652f5e897d00086'}}, {'rgw': {'c1.client.0': {'port': 8000}, 'c1.client.1': {'port': 8001}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 90}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-16_17:28:09', 'tube': 'smithi', 'user': 'smanjara', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-16T17:46:41.626 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2024-04-16T17:46:41.628 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2024-04-16T17:46:41.632 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2024-04-16T17:46:41.634 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2024-04-16T17:46:41.641 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:41.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:41.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:41.685 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T17:46:41.699 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:41.699 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T17:46:41.743 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T17:46:41.743 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 17:40:34.694554387 +0000 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 17:40:34.418556852 +0000 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 17:40:34.418556852 +0000 2024-04-16T17:46:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-04-16T17:46:41.794 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T17:46:41.844 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T17:46:41.844 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T17:46:41.844 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000349316 s, 1.5 MB/s 2024-04-16T17:46:41.845 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T17:46:41.892 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 17:40:35.166550170 +0000 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 17:40:34.930552278 +0000 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 17:40:34.930552278 +0000 2024-04-16T17:46:41.939 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-04-16T17:46:41.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T17:46:41.993 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T17:46:41.993 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T17:46:41.993 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00042814 s, 1.2 MB/s 2024-04-16T17:46:41.994 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T17:46:42.044 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-04-16T17:46:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T17:46:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:42.091 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-16T17:46:42.091 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:42.091 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 17:40:35.674545633 +0000 2024-04-16T17:46:42.092 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 17:40:35.418547920 +0000 2024-04-16T17:46:42.092 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 17:40:35.418547920 +0000 2024-04-16T17:46:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-04-16T17:46:42.092 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T17:46:42.145 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T17:46:42.146 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T17:46:42.146 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000414071 s, 1.2 MB/s 2024-04-16T17:46:42.147 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T17:46:42.196 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 17:40:36.182541096 +0000 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 17:40:35.938543275 +0000 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 17:40:35.938543275 +0000 2024-04-16T17:46:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-04-16T17:46:42.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T17:46:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T17:46:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T17:46:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000315144 s, 1.6 MB/s 2024-04-16T17:46:42.298 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T17:46:42.348 INFO:tasks.ceph:osd dev map: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:42.348 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:42.348 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T17:46:42.354 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T17:46:42.354 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-16 17:40:34.074505072 +0000 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-16 17:40:33.814507394 +0000 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-16 17:40:33.814507394 +0000 2024-04-16T17:46:42.402 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-04-16T17:46:42.407 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T17:46:42.457 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-16T17:46:42.457 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-16T17:46:42.457 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000441233 s, 1.2 MB/s 2024-04-16T17:46:42.458 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T17:46:42.507 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-16 17:40:34.558500749 +0000 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-16 17:40:34.290503143 +0000 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-16 17:40:34.290503143 +0000 2024-04-16T17:46:42.554 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-04-16T17:46:42.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T17:46:42.609 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-16T17:46:42.609 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-16T17:46:42.609 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000379162 s, 1.4 MB/s 2024-04-16T17:46:42.610 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T17:46:42.663 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 912 Links: 1 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-16 17:40:35.014496675 +0000 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-16 17:40:34.790498676 +0000 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-16 17:40:34.790498676 +0000 2024-04-16T17:46:42.710 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-04-16T17:46:42.710 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T17:46:42.764 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-16T17:46:42.764 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-16T17:46:42.764 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000338704 s, 1.5 MB/s 2024-04-16T17:46:42.765 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T17:46:42.820 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-16 17:40:35.478492531 +0000 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-16 17:40:35.226494781 +0000 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-16 17:40:35.226494781 +0000 2024-04-16T17:46:42.866 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-04-16T17:46:42.866 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T17:46:42.920 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-16T17:46:42.920 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-16T17:46:42.920 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000279384 s, 1.8 MB/s 2024-04-16T17:46:42.921 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T17:46:42.971 INFO:tasks.ceph:osd dev map: {'c2.osd.3': '/dev/vg_nvme/lv_1', 'c2.osd.4': '/dev/vg_nvme/lv_2', 'c2.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:42.971 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'c2.osd.3': '/dev/vg_nvme/lv_1', 'c2.osd.4': '/dev/vg_nvme/lv_2', 'c2.osd.5': '/dev/vg_nvme/lv_3'}} 2024-04-16T17:46:42.971 INFO:tasks.ceph:Generating config... 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] debug rgw lifecycle = 20 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] debug rgw notification = 20 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] debug rgw sync = 20 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw data log num shards = 4 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw data sync poll interval = 5 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw md log max shards = 4 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw meta sync poll interval = 5 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw sync data inject err probability = 0.1 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2024-04-16T17:46:42.973 INFO:tasks.ceph:[client] rgw sync meta inject err probability = 0.1 2024-04-16T17:46:42.974 INFO:tasks.ceph:[client] rgw sync obj etag verify = True 2024-04-16T17:46:42.974 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-16T17:46:42.974 INFO:tasks.ceph:[client] setuser = ceph 2024-04-16T17:46:42.974 INFO:tasks.ceph:[client.0] rgw data notify interval msec = 0 2024-04-16T17:46:42.974 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-16T17:46:42.974 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-16T17:46:42.974 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-16T17:46:42.974 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-16T17:46:42.974 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-16T17:46:42.974 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-16T17:46:42.974 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-16T17:46:42.974 INFO:tasks.ceph:[osd] rocksdb delete range threshold = 0 2024-04-16T17:46:42.974 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2024-04-16T17:46:42.974 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:42.975 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:42.975 INFO:tasks.ceph:Setting up c2.mon.a... 2024-04-16T17:46:42.975 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c2.keyring 2024-04-16T17:46:43.008 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/c2.keyring 2024-04-16T17:46:43.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c2.keyring 2024-04-16T17:46:43.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/c2.keyring 2024-04-16T17:46:43.159 DEBUG:tasks.ceph:Ceph mon addresses: [('c2.mon.a', '172.21.15.122')] 2024-04-16T17:46:43.159 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.122'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw lifecycle': 20, 'debug rgw notification': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw data sync poll interval': 5, 'rgw md log max shards': 4, 'rgw meta sync poll interval': 5, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'client.0': {'rgw data notify interval msec': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}} 2024-04-16T17:46:43.160 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:43.160 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T17:46:43.207 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.122 --print /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:43.285 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:46:43.285 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: generated fsid a05abb94-fa63-4b8c-8984-45e6159773f5 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 0 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:fsid a05abb94-fa63-4b8c-8984-45e6159773f5 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:last_changed 2024-04-16T17:46:43.288651+0000 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-04-16T17:46:43.288651+0000 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:min_mon_release 17 (quincy) 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:election_strategy: 1 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-16T17:46:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c2.monmap (1 monitors) 2024-04-16T17:46:43.288 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T17:46:43.335 INFO:tasks.ceph:Writing /etc/ceph/c2.conf for FSID a05abb94-fa63-4b8c-8984-45e6159773f5... 2024-04-16T17:46:43.336 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2024-04-16T17:46:43.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2024-04-16T17:46:43.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2024-04-16T17:46:43.380 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2024-04-16T17:46:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-16T17:46:43.384 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-16T17:46:43.384 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:43.384 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-16T17:46:43.384 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.384 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.385 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:43.386 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.387 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = 172.21.15.122 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = a05abb94-fa63-4b8c-8984-45e6159773f5 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.388 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-16T17:46:43.389 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.390 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-16T17:46:43.391 INFO:teuthology.orchestra.run.smithi083.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.392 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-16T17:46:43.393 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:43.394 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.395 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw = 20 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw notification = 20 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: debug rgw sync = 20 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: rgw crypt require ssl = False 2024-04-16T17:46:43.396 INFO:teuthology.orchestra.run.smithi083.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw curl low speed time = 300 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw data log num shards = 4 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw md log max shards = 4 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: setgroup = ceph 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: setuser = ceph 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-16T17:46:43.397 INFO:teuthology.orchestra.run.smithi083.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi083.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi083.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi083.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:43.398 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:43.399 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2024-04-16T17:46:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = 172.21.15.122 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = a05abb94-fa63-4b8c-8984-45e6159773f5 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug trim objects = true 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.403 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2024-04-16T17:46:43.404 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.405 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2024-04-16T17:46:43.406 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw = 20 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw notification = 20 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: debug rgw sync = 20 2024-04-16T17:46:43.407 INFO:teuthology.orchestra.run.smithi071.stdout: rgw crypt require ssl = False 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw curl low speed time = 300 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw data log num shards = 4 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw md log max shards = 4 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: setgroup = ceph 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: setuser = ceph 2024-04-16T17:46:43.408 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-04-16T17:46:43.409 INFO:teuthology.orchestra.run.smithi071.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-04-16T17:46:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-04-16T17:46:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = 172.21.15.122 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = a05abb94-fa63-4b8c-8984-45e6159773f5 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.416 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.417 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw = 20 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw notification = 20 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: debug rgw sync = 20 2024-04-16T17:46:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: rgw crypt require ssl = False 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw curl low speed time = 300 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw data log num shards = 4 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw md log max shards = 4 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: setgroup = ceph 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: setuser = ceph 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-04-16T17:46:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:43.427 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-16T17:46:43.427 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-16T17:46:43.428 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-16T17:46:43.429 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T17:46:43.430 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = 172.21.15.122 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = a05abb94-fa63-4b8c-8984-45e6159773f5 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-16T17:46:43.431 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-16T17:46:43.432 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-16T17:46:43.433 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-16T17:46:43.434 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: rocksdb delete range threshold = 0 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-16T17:46:43.435 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-16T17:46:43.436 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-16T17:46:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-16T17:46:43.438 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw = 20 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw lifecycle = 20 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw notification = 20 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: debug rgw sync = 20 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: rgw crypt require ssl = False 2024-04-16T17:46:43.439 INFO:teuthology.orchestra.run.smithi188.stdout: rgw crypt s3 kms backend = testing 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw curl low speed time = 300 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw data log num shards = 4 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw data sync poll interval = 5 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw md log max shards = 4 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw meta sync poll interval = 5 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync data inject err probability = 0.1 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync log trim interval = 0 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync meta inject err probability = 0.1 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: rgw sync obj etag verify = True 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: setgroup = ceph 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout: setuser = ceph 2024-04-16T17:46:43.440 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout: rgw data notify interval msec = 0 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout: osd_max_omap_entries_per_request = 10 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout: osd_max_omap_entries_per_request = 1000 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-04-16T17:46:43.441 INFO:teuthology.orchestra.run.smithi188.stdout: osd_max_omap_entries_per_request = 10000 2024-04-16T17:46:43.441 INFO:tasks.ceph:Creating admin key on c2.mon.a... 2024-04-16T17:46:43.441 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c2.keyring 2024-04-16T17:46:43.517 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-16T17:46:43.518 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:43.518 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/c2.keyring of=/dev/stdout 2024-04-16T17:46:43.524 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:43.524 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/c2.monmap of=/dev/stdout 2024-04-16T17:46:43.571 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T17:46:43.571 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:43.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/c2.keyring 2024-04-16T17:46:43.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/c2.keyring 2024-04-16T17:46:43.595 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T17:46:43.595 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:43.642 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T17:46:43.642 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:43.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/c2.keyring 2024-04-16T17:46:43.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/c2.keyring 2024-04-16T17:46:43.665 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-16T17:46:43.665 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:43.714 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T17:46:43.714 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:43.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/c2.keyring 2024-04-16T17:46:43.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/c2.keyring 2024-04-16T17:46:43.738 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:46:43.739 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:43.787 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T17:46:43.787 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:43.787 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/c2.keyring 2024-04-16T17:46:43.787 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/c2.keyring 2024-04-16T17:46:43.809 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:46:43.809 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:43.858 INFO:tasks.ceph:Setting up mon nodes... 2024-04-16T17:46:43.858 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-16T17:46:43.858 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/c2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c2-x/keyring 2024-04-16T17:46:43.942 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/c2-x/keyring 2024-04-16T17:46:43.951 INFO:tasks.ceph:Setting up mds nodes... 2024-04-16T17:46:43.951 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-16T17:46:43.951 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c2.client.0.keyring && sudo chmod 0644 /etc/ceph/c2.client.0.keyring 2024-04-16T17:46:43.987 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/c2.client.0.keyring 2024-04-16T17:46:43.999 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c2.client.1.keyring && sudo chmod 0644 /etc/ceph/c2.client.1.keyring 2024-04-16T17:46:44.029 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/c2.client.1.keyring 2024-04-16T17:46:44.040 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-16T17:46:44.041 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'c1.osd.3': '/dev/vg_nvme/lv_1', 'c1.osd.4': '/dev/vg_nvme/lv_2', 'c1.osd.5': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'c2.osd.3': '/dev/vg_nvme/lv_1', 'c2.osd.4': '/dev/vg_nvme/lv_2', 'c2.osd.5': '/dev/vg_nvme/lv_3'}} 2024-04-16T17:46:44.041 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/c2-0 2024-04-16T17:46:44.054 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:44.054 INFO:tasks.ceph:role: c2.osd.0 2024-04-16T17:46:44.054 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T17:46:44.054 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T17:46:44.120 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:44.120 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:44.121 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:44.570 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T17:46:44.572 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T17:46:44.572 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c2-0 2024-04-16T17:46:44.673 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-0 2024-04-16T17:46:44.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:44.683 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:44.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/c2-1 2024-04-16T17:46:44.736 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:44.736 INFO:tasks.ceph:role: c2.osd.1 2024-04-16T17:46:44.736 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T17:46:44.737 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T17:46:44.798 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:44.798 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:44.799 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:45.280 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T17:46:45.282 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T17:46:45.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c2-1 2024-04-16T17:46:45.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-1 2024-04-16T17:46:45.430 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:45.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:45.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/c2-2 2024-04-16T17:46:45.486 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:45.487 INFO:tasks.ceph:role: c2.osd.2 2024-04-16T17:46:45.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T17:46:45.487 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-16T17:46:45.551 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:45.551 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:45.551 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:45.551 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:45.551 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:45.551 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:45.552 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:45.552 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:45.552 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:45.552 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:46.026 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T17:46:46.029 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T17:46:46.029 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c2-2 2024-04-16T17:46:46.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-2 2024-04-16T17:46:46.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:46.183 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:46.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:46.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:46.299+0000 7f37b0b097c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-0/keyring: can't open /var/lib/ceph/osd/c2-0/keyring: (2) No such file or directory 2024-04-16T17:46:46.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:46.299+0000 7f37b0b097c0 -1 created new key in keyring /var/lib/ceph/osd/c2-0/keyring 2024-04-16T17:46:46.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:46.299+0000 7f37b0b097c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2024-04-16T17:46:46.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:46.299+0000 7f37b0b097c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2024-04-16T17:46:46.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:46.299+0000 7f37b0b097c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2024-04-16T17:46:46.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:46.299+0000 7f37b0b097c0 -1 bluestore(/var/lib/ceph/osd/c2-0) _read_fsid unparsable uuid 2024-04-16T17:46:48.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-0 2024-04-16T17:46:48.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:48.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:48.571+0000 7f41c3a497c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-1/keyring: can't open /var/lib/ceph/osd/c2-1/keyring: (2) No such file or directory 2024-04-16T17:46:48.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:48.571+0000 7f41c3a497c0 -1 created new key in keyring /var/lib/ceph/osd/c2-1/keyring 2024-04-16T17:46:48.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:48.571+0000 7f41c3a497c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2024-04-16T17:46:48.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:48.571+0000 7f41c3a497c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2024-04-16T17:46:48.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:48.571+0000 7f41c3a497c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2024-04-16T17:46:48.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:48.575+0000 7f41c3a497c0 -1 bluestore(/var/lib/ceph/osd/c2-1) _read_fsid unparsable uuid 2024-04-16T17:46:50.761 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-1 2024-04-16T17:46:50.814 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:50.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:50.895+0000 7f87e67417c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-2/keyring: can't open /var/lib/ceph/osd/c2-2/keyring: (2) No such file or directory 2024-04-16T17:46:50.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:50.895+0000 7f87e67417c0 -1 created new key in keyring /var/lib/ceph/osd/c2-2/keyring 2024-04-16T17:46:50.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:50.895+0000 7f87e67417c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2024-04-16T17:46:50.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:50.895+0000 7f87e67417c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2024-04-16T17:46:50.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:50.895+0000 7f87e67417c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2024-04-16T17:46:50.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:46:50.899+0000 7f87e67417c0 -1 bluestore(/var/lib/ceph/osd/c2-2) _read_fsid unparsable uuid 2024-04-16T17:46:53.089 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-2 2024-04-16T17:46:53.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/c2-3 2024-04-16T17:46:53.157 INFO:tasks.ceph:roles_to_devs: {'c2.osd.3': '/dev/vg_nvme/lv_1', 'c2.osd.4': '/dev/vg_nvme/lv_2', 'c2.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:53.157 INFO:tasks.ceph:role: c2.osd.3 2024-04-16T17:46:53.157 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T17:46:53.157 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T17:46:53.221 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:53.221 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:53.221 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:53.221 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:53.221 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:53.222 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:53.222 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:53.222 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:53.222 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:53.222 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:53.712 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-16T17:46:53.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-16T17:46:53.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c2-3 2024-04-16T17:46:53.875 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-3 2024-04-16T17:46:53.886 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:53.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:53.887 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/c2-4 2024-04-16T17:46:53.940 INFO:tasks.ceph:roles_to_devs: {'c2.osd.3': '/dev/vg_nvme/lv_1', 'c2.osd.4': '/dev/vg_nvme/lv_2', 'c2.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:53.940 INFO:tasks.ceph:role: c2.osd.4 2024-04-16T17:46:53.940 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T17:46:53.940 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:54.004 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:54.005 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:54.005 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:54.005 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:54.450 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-16T17:46:54.452 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-16T17:46:54.453 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c2-4 2024-04-16T17:46:54.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-4 2024-04-16T17:46:54.585 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:54.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:54.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/c2-5 2024-04-16T17:46:54.641 INFO:tasks.ceph:roles_to_devs: {'c2.osd.3': '/dev/vg_nvme/lv_1', 'c2.osd.4': '/dev/vg_nvme/lv_2', 'c2.osd.5': '/dev/vg_nvme/lv_3'} 2024-04-16T17:46:54.641 INFO:tasks.ceph:role: c2.osd.5 2024-04-16T17:46:54.641 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T17:46:54.641 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-16T17:46:54.705 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T17:46:54.706 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-16T17:46:54.706 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T17:46:54.706 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T17:46:55.169 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-16T17:46:55.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-16T17:46:55.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c2-5 2024-04-16T17:46:55.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-5 2024-04-16T17:46:55.311 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: /sbin/restorecon: command not found 2024-04-16T17:46:55.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:46:55.312 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:55.425 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:55.424+0000 7fd86aa447c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-3/keyring: can't open /var/lib/ceph/osd/c2-3/keyring: (2) No such file or directory 2024-04-16T17:46:55.428 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:55.424+0000 7fd86aa447c0 -1 created new key in keyring /var/lib/ceph/osd/c2-3/keyring 2024-04-16T17:46:55.428 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:55.424+0000 7fd86aa447c0 -1 bluestore(/var/lib/ceph/osd/c2-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-3/block: (2) No such file or directory 2024-04-16T17:46:55.428 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:55.424+0000 7fd86aa447c0 -1 bluestore(/var/lib/ceph/osd/c2-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-3/block: (2) No such file or directory 2024-04-16T17:46:55.429 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:55.424+0000 7fd86aa447c0 -1 bluestore(/var/lib/ceph/osd/c2-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-3/block: (2) No such file or directory 2024-04-16T17:46:55.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:55.432+0000 7fd86aa447c0 -1 bluestore(/var/lib/ceph/osd/c2-3) _read_fsid unparsable uuid 2024-04-16T17:46:57.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-3 2024-04-16T17:46:57.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:46:57.703 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:57.700+0000 7fdf747c57c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-4/keyring: can't open /var/lib/ceph/osd/c2-4/keyring: (2) No such file or directory 2024-04-16T17:46:57.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:57.700+0000 7fdf747c57c0 -1 created new key in keyring /var/lib/ceph/osd/c2-4/keyring 2024-04-16T17:46:57.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:57.700+0000 7fdf747c57c0 -1 bluestore(/var/lib/ceph/osd/c2-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-4/block: (2) No such file or directory 2024-04-16T17:46:57.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:57.700+0000 7fdf747c57c0 -1 bluestore(/var/lib/ceph/osd/c2-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-4/block: (2) No such file or directory 2024-04-16T17:46:57.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:57.700+0000 7fdf747c57c0 -1 bluestore(/var/lib/ceph/osd/c2-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-4/block: (2) No such file or directory 2024-04-16T17:46:57.705 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:46:57.700+0000 7fdf747c57c0 -1 bluestore(/var/lib/ceph/osd/c2-4) _read_fsid unparsable uuid 2024-04-16T17:46:59.888 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-4 2024-04-16T17:46:59.941 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:47:00.022 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:00.020+0000 7fc1baebc7c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-5/keyring: can't open /var/lib/ceph/osd/c2-5/keyring: (2) No such file or directory 2024-04-16T17:47:00.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:00.020+0000 7fc1baebc7c0 -1 created new key in keyring /var/lib/ceph/osd/c2-5/keyring 2024-04-16T17:47:00.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:00.020+0000 7fc1baebc7c0 -1 bluestore(/var/lib/ceph/osd/c2-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-5/block: (2) No such file or directory 2024-04-16T17:47:00.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:00.020+0000 7fc1baebc7c0 -1 bluestore(/var/lib/ceph/osd/c2-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-5/block: (2) No such file or directory 2024-04-16T17:47:00.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:00.020+0000 7fc1baebc7c0 -1 bluestore(/var/lib/ceph/osd/c2-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-5/block: (2) No such file or directory 2024-04-16T17:47:00.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:00.020+0000 7fc1baebc7c0 -1 bluestore(/var/lib/ceph/osd/c2-5) _read_fsid unparsable uuid 2024-04-16T17:47:02.208 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-5 2024-04-16T17:47:02.261 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-16T17:47:02.261 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:02.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/c2-0/keyring of=/dev/stdout 2024-04-16T17:47:02.275 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:02.276 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/c2-1/keyring of=/dev/stdout 2024-04-16T17:47:02.330 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:02.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/c2-2/keyring of=/dev/stdout 2024-04-16T17:47:02.385 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:02.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/c2-x/keyring of=/dev/stdout 2024-04-16T17:47:02.399 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:02.399 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/c2-3/keyring of=/dev/stdout 2024-04-16T17:47:02.453 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:02.453 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/c2-4/keyring of=/dev/stdout 2024-04-16T17:47:02.508 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:02.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/c2-5/keyring of=/dev/stdout 2024-04-16T17:47:02.564 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:02.564 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/c2.client.0.keyring of=/dev/stdout 2024-04-16T17:47:02.571 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:02.571 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/c2.client.1.keyring of=/dev/stdout 2024-04-16T17:47:02.610 INFO:tasks.ceph:Adding keys to all mons... 2024-04-16T17:47:02.610 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/c2.keyring 2024-04-16T17:47:02.624 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCGuR5ma+j4ERAAgIzdPxjHNq5Jy8jisS108w== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCIuR5m1zlfIhAAPLirW/TrsNgKb+6ZtZuygQ== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCKuR5mcielNRAAPG0bpZCskFfb/tFukdo2Ng== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCDuR5mpfFROBAA+Q8NMYqyRkBKzNT286DadA== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCPuR5mpUKDGRAA2sjPB6sVbqKObyBBfbj1Cw== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCRuR5ma8AQKhAAHDXl3XC1kB4d/HrSbDFfUQ== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCUuR5mhoZyARAAGWmHUU2sxPi5EpVF7nUpMw== 2024-04-16T17:47:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-04-16T17:47:02.626 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCDuR5mqmAJOxAAI9DthBB91WSEUpqGXwA/TA== 2024-04-16T17:47:02.626 INFO:teuthology.orchestra.run.smithi122.stdout:[client.1] 2024-04-16T17:47:02.626 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCEuR5mwSrmARAAwBld8pOsDw83/RWbfeLyFw== 2024-04-16T17:47:02.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:47:02.715 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:47:02.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:47:02.839 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T17:47:02.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:47:02.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:47:03.084 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T17:47:03.164 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T17:47:03.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T17:47:03.293 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-16T17:47:03.293 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/c2-a 2024-04-16T17:47:03.343 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c2 --mkfs -i a --monmap /home/ubuntu/cephtest/c2.monmap --keyring /etc/ceph/c2.keyring 2024-04-16T17:47:03.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c2-a 2024-04-16T17:47:03.479 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/c2.monmap 2024-04-16T17:47:03.525 INFO:tasks.ceph:Starting mon daemons in cluster c2... 2024-04-16T17:47:03.525 INFO:tasks.ceph.c2.mon.a:Restarting daemon 2024-04-16T17:47:03.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster c2 -i a 2024-04-16T17:47:03.567 INFO:tasks.ceph.c2.mon.a:Started 2024-04-16T17:47:03.567 INFO:tasks.ceph:Starting mgr daemons in cluster c2... 2024-04-16T17:47:03.567 INFO:tasks.ceph.c2.mgr.x:Restarting daemon 2024-04-16T17:47:03.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster c2 -i x 2024-04-16T17:47:03.570 INFO:tasks.ceph.c2.mgr.x:Started 2024-04-16T17:47:03.570 INFO:tasks.ceph:Setting crush tunables to default 2024-04-16T17:47:03.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster c2 osd crush tunables default 2024-04-16T17:47:03.775 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-04-16T17:47:03.792 INFO:tasks.ceph:check_enable_crimson: False 2024-04-16T17:47:03.792 INFO:tasks.ceph:Starting osd daemons in cluster c2... 2024-04-16T17:47:03.793 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:03.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/c2-0/fsid of=/dev/stdout 2024-04-16T17:47:03.804 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:03.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/c2-1/fsid of=/dev/stdout 2024-04-16T17:47:03.861 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T17:47:03.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/c2-2/fsid of=/dev/stdout 2024-04-16T17:47:03.912 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:03.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/c2-3/fsid of=/dev/stdout 2024-04-16T17:47:03.928 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:03.928 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/c2-4/fsid of=/dev/stdout 2024-04-16T17:47:03.971 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:03.968+0000 7f8a5cd54180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:47:03.980 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-16T17:47:03.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/c2-5/fsid of=/dev/stdout 2024-04-16T17:47:04.032 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster c2 osd new 27732649-3364-48f9-b5ec-bb7a1defa562 0 2024-04-16T17:47:04.284 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-04-16T17:47:04.302 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster c2 osd new 7190ae20-d829-427c-9b81-2ed841511361 1 2024-04-16T17:47:04.509 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-16T17:47:04.513 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:04.512+0000 7f8a5cd54180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:47:04.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster c2 osd new dab9cde9-4087-46d4-b2cc-dfd5f4143c64 2 2024-04-16T17:47:04.716 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-04-16T17:47:04.728 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:04.724+0000 7f8a5cd54180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:47:04.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster c2 osd new fc7625e4-67e0-4f24-b035-22e269110ec5 3 2024-04-16T17:47:04.937 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-04-16T17:47:04.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster c2 osd new 592c1327-3b3b-467d-a6b1-c677580b88db 4 2024-04-16T17:47:05.156 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-04-16T17:47:05.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster c2 osd new 1348c53c-a16a-4fb1-8e56-3b193328bb95 5 2024-04-16T17:47:05.319 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.316+0000 7f8a5cd54180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:47:05.358 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-04-16T17:47:05.374 INFO:tasks.ceph.c2.osd.0:Restarting daemon 2024-04-16T17:47:05.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 0 2024-04-16T17:47:05.377 INFO:tasks.ceph.c2.osd.0:Started 2024-04-16T17:47:05.377 INFO:tasks.ceph.c2.osd.1:Restarting daemon 2024-04-16T17:47:05.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 1 2024-04-16T17:47:05.379 INFO:tasks.ceph.c2.osd.1:Started 2024-04-16T17:47:05.379 INFO:tasks.ceph.c2.osd.2:Restarting daemon 2024-04-16T17:47:05.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 2 2024-04-16T17:47:05.382 INFO:tasks.ceph.c2.osd.2:Started 2024-04-16T17:47:05.382 INFO:tasks.ceph.c2.osd.3:Restarting daemon 2024-04-16T17:47:05.382 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 3 2024-04-16T17:47:05.384 INFO:tasks.ceph.c2.osd.3:Started 2024-04-16T17:47:05.384 INFO:tasks.ceph.c2.osd.4:Restarting daemon 2024-04-16T17:47:05.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 4 2024-04-16T17:47:05.386 INFO:tasks.ceph.c2.osd.4:Started 2024-04-16T17:47:05.386 INFO:tasks.ceph.c2.osd.5:Restarting daemon 2024-04-16T17:47:05.386 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 5 2024-04-16T17:47:05.388 INFO:tasks.ceph.c2.osd.5:Started 2024-04-16T17:47:05.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2024-04-16T17:47:05.406 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.404+0000 7f8a5cd54180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:47:05.475 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.472+0000 7f8a5cd54180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:47:05.552 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.548+0000 7f8a5cd54180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:47:05.577 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:05.577 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"fsid":"a05abb94-fa63-4b8c-8984-45e6159773f5","created":"2024-04-16T17:47:03.675290+0000","modified":"2024-04-16T17:47:05.359563+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T17:47:05.359563+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"27732649-3364-48f9-b5ec-bb7a1defa562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7190ae20-d829-427c-9b81-2ed841511361","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dab9cde9-4087-46d4-b2cc-dfd5f4143c64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fc7625e4-67e0-4f24-b035-22e269110ec5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"592c1327-3b3b-467d-a6b1-c677580b88db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1348c53c-a16a-4fb1-8e56-3b193328bb95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:47:05.594 INFO:tasks.ceph.ceph_manager.c2:[] 2024-04-16T17:47:05.594 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-16T17:47:05.603 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.600+0000 7f8a5cd54180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:47:05.679 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.676+0000 7f8a5cd54180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:47:05.731 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.728+0000 7f8a5cd54180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:47:05.880 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.876+0000 7f8a5cd54180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:47:05.940 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:05.936+0000 7f8a5cd54180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:47:05.992 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T17:47:05.991+0000 7ff3512127c0 -1 Falling back to public interface 2024-04-16T17:47:06.002 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T17:47:06.000+0000 7fc2f37007c0 -1 Falling back to public interface 2024-04-16T17:47:06.003 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T17:47:06.000+0000 7f7277ec57c0 -1 Falling back to public interface 2024-04-16T17:47:06.007 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T17:47:06.007+0000 7f21c035c7c0 -1 Falling back to public interface 2024-04-16T17:47:06.011 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T17:47:06.011+0000 7f0feb8d27c0 -1 Falling back to public interface 2024-04-16T17:47:06.025 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T17:47:06.024+0000 7f7b2271d7c0 -1 Falling back to public interface 2024-04-16T17:47:06.197 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2024-04-16T17:47:06.386 INFO:teuthology.misc.health.smithi122.stdout: 2024-04-16T17:47:06.387 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":8,"fsid":"a05abb94-fa63-4b8c-8984-45e6159773f5","created":"2024-04-16T17:47:03.675290+0000","modified":"2024-04-16T17:47:05.359563+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T17:47:05.359563+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"27732649-3364-48f9-b5ec-bb7a1defa562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7190ae20-d829-427c-9b81-2ed841511361","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dab9cde9-4087-46d4-b2cc-dfd5f4143c64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fc7625e4-67e0-4f24-b035-22e269110ec5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"592c1327-3b3b-467d-a6b1-c677580b88db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1348c53c-a16a-4fb1-8e56-3b193328bb95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:47:06.402 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-04-16T17:47:06.568 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-16T17:47:06.568 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-16T17:47:06.568 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr: from numpy import show_config as show_numpy_config 2024-04-16T17:47:06.611 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:06.608+0000 7f8a5cd54180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:47:06.744 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:06.740+0000 7f8a5cd54180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:47:06.890 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:06.888+0000 7f8a5cd54180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:47:06.953 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:06.952+0000 7f8a5cd54180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:47:07.086 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:07.084+0000 7f8a5cd54180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:47:07.691 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:07.688+0000 7f8a5cd54180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:47:07.765 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:07.764+0000 7f8a5cd54180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:47:07.946 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:07.944+0000 7f8a5cd54180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:47:08.019 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:08.016+0000 7f8a5cd54180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:47:08.303 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:08.300+0000 7f8a5cd54180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:47:08.364 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:08.360+0000 7f8a5cd54180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:47:08.546 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T17:47:08.547+0000 7ff3512127c0 -1 osd.1 0 log_to_monitors true 2024-04-16T17:47:08.812 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T17:47:08.808+0000 7f7277ec57c0 -1 osd.4 0 log_to_monitors true 2024-04-16T17:47:08.931 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T17:47:08.931+0000 7f21c035c7c0 -1 osd.2 0 log_to_monitors true 2024-04-16T17:47:09.057 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T17:47:09.055+0000 7f0feb8d27c0 -1 osd.0 0 log_to_monitors true 2024-04-16T17:47:09.058 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T17:47:09.056+0000 7f7b2271d7c0 -1 osd.5 0 log_to_monitors true 2024-04-16T17:47:09.108 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T17:47:09.104+0000 7fc2f37007c0 -1 osd.3 0 log_to_monitors true 2024-04-16T17:47:09.398 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T17:47:09.396+0000 7f8a041a1640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-16T17:47:11.822 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T17:47:11.819+0000 7f21bc305640 -1 osd.2 0 waiting for initial osdmap 2024-04-16T17:47:11.832 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T17:47:11.831+0000 7ff34d1bb640 -1 osd.1 0 waiting for initial osdmap 2024-04-16T17:47:11.841 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T17:47:11.839+0000 7f0fe787b640 -1 osd.0 0 waiting for initial osdmap 2024-04-16T17:47:11.844 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T17:47:11.843+0000 7f21b4139640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:47:11.855 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T17:47:11.855+0000 7ff344fef640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:47:11.870 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T17:47:11.871+0000 7f0fdf6af640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:47:11.891 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T17:47:11.888+0000 7f7273e6e640 -1 osd.4 0 waiting for initial osdmap 2024-04-16T17:47:11.909 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T17:47:11.908+0000 7f7b1e6c6640 -1 osd.5 0 waiting for initial osdmap 2024-04-16T17:47:11.914 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T17:47:11.912+0000 7fc2ef6a9640 -1 osd.3 0 waiting for initial osdmap 2024-04-16T17:47:11.918 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T17:47:11.916+0000 7f726bca2640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:47:11.934 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T17:47:11.932+0000 7f7b164fa640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:47:11.946 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T17:47:11.944+0000 7fc2e74dd640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:47:13.005 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2024-04-16T17:47:13.303 INFO:teuthology.misc.health.smithi122.stdout: 2024-04-16T17:47:13.303 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":11,"fsid":"a05abb94-fa63-4b8c-8984-45e6159773f5","created":"2024-04-16T17:47:03.675290+0000","modified":"2024-04-16T17:47:12.847163+0000","last_up_change":"2024-04-16T17:47:12.847163+0000","last_in_change":"2024-04-16T17:47:05.359563+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"27732649-3364-48f9-b5ec-bb7a1defa562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6817","nonce":2098402233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6819","nonce":2098402233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6823","nonce":2098402233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6821","nonce":2098402233}]},"public_addr":"172.21.15.122:6817/2098402233","cluster_addr":"172.21.15.122:6819/2098402233","heartbeat_back_addr":"172.21.15.122:6823/2098402233","heartbeat_front_addr":"172.21.15.122:6821/2098402233","state":["exists","up"]},{"osd":1,"uuid":"7190ae20-d829-427c-9b81-2ed841511361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6801","nonce":2106220236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6803","nonce":2106220236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6807","nonce":2106220236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6805","nonce":2106220236}]},"public_addr":"172.21.15.122:6801/2106220236","cluster_addr":"172.21.15.122:6803/2106220236","heartbeat_back_addr":"172.21.15.122:6807/2106220236","heartbeat_front_addr":"172.21.15.122:6805/2106220236","state":["exists","up"]},{"osd":2,"uuid":"dab9cde9-4087-46d4-b2cc-dfd5f4143c64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6809","nonce":2770906349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6811","nonce":2770906349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6815","nonce":2770906349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6813","nonce":2770906349}]},"public_addr":"172.21.15.122:6809/2770906349","cluster_addr":"172.21.15.122:6811/2770906349","heartbeat_back_addr":"172.21.15.122:6815/2770906349","heartbeat_front_addr":"172.21.15.122:6813/2770906349","state":["exists","up"]},{"osd":3,"uuid":"fc7625e4-67e0-4f24-b035-22e269110ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6802","nonce":3793059283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6806","nonce":3793059283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6814","nonce":3793059283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6810","nonce":3793059283}]},"public_addr":"172.21.15.188:6802/3793059283","cluster_addr":"172.21.15.188:6806/3793059283","heartbeat_back_addr":"172.21.15.188:6814/3793059283","heartbeat_front_addr":"172.21.15.188:6810/3793059283","state":["exists","up"]},{"osd":4,"uuid":"592c1327-3b3b-467d-a6b1-c677580b88db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6803","nonce":3538207696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6807","nonce":3538207696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6815","nonce":3538207696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6811","nonce":3538207696}]},"public_addr":"172.21.15.188:6803/3538207696","cluster_addr":"172.21.15.188:6807/3538207696","heartbeat_back_addr":"172.21.15.188:6815/3538207696","heartbeat_front_addr":"172.21.15.188:6811/3538207696","state":["exists","up"]},{"osd":5,"uuid":"1348c53c-a16a-4fb1-8e56-3b193328bb95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6817","nonce":2109975140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6819","nonce":2109975140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6823","nonce":2109975140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6821","nonce":2109975140}]},"public_addr":"172.21.15.188:6817/2109975140","cluster_addr":"172.21.15.188:6819/2109975140","heartbeat_back_addr":"172.21.15.188:6823/2109975140","heartbeat_front_addr":"172.21.15.188:6821/2109975140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:47:13.320 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-04-16T17:47:13.320 INFO:tasks.ceph:Creating RBD pool 2024-04-16T17:47:13.320 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster c2 osd pool create rbd 8 2024-04-16T17:47:13.848 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'rbd' created 2024-04-16T17:47:13.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster c2 osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-16T17:47:14.853 INFO:teuthology.orchestra.run.smithi122.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-16T17:47:14.880 INFO:tasks.ceph:Starting mds daemons in cluster c2... 2024-04-16T17:47:14.881 INFO:tasks.ceph:Waiting until c2 daemons up and pgs clean... 2024-04-16T17:47:14.881 INFO:tasks.ceph.ceph_manager.c2:waiting for mgr available 2024-04-16T17:47:14.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 mgr dump --format=json 2024-04-16T17:47:14.881 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-16T17:47:15.225 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:15.243 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":4,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":246140045},{"type":"v1","addr":"172.21.15.188:6825","nonce":246140045}]},"active_addr":"172.21.15.188:6825/246140045","active_change":"2024-04-16T17:47:08.386443+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":3099345557}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":1286277339}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":1895849417}]}]} 2024-04-16T17:47:15.245 INFO:tasks.ceph.ceph_manager.c2:mgr available! 2024-04-16T17:47:15.245 INFO:tasks.ceph.ceph_manager.c2:waiting for all up 2024-04-16T17:47:15.246 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2024-04-16T17:47:15.540 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:15.540 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"fsid":"a05abb94-fa63-4b8c-8984-45e6159773f5","created":"2024-04-16T17:47:03.675290+0000","modified":"2024-04-16T17:47:14.850866+0000","last_up_change":"2024-04-16T17:47:12.847163+0000","last_in_change":"2024-04-16T17:47:05.359563+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-16T17:47:13.622638+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-16T17:47:14.443765+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"27732649-3364-48f9-b5ec-bb7a1defa562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6817","nonce":2098402233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6819","nonce":2098402233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6823","nonce":2098402233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6821","nonce":2098402233}]},"public_addr":"172.21.15.122:6817/2098402233","cluster_addr":"172.21.15.122:6819/2098402233","heartbeat_back_addr":"172.21.15.122:6823/2098402233","heartbeat_front_addr":"172.21.15.122:6821/2098402233","state":["exists","up"]},{"osd":1,"uuid":"7190ae20-d829-427c-9b81-2ed841511361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6801","nonce":2106220236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6803","nonce":2106220236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6807","nonce":2106220236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6805","nonce":2106220236}]},"public_addr":"172.21.15.122:6801/2106220236","cluster_addr":"172.21.15.122:6803/2106220236","heartbeat_back_addr":"172.21.15.122:6807/2106220236","heartbeat_front_addr":"172.21.15.122:6805/2106220236","state":["exists","up"]},{"osd":2,"uuid":"dab9cde9-4087-46d4-b2cc-dfd5f4143c64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6809","nonce":2770906349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6811","nonce":2770906349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6815","nonce":2770906349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6813","nonce":2770906349}]},"public_addr":"172.21.15.122:6809/2770906349","cluster_addr":"172.21.15.122:6811/2770906349","heartbeat_back_addr":"172.21.15.122:6815/2770906349","heartbeat_front_addr":"172.21.15.122:6813/2770906349","state":["exists","up"]},{"osd":3,"uuid":"fc7625e4-67e0-4f24-b035-22e269110ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6802","nonce":3793059283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6806","nonce":3793059283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6814","nonce":3793059283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6810","nonce":3793059283}]},"public_addr":"172.21.15.188:6802/3793059283","cluster_addr":"172.21.15.188:6806/3793059283","heartbeat_back_addr":"172.21.15.188:6814/3793059283","heartbeat_front_addr":"172.21.15.188:6810/3793059283","state":["exists","up"]},{"osd":4,"uuid":"592c1327-3b3b-467d-a6b1-c677580b88db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6803","nonce":3538207696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6807","nonce":3538207696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6815","nonce":3538207696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6811","nonce":3538207696}]},"public_addr":"172.21.15.188:6803/3538207696","cluster_addr":"172.21.15.188:6807/3538207696","heartbeat_back_addr":"172.21.15.188:6815/3538207696","heartbeat_front_addr":"172.21.15.188:6811/3538207696","state":["exists","up"]},{"osd":5,"uuid":"1348c53c-a16a-4fb1-8e56-3b193328bb95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6817","nonce":2109975140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6819","nonce":2109975140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6823","nonce":2109975140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6821","nonce":2109975140}]},"public_addr":"172.21.15.188:6817/2109975140","cluster_addr":"172.21.15.188:6819/2109975140","heartbeat_back_addr":"172.21.15.188:6823/2109975140","heartbeat_front_addr":"172.21.15.188:6821/2109975140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:10.082669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:09.563488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:09.968329+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:10.089449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:09.844200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:10.096728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:47:15.559 INFO:tasks.ceph.ceph_manager.c2:all up! 2024-04-16T17:47:15.559 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2024-04-16T17:47:15.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:15.854 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"fsid":"a05abb94-fa63-4b8c-8984-45e6159773f5","created":"2024-04-16T17:47:03.675290+0000","modified":"2024-04-16T17:47:14.850866+0000","last_up_change":"2024-04-16T17:47:12.847163+0000","last_in_change":"2024-04-16T17:47:05.359563+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-16T17:47:13.622638+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-16T17:47:14.443765+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"27732649-3364-48f9-b5ec-bb7a1defa562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6817","nonce":2098402233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6819","nonce":2098402233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6823","nonce":2098402233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2098402233},{"type":"v1","addr":"172.21.15.122:6821","nonce":2098402233}]},"public_addr":"172.21.15.122:6817/2098402233","cluster_addr":"172.21.15.122:6819/2098402233","heartbeat_back_addr":"172.21.15.122:6823/2098402233","heartbeat_front_addr":"172.21.15.122:6821/2098402233","state":["exists","up"]},{"osd":1,"uuid":"7190ae20-d829-427c-9b81-2ed841511361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6801","nonce":2106220236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6803","nonce":2106220236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6807","nonce":2106220236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2106220236},{"type":"v1","addr":"172.21.15.122:6805","nonce":2106220236}]},"public_addr":"172.21.15.122:6801/2106220236","cluster_addr":"172.21.15.122:6803/2106220236","heartbeat_back_addr":"172.21.15.122:6807/2106220236","heartbeat_front_addr":"172.21.15.122:6805/2106220236","state":["exists","up"]},{"osd":2,"uuid":"dab9cde9-4087-46d4-b2cc-dfd5f4143c64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6809","nonce":2770906349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6811","nonce":2770906349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6815","nonce":2770906349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2770906349},{"type":"v1","addr":"172.21.15.122:6813","nonce":2770906349}]},"public_addr":"172.21.15.122:6809/2770906349","cluster_addr":"172.21.15.122:6811/2770906349","heartbeat_back_addr":"172.21.15.122:6815/2770906349","heartbeat_front_addr":"172.21.15.122:6813/2770906349","state":["exists","up"]},{"osd":3,"uuid":"fc7625e4-67e0-4f24-b035-22e269110ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6802","nonce":3793059283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6806","nonce":3793059283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6814","nonce":3793059283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3793059283},{"type":"v1","addr":"172.21.15.188:6810","nonce":3793059283}]},"public_addr":"172.21.15.188:6802/3793059283","cluster_addr":"172.21.15.188:6806/3793059283","heartbeat_back_addr":"172.21.15.188:6814/3793059283","heartbeat_front_addr":"172.21.15.188:6810/3793059283","state":["exists","up"]},{"osd":4,"uuid":"592c1327-3b3b-467d-a6b1-c677580b88db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6803","nonce":3538207696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6807","nonce":3538207696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6815","nonce":3538207696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":3538207696},{"type":"v1","addr":"172.21.15.188:6811","nonce":3538207696}]},"public_addr":"172.21.15.188:6803/3538207696","cluster_addr":"172.21.15.188:6807/3538207696","heartbeat_back_addr":"172.21.15.188:6815/3538207696","heartbeat_front_addr":"172.21.15.188:6811/3538207696","state":["exists","up"]},{"osd":5,"uuid":"1348c53c-a16a-4fb1-8e56-3b193328bb95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6817","nonce":2109975140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6819","nonce":2109975140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6823","nonce":2109975140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2109975140},{"type":"v1","addr":"172.21.15.188:6821","nonce":2109975140}]},"public_addr":"172.21.15.188:6817/2109975140","cluster_addr":"172.21.15.188:6819/2109975140","heartbeat_back_addr":"172.21.15.188:6823/2109975140","heartbeat_front_addr":"172.21.15.188:6821/2109975140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:10.082669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:09.563488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:09.968329+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:10.089449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:09.844200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:47:10.096728+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:47:15.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.0 flush_pg_stats 2024-04-16T17:47:15.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.1 flush_pg_stats 2024-04-16T17:47:15.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.2 flush_pg_stats 2024-04-16T17:47:15.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.3 flush_pg_stats 2024-04-16T17:47:15.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.4 flush_pg_stats 2024-04-16T17:47:15.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.5 flush_pg_stats 2024-04-16T17:47:16.060 INFO:teuthology.orchestra.run.smithi122.stdout:47244640261 2024-04-16T17:47:16.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.1 2024-04-16T17:47:16.079 INFO:teuthology.orchestra.run.smithi122.stdout:47244640261 2024-04-16T17:47:16.080 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.5 2024-04-16T17:47:16.109 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:16.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.2 2024-04-16T17:47:16.114 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:16.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.3 2024-04-16T17:47:16.116 INFO:teuthology.orchestra.run.smithi122.stdout:47244640259 2024-04-16T17:47:16.117 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.4 2024-04-16T17:47:16.130 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:16.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2024-04-16T17:47:16.471 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:16.490 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640261 got 47244640260 for osd.1 2024-04-16T17:47:16.508 INFO:teuthology.orchestra.run.smithi122.stdout:47244640259 2024-04-16T17:47:16.525 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640260 got 47244640259 for osd.0 2024-04-16T17:47:16.552 INFO:teuthology.orchestra.run.smithi122.stdout:47244640259 2024-04-16T17:47:16.560 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:16.570 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640260 got 47244640259 for osd.2 2024-04-16T17:47:16.583 INFO:teuthology.orchestra.run.smithi122.stdout:47244640259 2024-04-16T17:47:16.584 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640261 got 47244640260 for osd.5 2024-04-16T17:47:16.600 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640260 got 47244640259 for osd.3 2024-04-16T17:47:16.624 INFO:teuthology.orchestra.run.smithi122.stdout:47244640258 2024-04-16T17:47:16.640 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640259 got 47244640258 for osd.4 2024-04-16T17:47:17.491 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.1 2024-04-16T17:47:17.526 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2024-04-16T17:47:17.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.2 2024-04-16T17:47:17.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.5 2024-04-16T17:47:17.602 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.3 2024-04-16T17:47:17.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.4 2024-04-16T17:47:17.818 INFO:teuthology.orchestra.run.smithi122.stdout:47244640261 2024-04-16T17:47:17.836 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640261 got 47244640261 for osd.1 2024-04-16T17:47:17.836 DEBUG:teuthology.parallel:result is None 2024-04-16T17:47:17.966 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:17.976 INFO:teuthology.orchestra.run.smithi122.stdout:47244640259 2024-04-16T17:47:17.992 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640260 got 47244640260 for osd.2 2024-04-16T17:47:17.993 DEBUG:teuthology.parallel:result is None 2024-04-16T17:47:17.994 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640259 got 47244640259 for osd.4 2024-04-16T17:47:17.994 DEBUG:teuthology.parallel:result is None 2024-04-16T17:47:17.995 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:47244640260 2024-04-16T17:47:18.015 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640260 got 47244640260 for osd.0 2024-04-16T17:47:18.015 DEBUG:teuthology.parallel:result is None 2024-04-16T17:47:18.024 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640260 got 47244640260 for osd.3 2024-04-16T17:47:18.024 DEBUG:teuthology.parallel:result is None 2024-04-16T17:47:18.029 INFO:teuthology.orchestra.run.smithi122.stdout:47244640261 2024-04-16T17:47:18.045 INFO:tasks.ceph.ceph_manager.c2:need seq 47244640261 got 47244640261 for osd.5 2024-04-16T17:47:18.045 DEBUG:teuthology.parallel:result is None 2024-04-16T17:47:18.045 INFO:tasks.ceph.ceph_manager.c2:waiting for clean 2024-04-16T17:47:18.045 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2024-04-16T17:47:18.347 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:18.347 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-16T17:47:18.365 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-16T17:47:16.863868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":162832,"kb_used_data":696,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":628982768,"statfs":{"total":644245094400,"available":644078354432,"internally_reserved":0,"allocated":712704,"data_stored":199344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.468059"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.946826+0000","last_change":"2024-04-16T17:47:14.943872+0000","last_active":"2024-04-16T17:47:14.946826+0000","last_peered":"2024-04-16T17:47:14.946826+0000","last_clean":"2024-04-16T17:47:14.946826+0000","last_became_active":"2024-04-16T17:47:14.943110+0000","last_became_peered":"2024-04-16T17:47:14.943110+0000","last_unstale":"2024-04-16T17:47:14.946826+0000","last_undegraded":"2024-04-16T17:47:14.946826+0000","last_fullsized":"2024-04-16T17:47:14.946826+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:40:47.896235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.861341+0000","last_change":"2024-04-16T17:47:14.867267+0000","last_active":"2024-04-16T17:47:15.861341+0000","last_peered":"2024-04-16T17:47:15.861341+0000","last_clean":"2024-04-16T17:47:15.861341+0000","last_became_active":"2024-04-16T17:47:14.866544+0000","last_became_peered":"2024-04-16T17:47:14.866544+0000","last_unstale":"2024-04-16T17:47:15.861341+0000","last_undegraded":"2024-04-16T17:47:15.861341+0000","last_fullsized":"2024-04-16T17:47:15.861341+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:14.650446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.946753+0000","last_change":"2024-04-16T17:47:14.943802+0000","last_active":"2024-04-16T17:47:14.946753+0000","last_peered":"2024-04-16T17:47:14.946753+0000","last_clean":"2024-04-16T17:47:14.946753+0000","last_became_active":"2024-04-16T17:47:14.943102+0000","last_became_peered":"2024-04-16T17:47:14.943102+0000","last_unstale":"2024-04-16T17:47:14.946753+0000","last_undegraded":"2024-04-16T17:47:14.946753+0000","last_fullsized":"2024-04-16T17:47:14.946753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:29.991700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.860124+0000","last_change":"2024-04-16T17:47:14.866348+0000","last_active":"2024-04-16T17:47:15.860124+0000","last_peered":"2024-04-16T17:47:15.860124+0000","last_clean":"2024-04-16T17:47:15.860124+0000","last_became_active":"2024-04-16T17:47:14.865893+0000","last_became_peered":"2024-04-16T17:47:14.865893+0000","last_unstale":"2024-04-16T17:47:15.860124+0000","last_undegraded":"2024-04-16T17:47:15.860124+0000","last_fullsized":"2024-04-16T17:47:15.860124+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:14.650446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'64","reported_seq":97,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:16.029197+0000","last_change":"2024-04-16T17:47:15.866260+0000","last_active":"2024-04-16T17:47:16.029197+0000","last_peered":"2024-04-16T17:47:16.029197+0000","last_clean":"2024-04-16T17:47:16.029197+0000","last_became_active":"2024-04-16T17:47:15.865671+0000","last_became_peered":"2024-04-16T17:47:15.865671+0000","last_unstale":"2024-04-16T17:47:16.029197+0000","last_undegraded":"2024-04-16T17:47:16.029197+0000","last_fullsized":"2024-04-16T17:47:16.029197+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:14.850866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:14.850866+0000","last_clean_scrub_stamp":"2024-04-16T17:47:14.850866+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.939195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.860158+0000","last_change":"2024-04-16T17:47:14.867114+0000","last_active":"2024-04-16T17:47:15.860158+0000","last_peered":"2024-04-16T17:47:15.860158+0000","last_clean":"2024-04-16T17:47:15.860158+0000","last_became_active":"2024-04-16T17:47:14.866505+0000","last_became_peered":"2024-04-16T17:47:14.866505+0000","last_unstale":"2024-04-16T17:47:15.860158+0000","last_undegraded":"2024-04-16T17:47:15.860158+0000","last_fullsized":"2024-04-16T17:47:15.860158+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:24.817173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.871493+0000","last_change":"2024-04-16T17:47:14.869013+0000","last_active":"2024-04-16T17:47:14.871493+0000","last_peered":"2024-04-16T17:47:14.871493+0000","last_clean":"2024-04-16T17:47:14.871493+0000","last_became_active":"2024-04-16T17:47:14.868312+0000","last_became_peered":"2024-04-16T17:47:14.868312+0000","last_unstale":"2024-04-16T17:47:14.871493+0000","last_undegraded":"2024-04-16T17:47:14.871493+0000","last_fullsized":"2024-04-16T17:47:14.871493+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:46.184928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.868706+0000","last_change":"2024-04-16T17:47:14.867547+0000","last_active":"2024-04-16T17:47:14.868706+0000","last_peered":"2024-04-16T17:47:14.868706+0000","last_clean":"2024-04-16T17:47:14.868706+0000","last_became_active":"2024-04-16T17:47:14.866772+0000","last_became_peered":"2024-04-16T17:47:14.866772+0000","last_unstale":"2024-04-16T17:47:14.868706+0000","last_undegraded":"2024-04-16T17:47:14.868706+0000","last_fullsized":"2024-04-16T17:47:14.868706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:29.991700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.862042+0000","last_change":"2024-04-16T17:47:14.867407+0000","last_active":"2024-04-16T17:47:15.862042+0000","last_peered":"2024-04-16T17:47:15.862042+0000","last_clean":"2024-04-16T17:47:15.862042+0000","last_became_active":"2024-04-16T17:47:14.866677+0000","last_became_peered":"2024-04-16T17:47:14.866677+0000","last_unstale":"2024-04-16T17:47:15.862042+0000","last_undegraded":"2024-04-16T17:47:15.862042+0000","last_fullsized":"2024-04-16T17:47:15.862042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:24.817173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:47:18.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2024-04-16T17:47:18.644 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:18.644 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-16T17:47:18.660 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-16T17:47:18.396409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":162832,"kb_used_data":696,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":628982768,"statfs":{"total":644245094400,"available":644078354432,"internally_reserved":0,"allocated":712704,"data_stored":199344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000600"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.946826+0000","last_change":"2024-04-16T17:47:14.943872+0000","last_active":"2024-04-16T17:47:14.946826+0000","last_peered":"2024-04-16T17:47:14.946826+0000","last_clean":"2024-04-16T17:47:14.946826+0000","last_became_active":"2024-04-16T17:47:14.943110+0000","last_became_peered":"2024-04-16T17:47:14.943110+0000","last_unstale":"2024-04-16T17:47:14.946826+0000","last_undegraded":"2024-04-16T17:47:14.946826+0000","last_fullsized":"2024-04-16T17:47:14.946826+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:40:47.896235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.861341+0000","last_change":"2024-04-16T17:47:14.867267+0000","last_active":"2024-04-16T17:47:15.861341+0000","last_peered":"2024-04-16T17:47:15.861341+0000","last_clean":"2024-04-16T17:47:15.861341+0000","last_became_active":"2024-04-16T17:47:14.866544+0000","last_became_peered":"2024-04-16T17:47:14.866544+0000","last_unstale":"2024-04-16T17:47:15.861341+0000","last_undegraded":"2024-04-16T17:47:15.861341+0000","last_fullsized":"2024-04-16T17:47:15.861341+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:14.650446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.946753+0000","last_change":"2024-04-16T17:47:14.943802+0000","last_active":"2024-04-16T17:47:14.946753+0000","last_peered":"2024-04-16T17:47:14.946753+0000","last_clean":"2024-04-16T17:47:14.946753+0000","last_became_active":"2024-04-16T17:47:14.943102+0000","last_became_peered":"2024-04-16T17:47:14.943102+0000","last_unstale":"2024-04-16T17:47:14.946753+0000","last_undegraded":"2024-04-16T17:47:14.946753+0000","last_fullsized":"2024-04-16T17:47:14.946753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:29.991700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.860124+0000","last_change":"2024-04-16T17:47:14.866348+0000","last_active":"2024-04-16T17:47:15.860124+0000","last_peered":"2024-04-16T17:47:15.860124+0000","last_clean":"2024-04-16T17:47:15.860124+0000","last_became_active":"2024-04-16T17:47:14.865893+0000","last_became_peered":"2024-04-16T17:47:14.865893+0000","last_unstale":"2024-04-16T17:47:15.860124+0000","last_undegraded":"2024-04-16T17:47:15.860124+0000","last_fullsized":"2024-04-16T17:47:15.860124+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:14.650446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'64","reported_seq":97,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:16.029197+0000","last_change":"2024-04-16T17:47:15.866260+0000","last_active":"2024-04-16T17:47:16.029197+0000","last_peered":"2024-04-16T17:47:16.029197+0000","last_clean":"2024-04-16T17:47:16.029197+0000","last_became_active":"2024-04-16T17:47:15.865671+0000","last_became_peered":"2024-04-16T17:47:15.865671+0000","last_unstale":"2024-04-16T17:47:16.029197+0000","last_undegraded":"2024-04-16T17:47:16.029197+0000","last_fullsized":"2024-04-16T17:47:16.029197+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:14.850866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:14.850866+0000","last_clean_scrub_stamp":"2024-04-16T17:47:14.850866+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:32.939195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.860158+0000","last_change":"2024-04-16T17:47:14.867114+0000","last_active":"2024-04-16T17:47:15.860158+0000","last_peered":"2024-04-16T17:47:15.860158+0000","last_clean":"2024-04-16T17:47:15.860158+0000","last_became_active":"2024-04-16T17:47:14.866505+0000","last_became_peered":"2024-04-16T17:47:14.866505+0000","last_unstale":"2024-04-16T17:47:15.860158+0000","last_undegraded":"2024-04-16T17:47:15.860158+0000","last_fullsized":"2024-04-16T17:47:15.860158+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:24.817173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.871493+0000","last_change":"2024-04-16T17:47:14.869013+0000","last_active":"2024-04-16T17:47:14.871493+0000","last_peered":"2024-04-16T17:47:14.871493+0000","last_clean":"2024-04-16T17:47:14.871493+0000","last_became_active":"2024-04-16T17:47:14.868312+0000","last_became_peered":"2024-04-16T17:47:14.868312+0000","last_unstale":"2024-04-16T17:47:14.871493+0000","last_undegraded":"2024-04-16T17:47:14.871493+0000","last_fullsized":"2024-04-16T17:47:14.871493+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:46.184928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-04-16T17:47:14.868706+0000","last_change":"2024-04-16T17:47:14.867547+0000","last_active":"2024-04-16T17:47:14.868706+0000","last_peered":"2024-04-16T17:47:14.868706+0000","last_clean":"2024-04-16T17:47:14.868706+0000","last_became_active":"2024-04-16T17:47:14.866772+0000","last_became_peered":"2024-04-16T17:47:14.866772+0000","last_unstale":"2024-04-16T17:47:14.868706+0000","last_undegraded":"2024-04-16T17:47:14.868706+0000","last_fullsized":"2024-04-16T17:47:14.868706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:29.991700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-16T17:47:15.862042+0000","last_change":"2024-04-16T17:47:14.867407+0000","last_active":"2024-04-16T17:47:15.862042+0000","last_peered":"2024-04-16T17:47:15.862042+0000","last_clean":"2024-04-16T17:47:15.862042+0000","last_became_active":"2024-04-16T17:47:14.866677+0000","last_became_peered":"2024-04-16T17:47:14.866677+0000","last_unstale":"2024-04-16T17:47:15.862042+0000","last_undegraded":"2024-04-16T17:47:15.862042+0000","last_fullsized":"2024-04-16T17:47:15.862042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:13.848576+0000","last_clean_scrub_stamp":"2024-04-16T17:47:13.848576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:24.817173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27140,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830460,"statfs":{"total":107374182400,"available":107346391040,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27136,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104830464,"statfs":{"total":107374182400,"available":107346395136,"internally_reserved":0,"allocated":118784,"data_stored":33224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:47:18.661 INFO:tasks.ceph.ceph_manager.c2:clean! 2024-04-16T17:47:18.661 INFO:tasks.ceph:Waiting until ceph cluster c2 is healthy... 2024-04-16T17:47:18.661 INFO:tasks.ceph.ceph_manager.c2:wait_until_healthy 2024-04-16T17:47:18.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 health --format=json 2024-04-16T17:47:18.973 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T17:47:18.973 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T17:47:18.990 INFO:tasks.ceph.ceph_manager.c2:wait_until_healthy done 2024-04-16T17:47:18.990 INFO:teuthology.run_tasks:Running task rgw... 2024-04-16T17:47:19.005 DEBUG:tasks.rgw:config is {'c1.client.0': {'port': 8000}, 'c1.client.1': {'port': 8001}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}} 2024-04-16T17:47:19.006 DEBUG:tasks.rgw:client list is dict_keys(['c1.client.0', 'c1.client.1', 'c2.client.0', 'c2.client.1']) 2024-04-16T17:47:19.006 INFO:tasks.rgw:Creating data pools 2024-04-16T17:47:19.006 DEBUG:tasks.rgw:Obtaining remote for client c1.client.0 2024-04-16T17:47:19.006 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2024-04-16T17:47:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'default.rgw.buckets.data' created 2024-04-16T17:47:20.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2024-04-16T17:47:21.207 INFO:teuthology.orchestra.run.smithi071.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-16T17:47:21.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2024-04-16T17:47:22.215 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'default.rgw.buckets.index' created 2024-04-16T17:47:22.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2024-04-16T17:47:23.223 INFO:teuthology.orchestra.run.smithi071.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-16T17:47:23.260 DEBUG:tasks.rgw:Obtaining remote for client c1.client.1 2024-04-16T17:47:23.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2024-04-16T17:47:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'default.rgw.buckets.data' already exists 2024-04-16T17:47:23.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2024-04-16T17:47:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-16T17:47:24.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2024-04-16T17:47:24.565 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'default.rgw.buckets.index' already exists 2024-04-16T17:47:24.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2024-04-16T17:47:25.236 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-16T17:47:25.259 DEBUG:tasks.rgw:Obtaining remote for client c2.client.0 2024-04-16T17:47:25.260 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2024-04-16T17:47:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'default.rgw.buckets.data' created 2024-04-16T17:47:26.577 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2024-04-16T17:47:27.547 INFO:teuthology.orchestra.run.smithi122.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-16T17:47:27.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2024-04-16T17:47:28.558 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'default.rgw.buckets.index' created 2024-04-16T17:47:28.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2024-04-16T17:47:29.564 INFO:teuthology.orchestra.run.smithi122.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-16T17:47:29.603 DEBUG:tasks.rgw:Obtaining remote for client c2.client.1 2024-04-16T17:47:29.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2024-04-16T17:47:29.919 INFO:teuthology.orchestra.run.smithi188.stderr:pool 'default.rgw.buckets.data' already exists 2024-04-16T17:47:29.937 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2024-04-16T17:47:30.570 INFO:teuthology.orchestra.run.smithi188.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-16T17:47:30.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2024-04-16T17:47:30.912 INFO:teuthology.orchestra.run.smithi188.stderr:pool 'default.rgw.buckets.index' already exists 2024-04-16T17:47:30.931 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2024-04-16T17:47:31.571 INFO:teuthology.orchestra.run.smithi188.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-16T17:47:31.597 DEBUG:tasks.rgw:Pools created 2024-04-16T17:47:31.597 INFO:tasks.rgw:Configuring compression type = random 2024-04-16T17:47:31.597 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'list'] 2024-04-16T17:47:31.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'list'] 2024-04-16T17:47:31.598 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user list 2024-04-16T17:47:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:33.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:33.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 20 realm 2024-04-16T17:47:33.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-16T17:47:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.356+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.360+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.360+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.360+0000 7f8ff610e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:33.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.360+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.360+0000 7f8ff610e980 20 rados_obj.operate() r=0 bl.length=993 2024-04-16T17:47:33.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.360+0000 7f8ff610e980 20 searching for the correct realm 2024-04-16T17:47:33.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 RGWRados::pool_iterate: got zone_info.7ee8b98f-146e-4b53-83bb-d235c1177b69 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 RGWRados::pool_iterate: got default.zone. 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 RGWRados::pool_iterate: got zonegroup_info.81574f29-b3e2-4e4d-beeb-7e088a01321f 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:33.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.380+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 20 rados_obj.operate() r=0 bl.length=436 2024-04-16T17:47:33.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 20 zone default found 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 4 Realm: () 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 4 ZoneGroup: default (81574f29-b3e2-4e4d-beeb-7e088a01321f) 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 4 Zone: default (7ee8b98f-146e-4b53-83bb-d235c1177b69) 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 20 zonegroup default 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:33.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:33.384+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:35.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:35.312+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:35.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:35.312+0000 7f8ff610e980 20 rados->read ofs=0 len=0 2024-04-16T17:47:35.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:35.316+0000 7f8ff610e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:35.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:35.316+0000 7f8ff610e980 20 started sync module instance, tier type = 2024-04-16T17:47:35.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:35.316+0000 7f8ff610e980 20 started zone id=7ee8b98f-146e-4b53-83bb-d235c1177b69 (name=default) with tier type = 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=0 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=1 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=2 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=3 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=4 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=5 2024-04-16T17:47:37.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.352+0000 7f8ff610e980 20 add_watcher() i=6 2024-04-16T17:47:37.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 20 add_watcher() i=7 2024-04-16T17:47:37.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 2 all 8 watchers are set, enabling cache 2024-04-16T17:47:37.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-04-16T17:47:37.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-04-16T17:47:37.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:37.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-04-16T17:47:37.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-04-16T17:47:37.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.356+0000 7f8ff610e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:37.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-04-16T17:47:37.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-04-16T17:47:37.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:37.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-04-16T17:47:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-04-16T17:47:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-04-16T17:47:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.360+0000 7f8ff610e980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-04-16T17:47:37.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.364+0000 7f8ff610e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.372+0000 7f8ea5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.372+0000 7f8ff610e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:37.372+0000 7f8ff610e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:40.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.304+0000 7f8ff610e980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-04-16T17:47:40.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.304+0000 7f8ff610e980 5 note: GC not initialized 2024-04-16T17:47:40.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.400+0000 7f8ea4fe1640 20 reqs_thread_entry: start 2024-04-16T17:47:40.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.400+0000 7f8ff610e980 20 init_complete bucket index max shards: 11 2024-04-16T17:47:40.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.400+0000 7f8ff610e980 20 Filter name: none 2024-04-16T17:47:40.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.400+0000 7f8e0e7fc640 20 reqs_thread_entry: start 2024-04-16T17:47:40.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.448+0000 7f8ea5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.452+0000 7f8ff610e980 20 remove_watcher() i=0 2024-04-16T17:47:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.452+0000 7f8ff610e980 2 removed watcher, disabling cache 2024-04-16T17:47:40.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.456+0000 7f8ff610e980 20 remove_watcher() i=1 2024-04-16T17:47:40.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.456+0000 7f8ff610e980 20 remove_watcher() i=2 2024-04-16T17:47:40.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.464+0000 7f8ff610e980 20 remove_watcher() i=3 2024-04-16T17:47:40.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.468+0000 7f8ff610e980 20 remove_watcher() i=4 2024-04-16T17:47:40.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.468+0000 7f8ff610e980 20 remove_watcher() i=5 2024-04-16T17:47:40.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.472+0000 7f8ff610e980 20 remove_watcher() i=6 2024-04-16T17:47:40.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.476+0000 7f8ff610e980 20 remove_watcher() i=7 2024-04-16T17:47:40.494 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T17:47:40.494 DEBUG:tasks.util.rgw: json result: [] 2024-04-16T17:47:40.494 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:40.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:40.495 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-04-16T17:47:40.536 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:40.537 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.564+0000 7fd8f747c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.564+0000 7fd8f747c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:40.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:40.568+0000 7fd742ffd640 20 reqs_thread_entry: start 2024-04-16T17:47:40.590 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"7ee8b98f-146e-4b53-83bb-d235c1177b69","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-16T17:47:40.591 DEBUG:tasks.util.rgw: json result: {'id': '7ee8b98f-146e-4b53-83bb-d235c1177b69', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-16T17:47:40.591 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['user', 'list'] 2024-04-16T17:47:40.591 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'user', 'list'] 2024-04-16T17:47:40.591 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 user list 2024-04-16T17:47:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.697+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 realm 2024-04-16T17:47:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-16T17:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.701+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.705+0000 7fc271734980 20 rados_obj.operate() r=0 bl.length=1009 2024-04-16T17:47:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.705+0000 7fc271734980 20 searching for the correct realm 2024-04-16T17:47:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 RGWRados::pool_iterate: got zone_info.7ee8b98f-146e-4b53-83bb-d235c1177b69 2024-04-16T17:47:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-16T17:47:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 RGWRados::pool_iterate: got default.zone. 2024-04-16T17:47:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 RGWRados::pool_iterate: got zonegroup_info.81574f29-b3e2-4e4d-beeb-7e088a01321f 2024-04-16T17:47:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-16T17:47:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-16T17:47:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados_obj.operate() r=0 bl.length=436 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 zone default found 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 4 Realm: () 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 4 ZoneGroup: default (81574f29-b3e2-4e4d-beeb-7e088a01321f) 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 4 Zone: default (7ee8b98f-146e-4b53-83bb-d235c1177b69) 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 zonegroup default 2024-04-16T17:47:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.721+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 rados->read ofs=0 len=0 2024-04-16T17:47:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 started sync module instance, tier type = 2024-04-16T17:47:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.725+0000 7fc271734980 20 started zone id=7ee8b98f-146e-4b53-83bb-d235c1177b69 (name=default) with tier type = 2024-04-16T17:47:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=0 2024-04-16T17:47:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=1 2024-04-16T17:47:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=2 2024-04-16T17:47:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=3 2024-04-16T17:47:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=4 2024-04-16T17:47:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=5 2024-04-16T17:47:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.741+0000 7fc271734980 20 add_watcher() i=6 2024-04-16T17:47:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.745+0000 7fc271734980 20 add_watcher() i=7 2024-04-16T17:47:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.745+0000 7fc271734980 2 all 8 watchers are set, enabling cache 2024-04-16T17:47:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.749+0000 7fc125fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:40.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.749+0000 7fc271734980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:40.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.749+0000 7fc271734980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:40.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.749+0000 7fc271734980 5 note: GC not initialized 2024-04-16T17:47:40.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.757+0000 7fc124fe1640 20 reqs_thread_entry: start 2024-04-16T17:47:40.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.757+0000 7fc271734980 20 init_complete bucket index max shards: 11 2024-04-16T17:47:40.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.757+0000 7fc271734980 20 Filter name: none 2024-04-16T17:47:40.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.757+0000 7fc09e7fc640 20 reqs_thread_entry: start 2024-04-16T17:47:40.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.777+0000 7fc125fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.781+0000 7fc271734980 20 remove_watcher() i=0 2024-04-16T17:47:40.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.781+0000 7fc271734980 2 removed watcher, disabling cache 2024-04-16T17:47:40.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.781+0000 7fc271734980 20 remove_watcher() i=1 2024-04-16T17:47:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.781+0000 7fc271734980 20 remove_watcher() i=2 2024-04-16T17:47:40.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.785+0000 7fc271734980 20 remove_watcher() i=3 2024-04-16T17:47:40.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.789+0000 7fc271734980 20 remove_watcher() i=4 2024-04-16T17:47:40.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.789+0000 7fc271734980 20 remove_watcher() i=5 2024-04-16T17:47:40.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.793+0000 7fc271734980 20 remove_watcher() i=6 2024-04-16T17:47:40.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.793+0000 7fc271734980 20 remove_watcher() i=7 2024-04-16T17:47:40.810 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-16T17:47:40.810 DEBUG:tasks.util.rgw: json result: [] 2024-04-16T17:47:40.810 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:40.810 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:40.810 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-04-16T17:47:40.847 INFO:teuthology.orchestra.run.smithi083.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:40.848 INFO:teuthology.orchestra.run.smithi083.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.877+0000 7fcc5bcc7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:40.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.877+0000 7fcc5bcc7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:40.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T17:47:40.881+0000 7fcb177fe640 20 reqs_thread_entry: start 2024-04-16T17:47:40.902 INFO:teuthology.orchestra.run.smithi083.stdout:{"id":"7ee8b98f-146e-4b53-83bb-d235c1177b69","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-16T17:47:40.902 DEBUG:tasks.util.rgw: json result: {'id': '7ee8b98f-146e-4b53-83bb-d235c1177b69', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-16T17:47:40.903 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['user', 'list'] 2024-04-16T17:47:40.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'user', 'list'] 2024-04-16T17:47:40.903 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 user list 2024-04-16T17:47:41.036 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:41.036 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:42.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:42.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 realm 2024-04-16T17:47:42.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:42.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-16T17:47:42.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:42.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.694+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.698+0000 7ff90dccb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:42.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.698+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.698+0000 7ff90dccb980 20 rados_obj.operate() r=0 bl.length=993 2024-04-16T17:47:42.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.698+0000 7ff90dccb980 20 searching for the correct realm 2024-04-16T17:47:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 RGWRados::pool_iterate: got zone_info.92fafaa3-32f0-4a3d-be05-105158ba63df 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 RGWRados::pool_iterate: got default.zone. 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 RGWRados::pool_iterate: got zonegroup_info.13c892c8-2fce-4e74-bdf3-e188968141f0 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.718+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 20 rados_obj.operate() r=0 bl.length=436 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 20 zone default found 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 4 Realm: () 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 4 ZoneGroup: default (13c892c8-2fce-4e74-bdf3-e188968141f0) 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 4 Zone: default (92fafaa3-32f0-4a3d-be05-105158ba63df) 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 20 zonegroup default 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:42.722+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:44.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:44.658+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:44.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:44.658+0000 7ff90dccb980 20 rados->read ofs=0 len=0 2024-04-16T17:47:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:44.658+0000 7ff90dccb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:44.658+0000 7ff90dccb980 20 started sync module instance, tier type = 2024-04-16T17:47:44.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:44.658+0000 7ff90dccb980 20 started zone id=92fafaa3-32f0-4a3d-be05-105158ba63df (name=default) with tier type = 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=0 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=1 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=2 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=3 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=4 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=5 2024-04-16T17:47:46.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=6 2024-04-16T17:47:46.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 20 add_watcher() i=7 2024-04-16T17:47:46.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.694+0000 7ff90dccb980 2 all 8 watchers are set, enabling cache 2024-04-16T17:47:46.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-04-16T17:47:46.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-04-16T17:47:46.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:46.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-04-16T17:47:46.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-04-16T17:47:46.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:46.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-04-16T17:47:46.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-04-16T17:47:46.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:46.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-04-16T17:47:46.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-04-16T17:47:46.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-04-16T17:47:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.698+0000 7ff90dccb980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-04-16T17:47:46.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.702+0000 7ff90dccb980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-16T17:47:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.710+0000 7ff7bdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.710+0000 7ff90dccb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:46.710+0000 7ff90dccb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.658+0000 7ff90dccb980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-04-16T17:47:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.658+0000 7ff90dccb980 5 note: GC not initialized 2024-04-16T17:47:49.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.670+0000 7ff7bcfe1640 20 reqs_thread_entry: start 2024-04-16T17:47:49.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.674+0000 7ff90dccb980 20 init_complete bucket index max shards: 11 2024-04-16T17:47:49.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.674+0000 7ff90dccb980 20 Filter name: none 2024-04-16T17:47:49.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.674+0000 7ff7267fc640 20 reqs_thread_entry: start 2024-04-16T17:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.714+0000 7ff7bdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:49.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.718+0000 7ff90dccb980 20 remove_watcher() i=0 2024-04-16T17:47:49.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.718+0000 7ff90dccb980 2 removed watcher, disabling cache 2024-04-16T17:47:49.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.722+0000 7ff90dccb980 20 remove_watcher() i=1 2024-04-16T17:47:49.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.726+0000 7ff90dccb980 20 remove_watcher() i=2 2024-04-16T17:47:49.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.730+0000 7ff90dccb980 20 remove_watcher() i=3 2024-04-16T17:47:49.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.734+0000 7ff90dccb980 20 remove_watcher() i=4 2024-04-16T17:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.738+0000 7ff90dccb980 20 remove_watcher() i=5 2024-04-16T17:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.738+0000 7ff90dccb980 20 remove_watcher() i=6 2024-04-16T17:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.742+0000 7ff90dccb980 20 remove_watcher() i=7 2024-04-16T17:47:49.764 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-16T17:47:49.765 DEBUG:tasks.util.rgw: json result: [] 2024-04-16T17:47:49.765 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:49.765 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:49.765 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-04-16T17:47:49.807 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:49.807 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:49.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.834+0000 7f7378504980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:49.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.834+0000 7f7378504980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:49.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:47:49.838+0000 7f72397e2640 20 reqs_thread_entry: start 2024-04-16T17:47:49.861 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"92fafaa3-32f0-4a3d-be05-105158ba63df","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-16T17:47:49.861 DEBUG:tasks.util.rgw: json result: {'id': '92fafaa3-32f0-4a3d-be05-105158ba63df', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-16T17:47:49.861 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['user', 'list'] 2024-04-16T17:47:49.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'user', 'list'] 2024-04-16T17:47:49.861 DEBUG:teuthology.orchestra.run.smithi188:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 user list 2024-04-16T17:47:49.971 INFO:teuthology.orchestra.run.smithi188.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:49.971 INFO:teuthology.orchestra.run.smithi188.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:50.018 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.020 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.020 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 realm 2024-04-16T17:47:50.020 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-16T17:47:50.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.019+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.022 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.023+0000 7ff76a55f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:50.022 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.023+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.023+0000 7ff76a55f980 20 rados_obj.operate() r=0 bl.length=1009 2024-04-16T17:47:50.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.023+0000 7ff76a55f980 20 searching for the correct realm 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 RGWRados::pool_iterate: got zone_info.92fafaa3-32f0-4a3d-be05-105158ba63df 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 RGWRados::pool_iterate: got default.zone. 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 RGWRados::pool_iterate: got zonegroup_info.13c892c8-2fce-4e74-bdf3-e188968141f0 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-16T17:47:50.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.043 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.043 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados_obj.operate() r=0 bl.length=436 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 zone default found 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 4 Realm: () 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 4 ZoneGroup: default (13c892c8-2fce-4e74-bdf3-e188968141f0) 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 4 Zone: default (92fafaa3-32f0-4a3d-be05-105158ba63df) 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 zonegroup default 2024-04-16T17:47:50.044 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados->read ofs=0 len=0 2024-04-16T17:47:50.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T17:47:50.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 started sync module instance, tier type = 2024-04-16T17:47:50.046 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.043+0000 7ff76a55f980 20 started zone id=92fafaa3-32f0-4a3d-be05-105158ba63df (name=default) with tier type = 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=0 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=1 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=2 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=3 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=4 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=5 2024-04-16T17:47:50.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=6 2024-04-16T17:47:50.072 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 20 add_watcher() i=7 2024-04-16T17:47:50.072 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.071+0000 7ff76a55f980 2 all 8 watchers are set, enabling cache 2024-04-16T17:47:50.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.075+0000 7ff61e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:50.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.075+0000 7ff76a55f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:50.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.075+0000 7ff76a55f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:50.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.075+0000 7ff76a55f980 5 note: GC not initialized 2024-04-16T17:47:50.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.091+0000 7ff61d7e2640 20 reqs_thread_entry: start 2024-04-16T17:47:50.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.091+0000 7ff76a55f980 20 init_complete bucket index max shards: 11 2024-04-16T17:47:50.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.091+0000 7ff76a55f980 20 Filter name: none 2024-04-16T17:47:50.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.091+0000 7ff58effd640 20 reqs_thread_entry: start 2024-04-16T17:47:50.113 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.111+0000 7ff61e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T17:47:50.116 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.115+0000 7ff76a55f980 20 remove_watcher() i=0 2024-04-16T17:47:50.116 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.115+0000 7ff76a55f980 2 removed watcher, disabling cache 2024-04-16T17:47:50.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.119+0000 7ff76a55f980 20 remove_watcher() i=1 2024-04-16T17:47:50.121 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.119+0000 7ff76a55f980 20 remove_watcher() i=2 2024-04-16T17:47:50.123 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.123+0000 7ff76a55f980 20 remove_watcher() i=3 2024-04-16T17:47:50.124 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.123+0000 7ff76a55f980 20 remove_watcher() i=4 2024-04-16T17:47:50.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.127+0000 7ff76a55f980 20 remove_watcher() i=5 2024-04-16T17:47:50.130 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.131+0000 7ff76a55f980 20 remove_watcher() i=6 2024-04-16T17:47:50.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.131+0000 7ff76a55f980 20 remove_watcher() i=7 2024-04-16T17:47:50.148 INFO:teuthology.orchestra.run.smithi188.stdout:[] 2024-04-16T17:47:50.149 DEBUG:tasks.util.rgw: json result: [] 2024-04-16T17:47:50.149 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:50.149 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-16T17:47:50.149 DEBUG:teuthology.orchestra.run.smithi188:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-04-16T17:47:50.189 INFO:teuthology.orchestra.run.smithi188.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:50.189 INFO:teuthology.orchestra.run.smithi188.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:50.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.223+0000 7f2d90b6e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T17:47:50.222 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.223+0000 7f2d90b6e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T17:47:50.226 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-16T17:47:50.227+0000 7f2c50fe1640 20 reqs_thread_entry: start 2024-04-16T17:47:50.245 INFO:teuthology.orchestra.run.smithi188.stdout:{"id":"92fafaa3-32f0-4a3d-be05-105158ba63df","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-16T17:47:50.245 DEBUG:tasks.util.rgw: json result: {'id': '92fafaa3-32f0-4a3d-be05-105158ba63df', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-16T17:47:50.245 INFO:tasks.rgw:Starting rgw... 2024-04-16T17:47:50.245 INFO:tasks.rgw:rgw c1.client.0 config is {'port': 8000} 2024-04-16T17:47:50.245 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-16T17:47:50.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo echo -n http://smithi071.front.sepia.ceph.com:8000 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.260 INFO:teuthology.orchestra.run.smithi071.stdout:http://smithi071.front.sepia.ceph.com:8000 2024-04-16T17:47:50.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.314 INFO:tasks.rgw.c1.client.0:Restarting daemon 2024-04-16T17:47:50.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2024-04-16T17:47:50.356 INFO:tasks.rgw.c1.client.0:Started 2024-04-16T17:47:50.356 INFO:tasks.rgw:rgw c1.client.1 config is {'port': 8001} 2024-04-16T17:47:50.356 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-16T17:47:50.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo echo -n http://smithi083.front.sepia.ceph.com:8001 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.372 INFO:teuthology.orchestra.run.smithi083.stdout:http://smithi083.front.sepia.ceph.com:8001 2024-04-16T17:47:50.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.422 INFO:tasks.rgw.c1.client.1:Restarting daemon 2024-04-16T17:47:50.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2024-04-16T17:47:50.466 INFO:tasks.rgw.c1.client.1:Started 2024-04-16T17:47:50.466 INFO:tasks.rgw:rgw c2.client.0 config is {'port': 8000} 2024-04-16T17:47:50.466 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-16T17:47:50.466 DEBUG:teuthology.orchestra.run.smithi122:> sudo echo -n http://smithi122.front.sepia.ceph.com:8000 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.483 INFO:teuthology.orchestra.run.smithi122.stdout:http://smithi122.front.sepia.ceph.com:8000 2024-04-16T17:47:50.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.538 INFO:tasks.rgw.c2.client.0:Restarting daemon 2024-04-16T17:47:50.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T17:47:50.547 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:47:50.543+0000 7f9a742c7a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T17:47:50.584 INFO:tasks.rgw.c2.client.0:Started 2024-04-16T17:47:50.584 INFO:tasks.rgw:rgw c2.client.1 config is {'port': 8001} 2024-04-16T17:47:50.584 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-16T17:47:50.584 DEBUG:teuthology.orchestra.run.smithi188:> sudo echo -n http://smithi188.front.sepia.ceph.com:8001 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.600 INFO:teuthology.orchestra.run.smithi188.stdout:http://smithi188.front.sepia.ceph.com:8001 2024-04-16T17:47:50.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-16T17:47:50.658 INFO:tasks.rgw.c2.client.1:Restarting daemon 2024-04-16T17:47:50.658 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2024-04-16T17:47:50.703 INFO:tasks.rgw.c2.client.1:Started 2024-04-16T17:47:50.703 INFO:tasks.rgw:Polling c1.client.0 until it starts accepting connections on http://smithi071.front.sepia.ceph.com:8000/ 2024-04-16T17:47:50.703 DEBUG:teuthology.orchestra.run.smithi071:> curl http://smithi071.front.sepia.ceph.com:8000/ 2024-04-16T17:47:50.708 INFO:tasks.rgw.c1.client.1.smithi083.stdout:2024-04-16T17:47:50.704+0000 7fa70f8d9a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T17:47:50.727 INFO:teuthology.orchestra.run.smithi071.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:47:50.727 INFO:teuthology.orchestra.run.smithi071.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:47:50.729 INFO:teuthology.orchestra.run.smithi071.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 86774 0 --:--:-- --:--:-- --:--:-- 93500 2024-04-16T17:47:50.730 INFO:teuthology.orchestra.run.smithi071.stdout:anonymous 2024-04-16T17:47:50.730 INFO:tasks.rgw:Polling c1.client.1 until it starts accepting connections on http://smithi083.front.sepia.ceph.com:8001/ 2024-04-16T17:47:50.730 DEBUG:teuthology.orchestra.run.smithi083:> curl http://smithi083.front.sepia.ceph.com:8001/ 2024-04-16T17:47:50.794 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T17:47:50.794+0000 7f6ab0a2ba40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T17:47:50.955 INFO:tasks.rgw.c2.client.1.smithi188.stdout:2024-04-16T17:47:50.955+0000 7f45e12caa40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T17:47:51.026 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T17:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T17:47:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:curl: (7) Failed to connect to smithi083.front.sepia.ceph.com port 8001 after 0 ms: Connection refused 2024-04-16T17:47:52.027 DEBUG:teuthology.orchestra.run.smithi083:> curl http://smithi083.front.sepia.ceph.com:8001/ 2024-04-16T17:47:52.038 INFO:teuthology.orchestra.run.smithi083.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:47:52.039 INFO:teuthology.orchestra.run.smithi083.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:47:52.042 INFO:teuthology.orchestra.run.smithi083.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 51092 0 --:--:-- --:--:-- --:--:-- 62333 2024-04-16T17:47:52.043 INFO:teuthology.orchestra.run.smithi083.stdout:anonymous 2024-04-16T17:47:52.043 INFO:tasks.rgw:Polling c2.client.0 until it starts accepting connections on http://smithi122.front.sepia.ceph.com:8000/ 2024-04-16T17:47:52.043 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000/ 2024-04-16T17:47:52.132 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:47:52.132 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:47:52.135 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 52869 0 --:--:-- --:--:-- --:--:-- 62333 2024-04-16T17:47:52.136 INFO:teuthology.orchestra.run.smithi122.stdout:anonymous 2024-04-16T17:47:52.136 INFO:tasks.rgw:Polling c2.client.1 until it starts accepting connections on http://smithi188.front.sepia.ceph.com:8001/ 2024-04-16T17:47:52.137 DEBUG:teuthology.orchestra.run.smithi188:> curl http://smithi188.front.sepia.ceph.com:8001/ 2024-04-16T17:47:52.449 INFO:teuthology.orchestra.run.smithi188.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:47:52.449 INFO:teuthology.orchestra.run.smithi188.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:47:52.452 INFO:teuthology.orchestra.run.smithi188.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 51009 0 --:--:-- --:--:-- --:--:-- 62333 2024-04-16T17:47:52.453 INFO:teuthology.orchestra.run.smithi188.stdout:anonymous 2024-04-16T17:47:52.453 INFO:teuthology.run_tasks:Running task rgw-multisite... 2024-04-16T17:47:52.685 INFO:tasks.rgw_multisite:creating realm.. 2024-04-16T17:47:52.685 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:52.685 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:52.685 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 realm create --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:47:52.724 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:52.724 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:52.787 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","name":"test-realm","current_period":"2fe52312-e804-49a9-9525-abfdb920e903","epoch":1} 2024-04-16T17:47:52.788 INFO:tasks.rgw_multisite:creating master zonegroup.. 2024-04-16T17:47:52.788 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:52.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:52.788 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup set --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:47:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:52.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:52.903+0000 7fadaf0ef980 -1 old format 2024-04-16T17:47:52.925 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":[]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-04-16T17:47:52.925 INFO:tasks.rgw_multisite:creating master zone.. 2024-04-16T17:47:52.926 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--endpoints', 'http://smithi071.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:52.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--endpoints', 'http://smithi071.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:52.926 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --master --access-key GVUAE6PEZFSPBINC --secret KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK --endpoints http://smithi071.front.sepia.ceph.com:8000 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:47:53.009 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:53.009 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:NOTICE: set zone's realm_id=750bb778-ce9e-43d8-aa1d-59a6a3ecda80 2024-04-16T17:47:53.087 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","roles_pool":"test-zone1.rgw.meta:roles","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","notif_pool":"test-zone1.rgw.log:notif","topics_pool":"test-zone1.rgw.meta:topics","account_pool":"test-zone1.rgw.meta:accounts","group_pool":"test-zone1.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T17:47:53.087 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph osd pool create test-zone1.rgw.buckets.data 64 64 --cluster c1 2024-04-16T17:47:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'test-zone1.rgw.buckets.data' created 2024-04-16T17:47:54.151 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph osd pool application enable test-zone1.rgw.buckets.data rgw --cluster c1 2024-04-16T17:47:55.129 INFO:teuthology.orchestra.run.smithi071.stderr:enabled application 'rgw' on pool 'test-zone1.rgw.buckets.data' 2024-04-16T17:47:55.178 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:55.178 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:55.178 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:47:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:55.220 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:55.272 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","roles_pool":"test-zone1.rgw.meta:roles","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","notif_pool":"test-zone1.rgw.log:notif","topics_pool":"test-zone1.rgw.meta:topics","account_pool":"test-zone1.rgw.meta:accounts","group_pool":"test-zone1.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T17:47:55.272 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:55.272 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:47:55.273 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:47:55.347 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:47:55.348 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:47:55.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:47:55.391+0000 7f613fb20980 0 period (2fe52312-e804-49a9-9525-abfdb920e903 does not have zone 054102a3-8d94-4596-8b3b-6341eab312e1 configured 2024-04-16T17:48:02.372 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":1,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T17:48:02.373 DEBUG:tasks.rgw.c1.client.0:waiting for process to exit 2024-04-16T17:48:02.373 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T17:48:02.416 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:02.418+0000 7f9a70a7f640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground (PID: 27987) UID: 0 2024-04-16T17:48:02.417 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:02.418+0000 7f9a742c7a40 -1 shutting down 2024-04-16T17:48:05.940 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 is failed for ~0s 2024-04-16T17:48:08.474 INFO:tasks.rgw.c1.client.0:Stopped 2024-04-16T17:48:08.474 INFO:tasks.rgw.c1.client.0:Restarting daemon 2024-04-16T17:48:08.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2024-04-16T17:48:08.478 INFO:tasks.rgw.c1.client.0:Started 2024-04-16T17:48:08.478 DEBUG:teuthology.orchestra.run.smithi071:> curl http://smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:08.489 INFO:teuthology.orchestra.run.smithi071.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:48:08.489 INFO:teuthology.orchestra.run.smithi071.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:48:08.490 INFO:teuthology.orchestra.run.smithi071.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T17:48:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:curl: (7) Failed to connect to smithi071.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T17:48:08.490 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T17:48:08.713 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:08.714+0000 7f59dd900a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T17:48:09.492 DEBUG:teuthology.orchestra.run.smithi071:> curl http://smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:09.503 INFO:teuthology.orchestra.run.smithi071.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:48:09.503 INFO:teuthology.orchestra.run.smithi071.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:48:09.506 INFO:teuthology.orchestra.run.smithi071.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 59065 0 --:--:-- --:--:-- --:--:-- 62333 2024-04-16T17:48:09.506 INFO:teuthology.orchestra.run.smithi071.stdout:anonymous 2024-04-16T17:48:09.507 INFO:tasks.rgw_multisite:creating admin user.. 2024-04-16T17:48:09.507 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:09.507 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:09.507 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid realm-admin --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Realm Admin' --system --access-key GVUAE6PEZFSPBINC --secret KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:09.597 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:09.597 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:{"user_id":"realm-admin","display_name":"Realm Admin","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"realm-admin","access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK","active":true,"create_date":"2024-04-16T17:48:09.665255Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-16T17:48:09.665244Z","tags":[],"group_ids":[]} 2024-04-16T17:48:09.697 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2024-04-16T17:48:09.697 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi071.front.sepia.ceph.com:8000', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:09.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi071.front.sepia.ceph.com:8000', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:09.697 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi071.front.sepia.ceph.com:8000 --access-key GVUAE6PEZFSPBINC --secret KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:09.748 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:09.748 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:09.830 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","name":"test-realm","current_period":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":2} 2024-04-16T17:48:09.831 INFO:tasks.rgw_multisite:creating zone.. 2024-04-16T17:48:09.831 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--endpoints', 'http://smithi122.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:09.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--endpoints', 'http://smithi122.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:09.831 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --access-key GVUAE6PEZFSPBINC --secret KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK --endpoints http://smithi122.front.sepia.ceph.com:8000 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:NOTICE: set zone's realm_id=750bb778-ce9e-43d8-aa1d-59a6a3ecda80 2024-04-16T17:48:10.003 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","notif_pool":"test-zone2.rgw.log:notif","topics_pool":"test-zone2.rgw.meta:topics","account_pool":"test-zone2.rgw.meta:accounts","group_pool":"test-zone2.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T17:48:10.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph osd pool create test-zone2.rgw.buckets.data 64 64 --cluster c2 2024-04-16T17:48:10.705 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'test-zone2.rgw.buckets.data' created 2024-04-16T17:48:10.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph osd pool application enable test-zone2.rgw.buckets.data rgw --cluster c2 2024-04-16T17:48:11.706 INFO:teuthology.orchestra.run.smithi122.stderr:enabled application 'rgw' on pool 'test-zone2.rgw.buckets.data' 2024-04-16T17:48:11.741 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:11.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:11.742 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:11.783 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:11.783 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:11.832 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","notif_pool":"test-zone2.rgw.log:notif","topics_pool":"test-zone2.rgw.meta:topics","account_pool":"test-zone2.rgw.meta:accounts","group_pool":"test-zone2.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T17:48:11.832 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:11.832 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:11.832 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:11.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 has been restored 2024-04-16T17:48:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:11.954+0000 7fac6fece980 0 period (8c713f9b-78f8-4e05-979f-375a77da9531 does not have zone bd830684-b604-4841-ab52-8c82ee812281 configured 2024-04-16T17:48:18.873 INFO:teuthology.orchestra.run.smithi122.stderr:Sending period to new master zone 054102a3-8d94-4596-8b3b-6341eab312e1 2024-04-16T17:48:18.969 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":2,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T17:48:18.970 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2024-04-16T17:48:18.970 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T17:48:19.010 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T17:48:19.009+0000 7f6aad1e3640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground (PID: 28337) UID: 0 2024-04-16T17:48:19.011 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T17:48:19.009+0000 7f6ab0a2ba40 -1 shutting down 2024-04-16T17:48:20.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T17:48:25.071 INFO:tasks.rgw.c2.client.0:Stopped 2024-04-16T17:48:25.071 INFO:tasks.rgw.c2.client.0:Restarting daemon 2024-04-16T17:48:25.071 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T17:48:25.074 INFO:tasks.rgw.c2.client.0:Started 2024-04-16T17:48:25.074 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:25.085 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:48:25.086 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:48:25.086 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T17:48:25.086 INFO:teuthology.orchestra.run.smithi122.stderr:curl: (7) Failed to connect to smithi122.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T17:48:25.087 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T17:48:25.374 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T17:48:25.373+0000 7f4aa8055a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T17:48:26.087 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:26.099 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T17:48:26.099 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T17:48:26.102 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 66809 0 --:--:-- --:--:-- --:--:-- 93500 2024-04-16T17:48:26.103 INFO:teuthology.orchestra.run.smithi122.stdout:anonymous 2024-04-16T17:48:26.103 INFO:tasks.rgw_multisite:rgw multisite configuration completed 2024-04-16T17:48:26.103 INFO:teuthology.run_tasks:Running task rgw-multisite-tests... 2024-04-16T17:48:26.148 INFO:tasks.rgw_multisite_tests:importing tests from /home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi 2024-04-16T17:48:26.189 INFO:tasks.rgw_multisite_tests:creating test user.. 2024-04-16T17:48:26.189 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['account', 'create', '--account-id', 'RGW11111111111111111', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:26.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'account', 'create', '--account-id', 'RGW11111111111111111', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:26.190 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 account create --account-id RGW11111111111111111 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:26.355 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"RGW11111111111111111","tenant":"","name":"","email":"","quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"max_users":1000,"max_roles":1000,"max_groups":1000,"max_buckets":1000,"max_access_keys":4} 2024-04-16T17:48:26.356 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'rgw-multisite-test-user', '--account-id', 'RGW11111111111111111', '--account-root', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'TestUser', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:26.356 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'rgw-multisite-test-user', '--account-id', 'RGW11111111111111111', '--account-root', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'TestUser', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T17:48:26.356 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid rgw-multisite-test-user --account-id RGW11111111111111111 --account-root --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name TestUser --gen-access-key --gen-secret --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T17:48:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:26.494 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2024-04-16T17:48:26.536 INFO:teuthology.orchestra.run.smithi071.stdout:{"user_id":"rgw-multisite-test-user","display_name":"TestUser","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"rgw-multisite-test-user","access_key":"076QSHZWMHBH668NR2NY","secret_key":"tXFk6bwnKAELgSvaIQqifLSm2Qn4JV0EIkoKCBFe","active":true,"create_date":"2024-04-16T17:48:26.521863Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"root","mfa_ids":[],"account_id":"RGW11111111111111111","path":"/","create_date":"2024-04-16T17:48:26.521558Z","tags":[],"group_ids":[]} 2024-04-16T17:48:26.564 INFO:rgw_multi.tests:meta checkpoint 2024-04-16T17:48:26.565 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:26.565 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:26.565 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:26.601 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:26.601 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:26.696 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:26.697 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:26.697 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:26.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:26.697 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:26.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:26.809+0000 7f34ac134980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:26.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:26.813+0000 7f34ac134980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:26.846 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"building-full-sync-maps","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":0,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":0}},{"key":1,"val":{"state":0,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":0}},{"key":2,"val":{"state":0,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":0}},{"key":3,"val":{"state":0,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":0}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T17:48:26Z"} 2024-04-16T17:48:26.846 WARNING:rgw_multi.tests:zone test-zone2 has not started sync yet, state=building-full-sync-maps 2024-04-16T17:48:31.847 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:31.848 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:31.848 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:31.890 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:31.890 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:31.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:31.957+0000 7f5626f3c980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:31.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:31.961+0000 7f5626f3c980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:31.993 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:31Z"} 2024-04-16T17:48:31.993 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:31.993 INFO:rgw_multi.tests:meta checkpoint 2024-04-16T17:48:31.993 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:31.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:31.993 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:32.104 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:32.136 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:32.137 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:32.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:32.137 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:32.178 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:32.179 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:32.241+0000 7fd713927980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:32.245+0000 7fd713927980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:32.277 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:32Z"} 2024-04-16T17:48:32.278 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:32.278 INFO:tasks.rgw_multisite_tests:running rgw multisite tests on '/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi' with args=['tests.py', '-a', '!fails_on_rgw'] 2024-04-16T17:48:32.303 DEBUG:boto:Using access key provided by client. 2024-04-16T17:48:32.303 DEBUG:boto:Using secret key provided by client. 2024-04-16T17:48:32.305 DEBUG:botocore.hooks:Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane 2024-04-16T17:48:32.306 DEBUG:botocore.hooks:Changing event name from before-call.apigateway to before-call.api-gateway 2024-04-16T17:48:32.307 DEBUG:botocore.hooks:Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict 2024-04-16T17:48:32.308 DEBUG:botocore.hooks:Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration 2024-04-16T17:48:32.308 DEBUG:botocore.hooks:Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 2024-04-16T17:48:32.309 DEBUG:botocore.hooks:Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search 2024-04-16T17:48:32.310 DEBUG:botocore.hooks:Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section 2024-04-16T17:48:32.312 DEBUG:botocore.hooks:Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask 2024-04-16T17:48:32.312 DEBUG:botocore.hooks:Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section 2024-04-16T17:48:32.312 DEBUG:botocore.hooks:Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search 2024-04-16T17:48:32.313 DEBUG:botocore.hooks:Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section 2024-04-16T17:48:32.317 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/endpoints.json 2024-04-16T17:48:32.338 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/sdk-default-configuration.json 2024-04-16T17:48:32.339 DEBUG:botocore.hooks:Event choose-service-name: calling handler 2024-04-16T17:48:32.353 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/iam/2010-05-08/service-2.json.gz 2024-04-16T17:48:32.370 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/iam/2010-05-08/endpoint-rule-set-1.json.gz 2024-04-16T17:48:32.370 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/partitions.json 2024-04-16T17:48:32.374 DEBUG:botocore.hooks:Event creating-client-class.iam: calling handler 2024-04-16T17:48:32.375 DEBUG:botocore.regions:Using partition endpoint for iam, test-zonegroup: aws-global 2024-04-16T17:48:32.377 DEBUG:botocore.endpoint:Setting iam timeout as (60, 60) 2024-04-16T17:48:32.379 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/_retry.json 2024-04-16T17:48:32.379 DEBUG:botocore.client:Registering retry handlers for service: iam 2024-04-16T17:48:32.380 DEBUG:botocore.hooks:Event choose-service-name: calling handler 2024-04-16T17:48:32.386 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/s3/2006-03-01/service-2.json.gz 2024-04-16T17:48:32.398 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/s3/2006-03-01/endpoint-rule-set-1.json.gz 2024-04-16T17:48:32.403 DEBUG:botocore.hooks:Event creating-client-class.s3: calling handler 2024-04-16T17:48:32.404 DEBUG:botocore.hooks:Event creating-client-class.s3: calling handler ._handler at 0x7f3d40cd44c0> 2024-04-16T17:48:32.419 DEBUG:botocore.hooks:Event creating-client-class.s3: calling handler 2024-04-16T17:48:32.419 DEBUG:botocore.regions:Creating a regex based endpoint for s3, test-zonegroup 2024-04-16T17:48:32.420 DEBUG:botocore.endpoint:Setting s3 timeout as (60, 60) 2024-04-16T17:48:32.422 DEBUG:botocore.client:Registering retry handlers for service: s3 2024-04-16T17:48:32.423 DEBUG:botocore.utils:Registering S3 region redirector handler 2024-04-16T17:48:32.423 DEBUG:botocore.utils:Registering S3Express Identity Resolver 2024-04-16T17:48:32.424 DEBUG:botocore.hooks:Event choose-service-name: calling handler 2024-04-16T17:48:32.425 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/sns/2010-03-31/service-2.json.gz 2024-04-16T17:48:32.428 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/sns/2010-03-31/endpoint-rule-set-1.json.gz 2024-04-16T17:48:32.429 DEBUG:botocore.hooks:Event creating-client-class.sns: calling handler 2024-04-16T17:48:32.429 DEBUG:botocore.regions:Creating a regex based endpoint for sns, test-zonegroup 2024-04-16T17:48:32.431 DEBUG:botocore.endpoint:Setting sns timeout as (60, 60) 2024-04-16T17:48:32.432 DEBUG:botocore.client:Registering retry handlers for service: sns 2024-04-16T17:48:32.432 DEBUG:boto:Using access key provided by client. 2024-04-16T17:48:32.433 DEBUG:boto:Using secret key provided by client. 2024-04-16T17:48:32.433 DEBUG:botocore.hooks:Event choose-service-name: calling handler 2024-04-16T17:48:32.435 DEBUG:botocore.hooks:Event creating-client-class.iam: calling handler 2024-04-16T17:48:32.435 DEBUG:botocore.regions:Using partition endpoint for iam, test-zonegroup: aws-global 2024-04-16T17:48:32.436 DEBUG:botocore.endpoint:Setting iam timeout as (60, 60) 2024-04-16T17:48:32.438 DEBUG:botocore.client:Registering retry handlers for service: iam 2024-04-16T17:48:32.439 DEBUG:botocore.hooks:Event choose-service-name: calling handler 2024-04-16T17:48:32.440 DEBUG:botocore.hooks:Event creating-client-class.s3: calling handler 2024-04-16T17:48:32.440 DEBUG:botocore.hooks:Event creating-client-class.s3: calling handler ._handler at 0x7f3d40cd44c0> 2024-04-16T17:48:32.440 DEBUG:botocore.hooks:Event creating-client-class.s3: calling handler 2024-04-16T17:48:32.440 DEBUG:botocore.regions:Creating a regex based endpoint for s3, test-zonegroup 2024-04-16T17:48:32.441 DEBUG:botocore.endpoint:Setting s3 timeout as (60, 60) 2024-04-16T17:48:32.443 DEBUG:botocore.client:Registering retry handlers for service: s3 2024-04-16T17:48:32.443 DEBUG:botocore.utils:Registering S3 region redirector handler 2024-04-16T17:48:32.444 DEBUG:botocore.utils:Registering S3Express Identity Resolver 2024-04-16T17:48:32.444 DEBUG:botocore.hooks:Event choose-service-name: calling handler 2024-04-16T17:48:32.445 DEBUG:botocore.hooks:Event creating-client-class.sns: calling handler 2024-04-16T17:48:32.445 DEBUG:botocore.regions:Creating a regex based endpoint for sns, test-zonegroup 2024-04-16T17:48:32.446 DEBUG:botocore.endpoint:Setting sns timeout as (60, 60) 2024-04-16T17:48:32.447 DEBUG:botocore.client:Registering retry handlers for service: sns 2024-04-16T17:48:32.448 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-1 2024-04-16T17:48:32.448 DEBUG:boto:path=/lybtug-1/ 2024-04-16T17:48:32.448 DEBUG:boto:auth_path=/lybtug-1/ 2024-04-16T17:48:32.448 DEBUG:boto:Method: PUT 2024-04-16T17:48:32.448 DEBUG:boto:Path: /lybtug-1/ 2024-04-16T17:48:32.448 DEBUG:boto:Data: 2024-04-16T17:48:32.448 DEBUG:boto:Headers: {} 2024-04-16T17:48:32.449 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:32.449 DEBUG:boto:Port: 8000 2024-04-16T17:48:32.449 DEBUG:boto:Params: {} 2024-04-16T17:48:32.449 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T17:48:32.449 DEBUG:boto:Token: None 2024-04-16T17:48:32.449 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:32 GMT /lybtug-1/ 2024-04-16T17:48:32.449 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r6f0olrQlbWJtA2Z47ue0dzHN+M= 2024-04-16T17:48:32.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r6f0olrQlbWJtA2Z47ue0dzHN+M=', 'Content-Length': '0'} 2024-04-16T17:48:35.369 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005c33dd413137de9b-00661eb9f0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:35.370 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-2 2024-04-16T17:48:35.370 DEBUG:boto:path=/lybtug-2/ 2024-04-16T17:48:35.370 DEBUG:boto:auth_path=/lybtug-2/ 2024-04-16T17:48:35.370 DEBUG:boto:Method: PUT 2024-04-16T17:48:35.370 DEBUG:boto:Path: /lybtug-2/ 2024-04-16T17:48:35.370 DEBUG:boto:Data: 2024-04-16T17:48:35.370 DEBUG:boto:Headers: {} 2024-04-16T17:48:35.370 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:35.370 DEBUG:boto:Port: 8000 2024-04-16T17:48:35.371 DEBUG:boto:Params: {} 2024-04-16T17:48:35.371 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T17:48:35.371 DEBUG:boto:Token: None 2024-04-16T17:48:35.371 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:35 GMT /lybtug-2/ 2024-04-16T17:48:35.371 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K2dt9ejc6elqT8jftq99t0x7LWI= 2024-04-16T17:48:35.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K2dt9ejc6elqT8jftq99t0x7LWI=', 'Content-Length': '0'} 2024-04-16T17:48:37.547 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000dd27418b03e6868-00661eb9f3-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.547 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:37.547 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:37.548 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:37.591 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:37.591 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:37.690 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"1_1713289715.407643_46.1","last_update":"2024-04-16T17:48:35.407643Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:37.690 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289706.527539_11.1', 1: '1_1713289715.407643_46.1', 2: '', 3: ''} 2024-04-16T17:48:37.690 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:37.691 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:37.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:37.691 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:37.733 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:37.733 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:37.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:37.801+0000 7fe858577980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:37.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:37.805+0000 7fe858577980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:37.833 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289715.407643_46.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:35.407643Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:37Z"} 2024-04-16T17:48:37.833 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289715.407643_46.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:35.407643Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:37Z"} 2024-04-16T17:48:37.833 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289715.407643_46.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:35.407643Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:37.833 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289715.407643_46.1', 2: '', 3: ''} 2024-04-16T17:48:37.834 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289715.407643_46.1', 2: '', 3: ''} 2024-04-16T17:48:37.834 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:37.834 DEBUG:boto:path=/lybtug-1/ 2024-04-16T17:48:37.834 DEBUG:boto:auth_path=/lybtug-1/ 2024-04-16T17:48:37.834 DEBUG:boto:Method: HEAD 2024-04-16T17:48:37.834 DEBUG:boto:Path: /lybtug-1/ 2024-04-16T17:48:37.834 DEBUG:boto:Data: 2024-04-16T17:48:37.834 DEBUG:boto:Headers: {} 2024-04-16T17:48:37.834 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:37.834 DEBUG:boto:Port: 8000 2024-04-16T17:48:37.835 DEBUG:boto:Params: {} 2024-04-16T17:48:37.835 DEBUG:boto:Token: None 2024-04-16T17:48:37.835 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:37 GMT /lybtug-1/ 2024-04-16T17:48:37.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q4Wpjh0q1ZtR1GZtDPbS8XN8U+o= 2024-04-16T17:48:37.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q4Wpjh0q1ZtR1GZtDPbS8XN8U+o=', 'Content-Length': '0'} 2024-04-16T17:48:37.839 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dda602a4855a64e4-00661eb9f5-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.839 DEBUG:boto:path=/lybtug-2/ 2024-04-16T17:48:37.839 DEBUG:boto:auth_path=/lybtug-2/ 2024-04-16T17:48:37.839 DEBUG:boto:Method: HEAD 2024-04-16T17:48:37.839 DEBUG:boto:Path: /lybtug-2/ 2024-04-16T17:48:37.839 DEBUG:boto:Data: 2024-04-16T17:48:37.839 DEBUG:boto:Headers: {} 2024-04-16T17:48:37.840 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:37.840 DEBUG:boto:Port: 8000 2024-04-16T17:48:37.840 DEBUG:boto:Params: {} 2024-04-16T17:48:37.840 DEBUG:boto:Token: None 2024-04-16T17:48:37.840 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:37 GMT /lybtug-2/ 2024-04-16T17:48:37.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FNmJVXAYOH3YEp1g6bqB8iH0OVU= 2024-04-16T17:48:37.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FNmJVXAYOH3YEp1g6bqB8iH0OVU=', 'Content-Length': '0'} 2024-04-16T17:48:37.843 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000b453e9ab10d755e-00661eb9f5-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.844 DEBUG:boto:path=/lybtug-1/ 2024-04-16T17:48:37.844 DEBUG:boto:auth_path=/lybtug-1/ 2024-04-16T17:48:37.844 DEBUG:boto:Method: HEAD 2024-04-16T17:48:37.844 DEBUG:boto:Path: /lybtug-1/ 2024-04-16T17:48:37.844 DEBUG:boto:Data: 2024-04-16T17:48:37.844 DEBUG:boto:Headers: {} 2024-04-16T17:48:37.844 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:37.844 DEBUG:boto:Port: 8000 2024-04-16T17:48:37.844 DEBUG:boto:Params: {} 2024-04-16T17:48:37.844 DEBUG:boto:Token: None 2024-04-16T17:48:37.845 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:37 GMT /lybtug-1/ 2024-04-16T17:48:37.845 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q4Wpjh0q1ZtR1GZtDPbS8XN8U+o= 2024-04-16T17:48:37.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q4Wpjh0q1ZtR1GZtDPbS8XN8U+o=', 'Content-Length': '0'} 2024-04-16T17:48:37.848 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aaef26a2c9e90efe-00661eb9f5-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.848 DEBUG:boto:path=/lybtug-2/ 2024-04-16T17:48:37.848 DEBUG:boto:auth_path=/lybtug-2/ 2024-04-16T17:48:37.848 DEBUG:boto:Method: HEAD 2024-04-16T17:48:37.848 DEBUG:boto:Path: /lybtug-2/ 2024-04-16T17:48:37.849 DEBUG:boto:Data: 2024-04-16T17:48:37.849 DEBUG:boto:Headers: {} 2024-04-16T17:48:37.849 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:37.849 DEBUG:boto:Port: 8000 2024-04-16T17:48:37.849 DEBUG:boto:Params: {} 2024-04-16T17:48:37.849 DEBUG:boto:Token: None 2024-04-16T17:48:37.849 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:37 GMT /lybtug-2/ 2024-04-16T17:48:37.849 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FNmJVXAYOH3YEp1g6bqB8iH0OVU= 2024-04-16T17:48:37.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FNmJVXAYOH3YEp1g6bqB8iH0OVU=', 'Content-Length': '0'} 2024-04-16T17:48:37.852 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003adc7d7557f02423-00661eb9f5-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.853 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_create ... ok 2024-04-16T17:48:37.853 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-3 2024-04-16T17:48:37.853 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:37.853 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:37.854 DEBUG:boto:Method: PUT 2024-04-16T17:48:37.854 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:37.854 DEBUG:boto:Data: 2024-04-16T17:48:37.854 DEBUG:boto:Headers: {} 2024-04-16T17:48:37.854 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:37.854 DEBUG:boto:Port: 8000 2024-04-16T17:48:37.854 DEBUG:boto:Params: {} 2024-04-16T17:48:37.854 DEBUG:boto:Token: None 2024-04-16T17:48:37.854 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:37 GMT /lybtug-3/ 2024-04-16T17:48:37.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vGi61QwV4fRuCDrPyA8w0OD0D4U= 2024-04-16T17:48:37.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vGi61QwV4fRuCDrPyA8w0OD0D4U=', 'Content-Length': '0'} 2024-04-16T17:48:37.888 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002806c4ba08a0b81e-00661eb9f5-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.889 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-4 2024-04-16T17:48:37.889 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:37.889 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:37.889 DEBUG:boto:Method: PUT 2024-04-16T17:48:37.889 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:37.889 DEBUG:boto:Data: 2024-04-16T17:48:37.889 DEBUG:boto:Headers: {} 2024-04-16T17:48:37.889 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:37.889 DEBUG:boto:Port: 8000 2024-04-16T17:48:37.889 DEBUG:boto:Params: {} 2024-04-16T17:48:37.890 DEBUG:boto:Token: None 2024-04-16T17:48:37.890 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:37 GMT /lybtug-4/ 2024-04-16T17:48:37.890 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XXENLqtkznVYaUoHMjKfN64Cdms= 2024-04-16T17:48:37.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XXENLqtkznVYaUoHMjKfN64Cdms=', 'Content-Length': '0'} 2024-04-16T17:48:37.947 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a840d1384dcedbf5-00661eb9f5-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:37.948 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:37.948 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:37.948 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:37.988 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:37.988 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:38.054 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:38.085 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"1_1713289717.922253_54.1","last_update":"2024-04-16T17:48:37.922253Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:38.085 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289706.527539_11.1', 1: '1_1713289717.922253_54.1', 2: '', 3: ''} 2024-04-16T17:48:38.085 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:38.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.086 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:38.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:38.197+0000 7fe03b704980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:38.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:38.197+0000 7fe03b704980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:38.228 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289717.922253_54.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:37.922253Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:38Z"} 2024-04-16T17:48:38.229 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289717.922253_54.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:37.922253Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:38Z"} 2024-04-16T17:48:38.229 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289717.922253_54.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:37.922253Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:38.229 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289717.922253_54.1', 2: '', 3: ''} 2024-04-16T17:48:38.229 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289717.922253_54.1', 2: '', 3: ''} 2024-04-16T17:48:38.229 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:38.229 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.229 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.229 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.229 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.229 DEBUG:boto:Data: 2024-04-16T17:48:38.230 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.230 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.230 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.230 DEBUG:boto:Params: {} 2024-04-16T17:48:38.230 DEBUG:boto:Token: None 2024-04-16T17:48:38.230 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.230 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI= 2024-04-16T17:48:38.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI=', 'Content-Length': '0'} 2024-04-16T17:48:38.234 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007f6a6108c469728a-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.234 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.234 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.234 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.234 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.234 DEBUG:boto:Data: 2024-04-16T17:48:38.235 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.235 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.235 DEBUG:boto:Params: {} 2024-04-16T17:48:38.235 DEBUG:boto:Token: None 2024-04-16T17:48:38.235 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg= 2024-04-16T17:48:38.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg=', 'Content-Length': '0'} 2024-04-16T17:48:38.238 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b6fb4abcfbe51a83-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.239 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.239 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.239 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.239 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.239 DEBUG:boto:Data: 2024-04-16T17:48:38.239 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.239 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.239 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.239 DEBUG:boto:Params: {} 2024-04-16T17:48:38.240 DEBUG:boto:Token: None 2024-04-16T17:48:38.240 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI= 2024-04-16T17:48:38.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI=', 'Content-Length': '0'} 2024-04-16T17:48:38.243 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e543a076e0acc30d-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.243 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.243 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.243 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.243 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.244 DEBUG:boto:Data: 2024-04-16T17:48:38.244 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.244 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.244 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.244 DEBUG:boto:Params: {} 2024-04-16T17:48:38.244 DEBUG:boto:Token: None 2024-04-16T17:48:38.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg= 2024-04-16T17:48:38.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg=', 'Content-Length': '0'} 2024-04-16T17:48:38.247 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002f57e923abb7429e-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.247 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.247 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.247 DEBUG:boto:Method: PUT 2024-04-16T17:48:38.247 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.247 DEBUG:boto:Data: 2024-04-16T17:48:38.247 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.247 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.248 DEBUG:boto:Params: {} 2024-04-16T17:48:38.248 DEBUG:boto:Token: None 2024-04-16T17:48:38.248 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:St02WUMVuQaxhmIKxgHCnhqyLAo= 2024-04-16T17:48:38.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:St02WUMVuQaxhmIKxgHCnhqyLAo=', 'Content-Length': '0'} 2024-04-16T17:48:38.293 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003d9f94e07950ace6-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.293 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.293 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.293 DEBUG:boto:Method: PUT 2024-04-16T17:48:38.293 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.293 DEBUG:boto:Data: 2024-04-16T17:48:38.293 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.293 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.293 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.293 DEBUG:boto:Params: {} 2024-04-16T17:48:38.293 DEBUG:boto:Token: None 2024-04-16T17:48:38.294 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZZJpcmxQiLMt0iJt/UmsTqYDiEs= 2024-04-16T17:48:38.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZZJpcmxQiLMt0iJt/UmsTqYDiEs=', 'Content-Length': '0'} 2024-04-16T17:48:38.335 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000038bfbb4b844483d4-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.335 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.335 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.335 DEBUG:boto:Method: PUT 2024-04-16T17:48:38.335 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.335 DEBUG:boto:Data: 2024-04-16T17:48:38.336 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.336 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.336 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.336 DEBUG:boto:Params: {} 2024-04-16T17:48:38.336 DEBUG:boto:Token: None 2024-04-16T17:48:38.336 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:St02WUMVuQaxhmIKxgHCnhqyLAo= 2024-04-16T17:48:38.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:St02WUMVuQaxhmIKxgHCnhqyLAo=', 'Content-Length': '0'} 2024-04-16T17:48:38.404 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000f65690e5812f21a7-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.404 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.404 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.404 DEBUG:boto:Method: PUT 2024-04-16T17:48:38.404 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.404 DEBUG:boto:Data: 2024-04-16T17:48:38.405 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.405 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.405 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.405 DEBUG:boto:Params: {} 2024-04-16T17:48:38.405 DEBUG:boto:Token: None 2024-04-16T17:48:38.405 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZZJpcmxQiLMt0iJt/UmsTqYDiEs= 2024-04-16T17:48:38.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZZJpcmxQiLMt0iJt/UmsTqYDiEs=', 'Content-Length': '0'} 2024-04-16T17:48:38.514 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d5a56318e387dbdc-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.514 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.515 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.515 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.515 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.515 DEBUG:boto:Data: 2024-04-16T17:48:38.515 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.515 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.515 DEBUG:boto:Params: {} 2024-04-16T17:48:38.515 DEBUG:boto:Token: None 2024-04-16T17:48:38.515 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI= 2024-04-16T17:48:38.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI=', 'Content-Length': '0'} 2024-04-16T17:48:38.520 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000037cbf98201a4621b-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.521 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.521 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.521 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.521 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.521 DEBUG:boto:Data: 2024-04-16T17:48:38.521 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.521 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.521 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.521 DEBUG:boto:Params: {} 2024-04-16T17:48:38.521 DEBUG:boto:Token: None 2024-04-16T17:48:38.522 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg= 2024-04-16T17:48:38.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg=', 'Content-Length': '0'} 2024-04-16T17:48:38.526 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000013f70af33003d95e-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.526 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.527 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.527 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.527 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.527 DEBUG:boto:Data: 2024-04-16T17:48:38.527 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.527 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.527 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.527 DEBUG:boto:Params: {} 2024-04-16T17:48:38.527 DEBUG:boto:Token: None 2024-04-16T17:48:38.527 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI= 2024-04-16T17:48:38.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI=', 'Content-Length': '0'} 2024-04-16T17:48:38.533 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a798da31a9d91fe8-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.533 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.533 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.533 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.533 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.533 DEBUG:boto:Data: 2024-04-16T17:48:38.533 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.533 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.533 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.534 DEBUG:boto:Params: {} 2024-04-16T17:48:38.534 DEBUG:boto:Token: None 2024-04-16T17:48:38.534 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg= 2024-04-16T17:48:38.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg=', 'Content-Length': '0'} 2024-04-16T17:48:38.538 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f5ee70fd9ac2302c-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.538 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.539 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:38.583 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:38.583 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:38.676 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"1_1713289718.436731_70.1","last_update":"2024-04-16T17:48:38.436731Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:38.677 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289706.527539_11.1', 1: '1_1713289718.436731_70.1', 2: '', 3: ''} 2024-04-16T17:48:38.677 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:38.677 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.677 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:38.719 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:38.719 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:38.785+0000 7f2bc0144980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:38.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:38.789+0000 7f2bc0144980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:38.817 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289718.436731_70.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:38.436731Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:38Z"} 2024-04-16T17:48:38.817 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289718.436731_70.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:38.436731Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:38Z"} 2024-04-16T17:48:38.817 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289718.436731_70.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:38.436731Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:38.817 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289718.436731_70.1', 2: '', 3: ''} 2024-04-16T17:48:38.817 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289718.436731_70.1', 2: '', 3: ''} 2024-04-16T17:48:38.817 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:38.817 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.817 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.818 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.818 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.818 DEBUG:boto:Data: 2024-04-16T17:48:38.818 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.818 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.818 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.818 DEBUG:boto:Params: {} 2024-04-16T17:48:38.818 DEBUG:boto:Token: None 2024-04-16T17:48:38.818 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.819 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI= 2024-04-16T17:48:38.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI=', 'Content-Length': '0'} 2024-04-16T17:48:38.823 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ad4dde807160d199-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.823 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.823 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.823 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.823 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.823 DEBUG:boto:Data: 2024-04-16T17:48:38.823 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.823 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.823 DEBUG:boto:Params: {} 2024-04-16T17:48:38.824 DEBUG:boto:Token: None 2024-04-16T17:48:38.824 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg= 2024-04-16T17:48:38.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg=', 'Content-Length': '0'} 2024-04-16T17:48:38.827 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d73e66a8a6959520-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.827 DEBUG:boto:path=/lybtug-3/ 2024-04-16T17:48:38.828 DEBUG:boto:auth_path=/lybtug-3/ 2024-04-16T17:48:38.828 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.828 DEBUG:boto:Path: /lybtug-3/ 2024-04-16T17:48:38.828 DEBUG:boto:Data: 2024-04-16T17:48:38.828 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.828 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.828 DEBUG:boto:Params: {} 2024-04-16T17:48:38.828 DEBUG:boto:Token: None 2024-04-16T17:48:38.828 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-3/ 2024-04-16T17:48:38.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI= 2024-04-16T17:48:38.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rg1ityaOogSmESphLMhKwdwK2TI=', 'Content-Length': '0'} 2024-04-16T17:48:38.832 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000171342e186cb1300-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.832 DEBUG:boto:path=/lybtug-4/ 2024-04-16T17:48:38.832 DEBUG:boto:auth_path=/lybtug-4/ 2024-04-16T17:48:38.832 DEBUG:boto:Method: HEAD 2024-04-16T17:48:38.832 DEBUG:boto:Path: /lybtug-4/ 2024-04-16T17:48:38.832 DEBUG:boto:Data: 2024-04-16T17:48:38.833 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.833 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.833 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.833 DEBUG:boto:Params: {} 2024-04-16T17:48:38.833 DEBUG:boto:Token: None 2024-04-16T17:48:38.833 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:38 GMT /lybtug-4/ 2024-04-16T17:48:38.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg= 2024-04-16T17:48:38.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7GQ4sff+Exk7itKNnBYRGXzQ0fg=', 'Content-Length': '0'} 2024-04-16T17:48:38.835 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea61f6d0e1a20df7-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.836 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_recreate ... ok 2024-04-16T17:48:38.836 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-5 2024-04-16T17:48:38.837 DEBUG:boto:path=/lybtug-5/ 2024-04-16T17:48:38.837 DEBUG:boto:auth_path=/lybtug-5/ 2024-04-16T17:48:38.837 DEBUG:boto:Method: PUT 2024-04-16T17:48:38.837 DEBUG:boto:Path: /lybtug-5/ 2024-04-16T17:48:38.837 DEBUG:boto:Data: 2024-04-16T17:48:38.837 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.837 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:38.837 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.837 DEBUG:boto:Params: {} 2024-04-16T17:48:38.837 DEBUG:boto:Token: None 2024-04-16T17:48:38.838 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:38 GMT /lybtug-5/ 2024-04-16T17:48:38.838 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ClKlXwoCzAU1jYEHf1V/F09pX0= 2024-04-16T17:48:38.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ClKlXwoCzAU1jYEHf1V/F09pX0=', 'Content-Length': '0'} 2024-04-16T17:48:38.870 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d7d91c867e5e8045-00661eb9f6-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.870 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-6 2024-04-16T17:48:38.870 DEBUG:boto:path=/lybtug-6/ 2024-04-16T17:48:38.870 DEBUG:boto:auth_path=/lybtug-6/ 2024-04-16T17:48:38.870 DEBUG:boto:Method: PUT 2024-04-16T17:48:38.870 DEBUG:boto:Path: /lybtug-6/ 2024-04-16T17:48:38.870 DEBUG:boto:Data: 2024-04-16T17:48:38.870 DEBUG:boto:Headers: {} 2024-04-16T17:48:38.870 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:38.871 DEBUG:boto:Port: 8000 2024-04-16T17:48:38.871 DEBUG:boto:Params: {} 2024-04-16T17:48:38.871 DEBUG:boto:Token: None 2024-04-16T17:48:38.871 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:38 GMT /lybtug-6/ 2024-04-16T17:48:38.871 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wrT1o3k55J+1S4RXRywQWdTLFqM= 2024-04-16T17:48:38.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wrT1o3k55J+1S4RXRywQWdTLFqM=', 'Content-Length': '0'} 2024-04-16T17:48:38.928 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000cf0987ea39516c7e-00661eb9f6-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:38.928 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.928 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:38.928 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:38.970 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:38.970 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:39.036 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:39.067 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"1_1713289718.905541_78.1","last_update":"2024-04-16T17:48:38.905541Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:39.067 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289706.527539_11.1', 1: '1_1713289718.905541_78.1', 2: '', 3: ''} 2024-04-16T17:48:39.068 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:39.068 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:39.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:39.068 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:39.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:39.181+0000 7f104b08b980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:39.185+0000 7f104b08b980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:39.212 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289718.888919_75.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:38.888919Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:39Z"} 2024-04-16T17:48:39.212 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289718.888919_75.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:38.888919Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:39Z"} 2024-04-16T17:48:39.212 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289718.888919_75.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:38.888919Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:39.213 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289718.905541_78.1', 2: '', 3: ''} 2024-04-16T17:48:39.213 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289718.888919_75.1', 2: '', 3: ''} 2024-04-16T17:48:39.213 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1713289718.905541_78.1 target=1_1713289718.888919_75.1 2024-04-16T17:48:39.500 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.497+0000 7f58c368b640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.504 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58c7693640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58ca699640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58c8695640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58cce9e640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58c6e92640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58bfe84640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58c468d640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58ce6a1640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.501+0000 7f58c0685640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:39.505 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T17:48:39.505+0000 7f58c4e8e640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:48:44.213 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:44.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:44.214 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:44.256 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:44.256 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:44.325+0000 7f9ac5741980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:44.325+0000 7f9ac5741980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:44.356 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289718.905541_78.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:38.905541Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:44Z"} 2024-04-16T17:48:44.357 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289718.905541_78.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:38.905541Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:44Z"} 2024-04-16T17:48:44.357 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289718.905541_78.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:38.905541Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:44.357 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289718.905541_78.1', 2: '', 3: ''} 2024-04-16T17:48:44.357 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289718.905541_78.1', 2: '', 3: ''} 2024-04-16T17:48:44.357 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:44.357 DEBUG:boto:path=/lybtug-5/ 2024-04-16T17:48:44.357 DEBUG:boto:auth_path=/lybtug-5/ 2024-04-16T17:48:44.358 DEBUG:boto:Method: HEAD 2024-04-16T17:48:44.358 DEBUG:boto:Path: /lybtug-5/ 2024-04-16T17:48:44.358 DEBUG:boto:Data: 2024-04-16T17:48:44.358 DEBUG:boto:Headers: {} 2024-04-16T17:48:44.358 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:44.358 DEBUG:boto:Port: 8000 2024-04-16T17:48:44.358 DEBUG:boto:Params: {} 2024-04-16T17:48:44.358 DEBUG:boto:Token: None 2024-04-16T17:48:44.358 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:44 GMT /lybtug-5/ 2024-04-16T17:48:44.359 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M9BGJLqqj6kqNOw0A4v6eyEgs1Y= 2024-04-16T17:48:44.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M9BGJLqqj6kqNOw0A4v6eyEgs1Y=', 'Content-Length': '0'} 2024-04-16T17:48:44.363 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000623688054b5f8d6d-00661eb9fc-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:44.363 DEBUG:boto:path=/lybtug-6/ 2024-04-16T17:48:44.363 DEBUG:boto:auth_path=/lybtug-6/ 2024-04-16T17:48:44.363 DEBUG:boto:Method: HEAD 2024-04-16T17:48:44.363 DEBUG:boto:Path: /lybtug-6/ 2024-04-16T17:48:44.363 DEBUG:boto:Data: 2024-04-16T17:48:44.363 DEBUG:boto:Headers: {} 2024-04-16T17:48:44.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:44.363 DEBUG:boto:Port: 8000 2024-04-16T17:48:44.363 DEBUG:boto:Params: {} 2024-04-16T17:48:44.364 DEBUG:boto:Token: None 2024-04-16T17:48:44.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:44 GMT /lybtug-6/ 2024-04-16T17:48:44.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cdHF6XHylpzS7a+837COL6CTlJ0= 2024-04-16T17:48:44.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cdHF6XHylpzS7a+837COL6CTlJ0=', 'Content-Length': '0'} 2024-04-16T17:48:44.368 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000951b4b69917d2d7b-00661eb9fc-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:44.368 DEBUG:boto:path=/lybtug-5/ 2024-04-16T17:48:44.368 DEBUG:boto:auth_path=/lybtug-5/ 2024-04-16T17:48:44.368 DEBUG:boto:Method: HEAD 2024-04-16T17:48:44.368 DEBUG:boto:Path: /lybtug-5/ 2024-04-16T17:48:44.368 DEBUG:boto:Data: 2024-04-16T17:48:44.368 DEBUG:boto:Headers: {} 2024-04-16T17:48:44.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:44.368 DEBUG:boto:Port: 8000 2024-04-16T17:48:44.369 DEBUG:boto:Params: {} 2024-04-16T17:48:44.369 DEBUG:boto:Token: None 2024-04-16T17:48:44.369 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:44 GMT /lybtug-5/ 2024-04-16T17:48:44.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M9BGJLqqj6kqNOw0A4v6eyEgs1Y= 2024-04-16T17:48:44.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M9BGJLqqj6kqNOw0A4v6eyEgs1Y=', 'Content-Length': '0'} 2024-04-16T17:48:44.373 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c2f6ef22b7c31ff1-00661eb9fc-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:44.373 DEBUG:boto:path=/lybtug-6/ 2024-04-16T17:48:44.373 DEBUG:boto:auth_path=/lybtug-6/ 2024-04-16T17:48:44.373 DEBUG:boto:Method: HEAD 2024-04-16T17:48:44.373 DEBUG:boto:Path: /lybtug-6/ 2024-04-16T17:48:44.373 DEBUG:boto:Data: 2024-04-16T17:48:44.373 DEBUG:boto:Headers: {} 2024-04-16T17:48:44.373 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:44.373 DEBUG:boto:Port: 8000 2024-04-16T17:48:44.373 DEBUG:boto:Params: {} 2024-04-16T17:48:44.373 DEBUG:boto:Token: None 2024-04-16T17:48:44.374 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:44 GMT /lybtug-6/ 2024-04-16T17:48:44.374 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cdHF6XHylpzS7a+837COL6CTlJ0= 2024-04-16T17:48:44.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cdHF6XHylpzS7a+837COL6CTlJ0=', 'Content-Length': '0'} 2024-04-16T17:48:44.377 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000220f5b0cb4085d58-00661eb9fc-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:44.377 DEBUG:boto:path=/lybtug-5/ 2024-04-16T17:48:44.377 DEBUG:boto:auth_path=/lybtug-5/ 2024-04-16T17:48:44.377 DEBUG:boto:Method: DELETE 2024-04-16T17:48:44.377 DEBUG:boto:Path: /lybtug-5/ 2024-04-16T17:48:44.377 DEBUG:boto:Data: 2024-04-16T17:48:44.377 DEBUG:boto:Headers: {} 2024-04-16T17:48:44.377 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:44.377 DEBUG:boto:Port: 8000 2024-04-16T17:48:44.378 DEBUG:boto:Params: {} 2024-04-16T17:48:44.378 DEBUG:boto:Token: None 2024-04-16T17:48:44.378 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:48:44 GMT /lybtug-5/ 2024-04-16T17:48:44.378 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5wz70Qi/kTmPJEi04SMnxH/sdSE= 2024-04-16T17:48:44.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5wz70Qi/kTmPJEi04SMnxH/sdSE=', 'Content-Length': '0'} 2024-04-16T17:48:44.443 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000001e44decdfdec323e-00661eb9fc-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:44.443 DEBUG:boto:path=/lybtug-6/ 2024-04-16T17:48:44.443 DEBUG:boto:auth_path=/lybtug-6/ 2024-04-16T17:48:44.443 DEBUG:boto:Method: DELETE 2024-04-16T17:48:44.443 DEBUG:boto:Path: /lybtug-6/ 2024-04-16T17:48:44.443 DEBUG:boto:Data: 2024-04-16T17:48:44.443 DEBUG:boto:Headers: {} 2024-04-16T17:48:44.443 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:44.444 DEBUG:boto:Port: 8000 2024-04-16T17:48:44.444 DEBUG:boto:Params: {} 2024-04-16T17:48:44.444 DEBUG:boto:Token: None 2024-04-16T17:48:44.444 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:48:44 GMT /lybtug-6/ 2024-04-16T17:48:44.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jMPzsoBWLhSn1Ah58JJGIotfuLc= 2024-04-16T17:48:44.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jMPzsoBWLhSn1Ah58JJGIotfuLc=', 'Content-Length': '0'} 2024-04-16T17:48:44.583 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006d2e17fe259ba1a7-00661eb9fc-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:44.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:44.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:44.583 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:44.625 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:44.692 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:44.722 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"1_1713289724.521201_84.1","last_update":"2024-04-16T17:48:44.521201Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:44.722 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289706.527539_11.1', 1: '1_1713289724.521201_84.1', 2: '', 3: ''} 2024-04-16T17:48:44.723 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:44.723 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:44.723 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:44.723 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:44.768 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:44.768 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:44.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:44.837+0000 7f5d11092980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:44.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:44.837+0000 7f5d11092980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:44.869 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289724.436419_82.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:44.436419Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:44Z"} 2024-04-16T17:48:44.869 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289724.436419_82.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:44.436419Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:44Z"} 2024-04-16T17:48:44.869 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289724.436419_82.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:44.436419Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:44.870 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289724.521201_84.1', 2: '', 3: ''} 2024-04-16T17:48:44.870 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289724.436419_82.1', 2: '', 3: ''} 2024-04-16T17:48:44.870 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1713289724.521201_84.1 target=1_1713289724.436419_82.1 2024-04-16T17:48:49.871 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:49.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:49.872 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:49.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:49.980+0000 7f699f5d5980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:49.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:49.984+0000 7f699f5d5980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:50.014 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289724.521201_84.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:44.521201Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:49Z"} 2024-04-16T17:48:50.014 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289724.521201_84.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:44.521201Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:49Z"} 2024-04-16T17:48:50.015 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289724.521201_84.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:44.521201Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:50.015 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289724.521201_84.1', 2: '', 3: ''} 2024-04-16T17:48:50.015 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289724.521201_84.1', 2: '', 3: ''} 2024-04-16T17:48:50.015 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:50.016 DEBUG:boto:path=/lybtug-5/ 2024-04-16T17:48:50.016 DEBUG:boto:auth_path=/lybtug-5/ 2024-04-16T17:48:50.016 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.016 DEBUG:boto:Path: /lybtug-5/ 2024-04-16T17:48:50.016 DEBUG:boto:Data: 2024-04-16T17:48:50.016 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.016 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.016 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.016 DEBUG:boto:Params: {} 2024-04-16T17:48:50.017 DEBUG:boto:Token: None 2024-04-16T17:48:50.017 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-5/ 2024-04-16T17:48:50.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6aBewVmTB2iH3ENSayuASfou3pU= 2024-04-16T17:48:50.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6aBewVmTB2iH3ENSayuASfou3pU=', 'Content-Length': '0'} 2024-04-16T17:48:50.019 DEBUG:boto:Response headers: [('Content-Length', '250'), ('x-amz-request-id', 'tx0000098229c91b0225fcc-00661eba02-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.019 DEBUG:boto:path=/lybtug-6/ 2024-04-16T17:48:50.019 DEBUG:boto:auth_path=/lybtug-6/ 2024-04-16T17:48:50.019 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.019 DEBUG:boto:Path: /lybtug-6/ 2024-04-16T17:48:50.019 DEBUG:boto:Data: 2024-04-16T17:48:50.019 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.019 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.019 DEBUG:boto:Params: {} 2024-04-16T17:48:50.020 DEBUG:boto:Token: None 2024-04-16T17:48:50.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-6/ 2024-04-16T17:48:50.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Idcyf2hbvhkEfUAD5T944QWxwMI= 2024-04-16T17:48:50.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Idcyf2hbvhkEfUAD5T944QWxwMI=', 'Content-Length': '0'} 2024-04-16T17:48:50.021 DEBUG:boto:Response headers: [('Content-Length', '250'), ('x-amz-request-id', 'tx0000013f4f01f9badddb9-00661eba02-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.021 DEBUG:boto:path=/lybtug-5/ 2024-04-16T17:48:50.021 DEBUG:boto:auth_path=/lybtug-5/ 2024-04-16T17:48:50.022 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.022 DEBUG:boto:Path: /lybtug-5/ 2024-04-16T17:48:50.022 DEBUG:boto:Data: 2024-04-16T17:48:50.022 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.022 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.022 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.022 DEBUG:boto:Params: {} 2024-04-16T17:48:50.022 DEBUG:boto:Token: None 2024-04-16T17:48:50.022 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-5/ 2024-04-16T17:48:50.022 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6aBewVmTB2iH3ENSayuASfou3pU= 2024-04-16T17:48:50.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6aBewVmTB2iH3ENSayuASfou3pU=', 'Content-Length': '0'} 2024-04-16T17:48:50.024 DEBUG:boto:Response headers: [('Content-Length', '250'), ('x-amz-request-id', 'tx00000c2975b6b338b8604-00661eba02-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.025 DEBUG:boto:path=/lybtug-6/ 2024-04-16T17:48:50.025 DEBUG:boto:auth_path=/lybtug-6/ 2024-04-16T17:48:50.025 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.025 DEBUG:boto:Path: /lybtug-6/ 2024-04-16T17:48:50.025 DEBUG:boto:Data: 2024-04-16T17:48:50.025 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.025 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.025 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.025 DEBUG:boto:Params: {} 2024-04-16T17:48:50.025 DEBUG:boto:Token: None 2024-04-16T17:48:50.026 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-6/ 2024-04-16T17:48:50.026 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Idcyf2hbvhkEfUAD5T944QWxwMI= 2024-04-16T17:48:50.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Idcyf2hbvhkEfUAD5T944QWxwMI=', 'Content-Length': '0'} 2024-04-16T17:48:50.027 DEBUG:boto:Response headers: [('Content-Length', '250'), ('x-amz-request-id', 'tx000002d853041d0326b11-00661eba02-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.027 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_remove ... ok 2024-04-16T17:48:50.028 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-7 2024-04-16T17:48:50.028 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.028 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.028 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.028 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.028 DEBUG:boto:Data: 2024-04-16T17:48:50.028 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.028 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.029 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.029 DEBUG:boto:Params: {} 2024-04-16T17:48:50.029 DEBUG:boto:Token: None 2024-04-16T17:48:50.029 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2pTptaGK77ISDdfCgAgzfGyEycY= 2024-04-16T17:48:50.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2pTptaGK77ISDdfCgAgzfGyEycY=', 'Content-Length': '0'} 2024-04-16T17:48:50.065 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003c5e1f911d5d55c7-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.065 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-8 2024-04-16T17:48:50.065 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.065 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.065 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.065 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.066 DEBUG:boto:Data: 2024-04-16T17:48:50.066 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.066 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.066 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.066 DEBUG:boto:Params: {} 2024-04-16T17:48:50.066 DEBUG:boto:Token: None 2024-04-16T17:48:50.066 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.066 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wr1tqFitR/rwCZaO0jtz0Ptskzo= 2024-04-16T17:48:50.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wr1tqFitR/rwCZaO0jtz0Ptskzo=', 'Content-Length': '0'} 2024-04-16T17:48:50.119 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000001efd317e8584ad26-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.119 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.120 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.120 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.120 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.120 DEBUG:boto:Data: 2024-04-16T17:48:50.120 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.120 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.120 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.120 DEBUG:boto:Params: {} 2024-04-16T17:48:50.120 DEBUG:boto:Token: None 2024-04-16T17:48:50.120 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.124 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f672a1ba65e8fe56-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.124 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:48:50.125 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:48:50.125 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.125 DEBUG:boto:Path: /lybtug-7/myobj 2024-04-16T17:48:50.125 DEBUG:boto:Data: 2024-04-16T17:48:50.125 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.125 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.125 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.125 DEBUG:boto:Params: {} 2024-04-16T17:48:50.125 DEBUG:boto:Token: None 2024-04-16T17:48:50.125 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/myobj 2024-04-16T17:48:50.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bRKk4336reC+DPS9CY6Z87oW+rQ= 2024-04-16T17:48:50.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bRKk4336reC+DPS9CY6Z87oW+rQ='} 2024-04-16T17:48:50.148 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a6b570e3093982ce-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.149 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.149 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.149 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.149 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.149 DEBUG:boto:Data: 2024-04-16T17:48:50.149 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.149 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.149 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.149 DEBUG:boto:Params: {} 2024-04-16T17:48:50.149 DEBUG:boto:Token: None 2024-04-16T17:48:50.150 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.150 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.153 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008511928a75138f92-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.153 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:48:50.153 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:48:50.154 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.154 DEBUG:boto:Path: /lybtug-7/_myobj 2024-04-16T17:48:50.154 DEBUG:boto:Data: 2024-04-16T17:48:50.154 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.154 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.154 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.154 DEBUG:boto:Params: {} 2024-04-16T17:48:50.154 DEBUG:boto:Token: None 2024-04-16T17:48:50.154 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/_myobj 2024-04-16T17:48:50.154 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7FkHJ+7oS6+B13FGDk0LWZ+dLQ= 2024-04-16T17:48:50.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7FkHJ+7oS6+B13FGDk0LWZ+dLQ='} 2024-04-16T17:48:50.174 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a56cd22d8104c086-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.174 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.174 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.174 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.174 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.175 DEBUG:boto:Data: 2024-04-16T17:48:50.175 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.175 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.175 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.175 DEBUG:boto:Params: {} 2024-04-16T17:48:50.175 DEBUG:boto:Token: None 2024-04-16T17:48:50.175 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.175 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.179 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d937f9526ec6b8a5-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.179 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:48:50.179 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:48:50.179 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.179 DEBUG:boto:Path: /lybtug-7/%3A 2024-04-16T17:48:50.179 DEBUG:boto:Data: 2024-04-16T17:48:50.179 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.180 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.180 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.180 DEBUG:boto:Params: {} 2024-04-16T17:48:50.180 DEBUG:boto:Token: None 2024-04-16T17:48:50.180 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/%3A 2024-04-16T17:48:50.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X4x8wDMVB1UHoo8c0ViPglA0/DA= 2024-04-16T17:48:50.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X4x8wDMVB1UHoo8c0ViPglA0/DA='} 2024-04-16T17:48:50.223 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e765f72c10f8b635-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.223 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.223 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.223 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.223 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.223 DEBUG:boto:Data: 2024-04-16T17:48:50.223 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.224 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.224 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.224 DEBUG:boto:Params: {} 2024-04-16T17:48:50.224 DEBUG:boto:Token: None 2024-04-16T17:48:50.224 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.228 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cec75c06c9bb939d-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.228 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:48:50.228 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:48:50.229 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.229 DEBUG:boto:Path: /lybtug-7/%26 2024-04-16T17:48:50.229 DEBUG:boto:Data: 2024-04-16T17:48:50.229 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.229 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.229 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.229 DEBUG:boto:Params: {} 2024-04-16T17:48:50.229 DEBUG:boto:Token: None 2024-04-16T17:48:50.229 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/%26 2024-04-16T17:48:50.229 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vq3+15Ify136ZnSLRt/RGmesShw= 2024-04-16T17:48:50.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vq3+15Ify136ZnSLRt/RGmesShw='} 2024-04-16T17:48:50.241 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000921064771c2850c3-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.241 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.241 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.241 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.242 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.242 DEBUG:boto:Data: 2024-04-16T17:48:50.242 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.242 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.242 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.242 DEBUG:boto:Params: {} 2024-04-16T17:48:50.242 DEBUG:boto:Token: None 2024-04-16T17:48:50.242 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.242 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.245 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005ed694d1382fa0ef-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.246 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:48:50.246 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:48:50.246 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.246 DEBUG:boto:Path: /lybtug-7/. 2024-04-16T17:48:50.246 DEBUG:boto:Data: 2024-04-16T17:48:50.246 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.246 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.246 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.246 DEBUG:boto:Params: {} 2024-04-16T17:48:50.247 DEBUG:boto:Token: None 2024-04-16T17:48:50.247 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/. 2024-04-16T17:48:50.247 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oh2pf4n2X9e4USZYlM1jHH1OV4Q= 2024-04-16T17:48:50.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oh2pf4n2X9e4USZYlM1jHH1OV4Q='} 2024-04-16T17:48:50.290 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000902c80c572c4a904-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.291 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.291 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.291 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.291 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.291 DEBUG:boto:Data: 2024-04-16T17:48:50.291 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.291 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.291 DEBUG:boto:Params: {} 2024-04-16T17:48:50.291 DEBUG:boto:Token: None 2024-04-16T17:48:50.291 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.295 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea2132942a7e1d12-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.295 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:48:50.295 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:48:50.295 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.295 DEBUG:boto:Path: /lybtug-7/.. 2024-04-16T17:48:50.295 DEBUG:boto:Data: 2024-04-16T17:48:50.295 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.296 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.296 DEBUG:boto:Params: {} 2024-04-16T17:48:50.296 DEBUG:boto:Token: None 2024-04-16T17:48:50.296 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/.. 2024-04-16T17:48:50.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uhMGYui3v0xMUfWG26VZa7HyDo0= 2024-04-16T17:48:50.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uhMGYui3v0xMUfWG26VZa7HyDo0='} 2024-04-16T17:48:50.307 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bfdbbe0f67c9b0f7-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.307 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.307 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.307 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.307 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.307 DEBUG:boto:Data: 2024-04-16T17:48:50.307 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.308 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.308 DEBUG:boto:Params: {} 2024-04-16T17:48:50.308 DEBUG:boto:Token: None 2024-04-16T17:48:50.308 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.311 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f44e581e35c0e10c-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.312 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:48:50.312 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:48:50.312 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.312 DEBUG:boto:Path: /lybtug-7/... 2024-04-16T17:48:50.312 DEBUG:boto:Data: 2024-04-16T17:48:50.312 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.312 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.312 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.312 DEBUG:boto:Params: {} 2024-04-16T17:48:50.312 DEBUG:boto:Token: None 2024-04-16T17:48:50.313 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/... 2024-04-16T17:48:50.313 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LxEOnskKymppY1vyBv7o2j40cvc= 2024-04-16T17:48:50.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LxEOnskKymppY1vyBv7o2j40cvc='} 2024-04-16T17:48:50.354 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001db8869084333767-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.355 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.355 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.355 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.355 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.355 DEBUG:boto:Data: 2024-04-16T17:48:50.355 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.355 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.355 DEBUG:boto:Params: {} 2024-04-16T17:48:50.355 DEBUG:boto:Token: None 2024-04-16T17:48:50.355 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.359 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000004716ce7d384379f-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.360 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:48:50.360 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:48:50.360 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.360 DEBUG:boto:Path: /lybtug-7/.o 2024-04-16T17:48:50.360 DEBUG:boto:Data: 2024-04-16T17:48:50.360 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.360 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.360 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.360 DEBUG:boto:Params: {} 2024-04-16T17:48:50.360 DEBUG:boto:Token: None 2024-04-16T17:48:50.361 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/.o 2024-04-16T17:48:50.361 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajZdMjXiJrmm9UjLPfijaZV4F1M= 2024-04-16T17:48:50.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajZdMjXiJrmm9UjLPfijaZV4F1M='} 2024-04-16T17:48:50.369 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000700c07a28b6417a5-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.369 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:48:50.369 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:48:50.370 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.370 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:48:50.370 DEBUG:boto:Data: 2024-04-16T17:48:50.370 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.370 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.370 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.370 DEBUG:boto:Params: {} 2024-04-16T17:48:50.370 DEBUG:boto:Token: None 2024-04-16T17:48:50.370 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/ 2024-04-16T17:48:50.370 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs= 2024-04-16T17:48:50.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5uxbPgVv5/6cpXuef3uc25pvOFs=', 'Content-Length': '0'} 2024-04-16T17:48:50.373 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a1fd747105dc2f03-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.374 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:48:50.374 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:48:50.374 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.374 DEBUG:boto:Path: /lybtug-7/.o. 2024-04-16T17:48:50.374 DEBUG:boto:Data: 2024-04-16T17:48:50.374 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.374 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:48:50.375 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.375 DEBUG:boto:Params: {} 2024-04-16T17:48:50.375 DEBUG:boto:Token: None 2024-04-16T17:48:50.375 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-7/.o. 2024-04-16T17:48:50.375 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HlABdLN97OANG0zy9QuUNEfGrQE= 2024-04-16T17:48:50.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HlABdLN97OANG0zy9QuUNEfGrQE='} 2024-04-16T17:48:50.418 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ce85c845282a9745-00661eba02-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.419 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.419 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.419 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.419 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.419 DEBUG:boto:Data: 2024-04-16T17:48:50.419 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.419 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.419 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.419 DEBUG:boto:Params: {} 2024-04-16T17:48:50.419 DEBUG:boto:Token: None 2024-04-16T17:48:50.419 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.423 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000771411acb0da6565-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.424 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:48:50.424 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:48:50.424 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.424 DEBUG:boto:Path: /lybtug-8/myobj 2024-04-16T17:48:50.424 DEBUG:boto:Data: 2024-04-16T17:48:50.424 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.424 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.424 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.424 DEBUG:boto:Params: {} 2024-04-16T17:48:50.425 DEBUG:boto:Token: None 2024-04-16T17:48:50.425 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/myobj 2024-04-16T17:48:50.425 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KGjDl5YT/Q8GXjKHpbFFyTaSBME= 2024-04-16T17:48:50.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KGjDl5YT/Q8GXjKHpbFFyTaSBME='} 2024-04-16T17:48:50.450 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000964d507302867ff4-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.450 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.450 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.450 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.450 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.450 DEBUG:boto:Data: 2024-04-16T17:48:50.450 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.451 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.451 DEBUG:boto:Params: {} 2024-04-16T17:48:50.451 DEBUG:boto:Token: None 2024-04-16T17:48:50.451 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.451 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.455 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f11ca314d9beac86-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.455 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:48:50.455 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:48:50.455 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.455 DEBUG:boto:Path: /lybtug-8/_myobj 2024-04-16T17:48:50.455 DEBUG:boto:Data: 2024-04-16T17:48:50.455 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.455 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.455 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.456 DEBUG:boto:Params: {} 2024-04-16T17:48:50.456 DEBUG:boto:Token: None 2024-04-16T17:48:50.456 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/_myobj 2024-04-16T17:48:50.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sV7E1SG4TN7cOqP0HnbG8/9guMc= 2024-04-16T17:48:50.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sV7E1SG4TN7cOqP0HnbG8/9guMc='} 2024-04-16T17:48:50.499 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b1cccc99b1e8746f-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.499 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.499 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.499 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.499 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.499 DEBUG:boto:Data: 2024-04-16T17:48:50.499 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.499 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.499 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.499 DEBUG:boto:Params: {} 2024-04-16T17:48:50.500 DEBUG:boto:Token: None 2024-04-16T17:48:50.500 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.504 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002473fa16e28076b0-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.504 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:48:50.504 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:48:50.504 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.504 DEBUG:boto:Path: /lybtug-8/%3A 2024-04-16T17:48:50.505 DEBUG:boto:Data: 2024-04-16T17:48:50.505 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.505 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.505 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.505 DEBUG:boto:Params: {} 2024-04-16T17:48:50.505 DEBUG:boto:Token: None 2024-04-16T17:48:50.505 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/%3A 2024-04-16T17:48:50.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J5BFuEsyI0hGaKXnW74vlb6Zf8k= 2024-04-16T17:48:50.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J5BFuEsyI0hGaKXnW74vlb6Zf8k='} 2024-04-16T17:48:50.527 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b3f2940aa5e2f04d-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.527 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.527 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.527 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.527 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.527 DEBUG:boto:Data: 2024-04-16T17:48:50.528 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.528 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.528 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.528 DEBUG:boto:Params: {} 2024-04-16T17:48:50.528 DEBUG:boto:Token: None 2024-04-16T17:48:50.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.532 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000019099fc5298a9aec-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.532 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:48:50.532 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:48:50.532 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.532 DEBUG:boto:Path: /lybtug-8/%26 2024-04-16T17:48:50.532 DEBUG:boto:Data: 2024-04-16T17:48:50.532 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.533 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.533 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.533 DEBUG:boto:Params: {} 2024-04-16T17:48:50.533 DEBUG:boto:Token: None 2024-04-16T17:48:50.533 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/%26 2024-04-16T17:48:50.533 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xKtFkxgIpQxWpfveMcxVW0nQ4yY= 2024-04-16T17:48:50.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xKtFkxgIpQxWpfveMcxVW0nQ4yY='} 2024-04-16T17:48:50.578 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008e99f6b881c418a5-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.579 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.579 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.579 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.579 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.579 DEBUG:boto:Data: 2024-04-16T17:48:50.579 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.579 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.579 DEBUG:boto:Params: {} 2024-04-16T17:48:50.579 DEBUG:boto:Token: None 2024-04-16T17:48:50.580 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.583 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007e3e4fe160e5456b-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.583 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:48:50.584 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:48:50.584 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.584 DEBUG:boto:Path: /lybtug-8/. 2024-04-16T17:48:50.584 DEBUG:boto:Data: 2024-04-16T17:48:50.584 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.584 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.584 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.584 DEBUG:boto:Params: {} 2024-04-16T17:48:50.584 DEBUG:boto:Token: None 2024-04-16T17:48:50.585 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/. 2024-04-16T17:48:50.585 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VeL8CW5YGmhps6HcFD0lYf5iTIU= 2024-04-16T17:48:50.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VeL8CW5YGmhps6HcFD0lYf5iTIU='} 2024-04-16T17:48:50.605 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000004cd8ee2138939e7-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.606 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.606 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.606 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.606 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.606 DEBUG:boto:Data: 2024-04-16T17:48:50.606 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.606 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.606 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.606 DEBUG:boto:Params: {} 2024-04-16T17:48:50.606 DEBUG:boto:Token: None 2024-04-16T17:48:50.607 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.607 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.610 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000519cea1fe32b678f-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.610 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:48:50.611 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:48:50.611 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.611 DEBUG:boto:Path: /lybtug-8/.. 2024-04-16T17:48:50.611 DEBUG:boto:Data: 2024-04-16T17:48:50.611 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.611 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.611 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.611 DEBUG:boto:Params: {} 2024-04-16T17:48:50.611 DEBUG:boto:Token: None 2024-04-16T17:48:50.611 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/.. 2024-04-16T17:48:50.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kDey1LGyqKTF7v2bs4AdXzC/t3U= 2024-04-16T17:48:50.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kDey1LGyqKTF7v2bs4AdXzC/t3U='} 2024-04-16T17:48:50.654 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ba48a132837e3c5d-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.655 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.655 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.655 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.655 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.655 DEBUG:boto:Data: 2024-04-16T17:48:50.655 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.655 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.655 DEBUG:boto:Params: {} 2024-04-16T17:48:50.655 DEBUG:boto:Token: None 2024-04-16T17:48:50.655 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.661 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000074f81c00c60e5b83-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.662 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:48:50.662 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:48:50.662 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.662 DEBUG:boto:Path: /lybtug-8/... 2024-04-16T17:48:50.662 DEBUG:boto:Data: 2024-04-16T17:48:50.662 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.662 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.662 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.662 DEBUG:boto:Params: {} 2024-04-16T17:48:50.663 DEBUG:boto:Token: None 2024-04-16T17:48:50.663 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/... 2024-04-16T17:48:50.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XQCm1CgFJAKOj7aU7rm2lVMYVHE= 2024-04-16T17:48:50.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XQCm1CgFJAKOj7aU7rm2lVMYVHE='} 2024-04-16T17:48:50.673 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000063374dd3fad1a94b-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.673 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.673 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.673 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.673 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.673 DEBUG:boto:Data: 2024-04-16T17:48:50.673 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.673 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.673 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.674 DEBUG:boto:Params: {} 2024-04-16T17:48:50.674 DEBUG:boto:Token: None 2024-04-16T17:48:50.674 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.674 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.678 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002da960545f41c80f-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.678 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:48:50.678 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:48:50.678 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.678 DEBUG:boto:Path: /lybtug-8/.o 2024-04-16T17:48:50.679 DEBUG:boto:Data: 2024-04-16T17:48:50.679 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.679 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.679 DEBUG:boto:Params: {} 2024-04-16T17:48:50.679 DEBUG:boto:Token: None 2024-04-16T17:48:50.679 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/.o 2024-04-16T17:48:50.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OtC8njVmvY3I8FKpuHOpVonRXxM= 2024-04-16T17:48:50.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OtC8njVmvY3I8FKpuHOpVonRXxM='} 2024-04-16T17:48:50.722 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000039db9e759b92eec1-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.723 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:48:50.723 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:48:50.723 DEBUG:boto:Method: HEAD 2024-04-16T17:48:50.723 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:48:50.723 DEBUG:boto:Data: 2024-04-16T17:48:50.723 DEBUG:boto:Headers: {} 2024-04-16T17:48:50.723 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.723 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.723 DEBUG:boto:Params: {} 2024-04-16T17:48:50.723 DEBUG:boto:Token: None 2024-04-16T17:48:50.723 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/ 2024-04-16T17:48:50.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA= 2024-04-16T17:48:50.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qoeLos0NpQuhehB6/dcCXPnxLA=', 'Content-Length': '0'} 2024-04-16T17:48:50.727 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002151e6600a905d39-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.727 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:48:50.727 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:48:50.727 DEBUG:boto:Method: PUT 2024-04-16T17:48:50.728 DEBUG:boto:Path: /lybtug-8/.o. 2024-04-16T17:48:50.728 DEBUG:boto:Data: 2024-04-16T17:48:50.728 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:48:50.728 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:48:50.728 DEBUG:boto:Port: 8000 2024-04-16T17:48:50.728 DEBUG:boto:Params: {} 2024-04-16T17:48:50.728 DEBUG:boto:Token: None 2024-04-16T17:48:50.728 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:48:50 GMT /lybtug-8/.o. 2024-04-16T17:48:50.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUWZ4MS1gIK9KiboiTsLU35DGhU= 2024-04-16T17:48:50.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:48:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUWZ4MS1gIK9KiboiTsLU35DGhU='} 2024-04-16T17:48:50.742 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004c62b45396d02f95-00661eba02-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:48:50.743 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:50.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:50.743 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:50.785 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:50.785 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:50.856 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:48:50.888 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289706.527539_11.1","last_update":"2024-04-16T17:48:26.527539Z"},{"marker":"1_1713289730.100600_92.1","last_update":"2024-04-16T17:48:50.100600Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:48:50.888 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289706.527539_11.1', 1: '1_1713289730.100600_92.1', 2: '', 3: ''} 2024-04-16T17:48:50.888 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:48:50.888 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:50.888 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:48:50.888 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:48:50.931 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:50.931 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:50.996+0000 7fef008cf980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:50.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:51.000+0000 7fef008cf980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:48:51.029 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289730.100600_92.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:50.100600Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:51Z"} 2024-04-16T17:48:51.030 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289706.527539_11.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:48:26.527539Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289730.100600_92.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:48:50.100600Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:48:51Z"} 2024-04-16T17:48:51.030 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289706.527539_11.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:48:26.527539Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289730.100600_92.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:48:50.100600Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:48:51.030 DEBUG:rgw_multi.tests:log_status={0: '1_1713289706.527539_11.1', 1: '1_1713289730.100600_92.1', 2: '', 3: ''} 2024-04-16T17:48:51.030 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289706.527539_11.1', 1: '1_1713289730.100600_92.1', 2: '', 3: ''} 2024-04-16T17:48:51.030 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:48:51.030 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-7', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:48:51.030 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-7', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:48:51.031 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-7 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:48:51.113 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:48:51.113 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:48:51.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:51.188+0000 7fc84c1eb980 1 waiting to reach incremental sync.. 2024-04-16T17:48:56.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:48:56.192+0000 7fc84c1eb980 1 waiting to reach incremental sync.. 2024-04-16T17:49:01.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:01.192+0000 7fc84c1eb980 1 waiting to reach incremental sync.. 2024-04-16T17:49:06.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:06.196+0000 7fc84c1eb980 1 bucket sync caught up with source: 2024-04-16T17:49:06.196 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000002.11.6, , 00000000001.10.6, , 00000000001.9.6, 00000000002.7.6, 00000000001.7.6, 00000000001.12.6, 00000000001.2.6, ] 2024-04-16T17:49:06.196 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000002.11.6, , 00000000001.10.6, , 00000000001.9.6, 00000000002.7.6, 00000000001.7.6, 00000000001.12.6, 00000000001.2.6, ] 2024-04-16T17:49:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:06.196+0000 7fc84c1eb980 0 bucket checkpoint complete 2024-04-16T17:49:06.239 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-7 zones={test-zone2, test-zone1} 2024-04-16T17:49:06.239 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.239 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.239 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.240 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:49:06.240 DEBUG:boto:Data: 2024-04-16T17:49:06.240 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.240 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.240 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.240 DEBUG:boto:Params: {} 2024-04-16T17:49:06.240 DEBUG:boto:Token: None 2024-04-16T17:49:06.240 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xirxKeGKQ/LAnm1opN/6o8T2WAM= 2024-04-16T17:49:06.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xirxKeGKQ/LAnm1opN/6o8T2WAM=', 'Content-Length': '0'} 2024-04-16T17:49:06.245 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000101194cba4397774-00661eba12-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.245 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.245 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.245 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.245 DEBUG:boto:Path: /lybtug-7/ 2024-04-16T17:49:06.245 DEBUG:boto:Data: 2024-04-16T17:49:06.245 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.246 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.246 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.246 DEBUG:boto:Params: {} 2024-04-16T17:49:06.246 DEBUG:boto:Token: None 2024-04-16T17:49:06.246 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.246 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xirxKeGKQ/LAnm1opN/6o8T2WAM= 2024-04-16T17:49:06.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xirxKeGKQ/LAnm1opN/6o8T2WAM=', 'Content-Length': '0'} 2024-04-16T17:49:06.250 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000031221c05a8339fc8-00661eba12-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.250 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:49:06.250 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.250 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.251 DEBUG:boto:path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.251 DEBUG:boto:auth_path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.251 DEBUG:boto:Method: GET 2024-04-16T17:49:06.251 DEBUG:boto:Path: /lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.251 DEBUG:boto:Data: 2024-04-16T17:49:06.251 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.251 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.251 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.251 DEBUG:boto:Params: {} 2024-04-16T17:49:06.251 DEBUG:boto:Token: None 2024-04-16T17:49:06.251 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/?versions 2024-04-16T17:49:06.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano= 2024-04-16T17:49:06.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano=', 'Content-Length': '0'} 2024-04-16T17:49:06.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004732efcda570c987-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.256 DEBUG:boto:b'lybtug-7999false&nulltrue2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.258 DEBUG:tasks.rgw_multi.zone_rados:o=& 2024-04-16T17:49:06.258 DEBUG:tasks.rgw_multi.zone_rados:o=. 2024-04-16T17:49:06.258 DEBUG:tasks.rgw_multi.zone_rados:o=.. 2024-04-16T17:49:06.258 DEBUG:tasks.rgw_multi.zone_rados:o=... 2024-04-16T17:49:06.259 DEBUG:tasks.rgw_multi.zone_rados:o=.o 2024-04-16T17:49:06.259 DEBUG:tasks.rgw_multi.zone_rados:o=.o. 2024-04-16T17:49:06.259 DEBUG:tasks.rgw_multi.zone_rados:o=: 2024-04-16T17:49:06.259 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2024-04-16T17:49:06.259 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:06.259 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:49:06.259 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.259 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.259 DEBUG:boto:path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.259 DEBUG:boto:auth_path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.259 DEBUG:boto:Method: GET 2024-04-16T17:49:06.260 DEBUG:boto:Path: /lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.260 DEBUG:boto:Data: 2024-04-16T17:49:06.260 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.260 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.260 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.260 DEBUG:boto:Params: {} 2024-04-16T17:49:06.260 DEBUG:boto:Token: None 2024-04-16T17:49:06.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/?versions 2024-04-16T17:49:06.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano= 2024-04-16T17:49:06.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano=', 'Content-Length': '0'} 2024-04-16T17:49:06.264 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000977e9dda156295f3-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.265 DEBUG:boto:b'lybtug-7999false&nulltrue2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.265 DEBUG:tasks.rgw_multi.zone_rados:o=& 2024-04-16T17:49:06.265 DEBUG:tasks.rgw_multi.zone_rados:o=. 2024-04-16T17:49:06.265 DEBUG:tasks.rgw_multi.zone_rados:o=.. 2024-04-16T17:49:06.265 DEBUG:tasks.rgw_multi.zone_rados:o=... 2024-04-16T17:49:06.265 DEBUG:tasks.rgw_multi.zone_rados:o=.o 2024-04-16T17:49:06.266 DEBUG:tasks.rgw_multi.zone_rados:o=.o. 2024-04-16T17:49:06.266 DEBUG:tasks.rgw_multi.zone_rados:o=: 2024-04-16T17:49:06.266 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2024-04-16T17:49:06.266 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:06.266 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.266 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.266 DEBUG:boto:path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.266 DEBUG:boto:auth_path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.266 DEBUG:boto:Method: GET 2024-04-16T17:49:06.266 DEBUG:boto:Path: /lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.266 DEBUG:boto:Data: 2024-04-16T17:49:06.266 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.267 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.267 DEBUG:boto:Params: {} 2024-04-16T17:49:06.267 DEBUG:boto:Token: None 2024-04-16T17:49:06.267 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/?versions 2024-04-16T17:49:06.267 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano= 2024-04-16T17:49:06.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano=', 'Content-Length': '0'} 2024-04-16T17:49:06.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091b54c8664935361-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.271 DEBUG:boto:b'lybtug-7999false&nulltrue2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.272 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.272 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.272 DEBUG:boto:path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.272 DEBUG:boto:auth_path=/lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.272 DEBUG:boto:Method: GET 2024-04-16T17:49:06.273 DEBUG:boto:Path: /lybtug-7/?versions&max-keys=999 2024-04-16T17:49:06.273 DEBUG:boto:Data: 2024-04-16T17:49:06.273 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.273 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.273 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.273 DEBUG:boto:Params: {} 2024-04-16T17:49:06.273 DEBUG:boto:Token: None 2024-04-16T17:49:06.273 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/?versions 2024-04-16T17:49:06.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano= 2024-04-16T17:49:06.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q906iOnpIPVZv4okiRl3iql9ano=', 'Content-Length': '0'} 2024-04-16T17:49:06.277 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014a3f3fddd3fd83e-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.278 DEBUG:boto:b'lybtug-7999false&nulltrue2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.278 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2024-04-16T17:49:06.278 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.279 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.279 DEBUG:boto:path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.279 DEBUG:boto:auth_path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.279 DEBUG:boto:Method: GET 2024-04-16T17:49:06.279 DEBUG:boto:Path: /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.279 DEBUG:boto:Data: 2024-04-16T17:49:06.279 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.279 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.279 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.279 DEBUG:boto:Params: {} 2024-04-16T17:49:06.279 DEBUG:boto:Token: None 2024-04-16T17:49:06.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA= 2024-04-16T17:49:06.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA=', 'Content-Length': '0'} 2024-04-16T17:49:06.283 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002556cc60b5ad1efd-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.326 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.326 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.327 DEBUG:boto:path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.327 DEBUG:boto:auth_path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.327 DEBUG:boto:Method: GET 2024-04-16T17:49:06.327 DEBUG:boto:Path: /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.327 DEBUG:boto:Data: 2024-04-16T17:49:06.327 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.327 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.327 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.327 DEBUG:boto:Params: {} 2024-04-16T17:49:06.327 DEBUG:boto:Token: None 2024-04-16T17:49:06.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA= 2024-04-16T17:49:06.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA=', 'Content-Length': '0'} 2024-04-16T17:49:06.330 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000fdbc9608594d8e5c-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.331 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.331 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.331 DEBUG:boto:path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.331 DEBUG:boto:auth_path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.331 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.331 DEBUG:boto:Path: /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.331 DEBUG:boto:Data: 2024-04-16T17:49:06.331 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.331 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.331 DEBUG:boto:Params: {} 2024-04-16T17:49:06.332 DEBUG:boto:Token: None 2024-04-16T17:49:06.332 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxoNu66bbLZQJOV67gbF+7AsTIs= 2024-04-16T17:49:06.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxoNu66bbLZQJOV67gbF+7AsTIs=', 'Content-Length': '0'} 2024-04-16T17:49:06.337 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b34d3f715bce306a-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.337 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.338 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.338 DEBUG:boto:path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.338 DEBUG:boto:auth_path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.338 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.338 DEBUG:boto:Path: /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.338 DEBUG:boto:Data: 2024-04-16T17:49:06.338 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.338 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.338 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.338 DEBUG:boto:Params: {} 2024-04-16T17:49:06.339 DEBUG:boto:Token: None 2024-04-16T17:49:06.339 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.339 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxoNu66bbLZQJOV67gbF+7AsTIs= 2024-04-16T17:49:06.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxoNu66bbLZQJOV67gbF+7AsTIs=', 'Content-Length': '0'} 2024-04-16T17:49:06.342 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b5fd75614e579a8c-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.342 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2024-04-16T17:49:06.342 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.342 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.342 DEBUG:boto:path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.343 DEBUG:boto:auth_path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.343 DEBUG:boto:Method: GET 2024-04-16T17:49:06.343 DEBUG:boto:Path: /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.343 DEBUG:boto:Data: 2024-04-16T17:49:06.343 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.343 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.343 DEBUG:boto:Params: {} 2024-04-16T17:49:06.343 DEBUG:boto:Token: None 2024-04-16T17:49:06.343 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA= 2024-04-16T17:49:06.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA=', 'Content-Length': '0'} 2024-04-16T17:49:06.347 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b51de4cda6bc12de-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.347 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.347 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.347 DEBUG:boto:path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.347 DEBUG:boto:auth_path=/lybtug-7/%26?versionId=null 2024-04-16T17:49:06.347 DEBUG:boto:Method: GET 2024-04-16T17:49:06.347 DEBUG:boto:Path: /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.348 DEBUG:boto:Data: 2024-04-16T17:49:06.348 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.348 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.348 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.348 DEBUG:boto:Params: {} 2024-04-16T17:49:06.348 DEBUG:boto:Token: None 2024-04-16T17:49:06.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26?versionId=null 2024-04-16T17:49:06.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA= 2024-04-16T17:49:06.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tlUmVV8x6BvovA6JnO3swYICAvA=', 'Content-Length': '0'} 2024-04-16T17:49:06.351 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000045ada19dfde5dfbd-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.352 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.352 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.352 DEBUG:boto:path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.352 DEBUG:boto:auth_path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.352 DEBUG:boto:Method: GET 2024-04-16T17:49:06.352 DEBUG:boto:Path: /lybtug-7/?prefix=%26 2024-04-16T17:49:06.352 DEBUG:boto:Data: 2024-04-16T17:49:06.352 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.352 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.353 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.353 DEBUG:boto:Params: {} 2024-04-16T17:49:06.353 DEBUG:boto:Token: None 2024-04-16T17:49:06.353 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0ed694721838cc9-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.398 DEBUG:boto:b'lybtug-7&1000false&2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.399 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.399 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.399 DEBUG:boto:path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.399 DEBUG:boto:auth_path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.399 DEBUG:boto:Method: GET 2024-04-16T17:49:06.399 DEBUG:boto:Path: /lybtug-7/?prefix=%26 2024-04-16T17:49:06.400 DEBUG:boto:Data: 2024-04-16T17:49:06.400 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.400 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.400 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.400 DEBUG:boto:Params: {} 2024-04-16T17:49:06.400 DEBUG:boto:Token: None 2024-04-16T17:49:06.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3a2de6344e4ff4e-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.404 DEBUG:boto:b'lybtug-7&1000false&2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.405 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.405 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.405 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.405 DEBUG:boto:Path: /lybtug-7/%26 2024-04-16T17:49:06.405 DEBUG:boto:Data: 2024-04-16T17:49:06.405 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.405 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.405 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.405 DEBUG:boto:Params: {} 2024-04-16T17:49:06.405 DEBUG:boto:Token: None 2024-04-16T17:49:06.406 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26 2024-04-16T17:49:06.406 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qESs5UiEkqhKFHXPdQvJV652bKY= 2024-04-16T17:49:06.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qESs5UiEkqhKFHXPdQvJV652bKY=', 'Content-Length': '0'} 2024-04-16T17:49:06.409 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a961fefd1bd1e710-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.409 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.409 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.409 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.409 DEBUG:boto:Path: /lybtug-7/%26 2024-04-16T17:49:06.409 DEBUG:boto:Data: 2024-04-16T17:49:06.409 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.409 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.409 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.409 DEBUG:boto:Params: {} 2024-04-16T17:49:06.410 DEBUG:boto:Token: None 2024-04-16T17:49:06.410 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26 2024-04-16T17:49:06.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qESs5UiEkqhKFHXPdQvJV652bKY= 2024-04-16T17:49:06.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qESs5UiEkqhKFHXPdQvJV652bKY=', 'Content-Length': '0'} 2024-04-16T17:49:06.413 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009c90e649e7bcdb87-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.413 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2024-04-16T17:49:06.413 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.413 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.414 DEBUG:boto:Method: GET 2024-04-16T17:49:06.414 DEBUG:boto:Path: /lybtug-7/%26 2024-04-16T17:49:06.414 DEBUG:boto:Data: 2024-04-16T17:49:06.414 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.414 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.414 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.414 DEBUG:boto:Params: {} 2024-04-16T17:49:06.414 DEBUG:boto:Token: None 2024-04-16T17:49:06.414 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26 2024-04-16T17:49:06.414 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lybNTaTqNlqmd+QBpzD63ZrOfu8= 2024-04-16T17:49:06.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lybNTaTqNlqmd+QBpzD63ZrOfu8=', 'Content-Length': '0'} 2024-04-16T17:49:06.417 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a778ddd54276d78a-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.458 DEBUG:boto:path=/lybtug-7/%26 2024-04-16T17:49:06.458 DEBUG:boto:auth_path=/lybtug-7/%26 2024-04-16T17:49:06.459 DEBUG:boto:Method: GET 2024-04-16T17:49:06.459 DEBUG:boto:Path: /lybtug-7/%26 2024-04-16T17:49:06.459 DEBUG:boto:Data: 2024-04-16T17:49:06.459 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.459 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.459 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.459 DEBUG:boto:Params: {} 2024-04-16T17:49:06.459 DEBUG:boto:Token: None 2024-04-16T17:49:06.459 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/%26 2024-04-16T17:49:06.459 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lybNTaTqNlqmd+QBpzD63ZrOfu8= 2024-04-16T17:49:06.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lybNTaTqNlqmd+QBpzD63ZrOfu8=', 'Content-Length': '0'} 2024-04-16T17:49:06.462 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002f53d29b9301274a-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.463 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.463 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.463 DEBUG:boto:path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.463 DEBUG:boto:auth_path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.463 DEBUG:boto:Method: GET 2024-04-16T17:49:06.463 DEBUG:boto:Path: /lybtug-7/?prefix=%26 2024-04-16T17:49:06.463 DEBUG:boto:Data: 2024-04-16T17:49:06.463 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.463 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.463 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.463 DEBUG:boto:Params: {} 2024-04-16T17:49:06.464 DEBUG:boto:Token: None 2024-04-16T17:49:06.464 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086c90a0f5644eb21-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.468 DEBUG:boto:b'lybtug-7&1000false&2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.468 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.469 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.469 DEBUG:boto:path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.469 DEBUG:boto:auth_path=/lybtug-7/?prefix=%26 2024-04-16T17:49:06.469 DEBUG:boto:Method: GET 2024-04-16T17:49:06.469 DEBUG:boto:Path: /lybtug-7/?prefix=%26 2024-04-16T17:49:06.469 DEBUG:boto:Data: 2024-04-16T17:49:06.469 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.469 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.469 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.469 DEBUG:boto:Params: {} 2024-04-16T17:49:06.469 DEBUG:boto:Token: None 2024-04-16T17:49:06.470 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.470 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.473 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb397dbf89fa2516-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.473 DEBUG:boto:b'lybtug-7&1000false&2024-04-16T17:48:50.235Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.474 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=. 2024-04-16T17:49:06.474 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.474 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.474 DEBUG:boto:path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.474 DEBUG:boto:auth_path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.474 DEBUG:boto:Method: GET 2024-04-16T17:49:06.474 DEBUG:boto:Path: /lybtug-7/.?versionId=null 2024-04-16T17:49:06.474 DEBUG:boto:Data: 2024-04-16T17:49:06.474 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.475 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.475 DEBUG:boto:Params: {} 2024-04-16T17:49:06.475 DEBUG:boto:Token: None 2024-04-16T17:49:06.475 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.?versionId=null 2024-04-16T17:49:06.475 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM= 2024-04-16T17:49:06.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM=', 'Content-Length': '0'} 2024-04-16T17:49:06.478 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000076a36fd16d7848fe-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.518 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.518 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.519 DEBUG:boto:path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.519 DEBUG:boto:auth_path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.519 DEBUG:boto:Method: GET 2024-04-16T17:49:06.519 DEBUG:boto:Path: /lybtug-7/.?versionId=null 2024-04-16T17:49:06.519 DEBUG:boto:Data: 2024-04-16T17:49:06.519 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.519 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.519 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.519 DEBUG:boto:Params: {} 2024-04-16T17:49:06.519 DEBUG:boto:Token: None 2024-04-16T17:49:06.519 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.?versionId=null 2024-04-16T17:49:06.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM= 2024-04-16T17:49:06.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM=', 'Content-Length': '0'} 2024-04-16T17:49:06.523 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000aa6d5411b5a1fd8f-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.523 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.523 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.523 DEBUG:boto:path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.523 DEBUG:boto:auth_path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.523 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.523 DEBUG:boto:Path: /lybtug-7/.?versionId=null 2024-04-16T17:49:06.523 DEBUG:boto:Data: 2024-04-16T17:49:06.523 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.523 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.524 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.524 DEBUG:boto:Params: {} 2024-04-16T17:49:06.524 DEBUG:boto:Token: None 2024-04-16T17:49:06.524 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.?versionId=null 2024-04-16T17:49:06.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AEfyJSq5JdGSMw9TQiOp/5yE8q4= 2024-04-16T17:49:06.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AEfyJSq5JdGSMw9TQiOp/5yE8q4=', 'Content-Length': '0'} 2024-04-16T17:49:06.527 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e2d717d8a36468e5-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.527 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.527 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.527 DEBUG:boto:path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.527 DEBUG:boto:auth_path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.528 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.528 DEBUG:boto:Path: /lybtug-7/.?versionId=null 2024-04-16T17:49:06.528 DEBUG:boto:Data: 2024-04-16T17:49:06.528 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.528 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.528 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.528 DEBUG:boto:Params: {} 2024-04-16T17:49:06.528 DEBUG:boto:Token: None 2024-04-16T17:49:06.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.?versionId=null 2024-04-16T17:49:06.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AEfyJSq5JdGSMw9TQiOp/5yE8q4= 2024-04-16T17:49:06.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AEfyJSq5JdGSMw9TQiOp/5yE8q4=', 'Content-Length': '0'} 2024-04-16T17:49:06.531 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000029558847b776a8dc-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.532 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=. 2024-04-16T17:49:06.532 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.532 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.532 DEBUG:boto:path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.532 DEBUG:boto:auth_path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.532 DEBUG:boto:Method: GET 2024-04-16T17:49:06.532 DEBUG:boto:Path: /lybtug-7/.?versionId=null 2024-04-16T17:49:06.532 DEBUG:boto:Data: 2024-04-16T17:49:06.532 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.532 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.532 DEBUG:boto:Params: {} 2024-04-16T17:49:06.533 DEBUG:boto:Token: None 2024-04-16T17:49:06.533 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.?versionId=null 2024-04-16T17:49:06.533 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM= 2024-04-16T17:49:06.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM=', 'Content-Length': '0'} 2024-04-16T17:49:06.536 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002d3d834d0df90cf5-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.536 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.536 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.536 DEBUG:boto:path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.536 DEBUG:boto:auth_path=/lybtug-7/.?versionId=null 2024-04-16T17:49:06.536 DEBUG:boto:Method: GET 2024-04-16T17:49:06.536 DEBUG:boto:Path: /lybtug-7/.?versionId=null 2024-04-16T17:49:06.536 DEBUG:boto:Data: 2024-04-16T17:49:06.536 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.537 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.537 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.537 DEBUG:boto:Params: {} 2024-04-16T17:49:06.537 DEBUG:boto:Token: None 2024-04-16T17:49:06.537 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.?versionId=null 2024-04-16T17:49:06.537 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM= 2024-04-16T17:49:06.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ScMYlwF5OIdyOPiR/QGR1ZS+EdM=', 'Content-Length': '0'} 2024-04-16T17:49:06.540 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000266398540fbdaef4-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.582 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.583 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.583 DEBUG:boto:path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.583 DEBUG:boto:auth_path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.583 DEBUG:boto:Method: GET 2024-04-16T17:49:06.583 DEBUG:boto:Path: /lybtug-7/?prefix=. 2024-04-16T17:49:06.583 DEBUG:boto:Data: 2024-04-16T17:49:06.583 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.583 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.583 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.583 DEBUG:boto:Params: {} 2024-04-16T17:49:06.583 DEBUG:boto:Token: None 2024-04-16T17:49:06.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.589 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1e7ab7959b92f3c-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.589 DEBUG:boto:b'lybtug-7.1000false.2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.589 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.589 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.589 DEBUG:boto:path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.590 DEBUG:boto:auth_path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.590 DEBUG:boto:Method: GET 2024-04-16T17:49:06.590 DEBUG:boto:Path: /lybtug-7/?prefix=. 2024-04-16T17:49:06.590 DEBUG:boto:Data: 2024-04-16T17:49:06.590 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.590 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.590 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.590 DEBUG:boto:Params: {} 2024-04-16T17:49:06.590 DEBUG:boto:Token: None 2024-04-16T17:49:06.590 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.590 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.595 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000614dc3276b590a52-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.595 DEBUG:boto:b'lybtug-7.1000false.2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.595 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.595 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.595 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.595 DEBUG:boto:Path: /lybtug-7/. 2024-04-16T17:49:06.596 DEBUG:boto:Data: 2024-04-16T17:49:06.596 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.596 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.596 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.596 DEBUG:boto:Params: {} 2024-04-16T17:49:06.596 DEBUG:boto:Token: None 2024-04-16T17:49:06.596 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/. 2024-04-16T17:49:06.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SrSjvnv8jcZa1NNUrxAw/C050XM= 2024-04-16T17:49:06.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SrSjvnv8jcZa1NNUrxAw/C050XM=', 'Content-Length': '0'} 2024-04-16T17:49:06.599 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006973666f08c81d7e-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.599 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.599 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.599 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.600 DEBUG:boto:Path: /lybtug-7/. 2024-04-16T17:49:06.600 DEBUG:boto:Data: 2024-04-16T17:49:06.600 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.600 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.600 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.600 DEBUG:boto:Params: {} 2024-04-16T17:49:06.600 DEBUG:boto:Token: None 2024-04-16T17:49:06.600 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/. 2024-04-16T17:49:06.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SrSjvnv8jcZa1NNUrxAw/C050XM= 2024-04-16T17:49:06.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SrSjvnv8jcZa1NNUrxAw/C050XM=', 'Content-Length': '0'} 2024-04-16T17:49:06.603 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c73935295cc0b5b0-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=. 2024-04-16T17:49:06.604 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.604 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.604 DEBUG:boto:Method: GET 2024-04-16T17:49:06.604 DEBUG:boto:Path: /lybtug-7/. 2024-04-16T17:49:06.604 DEBUG:boto:Data: 2024-04-16T17:49:06.604 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.604 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.604 DEBUG:boto:Params: {} 2024-04-16T17:49:06.605 DEBUG:boto:Token: None 2024-04-16T17:49:06.605 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/. 2024-04-16T17:49:06.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dcxxBFCS72VUOe9QyUPHR2yM45s= 2024-04-16T17:49:06.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dcxxBFCS72VUOe9QyUPHR2yM45s=', 'Content-Length': '0'} 2024-04-16T17:49:06.608 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cfc5476d829713c2-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.650 DEBUG:boto:path=/lybtug-7/. 2024-04-16T17:49:06.650 DEBUG:boto:auth_path=/lybtug-7/. 2024-04-16T17:49:06.651 DEBUG:boto:Method: GET 2024-04-16T17:49:06.651 DEBUG:boto:Path: /lybtug-7/. 2024-04-16T17:49:06.651 DEBUG:boto:Data: 2024-04-16T17:49:06.651 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.651 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.651 DEBUG:boto:Params: {} 2024-04-16T17:49:06.651 DEBUG:boto:Token: None 2024-04-16T17:49:06.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/. 2024-04-16T17:49:06.651 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dcxxBFCS72VUOe9QyUPHR2yM45s= 2024-04-16T17:49:06.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dcxxBFCS72VUOe9QyUPHR2yM45s=', 'Content-Length': '0'} 2024-04-16T17:49:06.654 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007536ec84fcea2deb-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.655 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.655 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.655 DEBUG:boto:path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.655 DEBUG:boto:auth_path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.655 DEBUG:boto:Method: GET 2024-04-16T17:49:06.655 DEBUG:boto:Path: /lybtug-7/?prefix=. 2024-04-16T17:49:06.655 DEBUG:boto:Data: 2024-04-16T17:49:06.655 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.655 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.655 DEBUG:boto:Params: {} 2024-04-16T17:49:06.655 DEBUG:boto:Token: None 2024-04-16T17:49:06.656 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.660 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ce5ffafec5347c1-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.660 DEBUG:boto:b'lybtug-7.1000false.2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.660 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.660 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.660 DEBUG:boto:path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.661 DEBUG:boto:auth_path=/lybtug-7/?prefix=. 2024-04-16T17:49:06.661 DEBUG:boto:Method: GET 2024-04-16T17:49:06.661 DEBUG:boto:Path: /lybtug-7/?prefix=. 2024-04-16T17:49:06.661 DEBUG:boto:Data: 2024-04-16T17:49:06.661 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.661 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.661 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.661 DEBUG:boto:Params: {} 2024-04-16T17:49:06.661 DEBUG:boto:Token: None 2024-04-16T17:49:06.661 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.661 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001184aa87739cf549-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.666 DEBUG:boto:b'lybtug-7.1000false.2024-04-16T17:48:50.252Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.666 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.. 2024-04-16T17:49:06.667 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.667 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.667 DEBUG:boto:path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.667 DEBUG:boto:auth_path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.667 DEBUG:boto:Method: GET 2024-04-16T17:49:06.667 DEBUG:boto:Path: /lybtug-7/..?versionId=null 2024-04-16T17:49:06.667 DEBUG:boto:Data: 2024-04-16T17:49:06.667 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.667 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.667 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.667 DEBUG:boto:Params: {} 2024-04-16T17:49:06.667 DEBUG:boto:Token: None 2024-04-16T17:49:06.668 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/..?versionId=null 2024-04-16T17:49:06.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4= 2024-04-16T17:49:06.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4=', 'Content-Length': '0'} 2024-04-16T17:49:06.671 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000026400c2ed4bffb78-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.671 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.671 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.671 DEBUG:boto:path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.672 DEBUG:boto:auth_path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.672 DEBUG:boto:Method: GET 2024-04-16T17:49:06.672 DEBUG:boto:Path: /lybtug-7/..?versionId=null 2024-04-16T17:49:06.672 DEBUG:boto:Data: 2024-04-16T17:49:06.672 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.672 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.672 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.672 DEBUG:boto:Params: {} 2024-04-16T17:49:06.672 DEBUG:boto:Token: None 2024-04-16T17:49:06.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/..?versionId=null 2024-04-16T17:49:06.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4= 2024-04-16T17:49:06.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4=', 'Content-Length': '0'} 2024-04-16T17:49:06.675 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000afd0f92d41ed112f-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.718 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.719 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.719 DEBUG:boto:path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.719 DEBUG:boto:auth_path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.719 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.719 DEBUG:boto:Path: /lybtug-7/..?versionId=null 2024-04-16T17:49:06.719 DEBUG:boto:Data: 2024-04-16T17:49:06.719 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.720 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.720 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.720 DEBUG:boto:Params: {} 2024-04-16T17:49:06.720 DEBUG:boto:Token: None 2024-04-16T17:49:06.720 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/..?versionId=null 2024-04-16T17:49:06.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ten0ylzMkIM6n+JGO0Yd8ZNLfYE= 2024-04-16T17:49:06.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ten0ylzMkIM6n+JGO0Yd8ZNLfYE=', 'Content-Length': '0'} 2024-04-16T17:49:06.723 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e73720498b78b10c-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.724 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.724 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.724 DEBUG:boto:path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.724 DEBUG:boto:auth_path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.724 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.724 DEBUG:boto:Path: /lybtug-7/..?versionId=null 2024-04-16T17:49:06.724 DEBUG:boto:Data: 2024-04-16T17:49:06.724 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.724 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.724 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.724 DEBUG:boto:Params: {} 2024-04-16T17:49:06.725 DEBUG:boto:Token: None 2024-04-16T17:49:06.725 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/..?versionId=null 2024-04-16T17:49:06.725 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ten0ylzMkIM6n+JGO0Yd8ZNLfYE= 2024-04-16T17:49:06.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ten0ylzMkIM6n+JGO0Yd8ZNLfYE=', 'Content-Length': '0'} 2024-04-16T17:49:06.728 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000650925d96304ab9a-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.728 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.. 2024-04-16T17:49:06.728 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.728 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.728 DEBUG:boto:path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.728 DEBUG:boto:auth_path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.728 DEBUG:boto:Method: GET 2024-04-16T17:49:06.729 DEBUG:boto:Path: /lybtug-7/..?versionId=null 2024-04-16T17:49:06.729 DEBUG:boto:Data: 2024-04-16T17:49:06.729 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.729 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.729 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.729 DEBUG:boto:Params: {} 2024-04-16T17:49:06.729 DEBUG:boto:Token: None 2024-04-16T17:49:06.729 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/..?versionId=null 2024-04-16T17:49:06.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4= 2024-04-16T17:49:06.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4=', 'Content-Length': '0'} 2024-04-16T17:49:06.732 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000041b1bb997060cf34-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.774 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.775 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.775 DEBUG:boto:path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.775 DEBUG:boto:auth_path=/lybtug-7/..?versionId=null 2024-04-16T17:49:06.775 DEBUG:boto:Method: GET 2024-04-16T17:49:06.775 DEBUG:boto:Path: /lybtug-7/..?versionId=null 2024-04-16T17:49:06.775 DEBUG:boto:Data: 2024-04-16T17:49:06.775 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.775 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.775 DEBUG:boto:Params: {} 2024-04-16T17:49:06.775 DEBUG:boto:Token: None 2024-04-16T17:49:06.775 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/..?versionId=null 2024-04-16T17:49:06.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4= 2024-04-16T17:49:06.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iBfmHGXPclXKJGA5OVyImUnAOS4=', 'Content-Length': '0'} 2024-04-16T17:49:06.778 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e11a78ed1d8bf48a-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.779 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.779 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.779 DEBUG:boto:path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.779 DEBUG:boto:auth_path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.779 DEBUG:boto:Method: GET 2024-04-16T17:49:06.779 DEBUG:boto:Path: /lybtug-7/?prefix=.. 2024-04-16T17:49:06.779 DEBUG:boto:Data: 2024-04-16T17:49:06.779 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.779 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.779 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.779 DEBUG:boto:Params: {} 2024-04-16T17:49:06.780 DEBUG:boto:Token: None 2024-04-16T17:49:06.780 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.783 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047009fefd8071927-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.784 DEBUG:boto:b'lybtug-7..1000false..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.784 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.784 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.784 DEBUG:boto:path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.784 DEBUG:boto:auth_path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.784 DEBUG:boto:Method: GET 2024-04-16T17:49:06.784 DEBUG:boto:Path: /lybtug-7/?prefix=.. 2024-04-16T17:49:06.785 DEBUG:boto:Data: 2024-04-16T17:49:06.785 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.785 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.785 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.785 DEBUG:boto:Params: {} 2024-04-16T17:49:06.785 DEBUG:boto:Token: None 2024-04-16T17:49:06.785 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.785 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.789 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bcf31f330740ecbe-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.789 DEBUG:boto:b'lybtug-7..1000false..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.789 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.790 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.790 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.790 DEBUG:boto:Path: /lybtug-7/.. 2024-04-16T17:49:06.790 DEBUG:boto:Data: 2024-04-16T17:49:06.790 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.790 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.790 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.790 DEBUG:boto:Params: {} 2024-04-16T17:49:06.790 DEBUG:boto:Token: None 2024-04-16T17:49:06.790 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.. 2024-04-16T17:49:06.791 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z5bTFiYgMmKTnvZVfNM1zfEONeU= 2024-04-16T17:49:06.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z5bTFiYgMmKTnvZVfNM1zfEONeU=', 'Content-Length': '0'} 2024-04-16T17:49:06.793 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005cc77a8fe5628363-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.793 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.793 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.794 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.794 DEBUG:boto:Path: /lybtug-7/.. 2024-04-16T17:49:06.794 DEBUG:boto:Data: 2024-04-16T17:49:06.794 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.794 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.794 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.794 DEBUG:boto:Params: {} 2024-04-16T17:49:06.794 DEBUG:boto:Token: None 2024-04-16T17:49:06.794 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.. 2024-04-16T17:49:06.794 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z5bTFiYgMmKTnvZVfNM1zfEONeU= 2024-04-16T17:49:06.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z5bTFiYgMmKTnvZVfNM1zfEONeU=', 'Content-Length': '0'} 2024-04-16T17:49:06.797 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000acf26aba8ab307ff-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.797 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.. 2024-04-16T17:49:06.798 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.798 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.798 DEBUG:boto:Method: GET 2024-04-16T17:49:06.798 DEBUG:boto:Path: /lybtug-7/.. 2024-04-16T17:49:06.798 DEBUG:boto:Data: 2024-04-16T17:49:06.798 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.798 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.798 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.798 DEBUG:boto:Params: {} 2024-04-16T17:49:06.798 DEBUG:boto:Token: None 2024-04-16T17:49:06.799 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.. 2024-04-16T17:49:06.799 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ObCbTP3QISQenx7DmI9Zr0R/ro= 2024-04-16T17:49:06.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ObCbTP3QISQenx7DmI9Zr0R/ro=', 'Content-Length': '0'} 2024-04-16T17:49:06.802 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f946e052d34b22d2-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.802 DEBUG:boto:path=/lybtug-7/.. 2024-04-16T17:49:06.802 DEBUG:boto:auth_path=/lybtug-7/.. 2024-04-16T17:49:06.802 DEBUG:boto:Method: GET 2024-04-16T17:49:06.802 DEBUG:boto:Path: /lybtug-7/.. 2024-04-16T17:49:06.802 DEBUG:boto:Data: 2024-04-16T17:49:06.802 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.802 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.803 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.803 DEBUG:boto:Params: {} 2024-04-16T17:49:06.803 DEBUG:boto:Token: None 2024-04-16T17:49:06.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/.. 2024-04-16T17:49:06.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ObCbTP3QISQenx7DmI9Zr0R/ro= 2024-04-16T17:49:06.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ObCbTP3QISQenx7DmI9Zr0R/ro=', 'Content-Length': '0'} 2024-04-16T17:49:06.806 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000601a6252c7f24e2a-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.846 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.846 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.846 DEBUG:boto:path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.847 DEBUG:boto:auth_path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.847 DEBUG:boto:Method: GET 2024-04-16T17:49:06.847 DEBUG:boto:Path: /lybtug-7/?prefix=.. 2024-04-16T17:49:06.847 DEBUG:boto:Data: 2024-04-16T17:49:06.847 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.847 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.847 DEBUG:boto:Params: {} 2024-04-16T17:49:06.847 DEBUG:boto:Token: None 2024-04-16T17:49:06.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f2a01e9e11f9cd8-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.852 DEBUG:boto:b'lybtug-7..1000false..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.852 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.852 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.852 DEBUG:boto:path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.853 DEBUG:boto:auth_path=/lybtug-7/?prefix=.. 2024-04-16T17:49:06.853 DEBUG:boto:Method: GET 2024-04-16T17:49:06.853 DEBUG:boto:Path: /lybtug-7/?prefix=.. 2024-04-16T17:49:06.853 DEBUG:boto:Data: 2024-04-16T17:49:06.853 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.853 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.853 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.853 DEBUG:boto:Params: {} 2024-04-16T17:49:06.853 DEBUG:boto:Token: None 2024-04-16T17:49:06.853 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.857 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a09b23f3d040b0d0-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.858 DEBUG:boto:b'lybtug-7..1000false..2024-04-16T17:48:50.301Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.858 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=... 2024-04-16T17:49:06.858 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.858 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.858 DEBUG:boto:path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.858 DEBUG:boto:auth_path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.859 DEBUG:boto:Method: GET 2024-04-16T17:49:06.859 DEBUG:boto:Path: /lybtug-7/...?versionId=null 2024-04-16T17:49:06.859 DEBUG:boto:Data: 2024-04-16T17:49:06.859 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.859 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.859 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.859 DEBUG:boto:Params: {} 2024-04-16T17:49:06.859 DEBUG:boto:Token: None 2024-04-16T17:49:06.859 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/...?versionId=null 2024-04-16T17:49:06.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY= 2024-04-16T17:49:06.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY=', 'Content-Length': '0'} 2024-04-16T17:49:06.862 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e46ee40ac6c306fb-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.906 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.907 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.907 DEBUG:boto:path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.907 DEBUG:boto:auth_path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.907 DEBUG:boto:Method: GET 2024-04-16T17:49:06.907 DEBUG:boto:Path: /lybtug-7/...?versionId=null 2024-04-16T17:49:06.907 DEBUG:boto:Data: 2024-04-16T17:49:06.907 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.907 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.907 DEBUG:boto:Params: {} 2024-04-16T17:49:06.908 DEBUG:boto:Token: None 2024-04-16T17:49:06.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/...?versionId=null 2024-04-16T17:49:06.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY= 2024-04-16T17:49:06.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY=', 'Content-Length': '0'} 2024-04-16T17:49:06.911 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d174d54654ec2137-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.911 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.912 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.912 DEBUG:boto:path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.912 DEBUG:boto:auth_path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.912 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.912 DEBUG:boto:Path: /lybtug-7/...?versionId=null 2024-04-16T17:49:06.912 DEBUG:boto:Data: 2024-04-16T17:49:06.912 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.912 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.912 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.912 DEBUG:boto:Params: {} 2024-04-16T17:49:06.912 DEBUG:boto:Token: None 2024-04-16T17:49:06.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/...?versionId=null 2024-04-16T17:49:06.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZxR+yGQHnS8GnhTYAijOf7LcAcM= 2024-04-16T17:49:06.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZxR+yGQHnS8GnhTYAijOf7LcAcM=', 'Content-Length': '0'} 2024-04-16T17:49:06.915 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d937711fa8ed828c-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.916 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.916 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.916 DEBUG:boto:path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.916 DEBUG:boto:auth_path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.916 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.916 DEBUG:boto:Path: /lybtug-7/...?versionId=null 2024-04-16T17:49:06.916 DEBUG:boto:Data: 2024-04-16T17:49:06.916 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.916 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.916 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.917 DEBUG:boto:Params: {} 2024-04-16T17:49:06.917 DEBUG:boto:Token: None 2024-04-16T17:49:06.917 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/...?versionId=null 2024-04-16T17:49:06.917 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZxR+yGQHnS8GnhTYAijOf7LcAcM= 2024-04-16T17:49:06.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZxR+yGQHnS8GnhTYAijOf7LcAcM=', 'Content-Length': '0'} 2024-04-16T17:49:06.920 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005dfb2b5631c93180-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=... 2024-04-16T17:49:06.920 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.921 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.921 DEBUG:boto:path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.921 DEBUG:boto:auth_path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.921 DEBUG:boto:Method: GET 2024-04-16T17:49:06.921 DEBUG:boto:Path: /lybtug-7/...?versionId=null 2024-04-16T17:49:06.921 DEBUG:boto:Data: 2024-04-16T17:49:06.921 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.921 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.921 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.921 DEBUG:boto:Params: {} 2024-04-16T17:49:06.921 DEBUG:boto:Token: None 2024-04-16T17:49:06.921 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/...?versionId=null 2024-04-16T17:49:06.922 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY= 2024-04-16T17:49:06.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY=', 'Content-Length': '0'} 2024-04-16T17:49:06.925 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bfc14834d5d4d033-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.925 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.925 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.925 DEBUG:boto:path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.925 DEBUG:boto:auth_path=/lybtug-7/...?versionId=null 2024-04-16T17:49:06.926 DEBUG:boto:Method: GET 2024-04-16T17:49:06.926 DEBUG:boto:Path: /lybtug-7/...?versionId=null 2024-04-16T17:49:06.926 DEBUG:boto:Data: 2024-04-16T17:49:06.926 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.926 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.926 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.926 DEBUG:boto:Params: {} 2024-04-16T17:49:06.926 DEBUG:boto:Token: None 2024-04-16T17:49:06.926 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/...?versionId=null 2024-04-16T17:49:06.926 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY= 2024-04-16T17:49:06.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlUPlE6zhlKh5ahv52r1Ixwy3AY=', 'Content-Length': '0'} 2024-04-16T17:49:06.930 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f3b3012f8b5d7256-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.970 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.970 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.971 DEBUG:boto:path=/lybtug-7/?prefix=... 2024-04-16T17:49:06.971 DEBUG:boto:auth_path=/lybtug-7/?prefix=... 2024-04-16T17:49:06.971 DEBUG:boto:Method: GET 2024-04-16T17:49:06.971 DEBUG:boto:Path: /lybtug-7/?prefix=... 2024-04-16T17:49:06.971 DEBUG:boto:Data: 2024-04-16T17:49:06.971 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.971 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.971 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.971 DEBUG:boto:Params: {} 2024-04-16T17:49:06.971 DEBUG:boto:Token: None 2024-04-16T17:49:06.971 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.975 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007fce8b00a2d77ca1-00661eba12-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.976 DEBUG:boto:b'lybtug-7...1000false...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.976 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:06.976 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:06.976 DEBUG:boto:path=/lybtug-7/?prefix=... 2024-04-16T17:49:06.976 DEBUG:boto:auth_path=/lybtug-7/?prefix=... 2024-04-16T17:49:06.976 DEBUG:boto:Method: GET 2024-04-16T17:49:06.976 DEBUG:boto:Path: /lybtug-7/?prefix=... 2024-04-16T17:49:06.976 DEBUG:boto:Data: 2024-04-16T17:49:06.976 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.977 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.977 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.977 DEBUG:boto:Params: {} 2024-04-16T17:49:06.977 DEBUG:boto:Token: None 2024-04-16T17:49:06.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/ 2024-04-16T17:49:06.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg= 2024-04-16T17:49:06.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/TsezIeyxDxdEjIUDuekYlkYfZg=', 'Content-Length': '0'} 2024-04-16T17:49:06.981 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b055264f8b3a838a-00661eba12-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.981 DEBUG:boto:b'lybtug-7...1000false...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:06.982 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.982 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.982 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.982 DEBUG:boto:Path: /lybtug-7/... 2024-04-16T17:49:06.982 DEBUG:boto:Data: 2024-04-16T17:49:06.982 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.982 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.982 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.982 DEBUG:boto:Params: {} 2024-04-16T17:49:06.982 DEBUG:boto:Token: None 2024-04-16T17:49:06.983 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/... 2024-04-16T17:49:06.983 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t8MxaLnoBxwLnFJAXMu5Wj44Vrw= 2024-04-16T17:49:06.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t8MxaLnoBxwLnFJAXMu5Wj44Vrw=', 'Content-Length': '0'} 2024-04-16T17:49:06.986 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000039ce7a76de5b09fd-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.986 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.986 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.986 DEBUG:boto:Method: HEAD 2024-04-16T17:49:06.986 DEBUG:boto:Path: /lybtug-7/... 2024-04-16T17:49:06.986 DEBUG:boto:Data: 2024-04-16T17:49:06.986 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.986 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:06.986 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.986 DEBUG:boto:Params: {} 2024-04-16T17:49:06.987 DEBUG:boto:Token: None 2024-04-16T17:49:06.987 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/... 2024-04-16T17:49:06.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t8MxaLnoBxwLnFJAXMu5Wj44Vrw= 2024-04-16T17:49:06.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t8MxaLnoBxwLnFJAXMu5Wj44Vrw=', 'Content-Length': '0'} 2024-04-16T17:49:06.990 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005957508b81760d0f-00661eba12-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:06.990 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=... 2024-04-16T17:49:06.990 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:06.990 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:06.990 DEBUG:boto:Method: GET 2024-04-16T17:49:06.990 DEBUG:boto:Path: /lybtug-7/... 2024-04-16T17:49:06.990 DEBUG:boto:Data: 2024-04-16T17:49:06.990 DEBUG:boto:Headers: {} 2024-04-16T17:49:06.991 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:06.991 DEBUG:boto:Port: 8000 2024-04-16T17:49:06.991 DEBUG:boto:Params: {} 2024-04-16T17:49:06.991 DEBUG:boto:Token: None 2024-04-16T17:49:06.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:06 GMT /lybtug-7/... 2024-04-16T17:49:06.991 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0o/1cBrqJk1dOvaAZb5tFE9A9zs= 2024-04-16T17:49:06.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:06 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0o/1cBrqJk1dOvaAZb5tFE9A9zs=', 'Content-Length': '0'} 2024-04-16T17:49:06.994 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000984547b077b87240-00661eba12-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:06 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.034 DEBUG:boto:path=/lybtug-7/... 2024-04-16T17:49:07.034 DEBUG:boto:auth_path=/lybtug-7/... 2024-04-16T17:49:07.035 DEBUG:boto:Method: GET 2024-04-16T17:49:07.035 DEBUG:boto:Path: /lybtug-7/... 2024-04-16T17:49:07.035 DEBUG:boto:Data: 2024-04-16T17:49:07.035 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.035 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.035 DEBUG:boto:Params: {} 2024-04-16T17:49:07.035 DEBUG:boto:Token: None 2024-04-16T17:49:07.035 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/... 2024-04-16T17:49:07.035 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qDNITh8jpuc7nVIAJw2OXRlwKJs= 2024-04-16T17:49:07.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qDNITh8jpuc7nVIAJw2OXRlwKJs=', 'Content-Length': '0'} 2024-04-16T17:49:07.038 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bbd47906055bdb96-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.039 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.039 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.039 DEBUG:boto:path=/lybtug-7/?prefix=... 2024-04-16T17:49:07.039 DEBUG:boto:auth_path=/lybtug-7/?prefix=... 2024-04-16T17:49:07.039 DEBUG:boto:Method: GET 2024-04-16T17:49:07.039 DEBUG:boto:Path: /lybtug-7/?prefix=... 2024-04-16T17:49:07.039 DEBUG:boto:Data: 2024-04-16T17:49:07.039 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.039 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.039 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.040 DEBUG:boto:Params: {} 2024-04-16T17:49:07.040 DEBUG:boto:Token: None 2024-04-16T17:49:07.040 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000fb0ad7371d48fa1-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.044 DEBUG:boto:b'lybtug-7...1000false...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.045 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.045 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.045 DEBUG:boto:path=/lybtug-7/?prefix=... 2024-04-16T17:49:07.045 DEBUG:boto:auth_path=/lybtug-7/?prefix=... 2024-04-16T17:49:07.045 DEBUG:boto:Method: GET 2024-04-16T17:49:07.045 DEBUG:boto:Path: /lybtug-7/?prefix=... 2024-04-16T17:49:07.045 DEBUG:boto:Data: 2024-04-16T17:49:07.045 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.045 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.046 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.046 DEBUG:boto:Params: {} 2024-04-16T17:49:07.046 DEBUG:boto:Token: None 2024-04-16T17:49:07.046 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.046 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d23d0081fcf9008-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.090 DEBUG:boto:b'lybtug-7...1000false...2024-04-16T17:48:50.317Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o 2024-04-16T17:49:07.091 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.091 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.091 DEBUG:boto:path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.091 DEBUG:boto:auth_path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.091 DEBUG:boto:Method: GET 2024-04-16T17:49:07.091 DEBUG:boto:Path: /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.091 DEBUG:boto:Data: 2024-04-16T17:49:07.091 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.091 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.091 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.091 DEBUG:boto:Params: {} 2024-04-16T17:49:07.092 DEBUG:boto:Token: None 2024-04-16T17:49:07.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM= 2024-04-16T17:49:07.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM=', 'Content-Length': '0'} 2024-04-16T17:49:07.095 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000be86aba6fcc6e3e4-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.095 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.095 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.095 DEBUG:boto:path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.096 DEBUG:boto:auth_path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.096 DEBUG:boto:Method: GET 2024-04-16T17:49:07.096 DEBUG:boto:Path: /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.096 DEBUG:boto:Data: 2024-04-16T17:49:07.096 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.096 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.096 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.096 DEBUG:boto:Params: {} 2024-04-16T17:49:07.096 DEBUG:boto:Token: None 2024-04-16T17:49:07.096 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM= 2024-04-16T17:49:07.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM=', 'Content-Length': '0'} 2024-04-16T17:49:07.100 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000033a10cfdd63473ad-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.100 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.101 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.101 DEBUG:boto:path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.101 DEBUG:boto:auth_path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.101 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.101 DEBUG:boto:Path: /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.101 DEBUG:boto:Data: 2024-04-16T17:49:07.101 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.101 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.101 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.101 DEBUG:boto:Params: {} 2024-04-16T17:49:07.101 DEBUG:boto:Token: None 2024-04-16T17:49:07.102 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.102 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AVSo4ulOSag/Ev6F3//B61scHHE= 2024-04-16T17:49:07.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AVSo4ulOSag/Ev6F3//B61scHHE=', 'Content-Length': '0'} 2024-04-16T17:49:07.104 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a766aedeef2d625c-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.105 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.105 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.105 DEBUG:boto:path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.105 DEBUG:boto:auth_path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.105 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.105 DEBUG:boto:Path: /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.105 DEBUG:boto:Data: 2024-04-16T17:49:07.105 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.106 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.106 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.106 DEBUG:boto:Params: {} 2024-04-16T17:49:07.106 DEBUG:boto:Token: None 2024-04-16T17:49:07.106 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.106 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AVSo4ulOSag/Ev6F3//B61scHHE= 2024-04-16T17:49:07.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AVSo4ulOSag/Ev6F3//B61scHHE=', 'Content-Length': '0'} 2024-04-16T17:49:07.109 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005dbf04b2d8a05b7f-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.109 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o 2024-04-16T17:49:07.109 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.109 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.109 DEBUG:boto:path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.109 DEBUG:boto:auth_path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.110 DEBUG:boto:Method: GET 2024-04-16T17:49:07.110 DEBUG:boto:Path: /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.110 DEBUG:boto:Data: 2024-04-16T17:49:07.110 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.110 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.110 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.110 DEBUG:boto:Params: {} 2024-04-16T17:49:07.110 DEBUG:boto:Token: None 2024-04-16T17:49:07.110 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM= 2024-04-16T17:49:07.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM=', 'Content-Length': '0'} 2024-04-16T17:49:07.113 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cbd7517a1af25cd7-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.114 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.114 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.114 DEBUG:boto:path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.114 DEBUG:boto:auth_path=/lybtug-7/.o?versionId=null 2024-04-16T17:49:07.114 DEBUG:boto:Method: GET 2024-04-16T17:49:07.114 DEBUG:boto:Path: /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.114 DEBUG:boto:Data: 2024-04-16T17:49:07.114 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.114 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.114 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.114 DEBUG:boto:Params: {} 2024-04-16T17:49:07.115 DEBUG:boto:Token: None 2024-04-16T17:49:07.115 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o?versionId=null 2024-04-16T17:49:07.115 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM= 2024-04-16T17:49:07.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ELnGdcU8JgET2wg1sgI4uBcfrvM=', 'Content-Length': '0'} 2024-04-16T17:49:07.118 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c07d01d79c2959c-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.118 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.118 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.118 DEBUG:boto:path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.118 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.118 DEBUG:boto:Method: GET 2024-04-16T17:49:07.118 DEBUG:boto:Path: /lybtug-7/?prefix=.o 2024-04-16T17:49:07.119 DEBUG:boto:Data: 2024-04-16T17:49:07.119 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.119 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.119 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.119 DEBUG:boto:Params: {} 2024-04-16T17:49:07.119 DEBUG:boto:Token: None 2024-04-16T17:49:07.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2fb2a9dc36bf073-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.123 DEBUG:boto:b'lybtug-7.o1000false.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.124 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.124 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.124 DEBUG:boto:path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.124 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.124 DEBUG:boto:Method: GET 2024-04-16T17:49:07.124 DEBUG:boto:Path: /lybtug-7/?prefix=.o 2024-04-16T17:49:07.124 DEBUG:boto:Data: 2024-04-16T17:49:07.124 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.125 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.125 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.125 DEBUG:boto:Params: {} 2024-04-16T17:49:07.125 DEBUG:boto:Token: None 2024-04-16T17:49:07.125 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7a3c45642d17c00-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.129 DEBUG:boto:b'lybtug-7.o1000false.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.129 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.130 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.130 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.130 DEBUG:boto:Path: /lybtug-7/.o 2024-04-16T17:49:07.130 DEBUG:boto:Data: 2024-04-16T17:49:07.130 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.130 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.130 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.130 DEBUG:boto:Params: {} 2024-04-16T17:49:07.130 DEBUG:boto:Token: None 2024-04-16T17:49:07.130 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o 2024-04-16T17:49:07.131 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3LS7eplwxPDXJkdvfsGTR1XOtlU= 2024-04-16T17:49:07.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3LS7eplwxPDXJkdvfsGTR1XOtlU=', 'Content-Length': '0'} 2024-04-16T17:49:07.134 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003669ba94d50ae92e-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.134 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.134 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.134 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.134 DEBUG:boto:Path: /lybtug-7/.o 2024-04-16T17:49:07.134 DEBUG:boto:Data: 2024-04-16T17:49:07.134 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.134 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.134 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.135 DEBUG:boto:Params: {} 2024-04-16T17:49:07.135 DEBUG:boto:Token: None 2024-04-16T17:49:07.135 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o 2024-04-16T17:49:07.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3LS7eplwxPDXJkdvfsGTR1XOtlU= 2024-04-16T17:49:07.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3LS7eplwxPDXJkdvfsGTR1XOtlU=', 'Content-Length': '0'} 2024-04-16T17:49:07.138 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f74102f9bd549a05-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.138 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o 2024-04-16T17:49:07.138 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.138 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.138 DEBUG:boto:Method: GET 2024-04-16T17:49:07.138 DEBUG:boto:Path: /lybtug-7/.o 2024-04-16T17:49:07.139 DEBUG:boto:Data: 2024-04-16T17:49:07.139 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.139 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.139 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.139 DEBUG:boto:Params: {} 2024-04-16T17:49:07.139 DEBUG:boto:Token: None 2024-04-16T17:49:07.139 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o 2024-04-16T17:49:07.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oiYojCdhE7Bfgf0uU62xEe9Prkc= 2024-04-16T17:49:07.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oiYojCdhE7Bfgf0uU62xEe9Prkc=', 'Content-Length': '0'} 2024-04-16T17:49:07.142 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a8a2909bef005b42-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.182 DEBUG:boto:path=/lybtug-7/.o 2024-04-16T17:49:07.182 DEBUG:boto:auth_path=/lybtug-7/.o 2024-04-16T17:49:07.183 DEBUG:boto:Method: GET 2024-04-16T17:49:07.183 DEBUG:boto:Path: /lybtug-7/.o 2024-04-16T17:49:07.183 DEBUG:boto:Data: 2024-04-16T17:49:07.183 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.183 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.183 DEBUG:boto:Params: {} 2024-04-16T17:49:07.183 DEBUG:boto:Token: None 2024-04-16T17:49:07.183 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o 2024-04-16T17:49:07.183 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oiYojCdhE7Bfgf0uU62xEe9Prkc= 2024-04-16T17:49:07.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oiYojCdhE7Bfgf0uU62xEe9Prkc=', 'Content-Length': '0'} 2024-04-16T17:49:07.186 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000650d9053a6c82288-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.187 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.187 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.187 DEBUG:boto:path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.187 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.187 DEBUG:boto:Method: GET 2024-04-16T17:49:07.187 DEBUG:boto:Path: /lybtug-7/?prefix=.o 2024-04-16T17:49:07.187 DEBUG:boto:Data: 2024-04-16T17:49:07.187 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.187 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.187 DEBUG:boto:Params: {} 2024-04-16T17:49:07.187 DEBUG:boto:Token: None 2024-04-16T17:49:07.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007fbdad25fc704a41-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.192 DEBUG:boto:b'lybtug-7.o1000false.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.192 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.193 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.193 DEBUG:boto:path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.193 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o 2024-04-16T17:49:07.193 DEBUG:boto:Method: GET 2024-04-16T17:49:07.193 DEBUG:boto:Path: /lybtug-7/?prefix=.o 2024-04-16T17:49:07.193 DEBUG:boto:Data: 2024-04-16T17:49:07.193 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.193 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.193 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.193 DEBUG:boto:Params: {} 2024-04-16T17:49:07.193 DEBUG:boto:Token: None 2024-04-16T17:49:07.194 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.194 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.198 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038f42656e63257dd-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.198 DEBUG:boto:b'lybtug-7.o1000false.o2024-04-16T17:48:50.366Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.198 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o. 2024-04-16T17:49:07.198 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.198 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.198 DEBUG:boto:path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.199 DEBUG:boto:auth_path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.199 DEBUG:boto:Method: GET 2024-04-16T17:49:07.199 DEBUG:boto:Path: /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.199 DEBUG:boto:Data: 2024-04-16T17:49:07.199 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.199 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.199 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.199 DEBUG:boto:Params: {} 2024-04-16T17:49:07.199 DEBUG:boto:Token: None 2024-04-16T17:49:07.199 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.199 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk= 2024-04-16T17:49:07.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk=', 'Content-Length': '0'} 2024-04-16T17:49:07.203 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000019fd2d9d91872455-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.203 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.203 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.203 DEBUG:boto:path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.203 DEBUG:boto:auth_path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.204 DEBUG:boto:Method: GET 2024-04-16T17:49:07.204 DEBUG:boto:Path: /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.204 DEBUG:boto:Data: 2024-04-16T17:49:07.204 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.204 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.204 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.204 DEBUG:boto:Params: {} 2024-04-16T17:49:07.204 DEBUG:boto:Token: None 2024-04-16T17:49:07.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk= 2024-04-16T17:49:07.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk=', 'Content-Length': '0'} 2024-04-16T17:49:07.208 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000529f992b0d84a2c6-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.208 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.208 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.208 DEBUG:boto:path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.208 DEBUG:boto:auth_path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.208 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.208 DEBUG:boto:Path: /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.208 DEBUG:boto:Data: 2024-04-16T17:49:07.208 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.209 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.209 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.209 DEBUG:boto:Params: {} 2024-04-16T17:49:07.209 DEBUG:boto:Token: None 2024-04-16T17:49:07.209 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.209 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J4I3ifTWRH8ECQhKqctfMDA7xE4= 2024-04-16T17:49:07.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J4I3ifTWRH8ECQhKqctfMDA7xE4=', 'Content-Length': '0'} 2024-04-16T17:49:07.212 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000539f25a31c44a67c-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.212 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.212 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.212 DEBUG:boto:path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.212 DEBUG:boto:auth_path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.212 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.212 DEBUG:boto:Path: /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.213 DEBUG:boto:Data: 2024-04-16T17:49:07.213 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.213 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.213 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.213 DEBUG:boto:Params: {} 2024-04-16T17:49:07.213 DEBUG:boto:Token: None 2024-04-16T17:49:07.213 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J4I3ifTWRH8ECQhKqctfMDA7xE4= 2024-04-16T17:49:07.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J4I3ifTWRH8ECQhKqctfMDA7xE4=', 'Content-Length': '0'} 2024-04-16T17:49:07.216 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008a6d3a3e68632b19-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o. 2024-04-16T17:49:07.217 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.217 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.217 DEBUG:boto:path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.217 DEBUG:boto:auth_path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.217 DEBUG:boto:Method: GET 2024-04-16T17:49:07.217 DEBUG:boto:Path: /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.217 DEBUG:boto:Data: 2024-04-16T17:49:07.217 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.217 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.217 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.217 DEBUG:boto:Params: {} 2024-04-16T17:49:07.217 DEBUG:boto:Token: None 2024-04-16T17:49:07.218 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.218 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk= 2024-04-16T17:49:07.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk=', 'Content-Length': '0'} 2024-04-16T17:49:07.221 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000002bedfaa77a12b40-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.262 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.262 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.263 DEBUG:boto:path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.263 DEBUG:boto:auth_path=/lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.263 DEBUG:boto:Method: GET 2024-04-16T17:49:07.263 DEBUG:boto:Path: /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.263 DEBUG:boto:Data: 2024-04-16T17:49:07.263 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.263 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.263 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.263 DEBUG:boto:Params: {} 2024-04-16T17:49:07.263 DEBUG:boto:Token: None 2024-04-16T17:49:07.263 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o.?versionId=null 2024-04-16T17:49:07.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk= 2024-04-16T17:49:07.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vlEpawJ8n9tGKGhPJfFWEmFl0Gk=', 'Content-Length': '0'} 2024-04-16T17:49:07.267 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c4b3c658da79f955-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.267 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.267 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.267 DEBUG:boto:path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.267 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.267 DEBUG:boto:Method: GET 2024-04-16T17:49:07.267 DEBUG:boto:Path: /lybtug-7/?prefix=.o. 2024-04-16T17:49:07.268 DEBUG:boto:Data: 2024-04-16T17:49:07.268 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.268 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.268 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.268 DEBUG:boto:Params: {} 2024-04-16T17:49:07.268 DEBUG:boto:Token: None 2024-04-16T17:49:07.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000216e52599e083ff5-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.272 DEBUG:boto:b'lybtug-7.o.1000false.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.273 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.273 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.273 DEBUG:boto:path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.273 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.273 DEBUG:boto:Method: GET 2024-04-16T17:49:07.273 DEBUG:boto:Path: /lybtug-7/?prefix=.o. 2024-04-16T17:49:07.273 DEBUG:boto:Data: 2024-04-16T17:49:07.273 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.273 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.273 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.273 DEBUG:boto:Params: {} 2024-04-16T17:49:07.273 DEBUG:boto:Token: None 2024-04-16T17:49:07.274 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.274 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.278 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc289774a93f7503-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.318 DEBUG:boto:b'lybtug-7.o.1000false.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.319 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.319 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.319 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.319 DEBUG:boto:Path: /lybtug-7/.o. 2024-04-16T17:49:07.319 DEBUG:boto:Data: 2024-04-16T17:49:07.319 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.319 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.319 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.319 DEBUG:boto:Params: {} 2024-04-16T17:49:07.320 DEBUG:boto:Token: None 2024-04-16T17:49:07.320 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o. 2024-04-16T17:49:07.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7ztP0ftZKzW/YyCbPoxBX7BKHzg= 2024-04-16T17:49:07.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7ztP0ftZKzW/YyCbPoxBX7BKHzg=', 'Content-Length': '0'} 2024-04-16T17:49:07.323 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008add81b27d682f14-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.323 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.323 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.323 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.323 DEBUG:boto:Path: /lybtug-7/.o. 2024-04-16T17:49:07.323 DEBUG:boto:Data: 2024-04-16T17:49:07.324 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.324 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.324 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.324 DEBUG:boto:Params: {} 2024-04-16T17:49:07.324 DEBUG:boto:Token: None 2024-04-16T17:49:07.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o. 2024-04-16T17:49:07.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7ztP0ftZKzW/YyCbPoxBX7BKHzg= 2024-04-16T17:49:07.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7ztP0ftZKzW/YyCbPoxBX7BKHzg=', 'Content-Length': '0'} 2024-04-16T17:49:07.327 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000085b67c650869f8e4-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.327 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o. 2024-04-16T17:49:07.327 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.327 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.327 DEBUG:boto:Method: GET 2024-04-16T17:49:07.327 DEBUG:boto:Path: /lybtug-7/.o. 2024-04-16T17:49:07.328 DEBUG:boto:Data: 2024-04-16T17:49:07.328 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.328 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.328 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.328 DEBUG:boto:Params: {} 2024-04-16T17:49:07.328 DEBUG:boto:Token: None 2024-04-16T17:49:07.328 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o. 2024-04-16T17:49:07.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4JD66Xnr8+8+XV6Fbiq1MdkVORI= 2024-04-16T17:49:07.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4JD66Xnr8+8+XV6Fbiq1MdkVORI=', 'Content-Length': '0'} 2024-04-16T17:49:07.332 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000018d86f59ef8b9926-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.332 DEBUG:boto:path=/lybtug-7/.o. 2024-04-16T17:49:07.332 DEBUG:boto:auth_path=/lybtug-7/.o. 2024-04-16T17:49:07.332 DEBUG:boto:Method: GET 2024-04-16T17:49:07.332 DEBUG:boto:Path: /lybtug-7/.o. 2024-04-16T17:49:07.332 DEBUG:boto:Data: 2024-04-16T17:49:07.332 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.332 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.332 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.333 DEBUG:boto:Params: {} 2024-04-16T17:49:07.333 DEBUG:boto:Token: None 2024-04-16T17:49:07.333 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/.o. 2024-04-16T17:49:07.333 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4JD66Xnr8+8+XV6Fbiq1MdkVORI= 2024-04-16T17:49:07.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4JD66Xnr8+8+XV6Fbiq1MdkVORI=', 'Content-Length': '0'} 2024-04-16T17:49:07.336 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000eda75380d21cec32-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.336 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.336 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.336 DEBUG:boto:path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.336 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.336 DEBUG:boto:Method: GET 2024-04-16T17:49:07.336 DEBUG:boto:Path: /lybtug-7/?prefix=.o. 2024-04-16T17:49:07.336 DEBUG:boto:Data: 2024-04-16T17:49:07.337 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.337 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.337 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.337 DEBUG:boto:Params: {} 2024-04-16T17:49:07.337 DEBUG:boto:Token: None 2024-04-16T17:49:07.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.341 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da56e54e9478e4a3-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.382 DEBUG:boto:b'lybtug-7.o.1000false.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.383 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.383 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.383 DEBUG:boto:path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.383 DEBUG:boto:auth_path=/lybtug-7/?prefix=.o. 2024-04-16T17:49:07.383 DEBUG:boto:Method: GET 2024-04-16T17:49:07.383 DEBUG:boto:Path: /lybtug-7/?prefix=.o. 2024-04-16T17:49:07.383 DEBUG:boto:Data: 2024-04-16T17:49:07.383 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.383 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.383 DEBUG:boto:Params: {} 2024-04-16T17:49:07.384 DEBUG:boto:Token: None 2024-04-16T17:49:07.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.387 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8337cba57b8ebaa-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.387 DEBUG:boto:b'lybtug-7.o.1000false.o.2024-04-16T17:48:50.380Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.388 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2024-04-16T17:49:07.388 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.388 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.388 DEBUG:boto:path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.388 DEBUG:boto:auth_path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.388 DEBUG:boto:Method: GET 2024-04-16T17:49:07.388 DEBUG:boto:Path: /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.388 DEBUG:boto:Data: 2024-04-16T17:49:07.388 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.388 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.388 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.389 DEBUG:boto:Params: {} 2024-04-16T17:49:07.389 DEBUG:boto:Token: None 2024-04-16T17:49:07.389 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.389 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU= 2024-04-16T17:49:07.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU=', 'Content-Length': '0'} 2024-04-16T17:49:07.392 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000036c9fecf33c891fa-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.392 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.392 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.393 DEBUG:boto:path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.393 DEBUG:boto:auth_path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.393 DEBUG:boto:Method: GET 2024-04-16T17:49:07.393 DEBUG:boto:Path: /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.393 DEBUG:boto:Data: 2024-04-16T17:49:07.393 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.393 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.393 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.393 DEBUG:boto:Params: {} 2024-04-16T17:49:07.393 DEBUG:boto:Token: None 2024-04-16T17:49:07.393 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.393 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU= 2024-04-16T17:49:07.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU=', 'Content-Length': '0'} 2024-04-16T17:49:07.397 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c724e974f425b794-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.438 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.438 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.439 DEBUG:boto:path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.439 DEBUG:boto:auth_path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.439 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.439 DEBUG:boto:Path: /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.439 DEBUG:boto:Data: 2024-04-16T17:49:07.439 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.439 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.439 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.439 DEBUG:boto:Params: {} 2024-04-16T17:49:07.439 DEBUG:boto:Token: None 2024-04-16T17:49:07.439 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/uW+HFvwJLyG9hI9+oO2TIAF4q8= 2024-04-16T17:49:07.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/uW+HFvwJLyG9hI9+oO2TIAF4q8=', 'Content-Length': '0'} 2024-04-16T17:49:07.442 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ef4e88825c9204d3-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.443 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.443 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.443 DEBUG:boto:path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.443 DEBUG:boto:auth_path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.443 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.443 DEBUG:boto:Path: /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.443 DEBUG:boto:Data: 2024-04-16T17:49:07.443 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.443 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.443 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.443 DEBUG:boto:Params: {} 2024-04-16T17:49:07.444 DEBUG:boto:Token: None 2024-04-16T17:49:07.444 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/uW+HFvwJLyG9hI9+oO2TIAF4q8= 2024-04-16T17:49:07.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/uW+HFvwJLyG9hI9+oO2TIAF4q8=', 'Content-Length': '0'} 2024-04-16T17:49:07.447 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008d284badec6246a1-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.447 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2024-04-16T17:49:07.447 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.447 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.447 DEBUG:boto:path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.447 DEBUG:boto:auth_path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.448 DEBUG:boto:Method: GET 2024-04-16T17:49:07.448 DEBUG:boto:Path: /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.448 DEBUG:boto:Data: 2024-04-16T17:49:07.448 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.448 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.448 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.448 DEBUG:boto:Params: {} 2024-04-16T17:49:07.448 DEBUG:boto:Token: None 2024-04-16T17:49:07.448 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU= 2024-04-16T17:49:07.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU=', 'Content-Length': '0'} 2024-04-16T17:49:07.451 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002cc3df4e2be427c2-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.451 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.452 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.452 DEBUG:boto:path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.452 DEBUG:boto:auth_path=/lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.452 DEBUG:boto:Method: GET 2024-04-16T17:49:07.452 DEBUG:boto:Path: /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.452 DEBUG:boto:Data: 2024-04-16T17:49:07.452 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.452 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.452 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.452 DEBUG:boto:Params: {} 2024-04-16T17:49:07.452 DEBUG:boto:Token: None 2024-04-16T17:49:07.453 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A?versionId=null 2024-04-16T17:49:07.453 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU= 2024-04-16T17:49:07.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UnnPkkwUIQVDhkViIaEexDe8gkU=', 'Content-Length': '0'} 2024-04-16T17:49:07.456 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000044e31b5220a228f5-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.456 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.456 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.456 DEBUG:boto:path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.456 DEBUG:boto:auth_path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.456 DEBUG:boto:Method: GET 2024-04-16T17:49:07.457 DEBUG:boto:Path: /lybtug-7/?prefix=%3A 2024-04-16T17:49:07.457 DEBUG:boto:Data: 2024-04-16T17:49:07.457 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.457 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.457 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.457 DEBUG:boto:Params: {} 2024-04-16T17:49:07.457 DEBUG:boto:Token: None 2024-04-16T17:49:07.457 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.457 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.461 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e6d60bee4216db9-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.502 DEBUG:boto:b'lybtug-7:1000false:2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.503 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.503 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.503 DEBUG:boto:path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.503 DEBUG:boto:auth_path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.503 DEBUG:boto:Method: GET 2024-04-16T17:49:07.503 DEBUG:boto:Path: /lybtug-7/?prefix=%3A 2024-04-16T17:49:07.503 DEBUG:boto:Data: 2024-04-16T17:49:07.503 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.503 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.503 DEBUG:boto:Params: {} 2024-04-16T17:49:07.504 DEBUG:boto:Token: None 2024-04-16T17:49:07.504 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.508 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbb11f62044599c0-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.508 DEBUG:boto:b'lybtug-7:1000false:2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.508 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.508 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.508 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.508 DEBUG:boto:Path: /lybtug-7/%3A 2024-04-16T17:49:07.508 DEBUG:boto:Data: 2024-04-16T17:49:07.508 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.509 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.509 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.509 DEBUG:boto:Params: {} 2024-04-16T17:49:07.509 DEBUG:boto:Token: None 2024-04-16T17:49:07.509 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A 2024-04-16T17:49:07.509 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pvvErXQGksmFKCLq+XjO1b04ZX4= 2024-04-16T17:49:07.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pvvErXQGksmFKCLq+XjO1b04ZX4=', 'Content-Length': '0'} 2024-04-16T17:49:07.512 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b6e37b1ac5c32f97-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.512 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.512 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.513 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.513 DEBUG:boto:Path: /lybtug-7/%3A 2024-04-16T17:49:07.513 DEBUG:boto:Data: 2024-04-16T17:49:07.513 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.513 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.513 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.513 DEBUG:boto:Params: {} 2024-04-16T17:49:07.513 DEBUG:boto:Token: None 2024-04-16T17:49:07.513 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A 2024-04-16T17:49:07.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pvvErXQGksmFKCLq+XjO1b04ZX4= 2024-04-16T17:49:07.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pvvErXQGksmFKCLq+XjO1b04ZX4=', 'Content-Length': '0'} 2024-04-16T17:49:07.516 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007f6ea3c273c9a8ad-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.517 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2024-04-16T17:49:07.517 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.517 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.517 DEBUG:boto:Method: GET 2024-04-16T17:49:07.517 DEBUG:boto:Path: /lybtug-7/%3A 2024-04-16T17:49:07.517 DEBUG:boto:Data: 2024-04-16T17:49:07.517 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.517 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.517 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.517 DEBUG:boto:Params: {} 2024-04-16T17:49:07.518 DEBUG:boto:Token: None 2024-04-16T17:49:07.518 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A 2024-04-16T17:49:07.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fgyziPaHS4j/qRrTy1OT5vSHcgQ= 2024-04-16T17:49:07.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fgyziPaHS4j/qRrTy1OT5vSHcgQ=', 'Content-Length': '0'} 2024-04-16T17:49:07.521 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000064fad1a989cc0751-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.521 DEBUG:boto:path=/lybtug-7/%3A 2024-04-16T17:49:07.521 DEBUG:boto:auth_path=/lybtug-7/%3A 2024-04-16T17:49:07.521 DEBUG:boto:Method: GET 2024-04-16T17:49:07.521 DEBUG:boto:Path: /lybtug-7/%3A 2024-04-16T17:49:07.521 DEBUG:boto:Data: 2024-04-16T17:49:07.521 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.522 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.522 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.522 DEBUG:boto:Params: {} 2024-04-16T17:49:07.522 DEBUG:boto:Token: None 2024-04-16T17:49:07.522 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/%3A 2024-04-16T17:49:07.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fgyziPaHS4j/qRrTy1OT5vSHcgQ= 2024-04-16T17:49:07.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fgyziPaHS4j/qRrTy1OT5vSHcgQ=', 'Content-Length': '0'} 2024-04-16T17:49:07.525 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007dee8f73fa5f9cf2-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.566 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.566 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.567 DEBUG:boto:path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.567 DEBUG:boto:auth_path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.567 DEBUG:boto:Method: GET 2024-04-16T17:49:07.567 DEBUG:boto:Path: /lybtug-7/?prefix=%3A 2024-04-16T17:49:07.567 DEBUG:boto:Data: 2024-04-16T17:49:07.567 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.567 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.567 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.567 DEBUG:boto:Params: {} 2024-04-16T17:49:07.567 DEBUG:boto:Token: None 2024-04-16T17:49:07.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac2c5111f24e62ff-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.572 DEBUG:boto:b'lybtug-7:1000false:2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.572 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.572 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.572 DEBUG:boto:path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.572 DEBUG:boto:auth_path=/lybtug-7/?prefix=%3A 2024-04-16T17:49:07.572 DEBUG:boto:Method: GET 2024-04-16T17:49:07.572 DEBUG:boto:Path: /lybtug-7/?prefix=%3A 2024-04-16T17:49:07.573 DEBUG:boto:Data: 2024-04-16T17:49:07.573 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.573 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.573 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.573 DEBUG:boto:Params: {} 2024-04-16T17:49:07.573 DEBUG:boto:Token: None 2024-04-16T17:49:07.573 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.577 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed987d2bb62536eb-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.577 DEBUG:boto:b'lybtug-7:1000false:2024-04-16T17:48:50.185Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.578 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2024-04-16T17:49:07.578 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.578 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.578 DEBUG:boto:path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.578 DEBUG:boto:auth_path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.578 DEBUG:boto:Method: GET 2024-04-16T17:49:07.578 DEBUG:boto:Path: /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.578 DEBUG:boto:Data: 2024-04-16T17:49:07.578 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.578 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.578 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.578 DEBUG:boto:Params: {} 2024-04-16T17:49:07.579 DEBUG:boto:Token: None 2024-04-16T17:49:07.579 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.579 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA= 2024-04-16T17:49:07.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA=', 'Content-Length': '0'} 2024-04-16T17:49:07.582 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ca46c5eaafbf9541-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.626 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.626 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.626 DEBUG:boto:path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.627 DEBUG:boto:auth_path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.627 DEBUG:boto:Method: GET 2024-04-16T17:49:07.627 DEBUG:boto:Path: /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.627 DEBUG:boto:Data: 2024-04-16T17:49:07.627 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.627 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.627 DEBUG:boto:Params: {} 2024-04-16T17:49:07.627 DEBUG:boto:Token: None 2024-04-16T17:49:07.627 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.627 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA= 2024-04-16T17:49:07.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA=', 'Content-Length': '0'} 2024-04-16T17:49:07.631 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a0db9afd5fccf651-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.631 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.631 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.631 DEBUG:boto:path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.631 DEBUG:boto:auth_path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.631 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.632 DEBUG:boto:Path: /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.632 DEBUG:boto:Data: 2024-04-16T17:49:07.632 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.632 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.632 DEBUG:boto:Params: {} 2024-04-16T17:49:07.632 DEBUG:boto:Token: None 2024-04-16T17:49:07.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z98u7z94Yuolm1nUyNCeyO3B49U= 2024-04-16T17:49:07.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z98u7z94Yuolm1nUyNCeyO3B49U=', 'Content-Length': '0'} 2024-04-16T17:49:07.635 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ae1c3e0347e8774b-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.636 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.636 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.636 DEBUG:boto:path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.636 DEBUG:boto:auth_path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.636 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.636 DEBUG:boto:Path: /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.636 DEBUG:boto:Data: 2024-04-16T17:49:07.636 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.636 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.637 DEBUG:boto:Params: {} 2024-04-16T17:49:07.637 DEBUG:boto:Token: None 2024-04-16T17:49:07.637 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z98u7z94Yuolm1nUyNCeyO3B49U= 2024-04-16T17:49:07.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z98u7z94Yuolm1nUyNCeyO3B49U=', 'Content-Length': '0'} 2024-04-16T17:49:07.640 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ebece3296abbaaa1-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2024-04-16T17:49:07.640 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.640 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.640 DEBUG:boto:path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.641 DEBUG:boto:auth_path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.641 DEBUG:boto:Method: GET 2024-04-16T17:49:07.641 DEBUG:boto:Path: /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.641 DEBUG:boto:Data: 2024-04-16T17:49:07.641 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.641 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.641 DEBUG:boto:Params: {} 2024-04-16T17:49:07.641 DEBUG:boto:Token: None 2024-04-16T17:49:07.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA= 2024-04-16T17:49:07.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA=', 'Content-Length': '0'} 2024-04-16T17:49:07.645 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000fd99e6d30f9ec9e7-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.645 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.645 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.645 DEBUG:boto:path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.645 DEBUG:boto:auth_path=/lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.645 DEBUG:boto:Method: GET 2024-04-16T17:49:07.645 DEBUG:boto:Path: /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.645 DEBUG:boto:Data: 2024-04-16T17:49:07.646 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.646 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.646 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.646 DEBUG:boto:Params: {} 2024-04-16T17:49:07.646 DEBUG:boto:Token: None 2024-04-16T17:49:07.646 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj?versionId=null 2024-04-16T17:49:07.646 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA= 2024-04-16T17:49:07.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1fp2Hgmcy9HOfUZCLDlAJwnJxA=', 'Content-Length': '0'} 2024-04-16T17:49:07.649 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004ac053d3067e8743-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.690 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.690 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.691 DEBUG:boto:path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.691 DEBUG:boto:auth_path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.691 DEBUG:boto:Method: GET 2024-04-16T17:49:07.691 DEBUG:boto:Path: /lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.691 DEBUG:boto:Data: 2024-04-16T17:49:07.691 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.691 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.691 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.691 DEBUG:boto:Params: {} 2024-04-16T17:49:07.691 DEBUG:boto:Token: None 2024-04-16T17:49:07.691 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f0038c9d3208c4f-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.696 DEBUG:boto:b'lybtug-7_myobj1000false_myobj2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.696 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.696 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.696 DEBUG:boto:path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.696 DEBUG:boto:auth_path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.697 DEBUG:boto:Method: GET 2024-04-16T17:49:07.697 DEBUG:boto:Path: /lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.697 DEBUG:boto:Data: 2024-04-16T17:49:07.697 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.697 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.697 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.697 DEBUG:boto:Params: {} 2024-04-16T17:49:07.697 DEBUG:boto:Token: None 2024-04-16T17:49:07.697 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.697 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000272f35c6fc2b0c03-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.702 DEBUG:boto:b'lybtug-7_myobj1000false_myobj2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.702 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.703 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.703 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.703 DEBUG:boto:Path: /lybtug-7/_myobj 2024-04-16T17:49:07.703 DEBUG:boto:Data: 2024-04-16T17:49:07.703 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.703 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.703 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.703 DEBUG:boto:Params: {} 2024-04-16T17:49:07.703 DEBUG:boto:Token: None 2024-04-16T17:49:07.703 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj 2024-04-16T17:49:07.703 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZrivZFFaNKGclBmnP4dxG8VeY6k= 2024-04-16T17:49:07.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZrivZFFaNKGclBmnP4dxG8VeY6k=', 'Content-Length': '0'} 2024-04-16T17:49:07.706 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000906186d1a4308cb2-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.707 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.707 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.707 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.707 DEBUG:boto:Path: /lybtug-7/_myobj 2024-04-16T17:49:07.707 DEBUG:boto:Data: 2024-04-16T17:49:07.707 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.707 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.707 DEBUG:boto:Params: {} 2024-04-16T17:49:07.707 DEBUG:boto:Token: None 2024-04-16T17:49:07.708 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj 2024-04-16T17:49:07.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZrivZFFaNKGclBmnP4dxG8VeY6k= 2024-04-16T17:49:07.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZrivZFFaNKGclBmnP4dxG8VeY6k=', 'Content-Length': '0'} 2024-04-16T17:49:07.710 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000029a5e564ee57ac36-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.711 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2024-04-16T17:49:07.711 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.711 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.711 DEBUG:boto:Method: GET 2024-04-16T17:49:07.711 DEBUG:boto:Path: /lybtug-7/_myobj 2024-04-16T17:49:07.711 DEBUG:boto:Data: 2024-04-16T17:49:07.711 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.711 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.711 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.712 DEBUG:boto:Params: {} 2024-04-16T17:49:07.712 DEBUG:boto:Token: None 2024-04-16T17:49:07.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj 2024-04-16T17:49:07.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OlJW+0/RLl7rdMuBV4p2Klhqc04= 2024-04-16T17:49:07.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OlJW+0/RLl7rdMuBV4p2Klhqc04=', 'Content-Length': '0'} 2024-04-16T17:49:07.715 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000828c75b69cd8bbb4-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.758 DEBUG:boto:path=/lybtug-7/_myobj 2024-04-16T17:49:07.758 DEBUG:boto:auth_path=/lybtug-7/_myobj 2024-04-16T17:49:07.759 DEBUG:boto:Method: GET 2024-04-16T17:49:07.759 DEBUG:boto:Path: /lybtug-7/_myobj 2024-04-16T17:49:07.759 DEBUG:boto:Data: 2024-04-16T17:49:07.759 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.759 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.759 DEBUG:boto:Params: {} 2024-04-16T17:49:07.759 DEBUG:boto:Token: None 2024-04-16T17:49:07.759 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/_myobj 2024-04-16T17:49:07.759 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OlJW+0/RLl7rdMuBV4p2Klhqc04= 2024-04-16T17:49:07.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OlJW+0/RLl7rdMuBV4p2Klhqc04=', 'Content-Length': '0'} 2024-04-16T17:49:07.762 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000054b7cf0d0c16de40-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.763 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.763 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.763 DEBUG:boto:path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.763 DEBUG:boto:auth_path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.763 DEBUG:boto:Method: GET 2024-04-16T17:49:07.763 DEBUG:boto:Path: /lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.763 DEBUG:boto:Data: 2024-04-16T17:49:07.763 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.763 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.763 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.763 DEBUG:boto:Params: {} 2024-04-16T17:49:07.764 DEBUG:boto:Token: None 2024-04-16T17:49:07.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007287145aba783106-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.768 DEBUG:boto:b'lybtug-7_myobj1000false_myobj2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.769 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.769 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.769 DEBUG:boto:path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.769 DEBUG:boto:auth_path=/lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.769 DEBUG:boto:Method: GET 2024-04-16T17:49:07.769 DEBUG:boto:Path: /lybtug-7/?prefix=_myobj 2024-04-16T17:49:07.769 DEBUG:boto:Data: 2024-04-16T17:49:07.769 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.769 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.769 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.769 DEBUG:boto:Params: {} 2024-04-16T17:49:07.770 DEBUG:boto:Token: None 2024-04-16T17:49:07.770 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.770 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.773 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b8910d6393d0fcf-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.814 DEBUG:boto:b'lybtug-7_myobj1000false_myobj2024-04-16T17:48:50.159Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.815 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:07.815 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.815 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.815 DEBUG:boto:path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.815 DEBUG:boto:auth_path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.815 DEBUG:boto:Method: GET 2024-04-16T17:49:07.815 DEBUG:boto:Path: /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.815 DEBUG:boto:Data: 2024-04-16T17:49:07.815 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.815 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.815 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.815 DEBUG:boto:Params: {} 2024-04-16T17:49:07.816 DEBUG:boto:Token: None 2024-04-16T17:49:07.816 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c= 2024-04-16T17:49:07.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c=', 'Content-Length': '0'} 2024-04-16T17:49:07.819 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000057d3c906c31811b5-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.819 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.820 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.820 DEBUG:boto:path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.820 DEBUG:boto:auth_path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.820 DEBUG:boto:Method: GET 2024-04-16T17:49:07.820 DEBUG:boto:Path: /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.820 DEBUG:boto:Data: 2024-04-16T17:49:07.820 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.820 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.820 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.820 DEBUG:boto:Params: {} 2024-04-16T17:49:07.820 DEBUG:boto:Token: None 2024-04-16T17:49:07.820 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c= 2024-04-16T17:49:07.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c=', 'Content-Length': '0'} 2024-04-16T17:49:07.824 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007ab9fcd3a6255769-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.824 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.824 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.824 DEBUG:boto:path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.824 DEBUG:boto:auth_path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.824 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.824 DEBUG:boto:Path: /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.824 DEBUG:boto:Data: 2024-04-16T17:49:07.825 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.825 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.825 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.825 DEBUG:boto:Params: {} 2024-04-16T17:49:07.825 DEBUG:boto:Token: None 2024-04-16T17:49:07.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HQJfXBkOtXqRlfMVOCE1gDkHRbY= 2024-04-16T17:49:07.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HQJfXBkOtXqRlfMVOCE1gDkHRbY=', 'Content-Length': '0'} 2024-04-16T17:49:07.828 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004ca2de5f7ee07b4f-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.828 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.829 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.829 DEBUG:boto:path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.829 DEBUG:boto:auth_path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.829 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.829 DEBUG:boto:Path: /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.829 DEBUG:boto:Data: 2024-04-16T17:49:07.829 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.829 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.829 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.829 DEBUG:boto:Params: {} 2024-04-16T17:49:07.829 DEBUG:boto:Token: None 2024-04-16T17:49:07.830 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.830 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HQJfXBkOtXqRlfMVOCE1gDkHRbY= 2024-04-16T17:49:07.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HQJfXBkOtXqRlfMVOCE1gDkHRbY=', 'Content-Length': '0'} 2024-04-16T17:49:07.833 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b38520a004301394-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.833 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:07.833 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.833 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.833 DEBUG:boto:path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.833 DEBUG:boto:auth_path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.833 DEBUG:boto:Method: GET 2024-04-16T17:49:07.833 DEBUG:boto:Path: /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.833 DEBUG:boto:Data: 2024-04-16T17:49:07.834 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.834 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.834 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.834 DEBUG:boto:Params: {} 2024-04-16T17:49:07.834 DEBUG:boto:Token: None 2024-04-16T17:49:07.834 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.834 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c= 2024-04-16T17:49:07.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c=', 'Content-Length': '0'} 2024-04-16T17:49:07.837 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d755aa9915130897-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.878 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.879 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.879 DEBUG:boto:path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.879 DEBUG:boto:auth_path=/lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.879 DEBUG:boto:Method: GET 2024-04-16T17:49:07.879 DEBUG:boto:Path: /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.879 DEBUG:boto:Data: 2024-04-16T17:49:07.879 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.879 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.879 DEBUG:boto:Params: {} 2024-04-16T17:49:07.879 DEBUG:boto:Token: None 2024-04-16T17:49:07.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj?versionId=null 2024-04-16T17:49:07.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c= 2024-04-16T17:49:07.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G2DOyqwOPHE5mCOPjaCeSLoV83c=', 'Content-Length': '0'} 2024-04-16T17:49:07.883 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004224e3c47f38aebe-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.883 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.883 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.883 DEBUG:boto:path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:07.884 DEBUG:boto:auth_path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:07.884 DEBUG:boto:Method: GET 2024-04-16T17:49:07.884 DEBUG:boto:Path: /lybtug-7/?prefix=myobj 2024-04-16T17:49:07.884 DEBUG:boto:Data: 2024-04-16T17:49:07.884 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.884 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.884 DEBUG:boto:Params: {} 2024-04-16T17:49:07.884 DEBUG:boto:Token: None 2024-04-16T17:49:07.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.889 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ee71694af5b276b-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.889 DEBUG:boto:b'lybtug-7myobj1000falsemyobj2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.889 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.889 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.889 DEBUG:boto:path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:07.889 DEBUG:boto:auth_path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:07.889 DEBUG:boto:Method: GET 2024-04-16T17:49:07.890 DEBUG:boto:Path: /lybtug-7/?prefix=myobj 2024-04-16T17:49:07.890 DEBUG:boto:Data: 2024-04-16T17:49:07.890 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.890 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.890 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.890 DEBUG:boto:Params: {} 2024-04-16T17:49:07.890 DEBUG:boto:Token: None 2024-04-16T17:49:07.890 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.890 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.894 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edef6de5cc763cae-00661eba13-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.938 DEBUG:boto:b'lybtug-7myobj1000falsemyobj2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:07.939 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.939 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.939 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.939 DEBUG:boto:Path: /lybtug-7/myobj 2024-04-16T17:49:07.939 DEBUG:boto:Data: 2024-04-16T17:49:07.939 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.939 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.939 DEBUG:boto:Params: {} 2024-04-16T17:49:07.940 DEBUG:boto:Token: None 2024-04-16T17:49:07.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj 2024-04-16T17:49:07.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XEu7nebkxdVRMhIIAChOXTGAabY= 2024-04-16T17:49:07.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XEu7nebkxdVRMhIIAChOXTGAabY=', 'Content-Length': '0'} 2024-04-16T17:49:07.943 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000023d5a791b0828c9c-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.943 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.943 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.943 DEBUG:boto:Method: HEAD 2024-04-16T17:49:07.944 DEBUG:boto:Path: /lybtug-7/myobj 2024-04-16T17:49:07.944 DEBUG:boto:Data: 2024-04-16T17:49:07.944 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.944 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.944 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.944 DEBUG:boto:Params: {} 2024-04-16T17:49:07.944 DEBUG:boto:Token: None 2024-04-16T17:49:07.944 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj 2024-04-16T17:49:07.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XEu7nebkxdVRMhIIAChOXTGAabY= 2024-04-16T17:49:07.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XEu7nebkxdVRMhIIAChOXTGAabY=', 'Content-Length': '0'} 2024-04-16T17:49:07.947 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c686ee00eeceab6a-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:07.948 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.948 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.948 DEBUG:boto:Method: GET 2024-04-16T17:49:07.948 DEBUG:boto:Path: /lybtug-7/myobj 2024-04-16T17:49:07.948 DEBUG:boto:Data: 2024-04-16T17:49:07.948 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.948 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.948 DEBUG:boto:Params: {} 2024-04-16T17:49:07.948 DEBUG:boto:Token: None 2024-04-16T17:49:07.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj 2024-04-16T17:49:07.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tQfaUwesmldYLoubaHFl5VX2Aaw= 2024-04-16T17:49:07.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tQfaUwesmldYLoubaHFl5VX2Aaw=', 'Content-Length': '0'} 2024-04-16T17:49:07.952 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000549d22ddde450b8f-00661eba13-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.952 DEBUG:boto:path=/lybtug-7/myobj 2024-04-16T17:49:07.952 DEBUG:boto:auth_path=/lybtug-7/myobj 2024-04-16T17:49:07.952 DEBUG:boto:Method: GET 2024-04-16T17:49:07.952 DEBUG:boto:Path: /lybtug-7/myobj 2024-04-16T17:49:07.952 DEBUG:boto:Data: 2024-04-16T17:49:07.952 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:07.953 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.953 DEBUG:boto:Params: {} 2024-04-16T17:49:07.953 DEBUG:boto:Token: None 2024-04-16T17:49:07.953 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/myobj 2024-04-16T17:49:07.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tQfaUwesmldYLoubaHFl5VX2Aaw= 2024-04-16T17:49:07.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tQfaUwesmldYLoubaHFl5VX2Aaw=', 'Content-Length': '0'} 2024-04-16T17:49:07.956 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b0386b294f27f871-00661eba13-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:07.957 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:07.957 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:07.957 DEBUG:boto:path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:07.957 DEBUG:boto:auth_path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:07.957 DEBUG:boto:Method: GET 2024-04-16T17:49:07.957 DEBUG:boto:Path: /lybtug-7/?prefix=myobj 2024-04-16T17:49:07.957 DEBUG:boto:Data: 2024-04-16T17:49:07.957 DEBUG:boto:Headers: {} 2024-04-16T17:49:07.957 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:07.957 DEBUG:boto:Port: 8000 2024-04-16T17:49:07.957 DEBUG:boto:Params: {} 2024-04-16T17:49:07.957 DEBUG:boto:Token: None 2024-04-16T17:49:07.958 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:07 GMT /lybtug-7/ 2024-04-16T17:49:07.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U= 2024-04-16T17:49:07.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:07 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yeBjn2I4/A0P+dNngmbwlofgD0U=', 'Content-Length': '0'} 2024-04-16T17:49:07.961 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e3afd034c87445a-00661eba13-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:07 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.002 DEBUG:boto:b'lybtug-7myobj1000falsemyobj2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.003 DEBUG:boto:path=/lybtug-7/ 2024-04-16T17:49:08.003 DEBUG:boto:auth_path=/lybtug-7/ 2024-04-16T17:49:08.003 DEBUG:boto:path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:08.003 DEBUG:boto:auth_path=/lybtug-7/?prefix=myobj 2024-04-16T17:49:08.003 DEBUG:boto:Method: GET 2024-04-16T17:49:08.003 DEBUG:boto:Path: /lybtug-7/?prefix=myobj 2024-04-16T17:49:08.003 DEBUG:boto:Data: 2024-04-16T17:49:08.003 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.003 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.004 DEBUG:boto:Params: {} 2024-04-16T17:49:08.004 DEBUG:boto:Token: None 2024-04-16T17:49:08.004 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-7/ 2024-04-16T17:49:08.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:08REwQQv8J6/1PywAlHh/Nfxj+c= 2024-04-16T17:49:08.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:08REwQQv8J6/1PywAlHh/Nfxj+c=', 'Content-Length': '0'} 2024-04-16T17:49:08.008 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f10c0ae1d71094b7-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.008 DEBUG:boto:b'lybtug-7myobj1000falsemyobj2024-04-16T17:48:50.133Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.008 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-7 zones={test-zone2, test-zone1} 2024-04-16T17:49:08.009 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-8', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:08.009 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-8', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:08.009 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-8 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:49:08.059 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:08.059 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:08.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:08.133+0000 7f2437ab9980 1 bucket sync caught up with source: 2024-04-16T17:49:08.134 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.13.6, , 00000000001.5.6, , 00000000001.10.6, 00000000002.12.6, 00000000001.16.6, 00000000001.9.6, 00000000001.7.6, ] 2024-04-16T17:49:08.134 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.13.6, , 00000000001.5.6, , 00000000001.10.6, 00000000002.12.6, 00000000001.16.6, 00000000001.9.6, 00000000001.7.6, ] 2024-04-16T17:49:08.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:08.133+0000 7f2437ab9980 0 bucket checkpoint complete 2024-04-16T17:49:08.163 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-8 zones={test-zone1, test-zone2} 2024-04-16T17:49:08.164 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.164 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.164 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.164 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:49:08.164 DEBUG:boto:Data: 2024-04-16T17:49:08.164 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.164 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.164 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.164 DEBUG:boto:Params: {} 2024-04-16T17:49:08.164 DEBUG:boto:Token: None 2024-04-16T17:49:08.165 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.165 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aIyQa9rRwPUeFArkzu0IRDIWOpY= 2024-04-16T17:49:08.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aIyQa9rRwPUeFArkzu0IRDIWOpY=', 'Content-Length': '0'} 2024-04-16T17:49:08.168 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a7fdd0a2f124719f-00661eba14-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.168 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.168 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.169 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.169 DEBUG:boto:Path: /lybtug-8/ 2024-04-16T17:49:08.169 DEBUG:boto:Data: 2024-04-16T17:49:08.169 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.169 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.169 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.169 DEBUG:boto:Params: {} 2024-04-16T17:49:08.169 DEBUG:boto:Token: None 2024-04-16T17:49:08.169 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.169 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aIyQa9rRwPUeFArkzu0IRDIWOpY= 2024-04-16T17:49:08.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aIyQa9rRwPUeFArkzu0IRDIWOpY=', 'Content-Length': '0'} 2024-04-16T17:49:08.173 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000033affbc47eb50c9d-00661eba14-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.173 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:49:08.173 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.173 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.173 DEBUG:boto:path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.173 DEBUG:boto:auth_path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.173 DEBUG:boto:Method: GET 2024-04-16T17:49:08.174 DEBUG:boto:Path: /lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.174 DEBUG:boto:Data: 2024-04-16T17:49:08.174 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.174 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.174 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.174 DEBUG:boto:Params: {} 2024-04-16T17:49:08.174 DEBUG:boto:Token: None 2024-04-16T17:49:08.174 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/?versions 2024-04-16T17:49:08.174 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0= 2024-04-16T17:49:08.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0=', 'Content-Length': '0'} 2024-04-16T17:49:08.177 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000614388f92c9eaefc-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.178 DEBUG:boto:b'lybtug-8999false&nulltrue2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.178 DEBUG:tasks.rgw_multi.zone_rados:o=& 2024-04-16T17:49:08.178 DEBUG:tasks.rgw_multi.zone_rados:o=. 2024-04-16T17:49:08.178 DEBUG:tasks.rgw_multi.zone_rados:o=.. 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:o=... 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:o=.o 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:o=.o. 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:o=: 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:08.179 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:49:08.179 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.179 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.179 DEBUG:boto:path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.179 DEBUG:boto:auth_path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.179 DEBUG:boto:Method: GET 2024-04-16T17:49:08.180 DEBUG:boto:Path: /lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.180 DEBUG:boto:Data: 2024-04-16T17:49:08.180 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.180 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.180 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.180 DEBUG:boto:Params: {} 2024-04-16T17:49:08.180 DEBUG:boto:Token: None 2024-04-16T17:49:08.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/?versions 2024-04-16T17:49:08.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0= 2024-04-16T17:49:08.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0=', 'Content-Length': '0'} 2024-04-16T17:49:08.184 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fdffc4a14d9d1da-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.184 DEBUG:boto:b'lybtug-8999false&nulltrue2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=& 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=. 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=.. 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=... 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=.o 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=.o. 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=: 2024-04-16T17:49:08.185 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2024-04-16T17:49:08.186 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:08.186 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.186 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.186 DEBUG:boto:path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.186 DEBUG:boto:auth_path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.186 DEBUG:boto:Method: GET 2024-04-16T17:49:08.186 DEBUG:boto:Path: /lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.186 DEBUG:boto:Data: 2024-04-16T17:49:08.186 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.186 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.186 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.187 DEBUG:boto:Params: {} 2024-04-16T17:49:08.187 DEBUG:boto:Token: None 2024-04-16T17:49:08.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/?versions 2024-04-16T17:49:08.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0= 2024-04-16T17:49:08.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0=', 'Content-Length': '0'} 2024-04-16T17:49:08.190 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b592205642cc63f0-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.190 DEBUG:boto:b'lybtug-8999false&nulltrue2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.191 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.191 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.191 DEBUG:boto:path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.191 DEBUG:boto:auth_path=/lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.191 DEBUG:boto:Method: GET 2024-04-16T17:49:08.191 DEBUG:boto:Path: /lybtug-8/?versions&max-keys=999 2024-04-16T17:49:08.191 DEBUG:boto:Data: 2024-04-16T17:49:08.191 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.192 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.192 DEBUG:boto:Params: {} 2024-04-16T17:49:08.192 DEBUG:boto:Token: None 2024-04-16T17:49:08.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/?versions 2024-04-16T17:49:08.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0= 2024-04-16T17:49:08.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TB7vqRdww9wNCCAwJeTYTj4PPb0=', 'Content-Length': '0'} 2024-04-16T17:49:08.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea7c14bfc67fc082-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.196 DEBUG:boto:b'lybtug-8999false&nulltrue2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.nulltrue2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..nulltrue2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...nulltrue2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.onulltrue2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.nulltrue2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal:nulltrue2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal_myobjnulltrue2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalmyobjnulltrue2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.197 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2024-04-16T17:49:08.197 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.197 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.197 DEBUG:boto:path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.197 DEBUG:boto:auth_path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.198 DEBUG:boto:Method: GET 2024-04-16T17:49:08.198 DEBUG:boto:Path: /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.198 DEBUG:boto:Data: 2024-04-16T17:49:08.198 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.198 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.198 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.198 DEBUG:boto:Params: {} 2024-04-16T17:49:08.198 DEBUG:boto:Token: None 2024-04-16T17:49:08.198 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.198 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ= 2024-04-16T17:49:08.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ=', 'Content-Length': '0'} 2024-04-16T17:49:08.201 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e334632ffbe9f710-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.242 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.242 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.243 DEBUG:boto:path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.243 DEBUG:boto:auth_path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.243 DEBUG:boto:Method: GET 2024-04-16T17:49:08.243 DEBUG:boto:Path: /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.243 DEBUG:boto:Data: 2024-04-16T17:49:08.243 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.243 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.243 DEBUG:boto:Params: {} 2024-04-16T17:49:08.243 DEBUG:boto:Token: None 2024-04-16T17:49:08.244 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ= 2024-04-16T17:49:08.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ=', 'Content-Length': '0'} 2024-04-16T17:49:08.247 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000455a8162dc601bb4-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.247 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.247 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.247 DEBUG:boto:path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.247 DEBUG:boto:auth_path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.247 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.248 DEBUG:boto:Path: /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.248 DEBUG:boto:Data: 2024-04-16T17:49:08.248 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.248 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.248 DEBUG:boto:Params: {} 2024-04-16T17:49:08.248 DEBUG:boto:Token: None 2024-04-16T17:49:08.248 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pnPMZn4a24hXuRv2iHeZJBuiz0U= 2024-04-16T17:49:08.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pnPMZn4a24hXuRv2iHeZJBuiz0U=', 'Content-Length': '0'} 2024-04-16T17:49:08.251 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000c243b540cd57062-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.251 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.251 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.252 DEBUG:boto:path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.252 DEBUG:boto:auth_path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.252 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.252 DEBUG:boto:Path: /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.252 DEBUG:boto:Data: 2024-04-16T17:49:08.252 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.252 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.252 DEBUG:boto:Params: {} 2024-04-16T17:49:08.252 DEBUG:boto:Token: None 2024-04-16T17:49:08.253 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.253 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pnPMZn4a24hXuRv2iHeZJBuiz0U= 2024-04-16T17:49:08.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pnPMZn4a24hXuRv2iHeZJBuiz0U=', 'Content-Length': '0'} 2024-04-16T17:49:08.256 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007865dc246d1a13ca-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2024-04-16T17:49:08.256 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.256 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.256 DEBUG:boto:path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.256 DEBUG:boto:auth_path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.256 DEBUG:boto:Method: GET 2024-04-16T17:49:08.256 DEBUG:boto:Path: /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.256 DEBUG:boto:Data: 2024-04-16T17:49:08.257 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.257 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.257 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.257 DEBUG:boto:Params: {} 2024-04-16T17:49:08.257 DEBUG:boto:Token: None 2024-04-16T17:49:08.257 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ= 2024-04-16T17:49:08.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ=', 'Content-Length': '0'} 2024-04-16T17:49:08.260 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000233f81ee2fd9003f-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.302 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.302 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.303 DEBUG:boto:path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.303 DEBUG:boto:auth_path=/lybtug-8/%26?versionId=null 2024-04-16T17:49:08.303 DEBUG:boto:Method: GET 2024-04-16T17:49:08.303 DEBUG:boto:Path: /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.303 DEBUG:boto:Data: 2024-04-16T17:49:08.303 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.303 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.303 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.303 DEBUG:boto:Params: {} 2024-04-16T17:49:08.303 DEBUG:boto:Token: None 2024-04-16T17:49:08.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26?versionId=null 2024-04-16T17:49:08.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ= 2024-04-16T17:49:08.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U0nDglInkI4kLl2EYJZsVZvFZFQ=', 'Content-Length': '0'} 2024-04-16T17:49:08.307 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b943d82345396ffc-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.307 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.307 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.307 DEBUG:boto:path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.307 DEBUG:boto:auth_path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.308 DEBUG:boto:Method: GET 2024-04-16T17:49:08.308 DEBUG:boto:Path: /lybtug-8/?prefix=%26 2024-04-16T17:49:08.308 DEBUG:boto:Data: 2024-04-16T17:49:08.308 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.308 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.308 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.308 DEBUG:boto:Params: {} 2024-04-16T17:49:08.308 DEBUG:boto:Token: None 2024-04-16T17:49:08.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000524d62a2e3d94722-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.354 DEBUG:boto:b'lybtug-8&1000false&2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.355 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.355 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.355 DEBUG:boto:path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.355 DEBUG:boto:auth_path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.355 DEBUG:boto:Method: GET 2024-04-16T17:49:08.355 DEBUG:boto:Path: /lybtug-8/?prefix=%26 2024-04-16T17:49:08.355 DEBUG:boto:Data: 2024-04-16T17:49:08.355 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.355 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.356 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.356 DEBUG:boto:Params: {} 2024-04-16T17:49:08.356 DEBUG:boto:Token: None 2024-04-16T17:49:08.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000958963df87d4a208-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.360 DEBUG:boto:b'lybtug-8&1000false&2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.360 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.361 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.361 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.361 DEBUG:boto:Path: /lybtug-8/%26 2024-04-16T17:49:08.361 DEBUG:boto:Data: 2024-04-16T17:49:08.361 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.361 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.361 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.361 DEBUG:boto:Params: {} 2024-04-16T17:49:08.361 DEBUG:boto:Token: None 2024-04-16T17:49:08.361 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26 2024-04-16T17:49:08.362 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LGP/j/wTEzeV3BfRtuyp//MZG70= 2024-04-16T17:49:08.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LGP/j/wTEzeV3BfRtuyp//MZG70=', 'Content-Length': '0'} 2024-04-16T17:49:08.365 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c591b538abdda6f9-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.365 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.365 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.365 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.365 DEBUG:boto:Path: /lybtug-8/%26 2024-04-16T17:49:08.365 DEBUG:boto:Data: 2024-04-16T17:49:08.366 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.366 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.366 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.366 DEBUG:boto:Params: {} 2024-04-16T17:49:08.366 DEBUG:boto:Token: None 2024-04-16T17:49:08.366 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26 2024-04-16T17:49:08.366 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LGP/j/wTEzeV3BfRtuyp//MZG70= 2024-04-16T17:49:08.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LGP/j/wTEzeV3BfRtuyp//MZG70=', 'Content-Length': '0'} 2024-04-16T17:49:08.369 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001d474658ef9a4068-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.369 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2024-04-16T17:49:08.370 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.370 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.370 DEBUG:boto:Method: GET 2024-04-16T17:49:08.370 DEBUG:boto:Path: /lybtug-8/%26 2024-04-16T17:49:08.370 DEBUG:boto:Data: 2024-04-16T17:49:08.370 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.370 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.370 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.370 DEBUG:boto:Params: {} 2024-04-16T17:49:08.370 DEBUG:boto:Token: None 2024-04-16T17:49:08.370 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26 2024-04-16T17:49:08.371 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0kNCm/dkH8182RVVLTdiCdVnYgU= 2024-04-16T17:49:08.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0kNCm/dkH8182RVVLTdiCdVnYgU=', 'Content-Length': '0'} 2024-04-16T17:49:08.374 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d7b123d07e8b7980-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.414 DEBUG:boto:path=/lybtug-8/%26 2024-04-16T17:49:08.414 DEBUG:boto:auth_path=/lybtug-8/%26 2024-04-16T17:49:08.415 DEBUG:boto:Method: GET 2024-04-16T17:49:08.415 DEBUG:boto:Path: /lybtug-8/%26 2024-04-16T17:49:08.415 DEBUG:boto:Data: 2024-04-16T17:49:08.415 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.415 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.415 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.415 DEBUG:boto:Params: {} 2024-04-16T17:49:08.415 DEBUG:boto:Token: None 2024-04-16T17:49:08.415 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/%26 2024-04-16T17:49:08.415 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0kNCm/dkH8182RVVLTdiCdVnYgU= 2024-04-16T17:49:08.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0kNCm/dkH8182RVVLTdiCdVnYgU=', 'Content-Length': '0'} 2024-04-16T17:49:08.418 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cff3e5cdeb1c3d57-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.419 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.419 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.419 DEBUG:boto:path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.419 DEBUG:boto:auth_path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.419 DEBUG:boto:Method: GET 2024-04-16T17:49:08.419 DEBUG:boto:Path: /lybtug-8/?prefix=%26 2024-04-16T17:49:08.419 DEBUG:boto:Data: 2024-04-16T17:49:08.420 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.420 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.420 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.420 DEBUG:boto:Params: {} 2024-04-16T17:49:08.420 DEBUG:boto:Token: None 2024-04-16T17:49:08.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084c8bf089bcae5b3-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.466 DEBUG:boto:b'lybtug-8&1000false&2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.467 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.467 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.467 DEBUG:boto:path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.467 DEBUG:boto:auth_path=/lybtug-8/?prefix=%26 2024-04-16T17:49:08.467 DEBUG:boto:Method: GET 2024-04-16T17:49:08.467 DEBUG:boto:Path: /lybtug-8/?prefix=%26 2024-04-16T17:49:08.467 DEBUG:boto:Data: 2024-04-16T17:49:08.467 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.467 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.468 DEBUG:boto:Params: {} 2024-04-16T17:49:08.468 DEBUG:boto:Token: None 2024-04-16T17:49:08.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.472 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9bec15a8e18a37f-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.472 DEBUG:boto:b'lybtug-8&1000false&2024-04-16T17:48:50.538Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.472 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=. 2024-04-16T17:49:08.473 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.473 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.473 DEBUG:boto:path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.473 DEBUG:boto:auth_path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.473 DEBUG:boto:Method: GET 2024-04-16T17:49:08.473 DEBUG:boto:Path: /lybtug-8/.?versionId=null 2024-04-16T17:49:08.473 DEBUG:boto:Data: 2024-04-16T17:49:08.473 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.473 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.473 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.473 DEBUG:boto:Params: {} 2024-04-16T17:49:08.473 DEBUG:boto:Token: None 2024-04-16T17:49:08.474 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.?versionId=null 2024-04-16T17:49:08.474 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY= 2024-04-16T17:49:08.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY=', 'Content-Length': '0'} 2024-04-16T17:49:08.477 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000019fd77a3b77dddb7-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.477 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.477 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.477 DEBUG:boto:path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.477 DEBUG:boto:auth_path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.477 DEBUG:boto:Method: GET 2024-04-16T17:49:08.478 DEBUG:boto:Path: /lybtug-8/.?versionId=null 2024-04-16T17:49:08.478 DEBUG:boto:Data: 2024-04-16T17:49:08.478 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.478 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.478 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.478 DEBUG:boto:Params: {} 2024-04-16T17:49:08.478 DEBUG:boto:Token: None 2024-04-16T17:49:08.478 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.?versionId=null 2024-04-16T17:49:08.478 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY= 2024-04-16T17:49:08.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY=', 'Content-Length': '0'} 2024-04-16T17:49:08.481 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000701466dd568acfbc-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.482 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.482 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.482 DEBUG:boto:path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.482 DEBUG:boto:auth_path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.482 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.482 DEBUG:boto:Path: /lybtug-8/.?versionId=null 2024-04-16T17:49:08.482 DEBUG:boto:Data: 2024-04-16T17:49:08.482 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.482 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.483 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.483 DEBUG:boto:Params: {} 2024-04-16T17:49:08.483 DEBUG:boto:Token: None 2024-04-16T17:49:08.483 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.?versionId=null 2024-04-16T17:49:08.483 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eF+WdNhbt8vY+SCrPNm80u1bgaY= 2024-04-16T17:49:08.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eF+WdNhbt8vY+SCrPNm80u1bgaY=', 'Content-Length': '0'} 2024-04-16T17:49:08.486 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b28eaf662102a9d3-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.486 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.486 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.486 DEBUG:boto:path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.486 DEBUG:boto:auth_path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.487 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.487 DEBUG:boto:Path: /lybtug-8/.?versionId=null 2024-04-16T17:49:08.487 DEBUG:boto:Data: 2024-04-16T17:49:08.487 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.487 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.487 DEBUG:boto:Params: {} 2024-04-16T17:49:08.487 DEBUG:boto:Token: None 2024-04-16T17:49:08.487 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.?versionId=null 2024-04-16T17:49:08.487 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eF+WdNhbt8vY+SCrPNm80u1bgaY= 2024-04-16T17:49:08.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eF+WdNhbt8vY+SCrPNm80u1bgaY=', 'Content-Length': '0'} 2024-04-16T17:49:08.490 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000042dd37aa87716d8-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.491 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=. 2024-04-16T17:49:08.491 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.491 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.491 DEBUG:boto:path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.491 DEBUG:boto:auth_path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.491 DEBUG:boto:Method: GET 2024-04-16T17:49:08.491 DEBUG:boto:Path: /lybtug-8/.?versionId=null 2024-04-16T17:49:08.491 DEBUG:boto:Data: 2024-04-16T17:49:08.491 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.491 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.491 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.491 DEBUG:boto:Params: {} 2024-04-16T17:49:08.492 DEBUG:boto:Token: None 2024-04-16T17:49:08.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.?versionId=null 2024-04-16T17:49:08.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY= 2024-04-16T17:49:08.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY=', 'Content-Length': '0'} 2024-04-16T17:49:08.495 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a1333fc6527573c5-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.495 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.495 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.496 DEBUG:boto:path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.496 DEBUG:boto:auth_path=/lybtug-8/.?versionId=null 2024-04-16T17:49:08.496 DEBUG:boto:Method: GET 2024-04-16T17:49:08.496 DEBUG:boto:Path: /lybtug-8/.?versionId=null 2024-04-16T17:49:08.496 DEBUG:boto:Data: 2024-04-16T17:49:08.496 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.496 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.496 DEBUG:boto:Params: {} 2024-04-16T17:49:08.496 DEBUG:boto:Token: None 2024-04-16T17:49:08.496 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.?versionId=null 2024-04-16T17:49:08.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY= 2024-04-16T17:49:08.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q1nP89I76gz/PsP1OQv/9ITU4rY=', 'Content-Length': '0'} 2024-04-16T17:49:08.499 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bf23113931f6385e-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.500 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.500 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.500 DEBUG:boto:path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.500 DEBUG:boto:auth_path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.500 DEBUG:boto:Method: GET 2024-04-16T17:49:08.500 DEBUG:boto:Path: /lybtug-8/?prefix=. 2024-04-16T17:49:08.500 DEBUG:boto:Data: 2024-04-16T17:49:08.500 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.500 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.501 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.501 DEBUG:boto:Params: {} 2024-04-16T17:49:08.501 DEBUG:boto:Token: None 2024-04-16T17:49:08.501 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7218ea61bb5fd9c-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.505 DEBUG:boto:b'lybtug-8.1000false.2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.506 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.506 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.506 DEBUG:boto:path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.506 DEBUG:boto:auth_path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.506 DEBUG:boto:Method: GET 2024-04-16T17:49:08.506 DEBUG:boto:Path: /lybtug-8/?prefix=. 2024-04-16T17:49:08.506 DEBUG:boto:Data: 2024-04-16T17:49:08.506 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.506 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.506 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.506 DEBUG:boto:Params: {} 2024-04-16T17:49:08.507 DEBUG:boto:Token: None 2024-04-16T17:49:08.507 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.507 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.511 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000709d8fec2e3319f9-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.511 DEBUG:boto:b'lybtug-8.1000false.2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.512 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.512 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.512 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.512 DEBUG:boto:Path: /lybtug-8/. 2024-04-16T17:49:08.512 DEBUG:boto:Data: 2024-04-16T17:49:08.512 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.512 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.513 DEBUG:boto:Params: {} 2024-04-16T17:49:08.513 DEBUG:boto:Token: None 2024-04-16T17:49:08.513 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/. 2024-04-16T17:49:08.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RWlndNje6yOLi/OnZBkNSKTy3ag= 2024-04-16T17:49:08.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RWlndNje6yOLi/OnZBkNSKTy3ag=', 'Content-Length': '0'} 2024-04-16T17:49:08.516 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e3d51278640e7a84-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.516 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.516 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.516 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.517 DEBUG:boto:Path: /lybtug-8/. 2024-04-16T17:49:08.517 DEBUG:boto:Data: 2024-04-16T17:49:08.517 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.517 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.517 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.517 DEBUG:boto:Params: {} 2024-04-16T17:49:08.517 DEBUG:boto:Token: None 2024-04-16T17:49:08.517 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/. 2024-04-16T17:49:08.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RWlndNje6yOLi/OnZBkNSKTy3ag= 2024-04-16T17:49:08.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RWlndNje6yOLi/OnZBkNSKTy3ag=', 'Content-Length': '0'} 2024-04-16T17:49:08.520 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f9042ca89e13356a-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=. 2024-04-16T17:49:08.520 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.521 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.521 DEBUG:boto:Method: GET 2024-04-16T17:49:08.521 DEBUG:boto:Path: /lybtug-8/. 2024-04-16T17:49:08.521 DEBUG:boto:Data: 2024-04-16T17:49:08.521 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.521 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.521 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.521 DEBUG:boto:Params: {} 2024-04-16T17:49:08.521 DEBUG:boto:Token: None 2024-04-16T17:49:08.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/. 2024-04-16T17:49:08.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WKZTSf+jAPSIhp3a3EEsoN6xxZo= 2024-04-16T17:49:08.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WKZTSf+jAPSIhp3a3EEsoN6xxZo=', 'Content-Length': '0'} 2024-04-16T17:49:08.524 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c755c7b8840281c9-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.566 DEBUG:boto:path=/lybtug-8/. 2024-04-16T17:49:08.566 DEBUG:boto:auth_path=/lybtug-8/. 2024-04-16T17:49:08.567 DEBUG:boto:Method: GET 2024-04-16T17:49:08.567 DEBUG:boto:Path: /lybtug-8/. 2024-04-16T17:49:08.567 DEBUG:boto:Data: 2024-04-16T17:49:08.567 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.567 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.567 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.567 DEBUG:boto:Params: {} 2024-04-16T17:49:08.567 DEBUG:boto:Token: None 2024-04-16T17:49:08.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/. 2024-04-16T17:49:08.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WKZTSf+jAPSIhp3a3EEsoN6xxZo= 2024-04-16T17:49:08.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WKZTSf+jAPSIhp3a3EEsoN6xxZo=', 'Content-Length': '0'} 2024-04-16T17:49:08.570 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006db9b114bd4c6637-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.571 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.571 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.571 DEBUG:boto:path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.571 DEBUG:boto:auth_path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.571 DEBUG:boto:Method: GET 2024-04-16T17:49:08.571 DEBUG:boto:Path: /lybtug-8/?prefix=. 2024-04-16T17:49:08.571 DEBUG:boto:Data: 2024-04-16T17:49:08.571 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.571 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.571 DEBUG:boto:Params: {} 2024-04-16T17:49:08.572 DEBUG:boto:Token: None 2024-04-16T17:49:08.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053ca7332d179c249-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.576 DEBUG:boto:b'lybtug-8.1000false.2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.576 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.576 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.576 DEBUG:boto:path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.576 DEBUG:boto:auth_path=/lybtug-8/?prefix=. 2024-04-16T17:49:08.576 DEBUG:boto:Method: GET 2024-04-16T17:49:08.576 DEBUG:boto:Path: /lybtug-8/?prefix=. 2024-04-16T17:49:08.577 DEBUG:boto:Data: 2024-04-16T17:49:08.577 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.577 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.577 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.577 DEBUG:boto:Params: {} 2024-04-16T17:49:08.577 DEBUG:boto:Token: None 2024-04-16T17:49:08.577 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.581 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000286a60019de131af-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.581 DEBUG:boto:b'lybtug-8.1000false.2024-04-16T17:48:50.590Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.582 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.. 2024-04-16T17:49:08.582 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.582 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.582 DEBUG:boto:path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.582 DEBUG:boto:auth_path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.582 DEBUG:boto:Method: GET 2024-04-16T17:49:08.582 DEBUG:boto:Path: /lybtug-8/..?versionId=null 2024-04-16T17:49:08.583 DEBUG:boto:Data: 2024-04-16T17:49:08.583 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.583 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.583 DEBUG:boto:Params: {} 2024-04-16T17:49:08.583 DEBUG:boto:Token: None 2024-04-16T17:49:08.583 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/..?versionId=null 2024-04-16T17:49:08.583 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8= 2024-04-16T17:49:08.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8=', 'Content-Length': '0'} 2024-04-16T17:49:08.587 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006419ebf2b33cfa84-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.587 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.587 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.587 DEBUG:boto:path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.587 DEBUG:boto:auth_path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.587 DEBUG:boto:Method: GET 2024-04-16T17:49:08.587 DEBUG:boto:Path: /lybtug-8/..?versionId=null 2024-04-16T17:49:08.587 DEBUG:boto:Data: 2024-04-16T17:49:08.587 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.588 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.588 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.588 DEBUG:boto:Params: {} 2024-04-16T17:49:08.588 DEBUG:boto:Token: None 2024-04-16T17:49:08.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/..?versionId=null 2024-04-16T17:49:08.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8= 2024-04-16T17:49:08.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8=', 'Content-Length': '0'} 2024-04-16T17:49:08.591 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f618b11545c4ff48-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.592 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.592 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.592 DEBUG:boto:path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.592 DEBUG:boto:auth_path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.592 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.592 DEBUG:boto:Path: /lybtug-8/..?versionId=null 2024-04-16T17:49:08.592 DEBUG:boto:Data: 2024-04-16T17:49:08.592 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.592 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.592 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.592 DEBUG:boto:Params: {} 2024-04-16T17:49:08.593 DEBUG:boto:Token: None 2024-04-16T17:49:08.593 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/..?versionId=null 2024-04-16T17:49:08.593 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2SWpJNBJasGty8ovxyRe9hiILnM= 2024-04-16T17:49:08.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2SWpJNBJasGty8ovxyRe9hiILnM=', 'Content-Length': '0'} 2024-04-16T17:49:08.596 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007ef8fa269bfc0ec2-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.596 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.596 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.596 DEBUG:boto:path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.597 DEBUG:boto:auth_path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.597 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.597 DEBUG:boto:Path: /lybtug-8/..?versionId=null 2024-04-16T17:49:08.597 DEBUG:boto:Data: 2024-04-16T17:49:08.597 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.597 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.597 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.597 DEBUG:boto:Params: {} 2024-04-16T17:49:08.597 DEBUG:boto:Token: None 2024-04-16T17:49:08.597 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/..?versionId=null 2024-04-16T17:49:08.597 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2SWpJNBJasGty8ovxyRe9hiILnM= 2024-04-16T17:49:08.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2SWpJNBJasGty8ovxyRe9hiILnM=', 'Content-Length': '0'} 2024-04-16T17:49:08.600 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000088060457e5f8a081-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.601 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.. 2024-04-16T17:49:08.601 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.601 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.601 DEBUG:boto:path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.601 DEBUG:boto:auth_path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.601 DEBUG:boto:Method: GET 2024-04-16T17:49:08.601 DEBUG:boto:Path: /lybtug-8/..?versionId=null 2024-04-16T17:49:08.601 DEBUG:boto:Data: 2024-04-16T17:49:08.601 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.601 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.602 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.602 DEBUG:boto:Params: {} 2024-04-16T17:49:08.602 DEBUG:boto:Token: None 2024-04-16T17:49:08.602 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/..?versionId=null 2024-04-16T17:49:08.602 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8= 2024-04-16T17:49:08.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8=', 'Content-Length': '0'} 2024-04-16T17:49:08.605 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002ac00ea4a3031832-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.646 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.646 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.647 DEBUG:boto:path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.647 DEBUG:boto:auth_path=/lybtug-8/..?versionId=null 2024-04-16T17:49:08.647 DEBUG:boto:Method: GET 2024-04-16T17:49:08.647 DEBUG:boto:Path: /lybtug-8/..?versionId=null 2024-04-16T17:49:08.647 DEBUG:boto:Data: 2024-04-16T17:49:08.647 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.647 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.647 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.647 DEBUG:boto:Params: {} 2024-04-16T17:49:08.647 DEBUG:boto:Token: None 2024-04-16T17:49:08.647 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/..?versionId=null 2024-04-16T17:49:08.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8= 2024-04-16T17:49:08.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GxxypPL1b/TR2T+1Br40ziFRWH8=', 'Content-Length': '0'} 2024-04-16T17:49:08.651 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000016ccc76a613d7d52-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.651 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.651 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.651 DEBUG:boto:path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.651 DEBUG:boto:auth_path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.651 DEBUG:boto:Method: GET 2024-04-16T17:49:08.651 DEBUG:boto:Path: /lybtug-8/?prefix=.. 2024-04-16T17:49:08.652 DEBUG:boto:Data: 2024-04-16T17:49:08.652 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.652 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.652 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.652 DEBUG:boto:Params: {} 2024-04-16T17:49:08.652 DEBUG:boto:Token: None 2024-04-16T17:49:08.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.656 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001631d707eb7198ba-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.656 DEBUG:boto:b'lybtug-8..1000false..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.656 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.656 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.656 DEBUG:boto:path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.657 DEBUG:boto:auth_path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.657 DEBUG:boto:Method: GET 2024-04-16T17:49:08.657 DEBUG:boto:Path: /lybtug-8/?prefix=.. 2024-04-16T17:49:08.657 DEBUG:boto:Data: 2024-04-16T17:49:08.657 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.657 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.657 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.657 DEBUG:boto:Params: {} 2024-04-16T17:49:08.657 DEBUG:boto:Token: None 2024-04-16T17:49:08.657 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.661 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000535568264afd2ab4-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.661 DEBUG:boto:b'lybtug-8..1000false..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.662 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.662 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.662 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.662 DEBUG:boto:Path: /lybtug-8/.. 2024-04-16T17:49:08.662 DEBUG:boto:Data: 2024-04-16T17:49:08.662 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.662 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.662 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.662 DEBUG:boto:Params: {} 2024-04-16T17:49:08.663 DEBUG:boto:Token: None 2024-04-16T17:49:08.663 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.. 2024-04-16T17:49:08.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z6lad6xSKXHmiK74LX9OQc1MBvQ= 2024-04-16T17:49:08.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z6lad6xSKXHmiK74LX9OQc1MBvQ=', 'Content-Length': '0'} 2024-04-16T17:49:08.666 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b690e9016d21e763-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.666 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.666 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.666 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.667 DEBUG:boto:Path: /lybtug-8/.. 2024-04-16T17:49:08.667 DEBUG:boto:Data: 2024-04-16T17:49:08.667 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.667 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.667 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.667 DEBUG:boto:Params: {} 2024-04-16T17:49:08.667 DEBUG:boto:Token: None 2024-04-16T17:49:08.667 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.. 2024-04-16T17:49:08.667 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z6lad6xSKXHmiK74LX9OQc1MBvQ= 2024-04-16T17:49:08.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z6lad6xSKXHmiK74LX9OQc1MBvQ=', 'Content-Length': '0'} 2024-04-16T17:49:08.670 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002ea513c374d57427-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.. 2024-04-16T17:49:08.671 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.671 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.671 DEBUG:boto:Method: GET 2024-04-16T17:49:08.671 DEBUG:boto:Path: /lybtug-8/.. 2024-04-16T17:49:08.671 DEBUG:boto:Data: 2024-04-16T17:49:08.671 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.671 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.671 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.671 DEBUG:boto:Params: {} 2024-04-16T17:49:08.672 DEBUG:boto:Token: None 2024-04-16T17:49:08.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.. 2024-04-16T17:49:08.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qYNiDLciFYzr059PAsSRsAhtlPg= 2024-04-16T17:49:08.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qYNiDLciFYzr059PAsSRsAhtlPg=', 'Content-Length': '0'} 2024-04-16T17:49:08.675 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000019e81f11c4ff8d6d-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.675 DEBUG:boto:path=/lybtug-8/.. 2024-04-16T17:49:08.675 DEBUG:boto:auth_path=/lybtug-8/.. 2024-04-16T17:49:08.675 DEBUG:boto:Method: GET 2024-04-16T17:49:08.675 DEBUG:boto:Path: /lybtug-8/.. 2024-04-16T17:49:08.675 DEBUG:boto:Data: 2024-04-16T17:49:08.676 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.676 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.676 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.676 DEBUG:boto:Params: {} 2024-04-16T17:49:08.676 DEBUG:boto:Token: None 2024-04-16T17:49:08.676 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.. 2024-04-16T17:49:08.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qYNiDLciFYzr059PAsSRsAhtlPg= 2024-04-16T17:49:08.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qYNiDLciFYzr059PAsSRsAhtlPg=', 'Content-Length': '0'} 2024-04-16T17:49:08.679 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000056cf6fa23e17cc1d-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.722 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.722 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.722 DEBUG:boto:path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.723 DEBUG:boto:auth_path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.723 DEBUG:boto:Method: GET 2024-04-16T17:49:08.723 DEBUG:boto:Path: /lybtug-8/?prefix=.. 2024-04-16T17:49:08.723 DEBUG:boto:Data: 2024-04-16T17:49:08.723 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.723 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.723 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.723 DEBUG:boto:Params: {} 2024-04-16T17:49:08.723 DEBUG:boto:Token: None 2024-04-16T17:49:08.723 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.723 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.727 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000163ac31efed59c69-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.727 DEBUG:boto:b'lybtug-8..1000false..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.727 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.728 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.728 DEBUG:boto:path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.728 DEBUG:boto:auth_path=/lybtug-8/?prefix=.. 2024-04-16T17:49:08.728 DEBUG:boto:Method: GET 2024-04-16T17:49:08.728 DEBUG:boto:Path: /lybtug-8/?prefix=.. 2024-04-16T17:49:08.728 DEBUG:boto:Data: 2024-04-16T17:49:08.728 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.728 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.728 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.728 DEBUG:boto:Params: {} 2024-04-16T17:49:08.728 DEBUG:boto:Token: None 2024-04-16T17:49:08.729 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.733 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032118bb3f39034af-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.733 DEBUG:boto:b'lybtug-8..1000false..2024-04-16T17:48:50.617Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.733 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=... 2024-04-16T17:49:08.733 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.733 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.733 DEBUG:boto:path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.733 DEBUG:boto:auth_path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.734 DEBUG:boto:Method: GET 2024-04-16T17:49:08.734 DEBUG:boto:Path: /lybtug-8/...?versionId=null 2024-04-16T17:49:08.734 DEBUG:boto:Data: 2024-04-16T17:49:08.734 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.734 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.734 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.734 DEBUG:boto:Params: {} 2024-04-16T17:49:08.734 DEBUG:boto:Token: None 2024-04-16T17:49:08.734 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/...?versionId=null 2024-04-16T17:49:08.734 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE= 2024-04-16T17:49:08.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE=', 'Content-Length': '0'} 2024-04-16T17:49:08.737 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002c96dd4496d6b455-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.778 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.779 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.779 DEBUG:boto:path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.779 DEBUG:boto:auth_path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.779 DEBUG:boto:Method: GET 2024-04-16T17:49:08.779 DEBUG:boto:Path: /lybtug-8/...?versionId=null 2024-04-16T17:49:08.779 DEBUG:boto:Data: 2024-04-16T17:49:08.779 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.779 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.779 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.779 DEBUG:boto:Params: {} 2024-04-16T17:49:08.779 DEBUG:boto:Token: None 2024-04-16T17:49:08.779 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/...?versionId=null 2024-04-16T17:49:08.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE= 2024-04-16T17:49:08.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE=', 'Content-Length': '0'} 2024-04-16T17:49:08.782 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f045d7a3b18cc85a-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.783 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.783 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.783 DEBUG:boto:path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.783 DEBUG:boto:auth_path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.783 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.783 DEBUG:boto:Path: /lybtug-8/...?versionId=null 2024-04-16T17:49:08.783 DEBUG:boto:Data: 2024-04-16T17:49:08.783 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.784 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.784 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.784 DEBUG:boto:Params: {} 2024-04-16T17:49:08.784 DEBUG:boto:Token: None 2024-04-16T17:49:08.784 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/...?versionId=null 2024-04-16T17:49:08.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nS5YJPa8RdvEEkRA9QUeuFuG3DE= 2024-04-16T17:49:08.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nS5YJPa8RdvEEkRA9QUeuFuG3DE=', 'Content-Length': '0'} 2024-04-16T17:49:08.789 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009b18ae51765882b0-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.790 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.790 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.790 DEBUG:boto:path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.790 DEBUG:boto:auth_path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.790 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.790 DEBUG:boto:Path: /lybtug-8/...?versionId=null 2024-04-16T17:49:08.790 DEBUG:boto:Data: 2024-04-16T17:49:08.790 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.791 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.791 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.791 DEBUG:boto:Params: {} 2024-04-16T17:49:08.791 DEBUG:boto:Token: None 2024-04-16T17:49:08.791 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/...?versionId=null 2024-04-16T17:49:08.791 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nS5YJPa8RdvEEkRA9QUeuFuG3DE= 2024-04-16T17:49:08.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nS5YJPa8RdvEEkRA9QUeuFuG3DE=', 'Content-Length': '0'} 2024-04-16T17:49:08.794 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000061a03d30b7a32de6-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.794 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=... 2024-04-16T17:49:08.794 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.794 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.794 DEBUG:boto:path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.795 DEBUG:boto:auth_path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.795 DEBUG:boto:Method: GET 2024-04-16T17:49:08.795 DEBUG:boto:Path: /lybtug-8/...?versionId=null 2024-04-16T17:49:08.795 DEBUG:boto:Data: 2024-04-16T17:49:08.795 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.795 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.795 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.795 DEBUG:boto:Params: {} 2024-04-16T17:49:08.795 DEBUG:boto:Token: None 2024-04-16T17:49:08.795 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/...?versionId=null 2024-04-16T17:49:08.795 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE= 2024-04-16T17:49:08.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE=', 'Content-Length': '0'} 2024-04-16T17:49:08.798 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000532271e6a83f3bff-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.799 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.799 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.799 DEBUG:boto:path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.799 DEBUG:boto:auth_path=/lybtug-8/...?versionId=null 2024-04-16T17:49:08.799 DEBUG:boto:Method: GET 2024-04-16T17:49:08.799 DEBUG:boto:Path: /lybtug-8/...?versionId=null 2024-04-16T17:49:08.799 DEBUG:boto:Data: 2024-04-16T17:49:08.799 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.799 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.800 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.800 DEBUG:boto:Params: {} 2024-04-16T17:49:08.800 DEBUG:boto:Token: None 2024-04-16T17:49:08.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/...?versionId=null 2024-04-16T17:49:08.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE= 2024-04-16T17:49:08.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXEa2f4p5CMHC1KJ4uVcsmUnVkE=', 'Content-Length': '0'} 2024-04-16T17:49:08.803 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000068c0382da4513957-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.803 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.803 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.803 DEBUG:boto:path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.803 DEBUG:boto:auth_path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.803 DEBUG:boto:Method: GET 2024-04-16T17:49:08.803 DEBUG:boto:Path: /lybtug-8/?prefix=... 2024-04-16T17:49:08.804 DEBUG:boto:Data: 2024-04-16T17:49:08.804 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.804 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.804 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.804 DEBUG:boto:Params: {} 2024-04-16T17:49:08.804 DEBUG:boto:Token: None 2024-04-16T17:49:08.804 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.808 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9e6d24e9a5aa7d6-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.850 DEBUG:boto:b'lybtug-8...1000false...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.851 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.851 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.851 DEBUG:boto:path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.851 DEBUG:boto:auth_path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.851 DEBUG:boto:Method: GET 2024-04-16T17:49:08.851 DEBUG:boto:Path: /lybtug-8/?prefix=... 2024-04-16T17:49:08.851 DEBUG:boto:Data: 2024-04-16T17:49:08.851 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.851 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.851 DEBUG:boto:Params: {} 2024-04-16T17:49:08.852 DEBUG:boto:Token: None 2024-04-16T17:49:08.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dee60499a2db97a4-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.856 DEBUG:boto:b'lybtug-8...1000false...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.856 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.856 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.857 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.857 DEBUG:boto:Path: /lybtug-8/... 2024-04-16T17:49:08.857 DEBUG:boto:Data: 2024-04-16T17:49:08.857 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.857 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.857 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.857 DEBUG:boto:Params: {} 2024-04-16T17:49:08.857 DEBUG:boto:Token: None 2024-04-16T17:49:08.857 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/... 2024-04-16T17:49:08.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pWDWDAycQ9f7may5umDtg64uuJ0= 2024-04-16T17:49:08.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pWDWDAycQ9f7may5umDtg64uuJ0=', 'Content-Length': '0'} 2024-04-16T17:49:08.860 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dd456b187be1f209-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.860 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.861 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.861 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.861 DEBUG:boto:Path: /lybtug-8/... 2024-04-16T17:49:08.861 DEBUG:boto:Data: 2024-04-16T17:49:08.861 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.861 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.861 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.861 DEBUG:boto:Params: {} 2024-04-16T17:49:08.861 DEBUG:boto:Token: None 2024-04-16T17:49:08.861 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/... 2024-04-16T17:49:08.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pWDWDAycQ9f7may5umDtg64uuJ0= 2024-04-16T17:49:08.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pWDWDAycQ9f7may5umDtg64uuJ0=', 'Content-Length': '0'} 2024-04-16T17:49:08.864 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000034ccf91c26bb997-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.865 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=... 2024-04-16T17:49:08.865 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.865 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.865 DEBUG:boto:Method: GET 2024-04-16T17:49:08.865 DEBUG:boto:Path: /lybtug-8/... 2024-04-16T17:49:08.865 DEBUG:boto:Data: 2024-04-16T17:49:08.865 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.865 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.865 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.865 DEBUG:boto:Params: {} 2024-04-16T17:49:08.866 DEBUG:boto:Token: None 2024-04-16T17:49:08.866 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/... 2024-04-16T17:49:08.866 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CSn/EUpH2tKpDzY8XM4y5muK0HU= 2024-04-16T17:49:08.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CSn/EUpH2tKpDzY8XM4y5muK0HU=', 'Content-Length': '0'} 2024-04-16T17:49:08.869 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002823d2166edcea79-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.869 DEBUG:boto:path=/lybtug-8/... 2024-04-16T17:49:08.869 DEBUG:boto:auth_path=/lybtug-8/... 2024-04-16T17:49:08.869 DEBUG:boto:Method: GET 2024-04-16T17:49:08.869 DEBUG:boto:Path: /lybtug-8/... 2024-04-16T17:49:08.869 DEBUG:boto:Data: 2024-04-16T17:49:08.869 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.870 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.870 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.870 DEBUG:boto:Params: {} 2024-04-16T17:49:08.870 DEBUG:boto:Token: None 2024-04-16T17:49:08.870 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/... 2024-04-16T17:49:08.870 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CSn/EUpH2tKpDzY8XM4y5muK0HU= 2024-04-16T17:49:08.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CSn/EUpH2tKpDzY8XM4y5muK0HU=', 'Content-Length': '0'} 2024-04-16T17:49:08.873 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000029dd0c71374419e6-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.914 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.915 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.915 DEBUG:boto:path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.915 DEBUG:boto:auth_path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.915 DEBUG:boto:Method: GET 2024-04-16T17:49:08.915 DEBUG:boto:Path: /lybtug-8/?prefix=... 2024-04-16T17:49:08.915 DEBUG:boto:Data: 2024-04-16T17:49:08.915 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.915 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.915 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.915 DEBUG:boto:Params: {} 2024-04-16T17:49:08.915 DEBUG:boto:Token: None 2024-04-16T17:49:08.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7d82907217e30f8-00661eba14-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.920 DEBUG:boto:b'lybtug-8...1000false...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.920 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:08.920 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:08.920 DEBUG:boto:path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.920 DEBUG:boto:auth_path=/lybtug-8/?prefix=... 2024-04-16T17:49:08.921 DEBUG:boto:Method: GET 2024-04-16T17:49:08.921 DEBUG:boto:Path: /lybtug-8/?prefix=... 2024-04-16T17:49:08.921 DEBUG:boto:Data: 2024-04-16T17:49:08.921 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.921 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.921 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.921 DEBUG:boto:Params: {} 2024-04-16T17:49:08.921 DEBUG:boto:Token: None 2024-04-16T17:49:08.921 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/ 2024-04-16T17:49:08.922 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg= 2024-04-16T17:49:08.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9RhGqfztVw7DtLTotYUHMwRMhg=', 'Content-Length': '0'} 2024-04-16T17:49:08.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c47baf4296730ac0-00661eba14-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.925 DEBUG:boto:b'lybtug-8...1000false...2024-04-16T17:48:50.668Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:08.926 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o 2024-04-16T17:49:08.926 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:08.926 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:08.926 DEBUG:boto:path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.926 DEBUG:boto:auth_path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.926 DEBUG:boto:Method: GET 2024-04-16T17:49:08.926 DEBUG:boto:Path: /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.927 DEBUG:boto:Data: 2024-04-16T17:49:08.927 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.927 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.927 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.927 DEBUG:boto:Params: {} 2024-04-16T17:49:08.927 DEBUG:boto:Token: None 2024-04-16T17:49:08.927 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.927 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII= 2024-04-16T17:49:08.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII=', 'Content-Length': '0'} 2024-04-16T17:49:08.930 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000031aafc57da8eafae-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.974 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:08.974 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:08.975 DEBUG:boto:path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.975 DEBUG:boto:auth_path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.975 DEBUG:boto:Method: GET 2024-04-16T17:49:08.975 DEBUG:boto:Path: /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.975 DEBUG:boto:Data: 2024-04-16T17:49:08.975 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.975 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.975 DEBUG:boto:Params: {} 2024-04-16T17:49:08.975 DEBUG:boto:Token: None 2024-04-16T17:49:08.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII= 2024-04-16T17:49:08.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII=', 'Content-Length': '0'} 2024-04-16T17:49:08.979 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005f2be55ff911d5f7-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.979 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:08.979 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:08.979 DEBUG:boto:path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.980 DEBUG:boto:auth_path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.980 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.980 DEBUG:boto:Path: /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.980 DEBUG:boto:Data: 2024-04-16T17:49:08.980 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.980 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.980 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.980 DEBUG:boto:Params: {} 2024-04-16T17:49:08.980 DEBUG:boto:Token: None 2024-04-16T17:49:08.980 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VWmpM4DdDAjwcOLKZgp7DXD6Bj0= 2024-04-16T17:49:08.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VWmpM4DdDAjwcOLKZgp7DXD6Bj0=', 'Content-Length': '0'} 2024-04-16T17:49:08.983 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009858ffec94b267ac-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.984 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:08.984 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:08.984 DEBUG:boto:path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.984 DEBUG:boto:auth_path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.984 DEBUG:boto:Method: HEAD 2024-04-16T17:49:08.984 DEBUG:boto:Path: /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.984 DEBUG:boto:Data: 2024-04-16T17:49:08.984 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.985 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.985 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.985 DEBUG:boto:Params: {} 2024-04-16T17:49:08.985 DEBUG:boto:Token: None 2024-04-16T17:49:08.985 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VWmpM4DdDAjwcOLKZgp7DXD6Bj0= 2024-04-16T17:49:08.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VWmpM4DdDAjwcOLKZgp7DXD6Bj0=', 'Content-Length': '0'} 2024-04-16T17:49:08.988 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000052c3700deed6e989-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.988 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o 2024-04-16T17:49:08.988 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:08.989 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:08.989 DEBUG:boto:path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.989 DEBUG:boto:auth_path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.989 DEBUG:boto:Method: GET 2024-04-16T17:49:08.989 DEBUG:boto:Path: /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.989 DEBUG:boto:Data: 2024-04-16T17:49:08.989 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.989 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:08.989 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.989 DEBUG:boto:Params: {} 2024-04-16T17:49:08.990 DEBUG:boto:Token: None 2024-04-16T17:49:08.990 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.990 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII= 2024-04-16T17:49:08.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII=', 'Content-Length': '0'} 2024-04-16T17:49:08.993 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000af9881eb4c93d1e1-00661eba14-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:08.993 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:08.993 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:08.993 DEBUG:boto:path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.993 DEBUG:boto:auth_path=/lybtug-8/.o?versionId=null 2024-04-16T17:49:08.993 DEBUG:boto:Method: GET 2024-04-16T17:49:08.994 DEBUG:boto:Path: /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.994 DEBUG:boto:Data: 2024-04-16T17:49:08.994 DEBUG:boto:Headers: {} 2024-04-16T17:49:08.994 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:08.994 DEBUG:boto:Port: 8000 2024-04-16T17:49:08.994 DEBUG:boto:Params: {} 2024-04-16T17:49:08.994 DEBUG:boto:Token: None 2024-04-16T17:49:08.994 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:08 GMT /lybtug-8/.o?versionId=null 2024-04-16T17:49:08.994 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII= 2024-04-16T17:49:08.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NGamCCW93PmUs+egh7rCItrtRII=', 'Content-Length': '0'} 2024-04-16T17:49:08.998 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000592ef1e0cd59701d-00661eba14-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.038 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.039 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.039 DEBUG:boto:path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.039 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.039 DEBUG:boto:Method: GET 2024-04-16T17:49:09.039 DEBUG:boto:Path: /lybtug-8/?prefix=.o 2024-04-16T17:49:09.039 DEBUG:boto:Data: 2024-04-16T17:49:09.039 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.039 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.039 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.039 DEBUG:boto:Params: {} 2024-04-16T17:49:09.039 DEBUG:boto:Token: None 2024-04-16T17:49:09.040 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002727c5f62bccc691-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.044 DEBUG:boto:b'lybtug-8.o1000false.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.044 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.044 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.045 DEBUG:boto:path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.045 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.045 DEBUG:boto:Method: GET 2024-04-16T17:49:09.045 DEBUG:boto:Path: /lybtug-8/?prefix=.o 2024-04-16T17:49:09.045 DEBUG:boto:Data: 2024-04-16T17:49:09.045 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.045 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.045 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.045 DEBUG:boto:Params: {} 2024-04-16T17:49:09.045 DEBUG:boto:Token: None 2024-04-16T17:49:09.045 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.046 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ce8567fef6caf109-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.050 DEBUG:boto:b'lybtug-8.o1000false.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.050 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:09.051 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:09.051 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.051 DEBUG:boto:Path: /lybtug-8/.o 2024-04-16T17:49:09.051 DEBUG:boto:Data: 2024-04-16T17:49:09.051 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.051 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.051 DEBUG:boto:Params: {} 2024-04-16T17:49:09.051 DEBUG:boto:Token: None 2024-04-16T17:49:09.051 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o 2024-04-16T17:49:09.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lQuTZrh9mxLOnj59x3sJm3AUO0w= 2024-04-16T17:49:09.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lQuTZrh9mxLOnj59x3sJm3AUO0w=', 'Content-Length': '0'} 2024-04-16T17:49:09.054 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f36878a251f1d248-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.054 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:09.054 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:09.055 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.055 DEBUG:boto:Path: /lybtug-8/.o 2024-04-16T17:49:09.055 DEBUG:boto:Data: 2024-04-16T17:49:09.055 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.055 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.055 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.055 DEBUG:boto:Params: {} 2024-04-16T17:49:09.055 DEBUG:boto:Token: None 2024-04-16T17:49:09.055 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o 2024-04-16T17:49:09.055 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lQuTZrh9mxLOnj59x3sJm3AUO0w= 2024-04-16T17:49:09.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lQuTZrh9mxLOnj59x3sJm3AUO0w=', 'Content-Length': '0'} 2024-04-16T17:49:09.058 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000037aad49e022f0c11-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.059 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o 2024-04-16T17:49:09.059 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:09.059 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:09.059 DEBUG:boto:Method: GET 2024-04-16T17:49:09.059 DEBUG:boto:Path: /lybtug-8/.o 2024-04-16T17:49:09.059 DEBUG:boto:Data: 2024-04-16T17:49:09.059 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.059 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.059 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.059 DEBUG:boto:Params: {} 2024-04-16T17:49:09.060 DEBUG:boto:Token: None 2024-04-16T17:49:09.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o 2024-04-16T17:49:09.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pcqDhAHS36ktt1XKMWZTjEFyrWY= 2024-04-16T17:49:09.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pcqDhAHS36ktt1XKMWZTjEFyrWY=', 'Content-Length': '0'} 2024-04-16T17:49:09.062 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001c91a55849b67409-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.106 DEBUG:boto:path=/lybtug-8/.o 2024-04-16T17:49:09.106 DEBUG:boto:auth_path=/lybtug-8/.o 2024-04-16T17:49:09.107 DEBUG:boto:Method: GET 2024-04-16T17:49:09.107 DEBUG:boto:Path: /lybtug-8/.o 2024-04-16T17:49:09.107 DEBUG:boto:Data: 2024-04-16T17:49:09.107 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.107 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.107 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.107 DEBUG:boto:Params: {} 2024-04-16T17:49:09.107 DEBUG:boto:Token: None 2024-04-16T17:49:09.107 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o 2024-04-16T17:49:09.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pcqDhAHS36ktt1XKMWZTjEFyrWY= 2024-04-16T17:49:09.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pcqDhAHS36ktt1XKMWZTjEFyrWY=', 'Content-Length': '0'} 2024-04-16T17:49:09.111 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e4b25dbf618b92ed-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.111 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.111 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.111 DEBUG:boto:path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.111 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.111 DEBUG:boto:Method: GET 2024-04-16T17:49:09.111 DEBUG:boto:Path: /lybtug-8/?prefix=.o 2024-04-16T17:49:09.111 DEBUG:boto:Data: 2024-04-16T17:49:09.112 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.112 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.112 DEBUG:boto:Params: {} 2024-04-16T17:49:09.112 DEBUG:boto:Token: None 2024-04-16T17:49:09.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.116 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071cb8c0e8d83a4a8-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.116 DEBUG:boto:b'lybtug-8.o1000false.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.117 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.117 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.117 DEBUG:boto:path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.117 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o 2024-04-16T17:49:09.117 DEBUG:boto:Method: GET 2024-04-16T17:49:09.117 DEBUG:boto:Path: /lybtug-8/?prefix=.o 2024-04-16T17:49:09.117 DEBUG:boto:Data: 2024-04-16T17:49:09.117 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.118 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.118 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.118 DEBUG:boto:Params: {} 2024-04-16T17:49:09.118 DEBUG:boto:Token: None 2024-04-16T17:49:09.118 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.118 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.122 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c535d64b8e3bef19-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.166 DEBUG:boto:b'lybtug-8.o1000false.o2024-04-16T17:48:50.684Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.167 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o. 2024-04-16T17:49:09.167 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.167 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.167 DEBUG:boto:path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.167 DEBUG:boto:auth_path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.168 DEBUG:boto:Method: GET 2024-04-16T17:49:09.168 DEBUG:boto:Path: /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.168 DEBUG:boto:Data: 2024-04-16T17:49:09.168 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.168 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.168 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.168 DEBUG:boto:Params: {} 2024-04-16T17:49:09.168 DEBUG:boto:Token: None 2024-04-16T17:49:09.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI= 2024-04-16T17:49:09.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI=', 'Content-Length': '0'} 2024-04-16T17:49:09.172 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007c525965138e8f9b-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.172 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.172 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.172 DEBUG:boto:path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.172 DEBUG:boto:auth_path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.173 DEBUG:boto:Method: GET 2024-04-16T17:49:09.173 DEBUG:boto:Path: /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.173 DEBUG:boto:Data: 2024-04-16T17:49:09.173 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.173 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.173 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.173 DEBUG:boto:Params: {} 2024-04-16T17:49:09.173 DEBUG:boto:Token: None 2024-04-16T17:49:09.173 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.173 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI= 2024-04-16T17:49:09.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI=', 'Content-Length': '0'} 2024-04-16T17:49:09.176 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a0809ec1b66d9a41-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.176 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.177 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.177 DEBUG:boto:path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.177 DEBUG:boto:auth_path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.177 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.177 DEBUG:boto:Path: /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.177 DEBUG:boto:Data: 2024-04-16T17:49:09.177 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.177 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.177 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.177 DEBUG:boto:Params: {} 2024-04-16T17:49:09.177 DEBUG:boto:Token: None 2024-04-16T17:49:09.177 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.178 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RHNyJUrgWjJ9jXVYHGq0wmVnyB8= 2024-04-16T17:49:09.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RHNyJUrgWjJ9jXVYHGq0wmVnyB8=', 'Content-Length': '0'} 2024-04-16T17:49:09.180 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008c8033a7377fbeee-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.181 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.181 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.181 DEBUG:boto:path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.181 DEBUG:boto:auth_path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.181 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.181 DEBUG:boto:Path: /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.181 DEBUG:boto:Data: 2024-04-16T17:49:09.181 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.181 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.181 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.181 DEBUG:boto:Params: {} 2024-04-16T17:49:09.182 DEBUG:boto:Token: None 2024-04-16T17:49:09.182 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.182 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RHNyJUrgWjJ9jXVYHGq0wmVnyB8= 2024-04-16T17:49:09.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RHNyJUrgWjJ9jXVYHGq0wmVnyB8=', 'Content-Length': '0'} 2024-04-16T17:49:09.185 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005b3fc52982549eb0-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.185 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o. 2024-04-16T17:49:09.185 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.185 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.185 DEBUG:boto:path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.185 DEBUG:boto:auth_path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.185 DEBUG:boto:Method: GET 2024-04-16T17:49:09.186 DEBUG:boto:Path: /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.186 DEBUG:boto:Data: 2024-04-16T17:49:09.186 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.186 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.186 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.186 DEBUG:boto:Params: {} 2024-04-16T17:49:09.186 DEBUG:boto:Token: None 2024-04-16T17:49:09.186 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.186 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI= 2024-04-16T17:49:09.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI=', 'Content-Length': '0'} 2024-04-16T17:49:09.189 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b438e9f3382e00f0-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.190 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.190 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.190 DEBUG:boto:path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.190 DEBUG:boto:auth_path=/lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.190 DEBUG:boto:Method: GET 2024-04-16T17:49:09.190 DEBUG:boto:Path: /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.190 DEBUG:boto:Data: 2024-04-16T17:49:09.190 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.190 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.191 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.191 DEBUG:boto:Params: {} 2024-04-16T17:49:09.191 DEBUG:boto:Token: None 2024-04-16T17:49:09.191 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o.?versionId=null 2024-04-16T17:49:09.191 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI= 2024-04-16T17:49:09.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dv/bDhCk86Vk18UCoczRjS381FI=', 'Content-Length': '0'} 2024-04-16T17:49:09.194 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000623c4ff4425aa589-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.194 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.194 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.194 DEBUG:boto:path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.195 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.195 DEBUG:boto:Method: GET 2024-04-16T17:49:09.195 DEBUG:boto:Path: /lybtug-8/?prefix=.o. 2024-04-16T17:49:09.195 DEBUG:boto:Data: 2024-04-16T17:49:09.195 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.197 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.197 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.197 DEBUG:boto:Params: {} 2024-04-16T17:49:09.197 DEBUG:boto:Token: None 2024-04-16T17:49:09.198 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.198 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064919ddd15e16848-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.202 DEBUG:boto:b'lybtug-8.o.1000false.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.202 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.202 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.202 DEBUG:boto:path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.202 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.203 DEBUG:boto:Method: GET 2024-04-16T17:49:09.203 DEBUG:boto:Path: /lybtug-8/?prefix=.o. 2024-04-16T17:49:09.203 DEBUG:boto:Data: 2024-04-16T17:49:09.203 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.203 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.203 DEBUG:boto:Params: {} 2024-04-16T17:49:09.203 DEBUG:boto:Token: None 2024-04-16T17:49:09.203 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.203 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.207 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa22f96033bc4a1b-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.207 DEBUG:boto:b'lybtug-8.o.1000false.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.208 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.208 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.208 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.208 DEBUG:boto:Path: /lybtug-8/.o. 2024-04-16T17:49:09.208 DEBUG:boto:Data: 2024-04-16T17:49:09.208 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.208 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.208 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.208 DEBUG:boto:Params: {} 2024-04-16T17:49:09.209 DEBUG:boto:Token: None 2024-04-16T17:49:09.209 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o. 2024-04-16T17:49:09.209 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KVprhHzX6QsvB5GmJDsjf5fr7yc= 2024-04-16T17:49:09.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KVprhHzX6QsvB5GmJDsjf5fr7yc=', 'Content-Length': '0'} 2024-04-16T17:49:09.211 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000021e1f533ebcae16f-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.212 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.212 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.212 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.212 DEBUG:boto:Path: /lybtug-8/.o. 2024-04-16T17:49:09.212 DEBUG:boto:Data: 2024-04-16T17:49:09.212 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.212 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.212 DEBUG:boto:Params: {} 2024-04-16T17:49:09.213 DEBUG:boto:Token: None 2024-04-16T17:49:09.213 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o. 2024-04-16T17:49:09.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KVprhHzX6QsvB5GmJDsjf5fr7yc= 2024-04-16T17:49:09.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KVprhHzX6QsvB5GmJDsjf5fr7yc=', 'Content-Length': '0'} 2024-04-16T17:49:09.216 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c3765b9411168dc3-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=.o. 2024-04-16T17:49:09.216 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.216 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.216 DEBUG:boto:Method: GET 2024-04-16T17:49:09.216 DEBUG:boto:Path: /lybtug-8/.o. 2024-04-16T17:49:09.216 DEBUG:boto:Data: 2024-04-16T17:49:09.216 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.217 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.217 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.217 DEBUG:boto:Params: {} 2024-04-16T17:49:09.217 DEBUG:boto:Token: None 2024-04-16T17:49:09.217 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o. 2024-04-16T17:49:09.217 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9cd3vCLWZWRelGUBJ7f1SxEansM= 2024-04-16T17:49:09.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9cd3vCLWZWRelGUBJ7f1SxEansM=', 'Content-Length': '0'} 2024-04-16T17:49:09.220 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004a930f504f59b036-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.262 DEBUG:boto:path=/lybtug-8/.o. 2024-04-16T17:49:09.263 DEBUG:boto:auth_path=/lybtug-8/.o. 2024-04-16T17:49:09.263 DEBUG:boto:Method: GET 2024-04-16T17:49:09.263 DEBUG:boto:Path: /lybtug-8/.o. 2024-04-16T17:49:09.263 DEBUG:boto:Data: 2024-04-16T17:49:09.263 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.263 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.263 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.263 DEBUG:boto:Params: {} 2024-04-16T17:49:09.263 DEBUG:boto:Token: None 2024-04-16T17:49:09.263 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/.o. 2024-04-16T17:49:09.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9cd3vCLWZWRelGUBJ7f1SxEansM= 2024-04-16T17:49:09.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9cd3vCLWZWRelGUBJ7f1SxEansM=', 'Content-Length': '0'} 2024-04-16T17:49:09.266 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000054e6bddc9ddc14eb-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.267 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.267 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.267 DEBUG:boto:path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.267 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.267 DEBUG:boto:Method: GET 2024-04-16T17:49:09.267 DEBUG:boto:Path: /lybtug-8/?prefix=.o. 2024-04-16T17:49:09.267 DEBUG:boto:Data: 2024-04-16T17:49:09.267 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.267 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.267 DEBUG:boto:Params: {} 2024-04-16T17:49:09.268 DEBUG:boto:Token: None 2024-04-16T17:49:09.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8173a529992e3f5-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.272 DEBUG:boto:b'lybtug-8.o.1000false.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.272 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.272 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.272 DEBUG:boto:path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.272 DEBUG:boto:auth_path=/lybtug-8/?prefix=.o. 2024-04-16T17:49:09.272 DEBUG:boto:Method: GET 2024-04-16T17:49:09.272 DEBUG:boto:Path: /lybtug-8/?prefix=.o. 2024-04-16T17:49:09.272 DEBUG:boto:Data: 2024-04-16T17:49:09.273 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.273 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.273 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.273 DEBUG:boto:Params: {} 2024-04-16T17:49:09.273 DEBUG:boto:Token: None 2024-04-16T17:49:09.273 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.277 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bdb32d2b05eadf33-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.277 DEBUG:boto:b'lybtug-8.o.1000false.o.2024-04-16T17:48:50.733Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2024-04-16T17:49:09.277 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.278 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.278 DEBUG:boto:path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.278 DEBUG:boto:auth_path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.278 DEBUG:boto:Method: GET 2024-04-16T17:49:09.278 DEBUG:boto:Path: /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.278 DEBUG:boto:Data: 2024-04-16T17:49:09.278 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.278 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.278 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.278 DEBUG:boto:Params: {} 2024-04-16T17:49:09.278 DEBUG:boto:Token: None 2024-04-16T17:49:09.279 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.279 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ= 2024-04-16T17:49:09.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.281 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000be38a93c7afdb14b-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.282 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.282 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.282 DEBUG:boto:path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.282 DEBUG:boto:auth_path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.282 DEBUG:boto:Method: GET 2024-04-16T17:49:09.282 DEBUG:boto:Path: /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.282 DEBUG:boto:Data: 2024-04-16T17:49:09.282 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.282 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.283 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.283 DEBUG:boto:Params: {} 2024-04-16T17:49:09.283 DEBUG:boto:Token: None 2024-04-16T17:49:09.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ= 2024-04-16T17:49:09.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.286 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003c9bbbb2b6119c35-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.286 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.286 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.286 DEBUG:boto:path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.286 DEBUG:boto:auth_path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.287 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.287 DEBUG:boto:Path: /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.287 DEBUG:boto:Data: 2024-04-16T17:49:09.287 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.287 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.287 DEBUG:boto:Params: {} 2024-04-16T17:49:09.287 DEBUG:boto:Token: None 2024-04-16T17:49:09.287 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D3bRQQbvHO7Rs1cynYIYXbZtAZs= 2024-04-16T17:49:09.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D3bRQQbvHO7Rs1cynYIYXbZtAZs=', 'Content-Length': '0'} 2024-04-16T17:49:09.290 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000005ab382d413d8b78-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.291 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.291 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.291 DEBUG:boto:path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.291 DEBUG:boto:auth_path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.291 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.291 DEBUG:boto:Path: /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.291 DEBUG:boto:Data: 2024-04-16T17:49:09.291 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.291 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.291 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.292 DEBUG:boto:Params: {} 2024-04-16T17:49:09.292 DEBUG:boto:Token: None 2024-04-16T17:49:09.292 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D3bRQQbvHO7Rs1cynYIYXbZtAZs= 2024-04-16T17:49:09.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D3bRQQbvHO7Rs1cynYIYXbZtAZs=', 'Content-Length': '0'} 2024-04-16T17:49:09.295 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002dfc48fee448492d-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2024-04-16T17:49:09.295 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.295 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.295 DEBUG:boto:path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.295 DEBUG:boto:auth_path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.295 DEBUG:boto:Method: GET 2024-04-16T17:49:09.295 DEBUG:boto:Path: /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.296 DEBUG:boto:Data: 2024-04-16T17:49:09.296 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.296 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.296 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.296 DEBUG:boto:Params: {} 2024-04-16T17:49:09.296 DEBUG:boto:Token: None 2024-04-16T17:49:09.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ= 2024-04-16T17:49:09.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.300 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c02cab1468780563-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.342 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.342 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.343 DEBUG:boto:path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.343 DEBUG:boto:auth_path=/lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.343 DEBUG:boto:Method: GET 2024-04-16T17:49:09.343 DEBUG:boto:Path: /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.343 DEBUG:boto:Data: 2024-04-16T17:49:09.343 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.343 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.343 DEBUG:boto:Params: {} 2024-04-16T17:49:09.344 DEBUG:boto:Token: None 2024-04-16T17:49:09.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A?versionId=null 2024-04-16T17:49:09.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ= 2024-04-16T17:49:09.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1HdBBaiygjVs63GidUzguwuhguQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.347 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c6feb2f5db49f3b-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.347 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.347 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.347 DEBUG:boto:path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.347 DEBUG:boto:auth_path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.348 DEBUG:boto:Method: GET 2024-04-16T17:49:09.348 DEBUG:boto:Path: /lybtug-8/?prefix=%3A 2024-04-16T17:49:09.348 DEBUG:boto:Data: 2024-04-16T17:49:09.348 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.348 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.348 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.348 DEBUG:boto:Params: {} 2024-04-16T17:49:09.348 DEBUG:boto:Token: None 2024-04-16T17:49:09.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069d3af5c69eebb08-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.352 DEBUG:boto:b'lybtug-8:1000false:2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.353 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.353 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.353 DEBUG:boto:path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.353 DEBUG:boto:auth_path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.353 DEBUG:boto:Method: GET 2024-04-16T17:49:09.353 DEBUG:boto:Path: /lybtug-8/?prefix=%3A 2024-04-16T17:49:09.353 DEBUG:boto:Data: 2024-04-16T17:49:09.353 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.353 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.354 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.354 DEBUG:boto:Params: {} 2024-04-16T17:49:09.354 DEBUG:boto:Token: None 2024-04-16T17:49:09.354 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.354 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bdecbbc90aa0077-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.401 DEBUG:boto:b'lybtug-8:1000false:2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.402 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.402 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.402 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.402 DEBUG:boto:Path: /lybtug-8/%3A 2024-04-16T17:49:09.402 DEBUG:boto:Data: 2024-04-16T17:49:09.402 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.402 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.402 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.402 DEBUG:boto:Params: {} 2024-04-16T17:49:09.402 DEBUG:boto:Token: None 2024-04-16T17:49:09.403 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A 2024-04-16T17:49:09.403 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0be6P8gaNIwZzK9wyKZrcQZUXMI= 2024-04-16T17:49:09.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0be6P8gaNIwZzK9wyKZrcQZUXMI=', 'Content-Length': '0'} 2024-04-16T17:49:09.406 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000832799212cd0789c-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.406 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.406 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.406 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.406 DEBUG:boto:Path: /lybtug-8/%3A 2024-04-16T17:49:09.406 DEBUG:boto:Data: 2024-04-16T17:49:09.406 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.407 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.407 DEBUG:boto:Params: {} 2024-04-16T17:49:09.407 DEBUG:boto:Token: None 2024-04-16T17:49:09.407 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A 2024-04-16T17:49:09.407 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0be6P8gaNIwZzK9wyKZrcQZUXMI= 2024-04-16T17:49:09.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0be6P8gaNIwZzK9wyKZrcQZUXMI=', 'Content-Length': '0'} 2024-04-16T17:49:09.410 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a68ca6721dd72afa-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.410 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2024-04-16T17:49:09.411 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.411 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.411 DEBUG:boto:Method: GET 2024-04-16T17:49:09.411 DEBUG:boto:Path: /lybtug-8/%3A 2024-04-16T17:49:09.411 DEBUG:boto:Data: 2024-04-16T17:49:09.411 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.411 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.411 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.411 DEBUG:boto:Params: {} 2024-04-16T17:49:09.411 DEBUG:boto:Token: None 2024-04-16T17:49:09.411 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A 2024-04-16T17:49:09.411 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zChp3FRcMta2xD76ErJRcPt47HQ= 2024-04-16T17:49:09.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zChp3FRcMta2xD76ErJRcPt47HQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.414 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000079fb4b895a4cf79c-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.415 DEBUG:boto:path=/lybtug-8/%3A 2024-04-16T17:49:09.415 DEBUG:boto:auth_path=/lybtug-8/%3A 2024-04-16T17:49:09.415 DEBUG:boto:Method: GET 2024-04-16T17:49:09.415 DEBUG:boto:Path: /lybtug-8/%3A 2024-04-16T17:49:09.415 DEBUG:boto:Data: 2024-04-16T17:49:09.415 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.415 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.415 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.415 DEBUG:boto:Params: {} 2024-04-16T17:49:09.415 DEBUG:boto:Token: None 2024-04-16T17:49:09.416 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/%3A 2024-04-16T17:49:09.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zChp3FRcMta2xD76ErJRcPt47HQ= 2024-04-16T17:49:09.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zChp3FRcMta2xD76ErJRcPt47HQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.418 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d832c6207a135525-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.419 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.419 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.419 DEBUG:boto:path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.419 DEBUG:boto:auth_path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.419 DEBUG:boto:Method: GET 2024-04-16T17:49:09.419 DEBUG:boto:Path: /lybtug-8/?prefix=%3A 2024-04-16T17:49:09.419 DEBUG:boto:Data: 2024-04-16T17:49:09.419 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.419 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.419 DEBUG:boto:Params: {} 2024-04-16T17:49:09.420 DEBUG:boto:Token: None 2024-04-16T17:49:09.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.423 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079eb41f3a0c2b1d5-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.424 DEBUG:boto:b'lybtug-8:1000false:2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.424 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.424 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.424 DEBUG:boto:path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.424 DEBUG:boto:auth_path=/lybtug-8/?prefix=%3A 2024-04-16T17:49:09.424 DEBUG:boto:Method: GET 2024-04-16T17:49:09.424 DEBUG:boto:Path: /lybtug-8/?prefix=%3A 2024-04-16T17:49:09.425 DEBUG:boto:Data: 2024-04-16T17:49:09.425 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.425 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.425 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.425 DEBUG:boto:Params: {} 2024-04-16T17:49:09.425 DEBUG:boto:Token: None 2024-04-16T17:49:09.425 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.425 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb697d0b277edbd8-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.430 DEBUG:boto:b'lybtug-8:1000false:2024-04-16T17:48:50.511Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.430 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2024-04-16T17:49:09.431 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.431 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.431 DEBUG:boto:path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.431 DEBUG:boto:auth_path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.431 DEBUG:boto:Method: GET 2024-04-16T17:49:09.431 DEBUG:boto:Path: /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.431 DEBUG:boto:Data: 2024-04-16T17:49:09.431 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.431 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.431 DEBUG:boto:Params: {} 2024-04-16T17:49:09.432 DEBUG:boto:Token: None 2024-04-16T17:49:09.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ= 2024-04-16T17:49:09.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.435 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007541540dd2e73690-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.478 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.479 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.479 DEBUG:boto:path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.479 DEBUG:boto:auth_path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.479 DEBUG:boto:Method: GET 2024-04-16T17:49:09.479 DEBUG:boto:Path: /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.479 DEBUG:boto:Data: 2024-04-16T17:49:09.479 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.479 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.479 DEBUG:boto:Params: {} 2024-04-16T17:49:09.479 DEBUG:boto:Token: None 2024-04-16T17:49:09.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ= 2024-04-16T17:49:09.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.482 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000026741f520998ae71-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.483 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.483 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.483 DEBUG:boto:path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.483 DEBUG:boto:auth_path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.483 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.483 DEBUG:boto:Path: /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.483 DEBUG:boto:Data: 2024-04-16T17:49:09.483 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.484 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.484 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.484 DEBUG:boto:Params: {} 2024-04-16T17:49:09.484 DEBUG:boto:Token: None 2024-04-16T17:49:09.484 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:llIqdGBZn3jkYAHQExbYL1+IYY0= 2024-04-16T17:49:09.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:llIqdGBZn3jkYAHQExbYL1+IYY0=', 'Content-Length': '0'} 2024-04-16T17:49:09.487 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002cab8016ac1f3b1e-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.487 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.487 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.487 DEBUG:boto:path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.487 DEBUG:boto:auth_path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.487 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.488 DEBUG:boto:Path: /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.488 DEBUG:boto:Data: 2024-04-16T17:49:09.488 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.488 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.488 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.488 DEBUG:boto:Params: {} 2024-04-16T17:49:09.488 DEBUG:boto:Token: None 2024-04-16T17:49:09.488 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:llIqdGBZn3jkYAHQExbYL1+IYY0= 2024-04-16T17:49:09.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:llIqdGBZn3jkYAHQExbYL1+IYY0=', 'Content-Length': '0'} 2024-04-16T17:49:09.491 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002cfa29ec83c4d54b-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.491 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2024-04-16T17:49:09.491 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.491 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.492 DEBUG:boto:path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.492 DEBUG:boto:auth_path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.492 DEBUG:boto:Method: GET 2024-04-16T17:49:09.492 DEBUG:boto:Path: /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.492 DEBUG:boto:Data: 2024-04-16T17:49:09.492 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.492 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.492 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.492 DEBUG:boto:Params: {} 2024-04-16T17:49:09.492 DEBUG:boto:Token: None 2024-04-16T17:49:09.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ= 2024-04-16T17:49:09.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.495 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000010942f189ef3105-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.496 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.496 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.496 DEBUG:boto:path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.496 DEBUG:boto:auth_path=/lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.496 DEBUG:boto:Method: GET 2024-04-16T17:49:09.496 DEBUG:boto:Path: /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.496 DEBUG:boto:Data: 2024-04-16T17:49:09.496 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.496 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.497 DEBUG:boto:Params: {} 2024-04-16T17:49:09.497 DEBUG:boto:Token: None 2024-04-16T17:49:09.497 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj?versionId=null 2024-04-16T17:49:09.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ= 2024-04-16T17:49:09.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tXPYoRkpKlaOhvY0FZelj73V1JQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.500 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000069d7dc398e18c9ee-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.542 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.542 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.543 DEBUG:boto:path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.543 DEBUG:boto:auth_path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.543 DEBUG:boto:Method: GET 2024-04-16T17:49:09.543 DEBUG:boto:Path: /lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.543 DEBUG:boto:Data: 2024-04-16T17:49:09.543 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.543 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.543 DEBUG:boto:Params: {} 2024-04-16T17:49:09.543 DEBUG:boto:Token: None 2024-04-16T17:49:09.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035670b8e2ab3fcc4-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.548 DEBUG:boto:b'lybtug-8_myobj1000false_myobj2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.548 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.548 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.548 DEBUG:boto:path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.548 DEBUG:boto:auth_path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.549 DEBUG:boto:Method: GET 2024-04-16T17:49:09.549 DEBUG:boto:Path: /lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.549 DEBUG:boto:Data: 2024-04-16T17:49:09.549 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.549 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.549 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.549 DEBUG:boto:Params: {} 2024-04-16T17:49:09.549 DEBUG:boto:Token: None 2024-04-16T17:49:09.549 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.553 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b127d85c9964557-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.553 DEBUG:boto:b'lybtug-8_myobj1000false_myobj2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.554 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.554 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.554 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.554 DEBUG:boto:Path: /lybtug-8/_myobj 2024-04-16T17:49:09.554 DEBUG:boto:Data: 2024-04-16T17:49:09.554 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.554 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.554 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.554 DEBUG:boto:Params: {} 2024-04-16T17:49:09.554 DEBUG:boto:Token: None 2024-04-16T17:49:09.554 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj 2024-04-16T17:49:09.555 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XQMIVbM3Z2lC2sgM+2FDlDz5AiI= 2024-04-16T17:49:09.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XQMIVbM3Z2lC2sgM+2FDlDz5AiI=', 'Content-Length': '0'} 2024-04-16T17:49:09.557 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003b30acf68494e2c3-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.558 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.558 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.558 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.558 DEBUG:boto:Path: /lybtug-8/_myobj 2024-04-16T17:49:09.558 DEBUG:boto:Data: 2024-04-16T17:49:09.558 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.558 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.558 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.558 DEBUG:boto:Params: {} 2024-04-16T17:49:09.558 DEBUG:boto:Token: None 2024-04-16T17:49:09.559 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj 2024-04-16T17:49:09.559 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XQMIVbM3Z2lC2sgM+2FDlDz5AiI= 2024-04-16T17:49:09.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XQMIVbM3Z2lC2sgM+2FDlDz5AiI=', 'Content-Length': '0'} 2024-04-16T17:49:09.569 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005ef3b7f329962ee1-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.569 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2024-04-16T17:49:09.569 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.569 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.570 DEBUG:boto:Method: GET 2024-04-16T17:49:09.570 DEBUG:boto:Path: /lybtug-8/_myobj 2024-04-16T17:49:09.570 DEBUG:boto:Data: 2024-04-16T17:49:09.570 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.570 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.570 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.570 DEBUG:boto:Params: {} 2024-04-16T17:49:09.570 DEBUG:boto:Token: None 2024-04-16T17:49:09.570 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj 2024-04-16T17:49:09.570 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6VFjAiJiWNpOomsdOqQ/d1hDbCc= 2024-04-16T17:49:09.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6VFjAiJiWNpOomsdOqQ/d1hDbCc=', 'Content-Length': '0'} 2024-04-16T17:49:09.573 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002275d04bbfa57ef7-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.614 DEBUG:boto:path=/lybtug-8/_myobj 2024-04-16T17:49:09.614 DEBUG:boto:auth_path=/lybtug-8/_myobj 2024-04-16T17:49:09.615 DEBUG:boto:Method: GET 2024-04-16T17:49:09.615 DEBUG:boto:Path: /lybtug-8/_myobj 2024-04-16T17:49:09.615 DEBUG:boto:Data: 2024-04-16T17:49:09.615 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.615 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.615 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.615 DEBUG:boto:Params: {} 2024-04-16T17:49:09.615 DEBUG:boto:Token: None 2024-04-16T17:49:09.615 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/_myobj 2024-04-16T17:49:09.615 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6VFjAiJiWNpOomsdOqQ/d1hDbCc= 2024-04-16T17:49:09.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6VFjAiJiWNpOomsdOqQ/d1hDbCc=', 'Content-Length': '0'} 2024-04-16T17:49:09.618 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002af0a0c28bd2af10-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.619 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.619 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.619 DEBUG:boto:path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.619 DEBUG:boto:auth_path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.619 DEBUG:boto:Method: GET 2024-04-16T17:49:09.619 DEBUG:boto:Path: /lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.619 DEBUG:boto:Data: 2024-04-16T17:49:09.619 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.619 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.619 DEBUG:boto:Params: {} 2024-04-16T17:49:09.620 DEBUG:boto:Token: None 2024-04-16T17:49:09.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.623 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad69dcc9cc97ef75-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.624 DEBUG:boto:b'lybtug-8_myobj1000false_myobj2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.624 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.624 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.624 DEBUG:boto:path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.624 DEBUG:boto:auth_path=/lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.624 DEBUG:boto:Method: GET 2024-04-16T17:49:09.624 DEBUG:boto:Path: /lybtug-8/?prefix=_myobj 2024-04-16T17:49:09.625 DEBUG:boto:Data: 2024-04-16T17:49:09.625 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.625 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.625 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.625 DEBUG:boto:Params: {} 2024-04-16T17:49:09.625 DEBUG:boto:Token: None 2024-04-16T17:49:09.625 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.625 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.629 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ee53cd47c6516bc-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.670 DEBUG:boto:b'lybtug-8_myobj1000false_myobj2024-04-16T17:48:50.461Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:09.671 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.671 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.671 DEBUG:boto:path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.671 DEBUG:boto:auth_path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.671 DEBUG:boto:Method: GET 2024-04-16T17:49:09.671 DEBUG:boto:Path: /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.671 DEBUG:boto:Data: 2024-04-16T17:49:09.671 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.671 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.671 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.672 DEBUG:boto:Params: {} 2024-04-16T17:49:09.672 DEBUG:boto:Token: None 2024-04-16T17:49:09.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw= 2024-04-16T17:49:09.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw=', 'Content-Length': '0'} 2024-04-16T17:49:09.675 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000020d9c9d1713aee67-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.675 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.675 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.675 DEBUG:boto:path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.675 DEBUG:boto:auth_path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.675 DEBUG:boto:Method: GET 2024-04-16T17:49:09.676 DEBUG:boto:Path: /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.676 DEBUG:boto:Data: 2024-04-16T17:49:09.676 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.676 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.676 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.676 DEBUG:boto:Params: {} 2024-04-16T17:49:09.676 DEBUG:boto:Token: None 2024-04-16T17:49:09.676 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw= 2024-04-16T17:49:09.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw=', 'Content-Length': '0'} 2024-04-16T17:49:09.679 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e8deb298c5c76a94-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.680 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.680 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.680 DEBUG:boto:path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.680 DEBUG:boto:auth_path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.680 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.680 DEBUG:boto:Path: /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.680 DEBUG:boto:Data: 2024-04-16T17:49:09.680 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.680 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.680 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.680 DEBUG:boto:Params: {} 2024-04-16T17:49:09.681 DEBUG:boto:Token: None 2024-04-16T17:49:09.681 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.681 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mdYfzf9db3+3rFQF2xHDK1wh9Ec= 2024-04-16T17:49:09.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mdYfzf9db3+3rFQF2xHDK1wh9Ec=', 'Content-Length': '0'} 2024-04-16T17:49:09.683 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001111ec769b2f50e9-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.684 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.684 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.684 DEBUG:boto:path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.684 DEBUG:boto:auth_path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.684 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.684 DEBUG:boto:Path: /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.684 DEBUG:boto:Data: 2024-04-16T17:49:09.684 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.684 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.685 DEBUG:boto:Params: {} 2024-04-16T17:49:09.685 DEBUG:boto:Token: None 2024-04-16T17:49:09.685 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mdYfzf9db3+3rFQF2xHDK1wh9Ec= 2024-04-16T17:49:09.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mdYfzf9db3+3rFQF2xHDK1wh9Ec=', 'Content-Length': '0'} 2024-04-16T17:49:09.688 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000057ffe1d260875f35-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.688 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:09.688 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.688 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.688 DEBUG:boto:path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.688 DEBUG:boto:auth_path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.688 DEBUG:boto:Method: GET 2024-04-16T17:49:09.689 DEBUG:boto:Path: /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.689 DEBUG:boto:Data: 2024-04-16T17:49:09.689 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.689 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.689 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.689 DEBUG:boto:Params: {} 2024-04-16T17:49:09.689 DEBUG:boto:Token: None 2024-04-16T17:49:09.689 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.689 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw= 2024-04-16T17:49:09.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw=', 'Content-Length': '0'} 2024-04-16T17:49:09.692 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004ee8cf4af160a437-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.734 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.735 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.735 DEBUG:boto:path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.735 DEBUG:boto:auth_path=/lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.735 DEBUG:boto:Method: GET 2024-04-16T17:49:09.735 DEBUG:boto:Path: /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.735 DEBUG:boto:Data: 2024-04-16T17:49:09.735 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.735 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.735 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.735 DEBUG:boto:Params: {} 2024-04-16T17:49:09.735 DEBUG:boto:Token: None 2024-04-16T17:49:09.736 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj?versionId=null 2024-04-16T17:49:09.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw= 2024-04-16T17:49:09.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H04mZlFp5KyfiOZ61ZKO20v6viw=', 'Content-Length': '0'} 2024-04-16T17:49:09.738 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000026b1a76618044078-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.739 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.739 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.739 DEBUG:boto:path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.739 DEBUG:boto:auth_path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.739 DEBUG:boto:Method: GET 2024-04-16T17:49:09.739 DEBUG:boto:Path: /lybtug-8/?prefix=myobj 2024-04-16T17:49:09.739 DEBUG:boto:Data: 2024-04-16T17:49:09.739 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.740 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.740 DEBUG:boto:Params: {} 2024-04-16T17:49:09.740 DEBUG:boto:Token: None 2024-04-16T17:49:09.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e78b5bf33e6ffc4-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.744 DEBUG:boto:b'lybtug-8myobj1000falsemyobj2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.744 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.745 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.745 DEBUG:boto:path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.745 DEBUG:boto:auth_path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.745 DEBUG:boto:Method: GET 2024-04-16T17:49:09.745 DEBUG:boto:Path: /lybtug-8/?prefix=myobj 2024-04-16T17:49:09.745 DEBUG:boto:Data: 2024-04-16T17:49:09.745 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.745 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.745 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.745 DEBUG:boto:Params: {} 2024-04-16T17:49:09.745 DEBUG:boto:Token: None 2024-04-16T17:49:09.745 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.746 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b2a3af416619cb2-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.790 DEBUG:boto:b'lybtug-8myobj1000falsemyobj2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.791 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.791 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.791 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.791 DEBUG:boto:Path: /lybtug-8/myobj 2024-04-16T17:49:09.791 DEBUG:boto:Data: 2024-04-16T17:49:09.791 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.791 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.791 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.791 DEBUG:boto:Params: {} 2024-04-16T17:49:09.791 DEBUG:boto:Token: None 2024-04-16T17:49:09.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj 2024-04-16T17:49:09.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sszf1AabMUgxOXGk0rennU9IMVM= 2024-04-16T17:49:09.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sszf1AabMUgxOXGk0rennU9IMVM=', 'Content-Length': '0'} 2024-04-16T17:49:09.795 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000231c6186ac1cabb0-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.795 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.795 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.795 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.795 DEBUG:boto:Path: /lybtug-8/myobj 2024-04-16T17:49:09.795 DEBUG:boto:Data: 2024-04-16T17:49:09.795 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.796 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.796 DEBUG:boto:Params: {} 2024-04-16T17:49:09.796 DEBUG:boto:Token: None 2024-04-16T17:49:09.796 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj 2024-04-16T17:49:09.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sszf1AabMUgxOXGk0rennU9IMVM= 2024-04-16T17:49:09.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sszf1AabMUgxOXGk0rennU9IMVM=', 'Content-Length': '0'} 2024-04-16T17:49:09.799 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000390a756f571ebe89-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.799 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:09.799 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.799 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.799 DEBUG:boto:Method: GET 2024-04-16T17:49:09.800 DEBUG:boto:Path: /lybtug-8/myobj 2024-04-16T17:49:09.800 DEBUG:boto:Data: 2024-04-16T17:49:09.800 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.800 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.800 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.800 DEBUG:boto:Params: {} 2024-04-16T17:49:09.800 DEBUG:boto:Token: None 2024-04-16T17:49:09.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj 2024-04-16T17:49:09.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PuineKsefDg2+eXJL+XsFilnYjQ= 2024-04-16T17:49:09.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PuineKsefDg2+eXJL+XsFilnYjQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.803 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:48:54 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003e67273952e4866d-00661eba15-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.803 DEBUG:boto:path=/lybtug-8/myobj 2024-04-16T17:49:09.803 DEBUG:boto:auth_path=/lybtug-8/myobj 2024-04-16T17:49:09.804 DEBUG:boto:Method: GET 2024-04-16T17:49:09.804 DEBUG:boto:Path: /lybtug-8/myobj 2024-04-16T17:49:09.804 DEBUG:boto:Data: 2024-04-16T17:49:09.804 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.804 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.804 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.804 DEBUG:boto:Params: {} 2024-04-16T17:49:09.804 DEBUG:boto:Token: None 2024-04-16T17:49:09.804 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/myobj 2024-04-16T17:49:09.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PuineKsefDg2+eXJL+XsFilnYjQ= 2024-04-16T17:49:09.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PuineKsefDg2+eXJL+XsFilnYjQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.808 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:48:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d8440a5ad4fafda0-00661eba15-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.808 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.808 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.809 DEBUG:boto:path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.809 DEBUG:boto:auth_path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.809 DEBUG:boto:Method: GET 2024-04-16T17:49:09.809 DEBUG:boto:Path: /lybtug-8/?prefix=myobj 2024-04-16T17:49:09.809 DEBUG:boto:Data: 2024-04-16T17:49:09.809 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.809 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.809 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.809 DEBUG:boto:Params: {} 2024-04-16T17:49:09.809 DEBUG:boto:Token: None 2024-04-16T17:49:09.810 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.810 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.813 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054f0ec9656b6c4ba-00661eba15-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.854 DEBUG:boto:b'lybtug-8myobj1000falsemyobj2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.855 DEBUG:boto:path=/lybtug-8/ 2024-04-16T17:49:09.855 DEBUG:boto:auth_path=/lybtug-8/ 2024-04-16T17:49:09.855 DEBUG:boto:path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.855 DEBUG:boto:auth_path=/lybtug-8/?prefix=myobj 2024-04-16T17:49:09.855 DEBUG:boto:Method: GET 2024-04-16T17:49:09.855 DEBUG:boto:Path: /lybtug-8/?prefix=myobj 2024-04-16T17:49:09.855 DEBUG:boto:Data: 2024-04-16T17:49:09.855 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.855 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.855 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.855 DEBUG:boto:Params: {} 2024-04-16T17:49:09.856 DEBUG:boto:Token: None 2024-04-16T17:49:09.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:09 GMT /lybtug-8/ 2024-04-16T17:49:09.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0= 2024-04-16T17:49:09.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtPMUD6am4P7CWV1G3QjL0Fxpl0=', 'Content-Length': '0'} 2024-04-16T17:49:09.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c3fa8a93db6e2b7-00661eba15-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.860 DEBUG:boto:b'lybtug-8myobj1000falsemyobj2024-04-16T17:48:50.432Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:09.860 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-8 zones={test-zone1, test-zone2} 2024-04-16T17:49:09.860 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_sync ... ok 2024-04-16T17:49:09.861 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-9 2024-04-16T17:49:09.862 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:09.862 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:09.862 DEBUG:boto:Method: PUT 2024-04-16T17:49:09.862 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:09.862 DEBUG:boto:Data: 2024-04-16T17:49:09.862 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.862 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.862 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.862 DEBUG:boto:Params: {} 2024-04-16T17:49:09.862 DEBUG:boto:Token: None 2024-04-16T17:49:09.862 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:49:09 GMT /lybtug-9/ 2024-04-16T17:49:09.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lfZNpekzghXFEd1TIhNTeEVvB6A= 2024-04-16T17:49:09.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lfZNpekzghXFEd1TIhNTeEVvB6A=', 'Content-Length': '0'} 2024-04-16T17:49:09.897 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000021261cc695be8a2d-00661eba15-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.897 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-10 2024-04-16T17:49:09.897 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:09.898 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:09.898 DEBUG:boto:Method: PUT 2024-04-16T17:49:09.898 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:09.898 DEBUG:boto:Data: 2024-04-16T17:49:09.898 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.898 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.898 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.898 DEBUG:boto:Params: {} 2024-04-16T17:49:09.898 DEBUG:boto:Token: None 2024-04-16T17:49:09.898 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:49:09 GMT /lybtug-10/ 2024-04-16T17:49:09.898 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MgrrSMo10Dy4M7C4qoaehYRsVLE= 2024-04-16T17:49:09.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MgrrSMo10Dy4M7C4qoaehYRsVLE=', 'Content-Length': '0'} 2024-04-16T17:49:09.957 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009332a85f78cfeca2-00661eba15-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.957 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:09.957 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:09.958 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.958 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:09.958 DEBUG:boto:Data: 2024-04-16T17:49:09.958 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.958 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.958 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.958 DEBUG:boto:Params: {} 2024-04-16T17:49:09.958 DEBUG:boto:Token: None 2024-04-16T17:49:09.958 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-9/ 2024-04-16T17:49:09.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2j6Q1ZkhgeRQqcdgPlY9otuFkcQ= 2024-04-16T17:49:09.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2j6Q1ZkhgeRQqcdgPlY9otuFkcQ=', 'Content-Length': '0'} 2024-04-16T17:49:09.962 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000df5006168042bb8d-00661eba15-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.962 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:09.962 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:09.962 DEBUG:boto:Method: PUT 2024-04-16T17:49:09.962 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:09.962 DEBUG:boto:Data: 2024-04-16T17:49:09.963 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:09.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:09.963 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.963 DEBUG:boto:Params: {} 2024-04-16T17:49:09.963 DEBUG:boto:Token: None 2024-04-16T17:49:09.963 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:09 GMT /lybtug-9/myobj 2024-04-16T17:49:09.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J5DOJAb1Oh4AxUhtHntDTIuHW4Y= 2024-04-16T17:49:09.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J5DOJAb1Oh4AxUhtHntDTIuHW4Y='} 2024-04-16T17:49:09.977 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c2d7abd32f47fc56-00661eba15-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.977 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:09.977 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:09.977 DEBUG:boto:Method: HEAD 2024-04-16T17:49:09.977 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:09.977 DEBUG:boto:Data: 2024-04-16T17:49:09.977 DEBUG:boto:Headers: {} 2024-04-16T17:49:09.978 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.978 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.978 DEBUG:boto:Params: {} 2024-04-16T17:49:09.978 DEBUG:boto:Token: None 2024-04-16T17:49:09.978 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:09 GMT /lybtug-10/ 2024-04-16T17:49:09.978 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GydvPwuQDoa7iuLlaaySZ2jFuWs= 2024-04-16T17:49:09.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GydvPwuQDoa7iuLlaaySZ2jFuWs=', 'Content-Length': '0'} 2024-04-16T17:49:09.982 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002b4e19a97bb5d736-00661eba15-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:09.982 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:09.982 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:09.982 DEBUG:boto:Method: PUT 2024-04-16T17:49:09.982 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:09.983 DEBUG:boto:Data: 2024-04-16T17:49:09.983 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:09.983 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:09.983 DEBUG:boto:Port: 8000 2024-04-16T17:49:09.983 DEBUG:boto:Params: {} 2024-04-16T17:49:09.983 DEBUG:boto:Token: None 2024-04-16T17:49:09.983 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:09 GMT /lybtug-10/myobj 2024-04-16T17:49:09.983 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QDFF9v9TvdJlf30CzieaBlGEqQw= 2024-04-16T17:49:09.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QDFF9v9TvdJlf30CzieaBlGEqQw='} 2024-04-16T17:49:10.026 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008c50c09461e603e5-00661eba15-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:10.027 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:10.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:10.027 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:49:10.069 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:10.070 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:10.136 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:49:10.168 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289749.934452_37.1","last_update":"2024-04-16T17:49:09.934452Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:49:10.169 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289749.934452_37.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:10.169 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:49:10.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:10.169 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:10.169 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:49:10.211 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:10.211 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:10.280+0000 7fd0877f4980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:49:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:10.280+0000 7fd0877f4980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:49:10.311 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289749.934452_37.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:09.934452Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.875087_97.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.875087Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:10Z"} 2024-04-16T17:49:10.311 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289749.934452_37.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:09.934452Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.875087_97.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.875087Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:10Z"} 2024-04-16T17:49:10.311 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289749.934452_37.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.934452Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.875087_97.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.875087Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:49:10.311 DEBUG:rgw_multi.tests:log_status={0: '1_1713289749.934452_37.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:10.311 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289749.934452_37.1', 1: '1_1713289749.875087_97.1', 2: '', 3: ''} 2024-04-16T17:49:10.311 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1713289749.894324_100.1 target=1_1713289749.875087_97.1 2024-04-16T17:49:10.796 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T17:49:10.796+0000 7f4990de6640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T17:49:15.312 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:15.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:15.313 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:49:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:15.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:15.424+0000 7f8994527980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:49:15.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:15.424+0000 7f8994527980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:49:15.455 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289749.934452_37.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:09.934452Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:15Z"} 2024-04-16T17:49:15.455 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289749.934452_37.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:09.934452Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:15Z"} 2024-04-16T17:49:15.455 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289749.934452_37.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.934452Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:49:15.456 DEBUG:rgw_multi.tests:log_status={0: '1_1713289749.934452_37.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:15.456 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289749.934452_37.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:15.456 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:49:15.456 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-9', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:15.456 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-9', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:15.456 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-9 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:49:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:15.628+0000 7facea148980 1 bucket sync caught up with source: 2024-04-16T17:49:15.628 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.5.6, , , , , , , , , ] 2024-04-16T17:49:15.628 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000001.5.6, , , , , , , , , ] 2024-04-16T17:49:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:15.628+0000 7facea148980 0 bucket checkpoint complete 2024-04-16T17:49:15.654 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-9 zones={test-zone2, test-zone1} 2024-04-16T17:49:15.654 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.654 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.654 DEBUG:boto:Method: HEAD 2024-04-16T17:49:15.655 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:15.655 DEBUG:boto:Data: 2024-04-16T17:49:15.655 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.655 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.655 DEBUG:boto:Params: {} 2024-04-16T17:49:15.655 DEBUG:boto:Token: None 2024-04-16T17:49:15.655 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/ 2024-04-16T17:49:15.655 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BuLiUC5qJ5FIQLJ4XK4MoS7puZ0= 2024-04-16T17:49:15.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BuLiUC5qJ5FIQLJ4XK4MoS7puZ0=', 'Content-Length': '0'} 2024-04-16T17:49:15.658 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009f61f57ab2f15595-00661eba1b-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.658 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.658 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.658 DEBUG:boto:Method: HEAD 2024-04-16T17:49:15.659 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:15.659 DEBUG:boto:Data: 2024-04-16T17:49:15.659 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.659 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.659 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.659 DEBUG:boto:Params: {} 2024-04-16T17:49:15.659 DEBUG:boto:Token: None 2024-04-16T17:49:15.659 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/ 2024-04-16T17:49:15.659 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BuLiUC5qJ5FIQLJ4XK4MoS7puZ0= 2024-04-16T17:49:15.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BuLiUC5qJ5FIQLJ4XK4MoS7puZ0=', 'Content-Length': '0'} 2024-04-16T17:49:15.663 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f41fdd99df9fa90f-00661eba1b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.663 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:49:15.663 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.663 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.663 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.663 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.663 DEBUG:boto:Method: GET 2024-04-16T17:49:15.664 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.664 DEBUG:boto:Data: 2024-04-16T17:49:15.664 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.664 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.664 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.664 DEBUG:boto:Params: {} 2024-04-16T17:49:15.664 DEBUG:boto:Token: None 2024-04-16T17:49:15.664 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/?versions 2024-04-16T17:49:15.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634= 2024-04-16T17:49:15.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634=', 'Content-Length': '0'} 2024-04-16T17:49:15.667 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059bd9831f5b0f719-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.667 DEBUG:boto:b'lybtug-9999falsemyobjnulltrue2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.667 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:15.668 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:49:15.668 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.668 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.668 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.668 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.668 DEBUG:boto:Method: GET 2024-04-16T17:49:15.668 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.668 DEBUG:boto:Data: 2024-04-16T17:49:15.668 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.668 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.668 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.668 DEBUG:boto:Params: {} 2024-04-16T17:49:15.669 DEBUG:boto:Token: None 2024-04-16T17:49:15.669 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/?versions 2024-04-16T17:49:15.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634= 2024-04-16T17:49:15.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634=', 'Content-Length': '0'} 2024-04-16T17:49:15.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a92a87e9a9da80b4-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.673 DEBUG:boto:b'lybtug-9999falsemyobjnulltrue2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.673 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:15.673 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.673 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.673 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.673 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.673 DEBUG:boto:Method: GET 2024-04-16T17:49:15.673 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.673 DEBUG:boto:Data: 2024-04-16T17:49:15.674 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.674 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.674 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.674 DEBUG:boto:Params: {} 2024-04-16T17:49:15.674 DEBUG:boto:Token: None 2024-04-16T17:49:15.674 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/?versions 2024-04-16T17:49:15.674 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634= 2024-04-16T17:49:15.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634=', 'Content-Length': '0'} 2024-04-16T17:49:15.678 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac71e7f422a1a69a-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.678 DEBUG:boto:b'lybtug-9999falsemyobjnulltrue2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.678 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.678 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.678 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.678 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.678 DEBUG:boto:Method: GET 2024-04-16T17:49:15.678 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:15.679 DEBUG:boto:Data: 2024-04-16T17:49:15.679 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.679 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.679 DEBUG:boto:Params: {} 2024-04-16T17:49:15.679 DEBUG:boto:Token: None 2024-04-16T17:49:15.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/?versions 2024-04-16T17:49:15.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634= 2024-04-16T17:49:15.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:opfGg2Tl7+tKX7f+tsIB+iNP634=', 'Content-Length': '0'} 2024-04-16T17:49:15.683 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2d39ae699721275-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.726 DEBUG:boto:b'lybtug-9999falsemyobjnulltrue2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.727 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:15.727 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.727 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.727 DEBUG:boto:path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.727 DEBUG:boto:auth_path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.727 DEBUG:boto:Method: GET 2024-04-16T17:49:15.727 DEBUG:boto:Path: /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.727 DEBUG:boto:Data: 2024-04-16T17:49:15.727 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.727 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.728 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.728 DEBUG:boto:Params: {} 2024-04-16T17:49:15.728 DEBUG:boto:Token: None 2024-04-16T17:49:15.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0= 2024-04-16T17:49:15.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0=', 'Content-Length': '0'} 2024-04-16T17:49:15.731 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:10 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000043cbf546f77987a9-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.731 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.731 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.731 DEBUG:boto:path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.731 DEBUG:boto:auth_path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.731 DEBUG:boto:Method: GET 2024-04-16T17:49:15.731 DEBUG:boto:Path: /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.732 DEBUG:boto:Data: 2024-04-16T17:49:15.732 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.732 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.732 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.732 DEBUG:boto:Params: {} 2024-04-16T17:49:15.732 DEBUG:boto:Token: None 2024-04-16T17:49:15.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0= 2024-04-16T17:49:15.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0=', 'Content-Length': '0'} 2024-04-16T17:49:15.735 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000265eb2628ca8a2a0-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.735 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.736 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.736 DEBUG:boto:path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.736 DEBUG:boto:auth_path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.736 DEBUG:boto:Method: HEAD 2024-04-16T17:49:15.736 DEBUG:boto:Path: /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.736 DEBUG:boto:Data: 2024-04-16T17:49:15.736 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.736 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.736 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.736 DEBUG:boto:Params: {} 2024-04-16T17:49:15.736 DEBUG:boto:Token: None 2024-04-16T17:49:15.736 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XNxenG4O47xk1QOvQYo+ZKjZlok= 2024-04-16T17:49:15.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XNxenG4O47xk1QOvQYo+ZKjZlok=', 'Content-Length': '0'} 2024-04-16T17:49:15.739 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:10 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007f6b716dd01bdebd-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.740 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.740 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.740 DEBUG:boto:path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.740 DEBUG:boto:auth_path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.740 DEBUG:boto:Method: HEAD 2024-04-16T17:49:15.740 DEBUG:boto:Path: /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.740 DEBUG:boto:Data: 2024-04-16T17:49:15.740 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.740 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.740 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.740 DEBUG:boto:Params: {} 2024-04-16T17:49:15.741 DEBUG:boto:Token: None 2024-04-16T17:49:15.741 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.741 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XNxenG4O47xk1QOvQYo+ZKjZlok= 2024-04-16T17:49:15.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XNxenG4O47xk1QOvQYo+ZKjZlok=', 'Content-Length': '0'} 2024-04-16T17:49:15.744 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000aa233f7937af899-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:15.744 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.744 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.744 DEBUG:boto:path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.744 DEBUG:boto:auth_path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.745 DEBUG:boto:Method: GET 2024-04-16T17:49:15.745 DEBUG:boto:Path: /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.745 DEBUG:boto:Data: 2024-04-16T17:49:15.745 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.745 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.745 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.745 DEBUG:boto:Params: {} 2024-04-16T17:49:15.745 DEBUG:boto:Token: None 2024-04-16T17:49:15.745 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0= 2024-04-16T17:49:15.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0=', 'Content-Length': '0'} 2024-04-16T17:49:15.749 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:10 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e4964c0cfc605f5d-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.790 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.790 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.791 DEBUG:boto:path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.791 DEBUG:boto:auth_path=/lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.791 DEBUG:boto:Method: GET 2024-04-16T17:49:15.791 DEBUG:boto:Path: /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.791 DEBUG:boto:Data: 2024-04-16T17:49:15.791 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.791 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.791 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.791 DEBUG:boto:Params: {} 2024-04-16T17:49:15.791 DEBUG:boto:Token: None 2024-04-16T17:49:15.791 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj?versionId=null 2024-04-16T17:49:15.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0= 2024-04-16T17:49:15.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n9djDjboWRKUMowerZr/UCFzpD0=', 'Content-Length': '0'} 2024-04-16T17:49:15.794 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008d6ea9df02ec9413-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.795 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.795 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.795 DEBUG:boto:path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.795 DEBUG:boto:auth_path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.795 DEBUG:boto:Method: GET 2024-04-16T17:49:15.795 DEBUG:boto:Path: /lybtug-9/?prefix=myobj 2024-04-16T17:49:15.795 DEBUG:boto:Data: 2024-04-16T17:49:15.795 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.795 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.795 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.796 DEBUG:boto:Params: {} 2024-04-16T17:49:15.796 DEBUG:boto:Token: None 2024-04-16T17:49:15.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/ 2024-04-16T17:49:15.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc= 2024-04-16T17:49:15.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc=', 'Content-Length': '0'} 2024-04-16T17:49:15.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005fc8bcb8689a5fad-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.800 DEBUG:boto:b'lybtug-9myobj1000falsemyobj2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.800 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.800 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.800 DEBUG:boto:path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.800 DEBUG:boto:auth_path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.800 DEBUG:boto:Method: GET 2024-04-16T17:49:15.800 DEBUG:boto:Path: /lybtug-9/?prefix=myobj 2024-04-16T17:49:15.800 DEBUG:boto:Data: 2024-04-16T17:49:15.800 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.801 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.801 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.801 DEBUG:boto:Params: {} 2024-04-16T17:49:15.801 DEBUG:boto:Token: None 2024-04-16T17:49:15.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/ 2024-04-16T17:49:15.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc= 2024-04-16T17:49:15.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc=', 'Content-Length': '0'} 2024-04-16T17:49:15.805 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048ecabdb305c0fb3-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.846 DEBUG:boto:b'lybtug-9myobj1000falsemyobj2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.847 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.847 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.847 DEBUG:boto:Method: HEAD 2024-04-16T17:49:15.847 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:15.847 DEBUG:boto:Data: 2024-04-16T17:49:15.847 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.847 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.847 DEBUG:boto:Params: {} 2024-04-16T17:49:15.848 DEBUG:boto:Token: None 2024-04-16T17:49:15.848 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj 2024-04-16T17:49:15.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wLXAr/nvuibQ4QZVtlG4qF5r86c= 2024-04-16T17:49:15.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wLXAr/nvuibQ4QZVtlG4qF5r86c=', 'Content-Length': '0'} 2024-04-16T17:49:15.851 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:10 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b855f8eb473d40d8-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.851 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.851 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.851 DEBUG:boto:Method: HEAD 2024-04-16T17:49:15.851 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:15.851 DEBUG:boto:Data: 2024-04-16T17:49:15.852 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.852 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.852 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.852 DEBUG:boto:Params: {} 2024-04-16T17:49:15.852 DEBUG:boto:Token: None 2024-04-16T17:49:15.852 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj 2024-04-16T17:49:15.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wLXAr/nvuibQ4QZVtlG4qF5r86c= 2024-04-16T17:49:15.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wLXAr/nvuibQ4QZVtlG4qF5r86c=', 'Content-Length': '0'} 2024-04-16T17:49:15.855 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ec97de929acf12aa-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.855 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:15.856 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.856 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.856 DEBUG:boto:Method: GET 2024-04-16T17:49:15.856 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:15.856 DEBUG:boto:Data: 2024-04-16T17:49:15.856 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.856 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.856 DEBUG:boto:Params: {} 2024-04-16T17:49:15.856 DEBUG:boto:Token: None 2024-04-16T17:49:15.857 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj 2024-04-16T17:49:15.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5h5boE4XJCw15NHIU02as3JsdyA= 2024-04-16T17:49:15.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5h5boE4XJCw15NHIU02as3JsdyA=', 'Content-Length': '0'} 2024-04-16T17:49:15.859 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:10 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038973012b94e5bd6-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.902 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:15.902 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:15.903 DEBUG:boto:Method: GET 2024-04-16T17:49:15.903 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:15.903 DEBUG:boto:Data: 2024-04-16T17:49:15.903 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.903 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.903 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.903 DEBUG:boto:Params: {} 2024-04-16T17:49:15.903 DEBUG:boto:Token: None 2024-04-16T17:49:15.903 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/myobj 2024-04-16T17:49:15.903 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5h5boE4XJCw15NHIU02as3JsdyA= 2024-04-16T17:49:15.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5h5boE4XJCw15NHIU02as3JsdyA=', 'Content-Length': '0'} 2024-04-16T17:49:15.907 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007eda420b5ae4c42f-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.907 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.907 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.908 DEBUG:boto:path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.908 DEBUG:boto:auth_path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.908 DEBUG:boto:Method: GET 2024-04-16T17:49:15.908 DEBUG:boto:Path: /lybtug-9/?prefix=myobj 2024-04-16T17:49:15.908 DEBUG:boto:Data: 2024-04-16T17:49:15.908 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.908 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:15.908 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.908 DEBUG:boto:Params: {} 2024-04-16T17:49:15.908 DEBUG:boto:Token: None 2024-04-16T17:49:15.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/ 2024-04-16T17:49:15.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc= 2024-04-16T17:49:15.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc=', 'Content-Length': '0'} 2024-04-16T17:49:15.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002d4cce8b65e7942-00661eba1b-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.954 DEBUG:boto:b'lybtug-9myobj1000falsemyobj2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.955 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:15.955 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:15.955 DEBUG:boto:path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.955 DEBUG:boto:auth_path=/lybtug-9/?prefix=myobj 2024-04-16T17:49:15.955 DEBUG:boto:Method: GET 2024-04-16T17:49:15.955 DEBUG:boto:Path: /lybtug-9/?prefix=myobj 2024-04-16T17:49:15.955 DEBUG:boto:Data: 2024-04-16T17:49:15.955 DEBUG:boto:Headers: {} 2024-04-16T17:49:15.956 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:15.956 DEBUG:boto:Port: 8000 2024-04-16T17:49:15.956 DEBUG:boto:Params: {} 2024-04-16T17:49:15.956 DEBUG:boto:Token: None 2024-04-16T17:49:15.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:15 GMT /lybtug-9/ 2024-04-16T17:49:15.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc= 2024-04-16T17:49:15.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vZkd4G3C/xwsi/k8+B+qUFRInwc=', 'Content-Length': '0'} 2024-04-16T17:49:15.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007c7813ad6cc9409-00661eba1b-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:15.960 DEBUG:boto:b'lybtug-9myobj1000falsemyobj2024-04-16T17:49:09.968Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:15.960 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-9 zones={test-zone2, test-zone1} 2024-04-16T17:49:15.960 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-10', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:15.961 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-10', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:15.961 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-10 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:49:16.010 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:16.010 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:16.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:16.088+0000 7f4fe36c7980 1 bucket sync caught up with source: 2024-04-16T17:49:16.092 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.7.6, , , , , , , , , ] 2024-04-16T17:49:16.092 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.7.6, , , , , , , , , ] 2024-04-16T17:49:16.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:16.088+0000 7f4fe36c7980 0 bucket checkpoint complete 2024-04-16T17:49:16.121 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-10 zones={test-zone1, test-zone2} 2024-04-16T17:49:16.121 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.121 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.122 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.122 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:16.122 DEBUG:boto:Data: 2024-04-16T17:49:16.122 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.122 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.122 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.122 DEBUG:boto:Params: {} 2024-04-16T17:49:16.122 DEBUG:boto:Token: None 2024-04-16T17:49:16.122 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/ 2024-04-16T17:49:16.123 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IAy4vhuHfdoeS9KzpxKOageOAxI= 2024-04-16T17:49:16.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IAy4vhuHfdoeS9KzpxKOageOAxI=', 'Content-Length': '0'} 2024-04-16T17:49:16.125 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bd20bbc7eb906eff-00661eba1c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.125 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.126 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.126 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.126 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:16.126 DEBUG:boto:Data: 2024-04-16T17:49:16.126 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.126 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.126 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.126 DEBUG:boto:Params: {} 2024-04-16T17:49:16.126 DEBUG:boto:Token: None 2024-04-16T17:49:16.126 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/ 2024-04-16T17:49:16.127 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IAy4vhuHfdoeS9KzpxKOageOAxI= 2024-04-16T17:49:16.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IAy4vhuHfdoeS9KzpxKOageOAxI=', 'Content-Length': '0'} 2024-04-16T17:49:16.131 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000afc35b3f462ca055-00661eba1c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.131 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:49:16.131 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.131 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.131 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.131 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.131 DEBUG:boto:Method: GET 2024-04-16T17:49:16.131 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.131 DEBUG:boto:Data: 2024-04-16T17:49:16.132 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.132 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.132 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.132 DEBUG:boto:Params: {} 2024-04-16T17:49:16.132 DEBUG:boto:Token: None 2024-04-16T17:49:16.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/?versions 2024-04-16T17:49:16.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8= 2024-04-16T17:49:16.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8=', 'Content-Length': '0'} 2024-04-16T17:49:16.135 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2a33d290b7e5cdd-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.135 DEBUG:boto:b'lybtug-10999falsemyobjnulltrue2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.135 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:16.135 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:49:16.135 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.136 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.136 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.136 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.136 DEBUG:boto:Method: GET 2024-04-16T17:49:16.136 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.136 DEBUG:boto:Data: 2024-04-16T17:49:16.136 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.136 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.136 DEBUG:boto:Params: {} 2024-04-16T17:49:16.136 DEBUG:boto:Token: None 2024-04-16T17:49:16.137 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/?versions 2024-04-16T17:49:16.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8= 2024-04-16T17:49:16.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8=', 'Content-Length': '0'} 2024-04-16T17:49:16.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099f174668e5e8293-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.140 DEBUG:boto:b'lybtug-10999falsemyobjnulltrue2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.141 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2024-04-16T17:49:16.141 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.141 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.141 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.141 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.141 DEBUG:boto:Method: GET 2024-04-16T17:49:16.141 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.141 DEBUG:boto:Data: 2024-04-16T17:49:16.142 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.142 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.142 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.142 DEBUG:boto:Params: {} 2024-04-16T17:49:16.142 DEBUG:boto:Token: None 2024-04-16T17:49:16.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/?versions 2024-04-16T17:49:16.142 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8= 2024-04-16T17:49:16.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8=', 'Content-Length': '0'} 2024-04-16T17:49:16.145 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030e69b572427219a-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.145 DEBUG:boto:b'lybtug-10999falsemyobjnulltrue2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.146 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.146 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.146 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.146 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.146 DEBUG:boto:Method: GET 2024-04-16T17:49:16.146 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:16.146 DEBUG:boto:Data: 2024-04-16T17:49:16.146 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.146 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.146 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.146 DEBUG:boto:Params: {} 2024-04-16T17:49:16.147 DEBUG:boto:Token: None 2024-04-16T17:49:16.147 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/?versions 2024-04-16T17:49:16.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8= 2024-04-16T17:49:16.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UHB5lnRAKTFlXTGf1OmbKgJUWx8=', 'Content-Length': '0'} 2024-04-16T17:49:16.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008859e74c6b5b5904-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.151 DEBUG:boto:b'lybtug-10999falsemyobjnulltrue2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:16.151 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.151 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.151 DEBUG:boto:path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.151 DEBUG:boto:auth_path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.152 DEBUG:boto:Method: GET 2024-04-16T17:49:16.152 DEBUG:boto:Path: /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.152 DEBUG:boto:Data: 2024-04-16T17:49:16.152 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.152 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.152 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.152 DEBUG:boto:Params: {} 2024-04-16T17:49:16.152 DEBUG:boto:Token: None 2024-04-16T17:49:16.152 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0= 2024-04-16T17:49:16.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0=', 'Content-Length': '0'} 2024-04-16T17:49:16.154 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:14 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b303a6199d0fd4ae-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.155 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.155 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.155 DEBUG:boto:path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.155 DEBUG:boto:auth_path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.155 DEBUG:boto:Method: GET 2024-04-16T17:49:16.155 DEBUG:boto:Path: /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.155 DEBUG:boto:Data: 2024-04-16T17:49:16.155 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.155 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.156 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.156 DEBUG:boto:Params: {} 2024-04-16T17:49:16.156 DEBUG:boto:Token: None 2024-04-16T17:49:16.156 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0= 2024-04-16T17:49:16.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0=', 'Content-Length': '0'} 2024-04-16T17:49:16.159 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000332db27ead8d0bdb-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.202 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.203 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.203 DEBUG:boto:path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.203 DEBUG:boto:auth_path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.203 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.203 DEBUG:boto:Path: /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.203 DEBUG:boto:Data: 2024-04-16T17:49:16.203 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.203 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.203 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.203 DEBUG:boto:Params: {} 2024-04-16T17:49:16.203 DEBUG:boto:Token: None 2024-04-16T17:49:16.204 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q9MoNqt+07chnfkWpJveqp20WXA= 2024-04-16T17:49:16.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q9MoNqt+07chnfkWpJveqp20WXA=', 'Content-Length': '0'} 2024-04-16T17:49:16.207 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:14 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000055974daaaa7c52d2-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.207 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.207 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.207 DEBUG:boto:path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.207 DEBUG:boto:auth_path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.207 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.207 DEBUG:boto:Path: /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.207 DEBUG:boto:Data: 2024-04-16T17:49:16.208 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.208 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.208 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.208 DEBUG:boto:Params: {} 2024-04-16T17:49:16.208 DEBUG:boto:Token: None 2024-04-16T17:49:16.208 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q9MoNqt+07chnfkWpJveqp20WXA= 2024-04-16T17:49:16.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q9MoNqt+07chnfkWpJveqp20WXA=', 'Content-Length': '0'} 2024-04-16T17:49:16.211 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c4c38f6972edcb50-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.211 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:16.212 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.212 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.212 DEBUG:boto:path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.212 DEBUG:boto:auth_path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.212 DEBUG:boto:Method: GET 2024-04-16T17:49:16.212 DEBUG:boto:Path: /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.212 DEBUG:boto:Data: 2024-04-16T17:49:16.212 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.213 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.213 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.213 DEBUG:boto:Params: {} 2024-04-16T17:49:16.213 DEBUG:boto:Token: None 2024-04-16T17:49:16.213 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0= 2024-04-16T17:49:16.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0=', 'Content-Length': '0'} 2024-04-16T17:49:16.216 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:14 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a63af36db79e1dd9-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.258 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.258 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.259 DEBUG:boto:path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.259 DEBUG:boto:auth_path=/lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.259 DEBUG:boto:Method: GET 2024-04-16T17:49:16.259 DEBUG:boto:Path: /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.259 DEBUG:boto:Data: 2024-04-16T17:49:16.259 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.259 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.259 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.259 DEBUG:boto:Params: {} 2024-04-16T17:49:16.259 DEBUG:boto:Token: None 2024-04-16T17:49:16.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj?versionId=null 2024-04-16T17:49:16.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0= 2024-04-16T17:49:16.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0SjtzJE4FU3iQHwsU3sZxguecz0=', 'Content-Length': '0'} 2024-04-16T17:49:16.263 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002d22d88860e1228d-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.263 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.263 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.264 DEBUG:boto:path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.264 DEBUG:boto:auth_path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.264 DEBUG:boto:Method: GET 2024-04-16T17:49:16.264 DEBUG:boto:Path: /lybtug-10/?prefix=myobj 2024-04-16T17:49:16.264 DEBUG:boto:Data: 2024-04-16T17:49:16.264 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.264 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.264 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.264 DEBUG:boto:Params: {} 2024-04-16T17:49:16.264 DEBUG:boto:Token: None 2024-04-16T17:49:16.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/ 2024-04-16T17:49:16.265 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00= 2024-04-16T17:49:16.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00=', 'Content-Length': '0'} 2024-04-16T17:49:16.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7059e35c7992d20-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.269 DEBUG:boto:b'lybtug-10myobj1000falsemyobj2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.269 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.269 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.269 DEBUG:boto:path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.269 DEBUG:boto:auth_path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.269 DEBUG:boto:Method: GET 2024-04-16T17:49:16.269 DEBUG:boto:Path: /lybtug-10/?prefix=myobj 2024-04-16T17:49:16.270 DEBUG:boto:Data: 2024-04-16T17:49:16.270 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.270 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.270 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.270 DEBUG:boto:Params: {} 2024-04-16T17:49:16.270 DEBUG:boto:Token: None 2024-04-16T17:49:16.270 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/ 2024-04-16T17:49:16.270 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00= 2024-04-16T17:49:16.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00=', 'Content-Length': '0'} 2024-04-16T17:49:16.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066fded90b5f94f27-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.274 DEBUG:boto:b'lybtug-10myobj1000falsemyobj2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.275 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.275 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.275 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.275 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:16.275 DEBUG:boto:Data: 2024-04-16T17:49:16.275 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.276 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.276 DEBUG:boto:Params: {} 2024-04-16T17:49:16.276 DEBUG:boto:Token: None 2024-04-16T17:49:16.276 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj 2024-04-16T17:49:16.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jL4N1gjdJEtS/fBkxyf8BjxZis0= 2024-04-16T17:49:16.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jL4N1gjdJEtS/fBkxyf8BjxZis0=', 'Content-Length': '0'} 2024-04-16T17:49:16.279 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:14 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000c95c27e45d100fa-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.279 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.279 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.279 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.279 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:16.280 DEBUG:boto:Data: 2024-04-16T17:49:16.280 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.280 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.280 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.280 DEBUG:boto:Params: {} 2024-04-16T17:49:16.280 DEBUG:boto:Token: None 2024-04-16T17:49:16.280 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj 2024-04-16T17:49:16.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jL4N1gjdJEtS/fBkxyf8BjxZis0= 2024-04-16T17:49:16.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jL4N1gjdJEtS/fBkxyf8BjxZis0=', 'Content-Length': '0'} 2024-04-16T17:49:16.283 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006c375f1a9dadc467-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.283 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2024-04-16T17:49:16.283 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.284 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.284 DEBUG:boto:Method: GET 2024-04-16T17:49:16.284 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:16.284 DEBUG:boto:Data: 2024-04-16T17:49:16.284 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.284 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.284 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.284 DEBUG:boto:Params: {} 2024-04-16T17:49:16.284 DEBUG:boto:Token: None 2024-04-16T17:49:16.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj 2024-04-16T17:49:16.285 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pItbvypmHqAAMsQJal0EzRiEgig= 2024-04-16T17:49:16.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pItbvypmHqAAMsQJal0EzRiEgig=', 'Content-Length': '0'} 2024-04-16T17:49:16.288 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:49:14 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000000094c9ac4e375d8-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.330 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:16.331 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:16.331 DEBUG:boto:Method: GET 2024-04-16T17:49:16.331 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:16.331 DEBUG:boto:Data: 2024-04-16T17:49:16.331 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.331 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.331 DEBUG:boto:Params: {} 2024-04-16T17:49:16.331 DEBUG:boto:Token: None 2024-04-16T17:49:16.331 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/myobj 2024-04-16T17:49:16.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pItbvypmHqAAMsQJal0EzRiEgig= 2024-04-16T17:49:16.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pItbvypmHqAAMsQJal0EzRiEgig=', 'Content-Length': '0'} 2024-04-16T17:49:16.335 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a4fd2968bd612821-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.335 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.335 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.335 DEBUG:boto:path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.336 DEBUG:boto:auth_path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.336 DEBUG:boto:Method: GET 2024-04-16T17:49:16.336 DEBUG:boto:Path: /lybtug-10/?prefix=myobj 2024-04-16T17:49:16.336 DEBUG:boto:Data: 2024-04-16T17:49:16.336 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.336 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.336 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.336 DEBUG:boto:Params: {} 2024-04-16T17:49:16.336 DEBUG:boto:Token: None 2024-04-16T17:49:16.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/ 2024-04-16T17:49:16.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00= 2024-04-16T17:49:16.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00=', 'Content-Length': '0'} 2024-04-16T17:49:16.341 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2ed5c69aee977b7-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.342 DEBUG:boto:b'lybtug-10myobj1000falsemyobj2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.342 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:16.342 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:16.342 DEBUG:boto:path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.343 DEBUG:boto:auth_path=/lybtug-10/?prefix=myobj 2024-04-16T17:49:16.343 DEBUG:boto:Method: GET 2024-04-16T17:49:16.343 DEBUG:boto:Path: /lybtug-10/?prefix=myobj 2024-04-16T17:49:16.343 DEBUG:boto:Data: 2024-04-16T17:49:16.343 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:16.343 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.343 DEBUG:boto:Params: {} 2024-04-16T17:49:16.343 DEBUG:boto:Token: None 2024-04-16T17:49:16.343 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:16 GMT /lybtug-10/ 2024-04-16T17:49:16.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00= 2024-04-16T17:49:16.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ny/NHP2fZGh4YS0q1tULHGx5f00=', 'Content-Length': '0'} 2024-04-16T17:49:16.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b57671ac1081381-00661eba1c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.390 DEBUG:boto:b'lybtug-10myobj1000falsemyobj2024-04-16T17:49:09.988Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T17:49:16.391 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-10 zones={test-zone1, test-zone2} 2024-04-16T17:49:16.391 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:16.391 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:16.391 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.391 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:16.391 DEBUG:boto:Data: 2024-04-16T17:49:16.391 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.391 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.392 DEBUG:boto:Params: {} 2024-04-16T17:49:16.392 DEBUG:boto:Token: None 2024-04-16T17:49:16.392 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-9/ 2024-04-16T17:49:16.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:unBlMnc6Rqvp4bWcRhhQtrmgmQ8= 2024-04-16T17:49:16.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:unBlMnc6Rqvp4bWcRhhQtrmgmQ8=', 'Content-Length': '0'} 2024-04-16T17:49:16.396 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000383b57d91f1c6e59-00661eba1c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.396 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:16.396 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:16.396 DEBUG:boto:Method: HEAD 2024-04-16T17:49:16.396 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:16.396 DEBUG:boto:Data: 2024-04-16T17:49:16.396 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.396 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.396 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.397 DEBUG:boto:Params: {} 2024-04-16T17:49:16.397 DEBUG:boto:Token: None 2024-04-16T17:49:16.397 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:16 GMT /lybtug-9/myobj 2024-04-16T17:49:16.397 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:S593FibRAPZOEpdPjbS2rZ9RIGU= 2024-04-16T17:49:16.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:S593FibRAPZOEpdPjbS2rZ9RIGU=', 'Content-Length': '0'} 2024-04-16T17:49:16.400 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000002fb4ee57604838c-00661eba1c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.400 DEBUG:boto:path=/lybtug-9/myobj 2024-04-16T17:49:16.400 DEBUG:boto:auth_path=/lybtug-9/myobj 2024-04-16T17:49:16.400 DEBUG:boto:Method: DELETE 2024-04-16T17:49:16.401 DEBUG:boto:Path: /lybtug-9/myobj 2024-04-16T17:49:16.401 DEBUG:boto:Data: 2024-04-16T17:49:16.401 DEBUG:boto:Headers: {} 2024-04-16T17:49:16.401 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:16.401 DEBUG:boto:Port: 8000 2024-04-16T17:49:16.401 DEBUG:boto:Params: {} 2024-04-16T17:49:16.401 DEBUG:boto:Token: None 2024-04-16T17:49:16.401 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:49:16 GMT /lybtug-9/myobj 2024-04-16T17:49:16.401 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2yhZ0zz7Med+V4JXyYQOnBc0Kcs= 2024-04-16T17:49:16.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2yhZ0zz7Med+V4JXyYQOnBc0Kcs=', 'Content-Length': '0'} 2024-04-16T17:49:16.411 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000db315a8d1d9e06ba-00661eba1c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:16.411 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-9', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:16.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-9', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:16.411 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-9 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:49:16.462 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:16.462 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:16.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:16.540+0000 7f7d61912980 1 waiting for incremental sync to catch up: 2024-04-16T17:49:16.541 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.5.6, , , , , , , , , ] 2024-04-16T17:49:16.541 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000002.7.6, , , , , , , , , ] 2024-04-16T17:49:21.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:21.544+0000 7f7d61912980 1 waiting for incremental sync to catch up: 2024-04-16T17:49:21.544 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.5.6, , , , , , , , , ] 2024-04-16T17:49:21.544 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000002.7.6, , , , , , , , , ] 2024-04-16T17:49:26.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:26.548+0000 7f7d61912980 1 bucket sync caught up with source: 2024-04-16T17:49:26.547 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000002.7.6, , , , , , , , , ] 2024-04-16T17:49:26.547 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000002.7.6, , , , , , , , , ] 2024-04-16T17:49:26.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:26.548+0000 7f7d61912980 0 bucket checkpoint complete 2024-04-16T17:49:26.593 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-9 zones={test-zone2, test-zone1} 2024-04-16T17:49:26.593 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:26.594 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:26.594 DEBUG:boto:Method: HEAD 2024-04-16T17:49:26.594 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:26.594 DEBUG:boto:Data: 2024-04-16T17:49:26.594 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.594 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:26.594 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.594 DEBUG:boto:Params: {} 2024-04-16T17:49:26.594 DEBUG:boto:Token: None 2024-04-16T17:49:26.595 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:26 GMT /lybtug-9/ 2024-04-16T17:49:26.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pMT4btP+CP0bxdp3bvmyydfGrhw= 2024-04-16T17:49:26.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pMT4btP+CP0bxdp3bvmyydfGrhw=', 'Content-Length': '0'} 2024-04-16T17:49:26.598 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000678d1872224d1969-00661eba26-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.598 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:26.598 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:26.599 DEBUG:boto:Method: HEAD 2024-04-16T17:49:26.599 DEBUG:boto:Path: /lybtug-9/ 2024-04-16T17:49:26.599 DEBUG:boto:Data: 2024-04-16T17:49:26.599 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:26.599 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.599 DEBUG:boto:Params: {} 2024-04-16T17:49:26.599 DEBUG:boto:Token: None 2024-04-16T17:49:26.599 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:26 GMT /lybtug-9/ 2024-04-16T17:49:26.599 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pMT4btP+CP0bxdp3bvmyydfGrhw= 2024-04-16T17:49:26.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pMT4btP+CP0bxdp3bvmyydfGrhw=', 'Content-Length': '0'} 2024-04-16T17:49:26.603 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000062fab1b6e5697005-00661eba26-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.603 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:49:26.603 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:26.603 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:26.603 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.604 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.604 DEBUG:boto:Method: GET 2024-04-16T17:49:26.604 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.604 DEBUG:boto:Data: 2024-04-16T17:49:26.604 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:26.604 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.604 DEBUG:boto:Params: {} 2024-04-16T17:49:26.604 DEBUG:boto:Token: None 2024-04-16T17:49:26.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:26 GMT /lybtug-9/?versions 2024-04-16T17:49:26.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8= 2024-04-16T17:49:26.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8=', 'Content-Length': '0'} 2024-04-16T17:49:26.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007225a5f842d1e0a9-00661eba26-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.608 DEBUG:boto:b'lybtug-9999false' 2024-04-16T17:49:26.608 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:49:26.608 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:26.608 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:26.609 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.609 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.609 DEBUG:boto:Method: GET 2024-04-16T17:49:26.609 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.609 DEBUG:boto:Data: 2024-04-16T17:49:26.609 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.609 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:26.609 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.609 DEBUG:boto:Params: {} 2024-04-16T17:49:26.609 DEBUG:boto:Token: None 2024-04-16T17:49:26.609 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:26 GMT /lybtug-9/?versions 2024-04-16T17:49:26.610 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8= 2024-04-16T17:49:26.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8=', 'Content-Length': '0'} 2024-04-16T17:49:26.613 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009261eacceb7727f3-00661eba26-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.613 DEBUG:boto:b'lybtug-9999false' 2024-04-16T17:49:26.614 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:26.614 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:26.614 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.614 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.614 DEBUG:boto:Method: GET 2024-04-16T17:49:26.614 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.614 DEBUG:boto:Data: 2024-04-16T17:49:26.614 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.615 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:26.615 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.615 DEBUG:boto:Params: {} 2024-04-16T17:49:26.615 DEBUG:boto:Token: None 2024-04-16T17:49:26.615 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:26 GMT /lybtug-9/?versions 2024-04-16T17:49:26.615 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8= 2024-04-16T17:49:26.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8=', 'Content-Length': '0'} 2024-04-16T17:49:26.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b55bd76febcd5d97-00661eba26-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.619 DEBUG:boto:b'lybtug-9999false' 2024-04-16T17:49:26.619 DEBUG:boto:path=/lybtug-9/ 2024-04-16T17:49:26.619 DEBUG:boto:auth_path=/lybtug-9/ 2024-04-16T17:49:26.619 DEBUG:boto:path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.619 DEBUG:boto:auth_path=/lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.619 DEBUG:boto:Method: GET 2024-04-16T17:49:26.619 DEBUG:boto:Path: /lybtug-9/?versions&max-keys=999 2024-04-16T17:49:26.619 DEBUG:boto:Data: 2024-04-16T17:49:26.619 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.620 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:26.620 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.620 DEBUG:boto:Params: {} 2024-04-16T17:49:26.620 DEBUG:boto:Token: None 2024-04-16T17:49:26.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:26 GMT /lybtug-9/?versions 2024-04-16T17:49:26.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8= 2024-04-16T17:49:26.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s3OMSOSnKU+/NbXNTPDgTl2Syj8=', 'Content-Length': '0'} 2024-04-16T17:49:26.624 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000746c3c0b0a04bbd7-00661eba26-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.624 DEBUG:boto:b'lybtug-9999false' 2024-04-16T17:49:26.624 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-9 zones={test-zone2, test-zone1} 2024-04-16T17:49:26.624 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:26.624 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:26.624 DEBUG:boto:Method: HEAD 2024-04-16T17:49:26.624 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:26.625 DEBUG:boto:Data: 2024-04-16T17:49:26.625 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.625 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:26.625 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.625 DEBUG:boto:Params: {} 2024-04-16T17:49:26.625 DEBUG:boto:Token: None 2024-04-16T17:49:26.625 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:26 GMT /lybtug-10/ 2024-04-16T17:49:26.625 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I1mLBktB1wzIgvBLJ/F6l2D6A0k= 2024-04-16T17:49:26.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I1mLBktB1wzIgvBLJ/F6l2D6A0k=', 'Content-Length': '0'} 2024-04-16T17:49:26.629 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b98152deccccdcee-00661eba26-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.629 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:26.629 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:26.629 DEBUG:boto:Method: HEAD 2024-04-16T17:49:26.629 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:26.629 DEBUG:boto:Data: 2024-04-16T17:49:26.629 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.629 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:26.630 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.630 DEBUG:boto:Params: {} 2024-04-16T17:49:26.630 DEBUG:boto:Token: None 2024-04-16T17:49:26.630 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:26 GMT /lybtug-10/myobj 2024-04-16T17:49:26.630 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Y9osynOVBPtIH3yg13KspuR8bqQ= 2024-04-16T17:49:26.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Y9osynOVBPtIH3yg13KspuR8bqQ=', 'Content-Length': '0'} 2024-04-16T17:49:26.632 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:49:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000011c253b793036d68-00661eba26-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.633 DEBUG:boto:path=/lybtug-10/myobj 2024-04-16T17:49:26.633 DEBUG:boto:auth_path=/lybtug-10/myobj 2024-04-16T17:49:26.633 DEBUG:boto:Method: DELETE 2024-04-16T17:49:26.633 DEBUG:boto:Path: /lybtug-10/myobj 2024-04-16T17:49:26.633 DEBUG:boto:Data: 2024-04-16T17:49:26.633 DEBUG:boto:Headers: {} 2024-04-16T17:49:26.633 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:26.633 DEBUG:boto:Port: 8000 2024-04-16T17:49:26.633 DEBUG:boto:Params: {} 2024-04-16T17:49:26.633 DEBUG:boto:Token: None 2024-04-16T17:49:26.633 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:49:26 GMT /lybtug-10/myobj 2024-04-16T17:49:26.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VQog7aCeAdj/voGHgbB9HJ3zMhA= 2024-04-16T17:49:26.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VQog7aCeAdj/voGHgbB9HJ3zMhA=', 'Content-Length': '0'} 2024-04-16T17:49:26.641 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000045ef7516c023aa55-00661eba26-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:26.642 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-10', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:26.642 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-10', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:26.642 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-10 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:49:26.693 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:26.693 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:26.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:26.772+0000 7fb5ed3b6980 1 waiting for incremental sync to catch up: 2024-04-16T17:49:26.774 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.7.6, , , , , , , , , ] 2024-04-16T17:49:26.774 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.11.6, , , , , , , , , ] 2024-04-16T17:49:31.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:31.772+0000 7fb5ed3b6980 1 waiting for incremental sync to catch up: 2024-04-16T17:49:31.776 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.7.6, , , , , , , , , ] 2024-04-16T17:49:31.776 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.11.6, , , , , , , , , ] 2024-04-16T17:49:36.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:36.776+0000 7fb5ed3b6980 1 bucket sync caught up with source: 2024-04-16T17:49:36.779 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.11.6, , , , , , , , , ] 2024-04-16T17:49:36.779 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.11.6, , , , , , , , , ] 2024-04-16T17:49:36.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:49:36.776+0000 7fb5ed3b6980 0 bucket checkpoint complete 2024-04-16T17:49:36.823 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-10 zones={test-zone1, test-zone2} 2024-04-16T17:49:36.823 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:36.824 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:36.824 DEBUG:boto:Method: HEAD 2024-04-16T17:49:36.824 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:36.824 DEBUG:boto:Data: 2024-04-16T17:49:36.824 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:36.824 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.824 DEBUG:boto:Params: {} 2024-04-16T17:49:36.825 DEBUG:boto:Token: None 2024-04-16T17:49:36.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:36 GMT /lybtug-10/ 2024-04-16T17:49:36.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XcRHym21/6pjn3pitoPRnOO4N9M= 2024-04-16T17:49:36.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XcRHym21/6pjn3pitoPRnOO4N9M=', 'Content-Length': '0'} 2024-04-16T17:49:36.828 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ec2f92491d9f6880-00661eba30-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.829 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:36.829 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:36.829 DEBUG:boto:Method: HEAD 2024-04-16T17:49:36.829 DEBUG:boto:Path: /lybtug-10/ 2024-04-16T17:49:36.829 DEBUG:boto:Data: 2024-04-16T17:49:36.829 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.829 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:36.829 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.829 DEBUG:boto:Params: {} 2024-04-16T17:49:36.830 DEBUG:boto:Token: None 2024-04-16T17:49:36.830 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:36 GMT /lybtug-10/ 2024-04-16T17:49:36.830 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XcRHym21/6pjn3pitoPRnOO4N9M= 2024-04-16T17:49:36.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XcRHym21/6pjn3pitoPRnOO4N9M=', 'Content-Length': '0'} 2024-04-16T17:49:36.834 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000098d7ba5492b7823e-00661eba30-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.834 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:49:36.834 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:36.835 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:36.835 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.835 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.835 DEBUG:boto:Method: GET 2024-04-16T17:49:36.835 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.835 DEBUG:boto:Data: 2024-04-16T17:49:36.835 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.835 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:36.835 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.835 DEBUG:boto:Params: {} 2024-04-16T17:49:36.835 DEBUG:boto:Token: None 2024-04-16T17:49:36.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:36 GMT /lybtug-10/?versions 2024-04-16T17:49:36.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU= 2024-04-16T17:49:36.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU=', 'Content-Length': '0'} 2024-04-16T17:49:36.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002c95b4fd7ee57fc-00661eba30-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.882 DEBUG:boto:b'lybtug-10999false' 2024-04-16T17:49:36.883 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:49:36.883 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:36.883 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:36.883 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.883 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.883 DEBUG:boto:Method: GET 2024-04-16T17:49:36.883 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.884 DEBUG:boto:Data: 2024-04-16T17:49:36.884 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:36.884 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.884 DEBUG:boto:Params: {} 2024-04-16T17:49:36.884 DEBUG:boto:Token: None 2024-04-16T17:49:36.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:36 GMT /lybtug-10/?versions 2024-04-16T17:49:36.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU= 2024-04-16T17:49:36.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU=', 'Content-Length': '0'} 2024-04-16T17:49:36.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009ebe478941a0028-00661eba30-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.888 DEBUG:boto:b'lybtug-10999false' 2024-04-16T17:49:36.889 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:36.889 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:36.889 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.889 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.889 DEBUG:boto:Method: GET 2024-04-16T17:49:36.889 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.889 DEBUG:boto:Data: 2024-04-16T17:49:36.889 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.889 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:36.889 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.890 DEBUG:boto:Params: {} 2024-04-16T17:49:36.890 DEBUG:boto:Token: None 2024-04-16T17:49:36.890 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:36 GMT /lybtug-10/?versions 2024-04-16T17:49:36.890 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU= 2024-04-16T17:49:36.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU=', 'Content-Length': '0'} 2024-04-16T17:49:36.894 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddf2a1e851003adb-00661eba30-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.894 DEBUG:boto:b'lybtug-10999false' 2024-04-16T17:49:36.894 DEBUG:boto:path=/lybtug-10/ 2024-04-16T17:49:36.894 DEBUG:boto:auth_path=/lybtug-10/ 2024-04-16T17:49:36.895 DEBUG:boto:path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.895 DEBUG:boto:auth_path=/lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.895 DEBUG:boto:Method: GET 2024-04-16T17:49:36.895 DEBUG:boto:Path: /lybtug-10/?versions&max-keys=999 2024-04-16T17:49:36.895 DEBUG:boto:Data: 2024-04-16T17:49:36.895 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.895 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:36.895 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.895 DEBUG:boto:Params: {} 2024-04-16T17:49:36.895 DEBUG:boto:Token: None 2024-04-16T17:49:36.896 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:49:36 GMT /lybtug-10/?versions 2024-04-16T17:49:36.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU= 2024-04-16T17:49:36.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4C/uqFrjBYMMjUCPer4zeSBTzpU=', 'Content-Length': '0'} 2024-04-16T17:49:36.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004631fd295f007772-00661eba30-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.900 DEBUG:boto:b'lybtug-10999false' 2024-04-16T17:49:36.900 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-10 zones={test-zone1, test-zone2} 2024-04-16T17:49:36.900 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_delete ... ok 2024-04-16T17:49:36.901 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-11 2024-04-16T17:49:36.901 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:36.901 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:36.901 DEBUG:boto:Method: PUT 2024-04-16T17:49:36.902 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:36.902 DEBUG:boto:Data: 2024-04-16T17:49:36.902 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.902 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:36.902 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.902 DEBUG:boto:Params: {} 2024-04-16T17:49:36.902 DEBUG:boto:Token: None 2024-04-16T17:49:36.902 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:49:36 GMT /lybtug-11/ 2024-04-16T17:49:36.902 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1G7WmcqS9DJH0kZBehjdo1BMkd0= 2024-04-16T17:49:36.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1G7WmcqS9DJH0kZBehjdo1BMkd0=', 'Content-Length': '0'} 2024-04-16T17:49:36.936 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000299693dd133182d6-00661eba30-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.936 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-12 2024-04-16T17:49:36.936 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:36.937 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:36.937 DEBUG:boto:Method: PUT 2024-04-16T17:49:36.937 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:36.937 DEBUG:boto:Data: 2024-04-16T17:49:36.937 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.937 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:36.937 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.937 DEBUG:boto:Params: {} 2024-04-16T17:49:36.937 DEBUG:boto:Token: None 2024-04-16T17:49:36.937 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:49:36 GMT /lybtug-12/ 2024-04-16T17:49:36.938 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bQD9LZ+WD+9ItTqXemlug8QVF/U= 2024-04-16T17:49:36.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bQD9LZ+WD+9ItTqXemlug8QVF/U=', 'Content-Length': '0'} 2024-04-16T17:49:36.997 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c8d4ceaa73485b46-00661eba30-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:36.997 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:36.997 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:36.997 DEBUG:boto:Method: HEAD 2024-04-16T17:49:36.997 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:36.997 DEBUG:boto:Data: 2024-04-16T17:49:36.997 DEBUG:boto:Headers: {} 2024-04-16T17:49:36.997 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:36.997 DEBUG:boto:Port: 8000 2024-04-16T17:49:36.998 DEBUG:boto:Params: {} 2024-04-16T17:49:36.998 DEBUG:boto:Token: None 2024-04-16T17:49:36.998 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:36 GMT /lybtug-11/ 2024-04-16T17:49:36.998 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+4nI2cfescxoCQNgVybKJmkjjMA= 2024-04-16T17:49:36.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+4nI2cfescxoCQNgVybKJmkjjMA=', 'Content-Length': '0'} 2024-04-16T17:49:37.002 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d56e9bb04ba7bdef-00661eba30-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.002 DEBUG:boto:path=/lybtug-11/obj1 2024-04-16T17:49:37.002 DEBUG:boto:auth_path=/lybtug-11/obj1 2024-04-16T17:49:37.002 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.003 DEBUG:boto:Path: /lybtug-11/obj1 2024-04-16T17:49:37.003 DEBUG:boto:Data: 2024-04-16T17:49:37.003 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.003 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.003 DEBUG:boto:Params: {} 2024-04-16T17:49:37.003 DEBUG:boto:Token: None 2024-04-16T17:49:37.003 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj1 2024-04-16T17:49:37.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:INvo8xgnJQpOp3mxVp9On1G6eTg= 2024-04-16T17:49:37.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:INvo8xgnJQpOp3mxVp9On1G6eTg='} 2024-04-16T17:49:37.017 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003fb9fb8f177deaa5-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.017 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.017 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.017 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.018 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.018 DEBUG:boto:Data: 2024-04-16T17:49:37.018 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.018 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.018 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.018 DEBUG:boto:Params: {} 2024-04-16T17:49:37.018 DEBUG:boto:Token: None 2024-04-16T17:49:37.018 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.019 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.022 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b2be6a979b991ad3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.022 DEBUG:boto:path=/lybtug-11/obj2 2024-04-16T17:49:37.023 DEBUG:boto:auth_path=/lybtug-11/obj2 2024-04-16T17:49:37.023 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.023 DEBUG:boto:Path: /lybtug-11/obj2 2024-04-16T17:49:37.023 DEBUG:boto:Data: 2024-04-16T17:49:37.023 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.023 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.023 DEBUG:boto:Params: {} 2024-04-16T17:49:37.023 DEBUG:boto:Token: None 2024-04-16T17:49:37.023 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj2 2024-04-16T17:49:37.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:f3kCL/ITxfPGSoqD9X0pTss/foE= 2024-04-16T17:49:37.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:f3kCL/ITxfPGSoqD9X0pTss/foE='} 2024-04-16T17:49:37.035 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bcbfeda5c61972c8-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.035 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.036 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.036 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.036 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.036 DEBUG:boto:Data: 2024-04-16T17:49:37.036 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.037 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.037 DEBUG:boto:Params: {} 2024-04-16T17:49:37.037 DEBUG:boto:Token: None 2024-04-16T17:49:37.037 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.041 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cd8d97435f58a758-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.042 DEBUG:boto:path=/lybtug-11/obj3 2024-04-16T17:49:37.042 DEBUG:boto:auth_path=/lybtug-11/obj3 2024-04-16T17:49:37.042 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.042 DEBUG:boto:Path: /lybtug-11/obj3 2024-04-16T17:49:37.042 DEBUG:boto:Data: 2024-04-16T17:49:37.042 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.043 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.043 DEBUG:boto:Params: {} 2024-04-16T17:49:37.043 DEBUG:boto:Token: None 2024-04-16T17:49:37.043 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj3 2024-04-16T17:49:37.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8QnnPiS5DI7DvFe7Q0G5NRLgMko= 2024-04-16T17:49:37.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8QnnPiS5DI7DvFe7Q0G5NRLgMko='} 2024-04-16T17:49:37.087 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006728f057c4d3f1d9-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.087 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.087 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.087 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.087 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.087 DEBUG:boto:Data: 2024-04-16T17:49:37.087 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.088 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.088 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.088 DEBUG:boto:Params: {} 2024-04-16T17:49:37.088 DEBUG:boto:Token: None 2024-04-16T17:49:37.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.092 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a6acd8f7a1001632-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.092 DEBUG:boto:path=/lybtug-11/obj4 2024-04-16T17:49:37.092 DEBUG:boto:auth_path=/lybtug-11/obj4 2024-04-16T17:49:37.092 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.093 DEBUG:boto:Path: /lybtug-11/obj4 2024-04-16T17:49:37.093 DEBUG:boto:Data: 2024-04-16T17:49:37.093 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.093 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.093 DEBUG:boto:Params: {} 2024-04-16T17:49:37.093 DEBUG:boto:Token: None 2024-04-16T17:49:37.094 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj4 2024-04-16T17:49:37.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9sbutZVG3Ts+UM6lrDr3e7+dq1s= 2024-04-16T17:49:37.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9sbutZVG3Ts+UM6lrDr3e7+dq1s='} 2024-04-16T17:49:37.106 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005e140dfd74f62adf-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.106 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.106 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.106 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.106 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.107 DEBUG:boto:Data: 2024-04-16T17:49:37.107 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.107 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.107 DEBUG:boto:Params: {} 2024-04-16T17:49:37.107 DEBUG:boto:Token: None 2024-04-16T17:49:37.107 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.107 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.111 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ac0da4903729ff4b-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.111 DEBUG:boto:path=/lybtug-11/obj5 2024-04-16T17:49:37.111 DEBUG:boto:auth_path=/lybtug-11/obj5 2024-04-16T17:49:37.111 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.111 DEBUG:boto:Path: /lybtug-11/obj5 2024-04-16T17:49:37.111 DEBUG:boto:Data: 2024-04-16T17:49:37.112 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.112 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.112 DEBUG:boto:Params: {} 2024-04-16T17:49:37.112 DEBUG:boto:Token: None 2024-04-16T17:49:37.112 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj5 2024-04-16T17:49:37.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xF2QLJRFucQVqvjs5AISxDTvvm0= 2024-04-16T17:49:37.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xF2QLJRFucQVqvjs5AISxDTvvm0='} 2024-04-16T17:49:37.154 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c81ad494128c8c86-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.155 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.155 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.155 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.155 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.155 DEBUG:boto:Data: 2024-04-16T17:49:37.155 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.155 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.155 DEBUG:boto:Params: {} 2024-04-16T17:49:37.156 DEBUG:boto:Token: None 2024-04-16T17:49:37.156 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.160 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002896d6ee6b96d95f-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.160 DEBUG:boto:path=/lybtug-11/obj6 2024-04-16T17:49:37.160 DEBUG:boto:auth_path=/lybtug-11/obj6 2024-04-16T17:49:37.160 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.160 DEBUG:boto:Path: /lybtug-11/obj6 2024-04-16T17:49:37.160 DEBUG:boto:Data: 2024-04-16T17:49:37.160 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.160 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.161 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.161 DEBUG:boto:Params: {} 2024-04-16T17:49:37.161 DEBUG:boto:Token: None 2024-04-16T17:49:37.161 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj6 2024-04-16T17:49:37.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G0IYHPmx1nuoM8UuqPOkvdyiMSg= 2024-04-16T17:49:37.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G0IYHPmx1nuoM8UuqPOkvdyiMSg='} 2024-04-16T17:49:37.169 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fae36a1fe50ee0bb-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.169 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.170 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.170 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.170 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.170 DEBUG:boto:Data: 2024-04-16T17:49:37.170 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.170 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.170 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.170 DEBUG:boto:Params: {} 2024-04-16T17:49:37.170 DEBUG:boto:Token: None 2024-04-16T17:49:37.170 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.174 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000109c75f15d94feff-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.174 DEBUG:boto:path=/lybtug-11/obj7 2024-04-16T17:49:37.175 DEBUG:boto:auth_path=/lybtug-11/obj7 2024-04-16T17:49:37.175 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.175 DEBUG:boto:Path: /lybtug-11/obj7 2024-04-16T17:49:37.175 DEBUG:boto:Data: 2024-04-16T17:49:37.175 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.175 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.175 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.175 DEBUG:boto:Params: {} 2024-04-16T17:49:37.175 DEBUG:boto:Token: None 2024-04-16T17:49:37.175 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj7 2024-04-16T17:49:37.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bdqon9w5AFwbIsEGmU/Th7mkZF4= 2024-04-16T17:49:37.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bdqon9w5AFwbIsEGmU/Th7mkZF4='} 2024-04-16T17:49:37.218 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005fdaae6a9c7894d9-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.219 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.219 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.219 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.219 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.219 DEBUG:boto:Data: 2024-04-16T17:49:37.219 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.219 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.219 DEBUG:boto:Params: {} 2024-04-16T17:49:37.219 DEBUG:boto:Token: None 2024-04-16T17:49:37.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.223 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000652dfaffc6bcfc61-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.224 DEBUG:boto:path=/lybtug-11/obj8 2024-04-16T17:49:37.224 DEBUG:boto:auth_path=/lybtug-11/obj8 2024-04-16T17:49:37.224 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.224 DEBUG:boto:Path: /lybtug-11/obj8 2024-04-16T17:49:37.224 DEBUG:boto:Data: 2024-04-16T17:49:37.224 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.224 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.224 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.224 DEBUG:boto:Params: {} 2024-04-16T17:49:37.224 DEBUG:boto:Token: None 2024-04-16T17:49:37.225 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj8 2024-04-16T17:49:37.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tM1zL++4AQJbhxhsTZPJoV79Gwc= 2024-04-16T17:49:37.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tM1zL++4AQJbhxhsTZPJoV79Gwc='} 2024-04-16T17:49:37.233 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000de4df081249d3e73-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.233 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.234 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.234 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.234 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.234 DEBUG:boto:Data: 2024-04-16T17:49:37.234 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.234 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.234 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.234 DEBUG:boto:Params: {} 2024-04-16T17:49:37.234 DEBUG:boto:Token: None 2024-04-16T17:49:37.234 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.234 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.238 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000199e6af2f8859f2b-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.239 DEBUG:boto:path=/lybtug-11/obj9 2024-04-16T17:49:37.239 DEBUG:boto:auth_path=/lybtug-11/obj9 2024-04-16T17:49:37.239 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.239 DEBUG:boto:Path: /lybtug-11/obj9 2024-04-16T17:49:37.239 DEBUG:boto:Data: 2024-04-16T17:49:37.239 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.239 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.239 DEBUG:boto:Params: {} 2024-04-16T17:49:37.239 DEBUG:boto:Token: None 2024-04-16T17:49:37.240 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj9 2024-04-16T17:49:37.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dThEF7O+q7D6fpC5E+EMkaB584s= 2024-04-16T17:49:37.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dThEF7O+q7D6fpC5E+EMkaB584s='} 2024-04-16T17:49:37.286 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a603cb0aee0209a8-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.287 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.287 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.287 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.287 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.287 DEBUG:boto:Data: 2024-04-16T17:49:37.287 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.287 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.287 DEBUG:boto:Params: {} 2024-04-16T17:49:37.287 DEBUG:boto:Token: None 2024-04-16T17:49:37.288 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.291 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a8dafb5bddef76fc-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.291 DEBUG:boto:path=/lybtug-11/obj10 2024-04-16T17:49:37.291 DEBUG:boto:auth_path=/lybtug-11/obj10 2024-04-16T17:49:37.291 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.292 DEBUG:boto:Path: /lybtug-11/obj10 2024-04-16T17:49:37.292 DEBUG:boto:Data: 2024-04-16T17:49:37.292 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.292 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.292 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.292 DEBUG:boto:Params: {} 2024-04-16T17:49:37.292 DEBUG:boto:Token: None 2024-04-16T17:49:37.292 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj10 2024-04-16T17:49:37.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AvePScqxATQPCvOtmnlK4HCJlcY= 2024-04-16T17:49:37.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AvePScqxATQPCvOtmnlK4HCJlcY='} 2024-04-16T17:49:37.300 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000492476cecd5754cf-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.301 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.301 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.301 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.301 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.301 DEBUG:boto:Data: 2024-04-16T17:49:37.301 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.301 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.301 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.301 DEBUG:boto:Params: {} 2024-04-16T17:49:37.302 DEBUG:boto:Token: None 2024-04-16T17:49:37.302 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.305 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009f87e60327feb6b1-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.305 DEBUG:boto:path=/lybtug-11/obj11 2024-04-16T17:49:37.306 DEBUG:boto:auth_path=/lybtug-11/obj11 2024-04-16T17:49:37.306 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.306 DEBUG:boto:Path: /lybtug-11/obj11 2024-04-16T17:49:37.306 DEBUG:boto:Data: 2024-04-16T17:49:37.306 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.306 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.306 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.306 DEBUG:boto:Params: {} 2024-04-16T17:49:37.306 DEBUG:boto:Token: None 2024-04-16T17:49:37.307 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj11 2024-04-16T17:49:37.307 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ch/0xxJfLbL+SZnkcvBeTGHGXaA= 2024-04-16T17:49:37.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ch/0xxJfLbL+SZnkcvBeTGHGXaA='} 2024-04-16T17:49:37.350 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000066d3cad71a0658dd-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.351 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.351 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.351 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.351 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.351 DEBUG:boto:Data: 2024-04-16T17:49:37.351 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.351 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.351 DEBUG:boto:Params: {} 2024-04-16T17:49:37.352 DEBUG:boto:Token: None 2024-04-16T17:49:37.352 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.355 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009dfe4edc4de00ed7-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.356 DEBUG:boto:path=/lybtug-11/obj12 2024-04-16T17:49:37.356 DEBUG:boto:auth_path=/lybtug-11/obj12 2024-04-16T17:49:37.356 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.356 DEBUG:boto:Path: /lybtug-11/obj12 2024-04-16T17:49:37.356 DEBUG:boto:Data: 2024-04-16T17:49:37.356 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.356 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.356 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.356 DEBUG:boto:Params: {} 2024-04-16T17:49:37.356 DEBUG:boto:Token: None 2024-04-16T17:49:37.357 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj12 2024-04-16T17:49:37.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DfRtWdpKh+sZCDPItgpRJFqv0dg= 2024-04-16T17:49:37.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DfRtWdpKh+sZCDPItgpRJFqv0dg='} 2024-04-16T17:49:37.366 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d98a68fbb5bebc79-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.366 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.366 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.366 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.366 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.366 DEBUG:boto:Data: 2024-04-16T17:49:37.366 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.366 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.366 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.367 DEBUG:boto:Params: {} 2024-04-16T17:49:37.367 DEBUG:boto:Token: None 2024-04-16T17:49:37.367 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.367 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.371 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000000da18c3d9b77588-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.371 DEBUG:boto:path=/lybtug-11/obj13 2024-04-16T17:49:37.371 DEBUG:boto:auth_path=/lybtug-11/obj13 2024-04-16T17:49:37.371 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.371 DEBUG:boto:Path: /lybtug-11/obj13 2024-04-16T17:49:37.371 DEBUG:boto:Data: 2024-04-16T17:49:37.371 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.372 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.372 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.372 DEBUG:boto:Params: {} 2024-04-16T17:49:37.372 DEBUG:boto:Token: None 2024-04-16T17:49:37.372 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj13 2024-04-16T17:49:37.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Sa8PeFn7yU2nv5HtbT3vopvQb9Y= 2024-04-16T17:49:37.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sa8PeFn7yU2nv5HtbT3vopvQb9Y='} 2024-04-16T17:49:37.415 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000086d684f2400c99cd-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.415 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.415 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.415 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.415 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.415 DEBUG:boto:Data: 2024-04-16T17:49:37.415 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.415 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.415 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.415 DEBUG:boto:Params: {} 2024-04-16T17:49:37.415 DEBUG:boto:Token: None 2024-04-16T17:49:37.416 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.419 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f82b4bd3e45537f9-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.420 DEBUG:boto:path=/lybtug-11/obj14 2024-04-16T17:49:37.420 DEBUG:boto:auth_path=/lybtug-11/obj14 2024-04-16T17:49:37.420 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.420 DEBUG:boto:Path: /lybtug-11/obj14 2024-04-16T17:49:37.420 DEBUG:boto:Data: 2024-04-16T17:49:37.420 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.420 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.420 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.420 DEBUG:boto:Params: {} 2024-04-16T17:49:37.420 DEBUG:boto:Token: None 2024-04-16T17:49:37.421 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj14 2024-04-16T17:49:37.421 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:evB/39NHU64CyzNb8E14FONRWRE= 2024-04-16T17:49:37.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:evB/39NHU64CyzNb8E14FONRWRE='} 2024-04-16T17:49:37.429 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc5d7a0b887c9c59-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.429 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.430 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.430 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.430 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.430 DEBUG:boto:Data: 2024-04-16T17:49:37.430 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.430 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.430 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.430 DEBUG:boto:Params: {} 2024-04-16T17:49:37.430 DEBUG:boto:Token: None 2024-04-16T17:49:37.430 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.434 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a32f60f56a521a64-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.434 DEBUG:boto:path=/lybtug-11/obj15 2024-04-16T17:49:37.435 DEBUG:boto:auth_path=/lybtug-11/obj15 2024-04-16T17:49:37.435 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.435 DEBUG:boto:Path: /lybtug-11/obj15 2024-04-16T17:49:37.435 DEBUG:boto:Data: 2024-04-16T17:49:37.435 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.435 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.435 DEBUG:boto:Params: {} 2024-04-16T17:49:37.435 DEBUG:boto:Token: None 2024-04-16T17:49:37.435 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj15 2024-04-16T17:49:37.435 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I5wslan6/JElJqWvxeKYTsf+4TA= 2024-04-16T17:49:37.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I5wslan6/JElJqWvxeKYTsf+4TA='} 2024-04-16T17:49:37.478 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000064029d9cbf6a5390-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.479 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.479 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.479 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.479 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.479 DEBUG:boto:Data: 2024-04-16T17:49:37.479 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.479 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.479 DEBUG:boto:Params: {} 2024-04-16T17:49:37.479 DEBUG:boto:Token: None 2024-04-16T17:49:37.479 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.483 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002f957a443ec6d7c2-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.483 DEBUG:boto:path=/lybtug-11/obj16 2024-04-16T17:49:37.484 DEBUG:boto:auth_path=/lybtug-11/obj16 2024-04-16T17:49:37.484 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.484 DEBUG:boto:Path: /lybtug-11/obj16 2024-04-16T17:49:37.484 DEBUG:boto:Data: 2024-04-16T17:49:37.484 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.484 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.484 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.484 DEBUG:boto:Params: {} 2024-04-16T17:49:37.484 DEBUG:boto:Token: None 2024-04-16T17:49:37.484 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj16 2024-04-16T17:49:37.485 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GoETj96Lrbb2f4gv6Fcc32V1RZo= 2024-04-16T17:49:37.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GoETj96Lrbb2f4gv6Fcc32V1RZo='} 2024-04-16T17:49:37.496 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005548bef2e0bee951-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.496 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.496 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.496 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.496 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.496 DEBUG:boto:Data: 2024-04-16T17:49:37.496 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.496 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.496 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.497 DEBUG:boto:Params: {} 2024-04-16T17:49:37.497 DEBUG:boto:Token: None 2024-04-16T17:49:37.497 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.500 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cec63116b06c1896-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.500 DEBUG:boto:path=/lybtug-11/obj17 2024-04-16T17:49:37.501 DEBUG:boto:auth_path=/lybtug-11/obj17 2024-04-16T17:49:37.501 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.501 DEBUG:boto:Path: /lybtug-11/obj17 2024-04-16T17:49:37.501 DEBUG:boto:Data: 2024-04-16T17:49:37.501 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.501 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.501 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.501 DEBUG:boto:Params: {} 2024-04-16T17:49:37.501 DEBUG:boto:Token: None 2024-04-16T17:49:37.501 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj17 2024-04-16T17:49:37.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:me1hVUYxMp88mzZKMxHeZQmX5Pw= 2024-04-16T17:49:37.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:me1hVUYxMp88mzZKMxHeZQmX5Pw='} 2024-04-16T17:49:37.546 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000092301b145fc5b2eb-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.547 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.547 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.547 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.547 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.547 DEBUG:boto:Data: 2024-04-16T17:49:37.547 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.547 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.547 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.547 DEBUG:boto:Params: {} 2024-04-16T17:49:37.547 DEBUG:boto:Token: None 2024-04-16T17:49:37.547 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.551 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004c07fcc310a82df3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.551 DEBUG:boto:path=/lybtug-11/obj18 2024-04-16T17:49:37.551 DEBUG:boto:auth_path=/lybtug-11/obj18 2024-04-16T17:49:37.552 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.552 DEBUG:boto:Path: /lybtug-11/obj18 2024-04-16T17:49:37.552 DEBUG:boto:Data: 2024-04-16T17:49:37.552 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.552 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.552 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.552 DEBUG:boto:Params: {} 2024-04-16T17:49:37.552 DEBUG:boto:Token: None 2024-04-16T17:49:37.552 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj18 2024-04-16T17:49:37.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wr5FNon9mGeiiB2orsUQBdENxHA= 2024-04-16T17:49:37.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wr5FNon9mGeiiB2orsUQBdENxHA='} 2024-04-16T17:49:37.561 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c4a1af5c937c606b-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.561 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.561 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.561 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.561 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.561 DEBUG:boto:Data: 2024-04-16T17:49:37.561 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.561 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.561 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.561 DEBUG:boto:Params: {} 2024-04-16T17:49:37.562 DEBUG:boto:Token: None 2024-04-16T17:49:37.562 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.562 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.566 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '108'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000543970cd3d5520e4-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.566 DEBUG:boto:path=/lybtug-11/obj19 2024-04-16T17:49:37.566 DEBUG:boto:auth_path=/lybtug-11/obj19 2024-04-16T17:49:37.566 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.566 DEBUG:boto:Path: /lybtug-11/obj19 2024-04-16T17:49:37.566 DEBUG:boto:Data: 2024-04-16T17:49:37.566 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.567 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.567 DEBUG:boto:Params: {} 2024-04-16T17:49:37.567 DEBUG:boto:Token: None 2024-04-16T17:49:37.567 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj19 2024-04-16T17:49:37.567 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+SORG6Zu5yx9fvPAODwtcaUAEwo= 2024-04-16T17:49:37.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+SORG6Zu5yx9fvPAODwtcaUAEwo='} 2024-04-16T17:49:37.611 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b7db5e13668ff5ea-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.611 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.611 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.611 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.611 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.611 DEBUG:boto:Data: 2024-04-16T17:49:37.611 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.611 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.611 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.611 DEBUG:boto:Params: {} 2024-04-16T17:49:37.612 DEBUG:boto:Token: None 2024-04-16T17:49:37.612 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.615 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000078a470ca0541bc61-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.616 DEBUG:boto:path=/lybtug-11/obj20 2024-04-16T17:49:37.616 DEBUG:boto:auth_path=/lybtug-11/obj20 2024-04-16T17:49:37.616 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.616 DEBUG:boto:Path: /lybtug-11/obj20 2024-04-16T17:49:37.616 DEBUG:boto:Data: 2024-04-16T17:49:37.616 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.616 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.616 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.616 DEBUG:boto:Params: {} 2024-04-16T17:49:37.616 DEBUG:boto:Token: None 2024-04-16T17:49:37.617 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj20 2024-04-16T17:49:37.617 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZkE1gTVR7UarDXwAPV2f3iWvKBA= 2024-04-16T17:49:37.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZkE1gTVR7UarDXwAPV2f3iWvKBA='} 2024-04-16T17:49:37.628 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003dcaeac8b2cd7bc8-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.628 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.628 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.629 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.629 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.629 DEBUG:boto:Data: 2024-04-16T17:49:37.629 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.629 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.629 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.629 DEBUG:boto:Params: {} 2024-04-16T17:49:37.629 DEBUG:boto:Token: None 2024-04-16T17:49:37.629 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.629 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.633 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000e485fd929f81006-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.633 DEBUG:boto:path=/lybtug-11/obj21 2024-04-16T17:49:37.633 DEBUG:boto:auth_path=/lybtug-11/obj21 2024-04-16T17:49:37.634 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.634 DEBUG:boto:Path: /lybtug-11/obj21 2024-04-16T17:49:37.634 DEBUG:boto:Data: 2024-04-16T17:49:37.634 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.634 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.634 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.634 DEBUG:boto:Params: {} 2024-04-16T17:49:37.634 DEBUG:boto:Token: None 2024-04-16T17:49:37.634 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj21 2024-04-16T17:49:37.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3XcYtcsMEKG2Ndt6e43JhB4D3T4= 2024-04-16T17:49:37.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3XcYtcsMEKG2Ndt6e43JhB4D3T4='} 2024-04-16T17:49:37.678 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005b04a8003152359a-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.679 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.679 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.679 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.679 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.679 DEBUG:boto:Data: 2024-04-16T17:49:37.679 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.679 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.679 DEBUG:boto:Params: {} 2024-04-16T17:49:37.679 DEBUG:boto:Token: None 2024-04-16T17:49:37.680 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.683 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000d6824abaadeab79-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.684 DEBUG:boto:path=/lybtug-11/obj22 2024-04-16T17:49:37.684 DEBUG:boto:auth_path=/lybtug-11/obj22 2024-04-16T17:49:37.684 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.684 DEBUG:boto:Path: /lybtug-11/obj22 2024-04-16T17:49:37.684 DEBUG:boto:Data: 2024-04-16T17:49:37.684 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.684 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.684 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.684 DEBUG:boto:Params: {} 2024-04-16T17:49:37.684 DEBUG:boto:Token: None 2024-04-16T17:49:37.685 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj22 2024-04-16T17:49:37.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+55DjsEPpu0v0TFtmQNeFIX9Yj8= 2024-04-16T17:49:37.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+55DjsEPpu0v0TFtmQNeFIX9Yj8='} 2024-04-16T17:49:37.692 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000208ca2b65feacf4a-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.693 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.693 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.693 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.693 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.693 DEBUG:boto:Data: 2024-04-16T17:49:37.693 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.693 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.693 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.693 DEBUG:boto:Params: {} 2024-04-16T17:49:37.693 DEBUG:boto:Token: None 2024-04-16T17:49:37.693 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.694 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.697 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '132'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000051b78a348ac82b17-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.698 DEBUG:boto:path=/lybtug-11/obj23 2024-04-16T17:49:37.698 DEBUG:boto:auth_path=/lybtug-11/obj23 2024-04-16T17:49:37.698 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.698 DEBUG:boto:Path: /lybtug-11/obj23 2024-04-16T17:49:37.698 DEBUG:boto:Data: 2024-04-16T17:49:37.698 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.698 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.698 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.698 DEBUG:boto:Params: {} 2024-04-16T17:49:37.698 DEBUG:boto:Token: None 2024-04-16T17:49:37.699 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj23 2024-04-16T17:49:37.699 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kcJUutwtrqGuEC/hZo087+ce2W0= 2024-04-16T17:49:37.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kcJUutwtrqGuEC/hZo087+ce2W0='} 2024-04-16T17:49:37.742 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006cee05114966e1e3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.743 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.743 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.743 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.743 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.743 DEBUG:boto:Data: 2024-04-16T17:49:37.743 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.743 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.743 DEBUG:boto:Params: {} 2024-04-16T17:49:37.743 DEBUG:boto:Token: None 2024-04-16T17:49:37.744 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.748 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '138'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000af3f7f9026a45092-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.748 DEBUG:boto:path=/lybtug-11/obj24 2024-04-16T17:49:37.748 DEBUG:boto:auth_path=/lybtug-11/obj24 2024-04-16T17:49:37.748 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.748 DEBUG:boto:Path: /lybtug-11/obj24 2024-04-16T17:49:37.748 DEBUG:boto:Data: 2024-04-16T17:49:37.748 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.748 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.748 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.749 DEBUG:boto:Params: {} 2024-04-16T17:49:37.749 DEBUG:boto:Token: None 2024-04-16T17:49:37.749 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj24 2024-04-16T17:49:37.749 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0Io2owG7509qiM8EjpC8PQCUlww= 2024-04-16T17:49:37.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0Io2owG7509qiM8EjpC8PQCUlww='} 2024-04-16T17:49:37.758 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000062c36e860e0e3248-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.759 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.759 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.759 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.759 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.759 DEBUG:boto:Data: 2024-04-16T17:49:37.759 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.759 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.759 DEBUG:boto:Params: {} 2024-04-16T17:49:37.759 DEBUG:boto:Token: None 2024-04-16T17:49:37.759 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.763 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '144'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000059c59a501116edcd-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.763 DEBUG:boto:path=/lybtug-11/obj25 2024-04-16T17:49:37.764 DEBUG:boto:auth_path=/lybtug-11/obj25 2024-04-16T17:49:37.764 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.764 DEBUG:boto:Path: /lybtug-11/obj25 2024-04-16T17:49:37.764 DEBUG:boto:Data: 2024-04-16T17:49:37.764 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.764 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.764 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.764 DEBUG:boto:Params: {} 2024-04-16T17:49:37.764 DEBUG:boto:Token: None 2024-04-16T17:49:37.764 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj25 2024-04-16T17:49:37.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:apc+/MysuMt/Sm9EZQpYdPJHr3A= 2024-04-16T17:49:37.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:apc+/MysuMt/Sm9EZQpYdPJHr3A='} 2024-04-16T17:49:37.807 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009354657e465774bb-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.807 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.807 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.807 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.807 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.807 DEBUG:boto:Data: 2024-04-16T17:49:37.807 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.807 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.807 DEBUG:boto:Params: {} 2024-04-16T17:49:37.807 DEBUG:boto:Token: None 2024-04-16T17:49:37.808 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.812 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c4494081b93243c2-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.812 DEBUG:boto:path=/lybtug-11/obj26 2024-04-16T17:49:37.812 DEBUG:boto:auth_path=/lybtug-11/obj26 2024-04-16T17:49:37.812 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.812 DEBUG:boto:Path: /lybtug-11/obj26 2024-04-16T17:49:37.812 DEBUG:boto:Data: 2024-04-16T17:49:37.812 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.812 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.813 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.813 DEBUG:boto:Params: {} 2024-04-16T17:49:37.813 DEBUG:boto:Token: None 2024-04-16T17:49:37.813 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj26 2024-04-16T17:49:37.813 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I5doxI9wVVye8jw+F53XRY+cynQ= 2024-04-16T17:49:37.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I5doxI9wVVye8jw+F53XRY+cynQ='} 2024-04-16T17:49:37.822 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000ae13908654a22c4-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.822 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.822 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.822 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.822 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.822 DEBUG:boto:Data: 2024-04-16T17:49:37.822 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.822 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.822 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.823 DEBUG:boto:Params: {} 2024-04-16T17:49:37.823 DEBUG:boto:Token: None 2024-04-16T17:49:37.823 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.823 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.827 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '156'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000325cf4699ff159f3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.827 DEBUG:boto:path=/lybtug-11/obj27 2024-04-16T17:49:37.827 DEBUG:boto:auth_path=/lybtug-11/obj27 2024-04-16T17:49:37.827 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.827 DEBUG:boto:Path: /lybtug-11/obj27 2024-04-16T17:49:37.827 DEBUG:boto:Data: 2024-04-16T17:49:37.827 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.827 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.827 DEBUG:boto:Params: {} 2024-04-16T17:49:37.828 DEBUG:boto:Token: None 2024-04-16T17:49:37.828 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj27 2024-04-16T17:49:37.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SnSuX9KiFXyGSsc8uYlsQgmfUHg= 2024-04-16T17:49:37.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SnSuX9KiFXyGSsc8uYlsQgmfUHg='} 2024-04-16T17:49:37.870 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000017e41a052140b03f-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.871 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.871 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.871 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.871 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.871 DEBUG:boto:Data: 2024-04-16T17:49:37.871 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.871 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.871 DEBUG:boto:Params: {} 2024-04-16T17:49:37.871 DEBUG:boto:Token: None 2024-04-16T17:49:37.871 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.875 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003fe76d0ee5b4f82e-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.876 DEBUG:boto:path=/lybtug-11/obj28 2024-04-16T17:49:37.876 DEBUG:boto:auth_path=/lybtug-11/obj28 2024-04-16T17:49:37.876 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.876 DEBUG:boto:Path: /lybtug-11/obj28 2024-04-16T17:49:37.876 DEBUG:boto:Data: 2024-04-16T17:49:37.876 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.876 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.876 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.876 DEBUG:boto:Params: {} 2024-04-16T17:49:37.876 DEBUG:boto:Token: None 2024-04-16T17:49:37.877 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj28 2024-04-16T17:49:37.877 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g+uGI1z/un9nqYAUtaL7/HrQahQ= 2024-04-16T17:49:37.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g+uGI1z/un9nqYAUtaL7/HrQahQ='} 2024-04-16T17:49:37.885 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008a54faf378fd7e52-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.885 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.885 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.886 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.886 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.886 DEBUG:boto:Data: 2024-04-16T17:49:37.886 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.886 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.886 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.886 DEBUG:boto:Params: {} 2024-04-16T17:49:37.886 DEBUG:boto:Token: None 2024-04-16T17:49:37.886 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.890 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '168'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b07c907c41f43bb5-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.890 DEBUG:boto:path=/lybtug-11/obj29 2024-04-16T17:49:37.890 DEBUG:boto:auth_path=/lybtug-11/obj29 2024-04-16T17:49:37.891 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.891 DEBUG:boto:Path: /lybtug-11/obj29 2024-04-16T17:49:37.891 DEBUG:boto:Data: 2024-04-16T17:49:37.891 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.891 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.891 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.891 DEBUG:boto:Params: {} 2024-04-16T17:49:37.891 DEBUG:boto:Token: None 2024-04-16T17:49:37.891 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj29 2024-04-16T17:49:37.891 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cdhZqxTE5M/50AXFqynLGpxrJPw= 2024-04-16T17:49:37.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cdhZqxTE5M/50AXFqynLGpxrJPw='} 2024-04-16T17:49:37.934 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000738ff7419325bc36-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.935 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.935 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.935 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.935 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.935 DEBUG:boto:Data: 2024-04-16T17:49:37.935 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.935 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.935 DEBUG:boto:Params: {} 2024-04-16T17:49:37.935 DEBUG:boto:Token: None 2024-04-16T17:49:37.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.939 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000713060d5798afa8f-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.939 DEBUG:boto:path=/lybtug-11/obj30 2024-04-16T17:49:37.939 DEBUG:boto:auth_path=/lybtug-11/obj30 2024-04-16T17:49:37.940 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.940 DEBUG:boto:Path: /lybtug-11/obj30 2024-04-16T17:49:37.940 DEBUG:boto:Data: 2024-04-16T17:49:37.940 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.940 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.940 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.940 DEBUG:boto:Params: {} 2024-04-16T17:49:37.940 DEBUG:boto:Token: None 2024-04-16T17:49:37.940 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj30 2024-04-16T17:49:37.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MmPUD9GK4Yhh3jY0IFlnhH7ZkZE= 2024-04-16T17:49:37.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MmPUD9GK4Yhh3jY0IFlnhH7ZkZE='} 2024-04-16T17:49:37.950 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d9e8da8aa527bd30-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.950 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.950 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.950 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.950 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.950 DEBUG:boto:Data: 2024-04-16T17:49:37.950 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.951 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.951 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.951 DEBUG:boto:Params: {} 2024-04-16T17:49:37.951 DEBUG:boto:Token: None 2024-04-16T17:49:37.951 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:37.951 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:37.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:37.955 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e27564638851a9af-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.955 DEBUG:boto:path=/lybtug-11/obj31 2024-04-16T17:49:37.955 DEBUG:boto:auth_path=/lybtug-11/obj31 2024-04-16T17:49:37.955 DEBUG:boto:Method: PUT 2024-04-16T17:49:37.955 DEBUG:boto:Path: /lybtug-11/obj31 2024-04-16T17:49:37.955 DEBUG:boto:Data: 2024-04-16T17:49:37.955 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:37.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.955 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.955 DEBUG:boto:Params: {} 2024-04-16T17:49:37.956 DEBUG:boto:Token: None 2024-04-16T17:49:37.956 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/obj31 2024-04-16T17:49:37.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gO6fFoWp3CkcyLqKbNLpGD6zLaQ= 2024-04-16T17:49:37.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gO6fFoWp3CkcyLqKbNLpGD6zLaQ='} 2024-04-16T17:49:37.998 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005d8ce63aa48c249c-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:37.999 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:37.999 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:37.999 DEBUG:boto:Method: HEAD 2024-04-16T17:49:37.999 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:37.999 DEBUG:boto:Data: 2024-04-16T17:49:37.999 DEBUG:boto:Headers: {} 2024-04-16T17:49:37.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:37.999 DEBUG:boto:Port: 8000 2024-04-16T17:49:37.999 DEBUG:boto:Params: {} 2024-04-16T17:49:37.999 DEBUG:boto:Token: None 2024-04-16T17:49:37.999 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:37 GMT /lybtug-11/ 2024-04-16T17:49:38.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T17:49:38.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T17:49:38.003 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '186'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000baf7a586454c7959-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.003 DEBUG:boto:path=/lybtug-11/obj32 2024-04-16T17:49:38.003 DEBUG:boto:auth_path=/lybtug-11/obj32 2024-04-16T17:49:38.004 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.004 DEBUG:boto:Path: /lybtug-11/obj32 2024-04-16T17:49:38.004 DEBUG:boto:Data: 2024-04-16T17:49:38.004 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.004 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.004 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.004 DEBUG:boto:Params: {} 2024-04-16T17:49:38.004 DEBUG:boto:Token: None 2024-04-16T17:49:38.004 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj32 2024-04-16T17:49:38.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nSJXONsxbWbP5dggGtmfSWG4hX4= 2024-04-16T17:49:38.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nSJXONsxbWbP5dggGtmfSWG4hX4='} 2024-04-16T17:49:38.013 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000053e8086f7d64db64-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.013 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.013 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.013 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.013 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.013 DEBUG:boto:Data: 2024-04-16T17:49:38.013 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.014 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.014 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.014 DEBUG:boto:Params: {} 2024-04-16T17:49:38.014 DEBUG:boto:Token: None 2024-04-16T17:49:38.014 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.014 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.018 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '192'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000062b5011f27ea32e-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.018 DEBUG:boto:path=/lybtug-11/obj33 2024-04-16T17:49:38.018 DEBUG:boto:auth_path=/lybtug-11/obj33 2024-04-16T17:49:38.018 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.018 DEBUG:boto:Path: /lybtug-11/obj33 2024-04-16T17:49:38.018 DEBUG:boto:Data: 2024-04-16T17:49:38.018 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.018 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.018 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.019 DEBUG:boto:Params: {} 2024-04-16T17:49:38.019 DEBUG:boto:Token: None 2024-04-16T17:49:38.019 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj33 2024-04-16T17:49:38.019 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+OmWgNPuAPt2Tu1wMXyBHXTciu0= 2024-04-16T17:49:38.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+OmWgNPuAPt2Tu1wMXyBHXTciu0='} 2024-04-16T17:49:38.062 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000032e355110a42153-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.063 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.063 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.063 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.063 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.063 DEBUG:boto:Data: 2024-04-16T17:49:38.063 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.063 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.063 DEBUG:boto:Params: {} 2024-04-16T17:49:38.063 DEBUG:boto:Token: None 2024-04-16T17:49:38.064 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.067 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '198'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003b7b523a4cba74b8-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.067 DEBUG:boto:path=/lybtug-11/obj34 2024-04-16T17:49:38.067 DEBUG:boto:auth_path=/lybtug-11/obj34 2024-04-16T17:49:38.068 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.068 DEBUG:boto:Path: /lybtug-11/obj34 2024-04-16T17:49:38.068 DEBUG:boto:Data: 2024-04-16T17:49:38.068 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.068 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.068 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.068 DEBUG:boto:Params: {} 2024-04-16T17:49:38.068 DEBUG:boto:Token: None 2024-04-16T17:49:38.068 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj34 2024-04-16T17:49:38.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jWbZAPuTyoRnfZxADwO1cQtyvtg= 2024-04-16T17:49:38.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jWbZAPuTyoRnfZxADwO1cQtyvtg='} 2024-04-16T17:49:38.077 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000033f1ebfbcfb26d5-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.077 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.077 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.077 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.077 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.077 DEBUG:boto:Data: 2024-04-16T17:49:38.077 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.077 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.078 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.078 DEBUG:boto:Params: {} 2024-04-16T17:49:38.078 DEBUG:boto:Token: None 2024-04-16T17:49:38.078 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.078 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.082 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '204'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eb4932ad781b3221-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.082 DEBUG:boto:path=/lybtug-11/obj35 2024-04-16T17:49:38.082 DEBUG:boto:auth_path=/lybtug-11/obj35 2024-04-16T17:49:38.082 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.082 DEBUG:boto:Path: /lybtug-11/obj35 2024-04-16T17:49:38.082 DEBUG:boto:Data: 2024-04-16T17:49:38.082 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.083 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.083 DEBUG:boto:Params: {} 2024-04-16T17:49:38.083 DEBUG:boto:Token: None 2024-04-16T17:49:38.083 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj35 2024-04-16T17:49:38.083 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5UrE8rZHMsVivEMGEyvlYN624EQ= 2024-04-16T17:49:38.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5UrE8rZHMsVivEMGEyvlYN624EQ='} 2024-04-16T17:49:38.126 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e91d66a1ee085214-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.127 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.127 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.127 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.127 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.127 DEBUG:boto:Data: 2024-04-16T17:49:38.127 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.127 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.127 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.127 DEBUG:boto:Params: {} 2024-04-16T17:49:38.127 DEBUG:boto:Token: None 2024-04-16T17:49:38.127 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.132 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '210'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c84f03d3d5cfb396-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.132 DEBUG:boto:path=/lybtug-11/obj36 2024-04-16T17:49:38.132 DEBUG:boto:auth_path=/lybtug-11/obj36 2024-04-16T17:49:38.132 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.132 DEBUG:boto:Path: /lybtug-11/obj36 2024-04-16T17:49:38.132 DEBUG:boto:Data: 2024-04-16T17:49:38.132 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.133 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.133 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.133 DEBUG:boto:Params: {} 2024-04-16T17:49:38.133 DEBUG:boto:Token: None 2024-04-16T17:49:38.133 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj36 2024-04-16T17:49:38.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lcOQc8jSHyWsUQbMYtTPRKxKW2A= 2024-04-16T17:49:38.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lcOQc8jSHyWsUQbMYtTPRKxKW2A='} 2024-04-16T17:49:38.142 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b33d50fbb02ce03d-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.142 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.142 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.142 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.142 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.142 DEBUG:boto:Data: 2024-04-16T17:49:38.142 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.142 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.143 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.143 DEBUG:boto:Params: {} 2024-04-16T17:49:38.143 DEBUG:boto:Token: None 2024-04-16T17:49:38.143 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.143 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.147 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '36'), ('X-RGW-Bytes-Used', '216'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066e6a1b6c36cd40b-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.147 DEBUG:boto:path=/lybtug-11/obj37 2024-04-16T17:49:38.147 DEBUG:boto:auth_path=/lybtug-11/obj37 2024-04-16T17:49:38.147 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.147 DEBUG:boto:Path: /lybtug-11/obj37 2024-04-16T17:49:38.147 DEBUG:boto:Data: 2024-04-16T17:49:38.147 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.147 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.148 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.148 DEBUG:boto:Params: {} 2024-04-16T17:49:38.148 DEBUG:boto:Token: None 2024-04-16T17:49:38.148 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj37 2024-04-16T17:49:38.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XA5wa9RO68SvrNhKTtrmxlNampk= 2024-04-16T17:49:38.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XA5wa9RO68SvrNhKTtrmxlNampk='} 2024-04-16T17:49:38.190 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c1602d6e9d100448-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.191 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.191 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.191 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.191 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.191 DEBUG:boto:Data: 2024-04-16T17:49:38.191 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.191 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.191 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.191 DEBUG:boto:Params: {} 2024-04-16T17:49:38.191 DEBUG:boto:Token: None 2024-04-16T17:49:38.191 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.196 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '222'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f50e80b8c213adfa-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.196 DEBUG:boto:path=/lybtug-11/obj38 2024-04-16T17:49:38.196 DEBUG:boto:auth_path=/lybtug-11/obj38 2024-04-16T17:49:38.196 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.196 DEBUG:boto:Path: /lybtug-11/obj38 2024-04-16T17:49:38.196 DEBUG:boto:Data: 2024-04-16T17:49:38.196 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.197 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.197 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.197 DEBUG:boto:Params: {} 2024-04-16T17:49:38.197 DEBUG:boto:Token: None 2024-04-16T17:49:38.197 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj38 2024-04-16T17:49:38.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DZAhoytqZa+MlCf0ltR5AkPVUYI= 2024-04-16T17:49:38.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DZAhoytqZa+MlCf0ltR5AkPVUYI='} 2024-04-16T17:49:38.205 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc60bd14b4bbd99c-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.206 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.206 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.206 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.206 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.206 DEBUG:boto:Data: 2024-04-16T17:49:38.206 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.206 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.206 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.206 DEBUG:boto:Params: {} 2024-04-16T17:49:38.206 DEBUG:boto:Token: None 2024-04-16T17:49:38.207 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.210 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '228'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008558f53791349cdd-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.211 DEBUG:boto:path=/lybtug-11/obj39 2024-04-16T17:49:38.211 DEBUG:boto:auth_path=/lybtug-11/obj39 2024-04-16T17:49:38.211 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.211 DEBUG:boto:Path: /lybtug-11/obj39 2024-04-16T17:49:38.211 DEBUG:boto:Data: 2024-04-16T17:49:38.211 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.211 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.211 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.211 DEBUG:boto:Params: {} 2024-04-16T17:49:38.211 DEBUG:boto:Token: None 2024-04-16T17:49:38.211 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj39 2024-04-16T17:49:38.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BgryOXDWZCC5Tw6iL5yIwoHYKow= 2024-04-16T17:49:38.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BgryOXDWZCC5Tw6iL5yIwoHYKow='} 2024-04-16T17:49:38.254 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000aa2e1a2a48b9c9e3-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.255 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.255 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.255 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.255 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.255 DEBUG:boto:Data: 2024-04-16T17:49:38.255 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.255 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.255 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.255 DEBUG:boto:Params: {} 2024-04-16T17:49:38.255 DEBUG:boto:Token: None 2024-04-16T17:49:38.256 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.259 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '39'), ('X-RGW-Bytes-Used', '234'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b95192304f65decc-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.259 DEBUG:boto:path=/lybtug-11/obj40 2024-04-16T17:49:38.260 DEBUG:boto:auth_path=/lybtug-11/obj40 2024-04-16T17:49:38.260 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.260 DEBUG:boto:Path: /lybtug-11/obj40 2024-04-16T17:49:38.260 DEBUG:boto:Data: 2024-04-16T17:49:38.260 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.260 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.260 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.260 DEBUG:boto:Params: {} 2024-04-16T17:49:38.260 DEBUG:boto:Token: None 2024-04-16T17:49:38.260 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj40 2024-04-16T17:49:38.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IU6ydphsCCc72GlaYfFiCvC06oA= 2024-04-16T17:49:38.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IU6ydphsCCc72GlaYfFiCvC06oA='} 2024-04-16T17:49:38.269 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008471487ec0da0375-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.269 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.269 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.269 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.270 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.270 DEBUG:boto:Data: 2024-04-16T17:49:38.270 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.270 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.270 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.270 DEBUG:boto:Params: {} 2024-04-16T17:49:38.270 DEBUG:boto:Token: None 2024-04-16T17:49:38.270 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.270 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.274 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fb0bae1d8b7d40ab-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.275 DEBUG:boto:path=/lybtug-11/obj41 2024-04-16T17:49:38.275 DEBUG:boto:auth_path=/lybtug-11/obj41 2024-04-16T17:49:38.275 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.275 DEBUG:boto:Path: /lybtug-11/obj41 2024-04-16T17:49:38.275 DEBUG:boto:Data: 2024-04-16T17:49:38.275 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.275 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.275 DEBUG:boto:Params: {} 2024-04-16T17:49:38.275 DEBUG:boto:Token: None 2024-04-16T17:49:38.276 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj41 2024-04-16T17:49:38.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZMn0jhZIbwGTtr+RdBp3FPn1yzA= 2024-04-16T17:49:38.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZMn0jhZIbwGTtr+RdBp3FPn1yzA='} 2024-04-16T17:49:38.318 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007701e755dfefa32c-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.319 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.319 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.319 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.319 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.319 DEBUG:boto:Data: 2024-04-16T17:49:38.319 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.319 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.319 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.319 DEBUG:boto:Params: {} 2024-04-16T17:49:38.319 DEBUG:boto:Token: None 2024-04-16T17:49:38.320 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.323 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '41'), ('X-RGW-Bytes-Used', '246'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000a0940aaa2c43cba-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.324 DEBUG:boto:path=/lybtug-11/obj42 2024-04-16T17:49:38.324 DEBUG:boto:auth_path=/lybtug-11/obj42 2024-04-16T17:49:38.324 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.324 DEBUG:boto:Path: /lybtug-11/obj42 2024-04-16T17:49:38.324 DEBUG:boto:Data: 2024-04-16T17:49:38.324 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.324 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.324 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.324 DEBUG:boto:Params: {} 2024-04-16T17:49:38.324 DEBUG:boto:Token: None 2024-04-16T17:49:38.324 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj42 2024-04-16T17:49:38.325 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+GmPJavXsp3TlitVaBio8l+T/50= 2024-04-16T17:49:38.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+GmPJavXsp3TlitVaBio8l+T/50='} 2024-04-16T17:49:38.332 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000c9f920fcddeb739-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.333 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.333 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.333 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.333 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.333 DEBUG:boto:Data: 2024-04-16T17:49:38.333 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.333 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.333 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.333 DEBUG:boto:Params: {} 2024-04-16T17:49:38.333 DEBUG:boto:Token: None 2024-04-16T17:49:38.333 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.334 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.338 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '252'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b5d714269f7e4e56-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.338 DEBUG:boto:path=/lybtug-11/obj43 2024-04-16T17:49:38.338 DEBUG:boto:auth_path=/lybtug-11/obj43 2024-04-16T17:49:38.339 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.339 DEBUG:boto:Path: /lybtug-11/obj43 2024-04-16T17:49:38.339 DEBUG:boto:Data: 2024-04-16T17:49:38.339 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.339 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.339 DEBUG:boto:Params: {} 2024-04-16T17:49:38.339 DEBUG:boto:Token: None 2024-04-16T17:49:38.339 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj43 2024-04-16T17:49:38.339 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iEwlnTuoEhozkenpmnrLuv2y95M= 2024-04-16T17:49:38.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iEwlnTuoEhozkenpmnrLuv2y95M='} 2024-04-16T17:49:38.382 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ed9b0dfe86ff0cec-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.383 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.383 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.383 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.383 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.383 DEBUG:boto:Data: 2024-04-16T17:49:38.383 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.383 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.383 DEBUG:boto:Params: {} 2024-04-16T17:49:38.383 DEBUG:boto:Token: None 2024-04-16T17:49:38.383 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.387 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '43'), ('X-RGW-Bytes-Used', '258'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000798e2f2b653eaa00-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.387 DEBUG:boto:path=/lybtug-11/obj44 2024-04-16T17:49:38.388 DEBUG:boto:auth_path=/lybtug-11/obj44 2024-04-16T17:49:38.388 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.388 DEBUG:boto:Path: /lybtug-11/obj44 2024-04-16T17:49:38.388 DEBUG:boto:Data: 2024-04-16T17:49:38.388 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.388 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.388 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.388 DEBUG:boto:Params: {} 2024-04-16T17:49:38.388 DEBUG:boto:Token: None 2024-04-16T17:49:38.388 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj44 2024-04-16T17:49:38.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0cGSFZrkyomi5eD4FrrNQv3q9YE= 2024-04-16T17:49:38.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0cGSFZrkyomi5eD4FrrNQv3q9YE='} 2024-04-16T17:49:38.401 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004deb9fde98ec6856-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.401 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.401 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.401 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.401 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.401 DEBUG:boto:Data: 2024-04-16T17:49:38.401 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.402 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.402 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.402 DEBUG:boto:Params: {} 2024-04-16T17:49:38.402 DEBUG:boto:Token: None 2024-04-16T17:49:38.402 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.402 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.405 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '44'), ('X-RGW-Bytes-Used', '264'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050b5b141ce3b5c76-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.406 DEBUG:boto:path=/lybtug-11/obj45 2024-04-16T17:49:38.406 DEBUG:boto:auth_path=/lybtug-11/obj45 2024-04-16T17:49:38.406 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.406 DEBUG:boto:Path: /lybtug-11/obj45 2024-04-16T17:49:38.406 DEBUG:boto:Data: 2024-04-16T17:49:38.406 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.406 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.406 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.407 DEBUG:boto:Params: {} 2024-04-16T17:49:38.407 DEBUG:boto:Token: None 2024-04-16T17:49:38.407 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj45 2024-04-16T17:49:38.407 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ld29GMPp6mGph4y/TwWN5HtPH/Y= 2024-04-16T17:49:38.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ld29GMPp6mGph4y/TwWN5HtPH/Y='} 2024-04-16T17:49:38.450 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f94320fa674b89b9-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.451 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.451 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.451 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.451 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.451 DEBUG:boto:Data: 2024-04-16T17:49:38.451 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.451 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.451 DEBUG:boto:Params: {} 2024-04-16T17:49:38.451 DEBUG:boto:Token: None 2024-04-16T17:49:38.451 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.455 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '270'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005be30bb7bea1f009-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.455 DEBUG:boto:path=/lybtug-11/obj46 2024-04-16T17:49:38.455 DEBUG:boto:auth_path=/lybtug-11/obj46 2024-04-16T17:49:38.455 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.455 DEBUG:boto:Path: /lybtug-11/obj46 2024-04-16T17:49:38.456 DEBUG:boto:Data: 2024-04-16T17:49:38.456 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.456 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.456 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.456 DEBUG:boto:Params: {} 2024-04-16T17:49:38.456 DEBUG:boto:Token: None 2024-04-16T17:49:38.456 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj46 2024-04-16T17:49:38.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G7YUIIc/PbkafIVyUI8tDfvNP0o= 2024-04-16T17:49:38.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G7YUIIc/PbkafIVyUI8tDfvNP0o='} 2024-04-16T17:49:38.465 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000041b8d3d213d2fab6-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.465 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.465 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.465 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.465 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.465 DEBUG:boto:Data: 2024-04-16T17:49:38.465 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.465 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.465 DEBUG:boto:Params: {} 2024-04-16T17:49:38.466 DEBUG:boto:Token: None 2024-04-16T17:49:38.466 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.469 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '46'), ('X-RGW-Bytes-Used', '276'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000083267c889adc2b06-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.469 DEBUG:boto:path=/lybtug-11/obj47 2024-04-16T17:49:38.470 DEBUG:boto:auth_path=/lybtug-11/obj47 2024-04-16T17:49:38.470 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.470 DEBUG:boto:Path: /lybtug-11/obj47 2024-04-16T17:49:38.470 DEBUG:boto:Data: 2024-04-16T17:49:38.470 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.470 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.470 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.470 DEBUG:boto:Params: {} 2024-04-16T17:49:38.470 DEBUG:boto:Token: None 2024-04-16T17:49:38.470 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj47 2024-04-16T17:49:38.471 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6YFPKG9UijJYjMJj8SWbIyJwgvE= 2024-04-16T17:49:38.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6YFPKG9UijJYjMJj8SWbIyJwgvE='} 2024-04-16T17:49:38.514 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000043bd1b19d9892df2-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.515 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.515 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.515 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.515 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.515 DEBUG:boto:Data: 2024-04-16T17:49:38.515 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.515 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.515 DEBUG:boto:Params: {} 2024-04-16T17:49:38.515 DEBUG:boto:Token: None 2024-04-16T17:49:38.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.520 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '47'), ('X-RGW-Bytes-Used', '282'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bb9e5d24c573a955-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.520 DEBUG:boto:path=/lybtug-11/obj48 2024-04-16T17:49:38.520 DEBUG:boto:auth_path=/lybtug-11/obj48 2024-04-16T17:49:38.520 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.520 DEBUG:boto:Path: /lybtug-11/obj48 2024-04-16T17:49:38.520 DEBUG:boto:Data: 2024-04-16T17:49:38.520 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.521 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.521 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.521 DEBUG:boto:Params: {} 2024-04-16T17:49:38.521 DEBUG:boto:Token: None 2024-04-16T17:49:38.521 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj48 2024-04-16T17:49:38.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CxrUgzvMKChWJzYN4VX3FprBJnQ= 2024-04-16T17:49:38.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CxrUgzvMKChWJzYN4VX3FprBJnQ='} 2024-04-16T17:49:38.530 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fe4865108ca8c0e1-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.530 DEBUG:boto:path=/lybtug-11/ 2024-04-16T17:49:38.530 DEBUG:boto:auth_path=/lybtug-11/ 2024-04-16T17:49:38.530 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.530 DEBUG:boto:Path: /lybtug-11/ 2024-04-16T17:49:38.530 DEBUG:boto:Data: 2024-04-16T17:49:38.530 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.530 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.530 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.530 DEBUG:boto:Params: {} 2024-04-16T17:49:38.531 DEBUG:boto:Token: None 2024-04-16T17:49:38.531 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/ 2024-04-16T17:49:38.531 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T17:49:38.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T17:49:38.535 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '48'), ('X-RGW-Bytes-Used', '288'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000354404913400807e-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.535 DEBUG:boto:path=/lybtug-11/obj49 2024-04-16T17:49:38.535 DEBUG:boto:auth_path=/lybtug-11/obj49 2024-04-16T17:49:38.535 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.535 DEBUG:boto:Path: /lybtug-11/obj49 2024-04-16T17:49:38.535 DEBUG:boto:Data: 2024-04-16T17:49:38.535 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.536 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:49:38.536 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.536 DEBUG:boto:Params: {} 2024-04-16T17:49:38.536 DEBUG:boto:Token: None 2024-04-16T17:49:38.536 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-11/obj49 2024-04-16T17:49:38.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nmbb1GDSmAGbP4XVt6NRQ0l3N2k= 2024-04-16T17:49:38.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nmbb1GDSmAGbP4XVt6NRQ0l3N2k='} 2024-04-16T17:49:38.578 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000054358e9c3f76abb6-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.579 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.579 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.579 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.579 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.579 DEBUG:boto:Data: 2024-04-16T17:49:38.579 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.579 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.579 DEBUG:boto:Params: {} 2024-04-16T17:49:38.580 DEBUG:boto:Token: None 2024-04-16T17:49:38.580 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.584 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000080486a6223627db-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.584 DEBUG:boto:path=/lybtug-12/obj1 2024-04-16T17:49:38.584 DEBUG:boto:auth_path=/lybtug-12/obj1 2024-04-16T17:49:38.584 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.585 DEBUG:boto:Path: /lybtug-12/obj1 2024-04-16T17:49:38.585 DEBUG:boto:Data: 2024-04-16T17:49:38.585 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.585 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.585 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.585 DEBUG:boto:Params: {} 2024-04-16T17:49:38.585 DEBUG:boto:Token: None 2024-04-16T17:49:38.585 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj1 2024-04-16T17:49:38.585 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:erC5YB3G4fN1EmxF1CkZB+s/poQ= 2024-04-16T17:49:38.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:erC5YB3G4fN1EmxF1CkZB+s/poQ='} 2024-04-16T17:49:38.602 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000060921bc1ae71b5bb-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.602 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.602 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.602 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.603 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.603 DEBUG:boto:Data: 2024-04-16T17:49:38.603 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.603 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.603 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.603 DEBUG:boto:Params: {} 2024-04-16T17:49:38.603 DEBUG:boto:Token: None 2024-04-16T17:49:38.603 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.603 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.607 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fc5de81100e99ce9-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.608 DEBUG:boto:path=/lybtug-12/obj2 2024-04-16T17:49:38.608 DEBUG:boto:auth_path=/lybtug-12/obj2 2024-04-16T17:49:38.608 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.608 DEBUG:boto:Path: /lybtug-12/obj2 2024-04-16T17:49:38.608 DEBUG:boto:Data: 2024-04-16T17:49:38.608 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.608 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.608 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.608 DEBUG:boto:Params: {} 2024-04-16T17:49:38.608 DEBUG:boto:Token: None 2024-04-16T17:49:38.609 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj2 2024-04-16T17:49:38.609 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oCMc5d37rBC1ez90PzzVeS6lcsA= 2024-04-16T17:49:38.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oCMc5d37rBC1ez90PzzVeS6lcsA='} 2024-04-16T17:49:38.654 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009a2dfbc399227c59-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.655 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.655 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.655 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.655 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.655 DEBUG:boto:Data: 2024-04-16T17:49:38.655 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.655 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.655 DEBUG:boto:Params: {} 2024-04-16T17:49:38.655 DEBUG:boto:Token: None 2024-04-16T17:49:38.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.659 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050980cde1a8aab6f-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.660 DEBUG:boto:path=/lybtug-12/obj3 2024-04-16T17:49:38.660 DEBUG:boto:auth_path=/lybtug-12/obj3 2024-04-16T17:49:38.660 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.660 DEBUG:boto:Path: /lybtug-12/obj3 2024-04-16T17:49:38.660 DEBUG:boto:Data: 2024-04-16T17:49:38.660 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.660 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.660 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.660 DEBUG:boto:Params: {} 2024-04-16T17:49:38.660 DEBUG:boto:Token: None 2024-04-16T17:49:38.661 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj3 2024-04-16T17:49:38.661 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9w0RWRIEP8BHR42DwiIMtzg/ZaQ= 2024-04-16T17:49:38.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9w0RWRIEP8BHR42DwiIMtzg/ZaQ='} 2024-04-16T17:49:38.673 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b06eb23b25367837-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.673 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.674 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.674 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.674 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.674 DEBUG:boto:Data: 2024-04-16T17:49:38.674 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.674 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.674 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.674 DEBUG:boto:Params: {} 2024-04-16T17:49:38.674 DEBUG:boto:Token: None 2024-04-16T17:49:38.674 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.674 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.678 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a6afe447b0ab4b55-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.678 DEBUG:boto:path=/lybtug-12/obj4 2024-04-16T17:49:38.679 DEBUG:boto:auth_path=/lybtug-12/obj4 2024-04-16T17:49:38.679 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.679 DEBUG:boto:Path: /lybtug-12/obj4 2024-04-16T17:49:38.679 DEBUG:boto:Data: 2024-04-16T17:49:38.679 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.679 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.679 DEBUG:boto:Params: {} 2024-04-16T17:49:38.679 DEBUG:boto:Token: None 2024-04-16T17:49:38.680 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj4 2024-04-16T17:49:38.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z/njFxMvvB21eYeX3c6+kjcNAjE= 2024-04-16T17:49:38.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z/njFxMvvB21eYeX3c6+kjcNAjE='} 2024-04-16T17:49:38.722 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000034c14f09ab536ea7-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.723 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.723 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.723 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.723 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.723 DEBUG:boto:Data: 2024-04-16T17:49:38.723 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.723 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.723 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.723 DEBUG:boto:Params: {} 2024-04-16T17:49:38.723 DEBUG:boto:Token: None 2024-04-16T17:49:38.723 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.728 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000140249a9745b5dab-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.728 DEBUG:boto:path=/lybtug-12/obj5 2024-04-16T17:49:38.728 DEBUG:boto:auth_path=/lybtug-12/obj5 2024-04-16T17:49:38.728 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.728 DEBUG:boto:Path: /lybtug-12/obj5 2024-04-16T17:49:38.728 DEBUG:boto:Data: 2024-04-16T17:49:38.728 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.729 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.729 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.729 DEBUG:boto:Params: {} 2024-04-16T17:49:38.729 DEBUG:boto:Token: None 2024-04-16T17:49:38.729 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj5 2024-04-16T17:49:38.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4lpw7UymqwwH/b6nARG5Nx8dQFM= 2024-04-16T17:49:38.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4lpw7UymqwwH/b6nARG5Nx8dQFM='} 2024-04-16T17:49:38.738 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ca025750975ece77-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.738 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.738 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.738 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.738 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.738 DEBUG:boto:Data: 2024-04-16T17:49:38.738 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.738 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.738 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.739 DEBUG:boto:Params: {} 2024-04-16T17:49:38.739 DEBUG:boto:Token: None 2024-04-16T17:49:38.739 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.739 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.743 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009b93633364607282-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.743 DEBUG:boto:path=/lybtug-12/obj6 2024-04-16T17:49:38.743 DEBUG:boto:auth_path=/lybtug-12/obj6 2024-04-16T17:49:38.743 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.743 DEBUG:boto:Path: /lybtug-12/obj6 2024-04-16T17:49:38.743 DEBUG:boto:Data: 2024-04-16T17:49:38.743 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.743 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.743 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.744 DEBUG:boto:Params: {} 2024-04-16T17:49:38.744 DEBUG:boto:Token: None 2024-04-16T17:49:38.744 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj6 2024-04-16T17:49:38.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KB3/eHWkpWbNh7I7rb2ync5m/fI= 2024-04-16T17:49:38.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KB3/eHWkpWbNh7I7rb2ync5m/fI='} 2024-04-16T17:49:38.786 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ce891a0d5bf9d4be-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.787 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.787 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.787 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.787 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.787 DEBUG:boto:Data: 2024-04-16T17:49:38.787 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.787 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.787 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.787 DEBUG:boto:Params: {} 2024-04-16T17:49:38.787 DEBUG:boto:Token: None 2024-04-16T17:49:38.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.791 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000286b55b89be4dd96-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.791 DEBUG:boto:path=/lybtug-12/obj7 2024-04-16T17:49:38.792 DEBUG:boto:auth_path=/lybtug-12/obj7 2024-04-16T17:49:38.792 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.792 DEBUG:boto:Path: /lybtug-12/obj7 2024-04-16T17:49:38.792 DEBUG:boto:Data: 2024-04-16T17:49:38.792 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.792 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.792 DEBUG:boto:Params: {} 2024-04-16T17:49:38.792 DEBUG:boto:Token: None 2024-04-16T17:49:38.792 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj7 2024-04-16T17:49:38.793 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C+RP5TgEVTQ7pyr4sgU/ro6xAOY= 2024-04-16T17:49:38.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C+RP5TgEVTQ7pyr4sgU/ro6xAOY='} 2024-04-16T17:49:38.801 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008b29d2c920f62363-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.801 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.801 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.801 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.801 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.801 DEBUG:boto:Data: 2024-04-16T17:49:38.801 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.801 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.801 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.801 DEBUG:boto:Params: {} 2024-04-16T17:49:38.802 DEBUG:boto:Token: None 2024-04-16T17:49:38.802 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.802 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.808 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e483a227ea0ad272-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.808 DEBUG:boto:path=/lybtug-12/obj8 2024-04-16T17:49:38.809 DEBUG:boto:auth_path=/lybtug-12/obj8 2024-04-16T17:49:38.809 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.809 DEBUG:boto:Path: /lybtug-12/obj8 2024-04-16T17:49:38.809 DEBUG:boto:Data: 2024-04-16T17:49:38.809 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.809 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.809 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.809 DEBUG:boto:Params: {} 2024-04-16T17:49:38.809 DEBUG:boto:Token: None 2024-04-16T17:49:38.809 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj8 2024-04-16T17:49:38.810 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o7IQg0n4QVt50wxo30E4d+2qqvY= 2024-04-16T17:49:38.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o7IQg0n4QVt50wxo30E4d+2qqvY='} 2024-04-16T17:49:38.854 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006d7f876c692b636c-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.855 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.855 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.855 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.855 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.855 DEBUG:boto:Data: 2024-04-16T17:49:38.855 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.855 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.855 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.855 DEBUG:boto:Params: {} 2024-04-16T17:49:38.855 DEBUG:boto:Token: None 2024-04-16T17:49:38.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.859 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea57af861c2e113e-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.859 DEBUG:boto:path=/lybtug-12/obj9 2024-04-16T17:49:38.860 DEBUG:boto:auth_path=/lybtug-12/obj9 2024-04-16T17:49:38.860 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.860 DEBUG:boto:Path: /lybtug-12/obj9 2024-04-16T17:49:38.860 DEBUG:boto:Data: 2024-04-16T17:49:38.860 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.860 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.860 DEBUG:boto:Params: {} 2024-04-16T17:49:38.860 DEBUG:boto:Token: None 2024-04-16T17:49:38.860 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj9 2024-04-16T17:49:38.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zlQJdT2GGVBCCA+UHpgVWEbPrWY= 2024-04-16T17:49:38.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zlQJdT2GGVBCCA+UHpgVWEbPrWY='} 2024-04-16T17:49:38.872 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008ac957b1a61587b5-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.872 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.872 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.872 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.872 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.872 DEBUG:boto:Data: 2024-04-16T17:49:38.872 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.873 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.873 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.873 DEBUG:boto:Params: {} 2024-04-16T17:49:38.873 DEBUG:boto:Token: None 2024-04-16T17:49:38.873 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.873 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.877 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c6edd59d351272eb-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.877 DEBUG:boto:path=/lybtug-12/obj10 2024-04-16T17:49:38.877 DEBUG:boto:auth_path=/lybtug-12/obj10 2024-04-16T17:49:38.877 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.877 DEBUG:boto:Path: /lybtug-12/obj10 2024-04-16T17:49:38.877 DEBUG:boto:Data: 2024-04-16T17:49:38.877 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.877 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.878 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.878 DEBUG:boto:Params: {} 2024-04-16T17:49:38.878 DEBUG:boto:Token: None 2024-04-16T17:49:38.878 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj10 2024-04-16T17:49:38.878 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JDNJXQAOuCYUeihGaMlclEqYkkw= 2024-04-16T17:49:38.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JDNJXQAOuCYUeihGaMlclEqYkkw='} 2024-04-16T17:49:38.922 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000017deb5cd5d691009-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.923 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.923 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.923 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.923 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.923 DEBUG:boto:Data: 2024-04-16T17:49:38.923 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.923 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.923 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.923 DEBUG:boto:Params: {} 2024-04-16T17:49:38.923 DEBUG:boto:Token: None 2024-04-16T17:49:38.924 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.928 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ca61a069f551fcb8-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.928 DEBUG:boto:path=/lybtug-12/obj11 2024-04-16T17:49:38.928 DEBUG:boto:auth_path=/lybtug-12/obj11 2024-04-16T17:49:38.928 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.928 DEBUG:boto:Path: /lybtug-12/obj11 2024-04-16T17:49:38.928 DEBUG:boto:Data: 2024-04-16T17:49:38.928 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.928 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.928 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.928 DEBUG:boto:Params: {} 2024-04-16T17:49:38.929 DEBUG:boto:Token: None 2024-04-16T17:49:38.929 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj11 2024-04-16T17:49:38.929 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nmPyPwyWuRWV+AqvQU4bfiY6hA0= 2024-04-16T17:49:38.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nmPyPwyWuRWV+AqvQU4bfiY6hA0='} 2024-04-16T17:49:38.940 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000932b6909414e9bbf-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.940 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.940 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.940 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.940 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.940 DEBUG:boto:Data: 2024-04-16T17:49:38.940 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.940 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.941 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.941 DEBUG:boto:Params: {} 2024-04-16T17:49:38.941 DEBUG:boto:Token: None 2024-04-16T17:49:38.941 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.941 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.944 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008cd0d9b893cab4d0-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.945 DEBUG:boto:path=/lybtug-12/obj12 2024-04-16T17:49:38.945 DEBUG:boto:auth_path=/lybtug-12/obj12 2024-04-16T17:49:38.945 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.945 DEBUG:boto:Path: /lybtug-12/obj12 2024-04-16T17:49:38.945 DEBUG:boto:Data: 2024-04-16T17:49:38.945 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.945 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.945 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.945 DEBUG:boto:Params: {} 2024-04-16T17:49:38.946 DEBUG:boto:Token: None 2024-04-16T17:49:38.946 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj12 2024-04-16T17:49:38.946 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VTENEmyPuGlLlBZQKZcqVsLZUKA= 2024-04-16T17:49:38.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VTENEmyPuGlLlBZQKZcqVsLZUKA='} 2024-04-16T17:49:38.990 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004c5d5b60d8c061f7-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.991 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:38.991 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:38.991 DEBUG:boto:Method: HEAD 2024-04-16T17:49:38.991 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:38.991 DEBUG:boto:Data: 2024-04-16T17:49:38.991 DEBUG:boto:Headers: {} 2024-04-16T17:49:38.991 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.991 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.991 DEBUG:boto:Params: {} 2024-04-16T17:49:38.991 DEBUG:boto:Token: None 2024-04-16T17:49:38.992 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/ 2024-04-16T17:49:38.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T17:49:38.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T17:49:38.995 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006566105b09e0fb8e-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:38.996 DEBUG:boto:path=/lybtug-12/obj13 2024-04-16T17:49:38.996 DEBUG:boto:auth_path=/lybtug-12/obj13 2024-04-16T17:49:38.996 DEBUG:boto:Method: PUT 2024-04-16T17:49:38.996 DEBUG:boto:Path: /lybtug-12/obj13 2024-04-16T17:49:38.996 DEBUG:boto:Data: 2024-04-16T17:49:38.996 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:38.996 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:38.996 DEBUG:boto:Port: 8000 2024-04-16T17:49:38.996 DEBUG:boto:Params: {} 2024-04-16T17:49:38.996 DEBUG:boto:Token: None 2024-04-16T17:49:38.997 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:38 GMT /lybtug-12/obj13 2024-04-16T17:49:38.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ks25KBP7NfBbU62ZpsxUCY9jDmc= 2024-04-16T17:49:38.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ks25KBP7NfBbU62ZpsxUCY9jDmc='} 2024-04-16T17:49:39.005 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ec4dc29a991314ae-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.006 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.006 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.006 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.006 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.006 DEBUG:boto:Data: 2024-04-16T17:49:39.006 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.006 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.006 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.006 DEBUG:boto:Params: {} 2024-04-16T17:49:39.007 DEBUG:boto:Token: None 2024-04-16T17:49:39.007 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.007 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.010 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000437677509f4dfa90-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.010 DEBUG:boto:path=/lybtug-12/obj14 2024-04-16T17:49:39.011 DEBUG:boto:auth_path=/lybtug-12/obj14 2024-04-16T17:49:39.011 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.011 DEBUG:boto:Path: /lybtug-12/obj14 2024-04-16T17:49:39.011 DEBUG:boto:Data: 2024-04-16T17:49:39.011 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.011 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.011 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.011 DEBUG:boto:Params: {} 2024-04-16T17:49:39.011 DEBUG:boto:Token: None 2024-04-16T17:49:39.011 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj14 2024-04-16T17:49:39.011 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h405Y96cJgCmYdrPlnXx+m2u5PU= 2024-04-16T17:49:39.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h405Y96cJgCmYdrPlnXx+m2u5PU='} 2024-04-16T17:49:39.054 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001084fc3ad3836ca6-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.055 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.055 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.055 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.055 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.055 DEBUG:boto:Data: 2024-04-16T17:49:39.055 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.055 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.055 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.055 DEBUG:boto:Params: {} 2024-04-16T17:49:39.055 DEBUG:boto:Token: None 2024-04-16T17:49:39.055 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.059 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e4f245a8a7b1192e-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.059 DEBUG:boto:path=/lybtug-12/obj15 2024-04-16T17:49:39.060 DEBUG:boto:auth_path=/lybtug-12/obj15 2024-04-16T17:49:39.060 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.060 DEBUG:boto:Path: /lybtug-12/obj15 2024-04-16T17:49:39.060 DEBUG:boto:Data: 2024-04-16T17:49:39.060 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.060 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.060 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.060 DEBUG:boto:Params: {} 2024-04-16T17:49:39.060 DEBUG:boto:Token: None 2024-04-16T17:49:39.060 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj15 2024-04-16T17:49:39.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8HO35itNVKVDNmTY7SwyaCeKeQw= 2024-04-16T17:49:39.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8HO35itNVKVDNmTY7SwyaCeKeQw='} 2024-04-16T17:49:39.073 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004b8f3c4ca343e7b6-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.073 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.073 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.073 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.073 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.073 DEBUG:boto:Data: 2024-04-16T17:49:39.073 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.073 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.074 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.074 DEBUG:boto:Params: {} 2024-04-16T17:49:39.074 DEBUG:boto:Token: None 2024-04-16T17:49:39.074 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.074 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.077 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001945170028be4337-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.078 DEBUG:boto:path=/lybtug-12/obj16 2024-04-16T17:49:39.078 DEBUG:boto:auth_path=/lybtug-12/obj16 2024-04-16T17:49:39.078 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.078 DEBUG:boto:Path: /lybtug-12/obj16 2024-04-16T17:49:39.078 DEBUG:boto:Data: 2024-04-16T17:49:39.078 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.078 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.078 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.078 DEBUG:boto:Params: {} 2024-04-16T17:49:39.078 DEBUG:boto:Token: None 2024-04-16T17:49:39.079 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj16 2024-04-16T17:49:39.079 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O3AZMA91Jkb+S4AQ5VaIHz5FfTs= 2024-04-16T17:49:39.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O3AZMA91Jkb+S4AQ5VaIHz5FfTs='} 2024-04-16T17:49:39.122 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000de352dbcf1ca640c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.123 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.123 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.123 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.123 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.123 DEBUG:boto:Data: 2024-04-16T17:49:39.123 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.123 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.123 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.123 DEBUG:boto:Params: {} 2024-04-16T17:49:39.123 DEBUG:boto:Token: None 2024-04-16T17:49:39.124 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.128 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000098146b01e8ab5c1d-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.128 DEBUG:boto:path=/lybtug-12/obj17 2024-04-16T17:49:39.128 DEBUG:boto:auth_path=/lybtug-12/obj17 2024-04-16T17:49:39.128 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.128 DEBUG:boto:Path: /lybtug-12/obj17 2024-04-16T17:49:39.128 DEBUG:boto:Data: 2024-04-16T17:49:39.128 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.128 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.128 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.129 DEBUG:boto:Params: {} 2024-04-16T17:49:39.129 DEBUG:boto:Token: None 2024-04-16T17:49:39.129 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj17 2024-04-16T17:49:39.129 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a05ogLfDYDBkQZrdFW836hgCEM0= 2024-04-16T17:49:39.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a05ogLfDYDBkQZrdFW836hgCEM0='} 2024-04-16T17:49:39.137 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cb151fb28356e1e5-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.137 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.138 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.138 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.138 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.138 DEBUG:boto:Data: 2024-04-16T17:49:39.138 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.138 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.138 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.138 DEBUG:boto:Params: {} 2024-04-16T17:49:39.138 DEBUG:boto:Token: None 2024-04-16T17:49:39.138 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.142 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e9a1525895c8d8dc-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.142 DEBUG:boto:path=/lybtug-12/obj18 2024-04-16T17:49:39.143 DEBUG:boto:auth_path=/lybtug-12/obj18 2024-04-16T17:49:39.143 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.143 DEBUG:boto:Path: /lybtug-12/obj18 2024-04-16T17:49:39.143 DEBUG:boto:Data: 2024-04-16T17:49:39.143 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.143 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.143 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.143 DEBUG:boto:Params: {} 2024-04-16T17:49:39.143 DEBUG:boto:Token: None 2024-04-16T17:49:39.144 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj18 2024-04-16T17:49:39.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kM0a58UXDxt8inrdrJpCRWGaXiU= 2024-04-16T17:49:39.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kM0a58UXDxt8inrdrJpCRWGaXiU='} 2024-04-16T17:49:39.186 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cfe9bc2434da3428-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.187 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.187 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.187 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.187 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.187 DEBUG:boto:Data: 2024-04-16T17:49:39.187 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.187 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.187 DEBUG:boto:Params: {} 2024-04-16T17:49:39.187 DEBUG:boto:Token: None 2024-04-16T17:49:39.188 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.192 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '108'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000041915185d0ba2af0-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.192 DEBUG:boto:path=/lybtug-12/obj19 2024-04-16T17:49:39.192 DEBUG:boto:auth_path=/lybtug-12/obj19 2024-04-16T17:49:39.192 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.192 DEBUG:boto:Path: /lybtug-12/obj19 2024-04-16T17:49:39.192 DEBUG:boto:Data: 2024-04-16T17:49:39.192 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.192 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.192 DEBUG:boto:Params: {} 2024-04-16T17:49:39.193 DEBUG:boto:Token: None 2024-04-16T17:49:39.193 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj19 2024-04-16T17:49:39.193 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:beroYqmoXy6yynzh2fyqu7wHCNM= 2024-04-16T17:49:39.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:beroYqmoXy6yynzh2fyqu7wHCNM='} 2024-04-16T17:49:39.202 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ef5d81e78a84cb14-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.202 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.202 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.202 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.202 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.203 DEBUG:boto:Data: 2024-04-16T17:49:39.203 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.203 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.203 DEBUG:boto:Params: {} 2024-04-16T17:49:39.203 DEBUG:boto:Token: None 2024-04-16T17:49:39.203 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.203 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.207 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008398dd8f43928adc-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.207 DEBUG:boto:path=/lybtug-12/obj20 2024-04-16T17:49:39.207 DEBUG:boto:auth_path=/lybtug-12/obj20 2024-04-16T17:49:39.207 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.207 DEBUG:boto:Path: /lybtug-12/obj20 2024-04-16T17:49:39.207 DEBUG:boto:Data: 2024-04-16T17:49:39.207 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.208 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.208 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.208 DEBUG:boto:Params: {} 2024-04-16T17:49:39.208 DEBUG:boto:Token: None 2024-04-16T17:49:39.208 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj20 2024-04-16T17:49:39.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4uII/FhM4P82igOXnXlg3oykXQ4= 2024-04-16T17:49:39.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4uII/FhM4P82igOXnXlg3oykXQ4='} 2024-04-16T17:49:39.251 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000931bc5b33c3462d3-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.251 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.251 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.251 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.252 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.252 DEBUG:boto:Data: 2024-04-16T17:49:39.252 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.252 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.252 DEBUG:boto:Params: {} 2024-04-16T17:49:39.252 DEBUG:boto:Token: None 2024-04-16T17:49:39.252 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.256 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ad86e4943003d67-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.256 DEBUG:boto:path=/lybtug-12/obj21 2024-04-16T17:49:39.256 DEBUG:boto:auth_path=/lybtug-12/obj21 2024-04-16T17:49:39.257 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.257 DEBUG:boto:Path: /lybtug-12/obj21 2024-04-16T17:49:39.257 DEBUG:boto:Data: 2024-04-16T17:49:39.257 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.257 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.257 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.257 DEBUG:boto:Params: {} 2024-04-16T17:49:39.257 DEBUG:boto:Token: None 2024-04-16T17:49:39.257 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj21 2024-04-16T17:49:39.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9h9TvqyGxphQ/qKTuIB8hXqDWg= 2024-04-16T17:49:39.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9h9TvqyGxphQ/qKTuIB8hXqDWg='} 2024-04-16T17:49:39.269 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f49b6c483d5a337f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.269 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.269 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.269 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.269 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.269 DEBUG:boto:Data: 2024-04-16T17:49:39.270 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.270 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.270 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.270 DEBUG:boto:Params: {} 2024-04-16T17:49:39.270 DEBUG:boto:Token: None 2024-04-16T17:49:39.270 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.270 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.274 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dcbd023d86cb912f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.274 DEBUG:boto:path=/lybtug-12/obj22 2024-04-16T17:49:39.274 DEBUG:boto:auth_path=/lybtug-12/obj22 2024-04-16T17:49:39.274 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.275 DEBUG:boto:Path: /lybtug-12/obj22 2024-04-16T17:49:39.275 DEBUG:boto:Data: 2024-04-16T17:49:39.275 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.275 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.275 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.275 DEBUG:boto:Params: {} 2024-04-16T17:49:39.275 DEBUG:boto:Token: None 2024-04-16T17:49:39.275 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj22 2024-04-16T17:49:39.275 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9jP+eL2jQeYj7tjNF6nZKGYpJr4= 2024-04-16T17:49:39.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9jP+eL2jQeYj7tjNF6nZKGYpJr4='} 2024-04-16T17:49:39.322 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d1e7bafcc1aeebee-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.323 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.323 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.323 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.323 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.323 DEBUG:boto:Data: 2024-04-16T17:49:39.323 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.323 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.323 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.323 DEBUG:boto:Params: {} 2024-04-16T17:49:39.323 DEBUG:boto:Token: None 2024-04-16T17:49:39.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.327 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '132'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f7f6fe6b119a1c69-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.328 DEBUG:boto:path=/lybtug-12/obj23 2024-04-16T17:49:39.328 DEBUG:boto:auth_path=/lybtug-12/obj23 2024-04-16T17:49:39.328 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.328 DEBUG:boto:Path: /lybtug-12/obj23 2024-04-16T17:49:39.328 DEBUG:boto:Data: 2024-04-16T17:49:39.328 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.328 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.328 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.328 DEBUG:boto:Params: {} 2024-04-16T17:49:39.328 DEBUG:boto:Token: None 2024-04-16T17:49:39.329 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj23 2024-04-16T17:49:39.329 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UsOO3o618dGANrfPjzp371cljGw= 2024-04-16T17:49:39.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UsOO3o618dGANrfPjzp371cljGw='} 2024-04-16T17:49:39.337 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000a121842aba556b3-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.338 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.338 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.338 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.338 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.338 DEBUG:boto:Data: 2024-04-16T17:49:39.338 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.338 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.338 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.338 DEBUG:boto:Params: {} 2024-04-16T17:49:39.339 DEBUG:boto:Token: None 2024-04-16T17:49:39.339 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.339 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.343 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '138'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ffe60c5ab0e102c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.343 DEBUG:boto:path=/lybtug-12/obj24 2024-04-16T17:49:39.343 DEBUG:boto:auth_path=/lybtug-12/obj24 2024-04-16T17:49:39.343 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.343 DEBUG:boto:Path: /lybtug-12/obj24 2024-04-16T17:49:39.343 DEBUG:boto:Data: 2024-04-16T17:49:39.343 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.343 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.344 DEBUG:boto:Params: {} 2024-04-16T17:49:39.344 DEBUG:boto:Token: None 2024-04-16T17:49:39.344 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj24 2024-04-16T17:49:39.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qfZIwvEr00NXUVH8986B9+mIjJc= 2024-04-16T17:49:39.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qfZIwvEr00NXUVH8986B9+mIjJc='} 2024-04-16T17:49:39.386 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001008d77528cf78ea-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.387 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.387 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.387 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.387 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.387 DEBUG:boto:Data: 2024-04-16T17:49:39.387 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.387 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.387 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.387 DEBUG:boto:Params: {} 2024-04-16T17:49:39.388 DEBUG:boto:Token: None 2024-04-16T17:49:39.388 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.392 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '144'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000076fd282d70bea7f2-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.392 DEBUG:boto:path=/lybtug-12/obj25 2024-04-16T17:49:39.392 DEBUG:boto:auth_path=/lybtug-12/obj25 2024-04-16T17:49:39.392 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.392 DEBUG:boto:Path: /lybtug-12/obj25 2024-04-16T17:49:39.392 DEBUG:boto:Data: 2024-04-16T17:49:39.392 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.392 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.393 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.393 DEBUG:boto:Params: {} 2024-04-16T17:49:39.393 DEBUG:boto:Token: None 2024-04-16T17:49:39.393 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj25 2024-04-16T17:49:39.393 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O5prrqsNBgjudXXo9cetZDeMBO8= 2024-04-16T17:49:39.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O5prrqsNBgjudXXo9cetZDeMBO8='} 2024-04-16T17:49:39.401 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000004022a60c44fe727-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.402 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.402 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.402 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.402 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.402 DEBUG:boto:Data: 2024-04-16T17:49:39.402 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.402 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.402 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.403 DEBUG:boto:Params: {} 2024-04-16T17:49:39.403 DEBUG:boto:Token: None 2024-04-16T17:49:39.403 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.403 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.407 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000ba578b38a990bfb-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.407 DEBUG:boto:path=/lybtug-12/obj26 2024-04-16T17:49:39.407 DEBUG:boto:auth_path=/lybtug-12/obj26 2024-04-16T17:49:39.407 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.407 DEBUG:boto:Path: /lybtug-12/obj26 2024-04-16T17:49:39.407 DEBUG:boto:Data: 2024-04-16T17:49:39.407 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.408 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.408 DEBUG:boto:Params: {} 2024-04-16T17:49:39.408 DEBUG:boto:Token: None 2024-04-16T17:49:39.408 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj26 2024-04-16T17:49:39.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OXR1VgKiCFWJSZ59D4tCxK2bAkY= 2024-04-16T17:49:39.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OXR1VgKiCFWJSZ59D4tCxK2bAkY='} 2024-04-16T17:49:39.450 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c146bfc0929645e2-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.451 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.451 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.451 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.451 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.451 DEBUG:boto:Data: 2024-04-16T17:49:39.451 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.451 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.451 DEBUG:boto:Params: {} 2024-04-16T17:49:39.452 DEBUG:boto:Token: None 2024-04-16T17:49:39.452 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.456 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '156'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ab6f7c039ab2608c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.456 DEBUG:boto:path=/lybtug-12/obj27 2024-04-16T17:49:39.456 DEBUG:boto:auth_path=/lybtug-12/obj27 2024-04-16T17:49:39.456 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.457 DEBUG:boto:Path: /lybtug-12/obj27 2024-04-16T17:49:39.457 DEBUG:boto:Data: 2024-04-16T17:49:39.457 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.457 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.457 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.457 DEBUG:boto:Params: {} 2024-04-16T17:49:39.457 DEBUG:boto:Token: None 2024-04-16T17:49:39.457 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj27 2024-04-16T17:49:39.457 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5UwC3G6KrKM4plEDh+FWXuMZnY4= 2024-04-16T17:49:39.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5UwC3G6KrKM4plEDh+FWXuMZnY4='} 2024-04-16T17:49:39.466 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000eb3e607b6b6ad9a2-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.466 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.467 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.467 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.467 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.467 DEBUG:boto:Data: 2024-04-16T17:49:39.467 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.467 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.467 DEBUG:boto:Params: {} 2024-04-16T17:49:39.467 DEBUG:boto:Token: None 2024-04-16T17:49:39.468 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.471 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a599df499bbfdecc-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.471 DEBUG:boto:path=/lybtug-12/obj28 2024-04-16T17:49:39.472 DEBUG:boto:auth_path=/lybtug-12/obj28 2024-04-16T17:49:39.472 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.472 DEBUG:boto:Path: /lybtug-12/obj28 2024-04-16T17:49:39.472 DEBUG:boto:Data: 2024-04-16T17:49:39.472 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.472 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.472 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.472 DEBUG:boto:Params: {} 2024-04-16T17:49:39.472 DEBUG:boto:Token: None 2024-04-16T17:49:39.473 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj28 2024-04-16T17:49:39.473 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4t5qmsa+9k1lCWN+0oRFvoxHX84= 2024-04-16T17:49:39.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4t5qmsa+9k1lCWN+0oRFvoxHX84='} 2024-04-16T17:49:39.518 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000064b1b4cd8ae51b6e-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.519 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.519 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.519 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.519 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.519 DEBUG:boto:Data: 2024-04-16T17:49:39.519 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.519 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.519 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.520 DEBUG:boto:Params: {} 2024-04-16T17:49:39.520 DEBUG:boto:Token: None 2024-04-16T17:49:39.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.524 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '168'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000253fb594de5623ed-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.524 DEBUG:boto:path=/lybtug-12/obj29 2024-04-16T17:49:39.524 DEBUG:boto:auth_path=/lybtug-12/obj29 2024-04-16T17:49:39.524 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.524 DEBUG:boto:Path: /lybtug-12/obj29 2024-04-16T17:49:39.524 DEBUG:boto:Data: 2024-04-16T17:49:39.525 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.525 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.525 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.525 DEBUG:boto:Params: {} 2024-04-16T17:49:39.525 DEBUG:boto:Token: None 2024-04-16T17:49:39.525 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj29 2024-04-16T17:49:39.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6f/CsabvlTcnYBwoFnDQvco40Fw= 2024-04-16T17:49:39.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6f/CsabvlTcnYBwoFnDQvco40Fw='} 2024-04-16T17:49:39.534 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000076eb802062a56d1e-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.534 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.535 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.535 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.535 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.535 DEBUG:boto:Data: 2024-04-16T17:49:39.535 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.535 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.535 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.535 DEBUG:boto:Params: {} 2024-04-16T17:49:39.536 DEBUG:boto:Token: None 2024-04-16T17:49:39.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.540 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b23be95cadbe8e3f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.540 DEBUG:boto:path=/lybtug-12/obj30 2024-04-16T17:49:39.540 DEBUG:boto:auth_path=/lybtug-12/obj30 2024-04-16T17:49:39.540 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.540 DEBUG:boto:Path: /lybtug-12/obj30 2024-04-16T17:49:39.540 DEBUG:boto:Data: 2024-04-16T17:49:39.541 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.541 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.541 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.541 DEBUG:boto:Params: {} 2024-04-16T17:49:39.541 DEBUG:boto:Token: None 2024-04-16T17:49:39.541 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj30 2024-04-16T17:49:39.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K4o+j5fNVnRHrouyhD7ObCGxBAE= 2024-04-16T17:49:39.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K4o+j5fNVnRHrouyhD7ObCGxBAE='} 2024-04-16T17:49:39.586 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001bb2f64c15c4b5ad-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.587 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.587 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.587 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.587 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.587 DEBUG:boto:Data: 2024-04-16T17:49:39.587 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.587 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.587 DEBUG:boto:Params: {} 2024-04-16T17:49:39.588 DEBUG:boto:Token: None 2024-04-16T17:49:39.588 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.592 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000051d70031ad40bd59-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.592 DEBUG:boto:path=/lybtug-12/obj31 2024-04-16T17:49:39.592 DEBUG:boto:auth_path=/lybtug-12/obj31 2024-04-16T17:49:39.593 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.593 DEBUG:boto:Path: /lybtug-12/obj31 2024-04-16T17:49:39.593 DEBUG:boto:Data: 2024-04-16T17:49:39.593 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.593 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.593 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.593 DEBUG:boto:Params: {} 2024-04-16T17:49:39.593 DEBUG:boto:Token: None 2024-04-16T17:49:39.594 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj31 2024-04-16T17:49:39.594 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+buhP34IIsgxHxtErADz8eqruf4= 2024-04-16T17:49:39.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+buhP34IIsgxHxtErADz8eqruf4='} 2024-04-16T17:49:39.603 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bd9306ac08ecc15c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.603 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.603 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.603 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.603 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.603 DEBUG:boto:Data: 2024-04-16T17:49:39.604 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.604 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.604 DEBUG:boto:Params: {} 2024-04-16T17:49:39.604 DEBUG:boto:Token: None 2024-04-16T17:49:39.604 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.608 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '186'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000096b94c8d63beef8c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.608 DEBUG:boto:path=/lybtug-12/obj32 2024-04-16T17:49:39.609 DEBUG:boto:auth_path=/lybtug-12/obj32 2024-04-16T17:49:39.609 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.609 DEBUG:boto:Path: /lybtug-12/obj32 2024-04-16T17:49:39.609 DEBUG:boto:Data: 2024-04-16T17:49:39.609 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.609 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.609 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.609 DEBUG:boto:Params: {} 2024-04-16T17:49:39.610 DEBUG:boto:Token: None 2024-04-16T17:49:39.610 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj32 2024-04-16T17:49:39.610 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8zgihM9SxdBHDWIJplhjTlBCctw= 2024-04-16T17:49:39.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8zgihM9SxdBHDWIJplhjTlBCctw='} 2024-04-16T17:49:39.654 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a14118696c517bf1-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.655 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.655 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.655 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.655 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.655 DEBUG:boto:Data: 2024-04-16T17:49:39.655 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.656 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.656 DEBUG:boto:Params: {} 2024-04-16T17:49:39.656 DEBUG:boto:Token: None 2024-04-16T17:49:39.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.660 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '192'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fba722c0451c6169-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.660 DEBUG:boto:path=/lybtug-12/obj33 2024-04-16T17:49:39.660 DEBUG:boto:auth_path=/lybtug-12/obj33 2024-04-16T17:49:39.661 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.661 DEBUG:boto:Path: /lybtug-12/obj33 2024-04-16T17:49:39.661 DEBUG:boto:Data: 2024-04-16T17:49:39.661 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.661 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.661 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.661 DEBUG:boto:Params: {} 2024-04-16T17:49:39.662 DEBUG:boto:Token: None 2024-04-16T17:49:39.662 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj33 2024-04-16T17:49:39.662 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gu8Sv2z2FO3H76TRGXbSRxTvuek= 2024-04-16T17:49:39.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gu8Sv2z2FO3H76TRGXbSRxTvuek='} 2024-04-16T17:49:39.671 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005c29d62b8cb19034-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.671 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.671 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.672 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.672 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.672 DEBUG:boto:Data: 2024-04-16T17:49:39.672 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.672 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.672 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.672 DEBUG:boto:Params: {} 2024-04-16T17:49:39.672 DEBUG:boto:Token: None 2024-04-16T17:49:39.672 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.673 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.676 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '198'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000405cb94c0da0bc95-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.677 DEBUG:boto:path=/lybtug-12/obj34 2024-04-16T17:49:39.677 DEBUG:boto:auth_path=/lybtug-12/obj34 2024-04-16T17:49:39.677 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.677 DEBUG:boto:Path: /lybtug-12/obj34 2024-04-16T17:49:39.677 DEBUG:boto:Data: 2024-04-16T17:49:39.677 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.677 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.678 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.678 DEBUG:boto:Params: {} 2024-04-16T17:49:39.678 DEBUG:boto:Token: None 2024-04-16T17:49:39.678 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj34 2024-04-16T17:49:39.678 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lnr25DzwqtPlbz3hSkGtDA01qgg= 2024-04-16T17:49:39.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lnr25DzwqtPlbz3hSkGtDA01qgg='} 2024-04-16T17:49:39.723 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009fb1e4d6b1889a7f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.723 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.723 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.723 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.723 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.723 DEBUG:boto:Data: 2024-04-16T17:49:39.723 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.723 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.724 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.724 DEBUG:boto:Params: {} 2024-04-16T17:49:39.724 DEBUG:boto:Token: None 2024-04-16T17:49:39.724 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.728 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '204'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000215e2105d01c019c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.728 DEBUG:boto:path=/lybtug-12/obj35 2024-04-16T17:49:39.729 DEBUG:boto:auth_path=/lybtug-12/obj35 2024-04-16T17:49:39.729 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.729 DEBUG:boto:Path: /lybtug-12/obj35 2024-04-16T17:49:39.729 DEBUG:boto:Data: 2024-04-16T17:49:39.729 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.729 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.729 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.729 DEBUG:boto:Params: {} 2024-04-16T17:49:39.729 DEBUG:boto:Token: None 2024-04-16T17:49:39.730 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj35 2024-04-16T17:49:39.730 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lBuKujZGBqelRsRsir6OcSDmxKE= 2024-04-16T17:49:39.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lBuKujZGBqelRsRsir6OcSDmxKE='} 2024-04-16T17:49:39.740 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000af0d6bbd7805de06-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.740 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.740 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.740 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.740 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.740 DEBUG:boto:Data: 2024-04-16T17:49:39.740 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.741 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.741 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.741 DEBUG:boto:Params: {} 2024-04-16T17:49:39.741 DEBUG:boto:Token: None 2024-04-16T17:49:39.741 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.741 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.745 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '210'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000035f5a6bb4d54df58-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.745 DEBUG:boto:path=/lybtug-12/obj36 2024-04-16T17:49:39.746 DEBUG:boto:auth_path=/lybtug-12/obj36 2024-04-16T17:49:39.746 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.746 DEBUG:boto:Path: /lybtug-12/obj36 2024-04-16T17:49:39.746 DEBUG:boto:Data: 2024-04-16T17:49:39.746 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.746 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.746 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.746 DEBUG:boto:Params: {} 2024-04-16T17:49:39.746 DEBUG:boto:Token: None 2024-04-16T17:49:39.746 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj36 2024-04-16T17:49:39.747 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KNdsxJRdaTu/yT7E4yPJye3oH2o= 2024-04-16T17:49:39.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KNdsxJRdaTu/yT7E4yPJye3oH2o='} 2024-04-16T17:49:39.790 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000063b04fd489d27f6c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.791 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.791 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.791 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.791 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.791 DEBUG:boto:Data: 2024-04-16T17:49:39.791 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.791 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.791 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.791 DEBUG:boto:Params: {} 2024-04-16T17:49:39.792 DEBUG:boto:Token: None 2024-04-16T17:49:39.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.795 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '36'), ('X-RGW-Bytes-Used', '216'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ce4192012cae1b57-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.796 DEBUG:boto:path=/lybtug-12/obj37 2024-04-16T17:49:39.796 DEBUG:boto:auth_path=/lybtug-12/obj37 2024-04-16T17:49:39.796 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.796 DEBUG:boto:Path: /lybtug-12/obj37 2024-04-16T17:49:39.796 DEBUG:boto:Data: 2024-04-16T17:49:39.796 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.796 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.796 DEBUG:boto:Params: {} 2024-04-16T17:49:39.797 DEBUG:boto:Token: None 2024-04-16T17:49:39.797 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj37 2024-04-16T17:49:39.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/E9fB8/K3Fjcjnfz6Lgz4++qDTA= 2024-04-16T17:49:39.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/E9fB8/K3Fjcjnfz6Lgz4++qDTA='} 2024-04-16T17:49:39.806 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c1169d7a632249cf-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.806 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.807 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.807 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.807 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.807 DEBUG:boto:Data: 2024-04-16T17:49:39.807 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.807 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.807 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.807 DEBUG:boto:Params: {} 2024-04-16T17:49:39.807 DEBUG:boto:Token: None 2024-04-16T17:49:39.807 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.811 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '222'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000054f3a7d73d532887-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.811 DEBUG:boto:path=/lybtug-12/obj38 2024-04-16T17:49:39.812 DEBUG:boto:auth_path=/lybtug-12/obj38 2024-04-16T17:49:39.812 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.812 DEBUG:boto:Path: /lybtug-12/obj38 2024-04-16T17:49:39.812 DEBUG:boto:Data: 2024-04-16T17:49:39.812 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.812 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.812 DEBUG:boto:Params: {} 2024-04-16T17:49:39.812 DEBUG:boto:Token: None 2024-04-16T17:49:39.812 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj38 2024-04-16T17:49:39.813 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Bajf2c54tI6vxQBNOxzVqTBl+Xk= 2024-04-16T17:49:39.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Bajf2c54tI6vxQBNOxzVqTBl+Xk='} 2024-04-16T17:49:39.854 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fbd9178163cbe466-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.854 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.855 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.855 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.855 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.855 DEBUG:boto:Data: 2024-04-16T17:49:39.855 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.855 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.855 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.855 DEBUG:boto:Params: {} 2024-04-16T17:49:39.855 DEBUG:boto:Token: None 2024-04-16T17:49:39.855 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.859 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '228'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006c055fc2798fecc4-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.860 DEBUG:boto:path=/lybtug-12/obj39 2024-04-16T17:49:39.860 DEBUG:boto:auth_path=/lybtug-12/obj39 2024-04-16T17:49:39.860 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.860 DEBUG:boto:Path: /lybtug-12/obj39 2024-04-16T17:49:39.860 DEBUG:boto:Data: 2024-04-16T17:49:39.860 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.860 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.860 DEBUG:boto:Params: {} 2024-04-16T17:49:39.861 DEBUG:boto:Token: None 2024-04-16T17:49:39.861 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj39 2024-04-16T17:49:39.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6BK+U+NMclQLMTjDxBTsilLSCco= 2024-04-16T17:49:39.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6BK+U+NMclQLMTjDxBTsilLSCco='} 2024-04-16T17:49:39.869 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001cc78fd4f049f1bd-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.869 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.870 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.870 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.870 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.870 DEBUG:boto:Data: 2024-04-16T17:49:39.870 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.870 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.870 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.870 DEBUG:boto:Params: {} 2024-04-16T17:49:39.870 DEBUG:boto:Token: None 2024-04-16T17:49:39.870 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.870 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.874 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '39'), ('X-RGW-Bytes-Used', '234'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ee4918c648d2836f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.874 DEBUG:boto:path=/lybtug-12/obj40 2024-04-16T17:49:39.874 DEBUG:boto:auth_path=/lybtug-12/obj40 2024-04-16T17:49:39.875 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.875 DEBUG:boto:Path: /lybtug-12/obj40 2024-04-16T17:49:39.875 DEBUG:boto:Data: 2024-04-16T17:49:39.875 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.875 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.875 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.875 DEBUG:boto:Params: {} 2024-04-16T17:49:39.875 DEBUG:boto:Token: None 2024-04-16T17:49:39.875 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj40 2024-04-16T17:49:39.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M+ZyCE/nlAkQwpySMzsb/F76lSc= 2024-04-16T17:49:39.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M+ZyCE/nlAkQwpySMzsb/F76lSc='} 2024-04-16T17:49:39.918 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003a6f1505bbdee4f8-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.919 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.919 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.919 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.919 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.919 DEBUG:boto:Data: 2024-04-16T17:49:39.919 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.919 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.919 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.919 DEBUG:boto:Params: {} 2024-04-16T17:49:39.919 DEBUG:boto:Token: None 2024-04-16T17:49:39.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.923 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007508a346400e062b-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.923 DEBUG:boto:path=/lybtug-12/obj41 2024-04-16T17:49:39.923 DEBUG:boto:auth_path=/lybtug-12/obj41 2024-04-16T17:49:39.923 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.924 DEBUG:boto:Path: /lybtug-12/obj41 2024-04-16T17:49:39.924 DEBUG:boto:Data: 2024-04-16T17:49:39.924 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.924 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.924 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.924 DEBUG:boto:Params: {} 2024-04-16T17:49:39.924 DEBUG:boto:Token: None 2024-04-16T17:49:39.924 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj41 2024-04-16T17:49:39.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/oWMmpoVcge5Pi/nvHJ7NLWYmi8= 2024-04-16T17:49:39.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/oWMmpoVcge5Pi/nvHJ7NLWYmi8='} 2024-04-16T17:49:39.933 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004fe8b6c7aece44ba-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.933 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.933 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.933 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.933 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.933 DEBUG:boto:Data: 2024-04-16T17:49:39.934 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.934 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.934 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.934 DEBUG:boto:Params: {} 2024-04-16T17:49:39.934 DEBUG:boto:Token: None 2024-04-16T17:49:39.934 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.934 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.937 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '41'), ('X-RGW-Bytes-Used', '246'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009664f04fa0288b49-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.938 DEBUG:boto:path=/lybtug-12/obj42 2024-04-16T17:49:39.938 DEBUG:boto:auth_path=/lybtug-12/obj42 2024-04-16T17:49:39.938 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.938 DEBUG:boto:Path: /lybtug-12/obj42 2024-04-16T17:49:39.938 DEBUG:boto:Data: 2024-04-16T17:49:39.938 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.938 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.938 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.938 DEBUG:boto:Params: {} 2024-04-16T17:49:39.939 DEBUG:boto:Token: None 2024-04-16T17:49:39.939 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj42 2024-04-16T17:49:39.939 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9QE7OO+3z4u21SlnOr/g/Hv06pk= 2024-04-16T17:49:39.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9QE7OO+3z4u21SlnOr/g/Hv06pk='} 2024-04-16T17:49:39.982 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b715d47398094cdf-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.983 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.983 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.983 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.983 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.983 DEBUG:boto:Data: 2024-04-16T17:49:39.983 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.983 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.983 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.983 DEBUG:boto:Params: {} 2024-04-16T17:49:39.983 DEBUG:boto:Token: None 2024-04-16T17:49:39.983 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:39.987 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '252'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e22f45f6127ad540-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.987 DEBUG:boto:path=/lybtug-12/obj43 2024-04-16T17:49:39.987 DEBUG:boto:auth_path=/lybtug-12/obj43 2024-04-16T17:49:39.987 DEBUG:boto:Method: PUT 2024-04-16T17:49:39.987 DEBUG:boto:Path: /lybtug-12/obj43 2024-04-16T17:49:39.987 DEBUG:boto:Data: 2024-04-16T17:49:39.988 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:39.988 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.988 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.988 DEBUG:boto:Params: {} 2024-04-16T17:49:39.988 DEBUG:boto:Token: None 2024-04-16T17:49:39.988 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/obj43 2024-04-16T17:49:39.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/C7b9SxlyB0A0+nl1RbKY8fcbOc= 2024-04-16T17:49:39.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/C7b9SxlyB0A0+nl1RbKY8fcbOc='} 2024-04-16T17:49:39.997 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004d40b6f9f258281a-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:39.997 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:39.997 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:39.997 DEBUG:boto:Method: HEAD 2024-04-16T17:49:39.997 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:39.997 DEBUG:boto:Data: 2024-04-16T17:49:39.997 DEBUG:boto:Headers: {} 2024-04-16T17:49:39.997 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:39.997 DEBUG:boto:Port: 8000 2024-04-16T17:49:39.997 DEBUG:boto:Params: {} 2024-04-16T17:49:39.997 DEBUG:boto:Token: None 2024-04-16T17:49:39.998 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:39 GMT /lybtug-12/ 2024-04-16T17:49:39.998 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T17:49:39.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T17:49:40.002 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '43'), ('X-RGW-Bytes-Used', '258'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d741353817187778-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.003 DEBUG:boto:path=/lybtug-12/obj44 2024-04-16T17:49:40.003 DEBUG:boto:auth_path=/lybtug-12/obj44 2024-04-16T17:49:40.003 DEBUG:boto:Method: PUT 2024-04-16T17:49:40.003 DEBUG:boto:Path: /lybtug-12/obj44 2024-04-16T17:49:40.003 DEBUG:boto:Data: 2024-04-16T17:49:40.003 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:40.003 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.003 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.003 DEBUG:boto:Params: {} 2024-04-16T17:49:40.003 DEBUG:boto:Token: None 2024-04-16T17:49:40.003 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/obj44 2024-04-16T17:49:40.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ImFIyRmCm+FhnqR2GmII83H1Nnw= 2024-04-16T17:49:40.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ImFIyRmCm+FhnqR2GmII83H1Nnw='} 2024-04-16T17:49:40.050 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009bc9aff4bd59a729-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.051 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:40.051 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:40.051 DEBUG:boto:Method: HEAD 2024-04-16T17:49:40.051 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:40.051 DEBUG:boto:Data: 2024-04-16T17:49:40.051 DEBUG:boto:Headers: {} 2024-04-16T17:49:40.051 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.051 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.051 DEBUG:boto:Params: {} 2024-04-16T17:49:40.051 DEBUG:boto:Token: None 2024-04-16T17:49:40.051 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/ 2024-04-16T17:49:40.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T17:49:40.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T17:49:40.055 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '44'), ('X-RGW-Bytes-Used', '264'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000323333e3365adf67-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.055 DEBUG:boto:path=/lybtug-12/obj45 2024-04-16T17:49:40.055 DEBUG:boto:auth_path=/lybtug-12/obj45 2024-04-16T17:49:40.056 DEBUG:boto:Method: PUT 2024-04-16T17:49:40.056 DEBUG:boto:Path: /lybtug-12/obj45 2024-04-16T17:49:40.056 DEBUG:boto:Data: 2024-04-16T17:49:40.056 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:40.056 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.056 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.056 DEBUG:boto:Params: {} 2024-04-16T17:49:40.056 DEBUG:boto:Token: None 2024-04-16T17:49:40.056 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/obj45 2024-04-16T17:49:40.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WfKRfrDnz/QX0j4XNy9PNBsWWzo= 2024-04-16T17:49:40.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WfKRfrDnz/QX0j4XNy9PNBsWWzo='} 2024-04-16T17:49:40.065 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001720b0bb69e36212-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.066 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:40.066 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:40.066 DEBUG:boto:Method: HEAD 2024-04-16T17:49:40.066 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:40.066 DEBUG:boto:Data: 2024-04-16T17:49:40.066 DEBUG:boto:Headers: {} 2024-04-16T17:49:40.066 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.066 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.066 DEBUG:boto:Params: {} 2024-04-16T17:49:40.066 DEBUG:boto:Token: None 2024-04-16T17:49:40.067 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/ 2024-04-16T17:49:40.067 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T17:49:40.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T17:49:40.070 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '270'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000018d50b3cb255d1c1-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.071 DEBUG:boto:path=/lybtug-12/obj46 2024-04-16T17:49:40.071 DEBUG:boto:auth_path=/lybtug-12/obj46 2024-04-16T17:49:40.071 DEBUG:boto:Method: PUT 2024-04-16T17:49:40.071 DEBUG:boto:Path: /lybtug-12/obj46 2024-04-16T17:49:40.071 DEBUG:boto:Data: 2024-04-16T17:49:40.071 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:40.071 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.071 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.071 DEBUG:boto:Params: {} 2024-04-16T17:49:40.071 DEBUG:boto:Token: None 2024-04-16T17:49:40.072 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/obj46 2024-04-16T17:49:40.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+J/x07gmqU7xzSk6he3oJIu/yWo= 2024-04-16T17:49:40.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+J/x07gmqU7xzSk6he3oJIu/yWo='} 2024-04-16T17:49:40.114 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000900ebb6e2e2dd71a-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.115 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:40.115 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:40.115 DEBUG:boto:Method: HEAD 2024-04-16T17:49:40.115 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:40.115 DEBUG:boto:Data: 2024-04-16T17:49:40.115 DEBUG:boto:Headers: {} 2024-04-16T17:49:40.115 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.115 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.115 DEBUG:boto:Params: {} 2024-04-16T17:49:40.115 DEBUG:boto:Token: None 2024-04-16T17:49:40.115 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/ 2024-04-16T17:49:40.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T17:49:40.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T17:49:40.119 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '46'), ('X-RGW-Bytes-Used', '276'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b1a6c7f3b28a2a4c-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.119 DEBUG:boto:path=/lybtug-12/obj47 2024-04-16T17:49:40.120 DEBUG:boto:auth_path=/lybtug-12/obj47 2024-04-16T17:49:40.120 DEBUG:boto:Method: PUT 2024-04-16T17:49:40.120 DEBUG:boto:Path: /lybtug-12/obj47 2024-04-16T17:49:40.120 DEBUG:boto:Data: 2024-04-16T17:49:40.120 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:40.120 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.120 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.120 DEBUG:boto:Params: {} 2024-04-16T17:49:40.120 DEBUG:boto:Token: None 2024-04-16T17:49:40.120 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/obj47 2024-04-16T17:49:40.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AJg8CkuJpOwz0Vr/VrxND108N6E= 2024-04-16T17:49:40.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AJg8CkuJpOwz0Vr/VrxND108N6E='} 2024-04-16T17:49:40.128 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000b1f5c8a436baaf2-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.129 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:40.129 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:40.129 DEBUG:boto:Method: HEAD 2024-04-16T17:49:40.129 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:40.129 DEBUG:boto:Data: 2024-04-16T17:49:40.129 DEBUG:boto:Headers: {} 2024-04-16T17:49:40.129 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.129 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.129 DEBUG:boto:Params: {} 2024-04-16T17:49:40.129 DEBUG:boto:Token: None 2024-04-16T17:49:40.130 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/ 2024-04-16T17:49:40.130 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T17:49:40.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T17:49:40.133 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '47'), ('X-RGW-Bytes-Used', '282'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009e3f23378d48440d-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.133 DEBUG:boto:path=/lybtug-12/obj48 2024-04-16T17:49:40.134 DEBUG:boto:auth_path=/lybtug-12/obj48 2024-04-16T17:49:40.134 DEBUG:boto:Method: PUT 2024-04-16T17:49:40.134 DEBUG:boto:Path: /lybtug-12/obj48 2024-04-16T17:49:40.134 DEBUG:boto:Data: 2024-04-16T17:49:40.134 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:40.134 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.134 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.134 DEBUG:boto:Params: {} 2024-04-16T17:49:40.134 DEBUG:boto:Token: None 2024-04-16T17:49:40.135 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/obj48 2024-04-16T17:49:40.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w6BkmtTKXFifRjTOUWsqey83cQk= 2024-04-16T17:49:40.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w6BkmtTKXFifRjTOUWsqey83cQk='} 2024-04-16T17:49:40.178 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000090598eff8e15047-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.179 DEBUG:boto:path=/lybtug-12/ 2024-04-16T17:49:40.179 DEBUG:boto:auth_path=/lybtug-12/ 2024-04-16T17:49:40.179 DEBUG:boto:Method: HEAD 2024-04-16T17:49:40.179 DEBUG:boto:Path: /lybtug-12/ 2024-04-16T17:49:40.179 DEBUG:boto:Data: 2024-04-16T17:49:40.179 DEBUG:boto:Headers: {} 2024-04-16T17:49:40.179 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.179 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.179 DEBUG:boto:Params: {} 2024-04-16T17:49:40.179 DEBUG:boto:Token: None 2024-04-16T17:49:40.179 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/ 2024-04-16T17:49:40.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T17:49:40.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T17:49:40.183 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '48'), ('X-RGW-Bytes-Used', '288'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008f8ac638818e949c-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.183 DEBUG:boto:path=/lybtug-12/obj49 2024-04-16T17:49:40.184 DEBUG:boto:auth_path=/lybtug-12/obj49 2024-04-16T17:49:40.184 DEBUG:boto:Method: PUT 2024-04-16T17:49:40.184 DEBUG:boto:Path: /lybtug-12/obj49 2024-04-16T17:49:40.184 DEBUG:boto:Data: 2024-04-16T17:49:40.184 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T17:49:40.184 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:49:40.184 DEBUG:boto:Port: 8000 2024-04-16T17:49:40.184 DEBUG:boto:Params: {} 2024-04-16T17:49:40.184 DEBUG:boto:Token: None 2024-04-16T17:49:40.184 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 17:49:40 GMT /lybtug-12/obj49 2024-04-16T17:49:40.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:brTjrxfGqV7qAcziz1q1P/IaIyQ= 2024-04-16T17:49:40.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:brTjrxfGqV7qAcziz1q1P/IaIyQ='} 2024-04-16T17:49:40.193 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008a831b4da0e0e685-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:49:40.193 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:40.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:40.194 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:49:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:40.304 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:49:40.336 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713289776.972768_45.1","last_update":"2024-04-16T17:49:36.972768Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:49:40.336 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713289776.972768_45.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:40.336 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:49:40.336 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:40.337 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:49:40.337 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:49:40.379 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:40.379 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:40.447+0000 7faba102d980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:49:40.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:40.447+0000 7faba102d980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:49:40.479 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289776.972768_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:36.972768Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:40Z"} 2024-04-16T17:49:40.479 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289776.972768_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:36.972768Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:40Z"} 2024-04-16T17:49:40.480 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289776.972768_45.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:49:36.972768Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:49:40.480 DEBUG:rgw_multi.tests:log_status={0: '1_1713289776.972768_45.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:40.480 DEBUG:rgw_multi.tests:sync_status={0: '1_1713289776.972768_45.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:49:40.480 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:49:40.480 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-11', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:40.480 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-11', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:49:40.480 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-11 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:49:40.570 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:49:40.570 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:49:40.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:40.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:49:45.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:45.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:49:50.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:50.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:49:55.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:49:55.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:00.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:00.667+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:05.667+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:10.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:10.667+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:15.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:15.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:20.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:25.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:25.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:30.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:30.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:35.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:35.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:40.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:40.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:45.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:45.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:50.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:50.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:50:55.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:50:55.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:00.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:05.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:05.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:10.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:10.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:15.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:15.681+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:20.681+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:25.685+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:30.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:30.685+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:35.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:35.689+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:40.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:40.689+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:45.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:45.688+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:50.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:50.688+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:51:55.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:51:55.692+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:00.692+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:05.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:05.692+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:10.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:10.696+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:15.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:15.696+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:20.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:20.696+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:25.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:25.695+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:30.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:30.699+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:35.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:35.699+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:40.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:40.699+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:45.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:45.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:50.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:52:55.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:52:55.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:00.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:00.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:05.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:05.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:10.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:10.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:15.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:15.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:20.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:25.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:30.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:30.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:35.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:35.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:40.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:40.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:45.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:45.715+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:50.715+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:53:55.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:53:55.714+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:00.718+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:05.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:05.718+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:10.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:10.718+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:15.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:15.722+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:20.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:20.722+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:25.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:25.722+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:30.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:30.726+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:35.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:35.726+0000 7f52084e9980 -1 bucket checkpoint timed out waiting to reach incremental sync 2024-04-16T17:54:35.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:35.726+0000 7f52084e9980 0 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T17:54:35.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:35.726+0000 7f52084e9980 -1 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T17:54:35.777 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-16T17:54:35.782 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_object_delete ... FAIL 2024-04-16T17:54:35.783 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-13 2024-04-16T17:54:35.784 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:54:35.784 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:54:35.784 DEBUG:boto:Method: PUT 2024-04-16T17:54:35.784 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:54:35.784 DEBUG:boto:Data: 2024-04-16T17:54:35.784 DEBUG:boto:Headers: {} 2024-04-16T17:54:35.784 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:35.784 DEBUG:boto:Port: 8000 2024-04-16T17:54:35.784 DEBUG:boto:Params: {} 2024-04-16T17:54:35.785 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T17:54:35.785 DEBUG:boto:Token: None 2024-04-16T17:54:35.785 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:54:35 GMT /lybtug-13/ 2024-04-16T17:54:35.785 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:++fuBPXORt0/kHfOCF6LkwyU2hE= 2024-04-16T17:54:35.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:++fuBPXORt0/kHfOCF6LkwyU2hE=', 'Content-Length': '0'} 2024-04-16T17:54:35.824 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000aeedf62e55c5b590-00661ebb5b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:35.824 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-14 2024-04-16T17:54:35.824 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:54:35.824 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:54:35.824 DEBUG:boto:Method: PUT 2024-04-16T17:54:35.825 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:54:35.825 DEBUG:boto:Data: 2024-04-16T17:54:35.825 DEBUG:boto:Headers: {} 2024-04-16T17:54:35.825 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:35.825 DEBUG:boto:Port: 8000 2024-04-16T17:54:35.825 DEBUG:boto:Params: {} 2024-04-16T17:54:35.825 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T17:54:35.825 DEBUG:boto:Token: None 2024-04-16T17:54:35.825 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:54:35 GMT /lybtug-14/ 2024-04-16T17:54:35.826 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OtzrZE5hEaYjnY3CIflMrzMalFI= 2024-04-16T17:54:35.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OtzrZE5hEaYjnY3CIflMrzMalFI=', 'Content-Length': '0'} 2024-04-16T17:54:35.886 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006d350dd037692662-00661ebb5b-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:35.886 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:54:35.887 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:54:35.887 DEBUG:boto:path=/lybtug-13/?versioning 2024-04-16T17:54:35.887 DEBUG:boto:auth_path=/lybtug-13/?versioning 2024-04-16T17:54:35.887 DEBUG:boto:Method: PUT 2024-04-16T17:54:35.887 DEBUG:boto:Path: /lybtug-13/?versioning 2024-04-16T17:54:35.887 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:54:35.887 DEBUG:boto:Headers: {} 2024-04-16T17:54:35.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:35.887 DEBUG:boto:Port: 8000 2024-04-16T17:54:35.887 DEBUG:boto:Params: {} 2024-04-16T17:54:35.887 DEBUG:boto:Token: None 2024-04-16T17:54:35.888 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:54:35 GMT /lybtug-13/?versioning 2024-04-16T17:54:35.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N+dVFGy7iNQeIDIJcuc/jFxAsvg= 2024-04-16T17:54:35.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N+dVFGy7iNQeIDIJcuc/jFxAsvg=', 'Content-Length': '228'} 2024-04-16T17:54:35.900 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006e56ac4dbf3b3052-00661ebb5b-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:35.900 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:54:35.900 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:54:35.900 DEBUG:boto:path=/lybtug-14/?versioning 2024-04-16T17:54:35.900 DEBUG:boto:auth_path=/lybtug-14/?versioning 2024-04-16T17:54:35.900 DEBUG:boto:Method: PUT 2024-04-16T17:54:35.900 DEBUG:boto:Path: /lybtug-14/?versioning 2024-04-16T17:54:35.900 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:54:35.901 DEBUG:boto:Headers: {} 2024-04-16T17:54:35.901 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:35.901 DEBUG:boto:Port: 8000 2024-04-16T17:54:35.901 DEBUG:boto:Params: {} 2024-04-16T17:54:35.901 DEBUG:boto:Token: None 2024-04-16T17:54:35.901 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:54:35 GMT /lybtug-14/?versioning 2024-04-16T17:54:35.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yq5DE1ovUTK1SOKJKL6R8HrwU4c= 2024-04-16T17:54:35.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yq5DE1ovUTK1SOKJKL6R8HrwU4c=', 'Content-Length': '228'} 2024-04-16T17:54:35.919 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002558b33faf5df875-00661ebb5b-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:35.920 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:54:35.920 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:54:35.920 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:54:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:54:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:54:36.027 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:54:36.058 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290075.917177_111.1","last_update":"2024-04-16T17:54:35.917177Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:54:36.058 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290075.917177_111.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:54:36.058 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:54:36.058 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:54:36.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:54:36.059 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:54:36.104 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:54:36.104 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:54:36.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:36.170+0000 7f09a2d80980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:54:36.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:36.174+0000 7f09a2d80980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:54:36.204 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290075.803646_100.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:54:35.803646Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:54:36Z"} 2024-04-16T17:54:36.204 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290075.803646_100.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:54:35.803646Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:54:36Z"} 2024-04-16T17:54:36.204 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290075.803646_100.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:54:35.803646Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:54:36.205 DEBUG:rgw_multi.tests:log_status={0: '1_1713290075.917177_111.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:54:36.205 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290075.803646_100.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:54:36.205 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290075.917177_111.1 target=1_1713290075.803646_100.1 2024-04-16T17:54:41.206 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:54:41.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:54:41.207 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:54:41.250 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:54:41.250 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:54:41.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:41.318+0000 7f26c4bcb980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:54:41.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:41.318+0000 7f26c4bcb980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:54:41.348 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290075.917177_111.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:54:35.917177Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:54:41Z"} 2024-04-16T17:54:41.348 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290075.917177_111.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:54:35.917177Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:54:41Z"} 2024-04-16T17:54:41.348 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290075.917177_111.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:54:35.917177Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:54:41.348 DEBUG:rgw_multi.tests:log_status={0: '1_1713290075.917177_111.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:54:41.348 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290075.917177_111.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:54:41.349 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:54:41.349 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:54:41.349 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:54:41.349 DEBUG:boto:Method: HEAD 2024-04-16T17:54:41.349 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:54:41.349 DEBUG:boto:Data: 2024-04-16T17:54:41.349 DEBUG:boto:Headers: {} 2024-04-16T17:54:41.349 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:41.349 DEBUG:boto:Port: 8000 2024-04-16T17:54:41.349 DEBUG:boto:Params: {} 2024-04-16T17:54:41.350 DEBUG:boto:Token: None 2024-04-16T17:54:41.350 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:54:41 GMT /lybtug-13/ 2024-04-16T17:54:41.350 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pWmKOKFd1+lchbD2pcE3yXMa/Q= 2024-04-16T17:54:41.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:41 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pWmKOKFd1+lchbD2pcE3yXMa/Q=', 'Content-Length': '0'} 2024-04-16T17:54:41.354 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000058b4fa024040edcf-00661ebb61-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:41.354 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:54:41.354 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:54:41.355 DEBUG:boto:Method: PUT 2024-04-16T17:54:41.355 DEBUG:boto:Path: /lybtug-13/dummy 2024-04-16T17:54:41.355 DEBUG:boto:Data: 2024-04-16T17:54:41.355 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:54:41.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:41.355 DEBUG:boto:Port: 8000 2024-04-16T17:54:41.355 DEBUG:boto:Params: {} 2024-04-16T17:54:41.355 DEBUG:boto:Token: None 2024-04-16T17:54:41.355 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:54:41 GMT /lybtug-13/dummy 2024-04-16T17:54:41.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fc4NPaiqLciKCvik/nBn32CwnG4= 2024-04-16T17:54:41.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:41 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fc4NPaiqLciKCvik/nBn32CwnG4='} 2024-04-16T17:54:41.385 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-amz-request-id', 'tx00000b75c1e9fa44f8385-00661ebb61-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:41.385 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:54:41.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:54:41.385 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:54:41.436 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:54:41.436 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:54:41.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:41.510+0000 7fe8497c9980 1 waiting to reach incremental sync.. 2024-04-16T17:54:46.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:46.514+0000 7fe8497c9980 1 bucket sync caught up with source: 2024-04-16T17:54:46.515 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000002.19.11, , , , , ] 2024-04-16T17:54:46.515 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000002.19.11, , , , , ] 2024-04-16T17:54:46.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:46.514+0000 7fe8497c9980 0 bucket checkpoint complete 2024-04-16T17:54:46.556 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:54:46.557 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:54:46.557 DEBUG:boto:Method: HEAD 2024-04-16T17:54:46.557 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:54:46.557 DEBUG:boto:Data: 2024-04-16T17:54:46.557 DEBUG:boto:Headers: {} 2024-04-16T17:54:46.557 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:46.557 DEBUG:boto:Port: 8000 2024-04-16T17:54:46.557 DEBUG:boto:Params: {} 2024-04-16T17:54:46.558 DEBUG:boto:Token: None 2024-04-16T17:54:46.558 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:54:46 GMT /lybtug-14/ 2024-04-16T17:54:46.558 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Swtl+LmaWYiI1OjEQcQkBSc0qzM= 2024-04-16T17:54:46.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Swtl+LmaWYiI1OjEQcQkBSc0qzM=', 'Content-Length': '0'} 2024-04-16T17:54:46.561 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007dbfa469ea898dde-00661ebb66-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:46.562 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:54:46.562 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:54:46.562 DEBUG:boto:Method: PUT 2024-04-16T17:54:46.562 DEBUG:boto:Path: /lybtug-14/dummy 2024-04-16T17:54:46.562 DEBUG:boto:Data: 2024-04-16T17:54:46.562 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:54:46.562 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:46.562 DEBUG:boto:Port: 8000 2024-04-16T17:54:46.563 DEBUG:boto:Params: {} 2024-04-16T17:54:46.563 DEBUG:boto:Token: None 2024-04-16T17:54:46.563 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:54:46 GMT /lybtug-14/dummy 2024-04-16T17:54:46.563 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bI99qufVMKZsGhB0g5sZiM26qhk= 2024-04-16T17:54:46.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bI99qufVMKZsGhB0g5sZiM26qhk='} 2024-04-16T17:54:46.583 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-amz-request-id', 'tx00000d90bd491cb359753-00661ebb66-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:46.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:54:46.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:54:46.584 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:54:46.634 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:54:46.634 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:54:46.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:54:46.704+0000 7fc959337980 1 waiting to reach incremental sync.. 2024-04-16T17:54:51.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:54:51.708+0000 7fc959337980 1 bucket sync caught up with source: 2024-04-16T17:54:51.709 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , 00000000002.22.11, , , , , ] 2024-04-16T17:54:51.710 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , 00000000002.22.11, , , , , ] 2024-04-16T17:54:51.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:54:51.708+0000 7fc959337980 0 bucket checkpoint complete 2024-04-16T17:54:51.769 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:54:51.770 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:54:51.770 DEBUG:boto:Method: HEAD 2024-04-16T17:54:51.770 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:54:51.770 DEBUG:boto:Data: 2024-04-16T17:54:51.770 DEBUG:boto:Headers: {} 2024-04-16T17:54:51.770 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:51.770 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.770 DEBUG:boto:Params: {} 2024-04-16T17:54:51.770 DEBUG:boto:Token: None 2024-04-16T17:54:51.771 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/ 2024-04-16T17:54:51.771 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SeSk4eIYaTSowY9U0W9qcIrLIyE= 2024-04-16T17:54:51.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SeSk4eIYaTSowY9U0W9qcIrLIyE=', 'Content-Length': '0'} 2024-04-16T17:54:51.775 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004f842fb6a0a18669-00661ebb6b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.775 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.775 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.775 DEBUG:boto:Method: PUT 2024-04-16T17:54:51.775 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:54:51.775 DEBUG:boto:Data: 2024-04-16T17:54:51.775 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:51.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:51.776 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.776 DEBUG:boto:Params: {} 2024-04-16T17:54:51.776 DEBUG:boto:Token: None 2024-04-16T17:54:51.776 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:54:51.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JWkgbUzV4T378GY/IcR23WFGRQE= 2024-04-16T17:54:51.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JWkgbUzV4T378GY/IcR23WFGRQE='} 2024-04-16T17:54:51.801 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-amz-request-id', 'tx000002d0f159a3e6abe07-00661ebb6b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.801 DEBUG:rgw_multi.tests:version1 id=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:54:51.801 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.801 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.801 DEBUG:boto:Method: PUT 2024-04-16T17:54:51.801 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:54:51.802 DEBUG:boto:Data: 2024-04-16T17:54:51.802 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:51.802 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:51.802 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.802 DEBUG:boto:Params: {} 2024-04-16T17:54:51.802 DEBUG:boto:Token: None 2024-04-16T17:54:51.802 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:54:51.802 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:27VrAPwyoe4Y2tO+08+d4vSbtsc= 2024-04-16T17:54:51.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:27VrAPwyoe4Y2tO+08+d4vSbtsc='} 2024-04-16T17:54:51.823 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '61.BJXaFO9FpVabVNw4yYwvNpht6bG4'), ('x-amz-request-id', 'tx00000c2227585c01ca0b5-00661ebb6b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.823 DEBUG:rgw_multi.tests:version2 id=61.BJXaFO9FpVabVNw4yYwvNpht6bG4 2024-04-16T17:54:51.823 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.823 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.823 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=61.BJXaFO9FpVabVNw4yYwvNpht6bG4 2024-04-16T17:54:51.823 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=61.BJXaFO9FpVabVNw4yYwvNpht6bG4 2024-04-16T17:54:51.823 DEBUG:boto:Method: DELETE 2024-04-16T17:54:51.823 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=61.BJXaFO9FpVabVNw4yYwvNpht6bG4 2024-04-16T17:54:51.823 DEBUG:boto:Data: 2024-04-16T17:54:51.824 DEBUG:boto:Headers: {} 2024-04-16T17:54:51.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:51.824 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.824 DEBUG:boto:Params: {} 2024-04-16T17:54:51.824 DEBUG:boto:Token: None 2024-04-16T17:54:51.824 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone1?versionId=61.BJXaFO9FpVabVNw4yYwvNpht6bG4 2024-04-16T17:54:51.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DHJh5c/8CqYqeBztsZEcv8aw3zQ= 2024-04-16T17:54:51.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DHJh5c/8CqYqeBztsZEcv8aw3zQ=', 'Content-Length': '0'} 2024-04-16T17:54:51.847 DEBUG:boto:Response headers: [('x-amz-version-id', '61.BJXaFO9FpVabVNw4yYwvNpht6bG4'), ('x-amz-request-id', 'tx0000021de55280fd8bb99-00661ebb6b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.847 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.847 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.847 DEBUG:boto:Method: PUT 2024-04-16T17:54:51.847 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:54:51.847 DEBUG:boto:Data: 2024-04-16T17:54:51.847 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:51.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:51.848 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.848 DEBUG:boto:Params: {} 2024-04-16T17:54:51.848 DEBUG:boto:Token: None 2024-04-16T17:54:51.848 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:54:51.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmcETc1YovJj3iL68r8yn1qMn1g= 2024-04-16T17:54:51.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmcETc1YovJj3iL68r8yn1qMn1g='} 2024-04-16T17:54:51.890 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb'), ('x-amz-request-id', 'tx0000045e7a1e1bccc7dfb-00661ebb6b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.891 DEBUG:rgw_multi.tests:version3 id=.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb 2024-04-16T17:54:51.891 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.891 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:54:51.891 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb 2024-04-16T17:54:51.891 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb 2024-04-16T17:54:51.891 DEBUG:boto:Method: DELETE 2024-04-16T17:54:51.891 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb 2024-04-16T17:54:51.891 DEBUG:boto:Data: 2024-04-16T17:54:51.891 DEBUG:boto:Headers: {} 2024-04-16T17:54:51.891 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:51.891 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.891 DEBUG:boto:Params: {} 2024-04-16T17:54:51.892 DEBUG:boto:Token: None 2024-04-16T17:54:51.892 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone1?versionId=.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb 2024-04-16T17:54:51.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EyYxKFxLg0qh5JFKuIRtT5rj9OU= 2024-04-16T17:54:51.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EyYxKFxLg0qh5JFKuIRtT5rj9OU=', 'Content-Length': '0'} 2024-04-16T17:54:51.918 DEBUG:boto:Response headers: [('x-amz-version-id', '.Eqq-rtblFHXAraCMAqzlhDIkikD3Rb'), ('x-amz-request-id', 'tx000005d2522cded8cdfaf-00661ebb6b-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.918 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:54:51.918 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:54:51.918 DEBUG:boto:Method: HEAD 2024-04-16T17:54:51.918 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:54:51.918 DEBUG:boto:Data: 2024-04-16T17:54:51.918 DEBUG:boto:Headers: {} 2024-04-16T17:54:51.918 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:51.918 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.918 DEBUG:boto:Params: {} 2024-04-16T17:54:51.919 DEBUG:boto:Token: None 2024-04-16T17:54:51.919 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/ 2024-04-16T17:54:51.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SeSk4eIYaTSowY9U0W9qcIrLIyE= 2024-04-16T17:54:51.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SeSk4eIYaTSowY9U0W9qcIrLIyE=', 'Content-Length': '0'} 2024-04-16T17:54:51.923 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009e4d15c0b46c7ab5-00661ebb6b-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.923 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:51.923 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:51.923 DEBUG:boto:Method: PUT 2024-04-16T17:54:51.923 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:54:51.923 DEBUG:boto:Data: 2024-04-16T17:54:51.923 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:51.923 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:51.923 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.923 DEBUG:boto:Params: {} 2024-04-16T17:54:51.924 DEBUG:boto:Token: None 2024-04-16T17:54:51.924 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:54:51.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:poO5AH+GFUsw7AMiVlaujbG05Zw= 2024-04-16T17:54:51.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:poO5AH+GFUsw7AMiVlaujbG05Zw='} 2024-04-16T17:54:51.950 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-amz-request-id', 'tx000002c883f4297c201b2-00661ebb6b-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.950 DEBUG:rgw_multi.tests:version1 id=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:54:51.951 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:51.951 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:51.951 DEBUG:boto:Method: PUT 2024-04-16T17:54:51.951 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:54:51.951 DEBUG:boto:Data: 2024-04-16T17:54:51.951 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:51.951 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:51.951 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.951 DEBUG:boto:Params: {} 2024-04-16T17:54:51.951 DEBUG:boto:Token: None 2024-04-16T17:54:51.952 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:54:51.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ci6lsHmUrtQYYV3iqoIP0boWrRA= 2024-04-16T17:54:51.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ci6lsHmUrtQYYV3iqoIP0boWrRA='} 2024-04-16T17:54:51.994 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '16i0CHYnjRULc9cjIcq0.4inCvxlkXx'), ('x-amz-request-id', 'tx00000de0973d68505b3be-00661ebb6b-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:51.995 DEBUG:rgw_multi.tests:version2 id=16i0CHYnjRULc9cjIcq0.4inCvxlkXx 2024-04-16T17:54:51.995 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:51.995 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:51.995 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=16i0CHYnjRULc9cjIcq0.4inCvxlkXx 2024-04-16T17:54:51.995 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=16i0CHYnjRULc9cjIcq0.4inCvxlkXx 2024-04-16T17:54:51.995 DEBUG:boto:Method: DELETE 2024-04-16T17:54:51.995 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=16i0CHYnjRULc9cjIcq0.4inCvxlkXx 2024-04-16T17:54:51.995 DEBUG:boto:Data: 2024-04-16T17:54:51.995 DEBUG:boto:Headers: {} 2024-04-16T17:54:51.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:51.995 DEBUG:boto:Port: 8000 2024-04-16T17:54:51.996 DEBUG:boto:Params: {} 2024-04-16T17:54:51.996 DEBUG:boto:Token: None 2024-04-16T17:54:51.996 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:51 GMT /lybtug-13/obj-test-zone2?versionId=16i0CHYnjRULc9cjIcq0.4inCvxlkXx 2024-04-16T17:54:51.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vt4na/Uyj532rCVIOP/abSxN0+0= 2024-04-16T17:54:51.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vt4na/Uyj532rCVIOP/abSxN0+0=', 'Content-Length': '0'} 2024-04-16T17:54:52.018 DEBUG:boto:Response headers: [('x-amz-version-id', '16i0CHYnjRULc9cjIcq0.4inCvxlkXx'), ('x-amz-request-id', 'tx00000b4f1a461f4fa711e-00661ebb6b-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.019 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:52.019 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:52.019 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.019 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:54:52.019 DEBUG:boto:Data: 2024-04-16T17:54:52.020 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.020 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.020 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.020 DEBUG:boto:Params: {} 2024-04-16T17:54:52.020 DEBUG:boto:Token: None 2024-04-16T17:54:52.020 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:54:52.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JRDn0RXHEve8P39r+284r+pR0BA= 2024-04-16T17:54:52.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JRDn0RXHEve8P39r+284r+pR0BA='} 2024-04-16T17:54:52.043 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO'), ('x-amz-request-id', 'tx000000d62676b395df54e-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.043 DEBUG:rgw_multi.tests:version3 id=GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO 2024-04-16T17:54:52.043 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:52.043 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:54:52.043 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO 2024-04-16T17:54:52.043 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO 2024-04-16T17:54:52.043 DEBUG:boto:Method: DELETE 2024-04-16T17:54:52.043 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO 2024-04-16T17:54:52.043 DEBUG:boto:Data: 2024-04-16T17:54:52.044 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.044 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.044 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.044 DEBUG:boto:Params: {} 2024-04-16T17:54:52.044 DEBUG:boto:Token: None 2024-04-16T17:54:52.044 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:52 GMT /lybtug-13/obj-test-zone2?versionId=GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO 2024-04-16T17:54:52.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GkHP9oJ6UDfAJb2JkPr/ZeAtRGo= 2024-04-16T17:54:52.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GkHP9oJ6UDfAJb2JkPr/ZeAtRGo=', 'Content-Length': '0'} 2024-04-16T17:54:52.066 DEBUG:boto:Response headers: [('x-amz-version-id', 'GgQ8gj.vmm8RZfyxfEYXAcUnjJ2sMwO'), ('x-amz-request-id', 'tx0000010a51494c94084cb-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.066 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:54:52.067 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:54:52.067 DEBUG:boto:Method: HEAD 2024-04-16T17:54:52.067 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:54:52.067 DEBUG:boto:Data: 2024-04-16T17:54:52.067 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:52.067 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.067 DEBUG:boto:Params: {} 2024-04-16T17:54:52.067 DEBUG:boto:Token: None 2024-04-16T17:54:52.067 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/ 2024-04-16T17:54:52.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RBPsTrPSHnjXAyRJQguyFa74+kk= 2024-04-16T17:54:52.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RBPsTrPSHnjXAyRJQguyFa74+kk=', 'Content-Length': '0'} 2024-04-16T17:54:52.071 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000083e311dc51c9106a-00661ebb6c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.071 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.071 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.072 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.072 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:54:52.072 DEBUG:boto:Data: 2024-04-16T17:54:52.072 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:52.072 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.072 DEBUG:boto:Params: {} 2024-04-16T17:54:52.072 DEBUG:boto:Token: None 2024-04-16T17:54:52.073 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:54:52.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TInFwtnZ1XblBUMqPTNfs09tyo4= 2024-04-16T17:54:52.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TInFwtnZ1XblBUMqPTNfs09tyo4='} 2024-04-16T17:54:52.100 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-amz-request-id', 'tx000002c545b28fb09059e-00661ebb6c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.100 DEBUG:rgw_multi.tests:version1 id=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:54:52.100 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.101 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.101 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.101 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:54:52.101 DEBUG:boto:Data: 2024-04-16T17:54:52.101 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.101 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:52.101 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.101 DEBUG:boto:Params: {} 2024-04-16T17:54:52.101 DEBUG:boto:Token: None 2024-04-16T17:54:52.102 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:54:52.102 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jIUa2xCT6XvePggctXECuRBwN/o= 2024-04-16T17:54:52.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jIUa2xCT6XvePggctXECuRBwN/o='} 2024-04-16T17:54:52.146 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW'), ('x-amz-request-id', 'tx00000c2eae2c889f57fe1-00661ebb6c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.147 DEBUG:rgw_multi.tests:version2 id=JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW 2024-04-16T17:54:52.147 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.147 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.147 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW 2024-04-16T17:54:52.147 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW 2024-04-16T17:54:52.147 DEBUG:boto:Method: DELETE 2024-04-16T17:54:52.147 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW 2024-04-16T17:54:52.147 DEBUG:boto:Data: 2024-04-16T17:54:52.147 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.148 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:52.148 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.148 DEBUG:boto:Params: {} 2024-04-16T17:54:52.148 DEBUG:boto:Token: None 2024-04-16T17:54:52.148 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone1?versionId=JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW 2024-04-16T17:54:52.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VggcJ4V6g/o1DA5Z2ng8euB5uVk= 2024-04-16T17:54:52.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VggcJ4V6g/o1DA5Z2ng8euB5uVk=', 'Content-Length': '0'} 2024-04-16T17:54:52.174 DEBUG:boto:Response headers: [('x-amz-version-id', 'JnA4w2BeP50HtZ0BxZBIMT8yn1i.ArW'), ('x-amz-request-id', 'tx000008ba73be8f201e724-00661ebb6c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.174 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.174 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.174 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.174 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:54:52.174 DEBUG:boto:Data: 2024-04-16T17:54:52.174 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.175 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:52.175 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.175 DEBUG:boto:Params: {} 2024-04-16T17:54:52.175 DEBUG:boto:Token: None 2024-04-16T17:54:52.175 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:54:52.175 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q5uI0JJHlrVyVi1sFlQQd47RFGY= 2024-04-16T17:54:52.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q5uI0JJHlrVyVi1sFlQQd47RFGY='} 2024-04-16T17:54:52.199 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '6izSy1RWiMu.HskeezWjxgfSVNU08NG'), ('x-amz-request-id', 'tx0000052599f892a8c93aa-00661ebb6c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.199 DEBUG:rgw_multi.tests:version3 id=6izSy1RWiMu.HskeezWjxgfSVNU08NG 2024-04-16T17:54:52.199 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.199 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:54:52.199 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=6izSy1RWiMu.HskeezWjxgfSVNU08NG 2024-04-16T17:54:52.199 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=6izSy1RWiMu.HskeezWjxgfSVNU08NG 2024-04-16T17:54:52.199 DEBUG:boto:Method: DELETE 2024-04-16T17:54:52.200 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=6izSy1RWiMu.HskeezWjxgfSVNU08NG 2024-04-16T17:54:52.200 DEBUG:boto:Data: 2024-04-16T17:54:52.200 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.200 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:54:52.200 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.200 DEBUG:boto:Params: {} 2024-04-16T17:54:52.200 DEBUG:boto:Token: None 2024-04-16T17:54:52.200 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone1?versionId=6izSy1RWiMu.HskeezWjxgfSVNU08NG 2024-04-16T17:54:52.201 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C/m0vu6FlUA4fNyb3E6BPKReCK0= 2024-04-16T17:54:52.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C/m0vu6FlUA4fNyb3E6BPKReCK0=', 'Content-Length': '0'} 2024-04-16T17:54:52.225 DEBUG:boto:Response headers: [('x-amz-version-id', '6izSy1RWiMu.HskeezWjxgfSVNU08NG'), ('x-amz-request-id', 'tx00000cb8fecf57ddf80c1-00661ebb6c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.225 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:54:52.225 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:54:52.226 DEBUG:boto:Method: HEAD 2024-04-16T17:54:52.226 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:54:52.226 DEBUG:boto:Data: 2024-04-16T17:54:52.226 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.226 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.226 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.226 DEBUG:boto:Params: {} 2024-04-16T17:54:52.226 DEBUG:boto:Token: None 2024-04-16T17:54:52.227 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/ 2024-04-16T17:54:52.227 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RBPsTrPSHnjXAyRJQguyFa74+kk= 2024-04-16T17:54:52.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RBPsTrPSHnjXAyRJQguyFa74+kk=', 'Content-Length': '0'} 2024-04-16T17:54:52.230 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000a6ca8de9dd2f828-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.231 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.231 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.231 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.231 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:54:52.231 DEBUG:boto:Data: 2024-04-16T17:54:52.231 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.231 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.231 DEBUG:boto:Params: {} 2024-04-16T17:54:52.232 DEBUG:boto:Token: None 2024-04-16T17:54:52.232 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:54:52.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2nf3OgAqmCZunBa3jSUzodUHXKg= 2024-04-16T17:54:52.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2nf3OgAqmCZunBa3jSUzodUHXKg='} 2024-04-16T17:54:52.259 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-amz-request-id', 'tx00000f866d9c0ef2c85b8-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.259 DEBUG:rgw_multi.tests:version1 id=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:54:52.259 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.260 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.260 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.260 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:54:52.260 DEBUG:boto:Data: 2024-04-16T17:54:52.260 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.260 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.260 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.260 DEBUG:boto:Params: {} 2024-04-16T17:54:52.260 DEBUG:boto:Token: None 2024-04-16T17:54:52.261 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:54:52.261 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ej7GITHqwWdKOI9s0i6t4eVaAAg= 2024-04-16T17:54:52.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ej7GITHqwWdKOI9s0i6t4eVaAAg='} 2024-04-16T17:54:52.285 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'C5.7-lpkE1008toIWZXtgZUF292Khf5'), ('x-amz-request-id', 'tx000004da7fd46e92be1c6-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.285 DEBUG:rgw_multi.tests:version2 id=C5.7-lpkE1008toIWZXtgZUF292Khf5 2024-04-16T17:54:52.285 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.285 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.285 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=C5.7-lpkE1008toIWZXtgZUF292Khf5 2024-04-16T17:54:52.285 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=C5.7-lpkE1008toIWZXtgZUF292Khf5 2024-04-16T17:54:52.285 DEBUG:boto:Method: DELETE 2024-04-16T17:54:52.286 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=C5.7-lpkE1008toIWZXtgZUF292Khf5 2024-04-16T17:54:52.286 DEBUG:boto:Data: 2024-04-16T17:54:52.286 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.286 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.286 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.286 DEBUG:boto:Params: {} 2024-04-16T17:54:52.286 DEBUG:boto:Token: None 2024-04-16T17:54:52.286 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone2?versionId=C5.7-lpkE1008toIWZXtgZUF292Khf5 2024-04-16T17:54:52.286 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A3qxkZbDDjdbYw5WYnQr/+F5ML4= 2024-04-16T17:54:52.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A3qxkZbDDjdbYw5WYnQr/+F5ML4=', 'Content-Length': '0'} 2024-04-16T17:54:52.311 DEBUG:boto:Response headers: [('x-amz-version-id', 'C5.7-lpkE1008toIWZXtgZUF292Khf5'), ('x-amz-request-id', 'tx0000034ca63bdd38bc85e-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.311 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.311 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.311 DEBUG:boto:Method: PUT 2024-04-16T17:54:52.311 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:54:52.311 DEBUG:boto:Data: 2024-04-16T17:54:52.311 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:54:52.312 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.312 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.312 DEBUG:boto:Params: {} 2024-04-16T17:54:52.312 DEBUG:boto:Token: None 2024-04-16T17:54:52.312 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:54:52.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XxRIktHHK9p1pqxJ91NKVaTL9gk= 2024-04-16T17:54:52.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XxRIktHHK9p1pqxJ91NKVaTL9gk='} 2024-04-16T17:54:52.355 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '75wEbL3s8.SH0jnhPERrWMX8-FBu.rj'), ('x-amz-request-id', 'tx000006431f3d506382996-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.355 DEBUG:rgw_multi.tests:version3 id=75wEbL3s8.SH0jnhPERrWMX8-FBu.rj 2024-04-16T17:54:52.355 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.355 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:54:52.355 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=75wEbL3s8.SH0jnhPERrWMX8-FBu.rj 2024-04-16T17:54:52.355 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=75wEbL3s8.SH0jnhPERrWMX8-FBu.rj 2024-04-16T17:54:52.355 DEBUG:boto:Method: DELETE 2024-04-16T17:54:52.355 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=75wEbL3s8.SH0jnhPERrWMX8-FBu.rj 2024-04-16T17:54:52.355 DEBUG:boto:Data: 2024-04-16T17:54:52.356 DEBUG:boto:Headers: {} 2024-04-16T17:54:52.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:54:52.356 DEBUG:boto:Port: 8000 2024-04-16T17:54:52.356 DEBUG:boto:Params: {} 2024-04-16T17:54:52.356 DEBUG:boto:Token: None 2024-04-16T17:54:52.356 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:54:52 GMT /lybtug-14/obj-test-zone2?versionId=75wEbL3s8.SH0jnhPERrWMX8-FBu.rj 2024-04-16T17:54:52.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P9lW511eEFJ16n1ufC2q1PaEzSQ= 2024-04-16T17:54:52.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:54:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P9lW511eEFJ16n1ufC2q1PaEzSQ=', 'Content-Length': '0'} 2024-04-16T17:54:52.381 DEBUG:boto:Response headers: [('x-amz-version-id', '75wEbL3s8.SH0jnhPERrWMX8-FBu.rj'), ('x-amz-request-id', 'tx000008cecca5b6801ad0b-00661ebb6c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:54:52.381 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-13 2024-04-16T17:54:52.382 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:54:52.382 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:54:52.382 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:54:52.434 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:54:52.434 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:54:52.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:52.510+0000 7f86fb959980 1 waiting for incremental sync to catch up: 2024-04-16T17:54:52.510 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000010.51.6, , , , 00000000002.19.11, , , , , ] 2024-04-16T17:54:52.510 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000010.51.6, , , , 00000000003.21.10, , , , , ] 2024-04-16T17:54:57.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:54:57.514+0000 7f86fb959980 1 waiting for incremental sync to catch up: 2024-04-16T17:54:57.513 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000010.51.6, , 00000000002.33.11, , 00000000002.19.11, , , , , ] 2024-04-16T17:54:57.513 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000010.51.6, , , , 00000000003.21.10, , , , , ] 2024-04-16T17:55:02.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:02.514+0000 7f86fb959980 1 bucket sync caught up with source: 2024-04-16T17:55:02.515 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000010.51.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:02.515 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000010.51.6, , , , 00000000003.21.10, , , , , ] 2024-04-16T17:55:02.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:02.514+0000 7f86fb959980 0 bucket checkpoint complete 2024-04-16T17:55:02.559 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-13 2024-04-16T17:55:02.559 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:02.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:02.559 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-13 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:55:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:55:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:55:02.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:02.688+0000 7fd6c108f980 1 bucket sync caught up with source: 2024-04-16T17:55:02.690 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:02.691 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:02.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:02.688+0000 7fd6c108f980 0 bucket checkpoint complete 2024-04-16T17:55:02.720 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-13 zones={test-zone2, test-zone1} 2024-04-16T17:55:02.721 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.721 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.721 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.721 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:55:02.721 DEBUG:boto:Data: 2024-04-16T17:55:02.721 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.721 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.721 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.721 DEBUG:boto:Params: {} 2024-04-16T17:55:02.721 DEBUG:boto:Token: None 2024-04-16T17:55:02.722 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/ 2024-04-16T17:55:02.722 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:70l7b23UzSu8Jop1dZbHr8zJD3w= 2024-04-16T17:55:02.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:70l7b23UzSu8Jop1dZbHr8zJD3w=', 'Content-Length': '0'} 2024-04-16T17:55:02.725 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001566417f0313434f-00661ebb76-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.726 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.726 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.726 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.726 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:55:02.726 DEBUG:boto:Data: 2024-04-16T17:55:02.726 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.726 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.726 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.726 DEBUG:boto:Params: {} 2024-04-16T17:55:02.727 DEBUG:boto:Token: None 2024-04-16T17:55:02.727 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/ 2024-04-16T17:55:02.727 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:70l7b23UzSu8Jop1dZbHr8zJD3w= 2024-04-16T17:55:02.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:70l7b23UzSu8Jop1dZbHr8zJD3w=', 'Content-Length': '0'} 2024-04-16T17:55:02.730 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e9055e759ae72f9e-00661ebb76-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.730 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:55:02.730 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.730 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.730 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.730 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.731 DEBUG:boto:Method: GET 2024-04-16T17:55:02.731 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.731 DEBUG:boto:Data: 2024-04-16T17:55:02.731 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.731 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.731 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.731 DEBUG:boto:Params: {} 2024-04-16T17:55:02.731 DEBUG:boto:Token: None 2024-04-16T17:55:02.731 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/?versions 2024-04-16T17:55:02.731 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw= 2024-04-16T17:55:02.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw=', 'Content-Length': '0'} 2024-04-16T17:55:02.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f670c3ec70de76c8-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.736 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.736 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:55:02.736 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:55:02.737 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:55:02.737 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:55:02.737 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.737 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.737 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.737 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.737 DEBUG:boto:Method: GET 2024-04-16T17:55:02.737 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.737 DEBUG:boto:Data: 2024-04-16T17:55:02.737 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.737 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.738 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.738 DEBUG:boto:Params: {} 2024-04-16T17:55:02.738 DEBUG:boto:Token: None 2024-04-16T17:55:02.738 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/?versions 2024-04-16T17:55:02.738 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw= 2024-04-16T17:55:02.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw=', 'Content-Length': '0'} 2024-04-16T17:55:02.741 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cce9c046d91c2c6-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.741 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.742 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:55:02.742 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:55:02.742 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:55:02.742 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.742 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.742 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.742 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.742 DEBUG:boto:Method: GET 2024-04-16T17:55:02.742 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.742 DEBUG:boto:Data: 2024-04-16T17:55:02.743 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.743 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.743 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.743 DEBUG:boto:Params: {} 2024-04-16T17:55:02.743 DEBUG:boto:Token: None 2024-04-16T17:55:02.743 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/?versions 2024-04-16T17:55:02.743 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw= 2024-04-16T17:55:02.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw=', 'Content-Length': '0'} 2024-04-16T17:55:02.747 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001da41ba20b948487-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.748 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.748 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.748 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.748 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.748 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.749 DEBUG:boto:Method: GET 2024-04-16T17:55:02.749 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:55:02.749 DEBUG:boto:Data: 2024-04-16T17:55:02.749 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.749 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.749 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.749 DEBUG:boto:Params: {} 2024-04-16T17:55:02.749 DEBUG:boto:Token: None 2024-04-16T17:55:02.749 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/?versions 2024-04-16T17:55:02.750 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw= 2024-04-16T17:55:02.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmjtqYmKSnyN+N24QP+A0L8Y1Jw=', 'Content-Length': '0'} 2024-04-16T17:55:02.753 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038ec010838c063a4-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.753 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.754 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:55:02.754 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.754 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.754 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.754 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.754 DEBUG:boto:Method: GET 2024-04-16T17:55:02.754 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.755 DEBUG:boto:Data: 2024-04-16T17:55:02.755 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.755 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.755 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.755 DEBUG:boto:Params: {} 2024-04-16T17:55:02.755 DEBUG:boto:Token: None 2024-04-16T17:55:02.755 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.755 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo= 2024-04-16T17:55:02.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo=', 'Content-Length': '0'} 2024-04-16T17:55:02.758 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000943db6bfee5d7719-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.759 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.759 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.759 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.759 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.759 DEBUG:boto:Method: GET 2024-04-16T17:55:02.759 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.759 DEBUG:boto:Data: 2024-04-16T17:55:02.759 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.760 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.760 DEBUG:boto:Params: {} 2024-04-16T17:55:02.760 DEBUG:boto:Token: None 2024-04-16T17:55:02.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo= 2024-04-16T17:55:02.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo=', 'Content-Length': '0'} 2024-04-16T17:55:02.762 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000034bb54bc9a83ad5f-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.763 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.763 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.763 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.763 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.763 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.763 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.764 DEBUG:boto:Data: 2024-04-16T17:55:02.764 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.764 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.764 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.764 DEBUG:boto:Params: {} 2024-04-16T17:55:02.764 DEBUG:boto:Token: None 2024-04-16T17:55:02.764 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.765 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GlQJj9V5tkyVK0P72tVj/FD3Qls= 2024-04-16T17:55:02.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GlQJj9V5tkyVK0P72tVj/FD3Qls=', 'Content-Length': '0'} 2024-04-16T17:55:02.767 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f10bd97ebdbca519-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.768 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.768 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.768 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.768 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.768 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.768 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.768 DEBUG:boto:Data: 2024-04-16T17:55:02.768 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.768 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.768 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.768 DEBUG:boto:Params: {} 2024-04-16T17:55:02.769 DEBUG:boto:Token: None 2024-04-16T17:55:02.769 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.769 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GlQJj9V5tkyVK0P72tVj/FD3Qls= 2024-04-16T17:55:02.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GlQJj9V5tkyVK0P72tVj/FD3Qls=', 'Content-Length': '0'} 2024-04-16T17:55:02.772 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000066335e32d40ec32b-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.772 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:55:02.772 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.772 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.772 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.773 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.773 DEBUG:boto:Method: GET 2024-04-16T17:55:02.773 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.773 DEBUG:boto:Data: 2024-04-16T17:55:02.773 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.773 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.773 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.773 DEBUG:boto:Params: {} 2024-04-16T17:55:02.773 DEBUG:boto:Token: None 2024-04-16T17:55:02.773 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.773 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo= 2024-04-16T17:55:02.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo=', 'Content-Length': '0'} 2024-04-16T17:55:02.776 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a654b3466c3b9dae-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.777 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.777 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.777 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.777 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.777 DEBUG:boto:Method: GET 2024-04-16T17:55:02.777 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.777 DEBUG:boto:Data: 2024-04-16T17:55:02.777 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.777 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.777 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.777 DEBUG:boto:Params: {} 2024-04-16T17:55:02.778 DEBUG:boto:Token: None 2024-04-16T17:55:02.778 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.778 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo= 2024-04-16T17:55:02.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo=', 'Content-Length': '0'} 2024-04-16T17:55:02.781 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004d7d918927c682ae-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.781 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.781 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.781 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.781 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.781 DEBUG:boto:Method: GET 2024-04-16T17:55:02.782 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:55:02.782 DEBUG:boto:Data: 2024-04-16T17:55:02.782 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.782 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.782 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.782 DEBUG:boto:Params: {} 2024-04-16T17:55:02.782 DEBUG:boto:Token: None 2024-04-16T17:55:02.782 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/ 2024-04-16T17:55:02.782 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ= 2024-04-16T17:55:02.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ=', 'Content-Length': '0'} 2024-04-16T17:55:02.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007140e25cf09903f-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.830 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.831 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.831 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.831 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.831 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.831 DEBUG:boto:Method: GET 2024-04-16T17:55:02.831 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:55:02.831 DEBUG:boto:Data: 2024-04-16T17:55:02.831 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.831 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.831 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.831 DEBUG:boto:Params: {} 2024-04-16T17:55:02.831 DEBUG:boto:Token: None 2024-04-16T17:55:02.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/ 2024-04-16T17:55:02.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ= 2024-04-16T17:55:02.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ=', 'Content-Length': '0'} 2024-04-16T17:55:02.835 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c368dc5f2517a963-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.835 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.836 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.836 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.836 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.836 DEBUG:boto:Path: /lybtug-13/dummy 2024-04-16T17:55:02.836 DEBUG:boto:Data: 2024-04-16T17:55:02.836 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.836 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.836 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.836 DEBUG:boto:Params: {} 2024-04-16T17:55:02.837 DEBUG:boto:Token: None 2024-04-16T17:55:02.837 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy 2024-04-16T17:55:02.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J0DEGTVbq+w5u0PT19SsC461yYY= 2024-04-16T17:55:02.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J0DEGTVbq+w5u0PT19SsC461yYY=', 'Content-Length': '0'} 2024-04-16T17:55:02.841 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000140144926e752004-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.841 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.841 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.841 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.841 DEBUG:boto:Path: /lybtug-13/dummy 2024-04-16T17:55:02.841 DEBUG:boto:Data: 2024-04-16T17:55:02.841 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.842 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.842 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.842 DEBUG:boto:Params: {} 2024-04-16T17:55:02.842 DEBUG:boto:Token: None 2024-04-16T17:55:02.842 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy 2024-04-16T17:55:02.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J0DEGTVbq+w5u0PT19SsC461yYY= 2024-04-16T17:55:02.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J0DEGTVbq+w5u0PT19SsC461yYY=', 'Content-Length': '0'} 2024-04-16T17:55:02.846 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004812d111766db415-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.846 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:55:02.846 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.846 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.847 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.847 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.847 DEBUG:boto:Method: GET 2024-04-16T17:55:02.847 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.847 DEBUG:boto:Data: 2024-04-16T17:55:02.847 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.847 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.847 DEBUG:boto:Params: {} 2024-04-16T17:55:02.847 DEBUG:boto:Token: None 2024-04-16T17:55:02.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo= 2024-04-16T17:55:02.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo=', 'Content-Length': '0'} 2024-04-16T17:55:02.850 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c5c409217b5a2387-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.851 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:55:02.851 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:55:02.851 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.851 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.851 DEBUG:boto:Method: GET 2024-04-16T17:55:02.851 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.851 DEBUG:boto:Data: 2024-04-16T17:55:02.851 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.852 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.852 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.852 DEBUG:boto:Params: {} 2024-04-16T17:55:02.852 DEBUG:boto:Token: None 2024-04-16T17:55:02.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:55:02.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo= 2024-04-16T17:55:02.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qG1ybXjKiL2/ujLuk6TOcMA6pxo=', 'Content-Length': '0'} 2024-04-16T17:55:02.855 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003ac51759d7ec8d98-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.855 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.855 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.855 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.856 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.856 DEBUG:boto:Method: GET 2024-04-16T17:55:02.856 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:55:02.856 DEBUG:boto:Data: 2024-04-16T17:55:02.856 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.856 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.856 DEBUG:boto:Params: {} 2024-04-16T17:55:02.856 DEBUG:boto:Token: None 2024-04-16T17:55:02.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/ 2024-04-16T17:55:02.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ= 2024-04-16T17:55:02.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ=', 'Content-Length': '0'} 2024-04-16T17:55:02.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d74244f22273313d-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.902 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.903 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:02.903 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:02.903 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.903 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:55:02.903 DEBUG:boto:Method: GET 2024-04-16T17:55:02.903 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:55:02.903 DEBUG:boto:Data: 2024-04-16T17:55:02.903 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.903 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.903 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.903 DEBUG:boto:Params: {} 2024-04-16T17:55:02.904 DEBUG:boto:Token: None 2024-04-16T17:55:02.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/ 2024-04-16T17:55:02.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ= 2024-04-16T17:55:02.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bupXwdEx+GYya5M46rvTy4RQRIQ=', 'Content-Length': '0'} 2024-04-16T17:55:02.907 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6b256d271fd8886-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.908 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:02.908 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:55:02.908 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.908 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.908 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.908 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.908 DEBUG:boto:Method: GET 2024-04-16T17:55:02.908 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.909 DEBUG:boto:Data: 2024-04-16T17:55:02.909 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.909 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.909 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.909 DEBUG:boto:Params: {} 2024-04-16T17:55:02.909 DEBUG:boto:Token: None 2024-04-16T17:55:02.909 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t0YaWTxdaUxC7U6WCEg7wrWUIgY= 2024-04-16T17:55:02.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t0YaWTxdaUxC7U6WCEg7wrWUIgY=', 'Content-Length': '0'} 2024-04-16T17:55:02.913 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000300c6a53be002b37-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.954 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.955 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.955 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.955 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.955 DEBUG:boto:Method: GET 2024-04-16T17:55:02.955 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.955 DEBUG:boto:Data: 2024-04-16T17:55:02.955 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.955 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.955 DEBUG:boto:Params: {} 2024-04-16T17:55:02.955 DEBUG:boto:Token: None 2024-04-16T17:55:02.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t0YaWTxdaUxC7U6WCEg7wrWUIgY= 2024-04-16T17:55:02.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t0YaWTxdaUxC7U6WCEg7wrWUIgY=', 'Content-Length': '0'} 2024-04-16T17:55:02.959 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000096400d2c3543fa06-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.959 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.959 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.959 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.960 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.960 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.960 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.960 DEBUG:boto:Data: 2024-04-16T17:55:02.960 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.960 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.960 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.960 DEBUG:boto:Params: {} 2024-04-16T17:55:02.960 DEBUG:boto:Token: None 2024-04-16T17:55:02.960 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v55819Y6Bc0p6KAQ62WNuLDQb1o= 2024-04-16T17:55:02.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v55819Y6Bc0p6KAQ62WNuLDQb1o=', 'Content-Length': '0'} 2024-04-16T17:55:02.964 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d80fd1f20d04f410-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.965 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.965 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.965 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.965 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.965 DEBUG:boto:Method: HEAD 2024-04-16T17:55:02.965 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.965 DEBUG:boto:Data: 2024-04-16T17:55:02.965 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.965 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:02.965 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.965 DEBUG:boto:Params: {} 2024-04-16T17:55:02.966 DEBUG:boto:Token: None 2024-04-16T17:55:02.966 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.966 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v55819Y6Bc0p6KAQ62WNuLDQb1o= 2024-04-16T17:55:02.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v55819Y6Bc0p6KAQ62WNuLDQb1o=', 'Content-Length': '0'} 2024-04-16T17:55:02.969 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007caf370433cefcf0-00661ebb76-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:02.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:55:02.969 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.969 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:02.969 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.969 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.970 DEBUG:boto:Method: GET 2024-04-16T17:55:02.970 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.970 DEBUG:boto:Data: 2024-04-16T17:55:02.970 DEBUG:boto:Headers: {} 2024-04-16T17:55:02.970 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:02.970 DEBUG:boto:Port: 8000 2024-04-16T17:55:02.970 DEBUG:boto:Params: {} 2024-04-16T17:55:02.970 DEBUG:boto:Token: None 2024-04-16T17:55:02.970 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:02 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:02.970 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t0YaWTxdaUxC7U6WCEg7wrWUIgY= 2024-04-16T17:55:02.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t0YaWTxdaUxC7U6WCEg7wrWUIgY=', 'Content-Length': '0'} 2024-04-16T17:55:02.973 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000601e8f4cec2b3c72-00661ebb76-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.014 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.014 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.015 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.015 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.015 DEBUG:boto:Method: GET 2024-04-16T17:55:03.015 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.015 DEBUG:boto:Data: 2024-04-16T17:55:03.015 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.015 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.015 DEBUG:boto:Params: {} 2024-04-16T17:55:03.015 DEBUG:boto:Token: None 2024-04-16T17:55:03.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KzC563RBfvgt57tAYGwPxNYS+eI= 2024-04-16T17:55:03.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KzC563RBfvgt57tAYGwPxNYS+eI=', 'Content-Length': '0'} 2024-04-16T17:55:03.019 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006054a70d29f59bcf-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.019 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.019 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.019 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.019 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.019 DEBUG:boto:Method: GET 2024-04-16T17:55:03.019 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.019 DEBUG:boto:Data: 2024-04-16T17:55:03.019 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.020 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.020 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.020 DEBUG:boto:Params: {} 2024-04-16T17:55:03.020 DEBUG:boto:Token: None 2024-04-16T17:55:03.020 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b287a2eb03ad319-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.024 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.025 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.025 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.025 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.025 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.025 DEBUG:boto:Method: GET 2024-04-16T17:55:03.025 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.025 DEBUG:boto:Data: 2024-04-16T17:55:03.025 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.026 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.026 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.026 DEBUG:boto:Params: {} 2024-04-16T17:55:03.026 DEBUG:boto:Token: None 2024-04-16T17:55:03.026 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.026 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c5fc7e276fa76ca-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.030 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.030 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.031 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.031 DEBUG:boto:Method: HEAD 2024-04-16T17:55:03.031 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:55:03.031 DEBUG:boto:Data: 2024-04-16T17:55:03.031 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.031 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.031 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.031 DEBUG:boto:Params: {} 2024-04-16T17:55:03.031 DEBUG:boto:Token: None 2024-04-16T17:55:03.031 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:55:03.031 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ExGCjbmv2eZ8wxr4GEumQLaQ9lU= 2024-04-16T17:55:03.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ExGCjbmv2eZ8wxr4GEumQLaQ9lU=', 'Content-Length': '0'} 2024-04-16T17:55:03.035 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db51443cfb6c6215-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.035 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.035 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.035 DEBUG:boto:Method: HEAD 2024-04-16T17:55:03.036 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:55:03.036 DEBUG:boto:Data: 2024-04-16T17:55:03.036 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.036 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.036 DEBUG:boto:Params: {} 2024-04-16T17:55:03.036 DEBUG:boto:Token: None 2024-04-16T17:55:03.036 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:55:03.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ExGCjbmv2eZ8wxr4GEumQLaQ9lU= 2024-04-16T17:55:03.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ExGCjbmv2eZ8wxr4GEumQLaQ9lU=', 'Content-Length': '0'} 2024-04-16T17:55:03.040 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000249920886fe9e9cb-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:55:03.040 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.041 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.041 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.041 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.041 DEBUG:boto:Method: GET 2024-04-16T17:55:03.041 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.041 DEBUG:boto:Data: 2024-04-16T17:55:03.041 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.041 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.041 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.041 DEBUG:boto:Params: {} 2024-04-16T17:55:03.041 DEBUG:boto:Token: None 2024-04-16T17:55:03.042 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KzC563RBfvgt57tAYGwPxNYS+eI= 2024-04-16T17:55:03.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KzC563RBfvgt57tAYGwPxNYS+eI=', 'Content-Length': '0'} 2024-04-16T17:55:03.045 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000be7311be6f5a6d1b-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.086 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.086 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:03.087 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.087 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.087 DEBUG:boto:Method: GET 2024-04-16T17:55:03.087 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.087 DEBUG:boto:Data: 2024-04-16T17:55:03.087 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.087 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.087 DEBUG:boto:Params: {} 2024-04-16T17:55:03.087 DEBUG:boto:Token: None 2024-04-16T17:55:03.087 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:55:03.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KzC563RBfvgt57tAYGwPxNYS+eI= 2024-04-16T17:55:03.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KzC563RBfvgt57tAYGwPxNYS+eI=', 'Content-Length': '0'} 2024-04-16T17:55:03.091 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e07b67c989cbd524-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.091 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.091 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.091 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.091 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.091 DEBUG:boto:Method: GET 2024-04-16T17:55:03.091 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.091 DEBUG:boto:Data: 2024-04-16T17:55:03.092 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.092 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.092 DEBUG:boto:Params: {} 2024-04-16T17:55:03.092 DEBUG:boto:Token: None 2024-04-16T17:55:03.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.096 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063e11b5e1014b1dd-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.096 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.097 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.097 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.097 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.097 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.097 DEBUG:boto:Method: GET 2024-04-16T17:55:03.097 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:55:03.097 DEBUG:boto:Data: 2024-04-16T17:55:03.097 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.097 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.098 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.098 DEBUG:boto:Params: {} 2024-04-16T17:55:03.098 DEBUG:boto:Token: None 2024-04-16T17:55:03.098 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.098 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.102 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3271f66ae633bab-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.102 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.102 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:55:03.102 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.103 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.103 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.103 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.103 DEBUG:boto:Method: GET 2024-04-16T17:55:03.103 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.103 DEBUG:boto:Data: 2024-04-16T17:55:03.103 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.103 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.103 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.103 DEBUG:boto:Params: {} 2024-04-16T17:55:03.103 DEBUG:boto:Token: None 2024-04-16T17:55:03.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg= 2024-04-16T17:55:03.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg=', 'Content-Length': '0'} 2024-04-16T17:55:03.107 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000007c6f3558e0760d7-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.107 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.107 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.107 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.107 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.108 DEBUG:boto:Method: GET 2024-04-16T17:55:03.108 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.108 DEBUG:boto:Data: 2024-04-16T17:55:03.108 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.108 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.108 DEBUG:boto:Params: {} 2024-04-16T17:55:03.108 DEBUG:boto:Token: None 2024-04-16T17:55:03.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg= 2024-04-16T17:55:03.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg=', 'Content-Length': '0'} 2024-04-16T17:55:03.112 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0605292461c9e8c-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.112 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.112 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.112 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.112 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.112 DEBUG:boto:Method: HEAD 2024-04-16T17:55:03.113 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.113 DEBUG:boto:Data: 2024-04-16T17:55:03.113 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.113 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.113 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.113 DEBUG:boto:Params: {} 2024-04-16T17:55:03.113 DEBUG:boto:Token: None 2024-04-16T17:55:03.113 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oSov0Z3ZQOQj4iRtqVCvXV3FCCI= 2024-04-16T17:55:03.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oSov0Z3ZQOQj4iRtqVCvXV3FCCI=', 'Content-Length': '0'} 2024-04-16T17:55:03.116 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d723f5e4973c6eb7-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.116 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.117 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.117 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.117 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.117 DEBUG:boto:Method: HEAD 2024-04-16T17:55:03.117 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.117 DEBUG:boto:Data: 2024-04-16T17:55:03.117 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.117 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.117 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.117 DEBUG:boto:Params: {} 2024-04-16T17:55:03.117 DEBUG:boto:Token: None 2024-04-16T17:55:03.117 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.118 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oSov0Z3ZQOQj4iRtqVCvXV3FCCI= 2024-04-16T17:55:03.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oSov0Z3ZQOQj4iRtqVCvXV3FCCI=', 'Content-Length': '0'} 2024-04-16T17:55:03.121 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd98c3de3a66ed9b-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.121 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:55:03.121 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.121 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.121 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.121 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.121 DEBUG:boto:Method: GET 2024-04-16T17:55:03.121 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.121 DEBUG:boto:Data: 2024-04-16T17:55:03.122 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.122 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.122 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.122 DEBUG:boto:Params: {} 2024-04-16T17:55:03.122 DEBUG:boto:Token: None 2024-04-16T17:55:03.122 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.122 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg= 2024-04-16T17:55:03.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg=', 'Content-Length': '0'} 2024-04-16T17:55:03.125 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eb1f651587d0fc06-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.166 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.166 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.166 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.167 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.167 DEBUG:boto:Method: GET 2024-04-16T17:55:03.167 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.167 DEBUG:boto:Data: 2024-04-16T17:55:03.167 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.167 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.167 DEBUG:boto:Params: {} 2024-04-16T17:55:03.167 DEBUG:boto:Token: None 2024-04-16T17:55:03.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg= 2024-04-16T17:55:03.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg=', 'Content-Length': '0'} 2024-04-16T17:55:03.171 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000041e27b50f09320f1-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.214 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.215 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.215 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.215 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.215 DEBUG:boto:Method: GET 2024-04-16T17:55:03.215 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.215 DEBUG:boto:Data: 2024-04-16T17:55:03.215 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.215 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.215 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.215 DEBUG:boto:Params: {} 2024-04-16T17:55:03.216 DEBUG:boto:Token: None 2024-04-16T17:55:03.216 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.220 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b840fafb0339c7bd-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.220 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.221 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.221 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.221 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.221 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.221 DEBUG:boto:Method: GET 2024-04-16T17:55:03.221 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.221 DEBUG:boto:Data: 2024-04-16T17:55:03.221 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.221 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.221 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.221 DEBUG:boto:Params: {} 2024-04-16T17:55:03.222 DEBUG:boto:Token: None 2024-04-16T17:55:03.222 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.222 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.225 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000afd91d086ea07f85-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.266 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.267 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.267 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.267 DEBUG:boto:Method: HEAD 2024-04-16T17:55:03.267 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:55:03.267 DEBUG:boto:Data: 2024-04-16T17:55:03.267 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.267 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.267 DEBUG:boto:Params: {} 2024-04-16T17:55:03.267 DEBUG:boto:Token: None 2024-04-16T17:55:03.267 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:55:03.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tdozIEpgOQx0AIjZmHFP6JhfA4k= 2024-04-16T17:55:03.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tdozIEpgOQx0AIjZmHFP6JhfA4k=', 'Content-Length': '0'} 2024-04-16T17:55:03.271 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f1bca9856b4c1b2f-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.272 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.272 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.272 DEBUG:boto:Method: HEAD 2024-04-16T17:55:03.272 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:55:03.272 DEBUG:boto:Data: 2024-04-16T17:55:03.272 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.272 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.272 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.272 DEBUG:boto:Params: {} 2024-04-16T17:55:03.272 DEBUG:boto:Token: None 2024-04-16T17:55:03.273 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:55:03.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tdozIEpgOQx0AIjZmHFP6JhfA4k= 2024-04-16T17:55:03.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tdozIEpgOQx0AIjZmHFP6JhfA4k=', 'Content-Length': '0'} 2024-04-16T17:55:03.277 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f2800acf9e8f195b-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:55:03.277 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.277 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.277 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.277 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.278 DEBUG:boto:Method: GET 2024-04-16T17:55:03.278 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.278 DEBUG:boto:Data: 2024-04-16T17:55:03.278 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.278 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.278 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.278 DEBUG:boto:Params: {} 2024-04-16T17:55:03.278 DEBUG:boto:Token: None 2024-04-16T17:55:03.278 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.278 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg= 2024-04-16T17:55:03.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg=', 'Content-Length': '0'} 2024-04-16T17:55:03.282 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d671d6903db763b9-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.282 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.282 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:03.282 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.282 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.282 DEBUG:boto:Method: GET 2024-04-16T17:55:03.282 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.282 DEBUG:boto:Data: 2024-04-16T17:55:03.282 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.283 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.283 DEBUG:boto:Params: {} 2024-04-16T17:55:03.283 DEBUG:boto:Token: None 2024-04-16T17:55:03.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:55:03.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg= 2024-04-16T17:55:03.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1LvNtKyDKW7+NPhFd1sZpglrVfg=', 'Content-Length': '0'} 2024-04-16T17:55:03.286 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ee61b226666af948-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.330 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.330 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.330 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.331 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.331 DEBUG:boto:Method: GET 2024-04-16T17:55:03.331 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.331 DEBUG:boto:Data: 2024-04-16T17:55:03.331 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:03.331 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.331 DEBUG:boto:Params: {} 2024-04-16T17:55:03.331 DEBUG:boto:Token: None 2024-04-16T17:55:03.331 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005d9273ccfd994a2-00661ebb77-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.336 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.336 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:03.336 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:03.336 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.336 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.336 DEBUG:boto:Method: GET 2024-04-16T17:55:03.336 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:55:03.337 DEBUG:boto:Data: 2024-04-16T17:55:03.337 DEBUG:boto:Headers: {} 2024-04-16T17:55:03.337 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:03.337 DEBUG:boto:Port: 8000 2024-04-16T17:55:03.337 DEBUG:boto:Params: {} 2024-04-16T17:55:03.337 DEBUG:boto:Token: None 2024-04-16T17:55:03.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:03 GMT /lybtug-13/ 2024-04-16T17:55:03.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s= 2024-04-16T17:55:03.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:03 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyK64tMUCK3tm72AIPFYJ9Eri8s=', 'Content-Length': '0'} 2024-04-16T17:55:03.341 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4a97ea3b4cff208-00661ebb77-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:03 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:03.382 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:03.383 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-13 zones={test-zone2, test-zone1} 2024-04-16T17:55:03.383 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-14 2024-04-16T17:55:03.383 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:03.384 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:03.384 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-14 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:55:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:03.510+0000 7f99e3089980 1 bucket sync caught up with source: 2024-04-16T17:55:03.510 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000010.29.6, , 00000000002.4.11, , 00000000002.11.11, , , , , ] 2024-04-16T17:55:03.510 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000010.29.6, , 00000000002.4.11, , 00000000002.11.11, , , , , ] 2024-04-16T17:55:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:03.510+0000 7f99e3089980 0 bucket checkpoint complete 2024-04-16T17:55:03.539 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-14 2024-04-16T17:55:03.539 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:03.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:03.540 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:55:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:55:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:55:03.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:03.664+0000 7fb290405980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:03.666 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.50.11, , 00000000010.25.6, , 00000000002.22.11, , , , , ] 2024-04-16T17:55:03.666 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.50.11, , 00000000010.25.6, , 00000000003.24.10, , , , , ] 2024-04-16T17:55:08.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:08.668+0000 7fb290405980 1 bucket sync caught up with source: 2024-04-16T17:55:08.669 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.50.11, , 00000000010.25.6, , 00000000003.24.10, , , , , ] 2024-04-16T17:55:08.669 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.50.11, , 00000000010.25.6, , 00000000003.24.10, , , , , ] 2024-04-16T17:55:08.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:08.668+0000 7fb290405980 0 bucket checkpoint complete 2024-04-16T17:55:08.711 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-14 zones={test-zone2, test-zone1} 2024-04-16T17:55:08.711 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.712 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.712 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.712 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:55:08.712 DEBUG:boto:Data: 2024-04-16T17:55:08.712 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.712 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.712 DEBUG:boto:Params: {} 2024-04-16T17:55:08.712 DEBUG:boto:Token: None 2024-04-16T17:55:08.713 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/ 2024-04-16T17:55:08.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DhQsiOYZRTBjxm3chFWqXx6udp0= 2024-04-16T17:55:08.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DhQsiOYZRTBjxm3chFWqXx6udp0=', 'Content-Length': '0'} 2024-04-16T17:55:08.717 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a82c6a23d7299273-00661ebb7c-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.717 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.717 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.717 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.718 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:55:08.718 DEBUG:boto:Data: 2024-04-16T17:55:08.718 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.718 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.718 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.718 DEBUG:boto:Params: {} 2024-04-16T17:55:08.718 DEBUG:boto:Token: None 2024-04-16T17:55:08.718 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/ 2024-04-16T17:55:08.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DhQsiOYZRTBjxm3chFWqXx6udp0= 2024-04-16T17:55:08.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DhQsiOYZRTBjxm3chFWqXx6udp0=', 'Content-Length': '0'} 2024-04-16T17:55:08.722 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000e4350221176e01b-00661ebb7c-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.722 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:55:08.722 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.722 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.722 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.722 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.722 DEBUG:boto:Method: GET 2024-04-16T17:55:08.722 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.722 DEBUG:boto:Data: 2024-04-16T17:55:08.722 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.722 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.723 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.723 DEBUG:boto:Params: {} 2024-04-16T17:55:08.723 DEBUG:boto:Token: None 2024-04-16T17:55:08.723 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/?versions 2024-04-16T17:55:08.723 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo= 2024-04-16T17:55:08.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo=', 'Content-Length': '0'} 2024-04-16T17:55:08.727 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060cd141f6c95597e-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.727 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.728 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:55:08.728 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:55:08.728 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:55:08.728 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:55:08.728 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.728 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.728 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.728 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.729 DEBUG:boto:Method: GET 2024-04-16T17:55:08.729 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.729 DEBUG:boto:Data: 2024-04-16T17:55:08.729 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.729 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.729 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.729 DEBUG:boto:Params: {} 2024-04-16T17:55:08.729 DEBUG:boto:Token: None 2024-04-16T17:55:08.729 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/?versions 2024-04-16T17:55:08.730 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo= 2024-04-16T17:55:08.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo=', 'Content-Length': '0'} 2024-04-16T17:55:08.733 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6124bc2de122bd9-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.733 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.733 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:55:08.733 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:55:08.733 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:55:08.733 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.734 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.734 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.734 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.734 DEBUG:boto:Method: GET 2024-04-16T17:55:08.734 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.734 DEBUG:boto:Data: 2024-04-16T17:55:08.734 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.734 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.734 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.734 DEBUG:boto:Params: {} 2024-04-16T17:55:08.735 DEBUG:boto:Token: None 2024-04-16T17:55:08.735 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/?versions 2024-04-16T17:55:08.735 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo= 2024-04-16T17:55:08.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo=', 'Content-Length': '0'} 2024-04-16T17:55:08.739 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f392f004994f7ac7-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.782 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.783 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.783 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.783 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.783 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.783 DEBUG:boto:Method: GET 2024-04-16T17:55:08.783 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:55:08.783 DEBUG:boto:Data: 2024-04-16T17:55:08.783 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.783 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.783 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.784 DEBUG:boto:Params: {} 2024-04-16T17:55:08.784 DEBUG:boto:Token: None 2024-04-16T17:55:08.784 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/?versions 2024-04-16T17:55:08.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo= 2024-04-16T17:55:08.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TKHuP250YSQxhjss5ZkwAXx1NVo=', 'Content-Length': '0'} 2024-04-16T17:55:08.787 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000563c99c77036e938-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.788 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:55:08.788 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.788 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.788 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.788 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.789 DEBUG:boto:Method: GET 2024-04-16T17:55:08.789 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.789 DEBUG:boto:Data: 2024-04-16T17:55:08.789 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.789 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.789 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.789 DEBUG:boto:Params: {} 2024-04-16T17:55:08.789 DEBUG:boto:Token: None 2024-04-16T17:55:08.789 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.789 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4= 2024-04-16T17:55:08.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4=', 'Content-Length': '0'} 2024-04-16T17:55:08.792 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001bd1f970fd574bd9-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.793 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.793 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.793 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.793 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.793 DEBUG:boto:Method: GET 2024-04-16T17:55:08.793 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.793 DEBUG:boto:Data: 2024-04-16T17:55:08.793 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.793 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.793 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.793 DEBUG:boto:Params: {} 2024-04-16T17:55:08.793 DEBUG:boto:Token: None 2024-04-16T17:55:08.794 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.794 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4= 2024-04-16T17:55:08.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4=', 'Content-Length': '0'} 2024-04-16T17:55:08.796 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e360328073aa1602-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.797 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.797 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.797 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.797 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.797 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.797 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.797 DEBUG:boto:Data: 2024-04-16T17:55:08.797 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.797 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.797 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.798 DEBUG:boto:Params: {} 2024-04-16T17:55:08.798 DEBUG:boto:Token: None 2024-04-16T17:55:08.798 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.798 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TUKWut3ez3WS9OHJUK7sOD3N9A0= 2024-04-16T17:55:08.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TUKWut3ez3WS9OHJUK7sOD3N9A0=', 'Content-Length': '0'} 2024-04-16T17:55:08.801 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bacff5496aa4508e-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.801 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.801 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.801 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.801 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.801 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.801 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.801 DEBUG:boto:Data: 2024-04-16T17:55:08.802 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.802 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.802 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.802 DEBUG:boto:Params: {} 2024-04-16T17:55:08.802 DEBUG:boto:Token: None 2024-04-16T17:55:08.802 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.802 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TUKWut3ez3WS9OHJUK7sOD3N9A0= 2024-04-16T17:55:08.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TUKWut3ez3WS9OHJUK7sOD3N9A0=', 'Content-Length': '0'} 2024-04-16T17:55:08.805 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008d580a7ee8d4623c-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.805 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:55:08.805 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.805 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.805 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.805 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.805 DEBUG:boto:Method: GET 2024-04-16T17:55:08.806 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.806 DEBUG:boto:Data: 2024-04-16T17:55:08.806 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.806 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.806 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.806 DEBUG:boto:Params: {} 2024-04-16T17:55:08.806 DEBUG:boto:Token: None 2024-04-16T17:55:08.806 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.806 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4= 2024-04-16T17:55:08.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4=', 'Content-Length': '0'} 2024-04-16T17:55:08.809 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008187ff2588e241bf-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.809 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.809 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.809 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.809 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.809 DEBUG:boto:Method: GET 2024-04-16T17:55:08.810 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.810 DEBUG:boto:Data: 2024-04-16T17:55:08.810 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.810 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.810 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.810 DEBUG:boto:Params: {} 2024-04-16T17:55:08.810 DEBUG:boto:Token: None 2024-04-16T17:55:08.810 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.810 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4= 2024-04-16T17:55:08.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4=', 'Content-Length': '0'} 2024-04-16T17:55:08.813 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000102a8d6372a71b7e-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.813 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.813 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.813 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.814 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.814 DEBUG:boto:Method: GET 2024-04-16T17:55:08.814 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:55:08.814 DEBUG:boto:Data: 2024-04-16T17:55:08.814 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.814 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.814 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.814 DEBUG:boto:Params: {} 2024-04-16T17:55:08.814 DEBUG:boto:Token: None 2024-04-16T17:55:08.814 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/ 2024-04-16T17:55:08.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M= 2024-04-16T17:55:08.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M=', 'Content-Length': '0'} 2024-04-16T17:55:08.818 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7bedd0148467099-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.819 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.819 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.819 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.819 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.819 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.819 DEBUG:boto:Method: GET 2024-04-16T17:55:08.819 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:55:08.820 DEBUG:boto:Data: 2024-04-16T17:55:08.820 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.820 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.820 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.820 DEBUG:boto:Params: {} 2024-04-16T17:55:08.820 DEBUG:boto:Token: None 2024-04-16T17:55:08.820 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/ 2024-04-16T17:55:08.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M= 2024-04-16T17:55:08.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M=', 'Content-Length': '0'} 2024-04-16T17:55:08.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc9d746b1819783c-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.824 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.824 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.824 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.824 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.824 DEBUG:boto:Path: /lybtug-14/dummy 2024-04-16T17:55:08.824 DEBUG:boto:Data: 2024-04-16T17:55:08.824 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.825 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.825 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.825 DEBUG:boto:Params: {} 2024-04-16T17:55:08.825 DEBUG:boto:Token: None 2024-04-16T17:55:08.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy 2024-04-16T17:55:08.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bdj9c3FBNEkfTd/Wrnkt0rlEUrw= 2024-04-16T17:55:08.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bdj9c3FBNEkfTd/Wrnkt0rlEUrw=', 'Content-Length': '0'} 2024-04-16T17:55:08.829 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000067aac07fd7de95ee-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.829 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.829 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.829 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.829 DEBUG:boto:Path: /lybtug-14/dummy 2024-04-16T17:55:08.829 DEBUG:boto:Data: 2024-04-16T17:55:08.830 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.830 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.830 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.830 DEBUG:boto:Params: {} 2024-04-16T17:55:08.830 DEBUG:boto:Token: None 2024-04-16T17:55:08.830 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy 2024-04-16T17:55:08.830 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bdj9c3FBNEkfTd/Wrnkt0rlEUrw= 2024-04-16T17:55:08.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bdj9c3FBNEkfTd/Wrnkt0rlEUrw=', 'Content-Length': '0'} 2024-04-16T17:55:08.834 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000838190cb508e0e1c-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.834 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:55:08.835 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.835 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.835 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.835 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.835 DEBUG:boto:Method: GET 2024-04-16T17:55:08.835 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.835 DEBUG:boto:Data: 2024-04-16T17:55:08.835 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.835 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.835 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.835 DEBUG:boto:Params: {} 2024-04-16T17:55:08.836 DEBUG:boto:Token: None 2024-04-16T17:55:08.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4= 2024-04-16T17:55:08.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4=', 'Content-Length': '0'} 2024-04-16T17:55:08.839 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e24b6f8783e44dec-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.839 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:55:08.839 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:55:08.839 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.839 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.839 DEBUG:boto:Method: GET 2024-04-16T17:55:08.839 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.839 DEBUG:boto:Data: 2024-04-16T17:55:08.839 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.840 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.840 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.840 DEBUG:boto:Params: {} 2024-04-16T17:55:08.840 DEBUG:boto:Token: None 2024-04-16T17:55:08.840 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:55:08.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4= 2024-04-16T17:55:08.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfYy8cVLAWmLDdEvzZLDvEIyrT4=', 'Content-Length': '0'} 2024-04-16T17:55:08.843 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004fc4c3ca3f124ece-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.843 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.843 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.843 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.843 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.844 DEBUG:boto:Method: GET 2024-04-16T17:55:08.844 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:55:08.844 DEBUG:boto:Data: 2024-04-16T17:55:08.844 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.844 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.844 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.844 DEBUG:boto:Params: {} 2024-04-16T17:55:08.844 DEBUG:boto:Token: None 2024-04-16T17:55:08.844 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/ 2024-04-16T17:55:08.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M= 2024-04-16T17:55:08.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M=', 'Content-Length': '0'} 2024-04-16T17:55:08.848 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000241a556769510d19-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.890 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.891 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:08.891 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:08.891 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.891 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:55:08.891 DEBUG:boto:Method: GET 2024-04-16T17:55:08.891 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:55:08.891 DEBUG:boto:Data: 2024-04-16T17:55:08.891 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.891 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.891 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.892 DEBUG:boto:Params: {} 2024-04-16T17:55:08.892 DEBUG:boto:Token: None 2024-04-16T17:55:08.892 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/ 2024-04-16T17:55:08.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M= 2024-04-16T17:55:08.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QIBekbRdC2tNec+mco43kzINr8M=', 'Content-Length': '0'} 2024-04-16T17:55:08.895 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000396d4f190913666d-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.938 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:08.939 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:55:08.939 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.939 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.939 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.939 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.939 DEBUG:boto:Method: GET 2024-04-16T17:55:08.939 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.939 DEBUG:boto:Data: 2024-04-16T17:55:08.939 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.939 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.940 DEBUG:boto:Params: {} 2024-04-16T17:55:08.940 DEBUG:boto:Token: None 2024-04-16T17:55:08.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xIliMJfA7gsSIH1802rprsIyH7A= 2024-04-16T17:55:08.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xIliMJfA7gsSIH1802rprsIyH7A=', 'Content-Length': '0'} 2024-04-16T17:55:08.943 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000717fe356c04ef8d0-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.943 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.944 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.944 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.944 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.944 DEBUG:boto:Method: GET 2024-04-16T17:55:08.944 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.944 DEBUG:boto:Data: 2024-04-16T17:55:08.944 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.944 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.944 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.944 DEBUG:boto:Params: {} 2024-04-16T17:55:08.945 DEBUG:boto:Token: None 2024-04-16T17:55:08.945 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xIliMJfA7gsSIH1802rprsIyH7A= 2024-04-16T17:55:08.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xIliMJfA7gsSIH1802rprsIyH7A=', 'Content-Length': '0'} 2024-04-16T17:55:08.948 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000085a52df4a845bacd-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.948 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.949 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.949 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.949 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.949 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.949 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.949 DEBUG:boto:Data: 2024-04-16T17:55:08.949 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.949 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.949 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.949 DEBUG:boto:Params: {} 2024-04-16T17:55:08.949 DEBUG:boto:Token: None 2024-04-16T17:55:08.950 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.950 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sgv88Lwdn+w3eA47rDTjyRifiI8= 2024-04-16T17:55:08.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sgv88Lwdn+w3eA47rDTjyRifiI8=', 'Content-Length': '0'} 2024-04-16T17:55:08.953 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000005660918c62671c4-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.953 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.953 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.953 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.953 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.953 DEBUG:boto:Method: HEAD 2024-04-16T17:55:08.953 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.954 DEBUG:boto:Data: 2024-04-16T17:55:08.954 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.954 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:08.954 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.954 DEBUG:boto:Params: {} 2024-04-16T17:55:08.954 DEBUG:boto:Token: None 2024-04-16T17:55:08.954 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sgv88Lwdn+w3eA47rDTjyRifiI8= 2024-04-16T17:55:08.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sgv88Lwdn+w3eA47rDTjyRifiI8=', 'Content-Length': '0'} 2024-04-16T17:55:08.957 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e5fe248d082e75ce-00661ebb7c-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:08.957 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:55:08.957 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.957 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:08.957 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.957 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.958 DEBUG:boto:Method: GET 2024-04-16T17:55:08.958 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.958 DEBUG:boto:Data: 2024-04-16T17:55:08.958 DEBUG:boto:Headers: {} 2024-04-16T17:55:08.958 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:08.958 DEBUG:boto:Port: 8000 2024-04-16T17:55:08.958 DEBUG:boto:Params: {} 2024-04-16T17:55:08.958 DEBUG:boto:Token: None 2024-04-16T17:55:08.958 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:08 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:08.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xIliMJfA7gsSIH1802rprsIyH7A= 2024-04-16T17:55:08.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xIliMJfA7gsSIH1802rprsIyH7A=', 'Content-Length': '0'} 2024-04-16T17:55:08.961 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007570e516b7debe5b-00661ebb7c-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.002 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.002 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.002 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.003 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.003 DEBUG:boto:Method: GET 2024-04-16T17:55:09.003 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.003 DEBUG:boto:Data: 2024-04-16T17:55:09.003 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.003 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.003 DEBUG:boto:Params: {} 2024-04-16T17:55:09.003 DEBUG:boto:Token: None 2024-04-16T17:55:09.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P9/qUsxHgp4/5uUhHj0r87N3SJM= 2024-04-16T17:55:09.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P9/qUsxHgp4/5uUhHj0r87N3SJM=', 'Content-Length': '0'} 2024-04-16T17:55:09.006 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ab6e0e84df459c2-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.007 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.007 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.007 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.007 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.007 DEBUG:boto:Method: GET 2024-04-16T17:55:09.007 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.007 DEBUG:boto:Data: 2024-04-16T17:55:09.007 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.008 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.008 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.008 DEBUG:boto:Params: {} 2024-04-16T17:55:09.008 DEBUG:boto:Token: None 2024-04-16T17:55:09.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089189a06b7f0318b-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.012 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.012 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.012 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.013 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.013 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.013 DEBUG:boto:Method: GET 2024-04-16T17:55:09.013 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.013 DEBUG:boto:Data: 2024-04-16T17:55:09.013 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.013 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.013 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.013 DEBUG:boto:Params: {} 2024-04-16T17:55:09.013 DEBUG:boto:Token: None 2024-04-16T17:55:09.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.017 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079258919ba03b406-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.058 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.058 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.059 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.059 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.059 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:55:09.059 DEBUG:boto:Data: 2024-04-16T17:55:09.059 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.059 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.059 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.059 DEBUG:boto:Params: {} 2024-04-16T17:55:09.059 DEBUG:boto:Token: None 2024-04-16T17:55:09.060 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:55:09.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iWjiTnv0NYkhxG9K/SQzVwMed1U= 2024-04-16T17:55:09.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iWjiTnv0NYkhxG9K/SQzVwMed1U=', 'Content-Length': '0'} 2024-04-16T17:55:09.064 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006d101c09c502af5b-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.064 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.064 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.064 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.064 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:55:09.065 DEBUG:boto:Data: 2024-04-16T17:55:09.065 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.065 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.065 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.065 DEBUG:boto:Params: {} 2024-04-16T17:55:09.065 DEBUG:boto:Token: None 2024-04-16T17:55:09.065 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:55:09.065 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iWjiTnv0NYkhxG9K/SQzVwMed1U= 2024-04-16T17:55:09.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iWjiTnv0NYkhxG9K/SQzVwMed1U=', 'Content-Length': '0'} 2024-04-16T17:55:09.069 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f6851231dacaa774-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.069 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:55:09.069 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.070 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.070 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.070 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.070 DEBUG:boto:Method: GET 2024-04-16T17:55:09.070 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.070 DEBUG:boto:Data: 2024-04-16T17:55:09.070 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.070 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.070 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.070 DEBUG:boto:Params: {} 2024-04-16T17:55:09.070 DEBUG:boto:Token: None 2024-04-16T17:55:09.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P9/qUsxHgp4/5uUhHj0r87N3SJM= 2024-04-16T17:55:09.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P9/qUsxHgp4/5uUhHj0r87N3SJM=', 'Content-Length': '0'} 2024-04-16T17:55:09.074 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000083fbf6e91630ed46-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.114 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.114 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.115 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.115 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.115 DEBUG:boto:Method: GET 2024-04-16T17:55:09.115 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.115 DEBUG:boto:Data: 2024-04-16T17:55:09.115 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.115 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.115 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.115 DEBUG:boto:Params: {} 2024-04-16T17:55:09.115 DEBUG:boto:Token: None 2024-04-16T17:55:09.115 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:55:09.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P9/qUsxHgp4/5uUhHj0r87N3SJM= 2024-04-16T17:55:09.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P9/qUsxHgp4/5uUhHj0r87N3SJM=', 'Content-Length': '0'} 2024-04-16T17:55:09.118 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000038ff267f0a74db78-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.119 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.119 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.119 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.119 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.119 DEBUG:boto:Method: GET 2024-04-16T17:55:09.119 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.119 DEBUG:boto:Data: 2024-04-16T17:55:09.119 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.119 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.119 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.120 DEBUG:boto:Params: {} 2024-04-16T17:55:09.120 DEBUG:boto:Token: None 2024-04-16T17:55:09.120 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c293a55acc4e73bb-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.124 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.125 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.125 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.125 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.125 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.125 DEBUG:boto:Method: GET 2024-04-16T17:55:09.125 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:55:09.125 DEBUG:boto:Data: 2024-04-16T17:55:09.125 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.125 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.125 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.125 DEBUG:boto:Params: {} 2024-04-16T17:55:09.126 DEBUG:boto:Token: None 2024-04-16T17:55:09.126 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.126 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.130 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f04531b8dc75d311-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.130 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.130 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:55:09.130 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.130 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.131 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.131 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.131 DEBUG:boto:Method: GET 2024-04-16T17:55:09.131 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.131 DEBUG:boto:Data: 2024-04-16T17:55:09.131 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.131 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.131 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.131 DEBUG:boto:Params: {} 2024-04-16T17:55:09.131 DEBUG:boto:Token: None 2024-04-16T17:55:09.131 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.131 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ= 2024-04-16T17:55:09.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.134 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dfbe9d39b79196c2-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.135 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.135 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.135 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.135 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.135 DEBUG:boto:Method: GET 2024-04-16T17:55:09.135 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.135 DEBUG:boto:Data: 2024-04-16T17:55:09.135 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.135 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.136 DEBUG:boto:Params: {} 2024-04-16T17:55:09.136 DEBUG:boto:Token: None 2024-04-16T17:55:09.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ= 2024-04-16T17:55:09.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.139 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000066cbe27f0bd82765-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.182 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.182 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.182 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.183 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.183 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.183 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.183 DEBUG:boto:Data: 2024-04-16T17:55:09.183 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.183 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.183 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.183 DEBUG:boto:Params: {} 2024-04-16T17:55:09.183 DEBUG:boto:Token: None 2024-04-16T17:55:09.183 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rttdyc+6KOmCNSigej59BI/ty78= 2024-04-16T17:55:09.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rttdyc+6KOmCNSigej59BI/ty78=', 'Content-Length': '0'} 2024-04-16T17:55:09.186 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002cbb5e93cf8fc3f6-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.187 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.187 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.187 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.187 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.187 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.187 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.187 DEBUG:boto:Data: 2024-04-16T17:55:09.187 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.188 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.188 DEBUG:boto:Params: {} 2024-04-16T17:55:09.188 DEBUG:boto:Token: None 2024-04-16T17:55:09.188 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rttdyc+6KOmCNSigej59BI/ty78= 2024-04-16T17:55:09.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rttdyc+6KOmCNSigej59BI/ty78=', 'Content-Length': '0'} 2024-04-16T17:55:09.191 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005e1225857a73902d-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.191 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:55:09.191 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.191 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.192 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.192 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.192 DEBUG:boto:Method: GET 2024-04-16T17:55:09.192 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.192 DEBUG:boto:Data: 2024-04-16T17:55:09.192 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.192 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.192 DEBUG:boto:Params: {} 2024-04-16T17:55:09.192 DEBUG:boto:Token: None 2024-04-16T17:55:09.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.193 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ= 2024-04-16T17:55:09.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.196 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dc04b907040f8331-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.238 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.238 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.239 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.239 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.239 DEBUG:boto:Method: GET 2024-04-16T17:55:09.239 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.239 DEBUG:boto:Data: 2024-04-16T17:55:09.239 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.239 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.239 DEBUG:boto:Params: {} 2024-04-16T17:55:09.239 DEBUG:boto:Token: None 2024-04-16T17:55:09.239 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ= 2024-04-16T17:55:09.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.243 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000074f1e04013ad52b2-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.286 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.286 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.286 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.287 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.287 DEBUG:boto:Method: GET 2024-04-16T17:55:09.287 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.287 DEBUG:boto:Data: 2024-04-16T17:55:09.287 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.287 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.287 DEBUG:boto:Params: {} 2024-04-16T17:55:09.287 DEBUG:boto:Token: None 2024-04-16T17:55:09.287 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029a9942e51d90d53-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.292 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.292 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.292 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.292 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.293 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.293 DEBUG:boto:Method: GET 2024-04-16T17:55:09.293 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.293 DEBUG:boto:Data: 2024-04-16T17:55:09.293 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.293 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.293 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.293 DEBUG:boto:Params: {} 2024-04-16T17:55:09.293 DEBUG:boto:Token: None 2024-04-16T17:55:09.293 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.293 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.297 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f18e552ba8d2b22-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.297 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.297 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.297 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.298 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.298 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:55:09.298 DEBUG:boto:Data: 2024-04-16T17:55:09.298 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.298 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.298 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.298 DEBUG:boto:Params: {} 2024-04-16T17:55:09.298 DEBUG:boto:Token: None 2024-04-16T17:55:09.298 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:55:09.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Df/saJzKUFxw4gHHPrjfG05EknQ= 2024-04-16T17:55:09.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Df/saJzKUFxw4gHHPrjfG05EknQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.303 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000088d466acbdf24b41-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.303 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.303 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.303 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.303 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:55:09.303 DEBUG:boto:Data: 2024-04-16T17:55:09.303 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.304 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.304 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.304 DEBUG:boto:Params: {} 2024-04-16T17:55:09.304 DEBUG:boto:Token: None 2024-04-16T17:55:09.304 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:55:09.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Df/saJzKUFxw4gHHPrjfG05EknQ= 2024-04-16T17:55:09.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Df/saJzKUFxw4gHHPrjfG05EknQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.308 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b0e436edea166d6e-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.309 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:55:09.309 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.309 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.309 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.309 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.309 DEBUG:boto:Method: GET 2024-04-16T17:55:09.309 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.309 DEBUG:boto:Data: 2024-04-16T17:55:09.309 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.310 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.310 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.310 DEBUG:boto:Params: {} 2024-04-16T17:55:09.310 DEBUG:boto:Token: None 2024-04-16T17:55:09.310 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.310 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ= 2024-04-16T17:55:09.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.313 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000310bf1dc4a3874d8-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.313 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.314 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.314 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.314 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.314 DEBUG:boto:Method: GET 2024-04-16T17:55:09.314 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.314 DEBUG:boto:Data: 2024-04-16T17:55:09.314 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.314 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.314 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.314 DEBUG:boto:Params: {} 2024-04-16T17:55:09.314 DEBUG:boto:Token: None 2024-04-16T17:55:09.314 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:55:09.315 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ= 2024-04-16T17:55:09.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L67ijZvNUN50Qi82P7Bj11NH3XQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.317 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000005b6367de48e4048-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.358 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.359 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.359 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.359 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.359 DEBUG:boto:Method: GET 2024-04-16T17:55:09.359 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.359 DEBUG:boto:Data: 2024-04-16T17:55:09.359 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.359 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.360 DEBUG:boto:Params: {} 2024-04-16T17:55:09.360 DEBUG:boto:Token: None 2024-04-16T17:55:09.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e419199ac75f3243-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.364 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.365 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.365 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.365 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.365 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.365 DEBUG:boto:Method: GET 2024-04-16T17:55:09.365 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:55:09.365 DEBUG:boto:Data: 2024-04-16T17:55:09.365 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.365 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.365 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.365 DEBUG:boto:Params: {} 2024-04-16T17:55:09.366 DEBUG:boto:Token: None 2024-04-16T17:55:09.366 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.366 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU= 2024-04-16T17:55:09.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xCe+vGGQfPob+xYBtBiTgsQjAYU=', 'Content-Length': '0'} 2024-04-16T17:55:09.369 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007721a8da2924611a-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.370 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.370 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-14 zones={test-zone2, test-zone1} 2024-04-16T17:55:09.370 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:09.370 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:09.370 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.370 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:55:09.371 DEBUG:boto:Data: 2024-04-16T17:55:09.371 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.371 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.371 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.371 DEBUG:boto:Params: {} 2024-04-16T17:55:09.371 DEBUG:boto:Token: None 2024-04-16T17:55:09.371 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-13/ 2024-04-16T17:55:09.371 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VdmVYuMKyIamEAgcvzPXSpzz0oQ= 2024-04-16T17:55:09.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VdmVYuMKyIamEAgcvzPXSpzz0oQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.375 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008eeac25f71556c3e-00661ebb7d-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.375 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:09.375 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:09.375 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999&prefix=obj-test-zone1 2024-04-16T17:55:09.375 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999&prefix=obj-test-zone1 2024-04-16T17:55:09.375 DEBUG:boto:Method: GET 2024-04-16T17:55:09.375 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999&prefix=obj-test-zone1 2024-04-16T17:55:09.375 DEBUG:boto:Data: 2024-04-16T17:55:09.376 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.376 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.376 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.376 DEBUG:boto:Params: {} 2024-04-16T17:55:09.376 DEBUG:boto:Token: None 2024-04-16T17:55:09.376 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-13/?versions 2024-04-16T17:55:09.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CxQIdH2jJv5z1kwYJy1xcoWhLwE= 2024-04-16T17:55:09.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CxQIdH2jJv5z1kwYJy1xcoWhLwE=', 'Content-Length': '0'} 2024-04-16T17:55:09.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0e824ecce026419-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.381 DEBUG:boto:b'lybtug-13obj-test-zone1999falseobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.381 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:09.381 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:55:09.381 DEBUG:boto:path=/lybtug-13/obj-test-zone1?acl 2024-04-16T17:55:09.381 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?acl 2024-04-16T17:55:09.381 DEBUG:boto:Method: PUT 2024-04-16T17:55:09.381 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?acl 2024-04-16T17:55:09.382 DEBUG:boto:Data: 2024-04-16T17:55:09.382 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2024-04-16T17:55:09.382 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.382 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.382 DEBUG:boto:Params: {} 2024-04-16T17:55:09.382 DEBUG:boto:Token: None 2024-04-16T17:55:09.382 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:55:09 GMT x-amz-acl:public-read /lybtug-13/obj-test-zone1?acl 2024-04-16T17:55:09.382 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fkrVTS8t1S9NfRoRZnhU5UhLFwk= 2024-04-16T17:55:09.383 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fkrVTS8t1S9NfRoRZnhU5UhLFwk=', 'Content-Length': '0'} 2024-04-16T17:55:09.393 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004dfd7a5b40e345e5-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.393 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:09.393 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:09.393 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.394 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:55:09.394 DEBUG:boto:Data: 2024-04-16T17:55:09.394 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.394 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.394 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.394 DEBUG:boto:Params: {} 2024-04-16T17:55:09.394 DEBUG:boto:Token: None 2024-04-16T17:55:09.394 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-13/ 2024-04-16T17:55:09.394 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VdmVYuMKyIamEAgcvzPXSpzz0oQ= 2024-04-16T17:55:09.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VdmVYuMKyIamEAgcvzPXSpzz0oQ=', 'Content-Length': '0'} 2024-04-16T17:55:09.398 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cb3d7831b44973c4-00661ebb7d-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.398 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:55:09.398 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:55:09.399 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999&prefix=obj-test-zone2 2024-04-16T17:55:09.399 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999&prefix=obj-test-zone2 2024-04-16T17:55:09.399 DEBUG:boto:Method: GET 2024-04-16T17:55:09.399 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999&prefix=obj-test-zone2 2024-04-16T17:55:09.399 DEBUG:boto:Data: 2024-04-16T17:55:09.399 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.399 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.399 DEBUG:boto:Params: {} 2024-04-16T17:55:09.399 DEBUG:boto:Token: None 2024-04-16T17:55:09.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-13/?versions 2024-04-16T17:55:09.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CxQIdH2jJv5z1kwYJy1xcoWhLwE= 2024-04-16T17:55:09.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CxQIdH2jJv5z1kwYJy1xcoWhLwE=', 'Content-Length': '0'} 2024-04-16T17:55:09.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2dfd8d920af7493-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.446 DEBUG:boto:b'lybtug-13obj-test-zone2999falseobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.447 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:09.447 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:55:09.447 DEBUG:boto:path=/lybtug-13/obj-test-zone2?acl 2024-04-16T17:55:09.447 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?acl 2024-04-16T17:55:09.447 DEBUG:boto:Method: PUT 2024-04-16T17:55:09.447 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?acl 2024-04-16T17:55:09.447 DEBUG:boto:Data: 2024-04-16T17:55:09.447 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2024-04-16T17:55:09.447 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.447 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.448 DEBUG:boto:Params: {} 2024-04-16T17:55:09.448 DEBUG:boto:Token: None 2024-04-16T17:55:09.448 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:55:09 GMT x-amz-acl:public-read /lybtug-13/obj-test-zone2?acl 2024-04-16T17:55:09.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KHXuWpyeRDNO1b+MMcL/1ejIqpM= 2024-04-16T17:55:09.448 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KHXuWpyeRDNO1b+MMcL/1ejIqpM=', 'Content-Length': '0'} 2024-04-16T17:55:09.459 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003348ad6a83c0582f-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.459 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.459 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.459 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.460 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:55:09.460 DEBUG:boto:Data: 2024-04-16T17:55:09.460 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.460 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.460 DEBUG:boto:Params: {} 2024-04-16T17:55:09.460 DEBUG:boto:Token: None 2024-04-16T17:55:09.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/wKLn9KjKuxxLZ/QHCjCaa0sx5Q= 2024-04-16T17:55:09.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/wKLn9KjKuxxLZ/QHCjCaa0sx5Q=', 'Content-Length': '0'} 2024-04-16T17:55:09.463 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005dab2dff2c067108-00661ebb7d-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.464 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.464 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.464 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999&prefix=obj-test-zone1 2024-04-16T17:55:09.464 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999&prefix=obj-test-zone1 2024-04-16T17:55:09.464 DEBUG:boto:Method: GET 2024-04-16T17:55:09.464 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999&prefix=obj-test-zone1 2024-04-16T17:55:09.464 DEBUG:boto:Data: 2024-04-16T17:55:09.464 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.464 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.464 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.464 DEBUG:boto:Params: {} 2024-04-16T17:55:09.465 DEBUG:boto:Token: None 2024-04-16T17:55:09.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/?versions 2024-04-16T17:55:09.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LzW5r5XtWFcghB9el0h0axGNkJY= 2024-04-16T17:55:09.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LzW5r5XtWFcghB9el0h0axGNkJY=', 'Content-Length': '0'} 2024-04-16T17:55:09.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004eb324326b4fa56b-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.468 DEBUG:boto:b'lybtug-14obj-test-zone1999falseobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.468 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.469 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:55:09.469 DEBUG:boto:path=/lybtug-14/obj-test-zone1?acl 2024-04-16T17:55:09.469 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?acl 2024-04-16T17:55:09.469 DEBUG:boto:Method: PUT 2024-04-16T17:55:09.469 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?acl 2024-04-16T17:55:09.469 DEBUG:boto:Data: 2024-04-16T17:55:09.469 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2024-04-16T17:55:09.469 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:55:09.469 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.469 DEBUG:boto:Params: {} 2024-04-16T17:55:09.469 DEBUG:boto:Token: None 2024-04-16T17:55:09.469 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:55:09 GMT x-amz-acl:public-read /lybtug-14/obj-test-zone1?acl 2024-04-16T17:55:09.470 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AfEK8vfkpfjIDs1r/LjMn9FrLBg= 2024-04-16T17:55:09.470 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AfEK8vfkpfjIDs1r/LjMn9FrLBg=', 'Content-Length': '0'} 2024-04-16T17:55:09.479 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d6adab046cba297e-00661ebb7d-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.480 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.480 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.480 DEBUG:boto:Method: HEAD 2024-04-16T17:55:09.480 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:55:09.480 DEBUG:boto:Data: 2024-04-16T17:55:09.480 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.480 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.480 DEBUG:boto:Params: {} 2024-04-16T17:55:09.480 DEBUG:boto:Token: None 2024-04-16T17:55:09.481 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/ 2024-04-16T17:55:09.481 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/wKLn9KjKuxxLZ/QHCjCaa0sx5Q= 2024-04-16T17:55:09.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/wKLn9KjKuxxLZ/QHCjCaa0sx5Q=', 'Content-Length': '0'} 2024-04-16T17:55:09.484 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000de0b55287bddcb74-00661ebb7d-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.485 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:55:09.485 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:55:09.485 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999&prefix=obj-test-zone2 2024-04-16T17:55:09.485 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999&prefix=obj-test-zone2 2024-04-16T17:55:09.485 DEBUG:boto:Method: GET 2024-04-16T17:55:09.485 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999&prefix=obj-test-zone2 2024-04-16T17:55:09.485 DEBUG:boto:Data: 2024-04-16T17:55:09.485 DEBUG:boto:Headers: {} 2024-04-16T17:55:09.485 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.485 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.485 DEBUG:boto:Params: {} 2024-04-16T17:55:09.486 DEBUG:boto:Token: None 2024-04-16T17:55:09.486 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:55:09 GMT /lybtug-14/?versions 2024-04-16T17:55:09.486 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LzW5r5XtWFcghB9el0h0axGNkJY= 2024-04-16T17:55:09.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LzW5r5XtWFcghB9el0h0axGNkJY=', 'Content-Length': '0'} 2024-04-16T17:55:09.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c49ac74b88bc987a-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.534 DEBUG:boto:b'lybtug-14obj-test-zone2999falseobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:55:09.535 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.535 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:55:09.535 DEBUG:boto:path=/lybtug-14/obj-test-zone2?acl 2024-04-16T17:55:09.535 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?acl 2024-04-16T17:55:09.535 DEBUG:boto:Method: PUT 2024-04-16T17:55:09.535 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?acl 2024-04-16T17:55:09.535 DEBUG:boto:Data: 2024-04-16T17:55:09.535 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2024-04-16T17:55:09.535 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:55:09.535 DEBUG:boto:Port: 8000 2024-04-16T17:55:09.535 DEBUG:boto:Params: {} 2024-04-16T17:55:09.536 DEBUG:boto:Token: None 2024-04-16T17:55:09.536 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:55:09 GMT x-amz-acl:public-read /lybtug-14/obj-test-zone2?acl 2024-04-16T17:55:09.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2fE15qlQ5MQJxvtIerO5zFa2HK4= 2024-04-16T17:55:09.536 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:55:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2fE15qlQ5MQJxvtIerO5zFa2HK4=', 'Content-Length': '0'} 2024-04-16T17:55:09.547 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000036da6d1a20f89313-00661ebb7d-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:55:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:55:09.547 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-13 2024-04-16T17:55:09.548 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:09.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:09.548 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:55:09.598 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:55:09.598 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:55:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:09.674+0000 7f79d4720980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:09.674 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000010.51.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:09.675 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000011.54.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:14.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:14.678+0000 7f79d4720980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:14.677 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000010.51.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:14.677 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000011.54.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:19.681+0000 7f79d4720980 1 bucket sync caught up with source: 2024-04-16T17:55:19.679 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000011.54.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:19.680 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000011.54.6, , 00000000002.33.11, , 00000000003.21.10, , , , , ] 2024-04-16T17:55:19.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:55:19.681+0000 7f79d4720980 0 bucket checkpoint complete 2024-04-16T17:55:19.726 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-13 2024-04-16T17:55:19.727 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:19.727 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-13', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:55:19.727 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-13 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:55:19.778 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:55:19.778 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:55:19.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:19.852+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:19.854 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:19.854 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:24.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:24.856+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:24.856 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:24.856 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:29.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:29.856+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:29.859 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:29.859 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:34.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:34.859+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:34.861 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:34.861 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:39.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:39.863+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:39.863 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:39.863 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:44.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:44.863+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:44.866 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:44.866 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:49.867+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:49.868 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:49.868 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:54.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:54.871+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:54.870 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:54.871 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:59.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:55:59.871+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:55:59.873 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:55:59.873 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:56:04.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:04.875+0000 7f8d2c3e1980 1 waiting for incremental sync to catch up: 2024-04-16T17:56:04.875 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000010.46.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:56:04.875 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:56:09.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:09.879+0000 7f8d2c3e1980 1 bucket sync caught up with source: 2024-04-16T17:56:09.877 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:56:09.877 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.43.11, , 00000000011.53.6, , 00000000002.16.11, , , , , ] 2024-04-16T17:56:09.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:09.879+0000 7f8d2c3e1980 0 bucket checkpoint complete 2024-04-16T17:56:09.926 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-13 zones={test-zone2, test-zone1} 2024-04-16T17:56:09.926 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.927 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.927 DEBUG:boto:Method: HEAD 2024-04-16T17:56:09.927 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:56:09.927 DEBUG:boto:Data: 2024-04-16T17:56:09.927 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.927 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.927 DEBUG:boto:Params: {} 2024-04-16T17:56:09.928 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T17:56:09.928 DEBUG:boto:Token: None 2024-04-16T17:56:09.928 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/ 2024-04-16T17:56:09.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:42VP16uY1fkQ63QXgesewz66S70= 2024-04-16T17:56:09.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:42VP16uY1fkQ63QXgesewz66S70=', 'Content-Length': '0'} 2024-04-16T17:56:09.934 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a38cba13c9e87915-00661ebbb9-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.934 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.934 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.935 DEBUG:boto:Method: HEAD 2024-04-16T17:56:09.935 DEBUG:boto:Path: /lybtug-13/ 2024-04-16T17:56:09.935 DEBUG:boto:Data: 2024-04-16T17:56:09.935 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:09.935 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.935 DEBUG:boto:Params: {} 2024-04-16T17:56:09.935 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T17:56:09.935 DEBUG:boto:Token: None 2024-04-16T17:56:09.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/ 2024-04-16T17:56:09.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:42VP16uY1fkQ63QXgesewz66S70= 2024-04-16T17:56:09.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:42VP16uY1fkQ63QXgesewz66S70=', 'Content-Length': '0'} 2024-04-16T17:56:09.941 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c05e90167f147fd0-00661ebbb9-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.941 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:56:09.941 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.941 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.941 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.941 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.941 DEBUG:boto:Method: GET 2024-04-16T17:56:09.941 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.942 DEBUG:boto:Data: 2024-04-16T17:56:09.942 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.942 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.942 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.942 DEBUG:boto:Params: {} 2024-04-16T17:56:09.942 DEBUG:boto:Token: None 2024-04-16T17:56:09.942 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/?versions 2024-04-16T17:56:09.942 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY= 2024-04-16T17:56:09.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY=', 'Content-Length': '0'} 2024-04-16T17:56:09.946 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040a9688161738135-00661ebbb9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.946 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:09.947 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:09.947 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:56:09.947 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:56:09.947 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:56:09.947 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.948 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.948 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.948 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.948 DEBUG:boto:Method: GET 2024-04-16T17:56:09.948 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.948 DEBUG:boto:Data: 2024-04-16T17:56:09.948 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.948 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:09.948 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.948 DEBUG:boto:Params: {} 2024-04-16T17:56:09.948 DEBUG:boto:Token: None 2024-04-16T17:56:09.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/?versions 2024-04-16T17:56:09.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY= 2024-04-16T17:56:09.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY=', 'Content-Length': '0'} 2024-04-16T17:56:09.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037a571cabf0e820e-00661ebbb9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.953 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:09.953 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:09.953 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:56:09.953 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:56:09.953 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.954 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.954 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.954 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.954 DEBUG:boto:Method: GET 2024-04-16T17:56:09.954 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.954 DEBUG:boto:Data: 2024-04-16T17:56:09.954 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.954 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.954 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.954 DEBUG:boto:Params: {} 2024-04-16T17:56:09.954 DEBUG:boto:Token: None 2024-04-16T17:56:09.955 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/?versions 2024-04-16T17:56:09.955 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY= 2024-04-16T17:56:09.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY=', 'Content-Length': '0'} 2024-04-16T17:56:09.959 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a438ca5207f52757-00661ebbb9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.959 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:09.960 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.960 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.960 DEBUG:boto:path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.960 DEBUG:boto:auth_path=/lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.960 DEBUG:boto:Method: GET 2024-04-16T17:56:09.960 DEBUG:boto:Path: /lybtug-13/?versions&max-keys=999 2024-04-16T17:56:09.960 DEBUG:boto:Data: 2024-04-16T17:56:09.960 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.960 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:09.961 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.961 DEBUG:boto:Params: {} 2024-04-16T17:56:09.961 DEBUG:boto:Token: None 2024-04-16T17:56:09.961 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/?versions 2024-04-16T17:56:09.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY= 2024-04-16T17:56:09.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GsLxziRbgv0EQgyfA0EHMS6zouY=', 'Content-Length': '0'} 2024-04-16T17:56:09.965 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fee2514a7d08769d-00661ebbb9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.965 DEBUG:boto:b'lybtug-13999falsedummyi-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3Dtrue2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJtrue2024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2BddSazy2TorOsY9OlUrEenOITSq5no0true2024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:09.966 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:09.966 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:09.966 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:09.966 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.966 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.966 DEBUG:boto:Method: GET 2024-04-16T17:56:09.966 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.966 DEBUG:boto:Data: 2024-04-16T17:56:09.966 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.966 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.967 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.967 DEBUG:boto:Params: {} 2024-04-16T17:56:09.967 DEBUG:boto:Token: None 2024-04-16T17:56:09.967 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.967 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c= 2024-04-16T17:56:09.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c=', 'Content-Length': '0'} 2024-04-16T17:56:09.970 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000333b61ad32267ada-00661ebbb9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.970 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:09.971 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:09.971 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.971 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.971 DEBUG:boto:Method: GET 2024-04-16T17:56:09.971 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.971 DEBUG:boto:Data: 2024-04-16T17:56:09.971 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:09.971 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.971 DEBUG:boto:Params: {} 2024-04-16T17:56:09.971 DEBUG:boto:Token: None 2024-04-16T17:56:09.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c= 2024-04-16T17:56:09.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c=', 'Content-Length': '0'} 2024-04-16T17:56:09.974 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002ae640303967cd2c-00661ebbb9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.974 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:09.975 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:09.975 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.975 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.975 DEBUG:boto:Method: HEAD 2024-04-16T17:56:09.975 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.975 DEBUG:boto:Data: 2024-04-16T17:56:09.975 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.975 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.975 DEBUG:boto:Params: {} 2024-04-16T17:56:09.976 DEBUG:boto:Token: None 2024-04-16T17:56:09.976 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ooX6nZPJfWAf0dJ/xTv6gD8UOjc= 2024-04-16T17:56:09.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ooX6nZPJfWAf0dJ/xTv6gD8UOjc=', 'Content-Length': '0'} 2024-04-16T17:56:09.978 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c0842736c6478f36-00661ebbb9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.979 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:09.979 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:09.979 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.979 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.979 DEBUG:boto:Method: HEAD 2024-04-16T17:56:09.979 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.979 DEBUG:boto:Data: 2024-04-16T17:56:09.979 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.979 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:09.979 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.980 DEBUG:boto:Params: {} 2024-04-16T17:56:09.980 DEBUG:boto:Token: None 2024-04-16T17:56:09.980 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ooX6nZPJfWAf0dJ/xTv6gD8UOjc= 2024-04-16T17:56:09.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ooX6nZPJfWAf0dJ/xTv6gD8UOjc=', 'Content-Length': '0'} 2024-04-16T17:56:09.983 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000cec41c521f320d9-00661ebbb9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.983 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:09.983 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:09.983 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:09.983 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.983 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.983 DEBUG:boto:Method: GET 2024-04-16T17:56:09.983 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.984 DEBUG:boto:Data: 2024-04-16T17:56:09.984 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.984 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.984 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.984 DEBUG:boto:Params: {} 2024-04-16T17:56:09.984 DEBUG:boto:Token: None 2024-04-16T17:56:09.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c= 2024-04-16T17:56:09.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c=', 'Content-Length': '0'} 2024-04-16T17:56:09.987 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000074889d4497f94313-00661ebbb9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.987 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:09.987 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:09.988 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.988 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.988 DEBUG:boto:Method: GET 2024-04-16T17:56:09.988 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.988 DEBUG:boto:Data: 2024-04-16T17:56:09.988 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.988 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:09.988 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.988 DEBUG:boto:Params: {} 2024-04-16T17:56:09.988 DEBUG:boto:Token: None 2024-04-16T17:56:09.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:09.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c= 2024-04-16T17:56:09.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ug6dKR3eWMBZLHBHTd0colPAm8c=', 'Content-Length': '0'} 2024-04-16T17:56:09.991 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002f6b86da3b173cf3-00661ebbb9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:09.992 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:09.992 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:09.992 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:09.992 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:09.992 DEBUG:boto:Method: GET 2024-04-16T17:56:09.992 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:56:09.992 DEBUG:boto:Data: 2024-04-16T17:56:09.992 DEBUG:boto:Headers: {} 2024-04-16T17:56:09.992 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:09.993 DEBUG:boto:Port: 8000 2024-04-16T17:56:09.993 DEBUG:boto:Params: {} 2024-04-16T17:56:09.993 DEBUG:boto:Token: None 2024-04-16T17:56:09.993 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:09 GMT /lybtug-13/ 2024-04-16T17:56:09.993 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gyF5lZkKM9qsOnUsfwm6V4EpiOQ= 2024-04-16T17:56:09.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gyF5lZkKM9qsOnUsfwm6V4EpiOQ=', 'Content-Length': '0'} 2024-04-16T17:56:09.997 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc49e3d67cf611f3-00661ebbb9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:09 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.038 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.039 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.039 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.039 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:10.039 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:10.039 DEBUG:boto:Method: GET 2024-04-16T17:56:10.039 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:56:10.039 DEBUG:boto:Data: 2024-04-16T17:56:10.039 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.040 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.040 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.040 DEBUG:boto:Params: {} 2024-04-16T17:56:10.040 DEBUG:boto:Token: None 2024-04-16T17:56:10.040 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc1e0a2b3fe60139-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.044 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.044 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:10.045 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:10.045 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.045 DEBUG:boto:Path: /lybtug-13/dummy 2024-04-16T17:56:10.045 DEBUG:boto:Data: 2024-04-16T17:56:10.045 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.045 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.045 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.045 DEBUG:boto:Params: {} 2024-04-16T17:56:10.045 DEBUG:boto:Token: None 2024-04-16T17:56:10.045 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/dummy 2024-04-16T17:56:10.046 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lDAFvK1PItsJltXvXuSiJBJK61s= 2024-04-16T17:56:10.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lDAFvK1PItsJltXvXuSiJBJK61s=', 'Content-Length': '0'} 2024-04-16T17:56:10.050 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003ad4e98e75102e55-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.050 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:10.050 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:10.050 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.050 DEBUG:boto:Path: /lybtug-13/dummy 2024-04-16T17:56:10.050 DEBUG:boto:Data: 2024-04-16T17:56:10.050 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.050 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.050 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.051 DEBUG:boto:Params: {} 2024-04-16T17:56:10.051 DEBUG:boto:Token: None 2024-04-16T17:56:10.051 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/dummy 2024-04-16T17:56:10.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lDAFvK1PItsJltXvXuSiJBJK61s= 2024-04-16T17:56:10.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lDAFvK1PItsJltXvXuSiJBJK61s=', 'Content-Length': '0'} 2024-04-16T17:56:10.055 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002651665e3aea5bc5-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.055 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:10.055 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:10.055 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:10.055 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.055 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.055 DEBUG:boto:Method: GET 2024-04-16T17:56:10.055 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.055 DEBUG:boto:Data: 2024-04-16T17:56:10.055 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.056 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.056 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.056 DEBUG:boto:Params: {} 2024-04-16T17:56:10.056 DEBUG:boto:Token: None 2024-04-16T17:56:10.056 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WF+u+Q1Xycy5bzMb1bKS7AhQC2A= 2024-04-16T17:56:10.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WF+u+Q1Xycy5bzMb1bKS7AhQC2A=', 'Content-Length': '0'} 2024-04-16T17:56:10.059 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:42 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b3702611c1e8fc73-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.059 DEBUG:boto:path=/lybtug-13/dummy 2024-04-16T17:56:10.060 DEBUG:boto:auth_path=/lybtug-13/dummy 2024-04-16T17:56:10.060 DEBUG:boto:path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.060 DEBUG:boto:auth_path=/lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.060 DEBUG:boto:Method: GET 2024-04-16T17:56:10.060 DEBUG:boto:Path: /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.060 DEBUG:boto:Data: 2024-04-16T17:56:10.060 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.060 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.060 DEBUG:boto:Params: {} 2024-04-16T17:56:10.060 DEBUG:boto:Token: None 2024-04-16T17:56:10.061 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/dummy?versionId=i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D 2024-04-16T17:56:10.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WF+u+Q1Xycy5bzMb1bKS7AhQC2A= 2024-04-16T17:56:10.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WF+u+Q1Xycy5bzMb1bKS7AhQC2A=', 'Content-Length': '0'} 2024-04-16T17:56:10.064 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:41 GMT'), ('x-amz-version-id', 'i-zfjiVUaCPv.K1Z8vQLEz-CQRWqU3D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d8ae5fc5dd6d6ffb-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.064 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.064 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.064 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:10.064 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:10.064 DEBUG:boto:Method: GET 2024-04-16T17:56:10.064 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:56:10.064 DEBUG:boto:Data: 2024-04-16T17:56:10.065 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.065 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.065 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.065 DEBUG:boto:Params: {} 2024-04-16T17:56:10.065 DEBUG:boto:Token: None 2024-04-16T17:56:10.065 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.065 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e57496a6fccebec5-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.070 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.070 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.070 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.070 DEBUG:boto:path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:10.070 DEBUG:boto:auth_path=/lybtug-13/?prefix=dummy 2024-04-16T17:56:10.070 DEBUG:boto:Method: GET 2024-04-16T17:56:10.071 DEBUG:boto:Path: /lybtug-13/?prefix=dummy 2024-04-16T17:56:10.071 DEBUG:boto:Data: 2024-04-16T17:56:10.071 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.071 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.071 DEBUG:boto:Params: {} 2024-04-16T17:56:10.071 DEBUG:boto:Token: None 2024-04-16T17:56:10.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d31de814546afc77-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.118 DEBUG:boto:b'lybtug-13dummy1000falsedummy2024-04-16T17:54:41.362Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.119 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:56:10.119 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.119 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.119 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.119 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.119 DEBUG:boto:Method: GET 2024-04-16T17:56:10.119 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.119 DEBUG:boto:Data: 2024-04-16T17:56:10.119 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.119 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.119 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.119 DEBUG:boto:Params: {} 2024-04-16T17:56:10.120 DEBUG:boto:Token: None 2024-04-16T17:56:10.120 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ= 2024-04-16T17:56:10.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ=', 'Content-Length': '0'} 2024-04-16T17:56:10.123 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a51ee40489a9f43-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.166 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.166 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.167 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.167 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.167 DEBUG:boto:Method: GET 2024-04-16T17:56:10.167 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.167 DEBUG:boto:Data: 2024-04-16T17:56:10.167 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.167 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.167 DEBUG:boto:Params: {} 2024-04-16T17:56:10.167 DEBUG:boto:Token: None 2024-04-16T17:56:10.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ= 2024-04-16T17:56:10.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ=', 'Content-Length': '0'} 2024-04-16T17:56:10.171 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000713f64d62cee6443-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.171 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.171 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.171 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.171 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.172 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.172 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.172 DEBUG:boto:Data: 2024-04-16T17:56:10.172 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.172 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.172 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.172 DEBUG:boto:Params: {} 2024-04-16T17:56:10.172 DEBUG:boto:Token: None 2024-04-16T17:56:10.172 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aEFhDy2EzYiTYpoVThL2CBp/vTg= 2024-04-16T17:56:10.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aEFhDy2EzYiTYpoVThL2CBp/vTg=', 'Content-Length': '0'} 2024-04-16T17:56:10.175 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a436bd7afacb6c2f-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.176 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.176 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.176 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.176 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.176 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.176 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.176 DEBUG:boto:Data: 2024-04-16T17:56:10.176 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.176 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.176 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.176 DEBUG:boto:Params: {} 2024-04-16T17:56:10.177 DEBUG:boto:Token: None 2024-04-16T17:56:10.177 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aEFhDy2EzYiTYpoVThL2CBp/vTg= 2024-04-16T17:56:10.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aEFhDy2EzYiTYpoVThL2CBp/vTg=', 'Content-Length': '0'} 2024-04-16T17:56:10.180 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f6d6c2b8bfbcc3ce-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.180 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:56:10.180 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.180 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.180 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.181 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.181 DEBUG:boto:Method: GET 2024-04-16T17:56:10.181 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.181 DEBUG:boto:Data: 2024-04-16T17:56:10.181 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.181 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.181 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.181 DEBUG:boto:Params: {} 2024-04-16T17:56:10.181 DEBUG:boto:Token: None 2024-04-16T17:56:10.181 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.181 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ= 2024-04-16T17:56:10.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ=', 'Content-Length': '0'} 2024-04-16T17:56:10.184 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d291c0637397efbd-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.226 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.226 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.227 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.227 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.227 DEBUG:boto:Method: GET 2024-04-16T17:56:10.227 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.227 DEBUG:boto:Data: 2024-04-16T17:56:10.227 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.227 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.227 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.227 DEBUG:boto:Params: {} 2024-04-16T17:56:10.227 DEBUG:boto:Token: None 2024-04-16T17:56:10.227 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ= 2024-04-16T17:56:10.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ=', 'Content-Length': '0'} 2024-04-16T17:56:10.231 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e8901ac877601824-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.232 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.232 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.232 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.232 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.232 DEBUG:boto:Method: GET 2024-04-16T17:56:10.232 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.232 DEBUG:boto:Data: 2024-04-16T17:56:10.232 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.232 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.232 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.232 DEBUG:boto:Params: {} 2024-04-16T17:56:10.233 DEBUG:boto:Token: None 2024-04-16T17:56:10.233 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.233 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.237 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000861d9bf8eb61d616-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.237 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.237 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.238 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.238 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.238 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.238 DEBUG:boto:Method: GET 2024-04-16T17:56:10.238 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.238 DEBUG:boto:Data: 2024-04-16T17:56:10.238 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.238 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.238 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.238 DEBUG:boto:Params: {} 2024-04-16T17:56:10.238 DEBUG:boto:Token: None 2024-04-16T17:56:10.239 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.239 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.242 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007367c8a7f7afb72e-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.242 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.243 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.243 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.243 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.243 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:56:10.243 DEBUG:boto:Data: 2024-04-16T17:56:10.243 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.243 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.243 DEBUG:boto:Params: {} 2024-04-16T17:56:10.244 DEBUG:boto:Token: None 2024-04-16T17:56:10.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:56:10.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2jkrhh6Y10vv/MMWcd6z0Yjiggw= 2024-04-16T17:56:10.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2jkrhh6Y10vv/MMWcd6z0Yjiggw=', 'Content-Length': '0'} 2024-04-16T17:56:10.248 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003896948269867290-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.248 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.248 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.248 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.249 DEBUG:boto:Path: /lybtug-13/obj-test-zone1 2024-04-16T17:56:10.249 DEBUG:boto:Data: 2024-04-16T17:56:10.249 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.249 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.249 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.249 DEBUG:boto:Params: {} 2024-04-16T17:56:10.249 DEBUG:boto:Token: None 2024-04-16T17:56:10.249 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1 2024-04-16T17:56:10.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2jkrhh6Y10vv/MMWcd6z0Yjiggw= 2024-04-16T17:56:10.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2jkrhh6Y10vv/MMWcd6z0Yjiggw=', 'Content-Length': '0'} 2024-04-16T17:56:10.253 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058f1623e3b0dc365-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.253 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:56:10.253 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.253 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.253 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.254 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.254 DEBUG:boto:Method: GET 2024-04-16T17:56:10.254 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.254 DEBUG:boto:Data: 2024-04-16T17:56:10.254 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.254 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.254 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.254 DEBUG:boto:Params: {} 2024-04-16T17:56:10.254 DEBUG:boto:Token: None 2024-04-16T17:56:10.254 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.254 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ= 2024-04-16T17:56:10.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ=', 'Content-Length': '0'} 2024-04-16T17:56:10.257 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000077cbc2e3e5a2c507-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.302 DEBUG:boto:path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.303 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1 2024-04-16T17:56:10.303 DEBUG:boto:path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.303 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.303 DEBUG:boto:Method: GET 2024-04-16T17:56:10.303 DEBUG:boto:Path: /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.303 DEBUG:boto:Data: 2024-04-16T17:56:10.303 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.303 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.303 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.303 DEBUG:boto:Params: {} 2024-04-16T17:56:10.304 DEBUG:boto:Token: None 2024-04-16T17:56:10.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone1?versionId=yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ 2024-04-16T17:56:10.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ= 2024-04-16T17:56:10.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9YrSb5n+YI+4hRkL4DWrk2nn0jQ=', 'Content-Length': '0'} 2024-04-16T17:56:10.307 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'yO4XRM0iCwMBQImt4qJ2vv1o.aiFGKJ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000921c9942deb3f7c8-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.307 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.307 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.308 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.308 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.308 DEBUG:boto:Method: GET 2024-04-16T17:56:10.308 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.308 DEBUG:boto:Data: 2024-04-16T17:56:10.308 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.308 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.308 DEBUG:boto:Params: {} 2024-04-16T17:56:10.308 DEBUG:boto:Token: None 2024-04-16T17:56:10.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006e1f8a89f9062ea-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.313 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.313 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.313 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.313 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.313 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.314 DEBUG:boto:Method: GET 2024-04-16T17:56:10.314 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone1 2024-04-16T17:56:10.314 DEBUG:boto:Data: 2024-04-16T17:56:10.314 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.314 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.314 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.314 DEBUG:boto:Params: {} 2024-04-16T17:56:10.314 DEBUG:boto:Token: None 2024-04-16T17:56:10.314 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.314 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac26b2fc10c96e9d-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.318 DEBUG:boto:b'lybtug-13obj-test-zone11000falseobj-test-zone12024-04-16T17:54:51.783Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.319 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:56:10.319 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.319 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.319 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.319 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.320 DEBUG:boto:Method: GET 2024-04-16T17:56:10.320 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.320 DEBUG:boto:Data: 2024-04-16T17:56:10.320 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.320 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.320 DEBUG:boto:Params: {} 2024-04-16T17:56:10.320 DEBUG:boto:Token: None 2024-04-16T17:56:10.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo= 2024-04-16T17:56:10.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo=', 'Content-Length': '0'} 2024-04-16T17:56:10.324 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e1f539484d3620b8-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.324 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.324 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.324 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.325 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.325 DEBUG:boto:Method: GET 2024-04-16T17:56:10.325 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.325 DEBUG:boto:Data: 2024-04-16T17:56:10.325 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.325 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.325 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.325 DEBUG:boto:Params: {} 2024-04-16T17:56:10.325 DEBUG:boto:Token: None 2024-04-16T17:56:10.325 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.326 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo= 2024-04-16T17:56:10.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo=', 'Content-Length': '0'} 2024-04-16T17:56:10.329 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009d4cc958433ce9f9-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.329 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.329 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.330 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.330 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.330 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.330 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.330 DEBUG:boto:Data: 2024-04-16T17:56:10.330 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.330 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.330 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.330 DEBUG:boto:Params: {} 2024-04-16T17:56:10.330 DEBUG:boto:Token: None 2024-04-16T17:56:10.331 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.331 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BlNPVmc5wAyPaf7YMPiB536N6fE= 2024-04-16T17:56:10.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BlNPVmc5wAyPaf7YMPiB536N6fE=', 'Content-Length': '0'} 2024-04-16T17:56:10.334 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004c75807c3ab7ecdd-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.334 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.334 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.334 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.334 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.334 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.335 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.335 DEBUG:boto:Data: 2024-04-16T17:56:10.335 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.335 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.335 DEBUG:boto:Params: {} 2024-04-16T17:56:10.335 DEBUG:boto:Token: None 2024-04-16T17:56:10.335 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BlNPVmc5wAyPaf7YMPiB536N6fE= 2024-04-16T17:56:10.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BlNPVmc5wAyPaf7YMPiB536N6fE=', 'Content-Length': '0'} 2024-04-16T17:56:10.338 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b896fd871f646a33-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.338 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:56:10.339 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.339 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.339 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.339 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.339 DEBUG:boto:Method: GET 2024-04-16T17:56:10.339 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.339 DEBUG:boto:Data: 2024-04-16T17:56:10.339 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.339 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.339 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.339 DEBUG:boto:Params: {} 2024-04-16T17:56:10.340 DEBUG:boto:Token: None 2024-04-16T17:56:10.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo= 2024-04-16T17:56:10.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo=', 'Content-Length': '0'} 2024-04-16T17:56:10.343 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000045b1425f5a351a8a-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.386 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.387 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.387 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.387 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.387 DEBUG:boto:Method: GET 2024-04-16T17:56:10.387 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.387 DEBUG:boto:Data: 2024-04-16T17:56:10.387 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.387 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.387 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.387 DEBUG:boto:Params: {} 2024-04-16T17:56:10.388 DEBUG:boto:Token: None 2024-04-16T17:56:10.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo= 2024-04-16T17:56:10.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo=', 'Content-Length': '0'} 2024-04-16T17:56:10.391 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006a0a094cc9aa8a9f-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.434 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.435 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.435 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.435 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.435 DEBUG:boto:Method: GET 2024-04-16T17:56:10.435 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.435 DEBUG:boto:Data: 2024-04-16T17:56:10.435 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.435 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.435 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.435 DEBUG:boto:Params: {} 2024-04-16T17:56:10.435 DEBUG:boto:Token: None 2024-04-16T17:56:10.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a783b198cff6e42-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.440 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.441 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.441 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.441 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.441 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.441 DEBUG:boto:Method: GET 2024-04-16T17:56:10.441 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.441 DEBUG:boto:Data: 2024-04-16T17:56:10.441 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.441 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.441 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.441 DEBUG:boto:Params: {} 2024-04-16T17:56:10.442 DEBUG:boto:Token: None 2024-04-16T17:56:10.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000827505ff17e78a11-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.486 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.487 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.487 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.487 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.487 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:56:10.487 DEBUG:boto:Data: 2024-04-16T17:56:10.487 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.488 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.488 DEBUG:boto:Params: {} 2024-04-16T17:56:10.488 DEBUG:boto:Token: None 2024-04-16T17:56:10.488 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:56:10.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:74pqbSudrXS/99jHkS1FBAs/CaM= 2024-04-16T17:56:10.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:74pqbSudrXS/99jHkS1FBAs/CaM=', 'Content-Length': '0'} 2024-04-16T17:56:10.492 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001a2a4c1cb52d33df-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.492 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.493 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.493 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.493 DEBUG:boto:Path: /lybtug-13/obj-test-zone2 2024-04-16T17:56:10.493 DEBUG:boto:Data: 2024-04-16T17:56:10.493 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.493 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.493 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.493 DEBUG:boto:Params: {} 2024-04-16T17:56:10.493 DEBUG:boto:Token: None 2024-04-16T17:56:10.493 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2 2024-04-16T17:56:10.494 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:74pqbSudrXS/99jHkS1FBAs/CaM= 2024-04-16T17:56:10.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:74pqbSudrXS/99jHkS1FBAs/CaM=', 'Content-Length': '0'} 2024-04-16T17:56:10.498 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001e5c7679175acc6a-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.498 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:56:10.499 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.499 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.499 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.499 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.499 DEBUG:boto:Method: GET 2024-04-16T17:56:10.499 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.499 DEBUG:boto:Data: 2024-04-16T17:56:10.499 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.499 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.499 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.499 DEBUG:boto:Params: {} 2024-04-16T17:56:10.500 DEBUG:boto:Token: None 2024-04-16T17:56:10.500 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo= 2024-04-16T17:56:10.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo=', 'Content-Length': '0'} 2024-04-16T17:56:10.503 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fa43a53c751f8bd4-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.503 DEBUG:boto:path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.503 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2 2024-04-16T17:56:10.504 DEBUG:boto:path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.504 DEBUG:boto:auth_path=/lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.504 DEBUG:boto:Method: GET 2024-04-16T17:56:10.504 DEBUG:boto:Path: /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.504 DEBUG:boto:Data: 2024-04-16T17:56:10.504 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.504 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.504 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.504 DEBUG:boto:Params: {} 2024-04-16T17:56:10.504 DEBUG:boto:Token: None 2024-04-16T17:56:10.505 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/obj-test-zone2?versionId=BddSazy2TorOsY9OlUrEenOITSq5no0 2024-04-16T17:56:10.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo= 2024-04-16T17:56:10.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Twmv0RhT6aR1PuJKVnrVMRdezmo=', 'Content-Length': '0'} 2024-04-16T17:56:10.508 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:51 GMT'), ('x-amz-version-id', 'BddSazy2TorOsY9OlUrEenOITSq5no0'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c43e9895422fb3da-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.550 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.551 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.551 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.551 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.551 DEBUG:boto:Method: GET 2024-04-16T17:56:10.551 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.551 DEBUG:boto:Data: 2024-04-16T17:56:10.551 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.551 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.551 DEBUG:boto:Params: {} 2024-04-16T17:56:10.551 DEBUG:boto:Token: None 2024-04-16T17:56:10.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f495c6fa582180ed-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.556 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.556 DEBUG:boto:path=/lybtug-13/ 2024-04-16T17:56:10.557 DEBUG:boto:auth_path=/lybtug-13/ 2024-04-16T17:56:10.557 DEBUG:boto:path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.557 DEBUG:boto:auth_path=/lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.557 DEBUG:boto:Method: GET 2024-04-16T17:56:10.557 DEBUG:boto:Path: /lybtug-13/?prefix=obj-test-zone2 2024-04-16T17:56:10.557 DEBUG:boto:Data: 2024-04-16T17:56:10.557 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.557 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.557 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.557 DEBUG:boto:Params: {} 2024-04-16T17:56:10.557 DEBUG:boto:Token: None 2024-04-16T17:56:10.558 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-13/ 2024-04-16T17:56:10.558 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY= 2024-04-16T17:56:10.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEhGcPJ9Ec1KNzyNjCB4ZJV7gTY=', 'Content-Length': '0'} 2024-04-16T17:56:10.562 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069fda2156d438fad-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.602 DEBUG:boto:b'lybtug-13obj-test-zone21000falseobj-test-zone22024-04-16T17:54:51.931Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.603 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-13 zones={test-zone2, test-zone1} 2024-04-16T17:56:10.603 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-14 2024-04-16T17:56:10.603 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:10.603 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:10.603 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-14 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:56:10.653 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:10.654 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:10.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:10.729+0000 7f00c9725980 1 bucket sync caught up with source: 2024-04-16T17:56:10.730 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000011.36.6, , 00000000002.4.11, , 00000000002.11.11, , , , , ] 2024-04-16T17:56:10.730 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000011.36.6, , 00000000002.4.11, , 00000000002.11.11, , , , , ] 2024-04-16T17:56:10.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:10.729+0000 7f00c9725980 0 bucket checkpoint complete 2024-04-16T17:56:10.761 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-14 2024-04-16T17:56:10.761 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:10.762 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:10.762 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:56:10.813 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:10.813 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:10.895+0000 7f6c9c76a980 1 bucket sync caught up with source: 2024-04-16T17:56:10.892 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.50.11, , 00000000011.28.6, , 00000000003.24.10, , , , , ] 2024-04-16T17:56:10.892 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.50.11, , 00000000011.28.6, , 00000000003.24.10, , , , , ] 2024-04-16T17:56:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:10.895+0000 7f6c9c76a980 0 bucket checkpoint complete 2024-04-16T17:56:10.922 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-14 zones={test-zone2, test-zone1} 2024-04-16T17:56:10.922 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:10.923 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:10.923 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.923 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:56:10.923 DEBUG:boto:Data: 2024-04-16T17:56:10.923 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.923 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.923 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.923 DEBUG:boto:Params: {} 2024-04-16T17:56:10.923 DEBUG:boto:Token: None 2024-04-16T17:56:10.923 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-14/ 2024-04-16T17:56:10.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CLhMLgHnuoGYA6t8HqdNjSLw9t8= 2024-04-16T17:56:10.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CLhMLgHnuoGYA6t8HqdNjSLw9t8=', 'Content-Length': '0'} 2024-04-16T17:56:10.928 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000544e4395eb36a86e-00661ebbba-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.928 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:10.928 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:10.928 DEBUG:boto:Method: HEAD 2024-04-16T17:56:10.928 DEBUG:boto:Path: /lybtug-14/ 2024-04-16T17:56:10.928 DEBUG:boto:Data: 2024-04-16T17:56:10.928 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.928 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.928 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.928 DEBUG:boto:Params: {} 2024-04-16T17:56:10.928 DEBUG:boto:Token: None 2024-04-16T17:56:10.929 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:10 GMT /lybtug-14/ 2024-04-16T17:56:10.929 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CLhMLgHnuoGYA6t8HqdNjSLw9t8= 2024-04-16T17:56:10.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CLhMLgHnuoGYA6t8HqdNjSLw9t8=', 'Content-Length': '0'} 2024-04-16T17:56:10.931 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000010f3afad9956e3b0-00661ebbba-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.932 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:56:10.932 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:10.932 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:10.932 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.932 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.932 DEBUG:boto:Method: GET 2024-04-16T17:56:10.932 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.932 DEBUG:boto:Data: 2024-04-16T17:56:10.932 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.932 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.933 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.933 DEBUG:boto:Params: {} 2024-04-16T17:56:10.933 DEBUG:boto:Token: None 2024-04-16T17:56:10.933 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-14/?versions 2024-04-16T17:56:10.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo= 2024-04-16T17:56:10.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo=', 'Content-Length': '0'} 2024-04-16T17:56:10.937 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bddb5abc20d97fec-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.937 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.937 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:10.938 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:56:10.938 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:56:10.938 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:56:10.938 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:10.938 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:10.938 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.938 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.938 DEBUG:boto:Method: GET 2024-04-16T17:56:10.938 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.938 DEBUG:boto:Data: 2024-04-16T17:56:10.938 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.939 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.939 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.939 DEBUG:boto:Params: {} 2024-04-16T17:56:10.939 DEBUG:boto:Token: None 2024-04-16T17:56:10.939 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-14/?versions 2024-04-16T17:56:10.939 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo= 2024-04-16T17:56:10.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo=', 'Content-Length': '0'} 2024-04-16T17:56:10.942 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b240284e5b378fe-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.942 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.945 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:10.946 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2024-04-16T17:56:10.946 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2024-04-16T17:56:10.946 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:10.946 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:10.946 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.946 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.946 DEBUG:boto:Method: GET 2024-04-16T17:56:10.946 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.946 DEBUG:boto:Data: 2024-04-16T17:56:10.947 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.947 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:10.947 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.947 DEBUG:boto:Params: {} 2024-04-16T17:56:10.947 DEBUG:boto:Token: None 2024-04-16T17:56:10.947 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-14/?versions 2024-04-16T17:56:10.947 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo= 2024-04-16T17:56:10.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo=', 'Content-Length': '0'} 2024-04-16T17:56:10.951 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c600db73ac9f0d5-00661ebbba-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:10.994 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:10.995 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:10.995 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:10.995 DEBUG:boto:path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.995 DEBUG:boto:auth_path=/lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.996 DEBUG:boto:Method: GET 2024-04-16T17:56:10.996 DEBUG:boto:Path: /lybtug-14/?versions&max-keys=999 2024-04-16T17:56:10.996 DEBUG:boto:Data: 2024-04-16T17:56:10.996 DEBUG:boto:Headers: {} 2024-04-16T17:56:10.996 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:10.996 DEBUG:boto:Port: 8000 2024-04-16T17:56:10.996 DEBUG:boto:Params: {} 2024-04-16T17:56:10.996 DEBUG:boto:Token: None 2024-04-16T17:56:10.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:10 GMT /lybtug-14/?versions 2024-04-16T17:56:10.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo= 2024-04-16T17:56:10.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RR0cgKEbXlauxxUbzGk5CmiMIDo=', 'Content-Length': '0'} 2024-04-16T17:56:11.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008828af86ad96e795-00661ebbba-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.000 DEBUG:boto:b'lybtug-14999falsedummyZvDr1IgZL0JBvk3G839G9Gh84bK3lKAtrue2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj-test-zone1SzsCMP20u88Jh3p691Bm8VasxASFjRctrue2024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj-test-zone2EAA9o3GMykOCygP5VWluriq4wdFzh4otrue2024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.001 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:11.001 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.001 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.001 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.001 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.001 DEBUG:boto:Method: GET 2024-04-16T17:56:11.001 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.001 DEBUG:boto:Data: 2024-04-16T17:56:11.002 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.002 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.002 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.002 DEBUG:boto:Params: {} 2024-04-16T17:56:11.002 DEBUG:boto:Token: None 2024-04-16T17:56:11.002 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.002 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec= 2024-04-16T17:56:11.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec=', 'Content-Length': '0'} 2024-04-16T17:56:11.005 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009dbc9a099b5a123b-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.005 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.006 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.006 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.006 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.006 DEBUG:boto:Method: GET 2024-04-16T17:56:11.006 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.007 DEBUG:boto:Data: 2024-04-16T17:56:11.007 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.007 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.007 DEBUG:boto:Params: {} 2024-04-16T17:56:11.007 DEBUG:boto:Token: None 2024-04-16T17:56:11.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec= 2024-04-16T17:56:11.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec=', 'Content-Length': '0'} 2024-04-16T17:56:11.010 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000971a7c5e3b51c92-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.011 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.011 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.011 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.011 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.012 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.012 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.012 DEBUG:boto:Data: 2024-04-16T17:56:11.012 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.012 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.013 DEBUG:boto:Params: {} 2024-04-16T17:56:11.013 DEBUG:boto:Token: None 2024-04-16T17:56:11.013 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1hcEGwIylKg2eKRuusTyhYZmmRg= 2024-04-16T17:56:11.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1hcEGwIylKg2eKRuusTyhYZmmRg=', 'Content-Length': '0'} 2024-04-16T17:56:11.016 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002dfe26dfd95c8f39-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.016 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.017 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.017 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.017 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.017 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.017 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.017 DEBUG:boto:Data: 2024-04-16T17:56:11.017 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.017 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.017 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.017 DEBUG:boto:Params: {} 2024-04-16T17:56:11.017 DEBUG:boto:Token: None 2024-04-16T17:56:11.018 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.018 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1hcEGwIylKg2eKRuusTyhYZmmRg= 2024-04-16T17:56:11.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1hcEGwIylKg2eKRuusTyhYZmmRg=', 'Content-Length': '0'} 2024-04-16T17:56:11.020 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e09b97028b43a6bf-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.021 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:11.021 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.021 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.021 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.021 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.021 DEBUG:boto:Method: GET 2024-04-16T17:56:11.021 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.022 DEBUG:boto:Data: 2024-04-16T17:56:11.022 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.022 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.022 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.022 DEBUG:boto:Params: {} 2024-04-16T17:56:11.022 DEBUG:boto:Token: None 2024-04-16T17:56:11.022 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.022 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec= 2024-04-16T17:56:11.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec=', 'Content-Length': '0'} 2024-04-16T17:56:11.025 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000aff2e556a794502b-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.025 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.025 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.026 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.026 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.026 DEBUG:boto:Method: GET 2024-04-16T17:56:11.026 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.026 DEBUG:boto:Data: 2024-04-16T17:56:11.027 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.027 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.027 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.027 DEBUG:boto:Params: {} 2024-04-16T17:56:11.027 DEBUG:boto:Token: None 2024-04-16T17:56:11.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec= 2024-04-16T17:56:11.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec=', 'Content-Length': '0'} 2024-04-16T17:56:11.031 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006d966713243a3d24-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.032 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.032 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.032 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.032 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.032 DEBUG:boto:Method: GET 2024-04-16T17:56:11.033 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:56:11.033 DEBUG:boto:Data: 2024-04-16T17:56:11.033 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.033 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.033 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.034 DEBUG:boto:Params: {} 2024-04-16T17:56:11.034 DEBUG:boto:Token: None 2024-04-16T17:56:11.034 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.034 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.039 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000102725e733b67bfd-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.039 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.040 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.040 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.040 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.041 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.041 DEBUG:boto:Method: GET 2024-04-16T17:56:11.041 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:56:11.041 DEBUG:boto:Data: 2024-04-16T17:56:11.041 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.041 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.041 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.042 DEBUG:boto:Params: {} 2024-04-16T17:56:11.042 DEBUG:boto:Token: None 2024-04-16T17:56:11.042 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad52784ecd69f2cf-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.090 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.091 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.092 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.092 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.092 DEBUG:boto:Path: /lybtug-14/dummy 2024-04-16T17:56:11.092 DEBUG:boto:Data: 2024-04-16T17:56:11.092 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.093 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.093 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.093 DEBUG:boto:Params: {} 2024-04-16T17:56:11.093 DEBUG:boto:Token: None 2024-04-16T17:56:11.094 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy 2024-04-16T17:56:11.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3DHECGsVYCyBWDIlPy7jfdDX0xY= 2024-04-16T17:56:11.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3DHECGsVYCyBWDIlPy7jfdDX0xY=', 'Content-Length': '0'} 2024-04-16T17:56:11.098 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f9b20fa14b7bfb71-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.099 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.099 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.099 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.099 DEBUG:boto:Path: /lybtug-14/dummy 2024-04-16T17:56:11.100 DEBUG:boto:Data: 2024-04-16T17:56:11.100 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.100 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.100 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.100 DEBUG:boto:Params: {} 2024-04-16T17:56:11.101 DEBUG:boto:Token: None 2024-04-16T17:56:11.101 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy 2024-04-16T17:56:11.101 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3DHECGsVYCyBWDIlPy7jfdDX0xY= 2024-04-16T17:56:11.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3DHECGsVYCyBWDIlPy7jfdDX0xY=', 'Content-Length': '0'} 2024-04-16T17:56:11.106 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d48572ccb85580bd-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.106 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:11.106 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.106 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.107 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.107 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.107 DEBUG:boto:Method: GET 2024-04-16T17:56:11.107 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.107 DEBUG:boto:Data: 2024-04-16T17:56:11.107 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.107 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.108 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.108 DEBUG:boto:Params: {} 2024-04-16T17:56:11.108 DEBUG:boto:Token: None 2024-04-16T17:56:11.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec= 2024-04-16T17:56:11.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec=', 'Content-Length': '0'} 2024-04-16T17:56:11.111 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000938629604bf6ba42-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.111 DEBUG:boto:path=/lybtug-14/dummy 2024-04-16T17:56:11.112 DEBUG:boto:auth_path=/lybtug-14/dummy 2024-04-16T17:56:11.112 DEBUG:boto:path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.112 DEBUG:boto:auth_path=/lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.112 DEBUG:boto:Method: GET 2024-04-16T17:56:11.112 DEBUG:boto:Path: /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.112 DEBUG:boto:Data: 2024-04-16T17:56:11.112 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.112 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.112 DEBUG:boto:Params: {} 2024-04-16T17:56:11.112 DEBUG:boto:Token: None 2024-04-16T17:56:11.113 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/dummy?versionId=ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA 2024-04-16T17:56:11.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec= 2024-04-16T17:56:11.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t7oto9AZLqAMf5U7wvoZ26+r+Ec=', 'Content-Length': '0'} 2024-04-16T17:56:11.116 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:46 GMT'), ('x-amz-version-id', 'ZvDr1IgZL0JBvk3G839G9Gh84bK3lKA'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:50 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002af66fea868208a6-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.116 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.116 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.116 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.116 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.116 DEBUG:boto:Method: GET 2024-04-16T17:56:11.116 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:56:11.116 DEBUG:boto:Data: 2024-04-16T17:56:11.116 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.116 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.117 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.117 DEBUG:boto:Params: {} 2024-04-16T17:56:11.117 DEBUG:boto:Token: None 2024-04-16T17:56:11.117 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.117 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.121 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000950171a64f957203-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.121 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.121 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.122 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.122 DEBUG:boto:path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.122 DEBUG:boto:auth_path=/lybtug-14/?prefix=dummy 2024-04-16T17:56:11.122 DEBUG:boto:Method: GET 2024-04-16T17:56:11.122 DEBUG:boto:Path: /lybtug-14/?prefix=dummy 2024-04-16T17:56:11.122 DEBUG:boto:Data: 2024-04-16T17:56:11.122 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.122 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.122 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.122 DEBUG:boto:Params: {} 2024-04-16T17:56:11.123 DEBUG:boto:Token: None 2024-04-16T17:56:11.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.123 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018719ac5b30a69f5-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.166 DEBUG:boto:b'lybtug-14dummy1000falsedummy2024-04-16T17:54:46.570Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.167 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:56:11.167 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.167 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.167 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.167 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.167 DEBUG:boto:Method: GET 2024-04-16T17:56:11.167 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.167 DEBUG:boto:Data: 2024-04-16T17:56:11.168 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.168 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.168 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.168 DEBUG:boto:Params: {} 2024-04-16T17:56:11.168 DEBUG:boto:Token: None 2024-04-16T17:56:11.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY= 2024-04-16T17:56:11.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY=', 'Content-Length': '0'} 2024-04-16T17:56:11.171 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000073ff6de6bd14b900-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.171 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.171 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.172 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.172 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.172 DEBUG:boto:Method: GET 2024-04-16T17:56:11.172 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.172 DEBUG:boto:Data: 2024-04-16T17:56:11.172 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.172 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.172 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.172 DEBUG:boto:Params: {} 2024-04-16T17:56:11.172 DEBUG:boto:Token: None 2024-04-16T17:56:11.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.173 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY= 2024-04-16T17:56:11.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY=', 'Content-Length': '0'} 2024-04-16T17:56:11.176 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001735c6843a218942-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.176 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.176 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.176 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.176 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.176 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.176 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.176 DEBUG:boto:Data: 2024-04-16T17:56:11.177 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.177 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.177 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.177 DEBUG:boto:Params: {} 2024-04-16T17:56:11.177 DEBUG:boto:Token: None 2024-04-16T17:56:11.177 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eiBHcGfT0UHyYnMBBA/kQsqUpgQ= 2024-04-16T17:56:11.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eiBHcGfT0UHyYnMBBA/kQsqUpgQ=', 'Content-Length': '0'} 2024-04-16T17:56:11.180 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000029baea5335f0e317-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.180 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.180 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.180 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.180 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.180 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.180 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.180 DEBUG:boto:Data: 2024-04-16T17:56:11.180 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.181 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.181 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.181 DEBUG:boto:Params: {} 2024-04-16T17:56:11.181 DEBUG:boto:Token: None 2024-04-16T17:56:11.181 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.181 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eiBHcGfT0UHyYnMBBA/kQsqUpgQ= 2024-04-16T17:56:11.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eiBHcGfT0UHyYnMBBA/kQsqUpgQ=', 'Content-Length': '0'} 2024-04-16T17:56:11.184 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000065d3b94f4708ae86-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.184 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:56:11.184 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.184 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.184 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.184 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.184 DEBUG:boto:Method: GET 2024-04-16T17:56:11.185 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.185 DEBUG:boto:Data: 2024-04-16T17:56:11.185 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.185 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.185 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.185 DEBUG:boto:Params: {} 2024-04-16T17:56:11.185 DEBUG:boto:Token: None 2024-04-16T17:56:11.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY= 2024-04-16T17:56:11.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY=', 'Content-Length': '0'} 2024-04-16T17:56:11.188 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000868dc82a726bf158-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.230 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.230 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.230 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.231 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.231 DEBUG:boto:Method: GET 2024-04-16T17:56:11.231 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.231 DEBUG:boto:Data: 2024-04-16T17:56:11.231 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.231 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.231 DEBUG:boto:Params: {} 2024-04-16T17:56:11.231 DEBUG:boto:Token: None 2024-04-16T17:56:11.231 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.231 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY= 2024-04-16T17:56:11.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY=', 'Content-Length': '0'} 2024-04-16T17:56:11.235 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a6e9dd17ebc924a-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.235 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.235 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.235 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.236 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.236 DEBUG:boto:Method: GET 2024-04-16T17:56:11.236 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.236 DEBUG:boto:Data: 2024-04-16T17:56:11.236 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.236 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.236 DEBUG:boto:Params: {} 2024-04-16T17:56:11.236 DEBUG:boto:Token: None 2024-04-16T17:56:11.236 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ccc41fd90d00a240-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.240 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.241 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.241 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.241 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.241 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.241 DEBUG:boto:Method: GET 2024-04-16T17:56:11.241 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.241 DEBUG:boto:Data: 2024-04-16T17:56:11.241 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.241 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.242 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.242 DEBUG:boto:Params: {} 2024-04-16T17:56:11.242 DEBUG:boto:Token: None 2024-04-16T17:56:11.242 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.242 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.245 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd9026538f4adfea-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.286 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.286 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.287 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.287 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.287 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:56:11.287 DEBUG:boto:Data: 2024-04-16T17:56:11.287 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.287 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.287 DEBUG:boto:Params: {} 2024-04-16T17:56:11.287 DEBUG:boto:Token: None 2024-04-16T17:56:11.287 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:56:11.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KgP3CqXzHLD2yTVy/ruK4k2Z/og= 2024-04-16T17:56:11.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KgP3CqXzHLD2yTVy/ruK4k2Z/og=', 'Content-Length': '0'} 2024-04-16T17:56:11.292 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000845132f1cc8ca192-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.292 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.292 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.292 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.292 DEBUG:boto:Path: /lybtug-14/obj-test-zone1 2024-04-16T17:56:11.293 DEBUG:boto:Data: 2024-04-16T17:56:11.293 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.293 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.293 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.293 DEBUG:boto:Params: {} 2024-04-16T17:56:11.293 DEBUG:boto:Token: None 2024-04-16T17:56:11.293 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1 2024-04-16T17:56:11.293 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KgP3CqXzHLD2yTVy/ruK4k2Z/og= 2024-04-16T17:56:11.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KgP3CqXzHLD2yTVy/ruK4k2Z/og=', 'Content-Length': '0'} 2024-04-16T17:56:11.297 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c415322e06d164ea-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.297 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2024-04-16T17:56:11.297 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.298 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.298 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.298 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.298 DEBUG:boto:Method: GET 2024-04-16T17:56:11.298 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.298 DEBUG:boto:Data: 2024-04-16T17:56:11.298 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.298 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.298 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.298 DEBUG:boto:Params: {} 2024-04-16T17:56:11.299 DEBUG:boto:Token: None 2024-04-16T17:56:11.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY= 2024-04-16T17:56:11.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY=', 'Content-Length': '0'} 2024-04-16T17:56:11.302 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:53 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000042c93d4b488a63c7-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.342 DEBUG:boto:path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.343 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1 2024-04-16T17:56:11.343 DEBUG:boto:path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.343 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.343 DEBUG:boto:Method: GET 2024-04-16T17:56:11.343 DEBUG:boto:Path: /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.343 DEBUG:boto:Data: 2024-04-16T17:56:11.343 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.343 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.343 DEBUG:boto:Params: {} 2024-04-16T17:56:11.344 DEBUG:boto:Token: None 2024-04-16T17:56:11.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone1?versionId=SzsCMP20u88Jh3p691Bm8VasxASFjRc 2024-04-16T17:56:11.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY= 2024-04-16T17:56:11.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mU7lwgNhNw77966naNGQYNgP7jY=', 'Content-Length': '0'} 2024-04-16T17:56:11.347 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'SzsCMP20u88Jh3p691Bm8VasxASFjRc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000766643978e27d1fb-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.347 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.347 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.347 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.347 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.348 DEBUG:boto:Method: GET 2024-04-16T17:56:11.348 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.348 DEBUG:boto:Data: 2024-04-16T17:56:11.348 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.348 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.348 DEBUG:boto:Params: {} 2024-04-16T17:56:11.348 DEBUG:boto:Token: None 2024-04-16T17:56:11.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.353 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000afb1e7d6531288e9-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.394 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.395 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.395 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.395 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.395 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.395 DEBUG:boto:Method: GET 2024-04-16T17:56:11.395 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone1 2024-04-16T17:56:11.395 DEBUG:boto:Data: 2024-04-16T17:56:11.395 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.395 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.396 DEBUG:boto:Params: {} 2024-04-16T17:56:11.396 DEBUG:boto:Token: None 2024-04-16T17:56:11.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.399 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a91cbc4216a6ebbb-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.399 DEBUG:boto:b'lybtug-14obj-test-zone11000falseobj-test-zone12024-04-16T17:54:52.079Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.400 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:56:11.400 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.400 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.400 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.400 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.400 DEBUG:boto:Method: GET 2024-04-16T17:56:11.400 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.400 DEBUG:boto:Data: 2024-04-16T17:56:11.401 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.401 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.401 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.401 DEBUG:boto:Params: {} 2024-04-16T17:56:11.401 DEBUG:boto:Token: None 2024-04-16T17:56:11.401 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.401 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM= 2024-04-16T17:56:11.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM=', 'Content-Length': '0'} 2024-04-16T17:56:11.404 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dfc8919bd2806120-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.404 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.405 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.405 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.405 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.405 DEBUG:boto:Method: GET 2024-04-16T17:56:11.405 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.405 DEBUG:boto:Data: 2024-04-16T17:56:11.405 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.405 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.405 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.405 DEBUG:boto:Params: {} 2024-04-16T17:56:11.405 DEBUG:boto:Token: None 2024-04-16T17:56:11.406 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.406 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM= 2024-04-16T17:56:11.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM=', 'Content-Length': '0'} 2024-04-16T17:56:11.409 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000090a17f2472c0cdf-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.450 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.450 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.451 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.451 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.451 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.451 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.451 DEBUG:boto:Data: 2024-04-16T17:56:11.451 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.451 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.451 DEBUG:boto:Params: {} 2024-04-16T17:56:11.451 DEBUG:boto:Token: None 2024-04-16T17:56:11.451 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gp6XMLZ4mtUOcsQb3/0wdnU/y28= 2024-04-16T17:56:11.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gp6XMLZ4mtUOcsQb3/0wdnU/y28=', 'Content-Length': '0'} 2024-04-16T17:56:11.454 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000620bcd44e175101c-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.455 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.455 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.455 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.455 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.455 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.455 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.455 DEBUG:boto:Data: 2024-04-16T17:56:11.455 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.456 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.456 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.456 DEBUG:boto:Params: {} 2024-04-16T17:56:11.456 DEBUG:boto:Token: None 2024-04-16T17:56:11.456 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gp6XMLZ4mtUOcsQb3/0wdnU/y28= 2024-04-16T17:56:11.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gp6XMLZ4mtUOcsQb3/0wdnU/y28=', 'Content-Length': '0'} 2024-04-16T17:56:11.459 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005581aa6591ae9024-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:56:11.459 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.459 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.460 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.460 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.460 DEBUG:boto:Method: GET 2024-04-16T17:56:11.460 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.460 DEBUG:boto:Data: 2024-04-16T17:56:11.460 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.460 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.460 DEBUG:boto:Params: {} 2024-04-16T17:56:11.460 DEBUG:boto:Token: None 2024-04-16T17:56:11.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM= 2024-04-16T17:56:11.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM=', 'Content-Length': '0'} 2024-04-16T17:56:11.464 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002e1bd29b90d1260a-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.464 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.464 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.464 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.464 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.464 DEBUG:boto:Method: GET 2024-04-16T17:56:11.464 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.464 DEBUG:boto:Data: 2024-04-16T17:56:11.465 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.465 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.465 DEBUG:boto:Params: {} 2024-04-16T17:56:11.465 DEBUG:boto:Token: None 2024-04-16T17:56:11.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM= 2024-04-16T17:56:11.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM=', 'Content-Length': '0'} 2024-04-16T17:56:11.468 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ba85a87f4fdebfde-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.510 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.510 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.511 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.511 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.511 DEBUG:boto:Method: GET 2024-04-16T17:56:11.511 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.511 DEBUG:boto:Data: 2024-04-16T17:56:11.511 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.511 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.511 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.511 DEBUG:boto:Params: {} 2024-04-16T17:56:11.511 DEBUG:boto:Token: None 2024-04-16T17:56:11.512 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a808e45b1f7b18a-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.516 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.516 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.516 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.516 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.516 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.517 DEBUG:boto:Method: GET 2024-04-16T17:56:11.517 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.517 DEBUG:boto:Data: 2024-04-16T17:56:11.517 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.517 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.517 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.517 DEBUG:boto:Params: {} 2024-04-16T17:56:11.517 DEBUG:boto:Token: None 2024-04-16T17:56:11.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.521 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eabf302130e88f76-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.521 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.521 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.521 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.522 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.522 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:56:11.522 DEBUG:boto:Data: 2024-04-16T17:56:11.522 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.522 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.522 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.522 DEBUG:boto:Params: {} 2024-04-16T17:56:11.522 DEBUG:boto:Token: None 2024-04-16T17:56:11.522 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:56:11.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajv4x2PJKUOa96C8wWHP0Sd4HKI= 2024-04-16T17:56:11.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajv4x2PJKUOa96C8wWHP0Sd4HKI=', 'Content-Length': '0'} 2024-04-16T17:56:11.527 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002f46555427ff5a11-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.527 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.527 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.527 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.527 DEBUG:boto:Path: /lybtug-14/obj-test-zone2 2024-04-16T17:56:11.527 DEBUG:boto:Data: 2024-04-16T17:56:11.527 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.527 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.528 DEBUG:boto:Params: {} 2024-04-16T17:56:11.528 DEBUG:boto:Token: None 2024-04-16T17:56:11.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2 2024-04-16T17:56:11.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajv4x2PJKUOa96C8wWHP0Sd4HKI= 2024-04-16T17:56:11.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajv4x2PJKUOa96C8wWHP0Sd4HKI=', 'Content-Length': '0'} 2024-04-16T17:56:11.532 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b366e958052a40a5-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.532 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2024-04-16T17:56:11.533 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.533 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.533 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.533 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.533 DEBUG:boto:Method: GET 2024-04-16T17:56:11.533 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.533 DEBUG:boto:Data: 2024-04-16T17:56:11.533 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.533 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.533 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.533 DEBUG:boto:Params: {} 2024-04-16T17:56:11.534 DEBUG:boto:Token: None 2024-04-16T17:56:11.534 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM= 2024-04-16T17:56:11.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM=', 'Content-Length': '0'} 2024-04-16T17:56:11.537 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aa4f7e3368919145-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.578 DEBUG:boto:path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.579 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2 2024-04-16T17:56:11.579 DEBUG:boto:path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.579 DEBUG:boto:auth_path=/lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.579 DEBUG:boto:Method: GET 2024-04-16T17:56:11.579 DEBUG:boto:Path: /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.579 DEBUG:boto:Data: 2024-04-16T17:56:11.579 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.579 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.579 DEBUG:boto:Params: {} 2024-04-16T17:56:11.580 DEBUG:boto:Token: None 2024-04-16T17:56:11.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/obj-test-zone2?versionId=EAA9o3GMykOCygP5VWluriq4wdFzh4o 2024-04-16T17:56:11.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM= 2024-04-16T17:56:11.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgvJEDSX0gU5Z7Md/ACmiRrXXAM=', 'Content-Length': '0'} 2024-04-16T17:56:11.583 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:54:52 GMT'), ('x-amz-version-id', 'EAA9o3GMykOCygP5VWluriq4wdFzh4o'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:54:54 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b23749fdcedb216a-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.626 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.627 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.627 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.627 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.627 DEBUG:boto:Method: GET 2024-04-16T17:56:11.627 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.627 DEBUG:boto:Data: 2024-04-16T17:56:11.627 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.627 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:11.627 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.627 DEBUG:boto:Params: {} 2024-04-16T17:56:11.628 DEBUG:boto:Token: None 2024-04-16T17:56:11.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008445187897810f8c-00661ebbbb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.632 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.633 DEBUG:boto:path=/lybtug-14/ 2024-04-16T17:56:11.633 DEBUG:boto:auth_path=/lybtug-14/ 2024-04-16T17:56:11.633 DEBUG:boto:path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.633 DEBUG:boto:auth_path=/lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.633 DEBUG:boto:Method: GET 2024-04-16T17:56:11.633 DEBUG:boto:Path: /lybtug-14/?prefix=obj-test-zone2 2024-04-16T17:56:11.633 DEBUG:boto:Data: 2024-04-16T17:56:11.634 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.634 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.634 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.634 DEBUG:boto:Params: {} 2024-04-16T17:56:11.634 DEBUG:boto:Token: None 2024-04-16T17:56:11.634 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:11 GMT /lybtug-14/ 2024-04-16T17:56:11.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo= 2024-04-16T17:56:11.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pgoY0lKvdTMyiQZBPsKnMil+Peo=', 'Content-Length': '0'} 2024-04-16T17:56:11.637 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b489080f4ebf095a-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.638 DEBUG:boto:b'lybtug-14obj-test-zone21000falseobj-test-zone22024-04-16T17:54:52.239Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:11.638 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-14 zones={test-zone2, test-zone1} 2024-04-16T17:56:11.638 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_versioned_object_incremental_sync ... ok 2024-04-16T17:56:11.640 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:11.640 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:11.640 DEBUG:boto:Method: PUT 2024-04-16T17:56:11.640 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:11.640 DEBUG:boto:Data: 2024-04-16T17:56:11.640 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.640 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.640 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.641 DEBUG:boto:Params: {} 2024-04-16T17:56:11.641 DEBUG:boto:Token: None 2024-04-16T17:56:11.641 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:56:11 GMT /lybtug-15/ 2024-04-16T17:56:11.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lK/82qeFrJMptxvJpwqa5MfEVqo= 2024-04-16T17:56:11.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lK/82qeFrJMptxvJpwqa5MfEVqo=', 'Content-Length': '0'} 2024-04-16T17:56:11.676 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000dad8a9a9a1767a1c-00661ebbbb-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.677 DEBUG:rgw_multi.tests:created bucket=lybtug-15 2024-04-16T17:56:11.677 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:11.677 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:11.677 DEBUG:boto:path=/lybtug-15/?versioning 2024-04-16T17:56:11.677 DEBUG:boto:auth_path=/lybtug-15/?versioning 2024-04-16T17:56:11.677 DEBUG:boto:Method: PUT 2024-04-16T17:56:11.677 DEBUG:boto:Path: /lybtug-15/?versioning 2024-04-16T17:56:11.677 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:56:11.677 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.677 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.678 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.678 DEBUG:boto:Params: {} 2024-04-16T17:56:11.678 DEBUG:boto:Token: None 2024-04-16T17:56:11.678 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:56:11 GMT /lybtug-15/?versioning 2024-04-16T17:56:11.678 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bPBn6Tl+HBZww9tG9MCHpY1whUU= 2024-04-16T17:56:11.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bPBn6Tl+HBZww9tG9MCHpY1whUU=', 'Content-Length': '228'} 2024-04-16T17:56:11.691 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000005b9f1a5ce67d1dc-00661ebbbb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.691 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:11.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:11.691 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:56:11.735 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:11.735 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:11.802 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:56:11.833 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290171.693277_128.1","last_update":"2024-04-16T17:56:11.693277Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:56:11.833 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290171.693277_128.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:56:11.833 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:56:11.834 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:11.834 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:11.834 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:56:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:11.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:11.941+0000 7f281f731980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:56:11.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:11.945+0000 7f281f731980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:56:11.975 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290171.693277_128.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:56:11.693277Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:56:11Z"} 2024-04-16T17:56:11.975 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290171.693277_128.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:56:11.693277Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:56:11Z"} 2024-04-16T17:56:11.975 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290171.693277_128.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:56:11.693277Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:56:11.976 DEBUG:rgw_multi.tests:log_status={0: '1_1713290171.693277_128.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:56:11.976 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290171.693277_128.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:56:11.976 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:56:11.976 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:11.976 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:11.976 DEBUG:boto:Method: HEAD 2024-04-16T17:56:11.976 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:11.976 DEBUG:boto:Data: 2024-04-16T17:56:11.976 DEBUG:boto:Headers: {} 2024-04-16T17:56:11.976 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.977 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.977 DEBUG:boto:Params: {} 2024-04-16T17:56:11.977 DEBUG:boto:Token: None 2024-04-16T17:56:11.977 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:11 GMT /lybtug-15/ 2024-04-16T17:56:11.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lO/3JCG0QjmVVaDEaHnkOM8k5JE= 2024-04-16T17:56:11.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lO/3JCG0QjmVVaDEaHnkOM8k5JE=', 'Content-Length': '0'} 2024-04-16T17:56:11.980 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aaab4301f8ce20d4-00661ebbbb-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:11.981 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:11.981 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:11.981 DEBUG:boto:Method: PUT 2024-04-16T17:56:11.981 DEBUG:boto:Path: /lybtug-15/dummy 2024-04-16T17:56:11.981 DEBUG:boto:Data: 2024-04-16T17:56:11.981 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:56:11.981 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:11.981 DEBUG:boto:Port: 8000 2024-04-16T17:56:11.981 DEBUG:boto:Params: {} 2024-04-16T17:56:11.982 DEBUG:boto:Token: None 2024-04-16T17:56:11.982 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:56:11 GMT /lybtug-15/dummy 2024-04-16T17:56:11.982 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7yWRdMSyVPybQxgvZZBteI4lQFg= 2024-04-16T17:56:11.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7yWRdMSyVPybQxgvZZBteI4lQFg='} 2024-04-16T17:56:12.027 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-amz-request-id', 'tx00000042e4e3f676d2125-00661ebbbb-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:12.027 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-15 2024-04-16T17:56:12.027 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:12.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:12.027 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-15 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:56:12.073 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:12.073 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:12.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:12.153+0000 7fec6cd8d980 1 waiting to reach incremental sync.. 2024-04-16T17:56:17.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:17.157+0000 7fec6cd8d980 1 bucket sync caught up with source: 2024-04-16T17:56:17.156 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000002.14.11, , , , , ] 2024-04-16T17:56:17.156 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000002.14.11, , , , , ] 2024-04-16T17:56:17.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:17.157+0000 7fec6cd8d980 0 bucket checkpoint complete 2024-04-16T17:56:17.201 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-15 2024-04-16T17:56:17.201 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:17.201 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:17.201 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-15 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:56:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:17.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:17.335+0000 7f8aeb515980 1 waiting to reach incremental sync.. 2024-04-16T17:56:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:22.335+0000 7f8aeb515980 1 waiting to reach incremental sync.. 2024-04-16T17:56:27.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:27.335+0000 7f8aeb515980 1 waiting to reach incremental sync.. 2024-04-16T17:56:32.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:32.339+0000 7f8aeb515980 1 bucket sync caught up with source: 2024-04-16T17:56:32.337 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , 00000000002.19.11, , , , , ] 2024-04-16T17:56:32.337 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , 00000000002.19.11, , , , , ] 2024-04-16T17:56:32.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:32.339+0000 7f8aeb515980 0 bucket checkpoint complete 2024-04-16T17:56:32.387 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-15 zones={test-zone2, test-zone1} 2024-04-16T17:56:32.387 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.387 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.387 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.387 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.388 DEBUG:boto:Data: 2024-04-16T17:56:32.388 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.388 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.388 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.388 DEBUG:boto:Params: {} 2024-04-16T17:56:32.388 DEBUG:boto:Token: None 2024-04-16T17:56:32.388 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.392 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c9894800018a2f36-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.392 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.392 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.393 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.393 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.393 DEBUG:boto:Data: 2024-04-16T17:56:32.393 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.393 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.393 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.393 DEBUG:boto:Params: {} 2024-04-16T17:56:32.393 DEBUG:boto:Token: None 2024-04-16T17:56:32.394 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.394 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.398 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000047810e25b58c9747-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.398 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:56:32.398 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.398 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.398 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.398 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.398 DEBUG:boto:Method: GET 2024-04-16T17:56:32.399 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.399 DEBUG:boto:Data: 2024-04-16T17:56:32.399 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.399 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.399 DEBUG:boto:Params: {} 2024-04-16T17:56:32.399 DEBUG:boto:Token: None 2024-04-16T17:56:32.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/?versions 2024-04-16T17:56:32.399 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q= 2024-04-16T17:56:32.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q=', 'Content-Length': '0'} 2024-04-16T17:56:32.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfa15e92780d1f99-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.404 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.404 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:32.405 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:56:32.405 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.405 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.405 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.405 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.405 DEBUG:boto:Method: GET 2024-04-16T17:56:32.405 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.405 DEBUG:boto:Data: 2024-04-16T17:56:32.405 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.405 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.405 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.406 DEBUG:boto:Params: {} 2024-04-16T17:56:32.406 DEBUG:boto:Token: None 2024-04-16T17:56:32.406 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/?versions 2024-04-16T17:56:32.406 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q= 2024-04-16T17:56:32.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q=', 'Content-Length': '0'} 2024-04-16T17:56:32.409 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfca6fb9c7b3efce-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.410 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.410 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:32.410 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.410 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.410 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.410 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.411 DEBUG:boto:Method: GET 2024-04-16T17:56:32.411 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.411 DEBUG:boto:Data: 2024-04-16T17:56:32.411 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.411 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.411 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.411 DEBUG:boto:Params: {} 2024-04-16T17:56:32.411 DEBUG:boto:Token: None 2024-04-16T17:56:32.411 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/?versions 2024-04-16T17:56:32.411 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q= 2024-04-16T17:56:32.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q=', 'Content-Length': '0'} 2024-04-16T17:56:32.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c81a5c48d4db2432-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.415 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.416 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.416 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.416 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.416 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.416 DEBUG:boto:Method: GET 2024-04-16T17:56:32.416 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:32.416 DEBUG:boto:Data: 2024-04-16T17:56:32.416 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.417 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.417 DEBUG:boto:Params: {} 2024-04-16T17:56:32.417 DEBUG:boto:Token: None 2024-04-16T17:56:32.417 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/?versions 2024-04-16T17:56:32.417 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q= 2024-04-16T17:56:32.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P+EO9lmcWko3uybQPvEhJ8r7a6Q=', 'Content-Length': '0'} 2024-04-16T17:56:32.421 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000737209eeba0deabf-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.421 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.422 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:32.422 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.422 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.422 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.422 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.422 DEBUG:boto:Method: GET 2024-04-16T17:56:32.422 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.422 DEBUG:boto:Data: 2024-04-16T17:56:32.423 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.423 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.423 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.423 DEBUG:boto:Params: {} 2024-04-16T17:56:32.423 DEBUG:boto:Token: None 2024-04-16T17:56:32.423 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.423 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU= 2024-04-16T17:56:32.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU=', 'Content-Length': '0'} 2024-04-16T17:56:32.426 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000006fceb7e719e16e2-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.426 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.427 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.427 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.427 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.427 DEBUG:boto:Method: GET 2024-04-16T17:56:32.427 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.427 DEBUG:boto:Data: 2024-04-16T17:56:32.427 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.427 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.427 DEBUG:boto:Params: {} 2024-04-16T17:56:32.427 DEBUG:boto:Token: None 2024-04-16T17:56:32.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU= 2024-04-16T17:56:32.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU=', 'Content-Length': '0'} 2024-04-16T17:56:32.431 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000072a757c146492048-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.431 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.431 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.431 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.431 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.431 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.431 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.431 DEBUG:boto:Data: 2024-04-16T17:56:32.432 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.432 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.432 DEBUG:boto:Params: {} 2024-04-16T17:56:32.432 DEBUG:boto:Token: None 2024-04-16T17:56:32.432 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FWOcHbmJgiIx4DCe/DZl2twtafY= 2024-04-16T17:56:32.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FWOcHbmJgiIx4DCe/DZl2twtafY=', 'Content-Length': '0'} 2024-04-16T17:56:32.435 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fb375b9af1e49f9f-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.436 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.436 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.436 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.436 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.436 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.436 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.436 DEBUG:boto:Data: 2024-04-16T17:56:32.436 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.436 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.436 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.436 DEBUG:boto:Params: {} 2024-04-16T17:56:32.437 DEBUG:boto:Token: None 2024-04-16T17:56:32.437 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FWOcHbmJgiIx4DCe/DZl2twtafY= 2024-04-16T17:56:32.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FWOcHbmJgiIx4DCe/DZl2twtafY=', 'Content-Length': '0'} 2024-04-16T17:56:32.440 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000015729ede5a6e1f77-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.440 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:32.441 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.441 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.441 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.441 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.441 DEBUG:boto:Method: GET 2024-04-16T17:56:32.441 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.441 DEBUG:boto:Data: 2024-04-16T17:56:32.441 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.441 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.441 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.441 DEBUG:boto:Params: {} 2024-04-16T17:56:32.442 DEBUG:boto:Token: None 2024-04-16T17:56:32.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU= 2024-04-16T17:56:32.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU=', 'Content-Length': '0'} 2024-04-16T17:56:32.445 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a6ec574630fcc4a4-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.445 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.445 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.446 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.446 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.446 DEBUG:boto:Method: GET 2024-04-16T17:56:32.446 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.446 DEBUG:boto:Data: 2024-04-16T17:56:32.446 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.446 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.446 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.446 DEBUG:boto:Params: {} 2024-04-16T17:56:32.446 DEBUG:boto:Token: None 2024-04-16T17:56:32.447 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.447 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU= 2024-04-16T17:56:32.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU=', 'Content-Length': '0'} 2024-04-16T17:56:32.450 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009ad0be13919d567a-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.450 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.450 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.450 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.450 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.451 DEBUG:boto:Method: GET 2024-04-16T17:56:32.451 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:32.451 DEBUG:boto:Data: 2024-04-16T17:56:32.451 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.451 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.451 DEBUG:boto:Params: {} 2024-04-16T17:56:32.451 DEBUG:boto:Token: None 2024-04-16T17:56:32.451 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.451 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I= 2024-04-16T17:56:32.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I=', 'Content-Length': '0'} 2024-04-16T17:56:32.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004686290d89cb1556-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.498 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.499 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.499 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.499 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.499 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.499 DEBUG:boto:Method: GET 2024-04-16T17:56:32.499 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:32.499 DEBUG:boto:Data: 2024-04-16T17:56:32.499 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.499 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.499 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.500 DEBUG:boto:Params: {} 2024-04-16T17:56:32.500 DEBUG:boto:Token: None 2024-04-16T17:56:32.500 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I= 2024-04-16T17:56:32.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I=', 'Content-Length': '0'} 2024-04-16T17:56:32.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f4e337bd0f41a13-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.504 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.504 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.504 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.504 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.505 DEBUG:boto:Path: /lybtug-15/dummy 2024-04-16T17:56:32.505 DEBUG:boto:Data: 2024-04-16T17:56:32.505 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.505 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.505 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.505 DEBUG:boto:Params: {} 2024-04-16T17:56:32.505 DEBUG:boto:Token: None 2024-04-16T17:56:32.505 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy 2024-04-16T17:56:32.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aZNoqHUX70j7kmgBeOZN9dx/J1o= 2024-04-16T17:56:32.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aZNoqHUX70j7kmgBeOZN9dx/J1o=', 'Content-Length': '0'} 2024-04-16T17:56:32.509 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000a26ec8fda4e5db0-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.510 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.510 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.510 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.510 DEBUG:boto:Path: /lybtug-15/dummy 2024-04-16T17:56:32.510 DEBUG:boto:Data: 2024-04-16T17:56:32.510 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.510 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.510 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.510 DEBUG:boto:Params: {} 2024-04-16T17:56:32.511 DEBUG:boto:Token: None 2024-04-16T17:56:32.511 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy 2024-04-16T17:56:32.511 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aZNoqHUX70j7kmgBeOZN9dx/J1o= 2024-04-16T17:56:32.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aZNoqHUX70j7kmgBeOZN9dx/J1o=', 'Content-Length': '0'} 2024-04-16T17:56:32.515 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000742b00b795cddaf1-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:32.515 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.515 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.515 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.515 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.515 DEBUG:boto:Method: GET 2024-04-16T17:56:32.516 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.516 DEBUG:boto:Data: 2024-04-16T17:56:32.516 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.516 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.516 DEBUG:boto:Params: {} 2024-04-16T17:56:32.516 DEBUG:boto:Token: None 2024-04-16T17:56:32.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU= 2024-04-16T17:56:32.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU=', 'Content-Length': '0'} 2024-04-16T17:56:32.520 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a04bcef020d152fc-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.520 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:32.520 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:32.520 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.520 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.520 DEBUG:boto:Method: GET 2024-04-16T17:56:32.521 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.521 DEBUG:boto:Data: 2024-04-16T17:56:32.521 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.521 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.521 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.521 DEBUG:boto:Params: {} 2024-04-16T17:56:32.521 DEBUG:boto:Token: None 2024-04-16T17:56:32.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:32.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU= 2024-04-16T17:56:32.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ltCBCFibsywUlpaRzeUzHF9x/qU=', 'Content-Length': '0'} 2024-04-16T17:56:32.524 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b8bdccf986c3b74e-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.525 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.525 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.525 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.525 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.525 DEBUG:boto:Method: GET 2024-04-16T17:56:32.525 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:32.525 DEBUG:boto:Data: 2024-04-16T17:56:32.525 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.525 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.525 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.526 DEBUG:boto:Params: {} 2024-04-16T17:56:32.526 DEBUG:boto:Token: None 2024-04-16T17:56:32.526 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.526 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I= 2024-04-16T17:56:32.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I=', 'Content-Length': '0'} 2024-04-16T17:56:32.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074d598f09303071a-00661ebbd0-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.574 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.575 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.575 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.575 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.575 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:32.575 DEBUG:boto:Method: GET 2024-04-16T17:56:32.575 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:32.575 DEBUG:boto:Data: 2024-04-16T17:56:32.576 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.576 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.576 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.576 DEBUG:boto:Params: {} 2024-04-16T17:56:32.576 DEBUG:boto:Token: None 2024-04-16T17:56:32.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I= 2024-04-16T17:56:32.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O723FLJDGZ5K/ERJbX4gpXqp85I=', 'Content-Length': '0'} 2024-04-16T17:56:32.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cd491f7bc258fea-00661ebbd0-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.580 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:32.581 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-15 zones={test-zone2, test-zone1} 2024-04-16T17:56:32.581 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.581 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.581 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.581 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.581 DEBUG:boto:Data: 2024-04-16T17:56:32.581 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.581 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.582 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.582 DEBUG:boto:Params: {} 2024-04-16T17:56:32.582 DEBUG:boto:Token: None 2024-04-16T17:56:32.582 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.582 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.585 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cb330dc15ccf5f5a-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.586 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.586 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.586 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.586 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.586 DEBUG:boto:Data: 2024-04-16T17:56:32.586 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.586 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.586 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.586 DEBUG:boto:Params: {} 2024-04-16T17:56:32.586 DEBUG:boto:Token: None 2024-04-16T17:56:32.587 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.587 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.630 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-amz-request-id', 'tx00000ad1ade9649add81b-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.631 DEBUG:rgw_multi.tests:zone=test-zone1 version=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:32.631 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.631 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.631 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.631 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.631 DEBUG:boto:Data: 2024-04-16T17:56:32.631 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.631 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.631 DEBUG:boto:Params: {} 2024-04-16T17:56:32.631 DEBUG:boto:Token: None 2024-04-16T17:56:32.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.635 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000027d90908145c607b-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.635 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.635 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.636 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.636 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.636 DEBUG:boto:Data: 2024-04-16T17:56:32.636 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.636 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.636 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.636 DEBUG:boto:Params: {} 2024-04-16T17:56:32.636 DEBUG:boto:Token: None 2024-04-16T17:56:32.636 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.663 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-amz-request-id', 'tx00000e87a91146baf2e06-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.663 DEBUG:rgw_multi.tests:zone=test-zone2 version=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:32.663 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.663 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.663 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.663 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.664 DEBUG:boto:Data: 2024-04-16T17:56:32.664 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.664 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.664 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.664 DEBUG:boto:Params: {} 2024-04-16T17:56:32.664 DEBUG:boto:Token: None 2024-04-16T17:56:32.664 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.668 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '8'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e687dc0da8e44300-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.669 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.669 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.669 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.669 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.669 DEBUG:boto:Data: 2024-04-16T17:56:32.669 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.669 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.669 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.669 DEBUG:boto:Params: {} 2024-04-16T17:56:32.669 DEBUG:boto:Token: None 2024-04-16T17:56:32.670 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.670 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.693 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-amz-request-id', 'tx00000fa5ad6fc5a492d6c-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.693 DEBUG:rgw_multi.tests:zone=test-zone1 version=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:32.693 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.693 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.693 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.693 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.693 DEBUG:boto:Data: 2024-04-16T17:56:32.693 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.694 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.694 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.694 DEBUG:boto:Params: {} 2024-04-16T17:56:32.694 DEBUG:boto:Token: None 2024-04-16T17:56:32.694 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.694 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.698 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '8'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000242a2df689b71525-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.698 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.698 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.698 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.698 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.698 DEBUG:boto:Data: 2024-04-16T17:56:32.698 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.698 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.698 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.699 DEBUG:boto:Params: {} 2024-04-16T17:56:32.699 DEBUG:boto:Token: None 2024-04-16T17:56:32.699 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.699 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.723 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-amz-request-id', 'tx000005ba8c52e5bb12f8e-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.723 DEBUG:rgw_multi.tests:zone=test-zone2 version=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:32.723 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.723 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.723 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.723 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.723 DEBUG:boto:Data: 2024-04-16T17:56:32.724 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.724 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.724 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.724 DEBUG:boto:Params: {} 2024-04-16T17:56:32.724 DEBUG:boto:Token: None 2024-04-16T17:56:32.724 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.728 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001db8f3e7bc469933-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.728 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.728 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.728 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.729 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.729 DEBUG:boto:Data: 2024-04-16T17:56:32.729 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.729 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.729 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.729 DEBUG:boto:Params: {} 2024-04-16T17:56:32.729 DEBUG:boto:Token: None 2024-04-16T17:56:32.729 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.774 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-amz-request-id', 'tx0000097143b5206368236-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.775 DEBUG:rgw_multi.tests:zone=test-zone1 version=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:32.775 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.775 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.775 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.775 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.775 DEBUG:boto:Data: 2024-04-16T17:56:32.775 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.778 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.778 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.778 DEBUG:boto:Params: {} 2024-04-16T17:56:32.779 DEBUG:boto:Token: None 2024-04-16T17:56:32.779 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.779 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.782 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000027cb4570854b5022-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.783 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.783 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.783 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.783 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.783 DEBUG:boto:Data: 2024-04-16T17:56:32.783 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.783 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.783 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.783 DEBUG:boto:Params: {} 2024-04-16T17:56:32.783 DEBUG:boto:Token: None 2024-04-16T17:56:32.784 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.826 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-amz-request-id', 'tx000003dd246a378eed878-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.827 DEBUG:rgw_multi.tests:zone=test-zone2 version=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:32.827 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.827 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.827 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.827 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.827 DEBUG:boto:Data: 2024-04-16T17:56:32.827 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.827 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.827 DEBUG:boto:Params: {} 2024-04-16T17:56:32.827 DEBUG:boto:Token: None 2024-04-16T17:56:32.828 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.831 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004420ff611439c184-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.831 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.832 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.832 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.832 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.832 DEBUG:boto:Data: 2024-04-16T17:56:32.832 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.832 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.832 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.832 DEBUG:boto:Params: {} 2024-04-16T17:56:32.832 DEBUG:boto:Token: None 2024-04-16T17:56:32.832 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.855 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-amz-request-id', 'tx00000e7723fc75dad040f-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.855 DEBUG:rgw_multi.tests:zone=test-zone1 version=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:32.855 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.855 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.856 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.856 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.856 DEBUG:boto:Data: 2024-04-16T17:56:32.856 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.856 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.856 DEBUG:boto:Params: {} 2024-04-16T17:56:32.856 DEBUG:boto:Token: None 2024-04-16T17:56:32.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.860 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000013a06323bf186a30-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.860 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.861 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.861 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.861 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.861 DEBUG:boto:Data: 2024-04-16T17:56:32.861 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.861 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.861 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.861 DEBUG:boto:Params: {} 2024-04-16T17:56:32.862 DEBUG:boto:Token: None 2024-04-16T17:56:32.862 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.885 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-amz-request-id', 'tx00000689e5a86c61ab1ee-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.885 DEBUG:rgw_multi.tests:zone=test-zone2 version=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:32.885 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.885 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.885 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.885 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.885 DEBUG:boto:Data: 2024-04-16T17:56:32.885 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.885 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.886 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.886 DEBUG:boto:Params: {} 2024-04-16T17:56:32.886 DEBUG:boto:Token: None 2024-04-16T17:56:32.886 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.890 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '32'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ac8f5a813cc2ea7a-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.890 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.890 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.890 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.890 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.890 DEBUG:boto:Data: 2024-04-16T17:56:32.890 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.891 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.891 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.891 DEBUG:boto:Params: {} 2024-04-16T17:56:32.891 DEBUG:boto:Token: None 2024-04-16T17:56:32.891 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.891 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.914 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-amz-request-id', 'tx000003b7330c8214373bc-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.914 DEBUG:rgw_multi.tests:zone=test-zone1 version=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:32.915 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.915 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.915 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.915 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.915 DEBUG:boto:Data: 2024-04-16T17:56:32.915 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.915 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.915 DEBUG:boto:Params: {} 2024-04-16T17:56:32.915 DEBUG:boto:Token: None 2024-04-16T17:56:32.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.919 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '32'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000015cdfe96b6d334aa-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.920 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.920 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.920 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.920 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.920 DEBUG:boto:Data: 2024-04-16T17:56:32.920 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:32.920 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.921 DEBUG:boto:Params: {} 2024-04-16T17:56:32.921 DEBUG:boto:Token: None 2024-04-16T17:56:32.921 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:32.967 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-amz-request-id', 'tx0000024cb9e8fd4ffed04-00661ebbd0-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.967 DEBUG:rgw_multi.tests:zone=test-zone2 version=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:32.967 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:32.967 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:32.967 DEBUG:boto:Method: HEAD 2024-04-16T17:56:32.967 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:32.968 DEBUG:boto:Data: 2024-04-16T17:56:32.968 DEBUG:boto:Headers: {} 2024-04-16T17:56:32.968 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.968 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.968 DEBUG:boto:Params: {} 2024-04-16T17:56:32.968 DEBUG:boto:Token: None 2024-04-16T17:56:32.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/ 2024-04-16T17:56:32.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U= 2024-04-16T17:56:32.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/srnBk98UHfz3CvxxgQqLaU9A5U=', 'Content-Length': '0'} 2024-04-16T17:56:32.972 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '40'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050c98ba67cb9eb17-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:32.973 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:32.973 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:32.973 DEBUG:boto:Method: PUT 2024-04-16T17:56:32.973 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:32.973 DEBUG:boto:Data: 2024-04-16T17:56:32.973 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:32.973 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:32.973 DEBUG:boto:Port: 8000 2024-04-16T17:56:32.973 DEBUG:boto:Params: {} 2024-04-16T17:56:32.974 DEBUG:boto:Token: None 2024-04-16T17:56:32.974 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:32 GMT /lybtug-15/obj 2024-04-16T17:56:32.974 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI= 2024-04-16T17:56:32.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XpeRRfiftiQUDhWF6OVJxVWJNtI='} 2024-04-16T17:56:33.018 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-amz-request-id', 'tx000000e4133757c2e50bc-00661ebbd0-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.019 DEBUG:rgw_multi.tests:zone=test-zone1 version=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:33.019 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.019 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.019 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.019 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.019 DEBUG:boto:Data: 2024-04-16T17:56:33.019 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.019 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.019 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.019 DEBUG:boto:Params: {} 2024-04-16T17:56:33.020 DEBUG:boto:Token: None 2024-04-16T17:56:33.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.023 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '40'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006742c5560449bf05-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.024 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.024 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.024 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.024 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.024 DEBUG:boto:Data: 2024-04-16T17:56:33.024 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.024 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.024 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.025 DEBUG:boto:Params: {} 2024-04-16T17:56:33.025 DEBUG:boto:Token: None 2024-04-16T17:56:33.025 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.050 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-amz-request-id', 'tx00000e041c202794cd2bd-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.050 DEBUG:rgw_multi.tests:zone=test-zone2 version=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:33.050 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.051 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.051 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.051 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.051 DEBUG:boto:Data: 2024-04-16T17:56:33.051 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.051 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.051 DEBUG:boto:Params: {} 2024-04-16T17:56:33.051 DEBUG:boto:Token: None 2024-04-16T17:56:33.051 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.055 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000081c11d0d53b25d64-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.055 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.055 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.056 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.056 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.056 DEBUG:boto:Data: 2024-04-16T17:56:33.056 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.056 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.056 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.056 DEBUG:boto:Params: {} 2024-04-16T17:56:33.056 DEBUG:boto:Token: None 2024-04-16T17:56:33.056 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.081 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-amz-request-id', 'tx000006cc5f1b9bbeb83ef-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.081 DEBUG:rgw_multi.tests:zone=test-zone1 version=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:33.081 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.081 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.081 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.081 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.081 DEBUG:boto:Data: 2024-04-16T17:56:33.081 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.081 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.082 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.082 DEBUG:boto:Params: {} 2024-04-16T17:56:33.082 DEBUG:boto:Token: None 2024-04-16T17:56:33.082 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.082 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.086 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000728d21d8d51f9178-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.086 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.086 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.086 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.086 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.086 DEBUG:boto:Data: 2024-04-16T17:56:33.086 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.086 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.087 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.087 DEBUG:boto:Params: {} 2024-04-16T17:56:33.087 DEBUG:boto:Token: None 2024-04-16T17:56:33.087 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.087 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.110 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-amz-request-id', 'tx00000bd523283499eb3e5-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.111 DEBUG:rgw_multi.tests:zone=test-zone2 version=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:33.111 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.111 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.111 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.111 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.111 DEBUG:boto:Data: 2024-04-16T17:56:33.111 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.111 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.111 DEBUG:boto:Params: {} 2024-04-16T17:56:33.111 DEBUG:boto:Token: None 2024-04-16T17:56:33.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.115 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '56'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d4d27a0f3950a1c5-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.115 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.115 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.115 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.115 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.116 DEBUG:boto:Data: 2024-04-16T17:56:33.116 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.116 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.116 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.116 DEBUG:boto:Params: {} 2024-04-16T17:56:33.116 DEBUG:boto:Token: None 2024-04-16T17:56:33.116 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.159 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-amz-request-id', 'tx000006d25a59d50a3303a-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.159 DEBUG:rgw_multi.tests:zone=test-zone1 version=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:33.159 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.159 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.159 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.159 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.159 DEBUG:boto:Data: 2024-04-16T17:56:33.159 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.159 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.160 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.160 DEBUG:boto:Params: {} 2024-04-16T17:56:33.160 DEBUG:boto:Token: None 2024-04-16T17:56:33.160 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.164 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '56'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000013d4c7a79dacc2e8-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.164 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.164 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.164 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.164 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.165 DEBUG:boto:Data: 2024-04-16T17:56:33.165 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.165 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.165 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.165 DEBUG:boto:Params: {} 2024-04-16T17:56:33.165 DEBUG:boto:Token: None 2024-04-16T17:56:33.165 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.165 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.214 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-amz-request-id', 'tx0000051c75045a1b21f11-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.215 DEBUG:rgw_multi.tests:zone=test-zone2 version=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:33.215 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.215 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.215 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.216 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.216 DEBUG:boto:Data: 2024-04-16T17:56:33.216 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.216 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.216 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.216 DEBUG:boto:Params: {} 2024-04-16T17:56:33.216 DEBUG:boto:Token: None 2024-04-16T17:56:33.216 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.220 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '64'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000acfaec056751c8b6-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.220 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.220 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.221 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.221 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.221 DEBUG:boto:Data: 2024-04-16T17:56:33.221 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.221 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.221 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.221 DEBUG:boto:Params: {} 2024-04-16T17:56:33.221 DEBUG:boto:Token: None 2024-04-16T17:56:33.221 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.244 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-amz-request-id', 'tx00000ad0d27b94123e52b-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.244 DEBUG:rgw_multi.tests:zone=test-zone1 version=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:33.244 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.244 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.244 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.244 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.245 DEBUG:boto:Data: 2024-04-16T17:56:33.245 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.245 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.245 DEBUG:boto:Params: {} 2024-04-16T17:56:33.245 DEBUG:boto:Token: None 2024-04-16T17:56:33.245 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.249 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '64'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d18cacaa0f084e92-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.249 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.249 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.249 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.249 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.250 DEBUG:boto:Data: 2024-04-16T17:56:33.250 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.250 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.250 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.250 DEBUG:boto:Params: {} 2024-04-16T17:56:33.250 DEBUG:boto:Token: None 2024-04-16T17:56:33.250 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.250 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.273 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-amz-request-id', 'tx00000bf5cf5e662b6d99b-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.274 DEBUG:rgw_multi.tests:zone=test-zone2 version=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:33.274 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.274 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.274 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.274 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.274 DEBUG:boto:Data: 2024-04-16T17:56:33.274 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.274 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.274 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.274 DEBUG:boto:Params: {} 2024-04-16T17:56:33.275 DEBUG:boto:Token: None 2024-04-16T17:56:33.275 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.275 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.278 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007e7a6b7db4665666-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.279 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.279 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.279 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.279 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.279 DEBUG:boto:Data: 2024-04-16T17:56:33.279 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.279 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:33.279 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.279 DEBUG:boto:Params: {} 2024-04-16T17:56:33.279 DEBUG:boto:Token: None 2024-04-16T17:56:33.280 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.322 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-amz-request-id', 'tx00000f6d3af132bcb6574-00661ebbd1-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.323 DEBUG:rgw_multi.tests:zone=test-zone1 version=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:33.323 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:33.323 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:33.323 DEBUG:boto:Method: HEAD 2024-04-16T17:56:33.323 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:33.323 DEBUG:boto:Data: 2024-04-16T17:56:33.323 DEBUG:boto:Headers: {} 2024-04-16T17:56:33.323 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.323 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.323 DEBUG:boto:Params: {} 2024-04-16T17:56:33.323 DEBUG:boto:Token: None 2024-04-16T17:56:33.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/ 2024-04-16T17:56:33.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s= 2024-04-16T17:56:33.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ncb92qhfYl/iU/O2fzU0babF84s=', 'Content-Length': '0'} 2024-04-16T17:56:33.327 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000542ba821aba91b1d-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.327 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:33.328 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:33.328 DEBUG:boto:Method: PUT 2024-04-16T17:56:33.328 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:33.328 DEBUG:boto:Data: 2024-04-16T17:56:33.328 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2024-04-16T17:56:33.328 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:33.328 DEBUG:boto:Port: 8000 2024-04-16T17:56:33.328 DEBUG:boto:Params: {} 2024-04-16T17:56:33.328 DEBUG:boto:Token: None 2024-04-16T17:56:33.328 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Tue, 16 Apr 2024 17:56:33 GMT /lybtug-15/obj 2024-04-16T17:56:33.329 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0= 2024-04-16T17:56:33.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:56:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57LXGFFh85OZ9QdP+g8yDWbyD0='} 2024-04-16T17:56:33.370 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-amz-request-id', 'tx00000439e592d93695b9b-00661ebbd1-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:33.371 DEBUG:rgw_multi.tests:zone=test-zone2 version=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:33.371 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-15 2024-04-16T17:56:33.371 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:33.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:33.372 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-15 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:56:33.422 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:33.422 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:33.505+0000 7f04fd83e980 1 waiting for incremental sync to catch up: 2024-04-16T17:56:33.500 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000002.14.11, , , , , ] 2024-04-16T17:56:33.500 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000003.16.10, , , 00000000020.46.12, , ] 2024-04-16T17:56:38.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:38.505+0000 7f04fd83e980 1 waiting for incremental sync to catch up: 2024-04-16T17:56:38.503 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000002.14.11, , , 00000000031.77.12, , ] 2024-04-16T17:56:38.503 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000003.16.10, , , 00000000020.46.12, , ] 2024-04-16T17:56:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:43.509+0000 7f04fd83e980 1 waiting for incremental sync to catch up: 2024-04-16T17:56:43.504 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000002.14.11, , , 00000000031.77.12, , ] 2024-04-16T17:56:43.504 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000003.16.10, , , 00000000020.46.12, , ] 2024-04-16T17:56:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:48.509+0000 7f04fd83e980 1 bucket sync caught up with source: 2024-04-16T17:56:48.506 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000003.16.10, , , 00000000031.77.12, , ] 2024-04-16T17:56:48.506 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000003.16.10, , , 00000000020.46.12, , ] 2024-04-16T17:56:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:56:48.509+0000 7f04fd83e980 0 bucket checkpoint complete 2024-04-16T17:56:48.554 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-15 2024-04-16T17:56:48.554 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:48.555 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-15', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:56:48.555 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-15 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:56:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:48.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:48.682+0000 7f7922433980 1 waiting for incremental sync to catch up: 2024-04-16T17:56:48.681 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , 00000000002.19.11, , , 00000000020.71.12, , ] 2024-04-16T17:56:48.681 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , 00000000002.19.11, , , 00000000031.102.9, , ] 2024-04-16T17:56:53.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:53.686+0000 7f7922433980 1 bucket sync caught up with source: 2024-04-16T17:56:53.684 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , 00000000002.19.11, , , 00000000031.102.9, , ] 2024-04-16T17:56:53.684 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , 00000000002.19.11, , , 00000000031.102.9, , ] 2024-04-16T17:56:53.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:56:53.686+0000 7f7922433980 0 bucket checkpoint complete 2024-04-16T17:56:53.724 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-15 zones={test-zone2, test-zone1} 2024-04-16T17:56:53.724 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.725 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.725 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.725 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:53.725 DEBUG:boto:Data: 2024-04-16T17:56:53.725 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.725 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.725 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.725 DEBUG:boto:Params: {} 2024-04-16T17:56:53.725 DEBUG:boto:Token: None 2024-04-16T17:56:53.726 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/ 2024-04-16T17:56:53.726 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2FVIzprfaDea0Q08ngV2qpo8Isk= 2024-04-16T17:56:53.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2FVIzprfaDea0Q08ngV2qpo8Isk=', 'Content-Length': '0'} 2024-04-16T17:56:53.730 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '160'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d1e3024dd5cffc53-00661ebbe5-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.730 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.731 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.731 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.731 DEBUG:boto:Path: /lybtug-15/ 2024-04-16T17:56:53.731 DEBUG:boto:Data: 2024-04-16T17:56:53.731 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.731 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.731 DEBUG:boto:Params: {} 2024-04-16T17:56:53.731 DEBUG:boto:Token: None 2024-04-16T17:56:53.731 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/ 2024-04-16T17:56:53.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2FVIzprfaDea0Q08ngV2qpo8Isk= 2024-04-16T17:56:53.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2FVIzprfaDea0Q08ngV2qpo8Isk=', 'Content-Length': '0'} 2024-04-16T17:56:53.735 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '160'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f3ea72aefa787fea-00661ebbe5-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.735 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:56:53.735 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.735 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.736 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.736 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.736 DEBUG:boto:Method: GET 2024-04-16T17:56:53.736 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.736 DEBUG:boto:Data: 2024-04-16T17:56:53.736 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.736 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.736 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.736 DEBUG:boto:Params: {} 2024-04-16T17:56:53.736 DEBUG:boto:Token: None 2024-04-16T17:56:53.736 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/?versions 2024-04-16T17:56:53.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE= 2024-04-16T17:56:53.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE=', 'Content-Length': '0'} 2024-04-16T17:56:53.742 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a15953f159cbd9d-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.742 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111NormalobjXJZrDUSbKtRrUBGGF-xtn75HidxGDK6true2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjxVk3GPd16PUPBBjIR.27F0BQoWuNXbBfalse2024-04-16T17:56:33.287Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4false2024-04-16T17:56:33.258Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjNxNMkltnMt-ticfCow-qcCHP0ay9YInfalse2024-04-16T17:56:33.228Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj2qSUAJ6panT59CkpTafgVOmu.SunHYvfalse2024-04-16T17:56:33.123Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjjcONTBJ7GDxGwW4redxUYSvnuYmDy86false2024-04-16T17:56:33.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5g.23p6esxNXeTOkllHk-A-IislofLtfalse2024-04-16T17:56:33.063Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobjv.A7JAAtanjXLBS8soduLW6h.mvqob6false2024-04-16T17:56:33.095Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAzfalse2024-04-16T17:56:32.981Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjTi61G73jRSlmFy6mWBV7t7VETZRGeD1false2024-04-16T17:56:33.033Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjUgUlGbQIYdfiGb9XWOfJzocNEtE6XC5false2024-04-16T17:56:32.929Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjrC25L2Rwl6lKjQSpxdNI-R.WY975kFcfalse2024-04-16T17:56:32.898Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjCZaDb7ETzISKO2ue.6IoBIPJNOl.MDofalse2024-04-16T17:56:32.870Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjMK5bjd1abLVVPz6kMWrR-SDuiZ0vR65false2024-04-16T17:56:32.839Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjGbG1ol87fm4v9MJM91TmJaz.F1y08.Xfalse2024-04-16T17:56:32.736Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjXYCyjdHzLH98ukxgiSCKrzW90a8.VDzfalse2024-04-16T17:56:32.792Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjJxY.HRojXcWb.xNlB2VxGvKZuCNVhq8false2024-04-16T17:56:32.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjvJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZfalse2024-04-16T17:56:32.707Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjAERxD8InPQYZhCUcNH0LNchHJgyZ9lkfalse2024-04-16T17:56:32.645Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjtzjvpfN57rNIeCpjBOk9ZM3a5.7yTgzfalse2024-04-16T17:56:32.594Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.744 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.745 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:56:53.746 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.746 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.746 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.746 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.746 DEBUG:boto:Method: GET 2024-04-16T17:56:53.746 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.746 DEBUG:boto:Data: 2024-04-16T17:56:53.746 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.746 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.746 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.747 DEBUG:boto:Params: {} 2024-04-16T17:56:53.747 DEBUG:boto:Token: None 2024-04-16T17:56:53.747 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/?versions 2024-04-16T17:56:53.747 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE= 2024-04-16T17:56:53.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE=', 'Content-Length': '0'} 2024-04-16T17:56:53.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049af82ac9c9b5d8a-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.753 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111NormalobjXJZrDUSbKtRrUBGGF-xtn75HidxGDK6true2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjxVk3GPd16PUPBBjIR.27F0BQoWuNXbBfalse2024-04-16T17:56:33.287Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4false2024-04-16T17:56:33.258Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjNxNMkltnMt-ticfCow-qcCHP0ay9YInfalse2024-04-16T17:56:33.228Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj2qSUAJ6panT59CkpTafgVOmu.SunHYvfalse2024-04-16T17:56:33.123Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjjcONTBJ7GDxGwW4redxUYSvnuYmDy86false2024-04-16T17:56:33.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5g.23p6esxNXeTOkllHk-A-IislofLtfalse2024-04-16T17:56:33.063Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobjv.A7JAAtanjXLBS8soduLW6h.mvqob6false2024-04-16T17:56:33.095Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAzfalse2024-04-16T17:56:32.981Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjTi61G73jRSlmFy6mWBV7t7VETZRGeD1false2024-04-16T17:56:33.033Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjUgUlGbQIYdfiGb9XWOfJzocNEtE6XC5false2024-04-16T17:56:32.929Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjrC25L2Rwl6lKjQSpxdNI-R.WY975kFcfalse2024-04-16T17:56:32.898Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjCZaDb7ETzISKO2ue.6IoBIPJNOl.MDofalse2024-04-16T17:56:32.870Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjMK5bjd1abLVVPz6kMWrR-SDuiZ0vR65false2024-04-16T17:56:32.839Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjGbG1ol87fm4v9MJM91TmJaz.F1y08.Xfalse2024-04-16T17:56:32.736Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjXYCyjdHzLH98ukxgiSCKrzW90a8.VDzfalse2024-04-16T17:56:32.792Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjJxY.HRojXcWb.xNlB2VxGvKZuCNVhq8false2024-04-16T17:56:32.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjvJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZfalse2024-04-16T17:56:32.707Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjAERxD8InPQYZhCUcNH0LNchHJgyZ9lkfalse2024-04-16T17:56:32.645Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjtzjvpfN57rNIeCpjBOk9ZM3a5.7yTgzfalse2024-04-16T17:56:32.594Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.754 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:56:53.755 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.755 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.755 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.756 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.756 DEBUG:boto:Method: GET 2024-04-16T17:56:53.756 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.756 DEBUG:boto:Data: 2024-04-16T17:56:53.756 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.756 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.756 DEBUG:boto:Params: {} 2024-04-16T17:56:53.756 DEBUG:boto:Token: None 2024-04-16T17:56:53.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/?versions 2024-04-16T17:56:53.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE= 2024-04-16T17:56:53.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE=', 'Content-Length': '0'} 2024-04-16T17:56:53.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fe203159ee9eb2f-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.762 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111NormalobjXJZrDUSbKtRrUBGGF-xtn75HidxGDK6true2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjxVk3GPd16PUPBBjIR.27F0BQoWuNXbBfalse2024-04-16T17:56:33.287Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4false2024-04-16T17:56:33.258Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjNxNMkltnMt-ticfCow-qcCHP0ay9YInfalse2024-04-16T17:56:33.228Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj2qSUAJ6panT59CkpTafgVOmu.SunHYvfalse2024-04-16T17:56:33.123Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjjcONTBJ7GDxGwW4redxUYSvnuYmDy86false2024-04-16T17:56:33.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5g.23p6esxNXeTOkllHk-A-IislofLtfalse2024-04-16T17:56:33.063Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobjv.A7JAAtanjXLBS8soduLW6h.mvqob6false2024-04-16T17:56:33.095Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAzfalse2024-04-16T17:56:32.981Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjTi61G73jRSlmFy6mWBV7t7VETZRGeD1false2024-04-16T17:56:33.033Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjUgUlGbQIYdfiGb9XWOfJzocNEtE6XC5false2024-04-16T17:56:32.929Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjrC25L2Rwl6lKjQSpxdNI-R.WY975kFcfalse2024-04-16T17:56:32.898Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjCZaDb7ETzISKO2ue.6IoBIPJNOl.MDofalse2024-04-16T17:56:32.870Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjMK5bjd1abLVVPz6kMWrR-SDuiZ0vR65false2024-04-16T17:56:32.839Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjGbG1ol87fm4v9MJM91TmJaz.F1y08.Xfalse2024-04-16T17:56:32.736Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjXYCyjdHzLH98ukxgiSCKrzW90a8.VDzfalse2024-04-16T17:56:32.792Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjJxY.HRojXcWb.xNlB2VxGvKZuCNVhq8false2024-04-16T17:56:32.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjvJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZfalse2024-04-16T17:56:32.707Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjAERxD8InPQYZhCUcNH0LNchHJgyZ9lkfalse2024-04-16T17:56:32.645Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjtzjvpfN57rNIeCpjBOk9ZM3a5.7yTgzfalse2024-04-16T17:56:32.594Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.763 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.763 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.763 DEBUG:boto:path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.763 DEBUG:boto:auth_path=/lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.763 DEBUG:boto:Method: GET 2024-04-16T17:56:53.763 DEBUG:boto:Path: /lybtug-15/?versions&max-keys=999 2024-04-16T17:56:53.763 DEBUG:boto:Data: 2024-04-16T17:56:53.764 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.764 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.764 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.764 DEBUG:boto:Params: {} 2024-04-16T17:56:53.764 DEBUG:boto:Token: None 2024-04-16T17:56:53.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/?versions 2024-04-16T17:56:53.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE= 2024-04-16T17:56:53.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D5fmgod8XSrZyzbBNCtO6MfmINE=', 'Content-Length': '0'} 2024-04-16T17:56:53.769 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000535288025175f449-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.769 DEBUG:boto:b'lybtug-15999falsedummykwxenR7qU2DZ--Tko-Wax.D0U1iJfwztrue2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111NormalobjXJZrDUSbKtRrUBGGF-xtn75HidxGDK6true2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjxVk3GPd16PUPBBjIR.27F0BQoWuNXbBfalse2024-04-16T17:56:33.287Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4false2024-04-16T17:56:33.258Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjNxNMkltnMt-ticfCow-qcCHP0ay9YInfalse2024-04-16T17:56:33.228Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj2qSUAJ6panT59CkpTafgVOmu.SunHYvfalse2024-04-16T17:56:33.123Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjjcONTBJ7GDxGwW4redxUYSvnuYmDy86false2024-04-16T17:56:33.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj5g.23p6esxNXeTOkllHk-A-IislofLtfalse2024-04-16T17:56:33.063Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobjv.A7JAAtanjXLBS8soduLW6h.mvqob6false2024-04-16T17:56:33.095Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normalobj8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAzfalse2024-04-16T17:56:32.981Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjTi61G73jRSlmFy6mWBV7t7VETZRGeD1false2024-04-16T17:56:33.033Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjUgUlGbQIYdfiGb9XWOfJzocNEtE6XC5false2024-04-16T17:56:32.929Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjrC25L2Rwl6lKjQSpxdNI-R.WY975kFcfalse2024-04-16T17:56:32.898Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjCZaDb7ETzISKO2ue.6IoBIPJNOl.MDofalse2024-04-16T17:56:32.870Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjMK5bjd1abLVVPz6kMWrR-SDuiZ0vR65false2024-04-16T17:56:32.839Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjGbG1ol87fm4v9MJM91TmJaz.F1y08.Xfalse2024-04-16T17:56:32.736Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjXYCyjdHzLH98ukxgiSCKrzW90a8.VDzfalse2024-04-16T17:56:32.792Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjJxY.HRojXcWb.xNlB2VxGvKZuCNVhq8false2024-04-16T17:56:32.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjvJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZfalse2024-04-16T17:56:32.707Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjAERxD8InPQYZhCUcNH0LNchHJgyZ9lkfalse2024-04-16T17:56:32.645Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111NormalobjtzjvpfN57rNIeCpjBOk9ZM3a5.7yTgzfalse2024-04-16T17:56:32.594Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.770 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:53.771 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.771 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.771 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.771 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.771 DEBUG:boto:Method: GET 2024-04-16T17:56:53.771 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.771 DEBUG:boto:Data: 2024-04-16T17:56:53.771 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.771 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.771 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.772 DEBUG:boto:Params: {} 2024-04-16T17:56:53.772 DEBUG:boto:Token: None 2024-04-16T17:56:53.772 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM= 2024-04-16T17:56:53.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM=', 'Content-Length': '0'} 2024-04-16T17:56:53.775 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000086d7e6e6759307fc-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.775 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.776 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.776 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.776 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.776 DEBUG:boto:Method: GET 2024-04-16T17:56:53.776 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.776 DEBUG:boto:Data: 2024-04-16T17:56:53.776 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.776 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.776 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.776 DEBUG:boto:Params: {} 2024-04-16T17:56:53.776 DEBUG:boto:Token: None 2024-04-16T17:56:53.777 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM= 2024-04-16T17:56:53.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM=', 'Content-Length': '0'} 2024-04-16T17:56:53.780 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000798b9ce393a5892d-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.780 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.780 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.780 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.780 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.781 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.781 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.781 DEBUG:boto:Data: 2024-04-16T17:56:53.781 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.781 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.781 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.781 DEBUG:boto:Params: {} 2024-04-16T17:56:53.781 DEBUG:boto:Token: None 2024-04-16T17:56:53.781 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uH29+9cpOaAIyxHxe9RLJbuFydQ= 2024-04-16T17:56:53.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uH29+9cpOaAIyxHxe9RLJbuFydQ=', 'Content-Length': '0'} 2024-04-16T17:56:53.784 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000053279ebecfb0fc1a-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.785 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.785 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.785 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.785 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.785 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.785 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.785 DEBUG:boto:Data: 2024-04-16T17:56:53.785 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.785 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.785 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.786 DEBUG:boto:Params: {} 2024-04-16T17:56:53.786 DEBUG:boto:Token: None 2024-04-16T17:56:53.786 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.786 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uH29+9cpOaAIyxHxe9RLJbuFydQ= 2024-04-16T17:56:53.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uH29+9cpOaAIyxHxe9RLJbuFydQ=', 'Content-Length': '0'} 2024-04-16T17:56:53.789 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c902c4e78592c017-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.789 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:53.789 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.790 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.790 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.790 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.790 DEBUG:boto:Method: GET 2024-04-16T17:56:53.790 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.790 DEBUG:boto:Data: 2024-04-16T17:56:53.790 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.790 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.790 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.790 DEBUG:boto:Params: {} 2024-04-16T17:56:53.790 DEBUG:boto:Token: None 2024-04-16T17:56:53.791 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.791 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM= 2024-04-16T17:56:53.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM=', 'Content-Length': '0'} 2024-04-16T17:56:53.794 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000071e93f3148c455e5-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.794 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.794 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.794 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.794 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.794 DEBUG:boto:Method: GET 2024-04-16T17:56:53.795 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.795 DEBUG:boto:Data: 2024-04-16T17:56:53.795 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.795 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.795 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.795 DEBUG:boto:Params: {} 2024-04-16T17:56:53.795 DEBUG:boto:Token: None 2024-04-16T17:56:53.795 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.795 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM= 2024-04-16T17:56:53.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM=', 'Content-Length': '0'} 2024-04-16T17:56:53.798 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000083f68ab74e4210cc-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.798 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.798 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.799 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.799 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.799 DEBUG:boto:Method: GET 2024-04-16T17:56:53.799 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:53.799 DEBUG:boto:Data: 2024-04-16T17:56:53.799 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.799 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.799 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.799 DEBUG:boto:Params: {} 2024-04-16T17:56:53.799 DEBUG:boto:Token: None 2024-04-16T17:56:53.799 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/ 2024-04-16T17:56:53.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k= 2024-04-16T17:56:53.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k=', 'Content-Length': '0'} 2024-04-16T17:56:53.803 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f40b0fa2b436ba94-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.847 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.847 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.847 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.847 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.847 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.847 DEBUG:boto:Method: GET 2024-04-16T17:56:53.847 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:53.848 DEBUG:boto:Data: 2024-04-16T17:56:53.848 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.848 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.848 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.848 DEBUG:boto:Params: {} 2024-04-16T17:56:53.848 DEBUG:boto:Token: None 2024-04-16T17:56:53.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/ 2024-04-16T17:56:53.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k= 2024-04-16T17:56:53.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k=', 'Content-Length': '0'} 2024-04-16T17:56:53.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000679edd263f181183-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.852 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.853 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.853 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.853 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.853 DEBUG:boto:Path: /lybtug-15/dummy 2024-04-16T17:56:53.853 DEBUG:boto:Data: 2024-04-16T17:56:53.853 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.853 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.853 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.853 DEBUG:boto:Params: {} 2024-04-16T17:56:53.853 DEBUG:boto:Token: None 2024-04-16T17:56:53.854 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy 2024-04-16T17:56:53.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RC37XGTUMe3kBM1XC9LCW3nK/M8= 2024-04-16T17:56:53.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RC37XGTUMe3kBM1XC9LCW3nK/M8=', 'Content-Length': '0'} 2024-04-16T17:56:53.857 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006bb40690c1678dfc-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.858 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.858 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.858 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.858 DEBUG:boto:Path: /lybtug-15/dummy 2024-04-16T17:56:53.858 DEBUG:boto:Data: 2024-04-16T17:56:53.858 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.858 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.858 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.858 DEBUG:boto:Params: {} 2024-04-16T17:56:53.858 DEBUG:boto:Token: None 2024-04-16T17:56:53.859 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy 2024-04-16T17:56:53.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RC37XGTUMe3kBM1XC9LCW3nK/M8= 2024-04-16T17:56:53.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RC37XGTUMe3kBM1XC9LCW3nK/M8=', 'Content-Length': '0'} 2024-04-16T17:56:53.863 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007dc23adf3e472c58-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.863 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2024-04-16T17:56:53.863 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.863 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.863 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.863 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.863 DEBUG:boto:Method: GET 2024-04-16T17:56:53.863 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.864 DEBUG:boto:Data: 2024-04-16T17:56:53.864 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.864 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.864 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.864 DEBUG:boto:Params: {} 2024-04-16T17:56:53.864 DEBUG:boto:Token: None 2024-04-16T17:56:53.864 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM= 2024-04-16T17:56:53.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM=', 'Content-Length': '0'} 2024-04-16T17:56:53.867 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000079e3083268d5854-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.867 DEBUG:boto:path=/lybtug-15/dummy 2024-04-16T17:56:53.868 DEBUG:boto:auth_path=/lybtug-15/dummy 2024-04-16T17:56:53.868 DEBUG:boto:path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.868 DEBUG:boto:auth_path=/lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.868 DEBUG:boto:Method: GET 2024-04-16T17:56:53.868 DEBUG:boto:Path: /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.868 DEBUG:boto:Data: 2024-04-16T17:56:53.868 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.868 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.868 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.868 DEBUG:boto:Params: {} 2024-04-16T17:56:53.868 DEBUG:boto:Token: None 2024-04-16T17:56:53.868 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/dummy?versionId=kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz 2024-04-16T17:56:53.869 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM= 2024-04-16T17:56:53.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKzhqgzKNCxh9w6O/WXjcLRn2mM=', 'Content-Length': '0'} 2024-04-16T17:56:53.872 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:11 GMT'), ('x-amz-version-id', 'kwxenR7qU2DZ--Tko-Wax.D0U1iJfwz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002a71f56c958825b2-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.872 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.872 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.872 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.873 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.873 DEBUG:boto:Method: GET 2024-04-16T17:56:53.873 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:53.873 DEBUG:boto:Data: 2024-04-16T17:56:53.873 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.873 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.873 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.873 DEBUG:boto:Params: {} 2024-04-16T17:56:53.873 DEBUG:boto:Token: None 2024-04-16T17:56:53.873 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/ 2024-04-16T17:56:53.873 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k= 2024-04-16T17:56:53.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k=', 'Content-Length': '0'} 2024-04-16T17:56:53.877 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002dfd3510dd59bc62-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.877 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.878 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:53.878 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:53.878 DEBUG:boto:path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.878 DEBUG:boto:auth_path=/lybtug-15/?prefix=dummy 2024-04-16T17:56:53.878 DEBUG:boto:Method: GET 2024-04-16T17:56:53.878 DEBUG:boto:Path: /lybtug-15/?prefix=dummy 2024-04-16T17:56:53.878 DEBUG:boto:Data: 2024-04-16T17:56:53.878 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.878 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.878 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.878 DEBUG:boto:Params: {} 2024-04-16T17:56:53.879 DEBUG:boto:Token: None 2024-04-16T17:56:53.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/ 2024-04-16T17:56:53.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k= 2024-04-16T17:56:53.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T7iYiz3zRNY/G06ghcB5weNWg7k=', 'Content-Length': '0'} 2024-04-16T17:56:53.882 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084f685619585c69c-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.926 DEBUG:boto:b'lybtug-15dummy1000falsedummy2024-04-16T17:56:11.989Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:53.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:53.927 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:53.927 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:53.927 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.927 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.927 DEBUG:boto:Method: GET 2024-04-16T17:56:53.927 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.927 DEBUG:boto:Data: 2024-04-16T17:56:53.927 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.927 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.928 DEBUG:boto:Params: {} 2024-04-16T17:56:53.928 DEBUG:boto:Token: None 2024-04-16T17:56:53.928 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8= 2024-04-16T17:56:53.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8=', 'Content-Length': '0'} 2024-04-16T17:56:53.931 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ba53082e5932f540-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.931 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:53.931 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:53.931 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.931 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.932 DEBUG:boto:Method: GET 2024-04-16T17:56:53.932 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.932 DEBUG:boto:Data: 2024-04-16T17:56:53.932 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.932 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.932 DEBUG:boto:Params: {} 2024-04-16T17:56:53.932 DEBUG:boto:Token: None 2024-04-16T17:56:53.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8= 2024-04-16T17:56:53.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8=', 'Content-Length': '0'} 2024-04-16T17:56:53.936 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000272b2dcf6aed17f4-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.936 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:53.936 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:53.936 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.936 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.936 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.937 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.937 DEBUG:boto:Data: 2024-04-16T17:56:53.937 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.937 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.937 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.937 DEBUG:boto:Params: {} 2024-04-16T17:56:53.937 DEBUG:boto:Token: None 2024-04-16T17:56:53.937 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.937 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:004GSsmicUtMBRbqzonKlbrhpBA= 2024-04-16T17:56:53.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:004GSsmicUtMBRbqzonKlbrhpBA=', 'Content-Length': '0'} 2024-04-16T17:56:53.940 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005c13ff15595121c2-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.940 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:53.940 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:53.941 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.941 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.941 DEBUG:boto:Method: HEAD 2024-04-16T17:56:53.941 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.941 DEBUG:boto:Data: 2024-04-16T17:56:53.941 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.941 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.941 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.941 DEBUG:boto:Params: {} 2024-04-16T17:56:53.941 DEBUG:boto:Token: None 2024-04-16T17:56:53.941 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.942 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:004GSsmicUtMBRbqzonKlbrhpBA= 2024-04-16T17:56:53.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:004GSsmicUtMBRbqzonKlbrhpBA=', 'Content-Length': '0'} 2024-04-16T17:56:53.944 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e98884615d37ea5d-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.944 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:53.945 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:53.945 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:53.945 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.945 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.945 DEBUG:boto:Method: GET 2024-04-16T17:56:53.945 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.945 DEBUG:boto:Data: 2024-04-16T17:56:53.945 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.945 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:53.945 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.946 DEBUG:boto:Params: {} 2024-04-16T17:56:53.946 DEBUG:boto:Token: None 2024-04-16T17:56:53.946 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.946 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8= 2024-04-16T17:56:53.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8=', 'Content-Length': '0'} 2024-04-16T17:56:53.949 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000930f922c7a4f68c0-00661ebbe5-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:53.990 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:53.991 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:53.991 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.991 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.991 DEBUG:boto:Method: GET 2024-04-16T17:56:53.991 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.991 DEBUG:boto:Data: 2024-04-16T17:56:53.991 DEBUG:boto:Headers: {} 2024-04-16T17:56:53.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:53.991 DEBUG:boto:Port: 8000 2024-04-16T17:56:53.991 DEBUG:boto:Params: {} 2024-04-16T17:56:53.991 DEBUG:boto:Token: None 2024-04-16T17:56:53.992 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:53 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:53.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8= 2024-04-16T17:56:53.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I8/pd9XBY24BTRbY6h6B2lJ3Tr8=', 'Content-Length': '0'} 2024-04-16T17:56:53.995 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000ab653b6e58b2489-00661ebbe5-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.042 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.043 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.043 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.043 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.043 DEBUG:boto:Method: GET 2024-04-16T17:56:54.043 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.043 DEBUG:boto:Data: 2024-04-16T17:56:54.043 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.043 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.043 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.043 DEBUG:boto:Params: {} 2024-04-16T17:56:54.043 DEBUG:boto:Token: None 2024-04-16T17:56:54.044 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.048 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001aafe8363aad6047-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.048 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.049 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.049 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.049 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.049 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.049 DEBUG:boto:Method: GET 2024-04-16T17:56:54.049 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.049 DEBUG:boto:Data: 2024-04-16T17:56:54.049 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.050 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.050 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.050 DEBUG:boto:Params: {} 2024-04-16T17:56:54.050 DEBUG:boto:Token: None 2024-04-16T17:56:54.050 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.050 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.054 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2492b5dfa5a673c-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.098 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.099 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.099 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.099 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.099 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.099 DEBUG:boto:Data: 2024-04-16T17:56:54.099 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.099 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.099 DEBUG:boto:Params: {} 2024-04-16T17:56:54.099 DEBUG:boto:Token: None 2024-04-16T17:56:54.100 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.106 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007f6bf51893a26f2c-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.107 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.107 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.107 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.107 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.107 DEBUG:boto:Data: 2024-04-16T17:56:54.107 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.107 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.107 DEBUG:boto:Params: {} 2024-04-16T17:56:54.107 DEBUG:boto:Token: None 2024-04-16T17:56:54.108 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.114 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000903a7296198a40d7-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.114 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.114 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.114 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.114 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.114 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.115 DEBUG:boto:Method: GET 2024-04-16T17:56:54.115 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.115 DEBUG:boto:Data: 2024-04-16T17:56:54.115 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.115 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.115 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.115 DEBUG:boto:Params: {} 2024-04-16T17:56:54.115 DEBUG:boto:Token: None 2024-04-16T17:56:54.115 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.115 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.118 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000079918fa4209677fe-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.119 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.119 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.119 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.119 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.119 DEBUG:boto:Method: GET 2024-04-16T17:56:54.119 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.119 DEBUG:boto:Data: 2024-04-16T17:56:54.119 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.119 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.119 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.119 DEBUG:boto:Params: {} 2024-04-16T17:56:54.120 DEBUG:boto:Token: None 2024-04-16T17:56:54.120 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.123 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000054568ed52c1da5ac-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.170 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.170 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.171 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.171 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.171 DEBUG:boto:Method: GET 2024-04-16T17:56:54.171 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.171 DEBUG:boto:Data: 2024-04-16T17:56:54.171 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.171 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.171 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.171 DEBUG:boto:Params: {} 2024-04-16T17:56:54.171 DEBUG:boto:Token: None 2024-04-16T17:56:54.171 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072023995981de2e6-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.176 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.176 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.176 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.176 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.177 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.177 DEBUG:boto:Method: GET 2024-04-16T17:56:54.177 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.177 DEBUG:boto:Data: 2024-04-16T17:56:54.177 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.177 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.177 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.177 DEBUG:boto:Params: {} 2024-04-16T17:56:54.177 DEBUG:boto:Token: None 2024-04-16T17:56:54.177 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.178 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.182 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057dbbcc6d71fc73d-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.222 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.223 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.223 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.223 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.223 DEBUG:boto:path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.223 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.223 DEBUG:boto:Method: GET 2024-04-16T17:56:54.223 DEBUG:boto:Path: /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.223 DEBUG:boto:Data: 2024-04-16T17:56:54.223 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.224 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.224 DEBUG:boto:Params: {} 2024-04-16T17:56:54.224 DEBUG:boto:Token: None 2024-04-16T17:56:54.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA= 2024-04-16T17:56:54.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA=', 'Content-Length': '0'} 2024-04-16T17:56:54.228 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:38 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b2bd708a9b5c2c10-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.270 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.271 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.271 DEBUG:boto:path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.271 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.271 DEBUG:boto:Method: GET 2024-04-16T17:56:54.271 DEBUG:boto:Path: /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.271 DEBUG:boto:Data: 2024-04-16T17:56:54.271 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.271 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.271 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.271 DEBUG:boto:Params: {} 2024-04-16T17:56:54.271 DEBUG:boto:Token: None 2024-04-16T17:56:54.272 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.272 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA= 2024-04-16T17:56:54.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA=', 'Content-Length': '0'} 2024-04-16T17:56:54.274 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005a27962c514b916f-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.275 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.275 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.275 DEBUG:boto:path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.275 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.275 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.275 DEBUG:boto:Path: /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.275 DEBUG:boto:Data: 2024-04-16T17:56:54.275 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.275 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.276 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.276 DEBUG:boto:Params: {} 2024-04-16T17:56:54.276 DEBUG:boto:Token: None 2024-04-16T17:56:54.276 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8eh47/RocYD/jiwiPupiQw+HJok= 2024-04-16T17:56:54.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8eh47/RocYD/jiwiPupiQw+HJok=', 'Content-Length': '0'} 2024-04-16T17:56:54.279 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:38 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e0593d930451bdc0-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.279 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.279 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.279 DEBUG:boto:path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.279 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.279 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.279 DEBUG:boto:Path: /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.279 DEBUG:boto:Data: 2024-04-16T17:56:54.280 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.280 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.280 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.280 DEBUG:boto:Params: {} 2024-04-16T17:56:54.280 DEBUG:boto:Token: None 2024-04-16T17:56:54.280 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8eh47/RocYD/jiwiPupiQw+HJok= 2024-04-16T17:56:54.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8eh47/RocYD/jiwiPupiQw+HJok=', 'Content-Length': '0'} 2024-04-16T17:56:54.283 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d25c5c83120d2f21-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.283 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.283 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.283 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.283 DEBUG:boto:path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.283 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.283 DEBUG:boto:Method: GET 2024-04-16T17:56:54.284 DEBUG:boto:Path: /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.284 DEBUG:boto:Data: 2024-04-16T17:56:54.284 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.284 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.284 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.284 DEBUG:boto:Params: {} 2024-04-16T17:56:54.284 DEBUG:boto:Token: None 2024-04-16T17:56:54.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA= 2024-04-16T17:56:54.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA=', 'Content-Length': '0'} 2024-04-16T17:56:54.287 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:38 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b728c2364d2cd051-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.330 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.330 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.331 DEBUG:boto:path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.331 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.331 DEBUG:boto:Method: GET 2024-04-16T17:56:54.331 DEBUG:boto:Path: /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.331 DEBUG:boto:Data: 2024-04-16T17:56:54.331 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.331 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.331 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.331 DEBUG:boto:Params: {} 2024-04-16T17:56:54.331 DEBUG:boto:Token: None 2024-04-16T17:56:54.332 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=xVk3GPd16PUPBBjIR.27F0BQoWuNXbB 2024-04-16T17:56:54.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA= 2024-04-16T17:56:54.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GpP4EE6Z87XxFoaURIgR05Kw6FA=', 'Content-Length': '0'} 2024-04-16T17:56:54.335 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'xVk3GPd16PUPBBjIR.27F0BQoWuNXbB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a48cb964138e78e1-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.335 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.335 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.335 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.335 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.335 DEBUG:boto:Method: GET 2024-04-16T17:56:54.335 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.335 DEBUG:boto:Data: 2024-04-16T17:56:54.336 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.336 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.336 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.336 DEBUG:boto:Params: {} 2024-04-16T17:56:54.336 DEBUG:boto:Token: None 2024-04-16T17:56:54.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.340 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004dff3a12a67f9772-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.340 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.341 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.341 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.341 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.341 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.341 DEBUG:boto:Method: GET 2024-04-16T17:56:54.341 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.341 DEBUG:boto:Data: 2024-04-16T17:56:54.341 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.341 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.341 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.341 DEBUG:boto:Params: {} 2024-04-16T17:56:54.342 DEBUG:boto:Token: None 2024-04-16T17:56:54.342 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.342 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.346 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011970a0adfccdbdf-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.346 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.346 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.346 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.347 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.347 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.347 DEBUG:boto:Data: 2024-04-16T17:56:54.347 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.347 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.347 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.347 DEBUG:boto:Params: {} 2024-04-16T17:56:54.347 DEBUG:boto:Token: None 2024-04-16T17:56:54.347 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.347 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.353 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000dea6b86fac979f2-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.354 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.354 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.354 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.355 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.355 DEBUG:boto:Data: 2024-04-16T17:56:54.355 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.355 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.355 DEBUG:boto:Params: {} 2024-04-16T17:56:54.356 DEBUG:boto:Token: None 2024-04-16T17:56:54.356 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.362 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000085de2b95e2886e91-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.362 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.362 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.362 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.362 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.363 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.363 DEBUG:boto:Method: GET 2024-04-16T17:56:54.363 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.363 DEBUG:boto:Data: 2024-04-16T17:56:54.363 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.363 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.363 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.363 DEBUG:boto:Params: {} 2024-04-16T17:56:54.363 DEBUG:boto:Token: None 2024-04-16T17:56:54.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.363 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.366 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ddd5527648dfedc9-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.410 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.411 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.411 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.411 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.411 DEBUG:boto:Method: GET 2024-04-16T17:56:54.411 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.411 DEBUG:boto:Data: 2024-04-16T17:56:54.411 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.411 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.411 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.411 DEBUG:boto:Params: {} 2024-04-16T17:56:54.411 DEBUG:boto:Token: None 2024-04-16T17:56:54.412 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.415 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000847111c8a4cb10d9-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.458 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.459 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.459 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.459 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.459 DEBUG:boto:Method: GET 2024-04-16T17:56:54.459 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.459 DEBUG:boto:Data: 2024-04-16T17:56:54.459 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.459 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.459 DEBUG:boto:Params: {} 2024-04-16T17:56:54.460 DEBUG:boto:Token: None 2024-04-16T17:56:54.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011dc2e49d555935c-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.464 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.465 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.465 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.465 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.465 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.465 DEBUG:boto:Method: GET 2024-04-16T17:56:54.465 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.465 DEBUG:boto:Data: 2024-04-16T17:56:54.465 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.465 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.465 DEBUG:boto:Params: {} 2024-04-16T17:56:54.466 DEBUG:boto:Token: None 2024-04-16T17:56:54.466 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000300c5cc8370dde64-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.470 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.471 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.471 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.471 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.471 DEBUG:boto:path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.471 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.471 DEBUG:boto:Method: GET 2024-04-16T17:56:54.471 DEBUG:boto:Path: /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.471 DEBUG:boto:Data: 2024-04-16T17:56:54.471 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.472 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.472 DEBUG:boto:Params: {} 2024-04-16T17:56:54.472 DEBUG:boto:Token: None 2024-04-16T17:56:54.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk= 2024-04-16T17:56:54.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk=', 'Content-Length': '0'} 2024-04-16T17:56:54.475 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d70e5c6803735b24-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.475 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.475 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.476 DEBUG:boto:path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.476 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.476 DEBUG:boto:Method: GET 2024-04-16T17:56:54.476 DEBUG:boto:Path: /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.476 DEBUG:boto:Data: 2024-04-16T17:56:54.476 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.476 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.476 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.476 DEBUG:boto:Params: {} 2024-04-16T17:56:54.476 DEBUG:boto:Token: None 2024-04-16T17:56:54.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.477 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk= 2024-04-16T17:56:54.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk=', 'Content-Length': '0'} 2024-04-16T17:56:54.480 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004a206b050423d113-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.522 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.523 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.523 DEBUG:boto:path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.523 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.523 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.523 DEBUG:boto:Path: /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.523 DEBUG:boto:Data: 2024-04-16T17:56:54.523 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.523 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.523 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.523 DEBUG:boto:Params: {} 2024-04-16T17:56:54.523 DEBUG:boto:Token: None 2024-04-16T17:56:54.524 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/RU1HTMkPDIivcmSbHUb5nibXr8= 2024-04-16T17:56:54.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/RU1HTMkPDIivcmSbHUb5nibXr8=', 'Content-Length': '0'} 2024-04-16T17:56:54.526 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000067290986d1be466f-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.527 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.527 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.527 DEBUG:boto:path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.527 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.527 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.527 DEBUG:boto:Path: /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.527 DEBUG:boto:Data: 2024-04-16T17:56:54.527 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.527 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.527 DEBUG:boto:Params: {} 2024-04-16T17:56:54.528 DEBUG:boto:Token: None 2024-04-16T17:56:54.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/RU1HTMkPDIivcmSbHUb5nibXr8= 2024-04-16T17:56:54.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/RU1HTMkPDIivcmSbHUb5nibXr8=', 'Content-Length': '0'} 2024-04-16T17:56:54.531 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000882157cf6994e7a8-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.531 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.531 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.531 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.531 DEBUG:boto:path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.531 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.531 DEBUG:boto:Method: GET 2024-04-16T17:56:54.531 DEBUG:boto:Path: /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.532 DEBUG:boto:Data: 2024-04-16T17:56:54.532 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.532 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.532 DEBUG:boto:Params: {} 2024-04-16T17:56:54.532 DEBUG:boto:Token: None 2024-04-16T17:56:54.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk= 2024-04-16T17:56:54.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk=', 'Content-Length': '0'} 2024-04-16T17:56:54.535 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000514b69a116db4585-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.535 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.535 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.535 DEBUG:boto:path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.536 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.536 DEBUG:boto:Method: GET 2024-04-16T17:56:54.536 DEBUG:boto:Path: /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.536 DEBUG:boto:Data: 2024-04-16T17:56:54.536 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.536 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.536 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.536 DEBUG:boto:Params: {} 2024-04-16T17:56:54.536 DEBUG:boto:Token: None 2024-04-16T17:56:54.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4 2024-04-16T17:56:54.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk= 2024-04-16T17:56:54.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q0pr8kBGj4IbttcYBUym3i7enkk=', 'Content-Length': '0'} 2024-04-16T17:56:54.539 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5o5Drj.94Di6JW3pmOcIvEdyCo-VeN4'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ac8336de830475f5-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.582 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.582 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.583 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.583 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.583 DEBUG:boto:Method: GET 2024-04-16T17:56:54.583 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.583 DEBUG:boto:Data: 2024-04-16T17:56:54.583 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.583 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.583 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.583 DEBUG:boto:Params: {} 2024-04-16T17:56:54.583 DEBUG:boto:Token: None 2024-04-16T17:56:54.583 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.588 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b200b2464f705fa7-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.588 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.588 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.588 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.589 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.589 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.589 DEBUG:boto:Method: GET 2024-04-16T17:56:54.589 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.589 DEBUG:boto:Data: 2024-04-16T17:56:54.589 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.589 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.589 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.589 DEBUG:boto:Params: {} 2024-04-16T17:56:54.589 DEBUG:boto:Token: None 2024-04-16T17:56:54.589 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.590 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.593 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b24d632724388a7-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.594 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.594 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.594 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.594 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.594 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.594 DEBUG:boto:Data: 2024-04-16T17:56:54.594 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.595 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.595 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.595 DEBUG:boto:Params: {} 2024-04-16T17:56:54.595 DEBUG:boto:Token: None 2024-04-16T17:56:54.595 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.601 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006259e23f6cc35055-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.601 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.602 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.602 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.602 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.602 DEBUG:boto:Data: 2024-04-16T17:56:54.602 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.602 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.602 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.602 DEBUG:boto:Params: {} 2024-04-16T17:56:54.602 DEBUG:boto:Token: None 2024-04-16T17:56:54.602 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.603 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.609 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e4a973d14f432a18-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.609 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.609 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.609 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.610 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.610 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.610 DEBUG:boto:Method: GET 2024-04-16T17:56:54.610 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.610 DEBUG:boto:Data: 2024-04-16T17:56:54.610 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.610 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.610 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.610 DEBUG:boto:Params: {} 2024-04-16T17:56:54.610 DEBUG:boto:Token: None 2024-04-16T17:56:54.611 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.611 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.614 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000961d39c45904a5d2-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.654 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.655 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.655 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.655 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.655 DEBUG:boto:Method: GET 2024-04-16T17:56:54.655 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.655 DEBUG:boto:Data: 2024-04-16T17:56:54.655 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.655 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.655 DEBUG:boto:Params: {} 2024-04-16T17:56:54.656 DEBUG:boto:Token: None 2024-04-16T17:56:54.656 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.659 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008a5038e708510130-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.702 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.703 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.703 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.703 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.703 DEBUG:boto:Method: GET 2024-04-16T17:56:54.703 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.703 DEBUG:boto:Data: 2024-04-16T17:56:54.703 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.703 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.703 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.703 DEBUG:boto:Params: {} 2024-04-16T17:56:54.703 DEBUG:boto:Token: None 2024-04-16T17:56:54.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004dffeef80345524a-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.709 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.709 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.709 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.709 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.709 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.709 DEBUG:boto:Method: GET 2024-04-16T17:56:54.709 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.709 DEBUG:boto:Data: 2024-04-16T17:56:54.710 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.710 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.710 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.710 DEBUG:boto:Params: {} 2024-04-16T17:56:54.710 DEBUG:boto:Token: None 2024-04-16T17:56:54.710 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.710 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045b68b108feef25f-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.758 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.759 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.759 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.759 DEBUG:boto:path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.759 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.759 DEBUG:boto:Method: GET 2024-04-16T17:56:54.759 DEBUG:boto:Path: /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.759 DEBUG:boto:Data: 2024-04-16T17:56:54.759 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.759 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.760 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.760 DEBUG:boto:Params: {} 2024-04-16T17:56:54.760 DEBUG:boto:Token: None 2024-04-16T17:56:54.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs= 2024-04-16T17:56:54.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs=', 'Content-Length': '0'} 2024-04-16T17:56:54.763 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:37 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000562399441db9789b-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.806 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.806 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.806 DEBUG:boto:path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.807 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.807 DEBUG:boto:Method: GET 2024-04-16T17:56:54.807 DEBUG:boto:Path: /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.807 DEBUG:boto:Data: 2024-04-16T17:56:54.807 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.807 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.807 DEBUG:boto:Params: {} 2024-04-16T17:56:54.808 DEBUG:boto:Token: None 2024-04-16T17:56:54.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs= 2024-04-16T17:56:54.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs=', 'Content-Length': '0'} 2024-04-16T17:56:54.811 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001c0757265bb84307-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.811 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.811 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.811 DEBUG:boto:path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.811 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.811 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.812 DEBUG:boto:Path: /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.812 DEBUG:boto:Data: 2024-04-16T17:56:54.812 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.812 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.812 DEBUG:boto:Params: {} 2024-04-16T17:56:54.812 DEBUG:boto:Token: None 2024-04-16T17:56:54.812 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vG60AcCtILzS1avXemdRkWZwaPU= 2024-04-16T17:56:54.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vG60AcCtILzS1avXemdRkWZwaPU=', 'Content-Length': '0'} 2024-04-16T17:56:54.816 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:37 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000098fd89b0f6c1c8a2-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.816 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.816 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.816 DEBUG:boto:path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.816 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.816 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.816 DEBUG:boto:Path: /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.816 DEBUG:boto:Data: 2024-04-16T17:56:54.817 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.817 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.817 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.817 DEBUG:boto:Params: {} 2024-04-16T17:56:54.817 DEBUG:boto:Token: None 2024-04-16T17:56:54.817 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.817 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vG60AcCtILzS1avXemdRkWZwaPU= 2024-04-16T17:56:54.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vG60AcCtILzS1avXemdRkWZwaPU=', 'Content-Length': '0'} 2024-04-16T17:56:54.820 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001388de7caf6a5b2c-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.820 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.820 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.821 DEBUG:boto:path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.821 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.821 DEBUG:boto:Method: GET 2024-04-16T17:56:54.821 DEBUG:boto:Path: /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.821 DEBUG:boto:Data: 2024-04-16T17:56:54.821 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.821 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.821 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.821 DEBUG:boto:Params: {} 2024-04-16T17:56:54.821 DEBUG:boto:Token: None 2024-04-16T17:56:54.821 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.822 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs= 2024-04-16T17:56:54.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs=', 'Content-Length': '0'} 2024-04-16T17:56:54.825 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:37 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006d69449cfe8be4da-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.866 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.867 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.867 DEBUG:boto:path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.867 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.867 DEBUG:boto:Method: GET 2024-04-16T17:56:54.867 DEBUG:boto:Path: /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.867 DEBUG:boto:Data: 2024-04-16T17:56:54.867 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.867 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.867 DEBUG:boto:Params: {} 2024-04-16T17:56:54.867 DEBUG:boto:Token: None 2024-04-16T17:56:54.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=NxNMkltnMt-ticfCow-qcCHP0ay9YIn 2024-04-16T17:56:54.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs= 2024-04-16T17:56:54.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WnwR9tlnBHOaW2gceUiiowjmVjs=', 'Content-Length': '0'} 2024-04-16T17:56:54.870 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'NxNMkltnMt-ticfCow-qcCHP0ay9YIn'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000169274f1f82dfcc1-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.871 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.871 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.871 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.871 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.871 DEBUG:boto:Method: GET 2024-04-16T17:56:54.871 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.871 DEBUG:boto:Data: 2024-04-16T17:56:54.871 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.871 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.871 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.872 DEBUG:boto:Params: {} 2024-04-16T17:56:54.872 DEBUG:boto:Token: None 2024-04-16T17:56:54.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9e7466cb5e74073-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.918 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.919 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.919 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.919 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.919 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.919 DEBUG:boto:Method: GET 2024-04-16T17:56:54.919 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.919 DEBUG:boto:Data: 2024-04-16T17:56:54.919 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.919 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.920 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.920 DEBUG:boto:Params: {} 2024-04-16T17:56:54.920 DEBUG:boto:Token: None 2024-04-16T17:56:54.920 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.924 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005279e22e35fa467d-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.924 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.925 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.925 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.925 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.925 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.925 DEBUG:boto:Data: 2024-04-16T17:56:54.925 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.925 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.925 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.925 DEBUG:boto:Params: {} 2024-04-16T17:56:54.925 DEBUG:boto:Token: None 2024-04-16T17:56:54.925 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.926 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.932 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c53f08fcc5d5294c-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.932 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.932 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.932 DEBUG:boto:Method: HEAD 2024-04-16T17:56:54.932 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:54.932 DEBUG:boto:Data: 2024-04-16T17:56:54.933 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.933 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.933 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.933 DEBUG:boto:Params: {} 2024-04-16T17:56:54.933 DEBUG:boto:Token: None 2024-04-16T17:56:54.933 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj 2024-04-16T17:56:54.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw= 2024-04-16T17:56:54.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FTypGUjaCTde5pCkH5u6P1zyWZw=', 'Content-Length': '0'} 2024-04-16T17:56:54.939 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003aa7bafac92b3d04-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.939 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:54.939 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.939 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.940 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.940 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.940 DEBUG:boto:Method: GET 2024-04-16T17:56:54.940 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.940 DEBUG:boto:Data: 2024-04-16T17:56:54.940 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.940 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.940 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.940 DEBUG:boto:Params: {} 2024-04-16T17:56:54.940 DEBUG:boto:Token: None 2024-04-16T17:56:54.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.941 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.943 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a0a594f8ab1aeb4d-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.944 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:54.944 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:54.944 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.944 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.944 DEBUG:boto:Method: GET 2024-04-16T17:56:54.944 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.944 DEBUG:boto:Data: 2024-04-16T17:56:54.944 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.944 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.944 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.944 DEBUG:boto:Params: {} 2024-04-16T17:56:54.945 DEBUG:boto:Token: None 2024-04-16T17:56:54.945 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:54.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw= 2024-04-16T17:56:54.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JKFKovp9yCCV7UEVFav9yb94kzw=', 'Content-Length': '0'} 2024-04-16T17:56:54.948 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000040d7cc5b2bf4ec69-00661ebbe6-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.990 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.991 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.991 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.991 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.991 DEBUG:boto:Method: GET 2024-04-16T17:56:54.991 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.991 DEBUG:boto:Data: 2024-04-16T17:56:54.991 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.991 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:54.991 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.991 DEBUG:boto:Params: {} 2024-04-16T17:56:54.991 DEBUG:boto:Token: None 2024-04-16T17:56:54.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:54.996 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a1f159ff50f89e1-00661ebbe6-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:54.996 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:54.996 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:54.996 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:54.996 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.996 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:54.997 DEBUG:boto:Method: GET 2024-04-16T17:56:54.997 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:54.997 DEBUG:boto:Data: 2024-04-16T17:56:54.997 DEBUG:boto:Headers: {} 2024-04-16T17:56:54.997 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:54.997 DEBUG:boto:Port: 8000 2024-04-16T17:56:54.997 DEBUG:boto:Params: {} 2024-04-16T17:56:54.997 DEBUG:boto:Token: None 2024-04-16T17:56:54.997 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:54 GMT /lybtug-15/ 2024-04-16T17:56:54.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs= 2024-04-16T17:56:54.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjaFkIgzLVq5HAafU9ETKxwlWWs=', 'Content-Length': '0'} 2024-04-16T17:56:55.002 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000574fd390165737c7-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.002 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.003 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.003 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.003 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.003 DEBUG:boto:path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.003 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.003 DEBUG:boto:Method: GET 2024-04-16T17:56:55.003 DEBUG:boto:Path: /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.003 DEBUG:boto:Data: 2024-04-16T17:56:55.003 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.003 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.003 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.004 DEBUG:boto:Params: {} 2024-04-16T17:56:55.004 DEBUG:boto:Token: None 2024-04-16T17:56:55.004 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0= 2024-04-16T17:56:55.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0=', 'Content-Length': '0'} 2024-04-16T17:56:55.007 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:37 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007875225cc27349af-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.050 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.050 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.051 DEBUG:boto:path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.051 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.051 DEBUG:boto:Method: GET 2024-04-16T17:56:55.051 DEBUG:boto:Path: /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.051 DEBUG:boto:Data: 2024-04-16T17:56:55.051 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.051 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.051 DEBUG:boto:Params: {} 2024-04-16T17:56:55.051 DEBUG:boto:Token: None 2024-04-16T17:56:55.051 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0= 2024-04-16T17:56:55.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0=', 'Content-Length': '0'} 2024-04-16T17:56:55.055 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f71a7426f71957df-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.055 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.055 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.055 DEBUG:boto:path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.055 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.056 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.056 DEBUG:boto:Path: /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.056 DEBUG:boto:Data: 2024-04-16T17:56:55.056 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.056 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.056 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.056 DEBUG:boto:Params: {} 2024-04-16T17:56:55.056 DEBUG:boto:Token: None 2024-04-16T17:56:55.056 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lysH76gkyz2yUjbOa0gO/6GiT8I= 2024-04-16T17:56:55.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lysH76gkyz2yUjbOa0gO/6GiT8I=', 'Content-Length': '0'} 2024-04-16T17:56:55.060 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:37 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000839898c120493d50-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.060 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.060 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.060 DEBUG:boto:path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.060 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.060 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.060 DEBUG:boto:Path: /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.060 DEBUG:boto:Data: 2024-04-16T17:56:55.060 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.060 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.061 DEBUG:boto:Params: {} 2024-04-16T17:56:55.061 DEBUG:boto:Token: None 2024-04-16T17:56:55.061 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lysH76gkyz2yUjbOa0gO/6GiT8I= 2024-04-16T17:56:55.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lysH76gkyz2yUjbOa0gO/6GiT8I=', 'Content-Length': '0'} 2024-04-16T17:56:55.064 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000084fe4d929336b7ce-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.064 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.064 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.064 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.065 DEBUG:boto:path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.065 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.065 DEBUG:boto:Method: GET 2024-04-16T17:56:55.065 DEBUG:boto:Path: /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.065 DEBUG:boto:Data: 2024-04-16T17:56:55.065 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.065 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.065 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.065 DEBUG:boto:Params: {} 2024-04-16T17:56:55.065 DEBUG:boto:Token: None 2024-04-16T17:56:55.065 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.066 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0= 2024-04-16T17:56:55.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0=', 'Content-Length': '0'} 2024-04-16T17:56:55.069 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:37 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000573106944a8b4475-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.110 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.110 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.111 DEBUG:boto:path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.111 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.111 DEBUG:boto:Method: GET 2024-04-16T17:56:55.111 DEBUG:boto:Path: /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.111 DEBUG:boto:Data: 2024-04-16T17:56:55.111 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.111 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.111 DEBUG:boto:Params: {} 2024-04-16T17:56:55.111 DEBUG:boto:Token: None 2024-04-16T17:56:55.111 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=2qSUAJ6panT59CkpTafgVOmu.SunHYv 2024-04-16T17:56:55.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0= 2024-04-16T17:56:55.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkG3q42NJLnW4uDK+uxS6nFbWJ0=', 'Content-Length': '0'} 2024-04-16T17:56:55.115 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '2qSUAJ6panT59CkpTafgVOmu.SunHYv'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ad6391c492626f57-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.115 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.115 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.115 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.115 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.115 DEBUG:boto:Method: GET 2024-04-16T17:56:55.115 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.115 DEBUG:boto:Data: 2024-04-16T17:56:55.115 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.116 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.116 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.116 DEBUG:boto:Params: {} 2024-04-16T17:56:55.116 DEBUG:boto:Token: None 2024-04-16T17:56:55.116 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000276119c412416a58-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.162 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.163 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.163 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.163 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.163 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.163 DEBUG:boto:Method: GET 2024-04-16T17:56:55.163 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.163 DEBUG:boto:Data: 2024-04-16T17:56:55.163 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.163 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.164 DEBUG:boto:Params: {} 2024-04-16T17:56:55.164 DEBUG:boto:Token: None 2024-04-16T17:56:55.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddb036cb40fd6d35-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.168 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.169 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.169 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.169 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.169 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:55.169 DEBUG:boto:Data: 2024-04-16T17:56:55.169 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.169 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.169 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.169 DEBUG:boto:Params: {} 2024-04-16T17:56:55.170 DEBUG:boto:Token: None 2024-04-16T17:56:55.170 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj 2024-04-16T17:56:55.170 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4= 2024-04-16T17:56:55.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4=', 'Content-Length': '0'} 2024-04-16T17:56:55.176 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f815a50a49b9e524-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.176 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.176 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.176 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.176 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:55.176 DEBUG:boto:Data: 2024-04-16T17:56:55.176 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.176 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.177 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.177 DEBUG:boto:Params: {} 2024-04-16T17:56:55.177 DEBUG:boto:Token: None 2024-04-16T17:56:55.177 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj 2024-04-16T17:56:55.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4= 2024-04-16T17:56:55.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4=', 'Content-Length': '0'} 2024-04-16T17:56:55.184 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009edae5b96ed6c29b-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.184 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.184 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.184 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.184 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.184 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.184 DEBUG:boto:Method: GET 2024-04-16T17:56:55.185 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.185 DEBUG:boto:Data: 2024-04-16T17:56:55.185 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.185 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.185 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.185 DEBUG:boto:Params: {} 2024-04-16T17:56:55.185 DEBUG:boto:Token: None 2024-04-16T17:56:55.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00= 2024-04-16T17:56:55.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00=', 'Content-Length': '0'} 2024-04-16T17:56:55.189 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000001be5125e489e8c6-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.189 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.189 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.189 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.189 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.189 DEBUG:boto:Method: GET 2024-04-16T17:56:55.189 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.190 DEBUG:boto:Data: 2024-04-16T17:56:55.190 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.190 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.190 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.190 DEBUG:boto:Params: {} 2024-04-16T17:56:55.190 DEBUG:boto:Token: None 2024-04-16T17:56:55.190 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.190 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00= 2024-04-16T17:56:55.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00=', 'Content-Length': '0'} 2024-04-16T17:56:55.194 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a21bcfc4975e4fc5-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.234 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.234 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.235 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.235 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.235 DEBUG:boto:Method: GET 2024-04-16T17:56:55.235 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.235 DEBUG:boto:Data: 2024-04-16T17:56:55.235 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.235 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.235 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.235 DEBUG:boto:Params: {} 2024-04-16T17:56:55.235 DEBUG:boto:Token: None 2024-04-16T17:56:55.235 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000902889e66a4e435-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.240 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.241 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.241 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.241 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.241 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.241 DEBUG:boto:Method: GET 2024-04-16T17:56:55.241 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.241 DEBUG:boto:Data: 2024-04-16T17:56:55.241 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.241 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.241 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.242 DEBUG:boto:Params: {} 2024-04-16T17:56:55.242 DEBUG:boto:Token: None 2024-04-16T17:56:55.242 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.242 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.246 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a174c64e6d3def58-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.246 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.246 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.247 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.247 DEBUG:boto:path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.247 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.247 DEBUG:boto:Method: GET 2024-04-16T17:56:55.247 DEBUG:boto:Path: /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.247 DEBUG:boto:Data: 2024-04-16T17:56:55.247 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.247 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.247 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.247 DEBUG:boto:Params: {} 2024-04-16T17:56:55.247 DEBUG:boto:Token: None 2024-04-16T17:56:55.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io= 2024-04-16T17:56:55.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io=', 'Content-Length': '0'} 2024-04-16T17:56:55.250 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000008a57ad92db2d83e-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.294 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.294 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.294 DEBUG:boto:path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.295 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.295 DEBUG:boto:Method: GET 2024-04-16T17:56:55.295 DEBUG:boto:Path: /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.295 DEBUG:boto:Data: 2024-04-16T17:56:55.295 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.295 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.295 DEBUG:boto:Params: {} 2024-04-16T17:56:55.295 DEBUG:boto:Token: None 2024-04-16T17:56:55.295 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.295 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io= 2024-04-16T17:56:55.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io=', 'Content-Length': '0'} 2024-04-16T17:56:55.298 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ff199aa5d790d33-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.342 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.342 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.342 DEBUG:boto:path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.343 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.343 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.343 DEBUG:boto:Path: /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.343 DEBUG:boto:Data: 2024-04-16T17:56:55.343 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.343 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.343 DEBUG:boto:Params: {} 2024-04-16T17:56:55.343 DEBUG:boto:Token: None 2024-04-16T17:56:55.343 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2Fwpz0ziimYYTyjtGQQKgRDhoUg= 2024-04-16T17:56:55.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Fwpz0ziimYYTyjtGQQKgRDhoUg=', 'Content-Length': '0'} 2024-04-16T17:56:55.347 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009e31fad6cdd0f4f5-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.347 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.347 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.347 DEBUG:boto:path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.347 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.347 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.347 DEBUG:boto:Path: /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.347 DEBUG:boto:Data: 2024-04-16T17:56:55.347 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.348 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.348 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.348 DEBUG:boto:Params: {} 2024-04-16T17:56:55.348 DEBUG:boto:Token: None 2024-04-16T17:56:55.348 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2Fwpz0ziimYYTyjtGQQKgRDhoUg= 2024-04-16T17:56:55.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Fwpz0ziimYYTyjtGQQKgRDhoUg=', 'Content-Length': '0'} 2024-04-16T17:56:55.351 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000066dc975e1d3a8cec-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.351 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.351 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.351 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.351 DEBUG:boto:path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.351 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.352 DEBUG:boto:Method: GET 2024-04-16T17:56:55.352 DEBUG:boto:Path: /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.352 DEBUG:boto:Data: 2024-04-16T17:56:55.352 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.352 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.352 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.352 DEBUG:boto:Params: {} 2024-04-16T17:56:55.352 DEBUG:boto:Token: None 2024-04-16T17:56:55.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io= 2024-04-16T17:56:55.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io=', 'Content-Length': '0'} 2024-04-16T17:56:55.355 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000022fa3f5eaac453fd-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.355 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.356 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.356 DEBUG:boto:path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.356 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.356 DEBUG:boto:Method: GET 2024-04-16T17:56:55.356 DEBUG:boto:Path: /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.356 DEBUG:boto:Data: 2024-04-16T17:56:55.356 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.356 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.356 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.356 DEBUG:boto:Params: {} 2024-04-16T17:56:55.356 DEBUG:boto:Token: None 2024-04-16T17:56:55.357 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=jcONTBJ7GDxGwW4redxUYSvnuYmDy86 2024-04-16T17:56:55.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io= 2024-04-16T17:56:55.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6xrewFb351515Z8qMyXY45zH9Io=', 'Content-Length': '0'} 2024-04-16T17:56:55.360 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'jcONTBJ7GDxGwW4redxUYSvnuYmDy86'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000146a7675e164f6b3-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.402 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.402 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.403 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.403 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.403 DEBUG:boto:Method: GET 2024-04-16T17:56:55.403 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.403 DEBUG:boto:Data: 2024-04-16T17:56:55.403 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.403 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.403 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.403 DEBUG:boto:Params: {} 2024-04-16T17:56:55.403 DEBUG:boto:Token: None 2024-04-16T17:56:55.403 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.408 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012707659611256ef-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.408 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.408 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.408 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.409 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.409 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.409 DEBUG:boto:Method: GET 2024-04-16T17:56:55.409 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.409 DEBUG:boto:Data: 2024-04-16T17:56:55.409 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.409 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.409 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.409 DEBUG:boto:Params: {} 2024-04-16T17:56:55.409 DEBUG:boto:Token: None 2024-04-16T17:56:55.409 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016d66b77d2f9d1c3-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.414 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.414 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.414 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.414 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.414 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:55.414 DEBUG:boto:Data: 2024-04-16T17:56:55.415 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.415 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.415 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.415 DEBUG:boto:Params: {} 2024-04-16T17:56:55.415 DEBUG:boto:Token: None 2024-04-16T17:56:55.415 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj 2024-04-16T17:56:55.415 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4= 2024-04-16T17:56:55.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4=', 'Content-Length': '0'} 2024-04-16T17:56:55.421 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000121281edb1d66b56-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.421 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.422 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.422 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.422 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:55.422 DEBUG:boto:Data: 2024-04-16T17:56:55.422 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.422 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.422 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.422 DEBUG:boto:Params: {} 2024-04-16T17:56:55.422 DEBUG:boto:Token: None 2024-04-16T17:56:55.422 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj 2024-04-16T17:56:55.423 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4= 2024-04-16T17:56:55.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4=', 'Content-Length': '0'} 2024-04-16T17:56:55.429 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000069a1ea2c63cdb6-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.429 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.429 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.429 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.429 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.429 DEBUG:boto:Method: GET 2024-04-16T17:56:55.429 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.430 DEBUG:boto:Data: 2024-04-16T17:56:55.430 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.430 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.430 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.430 DEBUG:boto:Params: {} 2024-04-16T17:56:55.430 DEBUG:boto:Token: None 2024-04-16T17:56:55.430 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00= 2024-04-16T17:56:55.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00=', 'Content-Length': '0'} 2024-04-16T17:56:55.433 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ede7fa247a12c237-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.474 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.475 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.475 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.475 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.475 DEBUG:boto:Method: GET 2024-04-16T17:56:55.475 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.475 DEBUG:boto:Data: 2024-04-16T17:56:55.475 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.475 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.475 DEBUG:boto:Params: {} 2024-04-16T17:56:55.476 DEBUG:boto:Token: None 2024-04-16T17:56:55.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00= 2024-04-16T17:56:55.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00=', 'Content-Length': '0'} 2024-04-16T17:56:55.479 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d0c56c775f95e7ad-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.522 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.522 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.523 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.523 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.523 DEBUG:boto:Method: GET 2024-04-16T17:56:55.523 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.523 DEBUG:boto:Data: 2024-04-16T17:56:55.523 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.523 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.523 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.523 DEBUG:boto:Params: {} 2024-04-16T17:56:55.523 DEBUG:boto:Token: None 2024-04-16T17:56:55.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b82f372566e440e-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.528 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.529 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.529 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.529 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.529 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.529 DEBUG:boto:Method: GET 2024-04-16T17:56:55.529 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.530 DEBUG:boto:Data: 2024-04-16T17:56:55.530 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.530 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.530 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.530 DEBUG:boto:Params: {} 2024-04-16T17:56:55.530 DEBUG:boto:Token: None 2024-04-16T17:56:55.530 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.530 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8e43f0dbe412ee3-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.578 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.579 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.579 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.579 DEBUG:boto:path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.579 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.579 DEBUG:boto:Method: GET 2024-04-16T17:56:55.579 DEBUG:boto:Path: /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.579 DEBUG:boto:Data: 2024-04-16T17:56:55.579 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.579 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.580 DEBUG:boto:Params: {} 2024-04-16T17:56:55.580 DEBUG:boto:Token: None 2024-04-16T17:56:55.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M= 2024-04-16T17:56:55.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M=', 'Content-Length': '0'} 2024-04-16T17:56:55.584 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000424f0e2e9f1d2f8d-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.626 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.627 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.627 DEBUG:boto:path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.627 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.627 DEBUG:boto:Method: GET 2024-04-16T17:56:55.627 DEBUG:boto:Path: /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.627 DEBUG:boto:Data: 2024-04-16T17:56:55.627 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.627 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.627 DEBUG:boto:Params: {} 2024-04-16T17:56:55.628 DEBUG:boto:Token: None 2024-04-16T17:56:55.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M= 2024-04-16T17:56:55.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M=', 'Content-Length': '0'} 2024-04-16T17:56:55.631 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b367e1720467462a-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.631 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.631 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.632 DEBUG:boto:path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.632 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.632 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.632 DEBUG:boto:Path: /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.632 DEBUG:boto:Data: 2024-04-16T17:56:55.632 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.632 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.632 DEBUG:boto:Params: {} 2024-04-16T17:56:55.632 DEBUG:boto:Token: None 2024-04-16T17:56:55.633 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.633 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IyyT0+Bl+kPD9bveOUVpqSGBQ/c= 2024-04-16T17:56:55.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IyyT0+Bl+kPD9bveOUVpqSGBQ/c=', 'Content-Length': '0'} 2024-04-16T17:56:55.635 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cef905b7c1ebf254-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.636 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.636 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.636 DEBUG:boto:path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.636 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.636 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.636 DEBUG:boto:Path: /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.636 DEBUG:boto:Data: 2024-04-16T17:56:55.636 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.636 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.637 DEBUG:boto:Params: {} 2024-04-16T17:56:55.637 DEBUG:boto:Token: None 2024-04-16T17:56:55.637 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IyyT0+Bl+kPD9bveOUVpqSGBQ/c= 2024-04-16T17:56:55.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IyyT0+Bl+kPD9bveOUVpqSGBQ/c=', 'Content-Length': '0'} 2024-04-16T17:56:55.640 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7f21a2e0f63e52a-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.640 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.640 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.640 DEBUG:boto:path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.640 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.640 DEBUG:boto:Method: GET 2024-04-16T17:56:55.641 DEBUG:boto:Path: /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.641 DEBUG:boto:Data: 2024-04-16T17:56:55.641 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.641 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.641 DEBUG:boto:Params: {} 2024-04-16T17:56:55.641 DEBUG:boto:Token: None 2024-04-16T17:56:55.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M= 2024-04-16T17:56:55.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M=', 'Content-Length': '0'} 2024-04-16T17:56:55.644 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c46452421b1ae703-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.686 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.687 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.687 DEBUG:boto:path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.687 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.687 DEBUG:boto:Method: GET 2024-04-16T17:56:55.687 DEBUG:boto:Path: /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.687 DEBUG:boto:Data: 2024-04-16T17:56:55.687 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.687 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.687 DEBUG:boto:Params: {} 2024-04-16T17:56:55.688 DEBUG:boto:Token: None 2024-04-16T17:56:55.688 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=5g.23p6esxNXeTOkllHk-A-IislofLt 2024-04-16T17:56:55.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M= 2024-04-16T17:56:55.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EKrMTaH9eydnzkft/+DSlNSgg3M=', 'Content-Length': '0'} 2024-04-16T17:56:55.691 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', '5g.23p6esxNXeTOkllHk-A-IislofLt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a2be988c9335b91-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.691 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.692 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.692 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.692 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.692 DEBUG:boto:Method: GET 2024-04-16T17:56:55.692 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.692 DEBUG:boto:Data: 2024-04-16T17:56:55.692 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.692 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.692 DEBUG:boto:Params: {} 2024-04-16T17:56:55.692 DEBUG:boto:Token: None 2024-04-16T17:56:55.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.693 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.697 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014fb907ccba8124e-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.742 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.743 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.743 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.743 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.743 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.743 DEBUG:boto:Method: GET 2024-04-16T17:56:55.743 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.743 DEBUG:boto:Data: 2024-04-16T17:56:55.743 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.743 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.744 DEBUG:boto:Params: {} 2024-04-16T17:56:55.744 DEBUG:boto:Token: None 2024-04-16T17:56:55.744 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.748 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e0157d97c5863db-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.749 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.749 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.749 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.749 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.749 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:55.749 DEBUG:boto:Data: 2024-04-16T17:56:55.750 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.750 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.750 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.750 DEBUG:boto:Params: {} 2024-04-16T17:56:55.750 DEBUG:boto:Token: None 2024-04-16T17:56:55.750 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj 2024-04-16T17:56:55.750 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4= 2024-04-16T17:56:55.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4=', 'Content-Length': '0'} 2024-04-16T17:56:55.757 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005ad9ad5926a24c28-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.757 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.757 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.757 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.757 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:55.757 DEBUG:boto:Data: 2024-04-16T17:56:55.758 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.758 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.758 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.758 DEBUG:boto:Params: {} 2024-04-16T17:56:55.758 DEBUG:boto:Token: None 2024-04-16T17:56:55.758 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj 2024-04-16T17:56:55.758 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4= 2024-04-16T17:56:55.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UU2SZM5ZT6Sb+usGLSUh+4BygS4=', 'Content-Length': '0'} 2024-04-16T17:56:55.765 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b5200c124173bbd3-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.765 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.765 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.765 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.765 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.765 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.766 DEBUG:boto:Method: GET 2024-04-16T17:56:55.766 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.766 DEBUG:boto:Data: 2024-04-16T17:56:55.766 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.766 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.766 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.766 DEBUG:boto:Params: {} 2024-04-16T17:56:55.766 DEBUG:boto:Token: None 2024-04-16T17:56:55.766 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.766 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00= 2024-04-16T17:56:55.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00=', 'Content-Length': '0'} 2024-04-16T17:56:55.769 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c526b0afc99c3bf1-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.770 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.770 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.770 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.770 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.770 DEBUG:boto:Method: GET 2024-04-16T17:56:55.770 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.770 DEBUG:boto:Data: 2024-04-16T17:56:55.770 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.771 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.771 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.771 DEBUG:boto:Params: {} 2024-04-16T17:56:55.771 DEBUG:boto:Token: None 2024-04-16T17:56:55.771 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:55.771 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00= 2024-04-16T17:56:55.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxWg1lEf21G6fDWa5u4dRqCTW00=', 'Content-Length': '0'} 2024-04-16T17:56:55.774 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fdeaaf17fd73bb86-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.818 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.818 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.819 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.819 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.819 DEBUG:boto:Method: GET 2024-04-16T17:56:55.819 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.819 DEBUG:boto:Data: 2024-04-16T17:56:55.819 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.819 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.819 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.819 DEBUG:boto:Params: {} 2024-04-16T17:56:55.819 DEBUG:boto:Token: None 2024-04-16T17:56:55.820 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.824 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013f9f4ba2e7c41e7-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.824 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.824 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.825 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.825 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.825 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.825 DEBUG:boto:Method: GET 2024-04-16T17:56:55.825 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.825 DEBUG:boto:Data: 2024-04-16T17:56:55.825 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.825 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.825 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.825 DEBUG:boto:Params: {} 2024-04-16T17:56:55.826 DEBUG:boto:Token: None 2024-04-16T17:56:55.826 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.826 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032d06de8d87eb7c5-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.830 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.831 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.831 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.831 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.831 DEBUG:boto:path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.831 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.831 DEBUG:boto:Method: GET 2024-04-16T17:56:55.831 DEBUG:boto:Path: /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.831 DEBUG:boto:Data: 2024-04-16T17:56:55.831 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.831 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.832 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.832 DEBUG:boto:Params: {} 2024-04-16T17:56:55.832 DEBUG:boto:Token: None 2024-04-16T17:56:55.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8= 2024-04-16T17:56:55.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8=', 'Content-Length': '0'} 2024-04-16T17:56:55.835 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026aca7b787bc0c82-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.878 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.879 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.879 DEBUG:boto:path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.879 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.879 DEBUG:boto:Method: GET 2024-04-16T17:56:55.879 DEBUG:boto:Path: /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.879 DEBUG:boto:Data: 2024-04-16T17:56:55.879 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.879 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.879 DEBUG:boto:Params: {} 2024-04-16T17:56:55.879 DEBUG:boto:Token: None 2024-04-16T17:56:55.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8= 2024-04-16T17:56:55.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8=', 'Content-Length': '0'} 2024-04-16T17:56:55.883 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af9a6c02c7eb832e-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.926 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.927 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.927 DEBUG:boto:path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.927 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.927 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.927 DEBUG:boto:Path: /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.927 DEBUG:boto:Data: 2024-04-16T17:56:55.927 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.927 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.927 DEBUG:boto:Params: {} 2024-04-16T17:56:55.927 DEBUG:boto:Token: None 2024-04-16T17:56:55.928 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ah5S3l28qn65xlJqpg2jRWgkEQs= 2024-04-16T17:56:55.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ah5S3l28qn65xlJqpg2jRWgkEQs=', 'Content-Length': '0'} 2024-04-16T17:56:55.931 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002c6a4e353cc06b2c-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.931 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.931 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.931 DEBUG:boto:path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.931 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.931 DEBUG:boto:Method: HEAD 2024-04-16T17:56:55.931 DEBUG:boto:Path: /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.931 DEBUG:boto:Data: 2024-04-16T17:56:55.931 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.932 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.932 DEBUG:boto:Params: {} 2024-04-16T17:56:55.932 DEBUG:boto:Token: None 2024-04-16T17:56:55.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ah5S3l28qn65xlJqpg2jRWgkEQs= 2024-04-16T17:56:55.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ah5S3l28qn65xlJqpg2jRWgkEQs=', 'Content-Length': '0'} 2024-04-16T17:56:55.936 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089ccd7c1a7a0bb52-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.936 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:55.936 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.937 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.937 DEBUG:boto:path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.937 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.937 DEBUG:boto:Method: GET 2024-04-16T17:56:55.937 DEBUG:boto:Path: /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.937 DEBUG:boto:Data: 2024-04-16T17:56:55.937 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.937 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.937 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.937 DEBUG:boto:Params: {} 2024-04-16T17:56:55.937 DEBUG:boto:Token: None 2024-04-16T17:56:55.937 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.938 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8= 2024-04-16T17:56:55.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8=', 'Content-Length': '0'} 2024-04-16T17:56:55.944 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001283e9565a890b25-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.944 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:55.945 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:55.945 DEBUG:boto:path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.945 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.945 DEBUG:boto:Method: GET 2024-04-16T17:56:55.945 DEBUG:boto:Path: /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.945 DEBUG:boto:Data: 2024-04-16T17:56:55.945 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.945 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.945 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.945 DEBUG:boto:Params: {} 2024-04-16T17:56:55.945 DEBUG:boto:Token: None 2024-04-16T17:56:55.946 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/obj?versionId=v.A7JAAtanjXLBS8soduLW6h.mvqob6 2024-04-16T17:56:55.946 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8= 2024-04-16T17:56:55.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZIdcyDI1u+YC7QAFuIvBRYDNfY8=', 'Content-Length': '0'} 2024-04-16T17:56:55.949 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'v.A7JAAtanjXLBS8soduLW6h.mvqob6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000048cac4454d0d5c34-00661ebbe7-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.990 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.991 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.991 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.991 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.991 DEBUG:boto:Method: GET 2024-04-16T17:56:55.991 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.991 DEBUG:boto:Data: 2024-04-16T17:56:55.991 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.991 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:55.991 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.991 DEBUG:boto:Params: {} 2024-04-16T17:56:55.991 DEBUG:boto:Token: None 2024-04-16T17:56:55.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:55.996 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065712f20bdd6f130-00661ebbe7-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:55.996 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:55.996 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:55.997 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:55.997 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.997 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:55.997 DEBUG:boto:Method: GET 2024-04-16T17:56:55.997 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:55.997 DEBUG:boto:Data: 2024-04-16T17:56:55.997 DEBUG:boto:Headers: {} 2024-04-16T17:56:55.997 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:55.997 DEBUG:boto:Port: 8000 2024-04-16T17:56:55.997 DEBUG:boto:Params: {} 2024-04-16T17:56:55.997 DEBUG:boto:Token: None 2024-04-16T17:56:55.997 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:55 GMT /lybtug-15/ 2024-04-16T17:56:55.998 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac= 2024-04-16T17:56:55.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FYoWYRE9VsoO1V8UUIAR266xdac=', 'Content-Length': '0'} 2024-04-16T17:56:56.002 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e5c67ba2fcadc9d-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.002 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.002 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.002 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.003 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.003 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.003 DEBUG:boto:Data: 2024-04-16T17:56:56.003 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.003 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.003 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.003 DEBUG:boto:Params: {} 2024-04-16T17:56:56.003 DEBUG:boto:Token: None 2024-04-16T17:56:56.003 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.009 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f88b931b44a36987-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.009 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.010 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.010 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.010 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.010 DEBUG:boto:Data: 2024-04-16T17:56:56.010 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.010 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.010 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.010 DEBUG:boto:Params: {} 2024-04-16T17:56:56.010 DEBUG:boto:Token: None 2024-04-16T17:56:56.010 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.010 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.017 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d47e1afa7445665c-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.017 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.017 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.017 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.017 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.018 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.018 DEBUG:boto:Method: GET 2024-04-16T17:56:56.018 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.018 DEBUG:boto:Data: 2024-04-16T17:56:56.018 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.018 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.018 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.018 DEBUG:boto:Params: {} 2024-04-16T17:56:56.018 DEBUG:boto:Token: None 2024-04-16T17:56:56.018 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.018 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.021 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000042c197875c812d17-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.062 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.062 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.063 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.063 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.063 DEBUG:boto:Method: GET 2024-04-16T17:56:56.063 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.063 DEBUG:boto:Data: 2024-04-16T17:56:56.063 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.063 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.063 DEBUG:boto:Params: {} 2024-04-16T17:56:56.063 DEBUG:boto:Token: None 2024-04-16T17:56:56.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.066 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000850c3b25b7cf9d5a-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.110 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.111 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.111 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.111 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.111 DEBUG:boto:Method: GET 2024-04-16T17:56:56.111 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.111 DEBUG:boto:Data: 2024-04-16T17:56:56.111 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.111 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.111 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.111 DEBUG:boto:Params: {} 2024-04-16T17:56:56.112 DEBUG:boto:Token: None 2024-04-16T17:56:56.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.116 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008554c3e9eac3b3f2-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.117 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.117 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.117 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.117 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.117 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.117 DEBUG:boto:Method: GET 2024-04-16T17:56:56.117 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.117 DEBUG:boto:Data: 2024-04-16T17:56:56.118 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.118 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.118 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.118 DEBUG:boto:Params: {} 2024-04-16T17:56:56.118 DEBUG:boto:Token: None 2024-04-16T17:56:56.118 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.118 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.122 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000939c792e3d4035e5-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.166 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.167 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.167 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.167 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.167 DEBUG:boto:path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.167 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.167 DEBUG:boto:Method: GET 2024-04-16T17:56:56.167 DEBUG:boto:Path: /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.167 DEBUG:boto:Data: 2024-04-16T17:56:56.167 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.167 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.168 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.168 DEBUG:boto:Params: {} 2024-04-16T17:56:56.168 DEBUG:boto:Token: None 2024-04-16T17:56:56.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA= 2024-04-16T17:56:56.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA=', 'Content-Length': '0'} 2024-04-16T17:56:56.171 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000072d846cfdec2a62b-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.214 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.214 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.215 DEBUG:boto:path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.215 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.215 DEBUG:boto:Method: GET 2024-04-16T17:56:56.215 DEBUG:boto:Path: /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.215 DEBUG:boto:Data: 2024-04-16T17:56:56.215 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.215 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.215 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.215 DEBUG:boto:Params: {} 2024-04-16T17:56:56.215 DEBUG:boto:Token: None 2024-04-16T17:56:56.216 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA= 2024-04-16T17:56:56.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA=', 'Content-Length': '0'} 2024-04-16T17:56:56.219 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c33bc49a84e53ce8-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.219 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.219 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.220 DEBUG:boto:path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.220 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.220 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.220 DEBUG:boto:Path: /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.220 DEBUG:boto:Data: 2024-04-16T17:56:56.220 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.220 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.220 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.220 DEBUG:boto:Params: {} 2024-04-16T17:56:56.220 DEBUG:boto:Token: None 2024-04-16T17:56:56.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CF0XaSXOJFM3zzbIhmfWnAgpzf0= 2024-04-16T17:56:56.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CF0XaSXOJFM3zzbIhmfWnAgpzf0=', 'Content-Length': '0'} 2024-04-16T17:56:56.224 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b0bf809a962cfea5-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.226 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.226 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.226 DEBUG:boto:path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.227 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.227 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.227 DEBUG:boto:Path: /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.227 DEBUG:boto:Data: 2024-04-16T17:56:56.227 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.227 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.227 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.227 DEBUG:boto:Params: {} 2024-04-16T17:56:56.227 DEBUG:boto:Token: None 2024-04-16T17:56:56.227 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CF0XaSXOJFM3zzbIhmfWnAgpzf0= 2024-04-16T17:56:56.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CF0XaSXOJFM3zzbIhmfWnAgpzf0=', 'Content-Length': '0'} 2024-04-16T17:56:56.230 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000254a418dc70304f2-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.231 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.231 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.231 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.231 DEBUG:boto:path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.231 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.231 DEBUG:boto:Method: GET 2024-04-16T17:56:56.231 DEBUG:boto:Path: /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.231 DEBUG:boto:Data: 2024-04-16T17:56:56.231 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.232 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.232 DEBUG:boto:Params: {} 2024-04-16T17:56:56.232 DEBUG:boto:Token: None 2024-04-16T17:56:56.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA= 2024-04-16T17:56:56.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA=', 'Content-Length': '0'} 2024-04-16T17:56:56.235 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000356ef1dfb7d844db-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.278 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.279 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.279 DEBUG:boto:path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.279 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.279 DEBUG:boto:Method: GET 2024-04-16T17:56:56.279 DEBUG:boto:Path: /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.279 DEBUG:boto:Data: 2024-04-16T17:56:56.279 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.279 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.279 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.279 DEBUG:boto:Params: {} 2024-04-16T17:56:56.279 DEBUG:boto:Token: None 2024-04-16T17:56:56.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz 2024-04-16T17:56:56.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA= 2024-04-16T17:56:56.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MxBsCcse2T6CuUD2k1dyo66boNA=', 'Content-Length': '0'} 2024-04-16T17:56:56.283 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', '8QpLZjKQt.pZSv4y1ho6q1SCtJnBgAz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000a0dfe0b110b5c24-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.283 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.283 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.283 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.284 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.284 DEBUG:boto:Method: GET 2024-04-16T17:56:56.284 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.284 DEBUG:boto:Data: 2024-04-16T17:56:56.284 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.284 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.284 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.284 DEBUG:boto:Params: {} 2024-04-16T17:56:56.284 DEBUG:boto:Token: None 2024-04-16T17:56:56.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.289 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc74139a1f992cc7-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.334 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.335 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.335 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.335 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.335 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.335 DEBUG:boto:Method: GET 2024-04-16T17:56:56.335 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.335 DEBUG:boto:Data: 2024-04-16T17:56:56.335 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.335 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.336 DEBUG:boto:Params: {} 2024-04-16T17:56:56.336 DEBUG:boto:Token: None 2024-04-16T17:56:56.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.340 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d31ba6a62c6cf11-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.340 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.341 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.341 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.341 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.341 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.341 DEBUG:boto:Data: 2024-04-16T17:56:56.341 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.341 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.341 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.341 DEBUG:boto:Params: {} 2024-04-16T17:56:56.342 DEBUG:boto:Token: None 2024-04-16T17:56:56.342 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.342 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.348 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008aad89f563c4db41-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.349 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.349 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.349 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.349 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.349 DEBUG:boto:Data: 2024-04-16T17:56:56.349 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.349 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.349 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.349 DEBUG:boto:Params: {} 2024-04-16T17:56:56.349 DEBUG:boto:Token: None 2024-04-16T17:56:56.349 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.350 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.356 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000739c9c460f3ebd10-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.356 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.356 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.356 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.356 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.356 DEBUG:boto:Method: GET 2024-04-16T17:56:56.357 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.357 DEBUG:boto:Data: 2024-04-16T17:56:56.357 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.357 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.357 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.357 DEBUG:boto:Params: {} 2024-04-16T17:56:56.357 DEBUG:boto:Token: None 2024-04-16T17:56:56.357 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.360 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b512b5c6ea84280-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.361 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.361 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.361 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.361 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.361 DEBUG:boto:Method: GET 2024-04-16T17:56:56.361 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.361 DEBUG:boto:Data: 2024-04-16T17:56:56.361 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.361 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.361 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.362 DEBUG:boto:Params: {} 2024-04-16T17:56:56.362 DEBUG:boto:Token: None 2024-04-16T17:56:56.362 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.362 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.365 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002536e11c570f0f8f-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.406 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.406 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.407 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.407 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.407 DEBUG:boto:Method: GET 2024-04-16T17:56:56.407 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.407 DEBUG:boto:Data: 2024-04-16T17:56:56.407 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.407 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.407 DEBUG:boto:Params: {} 2024-04-16T17:56:56.407 DEBUG:boto:Token: None 2024-04-16T17:56:56.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007cfed71a89c00f8-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.412 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.412 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.413 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.413 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.413 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.413 DEBUG:boto:Method: GET 2024-04-16T17:56:56.413 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.413 DEBUG:boto:Data: 2024-04-16T17:56:56.413 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.413 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.413 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.413 DEBUG:boto:Params: {} 2024-04-16T17:56:56.414 DEBUG:boto:Token: None 2024-04-16T17:56:56.414 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.414 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e9aec031f8f6b26-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.418 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.419 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.419 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.419 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.419 DEBUG:boto:path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.419 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.419 DEBUG:boto:Method: GET 2024-04-16T17:56:56.420 DEBUG:boto:Path: /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.420 DEBUG:boto:Data: 2024-04-16T17:56:56.420 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.420 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.420 DEBUG:boto:Params: {} 2024-04-16T17:56:56.420 DEBUG:boto:Token: None 2024-04-16T17:56:56.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU= 2024-04-16T17:56:56.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU=', 'Content-Length': '0'} 2024-04-16T17:56:56.424 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000443bbbe44a6b487d-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.466 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.466 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.467 DEBUG:boto:path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.467 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.467 DEBUG:boto:Method: GET 2024-04-16T17:56:56.467 DEBUG:boto:Path: /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.467 DEBUG:boto:Data: 2024-04-16T17:56:56.467 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.467 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.467 DEBUG:boto:Params: {} 2024-04-16T17:56:56.468 DEBUG:boto:Token: None 2024-04-16T17:56:56.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU= 2024-04-16T17:56:56.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU=', 'Content-Length': '0'} 2024-04-16T17:56:56.471 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089da9a7fb642404c-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.514 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.514 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.515 DEBUG:boto:path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.515 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.515 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.515 DEBUG:boto:Path: /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.515 DEBUG:boto:Data: 2024-04-16T17:56:56.515 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.515 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.515 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.515 DEBUG:boto:Params: {} 2024-04-16T17:56:56.515 DEBUG:boto:Token: None 2024-04-16T17:56:56.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ck1h3mo/zX2tPtjT9KviADI7eE= 2024-04-16T17:56:56.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ck1h3mo/zX2tPtjT9KviADI7eE=', 'Content-Length': '0'} 2024-04-16T17:56:56.519 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e10b9c027e341932-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.519 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.519 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.519 DEBUG:boto:path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.519 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.519 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.519 DEBUG:boto:Path: /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.519 DEBUG:boto:Data: 2024-04-16T17:56:56.520 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.520 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.520 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.520 DEBUG:boto:Params: {} 2024-04-16T17:56:56.520 DEBUG:boto:Token: None 2024-04-16T17:56:56.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ck1h3mo/zX2tPtjT9KviADI7eE= 2024-04-16T17:56:56.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ck1h3mo/zX2tPtjT9KviADI7eE=', 'Content-Length': '0'} 2024-04-16T17:56:56.523 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b28e995e15b63383-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.524 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.524 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.524 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.524 DEBUG:boto:path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.524 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.524 DEBUG:boto:Method: GET 2024-04-16T17:56:56.524 DEBUG:boto:Path: /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.524 DEBUG:boto:Data: 2024-04-16T17:56:56.524 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.524 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.525 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.525 DEBUG:boto:Params: {} 2024-04-16T17:56:56.525 DEBUG:boto:Token: None 2024-04-16T17:56:56.525 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU= 2024-04-16T17:56:56.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU=', 'Content-Length': '0'} 2024-04-16T17:56:56.528 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000160a29a3923f0c04-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.528 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.529 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.529 DEBUG:boto:path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.529 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.529 DEBUG:boto:Method: GET 2024-04-16T17:56:56.529 DEBUG:boto:Path: /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.529 DEBUG:boto:Data: 2024-04-16T17:56:56.529 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.529 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.529 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.529 DEBUG:boto:Params: {} 2024-04-16T17:56:56.529 DEBUG:boto:Token: None 2024-04-16T17:56:56.530 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=Ti61G73jRSlmFy6mWBV7t7VETZRGeD1 2024-04-16T17:56:56.530 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU= 2024-04-16T17:56:56.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCRPORY1Cfp/X2RLylLqZMRRslU=', 'Content-Length': '0'} 2024-04-16T17:56:56.533 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'Ti61G73jRSlmFy6mWBV7t7VETZRGeD1'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000812a15bddf4c914e-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.574 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.575 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.575 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.575 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.575 DEBUG:boto:Method: GET 2024-04-16T17:56:56.575 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.575 DEBUG:boto:Data: 2024-04-16T17:56:56.575 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.575 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.575 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.575 DEBUG:boto:Params: {} 2024-04-16T17:56:56.575 DEBUG:boto:Token: None 2024-04-16T17:56:56.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c10a65501eee830-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.580 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.581 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.581 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.581 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.581 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.581 DEBUG:boto:Method: GET 2024-04-16T17:56:56.581 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.581 DEBUG:boto:Data: 2024-04-16T17:56:56.581 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.581 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.582 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.582 DEBUG:boto:Params: {} 2024-04-16T17:56:56.582 DEBUG:boto:Token: None 2024-04-16T17:56:56.582 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.582 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098807ff456a18225-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.587 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.587 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.588 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.588 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.588 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.588 DEBUG:boto:Data: 2024-04-16T17:56:56.588 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.588 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.588 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.588 DEBUG:boto:Params: {} 2024-04-16T17:56:56.588 DEBUG:boto:Token: None 2024-04-16T17:56:56.588 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.589 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.595 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af7910492c56d147-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.595 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.595 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.596 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.596 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.596 DEBUG:boto:Data: 2024-04-16T17:56:56.596 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.596 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.596 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.596 DEBUG:boto:Params: {} 2024-04-16T17:56:56.596 DEBUG:boto:Token: None 2024-04-16T17:56:56.596 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.602 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000acc9c685b2ec7183-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.603 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.603 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.603 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.603 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.603 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.603 DEBUG:boto:Method: GET 2024-04-16T17:56:56.603 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.603 DEBUG:boto:Data: 2024-04-16T17:56:56.603 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.603 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.604 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.604 DEBUG:boto:Params: {} 2024-04-16T17:56:56.604 DEBUG:boto:Token: None 2024-04-16T17:56:56.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.607 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b8842368f4554bb-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.650 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.650 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.651 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.651 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.651 DEBUG:boto:Method: GET 2024-04-16T17:56:56.651 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.651 DEBUG:boto:Data: 2024-04-16T17:56:56.651 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.651 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.651 DEBUG:boto:Params: {} 2024-04-16T17:56:56.651 DEBUG:boto:Token: None 2024-04-16T17:56:56.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.654 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008c99d78c5dc78fe0-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.702 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.703 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.703 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.703 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.703 DEBUG:boto:Method: GET 2024-04-16T17:56:56.703 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.703 DEBUG:boto:Data: 2024-04-16T17:56:56.703 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.703 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.703 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.703 DEBUG:boto:Params: {} 2024-04-16T17:56:56.703 DEBUG:boto:Token: None 2024-04-16T17:56:56.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003dd5442281841c31-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.709 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.709 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.709 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.709 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.709 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.709 DEBUG:boto:Method: GET 2024-04-16T17:56:56.710 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.710 DEBUG:boto:Data: 2024-04-16T17:56:56.710 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.710 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.710 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.710 DEBUG:boto:Params: {} 2024-04-16T17:56:56.710 DEBUG:boto:Token: None 2024-04-16T17:56:56.710 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.710 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091d592fc71f6139e-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.715 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.715 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.715 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.715 DEBUG:boto:path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.715 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.716 DEBUG:boto:Method: GET 2024-04-16T17:56:56.716 DEBUG:boto:Path: /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.716 DEBUG:boto:Data: 2024-04-16T17:56:56.716 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.716 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.716 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.716 DEBUG:boto:Params: {} 2024-04-16T17:56:56.716 DEBUG:boto:Token: None 2024-04-16T17:56:56.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI= 2024-04-16T17:56:56.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI=', 'Content-Length': '0'} 2024-04-16T17:56:56.719 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005497066cbbf2ffdf-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.720 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.720 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.720 DEBUG:boto:path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.720 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.720 DEBUG:boto:Method: GET 2024-04-16T17:56:56.720 DEBUG:boto:Path: /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.720 DEBUG:boto:Data: 2024-04-16T17:56:56.720 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.720 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.721 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.721 DEBUG:boto:Params: {} 2024-04-16T17:56:56.721 DEBUG:boto:Token: None 2024-04-16T17:56:56.721 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.721 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI= 2024-04-16T17:56:56.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI=', 'Content-Length': '0'} 2024-04-16T17:56:56.724 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002880043fb72e65b1-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.766 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.766 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.767 DEBUG:boto:path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.767 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.767 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.767 DEBUG:boto:Path: /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.767 DEBUG:boto:Data: 2024-04-16T17:56:56.767 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.767 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.767 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.767 DEBUG:boto:Params: {} 2024-04-16T17:56:56.767 DEBUG:boto:Token: None 2024-04-16T17:56:56.767 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kod9z2jlij9CNoQoNIg892h2CAI= 2024-04-16T17:56:56.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kod9z2jlij9CNoQoNIg892h2CAI=', 'Content-Length': '0'} 2024-04-16T17:56:56.771 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d08bc95150704c76-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.771 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.771 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.771 DEBUG:boto:path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.771 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.771 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.772 DEBUG:boto:Path: /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.772 DEBUG:boto:Data: 2024-04-16T17:56:56.772 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.772 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.772 DEBUG:boto:Params: {} 2024-04-16T17:56:56.772 DEBUG:boto:Token: None 2024-04-16T17:56:56.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kod9z2jlij9CNoQoNIg892h2CAI= 2024-04-16T17:56:56.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kod9z2jlij9CNoQoNIg892h2CAI=', 'Content-Length': '0'} 2024-04-16T17:56:56.775 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000012dff0f10d074dac-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.775 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.775 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.776 DEBUG:boto:path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.776 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.776 DEBUG:boto:Method: GET 2024-04-16T17:56:56.776 DEBUG:boto:Path: /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.776 DEBUG:boto:Data: 2024-04-16T17:56:56.776 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.776 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.776 DEBUG:boto:Params: {} 2024-04-16T17:56:56.776 DEBUG:boto:Token: None 2024-04-16T17:56:56.776 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI= 2024-04-16T17:56:56.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI=', 'Content-Length': '0'} 2024-04-16T17:56:56.779 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009849d1d4cbb90e52-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.780 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.780 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.780 DEBUG:boto:path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.780 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.780 DEBUG:boto:Method: GET 2024-04-16T17:56:56.780 DEBUG:boto:Path: /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.780 DEBUG:boto:Data: 2024-04-16T17:56:56.780 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.780 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.780 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.780 DEBUG:boto:Params: {} 2024-04-16T17:56:56.781 DEBUG:boto:Token: None 2024-04-16T17:56:56.781 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5 2024-04-16T17:56:56.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI= 2024-04-16T17:56:56.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4s5qnM5Oamt/HIit4bWxQf0jRpI=', 'Content-Length': '0'} 2024-04-16T17:56:56.784 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'UgUlGbQIYdfiGb9XWOfJzocNEtE6XC5'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b2754c3b1becfe4b-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.826 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.826 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.827 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.827 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.827 DEBUG:boto:Method: GET 2024-04-16T17:56:56.827 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.827 DEBUG:boto:Data: 2024-04-16T17:56:56.827 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.827 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.827 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.827 DEBUG:boto:Params: {} 2024-04-16T17:56:56.827 DEBUG:boto:Token: None 2024-04-16T17:56:56.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.832 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f68469c5047d6c9-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.832 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.833 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.833 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.833 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.833 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.833 DEBUG:boto:Method: GET 2024-04-16T17:56:56.833 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.833 DEBUG:boto:Data: 2024-04-16T17:56:56.833 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.833 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.833 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.833 DEBUG:boto:Params: {} 2024-04-16T17:56:56.833 DEBUG:boto:Token: None 2024-04-16T17:56:56.834 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.834 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c14df184c336ba1-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.838 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.839 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.839 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.839 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.839 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.839 DEBUG:boto:Data: 2024-04-16T17:56:56.839 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.839 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.839 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.839 DEBUG:boto:Params: {} 2024-04-16T17:56:56.839 DEBUG:boto:Token: None 2024-04-16T17:56:56.840 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.846 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b83b37f80ec4dfd9-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.847 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.847 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.847 DEBUG:boto:Method: HEAD 2024-04-16T17:56:56.847 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:56.847 DEBUG:boto:Data: 2024-04-16T17:56:56.847 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.847 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.847 DEBUG:boto:Params: {} 2024-04-16T17:56:56.847 DEBUG:boto:Token: None 2024-04-16T17:56:56.848 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj 2024-04-16T17:56:56.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0= 2024-04-16T17:56:56.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7u/ihgeaMT7xpPxzJnnuf7jhnJ0=', 'Content-Length': '0'} 2024-04-16T17:56:56.853 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ad1d1485ebeb3f7e-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.854 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.854 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.854 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.854 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.854 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.854 DEBUG:boto:Method: GET 2024-04-16T17:56:56.854 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.854 DEBUG:boto:Data: 2024-04-16T17:56:56.854 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.854 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.855 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.855 DEBUG:boto:Params: {} 2024-04-16T17:56:56.855 DEBUG:boto:Token: None 2024-04-16T17:56:56.855 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.855 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.858 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db643306311684e8-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.899 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.899 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.899 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.899 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.899 DEBUG:boto:Method: GET 2024-04-16T17:56:56.899 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.899 DEBUG:boto:Data: 2024-04-16T17:56:56.899 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.899 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.899 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.900 DEBUG:boto:Params: {} 2024-04-16T17:56:56.900 DEBUG:boto:Token: None 2024-04-16T17:56:56.900 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:56.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk= 2024-04-16T17:56:56.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KlmXPtiyTBn80J/zRlxsKqDrpxk=', 'Content-Length': '0'} 2024-04-16T17:56:56.903 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000869ef6ffc25a79fb-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.946 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.946 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.947 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.947 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.947 DEBUG:boto:Method: GET 2024-04-16T17:56:56.947 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.947 DEBUG:boto:Data: 2024-04-16T17:56:56.947 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.947 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.947 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.947 DEBUG:boto:Params: {} 2024-04-16T17:56:56.947 DEBUG:boto:Token: None 2024-04-16T17:56:56.947 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.947 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.951 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f7a64119a963315-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.952 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.952 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:56.952 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:56.952 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.952 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:56.952 DEBUG:boto:Method: GET 2024-04-16T17:56:56.952 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:56.952 DEBUG:boto:Data: 2024-04-16T17:56:56.952 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:56.953 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.953 DEBUG:boto:Params: {} 2024-04-16T17:56:56.953 DEBUG:boto:Token: None 2024-04-16T17:56:56.953 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/ 2024-04-16T17:56:56.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4= 2024-04-16T17:56:56.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z5aveCb325xonURPWhyINyPqHe4=', 'Content-Length': '0'} 2024-04-16T17:56:56.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ba64a6db62942d0-00661ebbe8-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:56.957 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:56.958 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:56.958 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:56.958 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:56.958 DEBUG:boto:path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:56.958 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:56.958 DEBUG:boto:Method: GET 2024-04-16T17:56:56.958 DEBUG:boto:Path: /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:56.958 DEBUG:boto:Data: 2024-04-16T17:56:56.958 DEBUG:boto:Headers: {} 2024-04-16T17:56:56.958 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:56.958 DEBUG:boto:Port: 8000 2024-04-16T17:56:56.958 DEBUG:boto:Params: {} 2024-04-16T17:56:56.959 DEBUG:boto:Token: None 2024-04-16T17:56:56.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:56 GMT /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:56.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XS36WdKhBxaBQZRILygrbcDFjVQ= 2024-04-16T17:56:56.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XS36WdKhBxaBQZRILygrbcDFjVQ=', 'Content-Length': '0'} 2024-04-16T17:56:56.962 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ead248bbfc04fd58-00661ebbe8-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.002 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.002 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.003 DEBUG:boto:path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.003 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.003 DEBUG:boto:Method: GET 2024-04-16T17:56:57.003 DEBUG:boto:Path: /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.003 DEBUG:boto:Data: 2024-04-16T17:56:57.003 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.003 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.003 DEBUG:boto:Params: {} 2024-04-16T17:56:57.003 DEBUG:boto:Token: None 2024-04-16T17:56:57.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/4sNGYJuFInNuYYLI5j+ZXKtQLI= 2024-04-16T17:56:57.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/4sNGYJuFInNuYYLI5j+ZXKtQLI=', 'Content-Length': '0'} 2024-04-16T17:56:57.007 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000039aa571e55d288c3-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.007 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.007 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.007 DEBUG:boto:path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.007 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.007 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.007 DEBUG:boto:Path: /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.007 DEBUG:boto:Data: 2024-04-16T17:56:57.007 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.008 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.008 DEBUG:boto:Params: {} 2024-04-16T17:56:57.008 DEBUG:boto:Token: None 2024-04-16T17:56:57.008 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6fjxhyMivY0YeGL2044/Xlij8U8= 2024-04-16T17:56:57.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6fjxhyMivY0YeGL2044/Xlij8U8=', 'Content-Length': '0'} 2024-04-16T17:56:57.011 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003140b73b5a791f84-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.011 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.011 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.011 DEBUG:boto:path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.012 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.012 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.012 DEBUG:boto:Path: /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.012 DEBUG:boto:Data: 2024-04-16T17:56:57.012 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.012 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.012 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.012 DEBUG:boto:Params: {} 2024-04-16T17:56:57.012 DEBUG:boto:Token: None 2024-04-16T17:56:57.012 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6fjxhyMivY0YeGL2044/Xlij8U8= 2024-04-16T17:56:57.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6fjxhyMivY0YeGL2044/Xlij8U8=', 'Content-Length': '0'} 2024-04-16T17:56:57.015 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a00e1ac97bcb4a76-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.015 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.015 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.016 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.016 DEBUG:boto:path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.016 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.016 DEBUG:boto:Method: GET 2024-04-16T17:56:57.016 DEBUG:boto:Path: /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.016 DEBUG:boto:Data: 2024-04-16T17:56:57.016 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.016 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.016 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.016 DEBUG:boto:Params: {} 2024-04-16T17:56:57.016 DEBUG:boto:Token: None 2024-04-16T17:56:57.017 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/4sNGYJuFInNuYYLI5j+ZXKtQLI= 2024-04-16T17:56:57.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/4sNGYJuFInNuYYLI5j+ZXKtQLI=', 'Content-Length': '0'} 2024-04-16T17:56:57.020 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cfdb27bcbab257ae-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.062 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.062 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.062 DEBUG:boto:path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.063 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.063 DEBUG:boto:Method: GET 2024-04-16T17:56:57.063 DEBUG:boto:Path: /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.063 DEBUG:boto:Data: 2024-04-16T17:56:57.063 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.063 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.063 DEBUG:boto:Params: {} 2024-04-16T17:56:57.063 DEBUG:boto:Token: None 2024-04-16T17:56:57.063 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=rC25L2Rwl6lKjQSpxdNI-R.WY975kFc 2024-04-16T17:56:57.063 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/4sNGYJuFInNuYYLI5j+ZXKtQLI= 2024-04-16T17:56:57.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/4sNGYJuFInNuYYLI5j+ZXKtQLI=', 'Content-Length': '0'} 2024-04-16T17:56:57.067 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'rC25L2Rwl6lKjQSpxdNI-R.WY975kFc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000002db377c80c30411-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.067 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.067 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.067 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.067 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.068 DEBUG:boto:Method: GET 2024-04-16T17:56:57.068 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.068 DEBUG:boto:Data: 2024-04-16T17:56:57.068 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.068 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.068 DEBUG:boto:Params: {} 2024-04-16T17:56:57.068 DEBUG:boto:Token: None 2024-04-16T17:56:57.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.073 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001718c4c859df6be-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.073 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.074 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.074 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.074 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.074 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.074 DEBUG:boto:Method: GET 2024-04-16T17:56:57.074 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.074 DEBUG:boto:Data: 2024-04-16T17:56:57.074 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.074 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.074 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.074 DEBUG:boto:Params: {} 2024-04-16T17:56:57.075 DEBUG:boto:Token: None 2024-04-16T17:56:57.075 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.075 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5a5f2ffad54a9a3-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.122 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.123 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.123 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.123 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.123 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.123 DEBUG:boto:Data: 2024-04-16T17:56:57.123 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.123 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.123 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.123 DEBUG:boto:Params: {} 2024-04-16T17:56:57.124 DEBUG:boto:Token: None 2024-04-16T17:56:57.124 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.130 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000032cf10c925726c83-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.131 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.131 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.131 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.131 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.131 DEBUG:boto:Data: 2024-04-16T17:56:57.131 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.131 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.131 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.131 DEBUG:boto:Params: {} 2024-04-16T17:56:57.131 DEBUG:boto:Token: None 2024-04-16T17:56:57.132 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.138 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009beed7a34b656b40-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.138 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.138 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.139 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.139 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.139 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.139 DEBUG:boto:Method: GET 2024-04-16T17:56:57.139 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.139 DEBUG:boto:Data: 2024-04-16T17:56:57.139 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.139 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.139 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.139 DEBUG:boto:Params: {} 2024-04-16T17:56:57.139 DEBUG:boto:Token: None 2024-04-16T17:56:57.139 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.143 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f294e821d1d84aca-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.143 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.143 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.143 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.143 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.143 DEBUG:boto:Method: GET 2024-04-16T17:56:57.143 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.143 DEBUG:boto:Data: 2024-04-16T17:56:57.144 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.144 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.144 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.144 DEBUG:boto:Params: {} 2024-04-16T17:56:57.144 DEBUG:boto:Token: None 2024-04-16T17:56:57.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.147 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000094b49903e683ab3b-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.190 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.191 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.191 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.191 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.191 DEBUG:boto:Method: GET 2024-04-16T17:56:57.191 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.191 DEBUG:boto:Data: 2024-04-16T17:56:57.191 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.191 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.191 DEBUG:boto:Params: {} 2024-04-16T17:56:57.191 DEBUG:boto:Token: None 2024-04-16T17:56:57.191 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000306e15e3d506b4e1-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.196 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.196 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.196 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.196 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.196 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.196 DEBUG:boto:Method: GET 2024-04-16T17:56:57.196 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.197 DEBUG:boto:Data: 2024-04-16T17:56:57.197 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.197 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.197 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.197 DEBUG:boto:Params: {} 2024-04-16T17:56:57.197 DEBUG:boto:Token: None 2024-04-16T17:56:57.197 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7665e4825d9c7e4-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.242 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.243 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.243 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.243 DEBUG:boto:path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.243 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.243 DEBUG:boto:Method: GET 2024-04-16T17:56:57.243 DEBUG:boto:Path: /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.243 DEBUG:boto:Data: 2024-04-16T17:56:57.243 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.244 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.244 DEBUG:boto:Params: {} 2024-04-16T17:56:57.244 DEBUG:boto:Token: None 2024-04-16T17:56:57.244 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80= 2024-04-16T17:56:57.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80=', 'Content-Length': '0'} 2024-04-16T17:56:57.247 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bc804d5fa4b14c4b-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.247 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.248 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.248 DEBUG:boto:path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.248 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.248 DEBUG:boto:Method: GET 2024-04-16T17:56:57.248 DEBUG:boto:Path: /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.248 DEBUG:boto:Data: 2024-04-16T17:56:57.248 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.248 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.248 DEBUG:boto:Params: {} 2024-04-16T17:56:57.248 DEBUG:boto:Token: None 2024-04-16T17:56:57.249 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80= 2024-04-16T17:56:57.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80=', 'Content-Length': '0'} 2024-04-16T17:56:57.252 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a0bc0591ad35c29e-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.298 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.299 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.299 DEBUG:boto:path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.299 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.299 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.299 DEBUG:boto:Path: /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.299 DEBUG:boto:Data: 2024-04-16T17:56:57.299 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.299 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.299 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.299 DEBUG:boto:Params: {} 2024-04-16T17:56:57.300 DEBUG:boto:Token: None 2024-04-16T17:56:57.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:syAex08jncKGuOtWe+q7kgd7VKE= 2024-04-16T17:56:57.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:syAex08jncKGuOtWe+q7kgd7VKE=', 'Content-Length': '0'} 2024-04-16T17:56:57.303 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd2fe168f03356b8-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.303 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.303 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.303 DEBUG:boto:path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.303 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.303 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.304 DEBUG:boto:Path: /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.304 DEBUG:boto:Data: 2024-04-16T17:56:57.304 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.304 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.304 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.304 DEBUG:boto:Params: {} 2024-04-16T17:56:57.304 DEBUG:boto:Token: None 2024-04-16T17:56:57.304 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:syAex08jncKGuOtWe+q7kgd7VKE= 2024-04-16T17:56:57.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:syAex08jncKGuOtWe+q7kgd7VKE=', 'Content-Length': '0'} 2024-04-16T17:56:57.307 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af83c00fa7603433-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.307 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.308 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.308 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.308 DEBUG:boto:path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.308 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.308 DEBUG:boto:Method: GET 2024-04-16T17:56:57.308 DEBUG:boto:Path: /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.308 DEBUG:boto:Data: 2024-04-16T17:56:57.308 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.308 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.308 DEBUG:boto:Params: {} 2024-04-16T17:56:57.308 DEBUG:boto:Token: None 2024-04-16T17:56:57.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80= 2024-04-16T17:56:57.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80=', 'Content-Length': '0'} 2024-04-16T17:56:57.312 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f72d7a5bb8c64245-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.354 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.354 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.355 DEBUG:boto:path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.355 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.355 DEBUG:boto:Method: GET 2024-04-16T17:56:57.355 DEBUG:boto:Path: /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.355 DEBUG:boto:Data: 2024-04-16T17:56:57.355 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.355 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.355 DEBUG:boto:Params: {} 2024-04-16T17:56:57.355 DEBUG:boto:Token: None 2024-04-16T17:56:57.355 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo 2024-04-16T17:56:57.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80= 2024-04-16T17:56:57.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oNxFDFq5a6jKRmvL4BWzzcn1g80=', 'Content-Length': '0'} 2024-04-16T17:56:57.359 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'CZaDb7ETzISKO2ue.6IoBIPJNOl.MDo'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001d212a18e42155c1-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.402 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.403 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.403 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.403 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.403 DEBUG:boto:Method: GET 2024-04-16T17:56:57.403 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.403 DEBUG:boto:Data: 2024-04-16T17:56:57.403 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.403 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.403 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.403 DEBUG:boto:Params: {} 2024-04-16T17:56:57.403 DEBUG:boto:Token: None 2024-04-16T17:56:57.403 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.408 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072218e8b29a6127d-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.408 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.409 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.409 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.409 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.409 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.409 DEBUG:boto:Method: GET 2024-04-16T17:56:57.409 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.409 DEBUG:boto:Data: 2024-04-16T17:56:57.409 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.409 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.409 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.409 DEBUG:boto:Params: {} 2024-04-16T17:56:57.410 DEBUG:boto:Token: None 2024-04-16T17:56:57.410 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4482f616d201154-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.414 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.414 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.414 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.415 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.415 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.415 DEBUG:boto:Data: 2024-04-16T17:56:57.415 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.415 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.415 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.415 DEBUG:boto:Params: {} 2024-04-16T17:56:57.415 DEBUG:boto:Token: None 2024-04-16T17:56:57.415 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.422 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000181992080d1ea8ae-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.422 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.422 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.422 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.422 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.422 DEBUG:boto:Data: 2024-04-16T17:56:57.422 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.422 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.423 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.423 DEBUG:boto:Params: {} 2024-04-16T17:56:57.423 DEBUG:boto:Token: None 2024-04-16T17:56:57.423 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.423 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.429 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000c8c03fcc683ab01-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.429 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.430 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.430 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.430 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.430 DEBUG:boto:Method: GET 2024-04-16T17:56:57.430 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.430 DEBUG:boto:Data: 2024-04-16T17:56:57.430 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.430 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.430 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.430 DEBUG:boto:Params: {} 2024-04-16T17:56:57.430 DEBUG:boto:Token: None 2024-04-16T17:56:57.431 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.431 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.433 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002462362c461cd555-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.434 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.434 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.434 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.434 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.434 DEBUG:boto:Method: GET 2024-04-16T17:56:57.434 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.434 DEBUG:boto:Data: 2024-04-16T17:56:57.434 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.435 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.435 DEBUG:boto:Params: {} 2024-04-16T17:56:57.435 DEBUG:boto:Token: None 2024-04-16T17:56:57.435 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.435 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.438 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000993c92cc4aa37d56-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.478 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.479 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.479 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.479 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.479 DEBUG:boto:Method: GET 2024-04-16T17:56:57.479 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.479 DEBUG:boto:Data: 2024-04-16T17:56:57.479 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.479 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.479 DEBUG:boto:Params: {} 2024-04-16T17:56:57.479 DEBUG:boto:Token: None 2024-04-16T17:56:57.479 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000693980a1577345b6-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.484 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.484 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.484 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.484 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.484 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.484 DEBUG:boto:Method: GET 2024-04-16T17:56:57.485 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.485 DEBUG:boto:Data: 2024-04-16T17:56:57.485 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.485 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.485 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.485 DEBUG:boto:Params: {} 2024-04-16T17:56:57.485 DEBUG:boto:Token: None 2024-04-16T17:56:57.485 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.485 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000328f1d08d93ab6c2-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.530 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.530 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.531 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.531 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.531 DEBUG:boto:path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.531 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.531 DEBUG:boto:Method: GET 2024-04-16T17:56:57.531 DEBUG:boto:Path: /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.531 DEBUG:boto:Data: 2024-04-16T17:56:57.531 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.531 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.532 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.532 DEBUG:boto:Params: {} 2024-04-16T17:56:57.532 DEBUG:boto:Token: None 2024-04-16T17:56:57.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og= 2024-04-16T17:56:57.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og=', 'Content-Length': '0'} 2024-04-16T17:56:57.536 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005053a0370329cde3-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.578 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.579 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.579 DEBUG:boto:path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.579 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.579 DEBUG:boto:Method: GET 2024-04-16T17:56:57.579 DEBUG:boto:Path: /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.579 DEBUG:boto:Data: 2024-04-16T17:56:57.579 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.579 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.579 DEBUG:boto:Params: {} 2024-04-16T17:56:57.579 DEBUG:boto:Token: None 2024-04-16T17:56:57.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og= 2024-04-16T17:56:57.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og=', 'Content-Length': '0'} 2024-04-16T17:56:57.583 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000074ebf9798ce109cb-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.583 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.583 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.583 DEBUG:boto:path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.583 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.583 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.584 DEBUG:boto:Path: /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.584 DEBUG:boto:Data: 2024-04-16T17:56:57.584 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.584 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.584 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.584 DEBUG:boto:Params: {} 2024-04-16T17:56:57.584 DEBUG:boto:Token: None 2024-04-16T17:56:57.584 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vwkNNKnPp8A2TVdxXNeq+6v3cvk= 2024-04-16T17:56:57.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vwkNNKnPp8A2TVdxXNeq+6v3cvk=', 'Content-Length': '0'} 2024-04-16T17:56:57.587 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007db311217788e15c-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.587 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.588 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.588 DEBUG:boto:path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.588 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.588 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.588 DEBUG:boto:Path: /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.588 DEBUG:boto:Data: 2024-04-16T17:56:57.588 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.588 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.588 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.588 DEBUG:boto:Params: {} 2024-04-16T17:56:57.588 DEBUG:boto:Token: None 2024-04-16T17:56:57.588 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.589 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vwkNNKnPp8A2TVdxXNeq+6v3cvk= 2024-04-16T17:56:57.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vwkNNKnPp8A2TVdxXNeq+6v3cvk=', 'Content-Length': '0'} 2024-04-16T17:56:57.591 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000030473454e45d5463-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.592 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.592 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.592 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.592 DEBUG:boto:path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.592 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.592 DEBUG:boto:Method: GET 2024-04-16T17:56:57.592 DEBUG:boto:Path: /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.592 DEBUG:boto:Data: 2024-04-16T17:56:57.592 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.592 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.592 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.593 DEBUG:boto:Params: {} 2024-04-16T17:56:57.593 DEBUG:boto:Token: None 2024-04-16T17:56:57.593 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.593 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og= 2024-04-16T17:56:57.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og=', 'Content-Length': '0'} 2024-04-16T17:56:57.596 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f5ebeeb2515c257c-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.638 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.639 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.639 DEBUG:boto:path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.639 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.639 DEBUG:boto:Method: GET 2024-04-16T17:56:57.639 DEBUG:boto:Path: /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.639 DEBUG:boto:Data: 2024-04-16T17:56:57.639 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.639 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.639 DEBUG:boto:Params: {} 2024-04-16T17:56:57.639 DEBUG:boto:Token: None 2024-04-16T17:56:57.639 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65 2024-04-16T17:56:57.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og= 2024-04-16T17:56:57.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aT0d5ycbx9OeMC0OwifuJfWX2Og=', 'Content-Length': '0'} 2024-04-16T17:56:57.643 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'MK5bjd1abLVVPz6kMWrR-SDuiZ0vR65'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bb63e060c6b34c71-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.643 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.643 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.643 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.643 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.644 DEBUG:boto:Method: GET 2024-04-16T17:56:57.644 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.644 DEBUG:boto:Data: 2024-04-16T17:56:57.644 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.644 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.644 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.644 DEBUG:boto:Params: {} 2024-04-16T17:56:57.644 DEBUG:boto:Token: None 2024-04-16T17:56:57.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.649 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3f28dce22b02d50-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.649 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.649 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.650 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.650 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.650 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.650 DEBUG:boto:Method: GET 2024-04-16T17:56:57.650 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.650 DEBUG:boto:Data: 2024-04-16T17:56:57.650 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.650 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.650 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.650 DEBUG:boto:Params: {} 2024-04-16T17:56:57.650 DEBUG:boto:Token: None 2024-04-16T17:56:57.650 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.651 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.655 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ef6559da1061216-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.655 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.655 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.656 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.656 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.656 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.656 DEBUG:boto:Data: 2024-04-16T17:56:57.656 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.656 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.656 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.656 DEBUG:boto:Params: {} 2024-04-16T17:56:57.656 DEBUG:boto:Token: None 2024-04-16T17:56:57.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.663 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d84a8811292fb7e8-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.663 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.663 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.663 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.663 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.663 DEBUG:boto:Data: 2024-04-16T17:56:57.663 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.664 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.664 DEBUG:boto:Params: {} 2024-04-16T17:56:57.664 DEBUG:boto:Token: None 2024-04-16T17:56:57.664 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.670 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050cc02b1c05b36ed-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.671 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.671 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.671 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.671 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.671 DEBUG:boto:Method: GET 2024-04-16T17:56:57.671 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.671 DEBUG:boto:Data: 2024-04-16T17:56:57.671 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.671 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.672 DEBUG:boto:Params: {} 2024-04-16T17:56:57.672 DEBUG:boto:Token: None 2024-04-16T17:56:57.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.675 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001029f7d725f4fb60-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.718 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.718 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.719 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.719 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.719 DEBUG:boto:Method: GET 2024-04-16T17:56:57.719 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.719 DEBUG:boto:Data: 2024-04-16T17:56:57.719 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.719 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.719 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.719 DEBUG:boto:Params: {} 2024-04-16T17:56:57.719 DEBUG:boto:Token: None 2024-04-16T17:56:57.719 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.722 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007a74553ced40a07b-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.766 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.766 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.767 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.767 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.767 DEBUG:boto:Method: GET 2024-04-16T17:56:57.767 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.767 DEBUG:boto:Data: 2024-04-16T17:56:57.767 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.767 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.767 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.767 DEBUG:boto:Params: {} 2024-04-16T17:56:57.767 DEBUG:boto:Token: None 2024-04-16T17:56:57.767 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.767 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bdff577595937a30-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.772 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.772 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.773 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.773 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.773 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.773 DEBUG:boto:Method: GET 2024-04-16T17:56:57.773 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.773 DEBUG:boto:Data: 2024-04-16T17:56:57.773 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.773 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.773 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.773 DEBUG:boto:Params: {} 2024-04-16T17:56:57.773 DEBUG:boto:Token: None 2024-04-16T17:56:57.774 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbeabdeff10f8551-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.778 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.779 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.779 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.779 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.779 DEBUG:boto:path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.779 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.779 DEBUG:boto:Method: GET 2024-04-16T17:56:57.779 DEBUG:boto:Path: /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.779 DEBUG:boto:Data: 2024-04-16T17:56:57.779 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.779 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.779 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.780 DEBUG:boto:Params: {} 2024-04-16T17:56:57.780 DEBUG:boto:Token: None 2024-04-16T17:56:57.780 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw= 2024-04-16T17:56:57.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw=', 'Content-Length': '0'} 2024-04-16T17:56:57.783 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d4e6df3daefcaccb-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.826 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.826 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.827 DEBUG:boto:path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.827 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.827 DEBUG:boto:Method: GET 2024-04-16T17:56:57.827 DEBUG:boto:Path: /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.827 DEBUG:boto:Data: 2024-04-16T17:56:57.827 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.827 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.827 DEBUG:boto:Params: {} 2024-04-16T17:56:57.827 DEBUG:boto:Token: None 2024-04-16T17:56:57.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.827 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw= 2024-04-16T17:56:57.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw=', 'Content-Length': '0'} 2024-04-16T17:56:57.831 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cd720a3f3e38966e-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.831 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.831 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.831 DEBUG:boto:path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.831 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.831 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.831 DEBUG:boto:Path: /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.832 DEBUG:boto:Data: 2024-04-16T17:56:57.832 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.832 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.832 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.832 DEBUG:boto:Params: {} 2024-04-16T17:56:57.832 DEBUG:boto:Token: None 2024-04-16T17:56:57.832 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tkIe0Hlb4dfPX/rPJxwVt4bJVcE= 2024-04-16T17:56:57.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tkIe0Hlb4dfPX/rPJxwVt4bJVcE=', 'Content-Length': '0'} 2024-04-16T17:56:57.835 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007083e79785a533f2-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.835 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.835 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.835 DEBUG:boto:path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.836 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.836 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.836 DEBUG:boto:Path: /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.836 DEBUG:boto:Data: 2024-04-16T17:56:57.836 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.836 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.836 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.836 DEBUG:boto:Params: {} 2024-04-16T17:56:57.836 DEBUG:boto:Token: None 2024-04-16T17:56:57.836 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tkIe0Hlb4dfPX/rPJxwVt4bJVcE= 2024-04-16T17:56:57.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tkIe0Hlb4dfPX/rPJxwVt4bJVcE=', 'Content-Length': '0'} 2024-04-16T17:56:57.839 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d356cef683036005-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.840 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.840 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.840 DEBUG:boto:path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.840 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.840 DEBUG:boto:Method: GET 2024-04-16T17:56:57.840 DEBUG:boto:Path: /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.840 DEBUG:boto:Data: 2024-04-16T17:56:57.840 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.840 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.840 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.840 DEBUG:boto:Params: {} 2024-04-16T17:56:57.841 DEBUG:boto:Token: None 2024-04-16T17:56:57.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw= 2024-04-16T17:56:57.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw=', 'Content-Length': '0'} 2024-04-16T17:56:57.844 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000084dec730685577b4-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.886 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.886 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.886 DEBUG:boto:path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.887 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.887 DEBUG:boto:Method: GET 2024-04-16T17:56:57.887 DEBUG:boto:Path: /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.887 DEBUG:boto:Data: 2024-04-16T17:56:57.887 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.887 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.887 DEBUG:boto:Params: {} 2024-04-16T17:56:57.887 DEBUG:boto:Token: None 2024-04-16T17:56:57.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=GbG1ol87fm4v9MJM91TmJaz.F1y08.X 2024-04-16T17:56:57.887 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw= 2024-04-16T17:56:57.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UKHese4vJVOmklr9w8NJR6SwmIw=', 'Content-Length': '0'} 2024-04-16T17:56:57.891 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'GbG1ol87fm4v9MJM91TmJaz.F1y08.X'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000927e128f5bf32dc9-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.891 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.891 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.891 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.891 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.891 DEBUG:boto:Method: GET 2024-04-16T17:56:57.891 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.891 DEBUG:boto:Data: 2024-04-16T17:56:57.892 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.892 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.892 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.892 DEBUG:boto:Params: {} 2024-04-16T17:56:57.892 DEBUG:boto:Token: None 2024-04-16T17:56:57.892 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084610d636de9daf1-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.897 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.898 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:57.898 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:57.898 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.898 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:57.898 DEBUG:boto:Method: GET 2024-04-16T17:56:57.898 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:57.898 DEBUG:boto:Data: 2024-04-16T17:56:57.898 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.898 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.898 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.898 DEBUG:boto:Params: {} 2024-04-16T17:56:57.899 DEBUG:boto:Token: None 2024-04-16T17:56:57.899 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/ 2024-04-16T17:56:57.899 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs= 2024-04-16T17:56:57.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EhrFEHx1x8wdBQIsrI7OMjJHcZs=', 'Content-Length': '0'} 2024-04-16T17:56:57.903 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0b7d356864c1324-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.950 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:57.951 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.951 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.951 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.951 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.951 DEBUG:boto:Data: 2024-04-16T17:56:57.951 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.951 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.951 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.951 DEBUG:boto:Params: {} 2024-04-16T17:56:57.951 DEBUG:boto:Token: None 2024-04-16T17:56:57.951 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.959 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000643265c140a4dbce-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.959 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.959 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.959 DEBUG:boto:Method: HEAD 2024-04-16T17:56:57.959 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:57.959 DEBUG:boto:Data: 2024-04-16T17:56:57.960 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.960 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.960 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.960 DEBUG:boto:Params: {} 2024-04-16T17:56:57.960 DEBUG:boto:Token: None 2024-04-16T17:56:57.960 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj 2024-04-16T17:56:57.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o= 2024-04-16T17:56:57.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3CmCLJwsPAlUpOwc0MaYYO4LP4o=', 'Content-Length': '0'} 2024-04-16T17:56:57.967 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000060dda108e979918e-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:57.967 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.967 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.967 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.967 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.967 DEBUG:boto:Method: GET 2024-04-16T17:56:57.967 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.968 DEBUG:boto:Data: 2024-04-16T17:56:57.968 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:57.968 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.968 DEBUG:boto:Params: {} 2024-04-16T17:56:57.968 DEBUG:boto:Token: None 2024-04-16T17:56:57.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.971 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002b3f923808339bd8-00661ebbe9-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:57.971 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:57.971 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:57.971 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.971 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.971 DEBUG:boto:Method: GET 2024-04-16T17:56:57.972 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.972 DEBUG:boto:Data: 2024-04-16T17:56:57.972 DEBUG:boto:Headers: {} 2024-04-16T17:56:57.972 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:57.972 DEBUG:boto:Port: 8000 2024-04-16T17:56:57.972 DEBUG:boto:Params: {} 2024-04-16T17:56:57.972 DEBUG:boto:Token: None 2024-04-16T17:56:57.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:57 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:57.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc= 2024-04-16T17:56:57.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlJlzWsgUnfuA2TGlR2rRZ8Zlhc=', 'Content-Length': '0'} 2024-04-16T17:56:57.975 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004c0ad8f9082b7792-00661ebbe9-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.018 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.018 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.019 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.019 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.019 DEBUG:boto:Method: GET 2024-04-16T17:56:58.019 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.019 DEBUG:boto:Data: 2024-04-16T17:56:58.019 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.019 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.019 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.019 DEBUG:boto:Params: {} 2024-04-16T17:56:58.019 DEBUG:boto:Token: None 2024-04-16T17:56:58.019 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.019 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.023 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054c81c78686b41e4-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.024 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.024 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.024 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.024 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.024 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.024 DEBUG:boto:Method: GET 2024-04-16T17:56:58.024 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.024 DEBUG:boto:Data: 2024-04-16T17:56:58.025 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.025 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.025 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.025 DEBUG:boto:Params: {} 2024-04-16T17:56:58.025 DEBUG:boto:Token: None 2024-04-16T17:56:58.025 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058555f51b875e4e8-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.070 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.071 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.071 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.071 DEBUG:boto:path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.071 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.071 DEBUG:boto:Method: GET 2024-04-16T17:56:58.071 DEBUG:boto:Path: /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.071 DEBUG:boto:Data: 2024-04-16T17:56:58.071 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.071 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.071 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.071 DEBUG:boto:Params: {} 2024-04-16T17:56:58.072 DEBUG:boto:Token: None 2024-04-16T17:56:58.072 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q= 2024-04-16T17:56:58.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q=', 'Content-Length': '0'} 2024-04-16T17:56:58.075 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009dd26c8f4c633d68-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.075 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.075 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.075 DEBUG:boto:path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.075 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.075 DEBUG:boto:Method: GET 2024-04-16T17:56:58.076 DEBUG:boto:Path: /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.076 DEBUG:boto:Data: 2024-04-16T17:56:58.076 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.076 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.076 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.076 DEBUG:boto:Params: {} 2024-04-16T17:56:58.076 DEBUG:boto:Token: None 2024-04-16T17:56:58.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q= 2024-04-16T17:56:58.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q=', 'Content-Length': '0'} 2024-04-16T17:56:58.080 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000720ee05f151f35b2-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.126 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.127 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.127 DEBUG:boto:path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.127 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.127 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.127 DEBUG:boto:Path: /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.127 DEBUG:boto:Data: 2024-04-16T17:56:58.127 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.127 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.127 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.127 DEBUG:boto:Params: {} 2024-04-16T17:56:58.127 DEBUG:boto:Token: None 2024-04-16T17:56:58.127 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Cv/WYeE1rQS+W4Ctq2pa+RbXSTQ= 2024-04-16T17:56:58.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Cv/WYeE1rQS+W4Ctq2pa+RbXSTQ=', 'Content-Length': '0'} 2024-04-16T17:56:58.131 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d6940525c56f5b93-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.131 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.131 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.131 DEBUG:boto:path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.131 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.131 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.131 DEBUG:boto:Path: /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.131 DEBUG:boto:Data: 2024-04-16T17:56:58.132 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.132 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.132 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.132 DEBUG:boto:Params: {} 2024-04-16T17:56:58.132 DEBUG:boto:Token: None 2024-04-16T17:56:58.132 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Cv/WYeE1rQS+W4Ctq2pa+RbXSTQ= 2024-04-16T17:56:58.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Cv/WYeE1rQS+W4Ctq2pa+RbXSTQ=', 'Content-Length': '0'} 2024-04-16T17:56:58.135 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068f1bfb70afdc3c9-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.135 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.135 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.136 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.136 DEBUG:boto:path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.136 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.136 DEBUG:boto:Method: GET 2024-04-16T17:56:58.136 DEBUG:boto:Path: /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.136 DEBUG:boto:Data: 2024-04-16T17:56:58.136 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.136 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.136 DEBUG:boto:Params: {} 2024-04-16T17:56:58.136 DEBUG:boto:Token: None 2024-04-16T17:56:58.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q= 2024-04-16T17:56:58.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q=', 'Content-Length': '0'} 2024-04-16T17:56:58.140 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bb128b259093366a-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.182 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.182 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.183 DEBUG:boto:path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.183 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.183 DEBUG:boto:Method: GET 2024-04-16T17:56:58.183 DEBUG:boto:Path: /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.183 DEBUG:boto:Data: 2024-04-16T17:56:58.183 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.183 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.183 DEBUG:boto:Params: {} 2024-04-16T17:56:58.183 DEBUG:boto:Token: None 2024-04-16T17:56:58.183 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XYCyjdHzLH98ukxgiSCKrzW90a8.VDz 2024-04-16T17:56:58.183 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q= 2024-04-16T17:56:58.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4EmuuNNZTtau/m4VxVUlmoDgR6Q=', 'Content-Length': '0'} 2024-04-16T17:56:58.187 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'XYCyjdHzLH98ukxgiSCKrzW90a8.VDz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004510de7598b7bbe7-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.230 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.230 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.230 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.231 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.231 DEBUG:boto:Method: GET 2024-04-16T17:56:58.231 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.231 DEBUG:boto:Data: 2024-04-16T17:56:58.231 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.231 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.231 DEBUG:boto:Params: {} 2024-04-16T17:56:58.231 DEBUG:boto:Token: None 2024-04-16T17:56:58.231 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.231 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf9d176b4667994f-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.236 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.236 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.236 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.236 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.236 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.236 DEBUG:boto:Method: GET 2024-04-16T17:56:58.236 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.236 DEBUG:boto:Data: 2024-04-16T17:56:58.236 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.237 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.237 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.237 DEBUG:boto:Params: {} 2024-04-16T17:56:58.237 DEBUG:boto:Token: None 2024-04-16T17:56:58.237 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.245 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e61a0615d4a680df-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.245 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.245 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.245 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.245 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.245 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.245 DEBUG:boto:Data: 2024-04-16T17:56:58.246 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.246 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.246 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.246 DEBUG:boto:Params: {} 2024-04-16T17:56:58.246 DEBUG:boto:Token: None 2024-04-16T17:56:58.246 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.246 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.251 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000f03892c26377bc4-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.251 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.251 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.251 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.251 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.251 DEBUG:boto:Data: 2024-04-16T17:56:58.251 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.251 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.251 DEBUG:boto:Params: {} 2024-04-16T17:56:58.252 DEBUG:boto:Token: None 2024-04-16T17:56:58.252 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.258 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000126368672f293748-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.258 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.258 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.258 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.258 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.258 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.258 DEBUG:boto:Method: GET 2024-04-16T17:56:58.258 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.258 DEBUG:boto:Data: 2024-04-16T17:56:58.259 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.259 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.259 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.259 DEBUG:boto:Params: {} 2024-04-16T17:56:58.259 DEBUG:boto:Token: None 2024-04-16T17:56:58.259 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.259 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig= 2024-04-16T17:56:58.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig=', 'Content-Length': '0'} 2024-04-16T17:56:58.261 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a83277ab71d613ca-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.261 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.261 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.261 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.261 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.261 DEBUG:boto:Method: GET 2024-04-16T17:56:58.262 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.262 DEBUG:boto:Data: 2024-04-16T17:56:58.262 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.262 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.262 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.262 DEBUG:boto:Params: {} 2024-04-16T17:56:58.262 DEBUG:boto:Token: None 2024-04-16T17:56:58.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig= 2024-04-16T17:56:58.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig=', 'Content-Length': '0'} 2024-04-16T17:56:58.265 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002ede376fc29ac819-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.306 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.306 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.307 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.307 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.307 DEBUG:boto:Method: GET 2024-04-16T17:56:58.307 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.307 DEBUG:boto:Data: 2024-04-16T17:56:58.307 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.307 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.307 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.307 DEBUG:boto:Params: {} 2024-04-16T17:56:58.307 DEBUG:boto:Token: None 2024-04-16T17:56:58.307 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.307 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.311 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad5f2f4252b56fbf-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.312 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.312 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.312 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.312 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.312 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.312 DEBUG:boto:Method: GET 2024-04-16T17:56:58.312 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.312 DEBUG:boto:Data: 2024-04-16T17:56:58.313 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.313 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.313 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.313 DEBUG:boto:Params: {} 2024-04-16T17:56:58.313 DEBUG:boto:Token: None 2024-04-16T17:56:58.313 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.313 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009155ca3e694cd22f-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.318 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.318 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.318 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.319 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.319 DEBUG:boto:path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.319 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.319 DEBUG:boto:Method: GET 2024-04-16T17:56:58.319 DEBUG:boto:Path: /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.319 DEBUG:boto:Data: 2024-04-16T17:56:58.319 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.319 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.319 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.319 DEBUG:boto:Params: {} 2024-04-16T17:56:58.319 DEBUG:boto:Token: None 2024-04-16T17:56:58.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0= 2024-04-16T17:56:58.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0=', 'Content-Length': '0'} 2024-04-16T17:56:58.323 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f12d00e8be7d47df-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.366 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.366 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.367 DEBUG:boto:path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.367 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.367 DEBUG:boto:Method: GET 2024-04-16T17:56:58.367 DEBUG:boto:Path: /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.367 DEBUG:boto:Data: 2024-04-16T17:56:58.367 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.367 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.367 DEBUG:boto:Params: {} 2024-04-16T17:56:58.367 DEBUG:boto:Token: None 2024-04-16T17:56:58.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0= 2024-04-16T17:56:58.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0=', 'Content-Length': '0'} 2024-04-16T17:56:58.370 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007fa73745f7a0976a-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.371 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.371 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.371 DEBUG:boto:path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.371 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.371 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.371 DEBUG:boto:Path: /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.371 DEBUG:boto:Data: 2024-04-16T17:56:58.371 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.372 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.372 DEBUG:boto:Params: {} 2024-04-16T17:56:58.372 DEBUG:boto:Token: None 2024-04-16T17:56:58.372 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WYwvyROAYs0Jz/FYblvMBVylVrg= 2024-04-16T17:56:58.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WYwvyROAYs0Jz/FYblvMBVylVrg=', 'Content-Length': '0'} 2024-04-16T17:56:58.375 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b747b3cd2bbeeb17-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.376 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.376 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.376 DEBUG:boto:path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.376 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.376 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.376 DEBUG:boto:Path: /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.376 DEBUG:boto:Data: 2024-04-16T17:56:58.376 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.376 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.376 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.377 DEBUG:boto:Params: {} 2024-04-16T17:56:58.377 DEBUG:boto:Token: None 2024-04-16T17:56:58.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WYwvyROAYs0Jz/FYblvMBVylVrg= 2024-04-16T17:56:58.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WYwvyROAYs0Jz/FYblvMBVylVrg=', 'Content-Length': '0'} 2024-04-16T17:56:58.380 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009c378f9a168dfdf1-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.380 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.380 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.381 DEBUG:boto:path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.381 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.381 DEBUG:boto:Method: GET 2024-04-16T17:56:58.381 DEBUG:boto:Path: /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.381 DEBUG:boto:Data: 2024-04-16T17:56:58.381 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.381 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.381 DEBUG:boto:Params: {} 2024-04-16T17:56:58.381 DEBUG:boto:Token: None 2024-04-16T17:56:58.381 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.382 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0= 2024-04-16T17:56:58.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0=', 'Content-Length': '0'} 2024-04-16T17:56:58.384 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c3e7b064b03fc3fc-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.426 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.426 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.427 DEBUG:boto:path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.427 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.427 DEBUG:boto:Method: GET 2024-04-16T17:56:58.427 DEBUG:boto:Path: /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.427 DEBUG:boto:Data: 2024-04-16T17:56:58.427 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.427 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.427 DEBUG:boto:Params: {} 2024-04-16T17:56:58.427 DEBUG:boto:Token: None 2024-04-16T17:56:58.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8 2024-04-16T17:56:58.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0= 2024-04-16T17:56:58.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Pe2ijz8nCv8el6bVyEc0CuRDeG0=', 'Content-Length': '0'} 2024-04-16T17:56:58.431 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'JxY.HRojXcWb.xNlB2VxGvKZuCNVhq8'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000319b1536b2afbb6e-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.431 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.431 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.431 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.431 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.431 DEBUG:boto:Method: GET 2024-04-16T17:56:58.431 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.431 DEBUG:boto:Data: 2024-04-16T17:56:58.432 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.432 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.432 DEBUG:boto:Params: {} 2024-04-16T17:56:58.432 DEBUG:boto:Token: None 2024-04-16T17:56:58.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f23e44ebe3581f6-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.478 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.479 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.479 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.479 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.479 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.479 DEBUG:boto:Method: GET 2024-04-16T17:56:58.479 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.479 DEBUG:boto:Data: 2024-04-16T17:56:58.479 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.479 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.480 DEBUG:boto:Params: {} 2024-04-16T17:56:58.480 DEBUG:boto:Token: None 2024-04-16T17:56:58.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a6f612202e7ccc6-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.484 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.484 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.484 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.485 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.485 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.485 DEBUG:boto:Data: 2024-04-16T17:56:58.485 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.485 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.485 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.485 DEBUG:boto:Params: {} 2024-04-16T17:56:58.485 DEBUG:boto:Token: None 2024-04-16T17:56:58.485 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.485 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.492 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000865e40309219b4e3-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.492 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.492 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.492 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.492 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.492 DEBUG:boto:Data: 2024-04-16T17:56:58.492 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.492 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.492 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.493 DEBUG:boto:Params: {} 2024-04-16T17:56:58.493 DEBUG:boto:Token: None 2024-04-16T17:56:58.493 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.499 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000005fb433cce09acc5-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.500 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.500 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.500 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.500 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.500 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.500 DEBUG:boto:Method: GET 2024-04-16T17:56:58.500 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.500 DEBUG:boto:Data: 2024-04-16T17:56:58.500 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.501 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.501 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.501 DEBUG:boto:Params: {} 2024-04-16T17:56:58.501 DEBUG:boto:Token: None 2024-04-16T17:56:58.501 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig= 2024-04-16T17:56:58.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig=', 'Content-Length': '0'} 2024-04-16T17:56:58.504 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f98aaed81992e489-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.504 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.505 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.505 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.505 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.505 DEBUG:boto:Method: GET 2024-04-16T17:56:58.505 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.505 DEBUG:boto:Data: 2024-04-16T17:56:58.505 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.505 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.505 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.505 DEBUG:boto:Params: {} 2024-04-16T17:56:58.505 DEBUG:boto:Token: None 2024-04-16T17:56:58.506 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.506 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig= 2024-04-16T17:56:58.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig=', 'Content-Length': '0'} 2024-04-16T17:56:58.508 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c77ca06af0207d0a-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.550 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.551 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.551 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.551 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.551 DEBUG:boto:Method: GET 2024-04-16T17:56:58.551 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.551 DEBUG:boto:Data: 2024-04-16T17:56:58.551 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.551 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.551 DEBUG:boto:Params: {} 2024-04-16T17:56:58.551 DEBUG:boto:Token: None 2024-04-16T17:56:58.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2d9c40202436967-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.557 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.557 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.557 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.557 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.557 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.558 DEBUG:boto:Method: GET 2024-04-16T17:56:58.558 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.558 DEBUG:boto:Data: 2024-04-16T17:56:58.558 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.558 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.558 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.558 DEBUG:boto:Params: {} 2024-04-16T17:56:58.558 DEBUG:boto:Token: None 2024-04-16T17:56:58.558 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.558 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.563 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038326303073be4ce-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.563 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.563 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.564 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.564 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.564 DEBUG:boto:path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.564 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.564 DEBUG:boto:Method: GET 2024-04-16T17:56:58.564 DEBUG:boto:Path: /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.564 DEBUG:boto:Data: 2024-04-16T17:56:58.564 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.564 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.564 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.565 DEBUG:boto:Params: {} 2024-04-16T17:56:58.565 DEBUG:boto:Token: None 2024-04-16T17:56:58.565 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.565 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4= 2024-04-16T17:56:58.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4=', 'Content-Length': '0'} 2024-04-16T17:56:58.568 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000004fdb28743b1387-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.614 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.615 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.615 DEBUG:boto:path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.615 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.615 DEBUG:boto:Method: GET 2024-04-16T17:56:58.615 DEBUG:boto:Path: /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.615 DEBUG:boto:Data: 2024-04-16T17:56:58.615 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.615 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.615 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.615 DEBUG:boto:Params: {} 2024-04-16T17:56:58.615 DEBUG:boto:Token: None 2024-04-16T17:56:58.616 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4= 2024-04-16T17:56:58.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4=', 'Content-Length': '0'} 2024-04-16T17:56:58.619 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d3577f4b5d71d798-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.662 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.662 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.663 DEBUG:boto:path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.663 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.663 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.663 DEBUG:boto:Path: /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.663 DEBUG:boto:Data: 2024-04-16T17:56:58.663 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.663 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.663 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.663 DEBUG:boto:Params: {} 2024-04-16T17:56:58.663 DEBUG:boto:Token: None 2024-04-16T17:56:58.664 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mkutIXvTIZ2rCDzzn6d+6GZAdZ4= 2024-04-16T17:56:58.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mkutIXvTIZ2rCDzzn6d+6GZAdZ4=', 'Content-Length': '0'} 2024-04-16T17:56:58.666 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000064c91a4d8a98f286-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.667 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.667 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.667 DEBUG:boto:path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.667 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.667 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.667 DEBUG:boto:Path: /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.667 DEBUG:boto:Data: 2024-04-16T17:56:58.667 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.668 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.668 DEBUG:boto:Params: {} 2024-04-16T17:56:58.668 DEBUG:boto:Token: None 2024-04-16T17:56:58.668 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mkutIXvTIZ2rCDzzn6d+6GZAdZ4= 2024-04-16T17:56:58.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mkutIXvTIZ2rCDzzn6d+6GZAdZ4=', 'Content-Length': '0'} 2024-04-16T17:56:58.671 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000099f68f342c8fac25-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.671 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.671 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.671 DEBUG:boto:path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.671 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.671 DEBUG:boto:Method: GET 2024-04-16T17:56:58.672 DEBUG:boto:Path: /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.672 DEBUG:boto:Data: 2024-04-16T17:56:58.672 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.672 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.672 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.672 DEBUG:boto:Params: {} 2024-04-16T17:56:58.672 DEBUG:boto:Token: None 2024-04-16T17:56:58.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4= 2024-04-16T17:56:58.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4=', 'Content-Length': '0'} 2024-04-16T17:56:58.675 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000038c7c922a5adcaa8-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.675 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.676 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.676 DEBUG:boto:path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.676 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.676 DEBUG:boto:Method: GET 2024-04-16T17:56:58.676 DEBUG:boto:Path: /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.676 DEBUG:boto:Data: 2024-04-16T17:56:58.676 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.676 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.676 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.676 DEBUG:boto:Params: {} 2024-04-16T17:56:58.676 DEBUG:boto:Token: None 2024-04-16T17:56:58.677 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ 2024-04-16T17:56:58.677 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4= 2024-04-16T17:56:58.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MtGKTGszsL2c7/V9twl2NswvDL4=', 'Content-Length': '0'} 2024-04-16T17:56:58.680 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'vJM2dN-SzDoWjVuCpoyOxbU.5BLOBHZ'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005db63391530fc347-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.726 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.726 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.727 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.727 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.727 DEBUG:boto:Method: GET 2024-04-16T17:56:58.727 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.727 DEBUG:boto:Data: 2024-04-16T17:56:58.727 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.727 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.727 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.727 DEBUG:boto:Params: {} 2024-04-16T17:56:58.727 DEBUG:boto:Token: None 2024-04-16T17:56:58.727 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.732 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eef3663f1fb19272-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.732 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.733 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.733 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.733 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.733 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.733 DEBUG:boto:Method: GET 2024-04-16T17:56:58.733 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.733 DEBUG:boto:Data: 2024-04-16T17:56:58.733 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.733 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.733 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.733 DEBUG:boto:Params: {} 2024-04-16T17:56:58.734 DEBUG:boto:Token: None 2024-04-16T17:56:58.734 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.734 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4560a3e8820adea-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.738 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.738 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.739 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.739 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.739 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.739 DEBUG:boto:Data: 2024-04-16T17:56:58.739 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.739 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.739 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.739 DEBUG:boto:Params: {} 2024-04-16T17:56:58.739 DEBUG:boto:Token: None 2024-04-16T17:56:58.739 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.739 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.746 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000582c356d9c1fdb30-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.747 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.747 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.747 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.747 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.747 DEBUG:boto:Data: 2024-04-16T17:56:58.747 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.747 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.747 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.748 DEBUG:boto:Params: {} 2024-04-16T17:56:58.748 DEBUG:boto:Token: None 2024-04-16T17:56:58.748 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.755 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004b07f31141af4978-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.755 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.755 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.755 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.755 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.755 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.755 DEBUG:boto:Method: GET 2024-04-16T17:56:58.755 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.755 DEBUG:boto:Data: 2024-04-16T17:56:58.755 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.756 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.756 DEBUG:boto:Params: {} 2024-04-16T17:56:58.756 DEBUG:boto:Token: None 2024-04-16T17:56:58.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig= 2024-04-16T17:56:58.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig=', 'Content-Length': '0'} 2024-04-16T17:56:58.759 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d10065d31e3bcedc-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.802 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.802 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.803 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.803 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.803 DEBUG:boto:Method: GET 2024-04-16T17:56:58.803 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.803 DEBUG:boto:Data: 2024-04-16T17:56:58.803 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.803 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.803 DEBUG:boto:Params: {} 2024-04-16T17:56:58.803 DEBUG:boto:Token: None 2024-04-16T17:56:58.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:58.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig= 2024-04-16T17:56:58.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rLLq+zcldbfirF6UGHlv8bzUAig=', 'Content-Length': '0'} 2024-04-16T17:56:58.807 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007367277c25079c75-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.850 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.850 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.851 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.851 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.851 DEBUG:boto:Method: GET 2024-04-16T17:56:58.851 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.851 DEBUG:boto:Data: 2024-04-16T17:56:58.851 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.851 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.851 DEBUG:boto:Params: {} 2024-04-16T17:56:58.851 DEBUG:boto:Token: None 2024-04-16T17:56:58.851 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.851 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000900f3998eee25089-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.856 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.856 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.856 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.856 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.856 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.857 DEBUG:boto:Method: GET 2024-04-16T17:56:58.857 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.857 DEBUG:boto:Data: 2024-04-16T17:56:58.857 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.857 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.857 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.857 DEBUG:boto:Params: {} 2024-04-16T17:56:58.857 DEBUG:boto:Token: None 2024-04-16T17:56:58.857 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.861 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000823a4f9f7b9bf3ba-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.861 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.862 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.862 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.862 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.862 DEBUG:boto:path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.862 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.862 DEBUG:boto:Method: GET 2024-04-16T17:56:58.862 DEBUG:boto:Path: /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.862 DEBUG:boto:Data: 2024-04-16T17:56:58.862 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.863 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.863 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.863 DEBUG:boto:Params: {} 2024-04-16T17:56:58.863 DEBUG:boto:Token: None 2024-04-16T17:56:58.863 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo= 2024-04-16T17:56:58.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo=', 'Content-Length': '0'} 2024-04-16T17:56:58.866 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7147f6ee9256c11-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.866 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.867 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.867 DEBUG:boto:path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.867 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.867 DEBUG:boto:Method: GET 2024-04-16T17:56:58.867 DEBUG:boto:Path: /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.867 DEBUG:boto:Data: 2024-04-16T17:56:58.867 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.867 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.867 DEBUG:boto:Params: {} 2024-04-16T17:56:58.867 DEBUG:boto:Token: None 2024-04-16T17:56:58.868 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo= 2024-04-16T17:56:58.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo=', 'Content-Length': '0'} 2024-04-16T17:56:58.871 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a2b0edde10e0103d-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.914 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.914 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.915 DEBUG:boto:path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.915 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.915 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.915 DEBUG:boto:Path: /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.915 DEBUG:boto:Data: 2024-04-16T17:56:58.915 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.915 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.915 DEBUG:boto:Params: {} 2024-04-16T17:56:58.915 DEBUG:boto:Token: None 2024-04-16T17:56:58.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cW1dbwndeRrdmeEVo2ZPNTOswzU= 2024-04-16T17:56:58.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cW1dbwndeRrdmeEVo2ZPNTOswzU=', 'Content-Length': '0'} 2024-04-16T17:56:58.919 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000620b64922bc37503-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.919 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.919 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.919 DEBUG:boto:path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.919 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.919 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.919 DEBUG:boto:Path: /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.919 DEBUG:boto:Data: 2024-04-16T17:56:58.920 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.920 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.920 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.920 DEBUG:boto:Params: {} 2024-04-16T17:56:58.920 DEBUG:boto:Token: None 2024-04-16T17:56:58.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cW1dbwndeRrdmeEVo2ZPNTOswzU= 2024-04-16T17:56:58.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cW1dbwndeRrdmeEVo2ZPNTOswzU=', 'Content-Length': '0'} 2024-04-16T17:56:58.923 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000077f00aae00dbe390-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.923 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:58.923 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.924 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.924 DEBUG:boto:path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.924 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.924 DEBUG:boto:Method: GET 2024-04-16T17:56:58.924 DEBUG:boto:Path: /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.924 DEBUG:boto:Data: 2024-04-16T17:56:58.924 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.924 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.924 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.924 DEBUG:boto:Params: {} 2024-04-16T17:56:58.924 DEBUG:boto:Token: None 2024-04-16T17:56:58.924 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.925 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo= 2024-04-16T17:56:58.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo=', 'Content-Length': '0'} 2024-04-16T17:56:58.928 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000483a65c842f8147e-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.928 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.928 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.928 DEBUG:boto:path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.928 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.929 DEBUG:boto:Method: GET 2024-04-16T17:56:58.929 DEBUG:boto:Path: /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.929 DEBUG:boto:Data: 2024-04-16T17:56:58.929 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.929 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.929 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.929 DEBUG:boto:Params: {} 2024-04-16T17:56:58.929 DEBUG:boto:Token: None 2024-04-16T17:56:58.929 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj?versionId=AERxD8InPQYZhCUcNH0LNchHJgyZ9lk 2024-04-16T17:56:58.929 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo= 2024-04-16T17:56:58.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a7c5CyfZ0wW+FE24Dfu5POMT5yo=', 'Content-Length': '0'} 2024-04-16T17:56:58.932 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'AERxD8InPQYZhCUcNH0LNchHJgyZ9lk'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:35 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d98e0ed794ac006f-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.974 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.974 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.975 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.975 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.975 DEBUG:boto:Method: GET 2024-04-16T17:56:58.975 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.975 DEBUG:boto:Data: 2024-04-16T17:56:58.975 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.975 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.975 DEBUG:boto:Params: {} 2024-04-16T17:56:58.975 DEBUG:boto:Token: None 2024-04-16T17:56:58.975 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.975 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.980 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4f9b1b8b622235d-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.980 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.981 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:58.981 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:58.981 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.981 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:58.981 DEBUG:boto:Method: GET 2024-04-16T17:56:58.981 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:58.981 DEBUG:boto:Data: 2024-04-16T17:56:58.981 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.981 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.981 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.981 DEBUG:boto:Params: {} 2024-04-16T17:56:58.982 DEBUG:boto:Token: None 2024-04-16T17:56:58.982 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/ 2024-04-16T17:56:58.982 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY= 2024-04-16T17:56:58.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wkFKX4k/Bicw/hkn66lQYU2bDHY=', 'Content-Length': '0'} 2024-04-16T17:56:58.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d40298a89cfaa40-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.986 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:58.986 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.987 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.987 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.987 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.987 DEBUG:boto:Data: 2024-04-16T17:56:58.987 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:58.987 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.987 DEBUG:boto:Params: {} 2024-04-16T17:56:58.987 DEBUG:boto:Token: None 2024-04-16T17:56:58.987 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:58.994 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f28abe5f9b89b106-00661ebbea-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:58.994 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:58.994 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:58.994 DEBUG:boto:Method: HEAD 2024-04-16T17:56:58.994 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:58.995 DEBUG:boto:Data: 2024-04-16T17:56:58.995 DEBUG:boto:Headers: {} 2024-04-16T17:56:58.995 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:58.995 DEBUG:boto:Port: 8000 2024-04-16T17:56:58.995 DEBUG:boto:Params: {} 2024-04-16T17:56:58.995 DEBUG:boto:Token: None 2024-04-16T17:56:58.995 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:58 GMT /lybtug-15/obj 2024-04-16T17:56:58.995 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM= 2024-04-16T17:56:58.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rYbZ3F1vzdZZniSRnSkH1Y1IutM=', 'Content-Length': '0'} 2024-04-16T17:56:59.001 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eab9a6ac59f83644-00661ebbea-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.001 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:59.001 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.001 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.002 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.002 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.002 DEBUG:boto:Method: GET 2024-04-16T17:56:59.002 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.002 DEBUG:boto:Data: 2024-04-16T17:56:59.002 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.002 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.002 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.002 DEBUG:boto:Params: {} 2024-04-16T17:56:59.002 DEBUG:boto:Token: None 2024-04-16T17:56:59.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY= 2024-04-16T17:56:59.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY=', 'Content-Length': '0'} 2024-04-16T17:56:59.005 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cfb761f1622449dc-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.046 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.047 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.047 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.047 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.047 DEBUG:boto:Method: GET 2024-04-16T17:56:59.047 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.047 DEBUG:boto:Data: 2024-04-16T17:56:59.047 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.047 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.047 DEBUG:boto:Params: {} 2024-04-16T17:56:59.048 DEBUG:boto:Token: None 2024-04-16T17:56:59.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY= 2024-04-16T17:56:59.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY=', 'Content-Length': '0'} 2024-04-16T17:56:59.051 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000051ba488f72fbc268-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.095 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:59.095 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:59.095 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.095 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.095 DEBUG:boto:Method: GET 2024-04-16T17:56:59.095 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:59.095 DEBUG:boto:Data: 2024-04-16T17:56:59.095 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.095 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.095 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.095 DEBUG:boto:Params: {} 2024-04-16T17:56:59.096 DEBUG:boto:Token: None 2024-04-16T17:56:59.096 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/ 2024-04-16T17:56:59.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0= 2024-04-16T17:56:59.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0=', 'Content-Length': '0'} 2024-04-16T17:56:59.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001fafe87facd31727-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.100 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:59.101 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:59.101 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:59.101 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.101 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.101 DEBUG:boto:Method: GET 2024-04-16T17:56:59.101 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:59.101 DEBUG:boto:Data: 2024-04-16T17:56:59.101 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.101 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.101 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.101 DEBUG:boto:Params: {} 2024-04-16T17:56:59.102 DEBUG:boto:Token: None 2024-04-16T17:56:59.102 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/ 2024-04-16T17:56:59.102 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0= 2024-04-16T17:56:59.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0=', 'Content-Length': '0'} 2024-04-16T17:56:59.106 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a801740fced2347f-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.106 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:59.107 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:59.107 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.107 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.107 DEBUG:boto:path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.107 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.107 DEBUG:boto:Method: GET 2024-04-16T17:56:59.107 DEBUG:boto:Path: /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.107 DEBUG:boto:Data: 2024-04-16T17:56:59.107 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.108 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.108 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.108 DEBUG:boto:Params: {} 2024-04-16T17:56:59.108 DEBUG:boto:Token: None 2024-04-16T17:56:59.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U= 2024-04-16T17:56:59.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U=', 'Content-Length': '0'} 2024-04-16T17:56:59.111 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003b269e3edc24bb2e-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.154 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.155 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.155 DEBUG:boto:path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.155 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.155 DEBUG:boto:Method: GET 2024-04-16T17:56:59.155 DEBUG:boto:Path: /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.155 DEBUG:boto:Data: 2024-04-16T17:56:59.155 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.155 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.155 DEBUG:boto:Params: {} 2024-04-16T17:56:59.156 DEBUG:boto:Token: None 2024-04-16T17:56:59.156 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U= 2024-04-16T17:56:59.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U=', 'Content-Length': '0'} 2024-04-16T17:56:59.159 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d68a920d6cce8cf4-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.159 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.160 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.160 DEBUG:boto:path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.160 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.160 DEBUG:boto:Method: HEAD 2024-04-16T17:56:59.160 DEBUG:boto:Path: /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.160 DEBUG:boto:Data: 2024-04-16T17:56:59.160 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.160 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.160 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.160 DEBUG:boto:Params: {} 2024-04-16T17:56:59.161 DEBUG:boto:Token: None 2024-04-16T17:56:59.161 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V1euBTkDNXHr8x2ZL51bHME5gFw= 2024-04-16T17:56:59.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V1euBTkDNXHr8x2ZL51bHME5gFw=', 'Content-Length': '0'} 2024-04-16T17:56:59.164 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c965424e8836b542-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.164 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.165 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.165 DEBUG:boto:path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.165 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.165 DEBUG:boto:Method: HEAD 2024-04-16T17:56:59.165 DEBUG:boto:Path: /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.165 DEBUG:boto:Data: 2024-04-16T17:56:59.165 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.165 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.165 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.165 DEBUG:boto:Params: {} 2024-04-16T17:56:59.165 DEBUG:boto:Token: None 2024-04-16T17:56:59.166 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.166 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V1euBTkDNXHr8x2ZL51bHME5gFw= 2024-04-16T17:56:59.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V1euBTkDNXHr8x2ZL51bHME5gFw=', 'Content-Length': '0'} 2024-04-16T17:56:59.169 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008fa5ffcb77e37236-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.169 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:59.169 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.169 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.169 DEBUG:boto:path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.169 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.169 DEBUG:boto:Method: GET 2024-04-16T17:56:59.169 DEBUG:boto:Path: /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.169 DEBUG:boto:Data: 2024-04-16T17:56:59.170 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.170 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.170 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.170 DEBUG:boto:Params: {} 2024-04-16T17:56:59.170 DEBUG:boto:Token: None 2024-04-16T17:56:59.170 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.170 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U= 2024-04-16T17:56:59.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U=', 'Content-Length': '0'} 2024-04-16T17:56:59.174 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002cfba30a452bf053-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.218 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.218 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.219 DEBUG:boto:path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.219 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.219 DEBUG:boto:Method: GET 2024-04-16T17:56:59.219 DEBUG:boto:Path: /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.219 DEBUG:boto:Data: 2024-04-16T17:56:59.219 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.219 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.219 DEBUG:boto:Params: {} 2024-04-16T17:56:59.219 DEBUG:boto:Token: None 2024-04-16T17:56:59.219 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz 2024-04-16T17:56:59.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U= 2024-04-16T17:56:59.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tJTL08c5G0mdaX580R4KJe25g+U=', 'Content-Length': '0'} 2024-04-16T17:56:59.223 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:32 GMT'), ('x-amz-version-id', 'tzjvpfN57rNIeCpjBOk9ZM3a5.7yTgz'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b3c3aceb980f4427-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.223 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:59.223 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:59.223 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.223 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.224 DEBUG:boto:Method: GET 2024-04-16T17:56:59.224 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:59.224 DEBUG:boto:Data: 2024-04-16T17:56:59.224 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.224 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.224 DEBUG:boto:Params: {} 2024-04-16T17:56:59.224 DEBUG:boto:Token: None 2024-04-16T17:56:59.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/ 2024-04-16T17:56:59.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0= 2024-04-16T17:56:59.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0=', 'Content-Length': '0'} 2024-04-16T17:56:59.228 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004142d87b921b0fbc-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.229 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:59.229 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:59.229 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:59.229 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.229 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.230 DEBUG:boto:Method: GET 2024-04-16T17:56:59.230 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:59.230 DEBUG:boto:Data: 2024-04-16T17:56:59.230 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.230 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.230 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.230 DEBUG:boto:Params: {} 2024-04-16T17:56:59.230 DEBUG:boto:Token: None 2024-04-16T17:56:59.230 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/ 2024-04-16T17:56:59.231 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0= 2024-04-16T17:56:59.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0=', 'Content-Length': '0'} 2024-04-16T17:56:59.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c1f1ec56dbe43ac-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.278 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:59.279 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.279 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.279 DEBUG:boto:Method: HEAD 2024-04-16T17:56:59.279 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:59.279 DEBUG:boto:Data: 2024-04-16T17:56:59.279 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.279 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.279 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.279 DEBUG:boto:Params: {} 2024-04-16T17:56:59.279 DEBUG:boto:Token: None 2024-04-16T17:56:59.279 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj 2024-04-16T17:56:59.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JcMXqBVWr0+jYvmnNwfEqHmD6sI= 2024-04-16T17:56:59.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JcMXqBVWr0+jYvmnNwfEqHmD6sI=', 'Content-Length': '0'} 2024-04-16T17:56:59.286 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058c53b86167a2862-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.287 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.287 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.287 DEBUG:boto:Method: HEAD 2024-04-16T17:56:59.287 DEBUG:boto:Path: /lybtug-15/obj 2024-04-16T17:56:59.287 DEBUG:boto:Data: 2024-04-16T17:56:59.287 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.287 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.287 DEBUG:boto:Params: {} 2024-04-16T17:56:59.287 DEBUG:boto:Token: None 2024-04-16T17:56:59.287 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj 2024-04-16T17:56:59.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JcMXqBVWr0+jYvmnNwfEqHmD6sI= 2024-04-16T17:56:59.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JcMXqBVWr0+jYvmnNwfEqHmD6sI=', 'Content-Length': '0'} 2024-04-16T17:56:59.294 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006cd77c1ebacf3433-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.294 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:56:59.294 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.294 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.294 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.294 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.294 DEBUG:boto:Method: GET 2024-04-16T17:56:59.294 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.295 DEBUG:boto:Data: 2024-04-16T17:56:59.295 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.295 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.295 DEBUG:boto:Params: {} 2024-04-16T17:56:59.295 DEBUG:boto:Token: None 2024-04-16T17:56:59.295 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.295 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY= 2024-04-16T17:56:59.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY=', 'Content-Length': '0'} 2024-04-16T17:56:59.298 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002fb13eb97c55c256-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.298 DEBUG:boto:path=/lybtug-15/obj 2024-04-16T17:56:59.298 DEBUG:boto:auth_path=/lybtug-15/obj 2024-04-16T17:56:59.298 DEBUG:boto:path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.299 DEBUG:boto:auth_path=/lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.299 DEBUG:boto:Method: GET 2024-04-16T17:56:59.299 DEBUG:boto:Path: /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.299 DEBUG:boto:Data: 2024-04-16T17:56:59.299 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.299 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.299 DEBUG:boto:Params: {} 2024-04-16T17:56:59.299 DEBUG:boto:Token: None 2024-04-16T17:56:59.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/obj?versionId=XJZrDUSbKtRrUBGGF-xtn75HidxGDK6 2024-04-16T17:56:59.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY= 2024-04-16T17:56:59.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sExf8eFbkQlWaHjTy3GBnJZjXpY=', 'Content-Length': '0'} 2024-04-16T17:56:59.302 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:56:33 GMT'), ('x-amz-version-id', 'XJZrDUSbKtRrUBGGF-xtn75HidxGDK6'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:56:36 GMT'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000144907c0bc14573f-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.346 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:59.346 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:59.347 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.347 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.347 DEBUG:boto:Method: GET 2024-04-16T17:56:59.347 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:59.347 DEBUG:boto:Data: 2024-04-16T17:56:59.347 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.347 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:56:59.347 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.347 DEBUG:boto:Params: {} 2024-04-16T17:56:59.347 DEBUG:boto:Token: None 2024-04-16T17:56:59.347 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/ 2024-04-16T17:56:59.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0= 2024-04-16T17:56:59.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0=', 'Content-Length': '0'} 2024-04-16T17:56:59.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2f861e035e49f1a-00661ebbeb-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.352 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:59.352 DEBUG:boto:path=/lybtug-15/ 2024-04-16T17:56:59.352 DEBUG:boto:auth_path=/lybtug-15/ 2024-04-16T17:56:59.352 DEBUG:boto:path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.352 DEBUG:boto:auth_path=/lybtug-15/?prefix=obj 2024-04-16T17:56:59.352 DEBUG:boto:Method: GET 2024-04-16T17:56:59.353 DEBUG:boto:Path: /lybtug-15/?prefix=obj 2024-04-16T17:56:59.353 DEBUG:boto:Data: 2024-04-16T17:56:59.353 DEBUG:boto:Headers: {} 2024-04-16T17:56:59.353 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:56:59.353 DEBUG:boto:Port: 8000 2024-04-16T17:56:59.353 DEBUG:boto:Params: {} 2024-04-16T17:56:59.353 DEBUG:boto:Token: None 2024-04-16T17:56:59.353 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:56:59 GMT /lybtug-15/ 2024-04-16T17:56:59.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0= 2024-04-16T17:56:59.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:56:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WFIli8wNBgn8Wh2zMyVPq4duja0=', 'Content-Length': '0'} 2024-04-16T17:56:59.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b3ba78b45fbe2f1-00661ebbeb-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:56:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:56:59.398 DEBUG:boto:b'lybtug-15obj1000falseobj2024-04-16T17:56:33.336Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDRGW11111111111111111Normal' 2024-04-16T17:56:59.399 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-15 zones={test-zone2, test-zone1} 2024-04-16T17:56:59.399 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2024-04-16T17:56:59.399 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.399 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.399 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:56:59.441 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:59.441 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:59.553 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"00000000000000000000:00000000000000005020","last_update":"2024-04-16T17:56:45.050649Z"},{"marker":"00000000000000000000:00000000000000002909","last_update":"2024-04-16T17:50:09.020245Z"},{"marker":"00000000000000000000:00000000000000002484","last_update":"2024-04-16T17:54:55.037491Z"},{"marker":"00000000000000000000:00000000000000004027","last_update":"2024-04-16T17:56:45.054711Z"}] 2024-04-16T17:56:59.554 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000005020', 1: '00000000000000000000:00000000000000002909', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000004027'} 2024-04-16T17:56:59.554 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T17:56:59.554 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.554 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.554 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:56:59.598 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:59.598 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:59.699 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005020","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:45.050649Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002909","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:50:09.020245Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004027","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:45.054711Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T17:56:59Z"} 2024-04-16T17:56:59.700 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005020","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:45.050649Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002909","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:50:09.020245Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004027","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:45.054711Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T17:56:59Z"} 2024-04-16T17:56:59.700 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005020', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:45.050649Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002909', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:50:09.020245Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002484', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:54:55.037491Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004027', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:45.054711Z'}}] 2024-04-16T17:56:59.700 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000005020', 1: '00000000000000000000:00000000000000002909', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000004027'} 2024-04-16T17:56:59.700 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005020', 1: '00000000000000000000:00000000000000002909', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000004027'} 2024-04-16T17:56:59.700 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T17:56:59.700 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2024-04-16T17:56:59.701 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.701 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.701 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:56:59.784 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:59.784 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:56:59.891 INFO:teuthology.orchestra.run.smithi122.stdout:[{"marker":"00000000000000000000:00000000000000004315","last_update":"2024-04-16T17:56:29.271725Z"},{"marker":"00000000000000000000:00000000000000002626","last_update":"2024-04-16T17:49:53.244956Z"},{"marker":"00000000000000000000:00000000000000002625","last_update":"2024-04-16T17:55:23.267655Z"},{"marker":"00000000000000000000:00000000000000003745","last_update":"2024-04-16T17:56:51.275629Z"}] 2024-04-16T17:56:59.892 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002626', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000003745'} 2024-04-16T17:56:59.892 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T17:56:59.892 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.892 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:56:59.892 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:56:59.937 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:56:59.937 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:00.034 INFO:teuthology.orchestra.run.smithi071.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004315","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:29.271725Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002626","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:53.244956Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002625","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T17:55:23.267655Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003745","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:51.275629Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T17:57:00Z"} 2024-04-16T17:57:00.034 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004315","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:29.271725Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002626","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:53.244956Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002625","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T17:55:23.267655Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003745","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:51.275629Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T17:57:00Z"} 2024-04-16T17:57:00.034 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004315', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:29.271725Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002626', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:53.244956Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002625', 'next_step_marker': '', 'total_entries': 7, 'pos': 0, 'timestamp': '2024-04-16T17:55:23.267655Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003745', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:51.275629Z'}}] 2024-04-16T17:57:00.034 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002626', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000003745'} 2024-04-16T17:57:00.034 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002626', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000003745'} 2024-04-16T17:57:00.034 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T17:57:00.035 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_concurrent_versioned_object_incremental_sync ... ok 2024-04-16T17:57:00.036 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:00.037 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:00.037 DEBUG:boto:Method: PUT 2024-04-16T17:57:00.037 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:00.037 DEBUG:boto:Data: 2024-04-16T17:57:00.037 DEBUG:boto:Headers: {} 2024-04-16T17:57:00.037 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:00.037 DEBUG:boto:Port: 8000 2024-04-16T17:57:00.037 DEBUG:boto:Params: {} 2024-04-16T17:57:00.037 DEBUG:boto:Token: None 2024-04-16T17:57:00.038 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:57:00 GMT /lybtug-16/ 2024-04-16T17:57:00.038 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gor9chlY8faZ5cPcI0lo6ImQDAk= 2024-04-16T17:57:00.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gor9chlY8faZ5cPcI0lo6ImQDAk=', 'Content-Length': '0'} 2024-04-16T17:57:00.068 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006c611f8407355f91-00661ebbec-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:00.068 DEBUG:rgw_multi.tests:created bucket=lybtug-16 2024-04-16T17:57:00.068 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:00.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:00.069 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:57:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:00.191 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:57:00.223 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290220.068077_136.1","last_update":"2024-04-16T17:57:00.068077Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:57:00.223 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290220.068077_136.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:00.223 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:57:00.224 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:00.224 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:00.224 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:57:00.266 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:00.266 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:00.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:00.336+0000 7fb94ee0e980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:57:00.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:00.340+0000 7fb94ee0e980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:57:00.367 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290220.068077_136.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:57:00.068077Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:57:00Z"} 2024-04-16T17:57:00.367 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290220.068077_136.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:57:00.068077Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:57:00Z"} 2024-04-16T17:57:00.367 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290220.068077_136.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:57:00.068077Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:57:00.367 DEBUG:rgw_multi.tests:log_status={0: '1_1713290220.068077_136.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:00.367 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290220.068077_136.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:00.367 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:57:00.368 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:00.368 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:00.368 DEBUG:boto:Method: HEAD 2024-04-16T17:57:00.368 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:00.368 DEBUG:boto:Data: 2024-04-16T17:57:00.368 DEBUG:boto:Headers: {} 2024-04-16T17:57:00.368 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:00.368 DEBUG:boto:Port: 8000 2024-04-16T17:57:00.368 DEBUG:boto:Params: {} 2024-04-16T17:57:00.368 DEBUG:boto:Token: None 2024-04-16T17:57:00.369 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:00 GMT /lybtug-16/ 2024-04-16T17:57:00.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kzrGPwA2WckC4SpAUMU3PAnjSjk= 2024-04-16T17:57:00.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kzrGPwA2WckC4SpAUMU3PAnjSjk=', 'Content-Length': '0'} 2024-04-16T17:57:00.373 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009f877c5ff1d182ed-00661ebbec-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:00.373 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:00.373 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:00.374 DEBUG:boto:Method: PUT 2024-04-16T17:57:00.374 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:00.374 DEBUG:boto:Data: 2024-04-16T17:57:00.374 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:57:00.374 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:00.374 DEBUG:boto:Port: 8000 2024-04-16T17:57:00.374 DEBUG:boto:Params: {} 2024-04-16T17:57:00.374 DEBUG:boto:Token: None 2024-04-16T17:57:00.374 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:57:00 GMT /lybtug-16/obj 2024-04-16T17:57:00.375 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vHHpREcviOSqD1sR31hANoPXdyo= 2024-04-16T17:57:00.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:57:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vHHpREcviOSqD1sR31hANoPXdyo='} 2024-04-16T17:57:00.388 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000004af2cf1fea31a68-00661ebbec-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:00.388 DEBUG:rgw_multi.tests:created initial version id=None 2024-04-16T17:57:00.389 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-16 2024-04-16T17:57:00.389 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:00.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:00.389 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:57:00.454 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:00.455 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:00.536+0000 7fbb8d2b4980 1 waiting to reach incremental sync.. 2024-04-16T17:57:05.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:05.540+0000 7fbb8d2b4980 1 bucket sync caught up with source: 2024-04-16T17:57:05.536 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.27.6, , ] 2024-04-16T17:57:05.536 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000001.27.6, , ] 2024-04-16T17:57:05.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:05.540+0000 7fbb8d2b4980 0 bucket checkpoint complete 2024-04-16T17:57:05.583 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-16 2024-04-16T17:57:05.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:05.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:05.584 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:57:05.633 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:05.634 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:05.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:05.710+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:10.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:10.710+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:15.714+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:20.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:20.714+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:25.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:25.718+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:30.718+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:35.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:35.718+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:40.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:40.722+0000 7fa651e27980 1 waiting to reach incremental sync.. 2024-04-16T17:57:45.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:45.722+0000 7fa651e27980 1 bucket sync caught up with source: 2024-04-16T17:57:45.722 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000001.36.6, , ] 2024-04-16T17:57:45.722 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000001.36.6, , ] 2024-04-16T17:57:45.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:45.722+0000 7fa651e27980 0 bucket checkpoint complete 2024-04-16T17:57:45.765 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-16 zones={test-zone2, test-zone1} 2024-04-16T17:57:45.765 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.765 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.765 DEBUG:boto:Method: HEAD 2024-04-16T17:57:45.765 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:45.765 DEBUG:boto:Data: 2024-04-16T17:57:45.766 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.766 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.766 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.766 DEBUG:boto:Params: {} 2024-04-16T17:57:45.766 DEBUG:boto:Token: None 2024-04-16T17:57:45.766 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/ 2024-04-16T17:57:45.767 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NcR7zLyL3pT++0yuotRpJ/7bTw0= 2024-04-16T17:57:45.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NcR7zLyL3pT++0yuotRpJ/7bTw0=', 'Content-Length': '0'} 2024-04-16T17:57:45.771 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005d5e4b0b3da9fa31-00661ebc19-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.771 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.771 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.771 DEBUG:boto:Method: HEAD 2024-04-16T17:57:45.771 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:45.771 DEBUG:boto:Data: 2024-04-16T17:57:45.771 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.772 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.772 DEBUG:boto:Params: {} 2024-04-16T17:57:45.772 DEBUG:boto:Token: None 2024-04-16T17:57:45.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/ 2024-04-16T17:57:45.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NcR7zLyL3pT++0yuotRpJ/7bTw0= 2024-04-16T17:57:45.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NcR7zLyL3pT++0yuotRpJ/7bTw0=', 'Content-Length': '0'} 2024-04-16T17:57:45.775 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a15e64a8f2ce04ea-00661ebc19-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.775 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:57:45.775 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.775 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.776 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.776 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.776 DEBUG:boto:Method: GET 2024-04-16T17:57:45.776 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.776 DEBUG:boto:Data: 2024-04-16T17:57:45.776 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.776 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.776 DEBUG:boto:Params: {} 2024-04-16T17:57:45.776 DEBUG:boto:Token: None 2024-04-16T17:57:45.776 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/?versions 2024-04-16T17:57:45.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs= 2024-04-16T17:57:45.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs=', 'Content-Length': '0'} 2024-04-16T17:57:45.780 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0b8947ee95a5a1c-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.822 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.823 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:57:45.823 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:57:45.823 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.823 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.823 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.823 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.824 DEBUG:boto:Method: GET 2024-04-16T17:57:45.824 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.824 DEBUG:boto:Data: 2024-04-16T17:57:45.824 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.824 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.824 DEBUG:boto:Params: {} 2024-04-16T17:57:45.824 DEBUG:boto:Token: None 2024-04-16T17:57:45.824 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/?versions 2024-04-16T17:57:45.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs= 2024-04-16T17:57:45.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs=', 'Content-Length': '0'} 2024-04-16T17:57:45.828 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012938b72df027cf2-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.829 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.829 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:57:45.829 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.829 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.829 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.829 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.829 DEBUG:boto:Method: GET 2024-04-16T17:57:45.829 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.830 DEBUG:boto:Data: 2024-04-16T17:57:45.830 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.830 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.830 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.830 DEBUG:boto:Params: {} 2024-04-16T17:57:45.830 DEBUG:boto:Token: None 2024-04-16T17:57:45.830 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/?versions 2024-04-16T17:57:45.830 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs= 2024-04-16T17:57:45.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs=', 'Content-Length': '0'} 2024-04-16T17:57:45.834 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe241c6a70fe8d1a-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.834 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.834 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.834 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.835 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.835 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.835 DEBUG:boto:Method: GET 2024-04-16T17:57:45.835 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:45.835 DEBUG:boto:Data: 2024-04-16T17:57:45.835 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.835 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.835 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.835 DEBUG:boto:Params: {} 2024-04-16T17:57:45.835 DEBUG:boto:Token: None 2024-04-16T17:57:45.835 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/?versions 2024-04-16T17:57:45.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs= 2024-04-16T17:57:45.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3cuXOaNpyQskrLlccKfShHFnrDs=', 'Content-Length': '0'} 2024-04-16T17:57:45.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000804b01a47787b48a-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.839 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:45.839 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.840 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.840 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.840 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.840 DEBUG:boto:Method: GET 2024-04-16T17:57:45.840 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.840 DEBUG:boto:Data: 2024-04-16T17:57:45.840 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.840 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.840 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.840 DEBUG:boto:Params: {} 2024-04-16T17:57:45.841 DEBUG:boto:Token: None 2024-04-16T17:57:45.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk= 2024-04-16T17:57:45.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk=', 'Content-Length': '0'} 2024-04-16T17:57:45.843 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001efdbd284d24c8a5-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.844 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.844 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.844 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.845 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.845 DEBUG:boto:Method: GET 2024-04-16T17:57:45.845 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.845 DEBUG:boto:Data: 2024-04-16T17:57:45.845 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.845 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.845 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.845 DEBUG:boto:Params: {} 2024-04-16T17:57:45.845 DEBUG:boto:Token: None 2024-04-16T17:57:45.845 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.845 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk= 2024-04-16T17:57:45.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk=', 'Content-Length': '0'} 2024-04-16T17:57:45.848 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ee2379c4fd94c859-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.849 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.849 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.849 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.849 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.849 DEBUG:boto:Method: HEAD 2024-04-16T17:57:45.849 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.849 DEBUG:boto:Data: 2024-04-16T17:57:45.849 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.849 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.849 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.849 DEBUG:boto:Params: {} 2024-04-16T17:57:45.850 DEBUG:boto:Token: None 2024-04-16T17:57:45.850 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.850 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vK+ZOK3yN159g7EMRSZ8CsGQs1k= 2024-04-16T17:57:45.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vK+ZOK3yN159g7EMRSZ8CsGQs1k=', 'Content-Length': '0'} 2024-04-16T17:57:45.852 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000ce08ee3a208071e-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.853 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.853 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.853 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.853 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.853 DEBUG:boto:Method: HEAD 2024-04-16T17:57:45.853 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.853 DEBUG:boto:Data: 2024-04-16T17:57:45.853 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.853 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.853 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.854 DEBUG:boto:Params: {} 2024-04-16T17:57:45.854 DEBUG:boto:Token: None 2024-04-16T17:57:45.854 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vK+ZOK3yN159g7EMRSZ8CsGQs1k= 2024-04-16T17:57:45.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vK+ZOK3yN159g7EMRSZ8CsGQs1k=', 'Content-Length': '0'} 2024-04-16T17:57:45.857 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b282ea4197b9ffba-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.857 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:45.857 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.857 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.857 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.858 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.858 DEBUG:boto:Method: GET 2024-04-16T17:57:45.858 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.858 DEBUG:boto:Data: 2024-04-16T17:57:45.858 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.858 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.858 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.858 DEBUG:boto:Params: {} 2024-04-16T17:57:45.858 DEBUG:boto:Token: None 2024-04-16T17:57:45.858 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk= 2024-04-16T17:57:45.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk=', 'Content-Length': '0'} 2024-04-16T17:57:45.861 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000030d4bcf0fa2ea60c-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.861 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.862 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.862 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.862 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:45.862 DEBUG:boto:Method: GET 2024-04-16T17:57:45.862 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.862 DEBUG:boto:Data: 2024-04-16T17:57:45.862 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.862 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.862 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.862 DEBUG:boto:Params: {} 2024-04-16T17:57:45.863 DEBUG:boto:Token: None 2024-04-16T17:57:45.863 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:45.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk= 2024-04-16T17:57:45.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2IgKdNlnvKgSLK6nJcBxF5ZSufk=', 'Content-Length': '0'} 2024-04-16T17:57:45.866 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e0ceea7fddd2c015-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.866 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.866 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.866 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.866 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.866 DEBUG:boto:Method: GET 2024-04-16T17:57:45.866 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:45.866 DEBUG:boto:Data: 2024-04-16T17:57:45.866 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.867 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.867 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.867 DEBUG:boto:Params: {} 2024-04-16T17:57:45.867 DEBUG:boto:Token: None 2024-04-16T17:57:45.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/ 2024-04-16T17:57:45.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE= 2024-04-16T17:57:45.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE=', 'Content-Length': '0'} 2024-04-16T17:57:45.871 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008276a46f6c72a672-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.871 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.871 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.871 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.871 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.871 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.871 DEBUG:boto:Method: GET 2024-04-16T17:57:45.872 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:45.872 DEBUG:boto:Data: 2024-04-16T17:57:45.872 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.872 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.872 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.872 DEBUG:boto:Params: {} 2024-04-16T17:57:45.872 DEBUG:boto:Token: None 2024-04-16T17:57:45.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/ 2024-04-16T17:57:45.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE= 2024-04-16T17:57:45.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE=', 'Content-Length': '0'} 2024-04-16T17:57:45.877 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e73248138f732d1-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.918 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.919 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.919 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.919 DEBUG:boto:Method: HEAD 2024-04-16T17:57:45.919 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:45.919 DEBUG:boto:Data: 2024-04-16T17:57:45.919 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.919 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.919 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.919 DEBUG:boto:Params: {} 2024-04-16T17:57:45.919 DEBUG:boto:Token: None 2024-04-16T17:57:45.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj 2024-04-16T17:57:45.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G9n8XuoHBQGUlhQb4UJ7OehNpeg= 2024-04-16T17:57:45.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G9n8XuoHBQGUlhQb4UJ7OehNpeg=', 'Content-Length': '0'} 2024-04-16T17:57:45.922 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f8d93cdc18f3fcfb-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.923 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.923 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.923 DEBUG:boto:Method: HEAD 2024-04-16T17:57:45.923 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:45.923 DEBUG:boto:Data: 2024-04-16T17:57:45.923 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.923 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.923 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.923 DEBUG:boto:Params: {} 2024-04-16T17:57:45.923 DEBUG:boto:Token: None 2024-04-16T17:57:45.923 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj 2024-04-16T17:57:45.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G9n8XuoHBQGUlhQb4UJ7OehNpeg= 2024-04-16T17:57:45.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G9n8XuoHBQGUlhQb4UJ7OehNpeg=', 'Content-Length': '0'} 2024-04-16T17:57:45.926 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000175573d2f8f8cd93-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.926 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:45.927 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.927 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.927 DEBUG:boto:Method: GET 2024-04-16T17:57:45.927 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:45.927 DEBUG:boto:Data: 2024-04-16T17:57:45.927 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.927 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.927 DEBUG:boto:Params: {} 2024-04-16T17:57:45.927 DEBUG:boto:Token: None 2024-04-16T17:57:45.927 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj 2024-04-16T17:57:45.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7HaXW/bG8u8Uy6NmiJLMYxvdVao= 2024-04-16T17:57:45.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7HaXW/bG8u8Uy6NmiJLMYxvdVao=', 'Content-Length': '0'} 2024-04-16T17:57:45.930 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000040754542245813ca-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.930 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:45.931 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:45.931 DEBUG:boto:Method: GET 2024-04-16T17:57:45.931 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:45.931 DEBUG:boto:Data: 2024-04-16T17:57:45.931 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.931 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.931 DEBUG:boto:Params: {} 2024-04-16T17:57:45.931 DEBUG:boto:Token: None 2024-04-16T17:57:45.931 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/obj 2024-04-16T17:57:45.931 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7HaXW/bG8u8Uy6NmiJLMYxvdVao= 2024-04-16T17:57:45.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7HaXW/bG8u8Uy6NmiJLMYxvdVao=', 'Content-Length': '0'} 2024-04-16T17:57:45.934 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000041a96ac02c9a6572-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.934 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.935 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.935 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.935 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.935 DEBUG:boto:Method: GET 2024-04-16T17:57:45.935 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:45.935 DEBUG:boto:Data: 2024-04-16T17:57:45.935 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:45.935 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.935 DEBUG:boto:Params: {} 2024-04-16T17:57:45.935 DEBUG:boto:Token: None 2024-04-16T17:57:45.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/ 2024-04-16T17:57:45.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE= 2024-04-16T17:57:45.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE=', 'Content-Length': '0'} 2024-04-16T17:57:45.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a1a06f89d193477-00661ebc19-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.982 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.983 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.983 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.983 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.983 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:45.983 DEBUG:boto:Method: GET 2024-04-16T17:57:45.983 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:45.983 DEBUG:boto:Data: 2024-04-16T17:57:45.983 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.984 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.984 DEBUG:boto:Params: {} 2024-04-16T17:57:45.984 DEBUG:boto:Token: None 2024-04-16T17:57:45.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/ 2024-04-16T17:57:45.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE= 2024-04-16T17:57:45.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lpjo8WmrEcHVj4SZR2Sow7ZhIWE=', 'Content-Length': '0'} 2024-04-16T17:57:45.988 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cab08d946f7f4ad2-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:45.988 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:45.988 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-16 zones={test-zone2, test-zone1} 2024-04-16T17:57:45.989 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:45.989 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:45.989 DEBUG:boto:path=/lybtug-16/?versioning 2024-04-16T17:57:45.989 DEBUG:boto:auth_path=/lybtug-16/?versioning 2024-04-16T17:57:45.989 DEBUG:boto:Method: PUT 2024-04-16T17:57:45.989 DEBUG:boto:Path: /lybtug-16/?versioning 2024-04-16T17:57:45.989 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:57:45.989 DEBUG:boto:Headers: {} 2024-04-16T17:57:45.989 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:45.989 DEBUG:boto:Port: 8000 2024-04-16T17:57:45.989 DEBUG:boto:Params: {} 2024-04-16T17:57:45.990 DEBUG:boto:Token: None 2024-04-16T17:57:45.990 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:57:45 GMT /lybtug-16/?versioning 2024-04-16T17:57:45.990 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eQAhif8YVVN+x6CgdKr7LWYCRNU= 2024-04-16T17:57:45.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eQAhif8YVVN+x6CgdKr7LWYCRNU=', 'Content-Length': '228'} 2024-04-16T17:57:46.002 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000078bfccef8d518fd4-00661ebc19-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:46.002 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:46.002 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:46.003 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:57:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:46.115 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:57:46.147 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290266.004984_143.1","last_update":"2024-04-16T17:57:46.004984Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:57:46.147 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290266.004984_143.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:46.147 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:57:46.148 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:46.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:46.148 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:57:46.191 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:46.191 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:46.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:46.264+0000 7fe1c8bf5980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:57:46.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:46.268+0000 7fe1c8bf5980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:57:46.293 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290266.004984_143.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:57:46.004984Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:57:46Z"} 2024-04-16T17:57:46.293 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290266.004984_143.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:57:46.004984Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:57:46Z"} 2024-04-16T17:57:46.293 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290266.004984_143.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:57:46.004984Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:57:46.294 DEBUG:rgw_multi.tests:log_status={0: '1_1713290266.004984_143.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:46.294 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290266.004984_143.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:46.294 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:57:46.294 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:46.294 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:46.294 DEBUG:boto:Method: HEAD 2024-04-16T17:57:46.294 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:46.294 DEBUG:boto:Data: 2024-04-16T17:57:46.294 DEBUG:boto:Headers: {} 2024-04-16T17:57:46.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:46.295 DEBUG:boto:Port: 8000 2024-04-16T17:57:46.295 DEBUG:boto:Params: {} 2024-04-16T17:57:46.295 DEBUG:boto:Token: None 2024-04-16T17:57:46.295 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:46 GMT /lybtug-16/ 2024-04-16T17:57:46.295 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q8ixbA/Ydg/yNJhLuvEsx8i8ox4= 2024-04-16T17:57:46.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q8ixbA/Ydg/yNJhLuvEsx8i8ox4=', 'Content-Length': '0'} 2024-04-16T17:57:46.299 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000022bbb7f319652c2a-00661ebc1a-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:46.299 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:46.299 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:46.299 DEBUG:boto:Method: PUT 2024-04-16T17:57:46.299 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:46.299 DEBUG:boto:Data: 2024-04-16T17:57:46.299 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:57:46.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:46.299 DEBUG:boto:Port: 8000 2024-04-16T17:57:46.300 DEBUG:boto:Params: {} 2024-04-16T17:57:46.300 DEBUG:boto:Token: None 2024-04-16T17:57:46.300 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:57:46 GMT /lybtug-16/obj 2024-04-16T17:57:46.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wo1RTlUgxA/THNt8UfaYnNl45aU= 2024-04-16T17:57:46.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:57:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wo1RTlUgxA/THNt8UfaYnNl45aU='} 2024-04-16T17:57:46.342 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-amz-request-id', 'tx000009cf2f0f82f8b3ddd-00661ebc1a-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:46.343 DEBUG:rgw_multi.tests:created new version id=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:46.343 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-16 2024-04-16T17:57:46.343 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:46.343 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:46.343 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:57:46.386 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:46.386 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:46.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:46.472+0000 7f35c1c96980 1 waiting for incremental sync to catch up: 2024-04-16T17:57:46.467 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.27.6, , ] 2024-04-16T17:57:46.467 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:57:51.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:51.472+0000 7f35c1c96980 1 bucket sync caught up with source: 2024-04-16T17:57:51.470 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:57:51.470 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:57:51.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:51.472+0000 7f35c1c96980 0 bucket checkpoint complete 2024-04-16T17:57:51.513 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-16 2024-04-16T17:57:51.513 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:51.514 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:51.514 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:57:51.564 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:51.564 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:51.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:51.645+0000 7f826944c980 1 waiting for incremental sync to catch up: 2024-04-16T17:57:51.644 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000001.36.6, , ] 2024-04-16T17:57:51.645 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000003.41.13, , ] 2024-04-16T17:57:56.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:56.649+0000 7f826944c980 1 bucket sync caught up with source: 2024-04-16T17:57:56.647 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000003.41.13, , ] 2024-04-16T17:57:56.647 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000003.41.13, , ] 2024-04-16T17:57:56.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:57:56.649+0000 7f826944c980 0 bucket checkpoint complete 2024-04-16T17:57:56.691 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-16 zones={test-zone2, test-zone1} 2024-04-16T17:57:56.691 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.691 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.691 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.691 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:56.692 DEBUG:boto:Data: 2024-04-16T17:57:56.692 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.692 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.692 DEBUG:boto:Params: {} 2024-04-16T17:57:56.692 DEBUG:boto:Token: None 2024-04-16T17:57:56.692 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:znt2KbySOyrn7vdubrN17hT+WNg= 2024-04-16T17:57:56.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:znt2KbySOyrn7vdubrN17hT+WNg=', 'Content-Length': '0'} 2024-04-16T17:57:56.696 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050c99bca8501017f-00661ebc24-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.697 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.697 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.697 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.697 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:56.697 DEBUG:boto:Data: 2024-04-16T17:57:56.697 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.697 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.697 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.698 DEBUG:boto:Params: {} 2024-04-16T17:57:56.698 DEBUG:boto:Token: None 2024-04-16T17:57:56.698 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.698 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:znt2KbySOyrn7vdubrN17hT+WNg= 2024-04-16T17:57:56.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:znt2KbySOyrn7vdubrN17hT+WNg=', 'Content-Length': '0'} 2024-04-16T17:57:56.701 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005050a6bcfca4429c-00661ebc24-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.701 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:57:56.701 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.702 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.702 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.702 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.702 DEBUG:boto:Method: GET 2024-04-16T17:57:56.702 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.702 DEBUG:boto:Data: 2024-04-16T17:57:56.702 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.702 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.702 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.702 DEBUG:boto:Params: {} 2024-04-16T17:57:56.702 DEBUG:boto:Token: None 2024-04-16T17:57:56.703 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/?versions 2024-04-16T17:57:56.703 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog= 2024-04-16T17:57:56.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog=', 'Content-Length': '0'} 2024-04-16T17:57:56.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a057bd653f4ebd8-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.707 DEBUG:boto:b'lybtug-16999falseobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyatrue2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobjnullfalse2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.708 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:57:56.708 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:57:56.708 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:57:56.708 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.708 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.708 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.708 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.708 DEBUG:boto:Method: GET 2024-04-16T17:57:56.709 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.709 DEBUG:boto:Data: 2024-04-16T17:57:56.709 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.709 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.709 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.709 DEBUG:boto:Params: {} 2024-04-16T17:57:56.709 DEBUG:boto:Token: None 2024-04-16T17:57:56.709 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/?versions 2024-04-16T17:57:56.709 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog= 2024-04-16T17:57:56.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog=', 'Content-Length': '0'} 2024-04-16T17:57:56.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001caefd63d63fc9fa-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.712 DEBUG:boto:b'lybtug-16999falseobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyatrue2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobjnullfalse2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.713 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:57:56.713 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:57:56.713 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.713 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.713 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.713 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.713 DEBUG:boto:Method: GET 2024-04-16T17:57:56.713 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.713 DEBUG:boto:Data: 2024-04-16T17:57:56.713 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.713 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.714 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.714 DEBUG:boto:Params: {} 2024-04-16T17:57:56.714 DEBUG:boto:Token: None 2024-04-16T17:57:56.714 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/?versions 2024-04-16T17:57:56.714 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog= 2024-04-16T17:57:56.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog=', 'Content-Length': '0'} 2024-04-16T17:57:56.718 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ebb4ca09a5b9f3f-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.718 DEBUG:boto:b'lybtug-16999falseobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyatrue2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobjnullfalse2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.718 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.718 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.719 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.719 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.719 DEBUG:boto:Method: GET 2024-04-16T17:57:56.719 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:57:56.719 DEBUG:boto:Data: 2024-04-16T17:57:56.719 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.719 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.719 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.719 DEBUG:boto:Params: {} 2024-04-16T17:57:56.719 DEBUG:boto:Token: None 2024-04-16T17:57:56.719 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/?versions 2024-04-16T17:57:56.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog= 2024-04-16T17:57:56.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GVs35g6HeXYq/6jJR/AC4qh4Eog=', 'Content-Length': '0'} 2024-04-16T17:57:56.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed356caae810afe1-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.723 DEBUG:boto:b'lybtug-16999falseobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyatrue2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobjnullfalse2024-04-16T17:57:00.382Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:56.723 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.723 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.723 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.723 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.724 DEBUG:boto:Method: GET 2024-04-16T17:57:56.724 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.724 DEBUG:boto:Data: 2024-04-16T17:57:56.724 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.724 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.724 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.724 DEBUG:boto:Params: {} 2024-04-16T17:57:56.724 DEBUG:boto:Token: None 2024-04-16T17:57:56.724 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.727 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000013b327f1df3ab7cf-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.728 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.728 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.728 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.728 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.728 DEBUG:boto:Method: GET 2024-04-16T17:57:56.728 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.728 DEBUG:boto:Data: 2024-04-16T17:57:56.728 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.728 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.728 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.729 DEBUG:boto:Params: {} 2024-04-16T17:57:56.729 DEBUG:boto:Token: None 2024-04-16T17:57:56.729 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.731 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000008889290da7a99ab-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.732 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.732 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.732 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.732 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.732 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.732 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.732 DEBUG:boto:Data: 2024-04-16T17:57:56.732 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.732 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.732 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.733 DEBUG:boto:Params: {} 2024-04-16T17:57:56.733 DEBUG:boto:Token: None 2024-04-16T17:57:56.733 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qhI7mDGA2EAyvPdW2clJzTSedrY= 2024-04-16T17:57:56.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qhI7mDGA2EAyvPdW2clJzTSedrY=', 'Content-Length': '0'} 2024-04-16T17:57:56.736 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005c5f58907de2f09d-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.736 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.736 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.736 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.736 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.736 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.737 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.737 DEBUG:boto:Data: 2024-04-16T17:57:56.737 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.737 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.737 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.737 DEBUG:boto:Params: {} 2024-04-16T17:57:56.737 DEBUG:boto:Token: None 2024-04-16T17:57:56.737 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qhI7mDGA2EAyvPdW2clJzTSedrY= 2024-04-16T17:57:56.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qhI7mDGA2EAyvPdW2clJzTSedrY=', 'Content-Length': '0'} 2024-04-16T17:57:56.740 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ab135cda5c1f6354-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:56.740 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.741 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.741 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.741 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.741 DEBUG:boto:Method: GET 2024-04-16T17:57:56.741 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.741 DEBUG:boto:Data: 2024-04-16T17:57:56.741 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.741 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.741 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.741 DEBUG:boto:Params: {} 2024-04-16T17:57:56.741 DEBUG:boto:Token: None 2024-04-16T17:57:56.742 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.742 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.745 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006ca3811a925598d6-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.745 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.745 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.745 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.745 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.745 DEBUG:boto:Method: GET 2024-04-16T17:57:56.745 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.746 DEBUG:boto:Data: 2024-04-16T17:57:56.746 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.746 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.746 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.746 DEBUG:boto:Params: {} 2024-04-16T17:57:56.746 DEBUG:boto:Token: None 2024-04-16T17:57:56.746 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.746 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.749 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003343531844fc4676-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.749 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.749 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.749 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.750 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.750 DEBUG:boto:Method: GET 2024-04-16T17:57:56.750 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.750 DEBUG:boto:Data: 2024-04-16T17:57:56.750 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.750 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.750 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.750 DEBUG:boto:Params: {} 2024-04-16T17:57:56.750 DEBUG:boto:Token: None 2024-04-16T17:57:56.751 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.751 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.754 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000620884d6b271d64d-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.798 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.799 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.799 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.799 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.799 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.799 DEBUG:boto:Method: GET 2024-04-16T17:57:56.799 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.799 DEBUG:boto:Data: 2024-04-16T17:57:56.799 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.800 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.800 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.800 DEBUG:boto:Params: {} 2024-04-16T17:57:56.800 DEBUG:boto:Token: None 2024-04-16T17:57:56.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000228c4cf247eebaaf-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.805 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.805 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.805 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.805 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.805 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:56.805 DEBUG:boto:Data: 2024-04-16T17:57:56.805 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.805 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.805 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.806 DEBUG:boto:Params: {} 2024-04-16T17:57:56.806 DEBUG:boto:Token: None 2024-04-16T17:57:56.806 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj 2024-04-16T17:57:56.806 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg= 2024-04-16T17:57:56.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg=', 'Content-Length': '0'} 2024-04-16T17:57:56.810 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000957f8a49faa43c47-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.810 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.810 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.810 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.810 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:56.810 DEBUG:boto:Data: 2024-04-16T17:57:56.810 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.810 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.811 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.811 DEBUG:boto:Params: {} 2024-04-16T17:57:56.811 DEBUG:boto:Token: None 2024-04-16T17:57:56.811 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj 2024-04-16T17:57:56.811 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg= 2024-04-16T17:57:56.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg=', 'Content-Length': '0'} 2024-04-16T17:57:56.815 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000141c5ae4d2b5cb79-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.815 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:56.815 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.815 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.815 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.816 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.816 DEBUG:boto:Method: GET 2024-04-16T17:57:56.816 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.816 DEBUG:boto:Data: 2024-04-16T17:57:56.816 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.816 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.816 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.816 DEBUG:boto:Params: {} 2024-04-16T17:57:56.816 DEBUG:boto:Token: None 2024-04-16T17:57:56.816 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.819 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000d962822c3f1910e-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.819 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.820 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.820 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.820 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.820 DEBUG:boto:Method: GET 2024-04-16T17:57:56.820 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.820 DEBUG:boto:Data: 2024-04-16T17:57:56.820 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.820 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.820 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.820 DEBUG:boto:Params: {} 2024-04-16T17:57:56.820 DEBUG:boto:Token: None 2024-04-16T17:57:56.821 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.824 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a0c278ff7c2428cb-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.824 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.824 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.824 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.824 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.824 DEBUG:boto:Method: GET 2024-04-16T17:57:56.824 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.824 DEBUG:boto:Data: 2024-04-16T17:57:56.824 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.824 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.825 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.825 DEBUG:boto:Params: {} 2024-04-16T17:57:56.825 DEBUG:boto:Token: None 2024-04-16T17:57:56.825 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000574455b2319c7d35-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.829 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.829 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.830 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.830 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.830 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.830 DEBUG:boto:Method: GET 2024-04-16T17:57:56.830 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.830 DEBUG:boto:Data: 2024-04-16T17:57:56.830 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.830 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.830 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.830 DEBUG:boto:Params: {} 2024-04-16T17:57:56.830 DEBUG:boto:Token: None 2024-04-16T17:57:56.831 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.831 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.834 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7bfa25d2d7d9132-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.878 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.878 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:56.879 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.879 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.879 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.879 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.879 DEBUG:boto:Method: GET 2024-04-16T17:57:56.879 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.879 DEBUG:boto:Data: 2024-04-16T17:57:56.879 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.879 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.880 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.880 DEBUG:boto:Params: {} 2024-04-16T17:57:56.880 DEBUG:boto:Token: None 2024-04-16T17:57:56.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c= 2024-04-16T17:57:56.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c=', 'Content-Length': '0'} 2024-04-16T17:57:56.883 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008cbc8ba329f72204-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.883 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.883 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.883 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.883 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.883 DEBUG:boto:Method: GET 2024-04-16T17:57:56.884 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.884 DEBUG:boto:Data: 2024-04-16T17:57:56.884 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.884 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.884 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.884 DEBUG:boto:Params: {} 2024-04-16T17:57:56.884 DEBUG:boto:Token: None 2024-04-16T17:57:56.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c= 2024-04-16T17:57:56.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c=', 'Content-Length': '0'} 2024-04-16T17:57:56.887 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000012f465440a7766bf-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.888 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.888 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.888 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.888 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.888 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.888 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.888 DEBUG:boto:Data: 2024-04-16T17:57:56.888 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.888 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.888 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.888 DEBUG:boto:Params: {} 2024-04-16T17:57:56.889 DEBUG:boto:Token: None 2024-04-16T17:57:56.889 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.889 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+pgqTA4oYvkCFrikQGP6nzdwNV4= 2024-04-16T17:57:56.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+pgqTA4oYvkCFrikQGP6nzdwNV4=', 'Content-Length': '0'} 2024-04-16T17:57:56.891 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000872a78cbce196382-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.892 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.892 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.892 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.892 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.892 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.892 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.892 DEBUG:boto:Data: 2024-04-16T17:57:56.892 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.892 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.893 DEBUG:boto:Params: {} 2024-04-16T17:57:56.893 DEBUG:boto:Token: None 2024-04-16T17:57:56.893 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.893 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+pgqTA4oYvkCFrikQGP6nzdwNV4= 2024-04-16T17:57:56.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+pgqTA4oYvkCFrikQGP6nzdwNV4=', 'Content-Length': '0'} 2024-04-16T17:57:56.896 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008b7254bd506b9035-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:56.896 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.896 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.896 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.897 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.897 DEBUG:boto:Method: GET 2024-04-16T17:57:56.897 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.897 DEBUG:boto:Data: 2024-04-16T17:57:56.897 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.897 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.897 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.897 DEBUG:boto:Params: {} 2024-04-16T17:57:56.897 DEBUG:boto:Token: None 2024-04-16T17:57:56.897 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.897 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c= 2024-04-16T17:57:56.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c=', 'Content-Length': '0'} 2024-04-16T17:57:56.900 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:03 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000070a48a86c274dbcf-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.901 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.901 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.901 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.901 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:57:56.901 DEBUG:boto:Method: GET 2024-04-16T17:57:56.901 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.901 DEBUG:boto:Data: 2024-04-16T17:57:56.901 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.901 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.901 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.902 DEBUG:boto:Params: {} 2024-04-16T17:57:56.902 DEBUG:boto:Token: None 2024-04-16T17:57:56.902 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:57:56.902 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c= 2024-04-16T17:57:56.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n2rQARWhB3wGSOkoWt8ZEgy8M2c=', 'Content-Length': '0'} 2024-04-16T17:57:56.905 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f526b05ba9719a8b-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.905 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.905 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.905 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.905 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.905 DEBUG:boto:Method: GET 2024-04-16T17:57:56.905 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.906 DEBUG:boto:Data: 2024-04-16T17:57:56.906 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.906 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.906 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.906 DEBUG:boto:Params: {} 2024-04-16T17:57:56.906 DEBUG:boto:Token: None 2024-04-16T17:57:56.906 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.906 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.910 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000131f569d9061f74a-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.954 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.955 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.955 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.955 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.955 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.955 DEBUG:boto:Method: GET 2024-04-16T17:57:56.955 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.955 DEBUG:boto:Data: 2024-04-16T17:57:56.955 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.956 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.956 DEBUG:boto:Params: {} 2024-04-16T17:57:56.956 DEBUG:boto:Token: None 2024-04-16T17:57:56.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000942d37dff8ac2a48-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.960 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.961 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.961 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.961 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.961 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:56.961 DEBUG:boto:Data: 2024-04-16T17:57:56.961 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.961 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.961 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.961 DEBUG:boto:Params: {} 2024-04-16T17:57:56.961 DEBUG:boto:Token: None 2024-04-16T17:57:56.962 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj 2024-04-16T17:57:56.962 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg= 2024-04-16T17:57:56.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg=', 'Content-Length': '0'} 2024-04-16T17:57:56.965 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fc4e06ed5c1d5e16-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.966 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.966 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.966 DEBUG:boto:Method: HEAD 2024-04-16T17:57:56.966 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:56.966 DEBUG:boto:Data: 2024-04-16T17:57:56.966 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.966 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.966 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.966 DEBUG:boto:Params: {} 2024-04-16T17:57:56.967 DEBUG:boto:Token: None 2024-04-16T17:57:56.967 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj 2024-04-16T17:57:56.967 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg= 2024-04-16T17:57:56.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kVF/pCiBgqaB8KcynqQFOeOnfzg=', 'Content-Length': '0'} 2024-04-16T17:57:56.971 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c45dcaae41d6fa34-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:57:56.971 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.971 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.972 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.972 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.972 DEBUG:boto:Method: GET 2024-04-16T17:57:56.972 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.972 DEBUG:boto:Data: 2024-04-16T17:57:56.972 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.972 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.972 DEBUG:boto:Params: {} 2024-04-16T17:57:56.972 DEBUG:boto:Token: None 2024-04-16T17:57:56.973 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.973 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.975 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000059a294551ae803c1-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.976 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:56.976 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:56.976 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.976 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.976 DEBUG:boto:Method: GET 2024-04-16T17:57:56.976 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.976 DEBUG:boto:Data: 2024-04-16T17:57:56.976 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.976 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.976 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.976 DEBUG:boto:Params: {} 2024-04-16T17:57:56.977 DEBUG:boto:Token: None 2024-04-16T17:57:56.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:57:56.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI= 2024-04-16T17:57:56.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/GztvqanB5/GDh3n7jd/LJupIjI=', 'Content-Length': '0'} 2024-04-16T17:57:56.979 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000284aa9675a455965-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.980 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.980 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.980 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.980 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.980 DEBUG:boto:Method: GET 2024-04-16T17:57:56.980 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.980 DEBUG:boto:Data: 2024-04-16T17:57:56.981 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.981 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:57:56.981 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.981 DEBUG:boto:Params: {} 2024-04-16T17:57:56.981 DEBUG:boto:Token: None 2024-04-16T17:57:56.981 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.985 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2e4aa8a1961a4c2-00661ebc24-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:56.986 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:56.986 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:56.986 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:56.986 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.986 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:57:56.986 DEBUG:boto:Method: GET 2024-04-16T17:57:56.986 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:57:56.987 DEBUG:boto:Data: 2024-04-16T17:57:56.987 DEBUG:boto:Headers: {} 2024-04-16T17:57:56.987 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:56.987 DEBUG:boto:Port: 8000 2024-04-16T17:57:56.987 DEBUG:boto:Params: {} 2024-04-16T17:57:56.987 DEBUG:boto:Token: None 2024-04-16T17:57:56.987 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:57:56 GMT /lybtug-16/ 2024-04-16T17:57:56.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc= 2024-04-16T17:57:56.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cGjvKisenRel4oQaHmVjtbKv7uc=', 'Content-Length': '0'} 2024-04-16T17:57:56.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1ff4a8e36f58e86-00661ebc24-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:57.034 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:57:57.035 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-16 zones={test-zone2, test-zone1} 2024-04-16T17:57:57.035 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:57.035 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:57.035 DEBUG:boto:path=/lybtug-16/?versioning 2024-04-16T17:57:57.035 DEBUG:boto:auth_path=/lybtug-16/?versioning 2024-04-16T17:57:57.035 DEBUG:boto:Method: PUT 2024-04-16T17:57:57.035 DEBUG:boto:Path: /lybtug-16/?versioning 2024-04-16T17:57:57.035 DEBUG:boto:Data: Suspended Disabled 2024-04-16T17:57:57.035 DEBUG:boto:Headers: {} 2024-04-16T17:57:57.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:57.036 DEBUG:boto:Port: 8000 2024-04-16T17:57:57.036 DEBUG:boto:Params: {} 2024-04-16T17:57:57.036 DEBUG:boto:Token: None 2024-04-16T17:57:57.036 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:57:57 GMT /lybtug-16/?versioning 2024-04-16T17:57:57.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vn3npHWoqBNIPnHvLWZ0M5SfYqQ= 2024-04-16T17:57:57.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vn3npHWoqBNIPnHvLWZ0M5SfYqQ=', 'Content-Length': '230'} 2024-04-16T17:57:57.050 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000125d29b5fb468518-00661ebc25-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:57.050 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:57.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:57.050 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:57:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:57.158 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:57:57.189 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290277.052282_146.1","last_update":"2024-04-16T17:57:57.052282Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:57:57.190 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290277.052282_146.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:57.190 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:57:57.190 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:57.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:57:57.190 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:57:57.233 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:57.233 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:57.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:57.304+0000 7f273ad43980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:57:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:57.308+0000 7f273ad43980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:57:57.333 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290277.052282_146.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:57:57.052282Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:57:57Z"} 2024-04-16T17:57:57.333 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290277.052282_146.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:57:57.052282Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:57:57Z"} 2024-04-16T17:57:57.333 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290277.052282_146.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:57:57.052282Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:57:57.333 DEBUG:rgw_multi.tests:log_status={0: '1_1713290277.052282_146.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:57.333 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290277.052282_146.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:57:57.333 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:57:57.334 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:57:57.334 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:57:57.334 DEBUG:boto:Method: HEAD 2024-04-16T17:57:57.334 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:57:57.334 DEBUG:boto:Data: 2024-04-16T17:57:57.334 DEBUG:boto:Headers: {} 2024-04-16T17:57:57.334 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:57.334 DEBUG:boto:Port: 8000 2024-04-16T17:57:57.334 DEBUG:boto:Params: {} 2024-04-16T17:57:57.335 DEBUG:boto:Token: None 2024-04-16T17:57:57.335 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:57:57 GMT /lybtug-16/ 2024-04-16T17:57:57.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OevOeZvYgf+37nwg1nhX4rdPLuI= 2024-04-16T17:57:57.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:57:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OevOeZvYgf+37nwg1nhX4rdPLuI=', 'Content-Length': '0'} 2024-04-16T17:57:57.339 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009cdfd3a8e205164e-00661ebc25-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:57.339 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:57:57.339 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:57:57.339 DEBUG:boto:Method: PUT 2024-04-16T17:57:57.339 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:57:57.340 DEBUG:boto:Data: 2024-04-16T17:57:57.340 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:57:57.340 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:57:57.340 DEBUG:boto:Port: 8000 2024-04-16T17:57:57.340 DEBUG:boto:Params: {} 2024-04-16T17:57:57.340 DEBUG:boto:Token: None 2024-04-16T17:57:57.340 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:57:57 GMT /lybtug-16/obj 2024-04-16T17:57:57.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VIAqXIJfmRdt6jZt3JNA3UIYFyM= 2024-04-16T17:57:57.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:57:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VIAqXIJfmRdt6jZt3JNA3UIYFyM='} 2024-04-16T17:57:57.369 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002a1d9a26d270f11f-00661ebc25-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:57:57.369 DEBUG:rgw_multi.tests:created null version id=None 2024-04-16T17:57:57.369 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-16 2024-04-16T17:57:57.369 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:57.369 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:57:57.370 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:57:57.421 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:57:57.421 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:57:57.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:57:57.504+0000 7fe6d0bd3980 1 waiting for incremental sync to catch up: 2024-04-16T17:57:57.500 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:57:57.500 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000005.34.18, , ] 2024-04-16T17:58:02.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:02.508+0000 7fe6d0bd3980 1 waiting for incremental sync to catch up: 2024-04-16T17:58:02.502 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:58:02.503 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000005.34.18, , ] 2024-04-16T17:58:07.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:07.508+0000 7fe6d0bd3980 1 waiting for incremental sync to catch up: 2024-04-16T17:58:07.505 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:58:07.505 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000005.34.18, , ] 2024-04-16T17:58:12.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:12.512+0000 7fe6d0bd3980 1 waiting for incremental sync to catch up: 2024-04-16T17:58:12.507 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.30.13, , ] 2024-04-16T17:58:12.507 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000005.34.18, , ] 2024-04-16T17:58:17.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:17.512+0000 7fe6d0bd3980 1 bucket sync caught up with source: 2024-04-16T17:58:17.509 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000005.34.18, , ] 2024-04-16T17:58:17.509 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000005.34.18, , ] 2024-04-16T17:58:17.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:17.512+0000 7fe6d0bd3980 0 bucket checkpoint complete 2024-04-16T17:58:17.556 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-16 2024-04-16T17:58:17.557 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:17.557 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:17.557 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:17.609 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:17.609 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:17.689+0000 7eff8fad0980 1 waiting for incremental sync to catch up: 2024-04-16T17:58:17.688 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000003.41.13, , ] 2024-04-16T17:58:17.688 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000005.47.18, , ] 2024-04-16T17:58:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:22.689+0000 7eff8fad0980 1 bucket sync caught up with source: 2024-04-16T17:58:22.690 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000005.47.18, , ] 2024-04-16T17:58:22.690 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000005.47.18, , ] 2024-04-16T17:58:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:22.689+0000 7eff8fad0980 0 bucket checkpoint complete 2024-04-16T17:58:22.730 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-16 zones={test-zone2, test-zone1} 2024-04-16T17:58:22.730 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.730 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.730 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.730 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:58:22.731 DEBUG:boto:Data: 2024-04-16T17:58:22.731 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.731 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.731 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.731 DEBUG:boto:Params: {} 2024-04-16T17:58:22.731 DEBUG:boto:Token: None 2024-04-16T17:58:22.731 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pV4u4RUvjfo8XwMVlQcRpQFOBVg= 2024-04-16T17:58:22.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pV4u4RUvjfo8XwMVlQcRpQFOBVg=', 'Content-Length': '0'} 2024-04-16T17:58:22.735 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006739551ead22e870-00661ebc3e-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.736 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.736 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.736 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.736 DEBUG:boto:Path: /lybtug-16/ 2024-04-16T17:58:22.736 DEBUG:boto:Data: 2024-04-16T17:58:22.736 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.736 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.736 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.736 DEBUG:boto:Params: {} 2024-04-16T17:58:22.736 DEBUG:boto:Token: None 2024-04-16T17:58:22.737 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pV4u4RUvjfo8XwMVlQcRpQFOBVg= 2024-04-16T17:58:22.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pV4u4RUvjfo8XwMVlQcRpQFOBVg=', 'Content-Length': '0'} 2024-04-16T17:58:22.740 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005362bbe9a0b42a1f-00661ebc3e-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.740 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:58:22.740 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.740 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.740 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.740 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.740 DEBUG:boto:Method: GET 2024-04-16T17:58:22.740 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.740 DEBUG:boto:Data: 2024-04-16T17:58:22.741 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.741 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.741 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.741 DEBUG:boto:Params: {} 2024-04-16T17:58:22.741 DEBUG:boto:Token: None 2024-04-16T17:58:22.741 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/?versions 2024-04-16T17:58:22.741 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k= 2024-04-16T17:58:22.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k=', 'Content-Length': '0'} 2024-04-16T17:58:22.746 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000268c4867ed9ad0e6-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.786 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyafalse2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.787 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:22.787 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:22.787 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:58:22.787 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.787 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.788 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.788 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.788 DEBUG:boto:Method: GET 2024-04-16T17:58:22.788 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.788 DEBUG:boto:Data: 2024-04-16T17:58:22.788 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.788 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.788 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.788 DEBUG:boto:Params: {} 2024-04-16T17:58:22.788 DEBUG:boto:Token: None 2024-04-16T17:58:22.788 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/?versions 2024-04-16T17:58:22.789 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k= 2024-04-16T17:58:22.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k=', 'Content-Length': '0'} 2024-04-16T17:58:22.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ece819ff407afae-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.792 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyafalse2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.793 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:22.793 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:22.793 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.793 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.793 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.793 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.793 DEBUG:boto:Method: GET 2024-04-16T17:58:22.793 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.794 DEBUG:boto:Data: 2024-04-16T17:58:22.794 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.794 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.794 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.794 DEBUG:boto:Params: {} 2024-04-16T17:58:22.794 DEBUG:boto:Token: None 2024-04-16T17:58:22.794 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/?versions 2024-04-16T17:58:22.794 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k= 2024-04-16T17:58:22.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k=', 'Content-Length': '0'} 2024-04-16T17:58:22.798 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d80a4213b2eb139-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.798 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyafalse2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.799 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.799 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.799 DEBUG:boto:path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.799 DEBUG:boto:auth_path=/lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.799 DEBUG:boto:Method: GET 2024-04-16T17:58:22.799 DEBUG:boto:Path: /lybtug-16/?versions&max-keys=999 2024-04-16T17:58:22.799 DEBUG:boto:Data: 2024-04-16T17:58:22.799 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.799 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.799 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.799 DEBUG:boto:Params: {} 2024-04-16T17:58:22.800 DEBUG:boto:Token: None 2024-04-16T17:58:22.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/?versions 2024-04-16T17:58:22.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k= 2024-04-16T17:58:22.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MRwyc+eH8+tmwmnSUDcG6YQpy0k=', 'Content-Length': '0'} 2024-04-16T17:58:22.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9469e3631637115-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.846 DEBUG:boto:b'lybtug-16999falseobjnulltrue2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normalobj8t0iJsZveSu6vPfdfR1Iae1xiH5yyyafalse2024-04-16T17:57:46.307Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.847 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:22.847 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.847 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.847 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.847 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.847 DEBUG:boto:Method: GET 2024-04-16T17:58:22.847 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.847 DEBUG:boto:Data: 2024-04-16T17:58:22.847 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.848 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.848 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.848 DEBUG:boto:Params: {} 2024-04-16T17:58:22.848 DEBUG:boto:Token: None 2024-04-16T17:58:22.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw= 2024-04-16T17:58:22.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw=', 'Content-Length': '0'} 2024-04-16T17:58:22.851 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004288a6c928da5bf1-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.851 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.852 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.852 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.852 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.852 DEBUG:boto:Method: GET 2024-04-16T17:58:22.852 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.852 DEBUG:boto:Data: 2024-04-16T17:58:22.852 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.852 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.852 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.852 DEBUG:boto:Params: {} 2024-04-16T17:58:22.852 DEBUG:boto:Token: None 2024-04-16T17:58:22.853 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.853 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw= 2024-04-16T17:58:22.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw=', 'Content-Length': '0'} 2024-04-16T17:58:22.856 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005d82748bc95abafb-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.856 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.856 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.856 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.856 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.856 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.856 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.856 DEBUG:boto:Data: 2024-04-16T17:58:22.856 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.857 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.857 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.857 DEBUG:boto:Params: {} 2024-04-16T17:58:22.857 DEBUG:boto:Token: None 2024-04-16T17:58:22.857 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4mZFGUX9k5eGP7P79+5764v6gGk= 2024-04-16T17:58:22.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4mZFGUX9k5eGP7P79+5764v6gGk=', 'Content-Length': '0'} 2024-04-16T17:58:22.860 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000083c91567fe16bac8-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.860 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.860 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.860 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.860 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.860 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.860 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.861 DEBUG:boto:Data: 2024-04-16T17:58:22.861 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.861 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.861 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.861 DEBUG:boto:Params: {} 2024-04-16T17:58:22.861 DEBUG:boto:Token: None 2024-04-16T17:58:22.861 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4mZFGUX9k5eGP7P79+5764v6gGk= 2024-04-16T17:58:22.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4mZFGUX9k5eGP7P79+5764v6gGk=', 'Content-Length': '0'} 2024-04-16T17:58:22.864 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000022cbed0de9411563-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.864 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:22.865 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.865 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.865 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.865 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.865 DEBUG:boto:Method: GET 2024-04-16T17:58:22.865 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.865 DEBUG:boto:Data: 2024-04-16T17:58:22.865 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.866 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.866 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.866 DEBUG:boto:Params: {} 2024-04-16T17:58:22.866 DEBUG:boto:Token: None 2024-04-16T17:58:22.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw= 2024-04-16T17:58:22.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw=', 'Content-Length': '0'} 2024-04-16T17:58:22.870 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004ad254e1423cff61-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.870 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.870 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.870 DEBUG:boto:path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.870 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=null 2024-04-16T17:58:22.871 DEBUG:boto:Method: GET 2024-04-16T17:58:22.871 DEBUG:boto:Path: /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.871 DEBUG:boto:Data: 2024-04-16T17:58:22.871 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.871 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.871 DEBUG:boto:Params: {} 2024-04-16T17:58:22.871 DEBUG:boto:Token: None 2024-04-16T17:58:22.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=null 2024-04-16T17:58:22.871 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw= 2024-04-16T17:58:22.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H+RrcjkSKLy0WN74aIlvY0ptxPw=', 'Content-Length': '0'} 2024-04-16T17:58:22.874 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007c87fa0e14457bac-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.875 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.875 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.875 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.875 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.875 DEBUG:boto:Method: GET 2024-04-16T17:58:22.875 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:22.875 DEBUG:boto:Data: 2024-04-16T17:58:22.875 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.875 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.875 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.876 DEBUG:boto:Params: {} 2024-04-16T17:58:22.876 DEBUG:boto:Token: None 2024-04-16T17:58:22.876 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI= 2024-04-16T17:58:22.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI=', 'Content-Length': '0'} 2024-04-16T17:58:22.880 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035cf4c74220c3005-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.923 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.923 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.923 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.923 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.923 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.924 DEBUG:boto:Method: GET 2024-04-16T17:58:22.924 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:22.924 DEBUG:boto:Data: 2024-04-16T17:58:22.924 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.924 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.924 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.924 DEBUG:boto:Params: {} 2024-04-16T17:58:22.924 DEBUG:boto:Token: None 2024-04-16T17:58:22.924 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI= 2024-04-16T17:58:22.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI=', 'Content-Length': '0'} 2024-04-16T17:58:22.928 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a1bc8484386617b-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.928 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.929 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.929 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.929 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.929 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:22.929 DEBUG:boto:Data: 2024-04-16T17:58:22.929 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.929 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.929 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.929 DEBUG:boto:Params: {} 2024-04-16T17:58:22.930 DEBUG:boto:Token: None 2024-04-16T17:58:22.930 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj 2024-04-16T17:58:22.930 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QBuRSoKv4DA3nXKA4rCB/4FTmCw= 2024-04-16T17:58:22.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QBuRSoKv4DA3nXKA4rCB/4FTmCw=', 'Content-Length': '0'} 2024-04-16T17:58:22.933 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000be464b4c78dcb9f9-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.934 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.934 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.934 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.934 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:22.934 DEBUG:boto:Data: 2024-04-16T17:58:22.934 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.934 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.934 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.934 DEBUG:boto:Params: {} 2024-04-16T17:58:22.934 DEBUG:boto:Token: None 2024-04-16T17:58:22.935 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj 2024-04-16T17:58:22.935 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QBuRSoKv4DA3nXKA4rCB/4FTmCw= 2024-04-16T17:58:22.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QBuRSoKv4DA3nXKA4rCB/4FTmCw=', 'Content-Length': '0'} 2024-04-16T17:58:22.938 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003e591dd72008aa0b-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.938 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:22.938 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.938 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.938 DEBUG:boto:Method: GET 2024-04-16T17:58:22.938 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:22.938 DEBUG:boto:Data: 2024-04-16T17:58:22.939 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.939 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.939 DEBUG:boto:Params: {} 2024-04-16T17:58:22.939 DEBUG:boto:Token: None 2024-04-16T17:58:22.939 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj 2024-04-16T17:58:22.939 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qvMMwK6aRGXZR9PAgbv1nXKHPtg= 2024-04-16T17:58:22.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qvMMwK6aRGXZR9PAgbv1nXKHPtg=', 'Content-Length': '0'} 2024-04-16T17:58:22.942 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009905ca25b220497d-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.942 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.942 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.942 DEBUG:boto:Method: GET 2024-04-16T17:58:22.943 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:22.943 DEBUG:boto:Data: 2024-04-16T17:58:22.943 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.943 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.943 DEBUG:boto:Params: {} 2024-04-16T17:58:22.943 DEBUG:boto:Token: None 2024-04-16T17:58:22.943 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj 2024-04-16T17:58:22.943 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qvMMwK6aRGXZR9PAgbv1nXKHPtg= 2024-04-16T17:58:22.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qvMMwK6aRGXZR9PAgbv1nXKHPtg=', 'Content-Length': '0'} 2024-04-16T17:58:22.946 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000069b9fbabbb8d36c3-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.946 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.946 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.947 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.947 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.947 DEBUG:boto:Method: GET 2024-04-16T17:58:22.947 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:22.947 DEBUG:boto:Data: 2024-04-16T17:58:22.947 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.947 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.947 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.947 DEBUG:boto:Params: {} 2024-04-16T17:58:22.947 DEBUG:boto:Token: None 2024-04-16T17:58:22.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI= 2024-04-16T17:58:22.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI=', 'Content-Length': '0'} 2024-04-16T17:58:22.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1c1700f7e45aa36-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.952 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.953 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.953 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.953 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.953 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.953 DEBUG:boto:Method: GET 2024-04-16T17:58:22.953 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:22.953 DEBUG:boto:Data: 2024-04-16T17:58:22.953 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.953 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.953 DEBUG:boto:Params: {} 2024-04-16T17:58:22.954 DEBUG:boto:Token: None 2024-04-16T17:58:22.954 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI= 2024-04-16T17:58:22.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI=', 'Content-Length': '0'} 2024-04-16T17:58:22.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038c7750efa8f40df-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.958 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:22.958 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:22.958 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.958 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.958 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.958 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.959 DEBUG:boto:Method: GET 2024-04-16T17:58:22.959 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.959 DEBUG:boto:Data: 2024-04-16T17:58:22.959 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.959 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.959 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.959 DEBUG:boto:Params: {} 2024-04-16T17:58:22.959 DEBUG:boto:Token: None 2024-04-16T17:58:22.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA= 2024-04-16T17:58:22.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA=', 'Content-Length': '0'} 2024-04-16T17:58:22.962 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000cf1a9075759d4763-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.963 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.963 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.963 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.963 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.963 DEBUG:boto:Method: GET 2024-04-16T17:58:22.963 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.963 DEBUG:boto:Data: 2024-04-16T17:58:22.963 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.963 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.964 DEBUG:boto:Params: {} 2024-04-16T17:58:22.964 DEBUG:boto:Token: None 2024-04-16T17:58:22.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA= 2024-04-16T17:58:22.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA=', 'Content-Length': '0'} 2024-04-16T17:58:22.967 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008b7196f0ef154f3d-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.967 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.967 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.967 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.967 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.967 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.967 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.968 DEBUG:boto:Data: 2024-04-16T17:58:22.968 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.968 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.968 DEBUG:boto:Params: {} 2024-04-16T17:58:22.968 DEBUG:boto:Token: None 2024-04-16T17:58:22.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:35zIX962C7WyLhD/STNoBn2zw+I= 2024-04-16T17:58:22.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:35zIX962C7WyLhD/STNoBn2zw+I=', 'Content-Length': '0'} 2024-04-16T17:58:22.971 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009bc01582508a6a86-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.971 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.972 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.972 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.972 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.972 DEBUG:boto:Method: HEAD 2024-04-16T17:58:22.972 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.972 DEBUG:boto:Data: 2024-04-16T17:58:22.972 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.972 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.972 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.972 DEBUG:boto:Params: {} 2024-04-16T17:58:22.972 DEBUG:boto:Token: None 2024-04-16T17:58:22.973 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.973 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:35zIX962C7WyLhD/STNoBn2zw+I= 2024-04-16T17:58:22.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:35zIX962C7WyLhD/STNoBn2zw+I=', 'Content-Length': '0'} 2024-04-16T17:58:22.976 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e1205c25a2858c65-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.976 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:22.976 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.976 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.976 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.977 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.977 DEBUG:boto:Method: GET 2024-04-16T17:58:22.977 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.977 DEBUG:boto:Data: 2024-04-16T17:58:22.977 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.977 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.977 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.977 DEBUG:boto:Params: {} 2024-04-16T17:58:22.977 DEBUG:boto:Token: None 2024-04-16T17:58:22.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA= 2024-04-16T17:58:22.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA=', 'Content-Length': '0'} 2024-04-16T17:58:22.980 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:57:48 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000df2fd2bffcaea0eb-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.981 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:22.981 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:22.981 DEBUG:boto:path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.981 DEBUG:boto:auth_path=/lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.981 DEBUG:boto:Method: GET 2024-04-16T17:58:22.981 DEBUG:boto:Path: /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.981 DEBUG:boto:Data: 2024-04-16T17:58:22.981 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.981 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:22.982 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.982 DEBUG:boto:Params: {} 2024-04-16T17:58:22.982 DEBUG:boto:Token: None 2024-04-16T17:58:22.982 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/obj?versionId=8t0iJsZveSu6vPfdfR1Iae1xiH5yyya 2024-04-16T17:58:22.982 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA= 2024-04-16T17:58:22.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GI1YHF6lSP7Fm37YF00vweA0byA=', 'Content-Length': '0'} 2024-04-16T17:58:22.985 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:46 GMT'), ('x-amz-version-id', '8t0iJsZveSu6vPfdfR1Iae1xiH5yyya'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006f18c7c5cb65cb6f-00661ebc3e-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:22.986 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:22.986 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:22.986 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.986 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:22.987 DEBUG:boto:Method: GET 2024-04-16T17:58:22.987 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:22.987 DEBUG:boto:Data: 2024-04-16T17:58:22.987 DEBUG:boto:Headers: {} 2024-04-16T17:58:22.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:22.988 DEBUG:boto:Port: 8000 2024-04-16T17:58:22.988 DEBUG:boto:Params: {} 2024-04-16T17:58:22.988 DEBUG:boto:Token: None 2024-04-16T17:58:22.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:22 GMT /lybtug-16/ 2024-04-16T17:58:22.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI= 2024-04-16T17:58:22.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VaOVfssIFtg9ZahS818DYZsbGZI=', 'Content-Length': '0'} 2024-04-16T17:58:22.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000586e519203a2ab81-00661ebc3e-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.034 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:23.035 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:23.035 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:23.035 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:23.035 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:23.035 DEBUG:boto:Method: GET 2024-04-16T17:58:23.035 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:23.035 DEBUG:boto:Data: 2024-04-16T17:58:23.036 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.036 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.036 DEBUG:boto:Params: {} 2024-04-16T17:58:23.036 DEBUG:boto:Token: None 2024-04-16T17:58:23.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/ 2024-04-16T17:58:23.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RrFPBg2cVED4smtOQrxp85gR7/Y= 2024-04-16T17:58:23.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RrFPBg2cVED4smtOQrxp85gR7/Y=', 'Content-Length': '0'} 2024-04-16T17:58:23.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1848298ff24be17-00661ebc3f-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.040 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:23.041 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:23.041 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:23.041 DEBUG:boto:Method: HEAD 2024-04-16T17:58:23.041 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:23.041 DEBUG:boto:Data: 2024-04-16T17:58:23.041 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.041 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.041 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.041 DEBUG:boto:Params: {} 2024-04-16T17:58:23.041 DEBUG:boto:Token: None 2024-04-16T17:58:23.042 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/obj 2024-04-16T17:58:23.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KJTKoZhby3JLCyrRIFTdsRLzl6E= 2024-04-16T17:58:23.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KJTKoZhby3JLCyrRIFTdsRLzl6E=', 'Content-Length': '0'} 2024-04-16T17:58:23.044 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000dcd20a5dd0088c2a-00661ebc3f-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.045 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:23.045 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:23.045 DEBUG:boto:Method: HEAD 2024-04-16T17:58:23.045 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:23.045 DEBUG:boto:Data: 2024-04-16T17:58:23.045 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.045 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.046 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.046 DEBUG:boto:Params: {} 2024-04-16T17:58:23.046 DEBUG:boto:Token: None 2024-04-16T17:58:23.046 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/obj 2024-04-16T17:58:23.046 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KJTKoZhby3JLCyrRIFTdsRLzl6E= 2024-04-16T17:58:23.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KJTKoZhby3JLCyrRIFTdsRLzl6E=', 'Content-Length': '0'} 2024-04-16T17:58:23.049 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000005fa5bd8575e5c37-00661ebc3f-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.049 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:23.049 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:23.050 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:23.050 DEBUG:boto:Method: GET 2024-04-16T17:58:23.050 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:23.050 DEBUG:boto:Data: 2024-04-16T17:58:23.050 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.050 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.050 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.050 DEBUG:boto:Params: {} 2024-04-16T17:58:23.050 DEBUG:boto:Token: None 2024-04-16T17:58:23.051 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/obj 2024-04-16T17:58:23.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:976WDodyiIm6OWynBVODF0D2NP8= 2024-04-16T17:58:23.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:976WDodyiIm6OWynBVODF0D2NP8=', 'Content-Length': '0'} 2024-04-16T17:58:23.053 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:13 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000b065aa9f03b8019-00661ebc3f-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.054 DEBUG:boto:path=/lybtug-16/obj 2024-04-16T17:58:23.054 DEBUG:boto:auth_path=/lybtug-16/obj 2024-04-16T17:58:23.054 DEBUG:boto:Method: GET 2024-04-16T17:58:23.054 DEBUG:boto:Path: /lybtug-16/obj 2024-04-16T17:58:23.054 DEBUG:boto:Data: 2024-04-16T17:58:23.054 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.054 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.054 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.054 DEBUG:boto:Params: {} 2024-04-16T17:58:23.054 DEBUG:boto:Token: None 2024-04-16T17:58:23.055 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/obj 2024-04-16T17:58:23.055 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:976WDodyiIm6OWynBVODF0D2NP8= 2024-04-16T17:58:23.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:976WDodyiIm6OWynBVODF0D2NP8=', 'Content-Length': '0'} 2024-04-16T17:58:23.058 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:57:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d8406ee87b647703-00661ebc3f-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.058 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:23.058 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:23.058 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:23.058 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:23.058 DEBUG:boto:Method: GET 2024-04-16T17:58:23.059 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:23.059 DEBUG:boto:Data: 2024-04-16T17:58:23.059 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.059 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.059 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.059 DEBUG:boto:Params: {} 2024-04-16T17:58:23.059 DEBUG:boto:Token: None 2024-04-16T17:58:23.059 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/ 2024-04-16T17:58:23.059 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RrFPBg2cVED4smtOQrxp85gR7/Y= 2024-04-16T17:58:23.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RrFPBg2cVED4smtOQrxp85gR7/Y=', 'Content-Length': '0'} 2024-04-16T17:58:23.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf4147b7d0954844-00661ebc3f-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.063 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:23.064 DEBUG:boto:path=/lybtug-16/ 2024-04-16T17:58:23.064 DEBUG:boto:auth_path=/lybtug-16/ 2024-04-16T17:58:23.064 DEBUG:boto:path=/lybtug-16/?prefix=obj 2024-04-16T17:58:23.064 DEBUG:boto:auth_path=/lybtug-16/?prefix=obj 2024-04-16T17:58:23.064 DEBUG:boto:Method: GET 2024-04-16T17:58:23.064 DEBUG:boto:Path: /lybtug-16/?prefix=obj 2024-04-16T17:58:23.064 DEBUG:boto:Data: 2024-04-16T17:58:23.064 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.065 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.065 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.065 DEBUG:boto:Params: {} 2024-04-16T17:58:23.065 DEBUG:boto:Token: None 2024-04-16T17:58:23.065 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:23 GMT /lybtug-16/ 2024-04-16T17:58:23.065 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RrFPBg2cVED4smtOQrxp85gR7/Y= 2024-04-16T17:58:23.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RrFPBg2cVED4smtOQrxp85gR7/Y=', 'Content-Length': '0'} 2024-04-16T17:58:23.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b15e3feffff90e7f-00661ebc3f-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.069 DEBUG:boto:b'lybtug-16obj1000falseobj2024-04-16T17:57:57.348Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:23.069 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-16 zones={test-zone2, test-zone1} 2024-04-16T17:58:23.070 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_version_suspended_incremental_sync ... ok 2024-04-16T17:58:23.071 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-17 2024-04-16T17:58:23.071 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:23.071 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:23.071 DEBUG:boto:Method: PUT 2024-04-16T17:58:23.072 DEBUG:boto:Path: /lybtug-17/ 2024-04-16T17:58:23.072 DEBUG:boto:Data: 2024-04-16T17:58:23.072 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.072 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.072 DEBUG:boto:Params: {} 2024-04-16T17:58:23.072 DEBUG:boto:Token: None 2024-04-16T17:58:23.072 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:23 GMT /lybtug-17/ 2024-04-16T17:58:23.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jt0lB3wUemRgCNsh7YoaYNCdSao= 2024-04-16T17:58:23.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jt0lB3wUemRgCNsh7YoaYNCdSao=', 'Content-Length': '0'} 2024-04-16T17:58:23.109 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000048ae6a7d42ea1047-00661ebc3f-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.109 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-18 2024-04-16T17:58:23.109 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:23.109 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:23.109 DEBUG:boto:Method: PUT 2024-04-16T17:58:23.109 DEBUG:boto:Path: /lybtug-18/ 2024-04-16T17:58:23.110 DEBUG:boto:Data: 2024-04-16T17:58:23.110 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.110 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.110 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.110 DEBUG:boto:Params: {} 2024-04-16T17:58:23.110 DEBUG:boto:Token: None 2024-04-16T17:58:23.110 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:23 GMT /lybtug-18/ 2024-04-16T17:58:23.110 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K7rGlCzo4YiqGk8JEIsIAlNTiUc= 2024-04-16T17:58:23.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K7rGlCzo4YiqGk8JEIsIAlNTiUc=', 'Content-Length': '0'} 2024-04-16T17:58:23.170 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000012f820ed74936242-00661ebc3f-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.170 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:23.171 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:23.171 DEBUG:boto:path=/lybtug-17/?versioning 2024-04-16T17:58:23.171 DEBUG:boto:auth_path=/lybtug-17/?versioning 2024-04-16T17:58:23.171 DEBUG:boto:Method: PUT 2024-04-16T17:58:23.171 DEBUG:boto:Path: /lybtug-17/?versioning 2024-04-16T17:58:23.171 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:58:23.171 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.171 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.171 DEBUG:boto:Params: {} 2024-04-16T17:58:23.172 DEBUG:boto:Token: None 2024-04-16T17:58:23.172 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:23 GMT /lybtug-17/?versioning 2024-04-16T17:58:23.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9GLTBKkHi9pm24LPzz0ptK+5CtY= 2024-04-16T17:58:23.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9GLTBKkHi9pm24LPzz0ptK+5CtY=', 'Content-Length': '228'} 2024-04-16T17:58:23.185 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000de82a5b7ac9483ea-00661ebc3f-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.185 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:23.185 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:23.185 DEBUG:boto:path=/lybtug-18/?versioning 2024-04-16T17:58:23.185 DEBUG:boto:auth_path=/lybtug-18/?versioning 2024-04-16T17:58:23.185 DEBUG:boto:Method: PUT 2024-04-16T17:58:23.185 DEBUG:boto:Path: /lybtug-18/?versioning 2024-04-16T17:58:23.186 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:58:23.186 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.186 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.186 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.186 DEBUG:boto:Params: {} 2024-04-16T17:58:23.186 DEBUG:boto:Token: None 2024-04-16T17:58:23.186 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:23 GMT /lybtug-18/?versioning 2024-04-16T17:58:23.186 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BVZhB7O8gexrRT1XNiiH8FO5y/k= 2024-04-16T17:58:23.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BVZhB7O8gexrRT1XNiiH8FO5y/k=', 'Content-Length': '228'} 2024-04-16T17:58:23.206 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000558f3ae9868de25a-00661ebc3f-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.206 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:23.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:23.206 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:58:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:23.318 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:58:23.348 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290303.204578_162.1","last_update":"2024-04-16T17:58:23.204578Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:58:23.348 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290303.204578_162.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:23.348 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:58:23.349 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:23.349 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:23.349 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:58:23.393 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:23.393 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:23.468+0000 7f5d2a0ca980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:58:23.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:23.468+0000 7f5d2a0ca980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:58:23.495 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290303.204578_162.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:23.204578Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:58:23Z"} 2024-04-16T17:58:23.496 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290303.204578_162.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:23.204578Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:58:23Z"} 2024-04-16T17:58:23.496 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290303.204578_162.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:23.204578Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:58:23.496 DEBUG:rgw_multi.tests:log_status={0: '1_1713290303.204578_162.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:23.496 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290303.204578_162.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:23.496 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:58:23.496 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:23.496 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:23.496 DEBUG:boto:Method: HEAD 2024-04-16T17:58:23.497 DEBUG:boto:Path: /lybtug-17/ 2024-04-16T17:58:23.497 DEBUG:boto:Data: 2024-04-16T17:58:23.497 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.497 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.497 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.497 DEBUG:boto:Params: {} 2024-04-16T17:58:23.497 DEBUG:boto:Token: None 2024-04-16T17:58:23.497 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:23 GMT /lybtug-17/ 2024-04-16T17:58:23.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iWCg8034Rhfqh/RUtrivaQm8JhU= 2024-04-16T17:58:23.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iWCg8034Rhfqh/RUtrivaQm8JhU=', 'Content-Length': '0'} 2024-04-16T17:58:23.502 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001ef48ee917d32d66-00661ebc3f-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.502 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:23.502 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:23.502 DEBUG:boto:Method: PUT 2024-04-16T17:58:23.502 DEBUG:boto:Path: /lybtug-17/obj 2024-04-16T17:58:23.502 DEBUG:boto:Data: 2024-04-16T17:58:23.502 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:58:23.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.503 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.503 DEBUG:boto:Params: {} 2024-04-16T17:58:23.503 DEBUG:boto:Token: None 2024-04-16T17:58:23.503 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:58:23 GMT /lybtug-17/obj 2024-04-16T17:58:23.503 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aVMBMdRdHcKoygJ8JhQfdy5R3DI= 2024-04-16T17:58:23.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aVMBMdRdHcKoygJ8JhQfdy5R3DI='} 2024-04-16T17:58:23.551 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-amz-request-id', 'tx00000d684a562e2fce425-00661ebc3f-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.551 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:23.551 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:23.552 DEBUG:boto:Method: HEAD 2024-04-16T17:58:23.552 DEBUG:boto:Path: /lybtug-17/ 2024-04-16T17:58:23.552 DEBUG:boto:Data: 2024-04-16T17:58:23.552 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.552 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.552 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.552 DEBUG:boto:Params: {} 2024-04-16T17:58:23.552 DEBUG:boto:Token: None 2024-04-16T17:58:23.552 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:23 GMT /lybtug-17/ 2024-04-16T17:58:23.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iWCg8034Rhfqh/RUtrivaQm8JhU= 2024-04-16T17:58:23.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iWCg8034Rhfqh/RUtrivaQm8JhU=', 'Content-Length': '0'} 2024-04-16T17:58:23.557 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e9445dd369ca1b0b-00661ebc3f-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.557 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:23.557 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:23.557 DEBUG:boto:Method: DELETE 2024-04-16T17:58:23.558 DEBUG:boto:Path: /lybtug-17/obj 2024-04-16T17:58:23.558 DEBUG:boto:Data: 2024-04-16T17:58:23.558 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.558 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:23.558 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.558 DEBUG:boto:Params: {} 2024-04-16T17:58:23.558 DEBUG:boto:Token: None 2024-04-16T17:58:23.558 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:58:23 GMT /lybtug-17/obj 2024-04-16T17:58:23.558 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g3KjXJOx+dhH+z4QfTdxl6V9B+M= 2024-04-16T17:58:23.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g3KjXJOx+dhH+z4QfTdxl6V9B+M=', 'Content-Length': '0'} 2024-04-16T17:58:23.576 DEBUG:boto:Response headers: [('x-amz-version-id', 'j-eQx3nIr1hzJrAZJsij-fi6KyinZzW'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx000008193267dc13da93a-00661ebc3f-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.577 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:23.577 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:23.577 DEBUG:boto:Method: HEAD 2024-04-16T17:58:23.577 DEBUG:boto:Path: /lybtug-18/ 2024-04-16T17:58:23.577 DEBUG:boto:Data: 2024-04-16T17:58:23.577 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.577 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.577 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.577 DEBUG:boto:Params: {} 2024-04-16T17:58:23.578 DEBUG:boto:Token: None 2024-04-16T17:58:23.578 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:23 GMT /lybtug-18/ 2024-04-16T17:58:23.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AMY3LBlZ0IGxb6/g5LiAJK8pL5A= 2024-04-16T17:58:23.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AMY3LBlZ0IGxb6/g5LiAJK8pL5A=', 'Content-Length': '0'} 2024-04-16T17:58:23.583 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eca37c1d0fc87c3f-00661ebc3f-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.583 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:23.583 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:23.584 DEBUG:boto:Method: PUT 2024-04-16T17:58:23.584 DEBUG:boto:Path: /lybtug-18/obj 2024-04-16T17:58:23.584 DEBUG:boto:Data: 2024-04-16T17:58:23.584 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:58:23.584 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.584 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.584 DEBUG:boto:Params: {} 2024-04-16T17:58:23.584 DEBUG:boto:Token: None 2024-04-16T17:58:23.584 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:58:23 GMT /lybtug-18/obj 2024-04-16T17:58:23.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WCv2dkNQ8p2OWdAhM1BED/9jvdo= 2024-04-16T17:58:23.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WCv2dkNQ8p2OWdAhM1BED/9jvdo='} 2024-04-16T17:58:23.630 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-amz-request-id', 'tx000009a978e86bdc0b223-00661ebc3f-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.631 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:23.631 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:23.631 DEBUG:boto:Method: HEAD 2024-04-16T17:58:23.631 DEBUG:boto:Path: /lybtug-18/ 2024-04-16T17:58:23.631 DEBUG:boto:Data: 2024-04-16T17:58:23.631 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.631 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.631 DEBUG:boto:Params: {} 2024-04-16T17:58:23.631 DEBUG:boto:Token: None 2024-04-16T17:58:23.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:23 GMT /lybtug-18/ 2024-04-16T17:58:23.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AMY3LBlZ0IGxb6/g5LiAJK8pL5A= 2024-04-16T17:58:23.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AMY3LBlZ0IGxb6/g5LiAJK8pL5A=', 'Content-Length': '0'} 2024-04-16T17:58:23.636 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eebe58e9ea8938aa-00661ebc3f-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.636 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:23.637 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:23.637 DEBUG:boto:Method: DELETE 2024-04-16T17:58:23.637 DEBUG:boto:Path: /lybtug-18/obj 2024-04-16T17:58:23.637 DEBUG:boto:Data: 2024-04-16T17:58:23.637 DEBUG:boto:Headers: {} 2024-04-16T17:58:23.637 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:23.637 DEBUG:boto:Port: 8000 2024-04-16T17:58:23.637 DEBUG:boto:Params: {} 2024-04-16T17:58:23.637 DEBUG:boto:Token: None 2024-04-16T17:58:23.637 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:58:23 GMT /lybtug-18/obj 2024-04-16T17:58:23.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PHhS1tG87q6koaXtyssOV4wkjXU= 2024-04-16T17:58:23.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PHhS1tG87q6koaXtyssOV4wkjXU=', 'Content-Length': '0'} 2024-04-16T17:58:23.654 DEBUG:boto:Response headers: [('x-amz-version-id', 'kxL8UdLpGom52nDXZ1cvHs35Et5HJsu'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx00000a8e3b3f756ec24d2-00661ebc3f-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:23.654 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-17 2024-04-16T17:58:23.654 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:23.654 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:23.655 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:23.704 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:23.704 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:23.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:23.788+0000 7f2c89e2e980 1 waiting to reach incremental sync.. 2024-04-16T17:58:28.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:28.787+0000 7f2c89e2e980 1 waiting to reach incremental sync.. 2024-04-16T17:58:33.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:33.787+0000 7f2c89e2e980 1 waiting to reach incremental sync.. 2024-04-16T17:58:38.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:38.791+0000 7f2c89e2e980 1 bucket sync caught up with source: 2024-04-16T17:58:38.788 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.22.12, , ] 2024-04-16T17:58:38.789 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000003.22.12, , ] 2024-04-16T17:58:38.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:38.791+0000 7f2c89e2e980 0 bucket checkpoint complete 2024-04-16T17:58:38.828 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-17 2024-04-16T17:58:38.828 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:38.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:38.829 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:38.880 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:38.880 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:38.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:38.957+0000 7fd385cb8980 1 waiting to reach incremental sync.. 2024-04-16T17:58:43.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:43.961+0000 7fd385cb8980 1 bucket sync caught up with source: 2024-04-16T17:58:43.962 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000002.29.6, , ] 2024-04-16T17:58:43.962 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000002.29.6, , ] 2024-04-16T17:58:43.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:43.961+0000 7fd385cb8980 0 bucket checkpoint complete 2024-04-16T17:58:44.004 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-17 zones={test-zone2, test-zone1} 2024-04-16T17:58:44.005 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.005 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.005 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.005 DEBUG:boto:Path: /lybtug-17/ 2024-04-16T17:58:44.005 DEBUG:boto:Data: 2024-04-16T17:58:44.005 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.005 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.005 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.006 DEBUG:boto:Params: {} 2024-04-16T17:58:44.006 DEBUG:boto:Token: None 2024-04-16T17:58:44.006 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/ 2024-04-16T17:58:44.006 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kQZPy+K3aJcRJu8BgfXfcaVFfPo= 2024-04-16T17:58:44.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kQZPy+K3aJcRJu8BgfXfcaVFfPo=', 'Content-Length': '0'} 2024-04-16T17:58:44.010 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aabbb4466784ec32-00661ebc54-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.011 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.011 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.011 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.011 DEBUG:boto:Path: /lybtug-17/ 2024-04-16T17:58:44.011 DEBUG:boto:Data: 2024-04-16T17:58:44.011 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.011 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.011 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.011 DEBUG:boto:Params: {} 2024-04-16T17:58:44.012 DEBUG:boto:Token: None 2024-04-16T17:58:44.012 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/ 2024-04-16T17:58:44.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kQZPy+K3aJcRJu8BgfXfcaVFfPo= 2024-04-16T17:58:44.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kQZPy+K3aJcRJu8BgfXfcaVFfPo=', 'Content-Length': '0'} 2024-04-16T17:58:44.015 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ac0a654e8cdb31d9-00661ebc54-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.015 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:58:44.015 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.015 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.015 DEBUG:boto:path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.016 DEBUG:boto:auth_path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.016 DEBUG:boto:Method: GET 2024-04-16T17:58:44.016 DEBUG:boto:Path: /lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.016 DEBUG:boto:Data: 2024-04-16T17:58:44.016 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.016 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.016 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.016 DEBUG:boto:Params: {} 2024-04-16T17:58:44.016 DEBUG:boto:Token: None 2024-04-16T17:58:44.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/?versions 2024-04-16T17:58:44.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE= 2024-04-16T17:58:44.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE=', 'Content-Length': '0'} 2024-04-16T17:58:44.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006eaba95ed54c4c39-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.066 DEBUG:boto:b'lybtug-17999falseobjj-eQx3nIr1hzJrAZJsij-fi6KyinZzWtrue2024-04-16T17:58:23.568ZRGW11111111111111111NormalobjQqi6E6molngLfg.kh0rZfNKt-DvVcK8false2024-04-16T17:58:23.511Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.067 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:58:44.067 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.067 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.067 DEBUG:boto:path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.067 DEBUG:boto:auth_path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.068 DEBUG:boto:Method: GET 2024-04-16T17:58:44.068 DEBUG:boto:Path: /lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.068 DEBUG:boto:Data: 2024-04-16T17:58:44.068 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.068 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.068 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.068 DEBUG:boto:Params: {} 2024-04-16T17:58:44.068 DEBUG:boto:Token: None 2024-04-16T17:58:44.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/?versions 2024-04-16T17:58:44.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE= 2024-04-16T17:58:44.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE=', 'Content-Length': '0'} 2024-04-16T17:58:44.072 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ac2ca09a9b24476-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.072 DEBUG:boto:b'lybtug-17999falseobjj-eQx3nIr1hzJrAZJsij-fi6KyinZzWtrue2024-04-16T17:58:23.568ZRGW11111111111111111NormalobjQqi6E6molngLfg.kh0rZfNKt-DvVcK8false2024-04-16T17:58:23.511Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.073 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.073 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.073 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.073 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.073 DEBUG:boto:path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.073 DEBUG:boto:auth_path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.073 DEBUG:boto:Method: GET 2024-04-16T17:58:44.073 DEBUG:boto:Path: /lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.073 DEBUG:boto:Data: 2024-04-16T17:58:44.073 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.074 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.074 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.074 DEBUG:boto:Params: {} 2024-04-16T17:58:44.074 DEBUG:boto:Token: None 2024-04-16T17:58:44.074 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/?versions 2024-04-16T17:58:44.074 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE= 2024-04-16T17:58:44.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE=', 'Content-Length': '0'} 2024-04-16T17:58:44.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023dde735eb75de0c-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.078 DEBUG:boto:b'lybtug-17999falseobjj-eQx3nIr1hzJrAZJsij-fi6KyinZzWtrue2024-04-16T17:58:23.568ZRGW11111111111111111NormalobjQqi6E6molngLfg.kh0rZfNKt-DvVcK8false2024-04-16T17:58:23.511Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.079 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.079 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.079 DEBUG:boto:path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.079 DEBUG:boto:auth_path=/lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.079 DEBUG:boto:Method: GET 2024-04-16T17:58:44.079 DEBUG:boto:Path: /lybtug-17/?versions&max-keys=999 2024-04-16T17:58:44.079 DEBUG:boto:Data: 2024-04-16T17:58:44.079 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.079 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.079 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.080 DEBUG:boto:Params: {} 2024-04-16T17:58:44.080 DEBUG:boto:Token: None 2024-04-16T17:58:44.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/?versions 2024-04-16T17:58:44.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE= 2024-04-16T17:58:44.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DxLKNEfBjLkEGwjH/FADsvf7rcE=', 'Content-Length': '0'} 2024-04-16T17:58:44.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097a28c7702dfe67a-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.126 DEBUG:boto:b'lybtug-17999falseobjj-eQx3nIr1hzJrAZJsij-fi6KyinZzWtrue2024-04-16T17:58:23.568ZRGW11111111111111111NormalobjQqi6E6molngLfg.kh0rZfNKt-DvVcK8false2024-04-16T17:58:23.511Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.127 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:44.127 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.127 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.127 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.127 DEBUG:boto:Path: /lybtug-17/obj 2024-04-16T17:58:44.127 DEBUG:boto:Data: 2024-04-16T17:58:44.127 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.127 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.127 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.128 DEBUG:boto:Params: {} 2024-04-16T17:58:44.128 DEBUG:boto:Token: None 2024-04-16T17:58:44.128 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj 2024-04-16T17:58:44.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI= 2024-04-16T17:58:44.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI=', 'Content-Length': '0'} 2024-04-16T17:58:44.132 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx000006886abd520dd5e07-00661ebc54-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.132 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.132 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.132 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.132 DEBUG:boto:Path: /lybtug-17/obj 2024-04-16T17:58:44.133 DEBUG:boto:Data: 2024-04-16T17:58:44.133 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.133 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.133 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.133 DEBUG:boto:Params: {} 2024-04-16T17:58:44.133 DEBUG:boto:Token: None 2024-04-16T17:58:44.133 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj 2024-04-16T17:58:44.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI= 2024-04-16T17:58:44.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI=', 'Content-Length': '0'} 2024-04-16T17:58:44.137 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx00000dabedd13e65094a5-00661ebc54-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.137 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:44.137 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.137 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.137 DEBUG:boto:path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.137 DEBUG:boto:auth_path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.137 DEBUG:boto:Method: GET 2024-04-16T17:58:44.138 DEBUG:boto:Path: /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.138 DEBUG:boto:Data: 2024-04-16T17:58:44.138 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.138 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.138 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.138 DEBUG:boto:Params: {} 2024-04-16T17:58:44.138 DEBUG:boto:Token: None 2024-04-16T17:58:44.138 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.138 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A= 2024-04-16T17:58:44.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A=', 'Content-Length': '0'} 2024-04-16T17:58:44.141 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:38 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000065996b5e604bd3a0-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.141 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.141 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.142 DEBUG:boto:path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.142 DEBUG:boto:auth_path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.142 DEBUG:boto:Method: GET 2024-04-16T17:58:44.142 DEBUG:boto:Path: /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.142 DEBUG:boto:Data: 2024-04-16T17:58:44.142 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.142 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.142 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.142 DEBUG:boto:Params: {} 2024-04-16T17:58:44.142 DEBUG:boto:Token: None 2024-04-16T17:58:44.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.143 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A= 2024-04-16T17:58:44.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A=', 'Content-Length': '0'} 2024-04-16T17:58:44.146 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a9209f7e150e4f17-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.146 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.146 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.146 DEBUG:boto:path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.146 DEBUG:boto:auth_path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.146 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.146 DEBUG:boto:Path: /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.146 DEBUG:boto:Data: 2024-04-16T17:58:44.147 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.147 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.147 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.147 DEBUG:boto:Params: {} 2024-04-16T17:58:44.147 DEBUG:boto:Token: None 2024-04-16T17:58:44.147 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1cSLU2dQkIl4hMadXxE/37qDmHY= 2024-04-16T17:58:44.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1cSLU2dQkIl4hMadXxE/37qDmHY=', 'Content-Length': '0'} 2024-04-16T17:58:44.150 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:38 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eef9b732dcf1d501-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.150 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.150 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.150 DEBUG:boto:path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.150 DEBUG:boto:auth_path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.151 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.151 DEBUG:boto:Path: /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.151 DEBUG:boto:Data: 2024-04-16T17:58:44.151 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.151 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.151 DEBUG:boto:Params: {} 2024-04-16T17:58:44.151 DEBUG:boto:Token: None 2024-04-16T17:58:44.151 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.151 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1cSLU2dQkIl4hMadXxE/37qDmHY= 2024-04-16T17:58:44.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1cSLU2dQkIl4hMadXxE/37qDmHY=', 'Content-Length': '0'} 2024-04-16T17:58:44.154 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ea9db39f84844d65-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.154 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:44.154 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.154 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.155 DEBUG:boto:path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.155 DEBUG:boto:auth_path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.155 DEBUG:boto:Method: GET 2024-04-16T17:58:44.155 DEBUG:boto:Path: /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.155 DEBUG:boto:Data: 2024-04-16T17:58:44.155 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.155 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.155 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.155 DEBUG:boto:Params: {} 2024-04-16T17:58:44.155 DEBUG:boto:Token: None 2024-04-16T17:58:44.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A= 2024-04-16T17:58:44.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A=', 'Content-Length': '0'} 2024-04-16T17:58:44.158 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:38 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003d8e387e92061426-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.159 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.159 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.159 DEBUG:boto:path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.159 DEBUG:boto:auth_path=/lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.159 DEBUG:boto:Method: GET 2024-04-16T17:58:44.159 DEBUG:boto:Path: /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.159 DEBUG:boto:Data: 2024-04-16T17:58:44.159 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.159 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.159 DEBUG:boto:Params: {} 2024-04-16T17:58:44.159 DEBUG:boto:Token: None 2024-04-16T17:58:44.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj?versionId=Qqi6E6molngLfg.kh0rZfNKt-DvVcK8 2024-04-16T17:58:44.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A= 2024-04-16T17:58:44.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+8B+EBuuclTxQdFXw0H2WTNOM6A=', 'Content-Length': '0'} 2024-04-16T17:58:44.162 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'Qqi6E6molngLfg.kh0rZfNKt-DvVcK8'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006c0b6ac092ef1436-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.163 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.163 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.163 DEBUG:boto:path=/lybtug-17/?prefix=obj 2024-04-16T17:58:44.163 DEBUG:boto:auth_path=/lybtug-17/?prefix=obj 2024-04-16T17:58:44.163 DEBUG:boto:Method: GET 2024-04-16T17:58:44.163 DEBUG:boto:Path: /lybtug-17/?prefix=obj 2024-04-16T17:58:44.163 DEBUG:boto:Data: 2024-04-16T17:58:44.163 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.163 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.163 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.164 DEBUG:boto:Params: {} 2024-04-16T17:58:44.165 DEBUG:boto:Token: None 2024-04-16T17:58:44.165 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/ 2024-04-16T17:58:44.165 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ucfhOceqB+K2lGO1fircpAeqL4w= 2024-04-16T17:58:44.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ucfhOceqB+K2lGO1fircpAeqL4w=', 'Content-Length': '0'} 2024-04-16T17:58:44.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adeb829f49ee2031-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.210 DEBUG:boto:b'lybtug-17obj1000false' 2024-04-16T17:58:44.211 DEBUG:boto:path=/lybtug-17/ 2024-04-16T17:58:44.211 DEBUG:boto:auth_path=/lybtug-17/ 2024-04-16T17:58:44.211 DEBUG:boto:path=/lybtug-17/?prefix=obj 2024-04-16T17:58:44.211 DEBUG:boto:auth_path=/lybtug-17/?prefix=obj 2024-04-16T17:58:44.211 DEBUG:boto:Method: GET 2024-04-16T17:58:44.211 DEBUG:boto:Path: /lybtug-17/?prefix=obj 2024-04-16T17:58:44.211 DEBUG:boto:Data: 2024-04-16T17:58:44.211 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.212 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.212 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.212 DEBUG:boto:Params: {} 2024-04-16T17:58:44.212 DEBUG:boto:Token: None 2024-04-16T17:58:44.212 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/ 2024-04-16T17:58:44.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ucfhOceqB+K2lGO1fircpAeqL4w= 2024-04-16T17:58:44.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ucfhOceqB+K2lGO1fircpAeqL4w=', 'Content-Length': '0'} 2024-04-16T17:58:44.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000607348c3df90be2f-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.216 DEBUG:boto:b'lybtug-17obj1000false' 2024-04-16T17:58:44.217 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.217 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.217 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.217 DEBUG:boto:Path: /lybtug-17/obj 2024-04-16T17:58:44.217 DEBUG:boto:Data: 2024-04-16T17:58:44.217 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.217 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.217 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.217 DEBUG:boto:Params: {} 2024-04-16T17:58:44.217 DEBUG:boto:Token: None 2024-04-16T17:58:44.218 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj 2024-04-16T17:58:44.218 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI= 2024-04-16T17:58:44.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI=', 'Content-Length': '0'} 2024-04-16T17:58:44.222 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx0000026b618c8df526e75-00661ebc54-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.222 DEBUG:boto:path=/lybtug-17/obj 2024-04-16T17:58:44.222 DEBUG:boto:auth_path=/lybtug-17/obj 2024-04-16T17:58:44.222 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.222 DEBUG:boto:Path: /lybtug-17/obj 2024-04-16T17:58:44.222 DEBUG:boto:Data: 2024-04-16T17:58:44.222 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.222 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.222 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.223 DEBUG:boto:Params: {} 2024-04-16T17:58:44.223 DEBUG:boto:Token: None 2024-04-16T17:58:44.223 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-17/obj 2024-04-16T17:58:44.223 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI= 2024-04-16T17:58:44.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WdrvBamkimKIEUVMKYug8yQoQhI=', 'Content-Length': '0'} 2024-04-16T17:58:44.226 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx00000979c34d242316367-00661ebc54-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.226 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-17 zones={test-zone2, test-zone1} 2024-04-16T17:58:44.227 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-18 2024-04-16T17:58:44.227 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:44.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:44.227 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:44.277 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:44.277 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:44.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:44.359+0000 7ff016b7e980 1 bucket sync caught up with source: 2024-04-16T17:58:44.355 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000002.16.6, , ] 2024-04-16T17:58:44.355 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000002.16.6, , ] 2024-04-16T17:58:44.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:44.359+0000 7ff016b7e980 0 bucket checkpoint complete 2024-04-16T17:58:44.385 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-18 2024-04-16T17:58:44.385 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:44.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:44.386 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-18 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:44.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:44.513+0000 7fc92e498980 1 bucket sync caught up with source: 2024-04-16T17:58:44.512 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000003.33.12, , ] 2024-04-16T17:58:44.512 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000003.33.12, , ] 2024-04-16T17:58:44.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:44.513+0000 7fc92e498980 0 bucket checkpoint complete 2024-04-16T17:58:44.541 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-18 zones={test-zone2, test-zone1} 2024-04-16T17:58:44.541 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.541 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.542 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.542 DEBUG:boto:Path: /lybtug-18/ 2024-04-16T17:58:44.542 DEBUG:boto:Data: 2024-04-16T17:58:44.542 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.542 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.542 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.542 DEBUG:boto:Params: {} 2024-04-16T17:58:44.542 DEBUG:boto:Token: None 2024-04-16T17:58:44.542 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/ 2024-04-16T17:58:44.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:53HbS08w8WJqkVG9ssSaGhukkOY= 2024-04-16T17:58:44.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:53HbS08w8WJqkVG9ssSaGhukkOY=', 'Content-Length': '0'} 2024-04-16T17:58:44.547 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bb70d2906a5f9cb9-00661ebc54-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.547 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.547 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.548 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.548 DEBUG:boto:Path: /lybtug-18/ 2024-04-16T17:58:44.548 DEBUG:boto:Data: 2024-04-16T17:58:44.548 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.548 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.548 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.548 DEBUG:boto:Params: {} 2024-04-16T17:58:44.548 DEBUG:boto:Token: None 2024-04-16T17:58:44.548 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/ 2024-04-16T17:58:44.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:53HbS08w8WJqkVG9ssSaGhukkOY= 2024-04-16T17:58:44.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:53HbS08w8WJqkVG9ssSaGhukkOY=', 'Content-Length': '0'} 2024-04-16T17:58:44.552 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000057cba0e59433537d-00661ebc54-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.552 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:58:44.552 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.553 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.553 DEBUG:boto:path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.553 DEBUG:boto:auth_path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.553 DEBUG:boto:Method: GET 2024-04-16T17:58:44.553 DEBUG:boto:Path: /lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.553 DEBUG:boto:Data: 2024-04-16T17:58:44.553 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.553 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.553 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.553 DEBUG:boto:Params: {} 2024-04-16T17:58:44.553 DEBUG:boto:Token: None 2024-04-16T17:58:44.554 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/?versions 2024-04-16T17:58:44.554 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA= 2024-04-16T17:58:44.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA=', 'Content-Length': '0'} 2024-04-16T17:58:44.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c94c5cc7c3904722-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.559 DEBUG:boto:b'lybtug-18999falseobjkxL8UdLpGom52nDXZ1cvHs35Et5HJsutrue2024-04-16T17:58:23.648ZRGW11111111111111111NormalobjSfP8zAAsFRyg4vxO6mzoG6FH6qdViO2false2024-04-16T17:58:23.594Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.559 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.560 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.560 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:58:44.560 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.560 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.560 DEBUG:boto:path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.560 DEBUG:boto:auth_path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.560 DEBUG:boto:Method: GET 2024-04-16T17:58:44.560 DEBUG:boto:Path: /lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.560 DEBUG:boto:Data: 2024-04-16T17:58:44.560 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.560 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.561 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.561 DEBUG:boto:Params: {} 2024-04-16T17:58:44.561 DEBUG:boto:Token: None 2024-04-16T17:58:44.561 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/?versions 2024-04-16T17:58:44.561 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA= 2024-04-16T17:58:44.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA=', 'Content-Length': '0'} 2024-04-16T17:58:44.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a80141953504fb5-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.565 DEBUG:boto:b'lybtug-18999falseobjkxL8UdLpGom52nDXZ1cvHs35Et5HJsutrue2024-04-16T17:58:23.648ZRGW11111111111111111NormalobjSfP8zAAsFRyg4vxO6mzoG6FH6qdViO2false2024-04-16T17:58:23.594Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.565 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.565 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:44.565 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.565 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.566 DEBUG:boto:path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.566 DEBUG:boto:auth_path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.566 DEBUG:boto:Method: GET 2024-04-16T17:58:44.567 DEBUG:boto:Path: /lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.567 DEBUG:boto:Data: 2024-04-16T17:58:44.567 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.567 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.567 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.567 DEBUG:boto:Params: {} 2024-04-16T17:58:44.567 DEBUG:boto:Token: None 2024-04-16T17:58:44.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/?versions 2024-04-16T17:58:44.567 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA= 2024-04-16T17:58:44.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA=', 'Content-Length': '0'} 2024-04-16T17:58:44.574 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7f2ada5bb049bcc-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.614 DEBUG:boto:b'lybtug-18999falseobjkxL8UdLpGom52nDXZ1cvHs35Et5HJsutrue2024-04-16T17:58:23.648ZRGW11111111111111111NormalobjSfP8zAAsFRyg4vxO6mzoG6FH6qdViO2false2024-04-16T17:58:23.594Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.615 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.615 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.615 DEBUG:boto:path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.615 DEBUG:boto:auth_path=/lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.615 DEBUG:boto:Method: GET 2024-04-16T17:58:44.615 DEBUG:boto:Path: /lybtug-18/?versions&max-keys=999 2024-04-16T17:58:44.616 DEBUG:boto:Data: 2024-04-16T17:58:44.616 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.616 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.616 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.616 DEBUG:boto:Params: {} 2024-04-16T17:58:44.616 DEBUG:boto:Token: None 2024-04-16T17:58:44.616 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/?versions 2024-04-16T17:58:44.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA= 2024-04-16T17:58:44.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+O2r01+QkWWb5VUYgiDkG6CuUiA=', 'Content-Length': '0'} 2024-04-16T17:58:44.621 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ab0b4a2a383fd25-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.621 DEBUG:boto:b'lybtug-18999falseobjkxL8UdLpGom52nDXZ1cvHs35Et5HJsutrue2024-04-16T17:58:23.648ZRGW11111111111111111NormalobjSfP8zAAsFRyg4vxO6mzoG6FH6qdViO2false2024-04-16T17:58:23.594Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDRGW11111111111111111Normal' 2024-04-16T17:58:44.621 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:44.621 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.622 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.622 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.622 DEBUG:boto:Path: /lybtug-18/obj 2024-04-16T17:58:44.622 DEBUG:boto:Data: 2024-04-16T17:58:44.622 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.622 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.622 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.622 DEBUG:boto:Params: {} 2024-04-16T17:58:44.623 DEBUG:boto:Token: None 2024-04-16T17:58:44.623 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj 2024-04-16T17:58:44.623 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI= 2024-04-16T17:58:44.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI=', 'Content-Length': '0'} 2024-04-16T17:58:44.626 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx0000030fdde1f120cc782-00661ebc54-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.627 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.627 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.627 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.627 DEBUG:boto:Path: /lybtug-18/obj 2024-04-16T17:58:44.627 DEBUG:boto:Data: 2024-04-16T17:58:44.627 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.627 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.627 DEBUG:boto:Params: {} 2024-04-16T17:58:44.628 DEBUG:boto:Token: None 2024-04-16T17:58:44.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj 2024-04-16T17:58:44.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI= 2024-04-16T17:58:44.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI=', 'Content-Length': '0'} 2024-04-16T17:58:44.631 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx0000081bdae0c10e6c417-00661ebc54-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:44.631 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.631 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.631 DEBUG:boto:path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.632 DEBUG:boto:auth_path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.632 DEBUG:boto:Method: GET 2024-04-16T17:58:44.632 DEBUG:boto:Path: /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.632 DEBUG:boto:Data: 2024-04-16T17:58:44.632 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.632 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.632 DEBUG:boto:Params: {} 2024-04-16T17:58:44.632 DEBUG:boto:Token: None 2024-04-16T17:58:44.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.633 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY= 2024-04-16T17:58:44.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY=', 'Content-Length': '0'} 2024-04-16T17:58:44.635 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fb6a7af629b782c7-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.636 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.636 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.636 DEBUG:boto:path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.636 DEBUG:boto:auth_path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.636 DEBUG:boto:Method: GET 2024-04-16T17:58:44.636 DEBUG:boto:Path: /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.636 DEBUG:boto:Data: 2024-04-16T17:58:44.636 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.636 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.636 DEBUG:boto:Params: {} 2024-04-16T17:58:44.637 DEBUG:boto:Token: None 2024-04-16T17:58:44.637 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY= 2024-04-16T17:58:44.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY=', 'Content-Length': '0'} 2024-04-16T17:58:44.640 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:27 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005edfc077ec2c7299-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.640 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.640 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.640 DEBUG:boto:path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.640 DEBUG:boto:auth_path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.640 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.640 DEBUG:boto:Path: /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.641 DEBUG:boto:Data: 2024-04-16T17:58:44.641 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.641 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.641 DEBUG:boto:Params: {} 2024-04-16T17:58:44.641 DEBUG:boto:Token: None 2024-04-16T17:58:44.641 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1fHHiNG8cOKIWnkRLplvNKF5qIs= 2024-04-16T17:58:44.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1fHHiNG8cOKIWnkRLplvNKF5qIs=', 'Content-Length': '0'} 2024-04-16T17:58:44.644 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000bc9ca082ab6cc0d-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.644 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.644 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.645 DEBUG:boto:path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.645 DEBUG:boto:auth_path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.645 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.645 DEBUG:boto:Path: /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.645 DEBUG:boto:Data: 2024-04-16T17:58:44.645 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.645 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.645 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.645 DEBUG:boto:Params: {} 2024-04-16T17:58:44.645 DEBUG:boto:Token: None 2024-04-16T17:58:44.646 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.646 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1fHHiNG8cOKIWnkRLplvNKF5qIs= 2024-04-16T17:58:44.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1fHHiNG8cOKIWnkRLplvNKF5qIs=', 'Content-Length': '0'} 2024-04-16T17:58:44.648 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:27 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a09aeed8b5048602-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.649 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:44.649 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.649 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.649 DEBUG:boto:path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.649 DEBUG:boto:auth_path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.649 DEBUG:boto:Method: GET 2024-04-16T17:58:44.649 DEBUG:boto:Path: /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.650 DEBUG:boto:Data: 2024-04-16T17:58:44.650 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.650 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.650 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.650 DEBUG:boto:Params: {} 2024-04-16T17:58:44.650 DEBUG:boto:Token: None 2024-04-16T17:58:44.650 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.650 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY= 2024-04-16T17:58:44.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY=', 'Content-Length': '0'} 2024-04-16T17:58:44.653 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000477cd9078d26e297-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.653 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.653 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.654 DEBUG:boto:path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.654 DEBUG:boto:auth_path=/lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.654 DEBUG:boto:Method: GET 2024-04-16T17:58:44.654 DEBUG:boto:Path: /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.654 DEBUG:boto:Data: 2024-04-16T17:58:44.654 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.654 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.654 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.654 DEBUG:boto:Params: {} 2024-04-16T17:58:44.655 DEBUG:boto:Token: None 2024-04-16T17:58:44.655 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj?versionId=SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2 2024-04-16T17:58:44.655 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY= 2024-04-16T17:58:44.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:45008MydYu/C4ZlYd4DRXQuRdXY=', 'Content-Length': '0'} 2024-04-16T17:58:44.657 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 17:58:23 GMT'), ('x-amz-version-id', 'SfP8zAAsFRyg4vxO6mzoG6FH6qdViO2'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 17:58:27 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000027e352c05b0ed7b5-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.657 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.658 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.658 DEBUG:boto:path=/lybtug-18/?prefix=obj 2024-04-16T17:58:44.658 DEBUG:boto:auth_path=/lybtug-18/?prefix=obj 2024-04-16T17:58:44.658 DEBUG:boto:Method: GET 2024-04-16T17:58:44.658 DEBUG:boto:Path: /lybtug-18/?prefix=obj 2024-04-16T17:58:44.658 DEBUG:boto:Data: 2024-04-16T17:58:44.658 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.658 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.658 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.659 DEBUG:boto:Params: {} 2024-04-16T17:58:44.659 DEBUG:boto:Token: None 2024-04-16T17:58:44.659 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/ 2024-04-16T17:58:44.659 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tEMcOdG49CCtz0ovE56cCASjfpg= 2024-04-16T17:58:44.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tEMcOdG49CCtz0ovE56cCASjfpg=', 'Content-Length': '0'} 2024-04-16T17:58:44.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000085dda6b41e62a0e2-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.706 DEBUG:boto:b'lybtug-18obj1000false' 2024-04-16T17:58:44.707 DEBUG:boto:path=/lybtug-18/ 2024-04-16T17:58:44.707 DEBUG:boto:auth_path=/lybtug-18/ 2024-04-16T17:58:44.707 DEBUG:boto:path=/lybtug-18/?prefix=obj 2024-04-16T17:58:44.707 DEBUG:boto:auth_path=/lybtug-18/?prefix=obj 2024-04-16T17:58:44.707 DEBUG:boto:Method: GET 2024-04-16T17:58:44.707 DEBUG:boto:Path: /lybtug-18/?prefix=obj 2024-04-16T17:58:44.707 DEBUG:boto:Data: 2024-04-16T17:58:44.707 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.708 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.708 DEBUG:boto:Params: {} 2024-04-16T17:58:44.708 DEBUG:boto:Token: None 2024-04-16T17:58:44.708 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/ 2024-04-16T17:58:44.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tEMcOdG49CCtz0ovE56cCASjfpg= 2024-04-16T17:58:44.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tEMcOdG49CCtz0ovE56cCASjfpg=', 'Content-Length': '0'} 2024-04-16T17:58:44.713 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000355bf040f8856fd1-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.714 DEBUG:boto:b'lybtug-18obj1000false' 2024-04-16T17:58:44.714 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.714 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.714 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.714 DEBUG:boto:Path: /lybtug-18/obj 2024-04-16T17:58:44.714 DEBUG:boto:Data: 2024-04-16T17:58:44.715 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.715 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.715 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.715 DEBUG:boto:Params: {} 2024-04-16T17:58:44.715 DEBUG:boto:Token: None 2024-04-16T17:58:44.715 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj 2024-04-16T17:58:44.715 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI= 2024-04-16T17:58:44.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI=', 'Content-Length': '0'} 2024-04-16T17:58:44.719 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx000007646a16b55066d23-00661ebc54-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.719 DEBUG:boto:path=/lybtug-18/obj 2024-04-16T17:58:44.719 DEBUG:boto:auth_path=/lybtug-18/obj 2024-04-16T17:58:44.719 DEBUG:boto:Method: HEAD 2024-04-16T17:58:44.719 DEBUG:boto:Path: /lybtug-18/obj 2024-04-16T17:58:44.719 DEBUG:boto:Data: 2024-04-16T17:58:44.720 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.720 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.720 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.720 DEBUG:boto:Params: {} 2024-04-16T17:58:44.720 DEBUG:boto:Token: None 2024-04-16T17:58:44.720 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:44 GMT /lybtug-18/obj 2024-04-16T17:58:44.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI= 2024-04-16T17:58:44.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T/1dcuHYlgXkXKVHB8ks+bLebZI=', 'Content-Length': '0'} 2024-04-16T17:58:44.724 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx0000067cc31aa7e724d7f-00661ebc54-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.724 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-18 zones={test-zone2, test-zone1} 2024-04-16T17:58:44.724 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_delete_marker_full_sync ... ok 2024-04-16T17:58:44.725 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-19 2024-04-16T17:58:44.725 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:44.725 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:44.726 DEBUG:boto:Method: PUT 2024-04-16T17:58:44.726 DEBUG:boto:Path: /lybtug-19/ 2024-04-16T17:58:44.726 DEBUG:boto:Data: 2024-04-16T17:58:44.726 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.726 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.726 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.726 DEBUG:boto:Params: {} 2024-04-16T17:58:44.726 DEBUG:boto:Token: None 2024-04-16T17:58:44.726 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:44 GMT /lybtug-19/ 2024-04-16T17:58:44.727 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I6u6MFeZsFgqvStgSAWgHyuy1eQ= 2024-04-16T17:58:44.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I6u6MFeZsFgqvStgSAWgHyuy1eQ=', 'Content-Length': '0'} 2024-04-16T17:58:44.761 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000bffd0ed0fdbe0d88-00661ebc54-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.761 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-20 2024-04-16T17:58:44.761 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:44.761 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:44.761 DEBUG:boto:Method: PUT 2024-04-16T17:58:44.762 DEBUG:boto:Path: /lybtug-20/ 2024-04-16T17:58:44.762 DEBUG:boto:Data: 2024-04-16T17:58:44.762 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.762 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.762 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.762 DEBUG:boto:Params: {} 2024-04-16T17:58:44.762 DEBUG:boto:Token: None 2024-04-16T17:58:44.762 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:44 GMT /lybtug-20/ 2024-04-16T17:58:44.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2SGeNrpJyNXXz/BAAKoIBnNWnWg= 2024-04-16T17:58:44.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2SGeNrpJyNXXz/BAAKoIBnNWnWg=', 'Content-Length': '0'} 2024-04-16T17:58:44.820 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000076467620bed2ca1a-00661ebc54-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.820 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:44.820 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:44.820 DEBUG:boto:path=/lybtug-19/?versioning 2024-04-16T17:58:44.820 DEBUG:boto:auth_path=/lybtug-19/?versioning 2024-04-16T17:58:44.820 DEBUG:boto:Method: PUT 2024-04-16T17:58:44.820 DEBUG:boto:Path: /lybtug-19/?versioning 2024-04-16T17:58:44.820 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:58:44.821 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.821 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.821 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.821 DEBUG:boto:Params: {} 2024-04-16T17:58:44.821 DEBUG:boto:Token: None 2024-04-16T17:58:44.821 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:44 GMT /lybtug-19/?versioning 2024-04-16T17:58:44.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sKHhtMiEq7hd2wbjtzzHIAv2IzQ= 2024-04-16T17:58:44.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sKHhtMiEq7hd2wbjtzzHIAv2IzQ=', 'Content-Length': '228'} 2024-04-16T17:58:44.833 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000060a31e9c67d25875-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.833 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:44.833 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:44.833 DEBUG:boto:path=/lybtug-19/?versioning 2024-04-16T17:58:44.833 DEBUG:boto:auth_path=/lybtug-19/?versioning 2024-04-16T17:58:44.834 DEBUG:boto:Method: PUT 2024-04-16T17:58:44.834 DEBUG:boto:Path: /lybtug-19/?versioning 2024-04-16T17:58:44.834 DEBUG:boto:Data: Suspended Disabled 2024-04-16T17:58:44.834 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.834 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:44.834 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.834 DEBUG:boto:Params: {} 2024-04-16T17:58:44.834 DEBUG:boto:Token: None 2024-04-16T17:58:44.834 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:44 GMT /lybtug-19/?versioning 2024-04-16T17:58:44.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sKHhtMiEq7hd2wbjtzzHIAv2IzQ= 2024-04-16T17:58:44.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sKHhtMiEq7hd2wbjtzzHIAv2IzQ=', 'Content-Length': '230'} 2024-04-16T17:58:44.847 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006f76080f9e3ffd8d-00661ebc54-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.848 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:44.848 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:44.848 DEBUG:boto:path=/lybtug-20/?versioning 2024-04-16T17:58:44.848 DEBUG:boto:auth_path=/lybtug-20/?versioning 2024-04-16T17:58:44.848 DEBUG:boto:Method: PUT 2024-04-16T17:58:44.848 DEBUG:boto:Path: /lybtug-20/?versioning 2024-04-16T17:58:44.848 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:58:44.848 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.848 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.849 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.849 DEBUG:boto:Params: {} 2024-04-16T17:58:44.849 DEBUG:boto:Token: None 2024-04-16T17:58:44.849 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:44 GMT /lybtug-20/?versioning 2024-04-16T17:58:44.849 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fwq4WxZyeSrn5eh/FniPXLtAWT4= 2024-04-16T17:58:44.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fwq4WxZyeSrn5eh/FniPXLtAWT4=', 'Content-Length': '228'} 2024-04-16T17:58:44.871 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e9166c3459dda440-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.872 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:44.872 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:44.872 DEBUG:boto:path=/lybtug-20/?versioning 2024-04-16T17:58:44.872 DEBUG:boto:auth_path=/lybtug-20/?versioning 2024-04-16T17:58:44.872 DEBUG:boto:Method: PUT 2024-04-16T17:58:44.872 DEBUG:boto:Path: /lybtug-20/?versioning 2024-04-16T17:58:44.872 DEBUG:boto:Data: Suspended Disabled 2024-04-16T17:58:44.872 DEBUG:boto:Headers: {} 2024-04-16T17:58:44.873 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:44.873 DEBUG:boto:Port: 8000 2024-04-16T17:58:44.873 DEBUG:boto:Params: {} 2024-04-16T17:58:44.873 DEBUG:boto:Token: None 2024-04-16T17:58:44.873 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:44 GMT /lybtug-20/?versioning 2024-04-16T17:58:44.873 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fwq4WxZyeSrn5eh/FniPXLtAWT4= 2024-04-16T17:58:44.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fwq4WxZyeSrn5eh/FniPXLtAWT4=', 'Content-Length': '230'} 2024-04-16T17:58:44.893 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000050304337605a2d64-00661ebc54-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:44.894 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:44.894 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:44.894 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:58:44.935 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:44.936 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:45.002 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:58:45.034 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290324.891185_179.1","last_update":"2024-04-16T17:58:44.891185Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:58:45.034 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290324.891185_179.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:45.034 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:58:45.035 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:45.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:45.035 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:58:45.082 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:45.082 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:45.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:45.151+0000 7f99ab2db980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:58:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:45.155+0000 7f99ab2db980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:58:45.180 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290324.891185_179.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:44.891185Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:58:45Z"} 2024-04-16T17:58:45.181 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290324.891185_179.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:44.891185Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:58:45Z"} 2024-04-16T17:58:45.181 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290324.891185_179.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:44.891185Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:58:45.181 DEBUG:rgw_multi.tests:log_status={0: '1_1713290324.891185_179.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:45.181 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290324.891185_179.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:45.181 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T17:58:45.181 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:45.181 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:45.182 DEBUG:boto:Method: HEAD 2024-04-16T17:58:45.182 DEBUG:boto:Path: /lybtug-19/ 2024-04-16T17:58:45.182 DEBUG:boto:Data: 2024-04-16T17:58:45.182 DEBUG:boto:Headers: {} 2024-04-16T17:58:45.182 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:45.182 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.182 DEBUG:boto:Params: {} 2024-04-16T17:58:45.182 DEBUG:boto:Token: None 2024-04-16T17:58:45.182 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:45 GMT /lybtug-19/ 2024-04-16T17:58:45.183 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H/fA2HXEbLdFRIM3/fy3xw4RG8A= 2024-04-16T17:58:45.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H/fA2HXEbLdFRIM3/fy3xw4RG8A=', 'Content-Length': '0'} 2024-04-16T17:58:45.188 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e3dfeb28f413b0ba-00661ebc55-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.189 DEBUG:boto:path=/lybtug-19/obj 2024-04-16T17:58:45.189 DEBUG:boto:auth_path=/lybtug-19/obj 2024-04-16T17:58:45.189 DEBUG:boto:Method: PUT 2024-04-16T17:58:45.189 DEBUG:boto:Path: /lybtug-19/obj 2024-04-16T17:58:45.189 DEBUG:boto:Data: 2024-04-16T17:58:45.189 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:58:45.189 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:45.189 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.189 DEBUG:boto:Params: {} 2024-04-16T17:58:45.189 DEBUG:boto:Token: None 2024-04-16T17:58:45.190 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:58:45 GMT /lybtug-19/obj 2024-04-16T17:58:45.190 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/fzSLCmkXqaacNq3yBR8Ide32VY= 2024-04-16T17:58:45.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/fzSLCmkXqaacNq3yBR8Ide32VY='} 2024-04-16T17:58:45.204 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cda4b8a13c1b24f6-00661ebc55-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.204 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:45.204 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:45.205 DEBUG:boto:Method: HEAD 2024-04-16T17:58:45.205 DEBUG:boto:Path: /lybtug-19/ 2024-04-16T17:58:45.205 DEBUG:boto:Data: 2024-04-16T17:58:45.205 DEBUG:boto:Headers: {} 2024-04-16T17:58:45.205 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:45.205 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.205 DEBUG:boto:Params: {} 2024-04-16T17:58:45.205 DEBUG:boto:Token: None 2024-04-16T17:58:45.205 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:45 GMT /lybtug-19/ 2024-04-16T17:58:45.205 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H/fA2HXEbLdFRIM3/fy3xw4RG8A= 2024-04-16T17:58:45.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H/fA2HXEbLdFRIM3/fy3xw4RG8A=', 'Content-Length': '0'} 2024-04-16T17:58:45.209 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001b9bc1caaf3143a6-00661ebc55-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.209 DEBUG:boto:path=/lybtug-19/obj 2024-04-16T17:58:45.209 DEBUG:boto:auth_path=/lybtug-19/obj 2024-04-16T17:58:45.209 DEBUG:boto:Method: DELETE 2024-04-16T17:58:45.209 DEBUG:boto:Path: /lybtug-19/obj 2024-04-16T17:58:45.209 DEBUG:boto:Data: 2024-04-16T17:58:45.209 DEBUG:boto:Headers: {} 2024-04-16T17:58:45.209 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:45.209 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.209 DEBUG:boto:Params: {} 2024-04-16T17:58:45.210 DEBUG:boto:Token: None 2024-04-16T17:58:45.210 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:58:45 GMT /lybtug-19/obj 2024-04-16T17:58:45.210 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eTPJm2zaGaH6tlPWmMXtyc5yN28= 2024-04-16T17:58:45.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eTPJm2zaGaH6tlPWmMXtyc5yN28=', 'Content-Length': '0'} 2024-04-16T17:58:45.233 DEBUG:boto:Response headers: [('x-amz-version-id', 'null'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx00000e8f16ecf6181f78e-00661ebc55-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.233 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:45.233 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:45.234 DEBUG:boto:Method: HEAD 2024-04-16T17:58:45.234 DEBUG:boto:Path: /lybtug-20/ 2024-04-16T17:58:45.234 DEBUG:boto:Data: 2024-04-16T17:58:45.234 DEBUG:boto:Headers: {} 2024-04-16T17:58:45.234 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:45.234 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.234 DEBUG:boto:Params: {} 2024-04-16T17:58:45.234 DEBUG:boto:Token: None 2024-04-16T17:58:45.235 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:45 GMT /lybtug-20/ 2024-04-16T17:58:45.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D53p4brgVw5Jcg/cfA5kEdp9yME= 2024-04-16T17:58:45.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D53p4brgVw5Jcg/cfA5kEdp9yME=', 'Content-Length': '0'} 2024-04-16T17:58:45.238 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000ed95d7305a72d1f-00661ebc55-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.239 DEBUG:boto:path=/lybtug-20/obj 2024-04-16T17:58:45.239 DEBUG:boto:auth_path=/lybtug-20/obj 2024-04-16T17:58:45.239 DEBUG:boto:Method: PUT 2024-04-16T17:58:45.239 DEBUG:boto:Path: /lybtug-20/obj 2024-04-16T17:58:45.239 DEBUG:boto:Data: 2024-04-16T17:58:45.239 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2024-04-16T17:58:45.239 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:45.239 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.240 DEBUG:boto:Params: {} 2024-04-16T17:58:45.240 DEBUG:boto:Token: None 2024-04-16T17:58:45.240 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Tue, 16 Apr 2024 17:58:45 GMT /lybtug-20/obj 2024-04-16T17:58:45.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:at2CCpeo/VinyTe+YIUjzcplyyo= 2024-04-16T17:58:45.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:at2CCpeo/VinyTe+YIUjzcplyyo='} 2024-04-16T17:58:45.253 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000082c7d6de10ba07dc-00661ebc55-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.253 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:45.254 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:45.254 DEBUG:boto:Method: HEAD 2024-04-16T17:58:45.254 DEBUG:boto:Path: /lybtug-20/ 2024-04-16T17:58:45.254 DEBUG:boto:Data: 2024-04-16T17:58:45.254 DEBUG:boto:Headers: {} 2024-04-16T17:58:45.254 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:45.254 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.254 DEBUG:boto:Params: {} 2024-04-16T17:58:45.255 DEBUG:boto:Token: None 2024-04-16T17:58:45.255 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:45 GMT /lybtug-20/ 2024-04-16T17:58:45.255 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D53p4brgVw5Jcg/cfA5kEdp9yME= 2024-04-16T17:58:45.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D53p4brgVw5Jcg/cfA5kEdp9yME=', 'Content-Length': '0'} 2024-04-16T17:58:45.258 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005efddcdfdd69e0c8-00661ebc55-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.259 DEBUG:boto:path=/lybtug-20/obj 2024-04-16T17:58:45.259 DEBUG:boto:auth_path=/lybtug-20/obj 2024-04-16T17:58:45.259 DEBUG:boto:Method: DELETE 2024-04-16T17:58:45.259 DEBUG:boto:Path: /lybtug-20/obj 2024-04-16T17:58:45.259 DEBUG:boto:Data: 2024-04-16T17:58:45.259 DEBUG:boto:Headers: {} 2024-04-16T17:58:45.259 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:45.259 DEBUG:boto:Port: 8000 2024-04-16T17:58:45.260 DEBUG:boto:Params: {} 2024-04-16T17:58:45.260 DEBUG:boto:Token: None 2024-04-16T17:58:45.260 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 17:58:45 GMT /lybtug-20/obj 2024-04-16T17:58:45.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kbm48yHbP4Cw90lq+10gpScUAQY= 2024-04-16T17:58:45.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kbm48yHbP4Cw90lq+10gpScUAQY=', 'Content-Length': '0'} 2024-04-16T17:58:45.282 DEBUG:boto:Response headers: [('x-amz-version-id', 'null'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx0000038eb3b96537dfaaf-00661ebc55-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:45.282 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-19 2024-04-16T17:58:45.283 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:45.283 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:45.283 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:45.332 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:45.332 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:45.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:45.415+0000 7f814bda2980 1 waiting to reach incremental sync.. 2024-04-16T17:58:50.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:50.415+0000 7f814bda2980 1 bucket sync caught up with source: 2024-04-16T17:58:50.413 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000003.33.7, , ] 2024-04-16T17:58:50.413 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000003.33.7, , ] 2024-04-16T17:58:50.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:50.415+0000 7f814bda2980 0 bucket checkpoint complete 2024-04-16T17:58:50.459 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-19 2024-04-16T17:58:50.459 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:50.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:50.459 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-19 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:50.509 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:50.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:50.585+0000 7f468f7fb980 1 waiting to reach incremental sync.. 2024-04-16T17:58:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:55.589+0000 7f468f7fb980 1 bucket sync caught up with source: 2024-04-16T17:58:55.589 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000001.30.14, , ] 2024-04-16T17:58:55.589 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000001.30.14, , ] 2024-04-16T17:58:55.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:55.589+0000 7f468f7fb980 0 bucket checkpoint complete 2024-04-16T17:58:55.633 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-19 zones={test-zone2, test-zone1} 2024-04-16T17:58:55.633 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:55.633 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:55.633 DEBUG:boto:Method: HEAD 2024-04-16T17:58:55.634 DEBUG:boto:Path: /lybtug-19/ 2024-04-16T17:58:55.634 DEBUG:boto:Data: 2024-04-16T17:58:55.634 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.634 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:55.634 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.634 DEBUG:boto:Params: {} 2024-04-16T17:58:55.635 DEBUG:boto:Token: None 2024-04-16T17:58:55.635 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/ 2024-04-16T17:58:55.635 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YJnc9aA2uvcIFwLr9t+x45WOMhU= 2024-04-16T17:58:55.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YJnc9aA2uvcIFwLr9t+x45WOMhU=', 'Content-Length': '0'} 2024-04-16T17:58:55.640 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000064ddecc2384f65c8-00661ebc5f-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.640 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:55.641 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:55.641 DEBUG:boto:Method: HEAD 2024-04-16T17:58:55.641 DEBUG:boto:Path: /lybtug-19/ 2024-04-16T17:58:55.641 DEBUG:boto:Data: 2024-04-16T17:58:55.641 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.641 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:55.642 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.642 DEBUG:boto:Params: {} 2024-04-16T17:58:55.642 DEBUG:boto:Token: None 2024-04-16T17:58:55.642 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/ 2024-04-16T17:58:55.643 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YJnc9aA2uvcIFwLr9t+x45WOMhU= 2024-04-16T17:58:55.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YJnc9aA2uvcIFwLr9t+x45WOMhU=', 'Content-Length': '0'} 2024-04-16T17:58:55.647 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000af623cc777826941-00661ebc5f-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.647 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:58:55.647 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:55.647 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:55.647 DEBUG:boto:path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.647 DEBUG:boto:auth_path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.647 DEBUG:boto:Method: GET 2024-04-16T17:58:55.648 DEBUG:boto:Path: /lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.648 DEBUG:boto:Data: 2024-04-16T17:58:55.648 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.648 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:55.648 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.648 DEBUG:boto:Params: {} 2024-04-16T17:58:55.648 DEBUG:boto:Token: None 2024-04-16T17:58:55.649 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/?versions 2024-04-16T17:58:55.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q= 2024-04-16T17:58:55.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q=', 'Content-Length': '0'} 2024-04-16T17:58:55.653 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4ed405f348840e1-00661ebc5f-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.653 DEBUG:boto:b'lybtug-19999falseobjnulltrue2024-04-16T17:58:45.218ZRGW11111111111111111Normal' 2024-04-16T17:58:55.653 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:55.654 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:58:55.654 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:55.654 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:55.654 DEBUG:boto:path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.654 DEBUG:boto:auth_path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.654 DEBUG:boto:Method: GET 2024-04-16T17:58:55.654 DEBUG:boto:Path: /lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.655 DEBUG:boto:Data: 2024-04-16T17:58:55.655 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:55.655 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.655 DEBUG:boto:Params: {} 2024-04-16T17:58:55.656 DEBUG:boto:Token: None 2024-04-16T17:58:55.656 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/?versions 2024-04-16T17:58:55.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q= 2024-04-16T17:58:55.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q=', 'Content-Length': '0'} 2024-04-16T17:58:55.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fdd7165ff0a36e42-00661ebc5f-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.659 DEBUG:boto:b'lybtug-19999falseobjnulltrue2024-04-16T17:58:45.218ZRGW11111111111111111Normal' 2024-04-16T17:58:55.659 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:55.659 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:55.659 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:55.659 DEBUG:boto:path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.660 DEBUG:boto:auth_path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.660 DEBUG:boto:Method: GET 2024-04-16T17:58:55.660 DEBUG:boto:Path: /lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.660 DEBUG:boto:Data: 2024-04-16T17:58:55.660 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.660 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:55.660 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.660 DEBUG:boto:Params: {} 2024-04-16T17:58:55.660 DEBUG:boto:Token: None 2024-04-16T17:58:55.660 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/?versions 2024-04-16T17:58:55.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q= 2024-04-16T17:58:55.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q=', 'Content-Length': '0'} 2024-04-16T17:58:55.664 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072e40e8fe961a3b7-00661ebc5f-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.665 DEBUG:boto:b'lybtug-19999falseobjnulltrue2024-04-16T17:58:45.218ZRGW11111111111111111Normal' 2024-04-16T17:58:55.665 DEBUG:boto:path=/lybtug-19/ 2024-04-16T17:58:55.665 DEBUG:boto:auth_path=/lybtug-19/ 2024-04-16T17:58:55.665 DEBUG:boto:path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.665 DEBUG:boto:auth_path=/lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.665 DEBUG:boto:Method: GET 2024-04-16T17:58:55.665 DEBUG:boto:Path: /lybtug-19/?versions&max-keys=999 2024-04-16T17:58:55.666 DEBUG:boto:Data: 2024-04-16T17:58:55.666 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.666 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:55.666 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.666 DEBUG:boto:Params: {} 2024-04-16T17:58:55.666 DEBUG:boto:Token: None 2024-04-16T17:58:55.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/?versions 2024-04-16T17:58:55.667 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q= 2024-04-16T17:58:55.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bEDQwcREvJx9E0SldgQ4lI0tC+Q=', 'Content-Length': '0'} 2024-04-16T17:58:55.670 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d0bafd5a05d1e5e-00661ebc5f-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.670 DEBUG:boto:b'lybtug-19999falseobjnulltrue2024-04-16T17:58:45.218ZRGW11111111111111111Normal' 2024-04-16T17:58:55.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:55.671 DEBUG:boto:path=/lybtug-19/obj 2024-04-16T17:58:55.671 DEBUG:boto:auth_path=/lybtug-19/obj 2024-04-16T17:58:55.671 DEBUG:boto:Method: HEAD 2024-04-16T17:58:55.671 DEBUG:boto:Path: /lybtug-19/obj 2024-04-16T17:58:55.671 DEBUG:boto:Data: 2024-04-16T17:58:55.671 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:55.671 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.671 DEBUG:boto:Params: {} 2024-04-16T17:58:55.671 DEBUG:boto:Token: None 2024-04-16T17:58:55.672 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/obj 2024-04-16T17:58:55.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2zQT+TWccQ2qx5NNwII3J7XHHKw= 2024-04-16T17:58:55.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2zQT+TWccQ2qx5NNwII3J7XHHKw=', 'Content-Length': '0'} 2024-04-16T17:58:55.674 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx00000fd167b43a9974365-00661ebc5f-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.674 DEBUG:boto:path=/lybtug-19/obj 2024-04-16T17:58:55.675 DEBUG:boto:auth_path=/lybtug-19/obj 2024-04-16T17:58:55.675 DEBUG:boto:Method: HEAD 2024-04-16T17:58:55.675 DEBUG:boto:Path: /lybtug-19/obj 2024-04-16T17:58:55.675 DEBUG:boto:Data: 2024-04-16T17:58:55.675 DEBUG:boto:Headers: {} 2024-04-16T17:58:55.675 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:55.675 DEBUG:boto:Port: 8000 2024-04-16T17:58:55.675 DEBUG:boto:Params: {} 2024-04-16T17:58:55.675 DEBUG:boto:Token: None 2024-04-16T17:58:55.675 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:55 GMT /lybtug-19/obj 2024-04-16T17:58:55.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2zQT+TWccQ2qx5NNwII3J7XHHKw= 2024-04-16T17:58:55.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2zQT+TWccQ2qx5NNwII3J7XHHKw=', 'Content-Length': '0'} 2024-04-16T17:58:55.678 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx00000f25bde60915d7420-00661ebc5f-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:55.678 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-19 zones={test-zone2, test-zone1} 2024-04-16T17:58:55.678 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-20 2024-04-16T17:58:55.678 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:55.679 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:55.679 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:55.730 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:55.730 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:55.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:55.815+0000 7f56750b9980 1 bucket sync caught up with source: 2024-04-16T17:58:55.812 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.31.14, , ] 2024-04-16T17:58:55.812 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000001.31.14, , ] 2024-04-16T17:58:55.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:55.815+0000 7f56750b9980 0 bucket checkpoint complete 2024-04-16T17:58:55.841 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-20 2024-04-16T17:58:55.841 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:55.841 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T17:58:55.841 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T17:58:55.892 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:55.892 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:55.973+0000 7f7fdd3b2980 1 bucket sync caught up with source: 2024-04-16T17:58:55.972 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000003.47.7, , ] 2024-04-16T17:58:55.972 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000003.47.7, , ] 2024-04-16T17:58:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T17:58:55.973+0000 7f7fdd3b2980 0 bucket checkpoint complete 2024-04-16T17:58:56.001 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-20 zones={test-zone2, test-zone1} 2024-04-16T17:58:56.002 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:56.002 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:56.002 DEBUG:boto:Method: HEAD 2024-04-16T17:58:56.002 DEBUG:boto:Path: /lybtug-20/ 2024-04-16T17:58:56.002 DEBUG:boto:Data: 2024-04-16T17:58:56.002 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.002 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.002 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.003 DEBUG:boto:Params: {} 2024-04-16T17:58:56.003 DEBUG:boto:Token: None 2024-04-16T17:58:56.003 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/ 2024-04-16T17:58:56.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sd0rv72y4u7twTPS19BpZkSDJZk= 2024-04-16T17:58:56.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sd0rv72y4u7twTPS19BpZkSDJZk=', 'Content-Length': '0'} 2024-04-16T17:58:56.007 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e4eb42580ef7f262-00661ebc60-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.007 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:56.007 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:56.007 DEBUG:boto:Method: HEAD 2024-04-16T17:58:56.007 DEBUG:boto:Path: /lybtug-20/ 2024-04-16T17:58:56.008 DEBUG:boto:Data: 2024-04-16T17:58:56.008 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.008 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.008 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.008 DEBUG:boto:Params: {} 2024-04-16T17:58:56.008 DEBUG:boto:Token: None 2024-04-16T17:58:56.008 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/ 2024-04-16T17:58:56.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sd0rv72y4u7twTPS19BpZkSDJZk= 2024-04-16T17:58:56.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sd0rv72y4u7twTPS19BpZkSDJZk=', 'Content-Length': '0'} 2024-04-16T17:58:56.011 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b91b25b7e628da3f-00661ebc60-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.011 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T17:58:56.011 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:56.012 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:56.012 DEBUG:boto:path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.012 DEBUG:boto:auth_path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.012 DEBUG:boto:Method: GET 2024-04-16T17:58:56.012 DEBUG:boto:Path: /lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.012 DEBUG:boto:Data: 2024-04-16T17:58:56.012 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.012 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.012 DEBUG:boto:Params: {} 2024-04-16T17:58:56.012 DEBUG:boto:Token: None 2024-04-16T17:58:56.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/?versions 2024-04-16T17:58:56.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo= 2024-04-16T17:58:56.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo=', 'Content-Length': '0'} 2024-04-16T17:58:56.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a88e835b266c4363-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.017 DEBUG:boto:b'lybtug-20999falseobjnulltrue2024-04-16T17:58:45.270ZRGW11111111111111111Normal' 2024-04-16T17:58:56.017 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:56.017 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T17:58:56.017 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:56.017 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:56.017 DEBUG:boto:path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.017 DEBUG:boto:auth_path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.018 DEBUG:boto:Method: GET 2024-04-16T17:58:56.018 DEBUG:boto:Path: /lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.018 DEBUG:boto:Data: 2024-04-16T17:58:56.018 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.018 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.018 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.018 DEBUG:boto:Params: {} 2024-04-16T17:58:56.018 DEBUG:boto:Token: None 2024-04-16T17:58:56.018 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/?versions 2024-04-16T17:58:56.018 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo= 2024-04-16T17:58:56.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo=', 'Content-Length': '0'} 2024-04-16T17:58:56.022 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e3e82179aa5febd-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.022 DEBUG:boto:b'lybtug-20999falseobjnulltrue2024-04-16T17:58:45.270ZRGW11111111111111111Normal' 2024-04-16T17:58:56.022 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2024-04-16T17:58:56.022 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:56.022 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:56.023 DEBUG:boto:path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.023 DEBUG:boto:auth_path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.023 DEBUG:boto:Method: GET 2024-04-16T17:58:56.023 DEBUG:boto:Path: /lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.023 DEBUG:boto:Data: 2024-04-16T17:58:56.023 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.023 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.023 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.023 DEBUG:boto:Params: {} 2024-04-16T17:58:56.023 DEBUG:boto:Token: None 2024-04-16T17:58:56.023 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/?versions 2024-04-16T17:58:56.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo= 2024-04-16T17:58:56.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo=', 'Content-Length': '0'} 2024-04-16T17:58:56.028 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cc1d5a9205ef5a6-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.028 DEBUG:boto:b'lybtug-20999falseobjnulltrue2024-04-16T17:58:45.270ZRGW11111111111111111Normal' 2024-04-16T17:58:56.029 DEBUG:boto:path=/lybtug-20/ 2024-04-16T17:58:56.029 DEBUG:boto:auth_path=/lybtug-20/ 2024-04-16T17:58:56.029 DEBUG:boto:path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.029 DEBUG:boto:auth_path=/lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.029 DEBUG:boto:Method: GET 2024-04-16T17:58:56.029 DEBUG:boto:Path: /lybtug-20/?versions&max-keys=999 2024-04-16T17:58:56.029 DEBUG:boto:Data: 2024-04-16T17:58:56.029 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.029 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.029 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.029 DEBUG:boto:Params: {} 2024-04-16T17:58:56.030 DEBUG:boto:Token: None 2024-04-16T17:58:56.030 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/?versions 2024-04-16T17:58:56.030 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo= 2024-04-16T17:58:56.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cYA/PSv3WdFD6224tNUWGnCSrAo=', 'Content-Length': '0'} 2024-04-16T17:58:56.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000574f1e37b956c457-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.033 DEBUG:boto:b'lybtug-20999falseobjnulltrue2024-04-16T17:58:45.270ZRGW11111111111111111Normal' 2024-04-16T17:58:56.033 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2024-04-16T17:58:56.033 DEBUG:boto:path=/lybtug-20/obj 2024-04-16T17:58:56.034 DEBUG:boto:auth_path=/lybtug-20/obj 2024-04-16T17:58:56.034 DEBUG:boto:Method: HEAD 2024-04-16T17:58:56.034 DEBUG:boto:Path: /lybtug-20/obj 2024-04-16T17:58:56.034 DEBUG:boto:Data: 2024-04-16T17:58:56.034 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.034 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.034 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.034 DEBUG:boto:Params: {} 2024-04-16T17:58:56.035 DEBUG:boto:Token: None 2024-04-16T17:58:56.035 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/obj 2024-04-16T17:58:56.035 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C5EJa97WqwsDs1OvGkT9xWXKQy8= 2024-04-16T17:58:56.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C5EJa97WqwsDs1OvGkT9xWXKQy8=', 'Content-Length': '0'} 2024-04-16T17:58:56.037 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx000000df5582b950a0f0a-00661ebc60-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.038 DEBUG:boto:path=/lybtug-20/obj 2024-04-16T17:58:56.038 DEBUG:boto:auth_path=/lybtug-20/obj 2024-04-16T17:58:56.038 DEBUG:boto:Method: HEAD 2024-04-16T17:58:56.038 DEBUG:boto:Path: /lybtug-20/obj 2024-04-16T17:58:56.038 DEBUG:boto:Data: 2024-04-16T17:58:56.038 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.038 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.038 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.038 DEBUG:boto:Params: {} 2024-04-16T17:58:56.038 DEBUG:boto:Token: None 2024-04-16T17:58:56.038 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 17:58:56 GMT /lybtug-20/obj 2024-04-16T17:58:56.039 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C5EJa97WqwsDs1OvGkT9xWXKQy8= 2024-04-16T17:58:56.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C5EJa97WqwsDs1OvGkT9xWXKQy8=', 'Content-Length': '0'} 2024-04-16T17:58:56.040 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx000008a9268cd79f5cd95-00661ebc60-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.041 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-20 zones={test-zone2, test-zone1} 2024-04-16T17:58:56.041 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_suspended_delete_marker_full_sync ... ok 2024-04-16T17:58:56.041 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-21 2024-04-16T17:58:56.042 DEBUG:boto:path=/lybtug-21/ 2024-04-16T17:58:56.042 DEBUG:boto:auth_path=/lybtug-21/ 2024-04-16T17:58:56.042 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.042 DEBUG:boto:Path: /lybtug-21/ 2024-04-16T17:58:56.042 DEBUG:boto:Data: 2024-04-16T17:58:56.042 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.042 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.042 DEBUG:boto:Params: {} 2024-04-16T17:58:56.042 DEBUG:boto:Token: None 2024-04-16T17:58:56.042 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-21/ 2024-04-16T17:58:56.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VoYfgGdprD+989vBONQIR24iHHs= 2024-04-16T17:58:56.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VoYfgGdprD+989vBONQIR24iHHs=', 'Content-Length': '0'} 2024-04-16T17:58:56.071 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000059aed116e188983c-00661ebc60-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.071 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-22 2024-04-16T17:58:56.071 DEBUG:boto:path=/lybtug-22/ 2024-04-16T17:58:56.071 DEBUG:boto:auth_path=/lybtug-22/ 2024-04-16T17:58:56.071 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.071 DEBUG:boto:Path: /lybtug-22/ 2024-04-16T17:58:56.072 DEBUG:boto:Data: 2024-04-16T17:58:56.072 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.072 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.072 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.072 DEBUG:boto:Params: {} 2024-04-16T17:58:56.072 DEBUG:boto:Token: None 2024-04-16T17:58:56.072 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-22/ 2024-04-16T17:58:56.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wMhhCc3Y237tLuHp9lGxUiz9v+o= 2024-04-16T17:58:56.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wMhhCc3Y237tLuHp9lGxUiz9v+o=', 'Content-Length': '0'} 2024-04-16T17:58:56.125 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000025837e7e21bdb432-00661ebc60-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.125 DEBUG:boto:path=/lybtug-21/ 2024-04-16T17:58:56.125 DEBUG:boto:auth_path=/lybtug-21/ 2024-04-16T17:58:56.125 DEBUG:boto:path=/lybtug-21/?versioning 2024-04-16T17:58:56.126 DEBUG:boto:auth_path=/lybtug-21/?versioning 2024-04-16T17:58:56.126 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.126 DEBUG:boto:Path: /lybtug-21/?versioning 2024-04-16T17:58:56.126 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:58:56.126 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.126 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.126 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.126 DEBUG:boto:Params: {} 2024-04-16T17:58:56.126 DEBUG:boto:Token: None 2024-04-16T17:58:56.126 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-21/?versioning 2024-04-16T17:58:56.127 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hdjpfln65+SfmUyIC3f1MW5MgO0= 2024-04-16T17:58:56.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hdjpfln65+SfmUyIC3f1MW5MgO0=', 'Content-Length': '228'} 2024-04-16T17:58:56.139 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000016c33072f637333d-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.139 DEBUG:boto:path=/lybtug-21/ 2024-04-16T17:58:56.140 DEBUG:boto:auth_path=/lybtug-21/ 2024-04-16T17:58:56.140 DEBUG:boto:path=/lybtug-21/?versioning 2024-04-16T17:58:56.140 DEBUG:boto:auth_path=/lybtug-21/?versioning 2024-04-16T17:58:56.140 DEBUG:boto:Method: GET 2024-04-16T17:58:56.140 DEBUG:boto:Path: /lybtug-21/?versioning 2024-04-16T17:58:56.140 DEBUG:boto:Data: 2024-04-16T17:58:56.140 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.140 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.140 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.140 DEBUG:boto:Params: {} 2024-04-16T17:58:56.140 DEBUG:boto:Token: None 2024-04-16T17:58:56.141 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-21/?versioning 2024-04-16T17:58:56.141 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XAzV/Ul9rhau1i//8fNiLy6Sw/Y= 2024-04-16T17:58:56.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XAzV/Ul9rhau1i//8fNiLy6Sw/Y=', 'Content-Length': '0'} 2024-04-16T17:58:56.142 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000054665aa8c5fe9d8-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '192'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.142 DEBUG:boto:EnabledDisabled 2024-04-16T17:58:56.143 DEBUG:boto:path=/lybtug-22/ 2024-04-16T17:58:56.143 DEBUG:boto:auth_path=/lybtug-22/ 2024-04-16T17:58:56.143 DEBUG:boto:path=/lybtug-22/?versioning 2024-04-16T17:58:56.143 DEBUG:boto:auth_path=/lybtug-22/?versioning 2024-04-16T17:58:56.143 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.144 DEBUG:boto:Path: /lybtug-22/?versioning 2024-04-16T17:58:56.144 DEBUG:boto:Data: Enabled Disabled 2024-04-16T17:58:56.144 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.144 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.144 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.144 DEBUG:boto:Params: {} 2024-04-16T17:58:56.144 DEBUG:boto:Token: None 2024-04-16T17:58:56.144 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-22/?versioning 2024-04-16T17:58:56.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yjNu7zSyEQwOIcfKcK8KplFiarA= 2024-04-16T17:58:56.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yjNu7zSyEQwOIcfKcK8KplFiarA=', 'Content-Length': '228'} 2024-04-16T17:58:56.166 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000999f71de7f5e2eef-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.166 DEBUG:boto:path=/lybtug-22/ 2024-04-16T17:58:56.166 DEBUG:boto:auth_path=/lybtug-22/ 2024-04-16T17:58:56.166 DEBUG:boto:path=/lybtug-22/?versioning 2024-04-16T17:58:56.166 DEBUG:boto:auth_path=/lybtug-22/?versioning 2024-04-16T17:58:56.166 DEBUG:boto:Method: GET 2024-04-16T17:58:56.166 DEBUG:boto:Path: /lybtug-22/?versioning 2024-04-16T17:58:56.166 DEBUG:boto:Data: 2024-04-16T17:58:56.167 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.167 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.167 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.167 DEBUG:boto:Params: {} 2024-04-16T17:58:56.167 DEBUG:boto:Token: None 2024-04-16T17:58:56.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-22/?versioning 2024-04-16T17:58:56.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ahUS0UdBPj7jwBmRYstOq8FBnrc= 2024-04-16T17:58:56.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ahUS0UdBPj7jwBmRYstOq8FBnrc=', 'Content-Length': '0'} 2024-04-16T17:58:56.169 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007175c6db90cf8a92-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '192'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.210 DEBUG:boto:EnabledDisabled 2024-04-16T17:58:56.210 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_versioning ... ok 2024-04-16T17:58:56.211 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-23 2024-04-16T17:58:56.211 DEBUG:boto:path=/lybtug-23/ 2024-04-16T17:58:56.211 DEBUG:boto:auth_path=/lybtug-23/ 2024-04-16T17:58:56.211 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.211 DEBUG:boto:Path: /lybtug-23/ 2024-04-16T17:58:56.211 DEBUG:boto:Data: 2024-04-16T17:58:56.211 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.211 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.212 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.212 DEBUG:boto:Params: {} 2024-04-16T17:58:56.212 DEBUG:boto:Token: None 2024-04-16T17:58:56.212 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-23/ 2024-04-16T17:58:56.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:avuXhZmDjbM0cDSD52NFUa02XRw= 2024-04-16T17:58:56.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:avuXhZmDjbM0cDSD52NFUa02XRw=', 'Content-Length': '0'} 2024-04-16T17:58:56.248 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000050e33b51290af1fa-00661ebc60-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.248 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-24 2024-04-16T17:58:56.248 DEBUG:boto:path=/lybtug-24/ 2024-04-16T17:58:56.249 DEBUG:boto:auth_path=/lybtug-24/ 2024-04-16T17:58:56.249 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.249 DEBUG:boto:Path: /lybtug-24/ 2024-04-16T17:58:56.249 DEBUG:boto:Data: 2024-04-16T17:58:56.249 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.249 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.249 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.249 DEBUG:boto:Params: {} 2024-04-16T17:58:56.249 DEBUG:boto:Token: None 2024-04-16T17:58:56.250 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-24/ 2024-04-16T17:58:56.250 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T54kT/SDJHEUEkt7thxW39PZRc0= 2024-04-16T17:58:56.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T54kT/SDJHEUEkt7thxW39PZRc0=', 'Content-Length': '0'} 2024-04-16T17:58:56.307 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000152cb9d402058aa6-00661ebc60-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.307 DEBUG:boto:path=/lybtug-23/ 2024-04-16T17:58:56.307 DEBUG:boto:auth_path=/lybtug-23/ 2024-04-16T17:58:56.307 DEBUG:boto:path=/lybtug-23/?acl 2024-04-16T17:58:56.307 DEBUG:boto:auth_path=/lybtug-23/?acl 2024-04-16T17:58:56.307 DEBUG:boto:Method: GET 2024-04-16T17:58:56.308 DEBUG:boto:Path: /lybtug-23/?acl 2024-04-16T17:58:56.308 DEBUG:boto:Data: 2024-04-16T17:58:56.308 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.308 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.308 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.308 DEBUG:boto:Params: {} 2024-04-16T17:58:56.308 DEBUG:boto:Token: None 2024-04-16T17:58:56.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-23/?acl 2024-04-16T17:58:56.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wa5nMyajxDyvaeVNH3xklGSrPLY= 2024-04-16T17:58:56.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wa5nMyajxDyvaeVNH3xklGSrPLY=', 'Content-Length': '0'} 2024-04-16T17:58:56.310 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000019f7ad9ebf70e93f-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '391'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.310 DEBUG:boto:path=/lybtug-23/ 2024-04-16T17:58:56.310 DEBUG:boto:auth_path=/lybtug-23/ 2024-04-16T17:58:56.311 DEBUG:boto:path=/lybtug-23/?acl 2024-04-16T17:58:56.311 DEBUG:boto:auth_path=/lybtug-23/?acl 2024-04-16T17:58:56.311 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.311 DEBUG:boto:Path: /lybtug-23/?acl 2024-04-16T17:58:56.311 DEBUG:boto:Data: 2024-04-16T17:58:56.311 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2024-04-16T17:58:56.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.311 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.311 DEBUG:boto:Params: {} 2024-04-16T17:58:56.311 DEBUG:boto:Token: None 2024-04-16T17:58:56.311 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT x-amz-acl:public-read /lybtug-23/?acl 2024-04-16T17:58:56.311 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LGSQ6kb2zpy6tj4/l1+L1v4QKUU= 2024-04-16T17:58:56.312 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LGSQ6kb2zpy6tj4/l1+L1v4QKUU=', 'Content-Length': '0'} 2024-04-16T17:58:56.325 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000148ca1306722c03f-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.325 DEBUG:boto:path=/lybtug-23/ 2024-04-16T17:58:56.325 DEBUG:boto:auth_path=/lybtug-23/ 2024-04-16T17:58:56.325 DEBUG:boto:path=/lybtug-23/?acl 2024-04-16T17:58:56.325 DEBUG:boto:auth_path=/lybtug-23/?acl 2024-04-16T17:58:56.325 DEBUG:boto:Method: GET 2024-04-16T17:58:56.325 DEBUG:boto:Path: /lybtug-23/?acl 2024-04-16T17:58:56.326 DEBUG:boto:Data: 2024-04-16T17:58:56.326 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.326 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.326 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.326 DEBUG:boto:Params: {} 2024-04-16T17:58:56.326 DEBUG:boto:Token: None 2024-04-16T17:58:56.326 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-23/?acl 2024-04-16T17:58:56.326 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wa5nMyajxDyvaeVNH3xklGSrPLY= 2024-04-16T17:58:56.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wa5nMyajxDyvaeVNH3xklGSrPLY=', 'Content-Length': '0'} 2024-04-16T17:58:56.328 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005e72cc200ba50a5f-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '583'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.371 DEBUG:boto:path=/lybtug-24/ 2024-04-16T17:58:56.371 DEBUG:boto:auth_path=/lybtug-24/ 2024-04-16T17:58:56.371 DEBUG:boto:path=/lybtug-24/?acl 2024-04-16T17:58:56.371 DEBUG:boto:auth_path=/lybtug-24/?acl 2024-04-16T17:58:56.371 DEBUG:boto:Method: GET 2024-04-16T17:58:56.371 DEBUG:boto:Path: /lybtug-24/?acl 2024-04-16T17:58:56.371 DEBUG:boto:Data: 2024-04-16T17:58:56.371 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.371 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.371 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.371 DEBUG:boto:Params: {} 2024-04-16T17:58:56.372 DEBUG:boto:Token: None 2024-04-16T17:58:56.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-24/?acl 2024-04-16T17:58:56.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZqFyoL+Ql+dppkwTv/fd42qc11s= 2024-04-16T17:58:56.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZqFyoL+Ql+dppkwTv/fd42qc11s=', 'Content-Length': '0'} 2024-04-16T17:58:56.374 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000015a8f06bb5dee6a-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '391'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.374 DEBUG:boto:path=/lybtug-24/ 2024-04-16T17:58:56.374 DEBUG:boto:auth_path=/lybtug-24/ 2024-04-16T17:58:56.374 DEBUG:boto:path=/lybtug-24/?acl 2024-04-16T17:58:56.374 DEBUG:boto:auth_path=/lybtug-24/?acl 2024-04-16T17:58:56.375 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.375 DEBUG:boto:Path: /lybtug-24/?acl 2024-04-16T17:58:56.375 DEBUG:boto:Data: 2024-04-16T17:58:56.375 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2024-04-16T17:58:56.375 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.375 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.375 DEBUG:boto:Params: {} 2024-04-16T17:58:56.375 DEBUG:boto:Token: None 2024-04-16T17:58:56.375 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT x-amz-acl:public-read /lybtug-24/?acl 2024-04-16T17:58:56.375 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JIgVq7JNxz1EcxJgmLdzsZn2QtE= 2024-04-16T17:58:56.375 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JIgVq7JNxz1EcxJgmLdzsZn2QtE=', 'Content-Length': '0'} 2024-04-16T17:58:56.397 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000001a05cbb3fcea5a5f-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.397 DEBUG:boto:path=/lybtug-24/ 2024-04-16T17:58:56.397 DEBUG:boto:auth_path=/lybtug-24/ 2024-04-16T17:58:56.397 DEBUG:boto:path=/lybtug-24/?acl 2024-04-16T17:58:56.397 DEBUG:boto:auth_path=/lybtug-24/?acl 2024-04-16T17:58:56.398 DEBUG:boto:Method: GET 2024-04-16T17:58:56.398 DEBUG:boto:Path: /lybtug-24/?acl 2024-04-16T17:58:56.398 DEBUG:boto:Data: 2024-04-16T17:58:56.398 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.398 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.398 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.398 DEBUG:boto:Params: {} 2024-04-16T17:58:56.398 DEBUG:boto:Token: None 2024-04-16T17:58:56.398 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-24/?acl 2024-04-16T17:58:56.398 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZqFyoL+Ql+dppkwTv/fd42qc11s= 2024-04-16T17:58:56.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZqFyoL+Ql+dppkwTv/fd42qc11s=', 'Content-Length': '0'} 2024-04-16T17:58:56.400 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000efccf34431bde5c3-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '583'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.400 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_acl ... ok 2024-04-16T17:58:56.401 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-25 2024-04-16T17:58:56.401 DEBUG:boto:path=/lybtug-25/ 2024-04-16T17:58:56.401 DEBUG:boto:auth_path=/lybtug-25/ 2024-04-16T17:58:56.401 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.401 DEBUG:boto:Path: /lybtug-25/ 2024-04-16T17:58:56.402 DEBUG:boto:Data: 2024-04-16T17:58:56.402 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.402 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.402 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.402 DEBUG:boto:Params: {} 2024-04-16T17:58:56.402 DEBUG:boto:Token: None 2024-04-16T17:58:56.402 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-25/ 2024-04-16T17:58:56.402 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jshy5g7E/J5d2FWvX782mmdeXNc= 2024-04-16T17:58:56.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jshy5g7E/J5d2FWvX782mmdeXNc=', 'Content-Length': '0'} 2024-04-16T17:58:56.438 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000055e708178dedbc00-00661ebc60-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.438 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-26 2024-04-16T17:58:56.438 DEBUG:boto:path=/lybtug-26/ 2024-04-16T17:58:56.438 DEBUG:boto:auth_path=/lybtug-26/ 2024-04-16T17:58:56.438 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.438 DEBUG:boto:Path: /lybtug-26/ 2024-04-16T17:58:56.438 DEBUG:boto:Data: 2024-04-16T17:58:56.438 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.439 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.439 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.439 DEBUG:boto:Params: {} 2024-04-16T17:58:56.439 DEBUG:boto:Token: None 2024-04-16T17:58:56.439 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:56 GMT /lybtug-26/ 2024-04-16T17:58:56.439 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j1GyXEqPANLbyFzSDDBzwMddlhw= 2024-04-16T17:58:56.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j1GyXEqPANLbyFzSDDBzwMddlhw=', 'Content-Length': '0'} 2024-04-16T17:58:56.505 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002d061a5ea342ac4b-00661ebc60-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.505 DEBUG:boto:path=/lybtug-25/ 2024-04-16T17:58:56.505 DEBUG:boto:auth_path=/lybtug-25/ 2024-04-16T17:58:56.505 DEBUG:boto:path=/lybtug-25/?cors 2024-04-16T17:58:56.505 DEBUG:boto:auth_path=/lybtug-25/?cors 2024-04-16T17:58:56.505 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.506 DEBUG:boto:Path: /lybtug-25/?cors 2024-04-16T17:58:56.506 DEBUG:boto:Data: DELETEhttps://www.example.com*3000 2024-04-16T17:58:56.506 DEBUG:boto:Headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml'} 2024-04-16T17:58:56.506 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.506 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.506 DEBUG:boto:Params: {} 2024-04-16T17:58:56.506 DEBUG:boto:Token: None 2024-04-16T17:58:56.506 DEBUG:boto:StringToSign: PUT c5gEzAIzM7zcw8WVE+AbrQ== text/xml Tue, 16 Apr 2024 17:58:56 GMT /lybtug-25/?cors 2024-04-16T17:58:56.507 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:myc8Gepm0ik6gG854EazVbkj9kc= 2024-04-16T17:58:56.507 DEBUG:boto:Final headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:myc8Gepm0ik6gG854EazVbkj9kc=', 'Content-Length': '218'} 2024-04-16T17:58:56.519 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000064cac86de72fa7c7-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.519 DEBUG:boto:path=/lybtug-25/ 2024-04-16T17:58:56.519 DEBUG:boto:auth_path=/lybtug-25/ 2024-04-16T17:58:56.520 DEBUG:boto:path=/lybtug-25/?cors 2024-04-16T17:58:56.520 DEBUG:boto:auth_path=/lybtug-25/?cors 2024-04-16T17:58:56.520 DEBUG:boto:Method: GET 2024-04-16T17:58:56.520 DEBUG:boto:Path: /lybtug-25/?cors 2024-04-16T17:58:56.520 DEBUG:boto:Data: 2024-04-16T17:58:56.520 DEBUG:boto:Headers: {} 2024-04-16T17:58:56.520 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:56.520 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.520 DEBUG:boto:Params: {} 2024-04-16T17:58:56.520 DEBUG:boto:Token: None 2024-04-16T17:58:56.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:56 GMT /lybtug-25/?cors 2024-04-16T17:58:56.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AbekBumRbDTL4zFO8RkteHRhgcU= 2024-04-16T17:58:56.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AbekBumRbDTL4zFO8RkteHRhgcU=', 'Content-Length': '0'} 2024-04-16T17:58:56.522 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000022c6a440b379231c-00661ebc60-4353-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '266'), ('Date', 'Tue, 16 Apr 2024 17:58:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:56.522 DEBUG:boto:b'DELETEhttps://www.example.com*3000' 2024-04-16T17:58:56.523 DEBUG:boto:path=/lybtug-26/ 2024-04-16T17:58:56.523 DEBUG:boto:auth_path=/lybtug-26/ 2024-04-16T17:58:56.523 DEBUG:boto:path=/lybtug-26/?cors 2024-04-16T17:58:56.523 DEBUG:boto:auth_path=/lybtug-26/?cors 2024-04-16T17:58:56.523 DEBUG:boto:Method: PUT 2024-04-16T17:58:56.523 DEBUG:boto:Path: /lybtug-26/?cors 2024-04-16T17:58:56.523 DEBUG:boto:Data: DELETEhttps://www.example.com*3000 2024-04-16T17:58:56.523 DEBUG:boto:Headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml'} 2024-04-16T17:58:56.523 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:56.523 DEBUG:boto:Port: 8000 2024-04-16T17:58:56.523 DEBUG:boto:Params: {} 2024-04-16T17:58:56.524 DEBUG:boto:Token: None 2024-04-16T17:58:56.524 DEBUG:boto:StringToSign: PUT c5gEzAIzM7zcw8WVE+AbrQ== text/xml Tue, 16 Apr 2024 17:58:56 GMT /lybtug-26/?cors 2024-04-16T17:58:56.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sbmBax4TdhtZrX2m/e/qPJNpM4A= 2024-04-16T17:58:56.524 DEBUG:boto:Final headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sbmBax4TdhtZrX2m/e/qPJNpM4A=', 'Content-Length': '218'} 2024-04-16T17:58:57.551 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b1aabc25b9a4986a-00661ebc60-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:57.551 DEBUG:boto:path=/lybtug-26/ 2024-04-16T17:58:57.551 DEBUG:boto:auth_path=/lybtug-26/ 2024-04-16T17:58:57.551 DEBUG:boto:path=/lybtug-26/?cors 2024-04-16T17:58:57.551 DEBUG:boto:auth_path=/lybtug-26/?cors 2024-04-16T17:58:57.552 DEBUG:boto:Method: GET 2024-04-16T17:58:57.552 DEBUG:boto:Path: /lybtug-26/?cors 2024-04-16T17:58:57.552 DEBUG:boto:Data: 2024-04-16T17:58:57.552 DEBUG:boto:Headers: {} 2024-04-16T17:58:57.552 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:57.552 DEBUG:boto:Port: 8000 2024-04-16T17:58:57.552 DEBUG:boto:Params: {} 2024-04-16T17:58:57.552 DEBUG:boto:Token: None 2024-04-16T17:58:57.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 17:58:57 GMT /lybtug-26/?cors 2024-04-16T17:58:57.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Aa59pUCttrnbUvLybWw7WCx+QHk= 2024-04-16T17:58:57.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Aa59pUCttrnbUvLybWw7WCx+QHk=', 'Content-Length': '0'} 2024-04-16T17:58:57.554 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000063cb9a5c88d9a2d8-00661ebc61-4335-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '266'), ('Date', 'Tue, 16 Apr 2024 17:58:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:57.598 DEBUG:boto:b'DELETEhttps://www.example.com*3000' 2024-04-16T17:58:57.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_cors ... ok 2024-04-16T17:58:57.599 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-27 2024-04-16T17:58:57.600 DEBUG:boto:path=/lybtug-27/ 2024-04-16T17:58:57.600 DEBUG:boto:auth_path=/lybtug-27/ 2024-04-16T17:58:57.600 DEBUG:boto:Method: PUT 2024-04-16T17:58:57.600 DEBUG:boto:Path: /lybtug-27/ 2024-04-16T17:58:57.600 DEBUG:boto:Data: 2024-04-16T17:58:57.600 DEBUG:boto:Headers: {} 2024-04-16T17:58:57.600 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T17:58:57.600 DEBUG:boto:Port: 8000 2024-04-16T17:58:57.600 DEBUG:boto:Params: {} 2024-04-16T17:58:57.600 DEBUG:boto:Token: None 2024-04-16T17:58:57.600 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:57 GMT /lybtug-27/ 2024-04-16T17:58:57.601 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtQb9Is1/+cPJRzB+ETVctXBn0I= 2024-04-16T17:58:57.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtQb9Is1/+cPJRzB+ETVctXBn0I=', 'Content-Length': '0'} 2024-04-16T17:58:57.638 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008addbfd71d46d321-00661ebc61-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:57.638 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-28 2024-04-16T17:58:57.639 DEBUG:boto:path=/lybtug-28/ 2024-04-16T17:58:57.639 DEBUG:boto:auth_path=/lybtug-28/ 2024-04-16T17:58:57.639 DEBUG:boto:Method: PUT 2024-04-16T17:58:57.639 DEBUG:boto:Path: /lybtug-28/ 2024-04-16T17:58:57.639 DEBUG:boto:Data: 2024-04-16T17:58:57.639 DEBUG:boto:Headers: {} 2024-04-16T17:58:57.639 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T17:58:57.639 DEBUG:boto:Port: 8000 2024-04-16T17:58:57.639 DEBUG:boto:Params: {} 2024-04-16T17:58:57.639 DEBUG:boto:Token: None 2024-04-16T17:58:57.640 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 17:58:57 GMT /lybtug-28/ 2024-04-16T17:58:57.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bZuf+zr8KhtE29PaZV/ZOPjmOlw= 2024-04-16T17:58:57.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:58:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bZuf+zr8KhtE29PaZV/ZOPjmOlw=', 'Content-Length': '0'} 2024-04-16T17:58:57.710 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000cb564247eadd61df-00661ebc61-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:58:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T17:58:57.710 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:57.710 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:57.711 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:58:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:57.819 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T17:58:57.852 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290337.679908_225.1","last_update":"2024-04-16T17:58:57.679908Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T17:58:57.852 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:57.852 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T17:58:57.853 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:57.853 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:58:57.853 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:58:57.894 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:58:57.895 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:58:57.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:57.967+0000 7efef2484980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:58:57.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:58:57.967+0000 7efef2484980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:58:57.995 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.096372_187.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.096372Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:58:57Z"} 2024-04-16T17:58:57.995 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.096372_187.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.096372Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:58:57Z"} 2024-04-16T17:58:57.996 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.096372_187.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.096372Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:58:57.996 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:57.996 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.096372_187.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:58:57.996 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.096372_187.1 2024-04-16T17:59:02.998 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:02.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:02.998 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:03.042 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:03.042 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:03.115+0000 7f61d45c9980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:03.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:03.131+0000 7f61d45c9980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:03.180 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.065204_184.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.065204Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:03Z"} 2024-04-16T17:59:03.180 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.065204_184.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.065204Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:03Z"} 2024-04-16T17:59:03.181 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.065204_184.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.065204Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:03.181 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:03.181 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.065204_184.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:03.181 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.065204_184.1 2024-04-16T17:59:08.181 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:08.182 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:08.182 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:08.233 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:08.233 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:08.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:08.303+0000 7fea2b294980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:08.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:08.303+0000 7fea2b294980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:08.331 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:08Z"} 2024-04-16T17:59:08.331 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:08Z"} 2024-04-16T17:59:08.331 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.134371_190.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.134371Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:08.331 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:08.332 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.134371_190.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:08.332 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.134371_190.1 2024-04-16T17:59:13.332 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:13.333 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:13.333 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:13.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:13.455+0000 7f4a7cc0a980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:13.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:13.459+0000 7f4a7cc0a980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:13.482 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:13Z"} 2024-04-16T17:59:13.482 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:13Z"} 2024-04-16T17:59:13.482 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.134371_190.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.134371Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:13.483 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:13.483 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.134371_190.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:13.483 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.134371_190.1 2024-04-16T17:59:18.484 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:18.485 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:18.485 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:18.527 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:18.527 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:18.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:18.639+0000 7feb77e82980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:18.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:18.639+0000 7feb77e82980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:18.663 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.090052_186.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.090052Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:18Z"} 2024-04-16T17:59:18.663 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.090052_186.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.090052Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:18Z"} 2024-04-16T17:59:18.663 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.090052_186.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.090052Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:18.663 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:18.663 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.090052_186.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:18.663 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.090052_186.1 2024-04-16T17:59:23.664 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:23.664 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:23.665 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:23.706 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:23.707 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:23.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:23.779+0000 7ff467480980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:23.783+0000 7ff467480980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:23.808 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.154151_192.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.154151Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:23Z"} 2024-04-16T17:59:23.808 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.154151_192.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.154151Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:23Z"} 2024-04-16T17:59:23.808 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.154151_192.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.154151Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:23.809 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:23.809 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.154151_192.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:23.809 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.154151_192.1 2024-04-16T17:59:28.810 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:28.810 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:28.811 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:28.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:28.943+0000 7f6de55a8980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:28.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:28.947+0000 7f6de55a8980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:28.970 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.154151_192.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.154151Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:28Z"} 2024-04-16T17:59:28.971 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.154151_192.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.154151Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:28Z"} 2024-04-16T17:59:28.971 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.154151_192.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.154151Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:28.971 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:28.971 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.154151_192.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:28.971 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.154151_192.1 2024-04-16T17:59:33.972 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:33.973 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:33.973 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:34.016 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:34.016 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:34.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:34.087+0000 7f6a4484b980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:34.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:34.091+0000 7f6a4484b980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:34.116 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.279336_200.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.279336Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:34Z"} 2024-04-16T17:59:34.116 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.279336_200.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.279336Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:34Z"} 2024-04-16T17:59:34.116 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.279336_200.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.279336Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:34.117 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:34.117 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.279336_200.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:34.117 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.279336_200.1 2024-04-16T17:59:39.117 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:39.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:39.118 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:39.156 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:39.156 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:39.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:39.227+0000 7fcf54351980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:39.231+0000 7fcf54351980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:39.256 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.062267_183.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.062267Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:39Z"} 2024-04-16T17:59:39.256 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.062267_183.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.062267Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:39Z"} 2024-04-16T17:59:39.256 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.062267_183.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.062267Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:39.256 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:39.256 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.062267_183.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:39.256 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.062267_183.1 2024-04-16T17:59:44.258 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:44.258 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:44.258 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:44.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:44.371+0000 7f46a39a6980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:44.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:44.375+0000 7f46a39a6980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:44.400 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:44Z"} 2024-04-16T17:59:44.400 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:44Z"} 2024-04-16T17:59:44.400 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.134371_190.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.134371Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:44.400 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:44.400 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.134371_190.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:44.400 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.134371_190.1 2024-04-16T17:59:49.402 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:49.402 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:49.402 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:49.451 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:49.451 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:49.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:49.523+0000 7fac6513a980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:49.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:49.523+0000 7fac6513a980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:49.551 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:49Z"} 2024-04-16T17:59:49.551 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.134371_190.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.134371Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:49Z"} 2024-04-16T17:59:49.552 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.134371_190.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.134371Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:49.552 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:49.552 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.134371_190.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:49.552 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.134371_190.1 2024-04-16T17:59:54.553 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:54.554 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:54.554 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:54.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:54.699+0000 7fc5955fe980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:54.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:54.699+0000 7fc5955fe980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:54.721 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.090052_186.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.090052Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:54Z"} 2024-04-16T17:59:54.721 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.090052_186.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.090052Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:54Z"} 2024-04-16T17:59:54.722 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.090052_186.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.090052Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:54.722 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:54.722 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.090052_186.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:54.722 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.090052_186.1 2024-04-16T17:59:59.722 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:59.723 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T17:59:59.723 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T17:59:59.765 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T17:59:59.765 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T17:59:59.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:59.839+0000 7efe96597980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T17:59:59.839+0000 7efe96597980 20 RGW-SYNC:meta: read sync status 2024-04-16T17:59:59.866 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.071742_185.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.071742Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:59Z"} 2024-04-16T17:59:59.866 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.071742_185.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.071742Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:59:59Z"} 2024-04-16T17:59:59.866 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.071742_185.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.071742Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T17:59:59.867 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:59.867 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.071742_185.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T17:59:59.867 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.071742_185.1 2024-04-16T18:00:04.868 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:04.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:04.868 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:04.912 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:04.913 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:04.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:04.986+0000 7fb3826ce980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:04.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:04.986+0000 7fb3826ce980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:05.014 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.071742_185.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.071742Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:04Z"} 2024-04-16T18:00:05.014 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.071742_185.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.071742Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:04Z"} 2024-04-16T18:00:05.015 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.071742_185.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.071742Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:00:05.015 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:05.015 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.071742_185.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:05.015 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.071742_185.1 2024-04-16T18:00:10.017 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:10.017 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:10.017 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:10.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:10.130+0000 7fd4e6254980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:10.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:10.134+0000 7fd4e6254980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:10.160 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.090052_186.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.090052Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:10Z"} 2024-04-16T18:00:10.160 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.090052_186.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.090052Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:10Z"} 2024-04-16T18:00:10.160 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.090052_186.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.090052Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:00:10.160 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:10.161 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.090052_186.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:10.161 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.090052_186.1 2024-04-16T18:00:15.161 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:15.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:15.162 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:15.211 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:15.211 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:15.306+0000 7fdccad60980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:15.310+0000 7fdccad60980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:15.329 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.106237_189.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.106237Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:15Z"} 2024-04-16T18:00:15.330 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290336.106237_189.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:56.106237Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:15Z"} 2024-04-16T18:00:15.330 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290336.106237_189.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:56.106237Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:00:15.330 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:15.330 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290336.106237_189.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:15.330 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290337.679908_225.1 target=1_1713290336.106237_189.1 2024-04-16T18:00:20.332 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:20.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:20.332 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:20.375 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:20.375 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:20.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:20.446+0000 7fd7a298e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:20.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:20.450+0000 7fd7a298e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:20.477 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290337.679908_225.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:57.679908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:20Z"} 2024-04-16T18:00:20.477 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290337.679908_225.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:58:57.679908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:20Z"} 2024-04-16T18:00:20.477 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290337.679908_225.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:58:57.679908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:00:20.477 DEBUG:rgw_multi.tests:log_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:20.477 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290337.679908_225.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:20.477 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:00:20.478 DEBUG:boto:path=/lybtug-27/ 2024-04-16T18:00:20.478 DEBUG:boto:auth_path=/lybtug-27/ 2024-04-16T18:00:20.478 DEBUG:boto:Method: HEAD 2024-04-16T18:00:20.478 DEBUG:boto:Path: /lybtug-27/ 2024-04-16T18:00:20.478 DEBUG:boto:Data: 2024-04-16T18:00:20.478 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.478 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.478 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.479 DEBUG:boto:Params: {} 2024-04-16T18:00:20.480 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:00:20.480 DEBUG:boto:Token: None 2024-04-16T18:00:20.480 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:00:20 GMT /lybtug-27/ 2024-04-16T18:00:20.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x6zCXwDa6FVw7BHNuvGKuuhWZNo= 2024-04-16T18:00:20.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x6zCXwDa6FVw7BHNuvGKuuhWZNo=', 'Content-Length': '0'} 2024-04-16T18:00:20.488 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000faf05368842a693d-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.488 DEBUG:boto:path=/lybtug-27/foo 2024-04-16T18:00:20.488 DEBUG:boto:auth_path=/lybtug-27/foo 2024-04-16T18:00:20.488 DEBUG:boto:Method: PUT 2024-04-16T18:00:20.488 DEBUG:boto:Path: /lybtug-27/foo 2024-04-16T18:00:20.488 DEBUG:boto:Data: 2024-04-16T18:00:20.489 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:00:20.489 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.489 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.489 DEBUG:boto:Params: {} 2024-04-16T18:00:20.489 DEBUG:boto:Token: None 2024-04-16T18:00:20.489 DEBUG:boto:StringToSign: PUT N7UdGUp1E+RbVvZSTy1R8g== application/octet-stream Tue, 16 Apr 2024 18:00:20 GMT /lybtug-27/foo 2024-04-16T18:00:20.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1fEvnyWSoUMIniMLxlI3Wi5LSEc= 2024-04-16T18:00:20.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1fEvnyWSoUMIniMLxlI3Wi5LSEc='} 2024-04-16T18:00:20.504 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"37b51d194a7513e45b56f6524f2d51f2"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000717d47c869f4161c-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.504 DEBUG:boto:path=/lybtug-27/ 2024-04-16T18:00:20.504 DEBUG:boto:auth_path=/lybtug-27/ 2024-04-16T18:00:20.504 DEBUG:boto:Method: DELETE 2024-04-16T18:00:20.504 DEBUG:boto:Path: /lybtug-27/ 2024-04-16T18:00:20.505 DEBUG:boto:Data: 2024-04-16T18:00:20.505 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.505 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.505 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.505 DEBUG:boto:Params: {} 2024-04-16T18:00:20.505 DEBUG:boto:Token: None 2024-04-16T18:00:20.505 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 18:00:20 GMT /lybtug-27/ 2024-04-16T18:00:20.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LxYMD7705Rnm8gR0yEpQNKRUU3o= 2024-04-16T18:00:20.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LxYMD7705Rnm8gR0yEpQNKRUU3o=', 'Content-Length': '0'} 2024-04-16T18:00:20.522 DEBUG:boto:Response headers: [('Content-Length', '253'), ('x-amz-request-id', 'tx00000f5b3e25512d4b9bb-00661ebcb4-4353-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.523 DEBUG:boto:path=/lybtug-28/ 2024-04-16T18:00:20.523 DEBUG:boto:auth_path=/lybtug-28/ 2024-04-16T18:00:20.524 DEBUG:boto:Method: HEAD 2024-04-16T18:00:20.524 DEBUG:boto:Path: /lybtug-28/ 2024-04-16T18:00:20.524 DEBUG:boto:Data: 2024-04-16T18:00:20.524 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.524 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:00:20.524 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.524 DEBUG:boto:Params: {} 2024-04-16T18:00:20.524 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:00:20.524 DEBUG:boto:Token: None 2024-04-16T18:00:20.525 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:00:20 GMT /lybtug-28/ 2024-04-16T18:00:20.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s8Sqnlw0y5ZywOWrhXn0i1oGNHk= 2024-04-16T18:00:20.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s8Sqnlw0y5ZywOWrhXn0i1oGNHk=', 'Content-Length': '0'} 2024-04-16T18:00:20.530 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003d6bc475e6afee68-00661ebcb4-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.530 DEBUG:boto:path=/lybtug-28/foo 2024-04-16T18:00:20.530 DEBUG:boto:auth_path=/lybtug-28/foo 2024-04-16T18:00:20.530 DEBUG:boto:Method: PUT 2024-04-16T18:00:20.531 DEBUG:boto:Path: /lybtug-28/foo 2024-04-16T18:00:20.531 DEBUG:boto:Data: 2024-04-16T18:00:20.531 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:00:20.531 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:00:20.531 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.531 DEBUG:boto:Params: {} 2024-04-16T18:00:20.531 DEBUG:boto:Token: None 2024-04-16T18:00:20.531 DEBUG:boto:StringToSign: PUT N7UdGUp1E+RbVvZSTy1R8g== application/octet-stream Tue, 16 Apr 2024 18:00:20 GMT /lybtug-28/foo 2024-04-16T18:00:20.531 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R6KZBVpYfz4E1EgMNtY8TuvQdoY= 2024-04-16T18:00:20.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R6KZBVpYfz4E1EgMNtY8TuvQdoY='} 2024-04-16T18:00:20.545 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"37b51d194a7513e45b56f6524f2d51f2"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000d292cc4dfd00f18-00661ebcb4-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.545 DEBUG:boto:path=/lybtug-28/ 2024-04-16T18:00:20.545 DEBUG:boto:auth_path=/lybtug-28/ 2024-04-16T18:00:20.545 DEBUG:boto:Method: DELETE 2024-04-16T18:00:20.545 DEBUG:boto:Path: /lybtug-28/ 2024-04-16T18:00:20.545 DEBUG:boto:Data: 2024-04-16T18:00:20.546 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.546 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:00:20.546 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.546 DEBUG:boto:Params: {} 2024-04-16T18:00:20.546 DEBUG:boto:Token: None 2024-04-16T18:00:20.546 DEBUG:boto:StringToSign: DELETE Tue, 16 Apr 2024 18:00:20 GMT /lybtug-28/ 2024-04-16T18:00:20.546 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jqY7d6D2sdketGawulDUG9SH0y8= 2024-04-16T18:00:20.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jqY7d6D2sdketGawulDUG9SH0y8=', 'Content-Length': '0'} 2024-04-16T18:00:20.564 DEBUG:boto:Response headers: [('Content-Length', '253'), ('x-amz-request-id', 'tx000000dd9df2d7e2f1b82-00661ebcb4-4335-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.564 DEBUG:boto:path=/lybtug-27/ 2024-04-16T18:00:20.565 DEBUG:boto:auth_path=/lybtug-27/ 2024-04-16T18:00:20.565 DEBUG:boto:Method: HEAD 2024-04-16T18:00:20.565 DEBUG:boto:Path: /lybtug-27/ 2024-04-16T18:00:20.565 DEBUG:boto:Data: 2024-04-16T18:00:20.565 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.565 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.565 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.565 DEBUG:boto:Params: {} 2024-04-16T18:00:20.565 DEBUG:boto:Token: None 2024-04-16T18:00:20.565 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:00:20 GMT /lybtug-27/ 2024-04-16T18:00:20.565 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x6zCXwDa6FVw7BHNuvGKuuhWZNo= 2024-04-16T18:00:20.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x6zCXwDa6FVw7BHNuvGKuuhWZNo=', 'Content-Length': '0'} 2024-04-16T18:00:20.570 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000db6590f7648dff93-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.570 DEBUG:boto:path=/lybtug-28/ 2024-04-16T18:00:20.570 DEBUG:boto:auth_path=/lybtug-28/ 2024-04-16T18:00:20.570 DEBUG:boto:Method: HEAD 2024-04-16T18:00:20.570 DEBUG:boto:Path: /lybtug-28/ 2024-04-16T18:00:20.570 DEBUG:boto:Data: 2024-04-16T18:00:20.570 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.570 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.570 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.570 DEBUG:boto:Params: {} 2024-04-16T18:00:20.571 DEBUG:boto:Token: None 2024-04-16T18:00:20.571 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:00:20 GMT /lybtug-28/ 2024-04-16T18:00:20.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s8Sqnlw0y5ZywOWrhXn0i1oGNHk= 2024-04-16T18:00:20.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s8Sqnlw0y5ZywOWrhXn0i1oGNHk=', 'Content-Length': '0'} 2024-04-16T18:00:20.574 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001d1f06504b33644a-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.575 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_delete_notempty ... ok 2024-04-16T18:00:20.576 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_period_incremental_sync ... SKIP: test_multi_period_incremental_sync skipped. Requires 3 or more zones in master zonegroup. 2024-04-16T18:00:20.577 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-29 2024-04-16T18:00:20.577 DEBUG:boto:path=/lybtug-29/ 2024-04-16T18:00:20.577 DEBUG:boto:auth_path=/lybtug-29/ 2024-04-16T18:00:20.577 DEBUG:boto:Method: PUT 2024-04-16T18:00:20.577 DEBUG:boto:Path: /lybtug-29/ 2024-04-16T18:00:20.577 DEBUG:boto:Data: 2024-04-16T18:00:20.577 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.577 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.577 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.577 DEBUG:boto:Params: {} 2024-04-16T18:00:20.577 DEBUG:boto:Token: None 2024-04-16T18:00:20.578 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:00:20 GMT /lybtug-29/ 2024-04-16T18:00:20.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4OVKcylqljWQ+0mBY2qFquFGaWM= 2024-04-16T18:00:20.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4OVKcylqljWQ+0mBY2qFquFGaWM=', 'Content-Length': '0'} 2024-04-16T18:00:20.611 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a765d76480ab6972-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.611 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-30 2024-04-16T18:00:20.611 DEBUG:boto:path=/lybtug-30/ 2024-04-16T18:00:20.611 DEBUG:boto:auth_path=/lybtug-30/ 2024-04-16T18:00:20.611 DEBUG:boto:Method: PUT 2024-04-16T18:00:20.612 DEBUG:boto:Path: /lybtug-30/ 2024-04-16T18:00:20.612 DEBUG:boto:Data: 2024-04-16T18:00:20.612 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.612 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:00:20.612 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.612 DEBUG:boto:Params: {} 2024-04-16T18:00:20.612 DEBUG:boto:Token: None 2024-04-16T18:00:20.612 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:00:20 GMT /lybtug-30/ 2024-04-16T18:00:20.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OKp3QwUuaHnrMHTHr8BrvFN5Iak= 2024-04-16T18:00:20.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OKp3QwUuaHnrMHTHr8BrvFN5Iak=', 'Content-Length': '0'} 2024-04-16T18:00:20.676 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005da2238b090fcdd1-00661ebcb4-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.676 DEBUG:boto:path=/lybtug-29/ 2024-04-16T18:00:20.676 DEBUG:boto:auth_path=/lybtug-29/ 2024-04-16T18:00:20.677 DEBUG:boto:Method: HEAD 2024-04-16T18:00:20.677 DEBUG:boto:Path: /lybtug-29/ 2024-04-16T18:00:20.677 DEBUG:boto:Data: 2024-04-16T18:00:20.677 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.677 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.677 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.677 DEBUG:boto:Params: {} 2024-04-16T18:00:20.677 DEBUG:boto:Token: None 2024-04-16T18:00:20.677 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:00:20 GMT /lybtug-29/ 2024-04-16T18:00:20.677 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P8lLno97jdyyBSZqDb2mVDoN8yk= 2024-04-16T18:00:20.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P8lLno97jdyyBSZqDb2mVDoN8yk=', 'Content-Length': '0'} 2024-04-16T18:00:20.681 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006c8e0fa493136aa4-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.681 DEBUG:boto:path=/lybtug-29/key 2024-04-16T18:00:20.681 DEBUG:boto:auth_path=/lybtug-29/key 2024-04-16T18:00:20.681 DEBUG:boto:Method: PUT 2024-04-16T18:00:20.682 DEBUG:boto:Path: /lybtug-29/key 2024-04-16T18:00:20.682 DEBUG:boto:Data: 2024-04-16T18:00:20.682 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue'} 2024-04-16T18:00:20.682 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:00:20.682 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.682 DEBUG:boto:Params: {} 2024-04-16T18:00:20.682 DEBUG:boto:Token: None 2024-04-16T18:00:20.682 DEBUG:boto:StringToSign: PUT hBotaJrYa9FhFEdFPCLG/A== application/octet-stream Tue, 16 Apr 2024 18:00:20 GMT /lybtug-29/key 2024-04-16T18:00:20.682 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vXGFx4cdiI7STaDsBwM0VgrBMDY= 2024-04-16T18:00:20.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vXGFx4cdiI7STaDsBwM0VgrBMDY='} 2024-04-16T18:00:20.696 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"841a2d689ad86bd1611447453c22c6fc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003a307b61a1278f50-00661ebcb4-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.696 DEBUG:boto:path=/lybtug-30/ 2024-04-16T18:00:20.696 DEBUG:boto:auth_path=/lybtug-30/ 2024-04-16T18:00:20.696 DEBUG:boto:Method: HEAD 2024-04-16T18:00:20.696 DEBUG:boto:Path: /lybtug-30/ 2024-04-16T18:00:20.697 DEBUG:boto:Data: 2024-04-16T18:00:20.697 DEBUG:boto:Headers: {} 2024-04-16T18:00:20.697 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:00:20.697 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.697 DEBUG:boto:Params: {} 2024-04-16T18:00:20.697 DEBUG:boto:Token: None 2024-04-16T18:00:20.697 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:00:20 GMT /lybtug-30/ 2024-04-16T18:00:20.697 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Hys3avBQk26T90ER2Gxu4QSOPE= 2024-04-16T18:00:20.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Hys3avBQk26T90ER2Gxu4QSOPE=', 'Content-Length': '0'} 2024-04-16T18:00:20.700 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005e24ef1c1d0a1ae7-00661ebcb4-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.701 DEBUG:boto:path=/lybtug-30/key 2024-04-16T18:00:20.701 DEBUG:boto:auth_path=/lybtug-30/key 2024-04-16T18:00:20.701 DEBUG:boto:Method: PUT 2024-04-16T18:00:20.701 DEBUG:boto:Path: /lybtug-30/key 2024-04-16T18:00:20.701 DEBUG:boto:Data: 2024-04-16T18:00:20.701 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue'} 2024-04-16T18:00:20.701 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:00:20.701 DEBUG:boto:Port: 8000 2024-04-16T18:00:20.701 DEBUG:boto:Params: {} 2024-04-16T18:00:20.701 DEBUG:boto:Token: None 2024-04-16T18:00:20.701 DEBUG:boto:StringToSign: PUT hBotaJrYa9FhFEdFPCLG/A== application/octet-stream Tue, 16 Apr 2024 18:00:20 GMT /lybtug-30/key 2024-04-16T18:00:20.702 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zSBo+dZMzNu3FXLl9chhSBIre6M= 2024-04-16T18:00:20.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:00:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zSBo+dZMzNu3FXLl9chhSBIre6M='} 2024-04-16T18:00:20.746 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"841a2d689ad86bd1611447453c22c6fc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f1ca7cad02585b28-00661ebcb4-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:00:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:00:20.747 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:20.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:20.747 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:20.788 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:20.788 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:00:20.889 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290420.653255_234.1","last_update":"2024-04-16T18:00:20.653255Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:00:20.889 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:20.889 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:00:20.890 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:20.890 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:20.890 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:20.934 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:20.934 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:21.002+0000 7f3024c2a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:21.006+0000 7f3024c2a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:21.033 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.593976_227.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.593976Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:21Z"} 2024-04-16T18:00:21.034 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.593976_227.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.593976Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:21Z"} 2024-04-16T18:00:21.034 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.593976_227.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.593976Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:00:21.034 DEBUG:rgw_multi.tests:log_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:21.034 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.593976_227.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:21.034 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290420.653255_234.1 target=1_1713290420.593976_227.1 2024-04-16T18:00:26.035 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.035 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:26.077 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:26.077 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:26.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:26.150+0000 7fac50276980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:26.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:00:26.154+0000 7fac50276980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:00:26.181 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:26Z"} 2024-04-16T18:00:26.181 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:00:26Z"} 2024-04-16T18:00:26.181 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:00:26.181 DEBUG:rgw_multi.tests:log_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:26.181 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:00:26.181 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:00:26.181 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2024-04-16T18:00:26.182 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.182 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.182 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:26.226 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:26.226 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:26.341 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"00000000000000000000:00000000000000005020","last_update":"2024-04-16T17:56:45.050649Z"},{"marker":"00000000000000000000:00000000000000003193","last_update":"2024-04-16T18:00:20.988842Z"},{"marker":"00000000000000000000:00000000000000002484","last_update":"2024-04-16T17:54:55.037491Z"},{"marker":"00000000000000000000:00000000000000006142","last_update":"2024-04-16T18:00:22.361740Z"}] 2024-04-16T18:00:26.341 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000005020', 1: '00000000000000000000:00000000000000003193', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000006142'} 2024-04-16T18:00:26.341 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:00:26.341 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.341 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.342 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:26.386 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:26.386 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:26.490 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005020","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:45.050649Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:00:26Z"} 2024-04-16T18:00:26.491 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005020","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:45.050649Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:00:26Z"} 2024-04-16T18:00:26.491 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005020', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:45.050649Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003193', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.988842Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002484', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:54:55.037491Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006142', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:22.361740Z'}}] 2024-04-16T18:00:26.491 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000005020', 1: '00000000000000000000:00000000000000003193', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000006142'} 2024-04-16T18:00:26.491 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005020', 1: '00000000000000000000:00000000000000003193', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000006142'} 2024-04-16T18:00:26.491 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:00:26.491 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2024-04-16T18:00:26.491 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.492 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.492 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:26.571 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:26.571 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:26.682 INFO:teuthology.orchestra.run.smithi122.stdout:[{"marker":"00000000000000000000:00000000000000004315","last_update":"2024-04-16T17:56:29.271725Z"},{"marker":"00000000000000000000:00000000000000002910","last_update":"2024-04-16T18:00:21.995367Z"},{"marker":"00000000000000000000:00000000000000002625","last_update":"2024-04-16T17:55:23.267655Z"},{"marker":"00000000000000000000:00000000000000005719","last_update":"2024-04-16T18:00:24.359063Z"}] 2024-04-16T18:00:26.683 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002910', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000005719'} 2024-04-16T18:00:26.683 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:00:26.683 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.683 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:26.683 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:26.728 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:26.831 INFO:teuthology.orchestra.run.smithi071.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004315","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:29.271725Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002910","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:21.995367Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002625","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T17:55:23.267655Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005578","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.721066Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:00:26Z"} 2024-04-16T18:00:26.831 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004315","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:29.271725Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002910","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:21.995367Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002625","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T17:55:23.267655Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005578","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.721066Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:00:26Z"} 2024-04-16T18:00:26.831 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004315', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:29.271725Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002910', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:21.995367Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002625', 'next_step_marker': '', 'total_entries': 7, 'pos': 0, 'timestamp': '2024-04-16T17:55:23.267655Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005578', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.721066Z'}}] 2024-04-16T18:00:26.831 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002910', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000005719'} 2024-04-16T18:00:26.831 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002910', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000005578'} 2024-04-16T18:00:26.832 WARNING:rgw_multi.tests:data of zone test-zone1 behind zone test-zone2: shard=3 master=00000000000000000000:00000000000000005719 target=00000000000000000000:00000000000000005578 2024-04-16T18:00:31.832 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:31.833 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:31.833 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:31.879 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:31.879 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:31.979 INFO:teuthology.orchestra.run.smithi071.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004315","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:29.271725Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002910","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:21.995367Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002625","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T17:55:23.267655Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005719","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:24.359063Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:00:31Z"} 2024-04-16T18:00:31.979 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004315","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:56:29.271725Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002910","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:21.995367Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002625","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T17:55:23.267655Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005719","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:24.359063Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:00:31Z"} 2024-04-16T18:00:31.979 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004315', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:56:29.271725Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002910', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:21.995367Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002625', 'next_step_marker': '', 'total_entries': 7, 'pos': 0, 'timestamp': '2024-04-16T17:55:23.267655Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005719', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:24.359063Z'}}] 2024-04-16T18:00:31.979 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002910', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000005719'} 2024-04-16T18:00:31.979 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000004315', 1: '00000000000000000000:00000000000000002910', 2: '00000000000000000000:00000000000000002625', 3: '00000000000000000000:00000000000000005719'} 2024-04-16T18:00:31.980 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:00:31.980 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:31.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:31.980 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:32.009 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:32.009 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:32.116 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"00000000000000000000:00000000000000005020","last_update":"2024-04-16T17:56:45.050649Z"},{"marker":"00000000000000000000:00000000000000003193","last_update":"2024-04-16T18:00:20.988842Z"},{"marker":"00000000000000000000:00000000000000002484","last_update":"2024-04-16T17:54:55.037491Z"},{"marker":"00000000000000000000:00000000000000006142","last_update":"2024-04-16T18:00:22.361740Z"}] 2024-04-16T18:00:32.117 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:32.117 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:32.117 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:00:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:32.191 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:32.305 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.305 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.305 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 0 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:32.481 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:00:32.481 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.482 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 1 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:32.646 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:00:32.646 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.647 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 2 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:32.720 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:32.720 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:32.806 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:00:32.806 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.806 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.806 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 3 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:32.884 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:32.885 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:32.971 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:00:32.971 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.971 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:32.972 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:33.120 INFO:teuthology.orchestra.run.smithi122.stdout:[{"marker":"00000000000000000000:00000000000000004315","last_update":"2024-04-16T17:56:29.271725Z"},{"marker":"00000000000000000000:00000000000000002910","last_update":"2024-04-16T18:00:21.995367Z"},{"marker":"00000000000000000000:00000000000000002625","last_update":"2024-04-16T17:55:23.267655Z"},{"marker":"00000000000000000000:00000000000000005719","last_update":"2024-04-16T18:00:24.359063Z"}] 2024-04-16T18:00:33.121 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:33.121 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:33.121 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:00:33.198 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:33.198 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:33.313 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.313 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 0 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:33.395 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:33.395 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:33.490 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-16T18:00:33.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.491 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 1 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:33.572 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:33.572 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:33.668 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-16T18:00:33.669 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.669 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 2 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:33.837 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-16T18:00:33.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:00:33.838 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 3 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:00:33.920 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:33.920 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:34.004 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-16T18:00:34.004 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_datalog_autotrim ... ok 2024-04-16T18:00:34.005 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:34.005 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:34.005 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=false --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:00:34.076 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:34.076 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:34.123 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","notif_pool":"test-zone2.rgw.log:notif","topics_pool":"test-zone2.rgw.meta:topics","account_pool":"test-zone2.rgw.meta:accounts","group_pool":"test-zone2.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T18:00:34.123 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:34.124 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:00:34.124 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:00:34.207 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:00:34.207 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:00:34.294 INFO:teuthology.orchestra.run.smithi122.stderr:Sending period to new master zone 054102a3-8d94-4596-8b3b-6341eab312e1 2024-04-16T18:00:34.383 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":3,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":false,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:00:34.384 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to false 2024-04-16T18:02:04.384 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-31 2024-04-16T18:02:04.385 DEBUG:boto:path=/lybtug-31/ 2024-04-16T18:02:04.385 DEBUG:boto:auth_path=/lybtug-31/ 2024-04-16T18:02:04.386 DEBUG:boto:Method: PUT 2024-04-16T18:02:04.386 DEBUG:boto:Path: /lybtug-31/ 2024-04-16T18:02:04.386 DEBUG:boto:Data: 2024-04-16T18:02:04.386 DEBUG:boto:Headers: {} 2024-04-16T18:02:04.386 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:02:04.386 DEBUG:boto:Port: 8000 2024-04-16T18:02:04.386 DEBUG:boto:Params: {} 2024-04-16T18:02:04.387 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:02:04.387 DEBUG:boto:Token: None 2024-04-16T18:02:04.387 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:02:04 GMT /lybtug-31/ 2024-04-16T18:02:04.387 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YJN42OSDrG9Rn6uOVVupniqIhHQ= 2024-04-16T18:02:04.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:02:04 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YJN42OSDrG9Rn6uOVVupniqIhHQ=', 'Content-Length': '0'} 2024-04-16T18:02:04.429 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009e09df66bc2eb66c-00661ebd1c-4699-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:02:04 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:02:04.429 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:02:04.429 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:02:04.429 DEBUG:boto:Method: PUT 2024-04-16T18:02:04.429 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:02:04.430 DEBUG:boto:Data: 2024-04-16T18:02:04.430 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:02:04.430 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:02:04.430 DEBUG:boto:Port: 8000 2024-04-16T18:02:04.430 DEBUG:boto:Params: {} 2024-04-16T18:02:04.430 DEBUG:boto:Token: None 2024-04-16T18:02:04.430 DEBUG:boto:StringToSign: PUT 3FCGuEcom6i4veFJuDiBdQ== application/octet-stream Tue, 16 Apr 2024 18:02:04 GMT /lybtug-31/testredirect 2024-04-16T18:02:04.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ok9TULVuieDnO+iUv5cXYBRApcA= 2024-04-16T18:02:04.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:02:04 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ok9TULVuieDnO+iUv5cXYBRApcA='} 2024-04-16T18:02:04.449 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000285175e4d6e0ae33-00661ebd1c-4699-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:02:04 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:02:04.449 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:04.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:04.450 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:04.490 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:02:04.592 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290524.429884_257.1","last_update":"2024-04-16T18:02:04.429884Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:02:04.592 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:04.593 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:02:04.593 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:04.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:04.593 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:04.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:04.713+0000 7fedd33db980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:04.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:04.717+0000 7fedd33db980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:04.741 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:04Z"} 2024-04-16T18:02:04.741 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:04Z"} 2024-04-16T18:02:04.741 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:04.741 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:04.742 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:04.742 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290420.653255_234.1 2024-04-16T18:02:09.743 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:09.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:09.743 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:09.787 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:09.787 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:09.864+0000 7ff111b89980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:09.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:09.868+0000 7ff111b89980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:09.892 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:09Z"} 2024-04-16T18:02:09.892 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:09Z"} 2024-04-16T18:02:09.892 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:09.892 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:09.892 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:09.893 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290420.653255_234.1 2024-04-16T18:02:14.893 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:14.894 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:14.894 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:15.012+0000 7f32d8e65980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:15.012+0000 7f32d8e65980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:15.038 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:15Z"} 2024-04-16T18:02:15.038 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:15Z"} 2024-04-16T18:02:15.038 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:15.038 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:15.038 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:15.038 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290420.653255_234.1 2024-04-16T18:02:20.039 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:20.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:20.040 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:20.082 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:20.083 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:20.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:20.160+0000 7fed23c5a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:20.160+0000 7fed23c5a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:20.186 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:20Z"} 2024-04-16T18:02:20.187 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:20Z"} 2024-04-16T18:02:20.187 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:20.187 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:20.187 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:20.187 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290420.653255_234.1 2024-04-16T18:02:25.188 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:25.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:25.188 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:25.231 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:25.231 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:25.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:25.308+0000 7f0236131980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:25.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:25.308+0000 7f0236131980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:25.334 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:25Z"} 2024-04-16T18:02:25.334 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:25Z"} 2024-04-16T18:02:25.334 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:25.334 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:25.334 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:25.334 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290420.653255_234.1 2024-04-16T18:02:30.335 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:30.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:30.336 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:30.379 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:30.379 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:30.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:30.456+0000 7fdbe0597980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:30.456+0000 7fdbe0597980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:30.481 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:30Z"} 2024-04-16T18:02:30.481 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290420.653255_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:00:20.653255Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:30Z"} 2024-04-16T18:02:30.481 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290420.653255_234.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.653255Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:30.481 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:30.481 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290420.653255_234.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:30.481 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290420.653255_234.1 2024-04-16T18:02:35.483 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:35.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:35.483 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:35.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:35.604+0000 7fc7cc88f980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:35.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:35.604+0000 7fc7cc88f980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:35.634 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.411312_248.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.411312Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:35Z"} 2024-04-16T18:02:35.634 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.411312_248.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.411312Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:35Z"} 2024-04-16T18:02:35.634 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.411312_248.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.411312Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:35.634 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:35.635 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.411312_248.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:35.635 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290524.429884_257.1 target=1_1713290524.411312_248.1 2024-04-16T18:02:40.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:40.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:02:40.636 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:02:40.680 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:40.680 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:40.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:40.756+0000 7fd6d6ff9980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:40.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:02:40.760+0000 7fd6d6ff9980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:02:40.785 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:40Z"} 2024-04-16T18:02:40.785 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:02:40Z"} 2024-04-16T18:02:40.785 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:02:40.785 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:40.785 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:02:40.786 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:02:40.786 DEBUG:boto:path=/lybtug-31/ 2024-04-16T18:02:40.786 DEBUG:boto:auth_path=/lybtug-31/ 2024-04-16T18:02:40.786 DEBUG:boto:Method: HEAD 2024-04-16T18:02:40.786 DEBUG:boto:Path: /lybtug-31/ 2024-04-16T18:02:40.786 DEBUG:boto:Data: 2024-04-16T18:02:40.786 DEBUG:boto:Headers: {} 2024-04-16T18:02:40.786 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:02:40.786 DEBUG:boto:Port: 8000 2024-04-16T18:02:40.787 DEBUG:boto:Params: {} 2024-04-16T18:02:40.787 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:02:40.787 DEBUG:boto:Token: None 2024-04-16T18:02:40.787 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:02:40 GMT /lybtug-31/ 2024-04-16T18:02:40.787 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/H7ZHNKiGWKa67tZo4+tQzTMlJI= 2024-04-16T18:02:40.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:02:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/H7ZHNKiGWKa67tZo4+tQzTMlJI=', 'Content-Length': '0'} 2024-04-16T18:02:40.795 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009585785b052dc037-00661ebd40-4837-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:02:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:02:40.795 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:02:40.795 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:02:40.795 DEBUG:boto:Method: HEAD 2024-04-16T18:02:40.795 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:02:40.795 DEBUG:boto:Data: 2024-04-16T18:02:40.795 DEBUG:boto:Headers: {} 2024-04-16T18:02:40.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:02:40.796 DEBUG:boto:Port: 8000 2024-04-16T18:02:40.796 DEBUG:boto:Params: {} 2024-04-16T18:02:40.796 DEBUG:boto:Token: None 2024-04-16T18:02:40.796 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:02:40 GMT /lybtug-31/testredirect 2024-04-16T18:02:40.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P6vNqbg5eSb7VOWPyN0uJdOqq9A= 2024-04-16T18:02:40.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:02:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P6vNqbg5eSb7VOWPyN0uJdOqq9A=', 'Content-Length': '0'} 2024-04-16T18:02:40.798 DEBUG:boto:Response headers: [('Content-Length', '248'), ('x-amz-request-id', 'tx00000ef77ecc996d95683-00661ebd40-4837-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:02:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:02:40.798 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=054102a3-8d94-4596-8b3b-6341eab312e1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:02:40.798 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=054102a3-8d94-4596-8b3b-6341eab312e1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:02:40.799 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone=054102a3-8d94-4596-8b3b-6341eab312e1 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:02:40.832 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:40.833 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:40.884 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","notif_pool":"test-zone2.rgw.log:notif","topics_pool":"test-zone2.rgw.meta:topics","account_pool":"test-zone2.rgw.meta:accounts","group_pool":"test-zone2.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T18:02:40.885 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:02:40.885 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:02:40.885 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:02:40.967 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:02:40.967 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:02:41.064 INFO:teuthology.orchestra.run.smithi122.stderr:Sending period to new master zone 054102a3-8d94-4596-8b3b-6341eab312e1 2024-04-16T18:02:41.167 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":4,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":false,"sync_from":[],"redirect_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:02:41.167 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "054102a3-8d94-4596-8b3b-6341eab312e1" 2024-04-16T18:04:11.168 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.169 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.169 DEBUG:boto:Method: HEAD 2024-04-16T18:04:11.169 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.170 DEBUG:boto:Data: 2024-04-16T18:04:11.170 DEBUG:boto:Headers: {} 2024-04-16T18:04:11.170 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:04:11.170 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.170 DEBUG:boto:Params: {} 2024-04-16T18:04:11.171 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.171 DEBUG:boto:Token: None 2024-04-16T18:04:11.171 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Bvxfj9mAyVcz0S6yb8SWPpsoAOM= 2024-04-16T18:04:11.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Bvxfj9mAyVcz0S6yb8SWPpsoAOM=', 'Content-Length': '0'} 2024-04-16T18:04:11.183 DEBUG:boto:Response headers: [('Location', 'http://smithi071.front.sepia.ceph.com:8000/lybtug-31/testredirect?'), ('x-amz-request-id', 'tx0000065c9505f7053cac1-00661ebd9b-4910-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.183 DEBUG:boto:Redirecting: http://smithi071.front.sepia.ceph.com/lybtug-31/testredirect 2024-04-16T18:04:11.184 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': '8000'} 2024-04-16T18:04:11.184 DEBUG:boto:Token: None 2024-04-16T18:04:11.184 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Bvxfj9mAyVcz0S6yb8SWPpsoAOM= 2024-04-16T18:04:11.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Bvxfj9mAyVcz0S6yb8SWPpsoAOM=', 'Content-Length': '0'} 2024-04-16T18:04:11.195 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:02:04 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx00000c7a34c22c52d06bc-00661ebd9b-4709-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.196 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.196 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.196 DEBUG:boto:Method: GET 2024-04-16T18:04:11.196 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.196 DEBUG:boto:Data: 2024-04-16T18:04:11.196 DEBUG:boto:Headers: {} 2024-04-16T18:04:11.196 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:04:11.196 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.196 DEBUG:boto:Params: {} 2024-04-16T18:04:11.197 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.197 DEBUG:boto:Token: None 2024-04-16T18:04:11.197 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.201 DEBUG:boto:Response headers: [('Location', 'http://smithi071.front.sepia.ceph.com:8000/lybtug-31/testredirect?'), ('x-amz-request-id', 'tx00000a03425847e4815ab-00661ebd9b-4910-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.201 DEBUG:boto:Redirecting: http://smithi071.front.sepia.ceph.com/lybtug-31/testredirect 2024-04-16T18:04:11.201 DEBUG:boto:Token: None 2024-04-16T18:04:11.202 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.202 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.205 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:02:04 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx00000cd0071347c11f971-00661ebd9b-4709-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.205 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.205 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.205 DEBUG:boto:Method: PUT 2024-04-16T18:04:11.206 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.206 DEBUG:boto:Data: 2024-04-16T18:04:11.206 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VpBzljOcorCZvRIkX5Nt3A==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:04:11.206 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:04:11.206 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.206 DEBUG:boto:Params: {} 2024-04-16T18:04:11.206 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.206 DEBUG:boto:Token: None 2024-04-16T18:04:11.207 DEBUG:boto:StringToSign: PUT VpBzljOcorCZvRIkX5Nt3A== application/octet-stream Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hqqjkGIRSzk3bx1KMMV+5A0id5k= 2024-04-16T18:04:11.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VpBzljOcorCZvRIkX5Nt3A==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hqqjkGIRSzk3bx1KMMV+5A0id5k='} 2024-04-16T18:04:11.231 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"56907396339ca2b099bd12245f936ddc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000028b12348e8433b23-00661ebd9b-4709-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.231 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.231 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.231 DEBUG:boto:Method: GET 2024-04-16T18:04:11.231 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.231 DEBUG:boto:Data: 2024-04-16T18:04:11.231 DEBUG:boto:Headers: {} 2024-04-16T18:04:11.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:04:11.231 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.232 DEBUG:boto:Params: {} 2024-04-16T18:04:11.232 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.232 DEBUG:boto:Token: None 2024-04-16T18:04:11.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.236 DEBUG:boto:Response headers: [('Location', 'http://smithi071.front.sepia.ceph.com:8000/lybtug-31/testredirect?'), ('x-amz-request-id', 'tx00000c6a64c588526b755-00661ebd9b-4910-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.236 DEBUG:boto:Redirecting: http://smithi071.front.sepia.ceph.com/lybtug-31/testredirect 2024-04-16T18:04:11.236 DEBUG:boto:Token: None 2024-04-16T18:04:11.237 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.242 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"56907396339ca2b099bd12245f936ddc"'), ('x-amz-request-id', 'tx00000bc1760e7555b478a-00661ebd9b-4709-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.242 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.242 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.242 DEBUG:boto:Method: PUT 2024-04-16T18:04:11.242 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.243 DEBUG:boto:Data: 2024-04-16T18:04:11.243 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'uk9S5NXZfBvPq4jGr+LM5g==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:04:11.243 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:04:11.243 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.243 DEBUG:boto:Params: {} 2024-04-16T18:04:11.243 DEBUG:boto:Token: None 2024-04-16T18:04:11.243 DEBUG:boto:StringToSign: PUT uk9S5NXZfBvPq4jGr+LM5g== application/octet-stream Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.243 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mQGPVSVRjmihw9A/10ze9X3Of9g= 2024-04-16T18:04:11.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'uk9S5NXZfBvPq4jGr+LM5g==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mQGPVSVRjmihw9A/10ze9X3Of9g='} 2024-04-16T18:04:11.257 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"ba4f52e4d5d97c1bcfab88c6afe2cce6"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000059d47d8f33ae1cb8-00661ebd9b-4709-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.257 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.257 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.257 DEBUG:boto:Method: GET 2024-04-16T18:04:11.257 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.257 DEBUG:boto:Data: 2024-04-16T18:04:11.257 DEBUG:boto:Headers: {} 2024-04-16T18:04:11.258 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:04:11.258 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.258 DEBUG:boto:Params: {} 2024-04-16T18:04:11.258 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.258 DEBUG:boto:Token: None 2024-04-16T18:04:11.258 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.258 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.262 DEBUG:boto:Response headers: [('Location', 'http://smithi071.front.sepia.ceph.com:8000/lybtug-31/testredirect?'), ('x-amz-request-id', 'tx0000002ecc87383055089-00661ebd9b-4910-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.262 DEBUG:boto:Redirecting: http://smithi071.front.sepia.ceph.com/lybtug-31/testredirect 2024-04-16T18:04:11.262 DEBUG:boto:Token: None 2024-04-16T18:04:11.263 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.266 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"ba4f52e4d5d97c1bcfab88c6afe2cce6"'), ('x-amz-request-id', 'tx00000d1ff24a4c3846e96-00661ebd9b-4709-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.266 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.266 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.266 DEBUG:boto:Method: PUT 2024-04-16T18:04:11.267 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.267 DEBUG:boto:Data: 2024-04-16T18:04:11.267 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6aKmvaJMgaK7Z2rfDZvVeg==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:04:11.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:04:11.267 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.267 DEBUG:boto:Params: {} 2024-04-16T18:04:11.267 DEBUG:boto:Token: None 2024-04-16T18:04:11.267 DEBUG:boto:StringToSign: PUT 6aKmvaJMgaK7Z2rfDZvVeg== application/octet-stream Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.267 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kivUKz6iXKp0OVXAnKSbB2ys0zk= 2024-04-16T18:04:11.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6aKmvaJMgaK7Z2rfDZvVeg==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kivUKz6iXKp0OVXAnKSbB2ys0zk='} 2024-04-16T18:04:11.281 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"e9a2a6bda24c81a2bb676adf0d9bd57a"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bd33cd493308c302-00661ebd9b-4709-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.281 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.282 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.282 DEBUG:boto:Method: GET 2024-04-16T18:04:11.282 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.282 DEBUG:boto:Data: 2024-04-16T18:04:11.282 DEBUG:boto:Headers: {} 2024-04-16T18:04:11.282 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:04:11.282 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.282 DEBUG:boto:Params: {} 2024-04-16T18:04:11.282 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.282 DEBUG:boto:Token: None 2024-04-16T18:04:11.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.287 DEBUG:boto:Response headers: [('Location', 'http://smithi071.front.sepia.ceph.com:8000/lybtug-31/testredirect?'), ('x-amz-request-id', 'tx0000012040012c132129f-00661ebd9b-4910-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.287 DEBUG:boto:Redirecting: http://smithi071.front.sepia.ceph.com/lybtug-31/testredirect 2024-04-16T18:04:11.287 DEBUG:boto:Token: None 2024-04-16T18:04:11.287 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.290 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"e9a2a6bda24c81a2bb676adf0d9bd57a"'), ('x-amz-request-id', 'tx00000a7bb1c523b96def0-00661ebd9b-4709-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.291 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.291 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.291 DEBUG:boto:Method: PUT 2024-04-16T18:04:11.291 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.291 DEBUG:boto:Data: 2024-04-16T18:04:11.291 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GbNgNQ5MAukCtgY+rsT5Xw==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:04:11.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:04:11.291 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.291 DEBUG:boto:Params: {} 2024-04-16T18:04:11.292 DEBUG:boto:Token: None 2024-04-16T18:04:11.292 DEBUG:boto:StringToSign: PUT GbNgNQ5MAukCtgY+rsT5Xw== application/octet-stream Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VZew+MTFSeVu5u1EmG32Gs4g36Y= 2024-04-16T18:04:11.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GbNgNQ5MAukCtgY+rsT5Xw==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VZew+MTFSeVu5u1EmG32Gs4g36Y='} 2024-04-16T18:04:11.334 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"19b360350e4c02e902b6063eaec4f95f"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000724ecd0fddb8341e-00661ebd9b-4709-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.335 DEBUG:boto:path=/lybtug-31/testredirect 2024-04-16T18:04:11.335 DEBUG:boto:auth_path=/lybtug-31/testredirect 2024-04-16T18:04:11.335 DEBUG:boto:Method: GET 2024-04-16T18:04:11.335 DEBUG:boto:Path: /lybtug-31/testredirect 2024-04-16T18:04:11.335 DEBUG:boto:Data: 2024-04-16T18:04:11.335 DEBUG:boto:Headers: {} 2024-04-16T18:04:11.335 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:04:11.335 DEBUG:boto:Port: 8000 2024-04-16T18:04:11.335 DEBUG:boto:Params: {} 2024-04-16T18:04:11.335 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:04:11.336 DEBUG:boto:Token: None 2024-04-16T18:04:11.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.340 DEBUG:boto:Response headers: [('Location', 'http://smithi071.front.sepia.ceph.com:8000/lybtug-31/testredirect?'), ('x-amz-request-id', 'tx0000099650e6b0d9954d0-00661ebd9b-4910-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.340 DEBUG:boto:Redirecting: http://smithi071.front.sepia.ceph.com/lybtug-31/testredirect 2024-04-16T18:04:11.340 DEBUG:boto:Token: None 2024-04-16T18:04:11.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:04:11 GMT /lybtug-31/testredirect 2024-04-16T18:04:11.341 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE= 2024-04-16T18:04:11.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:04:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyfx9MOOtl4RTAxOsoE+YAet0PE=', 'Content-Length': '0'} 2024-04-16T18:04:11.344 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"19b360350e4c02e902b6063eaec4f95f"'), ('x-amz-request-id', 'tx000006163693b05974831-00661ebd9b-4709-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:04:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:04:11.344 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:04:11.345 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:04:11.345 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=true --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:04:11.387 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:04:11.388 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:04:11.441 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","notif_pool":"test-zone2.rgw.log:notif","topics_pool":"test-zone2.rgw.meta:topics","account_pool":"test-zone2.rgw.meta:accounts","group_pool":"test-zone2.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T18:04:11.441 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:04:11.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:04:11.442 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:04:11.517 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:04:11.518 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:04:11.615 INFO:teuthology.orchestra.run.smithi122.stderr:Sending period to new master zone 054102a3-8d94-4596-8b3b-6341eab312e1 2024-04-16T18:04:11.711 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":5,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:04:11.712 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to true 2024-04-16T18:04:16.977 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T18:04:16.976+0000 7f58b8e76640 -1 WARNING: The OSD cluster is overloaded and struggling to complete ops. You need more capacity to serve this level of demand. 2024-04-16T18:05:41.713 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:05:41.714 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:05:41.714 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone= --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:05:41.756 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:05:41.756 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:05:41.813 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","notif_pool":"test-zone2.rgw.log:notif","topics_pool":"test-zone2.rgw.meta:topics","account_pool":"test-zone2.rgw.meta:accounts","group_pool":"test-zone2.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T18:05:41.813 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:05:41.814 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:05:41.814 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:05:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:05:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:05:41.993 INFO:teuthology.orchestra.run.smithi122.stderr:Sending period to new master zone 054102a3-8d94-4596-8b3b-6341eab312e1 2024-04-16T18:05:42.098 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":6,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:05:42.098 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "" 2024-04-16T18:07:12.100 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_zone_redirect ... ok 2024-04-16T18:07:12.102 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:12.102 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:12.103 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:12.145 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:12.146 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:12.214 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:07:12.245 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290524.429884_257.1","last_update":"2024-04-16T18:02:04.429884Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:07:12.245 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:12.245 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:07:12.246 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:12.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:12.246 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:12.289 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:12.290 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:12.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:12.362+0000 7fc04cd6d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:12.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:12.366+0000 7fc04cd6d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:12.392 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:12Z"} 2024-04-16T18:07:12.392 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:12Z"} 2024-04-16T18:07:12.392 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:12.392 DEBUG:rgw_multi.tests:log_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:12.392 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:12.392 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:07:12.393 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:12.393 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'GVUAE6PEZFSPBINC', '--secret', 'KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:12.393 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key GVUAE6PEZFSPBINC --secret KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:12.513 INFO:teuthology.orchestra.run.smithi122.stderr:NOTICE: set zone's realm_id=750bb778-ce9e-43d8-aa1d-59a6a3ecda80 2024-04-16T18:07:12.555 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"c26bf760-bb15-4e8e-9665-ecd8fdde67b4","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","roles_pool":"remove.rgw.meta:roles","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","notif_pool":"remove.rgw.log:notif","topics_pool":"remove.rgw.meta:topics","account_pool":"remove.rgw.meta:accounts","group_pool":"remove.rgw.meta:groups","system_key":{"access_key":"GVUAE6PEZFSPBINC","secret_key":"KeMj7JT23gY7IYe6J9r3uroRsmuI9mlK"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T18:07:12.555 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:12.555 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:12.556 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:12.638 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:12.638 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:12.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:12.686+0000 7f767e192980 0 period (8c713f9b-78f8-4e05-979f-375a77da9531 does not have zone c26bf760-bb15-4e8e-9665-ecd8fdde67b4 configured 2024-04-16T18:07:18.964 INFO:teuthology.orchestra.run.smithi122.stderr:Sending period to new master zone 054102a3-8d94-4596-8b3b-6341eab312e1 2024-04-16T18:07:19.068 INFO:teuthology.orchestra.run.smithi122.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":7,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"c26bf760-bb15-4e8e-9665-ecd8fdde67b4","name":"remove","endpoints":[],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065},{"key":"c26bf760-bb15-4e8e-9665-ecd8fdde67b4","val":3872306251}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:07:19.069 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.069 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.069 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:19.167 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-04-16T18:07:19.167 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.168 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.248 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:19.290 INFO:teuthology.orchestra.run.smithi071.stderr:zone name remove not found in zonegroup test-zonegroup 2024-04-16T18:07:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-16T18:07:19.307 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.307 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.307 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone delete --rgw-zone remove --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.344 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.344 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:19.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:19.418+0000 7facb2e46980 0 Removed zone from zonegroup test-zonegroup 2024-04-16T18:07:19.435 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.435 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.472 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.472 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:19.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:07:19.520+0000 7f5ed02ce980 0 WARNING: can't generate connection for zone c26bf760-bb15-4e8e-9665-ecd8fdde67b4 id remove: no endpoints defined 2024-04-16T18:07:19.613 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":8,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:07:19.614 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_zonegroup_remove ... ok 2024-04-16T18:07:19.615 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.615 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup create --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.685 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.685 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:19.736 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"71410d2a-9576-47c8-9b05-1abfcbf6e957","name":"remove_zg","api_name":"remove_zg","is_master":false,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":[]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-04-16T18:07:19.737 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.737 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.737 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:19.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:07:19.868+0000 7f7ee2709980 0 NOTICE: promoted remove as new master_zone of zonegroup remove_zg 2024-04-16T18:07:19.878 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"3f75f772-83a8-471b-908d-59d19a3834df","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","roles_pool":"remove.rgw.meta:roles","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","notif_pool":"remove.rgw.log:notif","topics_pool":"remove.rgw.meta:topics","account_pool":"remove.rgw.meta:accounts","group_pool":"remove.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80"} 2024-04-16T18:07:19.879 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:19.879 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:19.953 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:19.953 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:20.085 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":9,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"71410d2a-9576-47c8-9b05-1abfcbf6e957","name":"remove_zg","api_name":"remove_zg","is_master":false,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"3f75f772-83a8-471b-908d-59d19a3834df","zones":[{"id":"3f75f772-83a8-471b-908d-59d19a3834df","name":"remove","endpoints":[],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]},{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"3f75f772-83a8-471b-908d-59d19a3834df","val":848615525},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:07:20.085 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.085 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.085 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone delete --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:20.161 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:20.161 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:07:20.212+0000 7f7a0ca75980 0 NOTICE: removed master_zone of zonegroup remove_zg 2024-04-16T18:07:20.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:07:20.212+0000 7f7a0ca75980 0 Removed zone from zonegroup remove_zg 2024-04-16T18:07:20.231 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.231 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:20.304 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:20.304 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:20.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:07:20.356+0000 7f96197fc980 0 ERROR: zonegroup remove_zg has a non existent master zone 2024-04-16T18:07:20.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-16T18:07:20.367 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.367 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.368 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup delete --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:20.482 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.482 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:07:20.482 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:07:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:20.690 INFO:teuthology.orchestra.run.smithi071.stdout:{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","epoch":10,"predecessor_uuid":"2fe52312-e804-49a9-9525-abfdb920e903","sync_status":[],"period_map":{"id":"8c713f9b-78f8-4e05-979f-375a77da9531","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":true,"endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","zones":[{"id":"054102a3-8d94-4596-8b3b-6341eab312e1","name":"test-zone1","endpoints":["http://smithi071.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]},{"id":"bd830684-b604-4841-ab52-8c82ee812281","name":"test-zone2","endpoints":["http://smithi122.front.sepia.ceph.com:8000"],"log_meta":false,"log_data":true,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["compress-encrypted","notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]}],"short_zone_ids":[{"key":"054102a3-8d94-4596-8b3b-6341eab312e1","val":1171345946},{"key":"bd830684-b604-4841-ab52-8c82ee812281","val":666793065}]},"master_zonegroup":"test-zonegroup","master_zone":"054102a3-8d94-4596-8b3b-6341eab312e1","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"bucket_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false},"anonymous_ratelimit":{"max_read_ops":0,"max_write_ops":0,"max_read_bytes":0,"max_write_bytes":0,"enabled":false}},"realm_id":"750bb778-ce9e-43d8-aa1d-59a6a3ecda80","realm_epoch":2} 2024-04-16T18:07:20.691 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_zg_master_zone_delete ... ok 2024-04-16T18:07:20.692 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-32 2024-04-16T18:07:20.692 DEBUG:boto:path=/lybtug-32/ 2024-04-16T18:07:20.692 DEBUG:boto:auth_path=/lybtug-32/ 2024-04-16T18:07:20.692 DEBUG:boto:Method: PUT 2024-04-16T18:07:20.692 DEBUG:boto:Path: /lybtug-32/ 2024-04-16T18:07:20.693 DEBUG:boto:Data: 2024-04-16T18:07:20.693 DEBUG:boto:Headers: {} 2024-04-16T18:07:20.693 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:07:20.693 DEBUG:boto:Port: 8000 2024-04-16T18:07:20.693 DEBUG:boto:Params: {} 2024-04-16T18:07:20.693 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:07:20.694 DEBUG:boto:Token: None 2024-04-16T18:07:20.694 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:20 GMT /lybtug-32/ 2024-04-16T18:07:20.694 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mDIkXY57zQO/9DbkTLrLqzyKtME= 2024-04-16T18:07:20.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mDIkXY57zQO/9DbkTLrLqzyKtME=', 'Content-Length': '0'} 2024-04-16T18:07:21.502 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e3b3beeb4cab2ca0-00661ebe59-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:07:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:21.502 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-33 2024-04-16T18:07:21.503 DEBUG:boto:path=/lybtug-33/ 2024-04-16T18:07:21.503 DEBUG:boto:auth_path=/lybtug-33/ 2024-04-16T18:07:21.503 DEBUG:boto:Method: PUT 2024-04-16T18:07:21.503 DEBUG:boto:Path: /lybtug-33/ 2024-04-16T18:07:21.503 DEBUG:boto:Data: 2024-04-16T18:07:21.503 DEBUG:boto:Headers: {} 2024-04-16T18:07:21.503 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:07:21.503 DEBUG:boto:Port: 8000 2024-04-16T18:07:21.503 DEBUG:boto:Params: {} 2024-04-16T18:07:21.504 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:07:21.504 DEBUG:boto:Token: None 2024-04-16T18:07:21.504 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:21 GMT /lybtug-33/ 2024-04-16T18:07:21.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/Neu4bcEs/yGu23qzuh+MlVU7Ro= 2024-04-16T18:07:21.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/Neu4bcEs/yGu23qzuh+MlVU7Ro=', 'Content-Length': '0'} 2024-04-16T18:07:21.998 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000083cc5de1ad1fda6a-00661ebe59-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:07:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:21.999 DEBUG:boto:path=/lybtug-32/ 2024-04-16T18:07:21.999 DEBUG:boto:auth_path=/lybtug-32/ 2024-04-16T18:07:21.999 DEBUG:boto:path=/lybtug-32/?website 2024-04-16T18:07:21.999 DEBUG:boto:auth_path=/lybtug-32/?website 2024-04-16T18:07:21.999 DEBUG:boto:Method: PUT 2024-04-16T18:07:21.999 DEBUG:boto:Path: /lybtug-32/?website 2024-04-16T18:07:22.000 DEBUG:boto:Data: index.htmlerror.html 2024-04-16T18:07:22.000 DEBUG:boto:Headers: {} 2024-04-16T18:07:22.000 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:07:22.000 DEBUG:boto:Port: 8000 2024-04-16T18:07:22.000 DEBUG:boto:Params: {} 2024-04-16T18:07:22.000 DEBUG:boto:Token: None 2024-04-16T18:07:22.000 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:22 GMT /lybtug-32/?website 2024-04-16T18:07:22.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AmzQv1Psdzvi4CnjR3oAf8rQpt8= 2024-04-16T18:07:22.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AmzQv1Psdzvi4CnjR3oAf8rQpt8=', 'Content-Length': '241'} 2024-04-16T18:07:22.002 DEBUG:boto:Response headers: [('Content-Length', '221'), ('x-amz-request-id', 'tx000006fcdf710b6f89731-00661ebe5a-4804-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:07:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:22.003 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_set_bucket_website ... SKIP: test_set_bucket_website skipped. Requires rgw_enable_static_website = 1. 2024-04-16T18:07:22.003 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-34 2024-04-16T18:07:22.004 DEBUG:boto:path=/lybtug-34/ 2024-04-16T18:07:22.004 DEBUG:boto:auth_path=/lybtug-34/ 2024-04-16T18:07:22.004 DEBUG:boto:Method: PUT 2024-04-16T18:07:22.004 DEBUG:boto:Path: /lybtug-34/ 2024-04-16T18:07:22.004 DEBUG:boto:Data: 2024-04-16T18:07:22.004 DEBUG:boto:Headers: {} 2024-04-16T18:07:22.004 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:07:22.004 DEBUG:boto:Port: 8000 2024-04-16T18:07:22.004 DEBUG:boto:Params: {} 2024-04-16T18:07:22.005 DEBUG:boto:Token: None 2024-04-16T18:07:22.005 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:22 GMT /lybtug-34/ 2024-04-16T18:07:22.005 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SCBj/sO715itrCdoyx2892x6WHM= 2024-04-16T18:07:22.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SCBj/sO715itrCdoyx2892x6WHM=', 'Content-Length': '0'} 2024-04-16T18:07:22.039 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007190c9e86b2e1370-00661ebe5a-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:07:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:22.039 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-35 2024-04-16T18:07:22.039 DEBUG:boto:path=/lybtug-35/ 2024-04-16T18:07:22.040 DEBUG:boto:auth_path=/lybtug-35/ 2024-04-16T18:07:22.040 DEBUG:boto:Method: PUT 2024-04-16T18:07:22.040 DEBUG:boto:Path: /lybtug-35/ 2024-04-16T18:07:22.040 DEBUG:boto:Data: 2024-04-16T18:07:22.040 DEBUG:boto:Headers: {} 2024-04-16T18:07:22.040 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:07:22.040 DEBUG:boto:Port: 8000 2024-04-16T18:07:22.040 DEBUG:boto:Params: {} 2024-04-16T18:07:22.040 DEBUG:boto:Token: None 2024-04-16T18:07:22.040 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:22 GMT /lybtug-35/ 2024-04-16T18:07:22.041 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cJ6oLOOEcgkjgxDxDdnzO6rBGkQ= 2024-04-16T18:07:22.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cJ6oLOOEcgkjgxDxDdnzO6rBGkQ=', 'Content-Length': '0'} 2024-04-16T18:07:22.093 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000fdf79a8f1daf7c8-00661ebe5a-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:07:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:22.093 DEBUG:boto:path=/lybtug-34/ 2024-04-16T18:07:22.093 DEBUG:boto:auth_path=/lybtug-34/ 2024-04-16T18:07:22.093 DEBUG:boto:path=/lybtug-34/?policy 2024-04-16T18:07:22.094 DEBUG:boto:auth_path=/lybtug-34/?policy 2024-04-16T18:07:22.094 DEBUG:boto:Method: PUT 2024-04-16T18:07:22.094 DEBUG:boto:Path: /lybtug-34/?policy 2024-04-16T18:07:22.094 DEBUG:boto:Data: { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": "*" }] } 2024-04-16T18:07:22.094 DEBUG:boto:Headers: {} 2024-04-16T18:07:22.094 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:07:22.094 DEBUG:boto:Port: 8000 2024-04-16T18:07:22.094 DEBUG:boto:Params: {} 2024-04-16T18:07:22.094 DEBUG:boto:Token: None 2024-04-16T18:07:22.095 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:22 GMT /lybtug-34/?policy 2024-04-16T18:07:22.095 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7BjKx2/+OprMKj9ttAVTHlazeDQ= 2024-04-16T18:07:22.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7BjKx2/+OprMKj9ttAVTHlazeDQ=', 'Content-Length': '97'} 2024-04-16T18:07:22.107 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ae7354c471b9ce6e-00661ebe5a-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:07:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:22.107 DEBUG:boto:path=/lybtug-34/ 2024-04-16T18:07:22.107 DEBUG:boto:auth_path=/lybtug-34/ 2024-04-16T18:07:22.107 DEBUG:boto:path=/lybtug-34/?policy 2024-04-16T18:07:22.107 DEBUG:boto:auth_path=/lybtug-34/?policy 2024-04-16T18:07:22.108 DEBUG:boto:Method: GET 2024-04-16T18:07:22.108 DEBUG:boto:Path: /lybtug-34/?policy 2024-04-16T18:07:22.108 DEBUG:boto:Data: 2024-04-16T18:07:22.108 DEBUG:boto:Headers: {} 2024-04-16T18:07:22.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:07:22.108 DEBUG:boto:Port: 8000 2024-04-16T18:07:22.108 DEBUG:boto:Params: {} 2024-04-16T18:07:22.108 DEBUG:boto:Token: None 2024-04-16T18:07:22.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:07:22 GMT /lybtug-34/?policy 2024-04-16T18:07:22.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X1LjkOzV/i0tqKmVcuF2L2dbaQ= 2024-04-16T18:07:22.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X1LjkOzV/i0tqKmVcuF2L2dbaQ=', 'Content-Length': '0'} 2024-04-16T18:07:22.110 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006c241e47bdf10e55-00661ebe5a-4804-test-zone1'), ('Content-Type', 'application/json'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '97'), ('Date', 'Tue, 16 Apr 2024 18:07:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:22.110 DEBUG:boto:path=/lybtug-35/ 2024-04-16T18:07:22.111 DEBUG:boto:auth_path=/lybtug-35/ 2024-04-16T18:07:22.111 DEBUG:boto:path=/lybtug-35/?policy 2024-04-16T18:07:22.111 DEBUG:boto:auth_path=/lybtug-35/?policy 2024-04-16T18:07:22.111 DEBUG:boto:Method: PUT 2024-04-16T18:07:22.111 DEBUG:boto:Path: /lybtug-35/?policy 2024-04-16T18:07:22.111 DEBUG:boto:Data: { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": "*" }] } 2024-04-16T18:07:22.111 DEBUG:boto:Headers: {} 2024-04-16T18:07:22.111 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:07:22.111 DEBUG:boto:Port: 8000 2024-04-16T18:07:22.111 DEBUG:boto:Params: {} 2024-04-16T18:07:22.111 DEBUG:boto:Token: None 2024-04-16T18:07:22.112 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:22 GMT /lybtug-35/?policy 2024-04-16T18:07:22.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K3tDVWF0ZMNRJ9W9RBUoBp+ceAM= 2024-04-16T18:07:22.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K3tDVWF0ZMNRJ9W9RBUoBp+ceAM=', 'Content-Length': '97'} 2024-04-16T18:07:23.133 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000268253b0d2802d50-00661ebe5a-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:07:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:23.133 DEBUG:boto:path=/lybtug-35/ 2024-04-16T18:07:23.134 DEBUG:boto:auth_path=/lybtug-35/ 2024-04-16T18:07:23.134 DEBUG:boto:path=/lybtug-35/?policy 2024-04-16T18:07:23.134 DEBUG:boto:auth_path=/lybtug-35/?policy 2024-04-16T18:07:23.134 DEBUG:boto:Method: GET 2024-04-16T18:07:23.134 DEBUG:boto:Path: /lybtug-35/?policy 2024-04-16T18:07:23.134 DEBUG:boto:Data: 2024-04-16T18:07:23.134 DEBUG:boto:Headers: {} 2024-04-16T18:07:23.134 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:07:23.134 DEBUG:boto:Port: 8000 2024-04-16T18:07:23.134 DEBUG:boto:Params: {} 2024-04-16T18:07:23.135 DEBUG:boto:Token: None 2024-04-16T18:07:23.135 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:07:23 GMT /lybtug-35/?policy 2024-04-16T18:07:23.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MND+kNncdALqjnmIC9rZJC6+IlU= 2024-04-16T18:07:23.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MND+kNncdALqjnmIC9rZJC6+IlU=', 'Content-Length': '0'} 2024-04-16T18:07:23.136 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000247bf341f94fbfcc-00661ebe5b-4984-test-zone2'), ('Content-Type', 'application/json'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '97'), ('Date', 'Tue, 16 Apr 2024 18:07:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:23.137 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_set_bucket_policy ... ok 2024-04-16T18:07:23.137 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-36 2024-04-16T18:07:23.137 DEBUG:boto:path=/lybtug-36/ 2024-04-16T18:07:23.138 DEBUG:boto:auth_path=/lybtug-36/ 2024-04-16T18:07:23.138 DEBUG:boto:Method: PUT 2024-04-16T18:07:23.138 DEBUG:boto:Path: /lybtug-36/ 2024-04-16T18:07:23.138 DEBUG:boto:Data: 2024-04-16T18:07:23.138 DEBUG:boto:Headers: {} 2024-04-16T18:07:23.138 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:07:23.138 DEBUG:boto:Port: 8000 2024-04-16T18:07:23.138 DEBUG:boto:Params: {} 2024-04-16T18:07:23.138 DEBUG:boto:Token: None 2024-04-16T18:07:23.138 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:23 GMT /lybtug-36/ 2024-04-16T18:07:23.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1dOXZqQH9W8mhdDIyJFg97UXbhI= 2024-04-16T18:07:23.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1dOXZqQH9W8mhdDIyJFg97UXbhI=', 'Content-Length': '0'} 2024-04-16T18:07:23.175 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a80a2b51471db1f7-00661ebe5b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:07:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:23.175 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-37 2024-04-16T18:07:23.175 DEBUG:boto:path=/lybtug-37/ 2024-04-16T18:07:23.175 DEBUG:boto:auth_path=/lybtug-37/ 2024-04-16T18:07:23.175 DEBUG:boto:Method: PUT 2024-04-16T18:07:23.175 DEBUG:boto:Path: /lybtug-37/ 2024-04-16T18:07:23.175 DEBUG:boto:Data: 2024-04-16T18:07:23.176 DEBUG:boto:Headers: {} 2024-04-16T18:07:23.176 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:07:23.176 DEBUG:boto:Port: 8000 2024-04-16T18:07:23.176 DEBUG:boto:Params: {} 2024-04-16T18:07:23.176 DEBUG:boto:Token: None 2024-04-16T18:07:23.176 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:07:23 GMT /lybtug-37/ 2024-04-16T18:07:23.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1R7U4M8sSSq+tn9ZtF67V/wgbxA= 2024-04-16T18:07:23.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:07:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1R7U4M8sSSq+tn9ZtF67V/wgbxA=', 'Content-Length': '0'} 2024-04-16T18:07:23.239 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fc030c78e8475a9c-00661ebe5b-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:07:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:07:23.240 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:23.240 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:23.240 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:23.281 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:23.281 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:23.353 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:07:23.387 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290843.220252_338.1","last_update":"2024-04-16T18:07:23.220252Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:07:23.387 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:23.387 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:07:23.387 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:23.387 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:23.388 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:23.430 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:23.430 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:23.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:23.505+0000 7f0e822c1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:23.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:23.505+0000 7f0e822c1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:23.531 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:23Z"} 2024-04-16T18:07:23.532 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:23Z"} 2024-04-16T18:07:23.532 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:23.532 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:23.532 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:23.532 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:28.533 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:28.533 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:28.534 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:28.577 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:28.577 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:28.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:28.649+0000 7fa248cfd980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:28.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:28.653+0000 7fa248cfd980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:28.678 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:28Z"} 2024-04-16T18:07:28.679 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:28Z"} 2024-04-16T18:07:28.679 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:28.679 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:28.679 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:28.679 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:33.680 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:33.680 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:33.680 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:33.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:33.801+0000 7f6b6e2f5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:33.805+0000 7f6b6e2f5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:33.831 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:33Z"} 2024-04-16T18:07:33.832 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:33Z"} 2024-04-16T18:07:33.832 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:33.832 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:33.832 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:33.832 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:38.834 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:38.834 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:38.834 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:38.878 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:38.878 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:38.953+0000 7f819c71f980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:38.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:38.953+0000 7f819c71f980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:38.982 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:38Z"} 2024-04-16T18:07:38.982 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:38Z"} 2024-04-16T18:07:38.982 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:38.982 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:38.982 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:38.982 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:43.983 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:43.983 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:43.984 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:44.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:44.101+0000 7f3872486980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:44.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:44.105+0000 7f3872486980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:44.130 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:44Z"} 2024-04-16T18:07:44.130 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:44Z"} 2024-04-16T18:07:44.130 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:44.131 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:44.131 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:44.131 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:49.131 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:49.132 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:49.132 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:49.176 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:49.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:49.249+0000 7f5774eaa980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:49.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:49.253+0000 7f5774eaa980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:49.279 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:49Z"} 2024-04-16T18:07:49.279 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:49Z"} 2024-04-16T18:07:49.280 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:49.280 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:49.280 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:49.280 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:54.281 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:54.282 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:54.282 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:54.325 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:54.325 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:54.401+0000 7f7699444980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:54.405+0000 7f7699444980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:54.431 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:54Z"} 2024-04-16T18:07:54.431 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:54Z"} 2024-04-16T18:07:54.431 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:54.431 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:54.431 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:54.431 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:07:59.433 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:59.433 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:07:59.433 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:07:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:07:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:07:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:59.549+0000 7f5614824980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:07:59.549+0000 7f5614824980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:07:59.577 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:59Z"} 2024-04-16T18:07:59.577 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:07:59Z"} 2024-04-16T18:07:59.578 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:07:59.578 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:59.578 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:07:59.578 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:04.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:04.579 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:04.580 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:04.623 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:04.623 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:04.701+0000 7f53c3d01980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:04.705+0000 7f53c3d01980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:04.730 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:04Z"} 2024-04-16T18:08:04.731 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:04Z"} 2024-04-16T18:08:04.731 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:04.731 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:04.731 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:04.731 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:09.732 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:09.732 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:09.732 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:09.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:09.853+0000 7f481ed37980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:09.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:09.853+0000 7f481ed37980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:09.880 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:09Z"} 2024-04-16T18:08:09.880 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:09Z"} 2024-04-16T18:08:09.880 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:09.880 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:09.880 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:09.881 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:14.882 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:14.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:14.882 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:14.925 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:14.925 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:14.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:15.000+0000 7f488bd12980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:15.004+0000 7f488bd12980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:15.025 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:15Z"} 2024-04-16T18:08:15.026 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:15Z"} 2024-04-16T18:08:15.026 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:15.026 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:15.026 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:15.026 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:20.027 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:20.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:20.027 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:20.070 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:20.070 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:20.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:20.148+0000 7efeb0a7f980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:20.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:20.152+0000 7efeb0a7f980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:20.175 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:20Z"} 2024-04-16T18:08:20.175 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:20Z"} 2024-04-16T18:08:20.175 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:20.175 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:20.176 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:20.176 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:25.177 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:25.177 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:25.177 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:25.220 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:25.220 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:25.296+0000 7f09039c0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:25.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:25.300+0000 7f09039c0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:25.325 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:25Z"} 2024-04-16T18:08:25.325 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:25Z"} 2024-04-16T18:08:25.325 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:25.326 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:25.326 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:25.326 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:30.327 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:30.328 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:30.328 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:30.372 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:30.372 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:30.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:30.448+0000 7f75f57c2980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:30.452+0000 7f75f57c2980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:30.477 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:30Z"} 2024-04-16T18:08:30.477 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:30Z"} 2024-04-16T18:08:30.477 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:30.478 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:30.478 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:30.478 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:35.479 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:35.479 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:35.479 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:35.523 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:35.599+0000 7ff172e10980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:35.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:35.599+0000 7ff172e10980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:35.627 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:35Z"} 2024-04-16T18:08:35.627 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:35Z"} 2024-04-16T18:08:35.628 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:35.628 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:35.628 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:35.628 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:40.629 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:40.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:40.629 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:40.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:40.751+0000 7fc8ad5e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:40.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:40.751+0000 7fc8ad5e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:40.778 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:40Z"} 2024-04-16T18:08:40.778 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:40Z"} 2024-04-16T18:08:40.778 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:40.779 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:40.779 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:40.779 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:45.780 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:45.780 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:45.781 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:45.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:45.899+0000 7f8262396980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:45.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:45.903+0000 7f8262396980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:45.929 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:45Z"} 2024-04-16T18:08:45.929 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:45Z"} 2024-04-16T18:08:45.929 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:45.930 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:45.930 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:45.930 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:50.931 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:50.932 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:50.932 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:51.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:51.055+0000 7fd9bc645980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:51.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:51.055+0000 7fd9bc645980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:51.084 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:51Z"} 2024-04-16T18:08:51.084 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290524.429884_257.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:02:04.429884Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:51Z"} 2024-04-16T18:08:51.084 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290524.429884_257.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:02:04.429884Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:51.085 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:51.085 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290524.429884_257.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:51.085 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290524.429884_257.1 2024-04-16T18:08:56.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:56.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:08:56.087 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:08:56.129 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:08:56.129 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:08:56.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:56.223+0000 7f751c8c2980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:56.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:08:56.227+0000 7f751c8c2980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:08:56.257 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290841.494218_306.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:07:21.494218Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:56Z"} 2024-04-16T18:08:56.257 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290841.494218_306.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:07:21.494218Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:08:56Z"} 2024-04-16T18:08:56.257 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290841.494218_306.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:07:21.494218Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:08:56.258 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:56.258 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290841.494218_306.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:08:56.258 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290841.494218_306.1 2024-04-16T18:09:01.258 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:01.259 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:01.259 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:01.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:01.374+0000 7f0351ca1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:09:01.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:01.378+0000 7f0351ca1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:09:01.404 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290841.963846_313.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:07:21.963846Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:09:01Z"} 2024-04-16T18:09:01.405 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290841.963846_313.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:07:21.963846Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:09:01Z"} 2024-04-16T18:09:01.405 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290841.963846_313.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:07:21.963846Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:09:01.405 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:01.405 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290841.963846_313.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:01.405 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713290843.220252_338.1 target=1_1713290841.963846_313.1 2024-04-16T18:09:06.406 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:06.406 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:06.407 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:06.451 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:06.452 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:06.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:06.526+0000 7fd0547ec980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:09:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:06.526+0000 7fd0547ec980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:09:06.555 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290843.220252_338.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:07:23.220252Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:09:06Z"} 2024-04-16T18:09:06.555 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290843.220252_338.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:07:23.220252Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:09:06Z"} 2024-04-16T18:09:06.555 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290843.220252_338.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:07:23.220252Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:09:06.556 DEBUG:rgw_multi.tests:log_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:06.556 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290843.220252_338.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:06.556 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:09:06.556 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'lybtug-36', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:06.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'lybtug-36', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:06.557 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket lybtug-36 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:09:06.603 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:06.603 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:06.758 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'lybtug-37', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:06.758 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'lybtug-37', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:06.758 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket lybtug-37 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:09:06.799 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:06.799 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:06.953 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:06.953 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:06.953 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:07.030 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:07.031 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:07.138 INFO:teuthology.orchestra.run.smithi071.stdout:["sync.error-log.29","obj_delete_at_hint.0000000078","meta.history","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:2","sync.error-log.23","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:1","sync.error-log.8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:4","obj_delete_at_hint.0000000028","data_log.2.HFJ3QP7uNY/.0","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.3.retry","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:10","sync.error-log.30","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:5","obj_delete_at_hint.0000000040","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:10","sync.error-log.6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:8","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10","meta.log.8c713f9b-78f8-4e05-979f-375a77da9531.0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:8","obj_delete_at_hint.0000000015","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:7","obj_delete_at_hint.0000000069","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:6","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:9","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:5","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:8","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:5","obj_delete_at_hint.0000000024","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.1.retry","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:3","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:0","data_log.3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:7","data_loggenerations_metadata","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:10","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:3","sync.error-log.21","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:5","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:2","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:9","obj_delete_at_hint.0000000110","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:7","sync.error-log.0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:8","obj_delete_at_hint.0000000096","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:5","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:7","obj_delete_at_hint.0000000006","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21","obj_delete_at_hint.0000000029","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:2","obj_delete_at_hint.0000000089","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:8","obj_delete_at_hint.0000000022","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:9","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15","bilog.trim","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:1","obj_delete_at_hint.0000000016","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:10","obj_delete_at_hint.0000000041","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:9","obj_delete_at_hint.0000000018","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:10","obj_delete_at_hint.0000000092","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:9","sync.error-log.27","sync.error-log.4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:8","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31","obj_delete_at_hint.0000000014","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:10","obj_delete_at_hint.0000000112","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:10","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:6","sync.error-log.16","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:9","obj_delete_at_hint.0000000064","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:1","obj_delete_at_hint.0000000071","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:10","obj_delete_at_hint.0000000074","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:8","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:4","sync.error-log.17","obj_delete_at_hint.0000000125","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:1","obj_delete_at_hint.0000000082","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:5","meta.log.8c713f9b-78f8-4e05-979f-375a77da9531.1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:8","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:4","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:3","sync.error-log.24","data_log.0.hnzroe/K1/W.0","obj_delete_at_hint.0000000053","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:4","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:8","sync.error-log.7","sync.error-log.3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:7","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:10","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:2","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.0.retry","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:10","obj_delete_at_hint.0000000045","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:5","obj_delete_at_hint.0000000002","sync.error-log.12","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:0","obj_delete_at_hint.0000000116","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:5","sync.error-log.25","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:4","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14","obj_delete_at_hint.0000000034","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:0","obj_delete_at_hint.0000000101","sync.error-log.13","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:7","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:0","sync.error-log.14","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:3","obj_delete_at_hint.0000000066","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:9","obj_delete_at_hint.0000000068","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:1","obj_delete_at_hint.0000000085","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:2","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:5","sync.error-log.10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:3","obj_delete_at_hint.0000000118","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:10","data_log.1.urpH+yfuLxt.0","obj_delete_at_hint.0000000036","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:0","obj_delete_at_hint.0000000103","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:1","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.2.retry","obj_delete_at_hint.0000000119","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:3","obj_delete_at_hint.0000000098","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:9","obj_delete_at_hint.0000000027","obj_delete_at_hint.0000000019","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:0","obj_delete_at_hint.0000000039","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:5","obj_delete_at_hint.0000000100","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:0","obj_delete_at_hint.0000000093","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:6","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:2","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:3","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:8","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24","sync.error-log.2","data_log.0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:6","obj_delete_at_hint.0000000012","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:4","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:2","sync.error-log.11","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:4","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:7","obj_delete_at_hint.0000000020","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:8","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","sync.error-log.19","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:0","sync.error-log.28","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:9","obj_delete_at_hint.0000000072","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:1","obj_delete_at_hint.0000000062","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:4","obj_delete_at_hint.0000000115","sync.error-log.1","obj_delete_at_hint.0000000033","datalog.sync-status.bd830684-b604-4841-ab52-8c82ee812281","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:2","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:0","obj_delete_at_hint.0000000025","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:1","obj_delete_at_hint.0000000123","sync.error-log.31","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:2","sync.error-log.18","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:10","obj_delete_at_hint.0000000060","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9","obj_delete_at_hint.0000000109","sync.error-log.15","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:5","obj_delete_at_hint.0000000013","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:9","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:10","obj_delete_at_hint.0000000023","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:6","obj_delete_at_hint.0000000107","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:7","obj_delete_at_hint.0000000046","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:1","sync.error-log.26","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:9","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:7","sync.error-log.22","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:9","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:9","data_log.2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:10","obj_delete_at_hint.0000000067","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:2","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:10","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:9","sync.error-log.9","obj_delete_at_hint.0000000086","sync.error-log.5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:0","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:5","sync.error-log.20","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:5","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:7","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:3","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4:2","obj_delete_at_hint.0000000124","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:1","obj_delete_at_hint.0000000001","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:2","obj_delete_at_hint.0000000126","data_log.3.cKqsHiBRZOS.0","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:0","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:4","obj_delete_at_hint.0000000097","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:3","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1","bucket.full-sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4","datalog.sync-status.shard.bd830684-b604-4841-ab52-8c82ee812281.0","obj_delete_at_hint.0000000042","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:1","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1:2","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:6","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3:8","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:4","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-6:054102a3-8d94-4596-8b3b-6341eab312e1.4353.10:3","obj_delete_at_hint.0000000075","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:9","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:7","bucket.sync-status.bd830684-b604-4841-ab52-8c82ee812281:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:6"] 2024-04-16T18:09:07.171 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:07.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:07.171 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:07.210 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:07.211 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:07.319 INFO:teuthology.orchestra.run.smithi122.stdout:["bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:4","sync.error-log.29","obj_delete_at_hint.0000000078","meta.history","sync.error-log.23","obj_delete_at_hint.0000000070","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:10","obj_delete_at_hint.0000000104","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:6","obj_delete_at_hint.0000000026","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:1","sync.error-log.8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:6","obj_delete_at_hint.0000000028","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:2","sync.error-log.30","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:6","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22","obj_delete_at_hint.0000000040","sync.error-log.6","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:2","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:3","meta.log.8c713f9b-78f8-4e05-979f-375a77da9531.0","obj_delete_at_hint.0000000015","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:10","obj_delete_at_hint.0000000069","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:7","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14","obj_delete_at_hint.0000000035","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:8","obj_delete_at_hint.0000000037","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:8","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:8","obj_delete_at_hint.0000000031","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:4","data_log.3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:5","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:9","data_loggenerations_metadata","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:6","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:5","obj_delete_at_hint.0000000076","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:2","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:1","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:6","sync.error-log.21","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:6","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:1","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:5","obj_delete_at_hint.0000000090","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:3","obj_delete_at_hint.0000000110","sync.error-log.0","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:10","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:10","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.0.retry","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:3","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:10","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:10","bilog.trim","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:9","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:10","sync.error-log.27","sync.error-log.4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:1","obj_delete_at_hint.0000000014","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:6","sync.error-log.16","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:4","obj_delete_at_hint.0000000064","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:8","data_log.1.u+JN/wqyuVT.0","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:10","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:6","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","sync.error-log.17","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","meta.log.8c713f9b-78f8-4e05-979f-375a77da9531.1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:3","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:8","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:0","datalog.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:10","sync.error-log.24","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:0","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.2","obj_delete_at_hint.0000000053","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:7","obj_delete_at_hint.0000000091","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:7","obj_delete_at_hint.0000000065","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:2","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:6","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:6","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:9","sync.error-log.7","sync.error-log.3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:1","mdlog.sync-status.shard.2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:6","obj_delete_at_hint.0000000045","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:0","obj_delete_at_hint.0000000002","sync.error-log.12","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:10","obj_delete_at_hint.0000000116","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20","sync.error-log.25","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31","obj_delete_at_hint.0000000034","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:2","obj_delete_at_hint.0000000101","sync.error-log.13","mdlog.sync-status.shard.1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:6","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13","obj_delete_at_hint.0000000079","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:4","obj_delete_at_hint.0000000049","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:7","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:5","sync.error-log.14","obj_delete_at_hint.0000000066","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:2","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:1","obj_delete_at_hint.0000000068","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:7","obj_delete_at_hint.0000000085","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:8","obj_delete_at_hint.0000000073","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:8","obj_delete_at_hint.0000000038","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:4","sync.error-log.10","obj_delete_at_hint.0000000118","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:0","obj_delete_at_hint.0000000036","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:6","obj_delete_at_hint.0000000103","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:9","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:5","mdlog.sync-status.shard.3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:1","data_log.3.k0yiR6DJp0L.0","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:4","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:6","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21","mdlog.sync-status","obj_delete_at_hint.0000000019","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:8","obj_delete_at_hint.0000000039","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:9","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.3.retry","obj_delete_at_hint.0000000100","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:10","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:7","obj_delete_at_hint.0000000093","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:2","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:7","obj_delete_at_hint.0000000122","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:0","obj_delete_at_hint.0000000057","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:8","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","sync.error-log.2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:4","data_log.0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:4","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:8","sync.error-log.11","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:6","obj_delete_at_hint.0000000111","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:5","obj_delete_at_hint.0000000048","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:10","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17","obj_delete_at_hint.0000000020","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:10","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:6","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:2","sync.error-log.19","sync.error-log.28","obj_delete_at_hint.0000000072","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.0","obj_delete_at_hint.0000000062","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:5","obj_delete_at_hint.0000000115","sync.error-log.1","obj_delete_at_hint.0000000033","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-9:054102a3-8d94-4596-8b3b-6341eab312e1.4353.13:1","obj_delete_at_hint.0000000025","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:10","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:8","obj_delete_at_hint.0000000123","sync.error-log.31","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:2","sync.error-log.18","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:10","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","sync.error-log.15","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:10","obj_delete_at_hint.0000000013","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:8","obj_delete_at_hint.0000000051","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:9","obj_delete_at_hint.0000000106","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:1","obj_delete_at_hint.0000000017","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:7","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:5","obj_delete_at_hint.0000000023","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-27:054102a3-8d94-4596-8b3b-6341eab312e1.4353.31:5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:6","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:0","obj_delete_at_hint.0000000107","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-7:054102a3-8d94-4596-8b3b-6341eab312e1.4353.11:10","obj_delete_at_hint.0000000046","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:7","sync.error-log.26","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:4","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:9","sync.error-log.22","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1","data_log.2","obj_delete_at_hint.0000000067","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-17:054102a3-8d94-4596-8b3b-6341eab312e1.4353.21:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-13:054102a3-8d94-4596-8b3b-6341eab312e1.4353.17:3","sync.error-log.9","obj_delete_at_hint.0000000086","sync.error-log.5","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-31:054102a3-8d94-4596-8b3b-6341eab312e1.4699.1:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:10","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:8","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:7","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:5","sync.error-log.20","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-28:054102a3-8d94-4596-8b3b-6341eab312e1.4353.32:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:10","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.1.retry","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:1","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-20:054102a3-8d94-4596-8b3b-6341eab312e1.4353.24:0","bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-30:054102a3-8d94-4596-8b3b-6341eab312e1.4353.34:9","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-19:054102a3-8d94-4596-8b3b-6341eab312e1.4353.23:3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:9","obj_delete_at_hint.0000000124","data_log.0.sDZpjHKXmcv.0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-11:054102a3-8d94-4596-8b3b-6341eab312e1.4353.15:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-16:054102a3-8d94-4596-8b3b-6341eab312e1.4353.20:7","obj_delete_at_hint.0000000001","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.3","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:4","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-15:054102a3-8d94-4596-8b3b-6341eab312e1.4353.19:6","obj_delete_at_hint.0000000061","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:2","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-12:054102a3-8d94-4596-8b3b-6341eab312e1.4353.16:0","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-29:054102a3-8d94-4596-8b3b-6341eab312e1.4353.33:6","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-14:054102a3-8d94-4596-8b3b-6341eab312e1.4353.18:1","obj_delete_at_hint.0000000097","data_log.2.D7e1iR5DjC/.0","datalog.sync-status.shard.054102a3-8d94-4596-8b3b-6341eab312e1.2.retry","obj_delete_at_hint.0000000042","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-18:054102a3-8d94-4596-8b3b-6341eab312e1.4353.22:9","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-8:054102a3-8d94-4596-8b3b-6341eab312e1.4353.12:3","obj_delete_at_hint.0000000075","bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-10:054102a3-8d94-4596-8b3b-6341eab312e1.4353.14:5","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2024-04-16T18:09:07.353 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2024-04-16T18:09:07.354 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:07.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:07.354 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:07.396 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:07.396 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:07.510 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"00000000000000000000:00000000000000006281","last_update":"2024-04-16T18:09:06.917416Z"},{"marker":"00000000000000000000:00000000000000004036","last_update":"2024-04-16T18:09:06.920073Z"},{"marker":"00000000000000000000:00000000000000003045","last_update":"2024-04-16T18:09:06.912016Z"},{"marker":"00000000000000000000:00000000000000006983","last_update":"2024-04-16T18:09:06.914485Z"}] 2024-04-16T18:09:07.510 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000006281', 1: '00000000000000000000:00000000000000004036', 2: '00000000000000000000:00000000000000003045', 3: '00000000000000000000:00000000000000006983'} 2024-04-16T18:09:07.511 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:09:07.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:07.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:07.511 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:07.557 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:07.557 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:07.667 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005441","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:04:11.694490Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:07Z"} 2024-04-16T18:09:07.668 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005441","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:04:11.694490Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:07Z"} 2024-04-16T18:09:07.668 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005441', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:04:11.694490Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003193', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.988842Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002484', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:54:55.037491Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006142', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:22.361740Z'}}] 2024-04-16T18:09:07.668 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000006281', 1: '00000000000000000000:00000000000000004036', 2: '00000000000000000000:00000000000000003045', 3: '00000000000000000000:00000000000000006983'} 2024-04-16T18:09:07.668 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005441', 1: '00000000000000000000:00000000000000003193', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000006142'} 2024-04-16T18:09:07.669 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=00000000000000000000:00000000000000006281 target=00000000000000000000:00000000000000005441, shard=1 master=00000000000000000000:00000000000000004036 target=00000000000000000000:00000000000000003193, shard=2 master=00000000000000000000:00000000000000003045 target=00000000000000000000:00000000000000002484, shard=3 master=00000000000000000000:00000000000000006983 target=00000000000000000000:00000000000000006142 2024-04-16T18:09:12.669 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:12.670 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:12.670 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:12.717 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:12.717 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:12.826 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005441","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:04:11.694490Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:12Z"} 2024-04-16T18:09:12.826 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005441","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:04:11.694490Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:12Z"} 2024-04-16T18:09:12.826 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005441', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:04:11.694490Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003193', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.988842Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002484', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:54:55.037491Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006142', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:22.361740Z'}}] 2024-04-16T18:09:12.827 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000006281', 1: '00000000000000000000:00000000000000004036', 2: '00000000000000000000:00000000000000003045', 3: '00000000000000000000:00000000000000006983'} 2024-04-16T18:09:12.827 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005441', 1: '00000000000000000000:00000000000000003193', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000006142'} 2024-04-16T18:09:12.827 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=00000000000000000000:00000000000000006281 target=00000000000000000000:00000000000000005441, shard=1 master=00000000000000000000:00000000000000004036 target=00000000000000000000:00000000000000003193, shard=2 master=00000000000000000000:00000000000000003045 target=00000000000000000000:00000000000000002484, shard=3 master=00000000000000000000:00000000000000006983 target=00000000000000000000:00000000000000006142 2024-04-16T18:09:17.827 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:17.828 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:17.828 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:17.873 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:17.873 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:17.981 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005441","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:04:11.694490Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:17Z"} 2024-04-16T18:09:17.981 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005441","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:04:11.694490Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003193","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:20.988842Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000002484","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:54:55.037491Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006142","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:00:22.361740Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:17Z"} 2024-04-16T18:09:17.981 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005441', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:04:11.694490Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003193', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:20.988842Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002484', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:54:55.037491Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006142', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:00:22.361740Z'}}] 2024-04-16T18:09:17.982 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000006281', 1: '00000000000000000000:00000000000000004036', 2: '00000000000000000000:00000000000000003045', 3: '00000000000000000000:00000000000000006983'} 2024-04-16T18:09:17.982 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005441', 1: '00000000000000000000:00000000000000003193', 2: '00000000000000000000:00000000000000002484', 3: '00000000000000000000:00000000000000006142'} 2024-04-16T18:09:17.982 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=00000000000000000000:00000000000000006281 target=00000000000000000000:00000000000000005441, shard=1 master=00000000000000000000:00000000000000004036 target=00000000000000000000:00000000000000003193, shard=2 master=00000000000000000000:00000000000000003045 target=00000000000000000000:00000000000000002484, shard=3 master=00000000000000000000:00000000000000006983 target=00000000000000000000:00000000000000006142 2024-04-16T18:09:22.982 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:22.983 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:22.983 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:23.031 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:23.031 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:23.147 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006281","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.917416Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004036","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.920073Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003045","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.912016Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006983","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.914485Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:23Z"} 2024-04-16T18:09:23.147 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006281","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.917416Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004036","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.920073Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003045","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.912016Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006983","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:06.914485Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:09:23Z"} 2024-04-16T18:09:23.147 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006281', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:06.917416Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004036', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:06.920073Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003045', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:06.912016Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006983', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:06.914485Z'}}] 2024-04-16T18:09:23.148 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000006281', 1: '00000000000000000000:00000000000000004036', 2: '00000000000000000000:00000000000000003045', 3: '00000000000000000000:00000000000000006983'} 2024-04-16T18:09:23.148 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000006281', 1: '00000000000000000000:00000000000000004036', 2: '00000000000000000000:00000000000000003045', 3: '00000000000000000000:00000000000000006983'} 2024-04-16T18:09:23.148 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:09:23.148 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2024-04-16T18:09:23.148 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.148 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:23.181 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:23.181 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:23.291 INFO:teuthology.orchestra.run.smithi122.stdout:[{"marker":"00000000000000000000:00000000000000005296","last_update":"2024-04-16T18:09:09.268194Z"},{"marker":"00000000000000000000:00000000000000003753","last_update":"2024-04-16T18:09:09.271779Z"},{"marker":"00000000000000000000:00000000000000003186","last_update":"2024-04-16T18:09:09.261589Z"},{"marker":"00000000000000000000:00000000000000006560","last_update":"2024-04-16T18:09:09.264992Z"}] 2024-04-16T18:09:23.291 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000005296', 1: '00000000000000000000:00000000000000003753', 2: '00000000000000000000:00000000000000003186', 3: '00000000000000000000:00000000000000006560'} 2024-04-16T18:09:23.291 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:09:23.291 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.291 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.291 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:23.339 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:23.339 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:23.440 INFO:teuthology.orchestra.run.smithi071.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005296","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.268194Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003753","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.271779Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003186","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T18:09:09.261589Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006560","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.264992Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:09:23Z"} 2024-04-16T18:09:23.441 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005296","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.268194Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003753","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.271779Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003186","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T18:09:09.261589Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006560","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.264992Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:09:23Z"} 2024-04-16T18:09:23.441 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005296', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:09.268194Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003753', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:09.271779Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003186', 'next_step_marker': '', 'total_entries': 7, 'pos': 0, 'timestamp': '2024-04-16T18:09:09.261589Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006560', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:09.264992Z'}}] 2024-04-16T18:09:23.441 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000005296', 1: '00000000000000000000:00000000000000003753', 2: '00000000000000000000:00000000000000003186', 3: '00000000000000000000:00000000000000006560'} 2024-04-16T18:09:23.441 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005296', 1: '00000000000000000000:00000000000000003753', 2: '00000000000000000000:00000000000000003186', 3: '00000000000000000000:00000000000000006560'} 2024-04-16T18:09:23.441 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:09:23.442 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_disable ... ok 2024-04-16T18:09:23.443 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-38 2024-04-16T18:09:23.444 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:23.444 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:23.444 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.444 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:23.444 DEBUG:boto:Data: 2024-04-16T18:09:23.444 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.444 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.444 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.445 DEBUG:boto:Params: {} 2024-04-16T18:09:23.445 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:09:23.445 DEBUG:boto:Token: None 2024-04-16T18:09:23.446 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/ 2024-04-16T18:09:23.446 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EYGpRR3rzAUb+aFp1PM+UOLQDB8= 2024-04-16T18:09:23.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EYGpRR3rzAUb+aFp1PM+UOLQDB8=', 'Content-Length': '0'} 2024-04-16T18:09:23.477 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c35b3f39a9be0bfe-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.478 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-39 2024-04-16T18:09:23.478 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:23.478 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:23.478 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.478 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:23.478 DEBUG:boto:Data: 2024-04-16T18:09:23.478 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.479 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.479 DEBUG:boto:Params: {} 2024-04-16T18:09:23.479 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:09:23.479 DEBUG:boto:Token: None 2024-04-16T18:09:23.479 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/ 2024-04-16T18:09:23.479 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:otPzhHMxiQiyfVEjzSBtDfaR4AE= 2024-04-16T18:09:23.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:otPzhHMxiQiyfVEjzSBtDfaR4AE=', 'Content-Length': '0'} 2024-04-16T18:09:23.540 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ff5061a1ef35f6d9-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.540 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:23.540 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:23.540 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.540 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:23.540 DEBUG:boto:Data: 2024-04-16T18:09:23.541 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.541 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.541 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.541 DEBUG:boto:Params: {} 2024-04-16T18:09:23.541 DEBUG:boto:Token: None 2024-04-16T18:09:23.541 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/ 2024-04-16T18:09:23.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE= 2024-04-16T18:09:23.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE=', 'Content-Length': '0'} 2024-04-16T18:09:23.545 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c5b182667b0d8066-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.545 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:23.545 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:23.545 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.546 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:23.546 DEBUG:boto:Data: 2024-04-16T18:09:23.546 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.546 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.546 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.546 DEBUG:boto:Params: {} 2024-04-16T18:09:23.546 DEBUG:boto:Token: None 2024-04-16T18:09:23.546 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/obj1 2024-04-16T18:09:23.546 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gdkncwStulzDgGNcXt1u+hmsc8A= 2024-04-16T18:09:23.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gdkncwStulzDgGNcXt1u+hmsc8A='} 2024-04-16T18:09:23.562 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000de45dfe601c8def7-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.562 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:23.562 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:23.562 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.563 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:23.563 DEBUG:boto:Data: 2024-04-16T18:09:23.563 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.563 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.563 DEBUG:boto:Params: {} 2024-04-16T18:09:23.563 DEBUG:boto:Token: None 2024-04-16T18:09:23.563 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/ 2024-04-16T18:09:23.563 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE= 2024-04-16T18:09:23.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE=', 'Content-Length': '0'} 2024-04-16T18:09:23.567 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000590d134beea8d9e8-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.567 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:23.567 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:23.567 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.567 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:23.568 DEBUG:boto:Data: 2024-04-16T18:09:23.568 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.568 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.568 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.568 DEBUG:boto:Params: {} 2024-04-16T18:09:23.568 DEBUG:boto:Token: None 2024-04-16T18:09:23.568 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/obj2 2024-04-16T18:09:23.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gXDYX32mTRpf5Xtme0JHxjvf1Yc= 2024-04-16T18:09:23.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gXDYX32mTRpf5Xtme0JHxjvf1Yc='} 2024-04-16T18:09:23.610 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000239e2cda3cad2cd8-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.611 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:23.611 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:23.611 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.611 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:23.611 DEBUG:boto:Data: 2024-04-16T18:09:23.611 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.611 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.611 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.611 DEBUG:boto:Params: {} 2024-04-16T18:09:23.611 DEBUG:boto:Token: None 2024-04-16T18:09:23.611 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/ 2024-04-16T18:09:23.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE= 2024-04-16T18:09:23.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE=', 'Content-Length': '0'} 2024-04-16T18:09:23.615 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007c33728cba7f0b66-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.616 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:23.616 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:23.616 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.616 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:23.616 DEBUG:boto:Data: 2024-04-16T18:09:23.616 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.616 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.616 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.616 DEBUG:boto:Params: {} 2024-04-16T18:09:23.616 DEBUG:boto:Token: None 2024-04-16T18:09:23.616 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/obj3 2024-04-16T18:09:23.617 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eZTfV657BjR5r4fWJPUVsiTrYOY= 2024-04-16T18:09:23.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eZTfV657BjR5r4fWJPUVsiTrYOY='} 2024-04-16T18:09:23.628 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b4b4f45255579de5-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.628 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:23.628 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:23.628 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.629 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:23.629 DEBUG:boto:Data: 2024-04-16T18:09:23.629 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.629 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.629 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.629 DEBUG:boto:Params: {} 2024-04-16T18:09:23.629 DEBUG:boto:Token: None 2024-04-16T18:09:23.629 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/ 2024-04-16T18:09:23.629 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE= 2024-04-16T18:09:23.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8suPVmO6+qR/RtyJzi/u26RSgzE=', 'Content-Length': '0'} 2024-04-16T18:09:23.632 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000043e7d72b4422099-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.633 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:23.633 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:23.633 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.633 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:23.633 DEBUG:boto:Data: 2024-04-16T18:09:23.633 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.633 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:23.633 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.633 DEBUG:boto:Params: {} 2024-04-16T18:09:23.633 DEBUG:boto:Token: None 2024-04-16T18:09:23.633 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-38/obj4 2024-04-16T18:09:23.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YCnlLKyYvIUCSzlZ0Is2DSlf544= 2024-04-16T18:09:23.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YCnlLKyYvIUCSzlZ0Is2DSlf544='} 2024-04-16T18:09:23.678 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000baeb1bb6d72774d1-00661ebed3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.679 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:23.679 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:23.679 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.679 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:23.679 DEBUG:boto:Data: 2024-04-16T18:09:23.679 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.679 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.679 DEBUG:boto:Params: {} 2024-04-16T18:09:23.679 DEBUG:boto:Token: None 2024-04-16T18:09:23.680 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/ 2024-04-16T18:09:23.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U= 2024-04-16T18:09:23.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U=', 'Content-Length': '0'} 2024-04-16T18:09:23.685 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050e46c3545680189-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.685 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:23.685 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:23.685 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.685 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:09:23.685 DEBUG:boto:Data: 2024-04-16T18:09:23.685 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.685 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.686 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.686 DEBUG:boto:Params: {} 2024-04-16T18:09:23.686 DEBUG:boto:Token: None 2024-04-16T18:09:23.686 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/obj1 2024-04-16T18:09:23.686 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LN6kekPCkHI7EF6NpHaxigXUdsU= 2024-04-16T18:09:23.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LN6kekPCkHI7EF6NpHaxigXUdsU='} 2024-04-16T18:09:23.702 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005d677b0c73e25d84-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.703 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:23.703 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:23.703 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.703 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:23.703 DEBUG:boto:Data: 2024-04-16T18:09:23.703 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.703 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.703 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.703 DEBUG:boto:Params: {} 2024-04-16T18:09:23.703 DEBUG:boto:Token: None 2024-04-16T18:09:23.703 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/ 2024-04-16T18:09:23.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U= 2024-04-16T18:09:23.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U=', 'Content-Length': '0'} 2024-04-16T18:09:23.707 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008cc7cbe2ef187490-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.708 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:23.708 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:23.708 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.708 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:09:23.708 DEBUG:boto:Data: 2024-04-16T18:09:23.708 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.708 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.708 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.708 DEBUG:boto:Params: {} 2024-04-16T18:09:23.708 DEBUG:boto:Token: None 2024-04-16T18:09:23.708 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/obj2 2024-04-16T18:09:23.709 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:64CMWn5XXT5v9Qi6SEpbuhbs++A= 2024-04-16T18:09:23.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:64CMWn5XXT5v9Qi6SEpbuhbs++A='} 2024-04-16T18:09:23.750 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000406858f683b1236a-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.751 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:23.751 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:23.751 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.751 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:23.751 DEBUG:boto:Data: 2024-04-16T18:09:23.751 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.751 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.751 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.751 DEBUG:boto:Params: {} 2024-04-16T18:09:23.751 DEBUG:boto:Token: None 2024-04-16T18:09:23.751 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/ 2024-04-16T18:09:23.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U= 2024-04-16T18:09:23.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U=', 'Content-Length': '0'} 2024-04-16T18:09:23.755 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000073fa5d8ea440b5e5-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.756 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:23.756 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:23.756 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.756 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:09:23.756 DEBUG:boto:Data: 2024-04-16T18:09:23.756 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.756 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.756 DEBUG:boto:Params: {} 2024-04-16T18:09:23.756 DEBUG:boto:Token: None 2024-04-16T18:09:23.757 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/obj3 2024-04-16T18:09:23.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ohNZ9uCVOXatFLZao96pb5zY2ng= 2024-04-16T18:09:23.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ohNZ9uCVOXatFLZao96pb5zY2ng='} 2024-04-16T18:09:23.768 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006128b9726cac9cda-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.768 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:23.768 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:23.768 DEBUG:boto:Method: HEAD 2024-04-16T18:09:23.769 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:23.769 DEBUG:boto:Data: 2024-04-16T18:09:23.769 DEBUG:boto:Headers: {} 2024-04-16T18:09:23.769 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.769 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.769 DEBUG:boto:Params: {} 2024-04-16T18:09:23.769 DEBUG:boto:Token: None 2024-04-16T18:09:23.769 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/ 2024-04-16T18:09:23.769 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U= 2024-04-16T18:09:23.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JAu+BisGgRLl0CMp7TSI6hL744U=', 'Content-Length': '0'} 2024-04-16T18:09:23.773 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008a308cab4a0683b7-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.773 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:23.773 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:23.773 DEBUG:boto:Method: PUT 2024-04-16T18:09:23.773 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:09:23.774 DEBUG:boto:Data: 2024-04-16T18:09:23.774 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:23.774 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:23.774 DEBUG:boto:Port: 8000 2024-04-16T18:09:23.774 DEBUG:boto:Params: {} 2024-04-16T18:09:23.774 DEBUG:boto:Token: None 2024-04-16T18:09:23.774 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:23 GMT /lybtug-39/obj4 2024-04-16T18:09:23.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hyrlchiYUdi9riTxd1iT8clZRBs= 2024-04-16T18:09:23.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hyrlchiYUdi9riTxd1iT8clZRBs='} 2024-04-16T18:09:23.819 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009c1365c30ab09f00-00661ebed3-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:23.819 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.819 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:23.860 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:23.861 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:23.928 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:09:23.961 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713290963.519706_362.1","last_update":"2024-04-16T18:09:23.519706Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:09:23.961 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713290963.519706_362.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:23.961 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:09:23.961 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:09:23.962 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:09:24.004 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:24.004 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:24.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:24.078+0000 7fc23512b980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:09:24.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:24.082+0000 7fc23512b980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:09:24.108 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290963.519706_362.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:09:23.519706Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:09:24Z"} 2024-04-16T18:09:24.109 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713290963.519706_362.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:09:23.519706Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:09:24Z"} 2024-04-16T18:09:24.109 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713290963.519706_362.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:09:23.519706Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:09:24.109 DEBUG:rgw_multi.tests:log_status={0: '1_1713290963.519706_362.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:24.109 DEBUG:rgw_multi.tests:sync_status={0: '1_1713290963.519706_362.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:09:24.109 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:09:24.109 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-38 2024-04-16T18:09:24.109 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:24.109 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:24.110 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-38 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:09:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:24.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:24.282+0000 7f1c0d73b980 1 waiting to reach incremental sync.. 2024-04-16T18:09:29.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:29.285+0000 7f1c0d73b980 1 bucket sync caught up with source: 2024-04-16T18:09:29.281 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.21.6, , 00000000001.35.6, , 00000000001.22.6, , , 00000000001.31.6, , , ] 2024-04-16T18:09:29.282 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.21.6, , 00000000001.35.6, , 00000000001.22.6, , , 00000000001.31.6, , , ] 2024-04-16T18:09:29.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:29.285+0000 7f1c0d73b980 0 bucket checkpoint complete 2024-04-16T18:09:29.330 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-38 2024-04-16T18:09:29.330 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:29.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:29.331 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-38 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:09:29.382 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:29.382 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:29.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:29.461+0000 7f20b2a4d980 1 waiting to reach incremental sync.. 2024-04-16T18:09:34.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:34.461+0000 7f20b2a4d980 1 bucket sync caught up with source: 2024-04-16T18:09:34.461 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.1316.6, , 00000000001.1281.6, , 00000000001.654.6, , , 00000000001.866.6, , , ] 2024-04-16T18:09:34.462 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000001.1316.6, , 00000000001.1281.6, , 00000000001.654.6, , , 00000000001.866.6, , , ] 2024-04-16T18:09:34.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:34.461+0000 7f20b2a4d980 0 bucket checkpoint complete 2024-04-16T18:09:34.509 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-38 zones={test-zone2, test-zone1} 2024-04-16T18:09:34.509 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.509 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.509 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.509 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:34.509 DEBUG:boto:Data: 2024-04-16T18:09:34.509 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.509 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.509 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.510 DEBUG:boto:Params: {} 2024-04-16T18:09:34.510 DEBUG:boto:Token: None 2024-04-16T18:09:34.510 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.510 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zVaxKUiOanh9m41yygSK8br36mc= 2024-04-16T18:09:34.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zVaxKUiOanh9m41yygSK8br36mc=', 'Content-Length': '0'} 2024-04-16T18:09:34.514 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000084a95aab825c4bb1-00661ebede-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.514 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.514 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.514 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.514 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:34.515 DEBUG:boto:Data: 2024-04-16T18:09:34.515 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.515 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.515 DEBUG:boto:Params: {} 2024-04-16T18:09:34.515 DEBUG:boto:Token: None 2024-04-16T18:09:34.515 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.515 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zVaxKUiOanh9m41yygSK8br36mc= 2024-04-16T18:09:34.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zVaxKUiOanh9m41yygSK8br36mc=', 'Content-Length': '0'} 2024-04-16T18:09:34.519 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000297bfbe86fbf788e-00661ebede-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.519 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:09:34.519 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.519 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.519 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.519 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.519 DEBUG:boto:Method: GET 2024-04-16T18:09:34.519 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.520 DEBUG:boto:Data: 2024-04-16T18:09:34.520 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.520 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.520 DEBUG:boto:Params: {} 2024-04-16T18:09:34.520 DEBUG:boto:Token: None 2024-04-16T18:09:34.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/?versions 2024-04-16T18:09:34.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE= 2024-04-16T18:09:34.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE=', 'Content-Length': '0'} 2024-04-16T18:09:34.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004473dacd12757311-00661ebede-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.524 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.525 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:09:34.525 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:09:34.526 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:09:34.526 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:09:34.526 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:09:34.526 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.526 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.526 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.526 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.526 DEBUG:boto:Method: GET 2024-04-16T18:09:34.526 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.526 DEBUG:boto:Data: 2024-04-16T18:09:34.527 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.527 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.527 DEBUG:boto:Params: {} 2024-04-16T18:09:34.527 DEBUG:boto:Token: None 2024-04-16T18:09:34.527 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/?versions 2024-04-16T18:09:34.527 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE= 2024-04-16T18:09:34.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE=', 'Content-Length': '0'} 2024-04-16T18:09:34.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f75117a7fc8f9e61-00661ebede-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.531 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.531 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:09:34.531 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:09:34.531 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:09:34.531 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:09:34.531 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.531 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.531 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.532 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.532 DEBUG:boto:Method: GET 2024-04-16T18:09:34.532 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.532 DEBUG:boto:Data: 2024-04-16T18:09:34.532 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.532 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.532 DEBUG:boto:Params: {} 2024-04-16T18:09:34.532 DEBUG:boto:Token: None 2024-04-16T18:09:34.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/?versions 2024-04-16T18:09:34.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE= 2024-04-16T18:09:34.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE=', 'Content-Length': '0'} 2024-04-16T18:09:34.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0fe849b47715eb9-00661ebede-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.536 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.537 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.537 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.537 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.537 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.537 DEBUG:boto:Method: GET 2024-04-16T18:09:34.537 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:34.537 DEBUG:boto:Data: 2024-04-16T18:09:34.537 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.537 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.538 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.538 DEBUG:boto:Params: {} 2024-04-16T18:09:34.538 DEBUG:boto:Token: None 2024-04-16T18:09:34.538 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/?versions 2024-04-16T18:09:34.538 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE= 2024-04-16T18:09:34.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+rlkyuoM4x7L458mb/yGsjL/qOE=', 'Content-Length': '0'} 2024-04-16T18:09:34.541 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d050acad7fc5547-00661ebede-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.541 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.542 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:34.542 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.542 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.542 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.542 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.542 DEBUG:boto:Method: GET 2024-04-16T18:09:34.542 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.543 DEBUG:boto:Data: 2024-04-16T18:09:34.543 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.543 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.543 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.543 DEBUG:boto:Params: {} 2024-04-16T18:09:34.543 DEBUG:boto:Token: None 2024-04-16T18:09:34.543 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE= 2024-04-16T18:09:34.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE=', 'Content-Length': '0'} 2024-04-16T18:09:34.546 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c8912b3443255d7-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.547 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.547 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.547 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.547 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.547 DEBUG:boto:Method: GET 2024-04-16T18:09:34.547 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.547 DEBUG:boto:Data: 2024-04-16T18:09:34.547 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.547 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.547 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.547 DEBUG:boto:Params: {} 2024-04-16T18:09:34.548 DEBUG:boto:Token: None 2024-04-16T18:09:34.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE= 2024-04-16T18:09:34.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE=', 'Content-Length': '0'} 2024-04-16T18:09:34.550 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c31edf81b34f3890-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.551 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.551 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.551 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.551 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.551 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.551 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.551 DEBUG:boto:Data: 2024-04-16T18:09:34.551 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.551 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.551 DEBUG:boto:Params: {} 2024-04-16T18:09:34.552 DEBUG:boto:Token: None 2024-04-16T18:09:34.552 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xvG+mv9PcdJCI870t0qwQX3q7cg= 2024-04-16T18:09:34.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xvG+mv9PcdJCI870t0qwQX3q7cg=', 'Content-Length': '0'} 2024-04-16T18:09:34.554 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001774b5e000ce8c75-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.555 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.555 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.555 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.555 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.555 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.555 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.555 DEBUG:boto:Data: 2024-04-16T18:09:34.555 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.555 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.555 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.555 DEBUG:boto:Params: {} 2024-04-16T18:09:34.556 DEBUG:boto:Token: None 2024-04-16T18:09:34.556 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xvG+mv9PcdJCI870t0qwQX3q7cg= 2024-04-16T18:09:34.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xvG+mv9PcdJCI870t0qwQX3q7cg=', 'Content-Length': '0'} 2024-04-16T18:09:34.558 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000074b05d647ac72c7c-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.558 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:34.559 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.559 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.559 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.559 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.559 DEBUG:boto:Method: GET 2024-04-16T18:09:34.559 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.559 DEBUG:boto:Data: 2024-04-16T18:09:34.559 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.559 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.559 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.559 DEBUG:boto:Params: {} 2024-04-16T18:09:34.560 DEBUG:boto:Token: None 2024-04-16T18:09:34.560 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE= 2024-04-16T18:09:34.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE=', 'Content-Length': '0'} 2024-04-16T18:09:34.563 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004db3b54b6bb9609f-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.606 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.606 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.607 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.607 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.607 DEBUG:boto:Method: GET 2024-04-16T18:09:34.607 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.607 DEBUG:boto:Data: 2024-04-16T18:09:34.607 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.607 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.607 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.607 DEBUG:boto:Params: {} 2024-04-16T18:09:34.607 DEBUG:boto:Token: None 2024-04-16T18:09:34.607 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:34.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE= 2024-04-16T18:09:34.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wVb19zVYO3vhuNkHWGJ7HX2KxQE=', 'Content-Length': '0'} 2024-04-16T18:09:34.610 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000168140dea58ef48c-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.611 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.611 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.611 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.611 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.611 DEBUG:boto:Method: GET 2024-04-16T18:09:34.611 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:34.611 DEBUG:boto:Data: 2024-04-16T18:09:34.611 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.611 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.611 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.611 DEBUG:boto:Params: {} 2024-04-16T18:09:34.612 DEBUG:boto:Token: None 2024-04-16T18:09:34.612 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.615 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ece6ace8852cb9d7-00661ebede-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.616 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.616 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.616 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.616 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.616 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.616 DEBUG:boto:Method: GET 2024-04-16T18:09:34.616 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:34.617 DEBUG:boto:Data: 2024-04-16T18:09:34.617 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.617 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.617 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.617 DEBUG:boto:Params: {} 2024-04-16T18:09:34.617 DEBUG:boto:Token: None 2024-04-16T18:09:34.617 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.617 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.621 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d9e1c35b82592b4-00661ebede-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.621 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.622 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.622 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.622 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.622 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:34.622 DEBUG:boto:Data: 2024-04-16T18:09:34.622 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.622 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.622 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.623 DEBUG:boto:Params: {} 2024-04-16T18:09:34.623 DEBUG:boto:Token: None 2024-04-16T18:09:34.623 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1 2024-04-16T18:09:34.623 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ARwwHGaF3no+UMpaD+Y2CeFwalc= 2024-04-16T18:09:34.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ARwwHGaF3no+UMpaD+Y2CeFwalc=', 'Content-Length': '0'} 2024-04-16T18:09:34.625 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000066aa08a1af92d4bb-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.626 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.626 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.626 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.626 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:34.626 DEBUG:boto:Data: 2024-04-16T18:09:34.626 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.626 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.626 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.626 DEBUG:boto:Params: {} 2024-04-16T18:09:34.627 DEBUG:boto:Token: None 2024-04-16T18:09:34.627 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1 2024-04-16T18:09:34.627 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ARwwHGaF3no+UMpaD+Y2CeFwalc= 2024-04-16T18:09:34.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ARwwHGaF3no+UMpaD+Y2CeFwalc=', 'Content-Length': '0'} 2024-04-16T18:09:34.629 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ed999494457074a1-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.630 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:34.630 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.630 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.630 DEBUG:boto:Method: GET 2024-04-16T18:09:34.630 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:34.630 DEBUG:boto:Data: 2024-04-16T18:09:34.630 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.630 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.630 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.630 DEBUG:boto:Params: {} 2024-04-16T18:09:34.630 DEBUG:boto:Token: None 2024-04-16T18:09:34.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1 2024-04-16T18:09:34.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7p6bO3wal+XMy3IhIQZzT/RcpTE= 2024-04-16T18:09:34.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7p6bO3wal+XMy3IhIQZzT/RcpTE=', 'Content-Length': '0'} 2024-04-16T18:09:34.634 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000016b3f4ea26592439-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.678 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:34.678 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:34.679 DEBUG:boto:Method: GET 2024-04-16T18:09:34.679 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:34.679 DEBUG:boto:Data: 2024-04-16T18:09:34.679 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.679 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.679 DEBUG:boto:Params: {} 2024-04-16T18:09:34.679 DEBUG:boto:Token: None 2024-04-16T18:09:34.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj1 2024-04-16T18:09:34.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7p6bO3wal+XMy3IhIQZzT/RcpTE= 2024-04-16T18:09:34.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7p6bO3wal+XMy3IhIQZzT/RcpTE=', 'Content-Length': '0'} 2024-04-16T18:09:34.682 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ccf672df5c08f5d5-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.682 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.683 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.683 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.683 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.683 DEBUG:boto:Method: GET 2024-04-16T18:09:34.683 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:34.683 DEBUG:boto:Data: 2024-04-16T18:09:34.683 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.683 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.683 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.683 DEBUG:boto:Params: {} 2024-04-16T18:09:34.684 DEBUG:boto:Token: None 2024-04-16T18:09:34.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.687 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd9c95b40fe1c56c-00661ebede-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.730 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.731 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.731 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.731 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.731 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:34.731 DEBUG:boto:Method: GET 2024-04-16T18:09:34.731 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:34.731 DEBUG:boto:Data: 2024-04-16T18:09:34.731 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.731 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.731 DEBUG:boto:Params: {} 2024-04-16T18:09:34.732 DEBUG:boto:Token: None 2024-04-16T18:09:34.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c38db9888cebd408-00661ebede-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.736 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.736 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:34.736 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.736 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.736 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.736 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.736 DEBUG:boto:Method: GET 2024-04-16T18:09:34.737 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.737 DEBUG:boto:Data: 2024-04-16T18:09:34.737 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.737 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.737 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.737 DEBUG:boto:Params: {} 2024-04-16T18:09:34.737 DEBUG:boto:Token: None 2024-04-16T18:09:34.737 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4= 2024-04-16T18:09:34.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4=', 'Content-Length': '0'} 2024-04-16T18:09:34.740 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cd617427bc9f2d12-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.782 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.782 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.783 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.783 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.783 DEBUG:boto:Method: GET 2024-04-16T18:09:34.783 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.783 DEBUG:boto:Data: 2024-04-16T18:09:34.783 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.783 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.783 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.783 DEBUG:boto:Params: {} 2024-04-16T18:09:34.783 DEBUG:boto:Token: None 2024-04-16T18:09:34.783 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4= 2024-04-16T18:09:34.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4=', 'Content-Length': '0'} 2024-04-16T18:09:34.786 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000005a904281ffbf304-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.787 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.787 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.787 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.787 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.787 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.787 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.787 DEBUG:boto:Data: 2024-04-16T18:09:34.787 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.787 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.787 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.787 DEBUG:boto:Params: {} 2024-04-16T18:09:34.788 DEBUG:boto:Token: None 2024-04-16T18:09:34.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BhP6Cg+Ar/9RwjI/HSbst12HkjQ= 2024-04-16T18:09:34.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BhP6Cg+Ar/9RwjI/HSbst12HkjQ=', 'Content-Length': '0'} 2024-04-16T18:09:34.791 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000093445e020de0952d-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.791 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.791 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.791 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.792 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.792 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.792 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.792 DEBUG:boto:Data: 2024-04-16T18:09:34.792 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.792 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.792 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.792 DEBUG:boto:Params: {} 2024-04-16T18:09:34.792 DEBUG:boto:Token: None 2024-04-16T18:09:34.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BhP6Cg+Ar/9RwjI/HSbst12HkjQ= 2024-04-16T18:09:34.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BhP6Cg+Ar/9RwjI/HSbst12HkjQ=', 'Content-Length': '0'} 2024-04-16T18:09:34.795 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000070630c64abfabe12-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:34.795 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.795 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.795 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.796 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.796 DEBUG:boto:Method: GET 2024-04-16T18:09:34.796 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.796 DEBUG:boto:Data: 2024-04-16T18:09:34.796 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.796 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.796 DEBUG:boto:Params: {} 2024-04-16T18:09:34.796 DEBUG:boto:Token: None 2024-04-16T18:09:34.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4= 2024-04-16T18:09:34.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4=', 'Content-Length': '0'} 2024-04-16T18:09:34.799 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000045fbfd99ad9e2940-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.842 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.842 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.843 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.843 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.843 DEBUG:boto:Method: GET 2024-04-16T18:09:34.843 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.843 DEBUG:boto:Data: 2024-04-16T18:09:34.843 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.843 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.843 DEBUG:boto:Params: {} 2024-04-16T18:09:34.843 DEBUG:boto:Token: None 2024-04-16T18:09:34.844 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:34.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4= 2024-04-16T18:09:34.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14SL/GrGoYBellrek2daZabx1Q4=', 'Content-Length': '0'} 2024-04-16T18:09:34.846 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005010c01eb7287d09-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.847 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.847 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.847 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.847 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.847 DEBUG:boto:Method: GET 2024-04-16T18:09:34.847 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:34.847 DEBUG:boto:Data: 2024-04-16T18:09:34.847 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.847 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.848 DEBUG:boto:Params: {} 2024-04-16T18:09:34.848 DEBUG:boto:Token: None 2024-04-16T18:09:34.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c6677ea84fbf7a7-00661ebede-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.852 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.852 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.852 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.852 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.852 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.853 DEBUG:boto:Method: GET 2024-04-16T18:09:34.853 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:34.853 DEBUG:boto:Data: 2024-04-16T18:09:34.853 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.853 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.853 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.853 DEBUG:boto:Params: {} 2024-04-16T18:09:34.853 DEBUG:boto:Token: None 2024-04-16T18:09:34.853 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.853 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.857 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d9eac96422f4709-00661ebede-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.857 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.858 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.858 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.858 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.858 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:34.858 DEBUG:boto:Data: 2024-04-16T18:09:34.858 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.858 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.858 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.858 DEBUG:boto:Params: {} 2024-04-16T18:09:34.859 DEBUG:boto:Token: None 2024-04-16T18:09:34.859 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2 2024-04-16T18:09:34.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HPzD5Mm/V/E/CT9Y5ieUyWTesEw= 2024-04-16T18:09:34.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HPzD5Mm/V/E/CT9Y5ieUyWTesEw=', 'Content-Length': '0'} 2024-04-16T18:09:34.862 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000004957b4a514cc7bd-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.862 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.862 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.862 DEBUG:boto:Method: HEAD 2024-04-16T18:09:34.862 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:34.862 DEBUG:boto:Data: 2024-04-16T18:09:34.863 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.863 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.863 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.863 DEBUG:boto:Params: {} 2024-04-16T18:09:34.863 DEBUG:boto:Token: None 2024-04-16T18:09:34.863 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2 2024-04-16T18:09:34.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HPzD5Mm/V/E/CT9Y5ieUyWTesEw= 2024-04-16T18:09:34.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HPzD5Mm/V/E/CT9Y5ieUyWTesEw=', 'Content-Length': '0'} 2024-04-16T18:09:34.865 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000969bb17134d0c556-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.866 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:34.866 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.866 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.866 DEBUG:boto:Method: GET 2024-04-16T18:09:34.866 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:34.866 DEBUG:boto:Data: 2024-04-16T18:09:34.866 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.866 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.866 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.867 DEBUG:boto:Params: {} 2024-04-16T18:09:34.867 DEBUG:boto:Token: None 2024-04-16T18:09:34.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2 2024-04-16T18:09:34.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NFmnloIdUMoD7BOCIqTtdgdiQgQ= 2024-04-16T18:09:34.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NFmnloIdUMoD7BOCIqTtdgdiQgQ=', 'Content-Length': '0'} 2024-04-16T18:09:34.870 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000779f841334b3eb15-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.910 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:34.910 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:34.911 DEBUG:boto:Method: GET 2024-04-16T18:09:34.911 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:34.911 DEBUG:boto:Data: 2024-04-16T18:09:34.911 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.911 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.911 DEBUG:boto:Params: {} 2024-04-16T18:09:34.911 DEBUG:boto:Token: None 2024-04-16T18:09:34.911 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj2 2024-04-16T18:09:34.911 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NFmnloIdUMoD7BOCIqTtdgdiQgQ= 2024-04-16T18:09:34.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NFmnloIdUMoD7BOCIqTtdgdiQgQ=', 'Content-Length': '0'} 2024-04-16T18:09:34.914 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006fed6a70581a1112-00661ebede-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.914 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.915 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.915 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.915 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.915 DEBUG:boto:Method: GET 2024-04-16T18:09:34.915 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:34.915 DEBUG:boto:Data: 2024-04-16T18:09:34.915 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.915 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.915 DEBUG:boto:Params: {} 2024-04-16T18:09:34.916 DEBUG:boto:Token: None 2024-04-16T18:09:34.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066edbc38680b03a7-00661ebede-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.962 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.963 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:34.963 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:34.963 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.963 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:34.963 DEBUG:boto:Method: GET 2024-04-16T18:09:34.963 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:34.963 DEBUG:boto:Data: 2024-04-16T18:09:34.963 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:34.963 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.963 DEBUG:boto:Params: {} 2024-04-16T18:09:34.964 DEBUG:boto:Token: None 2024-04-16T18:09:34.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/ 2024-04-16T18:09:34.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU= 2024-04-16T18:09:34.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0X81G4cg+87NzGI11V+ZOC/L/RU=', 'Content-Length': '0'} 2024-04-16T18:09:34.968 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000156d5e6528ec981-00661ebede-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:34.968 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:34.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:34.969 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:34.969 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:34.969 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:34.969 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:34.969 DEBUG:boto:Method: GET 2024-04-16T18:09:34.969 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:34.969 DEBUG:boto:Data: 2024-04-16T18:09:34.969 DEBUG:boto:Headers: {} 2024-04-16T18:09:34.969 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:34.969 DEBUG:boto:Port: 8000 2024-04-16T18:09:34.969 DEBUG:boto:Params: {} 2024-04-16T18:09:34.970 DEBUG:boto:Token: None 2024-04-16T18:09:34.970 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:34 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:34.970 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GoL5uzGJD8xsUay9X6noSBx5ZUw= 2024-04-16T18:09:34.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GoL5uzGJD8xsUay9X6noSBx5ZUw=', 'Content-Length': '0'} 2024-04-16T18:09:34.973 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e577807f7b5020dd-00661ebede-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.014 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.015 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.015 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.015 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.015 DEBUG:boto:Method: GET 2024-04-16T18:09:35.015 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.015 DEBUG:boto:Data: 2024-04-16T18:09:35.015 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.015 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.015 DEBUG:boto:Params: {} 2024-04-16T18:09:35.015 DEBUG:boto:Token: None 2024-04-16T18:09:35.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UktOkjbQnPrdmjIQPfhRp2kqb+c= 2024-04-16T18:09:35.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UktOkjbQnPrdmjIQPfhRp2kqb+c=', 'Content-Length': '0'} 2024-04-16T18:09:35.019 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cace265d9d3c500a-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.019 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.019 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.019 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.019 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.019 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.019 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.019 DEBUG:boto:Data: 2024-04-16T18:09:35.019 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.020 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.020 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.020 DEBUG:boto:Params: {} 2024-04-16T18:09:35.020 DEBUG:boto:Token: None 2024-04-16T18:09:35.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zDqktaIlc5s0XynQ/QcQLHQvHLY= 2024-04-16T18:09:35.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zDqktaIlc5s0XynQ/QcQLHQvHLY=', 'Content-Length': '0'} 2024-04-16T18:09:35.023 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000886c7a1671d7257b-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.023 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.023 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.023 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.024 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.024 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.024 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.024 DEBUG:boto:Data: 2024-04-16T18:09:35.024 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.024 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.024 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.024 DEBUG:boto:Params: {} 2024-04-16T18:09:35.024 DEBUG:boto:Token: None 2024-04-16T18:09:35.024 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zDqktaIlc5s0XynQ/QcQLHQvHLY= 2024-04-16T18:09:35.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zDqktaIlc5s0XynQ/QcQLHQvHLY=', 'Content-Length': '0'} 2024-04-16T18:09:35.027 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a8109ef49627c9c1-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.027 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:35.027 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.028 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.028 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.028 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.028 DEBUG:boto:Method: GET 2024-04-16T18:09:35.028 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.028 DEBUG:boto:Data: 2024-04-16T18:09:35.028 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.028 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.028 DEBUG:boto:Params: {} 2024-04-16T18:09:35.028 DEBUG:boto:Token: None 2024-04-16T18:09:35.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UktOkjbQnPrdmjIQPfhRp2kqb+c= 2024-04-16T18:09:35.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UktOkjbQnPrdmjIQPfhRp2kqb+c=', 'Content-Length': '0'} 2024-04-16T18:09:35.032 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000edc5bc62ea92e3a5-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.074 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.074 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.075 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.075 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.075 DEBUG:boto:Method: GET 2024-04-16T18:09:35.075 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.075 DEBUG:boto:Data: 2024-04-16T18:09:35.075 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.075 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.075 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.075 DEBUG:boto:Params: {} 2024-04-16T18:09:35.075 DEBUG:boto:Token: None 2024-04-16T18:09:35.075 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:35.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UktOkjbQnPrdmjIQPfhRp2kqb+c= 2024-04-16T18:09:35.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UktOkjbQnPrdmjIQPfhRp2kqb+c=', 'Content-Length': '0'} 2024-04-16T18:09:35.079 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bd3fda5fe1370f0f-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.079 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.079 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.079 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.079 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.079 DEBUG:boto:Method: GET 2024-04-16T18:09:35.079 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:35.080 DEBUG:boto:Data: 2024-04-16T18:09:35.080 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.080 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.080 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.080 DEBUG:boto:Params: {} 2024-04-16T18:09:35.080 DEBUG:boto:Token: None 2024-04-16T18:09:35.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c40066805c947a88-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.084 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.085 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.085 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.085 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.085 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.085 DEBUG:boto:Method: GET 2024-04-16T18:09:35.085 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:35.085 DEBUG:boto:Data: 2024-04-16T18:09:35.085 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.085 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.085 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.085 DEBUG:boto:Params: {} 2024-04-16T18:09:35.086 DEBUG:boto:Token: None 2024-04-16T18:09:35.086 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.086 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.090 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0de1b82e84b118a-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.090 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.090 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.091 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.091 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.091 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:35.091 DEBUG:boto:Data: 2024-04-16T18:09:35.091 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.091 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.091 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.091 DEBUG:boto:Params: {} 2024-04-16T18:09:35.091 DEBUG:boto:Token: None 2024-04-16T18:09:35.091 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3 2024-04-16T18:09:35.091 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kP8eIZlETLo9kq5PrxQlrYnnjfs= 2024-04-16T18:09:35.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kP8eIZlETLo9kq5PrxQlrYnnjfs=', 'Content-Length': '0'} 2024-04-16T18:09:35.094 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000022011789e35cd06b-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.095 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.095 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.095 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.095 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:35.095 DEBUG:boto:Data: 2024-04-16T18:09:35.095 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.095 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.095 DEBUG:boto:Params: {} 2024-04-16T18:09:35.095 DEBUG:boto:Token: None 2024-04-16T18:09:35.096 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3 2024-04-16T18:09:35.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kP8eIZlETLo9kq5PrxQlrYnnjfs= 2024-04-16T18:09:35.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kP8eIZlETLo9kq5PrxQlrYnnjfs=', 'Content-Length': '0'} 2024-04-16T18:09:35.098 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000aea4f67c65d3b5d1-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.099 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:35.099 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.099 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.099 DEBUG:boto:Method: GET 2024-04-16T18:09:35.099 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:35.099 DEBUG:boto:Data: 2024-04-16T18:09:35.099 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.099 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.099 DEBUG:boto:Params: {} 2024-04-16T18:09:35.100 DEBUG:boto:Token: None 2024-04-16T18:09:35.100 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3 2024-04-16T18:09:35.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ouQajjvZjycg2g2TuoFVN2Sbe3g= 2024-04-16T18:09:35.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ouQajjvZjycg2g2TuoFVN2Sbe3g=', 'Content-Length': '0'} 2024-04-16T18:09:35.103 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000166c7723fc52350e-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.146 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:35.146 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:35.147 DEBUG:boto:Method: GET 2024-04-16T18:09:35.147 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:35.147 DEBUG:boto:Data: 2024-04-16T18:09:35.147 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.147 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.147 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.147 DEBUG:boto:Params: {} 2024-04-16T18:09:35.147 DEBUG:boto:Token: None 2024-04-16T18:09:35.147 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj3 2024-04-16T18:09:35.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ouQajjvZjycg2g2TuoFVN2Sbe3g= 2024-04-16T18:09:35.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ouQajjvZjycg2g2TuoFVN2Sbe3g=', 'Content-Length': '0'} 2024-04-16T18:09:35.150 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b1b3122c0012882e-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.151 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.151 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.151 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.151 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.151 DEBUG:boto:Method: GET 2024-04-16T18:09:35.151 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:35.151 DEBUG:boto:Data: 2024-04-16T18:09:35.151 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.151 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.151 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.152 DEBUG:boto:Params: {} 2024-04-16T18:09:35.152 DEBUG:boto:Token: None 2024-04-16T18:09:35.152 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.155 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006956f168f0ac37a5-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.156 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.156 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.156 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.156 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.156 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:35.156 DEBUG:boto:Method: GET 2024-04-16T18:09:35.156 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:35.157 DEBUG:boto:Data: 2024-04-16T18:09:35.157 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.157 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.157 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.157 DEBUG:boto:Params: {} 2024-04-16T18:09:35.157 DEBUG:boto:Token: None 2024-04-16T18:09:35.157 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.157 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.161 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097f570df85b76684-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.161 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.162 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:35.162 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.162 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.162 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.162 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.162 DEBUG:boto:Method: GET 2024-04-16T18:09:35.162 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.162 DEBUG:boto:Data: 2024-04-16T18:09:35.162 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.162 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.163 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.163 DEBUG:boto:Params: {} 2024-04-16T18:09:35.163 DEBUG:boto:Token: None 2024-04-16T18:09:35.163 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.163 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o= 2024-04-16T18:09:35.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o=', 'Content-Length': '0'} 2024-04-16T18:09:35.166 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009a00179a101d0a05-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.206 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.206 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.207 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.207 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.207 DEBUG:boto:Method: GET 2024-04-16T18:09:35.207 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.207 DEBUG:boto:Data: 2024-04-16T18:09:35.207 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.207 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.207 DEBUG:boto:Params: {} 2024-04-16T18:09:35.207 DEBUG:boto:Token: None 2024-04-16T18:09:35.207 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o= 2024-04-16T18:09:35.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o=', 'Content-Length': '0'} 2024-04-16T18:09:35.210 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002daf99f7108c988e-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.211 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.211 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.211 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.211 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.211 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.211 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.211 DEBUG:boto:Data: 2024-04-16T18:09:35.211 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.212 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.212 DEBUG:boto:Params: {} 2024-04-16T18:09:35.212 DEBUG:boto:Token: None 2024-04-16T18:09:35.212 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iYsRqy0wOpZqHej/AyJrkVCuFT4= 2024-04-16T18:09:35.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iYsRqy0wOpZqHej/AyJrkVCuFT4=', 'Content-Length': '0'} 2024-04-16T18:09:35.215 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b140f823fbc4e969-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.215 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.215 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.215 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.215 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.215 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.215 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.216 DEBUG:boto:Data: 2024-04-16T18:09:35.216 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.216 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.216 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.216 DEBUG:boto:Params: {} 2024-04-16T18:09:35.216 DEBUG:boto:Token: None 2024-04-16T18:09:35.216 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iYsRqy0wOpZqHej/AyJrkVCuFT4= 2024-04-16T18:09:35.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iYsRqy0wOpZqHej/AyJrkVCuFT4=', 'Content-Length': '0'} 2024-04-16T18:09:35.219 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000232ba775014fee31-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.219 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:35.219 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.220 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.220 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.220 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.220 DEBUG:boto:Method: GET 2024-04-16T18:09:35.220 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.220 DEBUG:boto:Data: 2024-04-16T18:09:35.220 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.220 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.220 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.220 DEBUG:boto:Params: {} 2024-04-16T18:09:35.220 DEBUG:boto:Token: None 2024-04-16T18:09:35.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o= 2024-04-16T18:09:35.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o=', 'Content-Length': '0'} 2024-04-16T18:09:35.223 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000067bc0cd2b6ed1efe-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.266 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.266 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.267 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.267 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.267 DEBUG:boto:Method: GET 2024-04-16T18:09:35.267 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.267 DEBUG:boto:Data: 2024-04-16T18:09:35.267 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.267 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.267 DEBUG:boto:Params: {} 2024-04-16T18:09:35.267 DEBUG:boto:Token: None 2024-04-16T18:09:35.267 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:35.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o= 2024-04-16T18:09:35.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mpjTTNeeVoXLkP/L+/5Dd+L7v9o=', 'Content-Length': '0'} 2024-04-16T18:09:35.271 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000223925d1a2dfc975-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.271 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.271 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.271 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.271 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.271 DEBUG:boto:Method: GET 2024-04-16T18:09:35.271 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:35.271 DEBUG:boto:Data: 2024-04-16T18:09:35.272 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.272 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.272 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.272 DEBUG:boto:Params: {} 2024-04-16T18:09:35.272 DEBUG:boto:Token: None 2024-04-16T18:09:35.272 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.272 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.276 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048005d85fff7eb2f-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.276 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.276 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.276 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.277 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.277 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.277 DEBUG:boto:Method: GET 2024-04-16T18:09:35.277 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:35.277 DEBUG:boto:Data: 2024-04-16T18:09:35.277 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.277 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.277 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.277 DEBUG:boto:Params: {} 2024-04-16T18:09:35.277 DEBUG:boto:Token: None 2024-04-16T18:09:35.277 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f412eb293bd202fd-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.282 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.282 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.282 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.282 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.282 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:35.282 DEBUG:boto:Data: 2024-04-16T18:09:35.283 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.283 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.283 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.283 DEBUG:boto:Params: {} 2024-04-16T18:09:35.283 DEBUG:boto:Token: None 2024-04-16T18:09:35.283 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4 2024-04-16T18:09:35.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXi35XGE4/bN5hl8DNFByeq8IPs= 2024-04-16T18:09:35.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXi35XGE4/bN5hl8DNFByeq8IPs=', 'Content-Length': '0'} 2024-04-16T18:09:35.286 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002768383e38e8a7a3-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.286 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.286 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.286 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.286 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:35.286 DEBUG:boto:Data: 2024-04-16T18:09:35.286 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.287 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.287 DEBUG:boto:Params: {} 2024-04-16T18:09:35.287 DEBUG:boto:Token: None 2024-04-16T18:09:35.287 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4 2024-04-16T18:09:35.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXi35XGE4/bN5hl8DNFByeq8IPs= 2024-04-16T18:09:35.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXi35XGE4/bN5hl8DNFByeq8IPs=', 'Content-Length': '0'} 2024-04-16T18:09:35.290 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b83b4b6498bb02b6-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:35.290 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.290 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.291 DEBUG:boto:Method: GET 2024-04-16T18:09:35.291 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:35.291 DEBUG:boto:Data: 2024-04-16T18:09:35.291 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.291 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.291 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.291 DEBUG:boto:Params: {} 2024-04-16T18:09:35.291 DEBUG:boto:Token: None 2024-04-16T18:09:35.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4 2024-04-16T18:09:35.291 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:figmtVxy/EOvihy2R3/XhOEgA7M= 2024-04-16T18:09:35.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:figmtVxy/EOvihy2R3/XhOEgA7M=', 'Content-Length': '0'} 2024-04-16T18:09:35.294 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001c571aed43f5afdf-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.338 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:35.338 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:35.339 DEBUG:boto:Method: GET 2024-04-16T18:09:35.339 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:35.339 DEBUG:boto:Data: 2024-04-16T18:09:35.339 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.339 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.339 DEBUG:boto:Params: {} 2024-04-16T18:09:35.339 DEBUG:boto:Token: None 2024-04-16T18:09:35.339 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/obj4 2024-04-16T18:09:35.339 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:figmtVxy/EOvihy2R3/XhOEgA7M= 2024-04-16T18:09:35.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:figmtVxy/EOvihy2R3/XhOEgA7M=', 'Content-Length': '0'} 2024-04-16T18:09:35.342 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a53233f6711cdf64-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.342 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.343 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.343 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.343 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.343 DEBUG:boto:Method: GET 2024-04-16T18:09:35.343 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:35.343 DEBUG:boto:Data: 2024-04-16T18:09:35.343 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.343 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.343 DEBUG:boto:Params: {} 2024-04-16T18:09:35.343 DEBUG:boto:Token: None 2024-04-16T18:09:35.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cade7a747d2192c-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.390 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.391 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:35.391 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:35.391 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.391 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:35.391 DEBUG:boto:Method: GET 2024-04-16T18:09:35.391 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:35.391 DEBUG:boto:Data: 2024-04-16T18:09:35.391 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.391 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.391 DEBUG:boto:Params: {} 2024-04-16T18:09:35.392 DEBUG:boto:Token: None 2024-04-16T18:09:35.392 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-38/ 2024-04-16T18:09:35.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw= 2024-04-16T18:09:35.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTBOlQ40EZsMfEPNC0R7JlZMWEw=', 'Content-Length': '0'} 2024-04-16T18:09:35.396 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000296df0aebbf18bbb-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.396 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.397 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-38 zones={test-zone2, test-zone1} 2024-04-16T18:09:35.397 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-39 2024-04-16T18:09:35.397 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:35.397 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:35.397 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-39 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:09:35.447 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:35.447 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:35.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:35.533+0000 7f1c8d073980 1 bucket sync caught up with source: 2024-04-16T18:09:35.605 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.29.6, , 00000000001.28.6, , 00000000001.94.6, , , 00000000001.25.6, , , ] 2024-04-16T18:09:35.605 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.29.6, , 00000000001.28.6, , 00000000001.94.6, , , 00000000001.25.6, , , ] 2024-04-16T18:09:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:35.533+0000 7f1c8d073980 0 bucket checkpoint complete 2024-04-16T18:09:35.606 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-39 2024-04-16T18:09:35.606 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:35.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:35.607 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-39 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:09:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:35.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:35.737+0000 7f9ced4c3980 1 bucket sync caught up with source: 2024-04-16T18:09:35.735 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.87.6, , 00000000001.373.6, , 00000000001.870.6, , , 00000000001.281.6, , , ] 2024-04-16T18:09:35.735 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000001.87.6, , 00000000001.373.6, , 00000000001.870.6, , , 00000000001.281.6, , , ] 2024-04-16T18:09:35.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:35.737+0000 7f9ced4c3980 0 bucket checkpoint complete 2024-04-16T18:09:35.768 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-39 zones={test-zone2, test-zone1} 2024-04-16T18:09:35.768 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.768 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.768 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.768 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:35.768 DEBUG:boto:Data: 2024-04-16T18:09:35.768 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.769 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.769 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.769 DEBUG:boto:Params: {} 2024-04-16T18:09:35.769 DEBUG:boto:Token: None 2024-04-16T18:09:35.769 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/ 2024-04-16T18:09:35.769 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RzIx8T1ig32F7YueHFYsp8ioMIU= 2024-04-16T18:09:35.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RzIx8T1ig32F7YueHFYsp8ioMIU=', 'Content-Length': '0'} 2024-04-16T18:09:35.773 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000062609a8f5f05877c-00661ebedf-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.773 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.773 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.773 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.773 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:35.773 DEBUG:boto:Data: 2024-04-16T18:09:35.773 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.774 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.774 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.774 DEBUG:boto:Params: {} 2024-04-16T18:09:35.774 DEBUG:boto:Token: None 2024-04-16T18:09:35.774 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/ 2024-04-16T18:09:35.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RzIx8T1ig32F7YueHFYsp8ioMIU= 2024-04-16T18:09:35.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RzIx8T1ig32F7YueHFYsp8ioMIU=', 'Content-Length': '0'} 2024-04-16T18:09:35.777 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c9eb214730025320-00661ebedf-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.777 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:09:35.777 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.778 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.778 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.778 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.778 DEBUG:boto:Method: GET 2024-04-16T18:09:35.778 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.778 DEBUG:boto:Data: 2024-04-16T18:09:35.778 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.778 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.778 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.778 DEBUG:boto:Params: {} 2024-04-16T18:09:35.779 DEBUG:boto:Token: None 2024-04-16T18:09:35.779 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/?versions 2024-04-16T18:09:35.779 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os= 2024-04-16T18:09:35.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os=', 'Content-Length': '0'} 2024-04-16T18:09:35.782 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000164f1ff1f0386a7c-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.783 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.783 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:09:35.783 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:09:35.783 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:09:35.783 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:09:35.783 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:09:35.784 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.784 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.784 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.784 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.784 DEBUG:boto:Method: GET 2024-04-16T18:09:35.784 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.784 DEBUG:boto:Data: 2024-04-16T18:09:35.784 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.784 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.784 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.784 DEBUG:boto:Params: {} 2024-04-16T18:09:35.784 DEBUG:boto:Token: None 2024-04-16T18:09:35.785 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/?versions 2024-04-16T18:09:35.785 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os= 2024-04-16T18:09:35.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os=', 'Content-Length': '0'} 2024-04-16T18:09:35.788 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000832eb10d501b26db-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.788 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.788 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:09:35.789 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:09:35.789 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:09:35.789 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:09:35.789 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.789 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.789 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.789 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.789 DEBUG:boto:Method: GET 2024-04-16T18:09:35.789 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.789 DEBUG:boto:Data: 2024-04-16T18:09:35.789 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.790 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.790 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.790 DEBUG:boto:Params: {} 2024-04-16T18:09:35.790 DEBUG:boto:Token: None 2024-04-16T18:09:35.790 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/?versions 2024-04-16T18:09:35.790 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os= 2024-04-16T18:09:35.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os=', 'Content-Length': '0'} 2024-04-16T18:09:35.794 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ab4263d67a5e9e9-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.795 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.795 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.795 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.795 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.795 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.795 DEBUG:boto:Method: GET 2024-04-16T18:09:35.795 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:09:35.796 DEBUG:boto:Data: 2024-04-16T18:09:35.796 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.796 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.796 DEBUG:boto:Params: {} 2024-04-16T18:09:35.796 DEBUG:boto:Token: None 2024-04-16T18:09:35.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/?versions 2024-04-16T18:09:35.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os= 2024-04-16T18:09:35.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r7BnAq7rIl7GfK9CxI1nbWEw8Os=', 'Content-Length': '0'} 2024-04-16T18:09:35.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017c64b8ed7fc5e5f-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.799 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:35.800 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.800 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.800 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.800 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.800 DEBUG:boto:Method: GET 2024-04-16T18:09:35.800 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.800 DEBUG:boto:Data: 2024-04-16T18:09:35.800 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.800 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.801 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.801 DEBUG:boto:Params: {} 2024-04-16T18:09:35.801 DEBUG:boto:Token: None 2024-04-16T18:09:35.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g= 2024-04-16T18:09:35.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g=', 'Content-Length': '0'} 2024-04-16T18:09:35.804 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000370464cf26dfe6f-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.846 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.846 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.847 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.847 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.847 DEBUG:boto:Method: GET 2024-04-16T18:09:35.847 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.847 DEBUG:boto:Data: 2024-04-16T18:09:35.847 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.847 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.847 DEBUG:boto:Params: {} 2024-04-16T18:09:35.847 DEBUG:boto:Token: None 2024-04-16T18:09:35.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g= 2024-04-16T18:09:35.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g=', 'Content-Length': '0'} 2024-04-16T18:09:35.850 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000010f47dc78309c535-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.851 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.851 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.851 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.851 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.851 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.851 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.851 DEBUG:boto:Data: 2024-04-16T18:09:35.851 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.852 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.852 DEBUG:boto:Params: {} 2024-04-16T18:09:35.852 DEBUG:boto:Token: None 2024-04-16T18:09:35.852 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NvcS6SxjHnGFyHWQYnN+1RuSodo= 2024-04-16T18:09:35.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NvcS6SxjHnGFyHWQYnN+1RuSodo=', 'Content-Length': '0'} 2024-04-16T18:09:35.855 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002a0534ce4f875bf1-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.855 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.855 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.855 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.855 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.855 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.855 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.855 DEBUG:boto:Data: 2024-04-16T18:09:35.855 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.856 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.856 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.856 DEBUG:boto:Params: {} 2024-04-16T18:09:35.856 DEBUG:boto:Token: None 2024-04-16T18:09:35.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NvcS6SxjHnGFyHWQYnN+1RuSodo= 2024-04-16T18:09:35.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NvcS6SxjHnGFyHWQYnN+1RuSodo=', 'Content-Length': '0'} 2024-04-16T18:09:35.859 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000841bcd80ea347bb1-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.859 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:35.859 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.859 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.859 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.859 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.860 DEBUG:boto:Method: GET 2024-04-16T18:09:35.860 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.860 DEBUG:boto:Data: 2024-04-16T18:09:35.860 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.860 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.860 DEBUG:boto:Params: {} 2024-04-16T18:09:35.860 DEBUG:boto:Token: None 2024-04-16T18:09:35.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g= 2024-04-16T18:09:35.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g=', 'Content-Length': '0'} 2024-04-16T18:09:35.863 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000514dc67c4d0e6f7b-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.864 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.864 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.864 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.864 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.864 DEBUG:boto:Method: GET 2024-04-16T18:09:35.864 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.864 DEBUG:boto:Data: 2024-04-16T18:09:35.864 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.864 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.864 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.864 DEBUG:boto:Params: {} 2024-04-16T18:09:35.865 DEBUG:boto:Token: None 2024-04-16T18:09:35.865 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:09:35.865 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g= 2024-04-16T18:09:35.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fxNSuvTi9S0SIbBA8Q6S7/u2p8g=', 'Content-Length': '0'} 2024-04-16T18:09:35.868 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bd34d45539eff07b-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.868 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.868 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.868 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.868 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.868 DEBUG:boto:Method: GET 2024-04-16T18:09:35.869 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:09:35.869 DEBUG:boto:Data: 2024-04-16T18:09:35.869 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.869 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.869 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.869 DEBUG:boto:Params: {} 2024-04-16T18:09:35.869 DEBUG:boto:Token: None 2024-04-16T18:09:35.869 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/ 2024-04-16T18:09:35.869 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw= 2024-04-16T18:09:35.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw=', 'Content-Length': '0'} 2024-04-16T18:09:35.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c40702e5c085a066-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.914 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.915 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.915 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.915 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.915 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.915 DEBUG:boto:Method: GET 2024-04-16T18:09:35.915 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:09:35.915 DEBUG:boto:Data: 2024-04-16T18:09:35.915 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.915 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.915 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.915 DEBUG:boto:Params: {} 2024-04-16T18:09:35.916 DEBUG:boto:Token: None 2024-04-16T18:09:35.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/ 2024-04-16T18:09:35.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw= 2024-04-16T18:09:35.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw=', 'Content-Length': '0'} 2024-04-16T18:09:35.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d93c37671bc569bc-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.919 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.920 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.920 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.920 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.920 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:09:35.920 DEBUG:boto:Data: 2024-04-16T18:09:35.920 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.920 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.920 DEBUG:boto:Params: {} 2024-04-16T18:09:35.921 DEBUG:boto:Token: None 2024-04-16T18:09:35.921 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1 2024-04-16T18:09:35.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjzTb6ZsZvSSzJ1QF7xcG+IMp84= 2024-04-16T18:09:35.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjzTb6ZsZvSSzJ1QF7xcG+IMp84=', 'Content-Length': '0'} 2024-04-16T18:09:35.923 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f1268e366feec8bf-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.924 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.924 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.924 DEBUG:boto:Method: HEAD 2024-04-16T18:09:35.924 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:09:35.924 DEBUG:boto:Data: 2024-04-16T18:09:35.924 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.924 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.924 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.924 DEBUG:boto:Params: {} 2024-04-16T18:09:35.924 DEBUG:boto:Token: None 2024-04-16T18:09:35.925 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1 2024-04-16T18:09:35.925 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fjzTb6ZsZvSSzJ1QF7xcG+IMp84= 2024-04-16T18:09:35.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fjzTb6ZsZvSSzJ1QF7xcG+IMp84=', 'Content-Length': '0'} 2024-04-16T18:09:35.927 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000732c1d59ddc74865-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:35.928 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.928 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.928 DEBUG:boto:Method: GET 2024-04-16T18:09:35.928 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:09:35.928 DEBUG:boto:Data: 2024-04-16T18:09:35.928 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.928 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.928 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.928 DEBUG:boto:Params: {} 2024-04-16T18:09:35.929 DEBUG:boto:Token: None 2024-04-16T18:09:35.929 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1 2024-04-16T18:09:35.929 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mh879YRNZHgSDhr4oKSzZL7S36s= 2024-04-16T18:09:35.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mh879YRNZHgSDhr4oKSzZL7S36s=', 'Content-Length': '0'} 2024-04-16T18:09:35.932 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001ad42fbed5c4e761-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.932 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:09:35.932 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:09:35.932 DEBUG:boto:Method: GET 2024-04-16T18:09:35.932 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:09:35.932 DEBUG:boto:Data: 2024-04-16T18:09:35.932 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.933 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.933 DEBUG:boto:Params: {} 2024-04-16T18:09:35.933 DEBUG:boto:Token: None 2024-04-16T18:09:35.933 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj1 2024-04-16T18:09:35.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mh879YRNZHgSDhr4oKSzZL7S36s= 2024-04-16T18:09:35.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mh879YRNZHgSDhr4oKSzZL7S36s=', 'Content-Length': '0'} 2024-04-16T18:09:35.936 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000021696b229a4e93e-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.978 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.979 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.979 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.979 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.979 DEBUG:boto:Method: GET 2024-04-16T18:09:35.979 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:09:35.979 DEBUG:boto:Data: 2024-04-16T18:09:35.979 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.979 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.979 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.979 DEBUG:boto:Params: {} 2024-04-16T18:09:35.979 DEBUG:boto:Token: None 2024-04-16T18:09:35.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/ 2024-04-16T18:09:35.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw= 2024-04-16T18:09:35.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw=', 'Content-Length': '0'} 2024-04-16T18:09:35.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ea6fd8a50049138-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.984 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.984 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:35.984 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:35.984 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.984 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:09:35.984 DEBUG:boto:Method: GET 2024-04-16T18:09:35.984 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:09:35.984 DEBUG:boto:Data: 2024-04-16T18:09:35.984 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.985 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:35.985 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.985 DEBUG:boto:Params: {} 2024-04-16T18:09:35.985 DEBUG:boto:Token: None 2024-04-16T18:09:35.985 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/ 2024-04-16T18:09:35.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw= 2024-04-16T18:09:35.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pIoq6Qp4oZ9lvG2MimK6lrMFmbw=', 'Content-Length': '0'} 2024-04-16T18:09:35.988 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d6f9cd79785dfe9e-00661ebedf-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:35.988 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:35.989 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:35.989 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:35.989 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:35.989 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:35.989 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:35.989 DEBUG:boto:Method: GET 2024-04-16T18:09:35.989 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:09:35.989 DEBUG:boto:Data: 2024-04-16T18:09:35.989 DEBUG:boto:Headers: {} 2024-04-16T18:09:35.990 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:35.990 DEBUG:boto:Port: 8000 2024-04-16T18:09:35.990 DEBUG:boto:Params: {} 2024-04-16T18:09:35.990 DEBUG:boto:Token: None 2024-04-16T18:09:35.990 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:35 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:09:35.990 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q005GI5lfP0BXLNpl++Cmkl13w4= 2024-04-16T18:09:35.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q005GI5lfP0BXLNpl++Cmkl13w4=', 'Content-Length': '0'} 2024-04-16T18:09:35.993 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f35cef0fe7361663-00661ebedf-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.034 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.034 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.035 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.035 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.035 DEBUG:boto:Method: GET 2024-04-16T18:09:36.035 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.035 DEBUG:boto:Data: 2024-04-16T18:09:36.035 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.035 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.035 DEBUG:boto:Params: {} 2024-04-16T18:09:36.035 DEBUG:boto:Token: None 2024-04-16T18:09:36.035 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywWBPXhrPmtYsMUrBcXqaNO1p+o= 2024-04-16T18:09:36.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywWBPXhrPmtYsMUrBcXqaNO1p+o=', 'Content-Length': '0'} 2024-04-16T18:09:36.038 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000008a37c2ea7cdfb73-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.082 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.083 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.083 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.083 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.083 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.083 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.083 DEBUG:boto:Data: 2024-04-16T18:09:36.083 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.083 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.083 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.083 DEBUG:boto:Params: {} 2024-04-16T18:09:36.083 DEBUG:boto:Token: None 2024-04-16T18:09:36.083 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TUpVZzSKSmDgWfuywwmfFR9AVw8= 2024-04-16T18:09:36.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TUpVZzSKSmDgWfuywwmfFR9AVw8=', 'Content-Length': '0'} 2024-04-16T18:09:36.086 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000024aa37f933d941e1-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.087 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.087 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.087 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.087 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.087 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.087 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.087 DEBUG:boto:Data: 2024-04-16T18:09:36.087 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.087 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.087 DEBUG:boto:Params: {} 2024-04-16T18:09:36.088 DEBUG:boto:Token: None 2024-04-16T18:09:36.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TUpVZzSKSmDgWfuywwmfFR9AVw8= 2024-04-16T18:09:36.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TUpVZzSKSmDgWfuywwmfFR9AVw8=', 'Content-Length': '0'} 2024-04-16T18:09:36.091 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bf44886c5ba1c63e-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:36.091 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.091 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.091 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.091 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.092 DEBUG:boto:Method: GET 2024-04-16T18:09:36.092 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.092 DEBUG:boto:Data: 2024-04-16T18:09:36.092 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.092 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.092 DEBUG:boto:Params: {} 2024-04-16T18:09:36.092 DEBUG:boto:Token: None 2024-04-16T18:09:36.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywWBPXhrPmtYsMUrBcXqaNO1p+o= 2024-04-16T18:09:36.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywWBPXhrPmtYsMUrBcXqaNO1p+o=', 'Content-Length': '0'} 2024-04-16T18:09:36.095 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000364bd85981a3115c-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.096 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.096 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.096 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.096 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.096 DEBUG:boto:Method: GET 2024-04-16T18:09:36.096 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.096 DEBUG:boto:Data: 2024-04-16T18:09:36.096 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.096 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.096 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.097 DEBUG:boto:Params: {} 2024-04-16T18:09:36.097 DEBUG:boto:Token: None 2024-04-16T18:09:36.097 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:09:36.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywWBPXhrPmtYsMUrBcXqaNO1p+o= 2024-04-16T18:09:36.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywWBPXhrPmtYsMUrBcXqaNO1p+o=', 'Content-Length': '0'} 2024-04-16T18:09:36.100 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000072117b566b2ca7c0-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.142 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.142 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.143 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.143 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.143 DEBUG:boto:Method: GET 2024-04-16T18:09:36.143 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:09:36.143 DEBUG:boto:Data: 2024-04-16T18:09:36.143 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.143 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.143 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.143 DEBUG:boto:Params: {} 2024-04-16T18:09:36.143 DEBUG:boto:Token: None 2024-04-16T18:09:36.143 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.147 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d38037ec3fb1994c-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.148 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.148 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.148 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.148 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.148 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.148 DEBUG:boto:Method: GET 2024-04-16T18:09:36.148 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:09:36.149 DEBUG:boto:Data: 2024-04-16T18:09:36.149 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.149 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.149 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.149 DEBUG:boto:Params: {} 2024-04-16T18:09:36.149 DEBUG:boto:Token: None 2024-04-16T18:09:36.149 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.149 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000684cd5c2779a04f3-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.153 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.153 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.153 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.153 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.153 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:09:36.153 DEBUG:boto:Data: 2024-04-16T18:09:36.154 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.154 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.154 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.154 DEBUG:boto:Params: {} 2024-04-16T18:09:36.154 DEBUG:boto:Token: None 2024-04-16T18:09:36.154 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2 2024-04-16T18:09:36.154 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U+IYp7f+H34GfZrlZXopDLaDdJ4= 2024-04-16T18:09:36.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U+IYp7f+H34GfZrlZXopDLaDdJ4=', 'Content-Length': '0'} 2024-04-16T18:09:36.157 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a06bb95d7f011f44-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.157 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.157 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.157 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.157 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:09:36.158 DEBUG:boto:Data: 2024-04-16T18:09:36.158 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.158 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.158 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.158 DEBUG:boto:Params: {} 2024-04-16T18:09:36.158 DEBUG:boto:Token: None 2024-04-16T18:09:36.158 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2 2024-04-16T18:09:36.158 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U+IYp7f+H34GfZrlZXopDLaDdJ4= 2024-04-16T18:09:36.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U+IYp7f+H34GfZrlZXopDLaDdJ4=', 'Content-Length': '0'} 2024-04-16T18:09:36.161 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bb08af2482251b16-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.161 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:36.161 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.162 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.162 DEBUG:boto:Method: GET 2024-04-16T18:09:36.162 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:09:36.162 DEBUG:boto:Data: 2024-04-16T18:09:36.162 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.162 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.162 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.162 DEBUG:boto:Params: {} 2024-04-16T18:09:36.162 DEBUG:boto:Token: None 2024-04-16T18:09:36.162 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2 2024-04-16T18:09:36.163 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RmFfriFaje4OZBufD5Ao1JAT+Xw= 2024-04-16T18:09:36.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RmFfriFaje4OZBufD5Ao1JAT+Xw=', 'Content-Length': '0'} 2024-04-16T18:09:36.165 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007db35f3f9f9911ae-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.166 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:09:36.166 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:09:36.166 DEBUG:boto:Method: GET 2024-04-16T18:09:36.166 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:09:36.166 DEBUG:boto:Data: 2024-04-16T18:09:36.166 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.166 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.166 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.166 DEBUG:boto:Params: {} 2024-04-16T18:09:36.167 DEBUG:boto:Token: None 2024-04-16T18:09:36.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj2 2024-04-16T18:09:36.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RmFfriFaje4OZBufD5Ao1JAT+Xw= 2024-04-16T18:09:36.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RmFfriFaje4OZBufD5Ao1JAT+Xw=', 'Content-Length': '0'} 2024-04-16T18:09:36.170 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000009b1a9eb374eaa85-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.170 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.170 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.171 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.171 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.171 DEBUG:boto:Method: GET 2024-04-16T18:09:36.171 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:09:36.171 DEBUG:boto:Data: 2024-04-16T18:09:36.171 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.171 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.171 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.171 DEBUG:boto:Params: {} 2024-04-16T18:09:36.171 DEBUG:boto:Token: None 2024-04-16T18:09:36.171 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000260d16a3b260aba-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.175 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.176 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.176 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.176 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.176 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:09:36.176 DEBUG:boto:Method: GET 2024-04-16T18:09:36.176 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:09:36.176 DEBUG:boto:Data: 2024-04-16T18:09:36.176 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.176 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.177 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.177 DEBUG:boto:Params: {} 2024-04-16T18:09:36.177 DEBUG:boto:Token: None 2024-04-16T18:09:36.177 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a433967ddcf1c3d-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.181 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.181 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:36.182 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.182 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.182 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.182 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.182 DEBUG:boto:Method: GET 2024-04-16T18:09:36.182 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.182 DEBUG:boto:Data: 2024-04-16T18:09:36.182 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.182 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.182 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.182 DEBUG:boto:Params: {} 2024-04-16T18:09:36.183 DEBUG:boto:Token: None 2024-04-16T18:09:36.183 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.183 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE= 2024-04-16T18:09:36.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE=', 'Content-Length': '0'} 2024-04-16T18:09:36.186 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000452ca4aed72e337d-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.226 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.226 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.226 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.227 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.227 DEBUG:boto:Method: GET 2024-04-16T18:09:36.227 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.227 DEBUG:boto:Data: 2024-04-16T18:09:36.227 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.227 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.227 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.227 DEBUG:boto:Params: {} 2024-04-16T18:09:36.227 DEBUG:boto:Token: None 2024-04-16T18:09:36.227 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE= 2024-04-16T18:09:36.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE=', 'Content-Length': '0'} 2024-04-16T18:09:36.231 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000021f0e55f15029092-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.231 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.231 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.231 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.231 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.231 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.232 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.232 DEBUG:boto:Data: 2024-04-16T18:09:36.232 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.232 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.232 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.232 DEBUG:boto:Params: {} 2024-04-16T18:09:36.232 DEBUG:boto:Token: None 2024-04-16T18:09:36.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WOewS+8AnTHxSu6nUZL7D0nJc6M= 2024-04-16T18:09:36.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WOewS+8AnTHxSu6nUZL7D0nJc6M=', 'Content-Length': '0'} 2024-04-16T18:09:36.235 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000fb24521d2e88a8c5-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.235 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.235 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.235 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.235 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.236 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.236 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.236 DEBUG:boto:Data: 2024-04-16T18:09:36.236 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.236 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.236 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.236 DEBUG:boto:Params: {} 2024-04-16T18:09:36.236 DEBUG:boto:Token: None 2024-04-16T18:09:36.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WOewS+8AnTHxSu6nUZL7D0nJc6M= 2024-04-16T18:09:36.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WOewS+8AnTHxSu6nUZL7D0nJc6M=', 'Content-Length': '0'} 2024-04-16T18:09:36.239 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001e36b8c2bc5d9202-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:36.239 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.240 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.240 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.240 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.240 DEBUG:boto:Method: GET 2024-04-16T18:09:36.240 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.240 DEBUG:boto:Data: 2024-04-16T18:09:36.240 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.240 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.240 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.240 DEBUG:boto:Params: {} 2024-04-16T18:09:36.240 DEBUG:boto:Token: None 2024-04-16T18:09:36.241 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE= 2024-04-16T18:09:36.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE=', 'Content-Length': '0'} 2024-04-16T18:09:36.243 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ce5995d77d95faf1-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.244 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.244 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.244 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.244 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.244 DEBUG:boto:Method: GET 2024-04-16T18:09:36.244 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.244 DEBUG:boto:Data: 2024-04-16T18:09:36.244 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.244 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.244 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.244 DEBUG:boto:Params: {} 2024-04-16T18:09:36.245 DEBUG:boto:Token: None 2024-04-16T18:09:36.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:09:36.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE= 2024-04-16T18:09:36.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GgOsn1bwU9V0Pk6Y3Rz/6tvNUUE=', 'Content-Length': '0'} 2024-04-16T18:09:36.248 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f04e9b2fb025f2db-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.290 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.291 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.291 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.291 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.291 DEBUG:boto:Method: GET 2024-04-16T18:09:36.291 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:09:36.291 DEBUG:boto:Data: 2024-04-16T18:09:36.291 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.291 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.291 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.291 DEBUG:boto:Params: {} 2024-04-16T18:09:36.291 DEBUG:boto:Token: None 2024-04-16T18:09:36.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.295 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000deefed3ab7c33938-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.295 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.296 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.296 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.296 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.296 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.296 DEBUG:boto:Method: GET 2024-04-16T18:09:36.296 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:09:36.296 DEBUG:boto:Data: 2024-04-16T18:09:36.296 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.296 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.297 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.297 DEBUG:boto:Params: {} 2024-04-16T18:09:36.297 DEBUG:boto:Token: None 2024-04-16T18:09:36.297 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.297 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2af11e53cc767e8-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.300 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.301 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.301 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.301 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.301 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:09:36.301 DEBUG:boto:Data: 2024-04-16T18:09:36.301 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.301 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.301 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.301 DEBUG:boto:Params: {} 2024-04-16T18:09:36.302 DEBUG:boto:Token: None 2024-04-16T18:09:36.302 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3 2024-04-16T18:09:36.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vjc8bidRdKwPepdd6Grt8hG+qzE= 2024-04-16T18:09:36.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vjc8bidRdKwPepdd6Grt8hG+qzE=', 'Content-Length': '0'} 2024-04-16T18:09:36.304 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004e3c646804d794e8-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.305 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.305 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.305 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.305 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:09:36.305 DEBUG:boto:Data: 2024-04-16T18:09:36.305 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.305 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.305 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.305 DEBUG:boto:Params: {} 2024-04-16T18:09:36.305 DEBUG:boto:Token: None 2024-04-16T18:09:36.305 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3 2024-04-16T18:09:36.306 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vjc8bidRdKwPepdd6Grt8hG+qzE= 2024-04-16T18:09:36.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vjc8bidRdKwPepdd6Grt8hG+qzE=', 'Content-Length': '0'} 2024-04-16T18:09:36.308 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c21c8e9db93f299f-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.309 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:36.309 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.309 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.309 DEBUG:boto:Method: GET 2024-04-16T18:09:36.309 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:09:36.309 DEBUG:boto:Data: 2024-04-16T18:09:36.309 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.309 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.309 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.309 DEBUG:boto:Params: {} 2024-04-16T18:09:36.310 DEBUG:boto:Token: None 2024-04-16T18:09:36.310 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3 2024-04-16T18:09:36.310 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OVqVXs6MKSGvpRj6kZSqA0cjdHM= 2024-04-16T18:09:36.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OVqVXs6MKSGvpRj6kZSqA0cjdHM=', 'Content-Length': '0'} 2024-04-16T18:09:36.312 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ccdf59e78394d60d-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.354 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:09:36.355 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:09:36.355 DEBUG:boto:Method: GET 2024-04-16T18:09:36.355 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:09:36.355 DEBUG:boto:Data: 2024-04-16T18:09:36.355 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.355 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.355 DEBUG:boto:Params: {} 2024-04-16T18:09:36.355 DEBUG:boto:Token: None 2024-04-16T18:09:36.355 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj3 2024-04-16T18:09:36.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OVqVXs6MKSGvpRj6kZSqA0cjdHM= 2024-04-16T18:09:36.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OVqVXs6MKSGvpRj6kZSqA0cjdHM=', 'Content-Length': '0'} 2024-04-16T18:09:36.358 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000094a833620aec645a-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.359 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.359 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.359 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.359 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.359 DEBUG:boto:Method: GET 2024-04-16T18:09:36.359 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:09:36.359 DEBUG:boto:Data: 2024-04-16T18:09:36.359 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.359 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.360 DEBUG:boto:Params: {} 2024-04-16T18:09:36.360 DEBUG:boto:Token: None 2024-04-16T18:09:36.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f371cd819b33ce1-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.364 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.364 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.364 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.365 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.365 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:09:36.365 DEBUG:boto:Method: GET 2024-04-16T18:09:36.365 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:09:36.365 DEBUG:boto:Data: 2024-04-16T18:09:36.365 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.365 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.365 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.365 DEBUG:boto:Params: {} 2024-04-16T18:09:36.365 DEBUG:boto:Token: None 2024-04-16T18:09:36.365 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.366 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.369 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a34619f5918ed621-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.410 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.411 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:36.411 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.411 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.411 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.411 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.411 DEBUG:boto:Method: GET 2024-04-16T18:09:36.411 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.411 DEBUG:boto:Data: 2024-04-16T18:09:36.411 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.411 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.411 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.412 DEBUG:boto:Params: {} 2024-04-16T18:09:36.412 DEBUG:boto:Token: None 2024-04-16T18:09:36.412 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU= 2024-04-16T18:09:36.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU=', 'Content-Length': '0'} 2024-04-16T18:09:36.415 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000019518e518d6733ad-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.415 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.415 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.415 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.416 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.416 DEBUG:boto:Method: GET 2024-04-16T18:09:36.416 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.416 DEBUG:boto:Data: 2024-04-16T18:09:36.416 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.416 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.416 DEBUG:boto:Params: {} 2024-04-16T18:09:36.416 DEBUG:boto:Token: None 2024-04-16T18:09:36.416 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU= 2024-04-16T18:09:36.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU=', 'Content-Length': '0'} 2024-04-16T18:09:36.419 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b08c2bdf42facf63-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.420 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.420 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.420 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.420 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.420 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.420 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.420 DEBUG:boto:Data: 2024-04-16T18:09:36.420 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.420 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.421 DEBUG:boto:Params: {} 2024-04-16T18:09:36.421 DEBUG:boto:Token: None 2024-04-16T18:09:36.421 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.421 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3y1A+lKIgtFccr+Io0vSmlgaVq8= 2024-04-16T18:09:36.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3y1A+lKIgtFccr+Io0vSmlgaVq8=', 'Content-Length': '0'} 2024-04-16T18:09:36.424 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001adf6f139a6db4ff-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.424 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.424 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.425 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.425 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.425 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.425 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.425 DEBUG:boto:Data: 2024-04-16T18:09:36.425 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.425 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.425 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.425 DEBUG:boto:Params: {} 2024-04-16T18:09:36.425 DEBUG:boto:Token: None 2024-04-16T18:09:36.425 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.426 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3y1A+lKIgtFccr+Io0vSmlgaVq8= 2024-04-16T18:09:36.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3y1A+lKIgtFccr+Io0vSmlgaVq8=', 'Content-Length': '0'} 2024-04-16T18:09:36.428 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c9524b89ce74913c-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:36.429 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.429 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.429 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.429 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.429 DEBUG:boto:Method: GET 2024-04-16T18:09:36.429 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.429 DEBUG:boto:Data: 2024-04-16T18:09:36.429 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.429 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.429 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.430 DEBUG:boto:Params: {} 2024-04-16T18:09:36.430 DEBUG:boto:Token: None 2024-04-16T18:09:36.430 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU= 2024-04-16T18:09:36.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU=', 'Content-Length': '0'} 2024-04-16T18:09:36.433 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003c2d01846cc9fed2-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.474 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.475 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.475 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.475 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.475 DEBUG:boto:Method: GET 2024-04-16T18:09:36.475 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.475 DEBUG:boto:Data: 2024-04-16T18:09:36.475 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.475 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.475 DEBUG:boto:Params: {} 2024-04-16T18:09:36.475 DEBUG:boto:Token: None 2024-04-16T18:09:36.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:09:36.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU= 2024-04-16T18:09:36.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o2rwohN+G0qpCNjtuIgYV0OI/IU=', 'Content-Length': '0'} 2024-04-16T18:09:36.479 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000931ababfb34c5c49-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.479 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.479 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.479 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.479 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.479 DEBUG:boto:Method: GET 2024-04-16T18:09:36.480 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:09:36.480 DEBUG:boto:Data: 2024-04-16T18:09:36.480 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.480 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.480 DEBUG:boto:Params: {} 2024-04-16T18:09:36.480 DEBUG:boto:Token: None 2024-04-16T18:09:36.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f167328b96d0f74-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.484 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.485 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.485 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.485 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.485 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.485 DEBUG:boto:Method: GET 2024-04-16T18:09:36.485 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:09:36.485 DEBUG:boto:Data: 2024-04-16T18:09:36.485 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.485 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.485 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.485 DEBUG:boto:Params: {} 2024-04-16T18:09:36.486 DEBUG:boto:Token: None 2024-04-16T18:09:36.486 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.486 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000690d06c7083d7e85-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.530 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.531 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.531 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.531 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.531 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:09:36.531 DEBUG:boto:Data: 2024-04-16T18:09:36.531 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.531 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.531 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.531 DEBUG:boto:Params: {} 2024-04-16T18:09:36.531 DEBUG:boto:Token: None 2024-04-16T18:09:36.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4 2024-04-16T18:09:36.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ByO+PyjFj8Pf5Z6C33EHSrz0joM= 2024-04-16T18:09:36.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ByO+PyjFj8Pf5Z6C33EHSrz0joM=', 'Content-Length': '0'} 2024-04-16T18:09:36.535 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000094ca2c16f702e7cf-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.535 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.535 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.535 DEBUG:boto:Method: HEAD 2024-04-16T18:09:36.535 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:09:36.535 DEBUG:boto:Data: 2024-04-16T18:09:36.536 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.536 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.536 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.536 DEBUG:boto:Params: {} 2024-04-16T18:09:36.536 DEBUG:boto:Token: None 2024-04-16T18:09:36.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4 2024-04-16T18:09:36.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ByO+PyjFj8Pf5Z6C33EHSrz0joM= 2024-04-16T18:09:36.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ByO+PyjFj8Pf5Z6C33EHSrz0joM=', 'Content-Length': '0'} 2024-04-16T18:09:36.539 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000377f63b43ad8ebe6-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.540 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:36.540 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.540 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.540 DEBUG:boto:Method: GET 2024-04-16T18:09:36.540 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:09:36.540 DEBUG:boto:Data: 2024-04-16T18:09:36.540 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.540 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.540 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.541 DEBUG:boto:Params: {} 2024-04-16T18:09:36.541 DEBUG:boto:Token: None 2024-04-16T18:09:36.541 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4 2024-04-16T18:09:36.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x2x+H2Ua/ZiR9W9Hlx6Sevzj2Uw= 2024-04-16T18:09:36.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x2x+H2Ua/ZiR9W9Hlx6Sevzj2Uw=', 'Content-Length': '0'} 2024-04-16T18:09:36.544 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007b5e9e28144a37ff-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.544 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:09:36.545 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:09:36.545 DEBUG:boto:Method: GET 2024-04-16T18:09:36.545 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:09:36.545 DEBUG:boto:Data: 2024-04-16T18:09:36.545 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.545 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.545 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.545 DEBUG:boto:Params: {} 2024-04-16T18:09:36.545 DEBUG:boto:Token: None 2024-04-16T18:09:36.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/obj4 2024-04-16T18:09:36.546 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x2x+H2Ua/ZiR9W9Hlx6Sevzj2Uw= 2024-04-16T18:09:36.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x2x+H2Ua/ZiR9W9Hlx6Sevzj2Uw=', 'Content-Length': '0'} 2024-04-16T18:09:36.549 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000074e886aa4821ecf2-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.549 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.549 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.549 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.550 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.550 DEBUG:boto:Method: GET 2024-04-16T18:09:36.550 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:09:36.550 DEBUG:boto:Data: 2024-04-16T18:09:36.550 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.550 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:36.550 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.550 DEBUG:boto:Params: {} 2024-04-16T18:09:36.550 DEBUG:boto:Token: None 2024-04-16T18:09:36.550 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.551 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.554 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000894f064da113d349-00661ebee0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.594 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.595 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:36.595 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:36.595 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.595 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:09:36.595 DEBUG:boto:Method: GET 2024-04-16T18:09:36.595 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:09:36.595 DEBUG:boto:Data: 2024-04-16T18:09:36.595 DEBUG:boto:Headers: {} 2024-04-16T18:09:36.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:36.595 DEBUG:boto:Port: 8000 2024-04-16T18:09:36.596 DEBUG:boto:Params: {} 2024-04-16T18:09:36.596 DEBUG:boto:Token: None 2024-04-16T18:09:36.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:36 GMT /lybtug-39/ 2024-04-16T18:09:36.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo= 2024-04-16T18:09:36.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YRq5+DVnkwA9KdLL4aelG3CwPmo=', 'Content-Length': '0'} 2024-04-16T18:09:36.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eeca065e67abf9e3-00661ebee0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:36.600 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:36.600 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-39 zones={test-zone2, test-zone1} 2024-04-16T18:09:36.600 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'lybtug-38', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:36.600 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'lybtug-38', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:36.601 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket lybtug-38 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:09:36.644 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:36.644 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:36.792 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'lybtug-38', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:36.792 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'lybtug-38', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:36.792 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket lybtug-38 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:09:36.874 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:36.874 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:37.023 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'lybtug-39', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:37.024 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'lybtug-39', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:37.024 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket lybtug-39 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:09:37.110 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:37.110 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:37.259 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'lybtug-39', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:37.259 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'lybtug-39', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:09:37.259 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket lybtug-39 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:09:37.343 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:37.343 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:37.493 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:37.493 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:37.493 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.493 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:37.493 DEBUG:boto:Data: 2024-04-16T18:09:37.493 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.493 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.493 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.493 DEBUG:boto:Params: {} 2024-04-16T18:09:37.493 DEBUG:boto:Token: None 2024-04-16T18:09:37.494 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/ 2024-04-16T18:09:37.494 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w= 2024-04-16T18:09:37.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w=', 'Content-Length': '0'} 2024-04-16T18:09:37.497 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a3bd03178d3aa06f-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.497 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:37.497 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:37.497 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.497 DEBUG:boto:Path: /lybtug-38/obj5 2024-04-16T18:09:37.497 DEBUG:boto:Data: 2024-04-16T18:09:37.497 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.498 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.498 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.498 DEBUG:boto:Params: {} 2024-04-16T18:09:37.498 DEBUG:boto:Token: None 2024-04-16T18:09:37.498 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/obj5 2024-04-16T18:09:37.498 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s4NWBPp6dzwk6rOCFgRxrf5hBRw= 2024-04-16T18:09:37.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s4NWBPp6dzwk6rOCFgRxrf5hBRw='} 2024-04-16T18:09:37.504 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a5468ee7e1887c22-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.505 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:37.505 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:37.505 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.505 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:37.505 DEBUG:boto:Data: 2024-04-16T18:09:37.505 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.505 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.505 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.505 DEBUG:boto:Params: {} 2024-04-16T18:09:37.505 DEBUG:boto:Token: None 2024-04-16T18:09:37.506 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/ 2024-04-16T18:09:37.506 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w= 2024-04-16T18:09:37.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w=', 'Content-Length': '0'} 2024-04-16T18:09:37.508 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003bfc572f778f430e-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.508 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:37.508 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:37.508 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.508 DEBUG:boto:Path: /lybtug-38/obj6 2024-04-16T18:09:37.509 DEBUG:boto:Data: 2024-04-16T18:09:37.509 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.509 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.509 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.509 DEBUG:boto:Params: {} 2024-04-16T18:09:37.509 DEBUG:boto:Token: None 2024-04-16T18:09:37.509 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/obj6 2024-04-16T18:09:37.509 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0Wghic5CZ+iWhwjbGuOQMaP0HuI= 2024-04-16T18:09:37.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0Wghic5CZ+iWhwjbGuOQMaP0HuI='} 2024-04-16T18:09:37.554 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004a57193e8f199a6a-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.555 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:37.555 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:37.555 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.555 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:37.555 DEBUG:boto:Data: 2024-04-16T18:09:37.555 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.555 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.555 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.555 DEBUG:boto:Params: {} 2024-04-16T18:09:37.555 DEBUG:boto:Token: None 2024-04-16T18:09:37.556 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/ 2024-04-16T18:09:37.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w= 2024-04-16T18:09:37.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w=', 'Content-Length': '0'} 2024-04-16T18:09:37.559 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f6b7cd2b5f226047-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.560 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:37.560 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:37.560 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.560 DEBUG:boto:Path: /lybtug-38/obj7 2024-04-16T18:09:37.560 DEBUG:boto:Data: 2024-04-16T18:09:37.560 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.560 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.560 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.560 DEBUG:boto:Params: {} 2024-04-16T18:09:37.560 DEBUG:boto:Token: None 2024-04-16T18:09:37.561 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/obj7 2024-04-16T18:09:37.561 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CVZ2t21OMl9IngcApnIuhw5HngA= 2024-04-16T18:09:37.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CVZ2t21OMl9IngcApnIuhw5HngA='} 2024-04-16T18:09:37.569 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009efd2363fe4ad587-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.569 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:37.569 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:37.570 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.570 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:37.570 DEBUG:boto:Data: 2024-04-16T18:09:37.570 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.570 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.570 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.570 DEBUG:boto:Params: {} 2024-04-16T18:09:37.570 DEBUG:boto:Token: None 2024-04-16T18:09:37.570 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/ 2024-04-16T18:09:37.570 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w= 2024-04-16T18:09:37.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqSrOsdhZgrQZhc2TlLThprT22w=', 'Content-Length': '0'} 2024-04-16T18:09:37.574 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006cc16895e86eb528-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.574 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:09:37.574 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:09:37.574 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.575 DEBUG:boto:Path: /lybtug-38/obj8 2024-04-16T18:09:37.575 DEBUG:boto:Data: 2024-04-16T18:09:37.575 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:37.575 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.575 DEBUG:boto:Params: {} 2024-04-16T18:09:37.575 DEBUG:boto:Token: None 2024-04-16T18:09:37.575 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-38/obj8 2024-04-16T18:09:37.575 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dMWiCNnXTHjj+4pS2iFGmH5teCs= 2024-04-16T18:09:37.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dMWiCNnXTHjj+4pS2iFGmH5teCs='} 2024-04-16T18:09:37.618 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ae00bd782d2dec5b-00661ebee1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.619 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:37.619 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:37.619 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.619 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:37.619 DEBUG:boto:Data: 2024-04-16T18:09:37.619 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.619 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.619 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.619 DEBUG:boto:Params: {} 2024-04-16T18:09:37.619 DEBUG:boto:Token: None 2024-04-16T18:09:37.620 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/ 2024-04-16T18:09:37.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio= 2024-04-16T18:09:37.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio=', 'Content-Length': '0'} 2024-04-16T18:09:37.624 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001a6722c3e89ab563-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.624 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:09:37.624 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:09:37.624 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.624 DEBUG:boto:Path: /lybtug-39/obj5 2024-04-16T18:09:37.624 DEBUG:boto:Data: 2024-04-16T18:09:37.625 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.625 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.625 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.625 DEBUG:boto:Params: {} 2024-04-16T18:09:37.625 DEBUG:boto:Token: None 2024-04-16T18:09:37.625 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/obj5 2024-04-16T18:09:37.625 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nS91haYEfp+jq8avxMBzywZ5qrg= 2024-04-16T18:09:37.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nS91haYEfp+jq8avxMBzywZ5qrg='} 2024-04-16T18:09:37.635 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ad5e7a22f5d1f623-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.636 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:37.636 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:37.636 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.636 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:37.636 DEBUG:boto:Data: 2024-04-16T18:09:37.636 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.636 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.636 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.636 DEBUG:boto:Params: {} 2024-04-16T18:09:37.636 DEBUG:boto:Token: None 2024-04-16T18:09:37.636 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/ 2024-04-16T18:09:37.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio= 2024-04-16T18:09:37.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio=', 'Content-Length': '0'} 2024-04-16T18:09:37.643 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004bb7835a4b445a46-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.643 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:09:37.643 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:09:37.643 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.643 DEBUG:boto:Path: /lybtug-39/obj6 2024-04-16T18:09:37.643 DEBUG:boto:Data: 2024-04-16T18:09:37.643 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.643 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.644 DEBUG:boto:Params: {} 2024-04-16T18:09:37.644 DEBUG:boto:Token: None 2024-04-16T18:09:37.644 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/obj6 2024-04-16T18:09:37.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HEMyZmSWikwoFVvmxHLT2ZSSQX4= 2024-04-16T18:09:37.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HEMyZmSWikwoFVvmxHLT2ZSSQX4='} 2024-04-16T18:09:37.687 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004391af9e94fc2b38-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.687 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:37.687 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:37.687 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.687 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:37.687 DEBUG:boto:Data: 2024-04-16T18:09:37.687 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.687 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.687 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.687 DEBUG:boto:Params: {} 2024-04-16T18:09:37.687 DEBUG:boto:Token: None 2024-04-16T18:09:37.688 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/ 2024-04-16T18:09:37.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio= 2024-04-16T18:09:37.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio=', 'Content-Length': '0'} 2024-04-16T18:09:37.691 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aa1e34fea1dc48c0-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.692 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:09:37.692 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:09:37.692 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.692 DEBUG:boto:Path: /lybtug-39/obj7 2024-04-16T18:09:37.692 DEBUG:boto:Data: 2024-04-16T18:09:37.692 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.692 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.692 DEBUG:boto:Params: {} 2024-04-16T18:09:37.692 DEBUG:boto:Token: None 2024-04-16T18:09:37.692 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/obj7 2024-04-16T18:09:37.693 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:i08fwZ3Zdddo98ZxZSRiVH65cEM= 2024-04-16T18:09:37.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:i08fwZ3Zdddo98ZxZSRiVH65cEM='} 2024-04-16T18:09:37.701 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000abf5a0ade63c39df-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.701 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:09:37.701 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:09:37.701 DEBUG:boto:Method: HEAD 2024-04-16T18:09:37.701 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:09:37.701 DEBUG:boto:Data: 2024-04-16T18:09:37.702 DEBUG:boto:Headers: {} 2024-04-16T18:09:37.702 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.702 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.702 DEBUG:boto:Params: {} 2024-04-16T18:09:37.702 DEBUG:boto:Token: None 2024-04-16T18:09:37.702 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/ 2024-04-16T18:09:37.702 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio= 2024-04-16T18:09:37.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iMYAhKFhXTIHt/pk2bVzaphNyio=', 'Content-Length': '0'} 2024-04-16T18:09:37.706 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000016a00cebf0e0d913-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.706 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:09:37.706 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:09:37.707 DEBUG:boto:Method: PUT 2024-04-16T18:09:37.707 DEBUG:boto:Path: /lybtug-39/obj8 2024-04-16T18:09:37.707 DEBUG:boto:Data: 2024-04-16T18:09:37.707 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:09:37.707 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:37.707 DEBUG:boto:Port: 8000 2024-04-16T18:09:37.707 DEBUG:boto:Params: {} 2024-04-16T18:09:37.707 DEBUG:boto:Token: None 2024-04-16T18:09:37.707 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:09:37 GMT /lybtug-39/obj8 2024-04-16T18:09:37.707 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EepbErCZVUzoePiUJaqFtyIkRkc= 2024-04-16T18:09:37.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:09:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EepbErCZVUzoePiUJaqFtyIkRkc='} 2024-04-16T18:09:37.750 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000969e460e079bd015-00661ebee1-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:37.751 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-38 2024-04-16T18:09:37.751 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:37.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:37.751 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-38 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:09:37.801 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:37.801 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:37.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:37.889+0000 7f6af766f980 1 waiting for incremental sync to catch up: 2024-04-16T18:09:37.884 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.21.6, , 00000000001.35.6, , 00000000001.22.6, , , 00000000001.31.6, , , ] 2024-04-16T18:09:37.884 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.25.6, 00000000002.8.1, 00000000004.41.6, 00000000002.30.1, 00000000004.26.6, 00000000002.45.1, 00000000002.25.1, 00000000004.37.6, 00000000002.26.1, 00000000002.31.1, 00000000002.46.1] 2024-04-16T18:09:42.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:42.893+0000 7f6af766f980 1 bucket sync caught up with source: 2024-04-16T18:09:42.887 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000004.25.6, 00000000002.8.1, 00000000004.41.6, 00000000002.30.1, 00000000004.26.6, 00000000002.45.1, 00000000002.25.1, 00000000004.37.6, 00000000002.26.1, 00000000002.31.1, 00000000002.46.1] 2024-04-16T18:09:42.887 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.25.6, 00000000002.8.1, 00000000004.41.6, 00000000002.30.1, 00000000004.26.6, 00000000002.45.1, 00000000002.25.1, 00000000004.37.6, 00000000002.26.1, 00000000002.31.1, 00000000002.46.1] 2024-04-16T18:09:42.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:09:42.893+0000 7f6af766f980 0 bucket checkpoint complete 2024-04-16T18:09:42.933 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-38 2024-04-16T18:09:42.933 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:42.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-38', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:09:42.933 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-38 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:09:42.984 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:09:42.984 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:09:43.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:43.065+0000 7f16413c9980 1 waiting for incremental sync to catch up: 2024-04-16T18:09:43.062 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.1316.6, , 00000000001.1281.6, , 00000000001.654.6, , , 00000000001.866.6, , , ] 2024-04-16T18:09:43.062 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000002.1318.6, , 00000000002.1283.6, , 00000000002.656.6, , , 00000000002.871.6, , , ] 2024-04-16T18:09:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:48.065+0000 7f16413c9980 1 waiting for incremental sync to catch up: 2024-04-16T18:09:48.065 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.1316.6, , 00000000001.1281.6, , 00000000001.654.6, , , 00000000001.866.6, , , ] 2024-04-16T18:09:48.065 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000002.1318.6, , 00000000002.1283.6, , 00000000002.656.6, , , 00000000002.871.6, , , ] 2024-04-16T18:09:53.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:53.069+0000 7f16413c9980 1 waiting for incremental sync to catch up: 2024-04-16T18:09:53.067 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000002.1318.6, , 00000000002.1283.6, , 00000000001.654.6, , , 00000000001.866.6, , , ] 2024-04-16T18:09:53.067 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000002.1318.6, , 00000000002.1283.6, , 00000000002.656.6, , , 00000000002.871.6, , , ] 2024-04-16T18:09:58.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:58.072+0000 7f16413c9980 1 bucket sync caught up with source: 2024-04-16T18:09:58.071 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000002.1318.6, , 00000000002.1283.6, , 00000000002.656.6, , , 00000000002.871.6, , , ] 2024-04-16T18:09:58.071 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000002.1318.6, , 00000000002.1283.6, , 00000000002.656.6, , , 00000000002.871.6, , , ] 2024-04-16T18:09:58.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:09:58.072+0000 7f16413c9980 0 bucket checkpoint complete 2024-04-16T18:09:58.118 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-38 zones={test-zone2, test-zone1} 2024-04-16T18:09:58.118 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.118 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.118 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.119 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:58.119 DEBUG:boto:Data: 2024-04-16T18:09:58.119 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.119 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.119 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.119 DEBUG:boto:Params: {} 2024-04-16T18:09:58.119 DEBUG:boto:Token: None 2024-04-16T18:09:58.120 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IRVGyHVfjdQBKgpS5JsODRwXkEc= 2024-04-16T18:09:58.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IRVGyHVfjdQBKgpS5JsODRwXkEc=', 'Content-Length': '0'} 2024-04-16T18:09:58.124 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006179babb77bf5747-00661ebef6-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.124 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.124 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.125 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.125 DEBUG:boto:Path: /lybtug-38/ 2024-04-16T18:09:58.125 DEBUG:boto:Data: 2024-04-16T18:09:58.125 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.125 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.125 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.125 DEBUG:boto:Params: {} 2024-04-16T18:09:58.125 DEBUG:boto:Token: None 2024-04-16T18:09:58.126 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.126 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IRVGyHVfjdQBKgpS5JsODRwXkEc= 2024-04-16T18:09:58.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IRVGyHVfjdQBKgpS5JsODRwXkEc=', 'Content-Length': '0'} 2024-04-16T18:09:58.130 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000031f16689f740f86a-00661ebef6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.130 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:09:58.130 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.131 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.131 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.131 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.131 DEBUG:boto:Method: GET 2024-04-16T18:09:58.131 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.131 DEBUG:boto:Data: 2024-04-16T18:09:58.131 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.131 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.131 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.132 DEBUG:boto:Params: {} 2024-04-16T18:09:58.132 DEBUG:boto:Token: None 2024-04-16T18:09:58.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/?versions 2024-04-16T18:09:58.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0= 2024-04-16T18:09:58.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0=', 'Content-Length': '0'} 2024-04-16T18:09:58.136 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023399d622a314bc1-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.137 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.137 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:09:58.138 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:09:58.139 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.139 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.139 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.139 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.139 DEBUG:boto:Method: GET 2024-04-16T18:09:58.139 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.139 DEBUG:boto:Data: 2024-04-16T18:09:58.139 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.140 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.140 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.140 DEBUG:boto:Params: {} 2024-04-16T18:09:58.140 DEBUG:boto:Token: None 2024-04-16T18:09:58.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/?versions 2024-04-16T18:09:58.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0= 2024-04-16T18:09:58.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0=', 'Content-Length': '0'} 2024-04-16T18:09:58.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091de004fc457684a-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.145 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.145 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:09:58.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:09:58.147 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.147 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.147 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.147 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.147 DEBUG:boto:Method: GET 2024-04-16T18:09:58.147 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.147 DEBUG:boto:Data: 2024-04-16T18:09:58.147 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.148 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.148 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.148 DEBUG:boto:Params: {} 2024-04-16T18:09:58.148 DEBUG:boto:Token: None 2024-04-16T18:09:58.148 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/?versions 2024-04-16T18:09:58.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0= 2024-04-16T18:09:58.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0=', 'Content-Length': '0'} 2024-04-16T18:09:58.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0108198571a47f6-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.153 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.153 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.154 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.154 DEBUG:boto:path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.154 DEBUG:boto:auth_path=/lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.154 DEBUG:boto:Method: GET 2024-04-16T18:09:58.154 DEBUG:boto:Path: /lybtug-38/?versions&max-keys=999 2024-04-16T18:09:58.154 DEBUG:boto:Data: 2024-04-16T18:09:58.154 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.154 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.154 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.155 DEBUG:boto:Params: {} 2024-04-16T18:09:58.155 DEBUG:boto:Token: None 2024-04-16T18:09:58.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/?versions 2024-04-16T18:09:58.155 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0= 2024-04-16T18:09:58.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/+euqjAChYW0FwF+PveJ8xvFH0=', 'Content-Length': '0'} 2024-04-16T18:09:58.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c73be0453b28fd1-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.160 DEBUG:boto:b'lybtug-38999falseobj1nulltrue2024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.161 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:58.161 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.161 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.161 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.161 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.161 DEBUG:boto:Method: GET 2024-04-16T18:09:58.162 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.162 DEBUG:boto:Data: 2024-04-16T18:09:58.162 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.162 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.162 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.162 DEBUG:boto:Params: {} 2024-04-16T18:09:58.162 DEBUG:boto:Token: None 2024-04-16T18:09:58.162 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.163 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU= 2024-04-16T18:09:58.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU=', 'Content-Length': '0'} 2024-04-16T18:09:58.166 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000008511cf46943c65b-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.207 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.207 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.207 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.207 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.207 DEBUG:boto:Method: GET 2024-04-16T18:09:58.207 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.207 DEBUG:boto:Data: 2024-04-16T18:09:58.207 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.208 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.208 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.208 DEBUG:boto:Params: {} 2024-04-16T18:09:58.208 DEBUG:boto:Token: None 2024-04-16T18:09:58.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU= 2024-04-16T18:09:58.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU=', 'Content-Length': '0'} 2024-04-16T18:09:58.211 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cfc282d3172f2f96-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.212 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.212 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.212 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.212 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.212 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.212 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.212 DEBUG:boto:Data: 2024-04-16T18:09:58.212 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.213 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.213 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.213 DEBUG:boto:Params: {} 2024-04-16T18:09:58.213 DEBUG:boto:Token: None 2024-04-16T18:09:58.213 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g8KrEsQSFMfcIf5yIsoaL9g0M7g= 2024-04-16T18:09:58.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g8KrEsQSFMfcIf5yIsoaL9g0M7g=', 'Content-Length': '0'} 2024-04-16T18:09:58.216 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005d526618a411b48c-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.217 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.217 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.217 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.217 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.217 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.217 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.217 DEBUG:boto:Data: 2024-04-16T18:09:58.218 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.218 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.218 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.218 DEBUG:boto:Params: {} 2024-04-16T18:09:58.218 DEBUG:boto:Token: None 2024-04-16T18:09:58.218 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.218 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g8KrEsQSFMfcIf5yIsoaL9g0M7g= 2024-04-16T18:09:58.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g8KrEsQSFMfcIf5yIsoaL9g0M7g=', 'Content-Length': '0'} 2024-04-16T18:09:58.222 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c35ccb3af4d548ef-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.222 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:58.222 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.222 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.222 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.223 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.223 DEBUG:boto:Method: GET 2024-04-16T18:09:58.223 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.223 DEBUG:boto:Data: 2024-04-16T18:09:58.223 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.223 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.223 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.223 DEBUG:boto:Params: {} 2024-04-16T18:09:58.223 DEBUG:boto:Token: None 2024-04-16T18:09:58.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU= 2024-04-16T18:09:58.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU=', 'Content-Length': '0'} 2024-04-16T18:09:58.227 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f0ed4930110e8123-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.270 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.271 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.271 DEBUG:boto:path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.271 DEBUG:boto:auth_path=/lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.271 DEBUG:boto:Method: GET 2024-04-16T18:09:58.271 DEBUG:boto:Path: /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.271 DEBUG:boto:Data: 2024-04-16T18:09:58.271 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.271 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.272 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.272 DEBUG:boto:Params: {} 2024-04-16T18:09:58.272 DEBUG:boto:Token: None 2024-04-16T18:09:58.272 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1?versionId=null 2024-04-16T18:09:58.272 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU= 2024-04-16T18:09:58.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D3LeH5ZZQgkQGcTp0kSHaq1P/RU=', 'Content-Length': '0'} 2024-04-16T18:09:58.275 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000df028493c0958405-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.276 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.276 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.276 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.276 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.276 DEBUG:boto:Method: GET 2024-04-16T18:09:58.276 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:58.276 DEBUG:boto:Data: 2024-04-16T18:09:58.276 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.277 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.277 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.277 DEBUG:boto:Params: {} 2024-04-16T18:09:58.277 DEBUG:boto:Token: None 2024-04-16T18:09:58.277 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008595ac7eb48aeca1-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.322 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.323 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.323 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.323 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.323 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.323 DEBUG:boto:Method: GET 2024-04-16T18:09:58.323 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:58.324 DEBUG:boto:Data: 2024-04-16T18:09:58.324 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.324 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.324 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.324 DEBUG:boto:Params: {} 2024-04-16T18:09:58.324 DEBUG:boto:Token: None 2024-04-16T18:09:58.324 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.329 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9f65bc2122e3c23-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.329 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.329 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.329 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.329 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.330 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:58.330 DEBUG:boto:Data: 2024-04-16T18:09:58.330 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.330 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.330 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.330 DEBUG:boto:Params: {} 2024-04-16T18:09:58.330 DEBUG:boto:Token: None 2024-04-16T18:09:58.331 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1 2024-04-16T18:09:58.331 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8fvCG3w/3RP8ihgjFzfzWFsX+N0= 2024-04-16T18:09:58.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8fvCG3w/3RP8ihgjFzfzWFsX+N0=', 'Content-Length': '0'} 2024-04-16T18:09:58.334 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d1a95aa5c45a8d88-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.334 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.334 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.335 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.335 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:58.335 DEBUG:boto:Data: 2024-04-16T18:09:58.335 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.335 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.335 DEBUG:boto:Params: {} 2024-04-16T18:09:58.335 DEBUG:boto:Token: None 2024-04-16T18:09:58.336 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1 2024-04-16T18:09:58.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8fvCG3w/3RP8ihgjFzfzWFsX+N0= 2024-04-16T18:09:58.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8fvCG3w/3RP8ihgjFzfzWFsX+N0=', 'Content-Length': '0'} 2024-04-16T18:09:58.339 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c3145a87ce38da41-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.339 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:09:58.340 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.340 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.340 DEBUG:boto:Method: GET 2024-04-16T18:09:58.340 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:58.340 DEBUG:boto:Data: 2024-04-16T18:09:58.340 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.340 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.340 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.340 DEBUG:boto:Params: {} 2024-04-16T18:09:58.341 DEBUG:boto:Token: None 2024-04-16T18:09:58.341 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1 2024-04-16T18:09:58.341 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XHqiPGp7wKiBIC7k0yFvkx5h3xU= 2024-04-16T18:09:58.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XHqiPGp7wKiBIC7k0yFvkx5h3xU=', 'Content-Length': '0'} 2024-04-16T18:09:58.344 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dc63afc137c444fc-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.386 DEBUG:boto:path=/lybtug-38/obj1 2024-04-16T18:09:58.387 DEBUG:boto:auth_path=/lybtug-38/obj1 2024-04-16T18:09:58.387 DEBUG:boto:Method: GET 2024-04-16T18:09:58.387 DEBUG:boto:Path: /lybtug-38/obj1 2024-04-16T18:09:58.387 DEBUG:boto:Data: 2024-04-16T18:09:58.387 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.387 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.387 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.387 DEBUG:boto:Params: {} 2024-04-16T18:09:58.388 DEBUG:boto:Token: None 2024-04-16T18:09:58.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj1 2024-04-16T18:09:58.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XHqiPGp7wKiBIC7k0yFvkx5h3xU= 2024-04-16T18:09:58.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XHqiPGp7wKiBIC7k0yFvkx5h3xU=', 'Content-Length': '0'} 2024-04-16T18:09:58.391 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000343ea0dda66c9169-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.391 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.392 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.392 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.392 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.392 DEBUG:boto:Method: GET 2024-04-16T18:09:58.392 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:58.392 DEBUG:boto:Data: 2024-04-16T18:09:58.392 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.392 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.393 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.393 DEBUG:boto:Params: {} 2024-04-16T18:09:58.393 DEBUG:boto:Token: None 2024-04-16T18:09:58.393 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.393 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.397 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af2fa54a055d6646-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.397 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.398 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.398 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.398 DEBUG:boto:path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.398 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj1 2024-04-16T18:09:58.398 DEBUG:boto:Method: GET 2024-04-16T18:09:58.398 DEBUG:boto:Path: /lybtug-38/?prefix=obj1 2024-04-16T18:09:58.399 DEBUG:boto:Data: 2024-04-16T18:09:58.399 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.399 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.399 DEBUG:boto:Params: {} 2024-04-16T18:09:58.399 DEBUG:boto:Token: None 2024-04-16T18:09:58.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.399 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.403 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000fed7dbe21fee4e8-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.404 DEBUG:boto:b'lybtug-38obj11000falseobj12024-04-16T18:09:23.556Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:58.404 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.404 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.405 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.405 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.405 DEBUG:boto:Method: GET 2024-04-16T18:09:58.405 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.405 DEBUG:boto:Data: 2024-04-16T18:09:58.405 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.405 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.405 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.405 DEBUG:boto:Params: {} 2024-04-16T18:09:58.406 DEBUG:boto:Token: None 2024-04-16T18:09:58.406 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.406 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto= 2024-04-16T18:09:58.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto=', 'Content-Length': '0'} 2024-04-16T18:09:58.409 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000099d0d62fd3fae331-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.450 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.451 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.451 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.451 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.451 DEBUG:boto:Method: GET 2024-04-16T18:09:58.451 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.451 DEBUG:boto:Data: 2024-04-16T18:09:58.451 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.452 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.452 DEBUG:boto:Params: {} 2024-04-16T18:09:58.452 DEBUG:boto:Token: None 2024-04-16T18:09:58.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto= 2024-04-16T18:09:58.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto=', 'Content-Length': '0'} 2024-04-16T18:09:58.455 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e327a968752b2cd8-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.455 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.456 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.456 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.456 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.456 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.456 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.456 DEBUG:boto:Data: 2024-04-16T18:09:58.456 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.456 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.457 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.457 DEBUG:boto:Params: {} 2024-04-16T18:09:58.457 DEBUG:boto:Token: None 2024-04-16T18:09:58.457 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.457 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/3qzr7uHj/5WMdSmGlW5b7AwELM= 2024-04-16T18:09:58.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/3qzr7uHj/5WMdSmGlW5b7AwELM=', 'Content-Length': '0'} 2024-04-16T18:09:58.460 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038acd00fd06d120e-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.460 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.461 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.461 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.461 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.461 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.461 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.461 DEBUG:boto:Data: 2024-04-16T18:09:58.461 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.461 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.462 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.462 DEBUG:boto:Params: {} 2024-04-16T18:09:58.462 DEBUG:boto:Token: None 2024-04-16T18:09:58.462 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.462 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/3qzr7uHj/5WMdSmGlW5b7AwELM= 2024-04-16T18:09:58.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/3qzr7uHj/5WMdSmGlW5b7AwELM=', 'Content-Length': '0'} 2024-04-16T18:09:58.465 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000190c1d5070849d3-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:58.466 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.466 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.466 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.466 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.467 DEBUG:boto:Method: GET 2024-04-16T18:09:58.467 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.467 DEBUG:boto:Data: 2024-04-16T18:09:58.467 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.467 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.467 DEBUG:boto:Params: {} 2024-04-16T18:09:58.467 DEBUG:boto:Token: None 2024-04-16T18:09:58.467 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto= 2024-04-16T18:09:58.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto=', 'Content-Length': '0'} 2024-04-16T18:09:58.471 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000083afe3419224880d-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.514 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.515 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.515 DEBUG:boto:path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.515 DEBUG:boto:auth_path=/lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.515 DEBUG:boto:Method: GET 2024-04-16T18:09:58.515 DEBUG:boto:Path: /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.515 DEBUG:boto:Data: 2024-04-16T18:09:58.515 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.515 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.516 DEBUG:boto:Params: {} 2024-04-16T18:09:58.516 DEBUG:boto:Token: None 2024-04-16T18:09:58.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2?versionId=null 2024-04-16T18:09:58.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto= 2024-04-16T18:09:58.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xuvRP9CYyaXwn4w/IeeAbyp0wto=', 'Content-Length': '0'} 2024-04-16T18:09:58.519 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000094ad7e1f41b2f7a7-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.520 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.520 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.520 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.520 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.520 DEBUG:boto:Method: GET 2024-04-16T18:09:58.520 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:58.520 DEBUG:boto:Data: 2024-04-16T18:09:58.520 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.521 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.521 DEBUG:boto:Params: {} 2024-04-16T18:09:58.521 DEBUG:boto:Token: None 2024-04-16T18:09:58.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.525 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7eb776a282ece21-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.566 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.567 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.567 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.567 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.567 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.567 DEBUG:boto:Method: GET 2024-04-16T18:09:58.568 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:58.568 DEBUG:boto:Data: 2024-04-16T18:09:58.568 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.568 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.568 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.568 DEBUG:boto:Params: {} 2024-04-16T18:09:58.568 DEBUG:boto:Token: None 2024-04-16T18:09:58.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.569 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.573 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043b55bb8605cf73b-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.573 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.574 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.574 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.574 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.574 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:58.574 DEBUG:boto:Data: 2024-04-16T18:09:58.574 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.574 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.575 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.575 DEBUG:boto:Params: {} 2024-04-16T18:09:58.575 DEBUG:boto:Token: None 2024-04-16T18:09:58.575 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2 2024-04-16T18:09:58.575 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BK7bSlcIt6iJJwJzO4ohaCSXJdM= 2024-04-16T18:09:58.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BK7bSlcIt6iJJwJzO4ohaCSXJdM=', 'Content-Length': '0'} 2024-04-16T18:09:58.578 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a140e217d07613e2-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.579 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.579 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.579 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.579 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:58.579 DEBUG:boto:Data: 2024-04-16T18:09:58.579 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.580 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.580 DEBUG:boto:Params: {} 2024-04-16T18:09:58.580 DEBUG:boto:Token: None 2024-04-16T18:09:58.580 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2 2024-04-16T18:09:58.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BK7bSlcIt6iJJwJzO4ohaCSXJdM= 2024-04-16T18:09:58.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BK7bSlcIt6iJJwJzO4ohaCSXJdM=', 'Content-Length': '0'} 2024-04-16T18:09:58.583 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004153f0f8d6941aef-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:09:58.584 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.584 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.584 DEBUG:boto:Method: GET 2024-04-16T18:09:58.584 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:58.584 DEBUG:boto:Data: 2024-04-16T18:09:58.584 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.584 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.585 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.585 DEBUG:boto:Params: {} 2024-04-16T18:09:58.585 DEBUG:boto:Token: None 2024-04-16T18:09:58.585 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2 2024-04-16T18:09:58.585 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oztf8KUmYfE6midaxNwa8GNSBf0= 2024-04-16T18:09:58.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oztf8KUmYfE6midaxNwa8GNSBf0=', 'Content-Length': '0'} 2024-04-16T18:09:58.588 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000114fb1c3c9010210-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.630 DEBUG:boto:path=/lybtug-38/obj2 2024-04-16T18:09:58.631 DEBUG:boto:auth_path=/lybtug-38/obj2 2024-04-16T18:09:58.631 DEBUG:boto:Method: GET 2024-04-16T18:09:58.631 DEBUG:boto:Path: /lybtug-38/obj2 2024-04-16T18:09:58.631 DEBUG:boto:Data: 2024-04-16T18:09:58.631 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.631 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.631 DEBUG:boto:Params: {} 2024-04-16T18:09:58.632 DEBUG:boto:Token: None 2024-04-16T18:09:58.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj2 2024-04-16T18:09:58.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oztf8KUmYfE6midaxNwa8GNSBf0= 2024-04-16T18:09:58.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oztf8KUmYfE6midaxNwa8GNSBf0=', 'Content-Length': '0'} 2024-04-16T18:09:58.635 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c76423bce13f26ff-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.636 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.636 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.636 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.636 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.636 DEBUG:boto:Method: GET 2024-04-16T18:09:58.636 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:58.637 DEBUG:boto:Data: 2024-04-16T18:09:58.637 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.637 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.637 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.637 DEBUG:boto:Params: {} 2024-04-16T18:09:58.637 DEBUG:boto:Token: None 2024-04-16T18:09:58.637 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.638 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.642 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aea52f71030949fe-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.642 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.643 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.643 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.643 DEBUG:boto:path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.643 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj2 2024-04-16T18:09:58.643 DEBUG:boto:Method: GET 2024-04-16T18:09:58.643 DEBUG:boto:Path: /lybtug-38/?prefix=obj2 2024-04-16T18:09:58.643 DEBUG:boto:Data: 2024-04-16T18:09:58.644 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.644 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.644 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.644 DEBUG:boto:Params: {} 2024-04-16T18:09:58.644 DEBUG:boto:Token: None 2024-04-16T18:09:58.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.648 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c20fec063e8806c-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.649 DEBUG:boto:b'lybtug-38obj21000falseobj22024-04-16T18:09:23.577Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.649 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:58.649 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.650 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.650 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.650 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.650 DEBUG:boto:Method: GET 2024-04-16T18:09:58.650 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.650 DEBUG:boto:Data: 2024-04-16T18:09:58.650 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.650 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.650 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.651 DEBUG:boto:Params: {} 2024-04-16T18:09:58.651 DEBUG:boto:Token: None 2024-04-16T18:09:58.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.651 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY= 2024-04-16T18:09:58.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY=', 'Content-Length': '0'} 2024-04-16T18:09:58.655 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005351604a39cd5118-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.698 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.699 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.699 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.699 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.699 DEBUG:boto:Method: GET 2024-04-16T18:09:58.699 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.699 DEBUG:boto:Data: 2024-04-16T18:09:58.699 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.700 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.700 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.700 DEBUG:boto:Params: {} 2024-04-16T18:09:58.700 DEBUG:boto:Token: None 2024-04-16T18:09:58.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY= 2024-04-16T18:09:58.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY=', 'Content-Length': '0'} 2024-04-16T18:09:58.704 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f807d30ab34572bb-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.704 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.704 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.705 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.705 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.705 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.705 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.705 DEBUG:boto:Data: 2024-04-16T18:09:58.705 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.705 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.705 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.705 DEBUG:boto:Params: {} 2024-04-16T18:09:58.706 DEBUG:boto:Token: None 2024-04-16T18:09:58.706 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.706 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2FONFAZLORuHNJAb3aufB0JaFe4= 2024-04-16T18:09:58.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2FONFAZLORuHNJAb3aufB0JaFe4=', 'Content-Length': '0'} 2024-04-16T18:09:58.709 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dc80ff65bffe7827-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.710 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.710 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.710 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.710 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.710 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.710 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.710 DEBUG:boto:Data: 2024-04-16T18:09:58.711 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.711 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.711 DEBUG:boto:Params: {} 2024-04-16T18:09:58.711 DEBUG:boto:Token: None 2024-04-16T18:09:58.711 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.711 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2FONFAZLORuHNJAb3aufB0JaFe4= 2024-04-16T18:09:58.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2FONFAZLORuHNJAb3aufB0JaFe4=', 'Content-Length': '0'} 2024-04-16T18:09:58.714 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e20ddc1a6fa6a785-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:58.715 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.715 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.715 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.715 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.715 DEBUG:boto:Method: GET 2024-04-16T18:09:58.716 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.716 DEBUG:boto:Data: 2024-04-16T18:09:58.716 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.716 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.716 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.716 DEBUG:boto:Params: {} 2024-04-16T18:09:58.716 DEBUG:boto:Token: None 2024-04-16T18:09:58.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY= 2024-04-16T18:09:58.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY=', 'Content-Length': '0'} 2024-04-16T18:09:58.720 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f927cad2e24e6bb2-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.762 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.763 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.763 DEBUG:boto:path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.763 DEBUG:boto:auth_path=/lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.763 DEBUG:boto:Method: GET 2024-04-16T18:09:58.763 DEBUG:boto:Path: /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.763 DEBUG:boto:Data: 2024-04-16T18:09:58.763 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.763 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.764 DEBUG:boto:Params: {} 2024-04-16T18:09:58.764 DEBUG:boto:Token: None 2024-04-16T18:09:58.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3?versionId=null 2024-04-16T18:09:58.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY= 2024-04-16T18:09:58.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ63n+RmPUKz3kh6IH2w1fLiboY=', 'Content-Length': '0'} 2024-04-16T18:09:58.767 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d67c60e4a58c5c0b-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.768 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.768 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.768 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.768 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.768 DEBUG:boto:Method: GET 2024-04-16T18:09:58.768 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:58.769 DEBUG:boto:Data: 2024-04-16T18:09:58.769 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.769 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.769 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.769 DEBUG:boto:Params: {} 2024-04-16T18:09:58.769 DEBUG:boto:Token: None 2024-04-16T18:09:58.769 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.769 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.774 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040902bddf0204f58-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.774 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.774 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.775 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.775 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.775 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.775 DEBUG:boto:Method: GET 2024-04-16T18:09:58.775 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:58.775 DEBUG:boto:Data: 2024-04-16T18:09:58.775 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.776 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.776 DEBUG:boto:Params: {} 2024-04-16T18:09:58.776 DEBUG:boto:Token: None 2024-04-16T18:09:58.776 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.780 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000745c69666ca7aaba-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.822 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.823 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.823 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.823 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.823 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:58.823 DEBUG:boto:Data: 2024-04-16T18:09:58.823 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.823 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.824 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.824 DEBUG:boto:Params: {} 2024-04-16T18:09:58.824 DEBUG:boto:Token: None 2024-04-16T18:09:58.824 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3 2024-04-16T18:09:58.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sPp7sE80LtSoHhKHixJeUz598L0= 2024-04-16T18:09:58.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sPp7sE80LtSoHhKHixJeUz598L0=', 'Content-Length': '0'} 2024-04-16T18:09:58.827 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000460c53f00e8561d2-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.827 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.827 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.828 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.828 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:58.828 DEBUG:boto:Data: 2024-04-16T18:09:58.828 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.828 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.828 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.828 DEBUG:boto:Params: {} 2024-04-16T18:09:58.828 DEBUG:boto:Token: None 2024-04-16T18:09:58.828 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3 2024-04-16T18:09:58.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sPp7sE80LtSoHhKHixJeUz598L0= 2024-04-16T18:09:58.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sPp7sE80LtSoHhKHixJeUz598L0=', 'Content-Length': '0'} 2024-04-16T18:09:58.832 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000154513a5ff05f773-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.832 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:09:58.832 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.832 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.832 DEBUG:boto:Method: GET 2024-04-16T18:09:58.832 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:58.832 DEBUG:boto:Data: 2024-04-16T18:09:58.832 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.833 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.833 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.833 DEBUG:boto:Params: {} 2024-04-16T18:09:58.833 DEBUG:boto:Token: None 2024-04-16T18:09:58.833 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3 2024-04-16T18:09:58.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Bs3++btxagRyAvcsAB+Rsq/BnZw= 2024-04-16T18:09:58.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Bs3++btxagRyAvcsAB+Rsq/BnZw=', 'Content-Length': '0'} 2024-04-16T18:09:58.836 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009994f9fba0fe9fec-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.878 DEBUG:boto:path=/lybtug-38/obj3 2024-04-16T18:09:58.879 DEBUG:boto:auth_path=/lybtug-38/obj3 2024-04-16T18:09:58.879 DEBUG:boto:Method: GET 2024-04-16T18:09:58.879 DEBUG:boto:Path: /lybtug-38/obj3 2024-04-16T18:09:58.879 DEBUG:boto:Data: 2024-04-16T18:09:58.879 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.879 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.879 DEBUG:boto:Params: {} 2024-04-16T18:09:58.879 DEBUG:boto:Token: None 2024-04-16T18:09:58.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj3 2024-04-16T18:09:58.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Bs3++btxagRyAvcsAB+Rsq/BnZw= 2024-04-16T18:09:58.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Bs3++btxagRyAvcsAB+Rsq/BnZw=', 'Content-Length': '0'} 2024-04-16T18:09:58.883 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000827b525ad2cd75a-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.883 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.883 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.883 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.883 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.883 DEBUG:boto:Method: GET 2024-04-16T18:09:58.883 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:58.884 DEBUG:boto:Data: 2024-04-16T18:09:58.884 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.884 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.884 DEBUG:boto:Params: {} 2024-04-16T18:09:58.884 DEBUG:boto:Token: None 2024-04-16T18:09:58.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000334a630ee432073e-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.930 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.931 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:58.931 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:58.931 DEBUG:boto:path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.931 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj3 2024-04-16T18:09:58.931 DEBUG:boto:Method: GET 2024-04-16T18:09:58.931 DEBUG:boto:Path: /lybtug-38/?prefix=obj3 2024-04-16T18:09:58.931 DEBUG:boto:Data: 2024-04-16T18:09:58.931 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.932 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.932 DEBUG:boto:Params: {} 2024-04-16T18:09:58.932 DEBUG:boto:Token: None 2024-04-16T18:09:58.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/ 2024-04-16T18:09:58.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE= 2024-04-16T18:09:58.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GXG+7m95ZofgHA+JjmPGUEK8YE=', 'Content-Length': '0'} 2024-04-16T18:09:58.936 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072024fc5237e9b21-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.937 DEBUG:boto:b'lybtug-38obj31000falseobj32024-04-16T18:09:23.625Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:58.937 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:58.937 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:58.937 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:58.937 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.937 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.937 DEBUG:boto:Method: GET 2024-04-16T18:09:58.937 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.938 DEBUG:boto:Data: 2024-04-16T18:09:58.938 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.938 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.938 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.938 DEBUG:boto:Params: {} 2024-04-16T18:09:58.938 DEBUG:boto:Token: None 2024-04-16T18:09:58.938 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.938 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZSWfJ3XhU2OQH55eXlcIHxOiPmk= 2024-04-16T18:09:58.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZSWfJ3XhU2OQH55eXlcIHxOiPmk=', 'Content-Length': '0'} 2024-04-16T18:09:58.941 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000021554dab496799ea-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.982 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:58.983 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:58.983 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.983 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.983 DEBUG:boto:Method: GET 2024-04-16T18:09:58.983 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.983 DEBUG:boto:Data: 2024-04-16T18:09:58.983 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.983 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.983 DEBUG:boto:Params: {} 2024-04-16T18:09:58.983 DEBUG:boto:Token: None 2024-04-16T18:09:58.983 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZSWfJ3XhU2OQH55eXlcIHxOiPmk= 2024-04-16T18:09:58.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZSWfJ3XhU2OQH55eXlcIHxOiPmk=', 'Content-Length': '0'} 2024-04-16T18:09:58.987 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003a76625e7a8f8a14-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.987 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:58.987 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:58.987 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.987 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.987 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.987 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.987 DEBUG:boto:Data: 2024-04-16T18:09:58.987 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.988 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.988 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.988 DEBUG:boto:Params: {} 2024-04-16T18:09:58.988 DEBUG:boto:Token: None 2024-04-16T18:09:58.988 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wL3GMF0WPVG8ZcpMQIGCYV4xgXo= 2024-04-16T18:09:58.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wL3GMF0WPVG8ZcpMQIGCYV4xgXo=', 'Content-Length': '0'} 2024-04-16T18:09:58.991 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001da2194783cb4536-00661ebef6-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.991 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:58.991 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:58.991 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.991 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.992 DEBUG:boto:Method: HEAD 2024-04-16T18:09:58.992 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.992 DEBUG:boto:Data: 2024-04-16T18:09:58.992 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.992 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:58.992 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.992 DEBUG:boto:Params: {} 2024-04-16T18:09:58.992 DEBUG:boto:Token: None 2024-04-16T18:09:58.992 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wL3GMF0WPVG8ZcpMQIGCYV4xgXo= 2024-04-16T18:09:58.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wL3GMF0WPVG8ZcpMQIGCYV4xgXo=', 'Content-Length': '0'} 2024-04-16T18:09:58.995 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c9d23091e5a01fb5-00661ebef6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:58.996 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:58.996 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:58.996 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:58.996 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.996 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.996 DEBUG:boto:Method: GET 2024-04-16T18:09:58.996 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.996 DEBUG:boto:Data: 2024-04-16T18:09:58.996 DEBUG:boto:Headers: {} 2024-04-16T18:09:58.996 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:58.996 DEBUG:boto:Port: 8000 2024-04-16T18:09:58.997 DEBUG:boto:Params: {} 2024-04-16T18:09:58.997 DEBUG:boto:Token: None 2024-04-16T18:09:58.997 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:58 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:58.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZSWfJ3XhU2OQH55eXlcIHxOiPmk= 2024-04-16T18:09:58.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZSWfJ3XhU2OQH55eXlcIHxOiPmk=', 'Content-Length': '0'} 2024-04-16T18:09:59.000 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e8031a74a773034b-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.042 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:59.042 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:59.042 DEBUG:boto:path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:59.043 DEBUG:boto:auth_path=/lybtug-38/obj4?versionId=null 2024-04-16T18:09:59.043 DEBUG:boto:Method: GET 2024-04-16T18:09:59.043 DEBUG:boto:Path: /lybtug-38/obj4?versionId=null 2024-04-16T18:09:59.043 DEBUG:boto:Data: 2024-04-16T18:09:59.043 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.043 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.043 DEBUG:boto:Params: {} 2024-04-16T18:09:59.043 DEBUG:boto:Token: None 2024-04-16T18:09:59.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj4?versionId=null 2024-04-16T18:09:59.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IWfSyVREJuiv/iNGT7roo/Tqf78= 2024-04-16T18:09:59.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IWfSyVREJuiv/iNGT7roo/Tqf78=', 'Content-Length': '0'} 2024-04-16T18:09:59.047 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005e97a36cf0776c21-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.047 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.047 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.047 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.047 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.047 DEBUG:boto:Method: GET 2024-04-16T18:09:59.047 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:59.047 DEBUG:boto:Data: 2024-04-16T18:09:59.047 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.047 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.048 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.048 DEBUG:boto:Params: {} 2024-04-16T18:09:59.048 DEBUG:boto:Token: None 2024-04-16T18:09:59.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054f0dde2cf2caa57-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.052 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.052 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.052 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.052 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.052 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.053 DEBUG:boto:Method: GET 2024-04-16T18:09:59.053 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:59.053 DEBUG:boto:Data: 2024-04-16T18:09:59.053 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.053 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.053 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.053 DEBUG:boto:Params: {} 2024-04-16T18:09:59.053 DEBUG:boto:Token: None 2024-04-16T18:09:59.053 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.053 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e27b89802f15f972-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.098 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.099 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:59.099 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:59.099 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.099 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:59.099 DEBUG:boto:Data: 2024-04-16T18:09:59.099 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.099 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.099 DEBUG:boto:Params: {} 2024-04-16T18:09:59.099 DEBUG:boto:Token: None 2024-04-16T18:09:59.099 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj4 2024-04-16T18:09:59.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PbKqeG6XpXFHG9XcRPm+8ENDC2E= 2024-04-16T18:09:59.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PbKqeG6XpXFHG9XcRPm+8ENDC2E=', 'Content-Length': '0'} 2024-04-16T18:09:59.102 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000c9b1ee485a62a21-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.103 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:59.103 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:59.103 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.103 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:59.103 DEBUG:boto:Data: 2024-04-16T18:09:59.103 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.103 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.103 DEBUG:boto:Params: {} 2024-04-16T18:09:59.103 DEBUG:boto:Token: None 2024-04-16T18:09:59.103 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj4 2024-04-16T18:09:59.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PbKqeG6XpXFHG9XcRPm+8ENDC2E= 2024-04-16T18:09:59.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PbKqeG6XpXFHG9XcRPm+8ENDC2E=', 'Content-Length': '0'} 2024-04-16T18:09:59.107 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e22b1bbc6faaf277-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.107 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:09:59.107 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:59.107 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:59.107 DEBUG:boto:Method: GET 2024-04-16T18:09:59.107 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:59.108 DEBUG:boto:Data: 2024-04-16T18:09:59.108 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.108 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.108 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.108 DEBUG:boto:Params: {} 2024-04-16T18:09:59.108 DEBUG:boto:Token: None 2024-04-16T18:09:59.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj4 2024-04-16T18:09:59.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1zxUjzj+ScFofbxdykY4tQEeZ4U= 2024-04-16T18:09:59.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1zxUjzj+ScFofbxdykY4tQEeZ4U=', 'Content-Length': '0'} 2024-04-16T18:09:59.111 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000564c469f7a3167b8-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.154 DEBUG:boto:path=/lybtug-38/obj4 2024-04-16T18:09:59.154 DEBUG:boto:auth_path=/lybtug-38/obj4 2024-04-16T18:09:59.155 DEBUG:boto:Method: GET 2024-04-16T18:09:59.155 DEBUG:boto:Path: /lybtug-38/obj4 2024-04-16T18:09:59.155 DEBUG:boto:Data: 2024-04-16T18:09:59.155 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.155 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.155 DEBUG:boto:Params: {} 2024-04-16T18:09:59.155 DEBUG:boto:Token: None 2024-04-16T18:09:59.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj4 2024-04-16T18:09:59.155 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1zxUjzj+ScFofbxdykY4tQEeZ4U= 2024-04-16T18:09:59.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1zxUjzj+ScFofbxdykY4tQEeZ4U=', 'Content-Length': '0'} 2024-04-16T18:09:59.158 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000099399b7cc53a51d8-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.159 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.159 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.159 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.159 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.159 DEBUG:boto:Method: GET 2024-04-16T18:09:59.159 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:59.159 DEBUG:boto:Data: 2024-04-16T18:09:59.159 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.159 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.159 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.160 DEBUG:boto:Params: {} 2024-04-16T18:09:59.160 DEBUG:boto:Token: None 2024-04-16T18:09:59.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e77863cd83f5bb7-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.206 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.207 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.207 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.207 DEBUG:boto:path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.207 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj4 2024-04-16T18:09:59.207 DEBUG:boto:Method: GET 2024-04-16T18:09:59.207 DEBUG:boto:Path: /lybtug-38/?prefix=obj4 2024-04-16T18:09:59.207 DEBUG:boto:Data: 2024-04-16T18:09:59.207 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.207 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.207 DEBUG:boto:Params: {} 2024-04-16T18:09:59.208 DEBUG:boto:Token: None 2024-04-16T18:09:59.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.212 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046427cdfd51fc5c4-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.212 DEBUG:boto:b'lybtug-38obj41000falseobj42024-04-16T18:09:23.641Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.212 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:09:59.212 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.212 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.212 DEBUG:boto:path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.213 DEBUG:boto:auth_path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.213 DEBUG:boto:Method: GET 2024-04-16T18:09:59.213 DEBUG:boto:Path: /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.213 DEBUG:boto:Data: 2024-04-16T18:09:59.213 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.213 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.213 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.213 DEBUG:boto:Params: {} 2024-04-16T18:09:59.213 DEBUG:boto:Token: None 2024-04-16T18:09:59.213 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg= 2024-04-16T18:09:59.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg=', 'Content-Length': '0'} 2024-04-16T18:09:59.217 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000473a70737e70eb8b-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.258 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.258 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.259 DEBUG:boto:path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.259 DEBUG:boto:auth_path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.259 DEBUG:boto:Method: GET 2024-04-16T18:09:59.259 DEBUG:boto:Path: /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.259 DEBUG:boto:Data: 2024-04-16T18:09:59.259 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.259 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.259 DEBUG:boto:Params: {} 2024-04-16T18:09:59.259 DEBUG:boto:Token: None 2024-04-16T18:09:59.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg= 2024-04-16T18:09:59.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg=', 'Content-Length': '0'} 2024-04-16T18:09:59.263 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f1eb95f77bdaca53-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.263 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.264 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.264 DEBUG:boto:path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.264 DEBUG:boto:auth_path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.264 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.264 DEBUG:boto:Path: /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.264 DEBUG:boto:Data: 2024-04-16T18:09:59.264 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.264 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.264 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.264 DEBUG:boto:Params: {} 2024-04-16T18:09:59.264 DEBUG:boto:Token: None 2024-04-16T18:09:59.265 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.265 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I8009o2zPY7iRZkVtLb9jnyV06M= 2024-04-16T18:09:59.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I8009o2zPY7iRZkVtLb9jnyV06M=', 'Content-Length': '0'} 2024-04-16T18:09:59.267 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005780520bf5153bd1-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.268 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.268 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.268 DEBUG:boto:path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.268 DEBUG:boto:auth_path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.268 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.268 DEBUG:boto:Path: /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.268 DEBUG:boto:Data: 2024-04-16T18:09:59.268 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.268 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.268 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.268 DEBUG:boto:Params: {} 2024-04-16T18:09:59.269 DEBUG:boto:Token: None 2024-04-16T18:09:59.269 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.269 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I8009o2zPY7iRZkVtLb9jnyV06M= 2024-04-16T18:09:59.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I8009o2zPY7iRZkVtLb9jnyV06M=', 'Content-Length': '0'} 2024-04-16T18:09:59.272 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000528125f8efc4c033-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.272 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:09:59.272 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.272 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.272 DEBUG:boto:path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.272 DEBUG:boto:auth_path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.272 DEBUG:boto:Method: GET 2024-04-16T18:09:59.273 DEBUG:boto:Path: /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.273 DEBUG:boto:Data: 2024-04-16T18:09:59.273 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.273 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.273 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.273 DEBUG:boto:Params: {} 2024-04-16T18:09:59.273 DEBUG:boto:Token: None 2024-04-16T18:09:59.273 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg= 2024-04-16T18:09:59.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg=', 'Content-Length': '0'} 2024-04-16T18:09:59.276 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001371e35de64528c4-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.318 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.319 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.319 DEBUG:boto:path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.319 DEBUG:boto:auth_path=/lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.319 DEBUG:boto:Method: GET 2024-04-16T18:09:59.319 DEBUG:boto:Path: /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.319 DEBUG:boto:Data: 2024-04-16T18:09:59.319 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.319 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.319 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.319 DEBUG:boto:Params: {} 2024-04-16T18:09:59.319 DEBUG:boto:Token: None 2024-04-16T18:09:59.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5?versionId=null 2024-04-16T18:09:59.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg= 2024-04-16T18:09:59.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qi3Ilwl4UNCSvZ282DWq5KxQeRg=', 'Content-Length': '0'} 2024-04-16T18:09:59.323 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ee366d57b5b32845-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.323 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.324 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.324 DEBUG:boto:path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.324 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.324 DEBUG:boto:Method: GET 2024-04-16T18:09:59.324 DEBUG:boto:Path: /lybtug-38/?prefix=obj5 2024-04-16T18:09:59.324 DEBUG:boto:Data: 2024-04-16T18:09:59.324 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.324 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.324 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.324 DEBUG:boto:Params: {} 2024-04-16T18:09:59.324 DEBUG:boto:Token: None 2024-04-16T18:09:59.324 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.325 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000182257ba0cd1e42b-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.328 DEBUG:boto:b'lybtug-38obj51000falseobj52024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.329 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.329 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.329 DEBUG:boto:path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.329 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.329 DEBUG:boto:Method: GET 2024-04-16T18:09:59.329 DEBUG:boto:Path: /lybtug-38/?prefix=obj5 2024-04-16T18:09:59.329 DEBUG:boto:Data: 2024-04-16T18:09:59.329 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.329 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.329 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.329 DEBUG:boto:Params: {} 2024-04-16T18:09:59.329 DEBUG:boto:Token: None 2024-04-16T18:09:59.330 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.330 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.333 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0f1db0649011a9a-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.374 DEBUG:boto:b'lybtug-38obj51000falseobj52024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.375 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.375 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.375 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.375 DEBUG:boto:Path: /lybtug-38/obj5 2024-04-16T18:09:59.375 DEBUG:boto:Data: 2024-04-16T18:09:59.375 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.375 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.375 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.375 DEBUG:boto:Params: {} 2024-04-16T18:09:59.375 DEBUG:boto:Token: None 2024-04-16T18:09:59.375 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5 2024-04-16T18:09:59.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aIqdRS6QVSTFLqQpQp2ddfnHFo4= 2024-04-16T18:09:59.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aIqdRS6QVSTFLqQpQp2ddfnHFo4=', 'Content-Length': '0'} 2024-04-16T18:09:59.378 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002a5fa1d3af74b065-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.379 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.379 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.379 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.379 DEBUG:boto:Path: /lybtug-38/obj5 2024-04-16T18:09:59.379 DEBUG:boto:Data: 2024-04-16T18:09:59.379 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.379 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.379 DEBUG:boto:Params: {} 2024-04-16T18:09:59.379 DEBUG:boto:Token: None 2024-04-16T18:09:59.379 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5 2024-04-16T18:09:59.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aIqdRS6QVSTFLqQpQp2ddfnHFo4= 2024-04-16T18:09:59.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aIqdRS6QVSTFLqQpQp2ddfnHFo4=', 'Content-Length': '0'} 2024-04-16T18:09:59.383 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c41882b639fc443d-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.383 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:09:59.383 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.383 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.383 DEBUG:boto:Method: GET 2024-04-16T18:09:59.383 DEBUG:boto:Path: /lybtug-38/obj5 2024-04-16T18:09:59.383 DEBUG:boto:Data: 2024-04-16T18:09:59.384 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.384 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.384 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.384 DEBUG:boto:Params: {} 2024-04-16T18:09:59.384 DEBUG:boto:Token: None 2024-04-16T18:09:59.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5 2024-04-16T18:09:59.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C/Yjgz65x3tNoiOXjUBCuDKK2pU= 2024-04-16T18:09:59.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C/Yjgz65x3tNoiOXjUBCuDKK2pU=', 'Content-Length': '0'} 2024-04-16T18:09:59.387 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000eda4dbb18e171bb4-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.430 DEBUG:boto:path=/lybtug-38/obj5 2024-04-16T18:09:59.431 DEBUG:boto:auth_path=/lybtug-38/obj5 2024-04-16T18:09:59.431 DEBUG:boto:Method: GET 2024-04-16T18:09:59.431 DEBUG:boto:Path: /lybtug-38/obj5 2024-04-16T18:09:59.431 DEBUG:boto:Data: 2024-04-16T18:09:59.431 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.431 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.431 DEBUG:boto:Params: {} 2024-04-16T18:09:59.431 DEBUG:boto:Token: None 2024-04-16T18:09:59.431 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj5 2024-04-16T18:09:59.431 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C/Yjgz65x3tNoiOXjUBCuDKK2pU= 2024-04-16T18:09:59.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C/Yjgz65x3tNoiOXjUBCuDKK2pU=', 'Content-Length': '0'} 2024-04-16T18:09:59.435 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000153342db5c07fd58-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.435 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.435 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.435 DEBUG:boto:path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.435 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.435 DEBUG:boto:Method: GET 2024-04-16T18:09:59.436 DEBUG:boto:Path: /lybtug-38/?prefix=obj5 2024-04-16T18:09:59.436 DEBUG:boto:Data: 2024-04-16T18:09:59.436 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.436 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.436 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.436 DEBUG:boto:Params: {} 2024-04-16T18:09:59.436 DEBUG:boto:Token: None 2024-04-16T18:09:59.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8e2c6381cff52bb-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.482 DEBUG:boto:b'lybtug-38obj51000falseobj52024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.483 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.483 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.483 DEBUG:boto:path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.483 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj5 2024-04-16T18:09:59.483 DEBUG:boto:Method: GET 2024-04-16T18:09:59.483 DEBUG:boto:Path: /lybtug-38/?prefix=obj5 2024-04-16T18:09:59.483 DEBUG:boto:Data: 2024-04-16T18:09:59.483 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.484 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.484 DEBUG:boto:Params: {} 2024-04-16T18:09:59.484 DEBUG:boto:Token: None 2024-04-16T18:09:59.484 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002873e5247fc296d7-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.488 DEBUG:boto:b'lybtug-38obj51000falseobj52024-04-16T18:09:37.506Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.488 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:09:59.488 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.488 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.488 DEBUG:boto:path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.489 DEBUG:boto:auth_path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.489 DEBUG:boto:Method: GET 2024-04-16T18:09:59.489 DEBUG:boto:Path: /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.489 DEBUG:boto:Data: 2024-04-16T18:09:59.489 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.489 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.489 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.489 DEBUG:boto:Params: {} 2024-04-16T18:09:59.489 DEBUG:boto:Token: None 2024-04-16T18:09:59.489 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow= 2024-04-16T18:09:59.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow=', 'Content-Length': '0'} 2024-04-16T18:09:59.493 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c10979d0a6163e43-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.534 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.534 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.535 DEBUG:boto:path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.535 DEBUG:boto:auth_path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.535 DEBUG:boto:Method: GET 2024-04-16T18:09:59.535 DEBUG:boto:Path: /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.535 DEBUG:boto:Data: 2024-04-16T18:09:59.535 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.535 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.535 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.535 DEBUG:boto:Params: {} 2024-04-16T18:09:59.535 DEBUG:boto:Token: None 2024-04-16T18:09:59.535 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.535 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow= 2024-04-16T18:09:59.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow=', 'Content-Length': '0'} 2024-04-16T18:09:59.538 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006e2ad318c26847d3-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.538 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.539 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.539 DEBUG:boto:path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.539 DEBUG:boto:auth_path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.539 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.539 DEBUG:boto:Path: /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.539 DEBUG:boto:Data: 2024-04-16T18:09:59.539 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.539 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.539 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.539 DEBUG:boto:Params: {} 2024-04-16T18:09:59.539 DEBUG:boto:Token: None 2024-04-16T18:09:59.540 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yo2vHGg7bQoA6xG79syVzXh5nGg= 2024-04-16T18:09:59.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yo2vHGg7bQoA6xG79syVzXh5nGg=', 'Content-Length': '0'} 2024-04-16T18:09:59.542 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000d22be31935e9dee-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.543 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.543 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.543 DEBUG:boto:path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.543 DEBUG:boto:auth_path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.543 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.543 DEBUG:boto:Path: /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.543 DEBUG:boto:Data: 2024-04-16T18:09:59.543 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.544 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.544 DEBUG:boto:Params: {} 2024-04-16T18:09:59.544 DEBUG:boto:Token: None 2024-04-16T18:09:59.544 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yo2vHGg7bQoA6xG79syVzXh5nGg= 2024-04-16T18:09:59.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yo2vHGg7bQoA6xG79syVzXh5nGg=', 'Content-Length': '0'} 2024-04-16T18:09:59.547 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d87f12001af4a28e-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.547 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:09:59.547 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.547 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.547 DEBUG:boto:path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.547 DEBUG:boto:auth_path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.547 DEBUG:boto:Method: GET 2024-04-16T18:09:59.547 DEBUG:boto:Path: /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.548 DEBUG:boto:Data: 2024-04-16T18:09:59.548 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.548 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.548 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.548 DEBUG:boto:Params: {} 2024-04-16T18:09:59.548 DEBUG:boto:Token: None 2024-04-16T18:09:59.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow= 2024-04-16T18:09:59.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow=', 'Content-Length': '0'} 2024-04-16T18:09:59.551 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005e85b4f73e425d98-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.594 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.594 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.595 DEBUG:boto:path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.595 DEBUG:boto:auth_path=/lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.595 DEBUG:boto:Method: GET 2024-04-16T18:09:59.595 DEBUG:boto:Path: /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.595 DEBUG:boto:Data: 2024-04-16T18:09:59.595 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.595 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.595 DEBUG:boto:Params: {} 2024-04-16T18:09:59.595 DEBUG:boto:Token: None 2024-04-16T18:09:59.595 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6?versionId=null 2024-04-16T18:09:59.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow= 2024-04-16T18:09:59.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6N81MXW6Nwoy882wXfiIwu6W2ow=', 'Content-Length': '0'} 2024-04-16T18:09:59.598 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d83fbd3b20459401-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.599 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.599 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.599 DEBUG:boto:path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.599 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.599 DEBUG:boto:Method: GET 2024-04-16T18:09:59.599 DEBUG:boto:Path: /lybtug-38/?prefix=obj6 2024-04-16T18:09:59.599 DEBUG:boto:Data: 2024-04-16T18:09:59.599 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.599 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.599 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.599 DEBUG:boto:Params: {} 2024-04-16T18:09:59.600 DEBUG:boto:Token: None 2024-04-16T18:09:59.600 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.603 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032b14c43e1478ef3-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.604 DEBUG:boto:b'lybtug-38obj61000falseobj62024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.604 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.604 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.604 DEBUG:boto:path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.604 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.604 DEBUG:boto:Method: GET 2024-04-16T18:09:59.604 DEBUG:boto:Path: /lybtug-38/?prefix=obj6 2024-04-16T18:09:59.604 DEBUG:boto:Data: 2024-04-16T18:09:59.604 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.605 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.605 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.605 DEBUG:boto:Params: {} 2024-04-16T18:09:59.605 DEBUG:boto:Token: None 2024-04-16T18:09:59.605 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006303a5b2fcda2c8c-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.650 DEBUG:boto:b'lybtug-38obj61000falseobj62024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.651 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.651 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.651 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.651 DEBUG:boto:Path: /lybtug-38/obj6 2024-04-16T18:09:59.651 DEBUG:boto:Data: 2024-04-16T18:09:59.651 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.651 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.651 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.651 DEBUG:boto:Params: {} 2024-04-16T18:09:59.651 DEBUG:boto:Token: None 2024-04-16T18:09:59.651 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6 2024-04-16T18:09:59.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:msx2VSzczaJJBYANpJy1aop6lEU= 2024-04-16T18:09:59.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:msx2VSzczaJJBYANpJy1aop6lEU=', 'Content-Length': '0'} 2024-04-16T18:09:59.654 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f065cf9116a8fba2-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.654 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.655 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.655 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.655 DEBUG:boto:Path: /lybtug-38/obj6 2024-04-16T18:09:59.655 DEBUG:boto:Data: 2024-04-16T18:09:59.655 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.655 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.655 DEBUG:boto:Params: {} 2024-04-16T18:09:59.655 DEBUG:boto:Token: None 2024-04-16T18:09:59.655 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6 2024-04-16T18:09:59.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:msx2VSzczaJJBYANpJy1aop6lEU= 2024-04-16T18:09:59.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:msx2VSzczaJJBYANpJy1aop6lEU=', 'Content-Length': '0'} 2024-04-16T18:09:59.658 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003aa4912d29dcff1a-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.659 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:09:59.659 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.659 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.659 DEBUG:boto:Method: GET 2024-04-16T18:09:59.659 DEBUG:boto:Path: /lybtug-38/obj6 2024-04-16T18:09:59.659 DEBUG:boto:Data: 2024-04-16T18:09:59.659 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.659 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.659 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.659 DEBUG:boto:Params: {} 2024-04-16T18:09:59.660 DEBUG:boto:Token: None 2024-04-16T18:09:59.660 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6 2024-04-16T18:09:59.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OpBODb/XvrR29NjXhkpsGEmEB0k= 2024-04-16T18:09:59.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OpBODb/XvrR29NjXhkpsGEmEB0k=', 'Content-Length': '0'} 2024-04-16T18:09:59.663 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009aa2566e4529f199-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.706 DEBUG:boto:path=/lybtug-38/obj6 2024-04-16T18:09:59.707 DEBUG:boto:auth_path=/lybtug-38/obj6 2024-04-16T18:09:59.707 DEBUG:boto:Method: GET 2024-04-16T18:09:59.707 DEBUG:boto:Path: /lybtug-38/obj6 2024-04-16T18:09:59.707 DEBUG:boto:Data: 2024-04-16T18:09:59.707 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.707 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.707 DEBUG:boto:Params: {} 2024-04-16T18:09:59.707 DEBUG:boto:Token: None 2024-04-16T18:09:59.707 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj6 2024-04-16T18:09:59.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OpBODb/XvrR29NjXhkpsGEmEB0k= 2024-04-16T18:09:59.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OpBODb/XvrR29NjXhkpsGEmEB0k=', 'Content-Length': '0'} 2024-04-16T18:09:59.710 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d747b1c66ab56430-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.711 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.711 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.711 DEBUG:boto:path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.711 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.711 DEBUG:boto:Method: GET 2024-04-16T18:09:59.711 DEBUG:boto:Path: /lybtug-38/?prefix=obj6 2024-04-16T18:09:59.711 DEBUG:boto:Data: 2024-04-16T18:09:59.711 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.712 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.712 DEBUG:boto:Params: {} 2024-04-16T18:09:59.712 DEBUG:boto:Token: None 2024-04-16T18:09:59.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e0098ee7a6ea0dd-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.758 DEBUG:boto:b'lybtug-38obj61000falseobj62024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.759 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.759 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.759 DEBUG:boto:path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.759 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj6 2024-04-16T18:09:59.759 DEBUG:boto:Method: GET 2024-04-16T18:09:59.759 DEBUG:boto:Path: /lybtug-38/?prefix=obj6 2024-04-16T18:09:59.759 DEBUG:boto:Data: 2024-04-16T18:09:59.759 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.759 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.759 DEBUG:boto:Params: {} 2024-04-16T18:09:59.760 DEBUG:boto:Token: None 2024-04-16T18:09:59.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4097805e32a8998-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.764 DEBUG:boto:b'lybtug-38obj61000falseobj62024-04-16T18:09:37.517Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:09:59.764 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.764 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.764 DEBUG:boto:path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.765 DEBUG:boto:auth_path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.765 DEBUG:boto:Method: GET 2024-04-16T18:09:59.765 DEBUG:boto:Path: /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.765 DEBUG:boto:Data: 2024-04-16T18:09:59.765 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.765 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.765 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.765 DEBUG:boto:Params: {} 2024-04-16T18:09:59.765 DEBUG:boto:Token: None 2024-04-16T18:09:59.765 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.766 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg= 2024-04-16T18:09:59.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg=', 'Content-Length': '0'} 2024-04-16T18:09:59.769 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000096cdef03f8d88962-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.810 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.810 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.811 DEBUG:boto:path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.811 DEBUG:boto:auth_path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.811 DEBUG:boto:Method: GET 2024-04-16T18:09:59.811 DEBUG:boto:Path: /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.811 DEBUG:boto:Data: 2024-04-16T18:09:59.811 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.811 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.811 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.811 DEBUG:boto:Params: {} 2024-04-16T18:09:59.811 DEBUG:boto:Token: None 2024-04-16T18:09:59.811 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg= 2024-04-16T18:09:59.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg=', 'Content-Length': '0'} 2024-04-16T18:09:59.814 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000185467acb973d99d-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.815 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.815 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.815 DEBUG:boto:path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.815 DEBUG:boto:auth_path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.815 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.815 DEBUG:boto:Path: /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.815 DEBUG:boto:Data: 2024-04-16T18:09:59.815 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.815 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.815 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.816 DEBUG:boto:Params: {} 2024-04-16T18:09:59.816 DEBUG:boto:Token: None 2024-04-16T18:09:59.816 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I4Vb5er+ok4qE1YywYnqQkl4QOo= 2024-04-16T18:09:59.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I4Vb5er+ok4qE1YywYnqQkl4QOo=', 'Content-Length': '0'} 2024-04-16T18:09:59.819 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000805e13953f38f5a-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.819 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.820 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.820 DEBUG:boto:path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.820 DEBUG:boto:auth_path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.820 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.820 DEBUG:boto:Path: /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.820 DEBUG:boto:Data: 2024-04-16T18:09:59.820 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.820 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.820 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.820 DEBUG:boto:Params: {} 2024-04-16T18:09:59.820 DEBUG:boto:Token: None 2024-04-16T18:09:59.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I4Vb5er+ok4qE1YywYnqQkl4QOo= 2024-04-16T18:09:59.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I4Vb5er+ok4qE1YywYnqQkl4QOo=', 'Content-Length': '0'} 2024-04-16T18:09:59.824 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002080296afa3e1cea-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.824 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:09:59.824 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.824 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.824 DEBUG:boto:path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.824 DEBUG:boto:auth_path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.824 DEBUG:boto:Method: GET 2024-04-16T18:09:59.824 DEBUG:boto:Path: /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.824 DEBUG:boto:Data: 2024-04-16T18:09:59.824 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.825 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.825 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.825 DEBUG:boto:Params: {} 2024-04-16T18:09:59.825 DEBUG:boto:Token: None 2024-04-16T18:09:59.825 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg= 2024-04-16T18:09:59.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg=', 'Content-Length': '0'} 2024-04-16T18:09:59.828 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000a02566d96e75011-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.870 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.871 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.871 DEBUG:boto:path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.871 DEBUG:boto:auth_path=/lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.871 DEBUG:boto:Method: GET 2024-04-16T18:09:59.871 DEBUG:boto:Path: /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.871 DEBUG:boto:Data: 2024-04-16T18:09:59.871 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.871 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.871 DEBUG:boto:Params: {} 2024-04-16T18:09:59.871 DEBUG:boto:Token: None 2024-04-16T18:09:59.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7?versionId=null 2024-04-16T18:09:59.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg= 2024-04-16T18:09:59.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IErwCNtKZbXZBWj7ab7/sVE3Zgg=', 'Content-Length': '0'} 2024-04-16T18:09:59.875 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008da4451980da62b3-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.875 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.875 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.875 DEBUG:boto:path=/lybtug-38/?prefix=obj7 2024-04-16T18:09:59.875 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj7 2024-04-16T18:09:59.875 DEBUG:boto:Method: GET 2024-04-16T18:09:59.875 DEBUG:boto:Path: /lybtug-38/?prefix=obj7 2024-04-16T18:09:59.876 DEBUG:boto:Data: 2024-04-16T18:09:59.876 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.876 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.876 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.876 DEBUG:boto:Params: {} 2024-04-16T18:09:59.876 DEBUG:boto:Token: None 2024-04-16T18:09:59.876 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.880 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000bcd97749deeb2ee-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.880 DEBUG:boto:b'lybtug-38obj71000falseobj72024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.880 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.881 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.881 DEBUG:boto:path=/lybtug-38/?prefix=obj7 2024-04-16T18:09:59.881 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj7 2024-04-16T18:09:59.881 DEBUG:boto:Method: GET 2024-04-16T18:09:59.881 DEBUG:boto:Path: /lybtug-38/?prefix=obj7 2024-04-16T18:09:59.881 DEBUG:boto:Data: 2024-04-16T18:09:59.881 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.881 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.881 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.881 DEBUG:boto:Params: {} 2024-04-16T18:09:59.881 DEBUG:boto:Token: None 2024-04-16T18:09:59.882 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.882 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087448177a80fe955-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.926 DEBUG:boto:b'lybtug-38obj71000falseobj72024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:09:59.927 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.927 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.927 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.927 DEBUG:boto:Path: /lybtug-38/obj7 2024-04-16T18:09:59.927 DEBUG:boto:Data: 2024-04-16T18:09:59.927 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.927 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.927 DEBUG:boto:Params: {} 2024-04-16T18:09:59.927 DEBUG:boto:Token: None 2024-04-16T18:09:59.928 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7 2024-04-16T18:09:59.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+QgcDmun5dt8UyCHz8Y7Lma5G70= 2024-04-16T18:09:59.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+QgcDmun5dt8UyCHz8Y7Lma5G70=', 'Content-Length': '0'} 2024-04-16T18:09:59.930 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dec49efb82341c30-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.930 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.931 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.931 DEBUG:boto:Method: HEAD 2024-04-16T18:09:59.931 DEBUG:boto:Path: /lybtug-38/obj7 2024-04-16T18:09:59.931 DEBUG:boto:Data: 2024-04-16T18:09:59.931 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.931 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.931 DEBUG:boto:Params: {} 2024-04-16T18:09:59.931 DEBUG:boto:Token: None 2024-04-16T18:09:59.931 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7 2024-04-16T18:09:59.931 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+QgcDmun5dt8UyCHz8Y7Lma5G70= 2024-04-16T18:09:59.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+QgcDmun5dt8UyCHz8Y7Lma5G70=', 'Content-Length': '0'} 2024-04-16T18:09:59.934 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000060283f74e41ca00c-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:09:59.935 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.935 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.935 DEBUG:boto:Method: GET 2024-04-16T18:09:59.935 DEBUG:boto:Path: /lybtug-38/obj7 2024-04-16T18:09:59.935 DEBUG:boto:Data: 2024-04-16T18:09:59.935 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.935 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.935 DEBUG:boto:Params: {} 2024-04-16T18:09:59.935 DEBUG:boto:Token: None 2024-04-16T18:09:59.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7 2024-04-16T18:09:59.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Tn7O9ioN2mQQCxcuZ/euvnV3NQ8= 2024-04-16T18:09:59.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Tn7O9ioN2mQQCxcuZ/euvnV3NQ8=', 'Content-Length': '0'} 2024-04-16T18:09:59.939 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:38 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000017a9cf1b3c114a5f-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.982 DEBUG:boto:path=/lybtug-38/obj7 2024-04-16T18:09:59.982 DEBUG:boto:auth_path=/lybtug-38/obj7 2024-04-16T18:09:59.983 DEBUG:boto:Method: GET 2024-04-16T18:09:59.983 DEBUG:boto:Path: /lybtug-38/obj7 2024-04-16T18:09:59.983 DEBUG:boto:Data: 2024-04-16T18:09:59.983 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:09:59.983 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.983 DEBUG:boto:Params: {} 2024-04-16T18:09:59.983 DEBUG:boto:Token: None 2024-04-16T18:09:59.983 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/obj7 2024-04-16T18:09:59.983 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Tn7O9ioN2mQQCxcuZ/euvnV3NQ8= 2024-04-16T18:09:59.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Tn7O9ioN2mQQCxcuZ/euvnV3NQ8=', 'Content-Length': '0'} 2024-04-16T18:09:59.986 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ab5a4c50bd29fdb8-00661ebef7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:09:59.986 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:09:59.986 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:09:59.987 DEBUG:boto:path=/lybtug-38/?prefix=obj7 2024-04-16T18:09:59.987 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj7 2024-04-16T18:09:59.987 DEBUG:boto:Method: GET 2024-04-16T18:09:59.987 DEBUG:boto:Path: /lybtug-38/?prefix=obj7 2024-04-16T18:09:59.987 DEBUG:boto:Data: 2024-04-16T18:09:59.987 DEBUG:boto:Headers: {} 2024-04-16T18:09:59.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:09:59.987 DEBUG:boto:Port: 8000 2024-04-16T18:09:59.987 DEBUG:boto:Params: {} 2024-04-16T18:09:59.987 DEBUG:boto:Token: None 2024-04-16T18:09:59.987 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:09:59 GMT /lybtug-38/ 2024-04-16T18:09:59.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg= 2024-04-16T18:09:59.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:09:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hp86rshJNIm+6tL1mmSxmJix1bg=', 'Content-Length': '0'} 2024-04-16T18:09:59.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055d7fc89d6aafc8a-00661ebef7-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:09:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.034 DEBUG:boto:b'lybtug-38obj71000falseobj72024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:00.035 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:10:00.035 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:10:00.035 DEBUG:boto:path=/lybtug-38/?prefix=obj7 2024-04-16T18:10:00.035 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj7 2024-04-16T18:10:00.035 DEBUG:boto:Method: GET 2024-04-16T18:10:00.035 DEBUG:boto:Path: /lybtug-38/?prefix=obj7 2024-04-16T18:10:00.035 DEBUG:boto:Data: 2024-04-16T18:10:00.035 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.035 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.035 DEBUG:boto:Params: {} 2024-04-16T18:10:00.036 DEBUG:boto:Token: None 2024-04-16T18:10:00.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/ 2024-04-16T18:10:00.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0= 2024-04-16T18:10:00.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0=', 'Content-Length': '0'} 2024-04-16T18:10:00.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061791892a495be12-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.040 DEBUG:boto:b'lybtug-38obj71000falseobj72024-04-16T18:09:37.568Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:00.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:00.041 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.041 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.041 DEBUG:boto:path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.041 DEBUG:boto:auth_path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.041 DEBUG:boto:Method: GET 2024-04-16T18:10:00.041 DEBUG:boto:Path: /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.041 DEBUG:boto:Data: 2024-04-16T18:10:00.041 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.041 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.042 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.042 DEBUG:boto:Params: {} 2024-04-16T18:10:00.042 DEBUG:boto:Token: None 2024-04-16T18:10:00.042 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc= 2024-04-16T18:10:00.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc=', 'Content-Length': '0'} 2024-04-16T18:10:00.045 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b785abaa737ea838-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.086 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.086 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.086 DEBUG:boto:path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.087 DEBUG:boto:auth_path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.087 DEBUG:boto:Method: GET 2024-04-16T18:10:00.087 DEBUG:boto:Path: /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.087 DEBUG:boto:Data: 2024-04-16T18:10:00.087 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.087 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.087 DEBUG:boto:Params: {} 2024-04-16T18:10:00.087 DEBUG:boto:Token: None 2024-04-16T18:10:00.087 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.087 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc= 2024-04-16T18:10:00.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc=', 'Content-Length': '0'} 2024-04-16T18:10:00.090 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008d7dcace45266638-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.091 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.091 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.091 DEBUG:boto:path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.091 DEBUG:boto:auth_path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.091 DEBUG:boto:Method: HEAD 2024-04-16T18:10:00.091 DEBUG:boto:Path: /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.091 DEBUG:boto:Data: 2024-04-16T18:10:00.091 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.091 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.092 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.092 DEBUG:boto:Params: {} 2024-04-16T18:10:00.092 DEBUG:boto:Token: None 2024-04-16T18:10:00.092 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wadgznasl2/hHD9jUXF2QJVYG0w= 2024-04-16T18:10:00.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wadgznasl2/hHD9jUXF2QJVYG0w=', 'Content-Length': '0'} 2024-04-16T18:10:00.095 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000326bc15cb71932e6-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.095 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.095 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.095 DEBUG:boto:path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.095 DEBUG:boto:auth_path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.095 DEBUG:boto:Method: HEAD 2024-04-16T18:10:00.095 DEBUG:boto:Path: /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.095 DEBUG:boto:Data: 2024-04-16T18:10:00.096 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.096 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.096 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.096 DEBUG:boto:Params: {} 2024-04-16T18:10:00.096 DEBUG:boto:Token: None 2024-04-16T18:10:00.096 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wadgznasl2/hHD9jUXF2QJVYG0w= 2024-04-16T18:10:00.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wadgznasl2/hHD9jUXF2QJVYG0w=', 'Content-Length': '0'} 2024-04-16T18:10:00.099 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002350ed629edead7e-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.100 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:00.100 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.100 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.100 DEBUG:boto:path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.100 DEBUG:boto:auth_path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.100 DEBUG:boto:Method: GET 2024-04-16T18:10:00.100 DEBUG:boto:Path: /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.100 DEBUG:boto:Data: 2024-04-16T18:10:00.100 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.100 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.100 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.101 DEBUG:boto:Params: {} 2024-04-16T18:10:00.101 DEBUG:boto:Token: None 2024-04-16T18:10:00.101 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.101 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc= 2024-04-16T18:10:00.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc=', 'Content-Length': '0'} 2024-04-16T18:10:00.104 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000071d7d7f05be63a09-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.146 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.146 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.146 DEBUG:boto:path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.147 DEBUG:boto:auth_path=/lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.147 DEBUG:boto:Method: GET 2024-04-16T18:10:00.147 DEBUG:boto:Path: /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.147 DEBUG:boto:Data: 2024-04-16T18:10:00.147 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.147 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.147 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.147 DEBUG:boto:Params: {} 2024-04-16T18:10:00.147 DEBUG:boto:Token: None 2024-04-16T18:10:00.147 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8?versionId=null 2024-04-16T18:10:00.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc= 2024-04-16T18:10:00.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OUnVus/rF6AXWHHytkq7YTGzBgc=', 'Content-Length': '0'} 2024-04-16T18:10:00.151 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003494d56c5c3cf018-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.151 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:10:00.151 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:10:00.151 DEBUG:boto:path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.151 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.151 DEBUG:boto:Method: GET 2024-04-16T18:10:00.151 DEBUG:boto:Path: /lybtug-38/?prefix=obj8 2024-04-16T18:10:00.151 DEBUG:boto:Data: 2024-04-16T18:10:00.152 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.152 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.152 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.152 DEBUG:boto:Params: {} 2024-04-16T18:10:00.152 DEBUG:boto:Token: None 2024-04-16T18:10:00.152 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/ 2024-04-16T18:10:00.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0= 2024-04-16T18:10:00.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0=', 'Content-Length': '0'} 2024-04-16T18:10:00.155 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a89ff53f18bedb5d-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.156 DEBUG:boto:b'lybtug-38obj81000falseobj82024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:00.156 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:10:00.156 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:10:00.156 DEBUG:boto:path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.156 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.156 DEBUG:boto:Method: GET 2024-04-16T18:10:00.156 DEBUG:boto:Path: /lybtug-38/?prefix=obj8 2024-04-16T18:10:00.156 DEBUG:boto:Data: 2024-04-16T18:10:00.157 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.157 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.157 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.157 DEBUG:boto:Params: {} 2024-04-16T18:10:00.157 DEBUG:boto:Token: None 2024-04-16T18:10:00.157 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/ 2024-04-16T18:10:00.157 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0= 2024-04-16T18:10:00.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0=', 'Content-Length': '0'} 2024-04-16T18:10:00.161 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b3a9a1a9bcd1cdb-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.202 DEBUG:boto:b'lybtug-38obj81000falseobj82024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:00.203 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.203 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.203 DEBUG:boto:Method: HEAD 2024-04-16T18:10:00.203 DEBUG:boto:Path: /lybtug-38/obj8 2024-04-16T18:10:00.203 DEBUG:boto:Data: 2024-04-16T18:10:00.203 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.203 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.203 DEBUG:boto:Params: {} 2024-04-16T18:10:00.203 DEBUG:boto:Token: None 2024-04-16T18:10:00.203 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8 2024-04-16T18:10:00.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bMYnZhSKgWN2q8p4NFTLRGO6FqM= 2024-04-16T18:10:00.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bMYnZhSKgWN2q8p4NFTLRGO6FqM=', 'Content-Length': '0'} 2024-04-16T18:10:00.207 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007890d4fdf6dd4f55-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.207 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.207 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.207 DEBUG:boto:Method: HEAD 2024-04-16T18:10:00.207 DEBUG:boto:Path: /lybtug-38/obj8 2024-04-16T18:10:00.207 DEBUG:boto:Data: 2024-04-16T18:10:00.207 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.208 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.208 DEBUG:boto:Params: {} 2024-04-16T18:10:00.208 DEBUG:boto:Token: None 2024-04-16T18:10:00.208 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8 2024-04-16T18:10:00.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bMYnZhSKgWN2q8p4NFTLRGO6FqM= 2024-04-16T18:10:00.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bMYnZhSKgWN2q8p4NFTLRGO6FqM=', 'Content-Length': '0'} 2024-04-16T18:10:00.211 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a5b1f4fa75cfb70f-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.211 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:00.211 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.212 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.212 DEBUG:boto:Method: GET 2024-04-16T18:10:00.212 DEBUG:boto:Path: /lybtug-38/obj8 2024-04-16T18:10:00.212 DEBUG:boto:Data: 2024-04-16T18:10:00.212 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.212 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.212 DEBUG:boto:Params: {} 2024-04-16T18:10:00.212 DEBUG:boto:Token: None 2024-04-16T18:10:00.212 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8 2024-04-16T18:10:00.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TiwEBD+HPnHvk5jRDrho+9ISins= 2024-04-16T18:10:00.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TiwEBD+HPnHvk5jRDrho+9ISins=', 'Content-Length': '0'} 2024-04-16T18:10:00.216 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-38:054102a3-8d94-4596-8b3b-6341eab312e1.4804.7'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d68155beb046f2ba-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.258 DEBUG:boto:path=/lybtug-38/obj8 2024-04-16T18:10:00.259 DEBUG:boto:auth_path=/lybtug-38/obj8 2024-04-16T18:10:00.259 DEBUG:boto:Method: GET 2024-04-16T18:10:00.259 DEBUG:boto:Path: /lybtug-38/obj8 2024-04-16T18:10:00.259 DEBUG:boto:Data: 2024-04-16T18:10:00.259 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.259 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.259 DEBUG:boto:Params: {} 2024-04-16T18:10:00.259 DEBUG:boto:Token: None 2024-04-16T18:10:00.259 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/obj8 2024-04-16T18:10:00.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TiwEBD+HPnHvk5jRDrho+9ISins= 2024-04-16T18:10:00.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TiwEBD+HPnHvk5jRDrho+9ISins=', 'Content-Length': '0'} 2024-04-16T18:10:00.263 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a9147b21daf076cd-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.263 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:10:00.263 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:10:00.263 DEBUG:boto:path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.263 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.263 DEBUG:boto:Method: GET 2024-04-16T18:10:00.263 DEBUG:boto:Path: /lybtug-38/?prefix=obj8 2024-04-16T18:10:00.263 DEBUG:boto:Data: 2024-04-16T18:10:00.264 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.264 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:00.264 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.264 DEBUG:boto:Params: {} 2024-04-16T18:10:00.264 DEBUG:boto:Token: None 2024-04-16T18:10:00.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/ 2024-04-16T18:10:00.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0= 2024-04-16T18:10:00.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0=', 'Content-Length': '0'} 2024-04-16T18:10:00.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c0b46ff6097a2c5-00661ebef8-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.310 DEBUG:boto:b'lybtug-38obj81000falseobj82024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:00.311 DEBUG:boto:path=/lybtug-38/ 2024-04-16T18:10:00.311 DEBUG:boto:auth_path=/lybtug-38/ 2024-04-16T18:10:00.311 DEBUG:boto:path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.311 DEBUG:boto:auth_path=/lybtug-38/?prefix=obj8 2024-04-16T18:10:00.311 DEBUG:boto:Method: GET 2024-04-16T18:10:00.311 DEBUG:boto:Path: /lybtug-38/?prefix=obj8 2024-04-16T18:10:00.311 DEBUG:boto:Data: 2024-04-16T18:10:00.311 DEBUG:boto:Headers: {} 2024-04-16T18:10:00.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:00.311 DEBUG:boto:Port: 8000 2024-04-16T18:10:00.311 DEBUG:boto:Params: {} 2024-04-16T18:10:00.312 DEBUG:boto:Token: None 2024-04-16T18:10:00.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:00 GMT /lybtug-38/ 2024-04-16T18:10:00.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0= 2024-04-16T18:10:00.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4OL4i5AoRiRTBOIKGW7RhA66wJ0=', 'Content-Length': '0'} 2024-04-16T18:10:00.316 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078a956640829e8ea-00661ebef8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:00.316 DEBUG:boto:b'lybtug-38obj81000falseobj82024-04-16T18:09:37.583Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:00.316 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-38 zones={test-zone2, test-zone1} 2024-04-16T18:10:00.316 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-39 2024-04-16T18:10:00.316 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:00.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:00.317 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-39 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:00.368 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:00.368 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:00.456+0000 7f5364e43980 1 waiting for incremental sync to catch up: 2024-04-16T18:10:00.449 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.33.1, 00000000002.23.1, 00000000003.32.1, 00000000002.35.1, 00000000003.96.1, 00000000002.39.1, 00000000002.20.1, 00000000003.27.1, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:00.449 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.35.6, 00000000002.23.1, 00000000004.34.6, 00000000002.35.1, 00000000004.98.6, 00000000002.39.1, 00000000002.20.1, 00000000004.29.6, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:05.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:05.456+0000 7f5364e43980 1 waiting for incremental sync to catch up: 2024-04-16T18:10:05.452 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.33.1, 00000000002.23.1, 00000000003.32.1, 00000000002.35.1, 00000000003.96.1, 00000000002.39.1, 00000000002.20.1, 00000000003.27.1, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:05.452 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.35.6, 00000000002.23.1, 00000000004.34.6, 00000000002.35.1, 00000000004.98.6, 00000000002.39.1, 00000000002.20.1, 00000000004.29.6, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:10.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:10.460+0000 7f5364e43980 1 waiting for incremental sync to catch up: 2024-04-16T18:10:10.454 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.33.1, 00000000002.23.1, 00000000003.32.1, 00000000002.35.1, 00000000003.96.1, 00000000002.39.1, 00000000002.20.1, 00000000003.27.1, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:10.454 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.35.6, 00000000002.23.1, 00000000004.34.6, 00000000002.35.1, 00000000004.98.6, 00000000002.39.1, 00000000002.20.1, 00000000004.29.6, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:15.460+0000 7f5364e43980 1 waiting for incremental sync to catch up: 2024-04-16T18:10:15.456 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.33.1, 00000000002.23.1, 00000000003.32.1, 00000000002.35.1, 00000000003.96.1, 00000000002.39.1, 00000000002.20.1, 00000000003.27.1, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:15.456 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.35.6, 00000000002.23.1, 00000000004.34.6, 00000000002.35.1, 00000000004.98.6, 00000000002.39.1, 00000000002.20.1, 00000000004.29.6, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:20.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:20.463+0000 7f5364e43980 1 bucket sync caught up with source: 2024-04-16T18:10:20.459 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000004.35.6, 00000000002.23.1, 00000000004.34.6, 00000000002.35.1, 00000000004.98.6, 00000000002.39.1, 00000000002.20.1, 00000000004.29.6, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:20.459 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.35.6, 00000000002.23.1, 00000000004.34.6, 00000000002.35.1, 00000000004.98.6, 00000000002.39.1, 00000000002.20.1, 00000000004.29.6, 00000000002.10.1, 00000000002.21.1, 00000000002.21.1] 2024-04-16T18:10:20.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:20.463+0000 7f5364e43980 0 bucket checkpoint complete 2024-04-16T18:10:20.501 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-39 2024-04-16T18:10:20.502 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:20.502 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-39', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:20.502 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-39 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:20.555 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:20.555 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:20.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:20.632+0000 7f3583b31980 1 bucket sync caught up with source: 2024-04-16T18:10:20.634 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000002.89.6, , 00000000002.375.6, , 00000000002.872.6, , , 00000000002.283.6, , , ] 2024-04-16T18:10:20.634 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000002.89.6, , 00000000002.375.6, , 00000000002.872.6, , , 00000000002.283.6, , , ] 2024-04-16T18:10:20.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:20.632+0000 7f3583b31980 0 bucket checkpoint complete 2024-04-16T18:10:20.664 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-39 zones={test-zone2, test-zone1} 2024-04-16T18:10:20.664 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.664 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.665 DEBUG:boto:Method: HEAD 2024-04-16T18:10:20.665 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:10:20.665 DEBUG:boto:Data: 2024-04-16T18:10:20.665 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.665 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.665 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.665 DEBUG:boto:Params: {} 2024-04-16T18:10:20.666 DEBUG:boto:Token: None 2024-04-16T18:10:20.666 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/ 2024-04-16T18:10:20.666 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OONL5hoVuHW264pLkLwEFjdVCpM= 2024-04-16T18:10:20.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OONL5hoVuHW264pLkLwEFjdVCpM=', 'Content-Length': '0'} 2024-04-16T18:10:20.670 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a86d605927ee1ead-00661ebf0c-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.671 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.671 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.671 DEBUG:boto:Method: HEAD 2024-04-16T18:10:20.671 DEBUG:boto:Path: /lybtug-39/ 2024-04-16T18:10:20.671 DEBUG:boto:Data: 2024-04-16T18:10:20.671 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.672 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.672 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.672 DEBUG:boto:Params: {} 2024-04-16T18:10:20.672 DEBUG:boto:Token: None 2024-04-16T18:10:20.672 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/ 2024-04-16T18:10:20.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OONL5hoVuHW264pLkLwEFjdVCpM= 2024-04-16T18:10:20.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OONL5hoVuHW264pLkLwEFjdVCpM=', 'Content-Length': '0'} 2024-04-16T18:10:20.676 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009433cde5d9589a0b-00661ebf0c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.676 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:10:20.676 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.676 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.677 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.677 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.677 DEBUG:boto:Method: GET 2024-04-16T18:10:20.677 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.677 DEBUG:boto:Data: 2024-04-16T18:10:20.677 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.677 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.677 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.677 DEBUG:boto:Params: {} 2024-04-16T18:10:20.678 DEBUG:boto:Token: None 2024-04-16T18:10:20.678 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/?versions 2024-04-16T18:10:20.678 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc= 2024-04-16T18:10:20.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc=', 'Content-Length': '0'} 2024-04-16T18:10:20.682 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081b2df1885e05729-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.683 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.683 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:10:20.684 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:10:20.685 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.685 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.685 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.685 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.685 DEBUG:boto:Method: GET 2024-04-16T18:10:20.685 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.685 DEBUG:boto:Data: 2024-04-16T18:10:20.685 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.686 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.686 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.686 DEBUG:boto:Params: {} 2024-04-16T18:10:20.686 DEBUG:boto:Token: None 2024-04-16T18:10:20.686 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/?versions 2024-04-16T18:10:20.686 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc= 2024-04-16T18:10:20.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc=', 'Content-Length': '0'} 2024-04-16T18:10:20.690 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5c4c5151bbd6e4b-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.690 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:10:20.691 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:10:20.692 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.692 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.692 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.692 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.692 DEBUG:boto:Method: GET 2024-04-16T18:10:20.692 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.692 DEBUG:boto:Data: 2024-04-16T18:10:20.692 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.693 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.693 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.693 DEBUG:boto:Params: {} 2024-04-16T18:10:20.693 DEBUG:boto:Token: None 2024-04-16T18:10:20.693 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/?versions 2024-04-16T18:10:20.693 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc= 2024-04-16T18:10:20.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc=', 'Content-Length': '0'} 2024-04-16T18:10:20.697 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000e4d04020b507ee5-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.698 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.698 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.698 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.699 DEBUG:boto:path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.699 DEBUG:boto:auth_path=/lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.699 DEBUG:boto:Method: GET 2024-04-16T18:10:20.699 DEBUG:boto:Path: /lybtug-39/?versions&max-keys=999 2024-04-16T18:10:20.699 DEBUG:boto:Data: 2024-04-16T18:10:20.699 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.699 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.699 DEBUG:boto:Params: {} 2024-04-16T18:10:20.700 DEBUG:boto:Token: None 2024-04-16T18:10:20.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/?versions 2024-04-16T18:10:20.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc= 2024-04-16T18:10:20.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xZLs519hK6VOGrjMU8quRUZ9OHc=', 'Content-Length': '0'} 2024-04-16T18:10:20.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095195c8693aff6db-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.704 DEBUG:boto:b'lybtug-39999falseobj1nulltrue2024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.704 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:20.704 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.705 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.705 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.705 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.705 DEBUG:boto:Method: GET 2024-04-16T18:10:20.705 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.705 DEBUG:boto:Data: 2024-04-16T18:10:20.705 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.705 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.705 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.706 DEBUG:boto:Params: {} 2024-04-16T18:10:20.706 DEBUG:boto:Token: None 2024-04-16T18:10:20.706 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.707 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc= 2024-04-16T18:10:20.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc=', 'Content-Length': '0'} 2024-04-16T18:10:20.710 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000723d556968716a6b-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.711 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.711 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.711 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.711 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.711 DEBUG:boto:Method: GET 2024-04-16T18:10:20.711 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.711 DEBUG:boto:Data: 2024-04-16T18:10:20.712 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.712 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.712 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.712 DEBUG:boto:Params: {} 2024-04-16T18:10:20.712 DEBUG:boto:Token: None 2024-04-16T18:10:20.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc= 2024-04-16T18:10:20.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc=', 'Content-Length': '0'} 2024-04-16T18:10:20.715 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000046961387b3614aaa-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.759 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.759 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.759 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.759 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.759 DEBUG:boto:Method: HEAD 2024-04-16T18:10:20.759 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.759 DEBUG:boto:Data: 2024-04-16T18:10:20.760 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.760 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.760 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.760 DEBUG:boto:Params: {} 2024-04-16T18:10:20.760 DEBUG:boto:Token: None 2024-04-16T18:10:20.760 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0q4mamRv6yXbzbMI5sWYLW5M5S4= 2024-04-16T18:10:20.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0q4mamRv6yXbzbMI5sWYLW5M5S4=', 'Content-Length': '0'} 2024-04-16T18:10:20.763 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009e4f4951d57295a1-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.764 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.764 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.764 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.764 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.764 DEBUG:boto:Method: HEAD 2024-04-16T18:10:20.764 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.764 DEBUG:boto:Data: 2024-04-16T18:10:20.765 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.765 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.765 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.765 DEBUG:boto:Params: {} 2024-04-16T18:10:20.765 DEBUG:boto:Token: None 2024-04-16T18:10:20.765 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.765 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0q4mamRv6yXbzbMI5sWYLW5M5S4= 2024-04-16T18:10:20.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0q4mamRv6yXbzbMI5sWYLW5M5S4=', 'Content-Length': '0'} 2024-04-16T18:10:20.769 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000923ad4eddff9b855-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.769 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:20.769 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.769 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.769 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.769 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.770 DEBUG:boto:Method: GET 2024-04-16T18:10:20.770 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.770 DEBUG:boto:Data: 2024-04-16T18:10:20.770 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.770 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.770 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.770 DEBUG:boto:Params: {} 2024-04-16T18:10:20.770 DEBUG:boto:Token: None 2024-04-16T18:10:20.771 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.771 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc= 2024-04-16T18:10:20.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc=', 'Content-Length': '0'} 2024-04-16T18:10:20.774 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cb24b5118147ac6b-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.814 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.815 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.815 DEBUG:boto:path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.815 DEBUG:boto:auth_path=/lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.815 DEBUG:boto:Method: GET 2024-04-16T18:10:20.815 DEBUG:boto:Path: /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.815 DEBUG:boto:Data: 2024-04-16T18:10:20.815 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.815 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.816 DEBUG:boto:Params: {} 2024-04-16T18:10:20.816 DEBUG:boto:Token: None 2024-04-16T18:10:20.816 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1?versionId=null 2024-04-16T18:10:20.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc= 2024-04-16T18:10:20.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PZQY4TQP6eNbhCnFRpDcQ2uqzFc=', 'Content-Length': '0'} 2024-04-16T18:10:20.819 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a868d06650f3296a-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.862 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.863 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.863 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.863 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.863 DEBUG:boto:Method: GET 2024-04-16T18:10:20.863 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:10:20.863 DEBUG:boto:Data: 2024-04-16T18:10:20.863 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.863 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.864 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.864 DEBUG:boto:Params: {} 2024-04-16T18:10:20.864 DEBUG:boto:Token: None 2024-04-16T18:10:20.864 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/ 2024-04-16T18:10:20.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA= 2024-04-16T18:10:20.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA=', 'Content-Length': '0'} 2024-04-16T18:10:20.869 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000754537ddd19a80b4-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.869 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.869 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.870 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.870 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.870 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.870 DEBUG:boto:Method: GET 2024-04-16T18:10:20.870 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:10:20.870 DEBUG:boto:Data: 2024-04-16T18:10:20.871 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.871 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.871 DEBUG:boto:Params: {} 2024-04-16T18:10:20.871 DEBUG:boto:Token: None 2024-04-16T18:10:20.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/ 2024-04-16T18:10:20.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA= 2024-04-16T18:10:20.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA=', 'Content-Length': '0'} 2024-04-16T18:10:20.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079024e29a8abda82-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.876 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.876 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.876 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.877 DEBUG:boto:Method: HEAD 2024-04-16T18:10:20.877 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:10:20.877 DEBUG:boto:Data: 2024-04-16T18:10:20.877 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.877 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.877 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.877 DEBUG:boto:Params: {} 2024-04-16T18:10:20.877 DEBUG:boto:Token: None 2024-04-16T18:10:20.878 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1 2024-04-16T18:10:20.878 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PbVKIgOB5sxu1gryDXZPlnyT1fI= 2024-04-16T18:10:20.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PbVKIgOB5sxu1gryDXZPlnyT1fI=', 'Content-Length': '0'} 2024-04-16T18:10:20.881 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005fc98b97d2b28ac9-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.881 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.881 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.881 DEBUG:boto:Method: HEAD 2024-04-16T18:10:20.882 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:10:20.882 DEBUG:boto:Data: 2024-04-16T18:10:20.882 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.882 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.882 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.882 DEBUG:boto:Params: {} 2024-04-16T18:10:20.882 DEBUG:boto:Token: None 2024-04-16T18:10:20.883 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1 2024-04-16T18:10:20.883 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PbVKIgOB5sxu1gryDXZPlnyT1fI= 2024-04-16T18:10:20.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PbVKIgOB5sxu1gryDXZPlnyT1fI=', 'Content-Length': '0'} 2024-04-16T18:10:20.886 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038d7ae384c80036f-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:20.886 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.887 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.887 DEBUG:boto:Method: GET 2024-04-16T18:10:20.887 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:10:20.887 DEBUG:boto:Data: 2024-04-16T18:10:20.887 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.887 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.887 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.887 DEBUG:boto:Params: {} 2024-04-16T18:10:20.888 DEBUG:boto:Token: None 2024-04-16T18:10:20.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1 2024-04-16T18:10:20.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J7xZpJilSQ7w9CRP8E8zb6mDWNg= 2024-04-16T18:10:20.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J7xZpJilSQ7w9CRP8E8zb6mDWNg=', 'Content-Length': '0'} 2024-04-16T18:10:20.891 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000fee72130f4cf82c3-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.891 DEBUG:boto:path=/lybtug-39/obj1 2024-04-16T18:10:20.891 DEBUG:boto:auth_path=/lybtug-39/obj1 2024-04-16T18:10:20.891 DEBUG:boto:Method: GET 2024-04-16T18:10:20.892 DEBUG:boto:Path: /lybtug-39/obj1 2024-04-16T18:10:20.892 DEBUG:boto:Data: 2024-04-16T18:10:20.892 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.892 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.892 DEBUG:boto:Params: {} 2024-04-16T18:10:20.892 DEBUG:boto:Token: None 2024-04-16T18:10:20.892 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj1 2024-04-16T18:10:20.893 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J7xZpJilSQ7w9CRP8E8zb6mDWNg= 2024-04-16T18:10:20.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J7xZpJilSQ7w9CRP8E8zb6mDWNg=', 'Content-Length': '0'} 2024-04-16T18:10:20.896 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b0c80c3b7818be38-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.938 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.939 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.939 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.939 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.939 DEBUG:boto:Method: GET 2024-04-16T18:10:20.939 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:10:20.939 DEBUG:boto:Data: 2024-04-16T18:10:20.939 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.940 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.940 DEBUG:boto:Params: {} 2024-04-16T18:10:20.940 DEBUG:boto:Token: None 2024-04-16T18:10:20.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/ 2024-04-16T18:10:20.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA= 2024-04-16T18:10:20.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA=', 'Content-Length': '0'} 2024-04-16T18:10:20.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e105d6088839f0e8-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.945 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.945 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:20.946 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:20.946 DEBUG:boto:path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.946 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj1 2024-04-16T18:10:20.946 DEBUG:boto:Method: GET 2024-04-16T18:10:20.946 DEBUG:boto:Path: /lybtug-39/?prefix=obj1 2024-04-16T18:10:20.946 DEBUG:boto:Data: 2024-04-16T18:10:20.946 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.947 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.947 DEBUG:boto:Params: {} 2024-04-16T18:10:20.947 DEBUG:boto:Token: None 2024-04-16T18:10:20.947 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/ 2024-04-16T18:10:20.947 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA= 2024-04-16T18:10:20.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SAfF7u+711QM4rc3D9dfyr4/BLA=', 'Content-Length': '0'} 2024-04-16T18:10:20.951 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c9e60fb3169cebd-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.952 DEBUG:boto:b'lybtug-39obj11000falseobj12024-04-16T18:09:23.699Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:20.952 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:20.952 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:20.952 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:20.953 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.953 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.953 DEBUG:boto:Method: GET 2024-04-16T18:10:20.953 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.953 DEBUG:boto:Data: 2024-04-16T18:10:20.953 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.953 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:20.953 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.954 DEBUG:boto:Params: {} 2024-04-16T18:10:20.954 DEBUG:boto:Token: None 2024-04-16T18:10:20.954 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ALqsRCzefaaxbJa4U52KaUe9r7k= 2024-04-16T18:10:20.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ALqsRCzefaaxbJa4U52KaUe9r7k=', 'Content-Length': '0'} 2024-04-16T18:10:20.957 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b0c983ebb8c46719-00661ebf0c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:20.958 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:20.958 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:20.958 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.958 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.958 DEBUG:boto:Method: GET 2024-04-16T18:10:20.958 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.958 DEBUG:boto:Data: 2024-04-16T18:10:20.959 DEBUG:boto:Headers: {} 2024-04-16T18:10:20.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:20.959 DEBUG:boto:Port: 8000 2024-04-16T18:10:20.959 DEBUG:boto:Params: {} 2024-04-16T18:10:20.959 DEBUG:boto:Token: None 2024-04-16T18:10:20.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:20 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:10:20.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ALqsRCzefaaxbJa4U52KaUe9r7k= 2024-04-16T18:10:20.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ALqsRCzefaaxbJa4U52KaUe9r7k=', 'Content-Length': '0'} 2024-04-16T18:10:20.962 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a50d1810691bdb1c-00661ebf0c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.006 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.007 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.007 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.007 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.007 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.007 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.007 DEBUG:boto:Data: 2024-04-16T18:10:21.007 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.008 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.008 DEBUG:boto:Params: {} 2024-04-16T18:10:21.008 DEBUG:boto:Token: None 2024-04-16T18:10:21.008 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pDJ3ZhgsoZ/fhrHLlKfPlcM5B1E= 2024-04-16T18:10:21.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pDJ3ZhgsoZ/fhrHLlKfPlcM5B1E=', 'Content-Length': '0'} 2024-04-16T18:10:21.011 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c0a3d482051d49aa-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.012 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.012 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.012 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.012 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.012 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.013 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.013 DEBUG:boto:Data: 2024-04-16T18:10:21.013 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.013 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.013 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.013 DEBUG:boto:Params: {} 2024-04-16T18:10:21.013 DEBUG:boto:Token: None 2024-04-16T18:10:21.013 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.014 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pDJ3ZhgsoZ/fhrHLlKfPlcM5B1E= 2024-04-16T18:10:21.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pDJ3ZhgsoZ/fhrHLlKfPlcM5B1E=', 'Content-Length': '0'} 2024-04-16T18:10:21.017 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002d75ac815b4ad3dc-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.017 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:21.017 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.017 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.017 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.018 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.018 DEBUG:boto:Method: GET 2024-04-16T18:10:21.018 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.018 DEBUG:boto:Data: 2024-04-16T18:10:21.018 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.018 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.018 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.018 DEBUG:boto:Params: {} 2024-04-16T18:10:21.019 DEBUG:boto:Token: None 2024-04-16T18:10:21.019 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.019 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c9YnXfuZtgRFeWU7MEI8hmkqxl4= 2024-04-16T18:10:21.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c9YnXfuZtgRFeWU7MEI8hmkqxl4=', 'Content-Length': '0'} 2024-04-16T18:10:21.022 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004e23eed0f9995229-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.062 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.063 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.063 DEBUG:boto:path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.063 DEBUG:boto:auth_path=/lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.063 DEBUG:boto:Method: GET 2024-04-16T18:10:21.063 DEBUG:boto:Path: /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.063 DEBUG:boto:Data: 2024-04-16T18:10:21.063 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.064 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.064 DEBUG:boto:Params: {} 2024-04-16T18:10:21.064 DEBUG:boto:Token: None 2024-04-16T18:10:21.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2?versionId=null 2024-04-16T18:10:21.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c9YnXfuZtgRFeWU7MEI8hmkqxl4= 2024-04-16T18:10:21.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c9YnXfuZtgRFeWU7MEI8hmkqxl4=', 'Content-Length': '0'} 2024-04-16T18:10:21.068 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006f677214a34931f0-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.110 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.111 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.111 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.111 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.111 DEBUG:boto:Method: GET 2024-04-16T18:10:21.111 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:10:21.111 DEBUG:boto:Data: 2024-04-16T18:10:21.111 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.111 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.112 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.112 DEBUG:boto:Params: {} 2024-04-16T18:10:21.112 DEBUG:boto:Token: None 2024-04-16T18:10:21.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.116 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca416a8ef0735a49-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.117 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.117 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.117 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.117 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.118 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.118 DEBUG:boto:Method: GET 2024-04-16T18:10:21.118 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:10:21.118 DEBUG:boto:Data: 2024-04-16T18:10:21.118 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.118 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.118 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.118 DEBUG:boto:Params: {} 2024-04-16T18:10:21.119 DEBUG:boto:Token: None 2024-04-16T18:10:21.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a46eeb234a6596f8-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.123 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.124 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.124 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.124 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.124 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:10:21.124 DEBUG:boto:Data: 2024-04-16T18:10:21.124 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.125 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.125 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.125 DEBUG:boto:Params: {} 2024-04-16T18:10:21.125 DEBUG:boto:Token: None 2024-04-16T18:10:21.125 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2 2024-04-16T18:10:21.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cnKuu6l8mXZMTFUxDrcdW+J34a8= 2024-04-16T18:10:21.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cnKuu6l8mXZMTFUxDrcdW+J34a8=', 'Content-Length': '0'} 2024-04-16T18:10:21.128 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004ef4458cd2349957-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.129 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.129 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.129 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.129 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:10:21.129 DEBUG:boto:Data: 2024-04-16T18:10:21.129 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.129 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.129 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.130 DEBUG:boto:Params: {} 2024-04-16T18:10:21.130 DEBUG:boto:Token: None 2024-04-16T18:10:21.130 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2 2024-04-16T18:10:21.130 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cnKuu6l8mXZMTFUxDrcdW+J34a8= 2024-04-16T18:10:21.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cnKuu6l8mXZMTFUxDrcdW+J34a8=', 'Content-Length': '0'} 2024-04-16T18:10:21.133 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bf6f2dcbba957285-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.133 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:21.134 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.134 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.134 DEBUG:boto:Method: GET 2024-04-16T18:10:21.134 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:10:21.134 DEBUG:boto:Data: 2024-04-16T18:10:21.134 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.134 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.135 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.135 DEBUG:boto:Params: {} 2024-04-16T18:10:21.135 DEBUG:boto:Token: None 2024-04-16T18:10:21.135 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2 2024-04-16T18:10:21.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SuK13BlLcvpd7VAZrvK9PAD0axY= 2024-04-16T18:10:21.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SuK13BlLcvpd7VAZrvK9PAD0axY=', 'Content-Length': '0'} 2024-04-16T18:10:21.138 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000075067c17256163ed-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.138 DEBUG:boto:path=/lybtug-39/obj2 2024-04-16T18:10:21.139 DEBUG:boto:auth_path=/lybtug-39/obj2 2024-04-16T18:10:21.139 DEBUG:boto:Method: GET 2024-04-16T18:10:21.139 DEBUG:boto:Path: /lybtug-39/obj2 2024-04-16T18:10:21.139 DEBUG:boto:Data: 2024-04-16T18:10:21.139 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.139 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.139 DEBUG:boto:Params: {} 2024-04-16T18:10:21.140 DEBUG:boto:Token: None 2024-04-16T18:10:21.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj2 2024-04-16T18:10:21.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SuK13BlLcvpd7VAZrvK9PAD0axY= 2024-04-16T18:10:21.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SuK13BlLcvpd7VAZrvK9PAD0axY=', 'Content-Length': '0'} 2024-04-16T18:10:21.143 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000884a1eb75326dd3c-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.186 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.187 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.187 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.187 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.187 DEBUG:boto:Method: GET 2024-04-16T18:10:21.187 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:10:21.187 DEBUG:boto:Data: 2024-04-16T18:10:21.187 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.188 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.188 DEBUG:boto:Params: {} 2024-04-16T18:10:21.188 DEBUG:boto:Token: None 2024-04-16T18:10:21.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053705aa89cdfca63-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.193 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.193 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.193 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.194 DEBUG:boto:path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.194 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj2 2024-04-16T18:10:21.194 DEBUG:boto:Method: GET 2024-04-16T18:10:21.194 DEBUG:boto:Path: /lybtug-39/?prefix=obj2 2024-04-16T18:10:21.194 DEBUG:boto:Data: 2024-04-16T18:10:21.194 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.194 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.194 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.194 DEBUG:boto:Params: {} 2024-04-16T18:10:21.195 DEBUG:boto:Token: None 2024-04-16T18:10:21.195 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.195 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.199 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043a2a6afc8514933-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.242 DEBUG:boto:b'lybtug-39obj21000falseobj22024-04-16T18:09:23.720Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:21.246 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.246 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.246 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.246 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.246 DEBUG:boto:Method: GET 2024-04-16T18:10:21.246 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.246 DEBUG:boto:Data: 2024-04-16T18:10:21.246 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.247 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.247 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.247 DEBUG:boto:Params: {} 2024-04-16T18:10:21.247 DEBUG:boto:Token: None 2024-04-16T18:10:21.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.247 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs= 2024-04-16T18:10:21.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs=', 'Content-Length': '0'} 2024-04-16T18:10:21.251 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000140486a59a35f285-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.251 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.251 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.252 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.252 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.252 DEBUG:boto:Method: GET 2024-04-16T18:10:21.252 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.252 DEBUG:boto:Data: 2024-04-16T18:10:21.253 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.253 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.253 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.253 DEBUG:boto:Params: {} 2024-04-16T18:10:21.253 DEBUG:boto:Token: None 2024-04-16T18:10:21.253 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.254 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs= 2024-04-16T18:10:21.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs=', 'Content-Length': '0'} 2024-04-16T18:10:21.257 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a451ea7b6aad6e4a-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.299 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.299 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.299 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.299 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.299 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.299 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.299 DEBUG:boto:Data: 2024-04-16T18:10:21.300 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.300 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.300 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.300 DEBUG:boto:Params: {} 2024-04-16T18:10:21.300 DEBUG:boto:Token: None 2024-04-16T18:10:21.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.301 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uIRwpCC0vmtH1lO4+Mgc2t4VRNI= 2024-04-16T18:10:21.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uIRwpCC0vmtH1lO4+Mgc2t4VRNI=', 'Content-Length': '0'} 2024-04-16T18:10:21.304 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bf1873eb4065d5c7-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.304 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.304 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.304 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.304 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.305 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.305 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.305 DEBUG:boto:Data: 2024-04-16T18:10:21.305 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.305 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.305 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.305 DEBUG:boto:Params: {} 2024-04-16T18:10:21.305 DEBUG:boto:Token: None 2024-04-16T18:10:21.305 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.306 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uIRwpCC0vmtH1lO4+Mgc2t4VRNI= 2024-04-16T18:10:21.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uIRwpCC0vmtH1lO4+Mgc2t4VRNI=', 'Content-Length': '0'} 2024-04-16T18:10:21.309 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007934ddd316ccdc8f-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.309 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:21.309 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.309 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.309 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.310 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.310 DEBUG:boto:Method: GET 2024-04-16T18:10:21.310 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.310 DEBUG:boto:Data: 2024-04-16T18:10:21.310 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.310 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.310 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.310 DEBUG:boto:Params: {} 2024-04-16T18:10:21.311 DEBUG:boto:Token: None 2024-04-16T18:10:21.311 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.311 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs= 2024-04-16T18:10:21.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs=', 'Content-Length': '0'} 2024-04-16T18:10:21.314 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000008a1344ffc5e59a0-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.354 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.355 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.355 DEBUG:boto:path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.355 DEBUG:boto:auth_path=/lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.355 DEBUG:boto:Method: GET 2024-04-16T18:10:21.355 DEBUG:boto:Path: /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.355 DEBUG:boto:Data: 2024-04-16T18:10:21.355 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.356 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.356 DEBUG:boto:Params: {} 2024-04-16T18:10:21.356 DEBUG:boto:Token: None 2024-04-16T18:10:21.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3?versionId=null 2024-04-16T18:10:21.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs= 2024-04-16T18:10:21.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mA/ixM5p07cbz/ssAfpELdWrnOs=', 'Content-Length': '0'} 2024-04-16T18:10:21.359 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000438b75df63a27b56-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.403 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.403 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.403 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.403 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.403 DEBUG:boto:Method: GET 2024-04-16T18:10:21.403 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:10:21.403 DEBUG:boto:Data: 2024-04-16T18:10:21.403 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.404 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.404 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.404 DEBUG:boto:Params: {} 2024-04-16T18:10:21.404 DEBUG:boto:Token: None 2024-04-16T18:10:21.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.409 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bedf2ce3607e353c-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.409 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.409 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.410 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.410 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.410 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.410 DEBUG:boto:Method: GET 2024-04-16T18:10:21.410 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:10:21.410 DEBUG:boto:Data: 2024-04-16T18:10:21.410 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.410 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.411 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.411 DEBUG:boto:Params: {} 2024-04-16T18:10:21.411 DEBUG:boto:Token: None 2024-04-16T18:10:21.411 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.411 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2fdbac7b3ef6caa-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.458 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.459 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.459 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.459 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.459 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:10:21.459 DEBUG:boto:Data: 2024-04-16T18:10:21.460 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.460 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.460 DEBUG:boto:Params: {} 2024-04-16T18:10:21.460 DEBUG:boto:Token: None 2024-04-16T18:10:21.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3 2024-04-16T18:10:21.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/wJRRpjTzu5cOlFZPGD9nep0ZBs= 2024-04-16T18:10:21.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/wJRRpjTzu5cOlFZPGD9nep0ZBs=', 'Content-Length': '0'} 2024-04-16T18:10:21.463 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000facd57829326e69d-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.464 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.464 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.464 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.464 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:10:21.464 DEBUG:boto:Data: 2024-04-16T18:10:21.464 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.465 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.465 DEBUG:boto:Params: {} 2024-04-16T18:10:21.465 DEBUG:boto:Token: None 2024-04-16T18:10:21.465 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3 2024-04-16T18:10:21.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/wJRRpjTzu5cOlFZPGD9nep0ZBs= 2024-04-16T18:10:21.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/wJRRpjTzu5cOlFZPGD9nep0ZBs=', 'Content-Length': '0'} 2024-04-16T18:10:21.468 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007b660a26e2b2ab17-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.469 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:21.469 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.469 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.469 DEBUG:boto:Method: GET 2024-04-16T18:10:21.469 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:10:21.469 DEBUG:boto:Data: 2024-04-16T18:10:21.469 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.470 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.470 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.470 DEBUG:boto:Params: {} 2024-04-16T18:10:21.470 DEBUG:boto:Token: None 2024-04-16T18:10:21.470 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3 2024-04-16T18:10:21.470 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:exdOp4BuwNd9uAljXvOpuVUv12M= 2024-04-16T18:10:21.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:exdOp4BuwNd9uAljXvOpuVUv12M=', 'Content-Length': '0'} 2024-04-16T18:10:21.474 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007e92c0982efb7fa8-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.474 DEBUG:boto:path=/lybtug-39/obj3 2024-04-16T18:10:21.475 DEBUG:boto:auth_path=/lybtug-39/obj3 2024-04-16T18:10:21.475 DEBUG:boto:Method: GET 2024-04-16T18:10:21.475 DEBUG:boto:Path: /lybtug-39/obj3 2024-04-16T18:10:21.475 DEBUG:boto:Data: 2024-04-16T18:10:21.475 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.475 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.475 DEBUG:boto:Params: {} 2024-04-16T18:10:21.475 DEBUG:boto:Token: None 2024-04-16T18:10:21.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj3 2024-04-16T18:10:21.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:exdOp4BuwNd9uAljXvOpuVUv12M= 2024-04-16T18:10:21.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:exdOp4BuwNd9uAljXvOpuVUv12M=', 'Content-Length': '0'} 2024-04-16T18:10:21.479 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e2eab347b199240f-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.522 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.523 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.523 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.523 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.523 DEBUG:boto:Method: GET 2024-04-16T18:10:21.523 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:10:21.523 DEBUG:boto:Data: 2024-04-16T18:10:21.523 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.523 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.524 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.524 DEBUG:boto:Params: {} 2024-04-16T18:10:21.524 DEBUG:boto:Token: None 2024-04-16T18:10:21.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f17c12f74b95ebe1-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.529 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.529 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.529 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.529 DEBUG:boto:path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.530 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj3 2024-04-16T18:10:21.530 DEBUG:boto:Method: GET 2024-04-16T18:10:21.530 DEBUG:boto:Path: /lybtug-39/?prefix=obj3 2024-04-16T18:10:21.530 DEBUG:boto:Data: 2024-04-16T18:10:21.530 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.531 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.531 DEBUG:boto:Params: {} 2024-04-16T18:10:21.531 DEBUG:boto:Token: None 2024-04-16T18:10:21.531 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.531 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.535 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1c727ebed0263d3-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.578 DEBUG:boto:b'lybtug-39obj31000falseobj32024-04-16T18:09:23.768Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:21.579 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.579 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.579 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.579 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.580 DEBUG:boto:Method: GET 2024-04-16T18:10:21.580 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.580 DEBUG:boto:Data: 2024-04-16T18:10:21.580 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.580 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.580 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.580 DEBUG:boto:Params: {} 2024-04-16T18:10:21.580 DEBUG:boto:Token: None 2024-04-16T18:10:21.581 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ= 2024-04-16T18:10:21.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ=', 'Content-Length': '0'} 2024-04-16T18:10:21.584 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f1452f1da51f80a4-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.584 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.585 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.585 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.585 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.585 DEBUG:boto:Method: GET 2024-04-16T18:10:21.585 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.585 DEBUG:boto:Data: 2024-04-16T18:10:21.586 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.586 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.586 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.586 DEBUG:boto:Params: {} 2024-04-16T18:10:21.586 DEBUG:boto:Token: None 2024-04-16T18:10:21.586 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.586 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ= 2024-04-16T18:10:21.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ=', 'Content-Length': '0'} 2024-04-16T18:10:21.590 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000549d2b57e29decd3-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.630 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.631 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.631 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.631 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.631 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.631 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.631 DEBUG:boto:Data: 2024-04-16T18:10:21.632 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.632 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.632 DEBUG:boto:Params: {} 2024-04-16T18:10:21.632 DEBUG:boto:Token: None 2024-04-16T18:10:21.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.633 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F1IrA3NOnkCahgR3YueKm6xufjQ= 2024-04-16T18:10:21.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F1IrA3NOnkCahgR3YueKm6xufjQ=', 'Content-Length': '0'} 2024-04-16T18:10:21.636 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e1356badef8ea1ca-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.636 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.636 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.636 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.636 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.637 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.637 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.637 DEBUG:boto:Data: 2024-04-16T18:10:21.637 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.637 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.637 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.637 DEBUG:boto:Params: {} 2024-04-16T18:10:21.637 DEBUG:boto:Token: None 2024-04-16T18:10:21.638 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.638 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F1IrA3NOnkCahgR3YueKm6xufjQ= 2024-04-16T18:10:21.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F1IrA3NOnkCahgR3YueKm6xufjQ=', 'Content-Length': '0'} 2024-04-16T18:10:21.642 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d1978bf0226daad9-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.642 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:21.642 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.642 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.643 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.643 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.643 DEBUG:boto:Method: GET 2024-04-16T18:10:21.643 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.643 DEBUG:boto:Data: 2024-04-16T18:10:21.643 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.643 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.644 DEBUG:boto:Params: {} 2024-04-16T18:10:21.644 DEBUG:boto:Token: None 2024-04-16T18:10:21.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ= 2024-04-16T18:10:21.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ=', 'Content-Length': '0'} 2024-04-16T18:10:21.647 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ada56477543af059-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.648 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.648 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.648 DEBUG:boto:path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.648 DEBUG:boto:auth_path=/lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.648 DEBUG:boto:Method: GET 2024-04-16T18:10:21.648 DEBUG:boto:Path: /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.648 DEBUG:boto:Data: 2024-04-16T18:10:21.648 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.649 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.649 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.649 DEBUG:boto:Params: {} 2024-04-16T18:10:21.649 DEBUG:boto:Token: None 2024-04-16T18:10:21.649 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4?versionId=null 2024-04-16T18:10:21.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ= 2024-04-16T18:10:21.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xh3L4MA/IgPljdd7UsPND3ZWInQ=', 'Content-Length': '0'} 2024-04-16T18:10:21.654 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d5596472ea414d92-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.695 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.695 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.695 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.695 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.695 DEBUG:boto:Method: GET 2024-04-16T18:10:21.695 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:10:21.695 DEBUG:boto:Data: 2024-04-16T18:10:21.696 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.696 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.696 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.696 DEBUG:boto:Params: {} 2024-04-16T18:10:21.696 DEBUG:boto:Token: None 2024-04-16T18:10:21.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.701 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd0ff0360e8ac67f-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.702 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.702 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.702 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.703 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.703 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.703 DEBUG:boto:Method: GET 2024-04-16T18:10:21.703 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:10:21.703 DEBUG:boto:Data: 2024-04-16T18:10:21.703 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.703 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.703 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.703 DEBUG:boto:Params: {} 2024-04-16T18:10:21.704 DEBUG:boto:Token: None 2024-04-16T18:10:21.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e69ae5126c026d86-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.750 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.751 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.751 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.751 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.751 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:10:21.752 DEBUG:boto:Data: 2024-04-16T18:10:21.752 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.752 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.752 DEBUG:boto:Params: {} 2024-04-16T18:10:21.752 DEBUG:boto:Token: None 2024-04-16T18:10:21.752 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4 2024-04-16T18:10:21.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Y5BJOcasvl5kv5bJml5MpuYVZn0= 2024-04-16T18:10:21.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Y5BJOcasvl5kv5bJml5MpuYVZn0=', 'Content-Length': '0'} 2024-04-16T18:10:21.756 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c8f917c38f2d47a5-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.756 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.756 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.756 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.757 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:10:21.757 DEBUG:boto:Data: 2024-04-16T18:10:21.757 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.757 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.757 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.757 DEBUG:boto:Params: {} 2024-04-16T18:10:21.757 DEBUG:boto:Token: None 2024-04-16T18:10:21.758 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4 2024-04-16T18:10:21.758 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Y5BJOcasvl5kv5bJml5MpuYVZn0= 2024-04-16T18:10:21.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Y5BJOcasvl5kv5bJml5MpuYVZn0=', 'Content-Length': '0'} 2024-04-16T18:10:21.762 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001651faae66fa1a7c-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.762 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:21.762 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.762 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.763 DEBUG:boto:Method: GET 2024-04-16T18:10:21.763 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:10:21.763 DEBUG:boto:Data: 2024-04-16T18:10:21.763 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.763 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.763 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.763 DEBUG:boto:Params: {} 2024-04-16T18:10:21.763 DEBUG:boto:Token: None 2024-04-16T18:10:21.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4 2024-04-16T18:10:21.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NDSa+x9IwfPXm/X6j7+vq9H2+kY= 2024-04-16T18:10:21.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NDSa+x9IwfPXm/X6j7+vq9H2+kY=', 'Content-Length': '0'} 2024-04-16T18:10:21.767 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000061e6735cb9a75ae6-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.810 DEBUG:boto:path=/lybtug-39/obj4 2024-04-16T18:10:21.811 DEBUG:boto:auth_path=/lybtug-39/obj4 2024-04-16T18:10:21.811 DEBUG:boto:Method: GET 2024-04-16T18:10:21.811 DEBUG:boto:Path: /lybtug-39/obj4 2024-04-16T18:10:21.811 DEBUG:boto:Data: 2024-04-16T18:10:21.811 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.811 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.811 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.812 DEBUG:boto:Params: {} 2024-04-16T18:10:21.812 DEBUG:boto:Token: None 2024-04-16T18:10:21.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj4 2024-04-16T18:10:21.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NDSa+x9IwfPXm/X6j7+vq9H2+kY= 2024-04-16T18:10:21.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NDSa+x9IwfPXm/X6j7+vq9H2+kY=', 'Content-Length': '0'} 2024-04-16T18:10:21.815 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b5e82e70e4e40771-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.859 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.859 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.859 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.859 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.859 DEBUG:boto:Method: GET 2024-04-16T18:10:21.859 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:10:21.859 DEBUG:boto:Data: 2024-04-16T18:10:21.860 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.860 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.860 DEBUG:boto:Params: {} 2024-04-16T18:10:21.860 DEBUG:boto:Token: None 2024-04-16T18:10:21.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042c2f73590dd210a-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.865 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.865 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.866 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.866 DEBUG:boto:path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.866 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj4 2024-04-16T18:10:21.866 DEBUG:boto:Method: GET 2024-04-16T18:10:21.866 DEBUG:boto:Path: /lybtug-39/?prefix=obj4 2024-04-16T18:10:21.866 DEBUG:boto:Data: 2024-04-16T18:10:21.866 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.866 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.867 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.867 DEBUG:boto:Params: {} 2024-04-16T18:10:21.867 DEBUG:boto:Token: None 2024-04-16T18:10:21.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.871 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b853a37914d6f63c-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.871 DEBUG:boto:b'lybtug-39obj41000falseobj42024-04-16T18:09:23.786Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.872 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:21.872 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:21.872 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:21.872 DEBUG:boto:path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.872 DEBUG:boto:auth_path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.873 DEBUG:boto:Method: GET 2024-04-16T18:10:21.873 DEBUG:boto:Path: /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.873 DEBUG:boto:Data: 2024-04-16T18:10:21.873 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.873 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.873 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.873 DEBUG:boto:Params: {} 2024-04-16T18:10:21.873 DEBUG:boto:Token: None 2024-04-16T18:10:21.874 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.874 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA= 2024-04-16T18:10:21.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA=', 'Content-Length': '0'} 2024-04-16T18:10:21.877 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ed3e6db606339e18-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.877 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:21.878 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:21.878 DEBUG:boto:path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.878 DEBUG:boto:auth_path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.878 DEBUG:boto:Method: GET 2024-04-16T18:10:21.878 DEBUG:boto:Path: /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.878 DEBUG:boto:Data: 2024-04-16T18:10:21.878 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.878 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.879 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.879 DEBUG:boto:Params: {} 2024-04-16T18:10:21.879 DEBUG:boto:Token: None 2024-04-16T18:10:21.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA= 2024-04-16T18:10:21.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA=', 'Content-Length': '0'} 2024-04-16T18:10:21.883 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c8ea45d17575c1a2-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.926 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:21.927 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:21.927 DEBUG:boto:path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.927 DEBUG:boto:auth_path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.927 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.927 DEBUG:boto:Path: /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.927 DEBUG:boto:Data: 2024-04-16T18:10:21.927 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.927 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.928 DEBUG:boto:Params: {} 2024-04-16T18:10:21.928 DEBUG:boto:Token: None 2024-04-16T18:10:21.928 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rZch8cP1ay7LKHwCGjaiZRUICj8= 2024-04-16T18:10:21.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rZch8cP1ay7LKHwCGjaiZRUICj8=', 'Content-Length': '0'} 2024-04-16T18:10:21.931 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005d878b026551dcc1-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.931 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:21.931 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:21.931 DEBUG:boto:path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.932 DEBUG:boto:auth_path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.932 DEBUG:boto:Method: HEAD 2024-04-16T18:10:21.932 DEBUG:boto:Path: /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.932 DEBUG:boto:Data: 2024-04-16T18:10:21.932 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.932 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.932 DEBUG:boto:Params: {} 2024-04-16T18:10:21.932 DEBUG:boto:Token: None 2024-04-16T18:10:21.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rZch8cP1ay7LKHwCGjaiZRUICj8= 2024-04-16T18:10:21.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rZch8cP1ay7LKHwCGjaiZRUICj8=', 'Content-Length': '0'} 2024-04-16T18:10:21.936 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ba90b117787503d9-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.936 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:21.936 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:21.936 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:21.936 DEBUG:boto:path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.936 DEBUG:boto:auth_path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.936 DEBUG:boto:Method: GET 2024-04-16T18:10:21.937 DEBUG:boto:Path: /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.937 DEBUG:boto:Data: 2024-04-16T18:10:21.937 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.937 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.937 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.937 DEBUG:boto:Params: {} 2024-04-16T18:10:21.937 DEBUG:boto:Token: None 2024-04-16T18:10:21.937 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.937 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA= 2024-04-16T18:10:21.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA=', 'Content-Length': '0'} 2024-04-16T18:10:21.941 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ddd7e5509ac43075-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.941 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:21.941 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:21.941 DEBUG:boto:path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.941 DEBUG:boto:auth_path=/lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.941 DEBUG:boto:Method: GET 2024-04-16T18:10:21.941 DEBUG:boto:Path: /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.941 DEBUG:boto:Data: 2024-04-16T18:10:21.942 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.942 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.942 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.942 DEBUG:boto:Params: {} 2024-04-16T18:10:21.942 DEBUG:boto:Token: None 2024-04-16T18:10:21.942 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/obj5?versionId=null 2024-04-16T18:10:21.942 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA= 2024-04-16T18:10:21.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcTj95nGyxI1jeHxGPu9NgF4QyA=', 'Content-Length': '0'} 2024-04-16T18:10:21.945 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000062ef018566df2d2a-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.986 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.986 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.987 DEBUG:boto:path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:21.987 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:21.987 DEBUG:boto:Method: GET 2024-04-16T18:10:21.987 DEBUG:boto:Path: /lybtug-39/?prefix=obj5 2024-04-16T18:10:21.987 DEBUG:boto:Data: 2024-04-16T18:10:21.987 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:21.987 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.987 DEBUG:boto:Params: {} 2024-04-16T18:10:21.988 DEBUG:boto:Token: None 2024-04-16T18:10:21.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f984eb8924f74453-00661ebf0d-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:21.992 DEBUG:boto:b'lybtug-39obj51000falseobj52024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:21.992 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:21.992 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:21.992 DEBUG:boto:path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:21.992 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:21.992 DEBUG:boto:Method: GET 2024-04-16T18:10:21.992 DEBUG:boto:Path: /lybtug-39/?prefix=obj5 2024-04-16T18:10:21.993 DEBUG:boto:Data: 2024-04-16T18:10:21.993 DEBUG:boto:Headers: {} 2024-04-16T18:10:21.993 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:21.993 DEBUG:boto:Port: 8000 2024-04-16T18:10:21.993 DEBUG:boto:Params: {} 2024-04-16T18:10:21.993 DEBUG:boto:Token: None 2024-04-16T18:10:21.993 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:21 GMT /lybtug-39/ 2024-04-16T18:10:21.993 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0= 2024-04-16T18:10:21.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esG5+or3uWe97NuGIGpWDjGRNz0=', 'Content-Length': '0'} 2024-04-16T18:10:21.997 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038399b713b32c047-00661ebf0d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.038 DEBUG:boto:b'lybtug-39obj51000falseobj52024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.039 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:22.039 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:22.039 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.039 DEBUG:boto:Path: /lybtug-39/obj5 2024-04-16T18:10:22.039 DEBUG:boto:Data: 2024-04-16T18:10:22.039 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.039 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.039 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.039 DEBUG:boto:Params: {} 2024-04-16T18:10:22.039 DEBUG:boto:Token: None 2024-04-16T18:10:22.039 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj5 2024-04-16T18:10:22.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:S+rUi4Eq5a1QPCguMIz+K9nEXGk= 2024-04-16T18:10:22.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:S+rUi4Eq5a1QPCguMIz+K9nEXGk=', 'Content-Length': '0'} 2024-04-16T18:10:22.043 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000884df9224d08c12e-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.043 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:22.043 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:22.043 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.043 DEBUG:boto:Path: /lybtug-39/obj5 2024-04-16T18:10:22.043 DEBUG:boto:Data: 2024-04-16T18:10:22.043 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.044 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.044 DEBUG:boto:Params: {} 2024-04-16T18:10:22.044 DEBUG:boto:Token: None 2024-04-16T18:10:22.044 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj5 2024-04-16T18:10:22.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:S+rUi4Eq5a1QPCguMIz+K9nEXGk= 2024-04-16T18:10:22.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:S+rUi4Eq5a1QPCguMIz+K9nEXGk=', 'Content-Length': '0'} 2024-04-16T18:10:22.047 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bae696b585c93ddd-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:22.047 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:22.048 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:22.048 DEBUG:boto:Method: GET 2024-04-16T18:10:22.048 DEBUG:boto:Path: /lybtug-39/obj5 2024-04-16T18:10:22.048 DEBUG:boto:Data: 2024-04-16T18:10:22.048 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.048 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.048 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.048 DEBUG:boto:Params: {} 2024-04-16T18:10:22.048 DEBUG:boto:Token: None 2024-04-16T18:10:22.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj5 2024-04-16T18:10:22.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C9W1KwJiSu3FtM222pGCH2xeiOk= 2024-04-16T18:10:22.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C9W1KwJiSu3FtM222pGCH2xeiOk=', 'Content-Length': '0'} 2024-04-16T18:10:22.052 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006dbbaff7c34e3cd7-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.094 DEBUG:boto:path=/lybtug-39/obj5 2024-04-16T18:10:22.094 DEBUG:boto:auth_path=/lybtug-39/obj5 2024-04-16T18:10:22.095 DEBUG:boto:Method: GET 2024-04-16T18:10:22.095 DEBUG:boto:Path: /lybtug-39/obj5 2024-04-16T18:10:22.095 DEBUG:boto:Data: 2024-04-16T18:10:22.095 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.095 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.095 DEBUG:boto:Params: {} 2024-04-16T18:10:22.095 DEBUG:boto:Token: None 2024-04-16T18:10:22.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj5 2024-04-16T18:10:22.095 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C9W1KwJiSu3FtM222pGCH2xeiOk= 2024-04-16T18:10:22.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C9W1KwJiSu3FtM222pGCH2xeiOk=', 'Content-Length': '0'} 2024-04-16T18:10:22.099 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bc33a782d5a9431a-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.142 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.143 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.143 DEBUG:boto:path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:22.143 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:22.143 DEBUG:boto:Method: GET 2024-04-16T18:10:22.143 DEBUG:boto:Path: /lybtug-39/?prefix=obj5 2024-04-16T18:10:22.143 DEBUG:boto:Data: 2024-04-16T18:10:22.143 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.143 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.143 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.143 DEBUG:boto:Params: {} 2024-04-16T18:10:22.143 DEBUG:boto:Token: None 2024-04-16T18:10:22.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.147 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013888df6861e8270-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.148 DEBUG:boto:b'lybtug-39obj51000falseobj52024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.148 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.148 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.148 DEBUG:boto:path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:22.148 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj5 2024-04-16T18:10:22.148 DEBUG:boto:Method: GET 2024-04-16T18:10:22.149 DEBUG:boto:Path: /lybtug-39/?prefix=obj5 2024-04-16T18:10:22.149 DEBUG:boto:Data: 2024-04-16T18:10:22.149 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.149 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.149 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.149 DEBUG:boto:Params: {} 2024-04-16T18:10:22.149 DEBUG:boto:Token: None 2024-04-16T18:10:22.149 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.149 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030ee6c43265249da-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.153 DEBUG:boto:b'lybtug-39obj51000falseobj52024-04-16T18:09:37.636Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.153 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:22.153 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.153 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.154 DEBUG:boto:path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.154 DEBUG:boto:auth_path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.154 DEBUG:boto:Method: GET 2024-04-16T18:10:22.154 DEBUG:boto:Path: /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.154 DEBUG:boto:Data: 2024-04-16T18:10:22.154 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.154 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.154 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.154 DEBUG:boto:Params: {} 2024-04-16T18:10:22.154 DEBUG:boto:Token: None 2024-04-16T18:10:22.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.155 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w= 2024-04-16T18:10:22.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w=', 'Content-Length': '0'} 2024-04-16T18:10:22.158 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007ef417199a62c5fb-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.158 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.158 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.158 DEBUG:boto:path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.159 DEBUG:boto:auth_path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.159 DEBUG:boto:Method: GET 2024-04-16T18:10:22.159 DEBUG:boto:Path: /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.159 DEBUG:boto:Data: 2024-04-16T18:10:22.159 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.159 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.159 DEBUG:boto:Params: {} 2024-04-16T18:10:22.159 DEBUG:boto:Token: None 2024-04-16T18:10:22.159 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.159 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w= 2024-04-16T18:10:22.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w=', 'Content-Length': '0'} 2024-04-16T18:10:22.163 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d0e7de1c09b1eedd-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.206 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.207 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.207 DEBUG:boto:path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.207 DEBUG:boto:auth_path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.207 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.207 DEBUG:boto:Path: /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.207 DEBUG:boto:Data: 2024-04-16T18:10:22.207 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.207 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.207 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.207 DEBUG:boto:Params: {} 2024-04-16T18:10:22.207 DEBUG:boto:Token: None 2024-04-16T18:10:22.208 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:csgSyEjtWdUg+5KQ0WzGiR9D8bE= 2024-04-16T18:10:22.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:csgSyEjtWdUg+5KQ0WzGiR9D8bE=', 'Content-Length': '0'} 2024-04-16T18:10:22.211 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ead4b36bb94fa295-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.211 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.211 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.211 DEBUG:boto:path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.211 DEBUG:boto:auth_path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.212 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.212 DEBUG:boto:Path: /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.212 DEBUG:boto:Data: 2024-04-16T18:10:22.212 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.212 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.212 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.212 DEBUG:boto:Params: {} 2024-04-16T18:10:22.212 DEBUG:boto:Token: None 2024-04-16T18:10:22.213 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:csgSyEjtWdUg+5KQ0WzGiR9D8bE= 2024-04-16T18:10:22.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:csgSyEjtWdUg+5KQ0WzGiR9D8bE=', 'Content-Length': '0'} 2024-04-16T18:10:22.216 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000013c336651b43be1d-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:22.216 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.216 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.217 DEBUG:boto:path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.217 DEBUG:boto:auth_path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.217 DEBUG:boto:Method: GET 2024-04-16T18:10:22.217 DEBUG:boto:Path: /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.217 DEBUG:boto:Data: 2024-04-16T18:10:22.217 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.217 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.217 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.218 DEBUG:boto:Params: {} 2024-04-16T18:10:22.218 DEBUG:boto:Token: None 2024-04-16T18:10:22.218 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.218 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w= 2024-04-16T18:10:22.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w=', 'Content-Length': '0'} 2024-04-16T18:10:22.221 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006943f9ed368c39b9-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.222 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.222 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.222 DEBUG:boto:path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.222 DEBUG:boto:auth_path=/lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.222 DEBUG:boto:Method: GET 2024-04-16T18:10:22.222 DEBUG:boto:Path: /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.222 DEBUG:boto:Data: 2024-04-16T18:10:22.222 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.222 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.222 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.222 DEBUG:boto:Params: {} 2024-04-16T18:10:22.223 DEBUG:boto:Token: None 2024-04-16T18:10:22.223 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6?versionId=null 2024-04-16T18:10:22.223 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w= 2024-04-16T18:10:22.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Vl5PCVYWnk2WnQnhLvwjwwtj1w=', 'Content-Length': '0'} 2024-04-16T18:10:22.226 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000050f179c6f71f6225-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.266 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.267 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.267 DEBUG:boto:path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.267 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.267 DEBUG:boto:Method: GET 2024-04-16T18:10:22.267 DEBUG:boto:Path: /lybtug-39/?prefix=obj6 2024-04-16T18:10:22.267 DEBUG:boto:Data: 2024-04-16T18:10:22.267 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.267 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.267 DEBUG:boto:Params: {} 2024-04-16T18:10:22.267 DEBUG:boto:Token: None 2024-04-16T18:10:22.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043e77a1e94683490-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.271 DEBUG:boto:b'lybtug-39obj61000falseobj62024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.272 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.272 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.272 DEBUG:boto:path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.272 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.272 DEBUG:boto:Method: GET 2024-04-16T18:10:22.272 DEBUG:boto:Path: /lybtug-39/?prefix=obj6 2024-04-16T18:10:22.272 DEBUG:boto:Data: 2024-04-16T18:10:22.272 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.272 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.272 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.272 DEBUG:boto:Params: {} 2024-04-16T18:10:22.273 DEBUG:boto:Token: None 2024-04-16T18:10:22.273 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.276 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016d49ac0e4a5d7db-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.322 DEBUG:boto:b'lybtug-39obj61000falseobj62024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.323 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.323 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.323 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.323 DEBUG:boto:Path: /lybtug-39/obj6 2024-04-16T18:10:22.323 DEBUG:boto:Data: 2024-04-16T18:10:22.323 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.323 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.323 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.323 DEBUG:boto:Params: {} 2024-04-16T18:10:22.324 DEBUG:boto:Token: None 2024-04-16T18:10:22.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6 2024-04-16T18:10:22.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XsCZQD8TfpBX/tlYK+9iQ0EzSS8= 2024-04-16T18:10:22.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XsCZQD8TfpBX/tlYK+9iQ0EzSS8=', 'Content-Length': '0'} 2024-04-16T18:10:22.327 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cd86e34def5201da-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.327 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.327 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.328 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.328 DEBUG:boto:Path: /lybtug-39/obj6 2024-04-16T18:10:22.328 DEBUG:boto:Data: 2024-04-16T18:10:22.328 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.328 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.328 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.328 DEBUG:boto:Params: {} 2024-04-16T18:10:22.328 DEBUG:boto:Token: None 2024-04-16T18:10:22.328 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6 2024-04-16T18:10:22.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XsCZQD8TfpBX/tlYK+9iQ0EzSS8= 2024-04-16T18:10:22.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XsCZQD8TfpBX/tlYK+9iQ0EzSS8=', 'Content-Length': '0'} 2024-04-16T18:10:22.332 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f74bbb883cdedec1-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.332 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:22.332 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.332 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.332 DEBUG:boto:Method: GET 2024-04-16T18:10:22.332 DEBUG:boto:Path: /lybtug-39/obj6 2024-04-16T18:10:22.332 DEBUG:boto:Data: 2024-04-16T18:10:22.332 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.332 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.333 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.333 DEBUG:boto:Params: {} 2024-04-16T18:10:22.333 DEBUG:boto:Token: None 2024-04-16T18:10:22.333 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6 2024-04-16T18:10:22.333 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TLy/aLXnzFhg9z3g5XpK/SBeBBE= 2024-04-16T18:10:22.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TLy/aLXnzFhg9z3g5XpK/SBeBBE=', 'Content-Length': '0'} 2024-04-16T18:10:22.336 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ae55fd656588efda-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.378 DEBUG:boto:path=/lybtug-39/obj6 2024-04-16T18:10:22.379 DEBUG:boto:auth_path=/lybtug-39/obj6 2024-04-16T18:10:22.379 DEBUG:boto:Method: GET 2024-04-16T18:10:22.379 DEBUG:boto:Path: /lybtug-39/obj6 2024-04-16T18:10:22.379 DEBUG:boto:Data: 2024-04-16T18:10:22.379 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.379 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.379 DEBUG:boto:Params: {} 2024-04-16T18:10:22.379 DEBUG:boto:Token: None 2024-04-16T18:10:22.379 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj6 2024-04-16T18:10:22.379 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TLy/aLXnzFhg9z3g5XpK/SBeBBE= 2024-04-16T18:10:22.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TLy/aLXnzFhg9z3g5XpK/SBeBBE=', 'Content-Length': '0'} 2024-04-16T18:10:22.383 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b3a4077e5d42ab85-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.426 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.426 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.427 DEBUG:boto:path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.427 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.427 DEBUG:boto:Method: GET 2024-04-16T18:10:22.427 DEBUG:boto:Path: /lybtug-39/?prefix=obj6 2024-04-16T18:10:22.427 DEBUG:boto:Data: 2024-04-16T18:10:22.427 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.427 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.427 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.427 DEBUG:boto:Params: {} 2024-04-16T18:10:22.427 DEBUG:boto:Token: None 2024-04-16T18:10:22.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.432 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005684b032e8866715-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.432 DEBUG:boto:b'lybtug-39obj61000falseobj62024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.432 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.432 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.432 DEBUG:boto:path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.433 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj6 2024-04-16T18:10:22.433 DEBUG:boto:Method: GET 2024-04-16T18:10:22.433 DEBUG:boto:Path: /lybtug-39/?prefix=obj6 2024-04-16T18:10:22.433 DEBUG:boto:Data: 2024-04-16T18:10:22.433 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.433 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.433 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.433 DEBUG:boto:Params: {} 2024-04-16T18:10:22.433 DEBUG:boto:Token: None 2024-04-16T18:10:22.433 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.434 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.438 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3fb21f528fb8955-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.438 DEBUG:boto:b'lybtug-39obj61000falseobj62024-04-16T18:09:37.655Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.438 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:22.438 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.438 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.438 DEBUG:boto:path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.439 DEBUG:boto:auth_path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.439 DEBUG:boto:Method: GET 2024-04-16T18:10:22.439 DEBUG:boto:Path: /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.439 DEBUG:boto:Data: 2024-04-16T18:10:22.439 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.439 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.439 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.439 DEBUG:boto:Params: {} 2024-04-16T18:10:22.439 DEBUG:boto:Token: None 2024-04-16T18:10:22.439 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI= 2024-04-16T18:10:22.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI=', 'Content-Length': '0'} 2024-04-16T18:10:22.443 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000098bad063bea5d11c-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.443 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.443 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.443 DEBUG:boto:path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.443 DEBUG:boto:auth_path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.444 DEBUG:boto:Method: GET 2024-04-16T18:10:22.444 DEBUG:boto:Path: /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.444 DEBUG:boto:Data: 2024-04-16T18:10:22.444 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.444 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.444 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.444 DEBUG:boto:Params: {} 2024-04-16T18:10:22.444 DEBUG:boto:Token: None 2024-04-16T18:10:22.444 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI= 2024-04-16T18:10:22.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI=', 'Content-Length': '0'} 2024-04-16T18:10:22.447 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003d4cb47b7888205f-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.490 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.490 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.491 DEBUG:boto:path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.491 DEBUG:boto:auth_path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.491 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.491 DEBUG:boto:Path: /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.491 DEBUG:boto:Data: 2024-04-16T18:10:22.491 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.491 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.491 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.491 DEBUG:boto:Params: {} 2024-04-16T18:10:22.491 DEBUG:boto:Token: None 2024-04-16T18:10:22.492 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GlTMica1R8NzDMYh5CVDbsHXsQc= 2024-04-16T18:10:22.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GlTMica1R8NzDMYh5CVDbsHXsQc=', 'Content-Length': '0'} 2024-04-16T18:10:22.495 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004836a2b6550f7b46-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.495 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.495 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.495 DEBUG:boto:path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.495 DEBUG:boto:auth_path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.495 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.495 DEBUG:boto:Path: /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.495 DEBUG:boto:Data: 2024-04-16T18:10:22.496 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.496 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.496 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.496 DEBUG:boto:Params: {} 2024-04-16T18:10:22.496 DEBUG:boto:Token: None 2024-04-16T18:10:22.496 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GlTMica1R8NzDMYh5CVDbsHXsQc= 2024-04-16T18:10:22.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GlTMica1R8NzDMYh5CVDbsHXsQc=', 'Content-Length': '0'} 2024-04-16T18:10:22.499 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005cd636e874063e87-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:22.499 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.499 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.499 DEBUG:boto:path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.500 DEBUG:boto:auth_path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.500 DEBUG:boto:Method: GET 2024-04-16T18:10:22.500 DEBUG:boto:Path: /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.500 DEBUG:boto:Data: 2024-04-16T18:10:22.500 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.500 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.500 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.500 DEBUG:boto:Params: {} 2024-04-16T18:10:22.500 DEBUG:boto:Token: None 2024-04-16T18:10:22.500 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI= 2024-04-16T18:10:22.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI=', 'Content-Length': '0'} 2024-04-16T18:10:22.503 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000059c6402be6267366-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.504 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.504 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.504 DEBUG:boto:path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.504 DEBUG:boto:auth_path=/lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.504 DEBUG:boto:Method: GET 2024-04-16T18:10:22.504 DEBUG:boto:Path: /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.504 DEBUG:boto:Data: 2024-04-16T18:10:22.504 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.504 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.504 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.505 DEBUG:boto:Params: {} 2024-04-16T18:10:22.505 DEBUG:boto:Token: None 2024-04-16T18:10:22.505 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7?versionId=null 2024-04-16T18:10:22.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI= 2024-04-16T18:10:22.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIkbki5iK/oLpFfc10p+oXn4dCI=', 'Content-Length': '0'} 2024-04-16T18:10:22.508 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000915a54cb3d8305f6-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.550 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.551 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.551 DEBUG:boto:path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.551 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.551 DEBUG:boto:Method: GET 2024-04-16T18:10:22.551 DEBUG:boto:Path: /lybtug-39/?prefix=obj7 2024-04-16T18:10:22.551 DEBUG:boto:Data: 2024-04-16T18:10:22.551 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.551 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.551 DEBUG:boto:Params: {} 2024-04-16T18:10:22.551 DEBUG:boto:Token: None 2024-04-16T18:10:22.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb704ed826d26189-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.556 DEBUG:boto:b'lybtug-39obj71000falseobj72024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.556 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.556 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.556 DEBUG:boto:path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.556 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.556 DEBUG:boto:Method: GET 2024-04-16T18:10:22.556 DEBUG:boto:Path: /lybtug-39/?prefix=obj7 2024-04-16T18:10:22.557 DEBUG:boto:Data: 2024-04-16T18:10:22.557 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.557 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.557 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.557 DEBUG:boto:Params: {} 2024-04-16T18:10:22.557 DEBUG:boto:Token: None 2024-04-16T18:10:22.557 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.557 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.561 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000221be27920f40842-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.561 DEBUG:boto:b'lybtug-39obj71000falseobj72024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.561 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.561 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.561 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.562 DEBUG:boto:Path: /lybtug-39/obj7 2024-04-16T18:10:22.562 DEBUG:boto:Data: 2024-04-16T18:10:22.562 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.562 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.562 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.562 DEBUG:boto:Params: {} 2024-04-16T18:10:22.562 DEBUG:boto:Token: None 2024-04-16T18:10:22.562 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7 2024-04-16T18:10:22.562 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WCd9r+ng8p9i+nw7vnpeKMCpsqc= 2024-04-16T18:10:22.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WCd9r+ng8p9i+nw7vnpeKMCpsqc=', 'Content-Length': '0'} 2024-04-16T18:10:22.566 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000549652045f6d6029-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.566 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.566 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.566 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.566 DEBUG:boto:Path: /lybtug-39/obj7 2024-04-16T18:10:22.566 DEBUG:boto:Data: 2024-04-16T18:10:22.566 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.566 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.567 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.567 DEBUG:boto:Params: {} 2024-04-16T18:10:22.567 DEBUG:boto:Token: None 2024-04-16T18:10:22.567 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7 2024-04-16T18:10:22.567 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WCd9r+ng8p9i+nw7vnpeKMCpsqc= 2024-04-16T18:10:22.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WCd9r+ng8p9i+nw7vnpeKMCpsqc=', 'Content-Length': '0'} 2024-04-16T18:10:22.570 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d812678ae0517d4d-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.570 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:22.570 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.570 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.570 DEBUG:boto:Method: GET 2024-04-16T18:10:22.571 DEBUG:boto:Path: /lybtug-39/obj7 2024-04-16T18:10:22.571 DEBUG:boto:Data: 2024-04-16T18:10:22.571 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.571 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.571 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.571 DEBUG:boto:Params: {} 2024-04-16T18:10:22.571 DEBUG:boto:Token: None 2024-04-16T18:10:22.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7 2024-04-16T18:10:22.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M5I3L9JMU+V7e/Gv7lUbnkUzb5Y= 2024-04-16T18:10:22.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M5I3L9JMU+V7e/Gv7lUbnkUzb5Y=', 'Content-Length': '0'} 2024-04-16T18:10:22.574 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000092a60ee55da19d8-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.618 DEBUG:boto:path=/lybtug-39/obj7 2024-04-16T18:10:22.619 DEBUG:boto:auth_path=/lybtug-39/obj7 2024-04-16T18:10:22.619 DEBUG:boto:Method: GET 2024-04-16T18:10:22.619 DEBUG:boto:Path: /lybtug-39/obj7 2024-04-16T18:10:22.619 DEBUG:boto:Data: 2024-04-16T18:10:22.619 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.619 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.619 DEBUG:boto:Params: {} 2024-04-16T18:10:22.619 DEBUG:boto:Token: None 2024-04-16T18:10:22.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj7 2024-04-16T18:10:22.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M5I3L9JMU+V7e/Gv7lUbnkUzb5Y= 2024-04-16T18:10:22.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M5I3L9JMU+V7e/Gv7lUbnkUzb5Y=', 'Content-Length': '0'} 2024-04-16T18:10:22.622 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e9a2f3d0ebe22f33-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.666 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.666 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.667 DEBUG:boto:path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.667 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.667 DEBUG:boto:Method: GET 2024-04-16T18:10:22.667 DEBUG:boto:Path: /lybtug-39/?prefix=obj7 2024-04-16T18:10:22.667 DEBUG:boto:Data: 2024-04-16T18:10:22.667 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.667 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.667 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.667 DEBUG:boto:Params: {} 2024-04-16T18:10:22.667 DEBUG:boto:Token: None 2024-04-16T18:10:22.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.674 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001bab6fb0b9b98fa-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.674 DEBUG:boto:b'lybtug-39obj71000falseobj72024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.675 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.675 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.675 DEBUG:boto:path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.675 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj7 2024-04-16T18:10:22.675 DEBUG:boto:Method: GET 2024-04-16T18:10:22.675 DEBUG:boto:Path: /lybtug-39/?prefix=obj7 2024-04-16T18:10:22.675 DEBUG:boto:Data: 2024-04-16T18:10:22.675 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.675 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.675 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.675 DEBUG:boto:Params: {} 2024-04-16T18:10:22.676 DEBUG:boto:Token: None 2024-04-16T18:10:22.676 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.679 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c758fde629cddd8-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.722 DEBUG:boto:b'lybtug-39obj71000falseobj72024-04-16T18:09:37.703Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:22.723 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.723 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.723 DEBUG:boto:path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.723 DEBUG:boto:auth_path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.723 DEBUG:boto:Method: GET 2024-04-16T18:10:22.723 DEBUG:boto:Path: /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.723 DEBUG:boto:Data: 2024-04-16T18:10:22.723 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.723 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.723 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.724 DEBUG:boto:Params: {} 2024-04-16T18:10:22.724 DEBUG:boto:Token: None 2024-04-16T18:10:22.724 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo= 2024-04-16T18:10:22.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo=', 'Content-Length': '0'} 2024-04-16T18:10:22.727 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000816614eca4396222-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.727 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.727 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.727 DEBUG:boto:path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.728 DEBUG:boto:auth_path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.728 DEBUG:boto:Method: GET 2024-04-16T18:10:22.728 DEBUG:boto:Path: /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.728 DEBUG:boto:Data: 2024-04-16T18:10:22.728 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.728 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.728 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.728 DEBUG:boto:Params: {} 2024-04-16T18:10:22.728 DEBUG:boto:Token: None 2024-04-16T18:10:22.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo= 2024-04-16T18:10:22.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo=', 'Content-Length': '0'} 2024-04-16T18:10:22.732 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000058575deabe51c824-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.774 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.774 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.775 DEBUG:boto:path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.775 DEBUG:boto:auth_path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.775 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.775 DEBUG:boto:Path: /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.775 DEBUG:boto:Data: 2024-04-16T18:10:22.775 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.775 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.775 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.775 DEBUG:boto:Params: {} 2024-04-16T18:10:22.775 DEBUG:boto:Token: None 2024-04-16T18:10:22.775 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uEEn3O16H5d+exwLEWPIWXQ6taE= 2024-04-16T18:10:22.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uEEn3O16H5d+exwLEWPIWXQ6taE=', 'Content-Length': '0'} 2024-04-16T18:10:22.778 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000060bd0d09b4534cb7-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.778 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.779 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.779 DEBUG:boto:path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.779 DEBUG:boto:auth_path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.779 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.779 DEBUG:boto:Path: /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.779 DEBUG:boto:Data: 2024-04-16T18:10:22.779 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.779 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.779 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.779 DEBUG:boto:Params: {} 2024-04-16T18:10:22.779 DEBUG:boto:Token: None 2024-04-16T18:10:22.780 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uEEn3O16H5d+exwLEWPIWXQ6taE= 2024-04-16T18:10:22.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uEEn3O16H5d+exwLEWPIWXQ6taE=', 'Content-Length': '0'} 2024-04-16T18:10:22.782 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007940cb1234a6ef77-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.783 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:22.783 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.783 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.783 DEBUG:boto:path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.783 DEBUG:boto:auth_path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.783 DEBUG:boto:Method: GET 2024-04-16T18:10:22.783 DEBUG:boto:Path: /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.783 DEBUG:boto:Data: 2024-04-16T18:10:22.783 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.784 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.784 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.784 DEBUG:boto:Params: {} 2024-04-16T18:10:22.784 DEBUG:boto:Token: None 2024-04-16T18:10:22.784 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo= 2024-04-16T18:10:22.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo=', 'Content-Length': '0'} 2024-04-16T18:10:22.787 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a28910fa1f518a2c-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.787 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.787 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.787 DEBUG:boto:path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.787 DEBUG:boto:auth_path=/lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.787 DEBUG:boto:Method: GET 2024-04-16T18:10:22.788 DEBUG:boto:Path: /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.788 DEBUG:boto:Data: 2024-04-16T18:10:22.788 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.788 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.788 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.788 DEBUG:boto:Params: {} 2024-04-16T18:10:22.788 DEBUG:boto:Token: None 2024-04-16T18:10:22.788 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8?versionId=null 2024-04-16T18:10:22.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo= 2024-04-16T18:10:22.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FkRl2HcLqThgGLWRzNIt6QMVrWo=', 'Content-Length': '0'} 2024-04-16T18:10:22.791 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d196d3a3c3e980e1-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.834 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.834 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.835 DEBUG:boto:path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.835 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.835 DEBUG:boto:Method: GET 2024-04-16T18:10:22.835 DEBUG:boto:Path: /lybtug-39/?prefix=obj8 2024-04-16T18:10:22.835 DEBUG:boto:Data: 2024-04-16T18:10:22.835 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.835 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.835 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.835 DEBUG:boto:Params: {} 2024-04-16T18:10:22.835 DEBUG:boto:Token: None 2024-04-16T18:10:22.835 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dec6b6a194e925d4-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.840 DEBUG:boto:b'lybtug-39obj81000falseobj82024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.840 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.840 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.840 DEBUG:boto:path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.840 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.840 DEBUG:boto:Method: GET 2024-04-16T18:10:22.841 DEBUG:boto:Path: /lybtug-39/?prefix=obj8 2024-04-16T18:10:22.841 DEBUG:boto:Data: 2024-04-16T18:10:22.841 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.841 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.841 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.841 DEBUG:boto:Params: {} 2024-04-16T18:10:22.841 DEBUG:boto:Token: None 2024-04-16T18:10:22.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.845 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ceda91ffa3fa92a-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.845 DEBUG:boto:b'lybtug-39obj81000falseobj82024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.845 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.846 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.846 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.846 DEBUG:boto:Path: /lybtug-39/obj8 2024-04-16T18:10:22.846 DEBUG:boto:Data: 2024-04-16T18:10:22.846 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.846 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.846 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.846 DEBUG:boto:Params: {} 2024-04-16T18:10:22.846 DEBUG:boto:Token: None 2024-04-16T18:10:22.846 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8 2024-04-16T18:10:22.846 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X0vtaMS8LKyEwbTm6mqHbc/JgVI= 2024-04-16T18:10:22.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X0vtaMS8LKyEwbTm6mqHbc/JgVI=', 'Content-Length': '0'} 2024-04-16T18:10:22.849 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000020ac8a3b3c9ccadd-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.849 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.849 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.849 DEBUG:boto:Method: HEAD 2024-04-16T18:10:22.850 DEBUG:boto:Path: /lybtug-39/obj8 2024-04-16T18:10:22.850 DEBUG:boto:Data: 2024-04-16T18:10:22.850 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.850 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.850 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.850 DEBUG:boto:Params: {} 2024-04-16T18:10:22.850 DEBUG:boto:Token: None 2024-04-16T18:10:22.850 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8 2024-04-16T18:10:22.850 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X0vtaMS8LKyEwbTm6mqHbc/JgVI= 2024-04-16T18:10:22.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X0vtaMS8LKyEwbTm6mqHbc/JgVI=', 'Content-Length': '0'} 2024-04-16T18:10:22.853 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a2d745f73b0a529e-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.854 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:22.854 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.854 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.854 DEBUG:boto:Method: GET 2024-04-16T18:10:22.854 DEBUG:boto:Path: /lybtug-39/obj8 2024-04-16T18:10:22.854 DEBUG:boto:Data: 2024-04-16T18:10:22.854 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.854 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.854 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.854 DEBUG:boto:Params: {} 2024-04-16T18:10:22.854 DEBUG:boto:Token: None 2024-04-16T18:10:22.855 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8 2024-04-16T18:10:22.855 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9ZMKWYmVlkj079UcIyL+9DA/kxY= 2024-04-16T18:10:22.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9ZMKWYmVlkj079UcIyL+9DA/kxY=', 'Content-Length': '0'} 2024-04-16T18:10:22.857 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000fbba282a86613020-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.858 DEBUG:boto:path=/lybtug-39/obj8 2024-04-16T18:10:22.858 DEBUG:boto:auth_path=/lybtug-39/obj8 2024-04-16T18:10:22.858 DEBUG:boto:Method: GET 2024-04-16T18:10:22.858 DEBUG:boto:Path: /lybtug-39/obj8 2024-04-16T18:10:22.858 DEBUG:boto:Data: 2024-04-16T18:10:22.858 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.858 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.858 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.859 DEBUG:boto:Params: {} 2024-04-16T18:10:22.859 DEBUG:boto:Token: None 2024-04-16T18:10:22.859 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/obj8 2024-04-16T18:10:22.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9ZMKWYmVlkj079UcIyL+9DA/kxY= 2024-04-16T18:10:22.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9ZMKWYmVlkj079UcIyL+9DA/kxY=', 'Content-Length': '0'} 2024-04-16T18:10:22.862 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:09:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-39:054102a3-8d94-4596-8b3b-6341eab312e1.4804.8'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:09:58 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000880f91facc35b903-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.902 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.903 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.903 DEBUG:boto:path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.903 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.903 DEBUG:boto:Method: GET 2024-04-16T18:10:22.903 DEBUG:boto:Path: /lybtug-39/?prefix=obj8 2024-04-16T18:10:22.903 DEBUG:boto:Data: 2024-04-16T18:10:22.903 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.903 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:22.903 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.903 DEBUG:boto:Params: {} 2024-04-16T18:10:22.903 DEBUG:boto:Token: None 2024-04-16T18:10:22.903 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.908 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d89734abfbbc929-00661ebf0e-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.908 DEBUG:boto:b'lybtug-39obj81000falseobj82024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.908 DEBUG:boto:path=/lybtug-39/ 2024-04-16T18:10:22.908 DEBUG:boto:auth_path=/lybtug-39/ 2024-04-16T18:10:22.909 DEBUG:boto:path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.909 DEBUG:boto:auth_path=/lybtug-39/?prefix=obj8 2024-04-16T18:10:22.909 DEBUG:boto:Method: GET 2024-04-16T18:10:22.909 DEBUG:boto:Path: /lybtug-39/?prefix=obj8 2024-04-16T18:10:22.909 DEBUG:boto:Data: 2024-04-16T18:10:22.909 DEBUG:boto:Headers: {} 2024-04-16T18:10:22.909 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:22.909 DEBUG:boto:Port: 8000 2024-04-16T18:10:22.909 DEBUG:boto:Params: {} 2024-04-16T18:10:22.909 DEBUG:boto:Token: None 2024-04-16T18:10:22.910 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:22 GMT /lybtug-39/ 2024-04-16T18:10:22.910 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4= 2024-04-16T18:10:22.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lpb8HpqNQbBDCHbunpudJcUS5D4=', 'Content-Length': '0'} 2024-04-16T18:10:22.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000526c1fd1a78ac232-00661ebf0e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:22.954 DEBUG:boto:b'lybtug-39obj81000falseobj82024-04-16T18:09:37.718Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:22.955 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-39 zones={test-zone2, test-zone1} 2024-04-16T18:10:22.955 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2024-04-16T18:10:22.955 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:22.955 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:22.956 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"00000000000000000000:00000000000000007961","last_update":"2024-04-16T18:09:37.458121Z"},{"marker":"00000000000000000000:00000000000000006281","last_update":"2024-04-16T18:10:17.384483Z"},{"marker":"00000000000000000000:00000000000000004725","last_update":"2024-04-16T18:10:17.388001Z"},{"marker":"00000000000000000000:00000000000000009783","last_update":"2024-04-16T18:10:17.391236Z"}] 2024-04-16T18:10:23.115 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000007961', 1: '00000000000000000000:00000000000000006281', 2: '00000000000000000000:00000000000000004725', 3: '00000000000000000000:00000000000000009783'} 2024-04-16T18:10:23.116 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:10:23.116 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.116 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.116 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:23.160 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:23.160 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:23.264 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000007961","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:37.458121Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006281","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:17.384483Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004725","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:17.388001Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000009783","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:17.391236Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:10:23Z"} 2024-04-16T18:10:23.265 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000007961","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:37.458121Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006281","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:17.384483Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004725","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:17.388001Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000009783","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:17.391236Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:10:23Z"} 2024-04-16T18:10:23.265 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000007961', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:37.458121Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006281', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:17.384483Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004725', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:17.388001Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000009783', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:17.391236Z'}}] 2024-04-16T18:10:23.265 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000007961', 1: '00000000000000000000:00000000000000006281', 2: '00000000000000000000:00000000000000004725', 3: '00000000000000000000:00000000000000009783'} 2024-04-16T18:10:23.265 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000007961', 1: '00000000000000000000:00000000000000006281', 2: '00000000000000000000:00000000000000004725', 3: '00000000000000000000:00000000000000009783'} 2024-04-16T18:10:23.265 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:10:23.265 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2024-04-16T18:10:23.265 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.265 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.266 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:23.343 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:23.344 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:23.459 INFO:teuthology.orchestra.run.smithi122.stdout:[{"marker":"00000000000000000000:00000000000000005296","last_update":"2024-04-16T18:09:09.268194Z"},{"marker":"00000000000000000000:00000000000000004314","last_update":"2024-04-16T18:09:55.840489Z"},{"marker":"00000000000000000000:00000000000000003746","last_update":"2024-04-16T18:09:55.844707Z"},{"marker":"00000000000000000000:00000000000000007680","last_update":"2024-04-16T18:09:55.848777Z"}] 2024-04-16T18:10:23.459 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000005296', 1: '00000000000000000000:00000000000000004314', 2: '00000000000000000000:00000000000000003746', 3: '00000000000000000000:00000000000000007680'} 2024-04-16T18:10:23.459 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:10:23.459 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.460 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.460 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:23.504 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:23.504 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:23.607 INFO:teuthology.orchestra.run.smithi071.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005296","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.268194Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004314","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:55.840489Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003746","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T18:09:55.844707Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000007680","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:55.848777Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:10:23Z"} 2024-04-16T18:10:23.608 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000005296","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:09.268194Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004314","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:55.840489Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000003746","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T18:09:55.844707Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000007680","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:09:55.848777Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:10:23Z"} 2024-04-16T18:10:23.608 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005296', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:09.268194Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004314', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:55.840489Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003746', 'next_step_marker': '', 'total_entries': 7, 'pos': 0, 'timestamp': '2024-04-16T18:09:55.844707Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000007680', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:09:55.848777Z'}}] 2024-04-16T18:10:23.608 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000005296', 1: '00000000000000000000:00000000000000004314', 2: '00000000000000000000:00000000000000003746', 3: '00000000000000000000:00000000000000007680'} 2024-04-16T18:10:23.608 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000005296', 1: '00000000000000000000:00000000000000004314', 2: '00000000000000000000:00000000000000003746', 3: '00000000000000000000:00000000000000007680'} 2024-04-16T18:10:23.608 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:10:23.608 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_enable_right_after_disable ... ok 2024-04-16T18:10:23.610 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-40 2024-04-16T18:10:23.610 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:23.610 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:23.610 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.610 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:23.610 DEBUG:boto:Data: 2024-04-16T18:10:23.610 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.610 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.611 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.611 DEBUG:boto:Params: {} 2024-04-16T18:10:23.611 DEBUG:boto:Token: None 2024-04-16T18:10:23.611 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/ 2024-04-16T18:10:23.611 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lb38981Y44KVj8/7a3/hbioTLpw= 2024-04-16T18:10:23.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lb38981Y44KVj8/7a3/hbioTLpw=', 'Content-Length': '0'} 2024-04-16T18:10:23.641 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007d9f660261488d52-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.641 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-41 2024-04-16T18:10:23.641 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:23.641 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:23.641 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.641 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:23.641 DEBUG:boto:Data: 2024-04-16T18:10:23.641 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.642 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.642 DEBUG:boto:Params: {} 2024-04-16T18:10:23.642 DEBUG:boto:Token: None 2024-04-16T18:10:23.642 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/ 2024-04-16T18:10:23.642 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9Lunge7g+1JLTNLnJGLNxPp7eZQ= 2024-04-16T18:10:23.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9Lunge7g+1JLTNLnJGLNxPp7eZQ=', 'Content-Length': '0'} 2024-04-16T18:10:23.698 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000061467a3b2eee7a5-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.699 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:23.699 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:23.699 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.699 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:23.699 DEBUG:boto:Data: 2024-04-16T18:10:23.699 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.699 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.699 DEBUG:boto:Params: {} 2024-04-16T18:10:23.699 DEBUG:boto:Token: None 2024-04-16T18:10:23.700 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/ 2024-04-16T18:10:23.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs= 2024-04-16T18:10:23.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs=', 'Content-Length': '0'} 2024-04-16T18:10:23.703 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066e2baeb4331fab4-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.703 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:23.703 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:23.703 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.703 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:23.703 DEBUG:boto:Data: 2024-04-16T18:10:23.703 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.703 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.704 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.704 DEBUG:boto:Params: {} 2024-04-16T18:10:23.704 DEBUG:boto:Token: None 2024-04-16T18:10:23.704 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/obj1 2024-04-16T18:10:23.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dU9NDGLWwnOF1K1PJRAITUA/3+0= 2024-04-16T18:10:23.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dU9NDGLWwnOF1K1PJRAITUA/3+0='} 2024-04-16T18:10:23.716 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000c2a94324e24ba94-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.716 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:23.716 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:23.716 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.716 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:23.716 DEBUG:boto:Data: 2024-04-16T18:10:23.716 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.716 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.716 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.717 DEBUG:boto:Params: {} 2024-04-16T18:10:23.717 DEBUG:boto:Token: None 2024-04-16T18:10:23.717 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/ 2024-04-16T18:10:23.717 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs= 2024-04-16T18:10:23.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs=', 'Content-Length': '0'} 2024-04-16T18:10:23.720 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a1736af73b1c0331-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.720 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:23.720 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:23.720 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.721 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:23.721 DEBUG:boto:Data: 2024-04-16T18:10:23.721 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.721 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.721 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.721 DEBUG:boto:Params: {} 2024-04-16T18:10:23.721 DEBUG:boto:Token: None 2024-04-16T18:10:23.721 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/obj2 2024-04-16T18:10:23.721 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cQId02Kve0Lg9Fv7VSHPjgKrtBU= 2024-04-16T18:10:23.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cQId02Kve0Lg9Fv7VSHPjgKrtBU='} 2024-04-16T18:10:23.767 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005d5e62cb49d817b1-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.767 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:23.767 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:23.767 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.767 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:23.767 DEBUG:boto:Data: 2024-04-16T18:10:23.767 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.767 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.767 DEBUG:boto:Params: {} 2024-04-16T18:10:23.768 DEBUG:boto:Token: None 2024-04-16T18:10:23.768 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/ 2024-04-16T18:10:23.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs= 2024-04-16T18:10:23.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs=', 'Content-Length': '0'} 2024-04-16T18:10:23.771 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000053a6003a201b1b29-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.772 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:23.772 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:23.772 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.772 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:23.772 DEBUG:boto:Data: 2024-04-16T18:10:23.772 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.772 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.772 DEBUG:boto:Params: {} 2024-04-16T18:10:23.772 DEBUG:boto:Token: None 2024-04-16T18:10:23.773 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/obj3 2024-04-16T18:10:23.773 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uiBR4+ZlNe3X/tjX5c2wbe101qY= 2024-04-16T18:10:23.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uiBR4+ZlNe3X/tjX5c2wbe101qY='} 2024-04-16T18:10:23.784 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006857464552315236-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.785 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:23.785 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:23.785 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.785 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:23.785 DEBUG:boto:Data: 2024-04-16T18:10:23.785 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.785 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.785 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.785 DEBUG:boto:Params: {} 2024-04-16T18:10:23.785 DEBUG:boto:Token: None 2024-04-16T18:10:23.786 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/ 2024-04-16T18:10:23.786 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs= 2024-04-16T18:10:23.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z7nLA9AuSa0NcvL2EFL46l7YUNs=', 'Content-Length': '0'} 2024-04-16T18:10:23.789 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000137ac19f7290ec3c-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.789 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:23.789 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:23.789 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.789 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:23.790 DEBUG:boto:Data: 2024-04-16T18:10:23.790 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.790 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:23.790 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.790 DEBUG:boto:Params: {} 2024-04-16T18:10:23.790 DEBUG:boto:Token: None 2024-04-16T18:10:23.790 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-40/obj4 2024-04-16T18:10:23.790 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mBojLN7sb8tpiTZJ4cFePILa7Do= 2024-04-16T18:10:23.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mBojLN7sb8tpiTZJ4cFePILa7Do='} 2024-04-16T18:10:23.834 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ee66bd0efc751a02-00661ebf0f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.835 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:23.835 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:23.835 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.835 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:23.835 DEBUG:boto:Data: 2024-04-16T18:10:23.835 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.835 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.835 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.835 DEBUG:boto:Params: {} 2024-04-16T18:10:23.835 DEBUG:boto:Token: None 2024-04-16T18:10:23.836 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/ 2024-04-16T18:10:23.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA= 2024-04-16T18:10:23.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA=', 'Content-Length': '0'} 2024-04-16T18:10:23.840 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c91de863f093b7b0-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.840 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:23.840 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:23.840 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.841 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:23.841 DEBUG:boto:Data: 2024-04-16T18:10:23.841 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.841 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.841 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.841 DEBUG:boto:Params: {} 2024-04-16T18:10:23.841 DEBUG:boto:Token: None 2024-04-16T18:10:23.841 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/obj1 2024-04-16T18:10:23.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sqSlZfMMafedXYs/N7eVSlrJ4Mo= 2024-04-16T18:10:23.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sqSlZfMMafedXYs/N7eVSlrJ4Mo='} 2024-04-16T18:10:23.886 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b73fc9ec9cc72d04-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.887 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:23.887 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:23.887 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.887 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:23.887 DEBUG:boto:Data: 2024-04-16T18:10:23.887 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.887 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.887 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.887 DEBUG:boto:Params: {} 2024-04-16T18:10:23.887 DEBUG:boto:Token: None 2024-04-16T18:10:23.887 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/ 2024-04-16T18:10:23.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA= 2024-04-16T18:10:23.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA=', 'Content-Length': '0'} 2024-04-16T18:10:23.891 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000011751658f6485448-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.891 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:23.891 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:23.892 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.892 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:23.892 DEBUG:boto:Data: 2024-04-16T18:10:23.892 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.892 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.892 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.892 DEBUG:boto:Params: {} 2024-04-16T18:10:23.892 DEBUG:boto:Token: None 2024-04-16T18:10:23.892 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/obj2 2024-04-16T18:10:23.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a/6rRFvRv4FjWni2vExqW0lT+c0= 2024-04-16T18:10:23.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a/6rRFvRv4FjWni2vExqW0lT+c0='} 2024-04-16T18:10:23.905 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003d1213efe76e5550-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.905 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:23.905 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:23.905 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.906 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:23.906 DEBUG:boto:Data: 2024-04-16T18:10:23.906 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.906 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.906 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.906 DEBUG:boto:Params: {} 2024-04-16T18:10:23.906 DEBUG:boto:Token: None 2024-04-16T18:10:23.906 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/ 2024-04-16T18:10:23.906 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA= 2024-04-16T18:10:23.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA=', 'Content-Length': '0'} 2024-04-16T18:10:23.910 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a4a2700c364d3d87-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.910 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:23.910 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:23.910 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.910 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:23.911 DEBUG:boto:Data: 2024-04-16T18:10:23.911 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.911 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.911 DEBUG:boto:Params: {} 2024-04-16T18:10:23.911 DEBUG:boto:Token: None 2024-04-16T18:10:23.911 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/obj3 2024-04-16T18:10:23.911 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P8u6wmJeQ2/RJ64WhlOQOTAgzxE= 2024-04-16T18:10:23.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P8u6wmJeQ2/RJ64WhlOQOTAgzxE='} 2024-04-16T18:10:23.954 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bf4e944a029b8ccf-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.955 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:23.955 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:23.955 DEBUG:boto:Method: HEAD 2024-04-16T18:10:23.955 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:23.955 DEBUG:boto:Data: 2024-04-16T18:10:23.955 DEBUG:boto:Headers: {} 2024-04-16T18:10:23.955 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.955 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.955 DEBUG:boto:Params: {} 2024-04-16T18:10:23.955 DEBUG:boto:Token: None 2024-04-16T18:10:23.955 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/ 2024-04-16T18:10:23.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA= 2024-04-16T18:10:23.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uGgTM+3XuoKyivjBiEI3Wb2AlAA=', 'Content-Length': '0'} 2024-04-16T18:10:23.959 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ccb4c08eaacbf727-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.960 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:23.960 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:23.960 DEBUG:boto:Method: PUT 2024-04-16T18:10:23.960 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:23.960 DEBUG:boto:Data: 2024-04-16T18:10:23.960 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:23.960 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:23.960 DEBUG:boto:Port: 8000 2024-04-16T18:10:23.960 DEBUG:boto:Params: {} 2024-04-16T18:10:23.960 DEBUG:boto:Token: None 2024-04-16T18:10:23.961 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:23 GMT /lybtug-41/obj4 2024-04-16T18:10:23.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LWQenxDCdP3rUs+8ZNxsjasIBMQ= 2024-04-16T18:10:23.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LWQenxDCdP3rUs+8ZNxsjasIBMQ='} 2024-04-16T18:10:23.973 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f0a235fe10885e62-00661ebf0f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:23.973 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.973 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:23.973 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:24.015 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:24.015 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:24.084 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:10:24.118 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291023.678388_399.1","last_update":"2024-04-16T18:10:23.678388Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:10:24.118 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291023.678388_399.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:24.118 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:10:24.118 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:24.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:24.119 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:24.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:24.243+0000 7f9877771980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:24.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:24.243+0000 7f9877771980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:24.271 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291023.678388_399.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:23.678388Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:24Z"} 2024-04-16T18:10:24.272 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291023.678388_399.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:23.678388Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:24Z"} 2024-04-16T18:10:24.272 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291023.678388_399.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:10:23.678388Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:10:24.272 DEBUG:rgw_multi.tests:log_status={0: '1_1713291023.678388_399.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:24.272 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291023.678388_399.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:24.272 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:10:24.272 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-40 2024-04-16T18:10:24.272 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:24.273 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:24.273 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-40 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:24.359 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:24.359 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:24.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:24.443+0000 7efd7e28d980 1 bucket sync caught up with source: 2024-04-16T18:10:24.441 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.33.6, , 00000000001.37.6, , 00000000001.42.6, , , 00000000001.25.6, , , ] 2024-04-16T18:10:24.441 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.33.6, , 00000000001.37.6, , 00000000001.42.6, , , 00000000001.25.6, , , ] 2024-04-16T18:10:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:24.443+0000 7efd7e28d980 0 bucket checkpoint complete 2024-04-16T18:10:24.470 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-40 2024-04-16T18:10:24.471 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:24.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:24.471 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-40 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:24.521 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:24.521 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:24.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:24.596+0000 7fecf4236980 1 waiting to reach incremental sync.. 2024-04-16T18:10:29.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:29.600+0000 7fecf4236980 1 bucket sync caught up with source: 2024-04-16T18:10:29.602 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.293.6, , 00000000001.388.6, , 00000000001.542.6, , , 00000000001.387.6, , , ] 2024-04-16T18:10:29.602 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000001.293.6, , 00000000001.388.6, , 00000000001.542.6, , , 00000000001.387.6, , , ] 2024-04-16T18:10:29.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:29.600+0000 7fecf4236980 0 bucket checkpoint complete 2024-04-16T18:10:29.645 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-40 zones={test-zone2, test-zone1} 2024-04-16T18:10:29.645 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.645 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.646 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.646 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:29.646 DEBUG:boto:Data: 2024-04-16T18:10:29.646 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.646 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.646 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.646 DEBUG:boto:Params: {} 2024-04-16T18:10:29.646 DEBUG:boto:Token: None 2024-04-16T18:10:29.647 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.647 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wm3hQODv5aOaPEhRUiQvo6OM9kA= 2024-04-16T18:10:29.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wm3hQODv5aOaPEhRUiQvo6OM9kA=', 'Content-Length': '0'} 2024-04-16T18:10:29.652 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000020c7d9b23e2ba85f-00661ebf15-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.652 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.652 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.652 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.652 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:29.652 DEBUG:boto:Data: 2024-04-16T18:10:29.653 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.653 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.653 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.653 DEBUG:boto:Params: {} 2024-04-16T18:10:29.653 DEBUG:boto:Token: None 2024-04-16T18:10:29.653 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.653 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wm3hQODv5aOaPEhRUiQvo6OM9kA= 2024-04-16T18:10:29.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wm3hQODv5aOaPEhRUiQvo6OM9kA=', 'Content-Length': '0'} 2024-04-16T18:10:29.657 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fba7768770136def-00661ebf15-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.657 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:10:29.657 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.657 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.657 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.657 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.658 DEBUG:boto:Method: GET 2024-04-16T18:10:29.658 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.658 DEBUG:boto:Data: 2024-04-16T18:10:29.658 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.658 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.658 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.658 DEBUG:boto:Params: {} 2024-04-16T18:10:29.658 DEBUG:boto:Token: None 2024-04-16T18:10:29.658 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/?versions 2024-04-16T18:10:29.659 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8= 2024-04-16T18:10:29.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8=', 'Content-Length': '0'} 2024-04-16T18:10:29.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007c33d74a6e2aac9-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.664 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.664 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:29.664 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:29.664 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:29.665 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:29.665 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:10:29.665 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.665 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.665 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.665 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.665 DEBUG:boto:Method: GET 2024-04-16T18:10:29.665 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.665 DEBUG:boto:Data: 2024-04-16T18:10:29.665 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.666 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.666 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.666 DEBUG:boto:Params: {} 2024-04-16T18:10:29.666 DEBUG:boto:Token: None 2024-04-16T18:10:29.666 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/?versions 2024-04-16T18:10:29.666 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8= 2024-04-16T18:10:29.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8=', 'Content-Length': '0'} 2024-04-16T18:10:29.670 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a88b14e1e3ec90f-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.670 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.671 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:29.671 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:29.671 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:29.671 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:29.671 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.671 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.671 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.671 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.672 DEBUG:boto:Method: GET 2024-04-16T18:10:29.672 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.672 DEBUG:boto:Data: 2024-04-16T18:10:29.672 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.672 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.672 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.672 DEBUG:boto:Params: {} 2024-04-16T18:10:29.672 DEBUG:boto:Token: None 2024-04-16T18:10:29.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/?versions 2024-04-16T18:10:29.673 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8= 2024-04-16T18:10:29.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8=', 'Content-Length': '0'} 2024-04-16T18:10:29.677 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091f7ed24100659ac-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.677 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.677 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.678 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.678 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.678 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.678 DEBUG:boto:Method: GET 2024-04-16T18:10:29.678 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:29.678 DEBUG:boto:Data: 2024-04-16T18:10:29.678 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.678 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.678 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.679 DEBUG:boto:Params: {} 2024-04-16T18:10:29.679 DEBUG:boto:Token: None 2024-04-16T18:10:29.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/?versions 2024-04-16T18:10:29.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8= 2024-04-16T18:10:29.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8bxbi/1tuLe/RHsdTF1phqHj8=', 'Content-Length': '0'} 2024-04-16T18:10:29.682 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020b4e11e037c1184-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.683 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.683 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:29.683 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.683 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.683 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.683 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.684 DEBUG:boto:Method: GET 2024-04-16T18:10:29.684 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.684 DEBUG:boto:Data: 2024-04-16T18:10:29.684 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.684 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.684 DEBUG:boto:Params: {} 2024-04-16T18:10:29.684 DEBUG:boto:Token: None 2024-04-16T18:10:29.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc= 2024-04-16T18:10:29.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc=', 'Content-Length': '0'} 2024-04-16T18:10:29.688 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000736ab165759ff678-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.730 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.731 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.731 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.731 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.731 DEBUG:boto:Method: GET 2024-04-16T18:10:29.731 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.731 DEBUG:boto:Data: 2024-04-16T18:10:29.731 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.731 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.731 DEBUG:boto:Params: {} 2024-04-16T18:10:29.732 DEBUG:boto:Token: None 2024-04-16T18:10:29.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc= 2024-04-16T18:10:29.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc=', 'Content-Length': '0'} 2024-04-16T18:10:29.735 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005a81e0621f5ea768-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.735 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.736 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.736 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.736 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.736 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.736 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.736 DEBUG:boto:Data: 2024-04-16T18:10:29.736 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.736 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.736 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.736 DEBUG:boto:Params: {} 2024-04-16T18:10:29.737 DEBUG:boto:Token: None 2024-04-16T18:10:29.737 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0z8FZR4SpT7uYsy8h84ijOheYu8= 2024-04-16T18:10:29.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0z8FZR4SpT7uYsy8h84ijOheYu8=', 'Content-Length': '0'} 2024-04-16T18:10:29.740 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008ca7958b69f9f8a4-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.740 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.741 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.741 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.741 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.741 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.741 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.741 DEBUG:boto:Data: 2024-04-16T18:10:29.741 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.741 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.741 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.741 DEBUG:boto:Params: {} 2024-04-16T18:10:29.742 DEBUG:boto:Token: None 2024-04-16T18:10:29.742 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.742 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0z8FZR4SpT7uYsy8h84ijOheYu8= 2024-04-16T18:10:29.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0z8FZR4SpT7uYsy8h84ijOheYu8=', 'Content-Length': '0'} 2024-04-16T18:10:29.745 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000027ae41bea4042890-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.745 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:29.745 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.746 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.746 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.746 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.746 DEBUG:boto:Method: GET 2024-04-16T18:10:29.746 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.746 DEBUG:boto:Data: 2024-04-16T18:10:29.746 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.746 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.746 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.746 DEBUG:boto:Params: {} 2024-04-16T18:10:29.747 DEBUG:boto:Token: None 2024-04-16T18:10:29.747 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.747 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc= 2024-04-16T18:10:29.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc=', 'Content-Length': '0'} 2024-04-16T18:10:29.750 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000059c78f4111ff0e79-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.750 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.751 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.751 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.751 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.751 DEBUG:boto:Method: GET 2024-04-16T18:10:29.751 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.751 DEBUG:boto:Data: 2024-04-16T18:10:29.751 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.751 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.751 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.752 DEBUG:boto:Params: {} 2024-04-16T18:10:29.752 DEBUG:boto:Token: None 2024-04-16T18:10:29.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:29.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc= 2024-04-16T18:10:29.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nhDJH5PbgucFHj4Al+tRFhSCiLc=', 'Content-Length': '0'} 2024-04-16T18:10:29.755 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004b723dc244a1740f-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.755 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.756 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.756 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.756 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.756 DEBUG:boto:Method: GET 2024-04-16T18:10:29.756 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:29.756 DEBUG:boto:Data: 2024-04-16T18:10:29.756 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.756 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.756 DEBUG:boto:Params: {} 2024-04-16T18:10:29.757 DEBUG:boto:Token: None 2024-04-16T18:10:29.757 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk= 2024-04-16T18:10:29.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk=', 'Content-Length': '0'} 2024-04-16T18:10:29.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000961410d9a66e90da-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.762 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.762 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.763 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.763 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.763 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.763 DEBUG:boto:Method: GET 2024-04-16T18:10:29.763 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:29.763 DEBUG:boto:Data: 2024-04-16T18:10:29.763 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.763 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.763 DEBUG:boto:Params: {} 2024-04-16T18:10:29.764 DEBUG:boto:Token: None 2024-04-16T18:10:29.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk= 2024-04-16T18:10:29.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk=', 'Content-Length': '0'} 2024-04-16T18:10:29.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6f74a0f4e64d461-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.810 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.811 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.811 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.811 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.811 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:29.811 DEBUG:boto:Data: 2024-04-16T18:10:29.811 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.812 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.812 DEBUG:boto:Params: {} 2024-04-16T18:10:29.812 DEBUG:boto:Token: None 2024-04-16T18:10:29.812 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1 2024-04-16T18:10:29.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xfu5Cl56kEHYGswE2RaRU16KupQ= 2024-04-16T18:10:29.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xfu5Cl56kEHYGswE2RaRU16KupQ=', 'Content-Length': '0'} 2024-04-16T18:10:29.816 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000230b3d6661ab98b9-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.816 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.816 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.816 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.816 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:29.816 DEBUG:boto:Data: 2024-04-16T18:10:29.817 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.817 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.817 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.817 DEBUG:boto:Params: {} 2024-04-16T18:10:29.817 DEBUG:boto:Token: None 2024-04-16T18:10:29.817 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1 2024-04-16T18:10:29.817 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xfu5Cl56kEHYGswE2RaRU16KupQ= 2024-04-16T18:10:29.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xfu5Cl56kEHYGswE2RaRU16KupQ=', 'Content-Length': '0'} 2024-04-16T18:10:29.820 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008cb90909dd1f16c7-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.821 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:29.821 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.821 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.821 DEBUG:boto:Method: GET 2024-04-16T18:10:29.821 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:29.821 DEBUG:boto:Data: 2024-04-16T18:10:29.821 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.821 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.822 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.822 DEBUG:boto:Params: {} 2024-04-16T18:10:29.822 DEBUG:boto:Token: None 2024-04-16T18:10:29.822 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1 2024-04-16T18:10:29.822 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m65mzXZ9jaYkqo3TLUAV5yPDGp0= 2024-04-16T18:10:29.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m65mzXZ9jaYkqo3TLUAV5yPDGp0=', 'Content-Length': '0'} 2024-04-16T18:10:29.826 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009ab7d691cbf32672-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.866 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:29.867 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:29.867 DEBUG:boto:Method: GET 2024-04-16T18:10:29.867 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:29.867 DEBUG:boto:Data: 2024-04-16T18:10:29.867 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.867 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.867 DEBUG:boto:Params: {} 2024-04-16T18:10:29.867 DEBUG:boto:Token: None 2024-04-16T18:10:29.868 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj1 2024-04-16T18:10:29.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m65mzXZ9jaYkqo3TLUAV5yPDGp0= 2024-04-16T18:10:29.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m65mzXZ9jaYkqo3TLUAV5yPDGp0=', 'Content-Length': '0'} 2024-04-16T18:10:29.871 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cd700b4ab82dc63f-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.872 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.872 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.872 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.872 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.872 DEBUG:boto:Method: GET 2024-04-16T18:10:29.872 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:29.872 DEBUG:boto:Data: 2024-04-16T18:10:29.872 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.872 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.872 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.872 DEBUG:boto:Params: {} 2024-04-16T18:10:29.873 DEBUG:boto:Token: None 2024-04-16T18:10:29.873 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.873 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk= 2024-04-16T18:10:29.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk=', 'Content-Length': '0'} 2024-04-16T18:10:29.877 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d8693a50675d548-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.877 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.877 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.878 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.878 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.878 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:29.878 DEBUG:boto:Method: GET 2024-04-16T18:10:29.878 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:29.878 DEBUG:boto:Data: 2024-04-16T18:10:29.878 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.878 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.879 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.879 DEBUG:boto:Params: {} 2024-04-16T18:10:29.879 DEBUG:boto:Token: None 2024-04-16T18:10:29.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk= 2024-04-16T18:10:29.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk=', 'Content-Length': '0'} 2024-04-16T18:10:29.883 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ef3e45e805d22af-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.883 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:29.883 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:29.883 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:29.883 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:29.884 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.884 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.884 DEBUG:boto:Method: GET 2024-04-16T18:10:29.884 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.884 DEBUG:boto:Data: 2024-04-16T18:10:29.884 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.884 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.884 DEBUG:boto:Params: {} 2024-04-16T18:10:29.884 DEBUG:boto:Token: None 2024-04-16T18:10:29.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.885 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI= 2024-04-16T18:10:29.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI=', 'Content-Length': '0'} 2024-04-16T18:10:29.888 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b142bca8e3d7c6dd-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.930 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:29.931 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:29.931 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.931 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.931 DEBUG:boto:Method: GET 2024-04-16T18:10:29.931 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.931 DEBUG:boto:Data: 2024-04-16T18:10:29.931 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.931 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.931 DEBUG:boto:Params: {} 2024-04-16T18:10:29.932 DEBUG:boto:Token: None 2024-04-16T18:10:29.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI= 2024-04-16T18:10:29.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI=', 'Content-Length': '0'} 2024-04-16T18:10:29.935 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a4aec20787d0a2c7-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.935 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:29.935 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:29.936 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.936 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.936 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.936 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.936 DEBUG:boto:Data: 2024-04-16T18:10:29.936 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.936 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.936 DEBUG:boto:Params: {} 2024-04-16T18:10:29.936 DEBUG:boto:Token: None 2024-04-16T18:10:29.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.937 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GhmZgQ/zMpbh/JuZAGgrApznAkE= 2024-04-16T18:10:29.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GhmZgQ/zMpbh/JuZAGgrApznAkE=', 'Content-Length': '0'} 2024-04-16T18:10:29.940 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ecf25a3b823e4ce1-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.940 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:29.940 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:29.940 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.940 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.940 DEBUG:boto:Method: HEAD 2024-04-16T18:10:29.940 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.940 DEBUG:boto:Data: 2024-04-16T18:10:29.941 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.941 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.941 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.941 DEBUG:boto:Params: {} 2024-04-16T18:10:29.941 DEBUG:boto:Token: None 2024-04-16T18:10:29.941 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.941 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GhmZgQ/zMpbh/JuZAGgrApznAkE= 2024-04-16T18:10:29.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GhmZgQ/zMpbh/JuZAGgrApznAkE=', 'Content-Length': '0'} 2024-04-16T18:10:29.944 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000226bcd931d5c4c18-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.945 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:29.945 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:29.945 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:29.945 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.945 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.945 DEBUG:boto:Method: GET 2024-04-16T18:10:29.945 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.945 DEBUG:boto:Data: 2024-04-16T18:10:29.945 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.945 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.945 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.946 DEBUG:boto:Params: {} 2024-04-16T18:10:29.946 DEBUG:boto:Token: None 2024-04-16T18:10:29.946 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.946 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI= 2024-04-16T18:10:29.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI=', 'Content-Length': '0'} 2024-04-16T18:10:29.949 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d5c66a6353cf1707-00661ebf15-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.990 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:29.991 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:29.991 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.991 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.991 DEBUG:boto:Method: GET 2024-04-16T18:10:29.991 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.991 DEBUG:boto:Data: 2024-04-16T18:10:29.991 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:29.991 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.991 DEBUG:boto:Params: {} 2024-04-16T18:10:29.991 DEBUG:boto:Token: None 2024-04-16T18:10:29.992 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:29.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI= 2024-04-16T18:10:29.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SCFc7orksySTz/q8+R22nqsckwI=', 'Content-Length': '0'} 2024-04-16T18:10:29.995 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000089ad0a49d84aee81-00661ebf15-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:29 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:29.995 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:29.995 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:29.996 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:29.996 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:29.996 DEBUG:boto:Method: GET 2024-04-16T18:10:29.996 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:29.996 DEBUG:boto:Data: 2024-04-16T18:10:29.996 DEBUG:boto:Headers: {} 2024-04-16T18:10:29.996 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:29.996 DEBUG:boto:Port: 8000 2024-04-16T18:10:29.996 DEBUG:boto:Params: {} 2024-04-16T18:10:29.996 DEBUG:boto:Token: None 2024-04-16T18:10:29.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:29 GMT /lybtug-40/ 2024-04-16T18:10:29.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk= 2024-04-16T18:10:29.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:29 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fNOzueQmo9xl1SN5R0xbG9zD/Gk=', 'Content-Length': '0'} 2024-04-16T18:10:30.001 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009719f25bf4878920-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.001 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.001 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.001 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.002 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:30.002 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:30.002 DEBUG:boto:Method: GET 2024-04-16T18:10:30.002 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:30.002 DEBUG:boto:Data: 2024-04-16T18:10:30.002 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.002 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.002 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.002 DEBUG:boto:Params: {} 2024-04-16T18:10:30.002 DEBUG:boto:Token: None 2024-04-16T18:10:30.002 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.006 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033734b0ddebc941e-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.007 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.007 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:30.007 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:30.007 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.007 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:30.007 DEBUG:boto:Data: 2024-04-16T18:10:30.007 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.007 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.008 DEBUG:boto:Params: {} 2024-04-16T18:10:30.008 DEBUG:boto:Token: None 2024-04-16T18:10:30.008 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj2 2024-04-16T18:10:30.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5iUdSHEr4XmDAo73Sn+VHsl6KSE= 2024-04-16T18:10:30.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5iUdSHEr4XmDAo73Sn+VHsl6KSE=', 'Content-Length': '0'} 2024-04-16T18:10:30.011 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000032fec93f1ea220d0-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.011 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:30.011 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:30.011 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.011 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:30.011 DEBUG:boto:Data: 2024-04-16T18:10:30.011 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.012 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.012 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.012 DEBUG:boto:Params: {} 2024-04-16T18:10:30.012 DEBUG:boto:Token: None 2024-04-16T18:10:30.012 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj2 2024-04-16T18:10:30.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5iUdSHEr4XmDAo73Sn+VHsl6KSE= 2024-04-16T18:10:30.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5iUdSHEr4XmDAo73Sn+VHsl6KSE=', 'Content-Length': '0'} 2024-04-16T18:10:30.015 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000097da11d9fc2e1e51-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.015 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:30.015 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:30.015 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:30.016 DEBUG:boto:Method: GET 2024-04-16T18:10:30.016 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:30.016 DEBUG:boto:Data: 2024-04-16T18:10:30.016 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.016 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.016 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.016 DEBUG:boto:Params: {} 2024-04-16T18:10:30.016 DEBUG:boto:Token: None 2024-04-16T18:10:30.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj2 2024-04-16T18:10:30.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LoTV2qAq2lJAN28rmY1xcF8qFk0= 2024-04-16T18:10:30.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LoTV2qAq2lJAN28rmY1xcF8qFk0=', 'Content-Length': '0'} 2024-04-16T18:10:30.020 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000302f3b2898ed87ed-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.062 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:30.063 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:30.063 DEBUG:boto:Method: GET 2024-04-16T18:10:30.063 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:30.063 DEBUG:boto:Data: 2024-04-16T18:10:30.063 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.063 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.063 DEBUG:boto:Params: {} 2024-04-16T18:10:30.063 DEBUG:boto:Token: None 2024-04-16T18:10:30.063 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj2 2024-04-16T18:10:30.063 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LoTV2qAq2lJAN28rmY1xcF8qFk0= 2024-04-16T18:10:30.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LoTV2qAq2lJAN28rmY1xcF8qFk0=', 'Content-Length': '0'} 2024-04-16T18:10:30.067 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000aa9d5b2157213861-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.067 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.067 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.067 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:30.067 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:30.067 DEBUG:boto:Method: GET 2024-04-16T18:10:30.067 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:30.067 DEBUG:boto:Data: 2024-04-16T18:10:30.068 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.068 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.068 DEBUG:boto:Params: {} 2024-04-16T18:10:30.068 DEBUG:boto:Token: None 2024-04-16T18:10:30.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.073 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cae2bbdeabc4495-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.073 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.073 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.073 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.074 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:30.074 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:30.074 DEBUG:boto:Method: GET 2024-04-16T18:10:30.074 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:30.074 DEBUG:boto:Data: 2024-04-16T18:10:30.074 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.074 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.074 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.074 DEBUG:boto:Params: {} 2024-04-16T18:10:30.074 DEBUG:boto:Token: None 2024-04-16T18:10:30.074 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.075 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3456593750d2fec-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.122 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.123 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:30.123 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.123 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.123 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.123 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.123 DEBUG:boto:Method: GET 2024-04-16T18:10:30.123 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.123 DEBUG:boto:Data: 2024-04-16T18:10:30.123 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.123 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.124 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.124 DEBUG:boto:Params: {} 2024-04-16T18:10:30.124 DEBUG:boto:Token: None 2024-04-16T18:10:30.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc= 2024-04-16T18:10:30.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc=', 'Content-Length': '0'} 2024-04-16T18:10:30.127 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000029c044f974e31ce-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.170 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.171 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.171 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.171 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.171 DEBUG:boto:Method: GET 2024-04-16T18:10:30.171 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.171 DEBUG:boto:Data: 2024-04-16T18:10:30.171 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.171 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.171 DEBUG:boto:Params: {} 2024-04-16T18:10:30.171 DEBUG:boto:Token: None 2024-04-16T18:10:30.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc= 2024-04-16T18:10:30.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc=', 'Content-Length': '0'} 2024-04-16T18:10:30.174 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007cfd07199bc823f7-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.175 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.175 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.175 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.175 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.175 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.175 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.175 DEBUG:boto:Data: 2024-04-16T18:10:30.175 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.175 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.175 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.176 DEBUG:boto:Params: {} 2024-04-16T18:10:30.176 DEBUG:boto:Token: None 2024-04-16T18:10:30.176 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X3XJymLfKMq35jZFXPJbm5mqYfw= 2024-04-16T18:10:30.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X3XJymLfKMq35jZFXPJbm5mqYfw=', 'Content-Length': '0'} 2024-04-16T18:10:30.179 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e7787a0d8f4beff7-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.179 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.179 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.179 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.179 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.179 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.179 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.179 DEBUG:boto:Data: 2024-04-16T18:10:30.180 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.180 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.180 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.180 DEBUG:boto:Params: {} 2024-04-16T18:10:30.180 DEBUG:boto:Token: None 2024-04-16T18:10:30.180 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X3XJymLfKMq35jZFXPJbm5mqYfw= 2024-04-16T18:10:30.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X3XJymLfKMq35jZFXPJbm5mqYfw=', 'Content-Length': '0'} 2024-04-16T18:10:30.183 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002bea91ed3a7b25d9-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.183 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:30.183 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.183 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.183 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.184 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.184 DEBUG:boto:Method: GET 2024-04-16T18:10:30.184 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.184 DEBUG:boto:Data: 2024-04-16T18:10:30.184 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.184 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.184 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.184 DEBUG:boto:Params: {} 2024-04-16T18:10:30.184 DEBUG:boto:Token: None 2024-04-16T18:10:30.184 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc= 2024-04-16T18:10:30.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc=', 'Content-Length': '0'} 2024-04-16T18:10:30.187 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009913094a7d443797-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.230 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.230 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.231 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.231 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.231 DEBUG:boto:Method: GET 2024-04-16T18:10:30.231 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.231 DEBUG:boto:Data: 2024-04-16T18:10:30.231 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.231 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.231 DEBUG:boto:Params: {} 2024-04-16T18:10:30.231 DEBUG:boto:Token: None 2024-04-16T18:10:30.231 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:30.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc= 2024-04-16T18:10:30.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5mTTlGJimYWgZX7Xk4WAb+SOLRc=', 'Content-Length': '0'} 2024-04-16T18:10:30.234 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000054f6627c239f4996-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.235 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.235 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.235 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.235 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.235 DEBUG:boto:Method: GET 2024-04-16T18:10:30.235 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:30.235 DEBUG:boto:Data: 2024-04-16T18:10:30.235 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.235 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.235 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.236 DEBUG:boto:Params: {} 2024-04-16T18:10:30.236 DEBUG:boto:Token: None 2024-04-16T18:10:30.236 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b8c62970d098b9e-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.240 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.241 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.241 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.241 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.241 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.241 DEBUG:boto:Method: GET 2024-04-16T18:10:30.241 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:30.241 DEBUG:boto:Data: 2024-04-16T18:10:30.241 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.241 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.241 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.241 DEBUG:boto:Params: {} 2024-04-16T18:10:30.242 DEBUG:boto:Token: None 2024-04-16T18:10:30.242 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.242 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.245 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005abc6b6b2570733e-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.245 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.246 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.246 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.246 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.246 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:30.246 DEBUG:boto:Data: 2024-04-16T18:10:30.246 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.246 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.246 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.247 DEBUG:boto:Params: {} 2024-04-16T18:10:30.247 DEBUG:boto:Token: None 2024-04-16T18:10:30.247 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3 2024-04-16T18:10:30.247 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rAft7QQ5k+kgvE1Jqft5Ie+pKrg= 2024-04-16T18:10:30.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rAft7QQ5k+kgvE1Jqft5Ie+pKrg=', 'Content-Length': '0'} 2024-04-16T18:10:30.250 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bf3662671a1639be-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.250 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.250 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.250 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.250 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:30.250 DEBUG:boto:Data: 2024-04-16T18:10:30.250 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.251 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.251 DEBUG:boto:Params: {} 2024-04-16T18:10:30.251 DEBUG:boto:Token: None 2024-04-16T18:10:30.251 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3 2024-04-16T18:10:30.251 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rAft7QQ5k+kgvE1Jqft5Ie+pKrg= 2024-04-16T18:10:30.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rAft7QQ5k+kgvE1Jqft5Ie+pKrg=', 'Content-Length': '0'} 2024-04-16T18:10:30.254 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d469e50496d0eb2b-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.254 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:30.254 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.254 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.254 DEBUG:boto:Method: GET 2024-04-16T18:10:30.255 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:30.255 DEBUG:boto:Data: 2024-04-16T18:10:30.255 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.255 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.255 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.255 DEBUG:boto:Params: {} 2024-04-16T18:10:30.255 DEBUG:boto:Token: None 2024-04-16T18:10:30.255 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3 2024-04-16T18:10:30.255 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kadsbo81RghQWMveJhYSHn9ZaQ0= 2024-04-16T18:10:30.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kadsbo81RghQWMveJhYSHn9ZaQ0=', 'Content-Length': '0'} 2024-04-16T18:10:30.258 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dc8455a67618684f-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.302 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:30.303 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:30.303 DEBUG:boto:Method: GET 2024-04-16T18:10:30.303 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:30.303 DEBUG:boto:Data: 2024-04-16T18:10:30.303 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.303 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.303 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.304 DEBUG:boto:Params: {} 2024-04-16T18:10:30.304 DEBUG:boto:Token: None 2024-04-16T18:10:30.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj3 2024-04-16T18:10:30.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kadsbo81RghQWMveJhYSHn9ZaQ0= 2024-04-16T18:10:30.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kadsbo81RghQWMveJhYSHn9ZaQ0=', 'Content-Length': '0'} 2024-04-16T18:10:30.307 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000013024eb52cf1b477-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.307 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.308 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.308 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.308 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.308 DEBUG:boto:Method: GET 2024-04-16T18:10:30.308 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:30.308 DEBUG:boto:Data: 2024-04-16T18:10:30.308 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.308 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.308 DEBUG:boto:Params: {} 2024-04-16T18:10:30.309 DEBUG:boto:Token: None 2024-04-16T18:10:30.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.313 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000479a554aac431713-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.354 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.355 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.355 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.355 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.355 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:30.355 DEBUG:boto:Method: GET 2024-04-16T18:10:30.355 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:30.355 DEBUG:boto:Data: 2024-04-16T18:10:30.355 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.356 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.356 DEBUG:boto:Params: {} 2024-04-16T18:10:30.356 DEBUG:boto:Token: None 2024-04-16T18:10:30.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c81e5f6f63ba65a-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.360 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.361 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:30.361 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.361 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.361 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.361 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.361 DEBUG:boto:Method: GET 2024-04-16T18:10:30.361 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.361 DEBUG:boto:Data: 2024-04-16T18:10:30.362 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.362 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.362 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.362 DEBUG:boto:Params: {} 2024-04-16T18:10:30.362 DEBUG:boto:Token: None 2024-04-16T18:10:30.362 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.362 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28= 2024-04-16T18:10:30.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28=', 'Content-Length': '0'} 2024-04-16T18:10:30.366 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000020c72d80e5dc8d8f-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.406 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.406 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.407 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.407 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.407 DEBUG:boto:Method: GET 2024-04-16T18:10:30.407 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.407 DEBUG:boto:Data: 2024-04-16T18:10:30.407 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.407 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.407 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.407 DEBUG:boto:Params: {} 2024-04-16T18:10:30.407 DEBUG:boto:Token: None 2024-04-16T18:10:30.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28= 2024-04-16T18:10:30.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28=', 'Content-Length': '0'} 2024-04-16T18:10:30.411 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000abe5378afbb0ecb0-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.411 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.411 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.412 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.412 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.412 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.412 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.412 DEBUG:boto:Data: 2024-04-16T18:10:30.412 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.412 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.412 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.412 DEBUG:boto:Params: {} 2024-04-16T18:10:30.412 DEBUG:boto:Token: None 2024-04-16T18:10:30.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.413 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZELVzWtZTRUsKRHSerrtU2D//6s= 2024-04-16T18:10:30.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZELVzWtZTRUsKRHSerrtU2D//6s=', 'Content-Length': '0'} 2024-04-16T18:10:30.416 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009eb24a1e1b7976a4-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.416 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.416 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.416 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.416 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.417 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.417 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.417 DEBUG:boto:Data: 2024-04-16T18:10:30.417 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.417 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.417 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.417 DEBUG:boto:Params: {} 2024-04-16T18:10:30.417 DEBUG:boto:Token: None 2024-04-16T18:10:30.417 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.417 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZELVzWtZTRUsKRHSerrtU2D//6s= 2024-04-16T18:10:30.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZELVzWtZTRUsKRHSerrtU2D//6s=', 'Content-Length': '0'} 2024-04-16T18:10:30.421 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b7bd4423f696e810-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.421 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:30.421 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.421 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.421 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.421 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.421 DEBUG:boto:Method: GET 2024-04-16T18:10:30.422 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.422 DEBUG:boto:Data: 2024-04-16T18:10:30.422 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.422 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.422 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.422 DEBUG:boto:Params: {} 2024-04-16T18:10:30.422 DEBUG:boto:Token: None 2024-04-16T18:10:30.422 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.422 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28= 2024-04-16T18:10:30.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28=', 'Content-Length': '0'} 2024-04-16T18:10:30.425 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000009b8e92c6ec701e-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.466 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.467 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.467 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.467 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.467 DEBUG:boto:Method: GET 2024-04-16T18:10:30.467 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.467 DEBUG:boto:Data: 2024-04-16T18:10:30.467 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.467 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.467 DEBUG:boto:Params: {} 2024-04-16T18:10:30.467 DEBUG:boto:Token: None 2024-04-16T18:10:30.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:30.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28= 2024-04-16T18:10:30.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lO+DL8j8QM9lb8rMKx0xB3Vpk28=', 'Content-Length': '0'} 2024-04-16T18:10:30.471 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c70c6dcfe5cc8ac9-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.472 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.472 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.472 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.472 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.472 DEBUG:boto:Method: GET 2024-04-16T18:10:30.472 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:30.472 DEBUG:boto:Data: 2024-04-16T18:10:30.472 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.472 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.472 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.472 DEBUG:boto:Params: {} 2024-04-16T18:10:30.473 DEBUG:boto:Token: None 2024-04-16T18:10:30.473 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.473 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.477 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe355702c6f586c1-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.477 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.478 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.478 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.478 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.478 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.478 DEBUG:boto:Method: GET 2024-04-16T18:10:30.478 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:30.478 DEBUG:boto:Data: 2024-04-16T18:10:30.479 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.479 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.479 DEBUG:boto:Params: {} 2024-04-16T18:10:30.479 DEBUG:boto:Token: None 2024-04-16T18:10:30.479 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.479 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.483 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b419a5a12f35bae-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.483 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.483 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.484 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.484 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.484 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:30.484 DEBUG:boto:Data: 2024-04-16T18:10:30.484 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.484 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.484 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.484 DEBUG:boto:Params: {} 2024-04-16T18:10:30.484 DEBUG:boto:Token: None 2024-04-16T18:10:30.484 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4 2024-04-16T18:10:30.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SSuV7MGkvZvSS0+RHmhEpQYoDcE= 2024-04-16T18:10:30.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SSuV7MGkvZvSS0+RHmhEpQYoDcE=', 'Content-Length': '0'} 2024-04-16T18:10:30.488 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e564782a388bf792-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.488 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.488 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.488 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.488 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:30.488 DEBUG:boto:Data: 2024-04-16T18:10:30.488 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.488 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.489 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.489 DEBUG:boto:Params: {} 2024-04-16T18:10:30.489 DEBUG:boto:Token: None 2024-04-16T18:10:30.489 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4 2024-04-16T18:10:30.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SSuV7MGkvZvSS0+RHmhEpQYoDcE= 2024-04-16T18:10:30.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SSuV7MGkvZvSS0+RHmhEpQYoDcE=', 'Content-Length': '0'} 2024-04-16T18:10:30.492 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b277c826620e52e1-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.492 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:30.492 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.493 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.493 DEBUG:boto:Method: GET 2024-04-16T18:10:30.493 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:30.493 DEBUG:boto:Data: 2024-04-16T18:10:30.493 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.493 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.493 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.493 DEBUG:boto:Params: {} 2024-04-16T18:10:30.493 DEBUG:boto:Token: None 2024-04-16T18:10:30.493 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4 2024-04-16T18:10:30.494 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zTJl1qDCdwJmTcOhrEaUqTLTaEA= 2024-04-16T18:10:30.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zTJl1qDCdwJmTcOhrEaUqTLTaEA=', 'Content-Length': '0'} 2024-04-16T18:10:30.497 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e4cc29759c3ed949-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.538 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:30.538 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:30.539 DEBUG:boto:Method: GET 2024-04-16T18:10:30.539 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:30.539 DEBUG:boto:Data: 2024-04-16T18:10:30.539 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.539 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.539 DEBUG:boto:Params: {} 2024-04-16T18:10:30.539 DEBUG:boto:Token: None 2024-04-16T18:10:30.539 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/obj4 2024-04-16T18:10:30.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zTJl1qDCdwJmTcOhrEaUqTLTaEA= 2024-04-16T18:10:30.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zTJl1qDCdwJmTcOhrEaUqTLTaEA=', 'Content-Length': '0'} 2024-04-16T18:10:30.543 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f55ee07d7a50ea99-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.543 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.543 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.543 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.543 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.543 DEBUG:boto:Method: GET 2024-04-16T18:10:30.544 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:30.544 DEBUG:boto:Data: 2024-04-16T18:10:30.544 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.544 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.544 DEBUG:boto:Params: {} 2024-04-16T18:10:30.544 DEBUG:boto:Token: None 2024-04-16T18:10:30.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cd50be5baae3cd6-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.550 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.550 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:30.550 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:30.550 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.550 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:30.550 DEBUG:boto:Method: GET 2024-04-16T18:10:30.551 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:30.551 DEBUG:boto:Data: 2024-04-16T18:10:30.551 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.551 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.551 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.551 DEBUG:boto:Params: {} 2024-04-16T18:10:30.551 DEBUG:boto:Token: None 2024-04-16T18:10:30.551 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-40/ 2024-04-16T18:10:30.551 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8= 2024-04-16T18:10:30.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vrdD1sb3pYYDmIo7tv53JNtmpc8=', 'Content-Length': '0'} 2024-04-16T18:10:30.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011332e87a99b1cf5-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.598 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.599 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-40 zones={test-zone2, test-zone1} 2024-04-16T18:10:30.599 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-41 2024-04-16T18:10:30.599 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:30.599 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:30.599 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-41 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:30.651 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:30.652 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:30.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:30.739+0000 7fe674d14980 1 bucket sync caught up with source: 2024-04-16T18:10:30.736 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.12.6, , 00000000001.27.6, , 00000000001.38.6, , , 00000000001.29.6, , , ] 2024-04-16T18:10:30.736 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.12.6, , 00000000001.27.6, , 00000000001.38.6, , , 00000000001.29.6, , , ] 2024-04-16T18:10:30.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:30.739+0000 7fe674d14980 0 bucket checkpoint complete 2024-04-16T18:10:30.766 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-41 2024-04-16T18:10:30.767 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:30.767 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:30.767 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-41 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:30.818 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:30.819 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:30.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:30.900+0000 7fd222726980 1 bucket sync caught up with source: 2024-04-16T18:10:30.900 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.570.6, , 00000000001.384.6, , 00000000001.102.6, , , 00000000001.604.6, , , ] 2024-04-16T18:10:30.900 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000001.570.6, , 00000000001.384.6, , 00000000001.102.6, , , 00000000001.604.6, , , ] 2024-04-16T18:10:30.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:30.900+0000 7fd222726980 0 bucket checkpoint complete 2024-04-16T18:10:30.931 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-41 zones={test-zone2, test-zone1} 2024-04-16T18:10:30.931 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:30.932 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:30.932 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.932 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:30.932 DEBUG:boto:Data: 2024-04-16T18:10:30.932 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.932 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.932 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.932 DEBUG:boto:Params: {} 2024-04-16T18:10:30.932 DEBUG:boto:Token: None 2024-04-16T18:10:30.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/ 2024-04-16T18:10:30.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6aOhGYCYXeyPQgrNKHy5vYwpfoU= 2024-04-16T18:10:30.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6aOhGYCYXeyPQgrNKHy5vYwpfoU=', 'Content-Length': '0'} 2024-04-16T18:10:30.937 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002220744d78e5ed34-00661ebf16-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.937 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:30.938 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:30.938 DEBUG:boto:Method: HEAD 2024-04-16T18:10:30.938 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:30.938 DEBUG:boto:Data: 2024-04-16T18:10:30.938 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.938 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.938 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.938 DEBUG:boto:Params: {} 2024-04-16T18:10:30.938 DEBUG:boto:Token: None 2024-04-16T18:10:30.938 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/ 2024-04-16T18:10:30.939 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6aOhGYCYXeyPQgrNKHy5vYwpfoU= 2024-04-16T18:10:30.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6aOhGYCYXeyPQgrNKHy5vYwpfoU=', 'Content-Length': '0'} 2024-04-16T18:10:30.941 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000017a7ceb1b54a018-00661ebf16-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.941 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:10:30.942 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:30.942 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:30.942 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.942 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.942 DEBUG:boto:Method: GET 2024-04-16T18:10:30.942 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.942 DEBUG:boto:Data: 2024-04-16T18:10:30.942 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.942 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.942 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.942 DEBUG:boto:Params: {} 2024-04-16T18:10:30.943 DEBUG:boto:Token: None 2024-04-16T18:10:30.943 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/?versions 2024-04-16T18:10:30.943 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ= 2024-04-16T18:10:30.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ=', 'Content-Length': '0'} 2024-04-16T18:10:30.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068d5059c4f7426dd-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.947 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.947 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:30.947 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:30.948 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:30.948 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:30.948 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:10:30.948 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:30.948 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:30.948 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.948 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.948 DEBUG:boto:Method: GET 2024-04-16T18:10:30.948 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.948 DEBUG:boto:Data: 2024-04-16T18:10:30.948 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.948 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.949 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.949 DEBUG:boto:Params: {} 2024-04-16T18:10:30.949 DEBUG:boto:Token: None 2024-04-16T18:10:30.949 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/?versions 2024-04-16T18:10:30.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ= 2024-04-16T18:10:30.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ=', 'Content-Length': '0'} 2024-04-16T18:10:30.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001361d3492965c3a1-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.952 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.952 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:30.952 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:30.952 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:30.952 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:30.953 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:30.953 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:30.953 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.953 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.953 DEBUG:boto:Method: GET 2024-04-16T18:10:30.953 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.953 DEBUG:boto:Data: 2024-04-16T18:10:30.953 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.953 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.953 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.954 DEBUG:boto:Params: {} 2024-04-16T18:10:30.954 DEBUG:boto:Token: None 2024-04-16T18:10:30.954 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/?versions 2024-04-16T18:10:30.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ= 2024-04-16T18:10:30.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ=', 'Content-Length': '0'} 2024-04-16T18:10:30.958 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f800e66c262ff163-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.958 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.959 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:30.959 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:30.959 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.959 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.959 DEBUG:boto:Method: GET 2024-04-16T18:10:30.959 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:30.959 DEBUG:boto:Data: 2024-04-16T18:10:30.959 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:30.959 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.960 DEBUG:boto:Params: {} 2024-04-16T18:10:30.960 DEBUG:boto:Token: None 2024-04-16T18:10:30.960 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/?versions 2024-04-16T18:10:30.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ= 2024-04-16T18:10:30.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q5IftQG6cJGYfnRyabqOfgfUwrQ=', 'Content-Length': '0'} 2024-04-16T18:10:30.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000382d59ec7220c0c8-00661ebf16-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:30.963 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:30.963 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:30.963 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:30.963 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:30.963 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:30.963 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:30.964 DEBUG:boto:Method: GET 2024-04-16T18:10:30.964 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:30.964 DEBUG:boto:Data: 2024-04-16T18:10:30.964 DEBUG:boto:Headers: {} 2024-04-16T18:10:30.964 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:30.964 DEBUG:boto:Port: 8000 2024-04-16T18:10:30.964 DEBUG:boto:Params: {} 2024-04-16T18:10:30.964 DEBUG:boto:Token: None 2024-04-16T18:10:30.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:30 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:30.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ErVJgg1yS9s1Ir/VCkr2Nwcb+5E= 2024-04-16T18:10:30.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:30 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ErVJgg1yS9s1Ir/VCkr2Nwcb+5E=', 'Content-Length': '0'} 2024-04-16T18:10:30.968 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e3176a7520425fe4-00661ebf16-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:30 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.010 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.010 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.011 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.011 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.011 DEBUG:boto:Method: GET 2024-04-16T18:10:31.011 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.011 DEBUG:boto:Data: 2024-04-16T18:10:31.011 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.011 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.011 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.011 DEBUG:boto:Params: {} 2024-04-16T18:10:31.011 DEBUG:boto:Token: None 2024-04-16T18:10:31.011 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hNDjS99LXfb+Utsl0waUzF3d4go= 2024-04-16T18:10:31.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hNDjS99LXfb+Utsl0waUzF3d4go=', 'Content-Length': '0'} 2024-04-16T18:10:31.014 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006fde55cc9c2e2d9a-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.015 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.015 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.015 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.015 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.015 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.015 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.015 DEBUG:boto:Data: 2024-04-16T18:10:31.015 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.015 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.015 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.015 DEBUG:boto:Params: {} 2024-04-16T18:10:31.016 DEBUG:boto:Token: None 2024-04-16T18:10:31.016 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EplFbBc0PcCJgArBIxv6DatovHQ= 2024-04-16T18:10:31.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EplFbBc0PcCJgArBIxv6DatovHQ=', 'Content-Length': '0'} 2024-04-16T18:10:31.019 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a1b0b9db2c337d58-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.019 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.019 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.019 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.019 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.019 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.019 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.019 DEBUG:boto:Data: 2024-04-16T18:10:31.019 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.020 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.020 DEBUG:boto:Params: {} 2024-04-16T18:10:31.020 DEBUG:boto:Token: None 2024-04-16T18:10:31.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EplFbBc0PcCJgArBIxv6DatovHQ= 2024-04-16T18:10:31.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EplFbBc0PcCJgArBIxv6DatovHQ=', 'Content-Length': '0'} 2024-04-16T18:10:31.023 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007f67e8df80357a08-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:31.023 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.023 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.024 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.024 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.024 DEBUG:boto:Method: GET 2024-04-16T18:10:31.024 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.024 DEBUG:boto:Data: 2024-04-16T18:10:31.024 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.024 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.024 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.024 DEBUG:boto:Params: {} 2024-04-16T18:10:31.024 DEBUG:boto:Token: None 2024-04-16T18:10:31.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hNDjS99LXfb+Utsl0waUzF3d4go= 2024-04-16T18:10:31.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hNDjS99LXfb+Utsl0waUzF3d4go=', 'Content-Length': '0'} 2024-04-16T18:10:31.027 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000744176451a733e6f-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.027 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.028 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.028 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.028 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.028 DEBUG:boto:Method: GET 2024-04-16T18:10:31.028 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.028 DEBUG:boto:Data: 2024-04-16T18:10:31.028 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.028 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.028 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.028 DEBUG:boto:Params: {} 2024-04-16T18:10:31.028 DEBUG:boto:Token: None 2024-04-16T18:10:31.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:31.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hNDjS99LXfb+Utsl0waUzF3d4go= 2024-04-16T18:10:31.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hNDjS99LXfb+Utsl0waUzF3d4go=', 'Content-Length': '0'} 2024-04-16T18:10:31.032 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d0181e1efe65f7e6-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.074 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.074 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.075 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.075 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.075 DEBUG:boto:Method: GET 2024-04-16T18:10:31.075 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:31.075 DEBUG:boto:Data: 2024-04-16T18:10:31.075 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.075 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.075 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.075 DEBUG:boto:Params: {} 2024-04-16T18:10:31.075 DEBUG:boto:Token: None 2024-04-16T18:10:31.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001391797399338b9b-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.080 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.080 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.080 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.080 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.081 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.081 DEBUG:boto:Method: GET 2024-04-16T18:10:31.081 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:31.081 DEBUG:boto:Data: 2024-04-16T18:10:31.081 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.081 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.081 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.081 DEBUG:boto:Params: {} 2024-04-16T18:10:31.081 DEBUG:boto:Token: None 2024-04-16T18:10:31.082 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.082 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021adad491be117f1-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.085 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.085 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.086 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.086 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.086 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:31.086 DEBUG:boto:Data: 2024-04-16T18:10:31.086 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.086 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.086 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.086 DEBUG:boto:Params: {} 2024-04-16T18:10:31.086 DEBUG:boto:Token: None 2024-04-16T18:10:31.087 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1 2024-04-16T18:10:31.087 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dcaea/G3HTwLJ/nuV2nMQxmRdQk= 2024-04-16T18:10:31.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dcaea/G3HTwLJ/nuV2nMQxmRdQk=', 'Content-Length': '0'} 2024-04-16T18:10:31.089 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c2ba03ab79a92a4d-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.090 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.090 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.090 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.090 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:31.090 DEBUG:boto:Data: 2024-04-16T18:10:31.090 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.090 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.090 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.090 DEBUG:boto:Params: {} 2024-04-16T18:10:31.091 DEBUG:boto:Token: None 2024-04-16T18:10:31.091 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1 2024-04-16T18:10:31.091 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dcaea/G3HTwLJ/nuV2nMQxmRdQk= 2024-04-16T18:10:31.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dcaea/G3HTwLJ/nuV2nMQxmRdQk=', 'Content-Length': '0'} 2024-04-16T18:10:31.094 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000053cbd1f6855d314d-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.094 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:31.094 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.095 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.095 DEBUG:boto:Method: GET 2024-04-16T18:10:31.095 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:31.095 DEBUG:boto:Data: 2024-04-16T18:10:31.095 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.095 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.095 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.095 DEBUG:boto:Params: {} 2024-04-16T18:10:31.095 DEBUG:boto:Token: None 2024-04-16T18:10:31.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1 2024-04-16T18:10:31.095 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rOzofqMZuoD+duVlAKpOPvS7uSM= 2024-04-16T18:10:31.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rOzofqMZuoD+duVlAKpOPvS7uSM=', 'Content-Length': '0'} 2024-04-16T18:10:31.099 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009efcff666bd5b5c0-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.142 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:31.143 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:31.143 DEBUG:boto:Method: GET 2024-04-16T18:10:31.143 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:31.143 DEBUG:boto:Data: 2024-04-16T18:10:31.143 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.143 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.143 DEBUG:boto:Params: {} 2024-04-16T18:10:31.143 DEBUG:boto:Token: None 2024-04-16T18:10:31.143 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj1 2024-04-16T18:10:31.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rOzofqMZuoD+duVlAKpOPvS7uSM= 2024-04-16T18:10:31.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rOzofqMZuoD+duVlAKpOPvS7uSM=', 'Content-Length': '0'} 2024-04-16T18:10:31.146 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f623061892421b30-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.190 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.191 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.191 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.191 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.191 DEBUG:boto:Method: GET 2024-04-16T18:10:31.191 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:31.191 DEBUG:boto:Data: 2024-04-16T18:10:31.191 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.191 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.191 DEBUG:boto:Params: {} 2024-04-16T18:10:31.191 DEBUG:boto:Token: None 2024-04-16T18:10:31.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d97da326a8d18dda-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.196 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.196 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.196 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.197 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.197 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:31.197 DEBUG:boto:Method: GET 2024-04-16T18:10:31.197 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:31.197 DEBUG:boto:Data: 2024-04-16T18:10:31.197 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.197 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.197 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.197 DEBUG:boto:Params: {} 2024-04-16T18:10:31.197 DEBUG:boto:Token: None 2024-04-16T18:10:31.198 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.198 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.201 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a68d4bc872bb9ca-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.202 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.202 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:31.202 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.202 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.202 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.202 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.203 DEBUG:boto:Method: GET 2024-04-16T18:10:31.203 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.203 DEBUG:boto:Data: 2024-04-16T18:10:31.203 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.203 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.203 DEBUG:boto:Params: {} 2024-04-16T18:10:31.203 DEBUG:boto:Token: None 2024-04-16T18:10:31.203 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.203 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo= 2024-04-16T18:10:31.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo=', 'Content-Length': '0'} 2024-04-16T18:10:31.206 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000003382a62abececec-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.207 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.207 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.207 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.207 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.207 DEBUG:boto:Method: GET 2024-04-16T18:10:31.207 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.207 DEBUG:boto:Data: 2024-04-16T18:10:31.207 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.208 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.208 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.208 DEBUG:boto:Params: {} 2024-04-16T18:10:31.208 DEBUG:boto:Token: None 2024-04-16T18:10:31.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo= 2024-04-16T18:10:31.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo=', 'Content-Length': '0'} 2024-04-16T18:10:31.211 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f5be749a7ebc925d-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.254 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.255 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.255 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.255 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.255 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.255 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.255 DEBUG:boto:Data: 2024-04-16T18:10:31.255 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.255 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.255 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.255 DEBUG:boto:Params: {} 2024-04-16T18:10:31.256 DEBUG:boto:Token: None 2024-04-16T18:10:31.256 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IlQDBwQnj4WVyVDXie2gtxjSHyw= 2024-04-16T18:10:31.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IlQDBwQnj4WVyVDXie2gtxjSHyw=', 'Content-Length': '0'} 2024-04-16T18:10:31.259 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c25696177bd8c11f-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.259 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.259 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.259 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.259 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.259 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.259 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.260 DEBUG:boto:Data: 2024-04-16T18:10:31.260 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.260 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.260 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.260 DEBUG:boto:Params: {} 2024-04-16T18:10:31.260 DEBUG:boto:Token: None 2024-04-16T18:10:31.260 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IlQDBwQnj4WVyVDXie2gtxjSHyw= 2024-04-16T18:10:31.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IlQDBwQnj4WVyVDXie2gtxjSHyw=', 'Content-Length': '0'} 2024-04-16T18:10:31.263 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b3c3e6dab5c591e3-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.263 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:31.264 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.264 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.264 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.264 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.264 DEBUG:boto:Method: GET 2024-04-16T18:10:31.264 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.264 DEBUG:boto:Data: 2024-04-16T18:10:31.264 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.264 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.265 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.265 DEBUG:boto:Params: {} 2024-04-16T18:10:31.265 DEBUG:boto:Token: None 2024-04-16T18:10:31.265 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.265 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo= 2024-04-16T18:10:31.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo=', 'Content-Length': '0'} 2024-04-16T18:10:31.268 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005ae9aba25a612852-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.268 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.269 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.269 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.269 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.269 DEBUG:boto:Method: GET 2024-04-16T18:10:31.269 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.269 DEBUG:boto:Data: 2024-04-16T18:10:31.269 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.269 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.269 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.269 DEBUG:boto:Params: {} 2024-04-16T18:10:31.270 DEBUG:boto:Token: None 2024-04-16T18:10:31.270 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:31.270 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo= 2024-04-16T18:10:31.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t+0QQoYoGrTbePFgIrNxE5WSMEo=', 'Content-Length': '0'} 2024-04-16T18:10:31.273 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bd24bcd8f94470ed-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.314 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.314 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.315 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.315 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.315 DEBUG:boto:Method: GET 2024-04-16T18:10:31.315 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:31.315 DEBUG:boto:Data: 2024-04-16T18:10:31.315 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.315 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.315 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.315 DEBUG:boto:Params: {} 2024-04-16T18:10:31.315 DEBUG:boto:Token: None 2024-04-16T18:10:31.316 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.320 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034f71eb9442887bf-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.320 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.320 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.321 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.321 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.321 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.321 DEBUG:boto:Method: GET 2024-04-16T18:10:31.321 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:31.321 DEBUG:boto:Data: 2024-04-16T18:10:31.321 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.321 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.321 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.321 DEBUG:boto:Params: {} 2024-04-16T18:10:31.322 DEBUG:boto:Token: None 2024-04-16T18:10:31.322 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.322 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.326 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082ce5ace681ffb42-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.326 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.326 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.326 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.327 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.327 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:31.327 DEBUG:boto:Data: 2024-04-16T18:10:31.327 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.327 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.327 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.327 DEBUG:boto:Params: {} 2024-04-16T18:10:31.327 DEBUG:boto:Token: None 2024-04-16T18:10:31.327 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2 2024-04-16T18:10:31.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DIovHuj+I9y8WPbD8l0jrIfuzLo= 2024-04-16T18:10:31.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DIovHuj+I9y8WPbD8l0jrIfuzLo=', 'Content-Length': '0'} 2024-04-16T18:10:31.330 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007b3a1ed1dc942558-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.331 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.331 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.331 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.331 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:31.331 DEBUG:boto:Data: 2024-04-16T18:10:31.331 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.331 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.331 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.332 DEBUG:boto:Params: {} 2024-04-16T18:10:31.332 DEBUG:boto:Token: None 2024-04-16T18:10:31.332 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2 2024-04-16T18:10:31.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DIovHuj+I9y8WPbD8l0jrIfuzLo= 2024-04-16T18:10:31.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DIovHuj+I9y8WPbD8l0jrIfuzLo=', 'Content-Length': '0'} 2024-04-16T18:10:31.335 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ca856a88d018052d-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.335 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:31.335 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.335 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.336 DEBUG:boto:Method: GET 2024-04-16T18:10:31.336 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:31.336 DEBUG:boto:Data: 2024-04-16T18:10:31.336 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.336 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.336 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.336 DEBUG:boto:Params: {} 2024-04-16T18:10:31.336 DEBUG:boto:Token: None 2024-04-16T18:10:31.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2 2024-04-16T18:10:31.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mJC2TCYlI1oeZZlU4HbCbeUOV0A= 2024-04-16T18:10:31.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mJC2TCYlI1oeZZlU4HbCbeUOV0A=', 'Content-Length': '0'} 2024-04-16T18:10:31.339 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000de987828dd932baa-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.386 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:31.387 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:31.387 DEBUG:boto:Method: GET 2024-04-16T18:10:31.387 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:31.387 DEBUG:boto:Data: 2024-04-16T18:10:31.387 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.387 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.387 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.387 DEBUG:boto:Params: {} 2024-04-16T18:10:31.387 DEBUG:boto:Token: None 2024-04-16T18:10:31.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj2 2024-04-16T18:10:31.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mJC2TCYlI1oeZZlU4HbCbeUOV0A= 2024-04-16T18:10:31.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mJC2TCYlI1oeZZlU4HbCbeUOV0A=', 'Content-Length': '0'} 2024-04-16T18:10:31.391 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038b2263d0d110215-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.434 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.434 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.435 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.435 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.435 DEBUG:boto:Method: GET 2024-04-16T18:10:31.435 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:31.435 DEBUG:boto:Data: 2024-04-16T18:10:31.435 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.435 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.435 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.435 DEBUG:boto:Params: {} 2024-04-16T18:10:31.435 DEBUG:boto:Token: None 2024-04-16T18:10:31.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e38149d0ec28c343-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.440 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.440 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.440 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.441 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.441 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:31.441 DEBUG:boto:Method: GET 2024-04-16T18:10:31.441 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:31.441 DEBUG:boto:Data: 2024-04-16T18:10:31.441 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.441 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.441 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.441 DEBUG:boto:Params: {} 2024-04-16T18:10:31.441 DEBUG:boto:Token: None 2024-04-16T18:10:31.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aca415b7de8e2d76-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.446 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.446 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:31.446 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.446 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.446 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.447 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.447 DEBUG:boto:Method: GET 2024-04-16T18:10:31.447 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.447 DEBUG:boto:Data: 2024-04-16T18:10:31.447 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.447 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.447 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.447 DEBUG:boto:Params: {} 2024-04-16T18:10:31.447 DEBUG:boto:Token: None 2024-04-16T18:10:31.447 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM= 2024-04-16T18:10:31.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM=', 'Content-Length': '0'} 2024-04-16T18:10:31.451 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003840a82dd8c0e44b-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.451 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.451 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.451 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.451 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.451 DEBUG:boto:Method: GET 2024-04-16T18:10:31.452 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.452 DEBUG:boto:Data: 2024-04-16T18:10:31.452 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.452 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.452 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.452 DEBUG:boto:Params: {} 2024-04-16T18:10:31.452 DEBUG:boto:Token: None 2024-04-16T18:10:31.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM= 2024-04-16T18:10:31.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM=', 'Content-Length': '0'} 2024-04-16T18:10:31.455 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000306264d2d609d2eb-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.498 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.499 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.499 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.499 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.499 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.499 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.499 DEBUG:boto:Data: 2024-04-16T18:10:31.499 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.499 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.499 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.500 DEBUG:boto:Params: {} 2024-04-16T18:10:31.500 DEBUG:boto:Token: None 2024-04-16T18:10:31.500 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HwVk6lcZ3ZOKgI8mefw3K7fYO40= 2024-04-16T18:10:31.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HwVk6lcZ3ZOKgI8mefw3K7fYO40=', 'Content-Length': '0'} 2024-04-16T18:10:31.503 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b8ea375bce0adf75-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.503 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.503 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.503 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.504 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.504 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.504 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.504 DEBUG:boto:Data: 2024-04-16T18:10:31.504 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.504 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.504 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.504 DEBUG:boto:Params: {} 2024-04-16T18:10:31.504 DEBUG:boto:Token: None 2024-04-16T18:10:31.505 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HwVk6lcZ3ZOKgI8mefw3K7fYO40= 2024-04-16T18:10:31.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HwVk6lcZ3ZOKgI8mefw3K7fYO40=', 'Content-Length': '0'} 2024-04-16T18:10:31.508 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000529006ef9133407-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:31.508 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.508 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.508 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.508 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.509 DEBUG:boto:Method: GET 2024-04-16T18:10:31.509 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.509 DEBUG:boto:Data: 2024-04-16T18:10:31.509 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.509 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.509 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.509 DEBUG:boto:Params: {} 2024-04-16T18:10:31.509 DEBUG:boto:Token: None 2024-04-16T18:10:31.509 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.510 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM= 2024-04-16T18:10:31.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM=', 'Content-Length': '0'} 2024-04-16T18:10:31.513 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000190122024ebb7785-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.513 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.513 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.513 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.513 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.513 DEBUG:boto:Method: GET 2024-04-16T18:10:31.513 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.514 DEBUG:boto:Data: 2024-04-16T18:10:31.514 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.514 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.514 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.514 DEBUG:boto:Params: {} 2024-04-16T18:10:31.514 DEBUG:boto:Token: None 2024-04-16T18:10:31.514 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:31.514 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM= 2024-04-16T18:10:31.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnwR0WjmHtz0qey08wF3jZJp2OM=', 'Content-Length': '0'} 2024-04-16T18:10:31.517 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b8227db5e04f42ac-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.558 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.559 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.559 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.559 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.559 DEBUG:boto:Method: GET 2024-04-16T18:10:31.559 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:31.559 DEBUG:boto:Data: 2024-04-16T18:10:31.559 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.559 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.560 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.560 DEBUG:boto:Params: {} 2024-04-16T18:10:31.560 DEBUG:boto:Token: None 2024-04-16T18:10:31.560 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008095b271c597f8be-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.564 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.564 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.565 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.565 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.565 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.565 DEBUG:boto:Method: GET 2024-04-16T18:10:31.565 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:31.565 DEBUG:boto:Data: 2024-04-16T18:10:31.565 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.565 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.565 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.565 DEBUG:boto:Params: {} 2024-04-16T18:10:31.566 DEBUG:boto:Token: None 2024-04-16T18:10:31.566 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.566 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000619f7d85d5574577-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.570 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.571 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.571 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.571 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.571 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:31.571 DEBUG:boto:Data: 2024-04-16T18:10:31.571 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.572 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.572 DEBUG:boto:Params: {} 2024-04-16T18:10:31.572 DEBUG:boto:Token: None 2024-04-16T18:10:31.572 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3 2024-04-16T18:10:31.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VgwKqfe6BWK+5XV89FFKoyndyx0= 2024-04-16T18:10:31.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VgwKqfe6BWK+5XV89FFKoyndyx0=', 'Content-Length': '0'} 2024-04-16T18:10:31.575 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008f190df12f62b8fc-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.575 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.575 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.576 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.576 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:31.576 DEBUG:boto:Data: 2024-04-16T18:10:31.576 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.576 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.576 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.576 DEBUG:boto:Params: {} 2024-04-16T18:10:31.576 DEBUG:boto:Token: None 2024-04-16T18:10:31.576 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3 2024-04-16T18:10:31.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VgwKqfe6BWK+5XV89FFKoyndyx0= 2024-04-16T18:10:31.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VgwKqfe6BWK+5XV89FFKoyndyx0=', 'Content-Length': '0'} 2024-04-16T18:10:31.579 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a1391b91539fdd79-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.580 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:31.580 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.580 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.580 DEBUG:boto:Method: GET 2024-04-16T18:10:31.580 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:31.581 DEBUG:boto:Data: 2024-04-16T18:10:31.581 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.581 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.581 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.581 DEBUG:boto:Params: {} 2024-04-16T18:10:31.581 DEBUG:boto:Token: None 2024-04-16T18:10:31.581 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3 2024-04-16T18:10:31.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jEkVP7k06/3f2vPJSRqTvyBA5pU= 2024-04-16T18:10:31.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jEkVP7k06/3f2vPJSRqTvyBA5pU=', 'Content-Length': '0'} 2024-04-16T18:10:31.584 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e8a8ad7f118de47e-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.626 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:31.627 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:31.627 DEBUG:boto:Method: GET 2024-04-16T18:10:31.627 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:31.627 DEBUG:boto:Data: 2024-04-16T18:10:31.627 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.627 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.627 DEBUG:boto:Params: {} 2024-04-16T18:10:31.627 DEBUG:boto:Token: None 2024-04-16T18:10:31.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj3 2024-04-16T18:10:31.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jEkVP7k06/3f2vPJSRqTvyBA5pU= 2024-04-16T18:10:31.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jEkVP7k06/3f2vPJSRqTvyBA5pU=', 'Content-Length': '0'} 2024-04-16T18:10:31.631 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000c1a17e359c4a053-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.678 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.679 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.679 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.679 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.679 DEBUG:boto:Method: GET 2024-04-16T18:10:31.679 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:31.679 DEBUG:boto:Data: 2024-04-16T18:10:31.679 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.679 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.679 DEBUG:boto:Params: {} 2024-04-16T18:10:31.680 DEBUG:boto:Token: None 2024-04-16T18:10:31.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.684 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f9e96076a9de8cf-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.684 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.685 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.685 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.685 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.685 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:31.685 DEBUG:boto:Method: GET 2024-04-16T18:10:31.685 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:31.685 DEBUG:boto:Data: 2024-04-16T18:10:31.685 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.685 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.685 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.686 DEBUG:boto:Params: {} 2024-04-16T18:10:31.686 DEBUG:boto:Token: None 2024-04-16T18:10:31.686 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.686 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.690 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fdd75a7093b2871-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.690 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.690 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:31.690 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.691 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.691 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.691 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.691 DEBUG:boto:Method: GET 2024-04-16T18:10:31.691 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.691 DEBUG:boto:Data: 2024-04-16T18:10:31.691 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.691 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.691 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.691 DEBUG:boto:Params: {} 2024-04-16T18:10:31.691 DEBUG:boto:Token: None 2024-04-16T18:10:31.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8= 2024-04-16T18:10:31.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8=', 'Content-Length': '0'} 2024-04-16T18:10:31.695 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003f26a72308b43ed6-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.695 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.695 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.695 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.695 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.695 DEBUG:boto:Method: GET 2024-04-16T18:10:31.696 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.696 DEBUG:boto:Data: 2024-04-16T18:10:31.696 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.696 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.696 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.696 DEBUG:boto:Params: {} 2024-04-16T18:10:31.696 DEBUG:boto:Token: None 2024-04-16T18:10:31.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8= 2024-04-16T18:10:31.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8=', 'Content-Length': '0'} 2024-04-16T18:10:31.699 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a544169a1d420d39-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.742 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.743 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.743 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.743 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.743 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.743 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.743 DEBUG:boto:Data: 2024-04-16T18:10:31.743 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.743 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.743 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.743 DEBUG:boto:Params: {} 2024-04-16T18:10:31.743 DEBUG:boto:Token: None 2024-04-16T18:10:31.744 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RmiOPTIFEpisKPnU1QMDoNx4IS8= 2024-04-16T18:10:31.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RmiOPTIFEpisKPnU1QMDoNx4IS8=', 'Content-Length': '0'} 2024-04-16T18:10:31.747 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d1803b032d46ff3f-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.747 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.747 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.747 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.747 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.747 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.747 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.747 DEBUG:boto:Data: 2024-04-16T18:10:31.747 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.748 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.748 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.748 DEBUG:boto:Params: {} 2024-04-16T18:10:31.748 DEBUG:boto:Token: None 2024-04-16T18:10:31.748 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RmiOPTIFEpisKPnU1QMDoNx4IS8= 2024-04-16T18:10:31.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RmiOPTIFEpisKPnU1QMDoNx4IS8=', 'Content-Length': '0'} 2024-04-16T18:10:31.751 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008eaa53c6e61d6a67-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.751 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:31.751 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.751 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.752 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.752 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.752 DEBUG:boto:Method: GET 2024-04-16T18:10:31.752 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.752 DEBUG:boto:Data: 2024-04-16T18:10:31.752 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.752 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.752 DEBUG:boto:Params: {} 2024-04-16T18:10:31.752 DEBUG:boto:Token: None 2024-04-16T18:10:31.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8= 2024-04-16T18:10:31.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8=', 'Content-Length': '0'} 2024-04-16T18:10:31.756 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000aef0880fa72a039c-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.756 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.756 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.756 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.756 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.756 DEBUG:boto:Method: GET 2024-04-16T18:10:31.756 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.757 DEBUG:boto:Data: 2024-04-16T18:10:31.757 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.757 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.757 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.757 DEBUG:boto:Params: {} 2024-04-16T18:10:31.757 DEBUG:boto:Token: None 2024-04-16T18:10:31.757 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:31.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8= 2024-04-16T18:10:31.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/9CcTveI6/3BPHTpMV7K0W3YQ/8=', 'Content-Length': '0'} 2024-04-16T18:10:31.760 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000084c690d87d86fbd1-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.802 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.802 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.803 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.803 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.803 DEBUG:boto:Method: GET 2024-04-16T18:10:31.803 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:31.803 DEBUG:boto:Data: 2024-04-16T18:10:31.803 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.803 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.803 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.803 DEBUG:boto:Params: {} 2024-04-16T18:10:31.803 DEBUG:boto:Token: None 2024-04-16T18:10:31.804 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000330d540793dad854-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.807 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.808 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.808 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.808 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.808 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.808 DEBUG:boto:Method: GET 2024-04-16T18:10:31.808 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:31.808 DEBUG:boto:Data: 2024-04-16T18:10:31.808 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.808 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.809 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.809 DEBUG:boto:Params: {} 2024-04-16T18:10:31.809 DEBUG:boto:Token: None 2024-04-16T18:10:31.809 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000085ccca08789714be-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.813 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.813 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.813 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.813 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.813 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:31.813 DEBUG:boto:Data: 2024-04-16T18:10:31.813 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.813 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.813 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.814 DEBUG:boto:Params: {} 2024-04-16T18:10:31.814 DEBUG:boto:Token: None 2024-04-16T18:10:31.814 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4 2024-04-16T18:10:31.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hkYf6hZXdFrxx3iW7PgYU5chDgI= 2024-04-16T18:10:31.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hkYf6hZXdFrxx3iW7PgYU5chDgI=', 'Content-Length': '0'} 2024-04-16T18:10:31.817 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000db43f383a744618e-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.817 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.817 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.817 DEBUG:boto:Method: HEAD 2024-04-16T18:10:31.817 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:31.817 DEBUG:boto:Data: 2024-04-16T18:10:31.818 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.818 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.818 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.818 DEBUG:boto:Params: {} 2024-04-16T18:10:31.818 DEBUG:boto:Token: None 2024-04-16T18:10:31.818 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4 2024-04-16T18:10:31.818 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hkYf6hZXdFrxx3iW7PgYU5chDgI= 2024-04-16T18:10:31.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hkYf6hZXdFrxx3iW7PgYU5chDgI=', 'Content-Length': '0'} 2024-04-16T18:10:31.821 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002eb44510bf741b78-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.821 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:31.822 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.822 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.822 DEBUG:boto:Method: GET 2024-04-16T18:10:31.822 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:31.822 DEBUG:boto:Data: 2024-04-16T18:10:31.822 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.822 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.822 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.822 DEBUG:boto:Params: {} 2024-04-16T18:10:31.822 DEBUG:boto:Token: None 2024-04-16T18:10:31.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4 2024-04-16T18:10:31.823 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OfbYS6ydsRguQ4QkABMaww+3QFM= 2024-04-16T18:10:31.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OfbYS6ydsRguQ4QkABMaww+3QFM=', 'Content-Length': '0'} 2024-04-16T18:10:31.825 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a1838db3018c419a-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.866 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:31.886 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:31.886 DEBUG:boto:Method: GET 2024-04-16T18:10:31.886 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:31.886 DEBUG:boto:Data: 2024-04-16T18:10:31.886 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.887 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.887 DEBUG:boto:Params: {} 2024-04-16T18:10:31.887 DEBUG:boto:Token: None 2024-04-16T18:10:31.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/obj4 2024-04-16T18:10:31.887 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OfbYS6ydsRguQ4QkABMaww+3QFM= 2024-04-16T18:10:31.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OfbYS6ydsRguQ4QkABMaww+3QFM=', 'Content-Length': '0'} 2024-04-16T18:10:31.890 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000537b479db749e9e6-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.930 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.931 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.931 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.931 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.931 DEBUG:boto:Method: GET 2024-04-16T18:10:31.931 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:31.931 DEBUG:boto:Data: 2024-04-16T18:10:31.931 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.931 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:31.931 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.931 DEBUG:boto:Params: {} 2024-04-16T18:10:31.931 DEBUG:boto:Token: None 2024-04-16T18:10:31.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.938 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000336df4f12220d8b6-00661ebf17-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.938 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.938 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:31.939 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:31.939 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.939 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:31.939 DEBUG:boto:Method: GET 2024-04-16T18:10:31.939 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:31.939 DEBUG:boto:Data: 2024-04-16T18:10:31.939 DEBUG:boto:Headers: {} 2024-04-16T18:10:31.939 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:31.939 DEBUG:boto:Port: 8000 2024-04-16T18:10:31.939 DEBUG:boto:Params: {} 2024-04-16T18:10:31.939 DEBUG:boto:Token: None 2024-04-16T18:10:31.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:31 GMT /lybtug-41/ 2024-04-16T18:10:31.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo= 2024-04-16T18:10:31.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:31 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GYIiCK6rDPbtcj2QFitnerisDgo=', 'Content-Length': '0'} 2024-04-16T18:10:31.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d93a994b476561a2-00661ebf17-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:31 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:31.986 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:31.987 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-41 zones={test-zone2, test-zone1} 2024-04-16T18:10:31.987 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'lybtug-40', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:31.987 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'lybtug-40', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:31.987 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket lybtug-40 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:10:32.031 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:32.031 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:32.191 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'lybtug-41', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:32.191 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'lybtug-41', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:32.191 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket lybtug-41 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:10:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:32.430 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:32.430 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:32.430 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:32.581 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:10:32.614 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291032.397129_410.1","last_update":"2024-04-16T18:10:32.397129Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:10:32.614 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291032.397129_410.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:32.614 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:10:32.614 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:32.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:32.615 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:32.657 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:32.735+0000 7f301a71c980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:32.735+0000 7f301a71c980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:32.762 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291023.678388_399.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:23.678388Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:32Z"} 2024-04-16T18:10:32.763 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291023.678388_399.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:23.678388Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:32Z"} 2024-04-16T18:10:32.763 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291023.678388_399.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:10:23.678388Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:10:32.763 DEBUG:rgw_multi.tests:log_status={0: '1_1713291032.397129_410.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:32.763 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291023.678388_399.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:32.763 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291032.397129_410.1 target=1_1713291023.678388_399.1 2024-04-16T18:10:37.764 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:37.765 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:37.765 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:37.809 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:37.809 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:37.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:37.883+0000 7f01fd745980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:37.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:37.887+0000 7f01fd745980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:37.917 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291032.397129_410.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:32.397129Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:37Z"} 2024-04-16T18:10:37.917 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291032.397129_410.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:32.397129Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:37Z"} 2024-04-16T18:10:37.917 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291032.397129_410.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:10:32.397129Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:10:37.917 DEBUG:rgw_multi.tests:log_status={0: '1_1713291032.397129_410.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:37.917 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291032.397129_410.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:37.917 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:10:37.917 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:37.918 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:37.918 DEBUG:boto:Method: HEAD 2024-04-16T18:10:37.918 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:37.918 DEBUG:boto:Data: 2024-04-16T18:10:37.918 DEBUG:boto:Headers: {} 2024-04-16T18:10:37.918 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.918 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.918 DEBUG:boto:Params: {} 2024-04-16T18:10:37.918 DEBUG:boto:Token: None 2024-04-16T18:10:37.919 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:37 GMT /lybtug-40/ 2024-04-16T18:10:37.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ChofPyb1RKOEhvzRV5u4wiAY98s= 2024-04-16T18:10:37.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ChofPyb1RKOEhvzRV5u4wiAY98s=', 'Content-Length': '0'} 2024-04-16T18:10:37.922 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a76040b022ef7bbf-00661ebf1d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:37.923 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:37.923 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:37.923 DEBUG:boto:Method: PUT 2024-04-16T18:10:37.923 DEBUG:boto:Path: /lybtug-40/obj5 2024-04-16T18:10:37.923 DEBUG:boto:Data: 2024-04-16T18:10:37.923 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:37.923 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.923 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.923 DEBUG:boto:Params: {} 2024-04-16T18:10:37.923 DEBUG:boto:Token: None 2024-04-16T18:10:37.924 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:37 GMT /lybtug-40/obj5 2024-04-16T18:10:37.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WXTG1dqnhK+yeSzWBt7iXKIF300= 2024-04-16T18:10:37.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WXTG1dqnhK+yeSzWBt7iXKIF300='} 2024-04-16T18:10:37.933 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fa59a4773d1683f9-00661ebf1d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:37.933 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:37.934 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:37.934 DEBUG:boto:Method: HEAD 2024-04-16T18:10:37.934 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:37.934 DEBUG:boto:Data: 2024-04-16T18:10:37.934 DEBUG:boto:Headers: {} 2024-04-16T18:10:37.934 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.934 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.934 DEBUG:boto:Params: {} 2024-04-16T18:10:37.934 DEBUG:boto:Token: None 2024-04-16T18:10:37.934 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:37 GMT /lybtug-40/ 2024-04-16T18:10:37.934 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ChofPyb1RKOEhvzRV5u4wiAY98s= 2024-04-16T18:10:37.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ChofPyb1RKOEhvzRV5u4wiAY98s=', 'Content-Length': '0'} 2024-04-16T18:10:37.938 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000db1b442649c4d7dd-00661ebf1d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:37.938 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:37.938 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:37.938 DEBUG:boto:Method: PUT 2024-04-16T18:10:37.938 DEBUG:boto:Path: /lybtug-40/obj6 2024-04-16T18:10:37.938 DEBUG:boto:Data: 2024-04-16T18:10:37.938 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:37.939 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.939 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.939 DEBUG:boto:Params: {} 2024-04-16T18:10:37.939 DEBUG:boto:Token: None 2024-04-16T18:10:37.939 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:37 GMT /lybtug-40/obj6 2024-04-16T18:10:37.939 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dw4+ExK4yB8N0otyOth1+/7U9to= 2024-04-16T18:10:37.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dw4+ExK4yB8N0otyOth1+/7U9to='} 2024-04-16T18:10:37.948 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001712cd05ca59af0d-00661ebf1d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:37.948 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:37.948 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:37.948 DEBUG:boto:Method: HEAD 2024-04-16T18:10:37.948 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:37.948 DEBUG:boto:Data: 2024-04-16T18:10:37.948 DEBUG:boto:Headers: {} 2024-04-16T18:10:37.948 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.948 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.948 DEBUG:boto:Params: {} 2024-04-16T18:10:37.949 DEBUG:boto:Token: None 2024-04-16T18:10:37.949 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:37 GMT /lybtug-40/ 2024-04-16T18:10:37.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ChofPyb1RKOEhvzRV5u4wiAY98s= 2024-04-16T18:10:37.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ChofPyb1RKOEhvzRV5u4wiAY98s=', 'Content-Length': '0'} 2024-04-16T18:10:37.952 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000579994cd698b8c5a-00661ebf1d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:37.952 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:37.952 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:37.952 DEBUG:boto:Method: PUT 2024-04-16T18:10:37.952 DEBUG:boto:Path: /lybtug-40/obj7 2024-04-16T18:10:37.953 DEBUG:boto:Data: 2024-04-16T18:10:37.953 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:37.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.953 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.953 DEBUG:boto:Params: {} 2024-04-16T18:10:37.953 DEBUG:boto:Token: None 2024-04-16T18:10:37.953 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:37 GMT /lybtug-40/obj7 2024-04-16T18:10:37.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NeSgqu1gZxQOUb2XBwseCeP1n0I= 2024-04-16T18:10:37.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NeSgqu1gZxQOUb2XBwseCeP1n0I='} 2024-04-16T18:10:37.998 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a08a6d1d3ddc133e-00661ebf1d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:37.999 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:37.999 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:37.999 DEBUG:boto:Method: HEAD 2024-04-16T18:10:37.999 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:37.999 DEBUG:boto:Data: 2024-04-16T18:10:37.999 DEBUG:boto:Headers: {} 2024-04-16T18:10:37.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:37.999 DEBUG:boto:Port: 8000 2024-04-16T18:10:37.999 DEBUG:boto:Params: {} 2024-04-16T18:10:38.000 DEBUG:boto:Token: None 2024-04-16T18:10:38.000 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:38 GMT /lybtug-40/ 2024-04-16T18:10:38.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dQaoles05PndzCmp0611M87+HX8= 2024-04-16T18:10:38.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dQaoles05PndzCmp0611M87+HX8=', 'Content-Length': '0'} 2024-04-16T18:10:38.003 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cdbc32ad46589086-00661ebf1e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.004 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:38.004 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:38.004 DEBUG:boto:Method: PUT 2024-04-16T18:10:38.004 DEBUG:boto:Path: /lybtug-40/obj8 2024-04-16T18:10:38.004 DEBUG:boto:Data: 2024-04-16T18:10:38.004 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:38.004 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:38.004 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.004 DEBUG:boto:Params: {} 2024-04-16T18:10:38.004 DEBUG:boto:Token: None 2024-04-16T18:10:38.005 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:38 GMT /lybtug-40/obj8 2024-04-16T18:10:38.005 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HM/g3wew9t0ReQTZz+RHwXKtYq0= 2024-04-16T18:10:38.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HM/g3wew9t0ReQTZz+RHwXKtYq0='} 2024-04-16T18:10:38.014 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000036795f715aed667b-00661ebf1e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.014 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:38.014 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:38.014 DEBUG:boto:Method: HEAD 2024-04-16T18:10:38.014 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:38.014 DEBUG:boto:Data: 2024-04-16T18:10:38.015 DEBUG:boto:Headers: {} 2024-04-16T18:10:38.015 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.015 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.015 DEBUG:boto:Params: {} 2024-04-16T18:10:38.015 DEBUG:boto:Token: None 2024-04-16T18:10:38.015 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/ 2024-04-16T18:10:38.015 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI= 2024-04-16T18:10:38.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI=', 'Content-Length': '0'} 2024-04-16T18:10:38.021 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006cc700a5c6dc9828-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.021 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:38.021 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:38.022 DEBUG:boto:Method: PUT 2024-04-16T18:10:38.022 DEBUG:boto:Path: /lybtug-41/obj5 2024-04-16T18:10:38.022 DEBUG:boto:Data: 2024-04-16T18:10:38.022 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:38.022 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.022 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.022 DEBUG:boto:Params: {} 2024-04-16T18:10:38.022 DEBUG:boto:Token: None 2024-04-16T18:10:38.022 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/obj5 2024-04-16T18:10:38.023 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xS6mOHUZOV1+POUFEbVfV4LZK+w= 2024-04-16T18:10:38.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xS6mOHUZOV1+POUFEbVfV4LZK+w='} 2024-04-16T18:10:38.031 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ff32a8f1b11c83db-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.031 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:38.032 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:38.032 DEBUG:boto:Method: HEAD 2024-04-16T18:10:38.032 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:38.032 DEBUG:boto:Data: 2024-04-16T18:10:38.032 DEBUG:boto:Headers: {} 2024-04-16T18:10:38.032 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.032 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.032 DEBUG:boto:Params: {} 2024-04-16T18:10:38.032 DEBUG:boto:Token: None 2024-04-16T18:10:38.032 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/ 2024-04-16T18:10:38.033 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI= 2024-04-16T18:10:38.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI=', 'Content-Length': '0'} 2024-04-16T18:10:38.036 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dff1b7c6eced6ab8-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.036 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:38.036 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:38.036 DEBUG:boto:Method: PUT 2024-04-16T18:10:38.036 DEBUG:boto:Path: /lybtug-41/obj6 2024-04-16T18:10:38.036 DEBUG:boto:Data: 2024-04-16T18:10:38.037 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:38.037 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.037 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.037 DEBUG:boto:Params: {} 2024-04-16T18:10:38.037 DEBUG:boto:Token: None 2024-04-16T18:10:38.037 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/obj6 2024-04-16T18:10:38.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XeGB7veNT0mYNC47T3i1jsNay7Q= 2024-04-16T18:10:38.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XeGB7veNT0mYNC47T3i1jsNay7Q='} 2024-04-16T18:10:38.082 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000058b085e5f78cc80d-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.083 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:38.083 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:38.083 DEBUG:boto:Method: HEAD 2024-04-16T18:10:38.083 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:38.083 DEBUG:boto:Data: 2024-04-16T18:10:38.083 DEBUG:boto:Headers: {} 2024-04-16T18:10:38.083 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.083 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.083 DEBUG:boto:Params: {} 2024-04-16T18:10:38.083 DEBUG:boto:Token: None 2024-04-16T18:10:38.084 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/ 2024-04-16T18:10:38.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI= 2024-04-16T18:10:38.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI=', 'Content-Length': '0'} 2024-04-16T18:10:38.087 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002938ea388a4f4b69-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.088 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:38.088 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:38.088 DEBUG:boto:Method: PUT 2024-04-16T18:10:38.088 DEBUG:boto:Path: /lybtug-41/obj7 2024-04-16T18:10:38.088 DEBUG:boto:Data: 2024-04-16T18:10:38.088 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:38.088 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.089 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.089 DEBUG:boto:Params: {} 2024-04-16T18:10:38.089 DEBUG:boto:Token: None 2024-04-16T18:10:38.089 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/obj7 2024-04-16T18:10:38.089 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4KS2FhEID2CIFNFzVeScjpBYpio= 2024-04-16T18:10:38.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4KS2FhEID2CIFNFzVeScjpBYpio='} 2024-04-16T18:10:38.098 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004a7a7ba5b4aec154-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.098 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:38.098 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:38.099 DEBUG:boto:Method: HEAD 2024-04-16T18:10:38.099 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:38.099 DEBUG:boto:Data: 2024-04-16T18:10:38.099 DEBUG:boto:Headers: {} 2024-04-16T18:10:38.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.099 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.099 DEBUG:boto:Params: {} 2024-04-16T18:10:38.099 DEBUG:boto:Token: None 2024-04-16T18:10:38.099 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/ 2024-04-16T18:10:38.099 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI= 2024-04-16T18:10:38.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b+v9d9Xsyh47iqQTIGg/RTErOyI=', 'Content-Length': '0'} 2024-04-16T18:10:38.103 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000064d99b15014442c2-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.103 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:38.103 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:38.103 DEBUG:boto:Method: PUT 2024-04-16T18:10:38.104 DEBUG:boto:Path: /lybtug-41/obj8 2024-04-16T18:10:38.104 DEBUG:boto:Data: 2024-04-16T18:10:38.104 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T18:10:38.104 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:38.104 DEBUG:boto:Port: 8000 2024-04-16T18:10:38.104 DEBUG:boto:Params: {} 2024-04-16T18:10:38.104 DEBUG:boto:Token: None 2024-04-16T18:10:38.104 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Tue, 16 Apr 2024 18:10:38 GMT /lybtug-41/obj8 2024-04-16T18:10:38.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ROij7By0o4KK2ASfGYncJpkECqQ= 2024-04-16T18:10:38.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ROij7By0o4KK2ASfGYncJpkECqQ='} 2024-04-16T18:10:38.147 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c9157e749fef89ec-00661ebf1e-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:38.147 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'lybtug-40', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:38.147 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'lybtug-40', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:38.147 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket lybtug-40 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:10:38.191 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:38.191 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:38.347 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'lybtug-41', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:38.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'lybtug-41', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:10:38.348 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket lybtug-41 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:10:38.434 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:38.434 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:38.593 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-40 2024-04-16T18:10:38.594 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:38.594 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:38.594 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-40 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:38.644 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:38.644 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:38.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:38.735+0000 7f3b09909980 1 waiting to reach incremental sync.. 2024-04-16T18:10:43.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:43.739+0000 7f3b09909980 1 bucket sync caught up with source: 2024-04-16T18:10:43.734 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000004.38.1, 00000000002.29.1, 00000000004.41.1, 00000000002.44.1, 00000000004.47.1, 00000000002.23.1, 00000000002.28.1, 00000000004.33.1, 00000000002.42.1, 00000000002.34.1, 00000000002.24.1] 2024-04-16T18:10:43.734 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.38.1, 00000000002.29.1, 00000000004.41.1, 00000000002.44.1, 00000000004.47.1, 00000000002.23.1, 00000000002.28.1, 00000000004.33.1, 00000000002.42.1, 00000000002.34.1, 00000000002.24.1] 2024-04-16T18:10:43.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:43.739+0000 7f3b09909980 0 bucket checkpoint complete 2024-04-16T18:10:43.780 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-40 2024-04-16T18:10:43.780 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:43.780 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-40', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:43.780 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-40 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:43.911+0000 7fba1618d980 1 bucket sync caught up with source: 2024-04-16T18:10:43.911 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000004.303.6, 00000000002.712.1, 00000000004.394.6, 00000000002.1296.1, 00000000004.552.6, 00000000002.835.1, 00000000002.449.1, 00000000004.403.6, 00000000002.887.1, 00000000002.399.1, 00000000002.998.1] 2024-04-16T18:10:43.911 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000004.303.6, 00000000002.712.1, 00000000004.394.6, 00000000002.1296.1, 00000000004.552.6, 00000000002.835.1, 00000000002.449.1, 00000000004.403.6, 00000000002.887.1, 00000000002.399.1, 00000000002.998.1] 2024-04-16T18:10:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:43.911+0000 7fba1618d980 0 bucket checkpoint complete 2024-04-16T18:10:43.943 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-40 zones={test-zone2, test-zone1} 2024-04-16T18:10:43.943 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:43.943 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:43.943 DEBUG:boto:Method: HEAD 2024-04-16T18:10:43.943 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:43.943 DEBUG:boto:Data: 2024-04-16T18:10:43.943 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.943 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:43.943 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.943 DEBUG:boto:Params: {} 2024-04-16T18:10:43.944 DEBUG:boto:Token: None 2024-04-16T18:10:43.944 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/ 2024-04-16T18:10:43.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YIALqsRnymVERECBzurxbluKyqg= 2024-04-16T18:10:43.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YIALqsRnymVERECBzurxbluKyqg=', 'Content-Length': '0'} 2024-04-16T18:10:43.948 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a98e243c3fa3e873-00661ebf23-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:43.949 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:43.949 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:43.949 DEBUG:boto:Method: HEAD 2024-04-16T18:10:43.949 DEBUG:boto:Path: /lybtug-40/ 2024-04-16T18:10:43.949 DEBUG:boto:Data: 2024-04-16T18:10:43.949 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.949 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:43.949 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.949 DEBUG:boto:Params: {} 2024-04-16T18:10:43.949 DEBUG:boto:Token: None 2024-04-16T18:10:43.950 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/ 2024-04-16T18:10:43.950 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YIALqsRnymVERECBzurxbluKyqg= 2024-04-16T18:10:43.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YIALqsRnymVERECBzurxbluKyqg=', 'Content-Length': '0'} 2024-04-16T18:10:43.952 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000511c58567367264c-00661ebf23-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:43.953 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:10:43.953 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:43.953 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:43.953 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.953 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.953 DEBUG:boto:Method: GET 2024-04-16T18:10:43.953 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.953 DEBUG:boto:Data: 2024-04-16T18:10:43.953 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.953 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:43.954 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.954 DEBUG:boto:Params: {} 2024-04-16T18:10:43.954 DEBUG:boto:Token: None 2024-04-16T18:10:43.954 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/?versions 2024-04-16T18:10:43.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8= 2024-04-16T18:10:43.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8=', 'Content-Length': '0'} 2024-04-16T18:10:43.958 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000894d19af247051f6-00661ebf23-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:43.959 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:43.959 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:43.959 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:43.959 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:10:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:10:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:10:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:10:43.960 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:10:43.960 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:43.960 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:43.960 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.960 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.960 DEBUG:boto:Method: GET 2024-04-16T18:10:43.960 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.961 DEBUG:boto:Data: 2024-04-16T18:10:43.961 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.961 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:43.961 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.961 DEBUG:boto:Params: {} 2024-04-16T18:10:43.961 DEBUG:boto:Token: None 2024-04-16T18:10:43.961 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/?versions 2024-04-16T18:10:43.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8= 2024-04-16T18:10:43.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8=', 'Content-Length': '0'} 2024-04-16T18:10:43.965 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000327b3f0528e07d85-00661ebf23-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:43.965 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:43.965 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:43.965 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:43.965 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:43.965 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:43.966 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:10:43.966 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:10:43.966 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:10:43.966 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:10:43.966 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:43.966 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:43.966 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.966 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.966 DEBUG:boto:Method: GET 2024-04-16T18:10:43.966 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.966 DEBUG:boto:Data: 2024-04-16T18:10:43.966 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:43.967 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.967 DEBUG:boto:Params: {} 2024-04-16T18:10:43.967 DEBUG:boto:Token: None 2024-04-16T18:10:43.967 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/?versions 2024-04-16T18:10:43.967 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8= 2024-04-16T18:10:43.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8=', 'Content-Length': '0'} 2024-04-16T18:10:43.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a98caa31fdc3838e-00661ebf23-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:43.972 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:43.973 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:43.973 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:43.973 DEBUG:boto:path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.973 DEBUG:boto:auth_path=/lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.973 DEBUG:boto:Method: GET 2024-04-16T18:10:43.973 DEBUG:boto:Path: /lybtug-40/?versions&max-keys=999 2024-04-16T18:10:43.973 DEBUG:boto:Data: 2024-04-16T18:10:43.973 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.974 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:43.974 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.974 DEBUG:boto:Params: {} 2024-04-16T18:10:43.974 DEBUG:boto:Token: None 2024-04-16T18:10:43.974 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/?versions 2024-04-16T18:10:43.974 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8= 2024-04-16T18:10:43.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FryjGKxdbvKSK1MwRfn3gxSZgW8=', 'Content-Length': '0'} 2024-04-16T18:10:43.977 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4e3003152bade13-00661ebf23-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:43.977 DEBUG:boto:b'lybtug-40999falseobj1nulltrue2024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:43.978 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:43.978 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:43.978 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:43.978 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:43.978 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:43.979 DEBUG:boto:Method: GET 2024-04-16T18:10:43.979 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:43.979 DEBUG:boto:Data: 2024-04-16T18:10:43.979 DEBUG:boto:Headers: {} 2024-04-16T18:10:43.979 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:43.979 DEBUG:boto:Port: 8000 2024-04-16T18:10:43.979 DEBUG:boto:Params: {} 2024-04-16T18:10:43.979 DEBUG:boto:Token: None 2024-04-16T18:10:43.979 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:43 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:43.979 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xivvpJBj+dbSJry3/fnBCxI7qlU= 2024-04-16T18:10:43.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xivvpJBj+dbSJry3/fnBCxI7qlU=', 'Content-Length': '0'} 2024-04-16T18:10:43.982 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b53d63e9cde266bc-00661ebf23-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.026 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.026 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.027 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.027 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.027 DEBUG:boto:Method: GET 2024-04-16T18:10:44.027 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.027 DEBUG:boto:Data: 2024-04-16T18:10:44.027 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.027 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.027 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.027 DEBUG:boto:Params: {} 2024-04-16T18:10:44.027 DEBUG:boto:Token: None 2024-04-16T18:10:44.027 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GmnjeFAh+WvH+XkhCPUQZ7SswX4= 2024-04-16T18:10:44.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GmnjeFAh+WvH+XkhCPUQZ7SswX4=', 'Content-Length': '0'} 2024-04-16T18:10:44.031 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c81096a451e4d51b-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.031 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.031 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.031 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.031 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.031 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.032 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.032 DEBUG:boto:Data: 2024-04-16T18:10:44.032 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.032 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.032 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.032 DEBUG:boto:Params: {} 2024-04-16T18:10:44.032 DEBUG:boto:Token: None 2024-04-16T18:10:44.032 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V5Gu0eir/k2YmyJKP6FB42kQ0ao= 2024-04-16T18:10:44.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V5Gu0eir/k2YmyJKP6FB42kQ0ao=', 'Content-Length': '0'} 2024-04-16T18:10:44.037 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b38c7dd0cb0116c6-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.038 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.038 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.038 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.038 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.038 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.038 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.038 DEBUG:boto:Data: 2024-04-16T18:10:44.038 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.039 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.039 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.039 DEBUG:boto:Params: {} 2024-04-16T18:10:44.039 DEBUG:boto:Token: None 2024-04-16T18:10:44.039 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.039 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V5Gu0eir/k2YmyJKP6FB42kQ0ao= 2024-04-16T18:10:44.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V5Gu0eir/k2YmyJKP6FB42kQ0ao=', 'Content-Length': '0'} 2024-04-16T18:10:44.042 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002fd96c7397592843-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:44.043 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.043 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.043 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.043 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.043 DEBUG:boto:Method: GET 2024-04-16T18:10:44.043 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.043 DEBUG:boto:Data: 2024-04-16T18:10:44.043 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.043 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.043 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.044 DEBUG:boto:Params: {} 2024-04-16T18:10:44.044 DEBUG:boto:Token: None 2024-04-16T18:10:44.044 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GmnjeFAh+WvH+XkhCPUQZ7SswX4= 2024-04-16T18:10:44.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GmnjeFAh+WvH+XkhCPUQZ7SswX4=', 'Content-Length': '0'} 2024-04-16T18:10:44.047 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003e26bf144afb7c79-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.090 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.090 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.091 DEBUG:boto:path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.091 DEBUG:boto:auth_path=/lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.091 DEBUG:boto:Method: GET 2024-04-16T18:10:44.091 DEBUG:boto:Path: /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.091 DEBUG:boto:Data: 2024-04-16T18:10:44.091 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.091 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.091 DEBUG:boto:Params: {} 2024-04-16T18:10:44.091 DEBUG:boto:Token: None 2024-04-16T18:10:44.091 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1?versionId=null 2024-04-16T18:10:44.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GmnjeFAh+WvH+XkhCPUQZ7SswX4= 2024-04-16T18:10:44.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GmnjeFAh+WvH+XkhCPUQZ7SswX4=', 'Content-Length': '0'} 2024-04-16T18:10:44.095 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a4c8cd2de30c4ddd-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.095 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.095 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.095 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.095 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.095 DEBUG:boto:Method: GET 2024-04-16T18:10:44.095 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:44.096 DEBUG:boto:Data: 2024-04-16T18:10:44.096 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.096 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.096 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.096 DEBUG:boto:Params: {} 2024-04-16T18:10:44.096 DEBUG:boto:Token: None 2024-04-16T18:10:44.096 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006de0f959e2a4a68-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.142 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.143 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.143 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.143 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.143 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.143 DEBUG:boto:Method: GET 2024-04-16T18:10:44.143 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:44.143 DEBUG:boto:Data: 2024-04-16T18:10:44.143 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.143 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.144 DEBUG:boto:Params: {} 2024-04-16T18:10:44.144 DEBUG:boto:Token: None 2024-04-16T18:10:44.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.147 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2f9441129e61add-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.148 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.148 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.148 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.148 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.148 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:44.148 DEBUG:boto:Data: 2024-04-16T18:10:44.148 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.148 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.149 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.149 DEBUG:boto:Params: {} 2024-04-16T18:10:44.149 DEBUG:boto:Token: None 2024-04-16T18:10:44.149 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1 2024-04-16T18:10:44.149 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CKb0076yN/oH/eiq5UA4ULfYTSA= 2024-04-16T18:10:44.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CKb0076yN/oH/eiq5UA4ULfYTSA=', 'Content-Length': '0'} 2024-04-16T18:10:44.152 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cab554b6e2c78250-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.152 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.152 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.152 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.152 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:44.152 DEBUG:boto:Data: 2024-04-16T18:10:44.152 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.152 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.153 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.153 DEBUG:boto:Params: {} 2024-04-16T18:10:44.153 DEBUG:boto:Token: None 2024-04-16T18:10:44.153 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1 2024-04-16T18:10:44.153 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CKb0076yN/oH/eiq5UA4ULfYTSA= 2024-04-16T18:10:44.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CKb0076yN/oH/eiq5UA4ULfYTSA=', 'Content-Length': '0'} 2024-04-16T18:10:44.156 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ed197dd014c39376-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.156 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:44.157 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.157 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.157 DEBUG:boto:Method: GET 2024-04-16T18:10:44.157 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:44.157 DEBUG:boto:Data: 2024-04-16T18:10:44.157 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.157 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.157 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.157 DEBUG:boto:Params: {} 2024-04-16T18:10:44.157 DEBUG:boto:Token: None 2024-04-16T18:10:44.158 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1 2024-04-16T18:10:44.158 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fTwicjS6Sw1QzQe/5R7lraAIF7g= 2024-04-16T18:10:44.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fTwicjS6Sw1QzQe/5R7lraAIF7g=', 'Content-Length': '0'} 2024-04-16T18:10:44.161 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f53745862a917865-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.202 DEBUG:boto:path=/lybtug-40/obj1 2024-04-16T18:10:44.202 DEBUG:boto:auth_path=/lybtug-40/obj1 2024-04-16T18:10:44.203 DEBUG:boto:Method: GET 2024-04-16T18:10:44.203 DEBUG:boto:Path: /lybtug-40/obj1 2024-04-16T18:10:44.203 DEBUG:boto:Data: 2024-04-16T18:10:44.203 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.203 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.203 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.203 DEBUG:boto:Params: {} 2024-04-16T18:10:44.203 DEBUG:boto:Token: None 2024-04-16T18:10:44.203 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj1 2024-04-16T18:10:44.203 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fTwicjS6Sw1QzQe/5R7lraAIF7g= 2024-04-16T18:10:44.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fTwicjS6Sw1QzQe/5R7lraAIF7g=', 'Content-Length': '0'} 2024-04-16T18:10:44.206 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007760112b2a8ffbc4-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.207 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.207 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.207 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.207 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.207 DEBUG:boto:Method: GET 2024-04-16T18:10:44.207 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:44.207 DEBUG:boto:Data: 2024-04-16T18:10:44.207 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.207 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.207 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.207 DEBUG:boto:Params: {} 2024-04-16T18:10:44.208 DEBUG:boto:Token: None 2024-04-16T18:10:44.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000084765deedc4d99a-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.254 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.255 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.255 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.255 DEBUG:boto:path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.255 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj1 2024-04-16T18:10:44.255 DEBUG:boto:Method: GET 2024-04-16T18:10:44.255 DEBUG:boto:Path: /lybtug-40/?prefix=obj1 2024-04-16T18:10:44.255 DEBUG:boto:Data: 2024-04-16T18:10:44.255 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.255 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.256 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.256 DEBUG:boto:Params: {} 2024-04-16T18:10:44.256 DEBUG:boto:Token: None 2024-04-16T18:10:44.256 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8f1626afb27056a-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.260 DEBUG:boto:b'lybtug-40obj11000falseobj12024-04-16T18:10:23.711Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.261 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:44.261 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.261 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.261 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.261 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.261 DEBUG:boto:Method: GET 2024-04-16T18:10:44.261 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.261 DEBUG:boto:Data: 2024-04-16T18:10:44.261 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.261 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.261 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.261 DEBUG:boto:Params: {} 2024-04-16T18:10:44.262 DEBUG:boto:Token: None 2024-04-16T18:10:44.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk= 2024-04-16T18:10:44.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk=', 'Content-Length': '0'} 2024-04-16T18:10:44.265 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000669a3b3494e2ac29-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.306 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.306 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.307 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.307 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.307 DEBUG:boto:Method: GET 2024-04-16T18:10:44.307 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.307 DEBUG:boto:Data: 2024-04-16T18:10:44.307 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.307 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.307 DEBUG:boto:Params: {} 2024-04-16T18:10:44.307 DEBUG:boto:Token: None 2024-04-16T18:10:44.307 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.307 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk= 2024-04-16T18:10:44.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk=', 'Content-Length': '0'} 2024-04-16T18:10:44.311 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cf357674c1154034-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.311 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.311 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.311 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.311 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.311 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.311 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.311 DEBUG:boto:Data: 2024-04-16T18:10:44.311 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.312 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.312 DEBUG:boto:Params: {} 2024-04-16T18:10:44.312 DEBUG:boto:Token: None 2024-04-16T18:10:44.312 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R/McoRtjA7u8u2/jtGspskXU77M= 2024-04-16T18:10:44.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R/McoRtjA7u8u2/jtGspskXU77M=', 'Content-Length': '0'} 2024-04-16T18:10:44.315 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001400664e7c78145b-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.315 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.315 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.315 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.315 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.316 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.316 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.316 DEBUG:boto:Data: 2024-04-16T18:10:44.316 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.316 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.316 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.316 DEBUG:boto:Params: {} 2024-04-16T18:10:44.316 DEBUG:boto:Token: None 2024-04-16T18:10:44.316 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R/McoRtjA7u8u2/jtGspskXU77M= 2024-04-16T18:10:44.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R/McoRtjA7u8u2/jtGspskXU77M=', 'Content-Length': '0'} 2024-04-16T18:10:44.319 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005c4a1aa2190e3fbc-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.319 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:44.319 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.319 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.319 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.320 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.320 DEBUG:boto:Method: GET 2024-04-16T18:10:44.320 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.320 DEBUG:boto:Data: 2024-04-16T18:10:44.320 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.320 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.320 DEBUG:boto:Params: {} 2024-04-16T18:10:44.320 DEBUG:boto:Token: None 2024-04-16T18:10:44.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk= 2024-04-16T18:10:44.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk=', 'Content-Length': '0'} 2024-04-16T18:10:44.323 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004e068aa366d71464-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.366 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.367 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.367 DEBUG:boto:path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.367 DEBUG:boto:auth_path=/lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.367 DEBUG:boto:Method: GET 2024-04-16T18:10:44.367 DEBUG:boto:Path: /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.367 DEBUG:boto:Data: 2024-04-16T18:10:44.367 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.367 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.367 DEBUG:boto:Params: {} 2024-04-16T18:10:44.367 DEBUG:boto:Token: None 2024-04-16T18:10:44.368 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2?versionId=null 2024-04-16T18:10:44.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk= 2024-04-16T18:10:44.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsQ/fdX/Hl7YuOhbWgU/IfsbQYk=', 'Content-Length': '0'} 2024-04-16T18:10:44.371 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d70d4e5778f2d8de-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.371 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.371 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.371 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.371 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.371 DEBUG:boto:Method: GET 2024-04-16T18:10:44.372 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:44.372 DEBUG:boto:Data: 2024-04-16T18:10:44.372 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.372 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.372 DEBUG:boto:Params: {} 2024-04-16T18:10:44.372 DEBUG:boto:Token: None 2024-04-16T18:10:44.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c463a7c5c3abe985-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.376 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.377 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.377 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.377 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.377 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.377 DEBUG:boto:Method: GET 2024-04-16T18:10:44.377 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:44.377 DEBUG:boto:Data: 2024-04-16T18:10:44.377 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.378 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.378 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.378 DEBUG:boto:Params: {} 2024-04-16T18:10:44.378 DEBUG:boto:Token: None 2024-04-16T18:10:44.378 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.378 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.382 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000293ef05aa1307743-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.382 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.382 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.382 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.383 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.383 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:44.383 DEBUG:boto:Data: 2024-04-16T18:10:44.383 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.383 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.383 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.383 DEBUG:boto:Params: {} 2024-04-16T18:10:44.383 DEBUG:boto:Token: None 2024-04-16T18:10:44.383 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2 2024-04-16T18:10:44.383 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jwdjdtUv9pkGdpjSP5Gs3BW/Fjs= 2024-04-16T18:10:44.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jwdjdtUv9pkGdpjSP5Gs3BW/Fjs=', 'Content-Length': '0'} 2024-04-16T18:10:44.386 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f42cd04a838e21a1-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.387 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.387 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.387 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.387 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:44.387 DEBUG:boto:Data: 2024-04-16T18:10:44.387 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.387 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.387 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.387 DEBUG:boto:Params: {} 2024-04-16T18:10:44.387 DEBUG:boto:Token: None 2024-04-16T18:10:44.387 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2 2024-04-16T18:10:44.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jwdjdtUv9pkGdpjSP5Gs3BW/Fjs= 2024-04-16T18:10:44.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jwdjdtUv9pkGdpjSP5Gs3BW/Fjs=', 'Content-Length': '0'} 2024-04-16T18:10:44.390 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009df117e806ef0e9c-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.391 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:44.391 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.391 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.391 DEBUG:boto:Method: GET 2024-04-16T18:10:44.391 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:44.391 DEBUG:boto:Data: 2024-04-16T18:10:44.391 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.391 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.391 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.391 DEBUG:boto:Params: {} 2024-04-16T18:10:44.391 DEBUG:boto:Token: None 2024-04-16T18:10:44.392 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2 2024-04-16T18:10:44.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Y8SCutNDvJbWyaeF0squcsuL/8= 2024-04-16T18:10:44.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Y8SCutNDvJbWyaeF0squcsuL/8=', 'Content-Length': '0'} 2024-04-16T18:10:44.395 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f5e75c33f9131166-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.438 DEBUG:boto:path=/lybtug-40/obj2 2024-04-16T18:10:44.439 DEBUG:boto:auth_path=/lybtug-40/obj2 2024-04-16T18:10:44.439 DEBUG:boto:Method: GET 2024-04-16T18:10:44.439 DEBUG:boto:Path: /lybtug-40/obj2 2024-04-16T18:10:44.439 DEBUG:boto:Data: 2024-04-16T18:10:44.439 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.439 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.439 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.439 DEBUG:boto:Params: {} 2024-04-16T18:10:44.439 DEBUG:boto:Token: None 2024-04-16T18:10:44.439 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj2 2024-04-16T18:10:44.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Y8SCutNDvJbWyaeF0squcsuL/8= 2024-04-16T18:10:44.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Y8SCutNDvJbWyaeF0squcsuL/8=', 'Content-Length': '0'} 2024-04-16T18:10:44.442 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c3b861256d055cf0-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.443 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.443 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.443 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.443 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.443 DEBUG:boto:Method: GET 2024-04-16T18:10:44.443 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:44.443 DEBUG:boto:Data: 2024-04-16T18:10:44.443 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.443 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.443 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.443 DEBUG:boto:Params: {} 2024-04-16T18:10:44.443 DEBUG:boto:Token: None 2024-04-16T18:10:44.444 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005250ad741415da8f-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.448 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.449 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.449 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.449 DEBUG:boto:path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.449 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj2 2024-04-16T18:10:44.449 DEBUG:boto:Method: GET 2024-04-16T18:10:44.449 DEBUG:boto:Path: /lybtug-40/?prefix=obj2 2024-04-16T18:10:44.449 DEBUG:boto:Data: 2024-04-16T18:10:44.449 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.449 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.450 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.450 DEBUG:boto:Params: {} 2024-04-16T18:10:44.450 DEBUG:boto:Token: None 2024-04-16T18:10:44.450 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.450 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.453 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007473fcfd03fe57d3-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.453 DEBUG:boto:b'lybtug-40obj21000falseobj22024-04-16T18:10:23.729Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.454 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:44.454 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.454 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.454 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.454 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.454 DEBUG:boto:Method: GET 2024-04-16T18:10:44.454 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.454 DEBUG:boto:Data: 2024-04-16T18:10:44.454 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.455 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.455 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.455 DEBUG:boto:Params: {} 2024-04-16T18:10:44.455 DEBUG:boto:Token: None 2024-04-16T18:10:44.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.455 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw= 2024-04-16T18:10:44.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw=', 'Content-Length': '0'} 2024-04-16T18:10:44.458 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000190428425bce0608-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.502 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.503 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.503 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.503 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.503 DEBUG:boto:Method: GET 2024-04-16T18:10:44.503 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.503 DEBUG:boto:Data: 2024-04-16T18:10:44.503 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.503 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.503 DEBUG:boto:Params: {} 2024-04-16T18:10:44.503 DEBUG:boto:Token: None 2024-04-16T18:10:44.503 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw= 2024-04-16T18:10:44.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw=', 'Content-Length': '0'} 2024-04-16T18:10:44.507 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d59d1bbda49f68e0-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.507 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.507 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.507 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.507 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.507 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.507 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.507 DEBUG:boto:Data: 2024-04-16T18:10:44.508 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.508 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.508 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.508 DEBUG:boto:Params: {} 2024-04-16T18:10:44.508 DEBUG:boto:Token: None 2024-04-16T18:10:44.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+tM1Op3uNnzYFGxWomRA9UY1Ch4= 2024-04-16T18:10:44.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+tM1Op3uNnzYFGxWomRA9UY1Ch4=', 'Content-Length': '0'} 2024-04-16T18:10:44.511 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ff059da6174f9d5a-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.511 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.511 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.512 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.512 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.512 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.512 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.512 DEBUG:boto:Data: 2024-04-16T18:10:44.512 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.512 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.512 DEBUG:boto:Params: {} 2024-04-16T18:10:44.512 DEBUG:boto:Token: None 2024-04-16T18:10:44.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+tM1Op3uNnzYFGxWomRA9UY1Ch4= 2024-04-16T18:10:44.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+tM1Op3uNnzYFGxWomRA9UY1Ch4=', 'Content-Length': '0'} 2024-04-16T18:10:44.515 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008620780ae2d267a7-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:44.515 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.516 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.516 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.516 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.516 DEBUG:boto:Method: GET 2024-04-16T18:10:44.516 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.516 DEBUG:boto:Data: 2024-04-16T18:10:44.516 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.516 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.516 DEBUG:boto:Params: {} 2024-04-16T18:10:44.516 DEBUG:boto:Token: None 2024-04-16T18:10:44.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw= 2024-04-16T18:10:44.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw=', 'Content-Length': '0'} 2024-04-16T18:10:44.519 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005736350bf2d1d985-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.562 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.563 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.563 DEBUG:boto:path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.563 DEBUG:boto:auth_path=/lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.563 DEBUG:boto:Method: GET 2024-04-16T18:10:44.563 DEBUG:boto:Path: /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.563 DEBUG:boto:Data: 2024-04-16T18:10:44.563 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.563 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.563 DEBUG:boto:Params: {} 2024-04-16T18:10:44.563 DEBUG:boto:Token: None 2024-04-16T18:10:44.563 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3?versionId=null 2024-04-16T18:10:44.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw= 2024-04-16T18:10:44.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjiNya0jY2clbGVZYnUEMZWmwlw=', 'Content-Length': '0'} 2024-04-16T18:10:44.567 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000014aec231e7e40f51-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.567 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.567 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.567 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.567 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.567 DEBUG:boto:Method: GET 2024-04-16T18:10:44.567 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:44.567 DEBUG:boto:Data: 2024-04-16T18:10:44.567 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.567 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.568 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.568 DEBUG:boto:Params: {} 2024-04-16T18:10:44.568 DEBUG:boto:Token: None 2024-04-16T18:10:44.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e4c9e91f74cfa1e-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.572 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.573 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.573 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.573 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.573 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.573 DEBUG:boto:Method: GET 2024-04-16T18:10:44.573 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:44.573 DEBUG:boto:Data: 2024-04-16T18:10:44.573 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.573 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.573 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.573 DEBUG:boto:Params: {} 2024-04-16T18:10:44.574 DEBUG:boto:Token: None 2024-04-16T18:10:44.574 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.574 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eca741b4fefc6233-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.578 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.578 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.578 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.578 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.579 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:44.579 DEBUG:boto:Data: 2024-04-16T18:10:44.579 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.579 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.579 DEBUG:boto:Params: {} 2024-04-16T18:10:44.579 DEBUG:boto:Token: None 2024-04-16T18:10:44.579 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3 2024-04-16T18:10:44.579 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1oLekVFhhgXLTBxDKtMq/HIBDtU= 2024-04-16T18:10:44.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1oLekVFhhgXLTBxDKtMq/HIBDtU=', 'Content-Length': '0'} 2024-04-16T18:10:44.582 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ef9f1ac9aeb6a2ab-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.582 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.582 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.582 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.582 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:44.582 DEBUG:boto:Data: 2024-04-16T18:10:44.582 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.583 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.583 DEBUG:boto:Params: {} 2024-04-16T18:10:44.583 DEBUG:boto:Token: None 2024-04-16T18:10:44.583 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3 2024-04-16T18:10:44.583 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1oLekVFhhgXLTBxDKtMq/HIBDtU= 2024-04-16T18:10:44.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1oLekVFhhgXLTBxDKtMq/HIBDtU=', 'Content-Length': '0'} 2024-04-16T18:10:44.586 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003448f269f1230991-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.586 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:44.586 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.586 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.586 DEBUG:boto:Method: GET 2024-04-16T18:10:44.586 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:44.586 DEBUG:boto:Data: 2024-04-16T18:10:44.587 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.587 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.587 DEBUG:boto:Params: {} 2024-04-16T18:10:44.587 DEBUG:boto:Token: None 2024-04-16T18:10:44.587 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3 2024-04-16T18:10:44.587 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uwnwbf0SI4hlB6AG1+uRIjCAHbE= 2024-04-16T18:10:44.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uwnwbf0SI4hlB6AG1+uRIjCAHbE=', 'Content-Length': '0'} 2024-04-16T18:10:44.590 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d97375a7f0986a19-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.630 DEBUG:boto:path=/lybtug-40/obj3 2024-04-16T18:10:44.630 DEBUG:boto:auth_path=/lybtug-40/obj3 2024-04-16T18:10:44.630 DEBUG:boto:Method: GET 2024-04-16T18:10:44.631 DEBUG:boto:Path: /lybtug-40/obj3 2024-04-16T18:10:44.631 DEBUG:boto:Data: 2024-04-16T18:10:44.631 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.631 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.631 DEBUG:boto:Params: {} 2024-04-16T18:10:44.631 DEBUG:boto:Token: None 2024-04-16T18:10:44.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj3 2024-04-16T18:10:44.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uwnwbf0SI4hlB6AG1+uRIjCAHbE= 2024-04-16T18:10:44.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uwnwbf0SI4hlB6AG1+uRIjCAHbE=', 'Content-Length': '0'} 2024-04-16T18:10:44.634 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000656bc596f74512ec-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.634 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.635 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.635 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.635 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.635 DEBUG:boto:Method: GET 2024-04-16T18:10:44.635 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:44.635 DEBUG:boto:Data: 2024-04-16T18:10:44.635 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.635 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.635 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.635 DEBUG:boto:Params: {} 2024-04-16T18:10:44.635 DEBUG:boto:Token: None 2024-04-16T18:10:44.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000476c7fc12abe00a5-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.686 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.687 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.687 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.687 DEBUG:boto:path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.687 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj3 2024-04-16T18:10:44.687 DEBUG:boto:Method: GET 2024-04-16T18:10:44.687 DEBUG:boto:Path: /lybtug-40/?prefix=obj3 2024-04-16T18:10:44.687 DEBUG:boto:Data: 2024-04-16T18:10:44.687 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.687 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.687 DEBUG:boto:Params: {} 2024-04-16T18:10:44.688 DEBUG:boto:Token: None 2024-04-16T18:10:44.688 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.692 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000501c279a10689948-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.692 DEBUG:boto:b'lybtug-40obj31000falseobj32024-04-16T18:10:23.780Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.692 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:44.692 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.693 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.693 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.693 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.693 DEBUG:boto:Method: GET 2024-04-16T18:10:44.693 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.693 DEBUG:boto:Data: 2024-04-16T18:10:44.693 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.693 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.693 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.693 DEBUG:boto:Params: {} 2024-04-16T18:10:44.693 DEBUG:boto:Token: None 2024-04-16T18:10:44.693 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.694 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU= 2024-04-16T18:10:44.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU=', 'Content-Length': '0'} 2024-04-16T18:10:44.697 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bfdbf3aa034780c6-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.738 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.739 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.739 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.739 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.739 DEBUG:boto:Method: GET 2024-04-16T18:10:44.739 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.739 DEBUG:boto:Data: 2024-04-16T18:10:44.739 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.739 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.739 DEBUG:boto:Params: {} 2024-04-16T18:10:44.739 DEBUG:boto:Token: None 2024-04-16T18:10:44.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU= 2024-04-16T18:10:44.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU=', 'Content-Length': '0'} 2024-04-16T18:10:44.743 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b258c7497ba6fbdf-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.743 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.743 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.744 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.744 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.744 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.744 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.744 DEBUG:boto:Data: 2024-04-16T18:10:44.744 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.744 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.744 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.744 DEBUG:boto:Params: {} 2024-04-16T18:10:44.744 DEBUG:boto:Token: None 2024-04-16T18:10:44.744 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:57OOLxbgiXDmcvZumctcZW8lGiY= 2024-04-16T18:10:44.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:57OOLxbgiXDmcvZumctcZW8lGiY=', 'Content-Length': '0'} 2024-04-16T18:10:44.747 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001eef3dd25dcaf7ba-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.748 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.748 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.748 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.748 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.748 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.748 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.748 DEBUG:boto:Data: 2024-04-16T18:10:44.748 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.748 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.749 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.749 DEBUG:boto:Params: {} 2024-04-16T18:10:44.749 DEBUG:boto:Token: None 2024-04-16T18:10:44.749 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.749 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:57OOLxbgiXDmcvZumctcZW8lGiY= 2024-04-16T18:10:44.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:57OOLxbgiXDmcvZumctcZW8lGiY=', 'Content-Length': '0'} 2024-04-16T18:10:44.752 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006b165366e12386dc-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:44.752 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.752 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.752 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.753 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.753 DEBUG:boto:Method: GET 2024-04-16T18:10:44.753 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.753 DEBUG:boto:Data: 2024-04-16T18:10:44.753 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.753 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.753 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.753 DEBUG:boto:Params: {} 2024-04-16T18:10:44.753 DEBUG:boto:Token: None 2024-04-16T18:10:44.753 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU= 2024-04-16T18:10:44.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU=', 'Content-Length': '0'} 2024-04-16T18:10:44.756 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008cdd836d924b622f-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.798 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.798 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.798 DEBUG:boto:path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.799 DEBUG:boto:auth_path=/lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.799 DEBUG:boto:Method: GET 2024-04-16T18:10:44.799 DEBUG:boto:Path: /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.799 DEBUG:boto:Data: 2024-04-16T18:10:44.799 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.799 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.799 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.799 DEBUG:boto:Params: {} 2024-04-16T18:10:44.799 DEBUG:boto:Token: None 2024-04-16T18:10:44.799 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4?versionId=null 2024-04-16T18:10:44.799 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU= 2024-04-16T18:10:44.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hKs+aAKIIug3X+upNaID85aBueU=', 'Content-Length': '0'} 2024-04-16T18:10:44.802 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000546c8ed3f4792279-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.803 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.803 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.803 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.803 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.803 DEBUG:boto:Method: GET 2024-04-16T18:10:44.803 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:44.803 DEBUG:boto:Data: 2024-04-16T18:10:44.803 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.803 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.803 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.803 DEBUG:boto:Params: {} 2024-04-16T18:10:44.804 DEBUG:boto:Token: None 2024-04-16T18:10:44.804 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.808 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000139ad794e713e5d4-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.808 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.808 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.808 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.808 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.808 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.808 DEBUG:boto:Method: GET 2024-04-16T18:10:44.808 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:44.809 DEBUG:boto:Data: 2024-04-16T18:10:44.809 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.809 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.809 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.809 DEBUG:boto:Params: {} 2024-04-16T18:10:44.809 DEBUG:boto:Token: None 2024-04-16T18:10:44.809 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.813 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4412e94d5b02a18-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.854 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.855 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.855 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.855 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.855 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:44.855 DEBUG:boto:Data: 2024-04-16T18:10:44.855 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.855 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.855 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.855 DEBUG:boto:Params: {} 2024-04-16T18:10:44.855 DEBUG:boto:Token: None 2024-04-16T18:10:44.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4 2024-04-16T18:10:44.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z7srTCadb5OFm+TsgbdbdFONScs= 2024-04-16T18:10:44.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z7srTCadb5OFm+TsgbdbdFONScs=', 'Content-Length': '0'} 2024-04-16T18:10:44.859 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009b6438e9b41a9c93-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.859 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.859 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.859 DEBUG:boto:Method: HEAD 2024-04-16T18:10:44.859 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:44.860 DEBUG:boto:Data: 2024-04-16T18:10:44.860 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.860 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.860 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.860 DEBUG:boto:Params: {} 2024-04-16T18:10:44.860 DEBUG:boto:Token: None 2024-04-16T18:10:44.860 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4 2024-04-16T18:10:44.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z7srTCadb5OFm+TsgbdbdFONScs= 2024-04-16T18:10:44.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z7srTCadb5OFm+TsgbdbdFONScs=', 'Content-Length': '0'} 2024-04-16T18:10:44.863 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f650e4cd187ad90c-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.864 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:44.864 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.864 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.864 DEBUG:boto:Method: GET 2024-04-16T18:10:44.864 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:44.864 DEBUG:boto:Data: 2024-04-16T18:10:44.864 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.864 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.865 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.865 DEBUG:boto:Params: {} 2024-04-16T18:10:44.865 DEBUG:boto:Token: None 2024-04-16T18:10:44.865 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4 2024-04-16T18:10:44.865 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B6tv3iaLDryE5TUN4B8fmJSeQGA= 2024-04-16T18:10:44.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B6tv3iaLDryE5TUN4B8fmJSeQGA=', 'Content-Length': '0'} 2024-04-16T18:10:44.868 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000079a713afa96085e7-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.910 DEBUG:boto:path=/lybtug-40/obj4 2024-04-16T18:10:44.910 DEBUG:boto:auth_path=/lybtug-40/obj4 2024-04-16T18:10:44.911 DEBUG:boto:Method: GET 2024-04-16T18:10:44.911 DEBUG:boto:Path: /lybtug-40/obj4 2024-04-16T18:10:44.911 DEBUG:boto:Data: 2024-04-16T18:10:44.911 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.911 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.911 DEBUG:boto:Params: {} 2024-04-16T18:10:44.911 DEBUG:boto:Token: None 2024-04-16T18:10:44.911 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj4 2024-04-16T18:10:44.911 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B6tv3iaLDryE5TUN4B8fmJSeQGA= 2024-04-16T18:10:44.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B6tv3iaLDryE5TUN4B8fmJSeQGA=', 'Content-Length': '0'} 2024-04-16T18:10:44.914 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000083d59065e4a2009b-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.915 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.915 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.915 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.915 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.915 DEBUG:boto:Method: GET 2024-04-16T18:10:44.915 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:44.915 DEBUG:boto:Data: 2024-04-16T18:10:44.915 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.916 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.916 DEBUG:boto:Params: {} 2024-04-16T18:10:44.916 DEBUG:boto:Token: None 2024-04-16T18:10:44.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8b5c74f898ca9d8-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.962 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.963 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:44.963 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:44.963 DEBUG:boto:path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.963 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj4 2024-04-16T18:10:44.963 DEBUG:boto:Method: GET 2024-04-16T18:10:44.963 DEBUG:boto:Path: /lybtug-40/?prefix=obj4 2024-04-16T18:10:44.963 DEBUG:boto:Data: 2024-04-16T18:10:44.963 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:44.964 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.964 DEBUG:boto:Params: {} 2024-04-16T18:10:44.964 DEBUG:boto:Token: None 2024-04-16T18:10:44.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/ 2024-04-16T18:10:44.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4= 2024-04-16T18:10:44.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCPv+oSI6dMUMwqyqc4lMmAaul4=', 'Content-Length': '0'} 2024-04-16T18:10:44.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec1f0ed325940653-00661ebf24-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:44.969 DEBUG:boto:b'lybtug-40obj41000falseobj42024-04-16T18:10:23.798Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:44.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:44.969 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:44.969 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:44.969 DEBUG:boto:path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:44.970 DEBUG:boto:auth_path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:44.970 DEBUG:boto:Method: GET 2024-04-16T18:10:44.970 DEBUG:boto:Path: /lybtug-40/obj5?versionId=null 2024-04-16T18:10:44.970 DEBUG:boto:Data: 2024-04-16T18:10:44.970 DEBUG:boto:Headers: {} 2024-04-16T18:10:44.970 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:44.970 DEBUG:boto:Port: 8000 2024-04-16T18:10:44.970 DEBUG:boto:Params: {} 2024-04-16T18:10:44.970 DEBUG:boto:Token: None 2024-04-16T18:10:44.970 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:44 GMT /lybtug-40/obj5?versionId=null 2024-04-16T18:10:44.971 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L/6+gpeMreDtOMOcyedCmHSaKuc= 2024-04-16T18:10:44.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L/6+gpeMreDtOMOcyedCmHSaKuc=', 'Content-Length': '0'} 2024-04-16T18:10:44.974 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004fd76f4bd23b6876-00661ebf24-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.014 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.014 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.015 DEBUG:boto:path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.015 DEBUG:boto:auth_path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.015 DEBUG:boto:Method: GET 2024-04-16T18:10:45.015 DEBUG:boto:Path: /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.015 DEBUG:boto:Data: 2024-04-16T18:10:45.015 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.015 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.015 DEBUG:boto:Params: {} 2024-04-16T18:10:45.015 DEBUG:boto:Token: None 2024-04-16T18:10:45.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lh7AFI7AvlyrYns3DnXthqr9mPk= 2024-04-16T18:10:45.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lh7AFI7AvlyrYns3DnXthqr9mPk=', 'Content-Length': '0'} 2024-04-16T18:10:45.019 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bfc1c13f93e86e62-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.019 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.019 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.019 DEBUG:boto:path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.019 DEBUG:boto:auth_path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.019 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.020 DEBUG:boto:Path: /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.020 DEBUG:boto:Data: 2024-04-16T18:10:45.020 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.020 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.020 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.020 DEBUG:boto:Params: {} 2024-04-16T18:10:45.020 DEBUG:boto:Token: None 2024-04-16T18:10:45.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FeA1FjtLJj9KN+3GX6/tybRRq+A= 2024-04-16T18:10:45.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FeA1FjtLJj9KN+3GX6/tybRRq+A=', 'Content-Length': '0'} 2024-04-16T18:10:45.023 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f49f12cc1c9da005-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.023 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.023 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.024 DEBUG:boto:path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.024 DEBUG:boto:auth_path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.024 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.024 DEBUG:boto:Path: /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.024 DEBUG:boto:Data: 2024-04-16T18:10:45.024 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.024 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.024 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.024 DEBUG:boto:Params: {} 2024-04-16T18:10:45.024 DEBUG:boto:Token: None 2024-04-16T18:10:45.025 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FeA1FjtLJj9KN+3GX6/tybRRq+A= 2024-04-16T18:10:45.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FeA1FjtLJj9KN+3GX6/tybRRq+A=', 'Content-Length': '0'} 2024-04-16T18:10:45.028 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000465b893fadb2799e-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.028 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:45.028 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.028 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.028 DEBUG:boto:path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.028 DEBUG:boto:auth_path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.028 DEBUG:boto:Method: GET 2024-04-16T18:10:45.028 DEBUG:boto:Path: /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.029 DEBUG:boto:Data: 2024-04-16T18:10:45.029 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.029 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.029 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.029 DEBUG:boto:Params: {} 2024-04-16T18:10:45.029 DEBUG:boto:Token: None 2024-04-16T18:10:45.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lh7AFI7AvlyrYns3DnXthqr9mPk= 2024-04-16T18:10:45.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lh7AFI7AvlyrYns3DnXthqr9mPk=', 'Content-Length': '0'} 2024-04-16T18:10:45.032 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e058b0f03c31ada1-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.074 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.074 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.075 DEBUG:boto:path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.075 DEBUG:boto:auth_path=/lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.075 DEBUG:boto:Method: GET 2024-04-16T18:10:45.075 DEBUG:boto:Path: /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.075 DEBUG:boto:Data: 2024-04-16T18:10:45.075 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.075 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.075 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.075 DEBUG:boto:Params: {} 2024-04-16T18:10:45.075 DEBUG:boto:Token: None 2024-04-16T18:10:45.075 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5?versionId=null 2024-04-16T18:10:45.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Lh7AFI7AvlyrYns3DnXthqr9mPk= 2024-04-16T18:10:45.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Lh7AFI7AvlyrYns3DnXthqr9mPk=', 'Content-Length': '0'} 2024-04-16T18:10:45.079 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009b2e9333b0cc93c0-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.079 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.079 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.079 DEBUG:boto:path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.079 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.079 DEBUG:boto:Method: GET 2024-04-16T18:10:45.079 DEBUG:boto:Path: /lybtug-40/?prefix=obj5 2024-04-16T18:10:45.080 DEBUG:boto:Data: 2024-04-16T18:10:45.080 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.080 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.080 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.080 DEBUG:boto:Params: {} 2024-04-16T18:10:45.080 DEBUG:boto:Token: None 2024-04-16T18:10:45.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001534dde2aed3a405-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.085 DEBUG:boto:b'lybtug-40obj51000falseobj52024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.085 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.085 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.085 DEBUG:boto:path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.085 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.085 DEBUG:boto:Method: GET 2024-04-16T18:10:45.085 DEBUG:boto:Path: /lybtug-40/?prefix=obj5 2024-04-16T18:10:45.086 DEBUG:boto:Data: 2024-04-16T18:10:45.086 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.086 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.086 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.086 DEBUG:boto:Params: {} 2024-04-16T18:10:45.086 DEBUG:boto:Token: None 2024-04-16T18:10:45.086 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.086 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.090 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002efd583f396472b4-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.130 DEBUG:boto:b'lybtug-40obj51000falseobj52024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.131 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.131 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.131 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.131 DEBUG:boto:Path: /lybtug-40/obj5 2024-04-16T18:10:45.131 DEBUG:boto:Data: 2024-04-16T18:10:45.131 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.131 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.131 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.131 DEBUG:boto:Params: {} 2024-04-16T18:10:45.131 DEBUG:boto:Token: None 2024-04-16T18:10:45.132 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5 2024-04-16T18:10:45.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hxth0zskqWSfkGOYNz65Wxur3yA= 2024-04-16T18:10:45.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hxth0zskqWSfkGOYNz65Wxur3yA=', 'Content-Length': '0'} 2024-04-16T18:10:45.135 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003a464797cef43902-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.135 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.135 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.135 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.135 DEBUG:boto:Path: /lybtug-40/obj5 2024-04-16T18:10:45.135 DEBUG:boto:Data: 2024-04-16T18:10:45.135 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.136 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.136 DEBUG:boto:Params: {} 2024-04-16T18:10:45.136 DEBUG:boto:Token: None 2024-04-16T18:10:45.136 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5 2024-04-16T18:10:45.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hxth0zskqWSfkGOYNz65Wxur3yA= 2024-04-16T18:10:45.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hxth0zskqWSfkGOYNz65Wxur3yA=', 'Content-Length': '0'} 2024-04-16T18:10:45.139 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000acbfb7828a55919b-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:45.139 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.139 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.139 DEBUG:boto:Method: GET 2024-04-16T18:10:45.139 DEBUG:boto:Path: /lybtug-40/obj5 2024-04-16T18:10:45.139 DEBUG:boto:Data: 2024-04-16T18:10:45.140 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.140 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.140 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.140 DEBUG:boto:Params: {} 2024-04-16T18:10:45.140 DEBUG:boto:Token: None 2024-04-16T18:10:45.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5 2024-04-16T18:10:45.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HM85NVYdOaDDX0G32+jSkdYkad0= 2024-04-16T18:10:45.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HM85NVYdOaDDX0G32+jSkdYkad0=', 'Content-Length': '0'} 2024-04-16T18:10:45.143 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d2521de5d10cbeb5-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.186 DEBUG:boto:path=/lybtug-40/obj5 2024-04-16T18:10:45.186 DEBUG:boto:auth_path=/lybtug-40/obj5 2024-04-16T18:10:45.187 DEBUG:boto:Method: GET 2024-04-16T18:10:45.187 DEBUG:boto:Path: /lybtug-40/obj5 2024-04-16T18:10:45.187 DEBUG:boto:Data: 2024-04-16T18:10:45.187 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.187 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.187 DEBUG:boto:Params: {} 2024-04-16T18:10:45.187 DEBUG:boto:Token: None 2024-04-16T18:10:45.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj5 2024-04-16T18:10:45.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HM85NVYdOaDDX0G32+jSkdYkad0= 2024-04-16T18:10:45.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HM85NVYdOaDDX0G32+jSkdYkad0=', 'Content-Length': '0'} 2024-04-16T18:10:45.190 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002fabdbecb9384c0e-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.191 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.191 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.191 DEBUG:boto:path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.191 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.191 DEBUG:boto:Method: GET 2024-04-16T18:10:45.191 DEBUG:boto:Path: /lybtug-40/?prefix=obj5 2024-04-16T18:10:45.191 DEBUG:boto:Data: 2024-04-16T18:10:45.191 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.191 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.191 DEBUG:boto:Params: {} 2024-04-16T18:10:45.192 DEBUG:boto:Token: None 2024-04-16T18:10:45.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044614d0a8e8df145-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.238 DEBUG:boto:b'lybtug-40obj51000falseobj52024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.239 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.239 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.239 DEBUG:boto:path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.239 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj5 2024-04-16T18:10:45.239 DEBUG:boto:Method: GET 2024-04-16T18:10:45.239 DEBUG:boto:Path: /lybtug-40/?prefix=obj5 2024-04-16T18:10:45.239 DEBUG:boto:Data: 2024-04-16T18:10:45.239 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.239 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.239 DEBUG:boto:Params: {} 2024-04-16T18:10:45.240 DEBUG:boto:Token: None 2024-04-16T18:10:45.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d208df36c0cd12d4-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.244 DEBUG:boto:b'lybtug-40obj51000falseobj52024-04-16T18:10:37.931Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.244 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:45.244 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.244 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.244 DEBUG:boto:path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.245 DEBUG:boto:auth_path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.245 DEBUG:boto:Method: GET 2024-04-16T18:10:45.245 DEBUG:boto:Path: /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.245 DEBUG:boto:Data: 2024-04-16T18:10:45.245 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.245 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.245 DEBUG:boto:Params: {} 2024-04-16T18:10:45.245 DEBUG:boto:Token: None 2024-04-16T18:10:45.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc= 2024-04-16T18:10:45.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc=', 'Content-Length': '0'} 2024-04-16T18:10:45.249 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000051b82f525f763fc4-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.294 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.294 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.295 DEBUG:boto:path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.295 DEBUG:boto:auth_path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.295 DEBUG:boto:Method: GET 2024-04-16T18:10:45.295 DEBUG:boto:Path: /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.295 DEBUG:boto:Data: 2024-04-16T18:10:45.295 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.295 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.295 DEBUG:boto:Params: {} 2024-04-16T18:10:45.295 DEBUG:boto:Token: None 2024-04-16T18:10:45.295 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc= 2024-04-16T18:10:45.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc=', 'Content-Length': '0'} 2024-04-16T18:10:45.299 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002299a1561df84ba5-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.299 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.299 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.299 DEBUG:boto:path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.299 DEBUG:boto:auth_path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.299 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.300 DEBUG:boto:Path: /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.300 DEBUG:boto:Data: 2024-04-16T18:10:45.300 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.300 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.300 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.300 DEBUG:boto:Params: {} 2024-04-16T18:10:45.300 DEBUG:boto:Token: None 2024-04-16T18:10:45.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xsim0+mUO7G+w+Fb01cZ+1Qox5U= 2024-04-16T18:10:45.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xsim0+mUO7G+w+Fb01cZ+1Qox5U=', 'Content-Length': '0'} 2024-04-16T18:10:45.303 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dd273c5a813d34fe-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.304 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.304 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.304 DEBUG:boto:path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.304 DEBUG:boto:auth_path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.304 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.304 DEBUG:boto:Path: /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.304 DEBUG:boto:Data: 2024-04-16T18:10:45.304 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.304 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.304 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.304 DEBUG:boto:Params: {} 2024-04-16T18:10:45.304 DEBUG:boto:Token: None 2024-04-16T18:10:45.305 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.305 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xsim0+mUO7G+w+Fb01cZ+1Qox5U= 2024-04-16T18:10:45.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xsim0+mUO7G+w+Fb01cZ+1Qox5U=', 'Content-Length': '0'} 2024-04-16T18:10:45.308 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d44e9ad817f8b4b2-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.308 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:45.308 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.308 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.308 DEBUG:boto:path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.308 DEBUG:boto:auth_path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.308 DEBUG:boto:Method: GET 2024-04-16T18:10:45.308 DEBUG:boto:Path: /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.308 DEBUG:boto:Data: 2024-04-16T18:10:45.309 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.309 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.309 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.309 DEBUG:boto:Params: {} 2024-04-16T18:10:45.309 DEBUG:boto:Token: None 2024-04-16T18:10:45.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc= 2024-04-16T18:10:45.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc=', 'Content-Length': '0'} 2024-04-16T18:10:45.312 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000d0631c9ef451166-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.354 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.354 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.355 DEBUG:boto:path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.355 DEBUG:boto:auth_path=/lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.355 DEBUG:boto:Method: GET 2024-04-16T18:10:45.355 DEBUG:boto:Path: /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.355 DEBUG:boto:Data: 2024-04-16T18:10:45.355 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.355 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.355 DEBUG:boto:Params: {} 2024-04-16T18:10:45.355 DEBUG:boto:Token: None 2024-04-16T18:10:45.355 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6?versionId=null 2024-04-16T18:10:45.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc= 2024-04-16T18:10:45.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4kAxQ3Pky09tITbNaKyjVaUazNc=', 'Content-Length': '0'} 2024-04-16T18:10:45.359 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ff4c8726f54bde47-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.359 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.359 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.359 DEBUG:boto:path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.359 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.359 DEBUG:boto:Method: GET 2024-04-16T18:10:45.359 DEBUG:boto:Path: /lybtug-40/?prefix=obj6 2024-04-16T18:10:45.359 DEBUG:boto:Data: 2024-04-16T18:10:45.360 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.360 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.360 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.360 DEBUG:boto:Params: {} 2024-04-16T18:10:45.360 DEBUG:boto:Token: None 2024-04-16T18:10:45.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4f6ffd85454d6f5-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.364 DEBUG:boto:b'lybtug-40obj61000falseobj62024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.364 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.364 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.365 DEBUG:boto:path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.365 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.365 DEBUG:boto:Method: GET 2024-04-16T18:10:45.365 DEBUG:boto:Path: /lybtug-40/?prefix=obj6 2024-04-16T18:10:45.365 DEBUG:boto:Data: 2024-04-16T18:10:45.365 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.365 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.365 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.365 DEBUG:boto:Params: {} 2024-04-16T18:10:45.366 DEBUG:boto:Token: None 2024-04-16T18:10:45.366 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.366 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000165a56ce30891bc5-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.410 DEBUG:boto:b'lybtug-40obj61000falseobj62024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.411 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.411 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.411 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.411 DEBUG:boto:Path: /lybtug-40/obj6 2024-04-16T18:10:45.411 DEBUG:boto:Data: 2024-04-16T18:10:45.411 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.411 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.412 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.412 DEBUG:boto:Params: {} 2024-04-16T18:10:45.412 DEBUG:boto:Token: None 2024-04-16T18:10:45.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6 2024-04-16T18:10:45.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9e6aBxS/ijZvepmdO31GpGZKnn0= 2024-04-16T18:10:45.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9e6aBxS/ijZvepmdO31GpGZKnn0=', 'Content-Length': '0'} 2024-04-16T18:10:45.415 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000008e4b2c91998d3df-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.415 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.416 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.416 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.416 DEBUG:boto:Path: /lybtug-40/obj6 2024-04-16T18:10:45.416 DEBUG:boto:Data: 2024-04-16T18:10:45.416 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.416 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.416 DEBUG:boto:Params: {} 2024-04-16T18:10:45.416 DEBUG:boto:Token: None 2024-04-16T18:10:45.416 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6 2024-04-16T18:10:45.417 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9e6aBxS/ijZvepmdO31GpGZKnn0= 2024-04-16T18:10:45.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9e6aBxS/ijZvepmdO31GpGZKnn0=', 'Content-Length': '0'} 2024-04-16T18:10:45.419 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003b5f1491510326c1-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:45.420 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.420 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.420 DEBUG:boto:Method: GET 2024-04-16T18:10:45.420 DEBUG:boto:Path: /lybtug-40/obj6 2024-04-16T18:10:45.420 DEBUG:boto:Data: 2024-04-16T18:10:45.420 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.420 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.421 DEBUG:boto:Params: {} 2024-04-16T18:10:45.421 DEBUG:boto:Token: None 2024-04-16T18:10:45.421 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6 2024-04-16T18:10:45.421 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t5e8ACyOaATN42soL/JZtWoVoW8= 2024-04-16T18:10:45.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t5e8ACyOaATN42soL/JZtWoVoW8=', 'Content-Length': '0'} 2024-04-16T18:10:45.424 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b507403189f4a9af-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.466 DEBUG:boto:path=/lybtug-40/obj6 2024-04-16T18:10:45.466 DEBUG:boto:auth_path=/lybtug-40/obj6 2024-04-16T18:10:45.467 DEBUG:boto:Method: GET 2024-04-16T18:10:45.467 DEBUG:boto:Path: /lybtug-40/obj6 2024-04-16T18:10:45.467 DEBUG:boto:Data: 2024-04-16T18:10:45.467 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.467 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.467 DEBUG:boto:Params: {} 2024-04-16T18:10:45.467 DEBUG:boto:Token: None 2024-04-16T18:10:45.467 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj6 2024-04-16T18:10:45.467 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t5e8ACyOaATN42soL/JZtWoVoW8= 2024-04-16T18:10:45.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t5e8ACyOaATN42soL/JZtWoVoW8=', 'Content-Length': '0'} 2024-04-16T18:10:45.470 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038655e2e2c7fc216-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.471 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.471 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.471 DEBUG:boto:path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.471 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.471 DEBUG:boto:Method: GET 2024-04-16T18:10:45.471 DEBUG:boto:Path: /lybtug-40/?prefix=obj6 2024-04-16T18:10:45.471 DEBUG:boto:Data: 2024-04-16T18:10:45.471 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.472 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.472 DEBUG:boto:Params: {} 2024-04-16T18:10:45.472 DEBUG:boto:Token: None 2024-04-16T18:10:45.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.477 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003166353594c73ce-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.518 DEBUG:boto:b'lybtug-40obj61000falseobj62024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.519 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.519 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.519 DEBUG:boto:path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.519 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj6 2024-04-16T18:10:45.519 DEBUG:boto:Method: GET 2024-04-16T18:10:45.519 DEBUG:boto:Path: /lybtug-40/?prefix=obj6 2024-04-16T18:10:45.519 DEBUG:boto:Data: 2024-04-16T18:10:45.519 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.520 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.520 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.520 DEBUG:boto:Params: {} 2024-04-16T18:10:45.520 DEBUG:boto:Token: None 2024-04-16T18:10:45.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adb4ec12cb1cb5dc-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.524 DEBUG:boto:b'lybtug-40obj61000falseobj62024-04-16T18:10:37.947Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.524 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:45.524 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.525 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.525 DEBUG:boto:path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.525 DEBUG:boto:auth_path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.525 DEBUG:boto:Method: GET 2024-04-16T18:10:45.525 DEBUG:boto:Path: /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.525 DEBUG:boto:Data: 2024-04-16T18:10:45.525 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.525 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.525 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.525 DEBUG:boto:Params: {} 2024-04-16T18:10:45.525 DEBUG:boto:Token: None 2024-04-16T18:10:45.526 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.526 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg= 2024-04-16T18:10:45.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg=', 'Content-Length': '0'} 2024-04-16T18:10:45.529 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f638a52328e4b575-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.570 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.570 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.571 DEBUG:boto:path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.571 DEBUG:boto:auth_path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.571 DEBUG:boto:Method: GET 2024-04-16T18:10:45.571 DEBUG:boto:Path: /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.571 DEBUG:boto:Data: 2024-04-16T18:10:45.571 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.571 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.571 DEBUG:boto:Params: {} 2024-04-16T18:10:45.571 DEBUG:boto:Token: None 2024-04-16T18:10:45.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg= 2024-04-16T18:10:45.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg=', 'Content-Length': '0'} 2024-04-16T18:10:45.575 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000071e8d7d413c19c5d-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.575 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.576 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.576 DEBUG:boto:path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.576 DEBUG:boto:auth_path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.576 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.576 DEBUG:boto:Path: /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.576 DEBUG:boto:Data: 2024-04-16T18:10:45.576 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.576 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.576 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.576 DEBUG:boto:Params: {} 2024-04-16T18:10:45.576 DEBUG:boto:Token: None 2024-04-16T18:10:45.577 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFJMa2BywhE/RZ63lznyoHtLEs= 2024-04-16T18:10:45.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFJMa2BywhE/RZ63lznyoHtLEs=', 'Content-Length': '0'} 2024-04-16T18:10:45.579 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000081581708f742522b-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.580 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.580 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.580 DEBUG:boto:path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.580 DEBUG:boto:auth_path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.580 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.580 DEBUG:boto:Path: /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.580 DEBUG:boto:Data: 2024-04-16T18:10:45.580 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.580 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.580 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.580 DEBUG:boto:Params: {} 2024-04-16T18:10:45.581 DEBUG:boto:Token: None 2024-04-16T18:10:45.581 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFJMa2BywhE/RZ63lznyoHtLEs= 2024-04-16T18:10:45.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFJMa2BywhE/RZ63lznyoHtLEs=', 'Content-Length': '0'} 2024-04-16T18:10:45.584 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000012179aa54ef5b88c-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:45.584 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.584 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.584 DEBUG:boto:path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.584 DEBUG:boto:auth_path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.584 DEBUG:boto:Method: GET 2024-04-16T18:10:45.585 DEBUG:boto:Path: /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.585 DEBUG:boto:Data: 2024-04-16T18:10:45.585 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.585 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.585 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.585 DEBUG:boto:Params: {} 2024-04-16T18:10:45.585 DEBUG:boto:Token: None 2024-04-16T18:10:45.585 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.585 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg= 2024-04-16T18:10:45.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg=', 'Content-Length': '0'} 2024-04-16T18:10:45.588 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009bc59ca5264a63aa-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.630 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.630 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.630 DEBUG:boto:path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.631 DEBUG:boto:auth_path=/lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.631 DEBUG:boto:Method: GET 2024-04-16T18:10:45.631 DEBUG:boto:Path: /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.631 DEBUG:boto:Data: 2024-04-16T18:10:45.631 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.631 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.631 DEBUG:boto:Params: {} 2024-04-16T18:10:45.631 DEBUG:boto:Token: None 2024-04-16T18:10:45.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7?versionId=null 2024-04-16T18:10:45.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg= 2024-04-16T18:10:45.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0JM50zP8TIH46Y84UWQm8W8XjIg=', 'Content-Length': '0'} 2024-04-16T18:10:45.634 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009f07b24f687aa825-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.635 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.635 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.635 DEBUG:boto:path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.635 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.635 DEBUG:boto:Method: GET 2024-04-16T18:10:45.635 DEBUG:boto:Path: /lybtug-40/?prefix=obj7 2024-04-16T18:10:45.635 DEBUG:boto:Data: 2024-04-16T18:10:45.635 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.635 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.635 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.635 DEBUG:boto:Params: {} 2024-04-16T18:10:45.636 DEBUG:boto:Token: None 2024-04-16T18:10:45.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e15631a23e53f819-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.640 DEBUG:boto:b'lybtug-40obj71000falseobj72024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.640 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.641 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.641 DEBUG:boto:path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.641 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.641 DEBUG:boto:Method: GET 2024-04-16T18:10:45.641 DEBUG:boto:Path: /lybtug-40/?prefix=obj7 2024-04-16T18:10:45.641 DEBUG:boto:Data: 2024-04-16T18:10:45.641 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.641 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.641 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.641 DEBUG:boto:Params: {} 2024-04-16T18:10:45.641 DEBUG:boto:Token: None 2024-04-16T18:10:45.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.642 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.645 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007bcd4d9dd811cf4-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.686 DEBUG:boto:b'lybtug-40obj71000falseobj72024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.687 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.687 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.687 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.687 DEBUG:boto:Path: /lybtug-40/obj7 2024-04-16T18:10:45.687 DEBUG:boto:Data: 2024-04-16T18:10:45.687 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.687 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.687 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.687 DEBUG:boto:Params: {} 2024-04-16T18:10:45.688 DEBUG:boto:Token: None 2024-04-16T18:10:45.688 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7 2024-04-16T18:10:45.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uV2ih6KJScx3H0CEFCLUKMIi1M0= 2024-04-16T18:10:45.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uV2ih6KJScx3H0CEFCLUKMIi1M0=', 'Content-Length': '0'} 2024-04-16T18:10:45.691 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e7e61a6f480e3116-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.691 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.691 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.691 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.691 DEBUG:boto:Path: /lybtug-40/obj7 2024-04-16T18:10:45.691 DEBUG:boto:Data: 2024-04-16T18:10:45.691 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.692 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.692 DEBUG:boto:Params: {} 2024-04-16T18:10:45.692 DEBUG:boto:Token: None 2024-04-16T18:10:45.692 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7 2024-04-16T18:10:45.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uV2ih6KJScx3H0CEFCLUKMIi1M0= 2024-04-16T18:10:45.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uV2ih6KJScx3H0CEFCLUKMIi1M0=', 'Content-Length': '0'} 2024-04-16T18:10:45.695 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f2df8ba59917d999-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.695 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:45.695 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.695 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.696 DEBUG:boto:Method: GET 2024-04-16T18:10:45.696 DEBUG:boto:Path: /lybtug-40/obj7 2024-04-16T18:10:45.696 DEBUG:boto:Data: 2024-04-16T18:10:45.696 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.696 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.696 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.696 DEBUG:boto:Params: {} 2024-04-16T18:10:45.696 DEBUG:boto:Token: None 2024-04-16T18:10:45.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7 2024-04-16T18:10:45.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kE6JEKkshJf509lBKA1Jve92800= 2024-04-16T18:10:45.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kE6JEKkshJf509lBKA1Jve92800=', 'Content-Length': '0'} 2024-04-16T18:10:45.699 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004d8506457aaab6b3-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.742 DEBUG:boto:path=/lybtug-40/obj7 2024-04-16T18:10:45.743 DEBUG:boto:auth_path=/lybtug-40/obj7 2024-04-16T18:10:45.743 DEBUG:boto:Method: GET 2024-04-16T18:10:45.743 DEBUG:boto:Path: /lybtug-40/obj7 2024-04-16T18:10:45.743 DEBUG:boto:Data: 2024-04-16T18:10:45.743 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.743 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.743 DEBUG:boto:Params: {} 2024-04-16T18:10:45.743 DEBUG:boto:Token: None 2024-04-16T18:10:45.743 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj7 2024-04-16T18:10:45.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kE6JEKkshJf509lBKA1Jve92800= 2024-04-16T18:10:45.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kE6JEKkshJf509lBKA1Jve92800=', 'Content-Length': '0'} 2024-04-16T18:10:45.746 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b9bde45adcc15f04-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.747 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.747 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.747 DEBUG:boto:path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.747 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.747 DEBUG:boto:Method: GET 2024-04-16T18:10:45.747 DEBUG:boto:Path: /lybtug-40/?prefix=obj7 2024-04-16T18:10:45.747 DEBUG:boto:Data: 2024-04-16T18:10:45.747 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.748 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.748 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.748 DEBUG:boto:Params: {} 2024-04-16T18:10:45.748 DEBUG:boto:Token: None 2024-04-16T18:10:45.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9695e8d63db7ef5-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.794 DEBUG:boto:b'lybtug-40obj71000falseobj72024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.795 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.795 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.795 DEBUG:boto:path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.795 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj7 2024-04-16T18:10:45.795 DEBUG:boto:Method: GET 2024-04-16T18:10:45.795 DEBUG:boto:Path: /lybtug-40/?prefix=obj7 2024-04-16T18:10:45.795 DEBUG:boto:Data: 2024-04-16T18:10:45.795 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.795 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.795 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.796 DEBUG:boto:Params: {} 2024-04-16T18:10:45.796 DEBUG:boto:Token: None 2024-04-16T18:10:45.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9bf65d3027229e4-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.800 DEBUG:boto:b'lybtug-40obj71000falseobj72024-04-16T18:10:37.961Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:45.800 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.800 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.800 DEBUG:boto:path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.800 DEBUG:boto:auth_path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.801 DEBUG:boto:Method: GET 2024-04-16T18:10:45.801 DEBUG:boto:Path: /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.801 DEBUG:boto:Data: 2024-04-16T18:10:45.801 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.801 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.801 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.801 DEBUG:boto:Params: {} 2024-04-16T18:10:45.801 DEBUG:boto:Token: None 2024-04-16T18:10:45.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y= 2024-04-16T18:10:45.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y=', 'Content-Length': '0'} 2024-04-16T18:10:45.804 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ce13868d0f826fa0-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.846 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.846 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.846 DEBUG:boto:path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.847 DEBUG:boto:auth_path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.847 DEBUG:boto:Method: GET 2024-04-16T18:10:45.847 DEBUG:boto:Path: /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.847 DEBUG:boto:Data: 2024-04-16T18:10:45.847 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.847 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.847 DEBUG:boto:Params: {} 2024-04-16T18:10:45.847 DEBUG:boto:Token: None 2024-04-16T18:10:45.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y= 2024-04-16T18:10:45.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y=', 'Content-Length': '0'} 2024-04-16T18:10:45.850 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009df0410c98bee24e-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.851 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.851 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.851 DEBUG:boto:path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.851 DEBUG:boto:auth_path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.851 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.851 DEBUG:boto:Path: /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.851 DEBUG:boto:Data: 2024-04-16T18:10:45.851 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.851 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.851 DEBUG:boto:Params: {} 2024-04-16T18:10:45.851 DEBUG:boto:Token: None 2024-04-16T18:10:45.852 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pfmZQPki8i2kS+Eb+4Htq257AqA= 2024-04-16T18:10:45.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pfmZQPki8i2kS+Eb+4Htq257AqA=', 'Content-Length': '0'} 2024-04-16T18:10:45.855 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000769d097b25407076-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.855 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.855 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.855 DEBUG:boto:path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.855 DEBUG:boto:auth_path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.855 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.856 DEBUG:boto:Path: /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.856 DEBUG:boto:Data: 2024-04-16T18:10:45.856 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.856 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.856 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.856 DEBUG:boto:Params: {} 2024-04-16T18:10:45.856 DEBUG:boto:Token: None 2024-04-16T18:10:45.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pfmZQPki8i2kS+Eb+4Htq257AqA= 2024-04-16T18:10:45.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pfmZQPki8i2kS+Eb+4Htq257AqA=', 'Content-Length': '0'} 2024-04-16T18:10:45.859 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c605265a60207f9e-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.859 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:45.859 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.859 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.859 DEBUG:boto:path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.859 DEBUG:boto:auth_path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.860 DEBUG:boto:Method: GET 2024-04-16T18:10:45.860 DEBUG:boto:Path: /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.860 DEBUG:boto:Data: 2024-04-16T18:10:45.860 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.860 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.860 DEBUG:boto:Params: {} 2024-04-16T18:10:45.860 DEBUG:boto:Token: None 2024-04-16T18:10:45.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y= 2024-04-16T18:10:45.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y=', 'Content-Length': '0'} 2024-04-16T18:10:45.864 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000036fd8642a0fd926c-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.906 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.906 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.906 DEBUG:boto:path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.907 DEBUG:boto:auth_path=/lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.907 DEBUG:boto:Method: GET 2024-04-16T18:10:45.907 DEBUG:boto:Path: /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.907 DEBUG:boto:Data: 2024-04-16T18:10:45.907 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.907 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.907 DEBUG:boto:Params: {} 2024-04-16T18:10:45.907 DEBUG:boto:Token: None 2024-04-16T18:10:45.907 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8?versionId=null 2024-04-16T18:10:45.907 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y= 2024-04-16T18:10:45.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ajK1eulUXOZn/40LB+FULdmAE8Y=', 'Content-Length': '0'} 2024-04-16T18:10:45.910 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004bf22bd6e07105a5-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.910 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.911 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.911 DEBUG:boto:path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:45.911 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:45.911 DEBUG:boto:Method: GET 2024-04-16T18:10:45.911 DEBUG:boto:Path: /lybtug-40/?prefix=obj8 2024-04-16T18:10:45.911 DEBUG:boto:Data: 2024-04-16T18:10:45.911 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.911 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.911 DEBUG:boto:Params: {} 2024-04-16T18:10:45.911 DEBUG:boto:Token: None 2024-04-16T18:10:45.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da8d52e07e46470f-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.916 DEBUG:boto:b'lybtug-40obj81000falseobj82024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.916 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:45.917 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:45.917 DEBUG:boto:path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:45.917 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:45.917 DEBUG:boto:Method: GET 2024-04-16T18:10:45.917 DEBUG:boto:Path: /lybtug-40/?prefix=obj8 2024-04-16T18:10:45.917 DEBUG:boto:Data: 2024-04-16T18:10:45.917 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.917 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.917 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.917 DEBUG:boto:Params: {} 2024-04-16T18:10:45.917 DEBUG:boto:Token: None 2024-04-16T18:10:45.917 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/ 2024-04-16T18:10:45.918 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4= 2024-04-16T18:10:45.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1N7AWts4swHTsWoWju0g8rASz4=', 'Content-Length': '0'} 2024-04-16T18:10:45.921 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f8b4cbc26896877-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.966 DEBUG:boto:b'lybtug-40obj81000falseobj82024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:45.967 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.967 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.967 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.967 DEBUG:boto:Path: /lybtug-40/obj8 2024-04-16T18:10:45.967 DEBUG:boto:Data: 2024-04-16T18:10:45.967 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.967 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.968 DEBUG:boto:Params: {} 2024-04-16T18:10:45.968 DEBUG:boto:Token: None 2024-04-16T18:10:45.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8 2024-04-16T18:10:45.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AH5hQY18V0Q1WPze7oHrAtZZBXU= 2024-04-16T18:10:45.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AH5hQY18V0Q1WPze7oHrAtZZBXU=', 'Content-Length': '0'} 2024-04-16T18:10:45.971 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a3879b8c29ce4f65-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.971 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.971 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.972 DEBUG:boto:Method: HEAD 2024-04-16T18:10:45.972 DEBUG:boto:Path: /lybtug-40/obj8 2024-04-16T18:10:45.972 DEBUG:boto:Data: 2024-04-16T18:10:45.972 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.972 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:45.972 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.972 DEBUG:boto:Params: {} 2024-04-16T18:10:45.972 DEBUG:boto:Token: None 2024-04-16T18:10:45.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8 2024-04-16T18:10:45.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AH5hQY18V0Q1WPze7oHrAtZZBXU= 2024-04-16T18:10:45.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AH5hQY18V0Q1WPze7oHrAtZZBXU=', 'Content-Length': '0'} 2024-04-16T18:10:45.975 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000094d61c60966bf99a-00661ebf25-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:45.976 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:45.976 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:45.976 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:45.976 DEBUG:boto:Method: GET 2024-04-16T18:10:45.976 DEBUG:boto:Path: /lybtug-40/obj8 2024-04-16T18:10:45.976 DEBUG:boto:Data: 2024-04-16T18:10:45.976 DEBUG:boto:Headers: {} 2024-04-16T18:10:45.976 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:45.976 DEBUG:boto:Port: 8000 2024-04-16T18:10:45.977 DEBUG:boto:Params: {} 2024-04-16T18:10:45.977 DEBUG:boto:Token: None 2024-04-16T18:10:45.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:45 GMT /lybtug-40/obj8 2024-04-16T18:10:45.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0ifjNztoiya5bvIHVSYyOWRmxXk= 2024-04-16T18:10:45.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0ifjNztoiya5bvIHVSYyOWRmxXk=', 'Content-Length': '0'} 2024-04-16T18:10:45.980 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-40:054102a3-8d94-4596-8b3b-6341eab312e1.4804.9'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c6a6440880f511f-00661ebf25-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.022 DEBUG:boto:path=/lybtug-40/obj8 2024-04-16T18:10:46.022 DEBUG:boto:auth_path=/lybtug-40/obj8 2024-04-16T18:10:46.023 DEBUG:boto:Method: GET 2024-04-16T18:10:46.023 DEBUG:boto:Path: /lybtug-40/obj8 2024-04-16T18:10:46.023 DEBUG:boto:Data: 2024-04-16T18:10:46.023 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.023 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.023 DEBUG:boto:Params: {} 2024-04-16T18:10:46.023 DEBUG:boto:Token: None 2024-04-16T18:10:46.023 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-40/obj8 2024-04-16T18:10:46.023 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lYsbvORmKJ0B8wSvCSfH/KohBrk= 2024-04-16T18:10:46.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lYsbvORmKJ0B8wSvCSfH/KohBrk=', 'Content-Length': '0'} 2024-04-16T18:10:46.026 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000045ffa9548f8e7e52-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.027 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:46.027 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:46.027 DEBUG:boto:path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:46.027 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:46.027 DEBUG:boto:Method: GET 2024-04-16T18:10:46.027 DEBUG:boto:Path: /lybtug-40/?prefix=obj8 2024-04-16T18:10:46.027 DEBUG:boto:Data: 2024-04-16T18:10:46.027 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.028 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.028 DEBUG:boto:Params: {} 2024-04-16T18:10:46.028 DEBUG:boto:Token: None 2024-04-16T18:10:46.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-40/ 2024-04-16T18:10:46.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DM732gKJaTPQxigl+0gNE1tyoOc= 2024-04-16T18:10:46.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DM732gKJaTPQxigl+0gNE1tyoOc=', 'Content-Length': '0'} 2024-04-16T18:10:46.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000742a97535fc23ea-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.074 DEBUG:boto:b'lybtug-40obj81000falseobj82024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.075 DEBUG:boto:path=/lybtug-40/ 2024-04-16T18:10:46.075 DEBUG:boto:auth_path=/lybtug-40/ 2024-04-16T18:10:46.075 DEBUG:boto:path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:46.075 DEBUG:boto:auth_path=/lybtug-40/?prefix=obj8 2024-04-16T18:10:46.075 DEBUG:boto:Method: GET 2024-04-16T18:10:46.075 DEBUG:boto:Path: /lybtug-40/?prefix=obj8 2024-04-16T18:10:46.075 DEBUG:boto:Data: 2024-04-16T18:10:46.075 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.075 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.076 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.076 DEBUG:boto:Params: {} 2024-04-16T18:10:46.076 DEBUG:boto:Token: None 2024-04-16T18:10:46.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-40/ 2024-04-16T18:10:46.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DM732gKJaTPQxigl+0gNE1tyoOc= 2024-04-16T18:10:46.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DM732gKJaTPQxigl+0gNE1tyoOc=', 'Content-Length': '0'} 2024-04-16T18:10:46.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091328bc05622479f-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.080 DEBUG:boto:b'lybtug-40obj81000falseobj82024-04-16T18:10:38.012Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.080 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-40 zones={test-zone2, test-zone1} 2024-04-16T18:10:46.081 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-41 2024-04-16T18:10:46.081 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:46.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:46.081 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-41 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:46.132 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:46.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:46.219+0000 7f3211a05980 1 bucket sync caught up with source: 2024-04-16T18:10:46.215 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000004.16.6, 00000000002.57.1, 00000000004.37.6, 00000000002.48.1, 00000000004.42.6, 00000000002.24.1, 00000000002.43.1, 00000000004.33.6, 00000000002.39.1, 00000000002.30.1, 00000000002.28.1] 2024-04-16T18:10:46.216 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000004.16.6, 00000000002.57.1, 00000000004.37.6, 00000000002.48.1, 00000000004.42.6, 00000000002.24.1, 00000000002.43.1, 00000000004.33.6, 00000000002.39.1, 00000000002.30.1, 00000000002.28.1] 2024-04-16T18:10:46.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:46.219+0000 7f3211a05980 0 bucket checkpoint complete 2024-04-16T18:10:46.246 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-41 2024-04-16T18:10:46.246 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:46.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-41', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:46.246 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-41 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:46.296 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:46.296 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:46.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:46.375+0000 7f6155d33980 1 bucket sync caught up with source: 2024-04-16T18:10:46.376 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000004.575.1, 00000000002.384.1, 00000000004.396.1, 00000000002.546.1, 00000000004.108.1, 00000000002.919.1, 00000000002.885.1, 00000000004.609.1, 00000000002.297.1, 00000000002.710.1, 00000000002.1327.1] 2024-04-16T18:10:46.376 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000004.575.1, 00000000002.384.1, 00000000004.396.1, 00000000002.546.1, 00000000004.108.1, 00000000002.919.1, 00000000002.885.1, 00000000004.609.1, 00000000002.297.1, 00000000002.710.1, 00000000002.1327.1] 2024-04-16T18:10:46.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:46.375+0000 7f6155d33980 0 bucket checkpoint complete 2024-04-16T18:10:46.406 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-41 zones={test-zone2, test-zone1} 2024-04-16T18:10:46.407 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.407 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.407 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.407 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:46.407 DEBUG:boto:Data: 2024-04-16T18:10:46.407 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.407 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.407 DEBUG:boto:Params: {} 2024-04-16T18:10:46.408 DEBUG:boto:Token: None 2024-04-16T18:10:46.408 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vDTNRqBYRFpnwRfpClPLEIyNmqY= 2024-04-16T18:10:46.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vDTNRqBYRFpnwRfpClPLEIyNmqY=', 'Content-Length': '0'} 2024-04-16T18:10:46.412 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001a32cfcf93079d00-00661ebf26-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.412 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.412 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.412 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.412 DEBUG:boto:Path: /lybtug-41/ 2024-04-16T18:10:46.412 DEBUG:boto:Data: 2024-04-16T18:10:46.412 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.413 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.413 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.413 DEBUG:boto:Params: {} 2024-04-16T18:10:46.413 DEBUG:boto:Token: None 2024-04-16T18:10:46.413 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.413 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vDTNRqBYRFpnwRfpClPLEIyNmqY= 2024-04-16T18:10:46.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vDTNRqBYRFpnwRfpClPLEIyNmqY=', 'Content-Length': '0'} 2024-04-16T18:10:46.416 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f66d20801330b3e7-00661ebf26-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.416 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:10:46.416 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.416 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.416 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.417 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.417 DEBUG:boto:Method: GET 2024-04-16T18:10:46.417 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.417 DEBUG:boto:Data: 2024-04-16T18:10:46.417 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.417 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.417 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.417 DEBUG:boto:Params: {} 2024-04-16T18:10:46.417 DEBUG:boto:Token: None 2024-04-16T18:10:46.417 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/?versions 2024-04-16T18:10:46.418 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4= 2024-04-16T18:10:46.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4=', 'Content-Length': '0'} 2024-04-16T18:10:46.421 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c650c44c27cefde0-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.421 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.422 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:46.422 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:46.422 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:46.422 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:46.423 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:10:46.423 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:10:46.423 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:10:46.423 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:10:46.423 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:10:46.423 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.423 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.423 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.423 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.423 DEBUG:boto:Method: GET 2024-04-16T18:10:46.423 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.424 DEBUG:boto:Data: 2024-04-16T18:10:46.424 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.424 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.424 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.424 DEBUG:boto:Params: {} 2024-04-16T18:10:46.424 DEBUG:boto:Token: None 2024-04-16T18:10:46.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/?versions 2024-04-16T18:10:46.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4= 2024-04-16T18:10:46.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4=', 'Content-Length': '0'} 2024-04-16T18:10:46.427 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007dd5baf63e75f812-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.427 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2024-04-16T18:10:46.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2024-04-16T18:10:46.429 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.429 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.429 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.429 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.429 DEBUG:boto:Method: GET 2024-04-16T18:10:46.429 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.429 DEBUG:boto:Data: 2024-04-16T18:10:46.429 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.429 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.429 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.430 DEBUG:boto:Params: {} 2024-04-16T18:10:46.430 DEBUG:boto:Token: None 2024-04-16T18:10:46.430 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/?versions 2024-04-16T18:10:46.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4= 2024-04-16T18:10:46.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4=', 'Content-Length': '0'} 2024-04-16T18:10:46.434 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021c02255131c117c-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.434 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.434 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.435 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.435 DEBUG:boto:path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.435 DEBUG:boto:auth_path=/lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.435 DEBUG:boto:Method: GET 2024-04-16T18:10:46.435 DEBUG:boto:Path: /lybtug-41/?versions&max-keys=999 2024-04-16T18:10:46.435 DEBUG:boto:Data: 2024-04-16T18:10:46.435 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.435 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.435 DEBUG:boto:Params: {} 2024-04-16T18:10:46.435 DEBUG:boto:Token: None 2024-04-16T18:10:46.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/?versions 2024-04-16T18:10:46.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4= 2024-04-16T18:10:46.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hCQ09PlIt7fLzUS662krqeMQ/I4=', 'Content-Length': '0'} 2024-04-16T18:10:46.438 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7dce28e4dcb848a-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.439 DEBUG:boto:b'lybtug-41999falseobj1nulltrue2024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj2nulltrue2024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj3nulltrue2024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj4nulltrue2024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj5nulltrue2024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj6nulltrue2024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj7nulltrue2024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normalobj8nulltrue2024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.439 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:46.439 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.439 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.439 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.440 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.440 DEBUG:boto:Method: GET 2024-04-16T18:10:46.440 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.440 DEBUG:boto:Data: 2024-04-16T18:10:46.440 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.440 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.440 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.440 DEBUG:boto:Params: {} 2024-04-16T18:10:46.440 DEBUG:boto:Token: None 2024-04-16T18:10:46.440 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw= 2024-04-16T18:10:46.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw=', 'Content-Length': '0'} 2024-04-16T18:10:46.443 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038600e06f2f26025-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.443 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.444 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.444 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.444 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.444 DEBUG:boto:Method: GET 2024-04-16T18:10:46.444 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.444 DEBUG:boto:Data: 2024-04-16T18:10:46.444 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.444 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.444 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.444 DEBUG:boto:Params: {} 2024-04-16T18:10:46.444 DEBUG:boto:Token: None 2024-04-16T18:10:46.445 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.445 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw= 2024-04-16T18:10:46.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw=', 'Content-Length': '0'} 2024-04-16T18:10:46.447 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b55328e3f7f94516-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.490 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.490 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.490 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.491 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.491 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.491 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.491 DEBUG:boto:Data: 2024-04-16T18:10:46.491 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.491 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.491 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.491 DEBUG:boto:Params: {} 2024-04-16T18:10:46.491 DEBUG:boto:Token: None 2024-04-16T18:10:46.491 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.491 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6a78h5OO9Iv8SO7RtYKzp7dXDPg= 2024-04-16T18:10:46.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6a78h5OO9Iv8SO7RtYKzp7dXDPg=', 'Content-Length': '0'} 2024-04-16T18:10:46.494 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c8e89c979b6228dc-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.495 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.495 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.495 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.495 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.495 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.495 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.495 DEBUG:boto:Data: 2024-04-16T18:10:46.495 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.495 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.495 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.496 DEBUG:boto:Params: {} 2024-04-16T18:10:46.496 DEBUG:boto:Token: None 2024-04-16T18:10:46.496 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6a78h5OO9Iv8SO7RtYKzp7dXDPg= 2024-04-16T18:10:46.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6a78h5OO9Iv8SO7RtYKzp7dXDPg=', 'Content-Length': '0'} 2024-04-16T18:10:46.499 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001d80b7e4e54fb71b-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:46.499 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.499 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.499 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.499 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.499 DEBUG:boto:Method: GET 2024-04-16T18:10:46.500 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.500 DEBUG:boto:Data: 2024-04-16T18:10:46.500 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.500 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.500 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.500 DEBUG:boto:Params: {} 2024-04-16T18:10:46.500 DEBUG:boto:Token: None 2024-04-16T18:10:46.500 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw= 2024-04-16T18:10:46.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw=', 'Content-Length': '0'} 2024-04-16T18:10:46.503 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d290842dd16d27c5-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.546 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.546 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.547 DEBUG:boto:path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.547 DEBUG:boto:auth_path=/lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.547 DEBUG:boto:Method: GET 2024-04-16T18:10:46.547 DEBUG:boto:Path: /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.547 DEBUG:boto:Data: 2024-04-16T18:10:46.548 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.548 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.548 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.548 DEBUG:boto:Params: {} 2024-04-16T18:10:46.548 DEBUG:boto:Token: None 2024-04-16T18:10:46.549 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1?versionId=null 2024-04-16T18:10:46.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw= 2024-04-16T18:10:46.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CY/k1OKRvROlrEvxZSNA59ngKvw=', 'Content-Length': '0'} 2024-04-16T18:10:46.552 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dc08c0b29526beee-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.552 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.552 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.552 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.553 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.553 DEBUG:boto:Method: GET 2024-04-16T18:10:46.553 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:46.553 DEBUG:boto:Data: 2024-04-16T18:10:46.553 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.553 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.553 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.553 DEBUG:boto:Params: {} 2024-04-16T18:10:46.553 DEBUG:boto:Token: None 2024-04-16T18:10:46.553 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.557 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e44779708fddb76-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.557 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.558 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.558 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.558 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.558 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.558 DEBUG:boto:Method: GET 2024-04-16T18:10:46.558 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:46.558 DEBUG:boto:Data: 2024-04-16T18:10:46.559 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.559 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.559 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.559 DEBUG:boto:Params: {} 2024-04-16T18:10:46.559 DEBUG:boto:Token: None 2024-04-16T18:10:46.559 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.559 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.563 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c2b6ca51d27dff3-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.563 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.563 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.563 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.563 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.563 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:46.564 DEBUG:boto:Data: 2024-04-16T18:10:46.564 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.564 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.564 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.564 DEBUG:boto:Params: {} 2024-04-16T18:10:46.564 DEBUG:boto:Token: None 2024-04-16T18:10:46.564 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1 2024-04-16T18:10:46.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qmIzsgrAqQbDZBZkjkCyrj3MNo= 2024-04-16T18:10:46.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qmIzsgrAqQbDZBZkjkCyrj3MNo=', 'Content-Length': '0'} 2024-04-16T18:10:46.567 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000051ca020903708c17-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.567 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.567 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.567 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.567 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:46.568 DEBUG:boto:Data: 2024-04-16T18:10:46.568 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.568 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.568 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.568 DEBUG:boto:Params: {} 2024-04-16T18:10:46.568 DEBUG:boto:Token: None 2024-04-16T18:10:46.568 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1 2024-04-16T18:10:46.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qmIzsgrAqQbDZBZkjkCyrj3MNo= 2024-04-16T18:10:46.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qmIzsgrAqQbDZBZkjkCyrj3MNo=', 'Content-Length': '0'} 2024-04-16T18:10:46.571 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ba1df0191e873ff8-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.571 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2024-04-16T18:10:46.571 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.571 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.571 DEBUG:boto:Method: GET 2024-04-16T18:10:46.572 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:46.572 DEBUG:boto:Data: 2024-04-16T18:10:46.572 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.572 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.572 DEBUG:boto:Params: {} 2024-04-16T18:10:46.572 DEBUG:boto:Token: None 2024-04-16T18:10:46.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1 2024-04-16T18:10:46.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PwRp/+oEIADAf1/O64HhEXvmMg4= 2024-04-16T18:10:46.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PwRp/+oEIADAf1/O64HhEXvmMg4=', 'Content-Length': '0'} 2024-04-16T18:10:46.575 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006605845d50558ac5-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.618 DEBUG:boto:path=/lybtug-41/obj1 2024-04-16T18:10:46.618 DEBUG:boto:auth_path=/lybtug-41/obj1 2024-04-16T18:10:46.619 DEBUG:boto:Method: GET 2024-04-16T18:10:46.619 DEBUG:boto:Path: /lybtug-41/obj1 2024-04-16T18:10:46.619 DEBUG:boto:Data: 2024-04-16T18:10:46.619 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.619 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.619 DEBUG:boto:Params: {} 2024-04-16T18:10:46.619 DEBUG:boto:Token: None 2024-04-16T18:10:46.619 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj1 2024-04-16T18:10:46.619 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PwRp/+oEIADAf1/O64HhEXvmMg4= 2024-04-16T18:10:46.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PwRp/+oEIADAf1/O64HhEXvmMg4=', 'Content-Length': '0'} 2024-04-16T18:10:46.622 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002854fe21a69d46d9-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.666 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.666 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.667 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.667 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.667 DEBUG:boto:Method: GET 2024-04-16T18:10:46.667 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:46.667 DEBUG:boto:Data: 2024-04-16T18:10:46.667 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.667 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.667 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.667 DEBUG:boto:Params: {} 2024-04-16T18:10:46.667 DEBUG:boto:Token: None 2024-04-16T18:10:46.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001841ba33254887eb-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.672 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.673 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.673 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.673 DEBUG:boto:path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.673 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj1 2024-04-16T18:10:46.673 DEBUG:boto:Method: GET 2024-04-16T18:10:46.673 DEBUG:boto:Path: /lybtug-41/?prefix=obj1 2024-04-16T18:10:46.673 DEBUG:boto:Data: 2024-04-16T18:10:46.673 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.673 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.674 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.674 DEBUG:boto:Params: {} 2024-04-16T18:10:46.674 DEBUG:boto:Token: None 2024-04-16T18:10:46.674 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.674 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.678 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083ed14747401da0a-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.678 DEBUG:boto:b'lybtug-41obj11000falseobj12024-04-16T18:10:23.852Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.678 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:46.678 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.678 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.679 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.679 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.679 DEBUG:boto:Method: GET 2024-04-16T18:10:46.679 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.679 DEBUG:boto:Data: 2024-04-16T18:10:46.679 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.679 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.679 DEBUG:boto:Params: {} 2024-04-16T18:10:46.679 DEBUG:boto:Token: None 2024-04-16T18:10:46.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI= 2024-04-16T18:10:46.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI=', 'Content-Length': '0'} 2024-04-16T18:10:46.682 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002739605db2e65f0e-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.683 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.683 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.683 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.683 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.683 DEBUG:boto:Method: GET 2024-04-16T18:10:46.683 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.683 DEBUG:boto:Data: 2024-04-16T18:10:46.683 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.683 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.683 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.684 DEBUG:boto:Params: {} 2024-04-16T18:10:46.684 DEBUG:boto:Token: None 2024-04-16T18:10:46.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI= 2024-04-16T18:10:46.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI=', 'Content-Length': '0'} 2024-04-16T18:10:46.687 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007e891c16bd03adaf-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.730 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.730 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.731 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.731 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.731 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.731 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.731 DEBUG:boto:Data: 2024-04-16T18:10:46.731 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.731 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.731 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.731 DEBUG:boto:Params: {} 2024-04-16T18:10:46.731 DEBUG:boto:Token: None 2024-04-16T18:10:46.732 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LwsBuLz0XAWcBaWCMlcxQfElBV8= 2024-04-16T18:10:46.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LwsBuLz0XAWcBaWCMlcxQfElBV8=', 'Content-Length': '0'} 2024-04-16T18:10:46.734 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000049d2a5cca1bce4ee-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.735 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.735 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.735 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.735 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.735 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.735 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.735 DEBUG:boto:Data: 2024-04-16T18:10:46.735 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.735 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.735 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.735 DEBUG:boto:Params: {} 2024-04-16T18:10:46.735 DEBUG:boto:Token: None 2024-04-16T18:10:46.736 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LwsBuLz0XAWcBaWCMlcxQfElBV8= 2024-04-16T18:10:46.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LwsBuLz0XAWcBaWCMlcxQfElBV8=', 'Content-Length': '0'} 2024-04-16T18:10:46.739 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000175230994ca0f271-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.739 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:46.739 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.739 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.739 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.739 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.739 DEBUG:boto:Method: GET 2024-04-16T18:10:46.739 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.739 DEBUG:boto:Data: 2024-04-16T18:10:46.740 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.740 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.740 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.740 DEBUG:boto:Params: {} 2024-04-16T18:10:46.740 DEBUG:boto:Token: None 2024-04-16T18:10:46.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI= 2024-04-16T18:10:46.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI=', 'Content-Length': '0'} 2024-04-16T18:10:46.743 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000685d1d1bd0d12fbd-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.743 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.743 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.744 DEBUG:boto:path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.744 DEBUG:boto:auth_path=/lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.744 DEBUG:boto:Method: GET 2024-04-16T18:10:46.744 DEBUG:boto:Path: /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.744 DEBUG:boto:Data: 2024-04-16T18:10:46.744 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.744 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.744 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.744 DEBUG:boto:Params: {} 2024-04-16T18:10:46.744 DEBUG:boto:Token: None 2024-04-16T18:10:46.744 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2?versionId=null 2024-04-16T18:10:46.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI= 2024-04-16T18:10:46.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nl5dIOmVgWhqB1pZPri55NybDYI=', 'Content-Length': '0'} 2024-04-16T18:10:46.747 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f9fd7ad9f926e393-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.790 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.791 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.791 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.791 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.791 DEBUG:boto:Method: GET 2024-04-16T18:10:46.791 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:46.791 DEBUG:boto:Data: 2024-04-16T18:10:46.791 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.791 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.791 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.791 DEBUG:boto:Params: {} 2024-04-16T18:10:46.791 DEBUG:boto:Token: None 2024-04-16T18:10:46.792 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000770ca24a60b4cb64-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.796 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.797 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.797 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.797 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.797 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.797 DEBUG:boto:Method: GET 2024-04-16T18:10:46.797 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:46.797 DEBUG:boto:Data: 2024-04-16T18:10:46.797 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.797 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.797 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.798 DEBUG:boto:Params: {} 2024-04-16T18:10:46.798 DEBUG:boto:Token: None 2024-04-16T18:10:46.798 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.798 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2194b40ccd4759a-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.802 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.802 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.802 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.802 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.802 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:46.802 DEBUG:boto:Data: 2024-04-16T18:10:46.802 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.803 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.803 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.803 DEBUG:boto:Params: {} 2024-04-16T18:10:46.803 DEBUG:boto:Token: None 2024-04-16T18:10:46.803 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2 2024-04-16T18:10:46.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aqn6E7lobOAxsAJDxuYunXrlBmw= 2024-04-16T18:10:46.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aqn6E7lobOAxsAJDxuYunXrlBmw=', 'Content-Length': '0'} 2024-04-16T18:10:46.806 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000020bc88ee709203ab-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.806 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.806 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.806 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.806 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:46.807 DEBUG:boto:Data: 2024-04-16T18:10:46.807 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.807 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.807 DEBUG:boto:Params: {} 2024-04-16T18:10:46.807 DEBUG:boto:Token: None 2024-04-16T18:10:46.807 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2 2024-04-16T18:10:46.807 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aqn6E7lobOAxsAJDxuYunXrlBmw= 2024-04-16T18:10:46.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aqn6E7lobOAxsAJDxuYunXrlBmw=', 'Content-Length': '0'} 2024-04-16T18:10:46.810 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b8796fd27755d4c2-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.810 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2024-04-16T18:10:46.810 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.810 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.810 DEBUG:boto:Method: GET 2024-04-16T18:10:46.810 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:46.811 DEBUG:boto:Data: 2024-04-16T18:10:46.811 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.811 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.811 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.811 DEBUG:boto:Params: {} 2024-04-16T18:10:46.811 DEBUG:boto:Token: None 2024-04-16T18:10:46.811 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2 2024-04-16T18:10:46.811 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0AtGUSU/a2Pphdp/WOnm+FsAL2M= 2024-04-16T18:10:46.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0AtGUSU/a2Pphdp/WOnm+FsAL2M=', 'Content-Length': '0'} 2024-04-16T18:10:46.814 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000928be09b7060e14f-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.858 DEBUG:boto:path=/lybtug-41/obj2 2024-04-16T18:10:46.858 DEBUG:boto:auth_path=/lybtug-41/obj2 2024-04-16T18:10:46.859 DEBUG:boto:Method: GET 2024-04-16T18:10:46.859 DEBUG:boto:Path: /lybtug-41/obj2 2024-04-16T18:10:46.859 DEBUG:boto:Data: 2024-04-16T18:10:46.859 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.859 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.859 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.859 DEBUG:boto:Params: {} 2024-04-16T18:10:46.859 DEBUG:boto:Token: None 2024-04-16T18:10:46.859 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj2 2024-04-16T18:10:46.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0AtGUSU/a2Pphdp/WOnm+FsAL2M= 2024-04-16T18:10:46.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0AtGUSU/a2Pphdp/WOnm+FsAL2M=', 'Content-Length': '0'} 2024-04-16T18:10:46.862 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000348e662e0f73cc80-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.906 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.907 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.907 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.907 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.907 DEBUG:boto:Method: GET 2024-04-16T18:10:46.907 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:46.907 DEBUG:boto:Data: 2024-04-16T18:10:46.907 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.907 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.907 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.907 DEBUG:boto:Params: {} 2024-04-16T18:10:46.907 DEBUG:boto:Token: None 2024-04-16T18:10:46.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.911 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f977b2efdd2e5b66-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.912 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.912 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:46.912 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:46.912 DEBUG:boto:path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.912 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj2 2024-04-16T18:10:46.912 DEBUG:boto:Method: GET 2024-04-16T18:10:46.912 DEBUG:boto:Path: /lybtug-41/?prefix=obj2 2024-04-16T18:10:46.913 DEBUG:boto:Data: 2024-04-16T18:10:46.913 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.913 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.913 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.913 DEBUG:boto:Params: {} 2024-04-16T18:10:46.913 DEBUG:boto:Token: None 2024-04-16T18:10:46.913 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/ 2024-04-16T18:10:46.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc= 2024-04-16T18:10:46.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxLmWUtbkKAtMMWMhUPIJ/m7zTc=', 'Content-Length': '0'} 2024-04-16T18:10:46.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000008ff660d4b0db9e-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.917 DEBUG:boto:b'lybtug-41obj21000falseobj22024-04-16T18:10:23.903Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:46.917 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:46.917 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:46.917 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:46.917 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.917 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.917 DEBUG:boto:Method: GET 2024-04-16T18:10:46.918 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.918 DEBUG:boto:Data: 2024-04-16T18:10:46.918 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.918 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.918 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.918 DEBUG:boto:Params: {} 2024-04-16T18:10:46.918 DEBUG:boto:Token: None 2024-04-16T18:10:46.918 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.918 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw= 2024-04-16T18:10:46.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw=', 'Content-Length': '0'} 2024-04-16T18:10:46.921 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000856201fdb55ba022-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.922 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:46.922 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:46.922 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.922 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.922 DEBUG:boto:Method: GET 2024-04-16T18:10:46.922 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.922 DEBUG:boto:Data: 2024-04-16T18:10:46.922 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.923 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.923 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.923 DEBUG:boto:Params: {} 2024-04-16T18:10:46.923 DEBUG:boto:Token: None 2024-04-16T18:10:46.923 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.923 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw= 2024-04-16T18:10:46.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw=', 'Content-Length': '0'} 2024-04-16T18:10:46.926 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000099dd9b69e0989402-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.966 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:46.966 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:46.967 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.967 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.967 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.967 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.967 DEBUG:boto:Data: 2024-04-16T18:10:46.967 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.967 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.967 DEBUG:boto:Params: {} 2024-04-16T18:10:46.967 DEBUG:boto:Token: None 2024-04-16T18:10:46.967 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fQcw0EW4ZQo71KyQIEk6KO5JkQ= 2024-04-16T18:10:46.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fQcw0EW4ZQo71KyQIEk6KO5JkQ=', 'Content-Length': '0'} 2024-04-16T18:10:46.970 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ee1e648f6242465d-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.971 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:46.971 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:46.971 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.971 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.971 DEBUG:boto:Method: HEAD 2024-04-16T18:10:46.971 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.971 DEBUG:boto:Data: 2024-04-16T18:10:46.971 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.971 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.972 DEBUG:boto:Params: {} 2024-04-16T18:10:46.972 DEBUG:boto:Token: None 2024-04-16T18:10:46.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fQcw0EW4ZQo71KyQIEk6KO5JkQ= 2024-04-16T18:10:46.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fQcw0EW4ZQo71KyQIEk6KO5JkQ=', 'Content-Length': '0'} 2024-04-16T18:10:46.975 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000017cbbaf6219088d0-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:46.975 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:46.975 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:46.975 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.975 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.975 DEBUG:boto:Method: GET 2024-04-16T18:10:46.975 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.975 DEBUG:boto:Data: 2024-04-16T18:10:46.976 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.976 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:46.976 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.976 DEBUG:boto:Params: {} 2024-04-16T18:10:46.976 DEBUG:boto:Token: None 2024-04-16T18:10:46.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw= 2024-04-16T18:10:46.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw=', 'Content-Length': '0'} 2024-04-16T18:10:46.979 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004b007f60ef54c891-00661ebf26-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:46.979 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:46.979 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:46.979 DEBUG:boto:path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.979 DEBUG:boto:auth_path=/lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.980 DEBUG:boto:Method: GET 2024-04-16T18:10:46.980 DEBUG:boto:Path: /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.980 DEBUG:boto:Data: 2024-04-16T18:10:46.980 DEBUG:boto:Headers: {} 2024-04-16T18:10:46.980 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:46.980 DEBUG:boto:Port: 8000 2024-04-16T18:10:46.980 DEBUG:boto:Params: {} 2024-04-16T18:10:46.980 DEBUG:boto:Token: None 2024-04-16T18:10:46.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:46 GMT /lybtug-41/obj3?versionId=null 2024-04-16T18:10:46.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw= 2024-04-16T18:10:46.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jhezal67nI4lfcwx01VfU1hnySw=', 'Content-Length': '0'} 2024-04-16T18:10:46.983 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f9c216460384148e-00661ebf26-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.026 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.026 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.027 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.027 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.027 DEBUG:boto:Method: GET 2024-04-16T18:10:47.027 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:47.027 DEBUG:boto:Data: 2024-04-16T18:10:47.027 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.027 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.027 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.027 DEBUG:boto:Params: {} 2024-04-16T18:10:47.027 DEBUG:boto:Token: None 2024-04-16T18:10:47.027 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d011d7e746aa177-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.032 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.032 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.032 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.032 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.032 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.032 DEBUG:boto:Method: GET 2024-04-16T18:10:47.032 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:47.033 DEBUG:boto:Data: 2024-04-16T18:10:47.033 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.033 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.033 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.033 DEBUG:boto:Params: {} 2024-04-16T18:10:47.033 DEBUG:boto:Token: None 2024-04-16T18:10:47.033 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.033 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.037 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d751a13103181883-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.078 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.079 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:47.079 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:47.079 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.079 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:47.079 DEBUG:boto:Data: 2024-04-16T18:10:47.079 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.079 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.079 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.079 DEBUG:boto:Params: {} 2024-04-16T18:10:47.079 DEBUG:boto:Token: None 2024-04-16T18:10:47.079 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj3 2024-04-16T18:10:47.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QS/UdSYCAAOiKgQhHv3f69aKSy0= 2024-04-16T18:10:47.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QS/UdSYCAAOiKgQhHv3f69aKSy0=', 'Content-Length': '0'} 2024-04-16T18:10:47.082 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000af13861899460d5a-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.082 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:47.083 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:47.083 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.083 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:47.083 DEBUG:boto:Data: 2024-04-16T18:10:47.083 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.083 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.083 DEBUG:boto:Params: {} 2024-04-16T18:10:47.083 DEBUG:boto:Token: None 2024-04-16T18:10:47.083 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj3 2024-04-16T18:10:47.083 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QS/UdSYCAAOiKgQhHv3f69aKSy0= 2024-04-16T18:10:47.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QS/UdSYCAAOiKgQhHv3f69aKSy0=', 'Content-Length': '0'} 2024-04-16T18:10:47.086 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004d7380927e2f77e3-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.087 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2024-04-16T18:10:47.087 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:47.087 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:47.087 DEBUG:boto:Method: GET 2024-04-16T18:10:47.087 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:47.087 DEBUG:boto:Data: 2024-04-16T18:10:47.087 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.087 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.087 DEBUG:boto:Params: {} 2024-04-16T18:10:47.088 DEBUG:boto:Token: None 2024-04-16T18:10:47.088 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj3 2024-04-16T18:10:47.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WRTZv836h86KHcNuXwQCXSG0F8M= 2024-04-16T18:10:47.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WRTZv836h86KHcNuXwQCXSG0F8M=', 'Content-Length': '0'} 2024-04-16T18:10:47.091 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e17a32553517c4fc-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.134 DEBUG:boto:path=/lybtug-41/obj3 2024-04-16T18:10:47.135 DEBUG:boto:auth_path=/lybtug-41/obj3 2024-04-16T18:10:47.135 DEBUG:boto:Method: GET 2024-04-16T18:10:47.135 DEBUG:boto:Path: /lybtug-41/obj3 2024-04-16T18:10:47.135 DEBUG:boto:Data: 2024-04-16T18:10:47.135 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.135 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.135 DEBUG:boto:Params: {} 2024-04-16T18:10:47.135 DEBUG:boto:Token: None 2024-04-16T18:10:47.135 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj3 2024-04-16T18:10:47.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WRTZv836h86KHcNuXwQCXSG0F8M= 2024-04-16T18:10:47.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WRTZv836h86KHcNuXwQCXSG0F8M=', 'Content-Length': '0'} 2024-04-16T18:10:47.138 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:28 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c308841a3c12d60e-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.182 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.182 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.183 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.183 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.183 DEBUG:boto:Method: GET 2024-04-16T18:10:47.183 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:47.183 DEBUG:boto:Data: 2024-04-16T18:10:47.183 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.183 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.183 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.183 DEBUG:boto:Params: {} 2024-04-16T18:10:47.183 DEBUG:boto:Token: None 2024-04-16T18:10:47.183 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c54d18697bca65dd-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.188 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.188 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.189 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.189 DEBUG:boto:path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.189 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj3 2024-04-16T18:10:47.189 DEBUG:boto:Method: GET 2024-04-16T18:10:47.189 DEBUG:boto:Path: /lybtug-41/?prefix=obj3 2024-04-16T18:10:47.189 DEBUG:boto:Data: 2024-04-16T18:10:47.189 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.189 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.189 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.189 DEBUG:boto:Params: {} 2024-04-16T18:10:47.189 DEBUG:boto:Token: None 2024-04-16T18:10:47.189 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.190 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e22620b6e1c6925b-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.193 DEBUG:boto:b'lybtug-41obj31000falseobj32024-04-16T18:10:23.922Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.193 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:47.194 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.194 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.194 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.194 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.194 DEBUG:boto:Method: GET 2024-04-16T18:10:47.194 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.194 DEBUG:boto:Data: 2024-04-16T18:10:47.194 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.194 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.194 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.194 DEBUG:boto:Params: {} 2024-04-16T18:10:47.194 DEBUG:boto:Token: None 2024-04-16T18:10:47.195 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.195 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c= 2024-04-16T18:10:47.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c=', 'Content-Length': '0'} 2024-04-16T18:10:47.198 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005fc5834c41498272-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.198 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.198 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.198 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.199 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.199 DEBUG:boto:Method: GET 2024-04-16T18:10:47.199 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.199 DEBUG:boto:Data: 2024-04-16T18:10:47.199 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.199 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.199 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.199 DEBUG:boto:Params: {} 2024-04-16T18:10:47.199 DEBUG:boto:Token: None 2024-04-16T18:10:47.199 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.199 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c= 2024-04-16T18:10:47.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c=', 'Content-Length': '0'} 2024-04-16T18:10:47.202 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006270bc61a4fe05e6-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.246 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.246 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.247 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.247 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.247 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.247 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.247 DEBUG:boto:Data: 2024-04-16T18:10:47.247 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.247 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.247 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.247 DEBUG:boto:Params: {} 2024-04-16T18:10:47.247 DEBUG:boto:Token: None 2024-04-16T18:10:47.247 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DvnlPkT8lx4+mBKnrw/bo5PgUZ8= 2024-04-16T18:10:47.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DvnlPkT8lx4+mBKnrw/bo5PgUZ8=', 'Content-Length': '0'} 2024-04-16T18:10:47.251 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004cd404adb3af780d-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.251 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.251 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.251 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.251 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.251 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.251 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.251 DEBUG:boto:Data: 2024-04-16T18:10:47.252 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.252 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.252 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.252 DEBUG:boto:Params: {} 2024-04-16T18:10:47.252 DEBUG:boto:Token: None 2024-04-16T18:10:47.252 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DvnlPkT8lx4+mBKnrw/bo5PgUZ8= 2024-04-16T18:10:47.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DvnlPkT8lx4+mBKnrw/bo5PgUZ8=', 'Content-Length': '0'} 2024-04-16T18:10:47.255 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002e847c46bde4b2e3-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.255 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:47.255 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.256 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.256 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.256 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.256 DEBUG:boto:Method: GET 2024-04-16T18:10:47.256 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.256 DEBUG:boto:Data: 2024-04-16T18:10:47.256 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.256 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.256 DEBUG:boto:Params: {} 2024-04-16T18:10:47.256 DEBUG:boto:Token: None 2024-04-16T18:10:47.257 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c= 2024-04-16T18:10:47.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c=', 'Content-Length': '0'} 2024-04-16T18:10:47.260 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b9547dc4cecaf53c-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.260 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.260 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.260 DEBUG:boto:path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.260 DEBUG:boto:auth_path=/lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.260 DEBUG:boto:Method: GET 2024-04-16T18:10:47.260 DEBUG:boto:Path: /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.261 DEBUG:boto:Data: 2024-04-16T18:10:47.261 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.261 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.261 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.261 DEBUG:boto:Params: {} 2024-04-16T18:10:47.261 DEBUG:boto:Token: None 2024-04-16T18:10:47.261 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4?versionId=null 2024-04-16T18:10:47.261 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c= 2024-04-16T18:10:47.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fWAULfN+Rl8ku6sIKGXu179s2+c=', 'Content-Length': '0'} 2024-04-16T18:10:47.264 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d756ead1b94cc413-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.306 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.306 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.307 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.307 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.307 DEBUG:boto:Method: GET 2024-04-16T18:10:47.307 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:47.307 DEBUG:boto:Data: 2024-04-16T18:10:47.307 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.307 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.307 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.307 DEBUG:boto:Params: {} 2024-04-16T18:10:47.307 DEBUG:boto:Token: None 2024-04-16T18:10:47.307 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f942eeb64d843611-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.312 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.312 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.312 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.312 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.312 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.312 DEBUG:boto:Method: GET 2024-04-16T18:10:47.313 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:47.313 DEBUG:boto:Data: 2024-04-16T18:10:47.313 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.313 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.313 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.313 DEBUG:boto:Params: {} 2024-04-16T18:10:47.313 DEBUG:boto:Token: None 2024-04-16T18:10:47.313 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.313 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.317 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021c6032496d93bfd-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.317 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.317 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.317 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.317 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.317 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:47.317 DEBUG:boto:Data: 2024-04-16T18:10:47.318 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.318 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.318 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.318 DEBUG:boto:Params: {} 2024-04-16T18:10:47.318 DEBUG:boto:Token: None 2024-04-16T18:10:47.318 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4 2024-04-16T18:10:47.318 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0pMSGfwCMdGU7LCCBgReHJLJsk= 2024-04-16T18:10:47.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0pMSGfwCMdGU7LCCBgReHJLJsk=', 'Content-Length': '0'} 2024-04-16T18:10:47.321 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000013af3e555d8e1b0e-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.321 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.321 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.321 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.321 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:47.321 DEBUG:boto:Data: 2024-04-16T18:10:47.322 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.322 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.322 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.322 DEBUG:boto:Params: {} 2024-04-16T18:10:47.322 DEBUG:boto:Token: None 2024-04-16T18:10:47.322 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4 2024-04-16T18:10:47.322 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0pMSGfwCMdGU7LCCBgReHJLJsk= 2024-04-16T18:10:47.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0pMSGfwCMdGU7LCCBgReHJLJsk=', 'Content-Length': '0'} 2024-04-16T18:10:47.325 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009bc19ed39a40d097-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2024-04-16T18:10:47.326 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.326 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.326 DEBUG:boto:Method: GET 2024-04-16T18:10:47.326 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:47.326 DEBUG:boto:Data: 2024-04-16T18:10:47.326 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.326 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.326 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.327 DEBUG:boto:Params: {} 2024-04-16T18:10:47.327 DEBUG:boto:Token: None 2024-04-16T18:10:47.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4 2024-04-16T18:10:47.327 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1wb+THumCwOfWXV8aEROSzJTkwU= 2024-04-16T18:10:47.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1wb+THumCwOfWXV8aEROSzJTkwU=', 'Content-Length': '0'} 2024-04-16T18:10:47.330 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005ec7827d7d735897-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.370 DEBUG:boto:path=/lybtug-41/obj4 2024-04-16T18:10:47.370 DEBUG:boto:auth_path=/lybtug-41/obj4 2024-04-16T18:10:47.371 DEBUG:boto:Method: GET 2024-04-16T18:10:47.371 DEBUG:boto:Path: /lybtug-41/obj4 2024-04-16T18:10:47.371 DEBUG:boto:Data: 2024-04-16T18:10:47.371 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.371 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.371 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.371 DEBUG:boto:Params: {} 2024-04-16T18:10:47.371 DEBUG:boto:Token: None 2024-04-16T18:10:47.371 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj4 2024-04-16T18:10:47.371 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1wb+THumCwOfWXV8aEROSzJTkwU= 2024-04-16T18:10:47.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1wb+THumCwOfWXV8aEROSzJTkwU=', 'Content-Length': '0'} 2024-04-16T18:10:47.375 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:23 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:24 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f8b959473e47366e-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.422 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.423 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.423 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.423 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.423 DEBUG:boto:Method: GET 2024-04-16T18:10:47.423 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:47.423 DEBUG:boto:Data: 2024-04-16T18:10:47.423 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.423 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.423 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.423 DEBUG:boto:Params: {} 2024-04-16T18:10:47.423 DEBUG:boto:Token: None 2024-04-16T18:10:47.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.428 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a40bdb907ccb1eba-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.428 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.428 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.428 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.429 DEBUG:boto:path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.429 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj4 2024-04-16T18:10:47.429 DEBUG:boto:Method: GET 2024-04-16T18:10:47.429 DEBUG:boto:Path: /lybtug-41/?prefix=obj4 2024-04-16T18:10:47.429 DEBUG:boto:Data: 2024-04-16T18:10:47.429 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.429 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.429 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.429 DEBUG:boto:Params: {} 2024-04-16T18:10:47.429 DEBUG:boto:Token: None 2024-04-16T18:10:47.429 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.433 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac86d780df440c72-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.474 DEBUG:boto:b'lybtug-41obj41000falseobj42024-04-16T18:10:23.971Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:47.475 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.475 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.475 DEBUG:boto:path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.475 DEBUG:boto:auth_path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.475 DEBUG:boto:Method: GET 2024-04-16T18:10:47.475 DEBUG:boto:Path: /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.475 DEBUG:boto:Data: 2024-04-16T18:10:47.475 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.475 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.476 DEBUG:boto:Params: {} 2024-04-16T18:10:47.476 DEBUG:boto:Token: None 2024-04-16T18:10:47.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc= 2024-04-16T18:10:47.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc=', 'Content-Length': '0'} 2024-04-16T18:10:47.479 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000da2a52c9389c7b04-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.479 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.479 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.479 DEBUG:boto:path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.479 DEBUG:boto:auth_path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.480 DEBUG:boto:Method: GET 2024-04-16T18:10:47.480 DEBUG:boto:Path: /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.480 DEBUG:boto:Data: 2024-04-16T18:10:47.480 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.480 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.480 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.480 DEBUG:boto:Params: {} 2024-04-16T18:10:47.480 DEBUG:boto:Token: None 2024-04-16T18:10:47.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc= 2024-04-16T18:10:47.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc=', 'Content-Length': '0'} 2024-04-16T18:10:47.483 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cd287416811c74ee-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.526 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.526 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.527 DEBUG:boto:path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.527 DEBUG:boto:auth_path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.527 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.527 DEBUG:boto:Path: /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.527 DEBUG:boto:Data: 2024-04-16T18:10:47.527 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.527 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.527 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.527 DEBUG:boto:Params: {} 2024-04-16T18:10:47.527 DEBUG:boto:Token: None 2024-04-16T18:10:47.527 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jXjF/5rXiCV/q1gyJFQgDRfkqA= 2024-04-16T18:10:47.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jXjF/5rXiCV/q1gyJFQgDRfkqA=', 'Content-Length': '0'} 2024-04-16T18:10:47.530 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000199e38fb5ee7945-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.531 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.531 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.531 DEBUG:boto:path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.531 DEBUG:boto:auth_path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.531 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.531 DEBUG:boto:Path: /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.531 DEBUG:boto:Data: 2024-04-16T18:10:47.531 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.531 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.531 DEBUG:boto:Params: {} 2024-04-16T18:10:47.532 DEBUG:boto:Token: None 2024-04-16T18:10:47.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jXjF/5rXiCV/q1gyJFQgDRfkqA= 2024-04-16T18:10:47.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jXjF/5rXiCV/q1gyJFQgDRfkqA=', 'Content-Length': '0'} 2024-04-16T18:10:47.535 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d91d9293e49189f9-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.535 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:47.535 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.535 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.535 DEBUG:boto:path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.535 DEBUG:boto:auth_path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.535 DEBUG:boto:Method: GET 2024-04-16T18:10:47.535 DEBUG:boto:Path: /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.535 DEBUG:boto:Data: 2024-04-16T18:10:47.536 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.536 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.536 DEBUG:boto:Params: {} 2024-04-16T18:10:47.536 DEBUG:boto:Token: None 2024-04-16T18:10:47.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc= 2024-04-16T18:10:47.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc=', 'Content-Length': '0'} 2024-04-16T18:10:47.539 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009da9de29ffcc2367-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.539 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.540 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.540 DEBUG:boto:path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.540 DEBUG:boto:auth_path=/lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.540 DEBUG:boto:Method: GET 2024-04-16T18:10:47.540 DEBUG:boto:Path: /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.540 DEBUG:boto:Data: 2024-04-16T18:10:47.540 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.540 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.540 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.540 DEBUG:boto:Params: {} 2024-04-16T18:10:47.540 DEBUG:boto:Token: None 2024-04-16T18:10:47.541 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5?versionId=null 2024-04-16T18:10:47.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc= 2024-04-16T18:10:47.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Rh5z5N+Zzwsxg4VRuOnnZCafRc=', 'Content-Length': '0'} 2024-04-16T18:10:47.544 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000255c3e3e0084618a-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.586 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.586 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.587 DEBUG:boto:path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.587 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.587 DEBUG:boto:Method: GET 2024-04-16T18:10:47.587 DEBUG:boto:Path: /lybtug-41/?prefix=obj5 2024-04-16T18:10:47.587 DEBUG:boto:Data: 2024-04-16T18:10:47.587 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.587 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.587 DEBUG:boto:Params: {} 2024-04-16T18:10:47.587 DEBUG:boto:Token: None 2024-04-16T18:10:47.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000282a0ea1a1a58bad-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.592 DEBUG:boto:b'lybtug-41obj51000falseobj52024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.592 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.593 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.593 DEBUG:boto:path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.593 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.593 DEBUG:boto:Method: GET 2024-04-16T18:10:47.593 DEBUG:boto:Path: /lybtug-41/?prefix=obj5 2024-04-16T18:10:47.593 DEBUG:boto:Data: 2024-04-16T18:10:47.593 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.593 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.593 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.593 DEBUG:boto:Params: {} 2024-04-16T18:10:47.593 DEBUG:boto:Token: None 2024-04-16T18:10:47.594 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.594 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.597 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2fb5996bb829d5e-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.597 DEBUG:boto:b'lybtug-41obj51000falseobj52024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.598 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.598 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.598 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.598 DEBUG:boto:Path: /lybtug-41/obj5 2024-04-16T18:10:47.598 DEBUG:boto:Data: 2024-04-16T18:10:47.598 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.598 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.598 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.599 DEBUG:boto:Params: {} 2024-04-16T18:10:47.599 DEBUG:boto:Token: None 2024-04-16T18:10:47.599 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5 2024-04-16T18:10:47.599 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C4pWXMWTNTOXKBlp2FOeXeN8wP0= 2024-04-16T18:10:47.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C4pWXMWTNTOXKBlp2FOeXeN8wP0=', 'Content-Length': '0'} 2024-04-16T18:10:47.602 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000afe3f290fd071285-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.602 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.602 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.602 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.602 DEBUG:boto:Path: /lybtug-41/obj5 2024-04-16T18:10:47.602 DEBUG:boto:Data: 2024-04-16T18:10:47.602 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.602 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.603 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.603 DEBUG:boto:Params: {} 2024-04-16T18:10:47.603 DEBUG:boto:Token: None 2024-04-16T18:10:47.603 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5 2024-04-16T18:10:47.603 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C4pWXMWTNTOXKBlp2FOeXeN8wP0= 2024-04-16T18:10:47.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C4pWXMWTNTOXKBlp2FOeXeN8wP0=', 'Content-Length': '0'} 2024-04-16T18:10:47.606 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000092bc1a082bb4005d-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.606 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2024-04-16T18:10:47.607 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.607 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.607 DEBUG:boto:Method: GET 2024-04-16T18:10:47.607 DEBUG:boto:Path: /lybtug-41/obj5 2024-04-16T18:10:47.607 DEBUG:boto:Data: 2024-04-16T18:10:47.607 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.607 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.607 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.607 DEBUG:boto:Params: {} 2024-04-16T18:10:47.607 DEBUG:boto:Token: None 2024-04-16T18:10:47.608 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5 2024-04-16T18:10:47.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A1wEoRK9eBOyYJTSUfRhWxoM5xY= 2024-04-16T18:10:47.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A1wEoRK9eBOyYJTSUfRhWxoM5xY=', 'Content-Length': '0'} 2024-04-16T18:10:47.611 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001f0203c57e83890b-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.611 DEBUG:boto:path=/lybtug-41/obj5 2024-04-16T18:10:47.611 DEBUG:boto:auth_path=/lybtug-41/obj5 2024-04-16T18:10:47.611 DEBUG:boto:Method: GET 2024-04-16T18:10:47.611 DEBUG:boto:Path: /lybtug-41/obj5 2024-04-16T18:10:47.612 DEBUG:boto:Data: 2024-04-16T18:10:47.612 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.612 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.612 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.612 DEBUG:boto:Params: {} 2024-04-16T18:10:47.612 DEBUG:boto:Token: None 2024-04-16T18:10:47.612 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj5 2024-04-16T18:10:47.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A1wEoRK9eBOyYJTSUfRhWxoM5xY= 2024-04-16T18:10:47.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A1wEoRK9eBOyYJTSUfRhWxoM5xY=', 'Content-Length': '0'} 2024-04-16T18:10:47.615 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005b461b67424b46a3-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.658 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.659 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.659 DEBUG:boto:path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.659 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.659 DEBUG:boto:Method: GET 2024-04-16T18:10:47.659 DEBUG:boto:Path: /lybtug-41/?prefix=obj5 2024-04-16T18:10:47.659 DEBUG:boto:Data: 2024-04-16T18:10:47.659 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.659 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.659 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.659 DEBUG:boto:Params: {} 2024-04-16T18:10:47.659 DEBUG:boto:Token: None 2024-04-16T18:10:47.659 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009426d28a30f81f0b-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.664 DEBUG:boto:b'lybtug-41obj51000falseobj52024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.664 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.664 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.664 DEBUG:boto:path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.664 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj5 2024-04-16T18:10:47.665 DEBUG:boto:Method: GET 2024-04-16T18:10:47.665 DEBUG:boto:Path: /lybtug-41/?prefix=obj5 2024-04-16T18:10:47.665 DEBUG:boto:Data: 2024-04-16T18:10:47.665 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.665 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.665 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.665 DEBUG:boto:Params: {} 2024-04-16T18:10:47.665 DEBUG:boto:Token: None 2024-04-16T18:10:47.665 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.665 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005249e4ad1961fc02-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.669 DEBUG:boto:b'lybtug-41obj51000falseobj52024-04-16T18:10:38.033Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.669 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:47.669 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.669 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.669 DEBUG:boto:path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.670 DEBUG:boto:auth_path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.670 DEBUG:boto:Method: GET 2024-04-16T18:10:47.670 DEBUG:boto:Path: /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.670 DEBUG:boto:Data: 2024-04-16T18:10:47.670 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.670 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.670 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.670 DEBUG:boto:Params: {} 2024-04-16T18:10:47.670 DEBUG:boto:Token: None 2024-04-16T18:10:47.670 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.670 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ= 2024-04-16T18:10:47.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ=', 'Content-Length': '0'} 2024-04-16T18:10:47.673 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000037a251d7a4cdeccd-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.714 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.714 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.715 DEBUG:boto:path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.715 DEBUG:boto:auth_path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.715 DEBUG:boto:Method: GET 2024-04-16T18:10:47.715 DEBUG:boto:Path: /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.715 DEBUG:boto:Data: 2024-04-16T18:10:47.715 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.715 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.715 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.715 DEBUG:boto:Params: {} 2024-04-16T18:10:47.715 DEBUG:boto:Token: None 2024-04-16T18:10:47.715 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ= 2024-04-16T18:10:47.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ=', 'Content-Length': '0'} 2024-04-16T18:10:47.719 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b4717080b55f4e91-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.762 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.762 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.763 DEBUG:boto:path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.763 DEBUG:boto:auth_path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.763 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.763 DEBUG:boto:Path: /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.763 DEBUG:boto:Data: 2024-04-16T18:10:47.763 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.763 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.763 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.763 DEBUG:boto:Params: {} 2024-04-16T18:10:47.763 DEBUG:boto:Token: None 2024-04-16T18:10:47.763 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RobD8zu4V5Eet2st0eYF5wCkAnE= 2024-04-16T18:10:47.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RobD8zu4V5Eet2st0eYF5wCkAnE=', 'Content-Length': '0'} 2024-04-16T18:10:47.766 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000961a0131b22353f6-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.767 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.767 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.767 DEBUG:boto:path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.767 DEBUG:boto:auth_path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.767 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.767 DEBUG:boto:Path: /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.767 DEBUG:boto:Data: 2024-04-16T18:10:47.767 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.767 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.768 DEBUG:boto:Params: {} 2024-04-16T18:10:47.768 DEBUG:boto:Token: None 2024-04-16T18:10:47.768 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RobD8zu4V5Eet2st0eYF5wCkAnE= 2024-04-16T18:10:47.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RobD8zu4V5Eet2st0eYF5wCkAnE=', 'Content-Length': '0'} 2024-04-16T18:10:47.771 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004d36595ba170fcbe-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.771 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:47.772 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.772 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.772 DEBUG:boto:path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.772 DEBUG:boto:auth_path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.772 DEBUG:boto:Method: GET 2024-04-16T18:10:47.772 DEBUG:boto:Path: /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.772 DEBUG:boto:Data: 2024-04-16T18:10:47.772 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.772 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.772 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.772 DEBUG:boto:Params: {} 2024-04-16T18:10:47.772 DEBUG:boto:Token: None 2024-04-16T18:10:47.773 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.773 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ= 2024-04-16T18:10:47.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ=', 'Content-Length': '0'} 2024-04-16T18:10:47.776 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000007a86370618f9986-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.776 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.776 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.776 DEBUG:boto:path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.776 DEBUG:boto:auth_path=/lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.776 DEBUG:boto:Method: GET 2024-04-16T18:10:47.777 DEBUG:boto:Path: /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.777 DEBUG:boto:Data: 2024-04-16T18:10:47.777 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.777 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.777 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.777 DEBUG:boto:Params: {} 2024-04-16T18:10:47.777 DEBUG:boto:Token: None 2024-04-16T18:10:47.777 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6?versionId=null 2024-04-16T18:10:47.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ= 2024-04-16T18:10:47.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXDfWrfQ5LNN5C9xPJslErwYqEQ=', 'Content-Length': '0'} 2024-04-16T18:10:47.780 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000099653810a2f550cd-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.822 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.822 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.823 DEBUG:boto:path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.823 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.823 DEBUG:boto:Method: GET 2024-04-16T18:10:47.823 DEBUG:boto:Path: /lybtug-41/?prefix=obj6 2024-04-16T18:10:47.823 DEBUG:boto:Data: 2024-04-16T18:10:47.823 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.823 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.823 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.823 DEBUG:boto:Params: {} 2024-04-16T18:10:47.823 DEBUG:boto:Token: None 2024-04-16T18:10:47.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031177e39b6629ec5-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.827 DEBUG:boto:b'lybtug-41obj61000falseobj62024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.827 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.828 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.828 DEBUG:boto:path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.828 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.828 DEBUG:boto:Method: GET 2024-04-16T18:10:47.828 DEBUG:boto:Path: /lybtug-41/?prefix=obj6 2024-04-16T18:10:47.828 DEBUG:boto:Data: 2024-04-16T18:10:47.828 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.828 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.828 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.828 DEBUG:boto:Params: {} 2024-04-16T18:10:47.828 DEBUG:boto:Token: None 2024-04-16T18:10:47.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.832 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5245fced98e1a5d-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.832 DEBUG:boto:b'lybtug-41obj61000falseobj62024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.832 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.833 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.833 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.833 DEBUG:boto:Path: /lybtug-41/obj6 2024-04-16T18:10:47.833 DEBUG:boto:Data: 2024-04-16T18:10:47.833 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.833 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.833 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.833 DEBUG:boto:Params: {} 2024-04-16T18:10:47.833 DEBUG:boto:Token: None 2024-04-16T18:10:47.833 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6 2024-04-16T18:10:47.834 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r3Gr0aObUDGRdqZ65Yl0Z66JVnU= 2024-04-16T18:10:47.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r3Gr0aObUDGRdqZ65Yl0Z66JVnU=', 'Content-Length': '0'} 2024-04-16T18:10:47.836 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000078d8c501db7abcc7-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.836 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.837 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.837 DEBUG:boto:Method: HEAD 2024-04-16T18:10:47.837 DEBUG:boto:Path: /lybtug-41/obj6 2024-04-16T18:10:47.837 DEBUG:boto:Data: 2024-04-16T18:10:47.837 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.837 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.837 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.837 DEBUG:boto:Params: {} 2024-04-16T18:10:47.837 DEBUG:boto:Token: None 2024-04-16T18:10:47.837 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6 2024-04-16T18:10:47.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r3Gr0aObUDGRdqZ65Yl0Z66JVnU= 2024-04-16T18:10:47.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r3Gr0aObUDGRdqZ65Yl0Z66JVnU=', 'Content-Length': '0'} 2024-04-16T18:10:47.841 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008073ca9d622937ae-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.841 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2024-04-16T18:10:47.841 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.841 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.841 DEBUG:boto:Method: GET 2024-04-16T18:10:47.841 DEBUG:boto:Path: /lybtug-41/obj6 2024-04-16T18:10:47.841 DEBUG:boto:Data: 2024-04-16T18:10:47.841 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.842 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.842 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.842 DEBUG:boto:Params: {} 2024-04-16T18:10:47.842 DEBUG:boto:Token: None 2024-04-16T18:10:47.842 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6 2024-04-16T18:10:47.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MwvDxCwTCl5TFiVX64k1LHCCcDg= 2024-04-16T18:10:47.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MwvDxCwTCl5TFiVX64k1LHCCcDg=', 'Content-Length': '0'} 2024-04-16T18:10:47.845 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a4343d2db26aad45-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.886 DEBUG:boto:path=/lybtug-41/obj6 2024-04-16T18:10:47.887 DEBUG:boto:auth_path=/lybtug-41/obj6 2024-04-16T18:10:47.887 DEBUG:boto:Method: GET 2024-04-16T18:10:47.887 DEBUG:boto:Path: /lybtug-41/obj6 2024-04-16T18:10:47.887 DEBUG:boto:Data: 2024-04-16T18:10:47.887 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.887 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.887 DEBUG:boto:Params: {} 2024-04-16T18:10:47.887 DEBUG:boto:Token: None 2024-04-16T18:10:47.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj6 2024-04-16T18:10:47.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MwvDxCwTCl5TFiVX64k1LHCCcDg= 2024-04-16T18:10:47.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MwvDxCwTCl5TFiVX64k1LHCCcDg=', 'Content-Length': '0'} 2024-04-16T18:10:47.891 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005905bbf72ad95df4-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.934 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.934 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.934 DEBUG:boto:path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.935 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.935 DEBUG:boto:Method: GET 2024-04-16T18:10:47.935 DEBUG:boto:Path: /lybtug-41/?prefix=obj6 2024-04-16T18:10:47.935 DEBUG:boto:Data: 2024-04-16T18:10:47.935 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.935 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.935 DEBUG:boto:Params: {} 2024-04-16T18:10:47.935 DEBUG:boto:Token: None 2024-04-16T18:10:47.935 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.935 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa2d9b2a27996c8a-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.939 DEBUG:boto:b'lybtug-41obj61000falseobj62024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.940 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:47.940 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:47.940 DEBUG:boto:path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.940 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj6 2024-04-16T18:10:47.940 DEBUG:boto:Method: GET 2024-04-16T18:10:47.940 DEBUG:boto:Path: /lybtug-41/?prefix=obj6 2024-04-16T18:10:47.940 DEBUG:boto:Data: 2024-04-16T18:10:47.940 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.940 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.940 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.940 DEBUG:boto:Params: {} 2024-04-16T18:10:47.941 DEBUG:boto:Token: None 2024-04-16T18:10:47.941 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/ 2024-04-16T18:10:47.941 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY= 2024-04-16T18:10:47.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xn2WkSnn6IO8TfcclYmZIYH96fY=', 'Content-Length': '0'} 2024-04-16T18:10:47.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000085521b978ddf651-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.986 DEBUG:boto:b'lybtug-41obj61000falseobj62024-04-16T18:10:38.047Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:47.987 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:47.987 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:47.987 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:47.987 DEBUG:boto:path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.987 DEBUG:boto:auth_path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.987 DEBUG:boto:Method: GET 2024-04-16T18:10:47.987 DEBUG:boto:Path: /lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.987 DEBUG:boto:Data: 2024-04-16T18:10:47.987 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:47.987 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.987 DEBUG:boto:Params: {} 2024-04-16T18:10:47.988 DEBUG:boto:Token: None 2024-04-16T18:10:47.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gus45H7IIV5jQxDu0+DIbbkLehM= 2024-04-16T18:10:47.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gus45H7IIV5jQxDu0+DIbbkLehM=', 'Content-Length': '0'} 2024-04-16T18:10:47.991 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000086994048d434040f-00661ebf27-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:47.991 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:47.991 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:47.991 DEBUG:boto:path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.991 DEBUG:boto:auth_path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.992 DEBUG:boto:Method: GET 2024-04-16T18:10:47.992 DEBUG:boto:Path: /lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.992 DEBUG:boto:Data: 2024-04-16T18:10:47.992 DEBUG:boto:Headers: {} 2024-04-16T18:10:47.992 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:47.992 DEBUG:boto:Port: 8000 2024-04-16T18:10:47.992 DEBUG:boto:Params: {} 2024-04-16T18:10:47.992 DEBUG:boto:Token: None 2024-04-16T18:10:47.992 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:47 GMT /lybtug-41/obj7?versionId=null 2024-04-16T18:10:47.993 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gus45H7IIV5jQxDu0+DIbbkLehM= 2024-04-16T18:10:47.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gus45H7IIV5jQxDu0+DIbbkLehM=', 'Content-Length': '0'} 2024-04-16T18:10:47.996 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006b8e03c6c2fd6171-00661ebf27-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.039 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.039 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.039 DEBUG:boto:path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.039 DEBUG:boto:auth_path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.039 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.039 DEBUG:boto:Path: /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.039 DEBUG:boto:Data: 2024-04-16T18:10:48.039 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.039 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.039 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.039 DEBUG:boto:Params: {} 2024-04-16T18:10:48.040 DEBUG:boto:Token: None 2024-04-16T18:10:48.040 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sR+gskL4j31Nt+uBvdS7pJKw6lE= 2024-04-16T18:10:48.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sR+gskL4j31Nt+uBvdS7pJKw6lE=', 'Content-Length': '0'} 2024-04-16T18:10:48.043 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ccef3e1229ee89e6-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.043 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.043 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.043 DEBUG:boto:path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.043 DEBUG:boto:auth_path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.043 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.044 DEBUG:boto:Path: /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.044 DEBUG:boto:Data: 2024-04-16T18:10:48.044 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.044 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.044 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.044 DEBUG:boto:Params: {} 2024-04-16T18:10:48.044 DEBUG:boto:Token: None 2024-04-16T18:10:48.044 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sR+gskL4j31Nt+uBvdS7pJKw6lE= 2024-04-16T18:10:48.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sR+gskL4j31Nt+uBvdS7pJKw6lE=', 'Content-Length': '0'} 2024-04-16T18:10:48.047 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004820a2a28ca97b0d-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.048 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:48.048 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.048 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.048 DEBUG:boto:path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.048 DEBUG:boto:auth_path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.048 DEBUG:boto:Method: GET 2024-04-16T18:10:48.048 DEBUG:boto:Path: /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.048 DEBUG:boto:Data: 2024-04-16T18:10:48.048 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.049 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.049 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.049 DEBUG:boto:Params: {} 2024-04-16T18:10:48.049 DEBUG:boto:Token: None 2024-04-16T18:10:48.049 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.049 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZhpQfoooxZl+EqspRKEBn6EAVFQ= 2024-04-16T18:10:48.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZhpQfoooxZl+EqspRKEBn6EAVFQ=', 'Content-Length': '0'} 2024-04-16T18:10:48.052 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a4493614f3d916c6-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.053 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.053 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.053 DEBUG:boto:path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.053 DEBUG:boto:auth_path=/lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.053 DEBUG:boto:Method: GET 2024-04-16T18:10:48.053 DEBUG:boto:Path: /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.053 DEBUG:boto:Data: 2024-04-16T18:10:48.053 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.053 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.054 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.054 DEBUG:boto:Params: {} 2024-04-16T18:10:48.054 DEBUG:boto:Token: None 2024-04-16T18:10:48.054 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7?versionId=null 2024-04-16T18:10:48.054 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZhpQfoooxZl+EqspRKEBn6EAVFQ= 2024-04-16T18:10:48.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZhpQfoooxZl+EqspRKEBn6EAVFQ=', 'Content-Length': '0'} 2024-04-16T18:10:48.057 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c8dca0bff60c1605-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.098 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.099 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.099 DEBUG:boto:path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.099 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.099 DEBUG:boto:Method: GET 2024-04-16T18:10:48.099 DEBUG:boto:Path: /lybtug-41/?prefix=obj7 2024-04-16T18:10:48.099 DEBUG:boto:Data: 2024-04-16T18:10:48.099 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.099 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.099 DEBUG:boto:Params: {} 2024-04-16T18:10:48.099 DEBUG:boto:Token: None 2024-04-16T18:10:48.100 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.104 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026ebec9a7a461f61-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.104 DEBUG:boto:b'lybtug-41obj71000falseobj72024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.105 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.105 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.105 DEBUG:boto:path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.105 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.105 DEBUG:boto:Method: GET 2024-04-16T18:10:48.105 DEBUG:boto:Path: /lybtug-41/?prefix=obj7 2024-04-16T18:10:48.105 DEBUG:boto:Data: 2024-04-16T18:10:48.105 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.105 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.105 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.105 DEBUG:boto:Params: {} 2024-04-16T18:10:48.106 DEBUG:boto:Token: None 2024-04-16T18:10:48.106 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.106 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b18befa9ee1a0872-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.110 DEBUG:boto:b'lybtug-41obj71000falseobj72024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.111 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.111 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.111 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.111 DEBUG:boto:Path: /lybtug-41/obj7 2024-04-16T18:10:48.111 DEBUG:boto:Data: 2024-04-16T18:10:48.111 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.111 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.111 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.111 DEBUG:boto:Params: {} 2024-04-16T18:10:48.112 DEBUG:boto:Token: None 2024-04-16T18:10:48.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7 2024-04-16T18:10:48.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zm6MOdI7AlZ3oyUMMq4aWorWtk4= 2024-04-16T18:10:48.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zm6MOdI7AlZ3oyUMMq4aWorWtk4=', 'Content-Length': '0'} 2024-04-16T18:10:48.115 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000a694bb4e42dcda6-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.115 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.115 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.115 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.115 DEBUG:boto:Path: /lybtug-41/obj7 2024-04-16T18:10:48.116 DEBUG:boto:Data: 2024-04-16T18:10:48.116 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.116 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.116 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.116 DEBUG:boto:Params: {} 2024-04-16T18:10:48.116 DEBUG:boto:Token: None 2024-04-16T18:10:48.116 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7 2024-04-16T18:10:48.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zm6MOdI7AlZ3oyUMMq4aWorWtk4= 2024-04-16T18:10:48.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zm6MOdI7AlZ3oyUMMq4aWorWtk4=', 'Content-Length': '0'} 2024-04-16T18:10:48.119 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000031872071f6aa6d0d-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.119 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2024-04-16T18:10:48.119 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.120 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.120 DEBUG:boto:Method: GET 2024-04-16T18:10:48.120 DEBUG:boto:Path: /lybtug-41/obj7 2024-04-16T18:10:48.120 DEBUG:boto:Data: 2024-04-16T18:10:48.120 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.120 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.120 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.120 DEBUG:boto:Params: {} 2024-04-16T18:10:48.120 DEBUG:boto:Token: None 2024-04-16T18:10:48.120 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7 2024-04-16T18:10:48.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ufSp7rLBT4E8dhw/fFn8PM3MU9k= 2024-04-16T18:10:48.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ufSp7rLBT4E8dhw/fFn8PM3MU9k=', 'Content-Length': '0'} 2024-04-16T18:10:48.124 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002dc59ec45b5f248c-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.166 DEBUG:boto:path=/lybtug-41/obj7 2024-04-16T18:10:48.167 DEBUG:boto:auth_path=/lybtug-41/obj7 2024-04-16T18:10:48.167 DEBUG:boto:Method: GET 2024-04-16T18:10:48.167 DEBUG:boto:Path: /lybtug-41/obj7 2024-04-16T18:10:48.167 DEBUG:boto:Data: 2024-04-16T18:10:48.167 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.167 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.167 DEBUG:boto:Params: {} 2024-04-16T18:10:48.167 DEBUG:boto:Token: None 2024-04-16T18:10:48.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj7 2024-04-16T18:10:48.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ufSp7rLBT4E8dhw/fFn8PM3MU9k= 2024-04-16T18:10:48.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ufSp7rLBT4E8dhw/fFn8PM3MU9k=', 'Content-Length': '0'} 2024-04-16T18:10:48.171 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e6733edb967e3c77-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.214 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.214 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.215 DEBUG:boto:path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.215 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.215 DEBUG:boto:Method: GET 2024-04-16T18:10:48.215 DEBUG:boto:Path: /lybtug-41/?prefix=obj7 2024-04-16T18:10:48.215 DEBUG:boto:Data: 2024-04-16T18:10:48.215 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.215 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.215 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.215 DEBUG:boto:Params: {} 2024-04-16T18:10:48.215 DEBUG:boto:Token: None 2024-04-16T18:10:48.215 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022b5b80505b26758-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.220 DEBUG:boto:b'lybtug-41obj71000falseobj72024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.220 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.220 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.220 DEBUG:boto:path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.220 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj7 2024-04-16T18:10:48.220 DEBUG:boto:Method: GET 2024-04-16T18:10:48.220 DEBUG:boto:Path: /lybtug-41/?prefix=obj7 2024-04-16T18:10:48.220 DEBUG:boto:Data: 2024-04-16T18:10:48.221 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.221 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.221 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.221 DEBUG:boto:Params: {} 2024-04-16T18:10:48.221 DEBUG:boto:Token: None 2024-04-16T18:10:48.221 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.225 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d98d2c86a09b63aa-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.266 DEBUG:boto:b'lybtug-41obj71000falseobj72024-04-16T18:10:38.099Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.267 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:48.267 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.267 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.267 DEBUG:boto:path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.267 DEBUG:boto:auth_path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.267 DEBUG:boto:Method: GET 2024-04-16T18:10:48.267 DEBUG:boto:Path: /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.267 DEBUG:boto:Data: 2024-04-16T18:10:48.267 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.268 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.268 DEBUG:boto:Params: {} 2024-04-16T18:10:48.268 DEBUG:boto:Token: None 2024-04-16T18:10:48.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks= 2024-04-16T18:10:48.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks=', 'Content-Length': '0'} 2024-04-16T18:10:48.271 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000e258f00cf092973-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.271 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.271 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.272 DEBUG:boto:path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.272 DEBUG:boto:auth_path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.272 DEBUG:boto:Method: GET 2024-04-16T18:10:48.272 DEBUG:boto:Path: /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.272 DEBUG:boto:Data: 2024-04-16T18:10:48.272 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.272 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.272 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.272 DEBUG:boto:Params: {} 2024-04-16T18:10:48.272 DEBUG:boto:Token: None 2024-04-16T18:10:48.272 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks= 2024-04-16T18:10:48.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks=', 'Content-Length': '0'} 2024-04-16T18:10:48.276 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d845a0c9256b0513-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.318 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.319 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.319 DEBUG:boto:path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.319 DEBUG:boto:auth_path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.319 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.319 DEBUG:boto:Path: /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.319 DEBUG:boto:Data: 2024-04-16T18:10:48.319 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.319 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.319 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.319 DEBUG:boto:Params: {} 2024-04-16T18:10:48.320 DEBUG:boto:Token: None 2024-04-16T18:10:48.320 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X0t2Oq8cOL1CFcc1rh4hzILJLu0= 2024-04-16T18:10:48.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X0t2Oq8cOL1CFcc1rh4hzILJLu0=', 'Content-Length': '0'} 2024-04-16T18:10:48.323 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b9ed4f2c2e1f7444-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.323 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.323 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.323 DEBUG:boto:path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.323 DEBUG:boto:auth_path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.323 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.323 DEBUG:boto:Path: /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.323 DEBUG:boto:Data: 2024-04-16T18:10:48.324 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.324 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.324 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.324 DEBUG:boto:Params: {} 2024-04-16T18:10:48.324 DEBUG:boto:Token: None 2024-04-16T18:10:48.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X0t2Oq8cOL1CFcc1rh4hzILJLu0= 2024-04-16T18:10:48.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X0t2Oq8cOL1CFcc1rh4hzILJLu0=', 'Content-Length': '0'} 2024-04-16T18:10:48.327 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f21137f748a19e7a-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.328 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:48.328 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.328 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.328 DEBUG:boto:path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.328 DEBUG:boto:auth_path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.328 DEBUG:boto:Method: GET 2024-04-16T18:10:48.328 DEBUG:boto:Path: /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.328 DEBUG:boto:Data: 2024-04-16T18:10:48.328 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.328 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.328 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.328 DEBUG:boto:Params: {} 2024-04-16T18:10:48.329 DEBUG:boto:Token: None 2024-04-16T18:10:48.329 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.329 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks= 2024-04-16T18:10:48.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks=', 'Content-Length': '0'} 2024-04-16T18:10:48.332 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000148d4cf63f3ab97c-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.332 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.332 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.332 DEBUG:boto:path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.332 DEBUG:boto:auth_path=/lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.332 DEBUG:boto:Method: GET 2024-04-16T18:10:48.332 DEBUG:boto:Path: /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.332 DEBUG:boto:Data: 2024-04-16T18:10:48.333 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.333 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.333 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.333 DEBUG:boto:Params: {} 2024-04-16T18:10:48.333 DEBUG:boto:Token: None 2024-04-16T18:10:48.333 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8?versionId=null 2024-04-16T18:10:48.333 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks= 2024-04-16T18:10:48.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7K61TlJK15dFKNTtD8jMWvg//Ks=', 'Content-Length': '0'} 2024-04-16T18:10:48.336 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ad66f825d0c1ec0a-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.378 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.378 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.379 DEBUG:boto:path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.379 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.379 DEBUG:boto:Method: GET 2024-04-16T18:10:48.379 DEBUG:boto:Path: /lybtug-41/?prefix=obj8 2024-04-16T18:10:48.379 DEBUG:boto:Data: 2024-04-16T18:10:48.379 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.379 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.379 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.379 DEBUG:boto:Params: {} 2024-04-16T18:10:48.379 DEBUG:boto:Token: None 2024-04-16T18:10:48.380 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.383 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4f9c6e280fd21fa-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.384 DEBUG:boto:b'lybtug-41obj81000falseobj82024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.384 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.384 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.385 DEBUG:boto:path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.385 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.385 DEBUG:boto:Method: GET 2024-04-16T18:10:48.385 DEBUG:boto:Path: /lybtug-41/?prefix=obj8 2024-04-16T18:10:48.385 DEBUG:boto:Data: 2024-04-16T18:10:48.385 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.385 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.385 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.385 DEBUG:boto:Params: {} 2024-04-16T18:10:48.385 DEBUG:boto:Token: None 2024-04-16T18:10:48.386 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.386 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.390 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008168ee2a1a36a66c-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.390 DEBUG:boto:b'lybtug-41obj81000falseobj82024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.390 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.391 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.391 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.391 DEBUG:boto:Path: /lybtug-41/obj8 2024-04-16T18:10:48.391 DEBUG:boto:Data: 2024-04-16T18:10:48.391 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.391 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.391 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.391 DEBUG:boto:Params: {} 2024-04-16T18:10:48.391 DEBUG:boto:Token: None 2024-04-16T18:10:48.391 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8 2024-04-16T18:10:48.391 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MbPuHSPZ6+bDwL6DhaknlyVthVI= 2024-04-16T18:10:48.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MbPuHSPZ6+bDwL6DhaknlyVthVI=', 'Content-Length': '0'} 2024-04-16T18:10:48.394 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006f4ba43860e81c3c-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.394 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.394 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.395 DEBUG:boto:Method: HEAD 2024-04-16T18:10:48.395 DEBUG:boto:Path: /lybtug-41/obj8 2024-04-16T18:10:48.395 DEBUG:boto:Data: 2024-04-16T18:10:48.395 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.395 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.395 DEBUG:boto:Params: {} 2024-04-16T18:10:48.395 DEBUG:boto:Token: None 2024-04-16T18:10:48.395 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8 2024-04-16T18:10:48.395 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MbPuHSPZ6+bDwL6DhaknlyVthVI= 2024-04-16T18:10:48.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MbPuHSPZ6+bDwL6DhaknlyVthVI=', 'Content-Length': '0'} 2024-04-16T18:10:48.398 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000190ed60855d0934f-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.399 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2024-04-16T18:10:48.399 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.399 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.399 DEBUG:boto:Method: GET 2024-04-16T18:10:48.399 DEBUG:boto:Path: /lybtug-41/obj8 2024-04-16T18:10:48.399 DEBUG:boto:Data: 2024-04-16T18:10:48.399 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.399 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.399 DEBUG:boto:Params: {} 2024-04-16T18:10:48.399 DEBUG:boto:Token: None 2024-04-16T18:10:48.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8 2024-04-16T18:10:48.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZcBpujHt1KSedO5hhqxmL67/NQ= 2024-04-16T18:10:48.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZcBpujHt1KSedO5hhqxmL67/NQ=', 'Content-Length': '0'} 2024-04-16T18:10:48.403 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000052789d8f0f7f1386-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.403 DEBUG:boto:path=/lybtug-41/obj8 2024-04-16T18:10:48.403 DEBUG:boto:auth_path=/lybtug-41/obj8 2024-04-16T18:10:48.403 DEBUG:boto:Method: GET 2024-04-16T18:10:48.403 DEBUG:boto:Path: /lybtug-41/obj8 2024-04-16T18:10:48.403 DEBUG:boto:Data: 2024-04-16T18:10:48.403 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.404 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.404 DEBUG:boto:Params: {} 2024-04-16T18:10:48.404 DEBUG:boto:Token: None 2024-04-16T18:10:48.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/obj8 2024-04-16T18:10:48.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZcBpujHt1KSedO5hhqxmL67/NQ= 2024-04-16T18:10:48.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZcBpujHt1KSedO5hhqxmL67/NQ=', 'Content-Length': '0'} 2024-04-16T18:10:48.407 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:38 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', 'bd830684-b604-4841-ab52-8c82ee812281:lybtug-41:054102a3-8d94-4596-8b3b-6341eab312e1.4804.10'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:39 GMT'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000081dd0a7aa64407a8-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.450 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.450 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.451 DEBUG:boto:path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.451 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.451 DEBUG:boto:Method: GET 2024-04-16T18:10:48.451 DEBUG:boto:Path: /lybtug-41/?prefix=obj8 2024-04-16T18:10:48.451 DEBUG:boto:Data: 2024-04-16T18:10:48.451 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:48.451 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.451 DEBUG:boto:Params: {} 2024-04-16T18:10:48.451 DEBUG:boto:Token: None 2024-04-16T18:10:48.451 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038148973b846d45a-00661ebf28-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.456 DEBUG:boto:b'lybtug-41obj81000falseobj82024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.456 DEBUG:boto:path=/lybtug-41/ 2024-04-16T18:10:48.456 DEBUG:boto:auth_path=/lybtug-41/ 2024-04-16T18:10:48.456 DEBUG:boto:path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.456 DEBUG:boto:auth_path=/lybtug-41/?prefix=obj8 2024-04-16T18:10:48.456 DEBUG:boto:Method: GET 2024-04-16T18:10:48.456 DEBUG:boto:Path: /lybtug-41/?prefix=obj8 2024-04-16T18:10:48.456 DEBUG:boto:Data: 2024-04-16T18:10:48.456 DEBUG:boto:Headers: {} 2024-04-16T18:10:48.457 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:48.457 DEBUG:boto:Port: 8000 2024-04-16T18:10:48.457 DEBUG:boto:Params: {} 2024-04-16T18:10:48.457 DEBUG:boto:Token: None 2024-04-16T18:10:48.457 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:48 GMT /lybtug-41/ 2024-04-16T18:10:48.457 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo= 2024-04-16T18:10:48.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dlj3dEIavu79PAvYojD0Y0MRoqo=', 'Content-Length': '0'} 2024-04-16T18:10:48.461 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca29ed11e4721d0f-00661ebf28-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:48.461 DEBUG:boto:b'lybtug-41obj81000falseobj82024-04-16T18:10:38.115Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDRGW11111111111111111Normal' 2024-04-16T18:10:48.461 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-41 zones={test-zone2, test-zone1} 2024-04-16T18:10:48.461 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2024-04-16T18:10:48.461 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.462 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:48.503 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:48.503 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:48.615 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"00000000000000000000:00000000000000009646","last_update":"2024-04-16T18:10:38.551636Z"},{"marker":"00000000000000000000:00000000000000008532","last_update":"2024-04-16T18:10:39.394998Z"},{"marker":"00000000000000000000:00000000000000006410","last_update":"2024-04-16T18:10:39.397985Z"},{"marker":"00000000000000000000:00000000000000012592","last_update":"2024-04-16T18:10:39.400594Z"}] 2024-04-16T18:10:48.615 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000009646', 1: '00000000000000000000:00000000000000008532', 2: '00000000000000000000:00000000000000006410', 3: '00000000000000000000:00000000000000012592'} 2024-04-16T18:10:48.615 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:10:48.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.616 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:48.660 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:48.660 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:48.765 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000009646","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:38.551636Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000008532","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.394998Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006410","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.397985Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000012592","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.400594Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:10:48Z"} 2024-04-16T18:10:48.765 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17208660468104013451},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000009646","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:38.551636Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000008532","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.394998Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006410","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.397985Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000012592","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.400594Z"}}]},"full_sync":{"total":0,"complete":0},"current_time":"2024-04-16T18:10:48Z"} 2024-04-16T18:10:48.766 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000009646', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:38.551636Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000008532', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.394998Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006410', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.397985Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000012592', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.400594Z'}}] 2024-04-16T18:10:48.766 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000009646', 1: '00000000000000000000:00000000000000008532', 2: '00000000000000000000:00000000000000006410', 3: '00000000000000000000:00000000000000012592'} 2024-04-16T18:10:48.766 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000009646', 1: '00000000000000000000:00000000000000008532', 2: '00000000000000000000:00000000000000006410', 3: '00000000000000000000:00000000000000012592'} 2024-04-16T18:10:48.766 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2024-04-16T18:10:48.766 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2024-04-16T18:10:48.766 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.766 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.766 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:48.961 INFO:teuthology.orchestra.run.smithi122.stdout:[{"marker":"00000000000000000000:00000000000000006981","last_update":"2024-04-16T18:10:39.853841Z"},{"marker":"00000000000000000000:00000000000000006002","last_update":"2024-04-16T18:10:39.857816Z"},{"marker":"00000000000000000000:00000000000000004869","last_update":"2024-04-16T18:10:39.861247Z"},{"marker":"00000000000000000000:00000000000000009365","last_update":"2024-04-16T18:10:39.865077Z"}] 2024-04-16T18:10:48.961 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000006981', 1: '00000000000000000000:00000000000000006002', 2: '00000000000000000000:00000000000000004869', 3: '00000000000000000000:00000000000000009365'} 2024-04-16T18:10:48.961 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:10:48.962 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:48.962 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:49.006 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:49.006 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:49.111 INFO:teuthology.orchestra.run.smithi071.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006981","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.853841Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006002","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.857816Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004869","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T18:10:39.861247Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000009365","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.865077Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:10:49Z"} 2024-04-16T18:10:49.112 DEBUG:rgw_multi.tests:current data sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":16595158658120093470},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006981","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.853841Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000006002","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.857816Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000004869","next_step_marker":"","total_entries":7,"pos":0,"timestamp":"2024-04-16T18:10:39.861247Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"00000000000000000000:00000000000000009365","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:10:39.865077Z"}}]},"full_sync":{"total":7,"complete":7},"current_time":"2024-04-16T18:10:49Z"} 2024-04-16T18:10:49.112 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006981', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.853841Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006002', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.857816Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004869', 'next_step_marker': '', 'total_entries': 7, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.861247Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000009365', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:10:39.865077Z'}}] 2024-04-16T18:10:49.112 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000006981', 1: '00000000000000000000:00000000000000006002', 2: '00000000000000000000:00000000000000004869', 3: '00000000000000000000:00000000000000009365'} 2024-04-16T18:10:49.112 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000006981', 1: '00000000000000000000:00000000000000006002', 2: '00000000000000000000:00000000000000004869', 3: '00000000000000000000:00000000000000009365'} 2024-04-16T18:10:49.112 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2024-04-16T18:10:49.112 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_disable_enable ... ok 2024-04-16T18:10:49.113 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-42 2024-04-16T18:10:49.114 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:10:49.114 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:10:49.114 DEBUG:boto:Method: PUT 2024-04-16T18:10:49.114 DEBUG:boto:Path: /lybtug-42/ 2024-04-16T18:10:49.114 DEBUG:boto:Data: 2024-04-16T18:10:49.114 DEBUG:boto:Headers: {} 2024-04-16T18:10:49.114 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:49.114 DEBUG:boto:Port: 8000 2024-04-16T18:10:49.114 DEBUG:boto:Params: {} 2024-04-16T18:10:49.114 DEBUG:boto:Token: None 2024-04-16T18:10:49.115 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:10:49 GMT /lybtug-42/ 2024-04-16T18:10:49.115 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hqfVdr0ATJqQcj4jOOC7hcLpmjA= 2024-04-16T18:10:49.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hqfVdr0ATJqQcj4jOOC7hcLpmjA=', 'Content-Length': '0'} 2024-04-16T18:10:49.146 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000f61bd633e2ac559c-00661ebf29-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:49.146 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=lybtug-43 2024-04-16T18:10:49.146 DEBUG:boto:path=/lybtug-43/ 2024-04-16T18:10:49.146 DEBUG:boto:auth_path=/lybtug-43/ 2024-04-16T18:10:49.146 DEBUG:boto:Method: PUT 2024-04-16T18:10:49.146 DEBUG:boto:Path: /lybtug-43/ 2024-04-16T18:10:49.147 DEBUG:boto:Data: 2024-04-16T18:10:49.147 DEBUG:boto:Headers: {} 2024-04-16T18:10:49.147 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:10:49.147 DEBUG:boto:Port: 8000 2024-04-16T18:10:49.147 DEBUG:boto:Params: {} 2024-04-16T18:10:49.147 DEBUG:boto:Token: None 2024-04-16T18:10:49.147 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:10:49 GMT /lybtug-43/ 2024-04-16T18:10:49.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lG3j6TBw0/qVMiIZRIQvIzGGZpo= 2024-04-16T18:10:49.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lG3j6TBw0/qVMiIZRIQvIzGGZpo=', 'Content-Length': '0'} 2024-04-16T18:10:49.206 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000666271c5f8aee67f-00661ebf29-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:10:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:49.206 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:49.206 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:49.206 DEBUG:boto:path=/lybtug-42/MULTIPART?uploads 2024-04-16T18:10:49.206 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploads 2024-04-16T18:10:49.207 DEBUG:boto:Method: POST 2024-04-16T18:10:49.207 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploads 2024-04-16T18:10:49.207 DEBUG:boto:Data: 2024-04-16T18:10:49.207 DEBUG:boto:Headers: {} 2024-04-16T18:10:49.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:49.207 DEBUG:boto:Port: 8000 2024-04-16T18:10:49.207 DEBUG:boto:Params: {} 2024-04-16T18:10:49.207 DEBUG:boto:Token: None 2024-04-16T18:10:49.207 DEBUG:boto:StringToSign: POST Tue, 16 Apr 2024 18:10:49 GMT /lybtug-42/MULTIPART?uploads 2024-04-16T18:10:49.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ACBywgqZjDNqPNbT4nXbDWu3AXU= 2024-04-16T18:10:49.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ACBywgqZjDNqPNbT4nXbDWu3AXU=', 'Content-Length': '0'} 2024-04-16T18:10:50.562 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a73c0e4b92af7d0d-00661ebf29-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '249'), ('Date', 'Tue, 16 Apr 2024 18:10:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:50.563 DEBUG:boto:b'lybtug-42MULTIPART2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq' 2024-04-16T18:10:50.606 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:50.607 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:50.607 DEBUG:boto:path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=1 2024-04-16T18:10:50.607 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=1 2024-04-16T18:10:50.607 DEBUG:boto:Method: PUT 2024-04-16T18:10:50.607 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=1 2024-04-16T18:10:50.607 DEBUG:boto:Data: 2024-04-16T18:10:50.607 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ebKBBg0ze5srhMzzkK3PdA==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2024-04-16T18:10:50.607 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:50.607 DEBUG:boto:Port: 8000 2024-04-16T18:10:50.608 DEBUG:boto:Params: {} 2024-04-16T18:10:50.608 DEBUG:boto:Token: None 2024-04-16T18:10:50.608 DEBUG:boto:StringToSign: PUT ebKBBg0ze5srhMzzkK3PdA== application/octet-stream Tue, 16 Apr 2024 18:10:50 GMT /lybtug-42/MULTIPART?partNumber=1&uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:50.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M1ZdV3j1xqqYQpEeIAvjzwY3xgc= 2024-04-16T18:10:50.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ebKBBg0ze5srhMzzkK3PdA==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M1ZdV3j1xqqYQpEeIAvjzwY3xgc='} 2024-04-16T18:10:50.754 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"79b281060d337b9b2b84ccf390adcf74"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001b16efd6915c74f0-00661ebf2a-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:50.780 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:50.781 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:50.781 DEBUG:boto:path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=2 2024-04-16T18:10:50.781 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=2 2024-04-16T18:10:50.781 DEBUG:boto:Method: PUT 2024-04-16T18:10:50.781 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=2 2024-04-16T18:10:50.781 DEBUG:boto:Data: 2024-04-16T18:10:50.781 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dIQ6OrGTo4m87YmUAtmdXw==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2024-04-16T18:10:50.781 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:50.781 DEBUG:boto:Port: 8000 2024-04-16T18:10:50.781 DEBUG:boto:Params: {} 2024-04-16T18:10:50.781 DEBUG:boto:Token: None 2024-04-16T18:10:50.782 DEBUG:boto:StringToSign: PUT dIQ6OrGTo4m87YmUAtmdXw== application/octet-stream Tue, 16 Apr 2024 18:10:50 GMT /lybtug-42/MULTIPART?partNumber=2&uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:50.782 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XyqsP4LlybvZlAEzNLUrZmNnC3M= 2024-04-16T18:10:50.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dIQ6OrGTo4m87YmUAtmdXw==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XyqsP4LlybvZlAEzNLUrZmNnC3M='} 2024-04-16T18:10:50.881 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"74843a3ab193a389bced899402d99d5f"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008c8b4b1067af7ff9-00661ebf2a-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:50.898 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:50.898 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:50.898 DEBUG:boto:path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=3 2024-04-16T18:10:50.898 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=3 2024-04-16T18:10:50.898 DEBUG:boto:Method: PUT 2024-04-16T18:10:50.899 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=3 2024-04-16T18:10:50.899 DEBUG:boto:Data: 2024-04-16T18:10:50.899 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'e4RW4edMN49Fhh9TYZ51tg==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2024-04-16T18:10:50.899 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:50.899 DEBUG:boto:Port: 8000 2024-04-16T18:10:50.899 DEBUG:boto:Params: {} 2024-04-16T18:10:50.899 DEBUG:boto:Token: None 2024-04-16T18:10:50.899 DEBUG:boto:StringToSign: PUT e4RW4edMN49Fhh9TYZ51tg== application/octet-stream Tue, 16 Apr 2024 18:10:50 GMT /lybtug-42/MULTIPART?partNumber=3&uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:50.899 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o/8XXtzrWjuqti1UrVfVtCqnLqI= 2024-04-16T18:10:50.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'e4RW4edMN49Fhh9TYZ51tg==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o/8XXtzrWjuqti1UrVfVtCqnLqI='} 2024-04-16T18:10:51.000 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"7b8456e1e74c378f45861f53619e75b6"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000c2d00320c1bf297-00661ebf2a-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:51.017 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:51.017 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:51.017 DEBUG:boto:path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=4 2024-04-16T18:10:51.017 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=4 2024-04-16T18:10:51.017 DEBUG:boto:Method: PUT 2024-04-16T18:10:51.017 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq&partNumber=4 2024-04-16T18:10:51.017 DEBUG:boto:Data: 2024-04-16T18:10:51.017 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'l+jDi4yPcIlVvEiQwDQZTQ==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2024-04-16T18:10:51.017 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:51.017 DEBUG:boto:Port: 8000 2024-04-16T18:10:51.017 DEBUG:boto:Params: {} 2024-04-16T18:10:51.018 DEBUG:boto:Token: None 2024-04-16T18:10:51.018 DEBUG:boto:StringToSign: PUT l+jDi4yPcIlVvEiQwDQZTQ== application/octet-stream Tue, 16 Apr 2024 18:10:51 GMT /lybtug-42/MULTIPART?partNumber=4&uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.018 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:byzYqqsDPoP41huvre90eSwKonA= 2024-04-16T18:10:51.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'l+jDi4yPcIlVvEiQwDQZTQ==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:10:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:byzYqqsDPoP41huvre90eSwKonA='} 2024-04-16T18:10:51.113 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"97e8c38b8c8f708955bc4890c034194d"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000565a176c9008aa08-00661ebf2b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:51.114 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:51.114 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:51.114 DEBUG:boto:path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.114 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.114 DEBUG:boto:Method: GET 2024-04-16T18:10:51.114 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.114 DEBUG:boto:Data: 2024-04-16T18:10:51.114 DEBUG:boto:Headers: {} 2024-04-16T18:10:51.114 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:51.114 DEBUG:boto:Port: 8000 2024-04-16T18:10:51.114 DEBUG:boto:Params: {} 2024-04-16T18:10:51.115 DEBUG:boto:Token: None 2024-04-16T18:10:51.115 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:10:51 GMT /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.115 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PVfxH5JAvC3JT0wNIxct4aoWgS0= 2024-04-16T18:10:51.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PVfxH5JAvC3JT0wNIxct4aoWgS0=', 'Content-Length': '0'} 2024-04-16T18:10:51.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000171583622e73bb4d-00661ebf2b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:51.163 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:10:51.163 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:10:51.163 DEBUG:boto:path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.163 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.163 DEBUG:boto:Method: POST 2024-04-16T18:10:51.163 DEBUG:boto:Path: /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.163 DEBUG:boto:Data: 1 "79b281060d337b9b2b84ccf390adcf74" 2 "74843a3ab193a389bced899402d99d5f" 3 "7b8456e1e74c378f45861f53619e75b6" 4 "97e8c38b8c8f708955bc4890c034194d" 2024-04-16T18:10:51.163 DEBUG:boto:Headers: {'Content-Type': 'text/xml'} 2024-04-16T18:10:51.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:10:51.163 DEBUG:boto:Port: 8000 2024-04-16T18:10:51.163 DEBUG:boto:Params: {} 2024-04-16T18:10:51.164 DEBUG:boto:Token: None 2024-04-16T18:10:51.164 DEBUG:boto:StringToSign: POST text/xml Tue, 16 Apr 2024 18:10:51 GMT /lybtug-42/MULTIPART?uploadId=2~3nXZRxcEGMvacfS5Aw0Et8ewE1i_gPq 2024-04-16T18:10:51.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r37LfzRx5Njb9CvW+mDv/qEiIs4= 2024-04-16T18:10:51.164 DEBUG:boto:Final headers: {'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:10:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r37LfzRx5Njb9CvW+mDv/qEiIs4=', 'Content-Length': '460'} 2024-04-16T18:10:51.197 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007de4085b14330653-00661ebf2b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '325'), ('Date', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:10:51.197 DEBUG:boto:http://smithi071.front.sepia.ceph.com:8000/lybtug-42/MULTIPARTlybtug-42MULTIPARTd07d411772772538cd4b657da660bfb5-4 2024-04-16T18:10:51.197 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:51.198 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:51.198 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:51.239 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:51.239 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:51.314 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:10:51.346 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291049.185475_429.1","last_update":"2024-04-16T18:10:49.185475Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:10:51.347 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291049.185475_429.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:51.347 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:10:51.347 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:51.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:10:51.347 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:10:51.390 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:51.390 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:51.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:51.462+0000 7f8b1d1fc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:51.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:51.466+0000 7f8b1d1fc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:10:51.495 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291049.185475_429.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:49.185475Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:51Z"} 2024-04-16T18:10:51.496 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291049.185475_429.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:10:49.185475Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:10:51Z"} 2024-04-16T18:10:51.496 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291049.185475_429.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:10:49.185475Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:10:51.496 DEBUG:rgw_multi.tests:log_status={0: '1_1713291049.185475_429.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:51.496 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291049.185475_429.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:10:51.496 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:10:51.496 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-42 2024-04-16T18:10:51.496 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-42', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:51.496 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-42', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:51.497 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-42 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:51.579 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:51.579 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:51.678+0000 7fd0e71cc980 1 waiting to reach incremental sync.. 2024-04-16T18:10:56.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:56.682+0000 7fd0e71cc980 1 bucket sync caught up with source: 2024-04-16T18:10:56.679 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000006.55.6, , , , , ] 2024-04-16T18:10:56.679 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000006.55.6, , , , , ] 2024-04-16T18:10:56.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:10:56.682+0000 7fd0e71cc980 0 bucket checkpoint complete 2024-04-16T18:10:56.724 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-42 2024-04-16T18:10:56.725 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-42', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:56.725 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-42', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:10:56.725 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-42 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:10:56.777 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:10:56.777 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:10:56.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:10:56.855+0000 7fbab1b25980 1 waiting to reach incremental sync.. 2024-04-16T18:11:01.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:01.859+0000 7fbab1b25980 1 bucket sync caught up with source: 2024-04-16T18:11:01.860 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , 00000000001.114.6, , , , , ] 2024-04-16T18:11:01.860 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , 00000000001.114.6, , , , , ] 2024-04-16T18:11:01.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:01.859+0000 7fbab1b25980 0 bucket checkpoint complete 2024-04-16T18:11:01.905 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-42 zones={test-zone2, test-zone1} 2024-04-16T18:11:01.905 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:01.905 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:01.905 DEBUG:boto:Method: HEAD 2024-04-16T18:11:01.905 DEBUG:boto:Path: /lybtug-42/ 2024-04-16T18:11:01.905 DEBUG:boto:Data: 2024-04-16T18:11:01.905 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.905 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:01.906 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.906 DEBUG:boto:Params: {} 2024-04-16T18:11:01.906 DEBUG:boto:Token: None 2024-04-16T18:11:01.906 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/ 2024-04-16T18:11:01.906 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtICaIS//yifWqZfPkE97UmBsHY= 2024-04-16T18:11:01.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtICaIS//yifWqZfPkE97UmBsHY=', 'Content-Length': '0'} 2024-04-16T18:11:01.910 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003c199418f62e5b2b-00661ebf35-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:01.910 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:01.911 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:01.911 DEBUG:boto:Method: HEAD 2024-04-16T18:11:01.911 DEBUG:boto:Path: /lybtug-42/ 2024-04-16T18:11:01.911 DEBUG:boto:Data: 2024-04-16T18:11:01.911 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:01.911 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.911 DEBUG:boto:Params: {} 2024-04-16T18:11:01.911 DEBUG:boto:Token: None 2024-04-16T18:11:01.911 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/ 2024-04-16T18:11:01.911 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mtICaIS//yifWqZfPkE97UmBsHY= 2024-04-16T18:11:01.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mtICaIS//yifWqZfPkE97UmBsHY=', 'Content-Length': '0'} 2024-04-16T18:11:01.915 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea8292961ac4e40a-00661ebf35-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:01.915 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:11:01.915 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:01.915 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:01.916 DEBUG:boto:path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.916 DEBUG:boto:auth_path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.916 DEBUG:boto:Method: GET 2024-04-16T18:11:01.916 DEBUG:boto:Path: /lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.916 DEBUG:boto:Data: 2024-04-16T18:11:01.916 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.916 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:01.916 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.916 DEBUG:boto:Params: {} 2024-04-16T18:11:01.916 DEBUG:boto:Token: None 2024-04-16T18:11:01.917 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/?versions 2024-04-16T18:11:01.917 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ= 2024-04-16T18:11:01.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ=', 'Content-Length': '0'} 2024-04-16T18:11:01.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009f4079b406ecfa9-00661ebf35-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:01.962 DEBUG:boto:b'lybtug-42999falseMULTIPARTnulltrue2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:01.963 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2024-04-16T18:11:01.963 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:11:01.963 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:01.963 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:01.963 DEBUG:boto:path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.963 DEBUG:boto:auth_path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.963 DEBUG:boto:Method: GET 2024-04-16T18:11:01.963 DEBUG:boto:Path: /lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.963 DEBUG:boto:Data: 2024-04-16T18:11:01.963 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:01.964 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.964 DEBUG:boto:Params: {} 2024-04-16T18:11:01.964 DEBUG:boto:Token: None 2024-04-16T18:11:01.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/?versions 2024-04-16T18:11:01.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ= 2024-04-16T18:11:01.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ=', 'Content-Length': '0'} 2024-04-16T18:11:01.968 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cf92fdfa3893f2f-00661ebf35-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:01.969 DEBUG:boto:b'lybtug-42999falseMULTIPARTnulltrue2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:01.969 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2024-04-16T18:11:01.969 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:01.969 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:01.969 DEBUG:boto:path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.969 DEBUG:boto:auth_path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.969 DEBUG:boto:Method: GET 2024-04-16T18:11:01.969 DEBUG:boto:Path: /lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.969 DEBUG:boto:Data: 2024-04-16T18:11:01.970 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.970 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:01.970 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.970 DEBUG:boto:Params: {} 2024-04-16T18:11:01.970 DEBUG:boto:Token: None 2024-04-16T18:11:01.970 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/?versions 2024-04-16T18:11:01.970 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ= 2024-04-16T18:11:01.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ=', 'Content-Length': '0'} 2024-04-16T18:11:01.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b532b5e6d3c090d3-00661ebf35-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:01.974 DEBUG:boto:b'lybtug-42999falseMULTIPARTnulltrue2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:01.974 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:01.975 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:01.975 DEBUG:boto:path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.975 DEBUG:boto:auth_path=/lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.975 DEBUG:boto:Method: GET 2024-04-16T18:11:01.975 DEBUG:boto:Path: /lybtug-42/?versions&max-keys=999 2024-04-16T18:11:01.975 DEBUG:boto:Data: 2024-04-16T18:11:01.975 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.975 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:01.975 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.975 DEBUG:boto:Params: {} 2024-04-16T18:11:01.975 DEBUG:boto:Token: None 2024-04-16T18:11:01.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/?versions 2024-04-16T18:11:01.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ= 2024-04-16T18:11:01.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dr6ujyftG23eGVpjyqP7Hh7SHKQ=', 'Content-Length': '0'} 2024-04-16T18:11:01.979 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094387650bce009fe-00661ebf35-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:01.980 DEBUG:boto:b'lybtug-42999falseMULTIPARTnulltrue2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:01.980 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2024-04-16T18:11:01.980 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:01.980 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:01.980 DEBUG:boto:path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:01.980 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:01.980 DEBUG:boto:Method: GET 2024-04-16T18:11:01.981 DEBUG:boto:Path: /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:01.981 DEBUG:boto:Data: 2024-04-16T18:11:01.981 DEBUG:boto:Headers: {} 2024-04-16T18:11:01.981 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:01.981 DEBUG:boto:Port: 8000 2024-04-16T18:11:01.981 DEBUG:boto:Params: {} 2024-04-16T18:11:01.981 DEBUG:boto:Token: None 2024-04-16T18:11:01.981 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:01 GMT /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:01.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zbD08LY8IIQw/brYDUHrWGspF/A= 2024-04-16T18:11:01.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zbD08LY8IIQw/brYDUHrWGspF/A=', 'Content-Length': '0'} 2024-04-16T18:11:01.990 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-42:054102a3-8d94-4596-8b3b-6341eab312e1.4804.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:54 GMT'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000ffe56447daa3ad6e-00661ebf35-4984-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.059 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.059 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.059 DEBUG:boto:path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.059 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.059 DEBUG:boto:Method: GET 2024-04-16T18:11:02.059 DEBUG:boto:Path: /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.059 DEBUG:boto:Data: 2024-04-16T18:11:02.059 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.059 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.059 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.060 DEBUG:boto:Params: {} 2024-04-16T18:11:02.060 DEBUG:boto:Token: None 2024-04-16T18:11:02.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZDhLk1uyby/ZCFRmtGQwIUq0Lms= 2024-04-16T18:11:02.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZDhLk1uyby/ZCFRmtGQwIUq0Lms=', 'Content-Length': '0'} 2024-04-16T18:11:02.083 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000002dcf1a48b85164bd-00661ebf36-4804-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.198 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.199 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.199 DEBUG:boto:path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.199 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.199 DEBUG:boto:Method: HEAD 2024-04-16T18:11:02.199 DEBUG:boto:Path: /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.199 DEBUG:boto:Data: 2024-04-16T18:11:02.199 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.199 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:02.199 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.199 DEBUG:boto:Params: {} 2024-04-16T18:11:02.199 DEBUG:boto:Token: None 2024-04-16T18:11:02.200 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lgZ2AY89NB+tZXLt02KuF5UlfKc= 2024-04-16T18:11:02.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgZ2AY89NB+tZXLt02KuF5UlfKc=', 'Content-Length': '0'} 2024-04-16T18:11:02.203 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-42:054102a3-8d94-4596-8b3b-6341eab312e1.4804.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:54 GMT'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000000e5daff3181e85a0-00661ebf36-4984-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.203 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.203 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.203 DEBUG:boto:path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.203 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.204 DEBUG:boto:Method: HEAD 2024-04-16T18:11:02.204 DEBUG:boto:Path: /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.204 DEBUG:boto:Data: 2024-04-16T18:11:02.204 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.204 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.204 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.204 DEBUG:boto:Params: {} 2024-04-16T18:11:02.204 DEBUG:boto:Token: None 2024-04-16T18:11:02.204 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lgZ2AY89NB+tZXLt02KuF5UlfKc= 2024-04-16T18:11:02.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgZ2AY89NB+tZXLt02KuF5UlfKc=', 'Content-Length': '0'} 2024-04-16T18:11:02.208 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000a6dfd18ec4f1370c-00661ebf36-4804-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2024-04-16T18:11:02.208 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.208 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.208 DEBUG:boto:path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.208 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.208 DEBUG:boto:Method: GET 2024-04-16T18:11:02.208 DEBUG:boto:Path: /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.208 DEBUG:boto:Data: 2024-04-16T18:11:02.209 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.209 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:02.209 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.209 DEBUG:boto:Params: {} 2024-04-16T18:11:02.209 DEBUG:boto:Token: None 2024-04-16T18:11:02.209 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.209 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZDhLk1uyby/ZCFRmtGQwIUq0Lms= 2024-04-16T18:11:02.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZDhLk1uyby/ZCFRmtGQwIUq0Lms=', 'Content-Length': '0'} 2024-04-16T18:11:02.216 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-42:054102a3-8d94-4596-8b3b-6341eab312e1.4804.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:54 GMT'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000a544d425fe777bb2-00661ebf36-4984-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.293 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.294 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.294 DEBUG:boto:path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.294 DEBUG:boto:auth_path=/lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.294 DEBUG:boto:Method: GET 2024-04-16T18:11:02.294 DEBUG:boto:Path: /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.294 DEBUG:boto:Data: 2024-04-16T18:11:02.294 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.294 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.294 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.294 DEBUG:boto:Params: {} 2024-04-16T18:11:02.295 DEBUG:boto:Token: None 2024-04-16T18:11:02.295 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART?versionId=null 2024-04-16T18:11:02.295 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZDhLk1uyby/ZCFRmtGQwIUq0Lms= 2024-04-16T18:11:02.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZDhLk1uyby/ZCFRmtGQwIUq0Lms=', 'Content-Length': '0'} 2024-04-16T18:11:02.319 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000002a9d8f52f1e5a33-00661ebf36-4804-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.433 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:02.434 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:02.434 DEBUG:boto:path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.434 DEBUG:boto:auth_path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.434 DEBUG:boto:Method: GET 2024-04-16T18:11:02.434 DEBUG:boto:Path: /lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.434 DEBUG:boto:Data: 2024-04-16T18:11:02.434 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.434 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:02.434 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.435 DEBUG:boto:Params: {} 2024-04-16T18:11:02.435 DEBUG:boto:Token: None 2024-04-16T18:11:02.435 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/ 2024-04-16T18:11:02.435 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg= 2024-04-16T18:11:02.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg=', 'Content-Length': '0'} 2024-04-16T18:11:02.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000210ab892898cf709-00661ebf36-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.439 DEBUG:boto:b'lybtug-42MULTIPART1000falseMULTIPART2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:02.440 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:02.440 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:02.440 DEBUG:boto:path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.440 DEBUG:boto:auth_path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.440 DEBUG:boto:Method: GET 2024-04-16T18:11:02.440 DEBUG:boto:Path: /lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.440 DEBUG:boto:Data: 2024-04-16T18:11:02.441 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.441 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.441 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.441 DEBUG:boto:Params: {} 2024-04-16T18:11:02.441 DEBUG:boto:Token: None 2024-04-16T18:11:02.441 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/ 2024-04-16T18:11:02.441 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg= 2024-04-16T18:11:02.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg=', 'Content-Length': '0'} 2024-04-16T18:11:02.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad97678327beb41d-00661ebf36-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.445 DEBUG:boto:b'lybtug-42MULTIPART1000falseMULTIPART2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:02.446 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.446 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.446 DEBUG:boto:Method: HEAD 2024-04-16T18:11:02.446 DEBUG:boto:Path: /lybtug-42/MULTIPART 2024-04-16T18:11:02.446 DEBUG:boto:Data: 2024-04-16T18:11:02.446 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.446 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:02.446 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.446 DEBUG:boto:Params: {} 2024-04-16T18:11:02.446 DEBUG:boto:Token: None 2024-04-16T18:11:02.447 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART 2024-04-16T18:11:02.447 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5kJIAlvqn/t7zjUsRIBH3juOIrg= 2024-04-16T18:11:02.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5kJIAlvqn/t7zjUsRIBH3juOIrg=', 'Content-Length': '0'} 2024-04-16T18:11:02.449 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-42:054102a3-8d94-4596-8b3b-6341eab312e1.4804.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:54 GMT'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000042e8fa9d8f985541-00661ebf36-4984-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.450 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.450 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.450 DEBUG:boto:Method: HEAD 2024-04-16T18:11:02.450 DEBUG:boto:Path: /lybtug-42/MULTIPART 2024-04-16T18:11:02.450 DEBUG:boto:Data: 2024-04-16T18:11:02.450 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.450 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.450 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.450 DEBUG:boto:Params: {} 2024-04-16T18:11:02.450 DEBUG:boto:Token: None 2024-04-16T18:11:02.451 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART 2024-04-16T18:11:02.451 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5kJIAlvqn/t7zjUsRIBH3juOIrg= 2024-04-16T18:11:02.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5kJIAlvqn/t7zjUsRIBH3juOIrg=', 'Content-Length': '0'} 2024-04-16T18:11:02.454 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000002374cf5685ddda2b-00661ebf36-4804-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.454 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2024-04-16T18:11:02.454 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.454 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.454 DEBUG:boto:Method: GET 2024-04-16T18:11:02.454 DEBUG:boto:Path: /lybtug-42/MULTIPART 2024-04-16T18:11:02.454 DEBUG:boto:Data: 2024-04-16T18:11:02.455 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.455 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:02.455 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.455 DEBUG:boto:Params: {} 2024-04-16T18:11:02.455 DEBUG:boto:Token: None 2024-04-16T18:11:02.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART 2024-04-16T18:11:02.455 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcPkDF3wbm+A36ILG4j3bbNxX/A= 2024-04-16T18:11:02.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcPkDF3wbm+A36ILG4j3bbNxX/A=', 'Content-Length': '0'} 2024-04-16T18:11:02.464 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-42:054102a3-8d94-4596-8b3b-6341eab312e1.4804.11'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:10:54 GMT'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000059862256845992ed-00661ebf36-4984-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.543 DEBUG:boto:path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.544 DEBUG:boto:auth_path=/lybtug-42/MULTIPART 2024-04-16T18:11:02.544 DEBUG:boto:Method: GET 2024-04-16T18:11:02.544 DEBUG:boto:Path: /lybtug-42/MULTIPART 2024-04-16T18:11:02.544 DEBUG:boto:Data: 2024-04-16T18:11:02.544 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.544 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.544 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.544 DEBUG:boto:Params: {} 2024-04-16T18:11:02.544 DEBUG:boto:Token: None 2024-04-16T18:11:02.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/MULTIPART 2024-04-16T18:11:02.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcPkDF3wbm+A36ILG4j3bbNxX/A= 2024-04-16T18:11:02.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcPkDF3wbm+A36ILG4j3bbNxX/A=', 'Content-Length': '0'} 2024-04-16T18:11:02.568 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:10:51 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000c9ba657ab1d26451-00661ebf36-4804-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.705 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:02.705 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:02.705 DEBUG:boto:path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.705 DEBUG:boto:auth_path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.705 DEBUG:boto:Method: GET 2024-04-16T18:11:02.705 DEBUG:boto:Path: /lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.705 DEBUG:boto:Data: 2024-04-16T18:11:02.705 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.705 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:02.705 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.706 DEBUG:boto:Params: {} 2024-04-16T18:11:02.706 DEBUG:boto:Token: None 2024-04-16T18:11:02.706 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/ 2024-04-16T18:11:02.706 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg= 2024-04-16T18:11:02.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg=', 'Content-Length': '0'} 2024-04-16T18:11:02.710 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000778ceafe2afdebf0-00661ebf36-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.710 DEBUG:boto:b'lybtug-42MULTIPART1000falseMULTIPART2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:02.710 DEBUG:boto:path=/lybtug-42/ 2024-04-16T18:11:02.711 DEBUG:boto:auth_path=/lybtug-42/ 2024-04-16T18:11:02.711 DEBUG:boto:path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.711 DEBUG:boto:auth_path=/lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.711 DEBUG:boto:Method: GET 2024-04-16T18:11:02.711 DEBUG:boto:Path: /lybtug-42/?prefix=MULTIPART 2024-04-16T18:11:02.711 DEBUG:boto:Data: 2024-04-16T18:11:02.711 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.711 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.711 DEBUG:boto:Params: {} 2024-04-16T18:11:02.711 DEBUG:boto:Token: None 2024-04-16T18:11:02.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:02 GMT /lybtug-42/ 2024-04-16T18:11:02.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg= 2024-04-16T18:11:02.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EFAUxKut5UUz/49dHJHDPMurvFg=', 'Content-Length': '0'} 2024-04-16T18:11:02.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab46ee754e8989b1-00661ebf36-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.716 DEBUG:boto:b'lybtug-42MULTIPART1000falseMULTIPART2024-04-16T18:10:51.178Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:02.716 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-42 zones={test-zone2, test-zone1} 2024-04-16T18:11:02.716 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multipart_object_sync ... ok 2024-04-16T18:11:02.717 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-44 2024-04-16T18:11:02.717 DEBUG:boto:path=/lybtug-44/ 2024-04-16T18:11:02.717 DEBUG:boto:auth_path=/lybtug-44/ 2024-04-16T18:11:02.717 DEBUG:boto:Method: PUT 2024-04-16T18:11:02.717 DEBUG:boto:Path: /lybtug-44/ 2024-04-16T18:11:02.718 DEBUG:boto:Data: 2024-04-16T18:11:02.718 DEBUG:boto:Headers: {} 2024-04-16T18:11:02.718 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.718 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.718 DEBUG:boto:Params: {} 2024-04-16T18:11:02.718 DEBUG:boto:Token: None 2024-04-16T18:11:02.718 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:11:02 GMT /lybtug-44/ 2024-04-16T18:11:02.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:asg87uNdAK9VTWa0QPqVyUn1pOQ= 2024-04-16T18:11:02.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:asg87uNdAK9VTWa0QPqVyUn1pOQ=', 'Content-Length': '0'} 2024-04-16T18:11:02.755 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000043d020b09ae510ac-00661ebf36-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.755 DEBUG:boto:path=/lybtug-44/testobj-sse-c 2024-04-16T18:11:02.756 DEBUG:boto:auth_path=/lybtug-44/testobj-sse-c 2024-04-16T18:11:02.756 DEBUG:boto:Method: PUT 2024-04-16T18:11:02.756 DEBUG:boto:Path: /lybtug-44/testobj-sse-c 2024-04-16T18:11:02.756 DEBUG:boto:Data: 2024-04-16T18:11:02.756 DEBUG:boto:Headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:11:02.756 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.756 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.756 DEBUG:boto:Params: {} 2024-04-16T18:11:02.756 DEBUG:boto:Token: None 2024-04-16T18:11:02.757 DEBUG:boto:StringToSign: PUT 3FCGuEcom6i4veFJuDiBdQ== application/octet-stream Tue, 16 Apr 2024 18:11:02 GMT x-amz-server-side-encryption-customer-algorithm:AES256 x-amz-server-side-encryption-customer-key:pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= x-amz-server-side-encryption-customer-key-md5:DWygnHRtgiJ77HCm+1rvHw== /lybtug-44/testobj-sse-c 2024-04-16T18:11:02.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FR3gch3wYBLu6Jk4Jn/cxUbnDho= 2024-04-16T18:11:02.757 DEBUG:boto:Final headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FR3gch3wYBLu6Jk4Jn/cxUbnDho='} 2024-04-16T18:11:02.802 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('Accept-Ranges', 'bytes'), ('x-amz-server-side-encryption-customer-algorithm', 'AES256'), ('x-amz-server-side-encryption-customer-key-MD5', 'DWygnHRtgiJ77HCm+1rvHw=='), ('x-amz-request-id', 'tx0000003b6c2cd9883bd5c-00661ebf36-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.803 DEBUG:boto:path=/lybtug-44/testobj-sse-kms 2024-04-16T18:11:02.803 DEBUG:boto:auth_path=/lybtug-44/testobj-sse-kms 2024-04-16T18:11:02.803 DEBUG:boto:Method: PUT 2024-04-16T18:11:02.803 DEBUG:boto:Path: /lybtug-44/testobj-sse-kms 2024-04-16T18:11:02.803 DEBUG:boto:Data: 2024-04-16T18:11:02.803 DEBUG:boto:Headers: {'x-amz-server-side-encryption': 'aws:kms', 'x-amz-server-side-encryption-aws-kms-key-id': 'testkey-1', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue'} 2024-04-16T18:11:02.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:02.803 DEBUG:boto:Port: 8000 2024-04-16T18:11:02.803 DEBUG:boto:Params: {} 2024-04-16T18:11:02.803 DEBUG:boto:Token: None 2024-04-16T18:11:02.804 DEBUG:boto:StringToSign: PUT 3FCGuEcom6i4veFJuDiBdQ== application/octet-stream Tue, 16 Apr 2024 18:11:02 GMT x-amz-server-side-encryption:aws:kms x-amz-server-side-encryption-aws-kms-key-id:testkey-1 /lybtug-44/testobj-sse-kms 2024-04-16T18:11:02.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/kOPwGs3LVy7H2O4o8K2YoLCEpY= 2024-04-16T18:11:02.804 DEBUG:boto:Final headers: {'x-amz-server-side-encryption': 'aws:kms', 'x-amz-server-side-encryption-aws-kms-key-id': 'testkey-1', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:02 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/kOPwGs3LVy7H2O4o8K2YoLCEpY='} 2024-04-16T18:11:02.892 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('Accept-Ranges', 'bytes'), ('x-amz-server-side-encryption', 'aws:kms'), ('x-amz-server-side-encryption-aws-kms-key-id', 'testkey-1'), ('x-amz-server-side-encryption-context', 'eyJhd3M6czM6YXJuIjoiYXJuOmF3czpzMzo6cmd3LW11bHRpc2l0ZS10ZXN0LXVzZXI6bHlidHVnLTQ0L3Rlc3RvYmotc3NlLWttcyJ9'), ('x-amz-request-id', 'tx000000c3f48df6c95bd51-00661ebf36-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:02.892 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:02.892 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:02.892 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:02.934 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:02.934 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:03.002 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:11:03.035 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291062.754727_435.1","last_update":"2024-04-16T18:11:02.754727Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:11:03.036 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291062.754727_435.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:03.036 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:11:03.036 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:03.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:03.036 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:03.078 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:03.081 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:03.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:03.154+0000 7f119d452980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:03.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:03.154+0000 7f119d452980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:03.184 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291062.735335_432.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:02.735335Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:03Z"} 2024-04-16T18:11:03.185 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291062.735335_432.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:02.735335Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:03Z"} 2024-04-16T18:11:03.185 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291062.735335_432.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:11:02.735335Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:11:03.185 DEBUG:rgw_multi.tests:log_status={0: '1_1713291062.754727_435.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:03.185 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291062.735335_432.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:03.185 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291062.754727_435.1 target=1_1713291062.735335_432.1 2024-04-16T18:11:08.186 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:08.186 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:08.186 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:08.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:08.302+0000 7f1b6fbab980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:08.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:08.306+0000 7f1b6fbab980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:08.335 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291062.754727_435.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:02.754727Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:08Z"} 2024-04-16T18:11:08.335 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291062.754727_435.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:02.754727Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:08Z"} 2024-04-16T18:11:08.335 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291062.754727_435.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:11:02.754727Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:11:08.335 DEBUG:rgw_multi.tests:log_status={0: '1_1713291062.754727_435.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:08.336 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291062.754727_435.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:08.336 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:11:08.336 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-44', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:08.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-44', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:08.336 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-44 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:08.425 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:08.425 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:08.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:08.506+0000 7fac83780980 1 bucket sync caught up with source: 2024-04-16T18:11:08.503 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.32.6, , , , 00000000001.28.6, , , , , ] 2024-04-16T18:11:08.503 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000001.32.6, , , , 00000000001.28.6, , , , , ] 2024-04-16T18:11:08.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:08.506+0000 7fac83780980 0 bucket checkpoint complete 2024-04-16T18:11:08.533 DEBUG:boto:path=/lybtug-44/ 2024-04-16T18:11:08.533 DEBUG:boto:auth_path=/lybtug-44/ 2024-04-16T18:11:08.533 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.533 DEBUG:boto:Path: /lybtug-44/ 2024-04-16T18:11:08.533 DEBUG:boto:Data: 2024-04-16T18:11:08.533 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.534 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:08.534 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.534 DEBUG:boto:Params: {} 2024-04-16T18:11:08.534 DEBUG:boto:Token: None 2024-04-16T18:11:08.534 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT /lybtug-44/ 2024-04-16T18:11:08.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fNQUF5Dd+mOE4sNfTBCdH4/nl5Y= 2024-04-16T18:11:08.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fNQUF5Dd+mOE4sNfTBCdH4/nl5Y=', 'Content-Length': '0'} 2024-04-16T18:11:08.538 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '1024'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000009969d62d815c1f-00661ebf3c-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.538 DEBUG:boto:path=/lybtug-44/testobj-sse-c 2024-04-16T18:11:08.538 DEBUG:boto:auth_path=/lybtug-44/testobj-sse-c 2024-04-16T18:11:08.538 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.538 DEBUG:boto:Path: /lybtug-44/testobj-sse-c 2024-04-16T18:11:08.538 DEBUG:boto:Data: 2024-04-16T18:11:08.538 DEBUG:boto:Headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw=='} 2024-04-16T18:11:08.538 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:08.538 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.539 DEBUG:boto:Params: {} 2024-04-16T18:11:08.539 DEBUG:boto:Token: None 2024-04-16T18:11:08.539 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT x-amz-server-side-encryption-customer-algorithm:AES256 x-amz-server-side-encryption-customer-key:pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= x-amz-server-side-encryption-customer-key-md5:DWygnHRtgiJ77HCm+1rvHw== /lybtug-44/testobj-sse-c 2024-04-16T18:11:08.539 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cb3etTDQ6r7jRW65zwLJliZzvGc= 2024-04-16T18:11:08.539 DEBUG:boto:Final headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cb3etTDQ6r7jRW65zwLJliZzvGc=', 'Content-Length': '0'} 2024-04-16T18:11:08.541 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption-customer-algorithm', 'AES256'), ('x-amz-server-side-encryption-customer-key-MD5', 'DWygnHRtgiJ77HCm+1rvHw=='), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-44:054102a3-8d94-4596-8b3b-6341eab312e1.4804.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:04 GMT'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx00000ea383df37bf94545-00661ebf3c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.541 DEBUG:boto:path=/lybtug-44/testobj-sse-c 2024-04-16T18:11:08.541 DEBUG:boto:auth_path=/lybtug-44/testobj-sse-c 2024-04-16T18:11:08.541 DEBUG:boto:Method: GET 2024-04-16T18:11:08.541 DEBUG:boto:Path: /lybtug-44/testobj-sse-c 2024-04-16T18:11:08.541 DEBUG:boto:Data: 2024-04-16T18:11:08.541 DEBUG:boto:Headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw=='} 2024-04-16T18:11:08.541 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:08.542 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.542 DEBUG:boto:Params: {} 2024-04-16T18:11:08.542 DEBUG:boto:Token: None 2024-04-16T18:11:08.542 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:08 GMT x-amz-server-side-encryption-customer-algorithm:AES256 x-amz-server-side-encryption-customer-key:pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= x-amz-server-side-encryption-customer-key-md5:DWygnHRtgiJ77HCm+1rvHw== /lybtug-44/testobj-sse-c 2024-04-16T18:11:08.542 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9kP4utJe0WUMcJMd52CJtwqwd0o= 2024-04-16T18:11:08.542 DEBUG:boto:Final headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9kP4utJe0WUMcJMd52CJtwqwd0o=', 'Content-Length': '0'} 2024-04-16T18:11:08.544 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption-customer-algorithm', 'AES256'), ('x-amz-server-side-encryption-customer-key-MD5', 'DWygnHRtgiJ77HCm+1rvHw=='), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-44:054102a3-8d94-4596-8b3b-6341eab312e1.4804.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:04 GMT'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx0000066eb90564af56cab-00661ebf3c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.544 DEBUG:boto:path=/lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.545 DEBUG:boto:auth_path=/lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.545 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.545 DEBUG:boto:Path: /lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.545 DEBUG:boto:Data: 2024-04-16T18:11:08.545 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.545 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:08.545 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.545 DEBUG:boto:Params: {} 2024-04-16T18:11:08.545 DEBUG:boto:Token: None 2024-04-16T18:11:08.545 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT /lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RPNBcbOj2RhJVEOT4iqGs+uGDXU= 2024-04-16T18:11:08.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RPNBcbOj2RhJVEOT4iqGs+uGDXU=', 'Content-Length': '0'} 2024-04-16T18:11:08.548 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption', 'aws:kms'), ('x-amz-server-side-encryption-aws-kms-key-id', 'testkey-1'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-44:054102a3-8d94-4596-8b3b-6341eab312e1.4804.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:04 GMT'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx0000031b92d4a21b4abb0-00661ebf3c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.548 DEBUG:boto:path=/lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.548 DEBUG:boto:auth_path=/lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.548 DEBUG:boto:Method: GET 2024-04-16T18:11:08.549 DEBUG:boto:Path: /lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.549 DEBUG:boto:Data: 2024-04-16T18:11:08.549 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.549 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:08.549 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.549 DEBUG:boto:Params: {} 2024-04-16T18:11:08.549 DEBUG:boto:Token: None 2024-04-16T18:11:08.549 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:08 GMT /lybtug-44/testobj-sse-kms 2024-04-16T18:11:08.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YSM18u4Q/944CovQ/4b44WXTqIw= 2024-04-16T18:11:08.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YSM18u4Q/944CovQ/4b44WXTqIw=', 'Content-Length': '0'} 2024-04-16T18:11:08.552 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption', 'aws:kms'), ('x-amz-server-side-encryption-aws-kms-key-id', 'testkey-1'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:02 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-44:054102a3-8d94-4596-8b3b-6341eab312e1.4804.13'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:04 GMT'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx000004f3b018598d718f4-00661ebf3c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.552 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_encrypted_object_sync ... ok 2024-04-16T18:11:08.553 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-45 2024-04-16T18:11:08.553 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:08.553 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:08.553 DEBUG:boto:Method: PUT 2024-04-16T18:11:08.553 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:08.553 DEBUG:boto:Data: 2024-04-16T18:11:08.553 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.554 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.554 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.554 DEBUG:boto:Params: {} 2024-04-16T18:11:08.554 DEBUG:boto:Token: None 2024-04-16T18:11:08.554 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/ 2024-04-16T18:11:08.554 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0mpW/RInioZWlgrmglY73PE3YEg= 2024-04-16T18:11:08.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0mpW/RInioZWlgrmglY73PE3YEg=', 'Content-Length': '0'} 2024-04-16T18:11:08.589 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006e1c192c350283e7-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.590 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:08.590 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:08.590 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.590 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:08.590 DEBUG:boto:Data: 2024-04-16T18:11:08.590 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.590 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.590 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.590 DEBUG:boto:Params: {} 2024-04-16T18:11:08.591 DEBUG:boto:Token: None 2024-04-16T18:11:08.591 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/ 2024-04-16T18:11:08.591 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k= 2024-04-16T18:11:08.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k=', 'Content-Length': '0'} 2024-04-16T18:11:08.594 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000c3f53b972266e55-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.595 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:08.595 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:08.595 DEBUG:boto:Method: PUT 2024-04-16T18:11:08.595 DEBUG:boto:Path: /lybtug-45/a 2024-04-16T18:11:08.595 DEBUG:boto:Data: 2024-04-16T18:11:08.595 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:08.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.595 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.595 DEBUG:boto:Params: {} 2024-04-16T18:11:08.595 DEBUG:boto:Token: None 2024-04-16T18:11:08.596 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/a 2024-04-16T18:11:08.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NZgMa33/5JvVhpHv0hPL/XNQWiM= 2024-04-16T18:11:08.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NZgMa33/5JvVhpHv0hPL/XNQWiM='} 2024-04-16T18:11:08.638 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e81ef56bf08cf573-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.639 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:08.639 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:08.639 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.639 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:08.639 DEBUG:boto:Data: 2024-04-16T18:11:08.639 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.639 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.639 DEBUG:boto:Params: {} 2024-04-16T18:11:08.639 DEBUG:boto:Token: None 2024-04-16T18:11:08.640 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/ 2024-04-16T18:11:08.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k= 2024-04-16T18:11:08.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k=', 'Content-Length': '0'} 2024-04-16T18:11:08.643 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006467b84119463505-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.644 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:08.644 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:08.644 DEBUG:boto:Method: PUT 2024-04-16T18:11:08.644 DEBUG:boto:Path: /lybtug-45/b 2024-04-16T18:11:08.644 DEBUG:boto:Data: 2024-04-16T18:11:08.644 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:08.644 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.644 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.644 DEBUG:boto:Params: {} 2024-04-16T18:11:08.644 DEBUG:boto:Token: None 2024-04-16T18:11:08.645 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/b 2024-04-16T18:11:08.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xhl8y1hlRpAGeSlgKd24o+o27os= 2024-04-16T18:11:08.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xhl8y1hlRpAGeSlgKd24o+o27os='} 2024-04-16T18:11:08.657 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000059301576e2e37b7a-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.657 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:08.657 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:08.657 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.657 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:08.657 DEBUG:boto:Data: 2024-04-16T18:11:08.657 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.657 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.657 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.658 DEBUG:boto:Params: {} 2024-04-16T18:11:08.658 DEBUG:boto:Token: None 2024-04-16T18:11:08.658 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/ 2024-04-16T18:11:08.658 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k= 2024-04-16T18:11:08.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k=', 'Content-Length': '0'} 2024-04-16T18:11:08.662 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ea84d4b3c34bde4-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.662 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:08.662 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:08.663 DEBUG:boto:Method: PUT 2024-04-16T18:11:08.663 DEBUG:boto:Path: /lybtug-45/c 2024-04-16T18:11:08.663 DEBUG:boto:Data: 2024-04-16T18:11:08.663 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:08.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.663 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.663 DEBUG:boto:Params: {} 2024-04-16T18:11:08.663 DEBUG:boto:Token: None 2024-04-16T18:11:08.663 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/c 2024-04-16T18:11:08.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WmP1Y5cuiUy2U05O5FoAp2q1QoM= 2024-04-16T18:11:08.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WmP1Y5cuiUy2U05O5FoAp2q1QoM='} 2024-04-16T18:11:08.706 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000091ecf4f46fda4d66-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.707 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:08.707 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:08.707 DEBUG:boto:Method: HEAD 2024-04-16T18:11:08.707 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:08.707 DEBUG:boto:Data: 2024-04-16T18:11:08.707 DEBUG:boto:Headers: {} 2024-04-16T18:11:08.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.707 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.707 DEBUG:boto:Params: {} 2024-04-16T18:11:08.707 DEBUG:boto:Token: None 2024-04-16T18:11:08.708 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/ 2024-04-16T18:11:08.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k= 2024-04-16T18:11:08.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N9XOeIJqMeThCHDGl7nII8QQi+k=', 'Content-Length': '0'} 2024-04-16T18:11:08.711 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004ded7da10df10a11-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.711 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:08.712 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:08.712 DEBUG:boto:Method: PUT 2024-04-16T18:11:08.712 DEBUG:boto:Path: /lybtug-45/d 2024-04-16T18:11:08.712 DEBUG:boto:Data: 2024-04-16T18:11:08.712 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:08.712 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:08.712 DEBUG:boto:Port: 8000 2024-04-16T18:11:08.712 DEBUG:boto:Params: {} 2024-04-16T18:11:08.712 DEBUG:boto:Token: None 2024-04-16T18:11:08.712 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:08 GMT /lybtug-45/d 2024-04-16T18:11:08.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G55DSK5WihuPRhcNrtGEt42Rfrk= 2024-04-16T18:11:08.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G55DSK5WihuPRhcNrtGEt42Rfrk='} 2024-04-16T18:11:08.725 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000964e6fab95d36fca-00661ebf3c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:08.725 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:08.725 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:08.725 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:08.835 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:11:08.867 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291068.589571_441.1","last_update":"2024-04-16T18:11:08.589571Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:11:08.868 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291068.589571_441.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:08.868 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:11:08.868 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:08.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:08.868 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:08.911 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:08.911 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:08.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:08.986+0000 7f324adb7980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:08.986+0000 7f324adb7980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:09.017 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291068.570322_438.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:08.570322Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:08Z"} 2024-04-16T18:11:09.017 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291068.570322_438.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:08.570322Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:08Z"} 2024-04-16T18:11:09.017 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291068.570322_438.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:11:08.570322Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:11:09.017 DEBUG:rgw_multi.tests:log_status={0: '1_1713291068.589571_441.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:09.017 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291068.570322_438.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:09.017 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291068.589571_441.1 target=1_1713291068.570322_438.1 2024-04-16T18:11:14.019 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:14.019 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:14.019 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:14.062 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:14.062 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:14.134+0000 7fb5b3ea4980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:14.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:14.138+0000 7fb5b3ea4980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:14.166 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291068.589571_441.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:08.589571Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:14Z"} 2024-04-16T18:11:14.167 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291068.589571_441.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:08.589571Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:14Z"} 2024-04-16T18:11:14.167 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291068.589571_441.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:11:08.589571Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:11:14.167 DEBUG:rgw_multi.tests:log_status={0: '1_1713291068.589571_441.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:14.167 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291068.589571_441.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:14.167 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:11:14.167 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-45 2024-04-16T18:11:14.167 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-45', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:14.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-45', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:14.168 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-45 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:14.342+0000 7f701560b980 1 bucket sync caught up with source: 2024-04-16T18:11:14.339 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.102.6, , , , 00000000001.60.6, , , 00000000001.62.6, , 00000000001.27.6] 2024-04-16T18:11:14.340 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000001.102.6, , , , 00000000001.60.6, , , 00000000001.62.6, , 00000000001.27.6] 2024-04-16T18:11:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:14.342+0000 7f701560b980 0 bucket checkpoint complete 2024-04-16T18:11:14.370 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-45 2024-04-16T18:11:14.370 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-45', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:14.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-45', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:14.371 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-45 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:14.422 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:14.422 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:14.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:14.503+0000 7f5708921980 1 bucket sync caught up with source: 2024-04-16T18:11:14.501 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.886.6, , , , 00000000001.119.6, , , 00000000001.397.6, , 00000000001.891.6] 2024-04-16T18:11:14.502 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.886.6, , , , 00000000001.119.6, , , 00000000001.397.6, , 00000000001.891.6] 2024-04-16T18:11:14.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:14.503+0000 7f5708921980 0 bucket checkpoint complete 2024-04-16T18:11:14.532 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-45 zones={test-zone2, test-zone1} 2024-04-16T18:11:14.532 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.532 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.533 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.533 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:14.533 DEBUG:boto:Data: 2024-04-16T18:11:14.533 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.533 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.533 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.533 DEBUG:boto:Params: {} 2024-04-16T18:11:14.534 DEBUG:boto:Token: None 2024-04-16T18:11:14.534 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pFAXu7Jl0zQ/UZdXkhQrJGEhmaw= 2024-04-16T18:11:14.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pFAXu7Jl0zQ/UZdXkhQrJGEhmaw=', 'Content-Length': '0'} 2024-04-16T18:11:14.538 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000640ff5679732954c-00661ebf42-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.538 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.538 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.539 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.539 DEBUG:boto:Path: /lybtug-45/ 2024-04-16T18:11:14.539 DEBUG:boto:Data: 2024-04-16T18:11:14.539 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.539 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.539 DEBUG:boto:Params: {} 2024-04-16T18:11:14.539 DEBUG:boto:Token: None 2024-04-16T18:11:14.540 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pFAXu7Jl0zQ/UZdXkhQrJGEhmaw= 2024-04-16T18:11:14.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pFAXu7Jl0zQ/UZdXkhQrJGEhmaw=', 'Content-Length': '0'} 2024-04-16T18:11:14.543 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006233fb73fcb4b881-00661ebf42-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.544 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:11:14.544 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.544 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.544 DEBUG:boto:path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.544 DEBUG:boto:auth_path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.544 DEBUG:boto:Method: GET 2024-04-16T18:11:14.544 DEBUG:boto:Path: /lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.544 DEBUG:boto:Data: 2024-04-16T18:11:14.545 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.545 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.545 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.545 DEBUG:boto:Params: {} 2024-04-16T18:11:14.545 DEBUG:boto:Token: None 2024-04-16T18:11:14.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/?versions 2024-04-16T18:11:14.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA= 2024-04-16T18:11:14.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA=', 'Content-Length': '0'} 2024-04-16T18:11:14.550 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0ac98e244238feb-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.550 DEBUG:boto:b'lybtug-45999falseanulltrue2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.551 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:11:14.551 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:11:14.551 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:11:14.551 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:11:14.551 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:11:14.551 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.551 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.552 DEBUG:boto:path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.552 DEBUG:boto:auth_path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.552 DEBUG:boto:Method: GET 2024-04-16T18:11:14.552 DEBUG:boto:Path: /lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.552 DEBUG:boto:Data: 2024-04-16T18:11:14.552 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.552 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.552 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.552 DEBUG:boto:Params: {} 2024-04-16T18:11:14.552 DEBUG:boto:Token: None 2024-04-16T18:11:14.553 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/?versions 2024-04-16T18:11:14.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA= 2024-04-16T18:11:14.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA=', 'Content-Length': '0'} 2024-04-16T18:11:14.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004172d511ae3ad549-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.556 DEBUG:boto:b'lybtug-45999falseanulltrue2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.557 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:11:14.557 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:11:14.557 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:11:14.557 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:11:14.557 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.557 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.558 DEBUG:boto:path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.558 DEBUG:boto:auth_path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.558 DEBUG:boto:Method: GET 2024-04-16T18:11:14.558 DEBUG:boto:Path: /lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.558 DEBUG:boto:Data: 2024-04-16T18:11:14.558 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.558 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.558 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.559 DEBUG:boto:Params: {} 2024-04-16T18:11:14.559 DEBUG:boto:Token: None 2024-04-16T18:11:14.559 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/?versions 2024-04-16T18:11:14.559 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA= 2024-04-16T18:11:14.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA=', 'Content-Length': '0'} 2024-04-16T18:11:14.563 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b333bbc74620db8-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.563 DEBUG:boto:b'lybtug-45999falseanulltrue2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.564 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.564 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.564 DEBUG:boto:path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.564 DEBUG:boto:auth_path=/lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.564 DEBUG:boto:Method: GET 2024-04-16T18:11:14.564 DEBUG:boto:Path: /lybtug-45/?versions&max-keys=999 2024-04-16T18:11:14.565 DEBUG:boto:Data: 2024-04-16T18:11:14.565 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.565 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.565 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.565 DEBUG:boto:Params: {} 2024-04-16T18:11:14.565 DEBUG:boto:Token: None 2024-04-16T18:11:14.565 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/?versions 2024-04-16T18:11:14.566 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA= 2024-04-16T18:11:14.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHzbYUh/GlUL1ivQ01+Vxiu/uIA=', 'Content-Length': '0'} 2024-04-16T18:11:14.569 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e51832091a9cceba-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.569 DEBUG:boto:b'lybtug-45999falseanulltrue2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.569 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:14.570 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.570 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.570 DEBUG:boto:path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.570 DEBUG:boto:auth_path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.570 DEBUG:boto:Method: GET 2024-04-16T18:11:14.570 DEBUG:boto:Path: /lybtug-45/a?versionId=null 2024-04-16T18:11:14.570 DEBUG:boto:Data: 2024-04-16T18:11:14.571 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.571 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.571 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.571 DEBUG:boto:Params: {} 2024-04-16T18:11:14.571 DEBUG:boto:Token: None 2024-04-16T18:11:14.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a?versionId=null 2024-04-16T18:11:14.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI= 2024-04-16T18:11:14.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI=', 'Content-Length': '0'} 2024-04-16T18:11:14.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b234d6428cf8c82-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.618 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.619 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.619 DEBUG:boto:path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.619 DEBUG:boto:auth_path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.619 DEBUG:boto:Method: GET 2024-04-16T18:11:14.619 DEBUG:boto:Path: /lybtug-45/a?versionId=null 2024-04-16T18:11:14.619 DEBUG:boto:Data: 2024-04-16T18:11:14.619 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.620 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.620 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.620 DEBUG:boto:Params: {} 2024-04-16T18:11:14.620 DEBUG:boto:Token: None 2024-04-16T18:11:14.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a?versionId=null 2024-04-16T18:11:14.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI= 2024-04-16T18:11:14.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI=', 'Content-Length': '0'} 2024-04-16T18:11:14.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a09ccdebaab5ba82-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.624 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.624 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.624 DEBUG:boto:path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.625 DEBUG:boto:auth_path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.625 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.625 DEBUG:boto:Path: /lybtug-45/a?versionId=null 2024-04-16T18:11:14.625 DEBUG:boto:Data: 2024-04-16T18:11:14.625 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.625 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.625 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.625 DEBUG:boto:Params: {} 2024-04-16T18:11:14.625 DEBUG:boto:Token: None 2024-04-16T18:11:14.626 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a?versionId=null 2024-04-16T18:11:14.626 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l+4OjvyWudH0uqF3DqEEdxyHHSs= 2024-04-16T18:11:14.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l+4OjvyWudH0uqF3DqEEdxyHHSs=', 'Content-Length': '0'} 2024-04-16T18:11:14.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7d17d5875b5e288-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.629 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.630 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.630 DEBUG:boto:path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.630 DEBUG:boto:auth_path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.630 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.630 DEBUG:boto:Path: /lybtug-45/a?versionId=null 2024-04-16T18:11:14.630 DEBUG:boto:Data: 2024-04-16T18:11:14.630 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.630 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.631 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.631 DEBUG:boto:Params: {} 2024-04-16T18:11:14.631 DEBUG:boto:Token: None 2024-04-16T18:11:14.631 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a?versionId=null 2024-04-16T18:11:14.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l+4OjvyWudH0uqF3DqEEdxyHHSs= 2024-04-16T18:11:14.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l+4OjvyWudH0uqF3DqEEdxyHHSs=', 'Content-Length': '0'} 2024-04-16T18:11:14.634 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001173b57c41517f0a-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.634 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:14.635 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.635 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.635 DEBUG:boto:path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.635 DEBUG:boto:auth_path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.635 DEBUG:boto:Method: GET 2024-04-16T18:11:14.635 DEBUG:boto:Path: /lybtug-45/a?versionId=null 2024-04-16T18:11:14.635 DEBUG:boto:Data: 2024-04-16T18:11:14.635 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.635 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.636 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.636 DEBUG:boto:Params: {} 2024-04-16T18:11:14.636 DEBUG:boto:Token: None 2024-04-16T18:11:14.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a?versionId=null 2024-04-16T18:11:14.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI= 2024-04-16T18:11:14.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI=', 'Content-Length': '0'} 2024-04-16T18:11:14.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f481db82a0e10828-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.640 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.640 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.640 DEBUG:boto:path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.640 DEBUG:boto:auth_path=/lybtug-45/a?versionId=null 2024-04-16T18:11:14.640 DEBUG:boto:Method: GET 2024-04-16T18:11:14.640 DEBUG:boto:Path: /lybtug-45/a?versionId=null 2024-04-16T18:11:14.640 DEBUG:boto:Data: 2024-04-16T18:11:14.640 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.640 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.640 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.641 DEBUG:boto:Params: {} 2024-04-16T18:11:14.641 DEBUG:boto:Token: None 2024-04-16T18:11:14.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a?versionId=null 2024-04-16T18:11:14.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI= 2024-04-16T18:11:14.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IxuLGba08uAZcmzOPA0s+UhXpMI=', 'Content-Length': '0'} 2024-04-16T18:11:14.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6bd1066920df9c2-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.645 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.645 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.645 DEBUG:boto:path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.645 DEBUG:boto:auth_path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.645 DEBUG:boto:Method: GET 2024-04-16T18:11:14.645 DEBUG:boto:Path: /lybtug-45/?prefix=a 2024-04-16T18:11:14.645 DEBUG:boto:Data: 2024-04-16T18:11:14.645 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.646 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.646 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.646 DEBUG:boto:Params: {} 2024-04-16T18:11:14.646 DEBUG:boto:Token: None 2024-04-16T18:11:14.646 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.646 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016d29cdbdf2644a7-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.650 DEBUG:boto:b'lybtug-45a1000falsea2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.650 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.651 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.651 DEBUG:boto:path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.651 DEBUG:boto:auth_path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.651 DEBUG:boto:Method: GET 2024-04-16T18:11:14.651 DEBUG:boto:Path: /lybtug-45/?prefix=a 2024-04-16T18:11:14.651 DEBUG:boto:Data: 2024-04-16T18:11:14.651 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.651 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.651 DEBUG:boto:Params: {} 2024-04-16T18:11:14.652 DEBUG:boto:Token: None 2024-04-16T18:11:14.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.656 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8079c46b72e9d8b-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.656 DEBUG:boto:b'lybtug-45a1000falsea2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.656 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.657 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.657 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.657 DEBUG:boto:Path: /lybtug-45/a 2024-04-16T18:11:14.657 DEBUG:boto:Data: 2024-04-16T18:11:14.657 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.657 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.657 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.657 DEBUG:boto:Params: {} 2024-04-16T18:11:14.657 DEBUG:boto:Token: None 2024-04-16T18:11:14.657 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a 2024-04-16T18:11:14.658 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r2Rv0eUzr4SBBCUi5Ls+OHSosq8= 2024-04-16T18:11:14.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r2Rv0eUzr4SBBCUi5Ls+OHSosq8=', 'Content-Length': '0'} 2024-04-16T18:11:14.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9d780c65f348ee0-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.661 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.661 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.661 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.661 DEBUG:boto:Path: /lybtug-45/a 2024-04-16T18:11:14.661 DEBUG:boto:Data: 2024-04-16T18:11:14.661 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.661 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.661 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.661 DEBUG:boto:Params: {} 2024-04-16T18:11:14.662 DEBUG:boto:Token: None 2024-04-16T18:11:14.662 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a 2024-04-16T18:11:14.662 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r2Rv0eUzr4SBBCUi5Ls+OHSosq8= 2024-04-16T18:11:14.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r2Rv0eUzr4SBBCUi5Ls+OHSosq8=', 'Content-Length': '0'} 2024-04-16T18:11:14.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d4458ae6f234d9c-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:14.665 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.665 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.665 DEBUG:boto:Method: GET 2024-04-16T18:11:14.665 DEBUG:boto:Path: /lybtug-45/a 2024-04-16T18:11:14.665 DEBUG:boto:Data: 2024-04-16T18:11:14.665 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.666 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.666 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.666 DEBUG:boto:Params: {} 2024-04-16T18:11:14.666 DEBUG:boto:Token: None 2024-04-16T18:11:14.666 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a 2024-04-16T18:11:14.666 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3rXmMmorst1fhqIBeqKiZj+oyRo= 2024-04-16T18:11:14.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3rXmMmorst1fhqIBeqKiZj+oyRo=', 'Content-Length': '0'} 2024-04-16T18:11:14.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000deb7c9fbed7fcbc8-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.710 DEBUG:boto:path=/lybtug-45/a 2024-04-16T18:11:14.711 DEBUG:boto:auth_path=/lybtug-45/a 2024-04-16T18:11:14.711 DEBUG:boto:Method: GET 2024-04-16T18:11:14.711 DEBUG:boto:Path: /lybtug-45/a 2024-04-16T18:11:14.711 DEBUG:boto:Data: 2024-04-16T18:11:14.711 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.711 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.711 DEBUG:boto:Params: {} 2024-04-16T18:11:14.711 DEBUG:boto:Token: None 2024-04-16T18:11:14.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/a 2024-04-16T18:11:14.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3rXmMmorst1fhqIBeqKiZj+oyRo= 2024-04-16T18:11:14.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3rXmMmorst1fhqIBeqKiZj+oyRo=', 'Content-Length': '0'} 2024-04-16T18:11:14.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004162be066dc94965-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.715 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.715 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.715 DEBUG:boto:path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.715 DEBUG:boto:auth_path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.715 DEBUG:boto:Method: GET 2024-04-16T18:11:14.715 DEBUG:boto:Path: /lybtug-45/?prefix=a 2024-04-16T18:11:14.716 DEBUG:boto:Data: 2024-04-16T18:11:14.716 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.716 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.716 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.716 DEBUG:boto:Params: {} 2024-04-16T18:11:14.716 DEBUG:boto:Token: None 2024-04-16T18:11:14.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f691dc846a055ada-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.762 DEBUG:boto:b'lybtug-45a1000falsea2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.763 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.763 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.763 DEBUG:boto:path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.763 DEBUG:boto:auth_path=/lybtug-45/?prefix=a 2024-04-16T18:11:14.763 DEBUG:boto:Method: GET 2024-04-16T18:11:14.763 DEBUG:boto:Path: /lybtug-45/?prefix=a 2024-04-16T18:11:14.763 DEBUG:boto:Data: 2024-04-16T18:11:14.763 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.763 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.764 DEBUG:boto:Params: {} 2024-04-16T18:11:14.764 DEBUG:boto:Token: None 2024-04-16T18:11:14.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040d7a3ce97294e4c-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.768 DEBUG:boto:b'lybtug-45a1000falsea2024-04-16T18:11:08.603Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.769 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:14.769 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.769 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.769 DEBUG:boto:path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.769 DEBUG:boto:auth_path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.769 DEBUG:boto:Method: GET 2024-04-16T18:11:14.769 DEBUG:boto:Path: /lybtug-45/b?versionId=null 2024-04-16T18:11:14.769 DEBUG:boto:Data: 2024-04-16T18:11:14.769 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.769 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.769 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.770 DEBUG:boto:Params: {} 2024-04-16T18:11:14.770 DEBUG:boto:Token: None 2024-04-16T18:11:14.770 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b?versionId=null 2024-04-16T18:11:14.770 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM= 2024-04-16T18:11:14.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM=', 'Content-Length': '0'} 2024-04-16T18:11:14.773 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d72181ae55335d8-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.814 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.815 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.815 DEBUG:boto:path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.815 DEBUG:boto:auth_path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.815 DEBUG:boto:Method: GET 2024-04-16T18:11:14.815 DEBUG:boto:Path: /lybtug-45/b?versionId=null 2024-04-16T18:11:14.815 DEBUG:boto:Data: 2024-04-16T18:11:14.815 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.815 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.815 DEBUG:boto:Params: {} 2024-04-16T18:11:14.815 DEBUG:boto:Token: None 2024-04-16T18:11:14.816 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b?versionId=null 2024-04-16T18:11:14.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM= 2024-04-16T18:11:14.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM=', 'Content-Length': '0'} 2024-04-16T18:11:14.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006009f058d8de6dd5-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.819 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.819 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.819 DEBUG:boto:path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.819 DEBUG:boto:auth_path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.820 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.820 DEBUG:boto:Path: /lybtug-45/b?versionId=null 2024-04-16T18:11:14.820 DEBUG:boto:Data: 2024-04-16T18:11:14.820 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.820 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.820 DEBUG:boto:Params: {} 2024-04-16T18:11:14.820 DEBUG:boto:Token: None 2024-04-16T18:11:14.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b?versionId=null 2024-04-16T18:11:14.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iabcAV4PyLD5weC191pPiMSQXBk= 2024-04-16T18:11:14.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iabcAV4PyLD5weC191pPiMSQXBk=', 'Content-Length': '0'} 2024-04-16T18:11:14.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054b6ab34766ab82a-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.824 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.824 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.824 DEBUG:boto:path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.824 DEBUG:boto:auth_path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.824 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.824 DEBUG:boto:Path: /lybtug-45/b?versionId=null 2024-04-16T18:11:14.825 DEBUG:boto:Data: 2024-04-16T18:11:14.825 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.825 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.825 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.825 DEBUG:boto:Params: {} 2024-04-16T18:11:14.825 DEBUG:boto:Token: None 2024-04-16T18:11:14.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b?versionId=null 2024-04-16T18:11:14.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iabcAV4PyLD5weC191pPiMSQXBk= 2024-04-16T18:11:14.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iabcAV4PyLD5weC191pPiMSQXBk=', 'Content-Length': '0'} 2024-04-16T18:11:14.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008366af51076772bd-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.829 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:14.829 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.829 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.829 DEBUG:boto:path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.829 DEBUG:boto:auth_path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.829 DEBUG:boto:Method: GET 2024-04-16T18:11:14.829 DEBUG:boto:Path: /lybtug-45/b?versionId=null 2024-04-16T18:11:14.829 DEBUG:boto:Data: 2024-04-16T18:11:14.829 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.830 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.830 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.830 DEBUG:boto:Params: {} 2024-04-16T18:11:14.830 DEBUG:boto:Token: None 2024-04-16T18:11:14.830 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b?versionId=null 2024-04-16T18:11:14.830 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM= 2024-04-16T18:11:14.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM=', 'Content-Length': '0'} 2024-04-16T18:11:14.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dedc9aa4ceea97c8-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.874 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.875 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.875 DEBUG:boto:path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.875 DEBUG:boto:auth_path=/lybtug-45/b?versionId=null 2024-04-16T18:11:14.875 DEBUG:boto:Method: GET 2024-04-16T18:11:14.875 DEBUG:boto:Path: /lybtug-45/b?versionId=null 2024-04-16T18:11:14.875 DEBUG:boto:Data: 2024-04-16T18:11:14.875 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.875 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.875 DEBUG:boto:Params: {} 2024-04-16T18:11:14.875 DEBUG:boto:Token: None 2024-04-16T18:11:14.876 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b?versionId=null 2024-04-16T18:11:14.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM= 2024-04-16T18:11:14.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VN8CqueiscdKz1gyZzJ6DEoYRfM=', 'Content-Length': '0'} 2024-04-16T18:11:14.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000519d0e7775ceaf5a-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.879 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.879 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.879 DEBUG:boto:path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.879 DEBUG:boto:auth_path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.879 DEBUG:boto:Method: GET 2024-04-16T18:11:14.880 DEBUG:boto:Path: /lybtug-45/?prefix=b 2024-04-16T18:11:14.880 DEBUG:boto:Data: 2024-04-16T18:11:14.880 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.880 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.880 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.880 DEBUG:boto:Params: {} 2024-04-16T18:11:14.880 DEBUG:boto:Token: None 2024-04-16T18:11:14.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f917637478ab2ed0-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.885 DEBUG:boto:b'lybtug-45b1000falseb2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.885 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.885 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.885 DEBUG:boto:path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.885 DEBUG:boto:auth_path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.885 DEBUG:boto:Method: GET 2024-04-16T18:11:14.885 DEBUG:boto:Path: /lybtug-45/?prefix=b 2024-04-16T18:11:14.886 DEBUG:boto:Data: 2024-04-16T18:11:14.886 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.886 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.886 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.886 DEBUG:boto:Params: {} 2024-04-16T18:11:14.886 DEBUG:boto:Token: None 2024-04-16T18:11:14.886 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061ace2bdb4022fb5-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.890 DEBUG:boto:b'lybtug-45b1000falseb2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.891 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.891 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.891 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.891 DEBUG:boto:Path: /lybtug-45/b 2024-04-16T18:11:14.891 DEBUG:boto:Data: 2024-04-16T18:11:14.891 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.891 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.891 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.891 DEBUG:boto:Params: {} 2024-04-16T18:11:14.891 DEBUG:boto:Token: None 2024-04-16T18:11:14.892 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b 2024-04-16T18:11:14.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R6zCGk7jUNkevtb7Kocz7qFuYa8= 2024-04-16T18:11:14.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R6zCGk7jUNkevtb7Kocz7qFuYa8=', 'Content-Length': '0'} 2024-04-16T18:11:14.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000158782bc11125613-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.895 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.895 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.895 DEBUG:boto:Method: HEAD 2024-04-16T18:11:14.895 DEBUG:boto:Path: /lybtug-45/b 2024-04-16T18:11:14.895 DEBUG:boto:Data: 2024-04-16T18:11:14.895 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.895 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.896 DEBUG:boto:Params: {} 2024-04-16T18:11:14.896 DEBUG:boto:Token: None 2024-04-16T18:11:14.896 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b 2024-04-16T18:11:14.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R6zCGk7jUNkevtb7Kocz7qFuYa8= 2024-04-16T18:11:14.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R6zCGk7jUNkevtb7Kocz7qFuYa8=', 'Content-Length': '0'} 2024-04-16T18:11:14.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ad2179842d045ee-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.900 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:14.900 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.900 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.900 DEBUG:boto:Method: GET 2024-04-16T18:11:14.900 DEBUG:boto:Path: /lybtug-45/b 2024-04-16T18:11:14.900 DEBUG:boto:Data: 2024-04-16T18:11:14.900 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.900 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.900 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.900 DEBUG:boto:Params: {} 2024-04-16T18:11:14.901 DEBUG:boto:Token: None 2024-04-16T18:11:14.901 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b 2024-04-16T18:11:14.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rNTG1xh4hTfnR34vnjTZRT+ddSI= 2024-04-16T18:11:14.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rNTG1xh4hTfnR34vnjTZRT+ddSI=', 'Content-Length': '0'} 2024-04-16T18:11:14.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3f6257a3a57eed1-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.946 DEBUG:boto:path=/lybtug-45/b 2024-04-16T18:11:14.946 DEBUG:boto:auth_path=/lybtug-45/b 2024-04-16T18:11:14.947 DEBUG:boto:Method: GET 2024-04-16T18:11:14.947 DEBUG:boto:Path: /lybtug-45/b 2024-04-16T18:11:14.947 DEBUG:boto:Data: 2024-04-16T18:11:14.947 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.947 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.947 DEBUG:boto:Params: {} 2024-04-16T18:11:14.947 DEBUG:boto:Token: None 2024-04-16T18:11:14.947 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/b 2024-04-16T18:11:14.947 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rNTG1xh4hTfnR34vnjTZRT+ddSI= 2024-04-16T18:11:14.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rNTG1xh4hTfnR34vnjTZRT+ddSI=', 'Content-Length': '0'} 2024-04-16T18:11:14.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017ab32b67270662e-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.951 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.951 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.951 DEBUG:boto:path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.951 DEBUG:boto:auth_path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.951 DEBUG:boto:Method: GET 2024-04-16T18:11:14.952 DEBUG:boto:Path: /lybtug-45/?prefix=b 2024-04-16T18:11:14.952 DEBUG:boto:Data: 2024-04-16T18:11:14.952 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.952 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.952 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.952 DEBUG:boto:Params: {} 2024-04-16T18:11:14.952 DEBUG:boto:Token: None 2024-04-16T18:11:14.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.956 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c685b66fcf06f26-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.956 DEBUG:boto:b'lybtug-45b1000falseb2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.957 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:14.957 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:14.957 DEBUG:boto:path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.957 DEBUG:boto:auth_path=/lybtug-45/?prefix=b 2024-04-16T18:11:14.957 DEBUG:boto:Method: GET 2024-04-16T18:11:14.957 DEBUG:boto:Path: /lybtug-45/?prefix=b 2024-04-16T18:11:14.958 DEBUG:boto:Data: 2024-04-16T18:11:14.958 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.958 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:14.958 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.958 DEBUG:boto:Params: {} 2024-04-16T18:11:14.958 DEBUG:boto:Token: None 2024-04-16T18:11:14.958 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/ 2024-04-16T18:11:14.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8= 2024-04-16T18:11:14.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2+0+pEWWCo3qlR06pplNxBdYG8=', 'Content-Length': '0'} 2024-04-16T18:11:14.962 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f31512c40cca49a-00661ebf42-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:14.963 DEBUG:boto:b'lybtug-45b1000falseb2024-04-16T18:11:08.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:14.963 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:14.963 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:14.963 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:14.963 DEBUG:boto:path=/lybtug-45/c?versionId=null 2024-04-16T18:11:14.963 DEBUG:boto:auth_path=/lybtug-45/c?versionId=null 2024-04-16T18:11:14.963 DEBUG:boto:Method: GET 2024-04-16T18:11:14.964 DEBUG:boto:Path: /lybtug-45/c?versionId=null 2024-04-16T18:11:14.964 DEBUG:boto:Data: 2024-04-16T18:11:14.964 DEBUG:boto:Headers: {} 2024-04-16T18:11:14.964 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:14.964 DEBUG:boto:Port: 8000 2024-04-16T18:11:14.964 DEBUG:boto:Params: {} 2024-04-16T18:11:14.964 DEBUG:boto:Token: None 2024-04-16T18:11:14.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:14 GMT /lybtug-45/c?versionId=null 2024-04-16T18:11:14.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o15RT8e84khnXJEZ+5vfUyE9bZw= 2024-04-16T18:11:14.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o15RT8e84khnXJEZ+5vfUyE9bZw=', 'Content-Length': '0'} 2024-04-16T18:11:14.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005647923790875f1-00661ebf42-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.010 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.011 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.011 DEBUG:boto:path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.011 DEBUG:boto:auth_path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.011 DEBUG:boto:Method: GET 2024-04-16T18:11:15.011 DEBUG:boto:Path: /lybtug-45/c?versionId=null 2024-04-16T18:11:15.011 DEBUG:boto:Data: 2024-04-16T18:11:15.011 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.011 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.011 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.011 DEBUG:boto:Params: {} 2024-04-16T18:11:15.012 DEBUG:boto:Token: None 2024-04-16T18:11:15.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c?versionId=null 2024-04-16T18:11:15.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AhHkXaQ8cz6WRSr+eA9X6fD1Q/w= 2024-04-16T18:11:15.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AhHkXaQ8cz6WRSr+eA9X6fD1Q/w=', 'Content-Length': '0'} 2024-04-16T18:11:15.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000095a6ce3f9dc7126-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.016 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.016 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.016 DEBUG:boto:path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.016 DEBUG:boto:auth_path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.016 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.016 DEBUG:boto:Path: /lybtug-45/c?versionId=null 2024-04-16T18:11:15.016 DEBUG:boto:Data: 2024-04-16T18:11:15.016 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.016 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.016 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.017 DEBUG:boto:Params: {} 2024-04-16T18:11:15.017 DEBUG:boto:Token: None 2024-04-16T18:11:15.017 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c?versionId=null 2024-04-16T18:11:15.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esOnmUg9f+w08qq5AkOaip2umD4= 2024-04-16T18:11:15.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esOnmUg9f+w08qq5AkOaip2umD4=', 'Content-Length': '0'} 2024-04-16T18:11:15.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a651e06f7986a685-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.020 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.021 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.021 DEBUG:boto:path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.021 DEBUG:boto:auth_path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.021 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.021 DEBUG:boto:Path: /lybtug-45/c?versionId=null 2024-04-16T18:11:15.021 DEBUG:boto:Data: 2024-04-16T18:11:15.021 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.021 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.021 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.021 DEBUG:boto:Params: {} 2024-04-16T18:11:15.021 DEBUG:boto:Token: None 2024-04-16T18:11:15.022 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c?versionId=null 2024-04-16T18:11:15.022 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:esOnmUg9f+w08qq5AkOaip2umD4= 2024-04-16T18:11:15.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:esOnmUg9f+w08qq5AkOaip2umD4=', 'Content-Length': '0'} 2024-04-16T18:11:15.025 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f3821b5ff7bab58-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.025 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:15.025 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.026 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.026 DEBUG:boto:path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.026 DEBUG:boto:auth_path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.026 DEBUG:boto:Method: GET 2024-04-16T18:11:15.026 DEBUG:boto:Path: /lybtug-45/c?versionId=null 2024-04-16T18:11:15.026 DEBUG:boto:Data: 2024-04-16T18:11:15.026 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.026 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.026 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.026 DEBUG:boto:Params: {} 2024-04-16T18:11:15.027 DEBUG:boto:Token: None 2024-04-16T18:11:15.027 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c?versionId=null 2024-04-16T18:11:15.027 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AhHkXaQ8cz6WRSr+eA9X6fD1Q/w= 2024-04-16T18:11:15.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AhHkXaQ8cz6WRSr+eA9X6fD1Q/w=', 'Content-Length': '0'} 2024-04-16T18:11:15.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a384fdd6525896c6-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.071 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.071 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.071 DEBUG:boto:path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.071 DEBUG:boto:auth_path=/lybtug-45/c?versionId=null 2024-04-16T18:11:15.071 DEBUG:boto:Method: GET 2024-04-16T18:11:15.071 DEBUG:boto:Path: /lybtug-45/c?versionId=null 2024-04-16T18:11:15.071 DEBUG:boto:Data: 2024-04-16T18:11:15.071 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.071 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.072 DEBUG:boto:Params: {} 2024-04-16T18:11:15.072 DEBUG:boto:Token: None 2024-04-16T18:11:15.072 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c?versionId=null 2024-04-16T18:11:15.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AhHkXaQ8cz6WRSr+eA9X6fD1Q/w= 2024-04-16T18:11:15.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AhHkXaQ8cz6WRSr+eA9X6fD1Q/w=', 'Content-Length': '0'} 2024-04-16T18:11:15.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002551651e70467fe4-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.076 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.076 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.076 DEBUG:boto:path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.076 DEBUG:boto:auth_path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.076 DEBUG:boto:Method: GET 2024-04-16T18:11:15.076 DEBUG:boto:Path: /lybtug-45/?prefix=c 2024-04-16T18:11:15.076 DEBUG:boto:Data: 2024-04-16T18:11:15.076 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.077 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.077 DEBUG:boto:Params: {} 2024-04-16T18:11:15.077 DEBUG:boto:Token: None 2024-04-16T18:11:15.077 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.077 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.082 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000712dbe981fc29c45-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.082 DEBUG:boto:b'lybtug-45c1000falsec2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.082 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.082 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.082 DEBUG:boto:path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.083 DEBUG:boto:auth_path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.083 DEBUG:boto:Method: GET 2024-04-16T18:11:15.083 DEBUG:boto:Path: /lybtug-45/?prefix=c 2024-04-16T18:11:15.083 DEBUG:boto:Data: 2024-04-16T18:11:15.083 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.083 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.083 DEBUG:boto:Params: {} 2024-04-16T18:11:15.083 DEBUG:boto:Token: None 2024-04-16T18:11:15.083 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.088 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059d41d1402330684-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.130 DEBUG:boto:b'lybtug-45c1000falsec2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.131 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.131 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.131 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.131 DEBUG:boto:Path: /lybtug-45/c 2024-04-16T18:11:15.131 DEBUG:boto:Data: 2024-04-16T18:11:15.131 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.131 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.131 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.132 DEBUG:boto:Params: {} 2024-04-16T18:11:15.132 DEBUG:boto:Token: None 2024-04-16T18:11:15.132 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c 2024-04-16T18:11:15.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:34/mPMgTZP6o9WQmUT35tiFBsps= 2024-04-16T18:11:15.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:34/mPMgTZP6o9WQmUT35tiFBsps=', 'Content-Length': '0'} 2024-04-16T18:11:15.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040610037bf27bf2c-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.135 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.135 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.135 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.135 DEBUG:boto:Path: /lybtug-45/c 2024-04-16T18:11:15.135 DEBUG:boto:Data: 2024-04-16T18:11:15.135 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.136 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.136 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.136 DEBUG:boto:Params: {} 2024-04-16T18:11:15.136 DEBUG:boto:Token: None 2024-04-16T18:11:15.136 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c 2024-04-16T18:11:15.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:34/mPMgTZP6o9WQmUT35tiFBsps= 2024-04-16T18:11:15.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:34/mPMgTZP6o9WQmUT35tiFBsps=', 'Content-Length': '0'} 2024-04-16T18:11:15.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092122087ffe560af-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:15.140 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.140 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.140 DEBUG:boto:Method: GET 2024-04-16T18:11:15.140 DEBUG:boto:Path: /lybtug-45/c 2024-04-16T18:11:15.140 DEBUG:boto:Data: 2024-04-16T18:11:15.140 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.140 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.140 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.140 DEBUG:boto:Params: {} 2024-04-16T18:11:15.140 DEBUG:boto:Token: None 2024-04-16T18:11:15.141 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c 2024-04-16T18:11:15.141 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sgrkaIlMHEv2Zo6R/RL2We4+/zk= 2024-04-16T18:11:15.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sgrkaIlMHEv2Zo6R/RL2We4+/zk=', 'Content-Length': '0'} 2024-04-16T18:11:15.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075cc7f3bb23a3c6b-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.186 DEBUG:boto:path=/lybtug-45/c 2024-04-16T18:11:15.186 DEBUG:boto:auth_path=/lybtug-45/c 2024-04-16T18:11:15.187 DEBUG:boto:Method: GET 2024-04-16T18:11:15.187 DEBUG:boto:Path: /lybtug-45/c 2024-04-16T18:11:15.187 DEBUG:boto:Data: 2024-04-16T18:11:15.187 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.187 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.187 DEBUG:boto:Params: {} 2024-04-16T18:11:15.187 DEBUG:boto:Token: None 2024-04-16T18:11:15.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/c 2024-04-16T18:11:15.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sgrkaIlMHEv2Zo6R/RL2We4+/zk= 2024-04-16T18:11:15.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sgrkaIlMHEv2Zo6R/RL2We4+/zk=', 'Content-Length': '0'} 2024-04-16T18:11:15.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019d13ea7898c2f0c-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.191 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.191 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.191 DEBUG:boto:path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.191 DEBUG:boto:auth_path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.191 DEBUG:boto:Method: GET 2024-04-16T18:11:15.191 DEBUG:boto:Path: /lybtug-45/?prefix=c 2024-04-16T18:11:15.191 DEBUG:boto:Data: 2024-04-16T18:11:15.192 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.192 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.192 DEBUG:boto:Params: {} 2024-04-16T18:11:15.192 DEBUG:boto:Token: None 2024-04-16T18:11:15.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b9fbb1bd8ec1a23-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.238 DEBUG:boto:b'lybtug-45c1000falsec2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.239 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.239 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.239 DEBUG:boto:path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.239 DEBUG:boto:auth_path=/lybtug-45/?prefix=c 2024-04-16T18:11:15.239 DEBUG:boto:Method: GET 2024-04-16T18:11:15.239 DEBUG:boto:Path: /lybtug-45/?prefix=c 2024-04-16T18:11:15.239 DEBUG:boto:Data: 2024-04-16T18:11:15.239 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.239 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.240 DEBUG:boto:Params: {} 2024-04-16T18:11:15.240 DEBUG:boto:Token: None 2024-04-16T18:11:15.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d36f323504d90baf-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.245 DEBUG:boto:b'lybtug-45c1000falsec2024-04-16T18:11:08.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:15.245 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.245 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.245 DEBUG:boto:path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.245 DEBUG:boto:auth_path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.246 DEBUG:boto:Method: GET 2024-04-16T18:11:15.246 DEBUG:boto:Path: /lybtug-45/d?versionId=null 2024-04-16T18:11:15.246 DEBUG:boto:Data: 2024-04-16T18:11:15.246 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.246 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.246 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.246 DEBUG:boto:Params: {} 2024-04-16T18:11:15.246 DEBUG:boto:Token: None 2024-04-16T18:11:15.246 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d?versionId=null 2024-04-16T18:11:15.246 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI= 2024-04-16T18:11:15.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI=', 'Content-Length': '0'} 2024-04-16T18:11:15.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cae053c96fc7c2c-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.290 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.291 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.291 DEBUG:boto:path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.291 DEBUG:boto:auth_path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.291 DEBUG:boto:Method: GET 2024-04-16T18:11:15.291 DEBUG:boto:Path: /lybtug-45/d?versionId=null 2024-04-16T18:11:15.291 DEBUG:boto:Data: 2024-04-16T18:11:15.291 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.291 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.291 DEBUG:boto:Params: {} 2024-04-16T18:11:15.292 DEBUG:boto:Token: None 2024-04-16T18:11:15.292 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d?versionId=null 2024-04-16T18:11:15.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI= 2024-04-16T18:11:15.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI=', 'Content-Length': '0'} 2024-04-16T18:11:15.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004601c83125f63f91-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.296 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.296 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.296 DEBUG:boto:path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.296 DEBUG:boto:auth_path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.296 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.296 DEBUG:boto:Path: /lybtug-45/d?versionId=null 2024-04-16T18:11:15.296 DEBUG:boto:Data: 2024-04-16T18:11:15.296 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.297 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.297 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.297 DEBUG:boto:Params: {} 2024-04-16T18:11:15.297 DEBUG:boto:Token: None 2024-04-16T18:11:15.297 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d?versionId=null 2024-04-16T18:11:15.297 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AyL9fSRv6xGIokX9UDmdwt5LDGI= 2024-04-16T18:11:15.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AyL9fSRv6xGIokX9UDmdwt5LDGI=', 'Content-Length': '0'} 2024-04-16T18:11:15.300 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c9807ca0ff0459c-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.301 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.301 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.301 DEBUG:boto:path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.301 DEBUG:boto:auth_path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.301 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.301 DEBUG:boto:Path: /lybtug-45/d?versionId=null 2024-04-16T18:11:15.301 DEBUG:boto:Data: 2024-04-16T18:11:15.301 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.301 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.301 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.302 DEBUG:boto:Params: {} 2024-04-16T18:11:15.302 DEBUG:boto:Token: None 2024-04-16T18:11:15.302 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d?versionId=null 2024-04-16T18:11:15.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AyL9fSRv6xGIokX9UDmdwt5LDGI= 2024-04-16T18:11:15.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AyL9fSRv6xGIokX9UDmdwt5LDGI=', 'Content-Length': '0'} 2024-04-16T18:11:15.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0cc8da767a0bd85-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.305 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:15.305 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.305 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.305 DEBUG:boto:path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.305 DEBUG:boto:auth_path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.305 DEBUG:boto:Method: GET 2024-04-16T18:11:15.306 DEBUG:boto:Path: /lybtug-45/d?versionId=null 2024-04-16T18:11:15.306 DEBUG:boto:Data: 2024-04-16T18:11:15.306 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.306 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.306 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.306 DEBUG:boto:Params: {} 2024-04-16T18:11:15.306 DEBUG:boto:Token: None 2024-04-16T18:11:15.306 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d?versionId=null 2024-04-16T18:11:15.306 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI= 2024-04-16T18:11:15.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI=', 'Content-Length': '0'} 2024-04-16T18:11:15.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da7b29612dca4b47-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.350 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.351 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.351 DEBUG:boto:path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.351 DEBUG:boto:auth_path=/lybtug-45/d?versionId=null 2024-04-16T18:11:15.351 DEBUG:boto:Method: GET 2024-04-16T18:11:15.351 DEBUG:boto:Path: /lybtug-45/d?versionId=null 2024-04-16T18:11:15.351 DEBUG:boto:Data: 2024-04-16T18:11:15.351 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.351 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.351 DEBUG:boto:Params: {} 2024-04-16T18:11:15.352 DEBUG:boto:Token: None 2024-04-16T18:11:15.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d?versionId=null 2024-04-16T18:11:15.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI= 2024-04-16T18:11:15.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WaHeuQjbcHg8REB80V1/mH/3dlI=', 'Content-Length': '0'} 2024-04-16T18:11:15.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf0ee1acd7cb942c-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.355 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.356 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.356 DEBUG:boto:path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.356 DEBUG:boto:auth_path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.356 DEBUG:boto:Method: GET 2024-04-16T18:11:15.356 DEBUG:boto:Path: /lybtug-45/?prefix=d 2024-04-16T18:11:15.356 DEBUG:boto:Data: 2024-04-16T18:11:15.356 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.356 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.356 DEBUG:boto:Params: {} 2024-04-16T18:11:15.357 DEBUG:boto:Token: None 2024-04-16T18:11:15.357 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031108bb2a6456eb8-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.361 DEBUG:boto:b'lybtug-45d1000falsed2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.361 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.362 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.362 DEBUG:boto:path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.362 DEBUG:boto:auth_path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.362 DEBUG:boto:Method: GET 2024-04-16T18:11:15.362 DEBUG:boto:Path: /lybtug-45/?prefix=d 2024-04-16T18:11:15.362 DEBUG:boto:Data: 2024-04-16T18:11:15.362 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.362 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.362 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.363 DEBUG:boto:Params: {} 2024-04-16T18:11:15.363 DEBUG:boto:Token: None 2024-04-16T18:11:15.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.363 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.367 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000438d362f498d0278-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.410 DEBUG:boto:b'lybtug-45d1000falsed2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.411 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.411 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.411 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.411 DEBUG:boto:Path: /lybtug-45/d 2024-04-16T18:11:15.411 DEBUG:boto:Data: 2024-04-16T18:11:15.411 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.411 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.412 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.412 DEBUG:boto:Params: {} 2024-04-16T18:11:15.412 DEBUG:boto:Token: None 2024-04-16T18:11:15.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d 2024-04-16T18:11:15.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:85QheRiNxUTzXmNK3U+7hEcN4+M= 2024-04-16T18:11:15.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:85QheRiNxUTzXmNK3U+7hEcN4+M=', 'Content-Length': '0'} 2024-04-16T18:11:15.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c208859672d5b6df-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.415 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.415 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.415 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.416 DEBUG:boto:Path: /lybtug-45/d 2024-04-16T18:11:15.416 DEBUG:boto:Data: 2024-04-16T18:11:15.416 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.416 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.416 DEBUG:boto:Params: {} 2024-04-16T18:11:15.416 DEBUG:boto:Token: None 2024-04-16T18:11:15.416 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d 2024-04-16T18:11:15.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:85QheRiNxUTzXmNK3U+7hEcN4+M= 2024-04-16T18:11:15.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:85QheRiNxUTzXmNK3U+7hEcN4+M=', 'Content-Length': '0'} 2024-04-16T18:11:15.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c93f8f3832ac6d1a-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:15.420 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.420 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.420 DEBUG:boto:Method: GET 2024-04-16T18:11:15.420 DEBUG:boto:Path: /lybtug-45/d 2024-04-16T18:11:15.420 DEBUG:boto:Data: 2024-04-16T18:11:15.421 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.421 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.421 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.421 DEBUG:boto:Params: {} 2024-04-16T18:11:15.421 DEBUG:boto:Token: None 2024-04-16T18:11:15.421 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d 2024-04-16T18:11:15.421 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fAcWnKb7wP19/333YJhLHVmw4fk= 2024-04-16T18:11:15.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fAcWnKb7wP19/333YJhLHVmw4fk=', 'Content-Length': '0'} 2024-04-16T18:11:15.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009dc1c251c0748f10-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.466 DEBUG:boto:path=/lybtug-45/d 2024-04-16T18:11:15.467 DEBUG:boto:auth_path=/lybtug-45/d 2024-04-16T18:11:15.467 DEBUG:boto:Method: GET 2024-04-16T18:11:15.467 DEBUG:boto:Path: /lybtug-45/d 2024-04-16T18:11:15.467 DEBUG:boto:Data: 2024-04-16T18:11:15.467 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.467 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.467 DEBUG:boto:Params: {} 2024-04-16T18:11:15.467 DEBUG:boto:Token: None 2024-04-16T18:11:15.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/d 2024-04-16T18:11:15.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fAcWnKb7wP19/333YJhLHVmw4fk= 2024-04-16T18:11:15.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fAcWnKb7wP19/333YJhLHVmw4fk=', 'Content-Length': '0'} 2024-04-16T18:11:15.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:08 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b16a68fdbc7f1698-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.471 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.471 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.471 DEBUG:boto:path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.471 DEBUG:boto:auth_path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.472 DEBUG:boto:Method: GET 2024-04-16T18:11:15.472 DEBUG:boto:Path: /lybtug-45/?prefix=d 2024-04-16T18:11:15.472 DEBUG:boto:Data: 2024-04-16T18:11:15.472 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.472 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:15.472 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.472 DEBUG:boto:Params: {} 2024-04-16T18:11:15.472 DEBUG:boto:Token: None 2024-04-16T18:11:15.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.473 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000082f62a29d2bf484-00661ebf43-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.522 DEBUG:boto:b'lybtug-45d1000falsed2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.523 DEBUG:boto:path=/lybtug-45/ 2024-04-16T18:11:15.523 DEBUG:boto:auth_path=/lybtug-45/ 2024-04-16T18:11:15.523 DEBUG:boto:path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.523 DEBUG:boto:auth_path=/lybtug-45/?prefix=d 2024-04-16T18:11:15.523 DEBUG:boto:Method: GET 2024-04-16T18:11:15.524 DEBUG:boto:Path: /lybtug-45/?prefix=d 2024-04-16T18:11:15.524 DEBUG:boto:Data: 2024-04-16T18:11:15.524 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.524 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.524 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.524 DEBUG:boto:Params: {} 2024-04-16T18:11:15.524 DEBUG:boto:Token: None 2024-04-16T18:11:15.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:15 GMT /lybtug-45/ 2024-04-16T18:11:15.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE= 2024-04-16T18:11:15.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7Bv806SBMsoVe+eyIj+q7XHm5eE=', 'Content-Length': '0'} 2024-04-16T18:11:15.529 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000dd4ac9dd3cf0f82-00661ebf43-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.529 DEBUG:boto:b'lybtug-45d1000falsed2024-04-16T18:11:08.720Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:15.530 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-45 zones={test-zone2, test-zone1} 2024-04-16T18:11:15.530 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:15.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:15.533 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:11:15.572 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:15.572 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:15.680 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-46 2024-04-16T18:11:15.680 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:15.681 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:15.681 DEBUG:boto:Method: PUT 2024-04-16T18:11:15.681 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:15.681 DEBUG:boto:Data: 2024-04-16T18:11:15.681 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.681 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.681 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.681 DEBUG:boto:Params: {} 2024-04-16T18:11:15.682 DEBUG:boto:Token: None 2024-04-16T18:11:15.682 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/ 2024-04-16T18:11:15.682 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pIBYSUoUqrmCYCk7m6NpEunw3vk= 2024-04-16T18:11:15.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pIBYSUoUqrmCYCk7m6NpEunw3vk=', 'Content-Length': '0'} 2024-04-16T18:11:15.713 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005dce5ec33826d5d4-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.714 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:15.714 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:15.714 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.714 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:15.714 DEBUG:boto:Data: 2024-04-16T18:11:15.715 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.715 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.715 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.715 DEBUG:boto:Params: {} 2024-04-16T18:11:15.715 DEBUG:boto:Token: None 2024-04-16T18:11:15.715 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/ 2024-04-16T18:11:15.715 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk= 2024-04-16T18:11:15.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk=', 'Content-Length': '0'} 2024-04-16T18:11:15.719 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000023c37de8dff106af-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.720 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:15.720 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:15.720 DEBUG:boto:Method: PUT 2024-04-16T18:11:15.720 DEBUG:boto:Path: /lybtug-46/a 2024-04-16T18:11:15.720 DEBUG:boto:Data: 2024-04-16T18:11:15.720 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:15.721 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.721 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.721 DEBUG:boto:Params: {} 2024-04-16T18:11:15.721 DEBUG:boto:Token: None 2024-04-16T18:11:15.721 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/a 2024-04-16T18:11:15.721 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CBwgAhTc+PohwwxDP+iUUIOyJ3k= 2024-04-16T18:11:15.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CBwgAhTc+PohwwxDP+iUUIOyJ3k='} 2024-04-16T18:11:15.767 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000303aaf3ff8a51d44-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.767 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:15.767 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:15.767 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.767 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:15.767 DEBUG:boto:Data: 2024-04-16T18:11:15.768 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.768 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.768 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.768 DEBUG:boto:Params: {} 2024-04-16T18:11:15.768 DEBUG:boto:Token: None 2024-04-16T18:11:15.768 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/ 2024-04-16T18:11:15.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk= 2024-04-16T18:11:15.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk=', 'Content-Length': '0'} 2024-04-16T18:11:15.772 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c417a669c901709f-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.772 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:15.772 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:15.773 DEBUG:boto:Method: PUT 2024-04-16T18:11:15.773 DEBUG:boto:Path: /lybtug-46/b 2024-04-16T18:11:15.773 DEBUG:boto:Data: 2024-04-16T18:11:15.773 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:15.773 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.773 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.773 DEBUG:boto:Params: {} 2024-04-16T18:11:15.773 DEBUG:boto:Token: None 2024-04-16T18:11:15.774 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/b 2024-04-16T18:11:15.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ihzp7XBCBi+vhDndP812eO7L7eA= 2024-04-16T18:11:15.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ihzp7XBCBi+vhDndP812eO7L7eA='} 2024-04-16T18:11:15.787 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008a5c47db37742fc4-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.787 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:15.787 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:15.787 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.787 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:15.788 DEBUG:boto:Data: 2024-04-16T18:11:15.788 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.788 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.788 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.788 DEBUG:boto:Params: {} 2024-04-16T18:11:15.788 DEBUG:boto:Token: None 2024-04-16T18:11:15.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/ 2024-04-16T18:11:15.789 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk= 2024-04-16T18:11:15.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk=', 'Content-Length': '0'} 2024-04-16T18:11:15.792 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000001c20051c83e0f82-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.792 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:15.793 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:15.793 DEBUG:boto:Method: PUT 2024-04-16T18:11:15.793 DEBUG:boto:Path: /lybtug-46/c 2024-04-16T18:11:15.793 DEBUG:boto:Data: 2024-04-16T18:11:15.793 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:15.793 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.793 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.793 DEBUG:boto:Params: {} 2024-04-16T18:11:15.794 DEBUG:boto:Token: None 2024-04-16T18:11:15.794 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/c 2024-04-16T18:11:15.794 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QA73PNTf/4qGve2xrsdhTLp2lF4= 2024-04-16T18:11:15.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QA73PNTf/4qGve2xrsdhTLp2lF4='} 2024-04-16T18:11:15.839 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000acc9b33629052e08-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.839 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:15.839 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:15.839 DEBUG:boto:Method: HEAD 2024-04-16T18:11:15.839 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:15.840 DEBUG:boto:Data: 2024-04-16T18:11:15.840 DEBUG:boto:Headers: {} 2024-04-16T18:11:15.840 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.840 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.840 DEBUG:boto:Params: {} 2024-04-16T18:11:15.840 DEBUG:boto:Token: None 2024-04-16T18:11:15.840 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/ 2024-04-16T18:11:15.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk= 2024-04-16T18:11:15.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:17mcshsiTXTQ1A3XRcapZWZAENk=', 'Content-Length': '0'} 2024-04-16T18:11:15.845 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000046edd9883712c8a9-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.845 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:15.845 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:15.845 DEBUG:boto:Method: PUT 2024-04-16T18:11:15.845 DEBUG:boto:Path: /lybtug-46/d 2024-04-16T18:11:15.846 DEBUG:boto:Data: 2024-04-16T18:11:15.846 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:15.846 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:15.846 DEBUG:boto:Port: 8000 2024-04-16T18:11:15.846 DEBUG:boto:Params: {} 2024-04-16T18:11:15.846 DEBUG:boto:Token: None 2024-04-16T18:11:15.847 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:15 GMT /lybtug-46/d 2024-04-16T18:11:15.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rk1osQ2m57FxUlDmlG/4jtr65po= 2024-04-16T18:11:15.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rk1osQ2m57FxUlDmlG/4jtr65po='} 2024-04-16T18:11:15.858 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006f95a6efd5b0cc13-00661ebf43-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:15.859 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:15.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:15.859 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:15.971 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:11:16.004 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291075.714237_446.1","last_update":"2024-04-16T18:11:15.714237Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:11:16.005 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291075.714237_446.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:16.005 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:11:16.005 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:16.005 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:16.005 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:16.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:16.122+0000 7fb7553cb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:16.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:16.122+0000 7fb7553cb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:16.153 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291075.714237_446.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:15.714237Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:16Z"} 2024-04-16T18:11:16.153 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291075.714237_446.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:15.714237Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:16Z"} 2024-04-16T18:11:16.153 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291075.714237_446.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:11:15.714237Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:11:16.153 DEBUG:rgw_multi.tests:log_status={0: '1_1713291075.714237_446.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:16.154 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291075.714237_446.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:16.154 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:11:16.154 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-46 2024-04-16T18:11:16.154 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-46', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:16.154 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-46', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:16.155 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-46 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:16.236 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:16.236 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:16.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:16.318+0000 7f5cba766980 1 waiting to reach incremental sync.. 2024-04-16T18:11:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:21.317+0000 7f5cba766980 1 waiting to reach incremental sync.. 2024-04-16T18:11:26.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:26.317+0000 7f5cba766980 1 waiting to reach incremental sync.. 2024-04-16T18:11:31.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:31.321+0000 7f5cba766980 1 waiting to reach incremental sync.. 2024-04-16T18:11:36.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:36.321+0000 7f5cba766980 1 waiting to reach incremental sync.. 2024-04-16T18:11:41.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:41.329+0000 7f5cba766980 1 bucket sync caught up with source: 2024-04-16T18:11:41.324 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.29.6, , , , 00000000001.49.6, , , 00000000001.63.6, , 00000000001.26.6] 2024-04-16T18:11:41.324 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000001.29.6, , , , 00000000001.49.6, , , 00000000001.63.6, , 00000000001.26.6] 2024-04-16T18:11:41.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:41.329+0000 7f5cba766980 0 bucket checkpoint complete 2024-04-16T18:11:41.375 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-46 2024-04-16T18:11:41.375 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-46', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:41.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-46', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:41.375 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-46 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:41.426 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:41.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:41.506+0000 7fcb58abb980 1 waiting to reach incremental sync.. 2024-04-16T18:11:46.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:46.510+0000 7fcb58abb980 1 bucket sync caught up with source: 2024-04-16T18:11:46.508 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.846.6, , , , 00000000001.1304.6, , , 00000000001.122.6, , 00000000001.931.6] 2024-04-16T18:11:46.508 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.846.6, , , , 00000000001.1304.6, , , 00000000001.122.6, , 00000000001.931.6] 2024-04-16T18:11:46.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:46.510+0000 7fcb58abb980 0 bucket checkpoint complete 2024-04-16T18:11:46.553 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-46 zones={test-zone2, test-zone1} 2024-04-16T18:11:46.553 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.554 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.554 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.554 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:46.554 DEBUG:boto:Data: 2024-04-16T18:11:46.554 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.554 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.554 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.554 DEBUG:boto:Params: {} 2024-04-16T18:11:46.554 DEBUG:boto:Token: None 2024-04-16T18:11:46.555 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.555 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2LpV1dwskYaG24IrJyZFssI9/6E= 2024-04-16T18:11:46.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2LpV1dwskYaG24IrJyZFssI9/6E=', 'Content-Length': '0'} 2024-04-16T18:11:46.559 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006fbc73cabbbbd2e3-00661ebf62-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.559 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.559 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.559 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.559 DEBUG:boto:Path: /lybtug-46/ 2024-04-16T18:11:46.559 DEBUG:boto:Data: 2024-04-16T18:11:46.559 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.559 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.560 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.560 DEBUG:boto:Params: {} 2024-04-16T18:11:46.560 DEBUG:boto:Token: None 2024-04-16T18:11:46.560 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2LpV1dwskYaG24IrJyZFssI9/6E= 2024-04-16T18:11:46.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2LpV1dwskYaG24IrJyZFssI9/6E=', 'Content-Length': '0'} 2024-04-16T18:11:46.563 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e032db5a54f62e1e-00661ebf62-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.563 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:11:46.564 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.564 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.564 DEBUG:boto:path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.564 DEBUG:boto:auth_path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.564 DEBUG:boto:Method: GET 2024-04-16T18:11:46.564 DEBUG:boto:Path: /lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.564 DEBUG:boto:Data: 2024-04-16T18:11:46.564 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.564 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.564 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.564 DEBUG:boto:Params: {} 2024-04-16T18:11:46.565 DEBUG:boto:Token: None 2024-04-16T18:11:46.565 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/?versions 2024-04-16T18:11:46.565 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY= 2024-04-16T18:11:46.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY=', 'Content-Length': '0'} 2024-04-16T18:11:46.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ea7fb76d23dfd16-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.569 DEBUG:boto:b'lybtug-46999falseanulltrue2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.569 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:11:46.569 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:11:46.569 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:11:46.569 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:11:46.570 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:11:46.570 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.570 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.570 DEBUG:boto:path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.570 DEBUG:boto:auth_path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.570 DEBUG:boto:Method: GET 2024-04-16T18:11:46.570 DEBUG:boto:Path: /lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.570 DEBUG:boto:Data: 2024-04-16T18:11:46.570 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.570 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.570 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.571 DEBUG:boto:Params: {} 2024-04-16T18:11:46.571 DEBUG:boto:Token: None 2024-04-16T18:11:46.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/?versions 2024-04-16T18:11:46.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY= 2024-04-16T18:11:46.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY=', 'Content-Length': '0'} 2024-04-16T18:11:46.574 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd15dc97d83761ac-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.575 DEBUG:boto:b'lybtug-46999falseanulltrue2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.575 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:11:46.575 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:11:46.575 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:11:46.575 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:11:46.575 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.575 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.576 DEBUG:boto:path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.576 DEBUG:boto:auth_path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.576 DEBUG:boto:Method: GET 2024-04-16T18:11:46.576 DEBUG:boto:Path: /lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.576 DEBUG:boto:Data: 2024-04-16T18:11:46.576 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.576 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.576 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.576 DEBUG:boto:Params: {} 2024-04-16T18:11:46.576 DEBUG:boto:Token: None 2024-04-16T18:11:46.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/?versions 2024-04-16T18:11:46.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY= 2024-04-16T18:11:46.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY=', 'Content-Length': '0'} 2024-04-16T18:11:46.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d22a862402f29a78-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.580 DEBUG:boto:b'lybtug-46999falseanulltrue2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.581 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.581 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.581 DEBUG:boto:path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.581 DEBUG:boto:auth_path=/lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.581 DEBUG:boto:Method: GET 2024-04-16T18:11:46.581 DEBUG:boto:Path: /lybtug-46/?versions&max-keys=999 2024-04-16T18:11:46.581 DEBUG:boto:Data: 2024-04-16T18:11:46.581 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.581 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.582 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.582 DEBUG:boto:Params: {} 2024-04-16T18:11:46.582 DEBUG:boto:Token: None 2024-04-16T18:11:46.582 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/?versions 2024-04-16T18:11:46.582 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY= 2024-04-16T18:11:46.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zTxcTFzkkWUsNP2t86VgMUhO2yY=', 'Content-Length': '0'} 2024-04-16T18:11:46.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7710b0ebf5856a1-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.586 DEBUG:boto:b'lybtug-46999falseanulltrue2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.586 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:46.586 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.586 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.586 DEBUG:boto:path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.587 DEBUG:boto:auth_path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.587 DEBUG:boto:Method: GET 2024-04-16T18:11:46.587 DEBUG:boto:Path: /lybtug-46/a?versionId=null 2024-04-16T18:11:46.587 DEBUG:boto:Data: 2024-04-16T18:11:46.587 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.587 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.587 DEBUG:boto:Params: {} 2024-04-16T18:11:46.587 DEBUG:boto:Token: None 2024-04-16T18:11:46.587 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a?versionId=null 2024-04-16T18:11:46.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8= 2024-04-16T18:11:46.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8=', 'Content-Length': '0'} 2024-04-16T18:11:46.590 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076e666bec52f326a-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.634 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.634 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.635 DEBUG:boto:path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.635 DEBUG:boto:auth_path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.635 DEBUG:boto:Method: GET 2024-04-16T18:11:46.635 DEBUG:boto:Path: /lybtug-46/a?versionId=null 2024-04-16T18:11:46.635 DEBUG:boto:Data: 2024-04-16T18:11:46.635 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.635 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.635 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.635 DEBUG:boto:Params: {} 2024-04-16T18:11:46.635 DEBUG:boto:Token: None 2024-04-16T18:11:46.635 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a?versionId=null 2024-04-16T18:11:46.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8= 2024-04-16T18:11:46.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8=', 'Content-Length': '0'} 2024-04-16T18:11:46.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ca25477f8278a5c-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.639 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.639 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.639 DEBUG:boto:path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.639 DEBUG:boto:auth_path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.639 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.639 DEBUG:boto:Path: /lybtug-46/a?versionId=null 2024-04-16T18:11:46.639 DEBUG:boto:Data: 2024-04-16T18:11:46.640 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.640 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.640 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.640 DEBUG:boto:Params: {} 2024-04-16T18:11:46.640 DEBUG:boto:Token: None 2024-04-16T18:11:46.640 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a?versionId=null 2024-04-16T18:11:46.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cZsmoqOBadT0kZxYNVJaY+iSNfY= 2024-04-16T18:11:46.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cZsmoqOBadT0kZxYNVJaY+iSNfY=', 'Content-Length': '0'} 2024-04-16T18:11:46.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed39d151a8d6fb86-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.643 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.643 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.643 DEBUG:boto:path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.644 DEBUG:boto:auth_path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.644 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.644 DEBUG:boto:Path: /lybtug-46/a?versionId=null 2024-04-16T18:11:46.644 DEBUG:boto:Data: 2024-04-16T18:11:46.644 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.644 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.644 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.644 DEBUG:boto:Params: {} 2024-04-16T18:11:46.644 DEBUG:boto:Token: None 2024-04-16T18:11:46.644 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a?versionId=null 2024-04-16T18:11:46.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cZsmoqOBadT0kZxYNVJaY+iSNfY= 2024-04-16T18:11:46.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cZsmoqOBadT0kZxYNVJaY+iSNfY=', 'Content-Length': '0'} 2024-04-16T18:11:46.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8ecf44e1586499a-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:46.648 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.648 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.648 DEBUG:boto:path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.648 DEBUG:boto:auth_path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.648 DEBUG:boto:Method: GET 2024-04-16T18:11:46.648 DEBUG:boto:Path: /lybtug-46/a?versionId=null 2024-04-16T18:11:46.648 DEBUG:boto:Data: 2024-04-16T18:11:46.648 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.648 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.648 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.649 DEBUG:boto:Params: {} 2024-04-16T18:11:46.649 DEBUG:boto:Token: None 2024-04-16T18:11:46.649 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a?versionId=null 2024-04-16T18:11:46.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8= 2024-04-16T18:11:46.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8=', 'Content-Length': '0'} 2024-04-16T18:11:46.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027bfd1c325ca176f-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.694 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.694 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.695 DEBUG:boto:path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.695 DEBUG:boto:auth_path=/lybtug-46/a?versionId=null 2024-04-16T18:11:46.695 DEBUG:boto:Method: GET 2024-04-16T18:11:46.695 DEBUG:boto:Path: /lybtug-46/a?versionId=null 2024-04-16T18:11:46.695 DEBUG:boto:Data: 2024-04-16T18:11:46.695 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.695 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.695 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.695 DEBUG:boto:Params: {} 2024-04-16T18:11:46.695 DEBUG:boto:Token: None 2024-04-16T18:11:46.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a?versionId=null 2024-04-16T18:11:46.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8= 2024-04-16T18:11:46.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qf93eCeMnRpq0nYAP6QWlvhRqC8=', 'Content-Length': '0'} 2024-04-16T18:11:46.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011a08f0c419d12d8-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.699 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.699 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.699 DEBUG:boto:path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.699 DEBUG:boto:auth_path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.699 DEBUG:boto:Method: GET 2024-04-16T18:11:46.700 DEBUG:boto:Path: /lybtug-46/?prefix=a 2024-04-16T18:11:46.700 DEBUG:boto:Data: 2024-04-16T18:11:46.700 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.700 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.700 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.700 DEBUG:boto:Params: {} 2024-04-16T18:11:46.700 DEBUG:boto:Token: None 2024-04-16T18:11:46.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8= 2024-04-16T18:11:46.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8=', 'Content-Length': '0'} 2024-04-16T18:11:46.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000daa3a350efe3192b-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.746 DEBUG:boto:b'lybtug-46a1000falsea2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.747 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.747 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.747 DEBUG:boto:path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.747 DEBUG:boto:auth_path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.747 DEBUG:boto:Method: GET 2024-04-16T18:11:46.747 DEBUG:boto:Path: /lybtug-46/?prefix=a 2024-04-16T18:11:46.747 DEBUG:boto:Data: 2024-04-16T18:11:46.747 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.747 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.747 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.747 DEBUG:boto:Params: {} 2024-04-16T18:11:46.748 DEBUG:boto:Token: None 2024-04-16T18:11:46.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8= 2024-04-16T18:11:46.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8=', 'Content-Length': '0'} 2024-04-16T18:11:46.751 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b12e0bda4762ff97-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.751 DEBUG:boto:b'lybtug-46a1000falsea2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.752 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.752 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.752 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.752 DEBUG:boto:Path: /lybtug-46/a 2024-04-16T18:11:46.752 DEBUG:boto:Data: 2024-04-16T18:11:46.752 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.752 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.752 DEBUG:boto:Params: {} 2024-04-16T18:11:46.752 DEBUG:boto:Token: None 2024-04-16T18:11:46.753 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a 2024-04-16T18:11:46.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DPTDu744M9Gmp+8BuSmanFOT1Vk= 2024-04-16T18:11:46.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DPTDu744M9Gmp+8BuSmanFOT1Vk=', 'Content-Length': '0'} 2024-04-16T18:11:46.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000157fe017a6a324fe-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.756 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.756 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.756 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.756 DEBUG:boto:Path: /lybtug-46/a 2024-04-16T18:11:46.756 DEBUG:boto:Data: 2024-04-16T18:11:46.756 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.756 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.756 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.756 DEBUG:boto:Params: {} 2024-04-16T18:11:46.757 DEBUG:boto:Token: None 2024-04-16T18:11:46.757 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a 2024-04-16T18:11:46.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DPTDu744M9Gmp+8BuSmanFOT1Vk= 2024-04-16T18:11:46.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DPTDu744M9Gmp+8BuSmanFOT1Vk=', 'Content-Length': '0'} 2024-04-16T18:11:46.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1c6a03587687381-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:46.760 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.760 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.760 DEBUG:boto:Method: GET 2024-04-16T18:11:46.760 DEBUG:boto:Path: /lybtug-46/a 2024-04-16T18:11:46.760 DEBUG:boto:Data: 2024-04-16T18:11:46.760 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.760 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.761 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.761 DEBUG:boto:Params: {} 2024-04-16T18:11:46.761 DEBUG:boto:Token: None 2024-04-16T18:11:46.761 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a 2024-04-16T18:11:46.761 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VwUd4oIjrvlB1xX0nXXKLzGSKAo= 2024-04-16T18:11:46.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VwUd4oIjrvlB1xX0nXXKLzGSKAo=', 'Content-Length': '0'} 2024-04-16T18:11:46.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e77450cf6d1564f-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.806 DEBUG:boto:path=/lybtug-46/a 2024-04-16T18:11:46.806 DEBUG:boto:auth_path=/lybtug-46/a 2024-04-16T18:11:46.807 DEBUG:boto:Method: GET 2024-04-16T18:11:46.807 DEBUG:boto:Path: /lybtug-46/a 2024-04-16T18:11:46.807 DEBUG:boto:Data: 2024-04-16T18:11:46.807 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.807 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.807 DEBUG:boto:Params: {} 2024-04-16T18:11:46.807 DEBUG:boto:Token: None 2024-04-16T18:11:46.807 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/a 2024-04-16T18:11:46.807 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VwUd4oIjrvlB1xX0nXXKLzGSKAo= 2024-04-16T18:11:46.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VwUd4oIjrvlB1xX0nXXKLzGSKAo=', 'Content-Length': '0'} 2024-04-16T18:11:46.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083ae4a31c05b196b-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.810 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.811 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.811 DEBUG:boto:path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.811 DEBUG:boto:auth_path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.811 DEBUG:boto:Method: GET 2024-04-16T18:11:46.811 DEBUG:boto:Path: /lybtug-46/?prefix=a 2024-04-16T18:11:46.811 DEBUG:boto:Data: 2024-04-16T18:11:46.811 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.811 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.811 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.811 DEBUG:boto:Params: {} 2024-04-16T18:11:46.811 DEBUG:boto:Token: None 2024-04-16T18:11:46.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8= 2024-04-16T18:11:46.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8=', 'Content-Length': '0'} 2024-04-16T18:11:46.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb4b0b46ea7a5b02-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.858 DEBUG:boto:b'lybtug-46a1000falsea2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.859 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.859 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.859 DEBUG:boto:path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.859 DEBUG:boto:auth_path=/lybtug-46/?prefix=a 2024-04-16T18:11:46.859 DEBUG:boto:Method: GET 2024-04-16T18:11:46.859 DEBUG:boto:Path: /lybtug-46/?prefix=a 2024-04-16T18:11:46.859 DEBUG:boto:Data: 2024-04-16T18:11:46.859 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.859 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.859 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.860 DEBUG:boto:Params: {} 2024-04-16T18:11:46.860 DEBUG:boto:Token: None 2024-04-16T18:11:46.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8= 2024-04-16T18:11:46.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8=', 'Content-Length': '0'} 2024-04-16T18:11:46.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096bc84e90a840c82-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.864 DEBUG:boto:b'lybtug-46a1000falsea2024-04-16T18:11:15.729Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.864 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:46.864 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.865 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.865 DEBUG:boto:path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.865 DEBUG:boto:auth_path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.865 DEBUG:boto:Method: GET 2024-04-16T18:11:46.865 DEBUG:boto:Path: /lybtug-46/b?versionId=null 2024-04-16T18:11:46.865 DEBUG:boto:Data: 2024-04-16T18:11:46.865 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.865 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.865 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.865 DEBUG:boto:Params: {} 2024-04-16T18:11:46.865 DEBUG:boto:Token: None 2024-04-16T18:11:46.866 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b?versionId=null 2024-04-16T18:11:46.866 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs= 2024-04-16T18:11:46.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs=', 'Content-Length': '0'} 2024-04-16T18:11:46.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005343cda043d3ede6-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.910 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.910 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.911 DEBUG:boto:path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.911 DEBUG:boto:auth_path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.911 DEBUG:boto:Method: GET 2024-04-16T18:11:46.911 DEBUG:boto:Path: /lybtug-46/b?versionId=null 2024-04-16T18:11:46.911 DEBUG:boto:Data: 2024-04-16T18:11:46.911 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.911 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.911 DEBUG:boto:Params: {} 2024-04-16T18:11:46.911 DEBUG:boto:Token: None 2024-04-16T18:11:46.911 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b?versionId=null 2024-04-16T18:11:46.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs= 2024-04-16T18:11:46.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs=', 'Content-Length': '0'} 2024-04-16T18:11:46.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fd78b7a40aec5b1-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.915 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.915 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.915 DEBUG:boto:path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.915 DEBUG:boto:auth_path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.915 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.915 DEBUG:boto:Path: /lybtug-46/b?versionId=null 2024-04-16T18:11:46.915 DEBUG:boto:Data: 2024-04-16T18:11:46.916 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.916 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.916 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.916 DEBUG:boto:Params: {} 2024-04-16T18:11:46.916 DEBUG:boto:Token: None 2024-04-16T18:11:46.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b?versionId=null 2024-04-16T18:11:46.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N+EmEnre/FgOyQ56N0iomFwy0Nw= 2024-04-16T18:11:46.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N+EmEnre/FgOyQ56N0iomFwy0Nw=', 'Content-Length': '0'} 2024-04-16T18:11:46.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003aee5c5e022b8cea-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.919 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.919 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.919 DEBUG:boto:path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.920 DEBUG:boto:auth_path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.920 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.920 DEBUG:boto:Path: /lybtug-46/b?versionId=null 2024-04-16T18:11:46.920 DEBUG:boto:Data: 2024-04-16T18:11:46.920 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.920 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.920 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.920 DEBUG:boto:Params: {} 2024-04-16T18:11:46.920 DEBUG:boto:Token: None 2024-04-16T18:11:46.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b?versionId=null 2024-04-16T18:11:46.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N+EmEnre/FgOyQ56N0iomFwy0Nw= 2024-04-16T18:11:46.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N+EmEnre/FgOyQ56N0iomFwy0Nw=', 'Content-Length': '0'} 2024-04-16T18:11:46.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076810e59cba7f5d0-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.924 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:46.924 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.924 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.924 DEBUG:boto:path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.924 DEBUG:boto:auth_path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.924 DEBUG:boto:Method: GET 2024-04-16T18:11:46.924 DEBUG:boto:Path: /lybtug-46/b?versionId=null 2024-04-16T18:11:46.924 DEBUG:boto:Data: 2024-04-16T18:11:46.924 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.924 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.924 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.925 DEBUG:boto:Params: {} 2024-04-16T18:11:46.925 DEBUG:boto:Token: None 2024-04-16T18:11:46.925 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b?versionId=null 2024-04-16T18:11:46.925 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs= 2024-04-16T18:11:46.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs=', 'Content-Length': '0'} 2024-04-16T18:11:46.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060d615030aa44770-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.970 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.970 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.970 DEBUG:boto:path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.971 DEBUG:boto:auth_path=/lybtug-46/b?versionId=null 2024-04-16T18:11:46.971 DEBUG:boto:Method: GET 2024-04-16T18:11:46.971 DEBUG:boto:Path: /lybtug-46/b?versionId=null 2024-04-16T18:11:46.971 DEBUG:boto:Data: 2024-04-16T18:11:46.971 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.971 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.971 DEBUG:boto:Params: {} 2024-04-16T18:11:46.971 DEBUG:boto:Token: None 2024-04-16T18:11:46.971 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b?versionId=null 2024-04-16T18:11:46.971 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs= 2024-04-16T18:11:46.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1oLZj2LsjgHbjZvBMu6lz5rt6Zs=', 'Content-Length': '0'} 2024-04-16T18:11:46.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c915edb65907f5f-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.975 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.975 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.975 DEBUG:boto:path=/lybtug-46/?prefix=b 2024-04-16T18:11:46.975 DEBUG:boto:auth_path=/lybtug-46/?prefix=b 2024-04-16T18:11:46.975 DEBUG:boto:Method: GET 2024-04-16T18:11:46.975 DEBUG:boto:Path: /lybtug-46/?prefix=b 2024-04-16T18:11:46.975 DEBUG:boto:Data: 2024-04-16T18:11:46.975 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.975 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.975 DEBUG:boto:Params: {} 2024-04-16T18:11:46.976 DEBUG:boto:Token: None 2024-04-16T18:11:46.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8= 2024-04-16T18:11:46.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8=', 'Content-Length': '0'} 2024-04-16T18:11:46.980 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ecc2b27811a6337-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.980 DEBUG:boto:b'lybtug-46b1000falseb2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.981 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:46.981 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:46.981 DEBUG:boto:path=/lybtug-46/?prefix=b 2024-04-16T18:11:46.981 DEBUG:boto:auth_path=/lybtug-46/?prefix=b 2024-04-16T18:11:46.981 DEBUG:boto:Method: GET 2024-04-16T18:11:46.981 DEBUG:boto:Path: /lybtug-46/?prefix=b 2024-04-16T18:11:46.981 DEBUG:boto:Data: 2024-04-16T18:11:46.981 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.981 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.981 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.981 DEBUG:boto:Params: {} 2024-04-16T18:11:46.982 DEBUG:boto:Token: None 2024-04-16T18:11:46.982 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/ 2024-04-16T18:11:46.982 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8= 2024-04-16T18:11:46.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dj0Aw0P57GtiSHzoX4viJWJeuf8=', 'Content-Length': '0'} 2024-04-16T18:11:46.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fed80d81b961451e-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.986 DEBUG:boto:b'lybtug-46b1000falseb2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:46.987 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.987 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.987 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.987 DEBUG:boto:Path: /lybtug-46/b 2024-04-16T18:11:46.987 DEBUG:boto:Data: 2024-04-16T18:11:46.987 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.987 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.987 DEBUG:boto:Params: {} 2024-04-16T18:11:46.987 DEBUG:boto:Token: None 2024-04-16T18:11:46.987 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b 2024-04-16T18:11:46.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:58hi2rtqeHUWHpkuPWAywP/od04= 2024-04-16T18:11:46.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:58hi2rtqeHUWHpkuPWAywP/od04=', 'Content-Length': '0'} 2024-04-16T18:11:46.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3cbabe5d1d97a29-00661ebf62-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.991 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.991 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.991 DEBUG:boto:Method: HEAD 2024-04-16T18:11:46.991 DEBUG:boto:Path: /lybtug-46/b 2024-04-16T18:11:46.991 DEBUG:boto:Data: 2024-04-16T18:11:46.991 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:46.991 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.991 DEBUG:boto:Params: {} 2024-04-16T18:11:46.991 DEBUG:boto:Token: None 2024-04-16T18:11:46.992 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b 2024-04-16T18:11:46.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:58hi2rtqeHUWHpkuPWAywP/od04= 2024-04-16T18:11:46.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:58hi2rtqeHUWHpkuPWAywP/od04=', 'Content-Length': '0'} 2024-04-16T18:11:46.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3aff156595e808a-00661ebf62-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:46.995 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:46.995 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:46.995 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:46.995 DEBUG:boto:Method: GET 2024-04-16T18:11:46.995 DEBUG:boto:Path: /lybtug-46/b 2024-04-16T18:11:46.995 DEBUG:boto:Data: 2024-04-16T18:11:46.995 DEBUG:boto:Headers: {} 2024-04-16T18:11:46.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:46.995 DEBUG:boto:Port: 8000 2024-04-16T18:11:46.995 DEBUG:boto:Params: {} 2024-04-16T18:11:46.995 DEBUG:boto:Token: None 2024-04-16T18:11:46.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:46 GMT /lybtug-46/b 2024-04-16T18:11:46.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K7NXkihsIeNleJ0YKr5F+cf8+yY= 2024-04-16T18:11:46.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K7NXkihsIeNleJ0YKr5F+cf8+yY=', 'Content-Length': '0'} 2024-04-16T18:11:46.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035adfe8f8d8fbe56-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.042 DEBUG:boto:path=/lybtug-46/b 2024-04-16T18:11:47.042 DEBUG:boto:auth_path=/lybtug-46/b 2024-04-16T18:11:47.043 DEBUG:boto:Method: GET 2024-04-16T18:11:47.043 DEBUG:boto:Path: /lybtug-46/b 2024-04-16T18:11:47.043 DEBUG:boto:Data: 2024-04-16T18:11:47.043 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.043 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.043 DEBUG:boto:Params: {} 2024-04-16T18:11:47.043 DEBUG:boto:Token: None 2024-04-16T18:11:47.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/b 2024-04-16T18:11:47.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8ZrzvJTxNx1ETfyyCUvLEBwFtcM= 2024-04-16T18:11:47.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8ZrzvJTxNx1ETfyyCUvLEBwFtcM=', 'Content-Length': '0'} 2024-04-16T18:11:47.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a9243ad46eb1e9f-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.047 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.047 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.047 DEBUG:boto:path=/lybtug-46/?prefix=b 2024-04-16T18:11:47.047 DEBUG:boto:auth_path=/lybtug-46/?prefix=b 2024-04-16T18:11:47.047 DEBUG:boto:Method: GET 2024-04-16T18:11:47.047 DEBUG:boto:Path: /lybtug-46/?prefix=b 2024-04-16T18:11:47.047 DEBUG:boto:Data: 2024-04-16T18:11:47.048 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.048 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.048 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.048 DEBUG:boto:Params: {} 2024-04-16T18:11:47.048 DEBUG:boto:Token: None 2024-04-16T18:11:47.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7a07ae9334cd422-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.053 DEBUG:boto:b'lybtug-46b1000falseb2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.053 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.053 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.053 DEBUG:boto:path=/lybtug-46/?prefix=b 2024-04-16T18:11:47.053 DEBUG:boto:auth_path=/lybtug-46/?prefix=b 2024-04-16T18:11:47.053 DEBUG:boto:Method: GET 2024-04-16T18:11:47.054 DEBUG:boto:Path: /lybtug-46/?prefix=b 2024-04-16T18:11:47.054 DEBUG:boto:Data: 2024-04-16T18:11:47.054 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.054 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.054 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.054 DEBUG:boto:Params: {} 2024-04-16T18:11:47.054 DEBUG:boto:Token: None 2024-04-16T18:11:47.054 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.054 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bb27a9956556186-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.102 DEBUG:boto:b'lybtug-46b1000falseb2024-04-16T18:11:15.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.103 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:47.103 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.103 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.103 DEBUG:boto:path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.103 DEBUG:boto:auth_path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.103 DEBUG:boto:Method: GET 2024-04-16T18:11:47.103 DEBUG:boto:Path: /lybtug-46/c?versionId=null 2024-04-16T18:11:47.103 DEBUG:boto:Data: 2024-04-16T18:11:47.103 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.103 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.103 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.104 DEBUG:boto:Params: {} 2024-04-16T18:11:47.104 DEBUG:boto:Token: None 2024-04-16T18:11:47.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c?versionId=null 2024-04-16T18:11:47.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0= 2024-04-16T18:11:47.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0=', 'Content-Length': '0'} 2024-04-16T18:11:47.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f9d029fbbbb66cc-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.150 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.150 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.151 DEBUG:boto:path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.151 DEBUG:boto:auth_path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.151 DEBUG:boto:Method: GET 2024-04-16T18:11:47.151 DEBUG:boto:Path: /lybtug-46/c?versionId=null 2024-04-16T18:11:47.151 DEBUG:boto:Data: 2024-04-16T18:11:47.151 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.151 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.151 DEBUG:boto:Params: {} 2024-04-16T18:11:47.151 DEBUG:boto:Token: None 2024-04-16T18:11:47.151 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c?versionId=null 2024-04-16T18:11:47.151 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0= 2024-04-16T18:11:47.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0=', 'Content-Length': '0'} 2024-04-16T18:11:47.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aab436b61019aee5-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.155 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.155 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.155 DEBUG:boto:path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.155 DEBUG:boto:auth_path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.155 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.155 DEBUG:boto:Path: /lybtug-46/c?versionId=null 2024-04-16T18:11:47.155 DEBUG:boto:Data: 2024-04-16T18:11:47.155 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.155 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.155 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.155 DEBUG:boto:Params: {} 2024-04-16T18:11:47.156 DEBUG:boto:Token: None 2024-04-16T18:11:47.156 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c?versionId=null 2024-04-16T18:11:47.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9B11jt1IEM4Ed3CJNcokEjzo894= 2024-04-16T18:11:47.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9B11jt1IEM4Ed3CJNcokEjzo894=', 'Content-Length': '0'} 2024-04-16T18:11:47.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1490d3700e3a986-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.159 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.159 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.159 DEBUG:boto:path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.159 DEBUG:boto:auth_path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.159 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.159 DEBUG:boto:Path: /lybtug-46/c?versionId=null 2024-04-16T18:11:47.159 DEBUG:boto:Data: 2024-04-16T18:11:47.159 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.159 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.160 DEBUG:boto:Params: {} 2024-04-16T18:11:47.160 DEBUG:boto:Token: None 2024-04-16T18:11:47.160 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c?versionId=null 2024-04-16T18:11:47.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9B11jt1IEM4Ed3CJNcokEjzo894= 2024-04-16T18:11:47.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9B11jt1IEM4Ed3CJNcokEjzo894=', 'Content-Length': '0'} 2024-04-16T18:11:47.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000543928101807219c-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:47.163 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.163 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.163 DEBUG:boto:path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.163 DEBUG:boto:auth_path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.163 DEBUG:boto:Method: GET 2024-04-16T18:11:47.163 DEBUG:boto:Path: /lybtug-46/c?versionId=null 2024-04-16T18:11:47.163 DEBUG:boto:Data: 2024-04-16T18:11:47.163 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.163 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.164 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.164 DEBUG:boto:Params: {} 2024-04-16T18:11:47.164 DEBUG:boto:Token: None 2024-04-16T18:11:47.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c?versionId=null 2024-04-16T18:11:47.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0= 2024-04-16T18:11:47.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0=', 'Content-Length': '0'} 2024-04-16T18:11:47.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b05904bc872a9610-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.210 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.210 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.211 DEBUG:boto:path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.211 DEBUG:boto:auth_path=/lybtug-46/c?versionId=null 2024-04-16T18:11:47.211 DEBUG:boto:Method: GET 2024-04-16T18:11:47.211 DEBUG:boto:Path: /lybtug-46/c?versionId=null 2024-04-16T18:11:47.211 DEBUG:boto:Data: 2024-04-16T18:11:47.211 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.211 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.211 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.211 DEBUG:boto:Params: {} 2024-04-16T18:11:47.211 DEBUG:boto:Token: None 2024-04-16T18:11:47.211 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c?versionId=null 2024-04-16T18:11:47.211 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0= 2024-04-16T18:11:47.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WTHqu6YybQPiYTjgY8pqw7VnLZ0=', 'Content-Length': '0'} 2024-04-16T18:11:47.214 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ac4a3c03ca2e231-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.214 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.215 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.215 DEBUG:boto:path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.215 DEBUG:boto:auth_path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.215 DEBUG:boto:Method: GET 2024-04-16T18:11:47.215 DEBUG:boto:Path: /lybtug-46/?prefix=c 2024-04-16T18:11:47.215 DEBUG:boto:Data: 2024-04-16T18:11:47.215 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.215 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.215 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.215 DEBUG:boto:Params: {} 2024-04-16T18:11:47.215 DEBUG:boto:Token: None 2024-04-16T18:11:47.216 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.220 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000e74dbea7a50e1c3-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.220 DEBUG:boto:b'lybtug-46c1000falsec2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.221 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.221 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.221 DEBUG:boto:path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.221 DEBUG:boto:auth_path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.221 DEBUG:boto:Method: GET 2024-04-16T18:11:47.221 DEBUG:boto:Path: /lybtug-46/?prefix=c 2024-04-16T18:11:47.221 DEBUG:boto:Data: 2024-04-16T18:11:47.221 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.221 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.221 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.222 DEBUG:boto:Params: {} 2024-04-16T18:11:47.222 DEBUG:boto:Token: None 2024-04-16T18:11:47.222 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.222 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.227 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7294dad54b3a9d7-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.227 DEBUG:boto:b'lybtug-46c1000falsec2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.227 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.227 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.227 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.227 DEBUG:boto:Path: /lybtug-46/c 2024-04-16T18:11:47.227 DEBUG:boto:Data: 2024-04-16T18:11:47.228 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.228 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.228 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.228 DEBUG:boto:Params: {} 2024-04-16T18:11:47.228 DEBUG:boto:Token: None 2024-04-16T18:11:47.228 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c 2024-04-16T18:11:47.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kO6HY7RKQu2ycKG7RndAeJ69l3g= 2024-04-16T18:11:47.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kO6HY7RKQu2ycKG7RndAeJ69l3g=', 'Content-Length': '0'} 2024-04-16T18:11:47.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee14a7a2e3c4a476-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.231 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.231 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.231 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.232 DEBUG:boto:Path: /lybtug-46/c 2024-04-16T18:11:47.232 DEBUG:boto:Data: 2024-04-16T18:11:47.232 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.232 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.232 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.232 DEBUG:boto:Params: {} 2024-04-16T18:11:47.232 DEBUG:boto:Token: None 2024-04-16T18:11:47.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c 2024-04-16T18:11:47.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kO6HY7RKQu2ycKG7RndAeJ69l3g= 2024-04-16T18:11:47.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kO6HY7RKQu2ycKG7RndAeJ69l3g=', 'Content-Length': '0'} 2024-04-16T18:11:47.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3af107a860c3868-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:47.235 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.235 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.235 DEBUG:boto:Method: GET 2024-04-16T18:11:47.235 DEBUG:boto:Path: /lybtug-46/c 2024-04-16T18:11:47.235 DEBUG:boto:Data: 2024-04-16T18:11:47.236 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.236 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.236 DEBUG:boto:Params: {} 2024-04-16T18:11:47.236 DEBUG:boto:Token: None 2024-04-16T18:11:47.236 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c 2024-04-16T18:11:47.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jMlAoG4e6Jgn5P+773SoHhTk4gw= 2024-04-16T18:11:47.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jMlAoG4e6Jgn5P+773SoHhTk4gw=', 'Content-Length': '0'} 2024-04-16T18:11:47.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:20 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028585f746264e1db-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.282 DEBUG:boto:path=/lybtug-46/c 2024-04-16T18:11:47.282 DEBUG:boto:auth_path=/lybtug-46/c 2024-04-16T18:11:47.283 DEBUG:boto:Method: GET 2024-04-16T18:11:47.283 DEBUG:boto:Path: /lybtug-46/c 2024-04-16T18:11:47.283 DEBUG:boto:Data: 2024-04-16T18:11:47.283 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.283 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.283 DEBUG:boto:Params: {} 2024-04-16T18:11:47.283 DEBUG:boto:Token: None 2024-04-16T18:11:47.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/c 2024-04-16T18:11:47.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jMlAoG4e6Jgn5P+773SoHhTk4gw= 2024-04-16T18:11:47.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jMlAoG4e6Jgn5P+773SoHhTk4gw=', 'Content-Length': '0'} 2024-04-16T18:11:47.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034dbabe1b0750ed8-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.287 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.287 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.287 DEBUG:boto:path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.287 DEBUG:boto:auth_path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.287 DEBUG:boto:Method: GET 2024-04-16T18:11:47.287 DEBUG:boto:Path: /lybtug-46/?prefix=c 2024-04-16T18:11:47.287 DEBUG:boto:Data: 2024-04-16T18:11:47.287 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.287 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.287 DEBUG:boto:Params: {} 2024-04-16T18:11:47.288 DEBUG:boto:Token: None 2024-04-16T18:11:47.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004578a313d0536bfd-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.292 DEBUG:boto:b'lybtug-46c1000falsec2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.292 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.292 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.292 DEBUG:boto:path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.292 DEBUG:boto:auth_path=/lybtug-46/?prefix=c 2024-04-16T18:11:47.293 DEBUG:boto:Method: GET 2024-04-16T18:11:47.293 DEBUG:boto:Path: /lybtug-46/?prefix=c 2024-04-16T18:11:47.293 DEBUG:boto:Data: 2024-04-16T18:11:47.293 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.293 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.293 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.293 DEBUG:boto:Params: {} 2024-04-16T18:11:47.293 DEBUG:boto:Token: None 2024-04-16T18:11:47.293 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.293 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.297 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000560d4118a825fb17-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.297 DEBUG:boto:b'lybtug-46c1000falsec2024-04-16T18:11:15.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.298 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:47.298 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.298 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.298 DEBUG:boto:path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.298 DEBUG:boto:auth_path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.298 DEBUG:boto:Method: GET 2024-04-16T18:11:47.298 DEBUG:boto:Path: /lybtug-46/d?versionId=null 2024-04-16T18:11:47.298 DEBUG:boto:Data: 2024-04-16T18:11:47.299 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.299 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.299 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.299 DEBUG:boto:Params: {} 2024-04-16T18:11:47.299 DEBUG:boto:Token: None 2024-04-16T18:11:47.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d?versionId=null 2024-04-16T18:11:47.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk= 2024-04-16T18:11:47.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk=', 'Content-Length': '0'} 2024-04-16T18:11:47.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcf928aadfd8b594-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.342 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.342 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.343 DEBUG:boto:path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.343 DEBUG:boto:auth_path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.343 DEBUG:boto:Method: GET 2024-04-16T18:11:47.343 DEBUG:boto:Path: /lybtug-46/d?versionId=null 2024-04-16T18:11:47.343 DEBUG:boto:Data: 2024-04-16T18:11:47.343 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.343 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.343 DEBUG:boto:Params: {} 2024-04-16T18:11:47.343 DEBUG:boto:Token: None 2024-04-16T18:11:47.343 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d?versionId=null 2024-04-16T18:11:47.343 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk= 2024-04-16T18:11:47.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk=', 'Content-Length': '0'} 2024-04-16T18:11:47.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8f21e07bb5d3f26-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.347 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.347 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.347 DEBUG:boto:path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.347 DEBUG:boto:auth_path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.347 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.347 DEBUG:boto:Path: /lybtug-46/d?versionId=null 2024-04-16T18:11:47.347 DEBUG:boto:Data: 2024-04-16T18:11:47.347 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.347 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.348 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.348 DEBUG:boto:Params: {} 2024-04-16T18:11:47.348 DEBUG:boto:Token: None 2024-04-16T18:11:47.348 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d?versionId=null 2024-04-16T18:11:47.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PU5zJzqpHNP2Rt9CE5QibEkKb6E= 2024-04-16T18:11:47.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PU5zJzqpHNP2Rt9CE5QibEkKb6E=', 'Content-Length': '0'} 2024-04-16T18:11:47.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007e5462c9b2f581c-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.351 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.351 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.351 DEBUG:boto:path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.351 DEBUG:boto:auth_path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.351 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.351 DEBUG:boto:Path: /lybtug-46/d?versionId=null 2024-04-16T18:11:47.351 DEBUG:boto:Data: 2024-04-16T18:11:47.352 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.352 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.352 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.352 DEBUG:boto:Params: {} 2024-04-16T18:11:47.352 DEBUG:boto:Token: None 2024-04-16T18:11:47.352 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d?versionId=null 2024-04-16T18:11:47.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PU5zJzqpHNP2Rt9CE5QibEkKb6E= 2024-04-16T18:11:47.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PU5zJzqpHNP2Rt9CE5QibEkKb6E=', 'Content-Length': '0'} 2024-04-16T18:11:47.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9728e58b9f882bb-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.355 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:47.355 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.355 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.356 DEBUG:boto:path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.356 DEBUG:boto:auth_path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.356 DEBUG:boto:Method: GET 2024-04-16T18:11:47.356 DEBUG:boto:Path: /lybtug-46/d?versionId=null 2024-04-16T18:11:47.356 DEBUG:boto:Data: 2024-04-16T18:11:47.356 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.356 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.356 DEBUG:boto:Params: {} 2024-04-16T18:11:47.356 DEBUG:boto:Token: None 2024-04-16T18:11:47.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d?versionId=null 2024-04-16T18:11:47.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk= 2024-04-16T18:11:47.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk=', 'Content-Length': '0'} 2024-04-16T18:11:47.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e52ef052448f2aeb-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.402 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.402 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.403 DEBUG:boto:path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.403 DEBUG:boto:auth_path=/lybtug-46/d?versionId=null 2024-04-16T18:11:47.403 DEBUG:boto:Method: GET 2024-04-16T18:11:47.403 DEBUG:boto:Path: /lybtug-46/d?versionId=null 2024-04-16T18:11:47.403 DEBUG:boto:Data: 2024-04-16T18:11:47.403 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.403 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.403 DEBUG:boto:Params: {} 2024-04-16T18:11:47.403 DEBUG:boto:Token: None 2024-04-16T18:11:47.403 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d?versionId=null 2024-04-16T18:11:47.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk= 2024-04-16T18:11:47.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CWsEUf92CE1jSrD7iYfOavc/hqk=', 'Content-Length': '0'} 2024-04-16T18:11:47.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008148a77a0cb6276b-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.407 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.407 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.407 DEBUG:boto:path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.407 DEBUG:boto:auth_path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.407 DEBUG:boto:Method: GET 2024-04-16T18:11:47.407 DEBUG:boto:Path: /lybtug-46/?prefix=d 2024-04-16T18:11:47.407 DEBUG:boto:Data: 2024-04-16T18:11:47.407 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.407 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.407 DEBUG:boto:Params: {} 2024-04-16T18:11:47.408 DEBUG:boto:Token: None 2024-04-16T18:11:47.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008acc64c0b6a5fb0e-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.412 DEBUG:boto:b'lybtug-46d1000falsed2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.413 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.413 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.413 DEBUG:boto:path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.413 DEBUG:boto:auth_path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.413 DEBUG:boto:Method: GET 2024-04-16T18:11:47.413 DEBUG:boto:Path: /lybtug-46/?prefix=d 2024-04-16T18:11:47.413 DEBUG:boto:Data: 2024-04-16T18:11:47.413 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.413 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.413 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.413 DEBUG:boto:Params: {} 2024-04-16T18:11:47.414 DEBUG:boto:Token: None 2024-04-16T18:11:47.414 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.414 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9e7c1fc154576d8-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.458 DEBUG:boto:b'lybtug-46d1000falsed2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.459 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.459 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.459 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.459 DEBUG:boto:Path: /lybtug-46/d 2024-04-16T18:11:47.459 DEBUG:boto:Data: 2024-04-16T18:11:47.459 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.459 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.460 DEBUG:boto:Params: {} 2024-04-16T18:11:47.460 DEBUG:boto:Token: None 2024-04-16T18:11:47.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d 2024-04-16T18:11:47.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ykd7LNGpCzTanyN2RyXyJnDtVOs= 2024-04-16T18:11:47.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ykd7LNGpCzTanyN2RyXyJnDtVOs=', 'Content-Length': '0'} 2024-04-16T18:11:47.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adf9b0630f726542-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.463 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.463 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.463 DEBUG:boto:Method: HEAD 2024-04-16T18:11:47.463 DEBUG:boto:Path: /lybtug-46/d 2024-04-16T18:11:47.463 DEBUG:boto:Data: 2024-04-16T18:11:47.463 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.464 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.464 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.464 DEBUG:boto:Params: {} 2024-04-16T18:11:47.464 DEBUG:boto:Token: None 2024-04-16T18:11:47.464 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d 2024-04-16T18:11:47.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ykd7LNGpCzTanyN2RyXyJnDtVOs= 2024-04-16T18:11:47.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ykd7LNGpCzTanyN2RyXyJnDtVOs=', 'Content-Length': '0'} 2024-04-16T18:11:47.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002392908e48579e2-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:47.467 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.468 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.468 DEBUG:boto:Method: GET 2024-04-16T18:11:47.468 DEBUG:boto:Path: /lybtug-46/d 2024-04-16T18:11:47.468 DEBUG:boto:Data: 2024-04-16T18:11:47.468 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.468 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.468 DEBUG:boto:Params: {} 2024-04-16T18:11:47.468 DEBUG:boto:Token: None 2024-04-16T18:11:47.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d 2024-04-16T18:11:47.469 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DzvPTIQEn3Xek2afFdIisJABYLM= 2024-04-16T18:11:47.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DzvPTIQEn3Xek2afFdIisJABYLM=', 'Content-Length': '0'} 2024-04-16T18:11:47.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-46:054102a3-8d94-4596-8b3b-6341eab312e1.4804.15'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092986a3ade6d47f7-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.514 DEBUG:boto:path=/lybtug-46/d 2024-04-16T18:11:47.514 DEBUG:boto:auth_path=/lybtug-46/d 2024-04-16T18:11:47.515 DEBUG:boto:Method: GET 2024-04-16T18:11:47.515 DEBUG:boto:Path: /lybtug-46/d 2024-04-16T18:11:47.515 DEBUG:boto:Data: 2024-04-16T18:11:47.515 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.515 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.515 DEBUG:boto:Params: {} 2024-04-16T18:11:47.515 DEBUG:boto:Token: None 2024-04-16T18:11:47.515 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/d 2024-04-16T18:11:47.515 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DzvPTIQEn3Xek2afFdIisJABYLM= 2024-04-16T18:11:47.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DzvPTIQEn3Xek2afFdIisJABYLM=', 'Content-Length': '0'} 2024-04-16T18:11:47.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:15 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c911fd3653dc2c5-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.518 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.519 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.519 DEBUG:boto:path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.519 DEBUG:boto:auth_path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.519 DEBUG:boto:Method: GET 2024-04-16T18:11:47.519 DEBUG:boto:Path: /lybtug-46/?prefix=d 2024-04-16T18:11:47.519 DEBUG:boto:Data: 2024-04-16T18:11:47.519 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.519 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:47.519 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.519 DEBUG:boto:Params: {} 2024-04-16T18:11:47.519 DEBUG:boto:Token: None 2024-04-16T18:11:47.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029c1295969ff6e7f-00661ebf63-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.566 DEBUG:boto:b'lybtug-46d1000falsed2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.567 DEBUG:boto:path=/lybtug-46/ 2024-04-16T18:11:47.567 DEBUG:boto:auth_path=/lybtug-46/ 2024-04-16T18:11:47.567 DEBUG:boto:path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.567 DEBUG:boto:auth_path=/lybtug-46/?prefix=d 2024-04-16T18:11:47.567 DEBUG:boto:Method: GET 2024-04-16T18:11:47.567 DEBUG:boto:Path: /lybtug-46/?prefix=d 2024-04-16T18:11:47.567 DEBUG:boto:Data: 2024-04-16T18:11:47.567 DEBUG:boto:Headers: {} 2024-04-16T18:11:47.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:47.567 DEBUG:boto:Port: 8000 2024-04-16T18:11:47.567 DEBUG:boto:Params: {} 2024-04-16T18:11:47.568 DEBUG:boto:Token: None 2024-04-16T18:11:47.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:47 GMT /lybtug-46/ 2024-04-16T18:11:47.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U= 2024-04-16T18:11:47.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:muBlFto5NGhhXAdlrrYeJ9Rgf6U=', 'Content-Length': '0'} 2024-04-16T18:11:47.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7618ce26d4d6a43-00661ebf63-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:47.572 DEBUG:boto:b'lybtug-46d1000falsed2024-04-16T18:11:15.854Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:47.572 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-46 zones={test-zone2, test-zone1} 2024-04-16T18:11:47.572 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:47.572 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:47.573 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 1 --rgw-sync-log-trim-min-cold-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:11:47.612 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:47.612 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:47.736 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:47.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:47.736 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-46 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:47.815 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:47.816 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:47.917 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:11:47.918 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:47.918 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:47.918 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:47.998 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:47.999 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:48.100 INFO:teuthology.orchestra.run.smithi071.stdout:[{"op_id":"1#00000000001.102.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.16726075910046676339","op":"write","object":"a","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:11:08.603780457Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14"}]},{"op_id":"5#00000000001.60.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10830716936616374218","op":"write","object":"d","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:11:08.720615120Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14"}]},{"op_id":"8#00000000001.62.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10515048560689040742","op":"write","object":"c","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:11:08.671567175Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14"}]},{"op_id":"10#00000000001.27.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6426660268614581114","op":"write","object":"b","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:11:08.652681151Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-45:054102a3-8d94-4596-8b3b-6341eab312e1.4804.14"}]}] 2024-04-16T18:11:48.101 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '999', '--rgw-sync-log-trim-min-cold-buckets', '999', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:48.101 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '999', '--rgw-sync-log-trim-min-cold-buckets', '999', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:48.101 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 999 --rgw-sync-log-trim-min-cold-buckets 999 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:11:48.175 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:48.176 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:48.878 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:48.881+0000 7f4a40745640 -1 req 55557711490417087 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-43:054102a3-8d94-4596-8b3b-6341eab312e1.4804.12,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-43:054102a3-8d94-4596-8b3b-6341eab312e1.4804.12,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:48.878 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:48.885+0000 7f4a10ee6640 -1 req 7450565648147883904 0.007999765s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-25:054102a3-8d94-4596-8b3b-6341eab312e1.4353.29,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-25:054102a3-8d94-4596-8b3b-6341eab312e1.4353.29,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:48.889 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:48.897+0000 7f4a086d5640 -1 req 5942632925262797834 0.007999764s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-34:054102a3-8d94-4596-8b3b-6341eab312e1.4804.3,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-34:054102a3-8d94-4596-8b3b-6341eab312e1.4804.3,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:48.891 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:48.897+0000 7f4a68795640 -1 req 7331326067742828928 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:48.944 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:48.949+0000 7f4a72faa640 -1 req 4244732700991267198 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-24:054102a3-8d94-4596-8b3b-6341eab312e1.4353.28,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-24:054102a3-8d94-4596-8b3b-6341eab312e1.4353.28,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:48.944 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:48.949+0000 7f4a34f2e640 -1 req 3397970669550954846 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-21:054102a3-8d94-4596-8b3b-6341eab312e1.4353.25,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-21:054102a3-8d94-4596-8b3b-6341eab312e1.4353.25,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.000 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.005+0000 7f4a11ee8640 -1 req 8412145451332716333 0.003999884s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-33:054102a3-8d94-4596-8b3b-6341eab312e1.4804.2,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-33:054102a3-8d94-4596-8b3b-6341eab312e1.4804.2,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.001 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.005+0000 7f4a1aefa640 -1 req 15115428322093332039 0.003999884s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.007 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.013+0000 7f49d7e74640 -1 req 16684681170204711652 0.007999764s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-26:054102a3-8d94-4596-8b3b-6341eab312e1.4353.30,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-26:054102a3-8d94-4596-8b3b-6341eab312e1.4353.30,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.107 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.113+0000 7f49fa6b9640 -1 req 17248798106278234227 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-32:054102a3-8d94-4596-8b3b-6341eab312e1.4804.1,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-32:054102a3-8d94-4596-8b3b-6341eab312e1.4804.1,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.120 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.125+0000 7f4a3ff44640 -1 req 15012354856661895992 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-35:054102a3-8d94-4596-8b3b-6341eab312e1.4804.4,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-35:054102a3-8d94-4596-8b3b-6341eab312e1.4804.4,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.219 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.225+0000 7f4a2570f640 -1 req 1428292960640768938 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-23:054102a3-8d94-4596-8b3b-6341eab312e1.4353.27,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-23:054102a3-8d94-4596-8b3b-6341eab312e1.4353.27,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.271 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.277+0000 7f49dce7e640 -1 req 6485194387815518372 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-5:054102a3-8d94-4596-8b3b-6341eab312e1.4353.9,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.356 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.361+0000 7f4a26f12640 -1 req 6227013847393348800 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-2:054102a3-8d94-4596-8b3b-6341eab312e1.4353.2,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.380 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.385+0000 7f49cd65f640 -1 req 806961164287164648 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-3:054102a3-8d94-4596-8b3b-6341eab312e1.4353.3,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.403 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:11:49.409+0000 7f4a41f48640 -1 req 8113751176057790736 0.003999882s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-22:054102a3-8d94-4596-8b3b-6341eab312e1.4353.26,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-22:054102a3-8d94-4596-8b3b-6341eab312e1.4353.26,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:11:49.556 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:49.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:49.556 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:49.640 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:49.641 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:49.741 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:11:49.742 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_index_log_trim ... ok 2024-04-16T18:11:49.743 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:49.743 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:49.743 DEBUG:boto:Method: PUT 2024-04-16T18:11:49.743 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:49.743 DEBUG:boto:Data: 2024-04-16T18:11:49.743 DEBUG:boto:Headers: {} 2024-04-16T18:11:49.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:49.744 DEBUG:boto:Port: 8000 2024-04-16T18:11:49.744 DEBUG:boto:Params: {} 2024-04-16T18:11:49.744 DEBUG:boto:Token: None 2024-04-16T18:11:49.744 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:11:49 GMT /lybtug-47/ 2024-04-16T18:11:49.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ejy9uZBBAPj9zkWtdv06pth62K0= 2024-04-16T18:11:49.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ejy9uZBBAPj9zkWtdv06pth62K0=', 'Content-Length': '0'} 2024-04-16T18:11:49.777 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004be6744dde22dcff-00661ebf65-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:49.777 DEBUG:rgw_multi.tests:created bucket=lybtug-47 2024-04-16T18:11:49.778 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:49.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:49.778 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:49.820 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:49.820 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:49.888 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:11:49.922 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291109.778031_453.1","last_update":"2024-04-16T18:11:49.778031Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:11:49.923 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291109.778031_453.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:49.923 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:11:49.923 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:49.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:11:49.923 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:11:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:50.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:50.041+0000 7f7a2c383980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:50.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:50.045+0000 7f7a2c383980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:11:50.071 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291109.778031_453.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:49.778031Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:50Z"} 2024-04-16T18:11:50.071 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291109.778031_453.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:11:49.778031Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:11:50Z"} 2024-04-16T18:11:50.071 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291109.778031_453.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:11:49.778031Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:11:50.071 DEBUG:rgw_multi.tests:log_status={0: '1_1713291109.778031_453.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:50.071 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291109.778031_453.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:11:50.071 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:11:50.072 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:50.072 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:50.072 DEBUG:boto:Method: HEAD 2024-04-16T18:11:50.072 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:50.072 DEBUG:boto:Data: 2024-04-16T18:11:50.072 DEBUG:boto:Headers: {} 2024-04-16T18:11:50.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.072 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.072 DEBUG:boto:Params: {} 2024-04-16T18:11:50.072 DEBUG:boto:Token: None 2024-04-16T18:11:50.072 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/ 2024-04-16T18:11:50.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE= 2024-04-16T18:11:50.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE=', 'Content-Length': '0'} 2024-04-16T18:11:50.077 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cb874a6710319807-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.077 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:50.077 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:50.078 DEBUG:boto:Method: PUT 2024-04-16T18:11:50.078 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:11:50.078 DEBUG:boto:Data: 2024-04-16T18:11:50.078 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:50.078 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.078 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.078 DEBUG:boto:Params: {} 2024-04-16T18:11:50.078 DEBUG:boto:Token: None 2024-04-16T18:11:50.078 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/a 2024-04-16T18:11:50.078 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MaGHyweTR9tFJ//U8WFMkVYXYEk= 2024-04-16T18:11:50.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MaGHyweTR9tFJ//U8WFMkVYXYEk='} 2024-04-16T18:11:50.092 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000035f16ceea2884311-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.093 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:50.093 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:50.093 DEBUG:boto:Method: HEAD 2024-04-16T18:11:50.093 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:50.093 DEBUG:boto:Data: 2024-04-16T18:11:50.093 DEBUG:boto:Headers: {} 2024-04-16T18:11:50.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.093 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.093 DEBUG:boto:Params: {} 2024-04-16T18:11:50.093 DEBUG:boto:Token: None 2024-04-16T18:11:50.093 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/ 2024-04-16T18:11:50.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE= 2024-04-16T18:11:50.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE=', 'Content-Length': '0'} 2024-04-16T18:11:50.097 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000da778b8d52fb2ffb-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.097 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:50.098 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:50.098 DEBUG:boto:Method: PUT 2024-04-16T18:11:50.098 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:11:50.098 DEBUG:boto:Data: 2024-04-16T18:11:50.098 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:50.098 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.098 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.098 DEBUG:boto:Params: {} 2024-04-16T18:11:50.098 DEBUG:boto:Token: None 2024-04-16T18:11:50.098 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/b 2024-04-16T18:11:50.099 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FqsRlkfZkxVElY8HX+jah6oBVnI= 2024-04-16T18:11:50.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FqsRlkfZkxVElY8HX+jah6oBVnI='} 2024-04-16T18:11:50.142 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fbbe96ab4c35246f-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.143 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:50.143 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:50.143 DEBUG:boto:Method: HEAD 2024-04-16T18:11:50.143 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:50.143 DEBUG:boto:Data: 2024-04-16T18:11:50.143 DEBUG:boto:Headers: {} 2024-04-16T18:11:50.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.143 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.143 DEBUG:boto:Params: {} 2024-04-16T18:11:50.143 DEBUG:boto:Token: None 2024-04-16T18:11:50.143 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/ 2024-04-16T18:11:50.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE= 2024-04-16T18:11:50.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE=', 'Content-Length': '0'} 2024-04-16T18:11:50.147 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000027c8dac4480234c2-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.147 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:50.147 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:50.147 DEBUG:boto:Method: PUT 2024-04-16T18:11:50.147 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:11:50.147 DEBUG:boto:Data: 2024-04-16T18:11:50.148 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:50.148 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.148 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.148 DEBUG:boto:Params: {} 2024-04-16T18:11:50.148 DEBUG:boto:Token: None 2024-04-16T18:11:50.148 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/c 2024-04-16T18:11:50.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M1yWPwScjR3moJx32kXdITpC8XA= 2024-04-16T18:11:50.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M1yWPwScjR3moJx32kXdITpC8XA='} 2024-04-16T18:11:50.159 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b83cbf24d761f9dd-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.159 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:50.159 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:50.160 DEBUG:boto:Method: HEAD 2024-04-16T18:11:50.160 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:50.160 DEBUG:boto:Data: 2024-04-16T18:11:50.160 DEBUG:boto:Headers: {} 2024-04-16T18:11:50.160 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.160 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.160 DEBUG:boto:Params: {} 2024-04-16T18:11:50.160 DEBUG:boto:Token: None 2024-04-16T18:11:50.160 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/ 2024-04-16T18:11:50.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE= 2024-04-16T18:11:50.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rlKQ5B20G30hV69CGBp0tXHtbxE=', 'Content-Length': '0'} 2024-04-16T18:11:50.164 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ac979a91f7ee226d-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.165 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:50.165 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:50.165 DEBUG:boto:Method: PUT 2024-04-16T18:11:50.165 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:11:50.165 DEBUG:boto:Data: 2024-04-16T18:11:50.165 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:50.165 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:50.165 DEBUG:boto:Port: 8000 2024-04-16T18:11:50.165 DEBUG:boto:Params: {} 2024-04-16T18:11:50.165 DEBUG:boto:Token: None 2024-04-16T18:11:50.166 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:50 GMT /lybtug-47/d 2024-04-16T18:11:50.166 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w8sFcGLWOgLI+pDDz5QP7RRkBtY= 2024-04-16T18:11:50.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w8sFcGLWOgLI+pDDz5QP7RRkBtY='} 2024-04-16T18:11:50.210 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005ff112d018518ac7-00661ebf66-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:50.211 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-47 2024-04-16T18:11:50.211 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:50.211 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:50.211 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-47 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:50.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:50.349+0000 7f811e2c6980 1 waiting to reach incremental sync.. 2024-04-16T18:11:55.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:55.352+0000 7f811e2c6980 1 bucket sync caught up with source: 2024-04-16T18:11:55.347 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.110.6, , , , 00000000001.55.6, , , 00000000001.41.6, , 00000000001.39.6] 2024-04-16T18:11:55.347 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000001.110.6, , , , 00000000001.55.6, , , 00000000001.41.6, , 00000000001.39.6] 2024-04-16T18:11:55.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:55.352+0000 7f811e2c6980 0 bucket checkpoint complete 2024-04-16T18:11:55.394 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-47 2024-04-16T18:11:55.394 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:55.394 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:55.394 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-47 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:55.444 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:55.444 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:55.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:55.526+0000 7fdef1c75980 1 bucket sync caught up with source: 2024-04-16T18:11:55.524 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.889.6, , , , 00000000001.1147.6, , , 00000000001.1342.6, , 00000000001.1340.6] 2024-04-16T18:11:55.524 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.889.6, , , , 00000000001.1147.6, , , 00000000001.1342.6, , 00000000001.1340.6] 2024-04-16T18:11:55.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:11:55.526+0000 7fdef1c75980 0 bucket checkpoint complete 2024-04-16T18:11:55.555 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-47 zones={test-zone2, test-zone1} 2024-04-16T18:11:55.555 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.555 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.556 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.556 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:55.556 DEBUG:boto:Data: 2024-04-16T18:11:55.556 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.556 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.556 DEBUG:boto:Params: {} 2024-04-16T18:11:55.556 DEBUG:boto:Token: None 2024-04-16T18:11:55.556 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FpwZMgOkJdx4it5ieX2+qX790tg= 2024-04-16T18:11:55.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FpwZMgOkJdx4it5ieX2+qX790tg=', 'Content-Length': '0'} 2024-04-16T18:11:55.561 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001c93beff5d725351-00661ebf6b-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.561 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.561 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.561 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.561 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:55.561 DEBUG:boto:Data: 2024-04-16T18:11:55.562 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.562 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.562 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.562 DEBUG:boto:Params: {} 2024-04-16T18:11:55.562 DEBUG:boto:Token: None 2024-04-16T18:11:55.562 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.562 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FpwZMgOkJdx4it5ieX2+qX790tg= 2024-04-16T18:11:55.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FpwZMgOkJdx4it5ieX2+qX790tg=', 'Content-Length': '0'} 2024-04-16T18:11:55.565 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000036d4512b7ee9749c-00661ebf6b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.565 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:11:55.565 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.565 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.565 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.565 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.566 DEBUG:boto:Method: GET 2024-04-16T18:11:55.566 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.566 DEBUG:boto:Data: 2024-04-16T18:11:55.566 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.566 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.566 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.566 DEBUG:boto:Params: {} 2024-04-16T18:11:55.566 DEBUG:boto:Token: None 2024-04-16T18:11:55.566 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/?versions 2024-04-16T18:11:55.566 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ= 2024-04-16T18:11:55.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ=', 'Content-Length': '0'} 2024-04-16T18:11:55.571 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019317e411bea5eed-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.571 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.572 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:11:55.572 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:11:55.572 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:11:55.572 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:11:55.572 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:11:55.572 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.572 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.572 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.572 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.572 DEBUG:boto:Method: GET 2024-04-16T18:11:55.572 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.572 DEBUG:boto:Data: 2024-04-16T18:11:55.573 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.573 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.573 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.573 DEBUG:boto:Params: {} 2024-04-16T18:11:55.573 DEBUG:boto:Token: None 2024-04-16T18:11:55.573 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/?versions 2024-04-16T18:11:55.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ= 2024-04-16T18:11:55.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ=', 'Content-Length': '0'} 2024-04-16T18:11:55.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046e797d601a5b52f-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.576 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.577 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:11:55.577 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:11:55.577 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:11:55.577 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:11:55.577 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.577 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.577 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.577 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.577 DEBUG:boto:Method: GET 2024-04-16T18:11:55.577 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.577 DEBUG:boto:Data: 2024-04-16T18:11:55.578 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.578 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.578 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.578 DEBUG:boto:Params: {} 2024-04-16T18:11:55.578 DEBUG:boto:Token: None 2024-04-16T18:11:55.578 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/?versions 2024-04-16T18:11:55.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ= 2024-04-16T18:11:55.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ=', 'Content-Length': '0'} 2024-04-16T18:11:55.582 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe8688fdeb016aa2-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.583 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.583 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.583 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.583 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.583 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.583 DEBUG:boto:Method: GET 2024-04-16T18:11:55.584 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:11:55.584 DEBUG:boto:Data: 2024-04-16T18:11:55.584 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.584 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.584 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.584 DEBUG:boto:Params: {} 2024-04-16T18:11:55.584 DEBUG:boto:Token: None 2024-04-16T18:11:55.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/?versions 2024-04-16T18:11:55.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ= 2024-04-16T18:11:55.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YfvK2qHVICXgQxhwIYnX+cRfqEQ=', 'Content-Length': '0'} 2024-04-16T18:11:55.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b46c7dc42bba6ce9-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.587 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.587 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:55.587 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.587 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.588 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.588 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.588 DEBUG:boto:Method: GET 2024-04-16T18:11:55.588 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:11:55.588 DEBUG:boto:Data: 2024-04-16T18:11:55.588 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.588 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.588 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.588 DEBUG:boto:Params: {} 2024-04-16T18:11:55.588 DEBUG:boto:Token: None 2024-04-16T18:11:55.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a?versionId=null 2024-04-16T18:11:55.589 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo= 2024-04-16T18:11:55.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo=', 'Content-Length': '0'} 2024-04-16T18:11:55.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9b159cc8e3af9fc-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.634 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.634 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.635 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.635 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.635 DEBUG:boto:Method: GET 2024-04-16T18:11:55.635 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:11:55.635 DEBUG:boto:Data: 2024-04-16T18:11:55.635 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.635 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.635 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.635 DEBUG:boto:Params: {} 2024-04-16T18:11:55.635 DEBUG:boto:Token: None 2024-04-16T18:11:55.635 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a?versionId=null 2024-04-16T18:11:55.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo= 2024-04-16T18:11:55.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo=', 'Content-Length': '0'} 2024-04-16T18:11:55.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cc18d153b26e2fd-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.639 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.639 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.639 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.639 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.639 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.639 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:11:55.639 DEBUG:boto:Data: 2024-04-16T18:11:55.640 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.640 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.640 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.640 DEBUG:boto:Params: {} 2024-04-16T18:11:55.640 DEBUG:boto:Token: None 2024-04-16T18:11:55.640 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a?versionId=null 2024-04-16T18:11:55.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eIV+AQ8oojdbbLaZb+pvQlPXbtg= 2024-04-16T18:11:55.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eIV+AQ8oojdbbLaZb+pvQlPXbtg=', 'Content-Length': '0'} 2024-04-16T18:11:55.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c16cf74d2c0f3a26-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.643 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.644 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.644 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.644 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.644 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.644 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:11:55.644 DEBUG:boto:Data: 2024-04-16T18:11:55.644 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.644 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.644 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.644 DEBUG:boto:Params: {} 2024-04-16T18:11:55.644 DEBUG:boto:Token: None 2024-04-16T18:11:55.645 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a?versionId=null 2024-04-16T18:11:55.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eIV+AQ8oojdbbLaZb+pvQlPXbtg= 2024-04-16T18:11:55.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eIV+AQ8oojdbbLaZb+pvQlPXbtg=', 'Content-Length': '0'} 2024-04-16T18:11:55.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000632e182941b1a4d3-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:55.648 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.648 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.648 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.648 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.648 DEBUG:boto:Method: GET 2024-04-16T18:11:55.648 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:11:55.648 DEBUG:boto:Data: 2024-04-16T18:11:55.649 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.649 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.649 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.649 DEBUG:boto:Params: {} 2024-04-16T18:11:55.649 DEBUG:boto:Token: None 2024-04-16T18:11:55.649 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a?versionId=null 2024-04-16T18:11:55.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo= 2024-04-16T18:11:55.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo=', 'Content-Length': '0'} 2024-04-16T18:11:55.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba694e43457ff669-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.694 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.694 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.695 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.695 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:11:55.695 DEBUG:boto:Method: GET 2024-04-16T18:11:55.695 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:11:55.695 DEBUG:boto:Data: 2024-04-16T18:11:55.695 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.695 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.695 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.695 DEBUG:boto:Params: {} 2024-04-16T18:11:55.695 DEBUG:boto:Token: None 2024-04-16T18:11:55.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a?versionId=null 2024-04-16T18:11:55.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo= 2024-04-16T18:11:55.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OKO6S1y1ETbIzvf8AWNCfqwbgGo=', 'Content-Length': '0'} 2024-04-16T18:11:55.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c6a5e2624ed496d-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.699 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.699 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.699 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.699 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.699 DEBUG:boto:Method: GET 2024-04-16T18:11:55.699 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:11:55.700 DEBUG:boto:Data: 2024-04-16T18:11:55.700 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.700 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.700 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.700 DEBUG:boto:Params: {} 2024-04-16T18:11:55.700 DEBUG:boto:Token: None 2024-04-16T18:11:55.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo= 2024-04-16T18:11:55.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo=', 'Content-Length': '0'} 2024-04-16T18:11:55.705 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000520ad267c0cd64d0-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.746 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.747 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.747 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.747 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.747 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.747 DEBUG:boto:Method: GET 2024-04-16T18:11:55.747 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:11:55.747 DEBUG:boto:Data: 2024-04-16T18:11:55.747 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.747 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.747 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.747 DEBUG:boto:Params: {} 2024-04-16T18:11:55.748 DEBUG:boto:Token: None 2024-04-16T18:11:55.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo= 2024-04-16T18:11:55.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo=', 'Content-Length': '0'} 2024-04-16T18:11:55.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbe863c37c3d5311-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.752 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.753 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.753 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.753 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.753 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:11:55.753 DEBUG:boto:Data: 2024-04-16T18:11:55.753 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.753 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.753 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.753 DEBUG:boto:Params: {} 2024-04-16T18:11:55.754 DEBUG:boto:Token: None 2024-04-16T18:11:55.754 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a 2024-04-16T18:11:55.754 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F2GHDfNCwU+YIdRU3AOo9GGJ/ok= 2024-04-16T18:11:55.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F2GHDfNCwU+YIdRU3AOo9GGJ/ok=', 'Content-Length': '0'} 2024-04-16T18:11:55.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c641d00afc8a688-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.757 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.757 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.757 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.757 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:11:55.757 DEBUG:boto:Data: 2024-04-16T18:11:55.757 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.758 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.758 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.758 DEBUG:boto:Params: {} 2024-04-16T18:11:55.758 DEBUG:boto:Token: None 2024-04-16T18:11:55.758 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a 2024-04-16T18:11:55.758 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F2GHDfNCwU+YIdRU3AOo9GGJ/ok= 2024-04-16T18:11:55.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F2GHDfNCwU+YIdRU3AOo9GGJ/ok=', 'Content-Length': '0'} 2024-04-16T18:11:55.761 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000822f6beae5f0825c-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.761 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:11:55.761 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.762 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.762 DEBUG:boto:Method: GET 2024-04-16T18:11:55.762 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:11:55.762 DEBUG:boto:Data: 2024-04-16T18:11:55.762 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.762 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.762 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.762 DEBUG:boto:Params: {} 2024-04-16T18:11:55.762 DEBUG:boto:Token: None 2024-04-16T18:11:55.763 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a 2024-04-16T18:11:55.763 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vMx0jdqbUtr20hU7Kx0TPknAbFs= 2024-04-16T18:11:55.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vMx0jdqbUtr20hU7Kx0TPknAbFs=', 'Content-Length': '0'} 2024-04-16T18:11:55.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3823fb02a40d1e7-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.806 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:11:55.806 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:11:55.807 DEBUG:boto:Method: GET 2024-04-16T18:11:55.807 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:11:55.807 DEBUG:boto:Data: 2024-04-16T18:11:55.807 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.807 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.807 DEBUG:boto:Params: {} 2024-04-16T18:11:55.807 DEBUG:boto:Token: None 2024-04-16T18:11:55.807 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/a 2024-04-16T18:11:55.807 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vMx0jdqbUtr20hU7Kx0TPknAbFs= 2024-04-16T18:11:55.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vMx0jdqbUtr20hU7Kx0TPknAbFs=', 'Content-Length': '0'} 2024-04-16T18:11:55.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c74eb15ee185fdbd-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.811 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.811 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.811 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.811 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.811 DEBUG:boto:Method: GET 2024-04-16T18:11:55.811 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:11:55.811 DEBUG:boto:Data: 2024-04-16T18:11:55.811 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.811 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.812 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.812 DEBUG:boto:Params: {} 2024-04-16T18:11:55.812 DEBUG:boto:Token: None 2024-04-16T18:11:55.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo= 2024-04-16T18:11:55.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo=', 'Content-Length': '0'} 2024-04-16T18:11:55.817 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050219e305dabe44f-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.817 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.817 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.817 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.817 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.817 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:11:55.817 DEBUG:boto:Method: GET 2024-04-16T18:11:55.818 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:11:55.818 DEBUG:boto:Data: 2024-04-16T18:11:55.818 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.818 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.818 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.818 DEBUG:boto:Params: {} 2024-04-16T18:11:55.818 DEBUG:boto:Token: None 2024-04-16T18:11:55.818 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.818 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo= 2024-04-16T18:11:55.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo=', 'Content-Length': '0'} 2024-04-16T18:11:55.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006749829a79168259-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.823 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.823 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:55.823 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.823 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.823 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.823 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.824 DEBUG:boto:Method: GET 2024-04-16T18:11:55.824 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:11:55.824 DEBUG:boto:Data: 2024-04-16T18:11:55.824 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.824 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.824 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.824 DEBUG:boto:Params: {} 2024-04-16T18:11:55.824 DEBUG:boto:Token: None 2024-04-16T18:11:55.824 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b?versionId=null 2024-04-16T18:11:55.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE= 2024-04-16T18:11:55.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE=', 'Content-Length': '0'} 2024-04-16T18:11:55.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2748426155058cf-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.870 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.870 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.871 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.871 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.871 DEBUG:boto:Method: GET 2024-04-16T18:11:55.871 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:11:55.871 DEBUG:boto:Data: 2024-04-16T18:11:55.871 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.871 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.871 DEBUG:boto:Params: {} 2024-04-16T18:11:55.871 DEBUG:boto:Token: None 2024-04-16T18:11:55.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b?versionId=null 2024-04-16T18:11:55.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE= 2024-04-16T18:11:55.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE=', 'Content-Length': '0'} 2024-04-16T18:11:55.875 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcce211f0e8c870e-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.875 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.875 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.875 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.876 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.876 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.876 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:11:55.876 DEBUG:boto:Data: 2024-04-16T18:11:55.876 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.876 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.876 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.876 DEBUG:boto:Params: {} 2024-04-16T18:11:55.876 DEBUG:boto:Token: None 2024-04-16T18:11:55.876 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b?versionId=null 2024-04-16T18:11:55.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vurqlIu6iZhICdY2mykns+uN2/o= 2024-04-16T18:11:55.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vurqlIu6iZhICdY2mykns+uN2/o=', 'Content-Length': '0'} 2024-04-16T18:11:55.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000092635393247d882-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.880 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.880 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.880 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.880 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.880 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.880 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:11:55.880 DEBUG:boto:Data: 2024-04-16T18:11:55.880 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.880 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.880 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.881 DEBUG:boto:Params: {} 2024-04-16T18:11:55.881 DEBUG:boto:Token: None 2024-04-16T18:11:55.881 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b?versionId=null 2024-04-16T18:11:55.881 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vurqlIu6iZhICdY2mykns+uN2/o= 2024-04-16T18:11:55.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vurqlIu6iZhICdY2mykns+uN2/o=', 'Content-Length': '0'} 2024-04-16T18:11:55.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1aaea2ff141575e-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.884 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:55.885 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.885 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.885 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.885 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.885 DEBUG:boto:Method: GET 2024-04-16T18:11:55.885 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:11:55.885 DEBUG:boto:Data: 2024-04-16T18:11:55.885 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.885 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.885 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.885 DEBUG:boto:Params: {} 2024-04-16T18:11:55.886 DEBUG:boto:Token: None 2024-04-16T18:11:55.886 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b?versionId=null 2024-04-16T18:11:55.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE= 2024-04-16T18:11:55.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE=', 'Content-Length': '0'} 2024-04-16T18:11:55.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea54975305e6f97d-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.930 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.930 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.931 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.931 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:11:55.931 DEBUG:boto:Method: GET 2024-04-16T18:11:55.931 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:11:55.931 DEBUG:boto:Data: 2024-04-16T18:11:55.931 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.931 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.931 DEBUG:boto:Params: {} 2024-04-16T18:11:55.931 DEBUG:boto:Token: None 2024-04-16T18:11:55.931 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b?versionId=null 2024-04-16T18:11:55.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE= 2024-04-16T18:11:55.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N5082LFSxvJlw/5xL3CWcLCz1YE=', 'Content-Length': '0'} 2024-04-16T18:11:55.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfd2cd33d1f3befb-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.935 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.935 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.935 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:11:55.935 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:11:55.935 DEBUG:boto:Method: GET 2024-04-16T18:11:55.936 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:11:55.936 DEBUG:boto:Data: 2024-04-16T18:11:55.936 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.936 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.936 DEBUG:boto:Params: {} 2024-04-16T18:11:55.936 DEBUG:boto:Token: None 2024-04-16T18:11:55.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo= 2024-04-16T18:11:55.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo=', 'Content-Length': '0'} 2024-04-16T18:11:55.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a0589badb15e3fd-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.982 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.983 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:55.983 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:55.983 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:11:55.983 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:11:55.983 DEBUG:boto:Method: GET 2024-04-16T18:11:55.983 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:11:55.983 DEBUG:boto:Data: 2024-04-16T18:11:55.983 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.983 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.984 DEBUG:boto:Params: {} 2024-04-16T18:11:55.984 DEBUG:boto:Token: None 2024-04-16T18:11:55.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/ 2024-04-16T18:11:55.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo= 2024-04-16T18:11:55.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UDdx+hKqvC/5Ms+GDSYPmEsRVDo=', 'Content-Length': '0'} 2024-04-16T18:11:55.988 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d499b4111b962a79-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.989 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:55.989 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.989 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.989 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.989 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:11:55.989 DEBUG:boto:Data: 2024-04-16T18:11:55.989 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.990 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.990 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.990 DEBUG:boto:Params: {} 2024-04-16T18:11:55.990 DEBUG:boto:Token: None 2024-04-16T18:11:55.990 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b 2024-04-16T18:11:55.990 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QOTiWEYr28Qmo396BBpuhPRHCZE= 2024-04-16T18:11:55.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QOTiWEYr28Qmo396BBpuhPRHCZE=', 'Content-Length': '0'} 2024-04-16T18:11:55.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb15e48bc84b8827-00661ebf6b-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.994 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.994 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.994 DEBUG:boto:Method: HEAD 2024-04-16T18:11:55.994 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:11:55.994 DEBUG:boto:Data: 2024-04-16T18:11:55.994 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.994 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:55.994 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.994 DEBUG:boto:Params: {} 2024-04-16T18:11:55.994 DEBUG:boto:Token: None 2024-04-16T18:11:55.995 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b 2024-04-16T18:11:55.995 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QOTiWEYr28Qmo396BBpuhPRHCZE= 2024-04-16T18:11:55.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QOTiWEYr28Qmo396BBpuhPRHCZE=', 'Content-Length': '0'} 2024-04-16T18:11:55.998 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c1bf5b0aed05fb6-00661ebf6b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:55.998 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:11:55.998 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:55.998 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:55.998 DEBUG:boto:Method: GET 2024-04-16T18:11:55.999 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:11:55.999 DEBUG:boto:Data: 2024-04-16T18:11:55.999 DEBUG:boto:Headers: {} 2024-04-16T18:11:55.999 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:55.999 DEBUG:boto:Port: 8000 2024-04-16T18:11:55.999 DEBUG:boto:Params: {} 2024-04-16T18:11:55.999 DEBUG:boto:Token: None 2024-04-16T18:11:55.999 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:55 GMT /lybtug-47/b 2024-04-16T18:11:55.999 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zauXx8vvpiGxdUe6mvDxJDHafgY= 2024-04-16T18:11:55.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zauXx8vvpiGxdUe6mvDxJDHafgY=', 'Content-Length': '0'} 2024-04-16T18:11:56.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d9b94e9ac2f36d9-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.046 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:11:56.046 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:11:56.047 DEBUG:boto:Method: GET 2024-04-16T18:11:56.047 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:11:56.047 DEBUG:boto:Data: 2024-04-16T18:11:56.047 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.047 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.047 DEBUG:boto:Params: {} 2024-04-16T18:11:56.047 DEBUG:boto:Token: None 2024-04-16T18:11:56.047 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/b 2024-04-16T18:11:56.047 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vQUysEY1ILXd+YUHxKD0PrtKzr8= 2024-04-16T18:11:56.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vQUysEY1ILXd+YUHxKD0PrtKzr8=', 'Content-Length': '0'} 2024-04-16T18:11:56.050 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6b5a9b1c65bec0e-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.051 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.051 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.051 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:11:56.051 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:11:56.051 DEBUG:boto:Method: GET 2024-04-16T18:11:56.051 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:11:56.051 DEBUG:boto:Data: 2024-04-16T18:11:56.051 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.052 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.052 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.052 DEBUG:boto:Params: {} 2024-04-16T18:11:56.052 DEBUG:boto:Token: None 2024-04-16T18:11:56.052 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3e06d1816b63ac1-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.057 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.057 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.057 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.057 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:11:56.058 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:11:56.058 DEBUG:boto:Method: GET 2024-04-16T18:11:56.058 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:11:56.058 DEBUG:boto:Data: 2024-04-16T18:11:56.058 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.058 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.058 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.058 DEBUG:boto:Params: {} 2024-04-16T18:11:56.058 DEBUG:boto:Token: None 2024-04-16T18:11:56.058 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.059 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045ab7e83e5512a91-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.063 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.063 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:56.063 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.064 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.064 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.064 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.064 DEBUG:boto:Method: GET 2024-04-16T18:11:56.064 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:11:56.064 DEBUG:boto:Data: 2024-04-16T18:11:56.064 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.064 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.064 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.064 DEBUG:boto:Params: {} 2024-04-16T18:11:56.064 DEBUG:boto:Token: None 2024-04-16T18:11:56.065 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c?versionId=null 2024-04-16T18:11:56.065 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY= 2024-04-16T18:11:56.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY=', 'Content-Length': '0'} 2024-04-16T18:11:56.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014cd9fe2bc41b79b-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.110 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.111 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.111 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.111 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.111 DEBUG:boto:Method: GET 2024-04-16T18:11:56.111 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:11:56.111 DEBUG:boto:Data: 2024-04-16T18:11:56.111 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.111 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.111 DEBUG:boto:Params: {} 2024-04-16T18:11:56.111 DEBUG:boto:Token: None 2024-04-16T18:11:56.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c?versionId=null 2024-04-16T18:11:56.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY= 2024-04-16T18:11:56.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY=', 'Content-Length': '0'} 2024-04-16T18:11:56.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a052504d76e54ee4-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.115 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.115 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.115 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.115 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.115 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.116 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:11:56.116 DEBUG:boto:Data: 2024-04-16T18:11:56.116 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.116 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.116 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.116 DEBUG:boto:Params: {} 2024-04-16T18:11:56.116 DEBUG:boto:Token: None 2024-04-16T18:11:56.116 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c?versionId=null 2024-04-16T18:11:56.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YNPUrhGG/nZ6eBI0BwvmTARVA4o= 2024-04-16T18:11:56.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YNPUrhGG/nZ6eBI0BwvmTARVA4o=', 'Content-Length': '0'} 2024-04-16T18:11:56.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d204d4b72e4289cf-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.119 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.120 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.120 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.120 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.120 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.120 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:11:56.120 DEBUG:boto:Data: 2024-04-16T18:11:56.120 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.120 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.120 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.120 DEBUG:boto:Params: {} 2024-04-16T18:11:56.120 DEBUG:boto:Token: None 2024-04-16T18:11:56.121 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c?versionId=null 2024-04-16T18:11:56.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YNPUrhGG/nZ6eBI0BwvmTARVA4o= 2024-04-16T18:11:56.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YNPUrhGG/nZ6eBI0BwvmTARVA4o=', 'Content-Length': '0'} 2024-04-16T18:11:56.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061baa649c30d5a3b-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.124 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:56.124 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.124 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.124 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.124 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.125 DEBUG:boto:Method: GET 2024-04-16T18:11:56.125 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:11:56.125 DEBUG:boto:Data: 2024-04-16T18:11:56.125 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.125 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.125 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.125 DEBUG:boto:Params: {} 2024-04-16T18:11:56.125 DEBUG:boto:Token: None 2024-04-16T18:11:56.125 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c?versionId=null 2024-04-16T18:11:56.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY= 2024-04-16T18:11:56.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY=', 'Content-Length': '0'} 2024-04-16T18:11:56.128 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000067b815392e164ef-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.170 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.171 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.171 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.171 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:11:56.171 DEBUG:boto:Method: GET 2024-04-16T18:11:56.171 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:11:56.171 DEBUG:boto:Data: 2024-04-16T18:11:56.171 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.171 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.171 DEBUG:boto:Params: {} 2024-04-16T18:11:56.171 DEBUG:boto:Token: None 2024-04-16T18:11:56.171 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c?versionId=null 2024-04-16T18:11:56.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY= 2024-04-16T18:11:56.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wG2HupCWLrbjQdA4gDgOkdjqVxY=', 'Content-Length': '0'} 2024-04-16T18:11:56.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081d0d9c6e5d1d793-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.175 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.175 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.175 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.175 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.176 DEBUG:boto:Method: GET 2024-04-16T18:11:56.176 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:11:56.176 DEBUG:boto:Data: 2024-04-16T18:11:56.176 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.176 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.176 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.176 DEBUG:boto:Params: {} 2024-04-16T18:11:56.176 DEBUG:boto:Token: None 2024-04-16T18:11:56.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000661193a67fa49291-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.222 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.223 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.223 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.223 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.223 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.223 DEBUG:boto:Method: GET 2024-04-16T18:11:56.223 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:11:56.223 DEBUG:boto:Data: 2024-04-16T18:11:56.223 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.223 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.223 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.223 DEBUG:boto:Params: {} 2024-04-16T18:11:56.224 DEBUG:boto:Token: None 2024-04-16T18:11:56.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.228 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4b3afdccef2057d-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.228 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.229 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.229 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.229 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.229 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:11:56.229 DEBUG:boto:Data: 2024-04-16T18:11:56.229 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.229 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.229 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.229 DEBUG:boto:Params: {} 2024-04-16T18:11:56.229 DEBUG:boto:Token: None 2024-04-16T18:11:56.230 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c 2024-04-16T18:11:56.230 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sqm3qXyqu0v3kEywJp3hNN6QhxE= 2024-04-16T18:11:56.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sqm3qXyqu0v3kEywJp3hNN6QhxE=', 'Content-Length': '0'} 2024-04-16T18:11:56.232 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003eb8749384f9f0c4-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.233 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.233 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.233 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.233 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:11:56.233 DEBUG:boto:Data: 2024-04-16T18:11:56.233 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.233 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.233 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.233 DEBUG:boto:Params: {} 2024-04-16T18:11:56.234 DEBUG:boto:Token: None 2024-04-16T18:11:56.234 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c 2024-04-16T18:11:56.234 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sqm3qXyqu0v3kEywJp3hNN6QhxE= 2024-04-16T18:11:56.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sqm3qXyqu0v3kEywJp3hNN6QhxE=', 'Content-Length': '0'} 2024-04-16T18:11:56.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efe0c8bb5db5c0e3-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.237 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:11:56.237 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.237 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.237 DEBUG:boto:Method: GET 2024-04-16T18:11:56.237 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:11:56.237 DEBUG:boto:Data: 2024-04-16T18:11:56.237 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.237 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.237 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.238 DEBUG:boto:Params: {} 2024-04-16T18:11:56.238 DEBUG:boto:Token: None 2024-04-16T18:11:56.238 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c 2024-04-16T18:11:56.238 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AJSrM/UuQl9yEasqw8oXXGVD/p4= 2024-04-16T18:11:56.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AJSrM/UuQl9yEasqw8oXXGVD/p4=', 'Content-Length': '0'} 2024-04-16T18:11:56.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba8519710680fa2c-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.282 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:11:56.283 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:11:56.283 DEBUG:boto:Method: GET 2024-04-16T18:11:56.283 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:11:56.283 DEBUG:boto:Data: 2024-04-16T18:11:56.283 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.283 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.283 DEBUG:boto:Params: {} 2024-04-16T18:11:56.283 DEBUG:boto:Token: None 2024-04-16T18:11:56.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/c 2024-04-16T18:11:56.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AJSrM/UuQl9yEasqw8oXXGVD/p4= 2024-04-16T18:11:56.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AJSrM/UuQl9yEasqw8oXXGVD/p4=', 'Content-Length': '0'} 2024-04-16T18:11:56.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0168ba2affbea67-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.287 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.287 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.287 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.287 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.287 DEBUG:boto:Method: GET 2024-04-16T18:11:56.287 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:11:56.287 DEBUG:boto:Data: 2024-04-16T18:11:56.287 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.287 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.287 DEBUG:boto:Params: {} 2024-04-16T18:11:56.288 DEBUG:boto:Token: None 2024-04-16T18:11:56.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000508491ed1cffceb1-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.292 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.293 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.293 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.293 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.293 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:11:56.293 DEBUG:boto:Method: GET 2024-04-16T18:11:56.293 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:11:56.293 DEBUG:boto:Data: 2024-04-16T18:11:56.293 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.293 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.293 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.293 DEBUG:boto:Params: {} 2024-04-16T18:11:56.294 DEBUG:boto:Token: None 2024-04-16T18:11:56.294 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf640429810d4200-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.298 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.299 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:56.299 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.299 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.299 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.299 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.299 DEBUG:boto:Method: GET 2024-04-16T18:11:56.299 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:11:56.299 DEBUG:boto:Data: 2024-04-16T18:11:56.299 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.299 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.300 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.300 DEBUG:boto:Params: {} 2024-04-16T18:11:56.300 DEBUG:boto:Token: None 2024-04-16T18:11:56.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d?versionId=null 2024-04-16T18:11:56.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0= 2024-04-16T18:11:56.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0=', 'Content-Length': '0'} 2024-04-16T18:11:56.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd28aad69d30886e-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.346 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.347 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.347 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.347 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.347 DEBUG:boto:Method: GET 2024-04-16T18:11:56.347 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:11:56.347 DEBUG:boto:Data: 2024-04-16T18:11:56.347 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.347 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.347 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.347 DEBUG:boto:Params: {} 2024-04-16T18:11:56.347 DEBUG:boto:Token: None 2024-04-16T18:11:56.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d?versionId=null 2024-04-16T18:11:56.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0= 2024-04-16T18:11:56.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0=', 'Content-Length': '0'} 2024-04-16T18:11:56.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7cdc50d8d543bd6-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.351 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.351 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.351 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.351 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.352 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.352 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:11:56.352 DEBUG:boto:Data: 2024-04-16T18:11:56.352 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.352 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.352 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.352 DEBUG:boto:Params: {} 2024-04-16T18:11:56.352 DEBUG:boto:Token: None 2024-04-16T18:11:56.352 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d?versionId=null 2024-04-16T18:11:56.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j0e//ju1YMRhaHBATMl8XVwdPmw= 2024-04-16T18:11:56.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j0e//ju1YMRhaHBATMl8XVwdPmw=', 'Content-Length': '0'} 2024-04-16T18:11:56.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034e6e846f39c3315-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.356 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.356 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.356 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.356 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.356 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.356 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:11:56.356 DEBUG:boto:Data: 2024-04-16T18:11:56.356 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.356 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.357 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.357 DEBUG:boto:Params: {} 2024-04-16T18:11:56.357 DEBUG:boto:Token: None 2024-04-16T18:11:56.357 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d?versionId=null 2024-04-16T18:11:56.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j0e//ju1YMRhaHBATMl8XVwdPmw= 2024-04-16T18:11:56.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j0e//ju1YMRhaHBATMl8XVwdPmw=', 'Content-Length': '0'} 2024-04-16T18:11:56.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096114b26ba7713bf-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.360 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:56.360 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.360 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.360 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.360 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.361 DEBUG:boto:Method: GET 2024-04-16T18:11:56.361 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:11:56.361 DEBUG:boto:Data: 2024-04-16T18:11:56.361 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.361 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.361 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.361 DEBUG:boto:Params: {} 2024-04-16T18:11:56.361 DEBUG:boto:Token: None 2024-04-16T18:11:56.361 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d?versionId=null 2024-04-16T18:11:56.361 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0= 2024-04-16T18:11:56.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0=', 'Content-Length': '0'} 2024-04-16T18:11:56.364 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006aa601b346dccc60-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.406 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.406 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.407 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.407 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:11:56.407 DEBUG:boto:Method: GET 2024-04-16T18:11:56.407 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:11:56.407 DEBUG:boto:Data: 2024-04-16T18:11:56.407 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.407 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.407 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.407 DEBUG:boto:Params: {} 2024-04-16T18:11:56.407 DEBUG:boto:Token: None 2024-04-16T18:11:56.407 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d?versionId=null 2024-04-16T18:11:56.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0= 2024-04-16T18:11:56.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O16TtbVRnufNMsKd9HaG0DO6XP0=', 'Content-Length': '0'} 2024-04-16T18:11:56.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064b9a09c1ad9f0a7-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.411 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.411 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.411 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.411 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.411 DEBUG:boto:Method: GET 2024-04-16T18:11:56.412 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:11:56.412 DEBUG:boto:Data: 2024-04-16T18:11:56.412 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.412 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.412 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.412 DEBUG:boto:Params: {} 2024-04-16T18:11:56.412 DEBUG:boto:Token: None 2024-04-16T18:11:56.412 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.417 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026e97abc394c598d-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.458 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.459 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.459 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.459 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.459 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.459 DEBUG:boto:Method: GET 2024-04-16T18:11:56.459 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:11:56.459 DEBUG:boto:Data: 2024-04-16T18:11:56.460 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.460 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.460 DEBUG:boto:Params: {} 2024-04-16T18:11:56.460 DEBUG:boto:Token: None 2024-04-16T18:11:56.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf307c8733c07908-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.465 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.465 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.465 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.465 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.465 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:11:56.465 DEBUG:boto:Data: 2024-04-16T18:11:56.466 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.466 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.466 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.466 DEBUG:boto:Params: {} 2024-04-16T18:11:56.466 DEBUG:boto:Token: None 2024-04-16T18:11:56.466 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d 2024-04-16T18:11:56.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uzbIDqG/xz3UlVr7g4OJbRIPBos= 2024-04-16T18:11:56.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uzbIDqG/xz3UlVr7g4OJbRIPBos=', 'Content-Length': '0'} 2024-04-16T18:11:56.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003039cc330433d481-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.470 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.470 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.470 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.470 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:11:56.470 DEBUG:boto:Data: 2024-04-16T18:11:56.470 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.470 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.470 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.470 DEBUG:boto:Params: {} 2024-04-16T18:11:56.471 DEBUG:boto:Token: None 2024-04-16T18:11:56.471 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d 2024-04-16T18:11:56.471 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uzbIDqG/xz3UlVr7g4OJbRIPBos= 2024-04-16T18:11:56.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uzbIDqG/xz3UlVr7g4OJbRIPBos=', 'Content-Length': '0'} 2024-04-16T18:11:56.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039c1e0f7682a6c95-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.474 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:11:56.474 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.474 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.474 DEBUG:boto:Method: GET 2024-04-16T18:11:56.474 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:11:56.475 DEBUG:boto:Data: 2024-04-16T18:11:56.475 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.475 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.475 DEBUG:boto:Params: {} 2024-04-16T18:11:56.475 DEBUG:boto:Token: None 2024-04-16T18:11:56.475 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d 2024-04-16T18:11:56.475 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3yWVNull9ZuIUj8hqiSgFRSZmaM= 2024-04-16T18:11:56.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3yWVNull9ZuIUj8hqiSgFRSZmaM=', 'Content-Length': '0'} 2024-04-16T18:11:56.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000070c6b9f64aa037c-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.518 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:11:56.518 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:11:56.518 DEBUG:boto:Method: GET 2024-04-16T18:11:56.519 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:11:56.519 DEBUG:boto:Data: 2024-04-16T18:11:56.519 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.519 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.519 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.519 DEBUG:boto:Params: {} 2024-04-16T18:11:56.519 DEBUG:boto:Token: None 2024-04-16T18:11:56.519 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/d 2024-04-16T18:11:56.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3yWVNull9ZuIUj8hqiSgFRSZmaM= 2024-04-16T18:11:56.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3yWVNull9ZuIUj8hqiSgFRSZmaM=', 'Content-Length': '0'} 2024-04-16T18:11:56.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7aa9b9104ef8835-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.523 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.523 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.523 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.523 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.523 DEBUG:boto:Method: GET 2024-04-16T18:11:56.523 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:11:56.523 DEBUG:boto:Data: 2024-04-16T18:11:56.523 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.523 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:11:56.523 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.524 DEBUG:boto:Params: {} 2024-04-16T18:11:56.524 DEBUG:boto:Token: None 2024-04-16T18:11:56.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.529 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043dbdfce211667bf-00661ebf6c-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.529 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.529 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.529 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.529 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.529 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:11:56.529 DEBUG:boto:Method: GET 2024-04-16T18:11:56.530 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:11:56.530 DEBUG:boto:Data: 2024-04-16T18:11:56.530 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.530 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.530 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.530 DEBUG:boto:Params: {} 2024-04-16T18:11:56.530 DEBUG:boto:Token: None 2024-04-16T18:11:56.530 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.530 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY= 2024-04-16T18:11:56.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GDsAamAR4iJ1trXm382pDD9UmqY=', 'Content-Length': '0'} 2024-04-16T18:11:56.535 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c122e41aaed2d5c1-00661ebf6c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:56.535 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:11:56.535 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-47 zones={test-zone2, test-zone1} 2024-04-16T18:11:56.535 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-47', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:56.535 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'lybtug-47', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:56.536 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket lybtug-47 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:11:56.578 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:56.578 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:56.666 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:11:56.667 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-47 2024-04-16T18:11:56.673 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 4 2024-04-16T18:11:56.775 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-47', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:56.775 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'reshard', '--bucket', 'lybtug-47', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:11:56.776 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket reshard --bucket lybtug-47 --num-shards 3 --yes-i-really-mean-it --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:11:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:56.903 INFO:teuthology.orchestra.run.smithi122.stdout:tenant: 2024-04-16T18:11:56.903 INFO:teuthology.orchestra.run.smithi122.stdout:bucket name: lybtug-47 2024-04-16T18:11:56.914 INFO:teuthology.orchestra.run.smithi122.stdout:total entries: 4 2024-04-16T18:11:56.996 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:56.996 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:56.996 DEBUG:boto:Method: HEAD 2024-04-16T18:11:56.996 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:56.997 DEBUG:boto:Data: 2024-04-16T18:11:56.997 DEBUG:boto:Headers: {} 2024-04-16T18:11:56.997 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:56.997 DEBUG:boto:Port: 8000 2024-04-16T18:11:56.997 DEBUG:boto:Params: {} 2024-04-16T18:11:56.997 DEBUG:boto:Token: None 2024-04-16T18:11:56.997 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:56 GMT /lybtug-47/ 2024-04-16T18:11:56.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LO7rzSyV/KGcQ1vJx6RC2vI4hWQ= 2024-04-16T18:11:56.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LO7rzSyV/KGcQ1vJx6RC2vI4hWQ=', 'Content-Length': '0'} 2024-04-16T18:11:57.001 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cedcd6c44dc006fa-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.001 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:11:57.001 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:11:57.001 DEBUG:boto:Method: PUT 2024-04-16T18:11:57.001 DEBUG:boto:Path: /lybtug-47/e 2024-04-16T18:11:57.001 DEBUG:boto:Data: 2024-04-16T18:11:57.002 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:57.002 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.002 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.002 DEBUG:boto:Params: {} 2024-04-16T18:11:57.002 DEBUG:boto:Token: None 2024-04-16T18:11:57.002 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/e 2024-04-16T18:11:57.002 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:scQjejHSE7hFgl3v3qNoYmIsXPo= 2024-04-16T18:11:57.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:scQjejHSE7hFgl3v3qNoYmIsXPo='} 2024-04-16T18:11:57.015 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000052fa5bac99492112-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.015 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:57.015 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:57.015 DEBUG:boto:Method: HEAD 2024-04-16T18:11:57.015 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:57.016 DEBUG:boto:Data: 2024-04-16T18:11:57.016 DEBUG:boto:Headers: {} 2024-04-16T18:11:57.016 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.016 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.016 DEBUG:boto:Params: {} 2024-04-16T18:11:57.016 DEBUG:boto:Token: None 2024-04-16T18:11:57.016 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/ 2024-04-16T18:11:57.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqLGNb53M1f+iY6RcAPPMy5anrE= 2024-04-16T18:11:57.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqLGNb53M1f+iY6RcAPPMy5anrE=', 'Content-Length': '0'} 2024-04-16T18:11:57.019 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a0d488f6bf886ce9-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.020 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:11:57.020 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:11:57.020 DEBUG:boto:Method: PUT 2024-04-16T18:11:57.020 DEBUG:boto:Path: /lybtug-47/f 2024-04-16T18:11:57.020 DEBUG:boto:Data: 2024-04-16T18:11:57.020 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:57.020 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.020 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.020 DEBUG:boto:Params: {} 2024-04-16T18:11:57.020 DEBUG:boto:Token: None 2024-04-16T18:11:57.021 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/f 2024-04-16T18:11:57.021 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uqjgg3D7WlaUJrGVzZMme7dbCyA= 2024-04-16T18:11:57.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uqjgg3D7WlaUJrGVzZMme7dbCyA='} 2024-04-16T18:11:57.062 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fd8eb1d750448503-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.063 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:57.063 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:57.063 DEBUG:boto:Method: HEAD 2024-04-16T18:11:57.063 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:57.063 DEBUG:boto:Data: 2024-04-16T18:11:57.063 DEBUG:boto:Headers: {} 2024-04-16T18:11:57.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.063 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.063 DEBUG:boto:Params: {} 2024-04-16T18:11:57.063 DEBUG:boto:Token: None 2024-04-16T18:11:57.064 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/ 2024-04-16T18:11:57.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqLGNb53M1f+iY6RcAPPMy5anrE= 2024-04-16T18:11:57.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqLGNb53M1f+iY6RcAPPMy5anrE=', 'Content-Length': '0'} 2024-04-16T18:11:57.066 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c85889e7b82cb168-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.067 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:11:57.067 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:11:57.067 DEBUG:boto:Method: PUT 2024-04-16T18:11:57.067 DEBUG:boto:Path: /lybtug-47/g 2024-04-16T18:11:57.067 DEBUG:boto:Data: 2024-04-16T18:11:57.067 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:57.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.067 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.067 DEBUG:boto:Params: {} 2024-04-16T18:11:57.068 DEBUG:boto:Token: None 2024-04-16T18:11:57.068 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/g 2024-04-16T18:11:57.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7lY1UbM+4/iDTo6rrUvtzNADT9Q= 2024-04-16T18:11:57.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7lY1UbM+4/iDTo6rrUvtzNADT9Q='} 2024-04-16T18:11:57.080 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bb0e44955f3b20c6-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.080 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:11:57.080 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:11:57.081 DEBUG:boto:Method: HEAD 2024-04-16T18:11:57.081 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:11:57.081 DEBUG:boto:Data: 2024-04-16T18:11:57.081 DEBUG:boto:Headers: {} 2024-04-16T18:11:57.081 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.081 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.081 DEBUG:boto:Params: {} 2024-04-16T18:11:57.081 DEBUG:boto:Token: None 2024-04-16T18:11:57.081 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/ 2024-04-16T18:11:57.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqLGNb53M1f+iY6RcAPPMy5anrE= 2024-04-16T18:11:57.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqLGNb53M1f+iY6RcAPPMy5anrE=', 'Content-Length': '0'} 2024-04-16T18:11:57.084 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ad754e6d2b0d319e-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.084 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:11:57.084 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:11:57.084 DEBUG:boto:Method: PUT 2024-04-16T18:11:57.085 DEBUG:boto:Path: /lybtug-47/h 2024-04-16T18:11:57.085 DEBUG:boto:Data: 2024-04-16T18:11:57.085 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:11:57.085 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:11:57.085 DEBUG:boto:Port: 8000 2024-04-16T18:11:57.085 DEBUG:boto:Params: {} 2024-04-16T18:11:57.085 DEBUG:boto:Token: None 2024-04-16T18:11:57.085 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:11:57 GMT /lybtug-47/h 2024-04-16T18:11:57.085 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uLYvT8BH2i1Ntl91R9kIRttVUIc= 2024-04-16T18:11:57.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:11:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uLYvT8BH2i1Ntl91R9kIRttVUIc='} 2024-04-16T18:11:57.130 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b150fc7132998ebb-00661ebf6d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:11:57.131 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-47 2024-04-16T18:11:57.131 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:57.131 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:11:57.131 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-47 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:11:57.182 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:11:57.182 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:11:57.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:11:57.264+0000 7fed9e0e6980 1 waiting to reach latest gen 1, on 0.. 2024-04-16T18:12:02.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:02.268+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:02.264 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , ] 2024-04-16T18:12:02.264 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:07.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:07.268+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:07.265 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , ] 2024-04-16T18:12:07.265 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:12.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:12.272+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:12.267 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, , ] 2024-04-16T18:12:12.267 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:17.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:17.272+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:17.268 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, ] 2024-04-16T18:12:17.268 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:22.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:22.272+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:22.270 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, ] 2024-04-16T18:12:22.270 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:27.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:27.275+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:27.271 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, ] 2024-04-16T18:12:27.272 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:32.275+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:32.273 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, ] 2024-04-16T18:12:32.273 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:37.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:37.279+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:37.275 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, ] 2024-04-16T18:12:37.275 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:42.279+0000 7fed9e0e6980 1 waiting for incremental sync to catch up: 2024-04-16T18:12:42.277 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, ] 2024-04-16T18:12:42.277 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:47.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:47.279+0000 7fed9e0e6980 1 bucket sync caught up with source: 2024-04-16T18:12:47.278 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:47.279 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.42.6, 00000000002.59.6, 00000000002.42.6] 2024-04-16T18:12:47.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:47.283+0000 7fed9e0e6980 0 bucket checkpoint complete 2024-04-16T18:12:47.326 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-47 2024-04-16T18:12:47.326 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:12:47.326 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-47', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:12:47.327 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-47 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:12:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:12:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:12:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:12:47.461+0000 7f650da7b980 1 bucket sync caught up with source: 2024-04-16T18:12:47.460 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000003.619.6, 00000000002.564.6, 00000000002.538.6] 2024-04-16T18:12:47.460 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000003.619.6, 00000000002.564.6, 00000000002.538.6] 2024-04-16T18:12:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:12:47.461+0000 7f650da7b980 0 bucket checkpoint complete 2024-04-16T18:12:47.491 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-47 zones={test-zone2, test-zone1} 2024-04-16T18:12:47.492 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.492 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.492 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.492 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:12:47.492 DEBUG:boto:Data: 2024-04-16T18:12:47.492 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.492 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.492 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.492 DEBUG:boto:Params: {} 2024-04-16T18:12:47.493 DEBUG:boto:Token: None 2024-04-16T18:12:47.493 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2Rv+hP+0YB99o9/MoHWszrt4Y0= 2024-04-16T18:12:47.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2Rv+hP+0YB99o9/MoHWszrt4Y0=', 'Content-Length': '0'} 2024-04-16T18:12:47.496 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000907a8781f6c51151-00661ebf9f-4984-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.496 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.496 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.496 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.497 DEBUG:boto:Path: /lybtug-47/ 2024-04-16T18:12:47.497 DEBUG:boto:Data: 2024-04-16T18:12:47.497 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.497 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.497 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.497 DEBUG:boto:Params: {} 2024-04-16T18:12:47.497 DEBUG:boto:Token: None 2024-04-16T18:12:47.497 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2Rv+hP+0YB99o9/MoHWszrt4Y0= 2024-04-16T18:12:47.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2Rv+hP+0YB99o9/MoHWszrt4Y0=', 'Content-Length': '0'} 2024-04-16T18:12:47.500 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000054fb0e3d08dd35b-00661ebf9f-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.500 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:12:47.500 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.500 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.500 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.500 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.500 DEBUG:boto:Method: GET 2024-04-16T18:12:47.500 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.501 DEBUG:boto:Data: 2024-04-16T18:12:47.501 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.501 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.501 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.501 DEBUG:boto:Params: {} 2024-04-16T18:12:47.501 DEBUG:boto:Token: None 2024-04-16T18:12:47.501 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/?versions 2024-04-16T18:12:47.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q= 2024-04-16T18:12:47.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092d05ddc104517ba-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.505 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.505 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:12:47.505 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:12:47.506 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:12:47.506 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.506 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.506 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.506 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.507 DEBUG:boto:Method: GET 2024-04-16T18:12:47.507 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.507 DEBUG:boto:Data: 2024-04-16T18:12:47.507 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.507 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.507 DEBUG:boto:Params: {} 2024-04-16T18:12:47.507 DEBUG:boto:Token: None 2024-04-16T18:12:47.507 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/?versions 2024-04-16T18:12:47.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q= 2024-04-16T18:12:47.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.510 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c407df3db18659d7-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.510 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:12:47.511 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:12:47.512 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.512 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.512 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.512 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.512 DEBUG:boto:Method: GET 2024-04-16T18:12:47.512 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.512 DEBUG:boto:Data: 2024-04-16T18:12:47.512 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.512 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.512 DEBUG:boto:Params: {} 2024-04-16T18:12:47.513 DEBUG:boto:Token: None 2024-04-16T18:12:47.513 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/?versions 2024-04-16T18:12:47.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q= 2024-04-16T18:12:47.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.516 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fa4ccae7a62e53c-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.516 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.517 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.517 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.517 DEBUG:boto:path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.517 DEBUG:boto:auth_path=/lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.517 DEBUG:boto:Method: GET 2024-04-16T18:12:47.517 DEBUG:boto:Path: /lybtug-47/?versions&max-keys=999 2024-04-16T18:12:47.518 DEBUG:boto:Data: 2024-04-16T18:12:47.518 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.518 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.518 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.518 DEBUG:boto:Params: {} 2024-04-16T18:12:47.518 DEBUG:boto:Token: None 2024-04-16T18:12:47.518 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/?versions 2024-04-16T18:12:47.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q= 2024-04-16T18:12:47.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7VbWT3YwF5m9zqYY2nFjHP4728Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.521 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052f45938844b0f84-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.521 DEBUG:boto:b'lybtug-47999falseanulltrue2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:12:47.522 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.522 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.522 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.522 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.522 DEBUG:boto:Method: GET 2024-04-16T18:12:47.522 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:12:47.522 DEBUG:boto:Data: 2024-04-16T18:12:47.522 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.522 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.522 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.523 DEBUG:boto:Params: {} 2024-04-16T18:12:47.523 DEBUG:boto:Token: None 2024-04-16T18:12:47.523 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a?versionId=null 2024-04-16T18:12:47.523 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q= 2024-04-16T18:12:47.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3bc279b3cabd198-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.566 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.567 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.567 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.567 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.567 DEBUG:boto:Method: GET 2024-04-16T18:12:47.567 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:12:47.567 DEBUG:boto:Data: 2024-04-16T18:12:47.567 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.567 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.567 DEBUG:boto:Params: {} 2024-04-16T18:12:47.568 DEBUG:boto:Token: None 2024-04-16T18:12:47.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a?versionId=null 2024-04-16T18:12:47.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q= 2024-04-16T18:12:47.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ff6fc788099038b-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.571 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.571 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.572 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.572 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.572 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.572 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:12:47.572 DEBUG:boto:Data: 2024-04-16T18:12:47.572 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.572 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.572 DEBUG:boto:Params: {} 2024-04-16T18:12:47.572 DEBUG:boto:Token: None 2024-04-16T18:12:47.572 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a?versionId=null 2024-04-16T18:12:47.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1mWXnCIUEPoUeW1x56PiMNDN1Q= 2024-04-16T18:12:47.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1mWXnCIUEPoUeW1x56PiMNDN1Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065d8e2906ca31ffb-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.576 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.576 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.576 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.576 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.576 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.576 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:12:47.576 DEBUG:boto:Data: 2024-04-16T18:12:47.576 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.576 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.577 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.577 DEBUG:boto:Params: {} 2024-04-16T18:12:47.577 DEBUG:boto:Token: None 2024-04-16T18:12:47.577 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a?versionId=null 2024-04-16T18:12:47.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1mWXnCIUEPoUeW1x56PiMNDN1Q= 2024-04-16T18:12:47.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1mWXnCIUEPoUeW1x56PiMNDN1Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000314a38bc3a7d5032-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.580 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:12:47.580 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.580 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.581 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.581 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.581 DEBUG:boto:Method: GET 2024-04-16T18:12:47.581 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:12:47.581 DEBUG:boto:Data: 2024-04-16T18:12:47.581 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.581 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.581 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.581 DEBUG:boto:Params: {} 2024-04-16T18:12:47.581 DEBUG:boto:Token: None 2024-04-16T18:12:47.581 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a?versionId=null 2024-04-16T18:12:47.582 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q= 2024-04-16T18:12:47.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a67512d7067e2f14-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.585 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.585 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.585 DEBUG:boto:path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.585 DEBUG:boto:auth_path=/lybtug-47/a?versionId=null 2024-04-16T18:12:47.585 DEBUG:boto:Method: GET 2024-04-16T18:12:47.585 DEBUG:boto:Path: /lybtug-47/a?versionId=null 2024-04-16T18:12:47.585 DEBUG:boto:Data: 2024-04-16T18:12:47.586 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.586 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.586 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.586 DEBUG:boto:Params: {} 2024-04-16T18:12:47.586 DEBUG:boto:Token: None 2024-04-16T18:12:47.586 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a?versionId=null 2024-04-16T18:12:47.586 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q= 2024-04-16T18:12:47.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wZsNZ7WPOdsad6eBbFqG+JYTe9Q=', 'Content-Length': '0'} 2024-04-16T18:12:47.590 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029114edd3a5c904d-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.630 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.630 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.630 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.631 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.631 DEBUG:boto:Method: GET 2024-04-16T18:12:47.631 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:12:47.631 DEBUG:boto:Data: 2024-04-16T18:12:47.631 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.631 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.631 DEBUG:boto:Params: {} 2024-04-16T18:12:47.631 DEBUG:boto:Token: None 2024-04-16T18:12:47.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.634 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e57409e8b4a09433-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.635 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.635 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.635 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.635 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.635 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.635 DEBUG:boto:Method: GET 2024-04-16T18:12:47.635 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:12:47.635 DEBUG:boto:Data: 2024-04-16T18:12:47.636 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.636 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.636 DEBUG:boto:Params: {} 2024-04-16T18:12:47.636 DEBUG:boto:Token: None 2024-04-16T18:12:47.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.639 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c030bf58802709f-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.639 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.640 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.640 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.640 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.640 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:12:47.640 DEBUG:boto:Data: 2024-04-16T18:12:47.640 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.640 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.640 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.640 DEBUG:boto:Params: {} 2024-04-16T18:12:47.640 DEBUG:boto:Token: None 2024-04-16T18:12:47.640 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a 2024-04-16T18:12:47.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s70cSyPZKWkkVrCr473D019F4/w= 2024-04-16T18:12:47.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s70cSyPZKWkkVrCr473D019F4/w=', 'Content-Length': '0'} 2024-04-16T18:12:47.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060d5c9e5e41e7868-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.644 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.644 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.644 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.644 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:12:47.644 DEBUG:boto:Data: 2024-04-16T18:12:47.644 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.644 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.645 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.645 DEBUG:boto:Params: {} 2024-04-16T18:12:47.645 DEBUG:boto:Token: None 2024-04-16T18:12:47.645 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a 2024-04-16T18:12:47.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s70cSyPZKWkkVrCr473D019F4/w= 2024-04-16T18:12:47.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s70cSyPZKWkkVrCr473D019F4/w=', 'Content-Length': '0'} 2024-04-16T18:12:47.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d828084af7d434e-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:12:47.648 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.649 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.649 DEBUG:boto:Method: GET 2024-04-16T18:12:47.649 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:12:47.649 DEBUG:boto:Data: 2024-04-16T18:12:47.649 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.649 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.649 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.649 DEBUG:boto:Params: {} 2024-04-16T18:12:47.649 DEBUG:boto:Token: None 2024-04-16T18:12:47.649 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a 2024-04-16T18:12:47.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7IgDYXOjRbQVlI67/wishBa+phA= 2024-04-16T18:12:47.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7IgDYXOjRbQVlI67/wishBa+phA=', 'Content-Length': '0'} 2024-04-16T18:12:47.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c05a5cbbd796167d-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.694 DEBUG:boto:path=/lybtug-47/a 2024-04-16T18:12:47.694 DEBUG:boto:auth_path=/lybtug-47/a 2024-04-16T18:12:47.695 DEBUG:boto:Method: GET 2024-04-16T18:12:47.695 DEBUG:boto:Path: /lybtug-47/a 2024-04-16T18:12:47.695 DEBUG:boto:Data: 2024-04-16T18:12:47.695 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.695 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.695 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.695 DEBUG:boto:Params: {} 2024-04-16T18:12:47.695 DEBUG:boto:Token: None 2024-04-16T18:12:47.695 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/a 2024-04-16T18:12:47.695 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7IgDYXOjRbQVlI67/wishBa+phA= 2024-04-16T18:12:47.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7IgDYXOjRbQVlI67/wishBa+phA=', 'Content-Length': '0'} 2024-04-16T18:12:47.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d23ad0c2dddcaa7-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.699 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.699 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.699 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.699 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.699 DEBUG:boto:Method: GET 2024-04-16T18:12:47.699 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:12:47.699 DEBUG:boto:Data: 2024-04-16T18:12:47.700 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.700 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.700 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.700 DEBUG:boto:Params: {} 2024-04-16T18:12:47.700 DEBUG:boto:Token: None 2024-04-16T18:12:47.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007841a5825aad62d1-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.704 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.704 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.704 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.704 DEBUG:boto:path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.704 DEBUG:boto:auth_path=/lybtug-47/?prefix=a 2024-04-16T18:12:47.704 DEBUG:boto:Method: GET 2024-04-16T18:12:47.704 DEBUG:boto:Path: /lybtug-47/?prefix=a 2024-04-16T18:12:47.704 DEBUG:boto:Data: 2024-04-16T18:12:47.705 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.705 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.705 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.705 DEBUG:boto:Params: {} 2024-04-16T18:12:47.705 DEBUG:boto:Token: None 2024-04-16T18:12:47.705 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.705 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.709 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7ad55efab182581-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.750 DEBUG:boto:b'lybtug-47a1000falsea2024-04-16T18:11:50.086Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.751 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:12:47.751 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.751 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.751 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.751 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.751 DEBUG:boto:Method: GET 2024-04-16T18:12:47.751 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:12:47.751 DEBUG:boto:Data: 2024-04-16T18:12:47.751 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.751 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.751 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.751 DEBUG:boto:Params: {} 2024-04-16T18:12:47.752 DEBUG:boto:Token: None 2024-04-16T18:12:47.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b?versionId=null 2024-04-16T18:12:47.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8= 2024-04-16T18:12:47.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8=', 'Content-Length': '0'} 2024-04-16T18:12:47.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000241103e1a1e82bf9-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.798 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.798 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.798 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.799 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.799 DEBUG:boto:Method: GET 2024-04-16T18:12:47.799 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:12:47.799 DEBUG:boto:Data: 2024-04-16T18:12:47.799 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.799 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.799 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.799 DEBUG:boto:Params: {} 2024-04-16T18:12:47.799 DEBUG:boto:Token: None 2024-04-16T18:12:47.799 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b?versionId=null 2024-04-16T18:12:47.799 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8= 2024-04-16T18:12:47.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8=', 'Content-Length': '0'} 2024-04-16T18:12:47.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d75ed6d1c95afc1b-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.803 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.803 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.803 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.803 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.803 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.803 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:12:47.803 DEBUG:boto:Data: 2024-04-16T18:12:47.804 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.804 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.804 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.804 DEBUG:boto:Params: {} 2024-04-16T18:12:47.804 DEBUG:boto:Token: None 2024-04-16T18:12:47.804 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b?versionId=null 2024-04-16T18:12:47.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tK80lWgVhC9BNo5uB97PK4r6J0o= 2024-04-16T18:12:47.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tK80lWgVhC9BNo5uB97PK4r6J0o=', 'Content-Length': '0'} 2024-04-16T18:12:47.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000902a089f2e923a60-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.807 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.807 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.807 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.808 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.808 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.808 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:12:47.808 DEBUG:boto:Data: 2024-04-16T18:12:47.808 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.808 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.808 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.808 DEBUG:boto:Params: {} 2024-04-16T18:12:47.808 DEBUG:boto:Token: None 2024-04-16T18:12:47.808 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b?versionId=null 2024-04-16T18:12:47.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tK80lWgVhC9BNo5uB97PK4r6J0o= 2024-04-16T18:12:47.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tK80lWgVhC9BNo5uB97PK4r6J0o=', 'Content-Length': '0'} 2024-04-16T18:12:47.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b070c862b7271624-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.812 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:12:47.812 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.812 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.812 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.812 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.812 DEBUG:boto:Method: GET 2024-04-16T18:12:47.812 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:12:47.812 DEBUG:boto:Data: 2024-04-16T18:12:47.812 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.812 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.813 DEBUG:boto:Params: {} 2024-04-16T18:12:47.813 DEBUG:boto:Token: None 2024-04-16T18:12:47.813 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b?versionId=null 2024-04-16T18:12:47.813 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8= 2024-04-16T18:12:47.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8=', 'Content-Length': '0'} 2024-04-16T18:12:47.816 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc7a8878ef44d09e-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.858 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.859 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.859 DEBUG:boto:path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.859 DEBUG:boto:auth_path=/lybtug-47/b?versionId=null 2024-04-16T18:12:47.859 DEBUG:boto:Method: GET 2024-04-16T18:12:47.859 DEBUG:boto:Path: /lybtug-47/b?versionId=null 2024-04-16T18:12:47.859 DEBUG:boto:Data: 2024-04-16T18:12:47.859 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.859 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.859 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.859 DEBUG:boto:Params: {} 2024-04-16T18:12:47.859 DEBUG:boto:Token: None 2024-04-16T18:12:47.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b?versionId=null 2024-04-16T18:12:47.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8= 2024-04-16T18:12:47.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xLcYs/IANEMv/kByGfJ34dGghe8=', 'Content-Length': '0'} 2024-04-16T18:12:47.863 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000486e8a6d80cb5d0b-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.864 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.864 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.864 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.864 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.864 DEBUG:boto:Method: GET 2024-04-16T18:12:47.864 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:12:47.864 DEBUG:boto:Data: 2024-04-16T18:12:47.864 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.864 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.865 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.865 DEBUG:boto:Params: {} 2024-04-16T18:12:47.865 DEBUG:boto:Token: None 2024-04-16T18:12:47.865 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.865 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000725dff9828f8677d-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.869 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.869 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.869 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.869 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.869 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.869 DEBUG:boto:Method: GET 2024-04-16T18:12:47.870 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:12:47.870 DEBUG:boto:Data: 2024-04-16T18:12:47.870 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.870 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.870 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.870 DEBUG:boto:Params: {} 2024-04-16T18:12:47.870 DEBUG:boto:Token: None 2024-04-16T18:12:47.870 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.870 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000143a8cf54f0eb5dd-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.874 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.874 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.874 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.874 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.874 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:12:47.874 DEBUG:boto:Data: 2024-04-16T18:12:47.874 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.875 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.875 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.875 DEBUG:boto:Params: {} 2024-04-16T18:12:47.875 DEBUG:boto:Token: None 2024-04-16T18:12:47.875 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b 2024-04-16T18:12:47.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:duut+bp1x860D+h/JIvz2EE7ow4= 2024-04-16T18:12:47.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duut+bp1x860D+h/JIvz2EE7ow4=', 'Content-Length': '0'} 2024-04-16T18:12:47.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a64a3055883070f9-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.878 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.878 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.878 DEBUG:boto:Method: HEAD 2024-04-16T18:12:47.879 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:12:47.879 DEBUG:boto:Data: 2024-04-16T18:12:47.879 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.879 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.879 DEBUG:boto:Params: {} 2024-04-16T18:12:47.879 DEBUG:boto:Token: None 2024-04-16T18:12:47.879 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b 2024-04-16T18:12:47.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:duut+bp1x860D+h/JIvz2EE7ow4= 2024-04-16T18:12:47.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duut+bp1x860D+h/JIvz2EE7ow4=', 'Content-Length': '0'} 2024-04-16T18:12:47.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0efd8fdc1622aa3-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.883 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:12:47.883 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.883 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.883 DEBUG:boto:Method: GET 2024-04-16T18:12:47.883 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:12:47.883 DEBUG:boto:Data: 2024-04-16T18:12:47.883 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.883 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.884 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.884 DEBUG:boto:Params: {} 2024-04-16T18:12:47.884 DEBUG:boto:Token: None 2024-04-16T18:12:47.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b 2024-04-16T18:12:47.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uJyyARxRzo2OjmucRwIAT4FUA9A= 2024-04-16T18:12:47.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uJyyARxRzo2OjmucRwIAT4FUA9A=', 'Content-Length': '0'} 2024-04-16T18:12:47.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbdc37917e1b5909-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.930 DEBUG:boto:path=/lybtug-47/b 2024-04-16T18:12:47.931 DEBUG:boto:auth_path=/lybtug-47/b 2024-04-16T18:12:47.931 DEBUG:boto:Method: GET 2024-04-16T18:12:47.931 DEBUG:boto:Path: /lybtug-47/b 2024-04-16T18:12:47.931 DEBUG:boto:Data: 2024-04-16T18:12:47.931 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.931 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.931 DEBUG:boto:Params: {} 2024-04-16T18:12:47.931 DEBUG:boto:Token: None 2024-04-16T18:12:47.931 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/b 2024-04-16T18:12:47.931 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uJyyARxRzo2OjmucRwIAT4FUA9A= 2024-04-16T18:12:47.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uJyyARxRzo2OjmucRwIAT4FUA9A=', 'Content-Length': '0'} 2024-04-16T18:12:47.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5a0ce308a01d722-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.935 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.935 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.935 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.935 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.935 DEBUG:boto:Method: GET 2024-04-16T18:12:47.936 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:12:47.936 DEBUG:boto:Data: 2024-04-16T18:12:47.936 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.936 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.936 DEBUG:boto:Params: {} 2024-04-16T18:12:47.936 DEBUG:boto:Token: None 2024-04-16T18:12:47.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc8b8472e51ef76c-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.982 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.983 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:47.983 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:47.983 DEBUG:boto:path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.983 DEBUG:boto:auth_path=/lybtug-47/?prefix=b 2024-04-16T18:12:47.983 DEBUG:boto:Method: GET 2024-04-16T18:12:47.983 DEBUG:boto:Path: /lybtug-47/?prefix=b 2024-04-16T18:12:47.983 DEBUG:boto:Data: 2024-04-16T18:12:47.983 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:47.983 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.983 DEBUG:boto:Params: {} 2024-04-16T18:12:47.984 DEBUG:boto:Token: None 2024-04-16T18:12:47.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/ 2024-04-16T18:12:47.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4= 2024-04-16T18:12:47.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QXCExk/VelsiMjKHn/v/N1VBXx4=', 'Content-Length': '0'} 2024-04-16T18:12:47.987 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000419dc9feb5fae8f7-00661ebf9f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:47.987 DEBUG:boto:b'lybtug-47b1000falseb2024-04-16T18:11:50.106Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:47.988 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:12:47.988 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:47.988 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:47.988 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:12:47.988 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:12:47.988 DEBUG:boto:Method: GET 2024-04-16T18:12:47.988 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:12:47.988 DEBUG:boto:Data: 2024-04-16T18:12:47.988 DEBUG:boto:Headers: {} 2024-04-16T18:12:47.988 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:47.988 DEBUG:boto:Port: 8000 2024-04-16T18:12:47.989 DEBUG:boto:Params: {} 2024-04-16T18:12:47.989 DEBUG:boto:Token: None 2024-04-16T18:12:47.989 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:47 GMT /lybtug-47/c?versionId=null 2024-04-16T18:12:47.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rcE16sAJyB35Y6V1GKa5EMTX5vw= 2024-04-16T18:12:47.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rcE16sAJyB35Y6V1GKa5EMTX5vw=', 'Content-Length': '0'} 2024-04-16T18:12:47.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000064bb9ce91f8a94c-00661ebf9f-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.034 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.034 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.034 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.035 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.035 DEBUG:boto:Method: GET 2024-04-16T18:12:48.035 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:12:48.035 DEBUG:boto:Data: 2024-04-16T18:12:48.035 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.035 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.035 DEBUG:boto:Params: {} 2024-04-16T18:12:48.035 DEBUG:boto:Token: None 2024-04-16T18:12:48.035 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c?versionId=null 2024-04-16T18:12:48.035 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7SxmRUrZqTy4n0qYtHjUBi88Q+c= 2024-04-16T18:12:48.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7SxmRUrZqTy4n0qYtHjUBi88Q+c=', 'Content-Length': '0'} 2024-04-16T18:12:48.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000df50aa06d491af-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.039 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.039 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.039 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.039 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.039 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.039 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:12:48.039 DEBUG:boto:Data: 2024-04-16T18:12:48.040 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.040 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.040 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.040 DEBUG:boto:Params: {} 2024-04-16T18:12:48.040 DEBUG:boto:Token: None 2024-04-16T18:12:48.040 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c?versionId=null 2024-04-16T18:12:48.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yZAzik8QTFt35qpFqw8moH/XyGQ= 2024-04-16T18:12:48.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yZAzik8QTFt35qpFqw8moH/XyGQ=', 'Content-Length': '0'} 2024-04-16T18:12:48.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c60a5fe40e14bf88-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.044 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.044 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.044 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.044 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.044 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.044 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:12:48.044 DEBUG:boto:Data: 2024-04-16T18:12:48.044 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.044 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.044 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.045 DEBUG:boto:Params: {} 2024-04-16T18:12:48.045 DEBUG:boto:Token: None 2024-04-16T18:12:48.045 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c?versionId=null 2024-04-16T18:12:48.045 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yZAzik8QTFt35qpFqw8moH/XyGQ= 2024-04-16T18:12:48.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yZAzik8QTFt35qpFqw8moH/XyGQ=', 'Content-Length': '0'} 2024-04-16T18:12:48.048 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f06ac7f2234e5e5-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.048 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:12:48.048 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.048 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.048 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.048 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.048 DEBUG:boto:Method: GET 2024-04-16T18:12:48.048 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:12:48.048 DEBUG:boto:Data: 2024-04-16T18:12:48.048 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.049 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.049 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.049 DEBUG:boto:Params: {} 2024-04-16T18:12:48.049 DEBUG:boto:Token: None 2024-04-16T18:12:48.049 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c?versionId=null 2024-04-16T18:12:48.049 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7SxmRUrZqTy4n0qYtHjUBi88Q+c= 2024-04-16T18:12:48.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7SxmRUrZqTy4n0qYtHjUBi88Q+c=', 'Content-Length': '0'} 2024-04-16T18:12:48.052 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be95818b1884a581-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.094 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.094 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.094 DEBUG:boto:path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.095 DEBUG:boto:auth_path=/lybtug-47/c?versionId=null 2024-04-16T18:12:48.095 DEBUG:boto:Method: GET 2024-04-16T18:12:48.095 DEBUG:boto:Path: /lybtug-47/c?versionId=null 2024-04-16T18:12:48.095 DEBUG:boto:Data: 2024-04-16T18:12:48.095 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.095 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.095 DEBUG:boto:Params: {} 2024-04-16T18:12:48.095 DEBUG:boto:Token: None 2024-04-16T18:12:48.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c?versionId=null 2024-04-16T18:12:48.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7SxmRUrZqTy4n0qYtHjUBi88Q+c= 2024-04-16T18:12:48.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7SxmRUrZqTy4n0qYtHjUBi88Q+c=', 'Content-Length': '0'} 2024-04-16T18:12:48.098 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3576ebff4f5fed4-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.098 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.099 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.099 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.099 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.099 DEBUG:boto:Method: GET 2024-04-16T18:12:48.099 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:12:48.099 DEBUG:boto:Data: 2024-04-16T18:12:48.099 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.099 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.099 DEBUG:boto:Params: {} 2024-04-16T18:12:48.099 DEBUG:boto:Token: None 2024-04-16T18:12:48.100 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.103 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037f069c931a16d6c-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.103 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.103 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.103 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.103 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.103 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.104 DEBUG:boto:Method: GET 2024-04-16T18:12:48.104 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:12:48.104 DEBUG:boto:Data: 2024-04-16T18:12:48.104 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.104 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.104 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.104 DEBUG:boto:Params: {} 2024-04-16T18:12:48.104 DEBUG:boto:Token: None 2024-04-16T18:12:48.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fffa213dfd542e9f-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.108 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.108 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.108 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.108 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.108 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:12:48.108 DEBUG:boto:Data: 2024-04-16T18:12:48.108 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.109 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.109 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.109 DEBUG:boto:Params: {} 2024-04-16T18:12:48.109 DEBUG:boto:Token: None 2024-04-16T18:12:48.109 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c 2024-04-16T18:12:48.109 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o1qzp2/AA7FReoulj7mBtKCUajY= 2024-04-16T18:12:48.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o1qzp2/AA7FReoulj7mBtKCUajY=', 'Content-Length': '0'} 2024-04-16T18:12:48.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036ca1c7bc3006b57-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.112 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.112 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.112 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.112 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:12:48.113 DEBUG:boto:Data: 2024-04-16T18:12:48.113 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.113 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.113 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.113 DEBUG:boto:Params: {} 2024-04-16T18:12:48.113 DEBUG:boto:Token: None 2024-04-16T18:12:48.113 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c 2024-04-16T18:12:48.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o1qzp2/AA7FReoulj7mBtKCUajY= 2024-04-16T18:12:48.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o1qzp2/AA7FReoulj7mBtKCUajY=', 'Content-Length': '0'} 2024-04-16T18:12:48.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070b7e2aa2e263f00-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.116 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:12:48.116 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.116 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.116 DEBUG:boto:Method: GET 2024-04-16T18:12:48.116 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:12:48.117 DEBUG:boto:Data: 2024-04-16T18:12:48.117 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.117 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.117 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.117 DEBUG:boto:Params: {} 2024-04-16T18:12:48.117 DEBUG:boto:Token: None 2024-04-16T18:12:48.117 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c 2024-04-16T18:12:48.117 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2jXgV+lf+zr9/Q4+JRV+3VR6y48= 2024-04-16T18:12:48.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2jXgV+lf+zr9/Q4+JRV+3VR6y48=', 'Content-Length': '0'} 2024-04-16T18:12:48.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000affd60fb87e17ce8-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.162 DEBUG:boto:path=/lybtug-47/c 2024-04-16T18:12:48.162 DEBUG:boto:auth_path=/lybtug-47/c 2024-04-16T18:12:48.163 DEBUG:boto:Method: GET 2024-04-16T18:12:48.163 DEBUG:boto:Path: /lybtug-47/c 2024-04-16T18:12:48.163 DEBUG:boto:Data: 2024-04-16T18:12:48.163 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.163 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.163 DEBUG:boto:Params: {} 2024-04-16T18:12:48.163 DEBUG:boto:Token: None 2024-04-16T18:12:48.163 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/c 2024-04-16T18:12:48.163 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2jXgV+lf+zr9/Q4+JRV+3VR6y48= 2024-04-16T18:12:48.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2jXgV+lf+zr9/Q4+JRV+3VR6y48=', 'Content-Length': '0'} 2024-04-16T18:12:48.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a7dd4000bdba4d2-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.167 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.167 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.167 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.167 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.167 DEBUG:boto:Method: GET 2024-04-16T18:12:48.167 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:12:48.167 DEBUG:boto:Data: 2024-04-16T18:12:48.167 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.168 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.168 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.168 DEBUG:boto:Params: {} 2024-04-16T18:12:48.168 DEBUG:boto:Token: None 2024-04-16T18:12:48.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.171 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d151144e7ed77162-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.171 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.172 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.172 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.172 DEBUG:boto:path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.172 DEBUG:boto:auth_path=/lybtug-47/?prefix=c 2024-04-16T18:12:48.172 DEBUG:boto:Method: GET 2024-04-16T18:12:48.172 DEBUG:boto:Path: /lybtug-47/?prefix=c 2024-04-16T18:12:48.172 DEBUG:boto:Data: 2024-04-16T18:12:48.172 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.172 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.173 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.173 DEBUG:boto:Params: {} 2024-04-16T18:12:48.173 DEBUG:boto:Token: None 2024-04-16T18:12:48.173 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.173 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac124b4f19d5ef86-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.218 DEBUG:boto:b'lybtug-47c1000falsec2024-04-16T18:11:50.155Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.218 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:12:48.219 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.219 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.219 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.219 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.219 DEBUG:boto:Method: GET 2024-04-16T18:12:48.219 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:12:48.219 DEBUG:boto:Data: 2024-04-16T18:12:48.219 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.219 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.219 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.219 DEBUG:boto:Params: {} 2024-04-16T18:12:48.220 DEBUG:boto:Token: None 2024-04-16T18:12:48.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d?versionId=null 2024-04-16T18:12:48.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA= 2024-04-16T18:12:48.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA=', 'Content-Length': '0'} 2024-04-16T18:12:48.222 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090b858492a794016-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.266 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.266 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.267 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.267 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.267 DEBUG:boto:Method: GET 2024-04-16T18:12:48.267 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:12:48.267 DEBUG:boto:Data: 2024-04-16T18:12:48.267 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.267 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.267 DEBUG:boto:Params: {} 2024-04-16T18:12:48.267 DEBUG:boto:Token: None 2024-04-16T18:12:48.267 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d?versionId=null 2024-04-16T18:12:48.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA= 2024-04-16T18:12:48.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA=', 'Content-Length': '0'} 2024-04-16T18:12:48.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005db840b2605f71d3-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.271 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.271 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.271 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.271 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.271 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.271 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:12:48.271 DEBUG:boto:Data: 2024-04-16T18:12:48.271 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.272 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.272 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.272 DEBUG:boto:Params: {} 2024-04-16T18:12:48.272 DEBUG:boto:Token: None 2024-04-16T18:12:48.272 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d?versionId=null 2024-04-16T18:12:48.272 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hq1kenqHF1y9MGnSfST/uWGspk0= 2024-04-16T18:12:48.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hq1kenqHF1y9MGnSfST/uWGspk0=', 'Content-Length': '0'} 2024-04-16T18:12:48.275 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3d9fac108570b0d-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.275 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.275 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.275 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.276 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.276 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.276 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:12:48.276 DEBUG:boto:Data: 2024-04-16T18:12:48.276 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.276 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.276 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.276 DEBUG:boto:Params: {} 2024-04-16T18:12:48.276 DEBUG:boto:Token: None 2024-04-16T18:12:48.276 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d?versionId=null 2024-04-16T18:12:48.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hq1kenqHF1y9MGnSfST/uWGspk0= 2024-04-16T18:12:48.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hq1kenqHF1y9MGnSfST/uWGspk0=', 'Content-Length': '0'} 2024-04-16T18:12:48.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095a1a84479d81653-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.279 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:12:48.279 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.280 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.280 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.280 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.280 DEBUG:boto:Method: GET 2024-04-16T18:12:48.280 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:12:48.280 DEBUG:boto:Data: 2024-04-16T18:12:48.280 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.280 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.280 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.280 DEBUG:boto:Params: {} 2024-04-16T18:12:48.280 DEBUG:boto:Token: None 2024-04-16T18:12:48.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d?versionId=null 2024-04-16T18:12:48.281 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA= 2024-04-16T18:12:48.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA=', 'Content-Length': '0'} 2024-04-16T18:12:48.284 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff0bcb4f42764b3a-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.326 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.326 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.326 DEBUG:boto:path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.327 DEBUG:boto:auth_path=/lybtug-47/d?versionId=null 2024-04-16T18:12:48.327 DEBUG:boto:Method: GET 2024-04-16T18:12:48.327 DEBUG:boto:Path: /lybtug-47/d?versionId=null 2024-04-16T18:12:48.327 DEBUG:boto:Data: 2024-04-16T18:12:48.327 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.327 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.327 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.327 DEBUG:boto:Params: {} 2024-04-16T18:12:48.327 DEBUG:boto:Token: None 2024-04-16T18:12:48.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d?versionId=null 2024-04-16T18:12:48.327 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA= 2024-04-16T18:12:48.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEqDTOdwHHQHwtXFnSUDnIUJBqA=', 'Content-Length': '0'} 2024-04-16T18:12:48.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4d46d6579655683-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.330 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.331 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.331 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.331 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.331 DEBUG:boto:Method: GET 2024-04-16T18:12:48.331 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:12:48.331 DEBUG:boto:Data: 2024-04-16T18:12:48.331 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.331 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.331 DEBUG:boto:Params: {} 2024-04-16T18:12:48.331 DEBUG:boto:Token: None 2024-04-16T18:12:48.331 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b94f95f34ae5f3c-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.335 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.335 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.335 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.335 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.335 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.336 DEBUG:boto:Method: GET 2024-04-16T18:12:48.336 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:12:48.336 DEBUG:boto:Data: 2024-04-16T18:12:48.336 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.336 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.336 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.336 DEBUG:boto:Params: {} 2024-04-16T18:12:48.336 DEBUG:boto:Token: None 2024-04-16T18:12:48.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.339 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4ebf75a8bcc9d05-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.339 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.340 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.340 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.340 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.340 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:12:48.340 DEBUG:boto:Data: 2024-04-16T18:12:48.340 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.340 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.340 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.340 DEBUG:boto:Params: {} 2024-04-16T18:12:48.340 DEBUG:boto:Token: None 2024-04-16T18:12:48.341 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d 2024-04-16T18:12:48.341 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfY5nHvN829aeJSJnuOWn7vbJ98= 2024-04-16T18:12:48.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfY5nHvN829aeJSJnuOWn7vbJ98=', 'Content-Length': '0'} 2024-04-16T18:12:48.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a996c00729d9d22a-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.344 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.344 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.344 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.344 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:12:48.344 DEBUG:boto:Data: 2024-04-16T18:12:48.344 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.344 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.344 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.344 DEBUG:boto:Params: {} 2024-04-16T18:12:48.345 DEBUG:boto:Token: None 2024-04-16T18:12:48.345 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d 2024-04-16T18:12:48.345 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfY5nHvN829aeJSJnuOWn7vbJ98= 2024-04-16T18:12:48.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfY5nHvN829aeJSJnuOWn7vbJ98=', 'Content-Length': '0'} 2024-04-16T18:12:48.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0c1247e2d846d09-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:12:48.348 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.348 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.348 DEBUG:boto:Method: GET 2024-04-16T18:12:48.348 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:12:48.348 DEBUG:boto:Data: 2024-04-16T18:12:48.348 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.348 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.349 DEBUG:boto:Params: {} 2024-04-16T18:12:48.349 DEBUG:boto:Token: None 2024-04-16T18:12:48.349 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d 2024-04-16T18:12:48.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:72zvWAJhlNHSKOACPMf51eQhRAk= 2024-04-16T18:12:48.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:72zvWAJhlNHSKOACPMf51eQhRAk=', 'Content-Length': '0'} 2024-04-16T18:12:48.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c68a289816b8e013-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.394 DEBUG:boto:path=/lybtug-47/d 2024-04-16T18:12:48.395 DEBUG:boto:auth_path=/lybtug-47/d 2024-04-16T18:12:48.395 DEBUG:boto:Method: GET 2024-04-16T18:12:48.395 DEBUG:boto:Path: /lybtug-47/d 2024-04-16T18:12:48.395 DEBUG:boto:Data: 2024-04-16T18:12:48.395 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.395 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.395 DEBUG:boto:Params: {} 2024-04-16T18:12:48.395 DEBUG:boto:Token: None 2024-04-16T18:12:48.395 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/d 2024-04-16T18:12:48.395 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:72zvWAJhlNHSKOACPMf51eQhRAk= 2024-04-16T18:12:48.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:72zvWAJhlNHSKOACPMf51eQhRAk=', 'Content-Length': '0'} 2024-04-16T18:12:48.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:50 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c628d2842b9fcfa-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.399 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.399 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.399 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.399 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.399 DEBUG:boto:Method: GET 2024-04-16T18:12:48.399 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:12:48.399 DEBUG:boto:Data: 2024-04-16T18:12:48.399 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.399 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.399 DEBUG:boto:Params: {} 2024-04-16T18:12:48.400 DEBUG:boto:Token: None 2024-04-16T18:12:48.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.403 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066f82a50c1651ea1-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.403 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.403 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.403 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.404 DEBUG:boto:path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.404 DEBUG:boto:auth_path=/lybtug-47/?prefix=d 2024-04-16T18:12:48.404 DEBUG:boto:Method: GET 2024-04-16T18:12:48.404 DEBUG:boto:Path: /lybtug-47/?prefix=d 2024-04-16T18:12:48.404 DEBUG:boto:Data: 2024-04-16T18:12:48.404 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.404 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.404 DEBUG:boto:Params: {} 2024-04-16T18:12:48.404 DEBUG:boto:Token: None 2024-04-16T18:12:48.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073b769ec4e3a62cd-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.408 DEBUG:boto:b'lybtug-47d1000falsed2024-04-16T18:11:50.173Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.408 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:12:48.408 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.408 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.408 DEBUG:boto:path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.408 DEBUG:boto:auth_path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.408 DEBUG:boto:Method: GET 2024-04-16T18:12:48.408 DEBUG:boto:Path: /lybtug-47/e?versionId=null 2024-04-16T18:12:48.409 DEBUG:boto:Data: 2024-04-16T18:12:48.409 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.409 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.409 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.409 DEBUG:boto:Params: {} 2024-04-16T18:12:48.409 DEBUG:boto:Token: None 2024-04-16T18:12:48.409 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e?versionId=null 2024-04-16T18:12:48.409 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew= 2024-04-16T18:12:48.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew=', 'Content-Length': '0'} 2024-04-16T18:12:48.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070b6bf2598a52d32-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.454 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.454 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.454 DEBUG:boto:path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.455 DEBUG:boto:auth_path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.455 DEBUG:boto:Method: GET 2024-04-16T18:12:48.455 DEBUG:boto:Path: /lybtug-47/e?versionId=null 2024-04-16T18:12:48.455 DEBUG:boto:Data: 2024-04-16T18:12:48.455 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.455 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.455 DEBUG:boto:Params: {} 2024-04-16T18:12:48.455 DEBUG:boto:Token: None 2024-04-16T18:12:48.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e?versionId=null 2024-04-16T18:12:48.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew= 2024-04-16T18:12:48.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew=', 'Content-Length': '0'} 2024-04-16T18:12:48.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf90cbe7f8357abf-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.459 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.459 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.459 DEBUG:boto:path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.459 DEBUG:boto:auth_path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.459 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.459 DEBUG:boto:Path: /lybtug-47/e?versionId=null 2024-04-16T18:12:48.459 DEBUG:boto:Data: 2024-04-16T18:12:48.459 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.459 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.459 DEBUG:boto:Params: {} 2024-04-16T18:12:48.459 DEBUG:boto:Token: None 2024-04-16T18:12:48.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e?versionId=null 2024-04-16T18:12:48.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t+flvJADrz5cGiAYzwotuh5USaY= 2024-04-16T18:12:48.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t+flvJADrz5cGiAYzwotuh5USaY=', 'Content-Length': '0'} 2024-04-16T18:12:48.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d213ce115c2fed88-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.463 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.463 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.463 DEBUG:boto:path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.463 DEBUG:boto:auth_path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.463 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.463 DEBUG:boto:Path: /lybtug-47/e?versionId=null 2024-04-16T18:12:48.463 DEBUG:boto:Data: 2024-04-16T18:12:48.464 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.464 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.464 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.464 DEBUG:boto:Params: {} 2024-04-16T18:12:48.464 DEBUG:boto:Token: None 2024-04-16T18:12:48.464 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e?versionId=null 2024-04-16T18:12:48.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t+flvJADrz5cGiAYzwotuh5USaY= 2024-04-16T18:12:48.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t+flvJADrz5cGiAYzwotuh5USaY=', 'Content-Length': '0'} 2024-04-16T18:12:48.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074c81a1a2734998b-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:12:48.467 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.467 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.467 DEBUG:boto:path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.467 DEBUG:boto:auth_path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.467 DEBUG:boto:Method: GET 2024-04-16T18:12:48.468 DEBUG:boto:Path: /lybtug-47/e?versionId=null 2024-04-16T18:12:48.468 DEBUG:boto:Data: 2024-04-16T18:12:48.468 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.468 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.468 DEBUG:boto:Params: {} 2024-04-16T18:12:48.468 DEBUG:boto:Token: None 2024-04-16T18:12:48.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e?versionId=null 2024-04-16T18:12:48.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew= 2024-04-16T18:12:48.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew=', 'Content-Length': '0'} 2024-04-16T18:12:48.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a88d8a15269b076-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.514 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.514 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.514 DEBUG:boto:path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.515 DEBUG:boto:auth_path=/lybtug-47/e?versionId=null 2024-04-16T18:12:48.515 DEBUG:boto:Method: GET 2024-04-16T18:12:48.515 DEBUG:boto:Path: /lybtug-47/e?versionId=null 2024-04-16T18:12:48.515 DEBUG:boto:Data: 2024-04-16T18:12:48.515 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.515 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.515 DEBUG:boto:Params: {} 2024-04-16T18:12:48.515 DEBUG:boto:Token: None 2024-04-16T18:12:48.515 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e?versionId=null 2024-04-16T18:12:48.515 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew= 2024-04-16T18:12:48.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wff+SDFWiogLccPQ1rIFBEBNIew=', 'Content-Length': '0'} 2024-04-16T18:12:48.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c110fbf57fdacbfd-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.519 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.519 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.519 DEBUG:boto:path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.519 DEBUG:boto:auth_path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.519 DEBUG:boto:Method: GET 2024-04-16T18:12:48.519 DEBUG:boto:Path: /lybtug-47/?prefix=e 2024-04-16T18:12:48.519 DEBUG:boto:Data: 2024-04-16T18:12:48.520 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.520 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.520 DEBUG:boto:Params: {} 2024-04-16T18:12:48.520 DEBUG:boto:Token: None 2024-04-16T18:12:48.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.523 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008aa9cd427920b67f-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.523 DEBUG:boto:b'lybtug-47e1000falsee2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.524 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.524 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.524 DEBUG:boto:path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.524 DEBUG:boto:auth_path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.524 DEBUG:boto:Method: GET 2024-04-16T18:12:48.524 DEBUG:boto:Path: /lybtug-47/?prefix=e 2024-04-16T18:12:48.524 DEBUG:boto:Data: 2024-04-16T18:12:48.524 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.524 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.524 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.524 DEBUG:boto:Params: {} 2024-04-16T18:12:48.524 DEBUG:boto:Token: None 2024-04-16T18:12:48.525 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d56a57dd9bb729ec-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.570 DEBUG:boto:b'lybtug-47e1000falsee2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.571 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.571 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.571 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.571 DEBUG:boto:Path: /lybtug-47/e 2024-04-16T18:12:48.571 DEBUG:boto:Data: 2024-04-16T18:12:48.571 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.571 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.571 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.571 DEBUG:boto:Params: {} 2024-04-16T18:12:48.571 DEBUG:boto:Token: None 2024-04-16T18:12:48.571 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e 2024-04-16T18:12:48.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YFgSX6W4HXC0QNHeefXQIUqkSRQ= 2024-04-16T18:12:48.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YFgSX6W4HXC0QNHeefXQIUqkSRQ=', 'Content-Length': '0'} 2024-04-16T18:12:48.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa82e608e590ecb2-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.575 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.575 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.575 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.575 DEBUG:boto:Path: /lybtug-47/e 2024-04-16T18:12:48.575 DEBUG:boto:Data: 2024-04-16T18:12:48.575 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.575 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.575 DEBUG:boto:Params: {} 2024-04-16T18:12:48.576 DEBUG:boto:Token: None 2024-04-16T18:12:48.576 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e 2024-04-16T18:12:48.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YFgSX6W4HXC0QNHeefXQIUqkSRQ= 2024-04-16T18:12:48.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YFgSX6W4HXC0QNHeefXQIUqkSRQ=', 'Content-Length': '0'} 2024-04-16T18:12:48.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000885cb6e85c67ee79-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:12:48.579 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.579 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.579 DEBUG:boto:Method: GET 2024-04-16T18:12:48.579 DEBUG:boto:Path: /lybtug-47/e 2024-04-16T18:12:48.579 DEBUG:boto:Data: 2024-04-16T18:12:48.579 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.579 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.579 DEBUG:boto:Params: {} 2024-04-16T18:12:48.580 DEBUG:boto:Token: None 2024-04-16T18:12:48.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e 2024-04-16T18:12:48.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bseiwOBYFcrl3WBNpNHTtQ8QM98= 2024-04-16T18:12:48.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bseiwOBYFcrl3WBNpNHTtQ8QM98=', 'Content-Length': '0'} 2024-04-16T18:12:48.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000606d0ba32c7b7144-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.626 DEBUG:boto:path=/lybtug-47/e 2024-04-16T18:12:48.627 DEBUG:boto:auth_path=/lybtug-47/e 2024-04-16T18:12:48.627 DEBUG:boto:Method: GET 2024-04-16T18:12:48.627 DEBUG:boto:Path: /lybtug-47/e 2024-04-16T18:12:48.627 DEBUG:boto:Data: 2024-04-16T18:12:48.627 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.627 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.627 DEBUG:boto:Params: {} 2024-04-16T18:12:48.627 DEBUG:boto:Token: None 2024-04-16T18:12:48.627 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/e 2024-04-16T18:12:48.627 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bseiwOBYFcrl3WBNpNHTtQ8QM98= 2024-04-16T18:12:48.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bseiwOBYFcrl3WBNpNHTtQ8QM98=', 'Content-Length': '0'} 2024-04-16T18:12:48.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d4d11a3484dd168-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.631 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.631 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.631 DEBUG:boto:path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.631 DEBUG:boto:auth_path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.631 DEBUG:boto:Method: GET 2024-04-16T18:12:48.631 DEBUG:boto:Path: /lybtug-47/?prefix=e 2024-04-16T18:12:48.631 DEBUG:boto:Data: 2024-04-16T18:12:48.631 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.631 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.631 DEBUG:boto:Params: {} 2024-04-16T18:12:48.631 DEBUG:boto:Token: None 2024-04-16T18:12:48.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.635 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a13f28b1ee3965a-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.678 DEBUG:boto:b'lybtug-47e1000falsee2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.679 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.679 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.679 DEBUG:boto:path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.679 DEBUG:boto:auth_path=/lybtug-47/?prefix=e 2024-04-16T18:12:48.679 DEBUG:boto:Method: GET 2024-04-16T18:12:48.679 DEBUG:boto:Path: /lybtug-47/?prefix=e 2024-04-16T18:12:48.679 DEBUG:boto:Data: 2024-04-16T18:12:48.679 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.679 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.679 DEBUG:boto:Params: {} 2024-04-16T18:12:48.680 DEBUG:boto:Token: None 2024-04-16T18:12:48.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.683 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001bc8ae44b9300d18-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.683 DEBUG:boto:b'lybtug-47e1000falsee2024-04-16T18:11:57.010Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.683 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:12:48.684 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.684 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.684 DEBUG:boto:path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.684 DEBUG:boto:auth_path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.684 DEBUG:boto:Method: GET 2024-04-16T18:12:48.684 DEBUG:boto:Path: /lybtug-47/f?versionId=null 2024-04-16T18:12:48.684 DEBUG:boto:Data: 2024-04-16T18:12:48.684 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.684 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.684 DEBUG:boto:Params: {} 2024-04-16T18:12:48.684 DEBUG:boto:Token: None 2024-04-16T18:12:48.685 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f?versionId=null 2024-04-16T18:12:48.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU= 2024-04-16T18:12:48.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU=', 'Content-Length': '0'} 2024-04-16T18:12:48.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db110f1d0dada693-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.730 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.731 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.731 DEBUG:boto:path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.731 DEBUG:boto:auth_path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.731 DEBUG:boto:Method: GET 2024-04-16T18:12:48.731 DEBUG:boto:Path: /lybtug-47/f?versionId=null 2024-04-16T18:12:48.731 DEBUG:boto:Data: 2024-04-16T18:12:48.731 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.731 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.731 DEBUG:boto:Params: {} 2024-04-16T18:12:48.731 DEBUG:boto:Token: None 2024-04-16T18:12:48.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f?versionId=null 2024-04-16T18:12:48.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU= 2024-04-16T18:12:48.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU=', 'Content-Length': '0'} 2024-04-16T18:12:48.735 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbaee575fffd27ef-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.735 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.735 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.735 DEBUG:boto:path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.735 DEBUG:boto:auth_path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.735 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.736 DEBUG:boto:Path: /lybtug-47/f?versionId=null 2024-04-16T18:12:48.736 DEBUG:boto:Data: 2024-04-16T18:12:48.736 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.736 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.736 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.736 DEBUG:boto:Params: {} 2024-04-16T18:12:48.736 DEBUG:boto:Token: None 2024-04-16T18:12:48.736 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f?versionId=null 2024-04-16T18:12:48.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eK6e8mcVvCdMAl5t9fNOffa6UD8= 2024-04-16T18:12:48.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eK6e8mcVvCdMAl5t9fNOffa6UD8=', 'Content-Length': '0'} 2024-04-16T18:12:48.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4785c0021e14de4-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.740 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.740 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.740 DEBUG:boto:path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.740 DEBUG:boto:auth_path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.740 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.740 DEBUG:boto:Path: /lybtug-47/f?versionId=null 2024-04-16T18:12:48.740 DEBUG:boto:Data: 2024-04-16T18:12:48.740 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.740 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.740 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.740 DEBUG:boto:Params: {} 2024-04-16T18:12:48.741 DEBUG:boto:Token: None 2024-04-16T18:12:48.741 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f?versionId=null 2024-04-16T18:12:48.741 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eK6e8mcVvCdMAl5t9fNOffa6UD8= 2024-04-16T18:12:48.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eK6e8mcVvCdMAl5t9fNOffa6UD8=', 'Content-Length': '0'} 2024-04-16T18:12:48.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bbe5d4deec640a7-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:12:48.744 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.744 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.744 DEBUG:boto:path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.744 DEBUG:boto:auth_path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.745 DEBUG:boto:Method: GET 2024-04-16T18:12:48.745 DEBUG:boto:Path: /lybtug-47/f?versionId=null 2024-04-16T18:12:48.745 DEBUG:boto:Data: 2024-04-16T18:12:48.745 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.745 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.745 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.745 DEBUG:boto:Params: {} 2024-04-16T18:12:48.745 DEBUG:boto:Token: None 2024-04-16T18:12:48.745 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f?versionId=null 2024-04-16T18:12:48.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU= 2024-04-16T18:12:48.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU=', 'Content-Length': '0'} 2024-04-16T18:12:48.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7be09abfbe52d17-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.790 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.790 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.791 DEBUG:boto:path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.791 DEBUG:boto:auth_path=/lybtug-47/f?versionId=null 2024-04-16T18:12:48.791 DEBUG:boto:Method: GET 2024-04-16T18:12:48.791 DEBUG:boto:Path: /lybtug-47/f?versionId=null 2024-04-16T18:12:48.791 DEBUG:boto:Data: 2024-04-16T18:12:48.791 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.791 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.791 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.791 DEBUG:boto:Params: {} 2024-04-16T18:12:48.791 DEBUG:boto:Token: None 2024-04-16T18:12:48.791 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f?versionId=null 2024-04-16T18:12:48.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU= 2024-04-16T18:12:48.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RywYosPosTNcmOve9Z/Zn5Zu6tU=', 'Content-Length': '0'} 2024-04-16T18:12:48.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000387f58157ee333f6-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.796 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.796 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.796 DEBUG:boto:path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.796 DEBUG:boto:auth_path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.796 DEBUG:boto:Method: GET 2024-04-16T18:12:48.796 DEBUG:boto:Path: /lybtug-47/?prefix=f 2024-04-16T18:12:48.796 DEBUG:boto:Data: 2024-04-16T18:12:48.796 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.796 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.796 DEBUG:boto:Params: {} 2024-04-16T18:12:48.796 DEBUG:boto:Token: None 2024-04-16T18:12:48.797 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.800 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b562c92ffb847f1e-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.800 DEBUG:boto:b'lybtug-47f1000falsef2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.801 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.801 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.801 DEBUG:boto:path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.801 DEBUG:boto:auth_path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.801 DEBUG:boto:Method: GET 2024-04-16T18:12:48.801 DEBUG:boto:Path: /lybtug-47/?prefix=f 2024-04-16T18:12:48.801 DEBUG:boto:Data: 2024-04-16T18:12:48.801 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.801 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.801 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.802 DEBUG:boto:Params: {} 2024-04-16T18:12:48.802 DEBUG:boto:Token: None 2024-04-16T18:12:48.802 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.802 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.805 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e57ebcee315c9ab-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.846 DEBUG:boto:b'lybtug-47f1000falsef2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.847 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.847 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.847 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.847 DEBUG:boto:Path: /lybtug-47/f 2024-04-16T18:12:48.847 DEBUG:boto:Data: 2024-04-16T18:12:48.847 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.848 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.848 DEBUG:boto:Params: {} 2024-04-16T18:12:48.848 DEBUG:boto:Token: None 2024-04-16T18:12:48.848 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f 2024-04-16T18:12:48.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Zt78LXeVqi7ZWtkZ+myPwtHSOPs= 2024-04-16T18:12:48.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Zt78LXeVqi7ZWtkZ+myPwtHSOPs=', 'Content-Length': '0'} 2024-04-16T18:12:48.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c1c7e7034f18319-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.852 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.852 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.852 DEBUG:boto:Method: HEAD 2024-04-16T18:12:48.852 DEBUG:boto:Path: /lybtug-47/f 2024-04-16T18:12:48.852 DEBUG:boto:Data: 2024-04-16T18:12:48.852 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.852 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.852 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.852 DEBUG:boto:Params: {} 2024-04-16T18:12:48.852 DEBUG:boto:Token: None 2024-04-16T18:12:48.853 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f 2024-04-16T18:12:48.853 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Zt78LXeVqi7ZWtkZ+myPwtHSOPs= 2024-04-16T18:12:48.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Zt78LXeVqi7ZWtkZ+myPwtHSOPs=', 'Content-Length': '0'} 2024-04-16T18:12:48.856 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e551d0c4a77ea3a8-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.856 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:12:48.856 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.856 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.856 DEBUG:boto:Method: GET 2024-04-16T18:12:48.856 DEBUG:boto:Path: /lybtug-47/f 2024-04-16T18:12:48.857 DEBUG:boto:Data: 2024-04-16T18:12:48.857 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.857 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.857 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.857 DEBUG:boto:Params: {} 2024-04-16T18:12:48.857 DEBUG:boto:Token: None 2024-04-16T18:12:48.857 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f 2024-04-16T18:12:48.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GbpLqmlmuJFyrz/N2WxDtW9us60= 2024-04-16T18:12:48.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GbpLqmlmuJFyrz/N2WxDtW9us60=', 'Content-Length': '0'} 2024-04-16T18:12:48.860 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c28b80f79f6881b-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.902 DEBUG:boto:path=/lybtug-47/f 2024-04-16T18:12:48.903 DEBUG:boto:auth_path=/lybtug-47/f 2024-04-16T18:12:48.903 DEBUG:boto:Method: GET 2024-04-16T18:12:48.903 DEBUG:boto:Path: /lybtug-47/f 2024-04-16T18:12:48.903 DEBUG:boto:Data: 2024-04-16T18:12:48.903 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.903 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.903 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.903 DEBUG:boto:Params: {} 2024-04-16T18:12:48.903 DEBUG:boto:Token: None 2024-04-16T18:12:48.903 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/f 2024-04-16T18:12:48.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GbpLqmlmuJFyrz/N2WxDtW9us60= 2024-04-16T18:12:48.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GbpLqmlmuJFyrz/N2WxDtW9us60=', 'Content-Length': '0'} 2024-04-16T18:12:48.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2658503a46c9566-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.907 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.907 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.907 DEBUG:boto:path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.907 DEBUG:boto:auth_path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.908 DEBUG:boto:Method: GET 2024-04-16T18:12:48.908 DEBUG:boto:Path: /lybtug-47/?prefix=f 2024-04-16T18:12:48.908 DEBUG:boto:Data: 2024-04-16T18:12:48.908 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.908 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.908 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.908 DEBUG:boto:Params: {} 2024-04-16T18:12:48.908 DEBUG:boto:Token: None 2024-04-16T18:12:48.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.912 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0ebcbdf0d7f723f-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.954 DEBUG:boto:b'lybtug-47f1000falsef2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.955 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:48.955 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:48.955 DEBUG:boto:path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.955 DEBUG:boto:auth_path=/lybtug-47/?prefix=f 2024-04-16T18:12:48.955 DEBUG:boto:Method: GET 2024-04-16T18:12:48.956 DEBUG:boto:Path: /lybtug-47/?prefix=f 2024-04-16T18:12:48.956 DEBUG:boto:Data: 2024-04-16T18:12:48.956 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.956 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:48.956 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.956 DEBUG:boto:Params: {} 2024-04-16T18:12:48.956 DEBUG:boto:Token: None 2024-04-16T18:12:48.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/ 2024-04-16T18:12:48.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g= 2024-04-16T18:12:48.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6y31O94gRkTQOHMz38swZTv5f8g=', 'Content-Length': '0'} 2024-04-16T18:12:48.959 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032072f7aa3601f85-00661ebfa0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:48.960 DEBUG:boto:b'lybtug-47f1000falsef2024-04-16T18:11:57.028Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:48.960 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:12:48.960 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:48.960 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:48.960 DEBUG:boto:path=/lybtug-47/g?versionId=null 2024-04-16T18:12:48.960 DEBUG:boto:auth_path=/lybtug-47/g?versionId=null 2024-04-16T18:12:48.961 DEBUG:boto:Method: GET 2024-04-16T18:12:48.961 DEBUG:boto:Path: /lybtug-47/g?versionId=null 2024-04-16T18:12:48.961 DEBUG:boto:Data: 2024-04-16T18:12:48.961 DEBUG:boto:Headers: {} 2024-04-16T18:12:48.961 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:48.961 DEBUG:boto:Port: 8000 2024-04-16T18:12:48.961 DEBUG:boto:Params: {} 2024-04-16T18:12:48.961 DEBUG:boto:Token: None 2024-04-16T18:12:48.961 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:48 GMT /lybtug-47/g?versionId=null 2024-04-16T18:12:48.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:izKS8TeXBUnD0Ei3tzJomk6j+Ac= 2024-04-16T18:12:48.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:izKS8TeXBUnD0Ei3tzJomk6j+Ac=', 'Content-Length': '0'} 2024-04-16T18:12:48.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:45 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0618002e0b45292-00661ebfa0-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.006 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.007 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.007 DEBUG:boto:path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.007 DEBUG:boto:auth_path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.007 DEBUG:boto:Method: GET 2024-04-16T18:12:49.007 DEBUG:boto:Path: /lybtug-47/g?versionId=null 2024-04-16T18:12:49.007 DEBUG:boto:Data: 2024-04-16T18:12:49.007 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.007 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.007 DEBUG:boto:Params: {} 2024-04-16T18:12:49.008 DEBUG:boto:Token: None 2024-04-16T18:12:49.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g?versionId=null 2024-04-16T18:12:49.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPXS2QTORDrT1LQ2M8PLFeevhZk= 2024-04-16T18:12:49.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPXS2QTORDrT1LQ2M8PLFeevhZk=', 'Content-Length': '0'} 2024-04-16T18:12:49.011 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a551b46e990fd3f-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.011 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.012 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.012 DEBUG:boto:path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.012 DEBUG:boto:auth_path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.012 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.012 DEBUG:boto:Path: /lybtug-47/g?versionId=null 2024-04-16T18:12:49.012 DEBUG:boto:Data: 2024-04-16T18:12:49.012 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.012 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.012 DEBUG:boto:Params: {} 2024-04-16T18:12:49.012 DEBUG:boto:Token: None 2024-04-16T18:12:49.013 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g?versionId=null 2024-04-16T18:12:49.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ok2VXtspRzcjG3dTfuiOKkSj4pk= 2024-04-16T18:12:49.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ok2VXtspRzcjG3dTfuiOKkSj4pk=', 'Content-Length': '0'} 2024-04-16T18:12:49.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:45 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044dd3e6c1ee69c73-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.016 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.016 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.016 DEBUG:boto:path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.016 DEBUG:boto:auth_path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.016 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.016 DEBUG:boto:Path: /lybtug-47/g?versionId=null 2024-04-16T18:12:49.016 DEBUG:boto:Data: 2024-04-16T18:12:49.017 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.017 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.017 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.017 DEBUG:boto:Params: {} 2024-04-16T18:12:49.017 DEBUG:boto:Token: None 2024-04-16T18:12:49.017 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g?versionId=null 2024-04-16T18:12:49.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ok2VXtspRzcjG3dTfuiOKkSj4pk= 2024-04-16T18:12:49.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ok2VXtspRzcjG3dTfuiOKkSj4pk=', 'Content-Length': '0'} 2024-04-16T18:12:49.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff16d551ebc6777e-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.020 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:12:49.020 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.020 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.020 DEBUG:boto:path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.020 DEBUG:boto:auth_path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.020 DEBUG:boto:Method: GET 2024-04-16T18:12:49.021 DEBUG:boto:Path: /lybtug-47/g?versionId=null 2024-04-16T18:12:49.021 DEBUG:boto:Data: 2024-04-16T18:12:49.021 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.021 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.021 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.021 DEBUG:boto:Params: {} 2024-04-16T18:12:49.021 DEBUG:boto:Token: None 2024-04-16T18:12:49.021 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g?versionId=null 2024-04-16T18:12:49.021 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPXS2QTORDrT1LQ2M8PLFeevhZk= 2024-04-16T18:12:49.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPXS2QTORDrT1LQ2M8PLFeevhZk=', 'Content-Length': '0'} 2024-04-16T18:12:49.024 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:45 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003c4ef1add47f2ac-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.066 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.067 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.067 DEBUG:boto:path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.067 DEBUG:boto:auth_path=/lybtug-47/g?versionId=null 2024-04-16T18:12:49.067 DEBUG:boto:Method: GET 2024-04-16T18:12:49.067 DEBUG:boto:Path: /lybtug-47/g?versionId=null 2024-04-16T18:12:49.067 DEBUG:boto:Data: 2024-04-16T18:12:49.067 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.067 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.067 DEBUG:boto:Params: {} 2024-04-16T18:12:49.068 DEBUG:boto:Token: None 2024-04-16T18:12:49.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g?versionId=null 2024-04-16T18:12:49.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPXS2QTORDrT1LQ2M8PLFeevhZk= 2024-04-16T18:12:49.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPXS2QTORDrT1LQ2M8PLFeevhZk=', 'Content-Length': '0'} 2024-04-16T18:12:49.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d095382526e09022-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.071 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.072 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.072 DEBUG:boto:path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.072 DEBUG:boto:auth_path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.072 DEBUG:boto:Method: GET 2024-04-16T18:12:49.072 DEBUG:boto:Path: /lybtug-47/?prefix=g 2024-04-16T18:12:49.072 DEBUG:boto:Data: 2024-04-16T18:12:49.072 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.072 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.072 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.072 DEBUG:boto:Params: {} 2024-04-16T18:12:49.072 DEBUG:boto:Token: None 2024-04-16T18:12:49.073 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e3eeb9590be4c72-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.076 DEBUG:boto:b'lybtug-47g1000falseg2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.077 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.077 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.077 DEBUG:boto:path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.077 DEBUG:boto:auth_path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.077 DEBUG:boto:Method: GET 2024-04-16T18:12:49.077 DEBUG:boto:Path: /lybtug-47/?prefix=g 2024-04-16T18:12:49.077 DEBUG:boto:Data: 2024-04-16T18:12:49.077 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.077 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.077 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.077 DEBUG:boto:Params: {} 2024-04-16T18:12:49.078 DEBUG:boto:Token: None 2024-04-16T18:12:49.078 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.078 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.081 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ac71520dd5dc4b8-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.122 DEBUG:boto:b'lybtug-47g1000falseg2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.123 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.123 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.123 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.123 DEBUG:boto:Path: /lybtug-47/g 2024-04-16T18:12:49.123 DEBUG:boto:Data: 2024-04-16T18:12:49.123 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.123 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.124 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.124 DEBUG:boto:Params: {} 2024-04-16T18:12:49.124 DEBUG:boto:Token: None 2024-04-16T18:12:49.124 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g 2024-04-16T18:12:49.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BFYh4c1NddgqisCG3ko3g3WhpEM= 2024-04-16T18:12:49.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BFYh4c1NddgqisCG3ko3g3WhpEM=', 'Content-Length': '0'} 2024-04-16T18:12:49.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:45 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1c7f33e1547bd0e-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.127 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.127 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.127 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.127 DEBUG:boto:Path: /lybtug-47/g 2024-04-16T18:12:49.128 DEBUG:boto:Data: 2024-04-16T18:12:49.128 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.128 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.128 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.128 DEBUG:boto:Params: {} 2024-04-16T18:12:49.128 DEBUG:boto:Token: None 2024-04-16T18:12:49.128 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g 2024-04-16T18:12:49.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BFYh4c1NddgqisCG3ko3g3WhpEM= 2024-04-16T18:12:49.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BFYh4c1NddgqisCG3ko3g3WhpEM=', 'Content-Length': '0'} 2024-04-16T18:12:49.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e29bdc81430e33c-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.131 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:12:49.131 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.131 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.132 DEBUG:boto:Method: GET 2024-04-16T18:12:49.132 DEBUG:boto:Path: /lybtug-47/g 2024-04-16T18:12:49.132 DEBUG:boto:Data: 2024-04-16T18:12:49.132 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.132 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.132 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.132 DEBUG:boto:Params: {} 2024-04-16T18:12:49.132 DEBUG:boto:Token: None 2024-04-16T18:12:49.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g 2024-04-16T18:12:49.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:925k1t+NPHgqLmBc7UoZJUU7sDo= 2024-04-16T18:12:49.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:925k1t+NPHgqLmBc7UoZJUU7sDo=', 'Content-Length': '0'} 2024-04-16T18:12:49.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:45 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9859affdef38fab-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.178 DEBUG:boto:path=/lybtug-47/g 2024-04-16T18:12:49.179 DEBUG:boto:auth_path=/lybtug-47/g 2024-04-16T18:12:49.179 DEBUG:boto:Method: GET 2024-04-16T18:12:49.179 DEBUG:boto:Path: /lybtug-47/g 2024-04-16T18:12:49.179 DEBUG:boto:Data: 2024-04-16T18:12:49.179 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.179 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.179 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.179 DEBUG:boto:Params: {} 2024-04-16T18:12:49.180 DEBUG:boto:Token: None 2024-04-16T18:12:49.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/g 2024-04-16T18:12:49.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:925k1t+NPHgqLmBc7UoZJUU7sDo= 2024-04-16T18:12:49.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:925k1t+NPHgqLmBc7UoZJUU7sDo=', 'Content-Length': '0'} 2024-04-16T18:12:49.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015d1fe28c3c2cae2-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.183 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.183 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.183 DEBUG:boto:path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.183 DEBUG:boto:auth_path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.183 DEBUG:boto:Method: GET 2024-04-16T18:12:49.184 DEBUG:boto:Path: /lybtug-47/?prefix=g 2024-04-16T18:12:49.184 DEBUG:boto:Data: 2024-04-16T18:12:49.184 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.184 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.184 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.184 DEBUG:boto:Params: {} 2024-04-16T18:12:49.184 DEBUG:boto:Token: None 2024-04-16T18:12:49.184 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.187 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000693ffb862a35faef-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.230 DEBUG:boto:b'lybtug-47g1000falseg2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.231 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.231 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.231 DEBUG:boto:path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.231 DEBUG:boto:auth_path=/lybtug-47/?prefix=g 2024-04-16T18:12:49.231 DEBUG:boto:Method: GET 2024-04-16T18:12:49.231 DEBUG:boto:Path: /lybtug-47/?prefix=g 2024-04-16T18:12:49.232 DEBUG:boto:Data: 2024-04-16T18:12:49.232 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.232 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.232 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.232 DEBUG:boto:Params: {} 2024-04-16T18:12:49.232 DEBUG:boto:Token: None 2024-04-16T18:12:49.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004cf715e92fa3e5e4-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.236 DEBUG:boto:b'lybtug-47g1000falseg2024-04-16T18:11:57.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.236 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:12:49.236 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.236 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.236 DEBUG:boto:path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.236 DEBUG:boto:auth_path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.236 DEBUG:boto:Method: GET 2024-04-16T18:12:49.236 DEBUG:boto:Path: /lybtug-47/h?versionId=null 2024-04-16T18:12:49.237 DEBUG:boto:Data: 2024-04-16T18:12:49.237 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.237 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.237 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.237 DEBUG:boto:Params: {} 2024-04-16T18:12:49.237 DEBUG:boto:Token: None 2024-04-16T18:12:49.237 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h?versionId=null 2024-04-16T18:12:49.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ= 2024-04-16T18:12:49.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ=', 'Content-Length': '0'} 2024-04-16T18:12:49.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a81f7481b1b8ab92-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.282 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.283 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.283 DEBUG:boto:path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.283 DEBUG:boto:auth_path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.283 DEBUG:boto:Method: GET 2024-04-16T18:12:49.283 DEBUG:boto:Path: /lybtug-47/h?versionId=null 2024-04-16T18:12:49.283 DEBUG:boto:Data: 2024-04-16T18:12:49.283 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.283 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.283 DEBUG:boto:Params: {} 2024-04-16T18:12:49.284 DEBUG:boto:Token: None 2024-04-16T18:12:49.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h?versionId=null 2024-04-16T18:12:49.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ= 2024-04-16T18:12:49.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ=', 'Content-Length': '0'} 2024-04-16T18:12:49.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000189c8fca3ad68aee-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.289 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.289 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.289 DEBUG:boto:path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.289 DEBUG:boto:auth_path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.289 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.289 DEBUG:boto:Path: /lybtug-47/h?versionId=null 2024-04-16T18:12:49.289 DEBUG:boto:Data: 2024-04-16T18:12:49.289 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.289 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.290 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.290 DEBUG:boto:Params: {} 2024-04-16T18:12:49.290 DEBUG:boto:Token: None 2024-04-16T18:12:49.290 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h?versionId=null 2024-04-16T18:12:49.290 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K1tUwpuUphDQuErV2w8sqYPfgs4= 2024-04-16T18:12:49.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K1tUwpuUphDQuErV2w8sqYPfgs4=', 'Content-Length': '0'} 2024-04-16T18:12:49.293 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7fd6013f76b5f68-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.293 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.293 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.293 DEBUG:boto:path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.293 DEBUG:boto:auth_path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.293 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.293 DEBUG:boto:Path: /lybtug-47/h?versionId=null 2024-04-16T18:12:49.294 DEBUG:boto:Data: 2024-04-16T18:12:49.294 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.294 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.294 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.294 DEBUG:boto:Params: {} 2024-04-16T18:12:49.294 DEBUG:boto:Token: None 2024-04-16T18:12:49.294 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h?versionId=null 2024-04-16T18:12:49.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K1tUwpuUphDQuErV2w8sqYPfgs4= 2024-04-16T18:12:49.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K1tUwpuUphDQuErV2w8sqYPfgs4=', 'Content-Length': '0'} 2024-04-16T18:12:49.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000399a9b95c44f79be-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.297 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:12:49.297 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.297 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.298 DEBUG:boto:path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.298 DEBUG:boto:auth_path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.298 DEBUG:boto:Method: GET 2024-04-16T18:12:49.298 DEBUG:boto:Path: /lybtug-47/h?versionId=null 2024-04-16T18:12:49.298 DEBUG:boto:Data: 2024-04-16T18:12:49.298 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.298 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.298 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.298 DEBUG:boto:Params: {} 2024-04-16T18:12:49.298 DEBUG:boto:Token: None 2024-04-16T18:12:49.298 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h?versionId=null 2024-04-16T18:12:49.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ= 2024-04-16T18:12:49.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ=', 'Content-Length': '0'} 2024-04-16T18:12:49.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005745947bd519bff5-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.342 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.343 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.343 DEBUG:boto:path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.343 DEBUG:boto:auth_path=/lybtug-47/h?versionId=null 2024-04-16T18:12:49.343 DEBUG:boto:Method: GET 2024-04-16T18:12:49.343 DEBUG:boto:Path: /lybtug-47/h?versionId=null 2024-04-16T18:12:49.343 DEBUG:boto:Data: 2024-04-16T18:12:49.343 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.343 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.344 DEBUG:boto:Params: {} 2024-04-16T18:12:49.344 DEBUG:boto:Token: None 2024-04-16T18:12:49.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h?versionId=null 2024-04-16T18:12:49.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ= 2024-04-16T18:12:49.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xag4hjmJXtBMQ7GMNmCHvkGyozQ=', 'Content-Length': '0'} 2024-04-16T18:12:49.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d73094da1056378-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.347 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.347 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.348 DEBUG:boto:path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.348 DEBUG:boto:auth_path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.348 DEBUG:boto:Method: GET 2024-04-16T18:12:49.348 DEBUG:boto:Path: /lybtug-47/?prefix=h 2024-04-16T18:12:49.348 DEBUG:boto:Data: 2024-04-16T18:12:49.348 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.348 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.348 DEBUG:boto:Params: {} 2024-04-16T18:12:49.348 DEBUG:boto:Token: None 2024-04-16T18:12:49.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000e4425a37ef5dc0b-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.394 DEBUG:boto:b'lybtug-47h1000falseh2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.395 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.395 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.395 DEBUG:boto:path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.395 DEBUG:boto:auth_path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.395 DEBUG:boto:Method: GET 2024-04-16T18:12:49.396 DEBUG:boto:Path: /lybtug-47/?prefix=h 2024-04-16T18:12:49.396 DEBUG:boto:Data: 2024-04-16T18:12:49.396 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.396 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.396 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.396 DEBUG:boto:Params: {} 2024-04-16T18:12:49.396 DEBUG:boto:Token: None 2024-04-16T18:12:49.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.400 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000725f088fb0dba589-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.400 DEBUG:boto:b'lybtug-47h1000falseh2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.400 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.400 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.401 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.401 DEBUG:boto:Path: /lybtug-47/h 2024-04-16T18:12:49.401 DEBUG:boto:Data: 2024-04-16T18:12:49.401 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.401 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.401 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.401 DEBUG:boto:Params: {} 2024-04-16T18:12:49.401 DEBUG:boto:Token: None 2024-04-16T18:12:49.401 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h 2024-04-16T18:12:49.401 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cDPpwK1orYcSXDuOU+1naCPbcrU= 2024-04-16T18:12:49.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cDPpwK1orYcSXDuOU+1naCPbcrU=', 'Content-Length': '0'} 2024-04-16T18:12:49.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001cb1e453d3435b5d-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.404 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.404 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.405 DEBUG:boto:Method: HEAD 2024-04-16T18:12:49.405 DEBUG:boto:Path: /lybtug-47/h 2024-04-16T18:12:49.405 DEBUG:boto:Data: 2024-04-16T18:12:49.405 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.405 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.405 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.405 DEBUG:boto:Params: {} 2024-04-16T18:12:49.405 DEBUG:boto:Token: None 2024-04-16T18:12:49.405 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h 2024-04-16T18:12:49.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cDPpwK1orYcSXDuOU+1naCPbcrU= 2024-04-16T18:12:49.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cDPpwK1orYcSXDuOU+1naCPbcrU=', 'Content-Length': '0'} 2024-04-16T18:12:49.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be1f76830ec05cb8-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.408 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:12:49.409 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.409 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.409 DEBUG:boto:Method: GET 2024-04-16T18:12:49.409 DEBUG:boto:Path: /lybtug-47/h 2024-04-16T18:12:49.409 DEBUG:boto:Data: 2024-04-16T18:12:49.409 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.409 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.409 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.409 DEBUG:boto:Params: {} 2024-04-16T18:12:49.409 DEBUG:boto:Token: None 2024-04-16T18:12:49.410 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h 2024-04-16T18:12:49.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BqKFFJtw9gqtrdn1i4xjl50dp/8= 2024-04-16T18:12:49.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BqKFFJtw9gqtrdn1i4xjl50dp/8=', 'Content-Length': '0'} 2024-04-16T18:12:49.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-47:054102a3-8d94-4596-8b3b-6341eab312e1.4804.16'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:12:11 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c9450af082b3b53-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.454 DEBUG:boto:path=/lybtug-47/h 2024-04-16T18:12:49.455 DEBUG:boto:auth_path=/lybtug-47/h 2024-04-16T18:12:49.455 DEBUG:boto:Method: GET 2024-04-16T18:12:49.455 DEBUG:boto:Path: /lybtug-47/h 2024-04-16T18:12:49.455 DEBUG:boto:Data: 2024-04-16T18:12:49.455 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.455 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.455 DEBUG:boto:Params: {} 2024-04-16T18:12:49.455 DEBUG:boto:Token: None 2024-04-16T18:12:49.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/h 2024-04-16T18:12:49.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BqKFFJtw9gqtrdn1i4xjl50dp/8= 2024-04-16T18:12:49.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BqKFFJtw9gqtrdn1i4xjl50dp/8=', 'Content-Length': '0'} 2024-04-16T18:12:49.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:11:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3e4386e88dc4546-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.459 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.460 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.460 DEBUG:boto:path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.460 DEBUG:boto:auth_path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.460 DEBUG:boto:Method: GET 2024-04-16T18:12:49.460 DEBUG:boto:Path: /lybtug-47/?prefix=h 2024-04-16T18:12:49.460 DEBUG:boto:Data: 2024-04-16T18:12:49.460 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:49.460 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.460 DEBUG:boto:Params: {} 2024-04-16T18:12:49.460 DEBUG:boto:Token: None 2024-04-16T18:12:49.461 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006af69c2f9d372a30-00661ebfa1-4984-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.506 DEBUG:boto:b'lybtug-47h1000falseh2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.507 DEBUG:boto:path=/lybtug-47/ 2024-04-16T18:12:49.507 DEBUG:boto:auth_path=/lybtug-47/ 2024-04-16T18:12:49.507 DEBUG:boto:path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.507 DEBUG:boto:auth_path=/lybtug-47/?prefix=h 2024-04-16T18:12:49.507 DEBUG:boto:Method: GET 2024-04-16T18:12:49.508 DEBUG:boto:Path: /lybtug-47/?prefix=h 2024-04-16T18:12:49.508 DEBUG:boto:Data: 2024-04-16T18:12:49.508 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.508 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.508 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.508 DEBUG:boto:Params: {} 2024-04-16T18:12:49.508 DEBUG:boto:Token: None 2024-04-16T18:12:49.508 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:12:49 GMT /lybtug-47/ 2024-04-16T18:12:49.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o= 2024-04-16T18:12:49.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iemdWmFMWeljpIpZoWKShQwvH/o=', 'Content-Length': '0'} 2024-04-16T18:12:49.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000620af66e7347329a-00661ebfa1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.512 DEBUG:boto:b'lybtug-47h1000falseh2024-04-16T18:11:57.093Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:12:49.512 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-47 zones={test-zone2, test-zone1} 2024-04-16T18:12:49.513 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_reshard_incremental ... ok 2024-04-16T18:12:49.513 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:49.514 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:49.514 DEBUG:boto:Method: PUT 2024-04-16T18:12:49.514 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:49.514 DEBUG:boto:Data: 2024-04-16T18:12:49.514 DEBUG:boto:Headers: {} 2024-04-16T18:12:49.514 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:49.514 DEBUG:boto:Port: 8000 2024-04-16T18:12:49.514 DEBUG:boto:Params: {} 2024-04-16T18:12:49.514 DEBUG:boto:Token: None 2024-04-16T18:12:49.514 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:12:49 GMT /lybtug-48/ 2024-04-16T18:12:49.514 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+iA4yVYg8p6QccqUsU0Lr63w5wk= 2024-04-16T18:12:49.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+iA4yVYg8p6QccqUsU0Lr63w5wk=', 'Content-Length': '0'} 2024-04-16T18:12:49.551 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000033a3a575aaf9acba-00661ebfa1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:49.551 DEBUG:rgw_multi.tests:created bucket=lybtug-48 2024-04-16T18:12:49.551 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:12:49.551 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:12:49.552 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:12:49.594 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:12:49.594 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:12:49.664 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:12:49.696 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291169.550435_470.1","last_update":"2024-04-16T18:12:49.550435Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:12:49.697 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291169.550435_470.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:12:49.697 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:12:49.697 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:12:49.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:12:49.697 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:12:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:12:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:12:49.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:49.815+0000 7f15fddcb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:12:49.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:49.819+0000 7f15fddcb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:12:49.848 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291169.550435_470.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:49.550435Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:12:49Z"} 2024-04-16T18:12:49.849 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291169.550435_470.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:49.550435Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:12:49Z"} 2024-04-16T18:12:49.849 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291169.550435_470.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:49.550435Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:12:49.849 DEBUG:rgw_multi.tests:log_status={0: '1_1713291169.550435_470.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:12:49.849 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291169.550435_470.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:12:49.849 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:12:49.849 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2024-04-16T18:12:49.850 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T18:12:49.890 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:12:49.895+0000 7f4aa480d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 31368) UID: 0 2024-04-16T18:12:49.891 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:12:49.895+0000 7f4aa8055a40 -1 shutting down 2024-04-16T18:12:52.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T18:12:55.952 INFO:tasks.rgw.c2.client.0:Stopped 2024-04-16T18:12:55.952 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:55.953 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:55.953 DEBUG:boto:Method: HEAD 2024-04-16T18:12:55.953 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:55.953 DEBUG:boto:Data: 2024-04-16T18:12:55.953 DEBUG:boto:Headers: {} 2024-04-16T18:12:55.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:55.953 DEBUG:boto:Port: 8000 2024-04-16T18:12:55.953 DEBUG:boto:Params: {} 2024-04-16T18:12:55.953 DEBUG:boto:Token: None 2024-04-16T18:12:55.953 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:55 GMT /lybtug-48/ 2024-04-16T18:12:55.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:euEugoFsL5DdxsaAhg+AHgJdSKM= 2024-04-16T18:12:55.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:euEugoFsL5DdxsaAhg+AHgJdSKM=', 'Content-Length': '0'} 2024-04-16T18:12:55.958 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a15885eaf02067f8-00661ebfa7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:55.958 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:12:55.958 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:12:55.958 DEBUG:boto:Method: PUT 2024-04-16T18:12:55.958 DEBUG:boto:Path: /lybtug-48/a 2024-04-16T18:12:55.958 DEBUG:boto:Data: 2024-04-16T18:12:55.958 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:55.958 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:55.959 DEBUG:boto:Port: 8000 2024-04-16T18:12:55.959 DEBUG:boto:Params: {} 2024-04-16T18:12:55.959 DEBUG:boto:Token: None 2024-04-16T18:12:55.959 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:55 GMT /lybtug-48/a 2024-04-16T18:12:55.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:clzHW9cRJKeUn8mug367q9yqj8Y= 2024-04-16T18:12:55.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:clzHW9cRJKeUn8mug367q9yqj8Y='} 2024-04-16T18:12:55.974 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000076d938288288b75b-00661ebfa7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:55.974 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:55.974 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:55.974 DEBUG:boto:Method: HEAD 2024-04-16T18:12:55.975 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:55.975 DEBUG:boto:Data: 2024-04-16T18:12:55.975 DEBUG:boto:Headers: {} 2024-04-16T18:12:55.975 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:55.975 DEBUG:boto:Port: 8000 2024-04-16T18:12:55.975 DEBUG:boto:Params: {} 2024-04-16T18:12:55.975 DEBUG:boto:Token: None 2024-04-16T18:12:55.975 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:55 GMT /lybtug-48/ 2024-04-16T18:12:55.975 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:euEugoFsL5DdxsaAhg+AHgJdSKM= 2024-04-16T18:12:55.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:euEugoFsL5DdxsaAhg+AHgJdSKM=', 'Content-Length': '0'} 2024-04-16T18:12:55.979 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000047a6fa4c285a8e0f-00661ebfa7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:55.979 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:12:55.980 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:12:55.980 DEBUG:boto:Method: PUT 2024-04-16T18:12:55.980 DEBUG:boto:Path: /lybtug-48/b 2024-04-16T18:12:55.980 DEBUG:boto:Data: 2024-04-16T18:12:55.980 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:55.980 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:55.980 DEBUG:boto:Port: 8000 2024-04-16T18:12:55.980 DEBUG:boto:Params: {} 2024-04-16T18:12:55.980 DEBUG:boto:Token: None 2024-04-16T18:12:55.980 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:55 GMT /lybtug-48/b 2024-04-16T18:12:55.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ouw4svHxRxgCB5eVJRi+8/PYvcg= 2024-04-16T18:12:55.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ouw4svHxRxgCB5eVJRi+8/PYvcg='} 2024-04-16T18:12:56.023 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000961c1f2348129047-00661ebfa7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.023 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:56.023 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:56.023 DEBUG:boto:Method: HEAD 2024-04-16T18:12:56.023 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:56.023 DEBUG:boto:Data: 2024-04-16T18:12:56.023 DEBUG:boto:Headers: {} 2024-04-16T18:12:56.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.024 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.024 DEBUG:boto:Params: {} 2024-04-16T18:12:56.024 DEBUG:boto:Token: None 2024-04-16T18:12:56.024 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/ 2024-04-16T18:12:56.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0= 2024-04-16T18:12:56.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0=', 'Content-Length': '0'} 2024-04-16T18:12:56.028 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fa4e57cb71965635-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.029 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:12:56.029 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:12:56.029 DEBUG:boto:Method: PUT 2024-04-16T18:12:56.029 DEBUG:boto:Path: /lybtug-48/c 2024-04-16T18:12:56.029 DEBUG:boto:Data: 2024-04-16T18:12:56.029 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:56.029 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.029 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.029 DEBUG:boto:Params: {} 2024-04-16T18:12:56.029 DEBUG:boto:Token: None 2024-04-16T18:12:56.030 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/c 2024-04-16T18:12:56.030 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gaFUI3iH0zK4+P5DAjwEBvbB/KE= 2024-04-16T18:12:56.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gaFUI3iH0zK4+P5DAjwEBvbB/KE='} 2024-04-16T18:12:56.041 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e5d954e0a745284b-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.042 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:56.042 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:56.042 DEBUG:boto:Method: HEAD 2024-04-16T18:12:56.042 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:56.042 DEBUG:boto:Data: 2024-04-16T18:12:56.042 DEBUG:boto:Headers: {} 2024-04-16T18:12:56.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.042 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.042 DEBUG:boto:Params: {} 2024-04-16T18:12:56.042 DEBUG:boto:Token: None 2024-04-16T18:12:56.043 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/ 2024-04-16T18:12:56.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0= 2024-04-16T18:12:56.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0=', 'Content-Length': '0'} 2024-04-16T18:12:56.046 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000694e3a147856fbaf-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.046 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:12:56.046 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:12:56.047 DEBUG:boto:Method: PUT 2024-04-16T18:12:56.047 DEBUG:boto:Path: /lybtug-48/d 2024-04-16T18:12:56.047 DEBUG:boto:Data: 2024-04-16T18:12:56.047 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:56.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.047 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.047 DEBUG:boto:Params: {} 2024-04-16T18:12:56.047 DEBUG:boto:Token: None 2024-04-16T18:12:56.047 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/d 2024-04-16T18:12:56.047 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e9tRLx7zk8lTjXimDIvzkL2qLWw= 2024-04-16T18:12:56.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e9tRLx7zk8lTjXimDIvzkL2qLWw='} 2024-04-16T18:12:56.091 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e030cc0c4688d790-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.091 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-48', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:12:56.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'lybtug-48', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:12:56.091 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket lybtug-48 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:12:56.134 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:12:56.134 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:12:56.223 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:12:56.223 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-48 2024-04-16T18:12:56.232 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 4 2024-04-16T18:12:56.330 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:56.330 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:56.330 DEBUG:boto:Method: HEAD 2024-04-16T18:12:56.330 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:56.330 DEBUG:boto:Data: 2024-04-16T18:12:56.330 DEBUG:boto:Headers: {} 2024-04-16T18:12:56.331 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.331 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.331 DEBUG:boto:Params: {} 2024-04-16T18:12:56.331 DEBUG:boto:Token: None 2024-04-16T18:12:56.331 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/ 2024-04-16T18:12:56.331 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0= 2024-04-16T18:12:56.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0=', 'Content-Length': '0'} 2024-04-16T18:12:56.334 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cb0818fe5524d5d0-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.334 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:12:56.334 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:12:56.334 DEBUG:boto:Method: PUT 2024-04-16T18:12:56.335 DEBUG:boto:Path: /lybtug-48/e 2024-04-16T18:12:56.335 DEBUG:boto:Data: 2024-04-16T18:12:56.335 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:56.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.335 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.335 DEBUG:boto:Params: {} 2024-04-16T18:12:56.335 DEBUG:boto:Token: None 2024-04-16T18:12:56.335 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/e 2024-04-16T18:12:56.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7i1OFf5SdMa3WM2TFHBtB2zY9h8= 2024-04-16T18:12:56.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7i1OFf5SdMa3WM2TFHBtB2zY9h8='} 2024-04-16T18:12:56.345 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008b052a7c543e57a1-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.346 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:56.346 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:56.346 DEBUG:boto:Method: HEAD 2024-04-16T18:12:56.346 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:56.346 DEBUG:boto:Data: 2024-04-16T18:12:56.346 DEBUG:boto:Headers: {} 2024-04-16T18:12:56.346 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.346 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.346 DEBUG:boto:Params: {} 2024-04-16T18:12:56.347 DEBUG:boto:Token: None 2024-04-16T18:12:56.347 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/ 2024-04-16T18:12:56.347 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0= 2024-04-16T18:12:56.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0=', 'Content-Length': '0'} 2024-04-16T18:12:56.349 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003809fc82cf1ac3a5-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.349 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:12:56.349 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:12:56.349 DEBUG:boto:Method: PUT 2024-04-16T18:12:56.350 DEBUG:boto:Path: /lybtug-48/f 2024-04-16T18:12:56.350 DEBUG:boto:Data: 2024-04-16T18:12:56.350 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:56.350 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.350 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.350 DEBUG:boto:Params: {} 2024-04-16T18:12:56.350 DEBUG:boto:Token: None 2024-04-16T18:12:56.350 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/f 2024-04-16T18:12:56.350 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F0Zi9Ut2GrPC8N+bCsf9ENij+lg= 2024-04-16T18:12:56.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F0Zi9Ut2GrPC8N+bCsf9ENij+lg='} 2024-04-16T18:12:56.360 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005adefd7f602ee929-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.360 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:56.360 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:56.361 DEBUG:boto:Method: HEAD 2024-04-16T18:12:56.361 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:56.361 DEBUG:boto:Data: 2024-04-16T18:12:56.361 DEBUG:boto:Headers: {} 2024-04-16T18:12:56.361 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.361 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.361 DEBUG:boto:Params: {} 2024-04-16T18:12:56.361 DEBUG:boto:Token: None 2024-04-16T18:12:56.361 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/ 2024-04-16T18:12:56.361 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0= 2024-04-16T18:12:56.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0=', 'Content-Length': '0'} 2024-04-16T18:12:56.364 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000609803a9b5d4840-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.364 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:12:56.364 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:12:56.364 DEBUG:boto:Method: PUT 2024-04-16T18:12:56.364 DEBUG:boto:Path: /lybtug-48/g 2024-04-16T18:12:56.365 DEBUG:boto:Data: 2024-04-16T18:12:56.365 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:56.365 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.365 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.365 DEBUG:boto:Params: {} 2024-04-16T18:12:56.365 DEBUG:boto:Token: None 2024-04-16T18:12:56.365 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/g 2024-04-16T18:12:56.365 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BWYdqoQIDsDCaGeQ+5it+PnXMZk= 2024-04-16T18:12:56.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BWYdqoQIDsDCaGeQ+5it+PnXMZk='} 2024-04-16T18:12:56.411 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f2b9a35ba844d60e-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.411 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:12:56.411 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:12:56.411 DEBUG:boto:Method: HEAD 2024-04-16T18:12:56.411 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:12:56.411 DEBUG:boto:Data: 2024-04-16T18:12:56.411 DEBUG:boto:Headers: {} 2024-04-16T18:12:56.411 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.412 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.412 DEBUG:boto:Params: {} 2024-04-16T18:12:56.412 DEBUG:boto:Token: None 2024-04-16T18:12:56.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/ 2024-04-16T18:12:56.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0= 2024-04-16T18:12:56.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AxlHryRgxemqhnx39nFcadPaPB0=', 'Content-Length': '0'} 2024-04-16T18:12:56.415 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eb596fbf4c0ccdc0-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.415 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:12:56.415 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:12:56.415 DEBUG:boto:Method: PUT 2024-04-16T18:12:56.415 DEBUG:boto:Path: /lybtug-48/h 2024-04-16T18:12:56.415 DEBUG:boto:Data: 2024-04-16T18:12:56.416 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:12:56.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:12:56.416 DEBUG:boto:Port: 8000 2024-04-16T18:12:56.416 DEBUG:boto:Params: {} 2024-04-16T18:12:56.416 DEBUG:boto:Token: None 2024-04-16T18:12:56.416 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:12:56 GMT /lybtug-48/h 2024-04-16T18:12:56.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C6itHYMrr91GAFSJUtvNNGSlkw4= 2024-04-16T18:12:56.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:12:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C6itHYMrr91GAFSJUtvNNGSlkw4='} 2024-04-16T18:12:56.425 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004a5a921156ee806c-00661ebfa8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:12:56.425 INFO:tasks.rgw.c2.client.0:Restarting daemon 2024-04-16T18:12:56.425 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T18:12:56.428 INFO:tasks.rgw.c2.client.0:Started 2024-04-16T18:12:56.428 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:56.440 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T18:12:56.441 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:12:56.441 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:12:56.441 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T18:12:56.441 INFO:teuthology.orchestra.run.smithi122.stderr:curl: (7) Failed to connect to smithi122.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T18:12:56.702 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:12:56.707+0000 7fbe7edb2a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T18:12:57.442 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:12:57.454 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:12:57.454 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:12:57.455 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 121k 0 --:--:-- --:--:-- --:--:-- 182k 2024-04-16T18:12:57.456 INFO:teuthology.orchestra.run.smithi122.stdout:anonymous 2024-04-16T18:12:57.456 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-48 2024-04-16T18:12:57.456 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-48', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:12:57.457 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-48', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:12:57.457 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-48 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:12:57.551 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:12:57.551 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:12:57.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:12:57.631+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:12:58.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2024-04-16T18:13:02.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:02.635+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:07.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:07.634+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:12.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:12.634+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:17.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:17.634+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:22.638+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:27.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:27.638+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:32.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:32.638+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:37.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:37.638+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:42.642+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:47.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:47.641+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:52.641+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:13:57.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:13:57.641+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:02.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:02.645+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:07.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:07.645+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:12.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:12.649+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:17.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:17.649+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:22.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:22.653+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:27.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:27.652+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:32.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:32.652+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:37.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:37.656+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:42.656+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:47.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:47.656+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:52.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:52.660+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:14:57.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:14:57.660+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:02.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:02.660+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:07.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:07.664+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:12.663+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:17.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:17.663+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:22.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:22.667+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:27.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:27.667+0000 7f32d3e6c980 1 waiting to reach incremental sync.. 2024-04-16T18:15:32.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:32.671+0000 7f32d3e6c980 1 bucket sync caught up with source: 2024-04-16T18:15:32.668 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000003.26.6, 00000000002.75.6, 00000000002.39.6] 2024-04-16T18:15:32.668 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000003.26.6, 00000000002.75.6, 00000000002.39.6] 2024-04-16T18:15:32.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:32.671+0000 7f32d3e6c980 0 bucket checkpoint complete 2024-04-16T18:15:32.712 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-48 2024-04-16T18:15:32.712 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-48', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:15:32.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-48', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:15:32.713 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-48 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:15:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:32.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:15:32.846+0000 7f80cc92f980 1 bucket sync caught up with source: 2024-04-16T18:15:32.844 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.897.6, , , 00000000001.724.6, 00000000001.720.6, , , 00000000002.1156.6, , 00000000002.1164.6] 2024-04-16T18:15:32.844 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.897.6, , , 00000000001.724.6, 00000000001.720.6, , , 00000000002.1156.6, , 00000000002.1164.6] 2024-04-16T18:15:32.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:15:32.846+0000 7f80cc92f980 0 bucket checkpoint complete 2024-04-16T18:15:32.874 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-48 zones={test-zone2, test-zone1} 2024-04-16T18:15:32.874 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:32.874 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:32.874 DEBUG:boto:Method: HEAD 2024-04-16T18:15:32.875 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:15:32.875 DEBUG:boto:Data: 2024-04-16T18:15:32.875 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.875 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:32.875 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.875 DEBUG:boto:Params: {} 2024-04-16T18:15:32.875 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:15:32.876 DEBUG:boto:Token: None 2024-04-16T18:15:32.876 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/ 2024-04-16T18:15:32.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GGA2XXl8SM7Icjgob8r1yiUg6C8= 2024-04-16T18:15:32.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GGA2XXl8SM7Icjgob8r1yiUg6C8=', 'Content-Length': '0'} 2024-04-16T18:15:32.887 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000274ecfc3311982fb-00661ec044-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.887 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:32.887 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:32.887 DEBUG:boto:Method: HEAD 2024-04-16T18:15:32.887 DEBUG:boto:Path: /lybtug-48/ 2024-04-16T18:15:32.887 DEBUG:boto:Data: 2024-04-16T18:15:32.887 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:32.888 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.888 DEBUG:boto:Params: {} 2024-04-16T18:15:32.888 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:15:32.888 DEBUG:boto:Token: None 2024-04-16T18:15:32.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/ 2024-04-16T18:15:32.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GGA2XXl8SM7Icjgob8r1yiUg6C8= 2024-04-16T18:15:32.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GGA2XXl8SM7Icjgob8r1yiUg6C8=', 'Content-Length': '0'} 2024-04-16T18:15:32.892 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003a58512e8f130215-00661ec044-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.893 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:15:32.893 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:32.893 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:32.893 DEBUG:boto:path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.893 DEBUG:boto:auth_path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.893 DEBUG:boto:Method: GET 2024-04-16T18:15:32.893 DEBUG:boto:Path: /lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.893 DEBUG:boto:Data: 2024-04-16T18:15:32.893 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.893 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:32.894 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.894 DEBUG:boto:Params: {} 2024-04-16T18:15:32.894 DEBUG:boto:Token: None 2024-04-16T18:15:32.894 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/?versions 2024-04-16T18:15:32.894 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk= 2024-04-16T18:15:32.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk=', 'Content-Length': '0'} 2024-04-16T18:15:32.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002185913509eabe00-00661ec044-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.898 DEBUG:boto:b'lybtug-48999falseanulltrue2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:32.899 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:15:32.900 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:15:32.900 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:32.900 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:32.900 DEBUG:boto:path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.900 DEBUG:boto:auth_path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.900 DEBUG:boto:Method: GET 2024-04-16T18:15:32.901 DEBUG:boto:Path: /lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.901 DEBUG:boto:Data: 2024-04-16T18:15:32.901 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.901 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:32.901 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.901 DEBUG:boto:Params: {} 2024-04-16T18:15:32.901 DEBUG:boto:Token: None 2024-04-16T18:15:32.901 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/?versions 2024-04-16T18:15:32.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk= 2024-04-16T18:15:32.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk=', 'Content-Length': '0'} 2024-04-16T18:15:32.904 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032a4c135727ef135-00661ec044-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.904 DEBUG:boto:b'lybtug-48999falseanulltrue2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:32.904 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:15:32.905 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:15:32.905 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:32.905 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:32.905 DEBUG:boto:path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.905 DEBUG:boto:auth_path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.905 DEBUG:boto:Method: GET 2024-04-16T18:15:32.906 DEBUG:boto:Path: /lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.906 DEBUG:boto:Data: 2024-04-16T18:15:32.906 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.906 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:32.906 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.906 DEBUG:boto:Params: {} 2024-04-16T18:15:32.906 DEBUG:boto:Token: None 2024-04-16T18:15:32.906 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/?versions 2024-04-16T18:15:32.906 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk= 2024-04-16T18:15:32.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk=', 'Content-Length': '0'} 2024-04-16T18:15:32.911 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e7b328ffa7e0731-00661ec044-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.911 DEBUG:boto:b'lybtug-48999falseanulltrue2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:32.911 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:32.912 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:32.912 DEBUG:boto:path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.912 DEBUG:boto:auth_path=/lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.912 DEBUG:boto:Method: GET 2024-04-16T18:15:32.912 DEBUG:boto:Path: /lybtug-48/?versions&max-keys=999 2024-04-16T18:15:32.912 DEBUG:boto:Data: 2024-04-16T18:15:32.912 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.912 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:32.912 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.912 DEBUG:boto:Params: {} 2024-04-16T18:15:32.912 DEBUG:boto:Token: None 2024-04-16T18:15:32.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/?versions 2024-04-16T18:15:32.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk= 2024-04-16T18:15:32.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gnvs94XqPqgUpwQeSqdUzVn9cSk=', 'Content-Length': '0'} 2024-04-16T18:15:32.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d07c5fabe8fbf84-00661ec044-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.915 DEBUG:boto:b'lybtug-48999falseanulltrue2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:32.916 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:15:32.916 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:32.916 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:32.916 DEBUG:boto:path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.916 DEBUG:boto:auth_path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.916 DEBUG:boto:Method: GET 2024-04-16T18:15:32.916 DEBUG:boto:Path: /lybtug-48/a?versionId=null 2024-04-16T18:15:32.916 DEBUG:boto:Data: 2024-04-16T18:15:32.917 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.917 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:32.917 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.917 DEBUG:boto:Params: {} 2024-04-16T18:15:32.917 DEBUG:boto:Token: None 2024-04-16T18:15:32.917 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/a?versionId=null 2024-04-16T18:15:32.917 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hxX/1W/Hy6xyfhpEfIjupnmJkyI= 2024-04-16T18:15:32.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hxX/1W/Hy6xyfhpEfIjupnmJkyI=', 'Content-Length': '0'} 2024-04-16T18:15:32.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c3ce297318e1971-00661ec044-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.962 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:32.962 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:32.963 DEBUG:boto:path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.963 DEBUG:boto:auth_path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.963 DEBUG:boto:Method: GET 2024-04-16T18:15:32.963 DEBUG:boto:Path: /lybtug-48/a?versionId=null 2024-04-16T18:15:32.963 DEBUG:boto:Data: 2024-04-16T18:15:32.963 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:32.963 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.963 DEBUG:boto:Params: {} 2024-04-16T18:15:32.963 DEBUG:boto:Token: None 2024-04-16T18:15:32.963 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/a?versionId=null 2024-04-16T18:15:32.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hxX/1W/Hy6xyfhpEfIjupnmJkyI= 2024-04-16T18:15:32.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hxX/1W/Hy6xyfhpEfIjupnmJkyI=', 'Content-Length': '0'} 2024-04-16T18:15:32.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020cc402bffe1b5a8-00661ec044-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.967 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:32.967 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:32.967 DEBUG:boto:path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.967 DEBUG:boto:auth_path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.967 DEBUG:boto:Method: HEAD 2024-04-16T18:15:32.967 DEBUG:boto:Path: /lybtug-48/a?versionId=null 2024-04-16T18:15:32.967 DEBUG:boto:Data: 2024-04-16T18:15:32.967 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:32.967 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.967 DEBUG:boto:Params: {} 2024-04-16T18:15:32.968 DEBUG:boto:Token: None 2024-04-16T18:15:32.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/a?versionId=null 2024-04-16T18:15:32.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:shxKRPwHfOKRfSR/enS6tkpW5zo= 2024-04-16T18:15:32.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shxKRPwHfOKRfSR/enS6tkpW5zo=', 'Content-Length': '0'} 2024-04-16T18:15:32.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f11d210c1a587aec-00661ec044-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.971 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:32.971 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:32.971 DEBUG:boto:path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.971 DEBUG:boto:auth_path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.971 DEBUG:boto:Method: HEAD 2024-04-16T18:15:32.971 DEBUG:boto:Path: /lybtug-48/a?versionId=null 2024-04-16T18:15:32.971 DEBUG:boto:Data: 2024-04-16T18:15:32.971 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:32.971 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.971 DEBUG:boto:Params: {} 2024-04-16T18:15:32.972 DEBUG:boto:Token: None 2024-04-16T18:15:32.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/a?versionId=null 2024-04-16T18:15:32.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:shxKRPwHfOKRfSR/enS6tkpW5zo= 2024-04-16T18:15:32.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shxKRPwHfOKRfSR/enS6tkpW5zo=', 'Content-Length': '0'} 2024-04-16T18:15:32.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c591ad3ea7cee11-00661ec044-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:32.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:15:32.975 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:32.975 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:32.975 DEBUG:boto:path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.975 DEBUG:boto:auth_path=/lybtug-48/a?versionId=null 2024-04-16T18:15:32.975 DEBUG:boto:Method: GET 2024-04-16T18:15:32.975 DEBUG:boto:Path: /lybtug-48/a?versionId=null 2024-04-16T18:15:32.975 DEBUG:boto:Data: 2024-04-16T18:15:32.975 DEBUG:boto:Headers: {} 2024-04-16T18:15:32.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:32.976 DEBUG:boto:Port: 8000 2024-04-16T18:15:32.976 DEBUG:boto:Params: {} 2024-04-16T18:15:32.976 DEBUG:boto:Token: None 2024-04-16T18:15:32.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:32 GMT /lybtug-48/a?versionId=null 2024-04-16T18:15:32.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hxX/1W/Hy6xyfhpEfIjupnmJkyI= 2024-04-16T18:15:32.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:32 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hxX/1W/Hy6xyfhpEfIjupnmJkyI=', 'Content-Length': '0'} 2024-04-16T18:15:32.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001952bdaf28fbfedb-00661ec044-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:32 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.022 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:33.023 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:33.023 DEBUG:boto:path=/lybtug-48/a?versionId=null 2024-04-16T18:15:33.023 DEBUG:boto:auth_path=/lybtug-48/a?versionId=null 2024-04-16T18:15:33.023 DEBUG:boto:Method: GET 2024-04-16T18:15:33.023 DEBUG:boto:Path: /lybtug-48/a?versionId=null 2024-04-16T18:15:33.023 DEBUG:boto:Data: 2024-04-16T18:15:33.023 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.023 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.023 DEBUG:boto:Params: {} 2024-04-16T18:15:33.023 DEBUG:boto:Token: None 2024-04-16T18:15:33.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/a?versionId=null 2024-04-16T18:15:33.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jWpL396T0AsiPDgfc3sL7UU8hwQ= 2024-04-16T18:15:33.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jWpL396T0AsiPDgfc3sL7UU8hwQ=', 'Content-Length': '0'} 2024-04-16T18:15:33.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000728da23d16f7da60-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.027 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.027 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.027 DEBUG:boto:path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.027 DEBUG:boto:auth_path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.027 DEBUG:boto:Method: GET 2024-04-16T18:15:33.027 DEBUG:boto:Path: /lybtug-48/?prefix=a 2024-04-16T18:15:33.027 DEBUG:boto:Data: 2024-04-16T18:15:33.028 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.028 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.028 DEBUG:boto:Params: {} 2024-04-16T18:15:33.028 DEBUG:boto:Token: None 2024-04-16T18:15:33.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044cc3279e0dc10c8-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.074 DEBUG:boto:b'lybtug-48a1000falsea2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.075 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.075 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.075 DEBUG:boto:path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.075 DEBUG:boto:auth_path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.075 DEBUG:boto:Method: GET 2024-04-16T18:15:33.075 DEBUG:boto:Path: /lybtug-48/?prefix=a 2024-04-16T18:15:33.075 DEBUG:boto:Data: 2024-04-16T18:15:33.075 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.075 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.075 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.076 DEBUG:boto:Params: {} 2024-04-16T18:15:33.076 DEBUG:boto:Token: None 2024-04-16T18:15:33.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000863bebf47712a768-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.079 DEBUG:boto:b'lybtug-48a1000falsea2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.079 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:33.080 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:33.080 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.080 DEBUG:boto:Path: /lybtug-48/a 2024-04-16T18:15:33.080 DEBUG:boto:Data: 2024-04-16T18:15:33.080 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.080 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.080 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.080 DEBUG:boto:Params: {} 2024-04-16T18:15:33.080 DEBUG:boto:Token: None 2024-04-16T18:15:33.080 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/a 2024-04-16T18:15:33.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bLKy7MDz4hG7da6dmabNM+kXJwg= 2024-04-16T18:15:33.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bLKy7MDz4hG7da6dmabNM+kXJwg=', 'Content-Length': '0'} 2024-04-16T18:15:33.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d05c2eeee6ad444c-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.084 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:33.084 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:33.084 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.084 DEBUG:boto:Path: /lybtug-48/a 2024-04-16T18:15:33.084 DEBUG:boto:Data: 2024-04-16T18:15:33.084 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.084 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.084 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.084 DEBUG:boto:Params: {} 2024-04-16T18:15:33.085 DEBUG:boto:Token: None 2024-04-16T18:15:33.085 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/a 2024-04-16T18:15:33.085 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bLKy7MDz4hG7da6dmabNM+kXJwg= 2024-04-16T18:15:33.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bLKy7MDz4hG7da6dmabNM+kXJwg=', 'Content-Length': '0'} 2024-04-16T18:15:33.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c84e206f6131ccae-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.088 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:15:33.088 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:33.088 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:33.088 DEBUG:boto:Method: GET 2024-04-16T18:15:33.088 DEBUG:boto:Path: /lybtug-48/a 2024-04-16T18:15:33.088 DEBUG:boto:Data: 2024-04-16T18:15:33.088 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.088 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.088 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.089 DEBUG:boto:Params: {} 2024-04-16T18:15:33.089 DEBUG:boto:Token: None 2024-04-16T18:15:33.089 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/a 2024-04-16T18:15:33.089 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EONTyUlGidWAjNyj8pj12CrUGGg= 2024-04-16T18:15:33.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EONTyUlGidWAjNyj8pj12CrUGGg=', 'Content-Length': '0'} 2024-04-16T18:15:33.092 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021ff073c546c7182-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.134 DEBUG:boto:path=/lybtug-48/a 2024-04-16T18:15:33.134 DEBUG:boto:auth_path=/lybtug-48/a 2024-04-16T18:15:33.135 DEBUG:boto:Method: GET 2024-04-16T18:15:33.135 DEBUG:boto:Path: /lybtug-48/a 2024-04-16T18:15:33.135 DEBUG:boto:Data: 2024-04-16T18:15:33.135 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.135 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.135 DEBUG:boto:Params: {} 2024-04-16T18:15:33.135 DEBUG:boto:Token: None 2024-04-16T18:15:33.135 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/a 2024-04-16T18:15:33.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EONTyUlGidWAjNyj8pj12CrUGGg= 2024-04-16T18:15:33.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EONTyUlGidWAjNyj8pj12CrUGGg=', 'Content-Length': '0'} 2024-04-16T18:15:33.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fc1e186e473a83f-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.138 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.139 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.139 DEBUG:boto:path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.139 DEBUG:boto:auth_path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.139 DEBUG:boto:Method: GET 2024-04-16T18:15:33.139 DEBUG:boto:Path: /lybtug-48/?prefix=a 2024-04-16T18:15:33.139 DEBUG:boto:Data: 2024-04-16T18:15:33.139 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.139 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.139 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.139 DEBUG:boto:Params: {} 2024-04-16T18:15:33.140 DEBUG:boto:Token: None 2024-04-16T18:15:33.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009865d92f2b967e18-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.186 DEBUG:boto:b'lybtug-48a1000falsea2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.187 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.187 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.187 DEBUG:boto:path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.187 DEBUG:boto:auth_path=/lybtug-48/?prefix=a 2024-04-16T18:15:33.187 DEBUG:boto:Method: GET 2024-04-16T18:15:33.187 DEBUG:boto:Path: /lybtug-48/?prefix=a 2024-04-16T18:15:33.187 DEBUG:boto:Data: 2024-04-16T18:15:33.187 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.187 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.187 DEBUG:boto:Params: {} 2024-04-16T18:15:33.188 DEBUG:boto:Token: None 2024-04-16T18:15:33.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.191 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f062eb811b1edeff-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.191 DEBUG:boto:b'lybtug-48a1000falsea2024-04-16T18:12:55.966Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.191 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:15:33.192 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.192 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.192 DEBUG:boto:path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.192 DEBUG:boto:auth_path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.192 DEBUG:boto:Method: GET 2024-04-16T18:15:33.192 DEBUG:boto:Path: /lybtug-48/b?versionId=null 2024-04-16T18:15:33.192 DEBUG:boto:Data: 2024-04-16T18:15:33.192 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.192 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.192 DEBUG:boto:Params: {} 2024-04-16T18:15:33.192 DEBUG:boto:Token: None 2024-04-16T18:15:33.193 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b?versionId=null 2024-04-16T18:15:33.193 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w= 2024-04-16T18:15:33.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w=', 'Content-Length': '0'} 2024-04-16T18:15:33.196 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:15:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4d17103716342c4-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.238 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.238 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.238 DEBUG:boto:path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.239 DEBUG:boto:auth_path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.239 DEBUG:boto:Method: GET 2024-04-16T18:15:33.239 DEBUG:boto:Path: /lybtug-48/b?versionId=null 2024-04-16T18:15:33.239 DEBUG:boto:Data: 2024-04-16T18:15:33.239 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.239 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.239 DEBUG:boto:Params: {} 2024-04-16T18:15:33.239 DEBUG:boto:Token: None 2024-04-16T18:15:33.239 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b?versionId=null 2024-04-16T18:15:33.239 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w= 2024-04-16T18:15:33.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w=', 'Content-Length': '0'} 2024-04-16T18:15:33.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd578b9687c03300-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.242 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.243 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.243 DEBUG:boto:path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.243 DEBUG:boto:auth_path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.243 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.243 DEBUG:boto:Path: /lybtug-48/b?versionId=null 2024-04-16T18:15:33.243 DEBUG:boto:Data: 2024-04-16T18:15:33.243 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.243 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.243 DEBUG:boto:Params: {} 2024-04-16T18:15:33.243 DEBUG:boto:Token: None 2024-04-16T18:15:33.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b?versionId=null 2024-04-16T18:15:33.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EJewATr2ulKbYGNJpsP1dlc3Y4k= 2024-04-16T18:15:33.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EJewATr2ulKbYGNJpsP1dlc3Y4k=', 'Content-Length': '0'} 2024-04-16T18:15:33.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:15:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4e4c143b90924ee-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.247 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.247 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.247 DEBUG:boto:path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.247 DEBUG:boto:auth_path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.247 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.247 DEBUG:boto:Path: /lybtug-48/b?versionId=null 2024-04-16T18:15:33.247 DEBUG:boto:Data: 2024-04-16T18:15:33.247 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.247 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.247 DEBUG:boto:Params: {} 2024-04-16T18:15:33.248 DEBUG:boto:Token: None 2024-04-16T18:15:33.248 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b?versionId=null 2024-04-16T18:15:33.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EJewATr2ulKbYGNJpsP1dlc3Y4k= 2024-04-16T18:15:33.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EJewATr2ulKbYGNJpsP1dlc3Y4k=', 'Content-Length': '0'} 2024-04-16T18:15:33.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b889aff0ccd7f924-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:15:33.251 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.251 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.251 DEBUG:boto:path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.251 DEBUG:boto:auth_path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.251 DEBUG:boto:Method: GET 2024-04-16T18:15:33.251 DEBUG:boto:Path: /lybtug-48/b?versionId=null 2024-04-16T18:15:33.252 DEBUG:boto:Data: 2024-04-16T18:15:33.252 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.252 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.252 DEBUG:boto:Params: {} 2024-04-16T18:15:33.252 DEBUG:boto:Token: None 2024-04-16T18:15:33.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b?versionId=null 2024-04-16T18:15:33.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w= 2024-04-16T18:15:33.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w=', 'Content-Length': '0'} 2024-04-16T18:15:33.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:15:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f23be272aa33d733-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.298 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.298 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.298 DEBUG:boto:path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.299 DEBUG:boto:auth_path=/lybtug-48/b?versionId=null 2024-04-16T18:15:33.299 DEBUG:boto:Method: GET 2024-04-16T18:15:33.299 DEBUG:boto:Path: /lybtug-48/b?versionId=null 2024-04-16T18:15:33.299 DEBUG:boto:Data: 2024-04-16T18:15:33.299 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.299 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.299 DEBUG:boto:Params: {} 2024-04-16T18:15:33.299 DEBUG:boto:Token: None 2024-04-16T18:15:33.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b?versionId=null 2024-04-16T18:15:33.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w= 2024-04-16T18:15:33.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o/Vf6sPb8k1AROVQs8CEkzNRk0w=', 'Content-Length': '0'} 2024-04-16T18:15:33.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021a2f2ca1b583230-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.302 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.303 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.303 DEBUG:boto:path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.303 DEBUG:boto:auth_path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.303 DEBUG:boto:Method: GET 2024-04-16T18:15:33.303 DEBUG:boto:Path: /lybtug-48/?prefix=b 2024-04-16T18:15:33.303 DEBUG:boto:Data: 2024-04-16T18:15:33.303 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.303 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.303 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.303 DEBUG:boto:Params: {} 2024-04-16T18:15:33.303 DEBUG:boto:Token: None 2024-04-16T18:15:33.303 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000057e22eba254e1db-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.307 DEBUG:boto:b'lybtug-48b1000falseb2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.308 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.308 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.308 DEBUG:boto:path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.308 DEBUG:boto:auth_path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.308 DEBUG:boto:Method: GET 2024-04-16T18:15:33.308 DEBUG:boto:Path: /lybtug-48/?prefix=b 2024-04-16T18:15:33.308 DEBUG:boto:Data: 2024-04-16T18:15:33.308 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.308 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.309 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.309 DEBUG:boto:Params: {} 2024-04-16T18:15:33.309 DEBUG:boto:Token: None 2024-04-16T18:15:33.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004769f5f05d7c3738-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.312 DEBUG:boto:b'lybtug-48b1000falseb2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.312 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.312 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.312 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.312 DEBUG:boto:Path: /lybtug-48/b 2024-04-16T18:15:33.312 DEBUG:boto:Data: 2024-04-16T18:15:33.313 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.313 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.313 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.313 DEBUG:boto:Params: {} 2024-04-16T18:15:33.313 DEBUG:boto:Token: None 2024-04-16T18:15:33.313 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b 2024-04-16T18:15:33.313 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e/w2NiE+ZTm+/1fv6LvdpCHQuhA= 2024-04-16T18:15:33.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e/w2NiE+ZTm+/1fv6LvdpCHQuhA=', 'Content-Length': '0'} 2024-04-16T18:15:33.316 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:15:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013aa115fb60d7b92-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.316 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.316 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.316 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.316 DEBUG:boto:Path: /lybtug-48/b 2024-04-16T18:15:33.317 DEBUG:boto:Data: 2024-04-16T18:15:33.317 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.317 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.317 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.317 DEBUG:boto:Params: {} 2024-04-16T18:15:33.317 DEBUG:boto:Token: None 2024-04-16T18:15:33.317 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b 2024-04-16T18:15:33.317 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e/w2NiE+ZTm+/1fv6LvdpCHQuhA= 2024-04-16T18:15:33.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e/w2NiE+ZTm+/1fv6LvdpCHQuhA=', 'Content-Length': '0'} 2024-04-16T18:15:33.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f6e829882c8283f-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.320 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:15:33.320 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.320 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.320 DEBUG:boto:Method: GET 2024-04-16T18:15:33.320 DEBUG:boto:Path: /lybtug-48/b 2024-04-16T18:15:33.320 DEBUG:boto:Data: 2024-04-16T18:15:33.321 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.321 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.321 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.321 DEBUG:boto:Params: {} 2024-04-16T18:15:33.321 DEBUG:boto:Token: None 2024-04-16T18:15:33.321 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b 2024-04-16T18:15:33.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:orp86apuhNEeDxPlcRZSgwiCPiI= 2024-04-16T18:15:33.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:orp86apuhNEeDxPlcRZSgwiCPiI=', 'Content-Length': '0'} 2024-04-16T18:15:33.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:15:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6d6b208572ceac1-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.366 DEBUG:boto:path=/lybtug-48/b 2024-04-16T18:15:33.366 DEBUG:boto:auth_path=/lybtug-48/b 2024-04-16T18:15:33.367 DEBUG:boto:Method: GET 2024-04-16T18:15:33.367 DEBUG:boto:Path: /lybtug-48/b 2024-04-16T18:15:33.367 DEBUG:boto:Data: 2024-04-16T18:15:33.367 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.367 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.367 DEBUG:boto:Params: {} 2024-04-16T18:15:33.367 DEBUG:boto:Token: None 2024-04-16T18:15:33.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/b 2024-04-16T18:15:33.367 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:orp86apuhNEeDxPlcRZSgwiCPiI= 2024-04-16T18:15:33.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:orp86apuhNEeDxPlcRZSgwiCPiI=', 'Content-Length': '0'} 2024-04-16T18:15:33.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:55 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000387b371be0d1ff53-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.371 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.371 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.371 DEBUG:boto:path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.371 DEBUG:boto:auth_path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.371 DEBUG:boto:Method: GET 2024-04-16T18:15:33.371 DEBUG:boto:Path: /lybtug-48/?prefix=b 2024-04-16T18:15:33.371 DEBUG:boto:Data: 2024-04-16T18:15:33.372 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.372 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.372 DEBUG:boto:Params: {} 2024-04-16T18:15:33.372 DEBUG:boto:Token: None 2024-04-16T18:15:33.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb8a427d760144ed-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.376 DEBUG:boto:b'lybtug-48b1000falseb2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.377 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.377 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.377 DEBUG:boto:path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.377 DEBUG:boto:auth_path=/lybtug-48/?prefix=b 2024-04-16T18:15:33.377 DEBUG:boto:Method: GET 2024-04-16T18:15:33.377 DEBUG:boto:Path: /lybtug-48/?prefix=b 2024-04-16T18:15:33.377 DEBUG:boto:Data: 2024-04-16T18:15:33.377 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.377 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.377 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.377 DEBUG:boto:Params: {} 2024-04-16T18:15:33.378 DEBUG:boto:Token: None 2024-04-16T18:15:33.378 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.378 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.381 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002198839fc171505c-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.381 DEBUG:boto:b'lybtug-48b1000falseb2024-04-16T18:12:55.988Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.381 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:15:33.382 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.382 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.382 DEBUG:boto:path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.382 DEBUG:boto:auth_path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.382 DEBUG:boto:Method: GET 2024-04-16T18:15:33.382 DEBUG:boto:Path: /lybtug-48/c?versionId=null 2024-04-16T18:15:33.382 DEBUG:boto:Data: 2024-04-16T18:15:33.382 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.382 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.382 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.382 DEBUG:boto:Params: {} 2024-04-16T18:15:33.383 DEBUG:boto:Token: None 2024-04-16T18:15:33.383 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c?versionId=null 2024-04-16T18:15:33.383 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik= 2024-04-16T18:15:33.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik=', 'Content-Length': '0'} 2024-04-16T18:15:33.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f713fcf3a63bca1b-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.426 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.427 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.427 DEBUG:boto:path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.427 DEBUG:boto:auth_path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.427 DEBUG:boto:Method: GET 2024-04-16T18:15:33.427 DEBUG:boto:Path: /lybtug-48/c?versionId=null 2024-04-16T18:15:33.427 DEBUG:boto:Data: 2024-04-16T18:15:33.427 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.427 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.427 DEBUG:boto:Params: {} 2024-04-16T18:15:33.428 DEBUG:boto:Token: None 2024-04-16T18:15:33.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c?versionId=null 2024-04-16T18:15:33.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik= 2024-04-16T18:15:33.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik=', 'Content-Length': '0'} 2024-04-16T18:15:33.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000213c6a2485c80170-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.432 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.432 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.432 DEBUG:boto:path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.432 DEBUG:boto:auth_path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.432 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.432 DEBUG:boto:Path: /lybtug-48/c?versionId=null 2024-04-16T18:15:33.432 DEBUG:boto:Data: 2024-04-16T18:15:33.432 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.432 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.432 DEBUG:boto:Params: {} 2024-04-16T18:15:33.433 DEBUG:boto:Token: None 2024-04-16T18:15:33.433 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c?versionId=null 2024-04-16T18:15:33.433 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ApqmVMHFyg721il3UxHX1tXNEVE= 2024-04-16T18:15:33.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ApqmVMHFyg721il3UxHX1tXNEVE=', 'Content-Length': '0'} 2024-04-16T18:15:33.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000367e8e7d037fe253-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.436 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.436 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.436 DEBUG:boto:path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.436 DEBUG:boto:auth_path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.436 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.436 DEBUG:boto:Path: /lybtug-48/c?versionId=null 2024-04-16T18:15:33.436 DEBUG:boto:Data: 2024-04-16T18:15:33.436 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.437 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.437 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.437 DEBUG:boto:Params: {} 2024-04-16T18:15:33.437 DEBUG:boto:Token: None 2024-04-16T18:15:33.437 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c?versionId=null 2024-04-16T18:15:33.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ApqmVMHFyg721il3UxHX1tXNEVE= 2024-04-16T18:15:33.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ApqmVMHFyg721il3UxHX1tXNEVE=', 'Content-Length': '0'} 2024-04-16T18:15:33.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066a50c8ca12337a4-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.440 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:15:33.441 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.441 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.441 DEBUG:boto:path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.441 DEBUG:boto:auth_path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.441 DEBUG:boto:Method: GET 2024-04-16T18:15:33.441 DEBUG:boto:Path: /lybtug-48/c?versionId=null 2024-04-16T18:15:33.441 DEBUG:boto:Data: 2024-04-16T18:15:33.441 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.441 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.441 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.441 DEBUG:boto:Params: {} 2024-04-16T18:15:33.441 DEBUG:boto:Token: None 2024-04-16T18:15:33.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c?versionId=null 2024-04-16T18:15:33.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik= 2024-04-16T18:15:33.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik=', 'Content-Length': '0'} 2024-04-16T18:15:33.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005828d13d5ee0554d-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.486 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.486 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.486 DEBUG:boto:path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.487 DEBUG:boto:auth_path=/lybtug-48/c?versionId=null 2024-04-16T18:15:33.487 DEBUG:boto:Method: GET 2024-04-16T18:15:33.487 DEBUG:boto:Path: /lybtug-48/c?versionId=null 2024-04-16T18:15:33.487 DEBUG:boto:Data: 2024-04-16T18:15:33.487 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.487 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.487 DEBUG:boto:Params: {} 2024-04-16T18:15:33.487 DEBUG:boto:Token: None 2024-04-16T18:15:33.487 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c?versionId=null 2024-04-16T18:15:33.487 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik= 2024-04-16T18:15:33.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9FYNfNV1qG6t0jI7SpK6elb/+ik=', 'Content-Length': '0'} 2024-04-16T18:15:33.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019bec2110e7327f8-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.491 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.491 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.491 DEBUG:boto:path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.491 DEBUG:boto:auth_path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.491 DEBUG:boto:Method: GET 2024-04-16T18:15:33.492 DEBUG:boto:Path: /lybtug-48/?prefix=c 2024-04-16T18:15:33.492 DEBUG:boto:Data: 2024-04-16T18:15:33.492 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.492 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.492 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.492 DEBUG:boto:Params: {} 2024-04-16T18:15:33.492 DEBUG:boto:Token: None 2024-04-16T18:15:33.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.496 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082a53808cf70dd81-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.496 DEBUG:boto:b'lybtug-48c1000falsec2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.497 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.497 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.497 DEBUG:boto:path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.497 DEBUG:boto:auth_path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.497 DEBUG:boto:Method: GET 2024-04-16T18:15:33.497 DEBUG:boto:Path: /lybtug-48/?prefix=c 2024-04-16T18:15:33.497 DEBUG:boto:Data: 2024-04-16T18:15:33.497 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.497 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.497 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.497 DEBUG:boto:Params: {} 2024-04-16T18:15:33.498 DEBUG:boto:Token: None 2024-04-16T18:15:33.498 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.498 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.501 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000936fe707e1944316-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.542 DEBUG:boto:b'lybtug-48c1000falsec2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.543 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.543 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.543 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.543 DEBUG:boto:Path: /lybtug-48/c 2024-04-16T18:15:33.543 DEBUG:boto:Data: 2024-04-16T18:15:33.543 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.543 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.543 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.543 DEBUG:boto:Params: {} 2024-04-16T18:15:33.543 DEBUG:boto:Token: None 2024-04-16T18:15:33.544 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c 2024-04-16T18:15:33.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MddcI15sgfYqX4yw+IQvRRPeSe4= 2024-04-16T18:15:33.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MddcI15sgfYqX4yw+IQvRRPeSe4=', 'Content-Length': '0'} 2024-04-16T18:15:33.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099c83321e60b971c-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.547 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.547 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.547 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.547 DEBUG:boto:Path: /lybtug-48/c 2024-04-16T18:15:33.547 DEBUG:boto:Data: 2024-04-16T18:15:33.547 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.547 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.547 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.547 DEBUG:boto:Params: {} 2024-04-16T18:15:33.547 DEBUG:boto:Token: None 2024-04-16T18:15:33.547 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c 2024-04-16T18:15:33.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MddcI15sgfYqX4yw+IQvRRPeSe4= 2024-04-16T18:15:33.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MddcI15sgfYqX4yw+IQvRRPeSe4=', 'Content-Length': '0'} 2024-04-16T18:15:33.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf0ddb70d14f12d3-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.551 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:15:33.551 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.551 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.551 DEBUG:boto:Method: GET 2024-04-16T18:15:33.551 DEBUG:boto:Path: /lybtug-48/c 2024-04-16T18:15:33.551 DEBUG:boto:Data: 2024-04-16T18:15:33.551 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.552 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.552 DEBUG:boto:Params: {} 2024-04-16T18:15:33.552 DEBUG:boto:Token: None 2024-04-16T18:15:33.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c 2024-04-16T18:15:33.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bgwCQnLBO8MtsuC1tVbEX5LA+NU= 2024-04-16T18:15:33.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bgwCQnLBO8MtsuC1tVbEX5LA+NU=', 'Content-Length': '0'} 2024-04-16T18:15:33.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000256d9d49e92cbd6e-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.598 DEBUG:boto:path=/lybtug-48/c 2024-04-16T18:15:33.598 DEBUG:boto:auth_path=/lybtug-48/c 2024-04-16T18:15:33.599 DEBUG:boto:Method: GET 2024-04-16T18:15:33.599 DEBUG:boto:Path: /lybtug-48/c 2024-04-16T18:15:33.599 DEBUG:boto:Data: 2024-04-16T18:15:33.599 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.599 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.599 DEBUG:boto:Params: {} 2024-04-16T18:15:33.599 DEBUG:boto:Token: None 2024-04-16T18:15:33.599 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/c 2024-04-16T18:15:33.599 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bgwCQnLBO8MtsuC1tVbEX5LA+NU= 2024-04-16T18:15:33.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bgwCQnLBO8MtsuC1tVbEX5LA+NU=', 'Content-Length': '0'} 2024-04-16T18:15:33.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096520d35707c97b2-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.603 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.603 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.603 DEBUG:boto:path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.603 DEBUG:boto:auth_path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.603 DEBUG:boto:Method: GET 2024-04-16T18:15:33.603 DEBUG:boto:Path: /lybtug-48/?prefix=c 2024-04-16T18:15:33.603 DEBUG:boto:Data: 2024-04-16T18:15:33.603 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.603 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.604 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.604 DEBUG:boto:Params: {} 2024-04-16T18:15:33.604 DEBUG:boto:Token: None 2024-04-16T18:15:33.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cee207a377424475-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.650 DEBUG:boto:b'lybtug-48c1000falsec2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.651 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.651 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.651 DEBUG:boto:path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.651 DEBUG:boto:auth_path=/lybtug-48/?prefix=c 2024-04-16T18:15:33.651 DEBUG:boto:Method: GET 2024-04-16T18:15:33.651 DEBUG:boto:Path: /lybtug-48/?prefix=c 2024-04-16T18:15:33.651 DEBUG:boto:Data: 2024-04-16T18:15:33.651 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.651 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.652 DEBUG:boto:Params: {} 2024-04-16T18:15:33.652 DEBUG:boto:Token: None 2024-04-16T18:15:33.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.655 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050772668bb19ecd6-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.655 DEBUG:boto:b'lybtug-48c1000falsec2024-04-16T18:12:56.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.655 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:15:33.655 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.655 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.655 DEBUG:boto:path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.656 DEBUG:boto:auth_path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.656 DEBUG:boto:Method: GET 2024-04-16T18:15:33.656 DEBUG:boto:Path: /lybtug-48/d?versionId=null 2024-04-16T18:15:33.656 DEBUG:boto:Data: 2024-04-16T18:15:33.656 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.656 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.656 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.656 DEBUG:boto:Params: {} 2024-04-16T18:15:33.656 DEBUG:boto:Token: None 2024-04-16T18:15:33.656 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d?versionId=null 2024-04-16T18:15:33.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU= 2024-04-16T18:15:33.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU=', 'Content-Length': '0'} 2024-04-16T18:15:33.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048236935f9145c33-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.702 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.702 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.703 DEBUG:boto:path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.703 DEBUG:boto:auth_path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.703 DEBUG:boto:Method: GET 2024-04-16T18:15:33.703 DEBUG:boto:Path: /lybtug-48/d?versionId=null 2024-04-16T18:15:33.703 DEBUG:boto:Data: 2024-04-16T18:15:33.703 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.703 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.703 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.703 DEBUG:boto:Params: {} 2024-04-16T18:15:33.703 DEBUG:boto:Token: None 2024-04-16T18:15:33.703 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d?versionId=null 2024-04-16T18:15:33.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU= 2024-04-16T18:15:33.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU=', 'Content-Length': '0'} 2024-04-16T18:15:33.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030e345ed6e7b7498-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.707 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.707 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.707 DEBUG:boto:path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.707 DEBUG:boto:auth_path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.707 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.708 DEBUG:boto:Path: /lybtug-48/d?versionId=null 2024-04-16T18:15:33.708 DEBUG:boto:Data: 2024-04-16T18:15:33.708 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.708 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.708 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.708 DEBUG:boto:Params: {} 2024-04-16T18:15:33.708 DEBUG:boto:Token: None 2024-04-16T18:15:33.708 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d?versionId=null 2024-04-16T18:15:33.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GpTiEfE+WH58w38TKMvaAHtm7F8= 2024-04-16T18:15:33.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GpTiEfE+WH58w38TKMvaAHtm7F8=', 'Content-Length': '0'} 2024-04-16T18:15:33.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e4c8200c8a56e33-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.712 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.712 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.712 DEBUG:boto:path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.712 DEBUG:boto:auth_path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.712 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.712 DEBUG:boto:Path: /lybtug-48/d?versionId=null 2024-04-16T18:15:33.712 DEBUG:boto:Data: 2024-04-16T18:15:33.712 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.712 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.712 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.712 DEBUG:boto:Params: {} 2024-04-16T18:15:33.713 DEBUG:boto:Token: None 2024-04-16T18:15:33.713 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d?versionId=null 2024-04-16T18:15:33.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GpTiEfE+WH58w38TKMvaAHtm7F8= 2024-04-16T18:15:33.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GpTiEfE+WH58w38TKMvaAHtm7F8=', 'Content-Length': '0'} 2024-04-16T18:15:33.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b5bb8cd5815f835-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:15:33.716 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.716 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.717 DEBUG:boto:path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.717 DEBUG:boto:auth_path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.717 DEBUG:boto:Method: GET 2024-04-16T18:15:33.717 DEBUG:boto:Path: /lybtug-48/d?versionId=null 2024-04-16T18:15:33.717 DEBUG:boto:Data: 2024-04-16T18:15:33.717 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.717 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.717 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.717 DEBUG:boto:Params: {} 2024-04-16T18:15:33.717 DEBUG:boto:Token: None 2024-04-16T18:15:33.717 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d?versionId=null 2024-04-16T18:15:33.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU= 2024-04-16T18:15:33.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU=', 'Content-Length': '0'} 2024-04-16T18:15:33.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000018c5fd8c79fc151-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.762 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.762 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.763 DEBUG:boto:path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.763 DEBUG:boto:auth_path=/lybtug-48/d?versionId=null 2024-04-16T18:15:33.763 DEBUG:boto:Method: GET 2024-04-16T18:15:33.763 DEBUG:boto:Path: /lybtug-48/d?versionId=null 2024-04-16T18:15:33.763 DEBUG:boto:Data: 2024-04-16T18:15:33.763 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.763 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.763 DEBUG:boto:Params: {} 2024-04-16T18:15:33.763 DEBUG:boto:Token: None 2024-04-16T18:15:33.763 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d?versionId=null 2024-04-16T18:15:33.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU= 2024-04-16T18:15:33.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:elCWqKIAEwR3OSvdkj3gcyy0GWU=', 'Content-Length': '0'} 2024-04-16T18:15:33.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dc7f7910f7d752b-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.767 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.767 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.767 DEBUG:boto:path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.768 DEBUG:boto:auth_path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.768 DEBUG:boto:Method: GET 2024-04-16T18:15:33.768 DEBUG:boto:Path: /lybtug-48/?prefix=d 2024-04-16T18:15:33.768 DEBUG:boto:Data: 2024-04-16T18:15:33.768 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.768 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.768 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.768 DEBUG:boto:Params: {} 2024-04-16T18:15:33.768 DEBUG:boto:Token: None 2024-04-16T18:15:33.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e254336882b55f1-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.773 DEBUG:boto:b'lybtug-48d1000falsed2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.773 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.773 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.773 DEBUG:boto:path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.773 DEBUG:boto:auth_path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.773 DEBUG:boto:Method: GET 2024-04-16T18:15:33.773 DEBUG:boto:Path: /lybtug-48/?prefix=d 2024-04-16T18:15:33.773 DEBUG:boto:Data: 2024-04-16T18:15:33.773 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.774 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.774 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.774 DEBUG:boto:Params: {} 2024-04-16T18:15:33.774 DEBUG:boto:Token: None 2024-04-16T18:15:33.774 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.777 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee015a087d7a2639-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.818 DEBUG:boto:b'lybtug-48d1000falsed2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.819 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.819 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.819 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.819 DEBUG:boto:Path: /lybtug-48/d 2024-04-16T18:15:33.819 DEBUG:boto:Data: 2024-04-16T18:15:33.819 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.819 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.819 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.819 DEBUG:boto:Params: {} 2024-04-16T18:15:33.819 DEBUG:boto:Token: None 2024-04-16T18:15:33.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d 2024-04-16T18:15:33.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hV15DDTBEV99tb8qaTTS3sHG1j0= 2024-04-16T18:15:33.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hV15DDTBEV99tb8qaTTS3sHG1j0=', 'Content-Length': '0'} 2024-04-16T18:15:33.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7a1f54c155a293a-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.823 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.823 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.823 DEBUG:boto:Method: HEAD 2024-04-16T18:15:33.823 DEBUG:boto:Path: /lybtug-48/d 2024-04-16T18:15:33.823 DEBUG:boto:Data: 2024-04-16T18:15:33.823 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.824 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.824 DEBUG:boto:Params: {} 2024-04-16T18:15:33.824 DEBUG:boto:Token: None 2024-04-16T18:15:33.824 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d 2024-04-16T18:15:33.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hV15DDTBEV99tb8qaTTS3sHG1j0= 2024-04-16T18:15:33.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hV15DDTBEV99tb8qaTTS3sHG1j0=', 'Content-Length': '0'} 2024-04-16T18:15:33.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bcb89803033e566-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.827 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:15:33.828 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.828 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.828 DEBUG:boto:Method: GET 2024-04-16T18:15:33.828 DEBUG:boto:Path: /lybtug-48/d 2024-04-16T18:15:33.828 DEBUG:boto:Data: 2024-04-16T18:15:33.828 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.828 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.828 DEBUG:boto:Params: {} 2024-04-16T18:15:33.828 DEBUG:boto:Token: None 2024-04-16T18:15:33.829 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d 2024-04-16T18:15:33.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:72FAYJdaauqHsDZgMDsMU9rAVv8= 2024-04-16T18:15:33.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:72FAYJdaauqHsDZgMDsMU9rAVv8=', 'Content-Length': '0'} 2024-04-16T18:15:33.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e200d37a40784938-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.874 DEBUG:boto:path=/lybtug-48/d 2024-04-16T18:15:33.874 DEBUG:boto:auth_path=/lybtug-48/d 2024-04-16T18:15:33.875 DEBUG:boto:Method: GET 2024-04-16T18:15:33.875 DEBUG:boto:Path: /lybtug-48/d 2024-04-16T18:15:33.875 DEBUG:boto:Data: 2024-04-16T18:15:33.875 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.875 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.875 DEBUG:boto:Params: {} 2024-04-16T18:15:33.875 DEBUG:boto:Token: None 2024-04-16T18:15:33.875 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/d 2024-04-16T18:15:33.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:72FAYJdaauqHsDZgMDsMU9rAVv8= 2024-04-16T18:15:33.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:72FAYJdaauqHsDZgMDsMU9rAVv8=', 'Content-Length': '0'} 2024-04-16T18:15:33.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ac2440364f63314-00661ec045-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.878 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.879 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.879 DEBUG:boto:path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.879 DEBUG:boto:auth_path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.879 DEBUG:boto:Method: GET 2024-04-16T18:15:33.879 DEBUG:boto:Path: /lybtug-48/?prefix=d 2024-04-16T18:15:33.879 DEBUG:boto:Data: 2024-04-16T18:15:33.879 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.879 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.879 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.879 DEBUG:boto:Params: {} 2024-04-16T18:15:33.879 DEBUG:boto:Token: None 2024-04-16T18:15:33.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f21fd8dec93297f4-00661ec045-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.926 DEBUG:boto:b'lybtug-48d1000falsed2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.962 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:33.962 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:33.963 DEBUG:boto:path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.963 DEBUG:boto:auth_path=/lybtug-48/?prefix=d 2024-04-16T18:15:33.963 DEBUG:boto:Method: GET 2024-04-16T18:15:33.963 DEBUG:boto:Path: /lybtug-48/?prefix=d 2024-04-16T18:15:33.963 DEBUG:boto:Data: 2024-04-16T18:15:33.963 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:33.963 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.963 DEBUG:boto:Params: {} 2024-04-16T18:15:33.963 DEBUG:boto:Token: None 2024-04-16T18:15:33.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/ 2024-04-16T18:15:33.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8= 2024-04-16T18:15:33.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pTE8CeXkY+6smCWBFMcDNSs4an8=', 'Content-Length': '0'} 2024-04-16T18:15:33.967 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b39174ce56d43492-00661ec045-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:33.967 DEBUG:boto:b'lybtug-48d1000falsed2024-04-16T18:12:56.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:33.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:15:33.967 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:33.967 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:33.967 DEBUG:boto:path=/lybtug-48/e?versionId=null 2024-04-16T18:15:33.968 DEBUG:boto:auth_path=/lybtug-48/e?versionId=null 2024-04-16T18:15:33.968 DEBUG:boto:Method: GET 2024-04-16T18:15:33.968 DEBUG:boto:Path: /lybtug-48/e?versionId=null 2024-04-16T18:15:33.968 DEBUG:boto:Data: 2024-04-16T18:15:33.968 DEBUG:boto:Headers: {} 2024-04-16T18:15:33.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:33.968 DEBUG:boto:Port: 8000 2024-04-16T18:15:33.968 DEBUG:boto:Params: {} 2024-04-16T18:15:33.968 DEBUG:boto:Token: None 2024-04-16T18:15:33.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:33 GMT /lybtug-48/e?versionId=null 2024-04-16T18:15:33.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LF9FDjzNGv9fPKhSGpS5X+e6N4U= 2024-04-16T18:15:33.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LF9FDjzNGv9fPKhSGpS5X+e6N4U=', 'Content-Length': '0'} 2024-04-16T18:15:33.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000282880275549acc2-00661ec045-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.014 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.014 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.015 DEBUG:boto:path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.015 DEBUG:boto:auth_path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.015 DEBUG:boto:Method: GET 2024-04-16T18:15:34.015 DEBUG:boto:Path: /lybtug-48/e?versionId=null 2024-04-16T18:15:34.015 DEBUG:boto:Data: 2024-04-16T18:15:34.015 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.015 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.015 DEBUG:boto:Params: {} 2024-04-16T18:15:34.015 DEBUG:boto:Token: None 2024-04-16T18:15:34.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e?versionId=null 2024-04-16T18:15:34.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyN9M/fAWL8jZNp9sgbqQnoV2lw= 2024-04-16T18:15:34.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyN9M/fAWL8jZNp9sgbqQnoV2lw=', 'Content-Length': '0'} 2024-04-16T18:15:34.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a02796808d77f41c-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.019 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.019 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.019 DEBUG:boto:path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.019 DEBUG:boto:auth_path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.020 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.020 DEBUG:boto:Path: /lybtug-48/e?versionId=null 2024-04-16T18:15:34.020 DEBUG:boto:Data: 2024-04-16T18:15:34.020 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.020 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.020 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.020 DEBUG:boto:Params: {} 2024-04-16T18:15:34.020 DEBUG:boto:Token: None 2024-04-16T18:15:34.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e?versionId=null 2024-04-16T18:15:34.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+95wA+Q5bzvFd3F3ycDvU9MJ7ZM= 2024-04-16T18:15:34.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+95wA+Q5bzvFd3F3ycDvU9MJ7ZM=', 'Content-Length': '0'} 2024-04-16T18:15:34.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000424fd87c06dd5275-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.023 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.024 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.024 DEBUG:boto:path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.024 DEBUG:boto:auth_path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.024 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.024 DEBUG:boto:Path: /lybtug-48/e?versionId=null 2024-04-16T18:15:34.024 DEBUG:boto:Data: 2024-04-16T18:15:34.024 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.024 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.024 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.024 DEBUG:boto:Params: {} 2024-04-16T18:15:34.024 DEBUG:boto:Token: None 2024-04-16T18:15:34.025 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e?versionId=null 2024-04-16T18:15:34.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+95wA+Q5bzvFd3F3ycDvU9MJ7ZM= 2024-04-16T18:15:34.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+95wA+Q5bzvFd3F3ycDvU9MJ7ZM=', 'Content-Length': '0'} 2024-04-16T18:15:34.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ece5d71417832ce9-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.028 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:15:34.028 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.028 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.028 DEBUG:boto:path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.028 DEBUG:boto:auth_path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.028 DEBUG:boto:Method: GET 2024-04-16T18:15:34.028 DEBUG:boto:Path: /lybtug-48/e?versionId=null 2024-04-16T18:15:34.028 DEBUG:boto:Data: 2024-04-16T18:15:34.029 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.029 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.029 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.029 DEBUG:boto:Params: {} 2024-04-16T18:15:34.029 DEBUG:boto:Token: None 2024-04-16T18:15:34.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e?versionId=null 2024-04-16T18:15:34.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyN9M/fAWL8jZNp9sgbqQnoV2lw= 2024-04-16T18:15:34.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyN9M/fAWL8jZNp9sgbqQnoV2lw=', 'Content-Length': '0'} 2024-04-16T18:15:34.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ffdff44d90ccd78-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.074 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.074 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.074 DEBUG:boto:path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.075 DEBUG:boto:auth_path=/lybtug-48/e?versionId=null 2024-04-16T18:15:34.075 DEBUG:boto:Method: GET 2024-04-16T18:15:34.075 DEBUG:boto:Path: /lybtug-48/e?versionId=null 2024-04-16T18:15:34.075 DEBUG:boto:Data: 2024-04-16T18:15:34.075 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.075 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.075 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.075 DEBUG:boto:Params: {} 2024-04-16T18:15:34.075 DEBUG:boto:Token: None 2024-04-16T18:15:34.075 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e?versionId=null 2024-04-16T18:15:34.075 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kyN9M/fAWL8jZNp9sgbqQnoV2lw= 2024-04-16T18:15:34.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kyN9M/fAWL8jZNp9sgbqQnoV2lw=', 'Content-Length': '0'} 2024-04-16T18:15:34.078 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e66863e4854e8c06-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.079 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.079 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.079 DEBUG:boto:path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.079 DEBUG:boto:auth_path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.079 DEBUG:boto:Method: GET 2024-04-16T18:15:34.079 DEBUG:boto:Path: /lybtug-48/?prefix=e 2024-04-16T18:15:34.079 DEBUG:boto:Data: 2024-04-16T18:15:34.079 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.079 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.079 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.080 DEBUG:boto:Params: {} 2024-04-16T18:15:34.080 DEBUG:boto:Token: None 2024-04-16T18:15:34.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2949919e94ff801-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.126 DEBUG:boto:b'lybtug-48e1000falsee2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.127 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.127 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.127 DEBUG:boto:path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.127 DEBUG:boto:auth_path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.127 DEBUG:boto:Method: GET 2024-04-16T18:15:34.127 DEBUG:boto:Path: /lybtug-48/?prefix=e 2024-04-16T18:15:34.127 DEBUG:boto:Data: 2024-04-16T18:15:34.127 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.127 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.127 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.127 DEBUG:boto:Params: {} 2024-04-16T18:15:34.128 DEBUG:boto:Token: None 2024-04-16T18:15:34.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000878229e0c71052ee-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.131 DEBUG:boto:b'lybtug-48e1000falsee2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.132 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.132 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.132 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.132 DEBUG:boto:Path: /lybtug-48/e 2024-04-16T18:15:34.132 DEBUG:boto:Data: 2024-04-16T18:15:34.132 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.132 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.132 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.132 DEBUG:boto:Params: {} 2024-04-16T18:15:34.132 DEBUG:boto:Token: None 2024-04-16T18:15:34.133 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e 2024-04-16T18:15:34.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h3o2JZW4z+J4NdkS98roTYLDEtM= 2024-04-16T18:15:34.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h3o2JZW4z+J4NdkS98roTYLDEtM=', 'Content-Length': '0'} 2024-04-16T18:15:34.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddf09fbcc94f57c6-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.136 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.136 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.136 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.136 DEBUG:boto:Path: /lybtug-48/e 2024-04-16T18:15:34.136 DEBUG:boto:Data: 2024-04-16T18:15:34.136 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.136 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.136 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.136 DEBUG:boto:Params: {} 2024-04-16T18:15:34.137 DEBUG:boto:Token: None 2024-04-16T18:15:34.137 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e 2024-04-16T18:15:34.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h3o2JZW4z+J4NdkS98roTYLDEtM= 2024-04-16T18:15:34.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h3o2JZW4z+J4NdkS98roTYLDEtM=', 'Content-Length': '0'} 2024-04-16T18:15:34.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca4d5b51101a3869-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.140 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:15:34.140 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.140 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.140 DEBUG:boto:Method: GET 2024-04-16T18:15:34.140 DEBUG:boto:Path: /lybtug-48/e 2024-04-16T18:15:34.140 DEBUG:boto:Data: 2024-04-16T18:15:34.140 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.140 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.141 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.141 DEBUG:boto:Params: {} 2024-04-16T18:15:34.141 DEBUG:boto:Token: None 2024-04-16T18:15:34.141 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e 2024-04-16T18:15:34.141 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PQQXaGRvESc+8a0PWQbVh/rnJW4= 2024-04-16T18:15:34.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PQQXaGRvESc+8a0PWQbVh/rnJW4=', 'Content-Length': '0'} 2024-04-16T18:15:34.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004863a787eb8a5629-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.186 DEBUG:boto:path=/lybtug-48/e 2024-04-16T18:15:34.186 DEBUG:boto:auth_path=/lybtug-48/e 2024-04-16T18:15:34.187 DEBUG:boto:Method: GET 2024-04-16T18:15:34.187 DEBUG:boto:Path: /lybtug-48/e 2024-04-16T18:15:34.187 DEBUG:boto:Data: 2024-04-16T18:15:34.187 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.187 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.187 DEBUG:boto:Params: {} 2024-04-16T18:15:34.187 DEBUG:boto:Token: None 2024-04-16T18:15:34.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/e 2024-04-16T18:15:34.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PQQXaGRvESc+8a0PWQbVh/rnJW4= 2024-04-16T18:15:34.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PQQXaGRvESc+8a0PWQbVh/rnJW4=', 'Content-Length': '0'} 2024-04-16T18:15:34.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075fe31762ecbb107-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.191 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.191 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.191 DEBUG:boto:path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.191 DEBUG:boto:auth_path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.191 DEBUG:boto:Method: GET 2024-04-16T18:15:34.191 DEBUG:boto:Path: /lybtug-48/?prefix=e 2024-04-16T18:15:34.191 DEBUG:boto:Data: 2024-04-16T18:15:34.191 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.191 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.191 DEBUG:boto:Params: {} 2024-04-16T18:15:34.192 DEBUG:boto:Token: None 2024-04-16T18:15:34.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.195 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009310b1f90de83f5e-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.238 DEBUG:boto:b'lybtug-48e1000falsee2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.239 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.239 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.239 DEBUG:boto:path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.239 DEBUG:boto:auth_path=/lybtug-48/?prefix=e 2024-04-16T18:15:34.239 DEBUG:boto:Method: GET 2024-04-16T18:15:34.239 DEBUG:boto:Path: /lybtug-48/?prefix=e 2024-04-16T18:15:34.239 DEBUG:boto:Data: 2024-04-16T18:15:34.239 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.239 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.239 DEBUG:boto:Params: {} 2024-04-16T18:15:34.240 DEBUG:boto:Token: None 2024-04-16T18:15:34.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7bab9a328fb1d69-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.243 DEBUG:boto:b'lybtug-48e1000falsee2024-04-16T18:12:56.342Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:15:34.244 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.244 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.244 DEBUG:boto:path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.244 DEBUG:boto:auth_path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.244 DEBUG:boto:Method: GET 2024-04-16T18:15:34.244 DEBUG:boto:Path: /lybtug-48/f?versionId=null 2024-04-16T18:15:34.244 DEBUG:boto:Data: 2024-04-16T18:15:34.244 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.244 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.244 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.244 DEBUG:boto:Params: {} 2024-04-16T18:15:34.244 DEBUG:boto:Token: None 2024-04-16T18:15:34.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f?versionId=null 2024-04-16T18:15:34.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac= 2024-04-16T18:15:34.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac=', 'Content-Length': '0'} 2024-04-16T18:15:34.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c5bf68dc1c15461-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.290 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.290 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.290 DEBUG:boto:path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.291 DEBUG:boto:auth_path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.291 DEBUG:boto:Method: GET 2024-04-16T18:15:34.291 DEBUG:boto:Path: /lybtug-48/f?versionId=null 2024-04-16T18:15:34.291 DEBUG:boto:Data: 2024-04-16T18:15:34.291 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.291 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.291 DEBUG:boto:Params: {} 2024-04-16T18:15:34.291 DEBUG:boto:Token: None 2024-04-16T18:15:34.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f?versionId=null 2024-04-16T18:15:34.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac= 2024-04-16T18:15:34.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac=', 'Content-Length': '0'} 2024-04-16T18:15:34.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bb03a5d8ed00bc2-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.295 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.295 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.295 DEBUG:boto:path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.295 DEBUG:boto:auth_path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.295 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.295 DEBUG:boto:Path: /lybtug-48/f?versionId=null 2024-04-16T18:15:34.295 DEBUG:boto:Data: 2024-04-16T18:15:34.295 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.295 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.295 DEBUG:boto:Params: {} 2024-04-16T18:15:34.296 DEBUG:boto:Token: None 2024-04-16T18:15:34.296 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f?versionId=null 2024-04-16T18:15:34.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3F13zxqpwyXyeP/Tx3VfRLbcrXw= 2024-04-16T18:15:34.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3F13zxqpwyXyeP/Tx3VfRLbcrXw=', 'Content-Length': '0'} 2024-04-16T18:15:34.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c237b06b236317a2-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.299 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.299 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.299 DEBUG:boto:path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.300 DEBUG:boto:auth_path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.300 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.300 DEBUG:boto:Path: /lybtug-48/f?versionId=null 2024-04-16T18:15:34.300 DEBUG:boto:Data: 2024-04-16T18:15:34.300 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.300 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.300 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.300 DEBUG:boto:Params: {} 2024-04-16T18:15:34.300 DEBUG:boto:Token: None 2024-04-16T18:15:34.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f?versionId=null 2024-04-16T18:15:34.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3F13zxqpwyXyeP/Tx3VfRLbcrXw= 2024-04-16T18:15:34.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3F13zxqpwyXyeP/Tx3VfRLbcrXw=', 'Content-Length': '0'} 2024-04-16T18:15:34.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b435b9d515a27c9b-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:15:34.303 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.304 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.304 DEBUG:boto:path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.304 DEBUG:boto:auth_path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.304 DEBUG:boto:Method: GET 2024-04-16T18:15:34.304 DEBUG:boto:Path: /lybtug-48/f?versionId=null 2024-04-16T18:15:34.304 DEBUG:boto:Data: 2024-04-16T18:15:34.304 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.304 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.304 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.304 DEBUG:boto:Params: {} 2024-04-16T18:15:34.304 DEBUG:boto:Token: None 2024-04-16T18:15:34.305 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f?versionId=null 2024-04-16T18:15:34.305 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac= 2024-04-16T18:15:34.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac=', 'Content-Length': '0'} 2024-04-16T18:15:34.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c835b191596d12e-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.350 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.350 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.351 DEBUG:boto:path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.351 DEBUG:boto:auth_path=/lybtug-48/f?versionId=null 2024-04-16T18:15:34.351 DEBUG:boto:Method: GET 2024-04-16T18:15:34.351 DEBUG:boto:Path: /lybtug-48/f?versionId=null 2024-04-16T18:15:34.351 DEBUG:boto:Data: 2024-04-16T18:15:34.351 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.351 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.351 DEBUG:boto:Params: {} 2024-04-16T18:15:34.351 DEBUG:boto:Token: None 2024-04-16T18:15:34.351 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f?versionId=null 2024-04-16T18:15:34.351 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac= 2024-04-16T18:15:34.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LnfW86tt4TDcVEILTt9lhWlD9Ac=', 'Content-Length': '0'} 2024-04-16T18:15:34.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026133c4959697273-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.355 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.355 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.355 DEBUG:boto:path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.355 DEBUG:boto:auth_path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.355 DEBUG:boto:Method: GET 2024-04-16T18:15:34.355 DEBUG:boto:Path: /lybtug-48/?prefix=f 2024-04-16T18:15:34.355 DEBUG:boto:Data: 2024-04-16T18:15:34.355 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.356 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.356 DEBUG:boto:Params: {} 2024-04-16T18:15:34.356 DEBUG:boto:Token: None 2024-04-16T18:15:34.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7db2180277feef5-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.360 DEBUG:boto:b'lybtug-48f1000falsef2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.361 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.361 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.361 DEBUG:boto:path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.361 DEBUG:boto:auth_path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.361 DEBUG:boto:Method: GET 2024-04-16T18:15:34.361 DEBUG:boto:Path: /lybtug-48/?prefix=f 2024-04-16T18:15:34.361 DEBUG:boto:Data: 2024-04-16T18:15:34.361 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.361 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.361 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.361 DEBUG:boto:Params: {} 2024-04-16T18:15:34.362 DEBUG:boto:Token: None 2024-04-16T18:15:34.362 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.362 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.365 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2d3a335ddb4422b-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.365 DEBUG:boto:b'lybtug-48f1000falsef2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.365 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.365 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.366 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.366 DEBUG:boto:Path: /lybtug-48/f 2024-04-16T18:15:34.366 DEBUG:boto:Data: 2024-04-16T18:15:34.366 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.366 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.366 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.366 DEBUG:boto:Params: {} 2024-04-16T18:15:34.366 DEBUG:boto:Token: None 2024-04-16T18:15:34.366 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f 2024-04-16T18:15:34.367 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:emrvgo1ishqKpJi3y3wLERQylDo= 2024-04-16T18:15:34.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:emrvgo1ishqKpJi3y3wLERQylDo=', 'Content-Length': '0'} 2024-04-16T18:15:34.370 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b64559302cb261ac-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.370 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.370 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.370 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.370 DEBUG:boto:Path: /lybtug-48/f 2024-04-16T18:15:34.370 DEBUG:boto:Data: 2024-04-16T18:15:34.370 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.370 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.371 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.371 DEBUG:boto:Params: {} 2024-04-16T18:15:34.371 DEBUG:boto:Token: None 2024-04-16T18:15:34.371 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f 2024-04-16T18:15:34.371 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:emrvgo1ishqKpJi3y3wLERQylDo= 2024-04-16T18:15:34.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:emrvgo1ishqKpJi3y3wLERQylDo=', 'Content-Length': '0'} 2024-04-16T18:15:34.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000072d877d5f7e6a24-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.374 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:15:34.374 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.374 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.374 DEBUG:boto:Method: GET 2024-04-16T18:15:34.374 DEBUG:boto:Path: /lybtug-48/f 2024-04-16T18:15:34.374 DEBUG:boto:Data: 2024-04-16T18:15:34.375 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.375 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.375 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.375 DEBUG:boto:Params: {} 2024-04-16T18:15:34.375 DEBUG:boto:Token: None 2024-04-16T18:15:34.375 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f 2024-04-16T18:15:34.375 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ws+ALR8rNb0WK7PpLereFT3/q/I= 2024-04-16T18:15:34.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ws+ALR8rNb0WK7PpLereFT3/q/I=', 'Content-Length': '0'} 2024-04-16T18:15:34.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078387fdba9c44d27-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.422 DEBUG:boto:path=/lybtug-48/f 2024-04-16T18:15:34.422 DEBUG:boto:auth_path=/lybtug-48/f 2024-04-16T18:15:34.423 DEBUG:boto:Method: GET 2024-04-16T18:15:34.423 DEBUG:boto:Path: /lybtug-48/f 2024-04-16T18:15:34.423 DEBUG:boto:Data: 2024-04-16T18:15:34.423 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.423 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.423 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.423 DEBUG:boto:Params: {} 2024-04-16T18:15:34.423 DEBUG:boto:Token: None 2024-04-16T18:15:34.423 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/f 2024-04-16T18:15:34.423 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ws+ALR8rNb0WK7PpLereFT3/q/I= 2024-04-16T18:15:34.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ws+ALR8rNb0WK7PpLereFT3/q/I=', 'Content-Length': '0'} 2024-04-16T18:15:34.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e321eb1e83224d6e-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.427 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.427 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.427 DEBUG:boto:path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.427 DEBUG:boto:auth_path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.427 DEBUG:boto:Method: GET 2024-04-16T18:15:34.427 DEBUG:boto:Path: /lybtug-48/?prefix=f 2024-04-16T18:15:34.427 DEBUG:boto:Data: 2024-04-16T18:15:34.428 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.428 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.428 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.428 DEBUG:boto:Params: {} 2024-04-16T18:15:34.428 DEBUG:boto:Token: None 2024-04-16T18:15:34.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.432 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e509c5b0350240c-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.474 DEBUG:boto:b'lybtug-48f1000falsef2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.475 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.475 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.475 DEBUG:boto:path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.475 DEBUG:boto:auth_path=/lybtug-48/?prefix=f 2024-04-16T18:15:34.475 DEBUG:boto:Method: GET 2024-04-16T18:15:34.475 DEBUG:boto:Path: /lybtug-48/?prefix=f 2024-04-16T18:15:34.475 DEBUG:boto:Data: 2024-04-16T18:15:34.475 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.475 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.475 DEBUG:boto:Params: {} 2024-04-16T18:15:34.476 DEBUG:boto:Token: None 2024-04-16T18:15:34.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.479 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000677444f666eb5c3-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.479 DEBUG:boto:b'lybtug-48f1000falsef2024-04-16T18:12:56.357Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:15:34.479 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.479 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.479 DEBUG:boto:path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.479 DEBUG:boto:auth_path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.480 DEBUG:boto:Method: GET 2024-04-16T18:15:34.480 DEBUG:boto:Path: /lybtug-48/g?versionId=null 2024-04-16T18:15:34.480 DEBUG:boto:Data: 2024-04-16T18:15:34.480 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.480 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.480 DEBUG:boto:Params: {} 2024-04-16T18:15:34.480 DEBUG:boto:Token: None 2024-04-16T18:15:34.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g?versionId=null 2024-04-16T18:15:34.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8= 2024-04-16T18:15:34.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8=', 'Content-Length': '0'} 2024-04-16T18:15:34.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000647bbb155c818261-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.526 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.526 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.527 DEBUG:boto:path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.527 DEBUG:boto:auth_path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.527 DEBUG:boto:Method: GET 2024-04-16T18:15:34.527 DEBUG:boto:Path: /lybtug-48/g?versionId=null 2024-04-16T18:15:34.527 DEBUG:boto:Data: 2024-04-16T18:15:34.527 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.527 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.527 DEBUG:boto:Params: {} 2024-04-16T18:15:34.527 DEBUG:boto:Token: None 2024-04-16T18:15:34.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g?versionId=null 2024-04-16T18:15:34.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8= 2024-04-16T18:15:34.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8=', 'Content-Length': '0'} 2024-04-16T18:15:34.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa027fe70c30005a-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.531 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.531 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.531 DEBUG:boto:path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.531 DEBUG:boto:auth_path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.531 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.531 DEBUG:boto:Path: /lybtug-48/g?versionId=null 2024-04-16T18:15:34.531 DEBUG:boto:Data: 2024-04-16T18:15:34.532 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.532 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.532 DEBUG:boto:Params: {} 2024-04-16T18:15:34.532 DEBUG:boto:Token: None 2024-04-16T18:15:34.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g?versionId=null 2024-04-16T18:15:34.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NZCGzzgaIrrUMa7LTg+V/5K9aYg= 2024-04-16T18:15:34.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NZCGzzgaIrrUMa7LTg+V/5K9aYg=', 'Content-Length': '0'} 2024-04-16T18:15:34.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000587e9694dba72770-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.535 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.535 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.535 DEBUG:boto:path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.535 DEBUG:boto:auth_path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.535 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.536 DEBUG:boto:Path: /lybtug-48/g?versionId=null 2024-04-16T18:15:34.536 DEBUG:boto:Data: 2024-04-16T18:15:34.536 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.536 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.536 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.536 DEBUG:boto:Params: {} 2024-04-16T18:15:34.536 DEBUG:boto:Token: None 2024-04-16T18:15:34.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g?versionId=null 2024-04-16T18:15:34.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NZCGzzgaIrrUMa7LTg+V/5K9aYg= 2024-04-16T18:15:34.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NZCGzzgaIrrUMa7LTg+V/5K9aYg=', 'Content-Length': '0'} 2024-04-16T18:15:34.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085e6dd7a5813c827-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.539 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:15:34.539 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.539 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.539 DEBUG:boto:path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.539 DEBUG:boto:auth_path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.539 DEBUG:boto:Method: GET 2024-04-16T18:15:34.540 DEBUG:boto:Path: /lybtug-48/g?versionId=null 2024-04-16T18:15:34.540 DEBUG:boto:Data: 2024-04-16T18:15:34.540 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.540 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.540 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.540 DEBUG:boto:Params: {} 2024-04-16T18:15:34.540 DEBUG:boto:Token: None 2024-04-16T18:15:34.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g?versionId=null 2024-04-16T18:15:34.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8= 2024-04-16T18:15:34.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8=', 'Content-Length': '0'} 2024-04-16T18:15:34.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcd76c4292af0762-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.586 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.586 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.587 DEBUG:boto:path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.587 DEBUG:boto:auth_path=/lybtug-48/g?versionId=null 2024-04-16T18:15:34.587 DEBUG:boto:Method: GET 2024-04-16T18:15:34.587 DEBUG:boto:Path: /lybtug-48/g?versionId=null 2024-04-16T18:15:34.587 DEBUG:boto:Data: 2024-04-16T18:15:34.587 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.587 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.587 DEBUG:boto:Params: {} 2024-04-16T18:15:34.587 DEBUG:boto:Token: None 2024-04-16T18:15:34.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g?versionId=null 2024-04-16T18:15:34.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8= 2024-04-16T18:15:34.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wyce2+Tpdf0ImTtt7DgPsusOYj8=', 'Content-Length': '0'} 2024-04-16T18:15:34.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b145c8b7f3fe950-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.591 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.591 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.591 DEBUG:boto:path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.591 DEBUG:boto:auth_path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.591 DEBUG:boto:Method: GET 2024-04-16T18:15:34.591 DEBUG:boto:Path: /lybtug-48/?prefix=g 2024-04-16T18:15:34.591 DEBUG:boto:Data: 2024-04-16T18:15:34.592 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.592 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.592 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.592 DEBUG:boto:Params: {} 2024-04-16T18:15:34.592 DEBUG:boto:Token: None 2024-04-16T18:15:34.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cd4efa04c2dcbb6-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.596 DEBUG:boto:b'lybtug-48g1000falseg2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.597 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.597 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.597 DEBUG:boto:path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.597 DEBUG:boto:auth_path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.597 DEBUG:boto:Method: GET 2024-04-16T18:15:34.597 DEBUG:boto:Path: /lybtug-48/?prefix=g 2024-04-16T18:15:34.597 DEBUG:boto:Data: 2024-04-16T18:15:34.597 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.597 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.597 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.598 DEBUG:boto:Params: {} 2024-04-16T18:15:34.598 DEBUG:boto:Token: None 2024-04-16T18:15:34.598 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.598 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.601 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff603e0f1197b711-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.602 DEBUG:boto:b'lybtug-48g1000falseg2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.602 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.602 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.602 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.602 DEBUG:boto:Path: /lybtug-48/g 2024-04-16T18:15:34.602 DEBUG:boto:Data: 2024-04-16T18:15:34.602 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.602 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.603 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.603 DEBUG:boto:Params: {} 2024-04-16T18:15:34.603 DEBUG:boto:Token: None 2024-04-16T18:15:34.603 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g 2024-04-16T18:15:34.603 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TV86qdncIIkwPeu7VkUXQNiWR98= 2024-04-16T18:15:34.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TV86qdncIIkwPeu7VkUXQNiWR98=', 'Content-Length': '0'} 2024-04-16T18:15:34.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd9b1fda70277875-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.606 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.606 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.606 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.606 DEBUG:boto:Path: /lybtug-48/g 2024-04-16T18:15:34.606 DEBUG:boto:Data: 2024-04-16T18:15:34.606 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.606 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.607 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.607 DEBUG:boto:Params: {} 2024-04-16T18:15:34.607 DEBUG:boto:Token: None 2024-04-16T18:15:34.607 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g 2024-04-16T18:15:34.607 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TV86qdncIIkwPeu7VkUXQNiWR98= 2024-04-16T18:15:34.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TV86qdncIIkwPeu7VkUXQNiWR98=', 'Content-Length': '0'} 2024-04-16T18:15:34.609 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e90cf49f7ef015ca-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.610 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:15:34.610 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.610 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.610 DEBUG:boto:Method: GET 2024-04-16T18:15:34.610 DEBUG:boto:Path: /lybtug-48/g 2024-04-16T18:15:34.610 DEBUG:boto:Data: 2024-04-16T18:15:34.610 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.610 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.611 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.611 DEBUG:boto:Params: {} 2024-04-16T18:15:34.611 DEBUG:boto:Token: None 2024-04-16T18:15:34.611 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g 2024-04-16T18:15:34.611 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3/LTpx+NkbN2ZS9kJaCp0XGAo8I= 2024-04-16T18:15:34.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3/LTpx+NkbN2ZS9kJaCp0XGAo8I=', 'Content-Length': '0'} 2024-04-16T18:15:34.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:27 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008592f4334f73044b-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.658 DEBUG:boto:path=/lybtug-48/g 2024-04-16T18:15:34.658 DEBUG:boto:auth_path=/lybtug-48/g 2024-04-16T18:15:34.659 DEBUG:boto:Method: GET 2024-04-16T18:15:34.659 DEBUG:boto:Path: /lybtug-48/g 2024-04-16T18:15:34.659 DEBUG:boto:Data: 2024-04-16T18:15:34.659 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.659 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.659 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.659 DEBUG:boto:Params: {} 2024-04-16T18:15:34.659 DEBUG:boto:Token: None 2024-04-16T18:15:34.659 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/g 2024-04-16T18:15:34.659 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3/LTpx+NkbN2ZS9kJaCp0XGAo8I= 2024-04-16T18:15:34.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3/LTpx+NkbN2ZS9kJaCp0XGAo8I=', 'Content-Length': '0'} 2024-04-16T18:15:34.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4ce3c60cc85ddb0-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.662 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.662 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.662 DEBUG:boto:path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.662 DEBUG:boto:auth_path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.663 DEBUG:boto:Method: GET 2024-04-16T18:15:34.663 DEBUG:boto:Path: /lybtug-48/?prefix=g 2024-04-16T18:15:34.663 DEBUG:boto:Data: 2024-04-16T18:15:34.663 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.663 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.663 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.663 DEBUG:boto:Params: {} 2024-04-16T18:15:34.663 DEBUG:boto:Token: None 2024-04-16T18:15:34.663 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.667 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047418555a02c9ae6-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.710 DEBUG:boto:b'lybtug-48g1000falseg2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.711 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.711 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.711 DEBUG:boto:path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.711 DEBUG:boto:auth_path=/lybtug-48/?prefix=g 2024-04-16T18:15:34.711 DEBUG:boto:Method: GET 2024-04-16T18:15:34.711 DEBUG:boto:Path: /lybtug-48/?prefix=g 2024-04-16T18:15:34.711 DEBUG:boto:Data: 2024-04-16T18:15:34.711 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.711 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.712 DEBUG:boto:Params: {} 2024-04-16T18:15:34.712 DEBUG:boto:Token: None 2024-04-16T18:15:34.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ebc4b3954ec4ec0a-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.715 DEBUG:boto:b'lybtug-48g1000falseg2024-04-16T18:12:56.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:15:34.715 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.716 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.716 DEBUG:boto:path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.716 DEBUG:boto:auth_path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.716 DEBUG:boto:Method: GET 2024-04-16T18:15:34.716 DEBUG:boto:Path: /lybtug-48/h?versionId=null 2024-04-16T18:15:34.716 DEBUG:boto:Data: 2024-04-16T18:15:34.716 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.716 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.716 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.716 DEBUG:boto:Params: {} 2024-04-16T18:15:34.716 DEBUG:boto:Token: None 2024-04-16T18:15:34.717 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h?versionId=null 2024-04-16T18:15:34.717 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc= 2024-04-16T18:15:34.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc=', 'Content-Length': '0'} 2024-04-16T18:15:34.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a432f68f8ae96998-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.762 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.762 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.762 DEBUG:boto:path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.763 DEBUG:boto:auth_path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.763 DEBUG:boto:Method: GET 2024-04-16T18:15:34.763 DEBUG:boto:Path: /lybtug-48/h?versionId=null 2024-04-16T18:15:34.763 DEBUG:boto:Data: 2024-04-16T18:15:34.763 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.763 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.763 DEBUG:boto:Params: {} 2024-04-16T18:15:34.763 DEBUG:boto:Token: None 2024-04-16T18:15:34.763 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h?versionId=null 2024-04-16T18:15:34.763 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc= 2024-04-16T18:15:34.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc=', 'Content-Length': '0'} 2024-04-16T18:15:34.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000446aa7f3dac773a8-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.767 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.767 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.767 DEBUG:boto:path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.767 DEBUG:boto:auth_path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.767 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.767 DEBUG:boto:Path: /lybtug-48/h?versionId=null 2024-04-16T18:15:34.767 DEBUG:boto:Data: 2024-04-16T18:15:34.767 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.768 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.768 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.768 DEBUG:boto:Params: {} 2024-04-16T18:15:34.768 DEBUG:boto:Token: None 2024-04-16T18:15:34.768 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h?versionId=null 2024-04-16T18:15:34.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xOqilkYwRxo0XPl08TExXJLjunA= 2024-04-16T18:15:34.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xOqilkYwRxo0XPl08TExXJLjunA=', 'Content-Length': '0'} 2024-04-16T18:15:34.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a713fd078644b33c-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.771 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.771 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.771 DEBUG:boto:path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.771 DEBUG:boto:auth_path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.771 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.772 DEBUG:boto:Path: /lybtug-48/h?versionId=null 2024-04-16T18:15:34.772 DEBUG:boto:Data: 2024-04-16T18:15:34.772 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.772 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.772 DEBUG:boto:Params: {} 2024-04-16T18:15:34.772 DEBUG:boto:Token: None 2024-04-16T18:15:34.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h?versionId=null 2024-04-16T18:15:34.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xOqilkYwRxo0XPl08TExXJLjunA= 2024-04-16T18:15:34.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xOqilkYwRxo0XPl08TExXJLjunA=', 'Content-Length': '0'} 2024-04-16T18:15:34.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000871509a7d007d704-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:15:34.775 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.776 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.776 DEBUG:boto:path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.776 DEBUG:boto:auth_path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.776 DEBUG:boto:Method: GET 2024-04-16T18:15:34.776 DEBUG:boto:Path: /lybtug-48/h?versionId=null 2024-04-16T18:15:34.776 DEBUG:boto:Data: 2024-04-16T18:15:34.776 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.776 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.776 DEBUG:boto:Params: {} 2024-04-16T18:15:34.776 DEBUG:boto:Token: None 2024-04-16T18:15:34.777 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h?versionId=null 2024-04-16T18:15:34.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc= 2024-04-16T18:15:34.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc=', 'Content-Length': '0'} 2024-04-16T18:15:34.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e7f4bdbb49e44ff-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.822 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.822 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.822 DEBUG:boto:path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.823 DEBUG:boto:auth_path=/lybtug-48/h?versionId=null 2024-04-16T18:15:34.823 DEBUG:boto:Method: GET 2024-04-16T18:15:34.823 DEBUG:boto:Path: /lybtug-48/h?versionId=null 2024-04-16T18:15:34.823 DEBUG:boto:Data: 2024-04-16T18:15:34.823 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.823 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.823 DEBUG:boto:Params: {} 2024-04-16T18:15:34.823 DEBUG:boto:Token: None 2024-04-16T18:15:34.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h?versionId=null 2024-04-16T18:15:34.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc= 2024-04-16T18:15:34.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G9VnIx/VlNcJj6kXqCPUf3iSxnc=', 'Content-Length': '0'} 2024-04-16T18:15:34.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094dc39d6b9442162-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.827 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.827 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.827 DEBUG:boto:path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.827 DEBUG:boto:auth_path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.827 DEBUG:boto:Method: GET 2024-04-16T18:15:34.827 DEBUG:boto:Path: /lybtug-48/?prefix=h 2024-04-16T18:15:34.827 DEBUG:boto:Data: 2024-04-16T18:15:34.827 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.827 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.828 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.828 DEBUG:boto:Params: {} 2024-04-16T18:15:34.828 DEBUG:boto:Token: None 2024-04-16T18:15:34.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.832 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092e18a37deeefe64-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.832 DEBUG:boto:b'lybtug-48h1000falseh2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.832 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.833 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.833 DEBUG:boto:path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.833 DEBUG:boto:auth_path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.833 DEBUG:boto:Method: GET 2024-04-16T18:15:34.833 DEBUG:boto:Path: /lybtug-48/?prefix=h 2024-04-16T18:15:34.833 DEBUG:boto:Data: 2024-04-16T18:15:34.833 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.833 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.833 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.833 DEBUG:boto:Params: {} 2024-04-16T18:15:34.833 DEBUG:boto:Token: None 2024-04-16T18:15:34.834 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.834 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.837 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e05d31e40b7bd02e-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.878 DEBUG:boto:b'lybtug-48h1000falseh2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.879 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.879 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.879 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.879 DEBUG:boto:Path: /lybtug-48/h 2024-04-16T18:15:34.879 DEBUG:boto:Data: 2024-04-16T18:15:34.879 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.879 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.879 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.880 DEBUG:boto:Params: {} 2024-04-16T18:15:34.880 DEBUG:boto:Token: None 2024-04-16T18:15:34.880 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h 2024-04-16T18:15:34.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZZ+Xql7PifQVALNaMArNo3MwPRM= 2024-04-16T18:15:34.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZZ+Xql7PifQVALNaMArNo3MwPRM=', 'Content-Length': '0'} 2024-04-16T18:15:34.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2e272f176498293-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.883 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.883 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.883 DEBUG:boto:Method: HEAD 2024-04-16T18:15:34.883 DEBUG:boto:Path: /lybtug-48/h 2024-04-16T18:15:34.883 DEBUG:boto:Data: 2024-04-16T18:15:34.883 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.884 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.884 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.884 DEBUG:boto:Params: {} 2024-04-16T18:15:34.884 DEBUG:boto:Token: None 2024-04-16T18:15:34.884 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h 2024-04-16T18:15:34.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZZ+Xql7PifQVALNaMArNo3MwPRM= 2024-04-16T18:15:34.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZZ+Xql7PifQVALNaMArNo3MwPRM=', 'Content-Length': '0'} 2024-04-16T18:15:34.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5eba17609bfcc15-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.887 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:15:34.887 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.887 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.887 DEBUG:boto:Method: GET 2024-04-16T18:15:34.887 DEBUG:boto:Path: /lybtug-48/h 2024-04-16T18:15:34.887 DEBUG:boto:Data: 2024-04-16T18:15:34.888 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.888 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.888 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.888 DEBUG:boto:Params: {} 2024-04-16T18:15:34.888 DEBUG:boto:Token: None 2024-04-16T18:15:34.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h 2024-04-16T18:15:34.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F39J/B3u5BcoNo5V1nMP+kWhe50= 2024-04-16T18:15:34.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F39J/B3u5BcoNo5V1nMP+kWhe50=', 'Content-Length': '0'} 2024-04-16T18:15:34.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-48:054102a3-8d94-4596-8b3b-6341eab312e1.4804.17'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:14:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a77ef41a61b62e2-00661ec046-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.934 DEBUG:boto:path=/lybtug-48/h 2024-04-16T18:15:34.935 DEBUG:boto:auth_path=/lybtug-48/h 2024-04-16T18:15:34.935 DEBUG:boto:Method: GET 2024-04-16T18:15:34.935 DEBUG:boto:Path: /lybtug-48/h 2024-04-16T18:15:34.935 DEBUG:boto:Data: 2024-04-16T18:15:34.935 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.935 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.935 DEBUG:boto:Params: {} 2024-04-16T18:15:34.935 DEBUG:boto:Token: None 2024-04-16T18:15:34.935 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/h 2024-04-16T18:15:34.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F39J/B3u5BcoNo5V1nMP+kWhe50= 2024-04-16T18:15:34.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F39J/B3u5BcoNo5V1nMP+kWhe50=', 'Content-Length': '0'} 2024-04-16T18:15:34.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:12:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e03b0ba288093c9c-00661ec046-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.938 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.939 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.939 DEBUG:boto:path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.939 DEBUG:boto:auth_path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.939 DEBUG:boto:Method: GET 2024-04-16T18:15:34.939 DEBUG:boto:Path: /lybtug-48/?prefix=h 2024-04-16T18:15:34.939 DEBUG:boto:Data: 2024-04-16T18:15:34.939 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.940 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:34.940 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.940 DEBUG:boto:Params: {} 2024-04-16T18:15:34.940 DEBUG:boto:Token: None 2024-04-16T18:15:34.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000485c14d089460696-00661ec046-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.986 DEBUG:boto:b'lybtug-48h1000falseh2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.987 DEBUG:boto:path=/lybtug-48/ 2024-04-16T18:15:34.987 DEBUG:boto:auth_path=/lybtug-48/ 2024-04-16T18:15:34.987 DEBUG:boto:path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.987 DEBUG:boto:auth_path=/lybtug-48/?prefix=h 2024-04-16T18:15:34.987 DEBUG:boto:Method: GET 2024-04-16T18:15:34.987 DEBUG:boto:Path: /lybtug-48/?prefix=h 2024-04-16T18:15:34.987 DEBUG:boto:Data: 2024-04-16T18:15:34.987 DEBUG:boto:Headers: {} 2024-04-16T18:15:34.987 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:34.987 DEBUG:boto:Port: 8000 2024-04-16T18:15:34.988 DEBUG:boto:Params: {} 2024-04-16T18:15:34.988 DEBUG:boto:Token: None 2024-04-16T18:15:34.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:34 GMT /lybtug-48/ 2024-04-16T18:15:34.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio= 2024-04-16T18:15:34.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZsZ2hvHlT3pBGjY4IQdz+jhLio=', 'Content-Length': '0'} 2024-04-16T18:15:34.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003419a9753122d534-00661ec046-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:34.991 DEBUG:boto:b'lybtug-48h1000falseh2024-04-16T18:12:56.423Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:15:34.991 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-48 zones={test-zone2, test-zone1} 2024-04-16T18:15:34.992 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_reshard_full ... ok 2024-04-16T18:15:34.993 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:34.994 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:34.994 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:35.105 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:35.138 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T18:15:35.139 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:15:35.139 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:35.139 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.139 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.139 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:35.255+0000 7f418611a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:35.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:35.255+0000 7f418611a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:35.286 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:35Z"} 2024-04-16T18:15:35.286 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:35Z"} 2024-04-16T18:15:35.286 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T18:15:35.286 DEBUG:rgw_multi.tests:log_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:15:35.286 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T18:15:35.287 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:35.287 DEBUG:boto:path=/ 2024-04-16T18:15:35.287 DEBUG:boto:auth_path=/ 2024-04-16T18:15:35.287 DEBUG:boto:Method: GET 2024-04-16T18:15:35.287 DEBUG:boto:Path: / 2024-04-16T18:15:35.287 DEBUG:boto:Data: 2024-04-16T18:15:35.287 DEBUG:boto:Headers: {} 2024-04-16T18:15:35.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.287 DEBUG:boto:Port: 8000 2024-04-16T18:15:35.287 DEBUG:boto:Params: {} 2024-04-16T18:15:35.287 DEBUG:boto:Token: None 2024-04-16T18:15:35.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:35 GMT / 2024-04-16T18:15:35.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uj98wPL2ZuARqEfURnedlRab42I= 2024-04-16T18:15:35.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uj98wPL2ZuARqEfURnedlRab42I=', 'Content-Length': '0'} 2024-04-16T18:15:35.291 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000989bf233ea0748c-00661ec047-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:35.292 DEBUG:boto:path=/ 2024-04-16T18:15:35.293 DEBUG:boto:auth_path=/ 2024-04-16T18:15:35.293 DEBUG:boto:Method: GET 2024-04-16T18:15:35.293 DEBUG:boto:Path: / 2024-04-16T18:15:35.293 DEBUG:boto:Data: 2024-04-16T18:15:35.293 DEBUG:boto:Headers: {} 2024-04-16T18:15:35.293 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.293 DEBUG:boto:Port: 8000 2024-04-16T18:15:35.293 DEBUG:boto:Params: {} 2024-04-16T18:15:35.293 DEBUG:boto:Token: None 2024-04-16T18:15:35.293 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:15:35 GMT / 2024-04-16T18:15:35.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uj98wPL2ZuARqEfURnedlRab42I= 2024-04-16T18:15:35.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uj98wPL2ZuARqEfURnedlRab42I=', 'Content-Length': '0'} 2024-04-16T18:15:35.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc002f84b0623d99-00661ec047-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:35 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:35.297 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_creation_time ... ok 2024-04-16T18:15:35.298 INFO:rgw_multi.tests:create role zone=test-zone1 name=roles-lybtug-1 2024-04-16T18:15:35.298 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateRole: calling handler 2024-04-16T18:15:35.299 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.299 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.300 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.300 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.300 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1', 'AssumeRolePolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.300 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler > 2024-04-16T18:15:35.301 DEBUG:botocore.hooks:Event choose-signer.iam.CreateRole: calling handler 2024-04-16T18:15:35.302 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.302 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 442fa7e1ba0a79321ebca45b5be24a997d01b3b77706ea7f9b9393f0f57d0a9e 2024-04-16T18:15:35.302 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request a3bd9900705bee7fc0f2ea61754723a4238d7454305dcf2e63d51a716064dbc4 2024-04-16T18:15:35.302 DEBUG:botocore.auth:Signature: 1f4735c9239e7886b428f89dc85e663e0866280d6e7c717bac402b3c6ca778f2 2024-04-16T18:15:35.303 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler 2024-04-16T18:15:35.303 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.331 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000051eaf5c869b20014-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '785', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.331 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}tx0000051eaf5c869b20014-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.333 DEBUG:botocore.hooks:Event needs-retry.iam.CreateRole: calling handler 2024-04-16T18:15:35.333 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.333 DEBUG:botocore.hooks:Event after-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.333 INFO:rgw_multi.tests:create role zone=test-zone2 name=roles-lybtug-2 2024-04-16T18:15:35.334 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateRole: calling handler 2024-04-16T18:15:35.334 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.334 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.334 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.335 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.335 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2', 'AssumeRolePolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.335 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler > 2024-04-16T18:15:35.335 DEBUG:botocore.hooks:Event choose-signer.iam.CreateRole: calling handler 2024-04-16T18:15:35.336 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.336 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 4f31e89ac8663a8973772eeb85a7329b9d666293b988f37b445c5ce05481aceb 2024-04-16T18:15:35.336 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request b2f64141c2e513f224bde51a0c9186476f0130b64bb6c84b7362023522a84c95 2024-04-16T18:15:35.336 DEBUG:botocore.auth:Signature: ce5829837c933356129c724de32003b90eddfa2e3167b44bf0745150a248e251 2024-04-16T18:15:35.336 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler 2024-04-16T18:15:35.336 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.381 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000067fd92a4d158c74-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '785', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.382 DEBUG:botocore.parsers:Response body: b'f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}tx00000067fd92a4d158c74-00661ec047-5397-test-zone2' 2024-04-16T18:15:35.383 DEBUG:botocore.hooks:Event needs-retry.iam.CreateRole: calling handler 2024-04-16T18:15:35.383 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.383 DEBUG:botocore.hooks:Event after-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.383 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.383 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:35.425 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:35.425 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:35.529 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"1_1713291335.325486_267.1","last_update":"2024-04-16T18:15:35.325486Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:35.530 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291335.325486_267.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:35.530 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:35.530 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.530 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:35.572 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:35.572 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:35.651+0000 7ff775fac980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:35.651+0000 7ff775fac980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:35.682 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291335.325486_267.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.325486Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:35Z"} 2024-04-16T18:15:35.682 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291335.325486_267.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.325486Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:35Z"} 2024-04-16T18:15:35.683 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291335.325486_267.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.325486Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:35.683 DEBUG:rgw_multi.tests:log_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291335.325486_267.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:35.683 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291335.325486_267.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:35.683 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:35.691 DEBUG:botocore.loaders:Loading JSON file: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/data/iam/2010-05-08/paginators-1.json 2024-04-16T18:15:35.692 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRoles: calling handler 2024-04-16T18:15:35.692 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.692 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.693 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:35.693 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:35.693 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRoles) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRoles', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.693 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler > 2024-04-16T18:15:35.694 DEBUG:botocore.hooks:Event choose-signer.iam.ListRoles: calling handler 2024-04-16T18:15:35.694 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.694 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 8e689dc6b69b17003f0460011582254d8643f0dfb48ac46e093a24df58170b3f 2024-04-16T18:15:35.694 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request d575ea85385bf7a53d8d00042f79ab5fd0ba2229df7b656564a0fb06c4228b17 2024-04-16T18:15:35.694 DEBUG:botocore.auth:Signature: b9b94a787399f3fa32bd7b071a9774af81dd4a0bd9a161454867f0c526ecdf34 2024-04-16T18:15:35.694 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler 2024-04-16T18:15:35.695 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.699 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000091313734a77c3325-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1519', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.699 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111falsetx0000091313734a77c3325-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.700 DEBUG:botocore.hooks:Event needs-retry.iam.ListRoles: calling handler 2024-04-16T18:15:35.700 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.701 DEBUG:botocore.hooks:Event after-call.iam.ListRoles: calling handler 2024-04-16T18:15:35.701 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRoles: calling handler 2024-04-16T18:15:35.701 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.701 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.701 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:35.702 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:35.702 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRoles) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRoles', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.702 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler > 2024-04-16T18:15:35.702 DEBUG:botocore.hooks:Event choose-signer.iam.ListRoles: calling handler 2024-04-16T18:15:35.703 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.703 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 8e689dc6b69b17003f0460011582254d8643f0dfb48ac46e093a24df58170b3f 2024-04-16T18:15:35.703 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 8d61c74895b87b9a6c60639c553df12b2e29026eb5e038b6e1b74307af97d338 2024-04-16T18:15:35.703 DEBUG:botocore.auth:Signature: 4a3f078a3ee4fd4e4251cafdfe3e7b262ce014e54f41734b64ed2de469e373b0 2024-04-16T18:15:35.703 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler 2024-04-16T18:15:35.703 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.710 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000891e0b67028758f6-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1519', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.710 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111falsetx00000891e0b67028758f6-00661ec047-5397-test-zone2' 2024-04-16T18:15:35.711 DEBUG:botocore.hooks:Event needs-retry.iam.ListRoles: calling handler 2024-04-16T18:15:35.711 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.711 DEBUG:botocore.hooks:Event after-call.iam.ListRoles: calling handler 2024-04-16T18:15:35.711 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:35.712 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.712 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.712 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.712 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.712 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.712 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:35.713 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:35.713 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.713 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 531de279f0b6986a98aec0ac85006960a2b886a52a23f3a97418ea909469aff9 2024-04-16T18:15:35.713 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 0d7f39effb531aa7ee9a29fb01e096d0d0a7bd4c9454af732c07ddb9257183c3 2024-04-16T18:15:35.713 DEBUG:botocore.auth:Signature: 392e083340327b4292c0ecba5b5d145f6fccacdb24d462448d463ba7f3ba69f1 2024-04-16T18:15:35.713 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:35.714 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.716 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ec5e80d8af43ccab-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.716 DEBUG:botocore.parsers:Response body: b'tx00000ec5e80d8af43ccab-00661ec047-4804-test-zone1853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:35.717 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:35.717 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.717 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:35.717 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:35.718 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.718 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.718 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.718 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.718 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.719 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:35.719 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:35.719 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.719 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 531de279f0b6986a98aec0ac85006960a2b886a52a23f3a97418ea909469aff9 2024-04-16T18:15:35.719 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 6b579aa631986a2dee14c4f32f811329ecfebc200623013644f0ece179df8752 2024-04-16T18:15:35.719 DEBUG:botocore.auth:Signature: a5f182afb2bd9feea66d43e9e60ab0c644b3617e3cebbe190dff097efb8fd5e1 2024-04-16T18:15:35.720 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:35.720 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.722 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ae99daa1335c55d2-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.722 DEBUG:botocore.parsers:Response body: b'tx00000ae99daa1335c55d2-00661ec047-5397-test-zone2853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:35.722 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:35.723 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.723 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:35.723 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.723 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.724 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.724 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.724 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.724 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.724 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:35.724 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.725 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.725 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date a1e5310e2ee0301edd6ccf49921d305b1fd2f5ba42bce13d1ba0ac6e4fae7892 2024-04-16T18:15:35.725 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 41d395303f3949b95ddf89a18ce1341b82c5289d5cecb4497e3ab13430af9b57 2024-04-16T18:15:35.725 DEBUG:botocore.auth:Signature: 47b2458dce4af27bd4d55a80add1554761f8cf9abc23fa627288de656238d565 2024-04-16T18:15:35.725 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.726 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.728 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006de3b7c94ae0f5b5-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.728 DEBUG:botocore.parsers:Response body: b'tx000006de3b7c94ae0f5b5-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.729 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.729 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.729 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.729 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.729 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.730 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.730 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.730 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.730 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.730 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:35.730 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.731 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.731 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date a1e5310e2ee0301edd6ccf49921d305b1fd2f5ba42bce13d1ba0ac6e4fae7892 2024-04-16T18:15:35.731 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 96c0284214cc8e249171cfb79248ec983baacb558a85593c3821f1a299991bf3 2024-04-16T18:15:35.731 DEBUG:botocore.auth:Signature: 563236238cee74d887af581c46509c519e4ae68855dfa0eda08a7192ee52baec 2024-04-16T18:15:35.731 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.732 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.734 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000bbe89e1ada2852bd-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.734 DEBUG:botocore.parsers:Response body: b'tx00000bbe89e1ada2852bd-00661ec047-5397-test-zone2' 2024-04-16T18:15:35.734 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.734 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.734 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.735 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.735 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.735 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.736 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.736 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.736 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.736 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:35.736 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.737 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.737 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date bc10540825b8174da6b0b6c6a0781cbbae8d1937f3ae57583a6120e809ff3750 2024-04-16T18:15:35.737 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request f5b11ce88ef443d908028a1a78ea862b3944c8a34b99ffea40d8ed4dc15b7034 2024-04-16T18:15:35.737 DEBUG:botocore.auth:Signature: 53e793aef259bc33b3f9c90621c7544ce19724cb47a53c8a7ad565f337c4f669 2024-04-16T18:15:35.737 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.737 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.740 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000410db888536638a2-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.740 DEBUG:botocore.parsers:Response body: b'tx00000410db888536638a2-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.740 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.741 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.741 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.741 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.741 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.741 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.742 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.742 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.742 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.742 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:35.742 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.743 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.743 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date bc10540825b8174da6b0b6c6a0781cbbae8d1937f3ae57583a6120e809ff3750 2024-04-16T18:15:35.743 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request c8f61e1177d95455dcaf5135a713015c91767fabd769948c02c80cedb4461976 2024-04-16T18:15:35.743 DEBUG:botocore.auth:Signature: 8d0405efb78b6986b00d9836d70229a014600c9a31eee2683428509106a58c32 2024-04-16T18:15:35.743 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.744 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.746 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000468804a9fdb5a615-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.746 DEBUG:botocore.parsers:Response body: b'tx00000468804a9fdb5a615-00661ec047-5397-test-zone2' 2024-04-16T18:15:35.747 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.747 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.747 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.747 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:35.747 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.747 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.747 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.748 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.748 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.748 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:35.748 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:35.748 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.748 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 02e2c37e0f639aed605a56f302249a2397fee3f10d1ef5436cec797a2864aea1 2024-04-16T18:15:35.748 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 7b8a58449f4e3236f797f6a975935e766020ae20e1e4e1fe19e4d0d43840594e 2024-04-16T18:15:35.749 DEBUG:botocore.auth:Signature: 10c8e03ddb10a74d8d3728b76dbfde91efe760c358691325aafa54197ec56b2f 2024-04-16T18:15:35.749 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:35.749 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.751 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000069f033db89fa04db-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.751 DEBUG:botocore.parsers:Response body: b'tx0000069f033db89fa04db-00661ec047-4804-test-zone1f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:35.752 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:35.752 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.752 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:35.752 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:35.752 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.752 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.753 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.753 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:35.753 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.753 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:35.753 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:35.753 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.754 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 02e2c37e0f639aed605a56f302249a2397fee3f10d1ef5436cec797a2864aea1 2024-04-16T18:15:35.754 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request a688bbf1a81213c30a1fc4ca820e7973bd8cc6d86a1ef5a4ca94b065fc9f27a7 2024-04-16T18:15:35.754 DEBUG:botocore.auth:Signature: 37cfc2c58d8635ac61aef33966db6a9664bca8db327ea79a802cee77ed1f74ba 2024-04-16T18:15:35.754 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:35.754 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.757 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000096046b6f20434081-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.757 DEBUG:botocore.parsers:Response body: b'tx0000096046b6f20434081-00661ec047-5397-test-zone2f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:35.757 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:35.757 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.758 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:35.758 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.758 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.758 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.758 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.758 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.759 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.759 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:35.759 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.759 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.759 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date a1d084b46fad81a075761bd5542a6a0e50950ec6311a68cd737f0f1de5b600c0 2024-04-16T18:15:35.759 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 657b63c96e11898489e67f965d7ab53df3aad1ad49c2f7544d8392b0e0247269 2024-04-16T18:15:35.760 DEBUG:botocore.auth:Signature: 76d61c3eced7375c6bf754d1e3ff9479643e8238f80c5e41a7c67816a6f6524b 2024-04-16T18:15:35.760 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.760 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.763 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a5614363939c1c16-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.763 DEBUG:botocore.parsers:Response body: b'tx00000a5614363939c1c16-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.763 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.764 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.764 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.764 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.764 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.764 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.764 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.764 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.764 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.765 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:35.765 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.765 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.765 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date a1d084b46fad81a075761bd5542a6a0e50950ec6311a68cd737f0f1de5b600c0 2024-04-16T18:15:35.765 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 8e87589e23becdd10d7c1abde051ed7c95bbd4106ab35c72697132f9ee76634f 2024-04-16T18:15:35.765 DEBUG:botocore.auth:Signature: d8ecae0b08a57c935b8a63fbc3cc809b1752849cceb29ad94792cd76d39ddc14 2024-04-16T18:15:35.765 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.766 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.768 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000306d8d800f141bf8-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.768 DEBUG:botocore.parsers:Response body: b'tx00000306d8d800f141bf8-00661ec047-5397-test-zone2' 2024-04-16T18:15:35.768 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.768 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.769 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:35.769 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.769 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.769 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.769 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.770 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.770 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.770 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:35.770 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.770 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.770 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 84b65f4bc9cead3a6678f4d4a45b599660046d1017f83ffab2a97c93c704c904 2024-04-16T18:15:35.771 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request dcb4b233fec2a030eb3ec302399243ff876f489e10cd931fc4cbdc7979e67406 2024-04-16T18:15:35.771 DEBUG:botocore.auth:Signature: 31ba84cd845def5e457b0094a1ca18e2f88c46d62edb1d5e41111b5dfe332886 2024-04-16T18:15:35.771 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.771 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.814 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ba39467aa3a21cc3-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.815 DEBUG:botocore.parsers:Response body: b'tx00000ba39467aa3a21cc3-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.815 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.815 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.815 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.815 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.815 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.815 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:35.816 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.816 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.816 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.816 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:35.816 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.816 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.817 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 84b65f4bc9cead3a6678f4d4a45b599660046d1017f83ffab2a97c93c704c904 2024-04-16T18:15:35.817 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 652eedc4f2738134f4aa0aed0116321c0cf50797ccce0ea830d4988b4aa4da51 2024-04-16T18:15:35.817 DEBUG:botocore.auth:Signature: 5e9e79eed71e271e072122fcdc559519eb8ddcdeef28d1168b1430007ce04dd2 2024-04-16T18:15:35.817 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.817 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.820 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000c5ad8b95729b34f8-00661ec047-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.820 DEBUG:botocore.parsers:Response body: b'tx00000c5ad8b95729b34f8-00661ec047-5397-test-zone2' 2024-04-16T18:15:35.820 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.821 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.821 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:35.821 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_role_sync ... ok 2024-04-16T18:15:35.821 INFO:rgw_multi.tests:create role zone=test-zone1 name=roles-lybtug-3 2024-04-16T18:15:35.822 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateRole: calling handler 2024-04-16T18:15:35.822 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:35.822 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:35.822 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.822 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.822 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-3', 'AssumeRolePolicyDocument': '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:35.822 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler > 2024-04-16T18:15:35.823 DEBUG:botocore.hooks:Event choose-signer.iam.CreateRole: calling handler 2024-04-16T18:15:35.823 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:35.823 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181535Z content-type;host;x-amz-date 46da7124adb8bef684229278ed03be8282426e24f2d4b3ac63bd252368217cca 2024-04-16T18:15:35.823 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181535Z 20240416/test-zonegroup/iam/aws4_request 55c53c1e28b97e9f300842ac222ffa6d4165fa5cdb572222e93a9b492f60e8e8 2024-04-16T18:15:35.823 DEBUG:botocore.auth:Signature: 911cf2252189fb328ced2c189b1630af8ba7acb25b1269877a4462440f02da08 2024-04-16T18:15:35.823 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler 2024-04-16T18:15:35.824 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:35.849 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000943003eee7e6d3a1-00661ec047-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '778', 'Date': 'Tue, 16 Apr 2024 18:15:35 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:35.849 DEBUG:botocore.parsers:Response body: b'8bea551d-865b-4b9e-b3d7-a440ef14cf0aroles-lybtug-3/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-32024-04-16T18:15:35.834Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}tx00000943003eee7e6d3a1-00661ec047-4804-test-zone1' 2024-04-16T18:15:35.849 DEBUG:botocore.hooks:Event needs-retry.iam.CreateRole: calling handler 2024-04-16T18:15:35.850 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:35.850 DEBUG:botocore.hooks:Event after-call.iam.CreateRole: calling handler 2024-04-16T18:15:35.850 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.850 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.850 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:35.891 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:35.891 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:35.960 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:35.992 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"1_1713291335.842842_269.1","last_update":"2024-04-16T18:15:35.842842Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:35.992 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291335.842842_269.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:35.992 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:35.992 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:35.993 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:36.036 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:36.036 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:36.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:36.111+0000 7f75a8348980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:36.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:36.111+0000 7f75a8348980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:36.142 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291335.842842_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.842842Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T18:15:36.143 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291335.842842_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.842842Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T18:15:36.143 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291335.842842_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.842842Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:36.143 DEBUG:rgw_multi.tests:log_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291335.842842_269.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.143 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291335.842842_269.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.143 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:36.143 INFO:rgw_multi.tests:checking if zone: test-zone1 has role: roles-lybtug-3 2024-04-16T18:15:36.143 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:36.143 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:36.143 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:36.144 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.144 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.144 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-3'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:36.144 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:36.144 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:36.145 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:36.145 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181536Z content-type;host;x-amz-date d3777ae24dcb0b99251a43a20b49ff8964459bed51d9858d33e4f6c619ce5e66 2024-04-16T18:15:36.145 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181536Z 20240416/test-zonegroup/iam/aws4_request cac3a1eaf9b088f7656073bebb622cfb8d4d6495cf70fb8f13ebe0dc6553da93 2024-04-16T18:15:36.145 DEBUG:botocore.auth:Signature: fb31c6f6cd56867c9b51da4031f28fe026d17faba66d38b2490c9d3266c2b4ad 2024-04-16T18:15:36.145 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:36.145 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:36.148 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000791d7c4999214e38-00661ec048-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '766', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.148 DEBUG:botocore.parsers:Response body: b'tx00000791d7c4999214e38-00661ec048-4804-test-zone18bea551d-865b-4b9e-b3d7-a440ef14cf0aroles-lybtug-3/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-32024-04-16T18:15:35.834Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2024-04-16T18:15:36.149 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:36.149 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:36.149 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:36.149 INFO:rgw_multi.tests:success, zone: test-zone1 has role: roles-lybtug-3 2024-04-16T18:15:36.149 INFO:rgw_multi.tests:checking if zone: test-zone2 has role: roles-lybtug-3 2024-04-16T18:15:36.150 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:36.150 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:36.150 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:36.150 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.150 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.150 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-3'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:36.150 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:36.151 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:36.151 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:36.151 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181536Z content-type;host;x-amz-date d3777ae24dcb0b99251a43a20b49ff8964459bed51d9858d33e4f6c619ce5e66 2024-04-16T18:15:36.151 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181536Z 20240416/test-zonegroup/iam/aws4_request 61e97da91b96254de44a921f6f14820861b1dae5cad70b08bd7c294b6b5fbe1b 2024-04-16T18:15:36.151 DEBUG:botocore.auth:Signature: 48ea5601fd10a773437a7545d9d2b5512a33388993cfd4bcced0b2480446a9df 2024-04-16T18:15:36.151 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:36.151 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:36.155 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000004bb65e10b4daf686-00661ec048-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '766', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.155 DEBUG:botocore.parsers:Response body: b'tx000004bb65e10b4daf686-00661ec048-5397-test-zone28bea551d-865b-4b9e-b3d7-a440ef14cf0aroles-lybtug-3/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-32024-04-16T18:15:35.834Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2024-04-16T18:15:36.155 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:36.156 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:36.156 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:36.156 INFO:rgw_multi.tests:success, zone: test-zone2 has role: roles-lybtug-3 2024-04-16T18:15:36.156 INFO:rgw_multi.tests:deleting role: roles-lybtug-3 2024-04-16T18:15:36.156 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteRole: calling handler 2024-04-16T18:15:36.156 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:36.157 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:36.157 DEBUG:botocore.hooks:Event before-call.iam.DeleteRole: calling handler 2024-04-16T18:15:36.157 DEBUG:botocore.hooks:Event before-call.iam.DeleteRole: calling handler 2024-04-16T18:15:36.157 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-3'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:36.157 DEBUG:botocore.hooks:Event request-created.iam.DeleteRole: calling handler > 2024-04-16T18:15:36.158 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteRole: calling handler 2024-04-16T18:15:36.158 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:36.158 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181536Z content-type;host;x-amz-date 4a515884c6dbce307e0239870471d855e611faf22f215bab42d3adf372cc7d86 2024-04-16T18:15:36.158 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181536Z 20240416/test-zonegroup/iam/aws4_request fbd9da313a2e30db5d8769a3da19665d2c57a08b638039ff371b579da8c9ce56 2024-04-16T18:15:36.158 DEBUG:botocore.auth:Signature: f4f5c906cbd3e7bb9fdde369ef0aa26c9aad7d2e491eb6023b2907e453ca48c8 2024-04-16T18:15:36.158 DEBUG:botocore.hooks:Event request-created.iam.DeleteRole: calling handler 2024-04-16T18:15:36.159 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:36.181 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000d72602f0a48199d7-00661ec048-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '151', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.181 DEBUG:botocore.parsers:Response body: b'tx00000d72602f0a48199d7-00661ec048-4804-test-zone1' 2024-04-16T18:15:36.182 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteRole: calling handler 2024-04-16T18:15:36.182 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:36.182 DEBUG:botocore.hooks:Event after-call.iam.DeleteRole: calling handler 2024-04-16T18:15:36.182 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.182 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.182 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:36.223 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:36.223 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:36.293 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:36.327 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:36.327 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.327 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:36.327 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.327 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.327 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:36.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:36.443+0000 7f1dc222d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:36.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:36.447+0000 7f1dc222d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:36.477 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T18:15:36.477 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T18:15:36.477 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:36.477 DEBUG:rgw_multi.tests:log_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.477 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.477 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:36.477 INFO:rgw_multi.tests:checking if zone: test-zone1 does not have role: roles-lybtug-3 2024-04-16T18:15:36.480 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:36.480 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:36.480 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:36.480 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.480 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.480 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-3'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:36.481 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:36.481 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:36.481 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:36.481 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181536Z content-type;host;x-amz-date d3777ae24dcb0b99251a43a20b49ff8964459bed51d9858d33e4f6c619ce5e66 2024-04-16T18:15:36.481 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181536Z 20240416/test-zonegroup/iam/aws4_request cac3a1eaf9b088f7656073bebb622cfb8d4d6495cf70fb8f13ebe0dc6553da93 2024-04-16T18:15:36.481 DEBUG:botocore.auth:Signature: fb31c6f6cd56867c9b51da4031f28fe026d17faba66d38b2490c9d3266c2b4ad 2024-04-16T18:15:36.481 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:36.482 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:36.486 DEBUG:botocore.parsers:Response headers: {'Content-Length': '328', 'x-amz-request-id': 'tx000005d91a041911b7fa7-00661ec048-4804-test-zone1', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.486 DEBUG:botocore.parsers:Response body: b'NoSuchEntityNo such RoleName in the tenanttx000005d91a041911b7fa7-00661ec048-4804-test-zone14804-test-zone1-test-zonegroup' 2024-04-16T18:15:36.486 DEBUG:botocore.parsers:Response headers: {'Content-Length': '328', 'x-amz-request-id': 'tx000005d91a041911b7fa7-00661ec048-4804-test-zone1', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.486 DEBUG:botocore.parsers:Response body: b'NoSuchEntityNo such RoleName in the tenanttx000005d91a041911b7fa7-00661ec048-4804-test-zone14804-test-zone1-test-zonegroup' 2024-04-16T18:15:36.487 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:36.487 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:36.487 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:36.487 INFO:rgw_multi.tests:success, zone: test-zone1 does not have role: roles-lybtug-3 2024-04-16T18:15:36.487 INFO:rgw_multi.tests:checking if zone: test-zone2 does not have role: roles-lybtug-3 2024-04-16T18:15:36.487 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:36.487 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:36.488 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:36.488 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.488 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:36.488 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-3'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:36.488 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:36.488 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:36.489 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:36.489 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181536Z content-type;host;x-amz-date d3777ae24dcb0b99251a43a20b49ff8964459bed51d9858d33e4f6c619ce5e66 2024-04-16T18:15:36.489 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181536Z 20240416/test-zonegroup/iam/aws4_request 61e97da91b96254de44a921f6f14820861b1dae5cad70b08bd7c294b6b5fbe1b 2024-04-16T18:15:36.489 DEBUG:botocore.auth:Signature: 48ea5601fd10a773437a7545d9d2b5512a33388993cfd4bcced0b2480446a9df 2024-04-16T18:15:36.489 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:36.489 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:36.492 DEBUG:botocore.parsers:Response headers: {'Content-Length': '328', 'x-amz-request-id': 'tx000005ba93ede9a19ae14-00661ec048-5397-test-zone2', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.492 DEBUG:botocore.parsers:Response body: b'NoSuchEntityNo such RoleName in the tenanttx000005ba93ede9a19ae14-00661ec048-5397-test-zone25397-test-zone2-test-zonegroup' 2024-04-16T18:15:36.494 DEBUG:botocore.parsers:Response headers: {'Content-Length': '328', 'x-amz-request-id': 'tx000005ba93ede9a19ae14-00661ec048-5397-test-zone2', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:36 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:36.494 DEBUG:botocore.parsers:Response body: b'NoSuchEntityNo such RoleName in the tenanttx000005ba93ede9a19ae14-00661ec048-5397-test-zone25397-test-zone2-test-zonegroup' 2024-04-16T18:15:36.494 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:36.494 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:36.495 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:36.495 INFO:rgw_multi.tests:success, zone: test-zone2 does not have role: roles-lybtug-3 2024-04-16T18:15:36.495 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_role_delete_sync ... ok 2024-04-16T18:15:36.495 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.496 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:36.607 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:36.639 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:36.640 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.640 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:36.640 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.640 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:36.640 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:36.683 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:36.684 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:36.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:36.755+0000 7f986d511980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:36.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:36.759+0000 7f986d511980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:36.787 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T18:15:36.787 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T18:15:36.787 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:36.787 DEBUG:rgw_multi.tests:log_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.787 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:36.787 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:41.789 INFO:rgw_multi.tests:create topic zone=test-zone1 name=topic-lybtug-1 2024-04-16T18:15:41.790 DEBUG:botocore.hooks:Event before-parameter-build.sns.CreateTopic: calling handler 2024-04-16T18:15:41.791 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:41.791 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:41.791 DEBUG:botocore.hooks:Event before-call.sns.CreateTopic: calling handler 2024-04-16T18:15:41.792 DEBUG:botocore.hooks:Event before-call.sns.CreateTopic: calling handler 2024-04-16T18:15:41.792 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'topic-lybtug-1', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'http://kaboom:9999', 'Attributes.entry.2.key': 'persistent', 'Attributes.entry.2.value': 'true'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:41.792 DEBUG:botocore.hooks:Event request-created.sns.CreateTopic: calling handler > 2024-04-16T18:15:41.793 DEBUG:botocore.hooks:Event choose-signer.sns.CreateTopic: calling handler 2024-04-16T18:15:41.793 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:41.793 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181541Z content-type;host;x-amz-date 6c1d177bfdd7e7dab29cb103da0c6e232c1871d8a55783668dd7f45c9a0b19fe 2024-04-16T18:15:41.794 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181541Z 20240416/test-zonegroup/sns/aws4_request 55362b45b03625ff9d7e750b901af6938205a68622f0f50ab5f5ef31a78c7c1f 2024-04-16T18:15:41.794 DEBUG:botocore.auth:Signature: c3d20d0790720573331d33d6ecdb5aea941c86d3dcf2f640835da70e864f7bf4 2024-04-16T18:15:41.794 DEBUG:botocore.hooks:Event request-created.sns.CreateTopic: calling handler 2024-04-16T18:15:41.794 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:41.821 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dd0f74954f23a6f4-00661ec04d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '337', 'Date': 'Tue, 16 Apr 2024 18:15:41 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:41.821 DEBUG:botocore.parsers:Response body: b'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1054102a3-8d94-4596-8b3b-6341eab312e1.4804.15929078591659419380' 2024-04-16T18:15:41.821 DEBUG:botocore.hooks:Event needs-retry.sns.CreateTopic: calling handler 2024-04-16T18:15:41.822 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:41.822 INFO:rgw_multi.tests:create topic zone=test-zone2 name=topic-lybtug-2 2024-04-16T18:15:41.822 DEBUG:botocore.hooks:Event before-parameter-build.sns.CreateTopic: calling handler 2024-04-16T18:15:41.823 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:41.823 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:41.823 DEBUG:botocore.hooks:Event before-call.sns.CreateTopic: calling handler 2024-04-16T18:15:41.823 DEBUG:botocore.hooks:Event before-call.sns.CreateTopic: calling handler 2024-04-16T18:15:41.823 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'topic-lybtug-2', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'http://kaboom:9999', 'Attributes.entry.2.key': 'persistent', 'Attributes.entry.2.value': 'true'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:41.824 DEBUG:botocore.hooks:Event request-created.sns.CreateTopic: calling handler > 2024-04-16T18:15:41.824 DEBUG:botocore.hooks:Event choose-signer.sns.CreateTopic: calling handler 2024-04-16T18:15:41.824 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:41.825 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181541Z content-type;host;x-amz-date 6774c3c7b156ceb70c08415911657cb1e511c662663d2f736ba5e37479485e4a 2024-04-16T18:15:41.825 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181541Z 20240416/test-zonegroup/sns/aws4_request 7e4f2589375c36f2378655630022d45f32b1a3bc382b5aee3e82c0b3e591acac 2024-04-16T18:15:41.825 DEBUG:botocore.auth:Signature: ed878a170108552237d6bde4dace52b6a9bc7bc31c1b143dd50a35cb8f382050 2024-04-16T18:15:41.825 DEBUG:botocore.hooks:Event request-created.sns.CreateTopic: calling handler 2024-04-16T18:15:41.825 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:41.868 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000eeaa0cec01c84729-00661ec04d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '337', 'Date': 'Tue, 16 Apr 2024 18:15:41 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:41.869 DEBUG:botocore.parsers:Response body: b'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2bd830684-b604-4841-ab52-8c82ee812281.5397.17197572334849378089' 2024-04-16T18:15:41.869 DEBUG:botocore.hooks:Event needs-retry.sns.CreateTopic: calling handler 2024-04-16T18:15:41.869 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:41.870 DEBUG:rgw_multi.tests:topic_arns: ['arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1', 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2'] 2024-04-16T18:15:41.870 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:41.870 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:41.870 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:41.911 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:41.911 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:41.981 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:42.017 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713291341.856004_360.1","last_update":"2024-04-16T18:15:41.856004Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:42.017 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:42.017 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:42.017 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:42.018 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:42.018 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:42.061 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:42.061 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:42.135+0000 7f5b3a4e7980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:42.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:42.135+0000 7f5b3a4e7980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:42.165 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T18:15:42.165 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T18:15:42.166 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291341.856004_360.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:41.856004Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:42.166 DEBUG:rgw_multi.tests:log_status={0: '1_1713291176.246608_485.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:42.166 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:42.166 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:42.167 DEBUG:botocore.hooks:Event before-parameter-build.sns.ListTopics: calling handler 2024-04-16T18:15:42.167 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:42.167 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:42.167 DEBUG:botocore.hooks:Event before-call.sns.ListTopics: calling handler 2024-04-16T18:15:42.168 DEBUG:botocore.hooks:Event before-call.sns.ListTopics: calling handler 2024-04-16T18:15:42.168 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListTopics) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListTopics', 'Version': '2010-03-31'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:42.168 DEBUG:botocore.hooks:Event request-created.sns.ListTopics: calling handler > 2024-04-16T18:15:42.168 DEBUG:botocore.hooks:Event choose-signer.sns.ListTopics: calling handler 2024-04-16T18:15:42.169 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:42.169 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181542Z content-type;host;x-amz-date 55e772bd3d832d8507965af84ab07fd17d9be3cbb788151ceb0d7fc2be442a54 2024-04-16T18:15:42.169 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181542Z 20240416/test-zonegroup/sns/aws4_request 08169e34f610bd48de5ab8908a2e55f23f608b87dc9166dc2494236a8fbb4909 2024-04-16T18:15:42.169 DEBUG:botocore.auth:Signature: f4c58d1fa15b8be2a932b3685ab7f4cbdf8f71ba5aea0e99cf92b35d615bc2ce 2024-04-16T18:15:42.170 DEBUG:botocore.hooks:Event request-created.sns.ListTopics: calling handler 2024-04-16T18:15:42.170 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:42.174 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000d6fd106ba491703b-00661ec04e-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1415', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:42.174 DEBUG:botocore.parsers:Response body: b'RGW11111111111111111topic-lybtug-1http://kaboom:9999Version=2010-03-31topic-lybtug-1FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1RGW11111111111111111topic-lybtug-2http://kaboom:9999Version=2010-03-31&rgwx-uid=RGW11111111111111111&rgwx-zonegroup=test-zonegrouptopic-lybtug-2FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2054102a3-8d94-4596-8b3b-6341eab312e1.4804.15491556347732914235' 2024-04-16T18:15:42.175 DEBUG:botocore.hooks:Event needs-retry.sns.ListTopics: calling handler 2024-04-16T18:15:42.175 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:42.175 DEBUG:rgw_multi.tests:topics for zone=test-zone1 = [{'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1'}, {'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2'}] 2024-04-16T18:15:42.175 DEBUG:botocore.hooks:Event before-parameter-build.sns.ListTopics: calling handler 2024-04-16T18:15:42.176 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:42.176 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:42.176 DEBUG:botocore.hooks:Event before-call.sns.ListTopics: calling handler 2024-04-16T18:15:42.176 DEBUG:botocore.hooks:Event before-call.sns.ListTopics: calling handler 2024-04-16T18:15:42.176 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListTopics) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListTopics', 'Version': '2010-03-31'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:42.177 DEBUG:botocore.hooks:Event request-created.sns.ListTopics: calling handler > 2024-04-16T18:15:42.177 DEBUG:botocore.hooks:Event choose-signer.sns.ListTopics: calling handler 2024-04-16T18:15:42.177 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:42.178 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181542Z content-type;host;x-amz-date 55e772bd3d832d8507965af84ab07fd17d9be3cbb788151ceb0d7fc2be442a54 2024-04-16T18:15:42.178 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181542Z 20240416/test-zonegroup/sns/aws4_request d88226fa1ea80c48fb7710f4ea147582ec6b3a4ef1b34e2c496b9a7f3d692c61 2024-04-16T18:15:42.178 DEBUG:botocore.auth:Signature: 4f274ce1b9862c6bdd52f5b5c3dc6d78567c0a9c7870c0ee80bf91a1ea0d9113 2024-04-16T18:15:42.178 DEBUG:botocore.hooks:Event request-created.sns.ListTopics: calling handler 2024-04-16T18:15:42.178 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:42.181 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000008006102a1c087900-00661ec04e-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1414', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:42.182 DEBUG:botocore.parsers:Response body: b'RGW11111111111111111topic-lybtug-1http://kaboom:9999Version=2010-03-31topic-lybtug-1FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1RGW11111111111111111topic-lybtug-2http://kaboom:9999Version=2010-03-31&rgwx-uid=RGW11111111111111111&rgwx-zonegroup=test-zonegrouptopic-lybtug-2FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2bd830684-b604-4841-ab52-8c82ee812281.5397.9225078659760027904' 2024-04-16T18:15:42.182 DEBUG:botocore.hooks:Event needs-retry.sns.ListTopics: calling handler 2024-04-16T18:15:42.182 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:42.182 DEBUG:rgw_multi.tests:topics for zone=test-zone2 = [{'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1'}, {'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2'}] 2024-04-16T18:15:42.183 DEBUG:boto:path=/lybtug-49/ 2024-04-16T18:15:42.183 DEBUG:boto:auth_path=/lybtug-49/ 2024-04-16T18:15:42.183 DEBUG:boto:Method: PUT 2024-04-16T18:15:42.183 DEBUG:boto:Path: /lybtug-49/ 2024-04-16T18:15:42.183 DEBUG:boto:Data: 2024-04-16T18:15:42.183 DEBUG:boto:Headers: {} 2024-04-16T18:15:42.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:42.183 DEBUG:boto:Port: 8000 2024-04-16T18:15:42.183 DEBUG:boto:Params: {} 2024-04-16T18:15:42.184 DEBUG:boto:Token: None 2024-04-16T18:15:42.184 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:15:42 GMT /lybtug-49/ 2024-04-16T18:15:42.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ogbg+rIpjzNXT51rB2PzI3GYhsY= 2024-04-16T18:15:42.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ogbg+rIpjzNXT51rB2PzI3GYhsY=', 'Content-Length': '0'} 2024-04-16T18:15:42.220 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000002c3f7561436c558-00661ec04e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:42.220 DEBUG:rgw_multi.tests:created bucket=lybtug-49 2024-04-16T18:15:42.221 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:42.221 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:42.221 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:42.262 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:42.263 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:42.332 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:42.367 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291342.220251_500.1","last_update":"2024-04-16T18:15:42.220251Z"},{"marker":"1_1713291341.856004_360.1","last_update":"2024-04-16T18:15:41.856004Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:42.368 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291342.220251_500.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:42.368 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:42.368 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:42.368 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:42.368 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:42.412 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:42.412 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:42.483+0000 7f8351580980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:42.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:42.487+0000 7f8351580980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:42.516 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291342.220251_500.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:42.220251Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T18:15:42.516 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291342.220251_500.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:42.220251Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T18:15:42.517 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291342.220251_500.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:42.220251Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291341.856004_360.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:41.856004Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:42.517 DEBUG:rgw_multi.tests:log_status={0: '1_1713291342.220251_500.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:42.517 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291342.220251_500.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:42.517 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:42.517 INFO:rgw_multi.tests:creating bucket notification for zone=test-zone1 name=bn-lybtug-1 2024-04-16T18:15:42.518 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.518 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.518 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.518 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.518 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.518 DEBUG:botocore.hooks:Event before-endpoint-resolution.s3: calling handler 2024-04-16T18:15:42.519 DEBUG:botocore.hooks:Event before-endpoint-resolution.s3: calling handler > 2024-04-16T18:15:42.519 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Bucket': 'lybtug-49', 'Region': 'test-zonegroup', 'UseFIPS': False, 'UseDualStack': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000', 'ForcePathStyle': True, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True, 'UseS3ExpressControlEndpoint': True} 2024-04-16T18:15:42.520 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000/lybtug-49 2024-04-16T18:15:42.520 DEBUG:botocore.regions:Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None" 2024-04-16T18:15:42.520 DEBUG:botocore.regions:Selected auth type "v4" as "v4" with signing context params: {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True} 2024-04-16T18:15:42.522 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.522 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.522 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.522 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.522 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutBucketNotificationConfiguration) with params: {'url_path': '?notification', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': b'bn-lybtug-1arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1s3:ObjectCreated:*', 'auth_path': '/lybtug-49/', 'url': 'http://smithi071.front.sepia.ceph.com:8000/lybtug-49?notification', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': 'v4', 's3_redirect': {'redirected': False, 'bucket': 'lybtug-49', 'params': {'Bucket': 'lybtug-49', 'NotificationConfiguration': {'TopicConfigurations': [{'Id': 'bn-lybtug-1', 'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1', 'Events': ['s3:ObjectCreated:*']}]}}}, 'S3Express': {'bucket_name': 'lybtug-49'}, 'signing': {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'test-zonegroup'}]}}} 2024-04-16T18:15:42.523 DEBUG:botocore.hooks:Event request-created.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.523 DEBUG:botocore.hooks:Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.523 DEBUG:botocore.hooks:Event before-sign.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.523 DEBUG:botocore.hooks:Event before-sign.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.524 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:42.524 DEBUG:botocore.auth:CanonicalRequest: PUT /lybtug-49 notification= host:smithi071.front.sepia.ceph.com:8000 x-amz-content-sha256:5e41bdd252e04b646cefb30bc3c736f83ae11b3ef056e4074db0a6801d176148 x-amz-date:20240416T181542Z host;x-amz-content-sha256;x-amz-date 5e41bdd252e04b646cefb30bc3c736f83ae11b3ef056e4074db0a6801d176148 2024-04-16T18:15:42.524 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181542Z 20240416/test-zonegroup/s3/aws4_request 316cba3cf9ca68e2e6debbd9a10452564dde96e82032104c6cad78f91e089ff6 2024-04-16T18:15:42.524 DEBUG:botocore.auth:Signature: 5211ece45ad4ec8557eeab0330a0275e93cab0872f22b6eac67639e61ab7d442 2024-04-16T18:15:42.524 DEBUG:botocore.hooks:Event request-created.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.524 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:42.545 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b7c26786192e4869-00661ec04e-4804-test-zone1', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '0', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:42.545 DEBUG:botocore.parsers:Response body: b'' 2024-04-16T18:15:42.546 DEBUG:botocore.hooks:Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.546 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:42.546 DEBUG:botocore.hooks:Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.546 INFO:rgw_multi.tests:creating bucket notification for zone=test-zone2 name=bn-lybtug-2 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-endpoint-resolution.s3: calling handler 2024-04-16T18:15:42.547 DEBUG:botocore.hooks:Event before-endpoint-resolution.s3: calling handler > 2024-04-16T18:15:42.548 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Bucket': 'lybtug-49', 'Region': 'test-zonegroup', 'UseFIPS': False, 'UseDualStack': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000', 'ForcePathStyle': True, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True, 'UseS3ExpressControlEndpoint': True} 2024-04-16T18:15:42.548 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000/lybtug-49 2024-04-16T18:15:42.548 DEBUG:botocore.regions:Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None" 2024-04-16T18:15:42.548 DEBUG:botocore.regions:Selected auth type "v4" as "v4" with signing context params: {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True} 2024-04-16T18:15:42.551 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.552 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.552 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.552 DEBUG:botocore.hooks:Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.552 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutBucketNotificationConfiguration) with params: {'url_path': '?notification', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': b'bn-lybtug-2arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2s3:ObjectCreated:*', 'auth_path': '/lybtug-49/', 'url': 'http://smithi122.front.sepia.ceph.com:8000/lybtug-49?notification', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': 'v4', 's3_redirect': {'redirected': False, 'bucket': 'lybtug-49', 'params': {'Bucket': 'lybtug-49', 'NotificationConfiguration': {'TopicConfigurations': [{'Id': 'bn-lybtug-2', 'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2', 'Events': ['s3:ObjectCreated:*']}]}}}, 'S3Express': {'bucket_name': 'lybtug-49'}, 'signing': {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'test-zonegroup'}]}}} 2024-04-16T18:15:42.552 DEBUG:botocore.hooks:Event request-created.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.552 DEBUG:botocore.hooks:Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.553 DEBUG:botocore.hooks:Event before-sign.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.553 DEBUG:botocore.hooks:Event before-sign.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:42.553 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:42.553 DEBUG:botocore.auth:CanonicalRequest: PUT /lybtug-49 notification= host:smithi122.front.sepia.ceph.com:8000 x-amz-content-sha256:c629530fae5787b7ee540d6c6306e59500adcac685bdbaa3124682d4eb30f587 x-amz-date:20240416T181542Z host;x-amz-content-sha256;x-amz-date c629530fae5787b7ee540d6c6306e59500adcac685bdbaa3124682d4eb30f587 2024-04-16T18:15:42.553 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181542Z 20240416/test-zonegroup/s3/aws4_request 01ed2c74f221e494d438ab0286aa9e81f2634244eb9f18cecd30358d1902b019 2024-04-16T18:15:42.553 DEBUG:botocore.auth:Signature: 2d4e32af45ef0d0395fcc373d1b315e90b02a84776aee15d84b887165d1d8f0a 2024-04-16T18:15:42.554 DEBUG:botocore.hooks:Event request-created.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:42.554 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.580 DEBUG:botocore.parsers:Response headers: {'Content-Length': '261', 'x-amz-request-id': 'tx00000f31bf77b15bd460e-00661ec04e-5397-test-zone2', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.581 DEBUG:botocore.parsers:Response body: b'ConcurrentModificationlybtug-49tx00000f31bf77b15bd460e-00661ec04e-5397-test-zone25397-test-zone2-test-zonegroup' 2024-04-16T18:15:43.584 DEBUG:botocore.hooks:Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T18:15:43.584 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.584 DEBUG:botocore.hooks:Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T18:15:43.586 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_topic_notification_sync ... ERROR 2024-04-16T18:15:43.587 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateUser: calling handler 2024-04-16T18:15:43.588 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.588 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.588 DEBUG:botocore.hooks:Event before-call.iam.CreateUser: calling handler 2024-04-16T18:15:43.589 DEBUG:botocore.hooks:Event before-call.iam.CreateUser: calling handler 2024-04-16T18:15:43.589 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateUser', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.589 DEBUG:botocore.hooks:Event request-created.iam.CreateUser: calling handler > 2024-04-16T18:15:43.589 DEBUG:botocore.hooks:Event choose-signer.iam.CreateUser: calling handler 2024-04-16T18:15:43.590 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.590 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date f7475b30454e8ffcb361ce7c694a8facda3014395dff383d48edb4ca989ef6b0 2024-04-16T18:15:43.590 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request c843c933768f4e1250d83034ea4520a55787244ec3e5dff01421bb00084414b9 2024-04-16T18:15:43.591 DEBUG:botocore.auth:Signature: 17e606006cd2c499c9f15fd088031004806b431a40b90732a2211893ad406af3 2024-04-16T18:15:43.591 DEBUG:botocore.hooks:Event request-created.iam.CreateUser: calling handler 2024-04-16T18:15:43.591 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.612 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a623a314c2c35907-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '499', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.612 DEBUG:botocore.parsers:Response body: b'/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone12024-04-16T18:15:43.601674Ztx00000a623a314c2c35907-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.613 DEBUG:botocore.hooks:Event needs-retry.iam.CreateUser: calling handler 2024-04-16T18:15:43.613 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.614 DEBUG:botocore.hooks:Event after-call.iam.CreateUser: calling handler 2024-04-16T18:15:43.614 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.614 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.615 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.615 DEBUG:botocore.hooks:Event before-call.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.615 DEBUG:botocore.hooks:Event before-call.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.615 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateAccessKey) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateAccessKey', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.616 DEBUG:botocore.hooks:Event request-created.iam.CreateAccessKey: calling handler > 2024-04-16T18:15:43.616 DEBUG:botocore.hooks:Event choose-signer.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.616 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.616 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 2d56745a01c086ed0203077dd4eef7bcaf00472bb3389da335fbf28f3732fcd0 2024-04-16T18:15:43.617 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request fd5c27875fc4e1435d418596cfb7755ac9b313fbfb4767cbc49f889e4f84d8f1 2024-04-16T18:15:43.617 DEBUG:botocore.auth:Signature: 75f719ac421225a209b49c74665d6c9f750cf7efa69a12bfe0f1e3c97376398a 2024-04-16T18:15:43.617 DEBUG:botocore.hooks:Event request-created.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.617 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.639 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000050abad302b40f937-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '547', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.640 DEBUG:botocore.parsers:Response body: b'test-zone183LIGVONPAK8I8ZAT2C6Active2024-04-16T18:15:43.627747Zs53QDXCDFv4SG1K2MyxJWkGiCrwBq7UGfFgOQrHBtx0000050abad302b40f937-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.640 DEBUG:botocore.hooks:Event needs-retry.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.640 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.641 DEBUG:botocore.hooks:Event after-call.iam.CreateAccessKey: calling handler 2024-04-16T18:15:43.641 DEBUG:botocore.hooks:Event before-parameter-build.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.641 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.641 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.642 DEBUG:botocore.hooks:Event before-call.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.642 DEBUG:botocore.hooks:Event before-call.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.642 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'PutUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone1', 'PolicyName': 'Allow', 'PolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.642 DEBUG:botocore.hooks:Event request-created.iam.PutUserPolicy: calling handler > 2024-04-16T18:15:43.643 DEBUG:botocore.hooks:Event choose-signer.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.643 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.643 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 17879975ab2692d579dfbda99a028060b29570530de8c9f87e064005597017ac 2024-04-16T18:15:43.643 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request a6904fbce9cf523c06fa31f5da0e8dc445ce9db892685085058739cce44f4023 2024-04-16T18:15:43.644 DEBUG:botocore.auth:Signature: 14eb99f9a8b70f0d30e8a0ca6161aab4ec56d3405c6ad894ad78375aa25786f8 2024-04-16T18:15:43.644 DEBUG:botocore.hooks:Event request-created.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.644 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.710 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b86b2adea5ed4bd1-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '207', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.710 DEBUG:botocore.parsers:Response body: b'tx00000b86b2adea5ed4bd1-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.711 DEBUG:botocore.hooks:Event needs-retry.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.711 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.711 DEBUG:botocore.hooks:Event after-call.iam.PutUserPolicy: calling handler 2024-04-16T18:15:43.712 DEBUG:botocore.hooks:Event before-parameter-build.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.712 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.712 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.712 DEBUG:botocore.hooks:Event before-call.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.713 DEBUG:botocore.hooks:Event before-call.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.713 DEBUG:botocore.endpoint:Making request for OperationModel(name=AttachUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AttachUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone1', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.713 DEBUG:botocore.hooks:Event request-created.iam.AttachUserPolicy: calling handler > 2024-04-16T18:15:43.713 DEBUG:botocore.hooks:Event choose-signer.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.714 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.714 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 6bae6a67a9faf220198a4254a71b23780816be124b7a0f3c4c5492c739caeaa2 2024-04-16T18:15:43.714 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 038d9d708eb18223e55faf72672ed618064073fdf4b4cad8e8204ca410946480 2024-04-16T18:15:43.714 DEBUG:botocore.auth:Signature: f95bb70be83d55069273fe7c063b328c1714b4b79532dbba04a74c721fb6cae6 2024-04-16T18:15:43.714 DEBUG:botocore.hooks:Event request-created.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.715 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.738 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000399bd28863c77358-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.738 DEBUG:botocore.parsers:Response body: b'tx00000399bd28863c77358-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.739 DEBUG:botocore.hooks:Event needs-retry.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.739 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.739 DEBUG:botocore.hooks:Event after-call.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:43.739 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateGroup: calling handler 2024-04-16T18:15:43.740 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.740 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.740 DEBUG:botocore.hooks:Event before-call.iam.CreateGroup: calling handler 2024-04-16T18:15:43.740 DEBUG:botocore.hooks:Event before-call.iam.CreateGroup: calling handler 2024-04-16T18:15:43.741 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.741 DEBUG:botocore.hooks:Event request-created.iam.CreateGroup: calling handler > 2024-04-16T18:15:43.741 DEBUG:botocore.hooks:Event choose-signer.iam.CreateGroup: calling handler 2024-04-16T18:15:43.742 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.742 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 624e928373ac8004a209ec47e507ab54a865f60c2f07522729232344f4ba6c8a 2024-04-16T18:15:43.742 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 7ed5adb47e142143da4900dff98f06115232551beb15aa132c54696191724911 2024-04-16T18:15:43.742 DEBUG:botocore.auth:Signature: 333899822ec2ff319342df4d2a9657c4f4fcbda3e1005887888ade365b9e9745 2024-04-16T18:15:43.742 DEBUG:botocore.hooks:Event request-created.iam.CreateGroup: calling handler 2024-04-16T18:15:43.743 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.763 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006e207174ef716406-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '458', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.763 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1tx000006e207174ef716406-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.763 DEBUG:botocore.hooks:Event needs-retry.iam.CreateGroup: calling handler 2024-04-16T18:15:43.764 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.764 DEBUG:botocore.hooks:Event after-call.iam.CreateGroup: calling handler 2024-04-16T18:15:43.764 DEBUG:botocore.hooks:Event before-parameter-build.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.764 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.765 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.765 DEBUG:botocore.hooks:Event before-call.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.765 DEBUG:botocore.hooks:Event before-call.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.765 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'PutGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone1', 'PolicyName': 'Allow', 'PolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.766 DEBUG:botocore.hooks:Event request-created.iam.PutGroupPolicy: calling handler > 2024-04-16T18:15:43.766 DEBUG:botocore.hooks:Event choose-signer.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.766 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.767 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date ad4c82339605c3b58275f685e51aaa8f175f9d7bfa75f418f3b90f7c7b4bf620 2024-04-16T18:15:43.767 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 4edb8dcc6f865988369df622dca11eac58d90ee0c069436e8d1b22dc8d231e51 2024-04-16T18:15:43.767 DEBUG:botocore.auth:Signature: a71f318c9e1ec5a90889e4424ccaab71b8db7ded926c6bd3ca550f8b881b457b 2024-04-16T18:15:43.767 DEBUG:botocore.hooks:Event request-created.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.767 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.822 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000079248d91b90466b5-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '247', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.823 DEBUG:botocore.parsers:Response body: b'tx0000079248d91b90466b5-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.823 DEBUG:botocore.hooks:Event needs-retry.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.823 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.823 DEBUG:botocore.hooks:Event after-call.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:43.824 DEBUG:botocore.hooks:Event before-parameter-build.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.824 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.824 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.824 DEBUG:botocore.hooks:Event before-call.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.825 DEBUG:botocore.hooks:Event before-call.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.825 DEBUG:botocore.endpoint:Making request for OperationModel(name=AttachGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AttachGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone1', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.825 DEBUG:botocore.hooks:Event request-created.iam.AttachGroupPolicy: calling handler > 2024-04-16T18:15:43.825 DEBUG:botocore.hooks:Event choose-signer.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.826 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.826 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 6650eb41f571a51390e5e4d8fe1d82748ac41b679798f9d70d71a7c097ba721b 2024-04-16T18:15:43.826 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 66210f30c775a60b41a0248e281f3d1b1fe60c5f74bb7bbf82d93e84946b2368 2024-04-16T18:15:43.826 DEBUG:botocore.auth:Signature: 4e9ce532d766cc167299d2541b1df792aa3c5ce939a2645a90685475cb488793 2024-04-16T18:15:43.826 DEBUG:botocore.hooks:Event request-created.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.827 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.839 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000004810e489f9d5fe68-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '253', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.839 DEBUG:botocore.parsers:Response body: b'tx000004810e489f9d5fe68-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.839 DEBUG:botocore.hooks:Event needs-retry.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.840 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.840 DEBUG:botocore.hooks:Event after-call.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:43.840 DEBUG:botocore.hooks:Event before-parameter-build.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.840 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.841 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.841 DEBUG:botocore.hooks:Event before-call.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.841 DEBUG:botocore.hooks:Event before-call.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.841 DEBUG:botocore.endpoint:Making request for OperationModel(name=AddUserToGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AddUserToGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone1', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.842 DEBUG:botocore.hooks:Event request-created.iam.AddUserToGroup: calling handler > 2024-04-16T18:15:43.842 DEBUG:botocore.hooks:Event choose-signer.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.842 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.842 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date b828f4a28c8d0ee119b54a503ae785618cc0a7bd2f1ba220c61d60c88189b9de 2024-04-16T18:15:43.843 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 0ad5311d52bf79590dc9a3257a0d526ebcb81a9e88b1a82e455fa4b20280bd4f 2024-04-16T18:15:43.843 DEBUG:botocore.auth:Signature: bc26c30856eef0b809016afee17f8e7fdcee32ecf0197702d3a86a2189fb686c 2024-04-16T18:15:43.843 DEBUG:botocore.hooks:Event request-created.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.843 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.861 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000003a7c9055a5de7a26-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '247', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.861 DEBUG:botocore.parsers:Response body: b'tx000003a7c9055a5de7a26-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.862 DEBUG:botocore.hooks:Event needs-retry.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.862 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.862 DEBUG:botocore.hooks:Event after-call.iam.AddUserToGroup: calling handler 2024-04-16T18:15:43.863 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateRole: calling handler 2024-04-16T18:15:43.863 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.863 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.863 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:43.863 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:43.863 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateRole', 'Version': '2010-05-08', 'RoleName': 'test-zone1', 'AssumeRolePolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.864 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler > 2024-04-16T18:15:43.864 DEBUG:botocore.hooks:Event choose-signer.iam.CreateRole: calling handler 2024-04-16T18:15:43.864 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.864 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date b8dfbdc932156a00bb2505dfd736bc41e7bb2b32934a250726a1d2ae8ac7126c 2024-04-16T18:15:43.865 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 37d1db9fd95666dda058b5df43e1d8271d185b8de225ea2a0abdda33cc1fb2d9 2024-04-16T18:15:43.865 DEBUG:botocore.auth:Signature: 134c065b899ddc45f2b4c11e56a94dbbea9a07ff35c6cf004004cc1317e11f19 2024-04-16T18:15:43.865 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler 2024-04-16T18:15:43.865 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.930 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000234d5794482cb449-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '777', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.931 DEBUG:botocore.parsers:Response body: b'7bf002ca-877a-41e4-ad14-3028bd7fdc7ftest-zone1/arn:aws:iam::RGW11111111111111111:role/test-zone12024-04-16T18:15:43.875Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}tx00000234d5794482cb449-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.931 DEBUG:botocore.hooks:Event needs-retry.iam.CreateRole: calling handler 2024-04-16T18:15:43.931 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.932 DEBUG:botocore.hooks:Event after-call.iam.CreateRole: calling handler 2024-04-16T18:15:43.932 DEBUG:botocore.hooks:Event before-parameter-build.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.932 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.932 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.933 DEBUG:botocore.hooks:Event before-call.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.933 DEBUG:botocore.hooks:Event before-call.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.933 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'PutRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone1', 'PolicyName': 'Allow', 'PolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.933 DEBUG:botocore.hooks:Event request-created.iam.PutRolePolicy: calling handler > 2024-04-16T18:15:43.934 DEBUG:botocore.hooks:Event choose-signer.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.934 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.934 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date fbdda2fe5860cad2b2ea0a554f4430d72305dbdc0b7565c6cbd1cba689429a4d 2024-04-16T18:15:43.935 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 4b20f5dcdc34a965ce880c2daef6b826122eb88e465503eca62e725366f053bd 2024-04-16T18:15:43.935 DEBUG:botocore.auth:Signature: 97a285404d3d33dd3009c8fee7813ac2551313b1b7e1c3c713d31c663b48a6e6 2024-04-16T18:15:43.935 DEBUG:botocore.hooks:Event request-created.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.935 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.951 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000000023b9d8d26dc3df-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '157', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.951 DEBUG:botocore.parsers:Response body: b'tx000000023b9d8d26dc3df-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.951 DEBUG:botocore.hooks:Event needs-retry.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.952 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.952 DEBUG:botocore.hooks:Event after-call.iam.PutRolePolicy: calling handler 2024-04-16T18:15:43.952 DEBUG:botocore.hooks:Event before-parameter-build.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.952 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.952 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:43.953 DEBUG:botocore.hooks:Event before-call.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.953 DEBUG:botocore.hooks:Event before-call.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.953 DEBUG:botocore.endpoint:Making request for OperationModel(name=AttachRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AttachRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone1', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.953 DEBUG:botocore.hooks:Event request-created.iam.AttachRolePolicy: calling handler > 2024-04-16T18:15:43.954 DEBUG:botocore.hooks:Event choose-signer.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.954 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.954 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 98ed48432946a0db032c442aa66c08bd0e2e471826a731d2e6807950820ab209 2024-04-16T18:15:43.954 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 96419e77a4344604219863c0b8003fd500a4d8d45f913aadfa0470deef343e1c 2024-04-16T18:15:43.954 DEBUG:botocore.auth:Signature: 7f87f20c8b3922225704ed2b533dea22618f274d73eb06581713f0caaec3be44 2024-04-16T18:15:43.955 DEBUG:botocore.hooks:Event request-created.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.955 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:43.969 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000c2a0829fe540b8ba-00661ec04f-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:43.969 DEBUG:botocore.parsers:Response body: b'tx00000c2a0829fe540b8ba-00661ec04f-4804-test-zone1' 2024-04-16T18:15:43.970 DEBUG:botocore.hooks:Event needs-retry.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.970 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:43.970 DEBUG:botocore.hooks:Event after-call.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:43.971 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateUser: calling handler 2024-04-16T18:15:43.971 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:43.971 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:43.971 DEBUG:botocore.hooks:Event before-call.iam.CreateUser: calling handler 2024-04-16T18:15:43.971 DEBUG:botocore.hooks:Event before-call.iam.CreateUser: calling handler 2024-04-16T18:15:43.972 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateUser', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:43.972 DEBUG:botocore.hooks:Event request-created.iam.CreateUser: calling handler > 2024-04-16T18:15:43.972 DEBUG:botocore.hooks:Event choose-signer.iam.CreateUser: calling handler 2024-04-16T18:15:43.973 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:43.973 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181543Z content-type;host;x-amz-date 2510cb179a00379b5bc09d3a2af02e7e7b61a5cb434f7fa528d9845bf96444f3 2024-04-16T18:15:43.973 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181543Z 20240416/test-zonegroup/iam/aws4_request 48add4fa784dba1612b5f9515f81a75ba6f05d071bb870eecfa2c858d68aef72 2024-04-16T18:15:43.973 DEBUG:botocore.auth:Signature: 2a63c76d9b1f01264680b300c1413921ebe6313a3e6fbbddfba364ed043035a2 2024-04-16T18:15:43.973 DEBUG:botocore.hooks:Event request-created.iam.CreateUser: calling handler 2024-04-16T18:15:43.973 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.003 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000001fbf02116b45efc5-00661ec04f-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '499', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.004 DEBUG:botocore.parsers:Response body: b'/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone22024-04-16T18:15:43.984255Ztx000001fbf02116b45efc5-00661ec04f-5397-test-zone2' 2024-04-16T18:15:44.004 DEBUG:botocore.hooks:Event needs-retry.iam.CreateUser: calling handler 2024-04-16T18:15:44.004 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.005 DEBUG:botocore.hooks:Event after-call.iam.CreateUser: calling handler 2024-04-16T18:15:44.005 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.005 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.005 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.006 DEBUG:botocore.hooks:Event before-call.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.006 DEBUG:botocore.hooks:Event before-call.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.006 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateAccessKey) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateAccessKey', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.007 DEBUG:botocore.hooks:Event request-created.iam.CreateAccessKey: calling handler > 2024-04-16T18:15:44.007 DEBUG:botocore.hooks:Event choose-signer.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.007 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.007 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date a0b5d5b3609514e9ba50d912aefb3a37850c6755f9d7985c4c4bb1c50f4d7f4b 2024-04-16T18:15:44.008 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request 6b3673843b42e94e96f9cf7317da89a664c7e2817cb3e9f8f07f0ea71f696209 2024-04-16T18:15:44.008 DEBUG:botocore.auth:Signature: 5622947e0948a2f4d0cb663300d967c2b7590c489d828a2edcedad3d982c3eec 2024-04-16T18:15:44.008 DEBUG:botocore.hooks:Event request-created.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.008 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.088 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000916e5e5d04333e99-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '547', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.088 DEBUG:botocore.parsers:Response body: b'test-zone261Y92QQ1X31D3HN60KSGActive2024-04-16T18:15:44.023770ZyhZ5qNiKAO7tdheYBlrNL3y2Xv3cSZ0gPmwZ2GqAtx00000916e5e5d04333e99-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.089 DEBUG:botocore.hooks:Event needs-retry.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.089 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.089 DEBUG:botocore.hooks:Event after-call.iam.CreateAccessKey: calling handler 2024-04-16T18:15:44.090 DEBUG:botocore.hooks:Event before-parameter-build.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.090 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.090 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.090 DEBUG:botocore.hooks:Event before-call.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.091 DEBUG:botocore.hooks:Event before-call.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.091 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'PutUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone2', 'PolicyName': 'Allow', 'PolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.091 DEBUG:botocore.hooks:Event request-created.iam.PutUserPolicy: calling handler > 2024-04-16T18:15:44.091 DEBUG:botocore.hooks:Event choose-signer.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.092 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.092 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date 85bc49ee9891ccaf4f94397e9650c15250b819764da6ef0a05cb2450c32e0d43 2024-04-16T18:15:44.092 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request 91e4448f4c5ffbef668e5900e7fdc07e3cf3637f5f9efa0f48cb5d357b399a65 2024-04-16T18:15:44.092 DEBUG:botocore.auth:Signature: 0a81b06c8082e0aa687f66d1ef6377479af3dd9075a0aaad2d18934198ecfb34 2024-04-16T18:15:44.092 DEBUG:botocore.hooks:Event request-created.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.093 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.218 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000002e64bed6f6c7e208-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '207', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.219 DEBUG:botocore.parsers:Response body: b'tx000002e64bed6f6c7e208-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.219 DEBUG:botocore.hooks:Event needs-retry.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.219 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.220 DEBUG:botocore.hooks:Event after-call.iam.PutUserPolicy: calling handler 2024-04-16T18:15:44.220 DEBUG:botocore.hooks:Event before-parameter-build.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.220 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.220 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.221 DEBUG:botocore.hooks:Event before-call.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.221 DEBUG:botocore.hooks:Event before-call.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.221 DEBUG:botocore.endpoint:Making request for OperationModel(name=AttachUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AttachUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone2', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.221 DEBUG:botocore.hooks:Event request-created.iam.AttachUserPolicy: calling handler > 2024-04-16T18:15:44.222 DEBUG:botocore.hooks:Event choose-signer.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.222 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.222 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date 9fa30c027a5dbbf088ce980f0476d5a9a21aa2fdc24be06e3279cba8bcdc470b 2024-04-16T18:15:44.223 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request babe300ccf98446ad1722a0e0736f95f7d62dce4ffd9fea7620f7ad5108f94ea 2024-04-16T18:15:44.223 DEBUG:botocore.auth:Signature: eb43de5794a4c185fa43ef78df494631aa9043c8224c2bc20691ed32c60d0826 2024-04-16T18:15:44.223 DEBUG:botocore.hooks:Event request-created.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.223 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.266 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000008384b50eed051422-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.266 DEBUG:botocore.parsers:Response body: b'tx000008384b50eed051422-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.267 DEBUG:botocore.hooks:Event needs-retry.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.267 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.267 DEBUG:botocore.hooks:Event after-call.iam.AttachUserPolicy: calling handler 2024-04-16T18:15:44.267 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateGroup: calling handler 2024-04-16T18:15:44.268 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.268 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.268 DEBUG:botocore.hooks:Event before-call.iam.CreateGroup: calling handler 2024-04-16T18:15:44.268 DEBUG:botocore.hooks:Event before-call.iam.CreateGroup: calling handler 2024-04-16T18:15:44.268 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.269 DEBUG:botocore.hooks:Event request-created.iam.CreateGroup: calling handler > 2024-04-16T18:15:44.269 DEBUG:botocore.hooks:Event choose-signer.iam.CreateGroup: calling handler 2024-04-16T18:15:44.269 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.270 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date 24a14e11b4493ec41142bbe4ace87bd6f70ba76e166d59411fd5edee5bcde329 2024-04-16T18:15:44.270 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request bffcc9a21c2d1ff1a449781b3f2b276352ce942a27dd637cfea496ae0844f00f 2024-04-16T18:15:44.270 DEBUG:botocore.auth:Signature: a1171655d43f2b3656286b663f9ba73837676ff40ace6c9c4089a694a35a0fd2 2024-04-16T18:15:44.270 DEBUG:botocore.hooks:Event request-created.iam.CreateGroup: calling handler 2024-04-16T18:15:44.271 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.349 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ae8b9592e1e635b8-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '458', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.349 DEBUG:botocore.parsers:Response body: b'/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2tx00000ae8b9592e1e635b8-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.350 DEBUG:botocore.hooks:Event needs-retry.iam.CreateGroup: calling handler 2024-04-16T18:15:44.350 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.350 DEBUG:botocore.hooks:Event after-call.iam.CreateGroup: calling handler 2024-04-16T18:15:44.351 DEBUG:botocore.hooks:Event before-parameter-build.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.351 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.351 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.351 DEBUG:botocore.hooks:Event before-call.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.352 DEBUG:botocore.hooks:Event before-call.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.352 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'PutGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone2', 'PolicyName': 'Allow', 'PolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.352 DEBUG:botocore.hooks:Event request-created.iam.PutGroupPolicy: calling handler > 2024-04-16T18:15:44.352 DEBUG:botocore.hooks:Event choose-signer.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.353 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.353 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date ab7d67234b7569246e67b8e19e7bdcd215174800a0c659b8f2e5f4ac095702b5 2024-04-16T18:15:44.353 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request 6954723e873c1e406aec32e8b37fc2f7f9ee21487889ff2f19603a9bec9a3059 2024-04-16T18:15:44.353 DEBUG:botocore.auth:Signature: 0a70cdc9f33d5c90959f7c1bcd5002d65ed8ac34b2303b9daf24b074e7e744f9 2024-04-16T18:15:44.354 DEBUG:botocore.hooks:Event request-created.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.354 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.458 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000047f5f9342ba0f570-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '247', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.459 DEBUG:botocore.parsers:Response body: b'tx0000047f5f9342ba0f570-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.459 DEBUG:botocore.hooks:Event needs-retry.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.459 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.459 DEBUG:botocore.hooks:Event after-call.iam.PutGroupPolicy: calling handler 2024-04-16T18:15:44.460 DEBUG:botocore.hooks:Event before-parameter-build.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.460 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.460 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.461 DEBUG:botocore.hooks:Event before-call.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.461 DEBUG:botocore.hooks:Event before-call.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.461 DEBUG:botocore.endpoint:Making request for OperationModel(name=AttachGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AttachGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone2', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.461 DEBUG:botocore.hooks:Event request-created.iam.AttachGroupPolicy: calling handler > 2024-04-16T18:15:44.461 DEBUG:botocore.hooks:Event choose-signer.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.462 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.462 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date 08d5ff802492dc9987af66bd4af8637b58974f1703349e3edf09538e4f34ef3a 2024-04-16T18:15:44.462 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request 638e159c8e23e2455064bcffd71bba73d029ba6d3845f39efaf96759bfcbb6c1 2024-04-16T18:15:44.462 DEBUG:botocore.auth:Signature: 7a420ee4f81bcc5baae56d626474343bc0e312ffd7f4a50cf1cd6b509f31f2a6 2024-04-16T18:15:44.462 DEBUG:botocore.hooks:Event request-created.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.463 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.482 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000916b2469dc9d2558-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '253', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.482 DEBUG:botocore.parsers:Response body: b'tx00000916b2469dc9d2558-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.482 DEBUG:botocore.hooks:Event needs-retry.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.483 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.483 DEBUG:botocore.hooks:Event after-call.iam.AttachGroupPolicy: calling handler 2024-04-16T18:15:44.483 DEBUG:botocore.hooks:Event before-parameter-build.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.483 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.483 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.484 DEBUG:botocore.hooks:Event before-call.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.484 DEBUG:botocore.hooks:Event before-call.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.484 DEBUG:botocore.endpoint:Making request for OperationModel(name=AddUserToGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AddUserToGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone2', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.484 DEBUG:botocore.hooks:Event request-created.iam.AddUserToGroup: calling handler > 2024-04-16T18:15:44.485 DEBUG:botocore.hooks:Event choose-signer.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.485 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.485 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date b1a71af0ad1d86e1dce0f1ed3358241dd82bb321ef00b4200a506a73f65811f4 2024-04-16T18:15:44.485 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request 65b0bf93fe495cef75e44ae59c556e3292fbeddfc146ffa90cde4648c0171e29 2024-04-16T18:15:44.485 DEBUG:botocore.auth:Signature: 4cf478c5652de7157dced46da011e9aec4977fa4e1f3fed00ba66d7e5535d8fd 2024-04-16T18:15:44.485 DEBUG:botocore.hooks:Event request-created.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.486 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.559 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000bdfad8f6478baad0-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '247', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.559 DEBUG:botocore.parsers:Response body: b'tx00000bdfad8f6478baad0-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.560 DEBUG:botocore.hooks:Event needs-retry.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.560 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.560 DEBUG:botocore.hooks:Event after-call.iam.AddUserToGroup: calling handler 2024-04-16T18:15:44.560 DEBUG:botocore.hooks:Event before-parameter-build.iam.CreateRole: calling handler 2024-04-16T18:15:44.561 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.561 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.561 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:44.561 DEBUG:botocore.hooks:Event before-call.iam.CreateRole: calling handler 2024-04-16T18:15:44.561 DEBUG:botocore.endpoint:Making request for OperationModel(name=CreateRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateRole', 'Version': '2010-05-08', 'RoleName': 'test-zone2', 'AssumeRolePolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.561 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler > 2024-04-16T18:15:44.562 DEBUG:botocore.hooks:Event choose-signer.iam.CreateRole: calling handler 2024-04-16T18:15:44.562 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.562 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date 3b15dc3cd5d30d4e069c24bb6bdc8bba7e68b341ab2726d98d123919ff657ea8 2024-04-16T18:15:44.562 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request d8989d8d0f69aeb2f4b370ab3d2477183a31f7926ac5e452f6f96835f1acc43f 2024-04-16T18:15:44.562 DEBUG:botocore.auth:Signature: e3f67c8db5cb4dc02e1a543ccd739d94492b35e3e306950a806e5fff8c91d1fd 2024-04-16T18:15:44.563 DEBUG:botocore.hooks:Event request-created.iam.CreateRole: calling handler 2024-04-16T18:15:44.563 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.690 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000cd2a4b36f2ba45d5-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '777', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.691 DEBUG:botocore.parsers:Response body: b'd2a10f29-d910-40b7-a1e5-bf7c2c78b9e5test-zone2/arn:aws:iam::RGW11111111111111111:role/test-zone22024-04-16T18:15:44.576Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}tx00000cd2a4b36f2ba45d5-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.691 DEBUG:botocore.hooks:Event needs-retry.iam.CreateRole: calling handler 2024-04-16T18:15:44.692 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.692 DEBUG:botocore.hooks:Event after-call.iam.CreateRole: calling handler 2024-04-16T18:15:44.692 DEBUG:botocore.hooks:Event before-parameter-build.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.692 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.693 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.693 DEBUG:botocore.hooks:Event before-call.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.693 DEBUG:botocore.hooks:Event before-call.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.693 DEBUG:botocore.endpoint:Making request for OperationModel(name=PutRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'PutRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone2', 'PolicyName': 'Allow', 'PolicyDocument': '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.693 DEBUG:botocore.hooks:Event request-created.iam.PutRolePolicy: calling handler > 2024-04-16T18:15:44.694 DEBUG:botocore.hooks:Event choose-signer.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.694 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.694 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date d375b5364696b79344588bf03ce90d2c993e120cfc9ed210ac5505fa6b6852b7 2024-04-16T18:15:44.694 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request 260c855858267d0fa10e217c33a8552199828a1ba3124ff29bf44ce1528b0334 2024-04-16T18:15:44.695 DEBUG:botocore.auth:Signature: 0e18e900418ed458714c453ca494c0c2a85e2c62ec62a169344aa64d517e9a48 2024-04-16T18:15:44.695 DEBUG:botocore.hooks:Event request-created.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.695 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.718 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006daebcea1f301263-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '157', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.718 DEBUG:botocore.parsers:Response body: b'tx000006daebcea1f301263-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.719 DEBUG:botocore.hooks:Event needs-retry.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.719 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.719 DEBUG:botocore.hooks:Event after-call.iam.PutRolePolicy: calling handler 2024-04-16T18:15:44.719 DEBUG:botocore.hooks:Event before-parameter-build.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.720 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:44.720 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:44.720 DEBUG:botocore.hooks:Event before-call.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.720 DEBUG:botocore.hooks:Event before-call.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.720 DEBUG:botocore.endpoint:Making request for OperationModel(name=AttachRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'AttachRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone2', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:44.721 DEBUG:botocore.hooks:Event request-created.iam.AttachRolePolicy: calling handler > 2024-04-16T18:15:44.721 DEBUG:botocore.hooks:Event choose-signer.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.721 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:44.722 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181544Z content-type;host;x-amz-date 769290fb146152a7afbdb9bd3a33d29951994cc1eb13e718908b74fc21378f03 2024-04-16T18:15:44.722 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181544Z 20240416/test-zonegroup/iam/aws4_request a03311566a0f4fe27c8fa0a200c5f183114df890470962067b76098725010354 2024-04-16T18:15:44.722 DEBUG:botocore.auth:Signature: 31da95b6d356c14574a686d7ccdaa3fea59222954aca8454ec55907ede135fb2 2024-04-16T18:15:44.722 DEBUG:botocore.hooks:Event request-created.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.722 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:44.788 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000001499e32d33bc4a6f-00661ec050-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:44 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:44.788 DEBUG:botocore.parsers:Response body: b'tx000001499e32d33bc4a6f-00661ec050-5397-test-zone2' 2024-04-16T18:15:44.789 DEBUG:botocore.hooks:Event needs-retry.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.789 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:44.789 DEBUG:botocore.hooks:Event after-call.iam.AttachRolePolicy: calling handler 2024-04-16T18:15:44.789 INFO:rgw_multi.tests:meta checkpoint 2024-04-16T18:15:44.789 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:44.789 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:44.790 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:44.832 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:44.832 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:44.935 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291343.860674_514.1","last_update":"2024-04-16T18:15:43.860674Z"},{"marker":"1_1713291344.507657_382.1","last_update":"2024-04-16T18:15:44.507657Z"},{"marker":"1_1713291344.741355_277.1","last_update":"2024-04-16T18:15:44.741355Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:44.936 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291343.860674_514.1', 1: '1_1713291344.507657_382.1', 2: '1_1713291344.741355_277.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:44.936 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:44.936 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:44.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:44.936 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:44.982 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:44.982 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:45.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:45.051+0000 7f8b2b1ac980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:45.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:45.055+0000 7f8b2b1ac980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:45.088 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291343.860674_514.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:43.860674Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291343.986341_370.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:43.986341Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291344.706762_274.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:44.706762Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:45Z"} 2024-04-16T18:15:45.088 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291343.860674_514.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:43.860674Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291343.986341_370.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:43.986341Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291344.706762_274.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:44.706762Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:45Z"} 2024-04-16T18:15:45.089 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291343.860674_514.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:43.860674Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291343.986341_370.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:43.986341Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291344.706762_274.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:44.706762Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:45.089 DEBUG:rgw_multi.tests:log_status={0: '1_1713291343.860674_514.1', 1: '1_1713291344.507657_382.1', 2: '1_1713291344.741355_277.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:45.089 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291343.860674_514.1', 1: '1_1713291343.986341_370.1', 2: '1_1713291344.706762_274.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:45.089 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1713291344.507657_382.1 target=1_1713291343.986341_370.1, shard=2 master=1_1713291344.741355_277.1 target=1_1713291344.706762_274.1 2024-04-16T18:15:50.091 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:50.093 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:50.093 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:50.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:50.211+0000 7f854abcf980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:50.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:50.211+0000 7f854abcf980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:50.242 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291343.860674_514.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:43.860674Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291344.507657_382.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:44.507657Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291344.741355_277.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:44.741355Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:50Z"} 2024-04-16T18:15:50.242 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291343.860674_514.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:43.860674Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291344.507657_382.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:44.507657Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291344.741355_277.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:44.741355Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:50Z"} 2024-04-16T18:15:50.243 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291343.860674_514.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:43.860674Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291344.507657_382.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:44.507657Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291344.741355_277.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:44.741355Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:50.243 DEBUG:rgw_multi.tests:log_status={0: '1_1713291343.860674_514.1', 1: '1_1713291344.507657_382.1', 2: '1_1713291344.741355_277.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:50.243 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291343.860674_514.1', 1: '1_1713291344.507657_382.1', 2: '1_1713291344.741355_277.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:50.243 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:50.243 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRoles: calling handler 2024-04-16T18:15:50.244 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.244 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.244 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:50.244 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:50.244 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRoles) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRoles', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.245 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler > 2024-04-16T18:15:50.245 DEBUG:botocore.hooks:Event choose-signer.iam.ListRoles: calling handler 2024-04-16T18:15:50.245 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.245 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 8e689dc6b69b17003f0460011582254d8643f0dfb48ac46e093a24df58170b3f 2024-04-16T18:15:50.245 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request b86a63d58be6ec05fe6dc7eee5be778b5f5c4c9b081c34aad7d9a87f2c82bc37 2024-04-16T18:15:50.245 DEBUG:botocore.auth:Signature: 20922dc72f7be95bf8f8183478ea4cde9fa0f08e6f3bea8997cd03afa50ae8aa 2024-04-16T18:15:50.246 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler 2024-04-16T18:15:50.246 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.251 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000084ae6fa71d910ee9-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '3633', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.251 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW111111111111111117bf002ca-877a-41e4-ad14-3028bd7fdc7ftest-zone1/arn:aws:iam::RGW11111111111111111:role/test-zone12024-04-16T18:15:43.875Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111Allow{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}arn:aws:iam::aws:policy/AmazonS3FullAccessd2a10f29-d910-40b7-a1e5-bf7c2c78b9e5test-zone2/arn:aws:iam::RGW11111111111111111:role/test-zone22024-04-16T18:15:44.576Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111Allow{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}arn:aws:iam::aws:policy/AmazonS3FullAccessfalsetx0000084ae6fa71d910ee9-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.252 DEBUG:botocore.hooks:Event needs-retry.iam.ListRoles: calling handler 2024-04-16T18:15:50.252 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.252 DEBUG:botocore.hooks:Event after-call.iam.ListRoles: calling handler 2024-04-16T18:15:50.252 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRoles: calling handler 2024-04-16T18:15:50.253 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.253 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.253 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:50.253 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:50.253 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRoles) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRoles', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.253 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler > 2024-04-16T18:15:50.253 DEBUG:botocore.hooks:Event choose-signer.iam.ListRoles: calling handler 2024-04-16T18:15:50.254 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.254 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 8e689dc6b69b17003f0460011582254d8643f0dfb48ac46e093a24df58170b3f 2024-04-16T18:15:50.254 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 2dfb6f5d5ed2a0ffdd6adfd019fe3b461f46ddfccfd4789aa4e6bbf76ed40610 2024-04-16T18:15:50.254 DEBUG:botocore.auth:Signature: 18ca290e1dcb9d63064d15ad1794a5dbf4fd45962a41ecb4541f193c78097f5b 2024-04-16T18:15:50.254 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler 2024-04-16T18:15:50.254 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.261 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000de23128ea8a5feae-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '3633', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.261 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW111111111111111117bf002ca-877a-41e4-ad14-3028bd7fdc7ftest-zone1/arn:aws:iam::RGW11111111111111111:role/test-zone12024-04-16T18:15:43.875Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111Allow{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}arn:aws:iam::aws:policy/AmazonS3FullAccessd2a10f29-d910-40b7-a1e5-bf7c2c78b9e5test-zone2/arn:aws:iam::RGW11111111111111111:role/test-zone22024-04-16T18:15:44.576Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111Allow{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "s3:*", "Resource": "*"}]}arn:aws:iam::aws:policy/AmazonS3FullAccessfalsetx00000de23128ea8a5feae-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.262 DEBUG:botocore.hooks:Event needs-retry.iam.ListRoles: calling handler 2024-04-16T18:15:50.262 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.262 DEBUG:botocore.hooks:Event after-call.iam.ListRoles: calling handler 2024-04-16T18:15:50.262 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.262 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.262 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.263 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.263 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.263 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.263 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.263 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.263 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.263 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 531de279f0b6986a98aec0ac85006960a2b886a52a23f3a97418ea909469aff9 2024-04-16T18:15:50.264 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 846f72ff90684a63a6620f2d935b75aa042c14f5eb6928240cb46f5fc217fac8 2024-04-16T18:15:50.264 DEBUG:botocore.auth:Signature: 8a33b7157d0853187bdd8fb245b32c53f5ccd188044145532fa7706f0ad4ef40 2024-04-16T18:15:50.264 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.264 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.310 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dc6a9e0bc5b5460c-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.311 DEBUG:botocore.parsers:Response body: b'tx00000dc6a9e0bc5b5460c-00661ec056-4804-test-zone1853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.311 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.311 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.311 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.311 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.312 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.312 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.312 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.312 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.312 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.312 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.312 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.313 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.313 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 531de279f0b6986a98aec0ac85006960a2b886a52a23f3a97418ea909469aff9 2024-04-16T18:15:50.313 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c7e8f84abde0acf80b91aef9b682ac4ee87bef46a607e24d6e40fb3ab041f5b8 2024-04-16T18:15:50.313 DEBUG:botocore.auth:Signature: fc82ea8f02309b40a5d20eaaf0d0f153612fbc561a6abb4a7a23d8130ef1eb18 2024-04-16T18:15:50.313 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.313 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.316 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000009856252b9dee68df-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.316 DEBUG:botocore.parsers:Response body: b'tx000009856252b9dee68df-00661ec056-5397-test-zone2853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.317 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.317 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.317 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.317 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.317 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.317 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.318 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.318 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.318 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.318 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.318 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.318 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.319 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date a1e5310e2ee0301edd6ccf49921d305b1fd2f5ba42bce13d1ba0ac6e4fae7892 2024-04-16T18:15:50.319 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 975bae3101f4443bde1c6a98b1de08c8110fa5e1fcae492b5824e96cc641cb9c 2024-04-16T18:15:50.319 DEBUG:botocore.auth:Signature: 4878f4add791dc01adf42096c064e71b3b6d00d6b8deb52bb4f7ae97570667a0 2024-04-16T18:15:50.319 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.319 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.322 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000000b2eb01b1f9d05ec-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.322 DEBUG:botocore.parsers:Response body: b'tx000000b2eb01b1f9d05ec-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.322 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.322 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.322 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.322 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.323 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.323 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.323 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.323 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.323 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.323 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.323 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.324 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.324 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date a1e5310e2ee0301edd6ccf49921d305b1fd2f5ba42bce13d1ba0ac6e4fae7892 2024-04-16T18:15:50.324 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 22111228e767ee82fe56e9326ce8f34dcb044439a8ec4ef985bd3bc11fb57ded 2024-04-16T18:15:50.324 DEBUG:botocore.auth:Signature: e45142e2755820b7f766c210d4f026ef266b6c434bb894fdfd7c846cf3d2a50d 2024-04-16T18:15:50.324 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.324 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.327 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000c72758ec33e3d4fc-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.327 DEBUG:botocore.parsers:Response body: b'tx00000c72758ec33e3d4fc-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.327 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.327 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.327 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.328 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.328 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.328 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.328 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.328 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.328 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.328 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.328 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.329 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.329 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date bc10540825b8174da6b0b6c6a0781cbbae8d1937f3ae57583a6120e809ff3750 2024-04-16T18:15:50.329 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 8e1689d103f4b231ef84f42f3695e44e1b235cb5b803b3187a0c9889f45f8259 2024-04-16T18:15:50.329 DEBUG:botocore.auth:Signature: 1443d4dac5602b75fc0eafe5c55943b958284edfc17d3ae081d872cca62a0b85 2024-04-16T18:15:50.329 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.329 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.332 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000815303cf5cf45707-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.332 DEBUG:botocore.parsers:Response body: b'tx00000815303cf5cf45707-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.332 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.332 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.332 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.333 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.333 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.333 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.333 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.333 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.333 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.333 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.333 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.334 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.334 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date bc10540825b8174da6b0b6c6a0781cbbae8d1937f3ae57583a6120e809ff3750 2024-04-16T18:15:50.334 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 95a8b3a4f14870f3a9c4c698ff350924dfdc2c834b90b869d52b492bcfaf387e 2024-04-16T18:15:50.334 DEBUG:botocore.auth:Signature: 00e5fc76a0086450f92be923b0d6dc22e52643094eff110af9aba18705a5699c 2024-04-16T18:15:50.334 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.334 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.378 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dc0213dc28c70566-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.379 DEBUG:botocore.parsers:Response body: b'tx00000dc0213dc28c70566-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.379 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.379 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.379 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.379 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.379 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.379 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.380 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.380 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.380 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.380 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.380 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.380 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.381 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 02e2c37e0f639aed605a56f302249a2397fee3f10d1ef5436cec797a2864aea1 2024-04-16T18:15:50.381 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 7ae29439568c191f26b7d40e10f7557b7de6ab58d46480a7121f532e2b12b153 2024-04-16T18:15:50.381 DEBUG:botocore.auth:Signature: dd48823c6133c8784bb1af55414748dfe79599d98478e8e105011918cb73836b 2024-04-16T18:15:50.381 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.381 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.384 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000004ec26b2c410cdcb-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.384 DEBUG:botocore.parsers:Response body: b'tx0000004ec26b2c410cdcb-00661ec056-4804-test-zone1f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.384 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.385 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.385 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.385 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.385 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.385 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.385 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.385 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.385 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.386 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.386 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.386 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.386 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 02e2c37e0f639aed605a56f302249a2397fee3f10d1ef5436cec797a2864aea1 2024-04-16T18:15:50.386 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 0477361b2ef629340528bbf9c419c8ff75376592f0f32bfca0770185188837d3 2024-04-16T18:15:50.386 DEBUG:botocore.auth:Signature: e12064d8a6395cd00415c0d0e773d9b9eecb2ca47a706dd7a4a6ce9e009890da 2024-04-16T18:15:50.387 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.387 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.389 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006e0755768edc10db-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.389 DEBUG:botocore.parsers:Response body: b'tx000006e0755768edc10db-00661ec056-5397-test-zone2f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.390 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.390 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.390 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.390 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.390 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.391 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.391 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.391 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.391 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.391 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.391 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.392 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.392 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date a1d084b46fad81a075761bd5542a6a0e50950ec6311a68cd737f0f1de5b600c0 2024-04-16T18:15:50.392 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 081d6c69d6986f8a1bf1b5b76bce1b132eb1a9b77b377ca6a06e91808e9c5399 2024-04-16T18:15:50.392 DEBUG:botocore.auth:Signature: 72cb0ccbeb5cbbe204caf1dcc051f93c0658e49ea8ddad2a22bbc389e5717d5f 2024-04-16T18:15:50.392 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.392 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.395 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000002edfb86a080cd08-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.395 DEBUG:botocore.parsers:Response body: b'tx0000002edfb86a080cd08-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.396 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.396 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.396 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.396 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.396 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.396 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.396 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.396 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.397 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.397 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.397 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.397 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.397 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date a1d084b46fad81a075761bd5542a6a0e50950ec6311a68cd737f0f1de5b600c0 2024-04-16T18:15:50.397 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 46beccda014024283e7238920cecf8d7e2d6087433876b0b598700db84701799 2024-04-16T18:15:50.397 DEBUG:botocore.auth:Signature: 30e7894b11f3eda12a0c362cf312980a929510f5ec8a8b0d105dd50c203ad960 2024-04-16T18:15:50.398 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.398 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.400 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000089fb2a67e94fd323-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.401 DEBUG:botocore.parsers:Response body: b'tx0000089fb2a67e94fd323-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.401 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.401 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.401 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.401 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.401 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.401 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.402 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.402 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.402 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.402 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.402 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.402 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.403 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 84b65f4bc9cead3a6678f4d4a45b599660046d1017f83ffab2a97c93c704c904 2024-04-16T18:15:50.403 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 6ce7a609b944b90ce09ec44aee54eec0b514a6e22c76fd60d7b58207ea1ad3a8 2024-04-16T18:15:50.403 DEBUG:botocore.auth:Signature: 90b7fe95047b9ad0dd4f432c23f40055d46d12e634fdc53cac44019609d9870e 2024-04-16T18:15:50.403 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.403 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.446 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006a41696a2fdb6bdf-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.446 DEBUG:botocore.parsers:Response body: b'tx000006a41696a2fdb6bdf-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.447 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.447 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.447 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.447 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.447 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.447 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.447 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.448 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.448 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.448 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.448 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.448 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.448 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 84b65f4bc9cead3a6678f4d4a45b599660046d1017f83ffab2a97c93c704c904 2024-04-16T18:15:50.449 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request d4d7a95f9b6faf1c211fc2b5bbab51e1770ba14df78797ad1e5fd90a88f05a7e 2024-04-16T18:15:50.449 DEBUG:botocore.auth:Signature: bad5d08ee40f9f398ba978a77052a583d4703d3d2af0f520059a87e286eb945c 2024-04-16T18:15:50.449 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.449 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.452 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000000dfe57eab282b949-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.452 DEBUG:botocore.parsers:Response body: b'tx000000dfe57eab282b949-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.452 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.452 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.452 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.452 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.453 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.453 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.453 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.453 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.453 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.453 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.453 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.454 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.454 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 7067bf4a2c2c30babee515f4ca93ddeff9e9141bcd70c45f18f5120d787f22f5 2024-04-16T18:15:50.454 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 9d84145f04ad8a5d5d2be5051653b18ecb585304fa034d7db7f0620f7f992926 2024-04-16T18:15:50.454 DEBUG:botocore.auth:Signature: dd232a0c459553974ff8d24c56dd8b98790fd1ec852a0394d2846490017e5e21 2024-04-16T18:15:50.454 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.454 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.457 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000740bb12d870d3eed-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '765', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.457 DEBUG:botocore.parsers:Response body: b'tx00000740bb12d870d3eed-00661ec056-4804-test-zone17bf002ca-877a-41e4-ad14-3028bd7fdc7ftest-zone1/arn:aws:iam::RGW11111111111111111:role/test-zone12024-04-16T18:15:43.875Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.458 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.458 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.458 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.458 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.458 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.458 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.458 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.459 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.459 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.459 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.459 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.459 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.459 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 7067bf4a2c2c30babee515f4ca93ddeff9e9141bcd70c45f18f5120d787f22f5 2024-04-16T18:15:50.459 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c47a40bd5ee4df3b4d0cfaf799759b4e2484f1bc89a08aee05badceaa27eb3e5 2024-04-16T18:15:50.460 DEBUG:botocore.auth:Signature: 8b44c1130ae5fb2b4f0a79724f4eaa4b42fcd34149145c569bd93da6c08c6488 2024-04-16T18:15:50.460 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.460 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.463 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000008be1264d25b0d447-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '765', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.463 DEBUG:botocore.parsers:Response body: b'tx000008be1264d25b0d447-00661ec056-5397-test-zone27bf002ca-877a-41e4-ad14-3028bd7fdc7ftest-zone1/arn:aws:iam::RGW11111111111111111:role/test-zone12024-04-16T18:15:43.875Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.465 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.465 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.465 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.466 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.466 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.466 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.466 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.466 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.467 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.467 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.467 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.467 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.467 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 38569227d58e6809847b4e4238520661b8b38646b59cb058130b7fc6c1ed64be 2024-04-16T18:15:50.467 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request b1f3ba1b3a4b7d603b0dcfaea6ec55dd73565f473660c71d4786da12cfc4c9c5 2024-04-16T18:15:50.468 DEBUG:botocore.auth:Signature: 599194edc60bb62ce03f7f2b33c4126db954a06d0bf63716d5c33d61e0f16160 2024-04-16T18:15:50.468 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.468 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.470 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000009b7d4d6990d7c807-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '261', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.471 DEBUG:botocore.parsers:Response body: b'tx000009b7d4d6990d7c807-00661ec056-4804-test-zone1Allow' 2024-04-16T18:15:50.471 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.471 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.471 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.471 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.471 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.471 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.472 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.472 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.472 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.472 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.472 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.472 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.472 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 38569227d58e6809847b4e4238520661b8b38646b59cb058130b7fc6c1ed64be 2024-04-16T18:15:50.473 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 41612f15af74ad3b9dee7163461ac552e2618c2543a198ba24b81b7ec602b9a7 2024-04-16T18:15:50.473 DEBUG:botocore.auth:Signature: 6d52160d46d2a2af0fe383898796e337bc86ec4a1df0d3c56619d666fac25b8f 2024-04-16T18:15:50.473 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.473 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.518 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ee883e5e2b4fe7fa-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '261', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.519 DEBUG:botocore.parsers:Response body: b'tx00000ee883e5e2b4fe7fa-00661ec056-5397-test-zone2Allow' 2024-04-16T18:15:50.519 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.519 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.519 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.519 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.520 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.520 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.520 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.520 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.520 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.520 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.520 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.521 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.521 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 8b09555836446b433b1809feb29e77128d8e49e7527669bc33c462424a43b8b4 2024-04-16T18:15:50.521 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 5cf8ad1cb7136f334e6b0710e9b7bd89df9a6f3ce4f940a3de6287972e027f4c 2024-04-16T18:15:50.521 DEBUG:botocore.auth:Signature: 0be820e1ab80e6e93ce8a3c5fb4f1a26f6252e33654f558ae1d35cd47c04cc7a 2024-04-16T18:15:50.521 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.521 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.524 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000002029aabf1e6bbc7f-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '456', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.524 DEBUG:botocore.parsers:Response body: b'tx000002029aabf1e6bbc7f-00661ec056-4804-test-zone1AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccess' 2024-04-16T18:15:50.525 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.525 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.525 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.525 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.525 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.525 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.525 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.526 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.526 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.526 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.526 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.526 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.526 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 8b09555836446b433b1809feb29e77128d8e49e7527669bc33c462424a43b8b4 2024-04-16T18:15:50.526 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 1296ca2f0db194cae51e650e2808c63a0d0f2f094fcaefba8348393e7b76a271 2024-04-16T18:15:50.527 DEBUG:botocore.auth:Signature: 653a45630597d7201299e95d78838764f419e52d4c3b8030f93154b646d9b801 2024-04-16T18:15:50.527 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.527 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.529 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000008f36ca95ef83afcd-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '456', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.530 DEBUG:botocore.parsers:Response body: b'tx000008f36ca95ef83afcd-00661ec056-5397-test-zone2AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccess' 2024-04-16T18:15:50.530 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.530 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.530 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.530 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.530 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.531 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.531 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.531 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.531 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.531 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.531 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.532 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.532 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date d1b3c02eab04a96ea4c1b62766790de704b6fcba4c966a27173500c60f5d6ff1 2024-04-16T18:15:50.532 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 1a1a2ca68f408fae14bd6eec0aa3fa3d4764a0b222eb44875a356b217db212a7 2024-04-16T18:15:50.532 DEBUG:botocore.auth:Signature: 64b9d8d63fcc677f16f31d0cc93c4b366df736e0e6f0d39c9319b3ff7b8c11b3 2024-04-16T18:15:50.532 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.532 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.535 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000bd054fe7db6fa023-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '765', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.535 DEBUG:botocore.parsers:Response body: b'tx00000bd054fe7db6fa023-00661ec056-4804-test-zone1d2a10f29-d910-40b7-a1e5-bf7c2c78b9e5test-zone2/arn:aws:iam::RGW11111111111111111:role/test-zone22024-04-16T18:15:44.576Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.535 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.535 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.535 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.536 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:50.536 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.536 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.536 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.536 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:50.536 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.537 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:50.537 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:50.537 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.537 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date d1b3c02eab04a96ea4c1b62766790de704b6fcba4c966a27173500c60f5d6ff1 2024-04-16T18:15:50.537 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c2e1328e8c8009560691fe101736a2276a7c621aa4e5826c96c67b913f6d528f 2024-04-16T18:15:50.537 DEBUG:botocore.auth:Signature: 6ef1f37f328870c64bdd9693f3d948d0b5fc877e23753ba8574916392e82b609 2024-04-16T18:15:50.537 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:50.538 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.540 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ed6fb5d44c5a129c-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '765', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.540 DEBUG:botocore.parsers:Response body: b'tx00000ed6fb5d44c5a129c-00661ec056-5397-test-zone2d2a10f29-d910-40b7-a1e5-bf7c2c78b9e5test-zone2/arn:aws:iam::RGW11111111111111111:role/test-zone22024-04-16T18:15:44.576Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:50.541 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:50.541 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.541 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:50.541 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.541 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.541 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.542 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.542 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.542 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.542 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.542 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.542 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.543 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 60679f059cf1aaaffeb2ba196691292bb365c33a340fc79c0064b5b350fd568f 2024-04-16T18:15:50.543 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 474845e811dcb38952277e80e47528ab82991ce0c7ead01835b0f60020c262f3 2024-04-16T18:15:50.543 DEBUG:botocore.auth:Signature: 64bf3aef40df9e600bdc6f46523b4fbd348fed665296e0314d91f1c634506722 2024-04-16T18:15:50.543 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.543 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.545 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dcc31dc1a8a45303-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '261', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.545 DEBUG:botocore.parsers:Response body: b'tx00000dcc31dc1a8a45303-00661ec056-4804-test-zone1Allow' 2024-04-16T18:15:50.546 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.546 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.546 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.546 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.546 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.546 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.547 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.547 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.547 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.547 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:50.547 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.547 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.547 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 60679f059cf1aaaffeb2ba196691292bb365c33a340fc79c0064b5b350fd568f 2024-04-16T18:15:50.548 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request ed1402cc2efe8a644ae0f999bd6dc7cf4015a3bc877c61acdf8fcd6b9e1fda8e 2024-04-16T18:15:50.548 DEBUG:botocore.auth:Signature: ede6385e49b88ddf35cfa33454ae2f47b535ed5361ee943e01b1add46201c78e 2024-04-16T18:15:50.548 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.548 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.551 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000cb6259d59e699952-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '261', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.551 DEBUG:botocore.parsers:Response body: b'tx00000cb6259d59e699952-00661ec056-5397-test-zone2Allow' 2024-04-16T18:15:50.551 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.551 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.551 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:50.551 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.552 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.552 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.552 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.552 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.552 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.552 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.552 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.553 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.553 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date e0f61ac7a0d39bc58cdbe7966ed858e3c10a35a6e4ff05a95164010ed084bae5 2024-04-16T18:15:50.553 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c6be0274077d6e7c5b2e8aaff5ff90bec34d5e1f3b14e1f500cf4e688e33bb4e 2024-04-16T18:15:50.553 DEBUG:botocore.auth:Signature: 6c91fb66f31c8fc994b22d2d8a4eb8ee1ebdbe713f1cb002c7295547023307c2 2024-04-16T18:15:50.553 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.553 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.556 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b2ffb56b17c2f6b9-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '456', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.556 DEBUG:botocore.parsers:Response body: b'tx00000b2ffb56b17c2f6b9-00661ec056-4804-test-zone1AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccess' 2024-04-16T18:15:50.556 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.556 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.556 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.557 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.557 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.557 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.557 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.557 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.557 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.557 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:50.558 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.558 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.558 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date e0f61ac7a0d39bc58cdbe7966ed858e3c10a35a6e4ff05a95164010ed084bae5 2024-04-16T18:15:50.558 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 4dd9aad8661220413f2dafe16d83736fabc69aa587411c6d9c8b513dc644f6f3 2024-04-16T18:15:50.558 DEBUG:botocore.auth:Signature: fae3dfb3693dcd134b1a2482a2fe596b3cf7c820a06d6d9653fd547e5d7ebc90 2024-04-16T18:15:50.558 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.559 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.561 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000002a824385f235c5a2-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '456', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.561 DEBUG:botocore.parsers:Response body: b'tx000002a824385f235c5a2-00661ec056-5397-test-zone2AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccess' 2024-04-16T18:15:50.562 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.562 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.562 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:50.562 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUsers: calling handler 2024-04-16T18:15:50.563 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.563 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.563 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:50.563 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:50.563 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUsers) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUsers', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.564 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler > 2024-04-16T18:15:50.564 DEBUG:botocore.hooks:Event choose-signer.iam.ListUsers: calling handler 2024-04-16T18:15:50.564 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.564 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date b6359072c78d70ebee1e81adcbab4f01bf2c23245fa365ef83fe8f1f955085e2 2024-04-16T18:15:50.564 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 75e3a6ff1d6fb81587ff6d6b1a3c0deee1c80bfd07fd407957d78349c12f87b4 2024-04-16T18:15:50.564 DEBUG:botocore.auth:Signature: d154365134bdcc5603ee80647032836315bc3662de2f1c494bb4910fd2a0132a 2024-04-16T18:15:50.565 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler 2024-04-16T18:15:50.565 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.569 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000f22a9ad84d91569e-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.569 DEBUG:botocore.parsers:Response body: b'/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone12024-04-16T18:15:43.601674Z/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone22024-04-16T18:15:43.985414Zfalse' 2024-04-16T18:15:50.570 DEBUG:botocore.hooks:Event needs-retry.iam.ListUsers: calling handler 2024-04-16T18:15:50.570 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.570 DEBUG:botocore.hooks:Event after-call.iam.ListUsers: calling handler 2024-04-16T18:15:50.571 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUsers: calling handler 2024-04-16T18:15:50.571 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.571 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.571 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:50.571 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:50.571 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUsers) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUsers', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.572 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler > 2024-04-16T18:15:50.572 DEBUG:botocore.hooks:Event choose-signer.iam.ListUsers: calling handler 2024-04-16T18:15:50.572 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.572 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date b6359072c78d70ebee1e81adcbab4f01bf2c23245fa365ef83fe8f1f955085e2 2024-04-16T18:15:50.572 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 35c1ae368094f0280625a07e6c03a4d2148ea6cff6faa3ec601cb4104517f532 2024-04-16T18:15:50.573 DEBUG:botocore.auth:Signature: a4497648cfcc1e27b51160e787e27e234d36e0aa7db185ccf4f4726d5b66beab 2024-04-16T18:15:50.573 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler 2024-04-16T18:15:50.573 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.577 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000d2bebc40857ad4cd-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.577 DEBUG:botocore.parsers:Response body: b'/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone12024-04-16T18:15:43.601674Z/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone22024-04-16T18:15:43.985414Zfalse' 2024-04-16T18:15:50.578 DEBUG:botocore.hooks:Event needs-retry.iam.ListUsers: calling handler 2024-04-16T18:15:50.578 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.578 DEBUG:botocore.hooks:Event after-call.iam.ListUsers: calling handler 2024-04-16T18:15:50.578 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetUser: calling handler 2024-04-16T18:15:50.578 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.579 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.579 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.579 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.579 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetUser', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.579 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler > 2024-04-16T18:15:50.579 DEBUG:botocore.hooks:Event choose-signer.iam.GetUser: calling handler 2024-04-16T18:15:50.580 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.580 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 36be43c886972026dc9392cd2c57a48595fcb211bd6cc645e01ae503a91b5b02 2024-04-16T18:15:50.580 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request ff80b04ccc3e5956be8caecbc406cbffa9130cfe5e156ce3be214848f623582c 2024-04-16T18:15:50.580 DEBUG:botocore.auth:Signature: 30c7e391f35139e78c7fad26f5a9b91f84fa1f3156a2077e7d04d2e3570436ad 2024-04-16T18:15:50.580 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler 2024-04-16T18:15:50.580 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.630 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000001de583b696bf1837-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '487', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.631 DEBUG:botocore.parsers:Response body: b'/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone12024-04-16T18:15:43.601674Ztx000001de583b696bf1837-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.631 DEBUG:botocore.hooks:Event needs-retry.iam.GetUser: calling handler 2024-04-16T18:15:50.631 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.631 DEBUG:botocore.hooks:Event after-call.iam.GetUser: calling handler 2024-04-16T18:15:50.632 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetUser: calling handler 2024-04-16T18:15:50.632 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.632 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.632 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.632 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.632 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetUser', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.633 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler > 2024-04-16T18:15:50.633 DEBUG:botocore.hooks:Event choose-signer.iam.GetUser: calling handler 2024-04-16T18:15:50.633 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.633 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 36be43c886972026dc9392cd2c57a48595fcb211bd6cc645e01ae503a91b5b02 2024-04-16T18:15:50.633 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c97e0233610682d2ac48f118bcd8157b031dcacce639a46713e5086f4d53ed94 2024-04-16T18:15:50.633 DEBUG:botocore.auth:Signature: 4a44a57822862446edef9d8d93134b84ec78fd1ea460d62d30f5b22ceae21564 2024-04-16T18:15:50.634 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler 2024-04-16T18:15:50.634 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.637 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ec2e885a953d7e0e-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '487', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.637 DEBUG:botocore.parsers:Response body: b'/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone12024-04-16T18:15:43.601674Ztx00000ec2e885a953d7e0e-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.638 DEBUG:botocore.hooks:Event needs-retry.iam.GetUser: calling handler 2024-04-16T18:15:50.638 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.638 DEBUG:botocore.hooks:Event after-call.iam.GetUser: calling handler 2024-04-16T18:15:50.639 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.639 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.639 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.639 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.640 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.640 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAccessKeys) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAccessKeys', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.640 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler > 2024-04-16T18:15:50.640 DEBUG:botocore.hooks:Event choose-signer.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.640 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.641 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 919fd997a1071a9eb8adeb0d8e2a852ea3984fffd61b7a1dc133995556ffb778 2024-04-16T18:15:50.641 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 50eb62e244799cdf316ec18a35dba0e724ca82f5f370e970660dab73ce605faf 2024-04-16T18:15:50.641 DEBUG:botocore.auth:Signature: f47f4bf7e30518aa3aeee5a13ea893bb68b90603adac3c08eec096688df9dab2 2024-04-16T18:15:50.641 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.641 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.645 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000437240e1763b7176-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.645 DEBUG:botocore.parsers:Response body: b'test-zone1test-zone183LIGVONPAK8I8ZAT2C6Active2024-04-16T18:15:43.627747Zfalse' 2024-04-16T18:15:50.646 DEBUG:botocore.hooks:Event needs-retry.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.646 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.646 DEBUG:botocore.hooks:Event after-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.647 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.647 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.647 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.647 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.647 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.647 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAccessKeys) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAccessKeys', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.648 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler > 2024-04-16T18:15:50.648 DEBUG:botocore.hooks:Event choose-signer.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.648 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.648 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 919fd997a1071a9eb8adeb0d8e2a852ea3984fffd61b7a1dc133995556ffb778 2024-04-16T18:15:50.648 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c3a5f49a00b271dbbaae35436c72af83521ddcc356701457396fb0188527a7ac 2024-04-16T18:15:50.649 DEBUG:botocore.auth:Signature: ddac49cbf8ab47dbf6e7d3d5bb69c1c00bf0ff49b312093267f157414a2229ad 2024-04-16T18:15:50.649 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.649 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.653 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000472ed4d20a724a50-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.653 DEBUG:botocore.parsers:Response body: b'test-zone1test-zone183LIGVONPAK8I8ZAT2C6Active2024-04-16T18:15:43.627747Zfalse' 2024-04-16T18:15:50.653 DEBUG:botocore.hooks:Event needs-retry.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.653 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.654 DEBUG:botocore.hooks:Event after-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.654 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.654 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.654 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.655 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.655 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.655 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupsForUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupsForUser', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.655 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler > 2024-04-16T18:15:50.655 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.656 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.656 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date d4742f49a0da68c5bb72a1260c8ca465b619be78c1f1b6a677d6fd1e87406c72 2024-04-16T18:15:50.656 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 26c1c1ea8c03fbb8e17ff346d4d2406f190f58420fde62f789c8b11b98b0635d 2024-04-16T18:15:50.656 DEBUG:botocore.auth:Signature: 3600e53b65f066ec171dc13eedec191e4fecbf189785720b423127ec2bfac8d6 2024-04-16T18:15:50.656 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.656 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.660 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ce73629db8d47e47-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '533', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.661 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1falsetx00000ce73629db8d47e47-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.661 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.661 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.661 DEBUG:botocore.hooks:Event after-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.662 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.662 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.662 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.662 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.662 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.662 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupsForUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupsForUser', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.663 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler > 2024-04-16T18:15:50.663 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.663 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.663 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date d4742f49a0da68c5bb72a1260c8ca465b619be78c1f1b6a677d6fd1e87406c72 2024-04-16T18:15:50.663 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 27898e884f3e28a3161eda7f772bfdae74d8159efd7e2fd391d7743335e3f960 2024-04-16T18:15:50.664 DEBUG:botocore.auth:Signature: 70f0ca594ed7a25c017e6003cc758d247fb20f5818433875a459acabb0b37685 2024-04-16T18:15:50.664 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.664 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.710 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000050b2449e0ca6090a-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '533', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.711 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1falsetx0000050b2449e0ca6090a-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.711 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.711 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.711 DEBUG:botocore.hooks:Event after-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.712 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.712 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.712 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.712 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.713 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.713 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.713 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler > 2024-04-16T18:15:50.713 DEBUG:botocore.hooks:Event choose-signer.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.714 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.714 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 4d3f921f2ff07e303538de70576b9876f48b3a5ab3f9a483a2b0b18fb63ec13d 2024-04-16T18:15:50.714 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request dffc80d2a6f16b16e5f9a3d605feb73b8d9d6ae2da5a1c45209f1a10e6e17648 2024-04-16T18:15:50.714 DEBUG:botocore.auth:Signature: aa4318beb36db1189bc35d8ffaa99142b471c7b596aab051d446ab1d70c5dc11 2024-04-16T18:15:50.714 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.714 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.718 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b23d920f878ceaff-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '343', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.718 DEBUG:botocore.parsers:Response body: b'tx00000b23d920f878ceaff-00661ec056-4804-test-zone1Allowfalse' 2024-04-16T18:15:50.719 DEBUG:botocore.hooks:Event needs-retry.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.719 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.719 DEBUG:botocore.hooks:Event after-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.719 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.720 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.720 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.720 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.720 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.720 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.721 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler > 2024-04-16T18:15:50.721 DEBUG:botocore.hooks:Event choose-signer.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.721 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.721 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 4d3f921f2ff07e303538de70576b9876f48b3a5ab3f9a483a2b0b18fb63ec13d 2024-04-16T18:15:50.721 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request fa693e03b15ab59ad9eeed49a22e94b08a0d3218d41ae82fd979eac6099f6bc0 2024-04-16T18:15:50.722 DEBUG:botocore.auth:Signature: d21a71d86dfa2a7c8b39e8060271ea9ea4fe9ea90002b61f44ed0254384da1c0 2024-04-16T18:15:50.722 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.722 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.726 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000004c369661f6bcf319-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '343', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.726 DEBUG:botocore.parsers:Response body: b'tx000004c369661f6bcf319-00661ec056-5397-test-zone2Allowfalse' 2024-04-16T18:15:50.726 DEBUG:botocore.hooks:Event needs-retry.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.726 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.727 DEBUG:botocore.hooks:Event after-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.727 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.727 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.727 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.728 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.728 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.728 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.728 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler > 2024-04-16T18:15:50.728 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.729 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.729 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 49799e0d599ed5d7467f14f6a5435b14aad4e9b68dd234a6844ecab21760f18f 2024-04-16T18:15:50.729 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 1a02c64685e660ff7a14efc7558137a404e8cc5bb2f2c713c6ad6e4482d9b019 2024-04-16T18:15:50.729 DEBUG:botocore.auth:Signature: 69d3b615bb1d45fa86983535e107e3687ae1dd2ccc8c50a9e3ece6d0849c7cca 2024-04-16T18:15:50.729 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.729 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.774 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a6e743459648bafa-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '488', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.774 DEBUG:botocore.parsers:Response body: b'tx00000a6e743459648bafa-00661ec056-4804-test-zone1AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalse' 2024-04-16T18:15:50.775 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.775 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.775 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.775 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.776 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.776 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.776 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.776 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.776 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.776 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler > 2024-04-16T18:15:50.777 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.777 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.777 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 49799e0d599ed5d7467f14f6a5435b14aad4e9b68dd234a6844ecab21760f18f 2024-04-16T18:15:50.777 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 1e7ca425e0d978cac0e65358f00d0d03e5105c6cc28fb30d20ff46d7a1b516a2 2024-04-16T18:15:50.777 DEBUG:botocore.auth:Signature: 392be295991fb3684bd9710d44d96fa5d225ab4fa5564e8fcf1a7150ae23f35b 2024-04-16T18:15:50.777 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.778 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.781 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000000a55691db17ac746-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '488', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.781 DEBUG:botocore.parsers:Response body: b'tx000000a55691db17ac746-00661ec056-5397-test-zone2AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalse' 2024-04-16T18:15:50.782 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.782 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.782 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.782 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetUser: calling handler 2024-04-16T18:15:50.783 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.783 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.783 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.783 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.783 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetUser', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.783 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler > 2024-04-16T18:15:50.783 DEBUG:botocore.hooks:Event choose-signer.iam.GetUser: calling handler 2024-04-16T18:15:50.784 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.784 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 8d30a3f507fe42ace157bf649ae227bc4005724392fed2f2ba1e19ac20f57f65 2024-04-16T18:15:50.784 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request e16c07847934957ca405308c462d8bdb7d6208966ba442be62b00bdedd1c8d7e 2024-04-16T18:15:50.784 DEBUG:botocore.auth:Signature: aeb6117effb8d77210f9bfd50647c0bf4f09838573f89548daaa733f2568fd00 2024-04-16T18:15:50.784 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler 2024-04-16T18:15:50.784 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.788 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000012bc827f61454c39-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '487', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.788 DEBUG:botocore.parsers:Response body: b'/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone22024-04-16T18:15:43.985414Ztx0000012bc827f61454c39-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.788 DEBUG:botocore.hooks:Event needs-retry.iam.GetUser: calling handler 2024-04-16T18:15:50.789 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.789 DEBUG:botocore.hooks:Event after-call.iam.GetUser: calling handler 2024-04-16T18:15:50.789 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetUser: calling handler 2024-04-16T18:15:50.789 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.789 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.789 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.789 DEBUG:botocore.hooks:Event before-call.iam.GetUser: calling handler 2024-04-16T18:15:50.789 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetUser', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.790 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler > 2024-04-16T18:15:50.790 DEBUG:botocore.hooks:Event choose-signer.iam.GetUser: calling handler 2024-04-16T18:15:50.790 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.790 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 8d30a3f507fe42ace157bf649ae227bc4005724392fed2f2ba1e19ac20f57f65 2024-04-16T18:15:50.790 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 6ee4465b15ba9c2a2e7b9f0e9e615fbbb3ef99a44ac7bfa6144aff956f76a4cb 2024-04-16T18:15:50.790 DEBUG:botocore.auth:Signature: 0475988d8a7a10ec97f9ea6aa2aeb760f829224c91d651acbdef76d14b7ed300 2024-04-16T18:15:50.791 DEBUG:botocore.hooks:Event request-created.iam.GetUser: calling handler 2024-04-16T18:15:50.791 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.795 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000002a1bee16f6e94623-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '487', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.795 DEBUG:botocore.parsers:Response body: b'/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone22024-04-16T18:15:43.985414Ztx000002a1bee16f6e94623-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.795 DEBUG:botocore.hooks:Event needs-retry.iam.GetUser: calling handler 2024-04-16T18:15:50.795 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.795 DEBUG:botocore.hooks:Event after-call.iam.GetUser: calling handler 2024-04-16T18:15:50.796 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.796 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.796 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.796 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.796 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.796 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAccessKeys) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAccessKeys', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.797 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler > 2024-04-16T18:15:50.797 DEBUG:botocore.hooks:Event choose-signer.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.797 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.797 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 575b893e712663944c456f61f1aabd27df830cbe7212c952bf0cd211f66fc279 2024-04-16T18:15:50.797 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 92bf6077bfb7f52d52152dad78098bc8b7341c6531ad818dbef13a72e9dcc994 2024-04-16T18:15:50.797 DEBUG:botocore.auth:Signature: 5025287a14758064e01796c6e07a7eaeb6aae80c79c67a0782c348f675adb79c 2024-04-16T18:15:50.797 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.798 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.801 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx000006d19a906aaeae787-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.801 DEBUG:botocore.parsers:Response body: b'test-zone2test-zone261Y92QQ1X31D3HN60KSGActive2024-04-16T18:15:44.023770Zfalse' 2024-04-16T18:15:50.802 DEBUG:botocore.hooks:Event needs-retry.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.802 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.802 DEBUG:botocore.hooks:Event after-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.802 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.802 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.803 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.803 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.803 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.803 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAccessKeys) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAccessKeys', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.803 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler > 2024-04-16T18:15:50.803 DEBUG:botocore.hooks:Event choose-signer.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.803 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.804 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 575b893e712663944c456f61f1aabd27df830cbe7212c952bf0cd211f66fc279 2024-04-16T18:15:50.804 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 5fe04d6873030638113ff20b7e1de1ec3c1185b3b20d11d58accd20dc55c29ae 2024-04-16T18:15:50.804 DEBUG:botocore.auth:Signature: 7ad97200e136a29bf15f5a1f8b6c214a7de00936d8d650eef26bd8a9eb0b5a6a 2024-04-16T18:15:50.804 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.804 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.808 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx000007acf1019f519bbba-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.808 DEBUG:botocore.parsers:Response body: b'test-zone2test-zone261Y92QQ1X31D3HN60KSGActive2024-04-16T18:15:44.023770Zfalse' 2024-04-16T18:15:50.809 DEBUG:botocore.hooks:Event needs-retry.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.809 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.809 DEBUG:botocore.hooks:Event after-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:50.809 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.809 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.809 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.810 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.810 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.810 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupsForUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupsForUser', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.810 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler > 2024-04-16T18:15:50.810 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.810 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.810 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date ecf8aee7706b736153a5a7d48afdc1ac23c4def3724f6ced3ccd7e69933ba573 2024-04-16T18:15:50.811 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request c9be7f22ccbe5313e33c8da1474dc407dc956491cff4c84c1c39059f474f5a4b 2024-04-16T18:15:50.811 DEBUG:botocore.auth:Signature: b6c550ffd80c0e392f7b77cd1389dda0d441a4c9daa3b1e047815d8c0a339c9e 2024-04-16T18:15:50.811 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.811 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.815 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b723509302734c88-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '533', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.815 DEBUG:botocore.parsers:Response body: b'/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2falsetx00000b723509302734c88-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.816 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.816 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.816 DEBUG:botocore.hooks:Event after-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.816 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.816 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.816 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.816 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.817 DEBUG:botocore.hooks:Event before-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.817 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupsForUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupsForUser', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.817 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler > 2024-04-16T18:15:50.817 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.817 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.817 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date ecf8aee7706b736153a5a7d48afdc1ac23c4def3724f6ced3ccd7e69933ba573 2024-04-16T18:15:50.817 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 9e9a1b1bd21e2fabf3b4f7043249d4306d6dc4f9919280d10eca75ed6b5e95bd 2024-04-16T18:15:50.818 DEBUG:botocore.auth:Signature: 539f4bc547d69509a2d12b05d03d84a535242520b4a379e0cde7933f5ee15905 2024-04-16T18:15:50.818 DEBUG:botocore.hooks:Event request-created.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.818 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.862 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000f0fd7c99e00c9bd8-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '533', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.863 DEBUG:botocore.parsers:Response body: b'/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2falsetx00000f0fd7c99e00c9bd8-00661ec056-5397-test-zone2' 2024-04-16T18:15:50.863 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.863 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.863 DEBUG:botocore.hooks:Event after-call.iam.ListGroupsForUser: calling handler 2024-04-16T18:15:50.863 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.863 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.864 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.864 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.864 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.864 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.864 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler > 2024-04-16T18:15:50.864 DEBUG:botocore.hooks:Event choose-signer.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.864 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.865 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 683bb38491391414b27b590de03643d3067ab3a39c7a8909fc0ae8ca88576c7c 2024-04-16T18:15:50.865 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 9871a52446fa173e88a90812549e9c8f3ea7a5ce4a2f1b4944966008fb3ed0c2 2024-04-16T18:15:50.865 DEBUG:botocore.auth:Signature: 55146c0d0009ce8527c400462c4bf8032109c0b99dcd43a37e1937458c292bdc 2024-04-16T18:15:50.865 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.865 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.869 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000129c2237c6a4e0ed-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '343', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.869 DEBUG:botocore.parsers:Response body: b'tx00000129c2237c6a4e0ed-00661ec056-4804-test-zone1Allowfalse' 2024-04-16T18:15:50.870 DEBUG:botocore.hooks:Event needs-retry.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.870 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.870 DEBUG:botocore.hooks:Event after-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.870 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.870 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.870 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.870 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.871 DEBUG:botocore.hooks:Event before-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.871 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.871 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler > 2024-04-16T18:15:50.871 DEBUG:botocore.hooks:Event choose-signer.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.871 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.871 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 683bb38491391414b27b590de03643d3067ab3a39c7a8909fc0ae8ca88576c7c 2024-04-16T18:15:50.871 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 7e89af2f622bd02a4f54351c28e1090f3751ea7b52cb7ed2b978429dd3702345 2024-04-16T18:15:50.872 DEBUG:botocore.auth:Signature: 36f307977328ebe1f2b73cca11a94f47643e7ede30471322c9cedd35543a0563 2024-04-16T18:15:50.872 DEBUG:botocore.hooks:Event request-created.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.872 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.876 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006be01cd65d8bd066-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '343', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.876 DEBUG:botocore.parsers:Response body: b'tx000006be01cd65d8bd066-00661ec056-5397-test-zone2Allowfalse' 2024-04-16T18:15:50.877 DEBUG:botocore.hooks:Event needs-retry.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.877 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.877 DEBUG:botocore.hooks:Event after-call.iam.ListUserPolicies: calling handler 2024-04-16T18:15:50.877 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.877 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.877 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.878 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.878 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.878 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.878 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler > 2024-04-16T18:15:50.878 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.879 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.879 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 3dace716e8edf34ab8446c6b1da0096d88964c8d134639d581b86139d8c0a818 2024-04-16T18:15:50.879 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 57a8edacc301cc2e9974d256682dd0e30cfe02ab32bd5d8b67bec1da3f841464 2024-04-16T18:15:50.879 DEBUG:botocore.auth:Signature: d19a22daadc6e89b36c7caff221aba922867daca96047d05e55257369fa200cd 2024-04-16T18:15:50.879 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.879 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.926 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000009041cc04de6de7f5-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '488', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.927 DEBUG:botocore.parsers:Response body: b'tx000009041cc04de6de7f5-00661ec056-4804-test-zone1AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalse' 2024-04-16T18:15:50.927 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.927 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.927 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.927 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.927 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.927 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.928 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.928 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.928 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedUserPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedUserPolicies', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.928 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler > 2024-04-16T18:15:50.928 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.928 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.928 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 3dace716e8edf34ab8446c6b1da0096d88964c8d134639d581b86139d8c0a818 2024-04-16T18:15:50.929 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 50ef132e121b5c3a379c454893eb4ed60380d714ffb297f9a1a294c6c8d9c98e 2024-04-16T18:15:50.929 DEBUG:botocore.auth:Signature: f764bb6f1aa2851b271e3f0979638a3b901dc861db4813e227959eec58d329d9 2024-04-16T18:15:50.929 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.929 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.933 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000652003fc1d4ccad0-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '488', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.933 DEBUG:botocore.parsers:Response body: b'tx00000652003fc1d4ccad0-00661ec056-5397-test-zone2AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalse' 2024-04-16T18:15:50.933 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.934 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.934 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedUserPolicies: calling handler 2024-04-16T18:15:50.934 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroups: calling handler 2024-04-16T18:15:50.934 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.934 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.935 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:50.935 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:50.935 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroups) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroups', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.935 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler > 2024-04-16T18:15:50.935 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroups: calling handler 2024-04-16T18:15:50.936 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.936 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 5f776d91509b9c99b8cb5eb5d6d4a787a33ae41c8cd6e7b69effca69080e1e1f 2024-04-16T18:15:50.936 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request a2bc4d7120057183e212d5564af8611037ad09e614dc96731d5d0150445004d7 2024-04-16T18:15:50.936 DEBUG:botocore.auth:Signature: 46c706857658525a09f82b1ca39f41ed46cf4dbdd5d691d912a21fb3b397c6ea 2024-04-16T18:15:50.936 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler 2024-04-16T18:15:50.936 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.940 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000e3aaf225c956eb00-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.940 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2false' 2024-04-16T18:15:50.941 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroups: calling handler 2024-04-16T18:15:50.941 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.941 DEBUG:botocore.hooks:Event after-call.iam.ListGroups: calling handler 2024-04-16T18:15:50.941 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroups: calling handler 2024-04-16T18:15:50.941 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.942 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.942 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:50.942 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:50.942 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroups) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroups', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.942 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler > 2024-04-16T18:15:50.943 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroups: calling handler 2024-04-16T18:15:50.943 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.943 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 5f776d91509b9c99b8cb5eb5d6d4a787a33ae41c8cd6e7b69effca69080e1e1f 2024-04-16T18:15:50.943 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request bf91c4dc079e14af7dc2cf741e59d09b6962cd8fec7e8dbc66a4d75746c3902b 2024-04-16T18:15:50.943 DEBUG:botocore.auth:Signature: 9b3c25ce7225e0df03a5a012827fdf8392e161c1dfdb681127d1490c8e4a9d0a 2024-04-16T18:15:50.943 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler 2024-04-16T18:15:50.944 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.948 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx000004deec4aab5a96f51-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.948 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2false' 2024-04-16T18:15:50.948 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroups: calling handler 2024-04-16T18:15:50.949 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.949 DEBUG:botocore.hooks:Event after-call.iam.ListGroups: calling handler 2024-04-16T18:15:50.949 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetGroup: calling handler 2024-04-16T18:15:50.949 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.949 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:50.950 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:50.950 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:50.950 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.950 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler > 2024-04-16T18:15:50.951 DEBUG:botocore.hooks:Event choose-signer.iam.GetGroup: calling handler 2024-04-16T18:15:50.951 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.951 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 96c30390354a0b67fe3fad2562a6dea8125351a0e25f82d61db2010f5b8b25a3 2024-04-16T18:15:50.951 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request f16a93db62fa76c11e6f2eeb789efd4ff44d618e81d1e8f6353b31d610154b15 2024-04-16T18:15:50.951 DEBUG:botocore.auth:Signature: d12c67f91c85d344312439c89cba896cb6c0d73f9a2c7f43fbf0a324b4f0180b 2024-04-16T18:15:50.951 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler 2024-04-16T18:15:50.952 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:50.956 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000c3613a393ae9585f-00661ec056-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '668', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:50.956 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone1falsetx00000c3613a393ae9585f-00661ec056-4804-test-zone1' 2024-04-16T18:15:50.957 DEBUG:botocore.hooks:Event needs-retry.iam.GetGroup: calling handler 2024-04-16T18:15:50.957 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:50.957 DEBUG:botocore.hooks:Event after-call.iam.GetGroup: calling handler 2024-04-16T18:15:50.957 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetGroup: calling handler 2024-04-16T18:15:50.958 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:50.958 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:50.958 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:50.958 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:50.958 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:50.959 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler > 2024-04-16T18:15:50.959 DEBUG:botocore.hooks:Event choose-signer.iam.GetGroup: calling handler 2024-04-16T18:15:50.959 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:50.959 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181550Z content-type;host;x-amz-date 96c30390354a0b67fe3fad2562a6dea8125351a0e25f82d61db2010f5b8b25a3 2024-04-16T18:15:50.959 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181550Z 20240416/test-zonegroup/iam/aws4_request 9bbf0aa198abe34acfa1b1eec264e661da05c0a6d2a2862278f67471bef6d3e9 2024-04-16T18:15:50.959 DEBUG:botocore.auth:Signature: 90dcd50f620d90dcc96ba8227386a29939cb4e9c071515644968a9060a81085d 2024-04-16T18:15:50.960 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler 2024-04-16T18:15:50.960 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.006 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000009bb4fee0a0c13b4c-00661ec056-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '668', 'Date': 'Tue, 16 Apr 2024 18:15:50 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.007 DEBUG:botocore.parsers:Response body: b'/test-zone1b8c1f40c-912a-40dd-89a3-f08b07ad100barn:aws:iam::RGW11111111111111111:group/test-zone1/test-zone1b891e439-4f7c-4b24-b442-873212d26c57arn:aws:iam::RGW11111111111111111:user/test-zone1falsetx000009bb4fee0a0c13b4c-00661ec056-5397-test-zone2' 2024-04-16T18:15:51.007 DEBUG:botocore.hooks:Event needs-retry.iam.GetGroup: calling handler 2024-04-16T18:15:51.007 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.007 DEBUG:botocore.hooks:Event after-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.008 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.008 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.008 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.008 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.009 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.009 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.009 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler > 2024-04-16T18:15:51.009 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.009 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.010 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 7811b9d5c4ca8b1e9d0155d32c2077f72a8b205c19c3c2277df8f5e6e074f9aa 2024-04-16T18:15:51.010 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 174b54be7d2026f056fce3cc0e86e82fac14716d67f0415b82696822da3f1418 2024-04-16T18:15:51.010 DEBUG:botocore.auth:Signature: f1c12512820c268fc48fdbc4028cd75614e9d6c0a094ffd5f8f58c7e58683b3c 2024-04-16T18:15:51.010 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.010 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.013 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000064824644379ecafe-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '385', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.013 DEBUG:botocore.parsers:Response body: b'Allowfalsetx0000064824644379ecafe-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.014 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.014 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.014 DEBUG:botocore.hooks:Event after-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.014 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.014 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.015 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.015 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.015 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.015 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.015 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler > 2024-04-16T18:15:51.016 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.016 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.016 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 7811b9d5c4ca8b1e9d0155d32c2077f72a8b205c19c3c2277df8f5e6e074f9aa 2024-04-16T18:15:51.016 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request eceb6c56b50814474f8501d261ad7668e564b7c33d994fea1a649d978beaa69c 2024-04-16T18:15:51.016 DEBUG:botocore.auth:Signature: a9cc804f509b891d6e67836c42875b2c070fcf01c08e93dc29e1b97d5000c7a8 2024-04-16T18:15:51.016 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.017 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.019 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a3e63f6c33054e86-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '385', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.020 DEBUG:botocore.parsers:Response body: b'Allowfalsetx00000a3e63f6c33054e86-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.020 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.020 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.020 DEBUG:botocore.hooks:Event after-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.021 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.021 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.021 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.021 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.022 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.022 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.022 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler > 2024-04-16T18:15:51.022 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.023 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.023 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date d3cad5863c5641759e4bcb9ff168c45d0bfbea13d35654ca45486df7605c4b98 2024-04-16T18:15:51.023 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 646c275e7e694789ea687dbc95a89d5f8b22371368dd021571dd7745c88e7885 2024-04-16T18:15:51.023 DEBUG:botocore.auth:Signature: b48313f268b7ce46144bd299f8ca83027b7ac43e3e1ecb023727fdf1b99dc4c6 2024-04-16T18:15:51.023 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.023 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.026 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b102e94fa83de23c-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '530', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.026 DEBUG:botocore.parsers:Response body: b'AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalsetx00000b102e94fa83de23c-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.027 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.027 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.027 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.027 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.028 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.028 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.028 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.028 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.028 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.029 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler > 2024-04-16T18:15:51.029 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.029 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.029 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date d3cad5863c5641759e4bcb9ff168c45d0bfbea13d35654ca45486df7605c4b98 2024-04-16T18:15:51.029 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request f4ad154169a8d0970d777c0ccbe772e55132b5520f07a7d831ad855d317faaf5 2024-04-16T18:15:51.029 DEBUG:botocore.auth:Signature: 21d2e5a2e200c3e2640e7e61c03e3141d454ee4b69a5d50143a32410ebef44e3 2024-04-16T18:15:51.030 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.030 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.032 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000038460fde1e5fed30-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '530', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.033 DEBUG:botocore.parsers:Response body: b'AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalsetx0000038460fde1e5fed30-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.033 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.033 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.033 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.033 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetGroup: calling handler 2024-04-16T18:15:51.034 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.034 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.034 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.034 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.034 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.034 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler > 2024-04-16T18:15:51.034 DEBUG:botocore.hooks:Event choose-signer.iam.GetGroup: calling handler 2024-04-16T18:15:51.035 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.035 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 0de1abeeecebec53ce7c53e52f1ad78c5754c2ec5c9b2c6a3da55177a99211d3 2024-04-16T18:15:51.035 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 0e8cbd401a0c61305be49914f94045f6174ccc32906da73519c50ed4b84e9231 2024-04-16T18:15:51.035 DEBUG:botocore.auth:Signature: 85bd8d7fcba7219b89a09676aca5b5f0a046d95f49f98a14e843403e4e03584c 2024-04-16T18:15:51.035 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler 2024-04-16T18:15:51.035 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.039 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a2bfea6a4070b4c4-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '668', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.040 DEBUG:botocore.parsers:Response body: b'/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone2falsetx00000a2bfea6a4070b4c4-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.040 DEBUG:botocore.hooks:Event needs-retry.iam.GetGroup: calling handler 2024-04-16T18:15:51.040 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.040 DEBUG:botocore.hooks:Event after-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.040 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetGroup: calling handler 2024-04-16T18:15:51.040 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.041 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.041 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.041 DEBUG:botocore.hooks:Event before-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.041 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.041 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler > 2024-04-16T18:15:51.041 DEBUG:botocore.hooks:Event choose-signer.iam.GetGroup: calling handler 2024-04-16T18:15:51.042 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.042 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 0de1abeeecebec53ce7c53e52f1ad78c5754c2ec5c9b2c6a3da55177a99211d3 2024-04-16T18:15:51.042 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request be24a6e8395362275f00e39a21da3dc9175785a78490d403ad31ff18b34fb0ff 2024-04-16T18:15:51.042 DEBUG:botocore.auth:Signature: 36b0eb40444bd6aabbd5a24f28793ae38aabd6ccbbb4893989abe08ee53a9741 2024-04-16T18:15:51.042 DEBUG:botocore.hooks:Event request-created.iam.GetGroup: calling handler 2024-04-16T18:15:51.042 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.046 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000630711b4be1c297e-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '668', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.046 DEBUG:botocore.parsers:Response body: b'/test-zone2dad4eadd-1aff-4576-b401-079a76d41be1arn:aws:iam::RGW11111111111111111:group/test-zone2/test-zone2a5fe26e3-3653-4b1d-94ce-c90b782066f6arn:aws:iam::RGW11111111111111111:user/test-zone2falsetx00000630711b4be1c297e-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.046 DEBUG:botocore.hooks:Event needs-retry.iam.GetGroup: calling handler 2024-04-16T18:15:51.047 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.047 DEBUG:botocore.hooks:Event after-call.iam.GetGroup: calling handler 2024-04-16T18:15:51.047 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.047 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.047 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.047 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.048 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.048 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.048 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler > 2024-04-16T18:15:51.048 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.048 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.048 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date b8c9ebd42f207eccd0f1ac4735e5608d9b5aca42e16ba14ba414850016f19bb6 2024-04-16T18:15:51.048 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request fdb2cba137683b629721daa29016a4606173ad2a8a1e3bd3e47994fcee3e4f92 2024-04-16T18:15:51.049 DEBUG:botocore.auth:Signature: d2824b99250a89a5ae4968ca52e2fc4e45474cc5500dac928d7ec3537bce1383 2024-04-16T18:15:51.049 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.049 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.094 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000e6804981c6b64280-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '385', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.095 DEBUG:botocore.parsers:Response body: b'Allowfalsetx00000e6804981c6b64280-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.095 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.095 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.096 DEBUG:botocore.hooks:Event after-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.096 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.096 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.096 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.096 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.096 DEBUG:botocore.hooks:Event before-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.096 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.097 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler > 2024-04-16T18:15:51.097 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.097 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.098 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date b8c9ebd42f207eccd0f1ac4735e5608d9b5aca42e16ba14ba414850016f19bb6 2024-04-16T18:15:51.098 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 0b461a0e55c281a89722e34ebe85859299e54bf920a15f89b0a698d918da4896 2024-04-16T18:15:51.098 DEBUG:botocore.auth:Signature: a31869355a4e1d312c2a652aff1487f4a62913a84910e20e1f02667845e526e6 2024-04-16T18:15:51.098 DEBUG:botocore.hooks:Event request-created.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.098 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.101 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000ab5bb2c574ff7a8c-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '385', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.101 DEBUG:botocore.parsers:Response body: b'Allowfalsetx00000ab5bb2c574ff7a8c-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.101 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.101 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.101 DEBUG:botocore.hooks:Event after-call.iam.ListGroupPolicies: calling handler 2024-04-16T18:15:51.102 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.102 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.102 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.102 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.102 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.102 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.103 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler > 2024-04-16T18:15:51.103 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.103 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.103 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 4cdafc9ca13a31ddd9a7a3bfb5e4a2230e855a4b6f877fc7873c6f03bf7ad991 2024-04-16T18:15:51.103 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request a6a93fc35baad612689d7fc651a52b15a498f73aec5f63d1e728790c2ab6d84e 2024-04-16T18:15:51.103 DEBUG:botocore.auth:Signature: fd208fcd1e6eb44c789a852d5fe213f7a949659015bc7ab330b3bdf8855e8fb1 2024-04-16T18:15:51.104 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.104 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.107 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000d3c70d241346a21f-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '530', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.107 DEBUG:botocore.parsers:Response body: b'AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalsetx00000d3c70d241346a21f-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.107 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.107 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.107 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.107 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.107 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.108 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.108 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.108 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.108 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedGroupPolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedGroupPolicies', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.108 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler > 2024-04-16T18:15:51.108 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.109 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.109 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 4cdafc9ca13a31ddd9a7a3bfb5e4a2230e855a4b6f877fc7873c6f03bf7ad991 2024-04-16T18:15:51.109 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 9c3d45f92f1dd28949f46373cb0c6c3f4e1310a01792fa4f31a6a480125e9054 2024-04-16T18:15:51.109 DEBUG:botocore.auth:Signature: 6ef835c54a9764cb5493675c54cb644eee5dd5dcf39d03c6d04256701e53242a 2024-04-16T18:15:51.109 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.109 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.112 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006a3b01c2e6f70093-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '530', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.112 DEBUG:botocore.parsers:Response body: b'AmazonS3FullAccessarn:aws:iam::aws:policy/AmazonS3FullAccessfalsetx000006a3b01c2e6f70093-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.112 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.112 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.112 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedGroupPolicies: calling handler 2024-04-16T18:15:51.113 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.113 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.113 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.113 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.113 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.114 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListOpenIDConnectProviders) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListOpenIDConnectProviders', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.114 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler > 2024-04-16T18:15:51.114 DEBUG:botocore.hooks:Event choose-signer.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.114 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.114 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 6be253d5bef7fe9cb4ed0d00d8a18351f59593888e86906ef784f147279adb10 2024-04-16T18:15:51.115 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 0de2716c1556ec3a2a489530ff9942bbbbdaa21dc1fa79e149d0f69dab5fa603 2024-04-16T18:15:51.115 DEBUG:botocore.auth:Signature: 7976ae9f575b0a8defc267169f448ecaa29edf46ce89b2b6dd0b02f9c376f476 2024-04-16T18:15:51.115 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.115 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.146 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000004a5330318db695a4-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '357', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.146 DEBUG:botocore.parsers:Response body: b'tx000004a5330318db695a4-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.146 DEBUG:botocore.hooks:Event needs-retry.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.147 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.147 DEBUG:botocore.hooks:Event after-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.147 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.147 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.147 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.148 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.148 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.148 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListOpenIDConnectProviders) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListOpenIDConnectProviders', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.148 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler > 2024-04-16T18:15:51.148 DEBUG:botocore.hooks:Event choose-signer.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.148 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.149 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 6be253d5bef7fe9cb4ed0d00d8a18351f59593888e86906ef784f147279adb10 2024-04-16T18:15:51.149 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 3c100d6210a5c7b205f8175fbc0bb39d7cd7a057f83bd15887aa828e09810c46 2024-04-16T18:15:51.149 DEBUG:botocore.auth:Signature: 3255497f3a13d30d9d1cb47d69b477ab6bf475a6d9dd5c4ecdb36d45bcbb8104 2024-04-16T18:15:51.149 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.149 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.178 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000004e327f6ef1cff9e4-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '357', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.178 DEBUG:botocore.parsers:Response body: b'tx000004e327f6ef1cff9e4-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.178 DEBUG:botocore.hooks:Event needs-retry.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.178 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.178 DEBUG:botocore.hooks:Event after-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:51.179 DEBUG:botocore.hooks:Event before-parameter-build.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.179 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.179 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.179 DEBUG:botocore.hooks:Event before-call.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.179 DEBUG:botocore.hooks:Event before-call.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.179 DEBUG:botocore.endpoint:Making request for OperationModel(name=DetachRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DetachRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone1', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.180 DEBUG:botocore.hooks:Event request-created.iam.DetachRolePolicy: calling handler > 2024-04-16T18:15:51.180 DEBUG:botocore.hooks:Event choose-signer.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.180 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.180 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date bd73383d5317fe146a37b32cc167dca318a97704e67083ae893e94c203174404 2024-04-16T18:15:51.180 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 841ddd047ed216e2c6bc30d7072608aaa0104c97d28e8de47b8d95daefec2e8d 2024-04-16T18:15:51.181 DEBUG:botocore.auth:Signature: 533ab61ec3f665378b0fd130af912a27ac878cac54670ff6afd7ca4553dc8a60 2024-04-16T18:15:51.181 DEBUG:botocore.hooks:Event request-created.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.181 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.238 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000096b2402dc6607d6a-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.238 DEBUG:botocore.parsers:Response body: b'tx0000096b2402dc6607d6a-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.239 DEBUG:botocore.hooks:Event needs-retry.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.239 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.239 DEBUG:botocore.hooks:Event after-call.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.239 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.239 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.240 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.240 DEBUG:botocore.hooks:Event before-call.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.240 DEBUG:botocore.hooks:Event before-call.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.240 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone1', 'PolicyName': 'Allow'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.240 DEBUG:botocore.hooks:Event request-created.iam.DeleteRolePolicy: calling handler > 2024-04-16T18:15:51.241 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.241 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.241 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date ff2b8e08f5bd2f6e968bc5d2892f7a2005ce31351a054fa3113914bd584e4a51 2024-04-16T18:15:51.241 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 053153130d53bc3f9a33038c45e65e484950b73607949ebba943cbbf54727d98 2024-04-16T18:15:51.241 DEBUG:botocore.auth:Signature: 253be27f9e6cd10e4720bca146b859c8476d9cb4f8d1b48397fbe92ef821c7b1 2024-04-16T18:15:51.241 DEBUG:botocore.hooks:Event request-created.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.242 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.256 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a9666ecbe25125eb-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '163', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.256 DEBUG:botocore.parsers:Response body: b'tx00000a9666ecbe25125eb-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.256 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.256 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.256 DEBUG:botocore.hooks:Event after-call.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.257 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteRole: calling handler 2024-04-16T18:15:51.257 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.257 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.257 DEBUG:botocore.hooks:Event before-call.iam.DeleteRole: calling handler 2024-04-16T18:15:51.257 DEBUG:botocore.hooks:Event before-call.iam.DeleteRole: calling handler 2024-04-16T18:15:51.257 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteRole', 'Version': '2010-05-08', 'RoleName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.257 DEBUG:botocore.hooks:Event request-created.iam.DeleteRole: calling handler > 2024-04-16T18:15:51.258 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteRole: calling handler 2024-04-16T18:15:51.258 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.258 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date eb0db1c076fced9cc6f71d7a5176de1b17d1027791a1e35ab50e96286833a6ef 2024-04-16T18:15:51.258 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request d730b37607825d375d1edd2c6a901cccd9ac347b2d7aab78715dd3f02157f8f7 2024-04-16T18:15:51.258 DEBUG:botocore.auth:Signature: 30f64b34af98a3319b7e22ce76bdfbe29cc192b2474cff3aa4681db9ad5273a3 2024-04-16T18:15:51.258 DEBUG:botocore.hooks:Event request-created.iam.DeleteRole: calling handler 2024-04-16T18:15:51.259 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.279 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b2b2a7709355bc81-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '151', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.279 DEBUG:botocore.parsers:Response body: b'tx00000b2b2a7709355bc81-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.279 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteRole: calling handler 2024-04-16T18:15:51.280 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.280 DEBUG:botocore.hooks:Event after-call.iam.DeleteRole: calling handler 2024-04-16T18:15:51.280 DEBUG:botocore.hooks:Event before-parameter-build.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.280 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.280 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.281 DEBUG:botocore.hooks:Event before-call.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.281 DEBUG:botocore.hooks:Event before-call.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.281 DEBUG:botocore.endpoint:Making request for OperationModel(name=RemoveUserFromGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'RemoveUserFromGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone1', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.281 DEBUG:botocore.hooks:Event request-created.iam.RemoveUserFromGroup: calling handler > 2024-04-16T18:15:51.281 DEBUG:botocore.hooks:Event choose-signer.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.282 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.282 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 8101fbe751c2510228d0196ca3a189db821341cab247a56bf56b1a8de94be62f 2024-04-16T18:15:51.282 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request eb2e0a14d3df2a382bb611dd765d173b7a5399de81726ba49efef921438b8046 2024-04-16T18:15:51.282 DEBUG:botocore.auth:Signature: aa765f39477feb7d03656cfb60ee6228b1cede5ed3142fa7d6dffc421f5b4092 2024-04-16T18:15:51.282 DEBUG:botocore.hooks:Event request-created.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.282 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.342 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000345118599ee78f54-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '257', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.343 DEBUG:botocore.parsers:Response body: b'tx00000345118599ee78f54-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.343 DEBUG:botocore.hooks:Event needs-retry.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.343 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.343 DEBUG:botocore.hooks:Event after-call.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.343 DEBUG:botocore.hooks:Event before-parameter-build.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.344 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.344 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.344 DEBUG:botocore.hooks:Event before-call.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.344 DEBUG:botocore.hooks:Event before-call.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.344 DEBUG:botocore.endpoint:Making request for OperationModel(name=DetachGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DetachGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone1', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.345 DEBUG:botocore.hooks:Event request-created.iam.DetachGroupPolicy: calling handler > 2024-04-16T18:15:51.345 DEBUG:botocore.hooks:Event choose-signer.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.345 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.345 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 93eba8cc47d76370fe5425ab749ce1edbed126d87b12ab5216df76b39f891c29 2024-04-16T18:15:51.345 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 3c3d9b32d1f6e520fede70303f8801b42d68fc4ed99e43f09fdd9fefc1cef2b8 2024-04-16T18:15:51.345 DEBUG:botocore.auth:Signature: 41aed8d3a4e20b4a255561765ba5bb4ed55631e7b62538a3b17860f8bce800f2 2024-04-16T18:15:51.346 DEBUG:botocore.hooks:Event request-created.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.346 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.356 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000f0083e7f1a60d60e-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '253', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.357 DEBUG:botocore.parsers:Response body: b'tx00000f0083e7f1a60d60e-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.357 DEBUG:botocore.hooks:Event needs-retry.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.357 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.357 DEBUG:botocore.hooks:Event after-call.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.358 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.358 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.358 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.358 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.359 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.359 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone1', 'PolicyName': 'Allow'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.359 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroupPolicy: calling handler > 2024-04-16T18:15:51.359 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.360 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.360 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date bcb6e0830d4e4b57d0c03ed18cd3a27f9eed72fb600ce4b187dfdf4f5a23a0f1 2024-04-16T18:15:51.360 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 1b8b74ac4252319067bbc3b223c002a2a8034818a56eb763c153d8eef7feb90e 2024-04-16T18:15:51.360 DEBUG:botocore.auth:Signature: d69fbec7ee55a19642c31276dd27334fbbfb15dad3de1bf12a303b96ca486ce6 2024-04-16T18:15:51.360 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.360 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.372 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000008ca55bb845be06fe-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '253', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.372 DEBUG:botocore.parsers:Response body: b'tx000008ca55bb845be06fe-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.373 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.373 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.373 DEBUG:botocore.hooks:Event after-call.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.373 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.374 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.374 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.374 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.374 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.374 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.374 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroup: calling handler > 2024-04-16T18:15:51.375 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.375 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.375 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date fe2ed6547b0314458d41caf46f8196bcd62a800e25af60d633b29183f00efad6 2024-04-16T18:15:51.375 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 8acec1890c5517321b7ddebb43aaff2d882a1330779b727b5bcc78e7be77d42f 2024-04-16T18:15:51.375 DEBUG:botocore.auth:Signature: 7ad1efa081cab24fa8778b431daf2aff9ab01691a5b3d18d4a5bdfbd43237eb9 2024-04-16T18:15:51.376 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.376 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.438 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000084f923516d193dff-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '241', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.438 DEBUG:botocore.parsers:Response body: b'tx0000084f923516d193dff-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.439 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.439 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.439 DEBUG:botocore.hooks:Event after-call.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.439 DEBUG:botocore.hooks:Event before-parameter-build.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.440 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.440 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.440 DEBUG:botocore.hooks:Event before-call.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.440 DEBUG:botocore.hooks:Event before-call.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.440 DEBUG:botocore.endpoint:Making request for OperationModel(name=DetachUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DetachUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone1', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.441 DEBUG:botocore.hooks:Event request-created.iam.DetachUserPolicy: calling handler > 2024-04-16T18:15:51.441 DEBUG:botocore.hooks:Event choose-signer.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.441 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.441 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date e4eb10f876ac99edb55b6e38ac859bdca8126307348276c5e6af1c89db67d5ef 2024-04-16T18:15:51.441 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 5911f358b1cd4761076770e44fb5564aaa4264f38cde33f7b6cd3051a414d5b7 2024-04-16T18:15:51.442 DEBUG:botocore.auth:Signature: e8cc738e30ec2c28969df4169954e45c6706283443e4915a2b6413458aff4eb9 2024-04-16T18:15:51.442 DEBUG:botocore.hooks:Event request-created.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.442 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.466 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b6cd0dca455764bb-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.466 DEBUG:botocore.parsers:Response body: b'tx00000b6cd0dca455764bb-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.466 DEBUG:botocore.hooks:Event needs-retry.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.467 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.467 DEBUG:botocore.hooks:Event after-call.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:51.467 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.467 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.467 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.468 DEBUG:botocore.hooks:Event before-call.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.468 DEBUG:botocore.hooks:Event before-call.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.468 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone1', 'PolicyName': 'Allow'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.468 DEBUG:botocore.hooks:Event request-created.iam.DeleteUserPolicy: calling handler > 2024-04-16T18:15:51.468 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.469 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.469 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date a8978949ce38abd713a4ef1765a7b6091a66572aed9d67f96ac31a077d7e0d99 2024-04-16T18:15:51.469 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 9eaad4c29310484802f36a6170d7133cf39e860e536ee22f2e1d69203b8032e5 2024-04-16T18:15:51.469 DEBUG:botocore.auth:Signature: 5c0eabf164eebfb72002259180fa2e7b3fb4810b9830697bf45c054fab3725ba 2024-04-16T18:15:51.469 DEBUG:botocore.hooks:Event request-created.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.470 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.493 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000021d50597497088a5-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '217', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.493 DEBUG:botocore.parsers:Response body: b'tx0000021d50597497088a5-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.493 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.494 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.494 DEBUG:botocore.hooks:Event after-call.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:51.494 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.494 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.494 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.494 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.495 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.495 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAccessKeys) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAccessKeys', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.497 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler > 2024-04-16T18:15:51.497 DEBUG:botocore.hooks:Event choose-signer.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.498 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.498 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 919fd997a1071a9eb8adeb0d8e2a852ea3984fffd61b7a1dc133995556ffb778 2024-04-16T18:15:51.498 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 141765d2c4f97f9b27ba37e79b7c3bdb63719df7fc444f5fdba9bb9ad773cdd4 2024-04-16T18:15:51.498 DEBUG:botocore.auth:Signature: 5c5108df7a6f30a7c5bb45990aa617547a4ad8c8123d35a126154bc7cd38497a 2024-04-16T18:15:51.498 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.498 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.542 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000a6ed2cf49e92d496-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.543 DEBUG:botocore.parsers:Response body: b'test-zone1test-zone183LIGVONPAK8I8ZAT2C6Active2024-04-16T18:15:43.627747Zfalse' 2024-04-16T18:15:51.543 DEBUG:botocore.hooks:Event needs-retry.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.543 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.543 DEBUG:botocore.hooks:Event after-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:51.544 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.544 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.544 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.544 DEBUG:botocore.hooks:Event before-call.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.544 DEBUG:botocore.hooks:Event before-call.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.545 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteAccessKey) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteAccessKey', 'Version': '2010-05-08', 'UserName': 'test-zone1', 'AccessKeyId': '83LIGVONPAK8I8ZAT2C6'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.545 DEBUG:botocore.hooks:Event request-created.iam.DeleteAccessKey: calling handler > 2024-04-16T18:15:51.545 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.545 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.546 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 67ef4d9384fd40748b82ba73da3d0bdf2aecec885cf9ededfda05d80bb20497e 2024-04-16T18:15:51.546 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request bca7bd9f6430ead1dd5d37c0fb942b9a25f6394bf2a5c473785735d96cf67f38 2024-04-16T18:15:51.546 DEBUG:botocore.auth:Signature: eda26e620b3105084ccfdb1c01c58900a4283c7f6288f3e2f5dfe3f30eaaaeaf 2024-04-16T18:15:51.546 DEBUG:botocore.hooks:Event request-created.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.546 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.567 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000001a5992fb741d991-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '249', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.567 DEBUG:botocore.parsers:Response body: b'tx0000001a5992fb741d991-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.567 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.567 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.567 DEBUG:botocore.hooks:Event after-call.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:51.568 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteUser: calling handler 2024-04-16T18:15:51.568 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.568 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:51.568 DEBUG:botocore.hooks:Event before-call.iam.DeleteUser: calling handler 2024-04-16T18:15:51.568 DEBUG:botocore.hooks:Event before-call.iam.DeleteUser: calling handler 2024-04-16T18:15:51.569 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteUser', 'Version': '2010-05-08', 'UserName': 'test-zone1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.569 DEBUG:botocore.hooks:Event request-created.iam.DeleteUser: calling handler > 2024-04-16T18:15:51.569 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteUser: calling handler 2024-04-16T18:15:51.569 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.569 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 73b6ad4b3e5d11fba46ac1e98e9ba27cc1964c1d8192f1fa1a1bf17be5282a09 2024-04-16T18:15:51.570 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 4d0a589b1f3ae44bc938271ccbd1a77d07e5df095ef401e57b234142ba19cae9 2024-04-16T18:15:51.570 DEBUG:botocore.auth:Signature: 7e9b9df23af119682723ec24b22514725ea11a48640feed3bddc22d4ea6b8230 2024-04-16T18:15:51.570 DEBUG:botocore.hooks:Event request-created.iam.DeleteUser: calling handler 2024-04-16T18:15:51.570 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.587 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000002fe5c08f69f3cc68-00661ec057-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.587 DEBUG:botocore.parsers:Response body: b'tx000002fe5c08f69f3cc68-00661ec057-4804-test-zone1' 2024-04-16T18:15:51.588 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteUser: calling handler 2024-04-16T18:15:51.588 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.588 DEBUG:botocore.hooks:Event after-call.iam.DeleteUser: calling handler 2024-04-16T18:15:51.588 DEBUG:botocore.hooks:Event before-parameter-build.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.589 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.589 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.589 DEBUG:botocore.hooks:Event before-call.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.589 DEBUG:botocore.hooks:Event before-call.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.589 DEBUG:botocore.endpoint:Making request for OperationModel(name=DetachRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DetachRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone2', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.589 DEBUG:botocore.hooks:Event request-created.iam.DetachRolePolicy: calling handler > 2024-04-16T18:15:51.590 DEBUG:botocore.hooks:Event choose-signer.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.590 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.590 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date c325a97cec49c93483b73d102a4ee669ab364fdbcde01894c7995f875c5e42ea 2024-04-16T18:15:51.590 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request d54015c2b696582a9016e5b368155ec4ce66706ab22250b56b011ab627acdb50 2024-04-16T18:15:51.591 DEBUG:botocore.auth:Signature: df45eb5d280c7ef6eb59b06ca65e7b122e38ed96b0042676a725a349c4ffd4e1 2024-04-16T18:15:51.591 DEBUG:botocore.hooks:Event request-created.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.591 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.613 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b8dbce84ea3a8c40-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.613 DEBUG:botocore.parsers:Response body: b'tx00000b8dbce84ea3a8c40-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.614 DEBUG:botocore.hooks:Event needs-retry.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.614 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.614 DEBUG:botocore.hooks:Event after-call.iam.DetachRolePolicy: calling handler 2024-04-16T18:15:51.614 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.614 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.615 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.615 DEBUG:botocore.hooks:Event before-call.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.615 DEBUG:botocore.hooks:Event before-call.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.615 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteRolePolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteRolePolicy', 'Version': '2010-05-08', 'RoleName': 'test-zone2', 'PolicyName': 'Allow'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.615 DEBUG:botocore.hooks:Event request-created.iam.DeleteRolePolicy: calling handler > 2024-04-16T18:15:51.616 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.616 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.616 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 0e711179b95df7649427669dd02fc5b80c804175ee6ff14feb947361680d8f4c 2024-04-16T18:15:51.616 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 156337c78ef1a1d00997618c81b627cd1f89c2e4d1175a9fd749523e7648221b 2024-04-16T18:15:51.616 DEBUG:botocore.auth:Signature: caa7648951ea87e96e663ad6259955cd710afe2a2ed24a4329b4c438c3d9df2d 2024-04-16T18:15:51.617 DEBUG:botocore.hooks:Event request-created.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.617 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.722 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000025f5773d14f958dd-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '163', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.723 DEBUG:botocore.parsers:Response body: b'tx0000025f5773d14f958dd-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.723 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.723 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.723 DEBUG:botocore.hooks:Event after-call.iam.DeleteRolePolicy: calling handler 2024-04-16T18:15:51.723 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteRole: calling handler 2024-04-16T18:15:51.724 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.724 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.725 DEBUG:botocore.hooks:Event before-call.iam.DeleteRole: calling handler 2024-04-16T18:15:51.725 DEBUG:botocore.hooks:Event before-call.iam.DeleteRole: calling handler 2024-04-16T18:15:51.725 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteRole', 'Version': '2010-05-08', 'RoleName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.725 DEBUG:botocore.hooks:Event request-created.iam.DeleteRole: calling handler > 2024-04-16T18:15:51.725 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteRole: calling handler 2024-04-16T18:15:51.726 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.726 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 832fae51b1d10776be58658abbb6e5d165d2e58ca758e9bedbbbabf3d441ea56 2024-04-16T18:15:51.726 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 7b3a93fd92c4adfabebf252530fc832b5123c063567afe83aa77160f885b3b60 2024-04-16T18:15:51.726 DEBUG:botocore.auth:Signature: e86c403f2ca3adec58d1661d71c00dacca16371e507be8a7fed7c500291b1337 2024-04-16T18:15:51.726 DEBUG:botocore.hooks:Event request-created.iam.DeleteRole: calling handler 2024-04-16T18:15:51.727 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.763 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000007079552699a70c3d-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '151', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.763 DEBUG:botocore.parsers:Response body: b'tx000007079552699a70c3d-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.764 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteRole: calling handler 2024-04-16T18:15:51.764 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.764 DEBUG:botocore.hooks:Event after-call.iam.DeleteRole: calling handler 2024-04-16T18:15:51.764 DEBUG:botocore.hooks:Event before-parameter-build.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.765 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.765 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.765 DEBUG:botocore.hooks:Event before-call.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.765 DEBUG:botocore.hooks:Event before-call.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.765 DEBUG:botocore.endpoint:Making request for OperationModel(name=RemoveUserFromGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'RemoveUserFromGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone2', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.766 DEBUG:botocore.hooks:Event request-created.iam.RemoveUserFromGroup: calling handler > 2024-04-16T18:15:51.766 DEBUG:botocore.hooks:Event choose-signer.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.766 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.766 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date f37f0ec6c347966013e62f32fd007623cac80b2e0db8e71ca06331c77d3b3c28 2024-04-16T18:15:51.766 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 307205c55f3177d36a5148e227413aa56a9e18071ea454541b3d8ac29a690d47 2024-04-16T18:15:51.767 DEBUG:botocore.auth:Signature: 1c307a33d79c76564fe566ef1bc10c84c2264121b341565ee77461ebd1a2bff6 2024-04-16T18:15:51.767 DEBUG:botocore.hooks:Event request-created.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.767 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.840 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000a2131bbba0b9aa22-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '257', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.840 DEBUG:botocore.parsers:Response body: b'tx00000a2131bbba0b9aa22-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.840 DEBUG:botocore.hooks:Event needs-retry.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.841 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.841 DEBUG:botocore.hooks:Event after-call.iam.RemoveUserFromGroup: calling handler 2024-04-16T18:15:51.841 DEBUG:botocore.hooks:Event before-parameter-build.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.841 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.841 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.842 DEBUG:botocore.hooks:Event before-call.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.842 DEBUG:botocore.hooks:Event before-call.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.842 DEBUG:botocore.endpoint:Making request for OperationModel(name=DetachGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DetachGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone2', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.842 DEBUG:botocore.hooks:Event request-created.iam.DetachGroupPolicy: calling handler > 2024-04-16T18:15:51.842 DEBUG:botocore.hooks:Event choose-signer.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.843 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.843 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date d10497e442384ad8daba8c183ca9a44d1658b63c4ded1005678646a32710f983 2024-04-16T18:15:51.843 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request da619a5e9b7adf49ed5568e3f5abe8d93445dca1e07fa74da46f7c71b863a108 2024-04-16T18:15:51.843 DEBUG:botocore.auth:Signature: 4b964e813c694e26f341c4ffc8d452a968cca23df2f97e032ef470214c4e2a66 2024-04-16T18:15:51.843 DEBUG:botocore.hooks:Event request-created.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.844 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.950 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000003383898a336859ac-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '253', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.951 DEBUG:botocore.parsers:Response body: b'tx000003383898a336859ac-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.951 DEBUG:botocore.hooks:Event needs-retry.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.951 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.951 DEBUG:botocore.hooks:Event after-call.iam.DetachGroupPolicy: calling handler 2024-04-16T18:15:51.951 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.952 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.952 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.952 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.952 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.952 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteGroupPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteGroupPolicy', 'Version': '2010-05-08', 'GroupName': 'test-zone2', 'PolicyName': 'Allow'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.953 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroupPolicy: calling handler > 2024-04-16T18:15:51.953 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.953 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.953 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date bc90399dc585c9565b8e09c006ff5a452f25f284492b3177a00a6816d35ed452 2024-04-16T18:15:51.953 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request 2403ea320eb2a106d8ed91db4aa958776f18e991ed08431ee3ab521868687895 2024-04-16T18:15:51.953 DEBUG:botocore.auth:Signature: 5851c9bc00214e939be11aa606c5ba96fcd672835b5dd97c40332bdcf1eff6e1 2024-04-16T18:15:51.954 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.954 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:51.974 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000867667fe7573a088-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '253', 'Date': 'Tue, 16 Apr 2024 18:15:51 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:51.974 DEBUG:botocore.parsers:Response body: b'tx00000867667fe7573a088-00661ec057-5397-test-zone2' 2024-04-16T18:15:51.974 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.974 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:51.975 DEBUG:botocore.hooks:Event after-call.iam.DeleteGroupPolicy: calling handler 2024-04-16T18:15:51.975 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.975 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:51.975 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:51.975 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.975 DEBUG:botocore.hooks:Event before-call.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.976 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteGroup) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteGroup', 'Version': '2010-05-08', 'GroupName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:51.976 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroup: calling handler > 2024-04-16T18:15:51.976 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.976 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:51.976 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181551Z content-type;host;x-amz-date 44bf744101969da17efb1509be4f7be6d0a4c5a9f8dcfdeffea997cc24d810c7 2024-04-16T18:15:51.977 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181551Z 20240416/test-zonegroup/iam/aws4_request d477519d37e3d2c528ddd13abeead965785e2c01ed6857c1eaa4741fb4537450 2024-04-16T18:15:51.977 DEBUG:botocore.auth:Signature: be5993103cd66cb6cf58d9b37a2cd1f5e24805fe6c010072aae337e4c445737e 2024-04-16T18:15:51.977 DEBUG:botocore.hooks:Event request-created.iam.DeleteGroup: calling handler 2024-04-16T18:15:51.977 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:52.063 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000029b54d99551e674d-00661ec057-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '241', 'Date': 'Tue, 16 Apr 2024 18:15:52 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:52.063 DEBUG:botocore.parsers:Response body: b'tx0000029b54d99551e674d-00661ec057-5397-test-zone2' 2024-04-16T18:15:52.063 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteGroup: calling handler 2024-04-16T18:15:52.063 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:52.064 DEBUG:botocore.hooks:Event after-call.iam.DeleteGroup: calling handler 2024-04-16T18:15:52.064 DEBUG:botocore.hooks:Event before-parameter-build.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.064 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:52.064 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:52.065 DEBUG:botocore.hooks:Event before-call.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.065 DEBUG:botocore.hooks:Event before-call.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.065 DEBUG:botocore.endpoint:Making request for OperationModel(name=DetachUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DetachUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone2', 'PolicyArn': 'arn:aws:iam::aws:policy/AmazonS3FullAccess'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:52.065 DEBUG:botocore.hooks:Event request-created.iam.DetachUserPolicy: calling handler > 2024-04-16T18:15:52.065 DEBUG:botocore.hooks:Event choose-signer.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.066 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:52.066 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181552Z content-type;host;x-amz-date e19d91a90ba0f4a2add980542fe58bb8ea7de111e90548b27ec9798cc678a117 2024-04-16T18:15:52.066 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181552Z 20240416/test-zonegroup/iam/aws4_request c2b6d2ee1cd88a38a3ec76151c19c3cc1aec190af8e32fdb98563bd9ba9c3da6 2024-04-16T18:15:52.066 DEBUG:botocore.auth:Signature: ad8a89586aa0f07edad219ed69541f20085c3558ef53ccb0e5f97cb84ea642d3 2024-04-16T18:15:52.066 DEBUG:botocore.hooks:Event request-created.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.067 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:52.190 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dce89851e13170c9-00661ec058-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '213', 'Date': 'Tue, 16 Apr 2024 18:15:52 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:52.190 DEBUG:botocore.parsers:Response body: b'tx00000dce89851e13170c9-00661ec058-5397-test-zone2' 2024-04-16T18:15:52.191 DEBUG:botocore.hooks:Event needs-retry.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.191 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:52.191 DEBUG:botocore.hooks:Event after-call.iam.DetachUserPolicy: calling handler 2024-04-16T18:15:52.191 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.192 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:52.192 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:52.192 DEBUG:botocore.hooks:Event before-call.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.192 DEBUG:botocore.hooks:Event before-call.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.192 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteUserPolicy) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteUserPolicy', 'Version': '2010-05-08', 'UserName': 'test-zone2', 'PolicyName': 'Allow'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:52.193 DEBUG:botocore.hooks:Event request-created.iam.DeleteUserPolicy: calling handler > 2024-04-16T18:15:52.193 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.193 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:52.193 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181552Z content-type;host;x-amz-date d2741fea5e8fde2129ad86dfc4cf8cd0689aa8930033a952888845632f37aeac 2024-04-16T18:15:52.193 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181552Z 20240416/test-zonegroup/iam/aws4_request fec3c00795095316f699a7444c553bbc62dbcbf41556eb08fb219ac75a360946 2024-04-16T18:15:52.194 DEBUG:botocore.auth:Signature: 3984536b3ab1ad8dc31c7b028703cd282e0887b6a8618e09387fc8d64418b35b 2024-04-16T18:15:52.194 DEBUG:botocore.hooks:Event request-created.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.194 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:52.238 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000b2e0abfbacf62dde-00661ec058-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '217', 'Date': 'Tue, 16 Apr 2024 18:15:52 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:52.238 DEBUG:botocore.parsers:Response body: b'tx00000b2e0abfbacf62dde-00661ec058-5397-test-zone2' 2024-04-16T18:15:52.239 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.239 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:52.239 DEBUG:botocore.hooks:Event after-call.iam.DeleteUserPolicy: calling handler 2024-04-16T18:15:52.239 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.239 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:52.239 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:52.240 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.240 DEBUG:botocore.hooks:Event before-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.240 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAccessKeys) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAccessKeys', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:52.240 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler > 2024-04-16T18:15:52.240 DEBUG:botocore.hooks:Event choose-signer.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.240 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:52.241 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181552Z content-type;host;x-amz-date 575b893e712663944c456f61f1aabd27df830cbe7212c952bf0cd211f66fc279 2024-04-16T18:15:52.241 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181552Z 20240416/test-zonegroup/iam/aws4_request 02562b4fcc2180723e19fe523600a565db3711acad4ddb18bef19296173dd9fb 2024-04-16T18:15:52.241 DEBUG:botocore.auth:Signature: b423f692e314456a4f5e93b8711d566cd296652074c0092c858992bec2b69951 2024-04-16T18:15:52.241 DEBUG:botocore.hooks:Event request-created.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.241 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:52.245 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx000000e9781383eeb52cd-00661ec058-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:52 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:52.245 DEBUG:botocore.parsers:Response body: b'test-zone2test-zone261Y92QQ1X31D3HN60KSGActive2024-04-16T18:15:44.023770Zfalse' 2024-04-16T18:15:52.246 DEBUG:botocore.hooks:Event needs-retry.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.246 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:52.246 DEBUG:botocore.hooks:Event after-call.iam.ListAccessKeys: calling handler 2024-04-16T18:15:52.246 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.246 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:52.246 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:52.247 DEBUG:botocore.hooks:Event before-call.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.247 DEBUG:botocore.hooks:Event before-call.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.247 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteAccessKey) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteAccessKey', 'Version': '2010-05-08', 'UserName': 'test-zone2', 'AccessKeyId': '61Y92QQ1X31D3HN60KSG'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:52.247 DEBUG:botocore.hooks:Event request-created.iam.DeleteAccessKey: calling handler > 2024-04-16T18:15:52.247 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.248 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:52.248 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181552Z content-type;host;x-amz-date bbf1332a826233643d3580a7246ed106cd4df6700271c3db465400c51f47f229 2024-04-16T18:15:52.248 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181552Z 20240416/test-zonegroup/iam/aws4_request 6d3ef2ae2438e011f732d8ee814feba23e9127aad3d7b864dd0fd47a85e1497a 2024-04-16T18:15:52.248 DEBUG:botocore.auth:Signature: 9e293689f40fd613b5fd9dd86556e2d4131d3cb09451b258fb22e057ba46bb9b 2024-04-16T18:15:52.248 DEBUG:botocore.hooks:Event request-created.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.248 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:52.366 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000067189090df156d42-00661ec058-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '249', 'Date': 'Tue, 16 Apr 2024 18:15:52 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:52.367 DEBUG:botocore.parsers:Response body: b'tx0000067189090df156d42-00661ec058-5397-test-zone2' 2024-04-16T18:15:52.367 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.367 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:52.367 DEBUG:botocore.hooks:Event after-call.iam.DeleteAccessKey: calling handler 2024-04-16T18:15:52.367 DEBUG:botocore.hooks:Event before-parameter-build.iam.DeleteUser: calling handler 2024-04-16T18:15:52.368 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:52.368 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:52.368 DEBUG:botocore.hooks:Event before-call.iam.DeleteUser: calling handler 2024-04-16T18:15:52.368 DEBUG:botocore.hooks:Event before-call.iam.DeleteUser: calling handler 2024-04-16T18:15:52.368 DEBUG:botocore.endpoint:Making request for OperationModel(name=DeleteUser) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'DeleteUser', 'Version': '2010-05-08', 'UserName': 'test-zone2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:52.368 DEBUG:botocore.hooks:Event request-created.iam.DeleteUser: calling handler > 2024-04-16T18:15:52.369 DEBUG:botocore.hooks:Event choose-signer.iam.DeleteUser: calling handler 2024-04-16T18:15:52.369 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:52.369 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181552Z content-type;host;x-amz-date 5cec3e80938f8d7af48456f647335d80c040d9c550731683093602f874824e23 2024-04-16T18:15:52.369 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181552Z 20240416/test-zonegroup/iam/aws4_request 4c183858fa46b4f7ba459e66704aa2faaf2d02371e11d3852249c3c1b610b666 2024-04-16T18:15:52.369 DEBUG:botocore.auth:Signature: e1a3c9b945b4a10cd2f5c8b1c64a19697f9f9e497b6abd94eec2d4304a26dbfd 2024-04-16T18:15:52.369 DEBUG:botocore.hooks:Event request-created.iam.DeleteUser: calling handler 2024-04-16T18:15:52.370 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:52.398 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dcd88727a3c3fc95-00661ec058-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:52 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:52.398 DEBUG:botocore.parsers:Response body: b'tx00000dcd88727a3c3fc95-00661ec058-5397-test-zone2' 2024-04-16T18:15:52.399 DEBUG:botocore.hooks:Event needs-retry.iam.DeleteUser: calling handler 2024-04-16T18:15:52.399 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:52.399 DEBUG:botocore.hooks:Event after-call.iam.DeleteUser: calling handler 2024-04-16T18:15:52.399 INFO:rgw_multi.tests:meta checkpoint 2024-04-16T18:15:52.399 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:52.400 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:52.400 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:52.442 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:52.442 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:52.511 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:52.545 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291351.590025_524.1","last_update":"2024-04-16T18:15:51.590025Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:52.545 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291351.590025_524.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:52.545 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:52.545 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:52.545 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:52.546 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:52.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:52.659+0000 7fcf0f61a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:52.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:52.663+0000 7fcf0f61a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:52.691 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291351.452859_517.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:51.452859Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:52Z"} 2024-04-16T18:15:52.691 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291351.452859_517.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:51.452859Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:52Z"} 2024-04-16T18:15:52.691 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291351.452859_517.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.452859Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:52.691 DEBUG:rgw_multi.tests:log_status={0: '1_1713291351.590025_524.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:52.691 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291351.452859_517.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:52.691 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291351.590025_524.1 target=1_1713291351.452859_517.1 2024-04-16T18:15:57.692 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:57.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:57.693 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:57.739 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:57.739 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:57.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:57.815+0000 7f51dd979980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:57.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:57.815+0000 7f51dd979980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:57.848 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291351.590025_524.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:51.590025Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:57Z"} 2024-04-16T18:15:57.849 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291351.590025_524.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:51.590025Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:57Z"} 2024-04-16T18:15:57.849 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291351.590025_524.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.590025Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:57.849 DEBUG:rgw_multi.tests:log_status={0: '1_1713291351.590025_524.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:57.849 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291351.590025_524.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:57.849 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:57.850 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRoles: calling handler 2024-04-16T18:15:57.850 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.851 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.851 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:57.851 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:57.851 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRoles) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRoles', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.852 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler > 2024-04-16T18:15:57.852 DEBUG:botocore.hooks:Event choose-signer.iam.ListRoles: calling handler 2024-04-16T18:15:57.852 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.853 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 8e689dc6b69b17003f0460011582254d8643f0dfb48ac46e093a24df58170b3f 2024-04-16T18:15:57.853 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 304d5c33e768ab50f22c23f91d881083814d13cfcf2188dc707df1e8d0d42c4d 2024-04-16T18:15:57.853 DEBUG:botocore.auth:Signature: c046eba48c303bd335d704c4c4647fe6ec66729edf0bd0905f6915cc0a60e5bf 2024-04-16T18:15:57.853 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler 2024-04-16T18:15:57.854 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.859 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000d74687acd1d42c50-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1519', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.859 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111falsetx00000d74687acd1d42c50-00661ec05d-4804-test-zone1' 2024-04-16T18:15:57.860 DEBUG:botocore.hooks:Event needs-retry.iam.ListRoles: calling handler 2024-04-16T18:15:57.860 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.860 DEBUG:botocore.hooks:Event after-call.iam.ListRoles: calling handler 2024-04-16T18:15:57.861 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRoles: calling handler 2024-04-16T18:15:57.861 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.861 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:57.861 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:57.862 DEBUG:botocore.hooks:Event before-call.iam.ListRoles: calling handler 2024-04-16T18:15:57.862 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRoles) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRoles', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.862 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler > 2024-04-16T18:15:57.862 DEBUG:botocore.hooks:Event choose-signer.iam.ListRoles: calling handler 2024-04-16T18:15:57.863 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.863 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 8e689dc6b69b17003f0460011582254d8643f0dfb48ac46e093a24df58170b3f 2024-04-16T18:15:57.863 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 4e8f2a1b28750f0d57009db69c491dd6fbf36225a1f00a8f335c2b19b30683bf 2024-04-16T18:15:57.863 DEBUG:botocore.auth:Signature: a922b7250873a57da40663839fe87a44e0fb8ea133d1eab0396f5c587fa1adfe 2024-04-16T18:15:57.863 DEBUG:botocore.hooks:Event request-created.iam.ListRoles: calling handler 2024-04-16T18:15:57.864 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.867 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000005147255266ab5b50-00661ec05d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1519', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.868 DEBUG:botocore.parsers:Response body: b'853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}RGW11111111111111111falsetx000005147255266ab5b50-00661ec05d-5397-test-zone2' 2024-04-16T18:15:57.868 DEBUG:botocore.hooks:Event needs-retry.iam.ListRoles: calling handler 2024-04-16T18:15:57.869 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.869 DEBUG:botocore.hooks:Event after-call.iam.ListRoles: calling handler 2024-04-16T18:15:57.869 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:57.869 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.869 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.870 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.870 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.870 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.870 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:57.871 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:57.871 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.871 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 531de279f0b6986a98aec0ac85006960a2b886a52a23f3a97418ea909469aff9 2024-04-16T18:15:57.871 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 5197e7eb9b48b9cd1963fbfe0f43f5f074134f70fe49102a2bf9374357560d4f 2024-04-16T18:15:57.872 DEBUG:botocore.auth:Signature: 5deb450dee87bfd607ad0448dfd2ca9b69967f8d51cf95c63c03bc077a759f9d 2024-04-16T18:15:57.872 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:57.872 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.919 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000d94aab8b0561268d-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.919 DEBUG:botocore.parsers:Response body: b'tx00000d94aab8b0561268d-00661ec05d-4804-test-zone1853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:57.920 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:57.920 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.920 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:57.920 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:57.921 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.921 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:57.921 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.921 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.921 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.922 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:57.922 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:57.922 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.923 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 531de279f0b6986a98aec0ac85006960a2b886a52a23f3a97418ea909469aff9 2024-04-16T18:15:57.923 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request f5f4c315024c9674fb3b354dfd1f43e466c884b53e36956079eff442c81e0f9d 2024-04-16T18:15:57.923 DEBUG:botocore.auth:Signature: d7f3dcd6f9a9b99c3cff29ab796a8fc6ef0754e1748e9edbded7ab9f1b31389e 2024-04-16T18:15:57.923 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:57.924 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.926 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000db53517088c58d7a-00661ec05d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.926 DEBUG:botocore.parsers:Response body: b'tx00000db53517088c58d7a-00661ec05d-5397-test-zone2853c17fc-ae46-4972-9dd7-451fc7741550roles-lybtug-1/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-12024-04-16T18:15:35.317Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:57.927 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:57.927 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.927 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:57.928 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.928 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.928 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.928 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.929 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.929 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.929 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:57.929 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.930 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.930 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date a1e5310e2ee0301edd6ccf49921d305b1fd2f5ba42bce13d1ba0ac6e4fae7892 2024-04-16T18:15:57.930 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request f8f3694e71da6ff5095ec2c99846a461c03ede6ca5bec2547e718a16409f9b4b 2024-04-16T18:15:57.930 DEBUG:botocore.auth:Signature: ff5235bce8478401bdf0114956f02671b06afd0a0edbbfdb0775afcc67a545e9 2024-04-16T18:15:57.930 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.931 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.934 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000e265d9da0fef4160-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.934 DEBUG:botocore.parsers:Response body: b'tx00000e265d9da0fef4160-00661ec05d-4804-test-zone1' 2024-04-16T18:15:57.934 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.934 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.934 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.935 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.935 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.935 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:57.935 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.935 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.936 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.936 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:57.936 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.936 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.937 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date a1e5310e2ee0301edd6ccf49921d305b1fd2f5ba42bce13d1ba0ac6e4fae7892 2024-04-16T18:15:57.937 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 023b73353a0b008ac7bedf477895270accf6fe4926c7229afb13e83f42d8c848 2024-04-16T18:15:57.937 DEBUG:botocore.auth:Signature: b01b11bf683594b8c51be1c202d49e4c57426a3eb2a1c98ab6835e2ae353ba05 2024-04-16T18:15:57.937 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.937 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.940 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000494e62614d0164d3-00661ec05d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.940 DEBUG:botocore.parsers:Response body: b'tx00000494e62614d0164d3-00661ec05d-5397-test-zone2' 2024-04-16T18:15:57.941 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.941 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.941 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.941 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.942 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.942 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.942 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.942 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.942 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.943 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:57.943 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.943 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.943 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date bc10540825b8174da6b0b6c6a0781cbbae8d1937f3ae57583a6120e809ff3750 2024-04-16T18:15:57.943 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 3b1b9a6857e0d24a45cf3127b688277f510659ecba0198e1e2c2e170b46c8a19 2024-04-16T18:15:57.944 DEBUG:botocore.auth:Signature: 005fbea6c8c6e1a73df131f69d6c7a16cb47ace2ffbb50d03a70e672bd4d939c 2024-04-16T18:15:57.944 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.944 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.947 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006db82600f2af3813-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.947 DEBUG:botocore.parsers:Response body: b'tx000006db82600f2af3813-00661ec05d-4804-test-zone1' 2024-04-16T18:15:57.947 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.948 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.948 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.948 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.948 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.948 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:57.948 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.949 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.949 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-1'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.949 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:57.949 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.950 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.950 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date bc10540825b8174da6b0b6c6a0781cbbae8d1937f3ae57583a6120e809ff3750 2024-04-16T18:15:57.950 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request e439a47e17d8613c49204cca65e742ad3b03459fd8f198f337888fd5a88847c4 2024-04-16T18:15:57.950 DEBUG:botocore.auth:Signature: 26021f3d4707cb06ea16035921a25213355577215374ef752be0bb6bfc50f228 2024-04-16T18:15:57.950 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.951 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.954 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000dd8903ce269138eb-00661ec05d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.954 DEBUG:botocore.parsers:Response body: b'tx00000dd8903ce269138eb-00661ec05d-5397-test-zone2' 2024-04-16T18:15:57.955 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.955 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.955 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.955 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:57.955 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.955 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.956 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.956 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.956 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.956 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:57.956 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:57.957 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.957 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 02e2c37e0f639aed605a56f302249a2397fee3f10d1ef5436cec797a2864aea1 2024-04-16T18:15:57.957 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 10e38449da57e59f5e42291109653d0d61bada3a47dde05f174165852709e9af 2024-04-16T18:15:57.957 DEBUG:botocore.auth:Signature: f04f4fba070c2606ae3e8a78a932f290570624f0ef0edc0e5a3b9fd666c240cc 2024-04-16T18:15:57.957 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:57.958 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.961 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000020d2093ef7724c86-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.961 DEBUG:botocore.parsers:Response body: b'tx0000020d2093ef7724c86-00661ec05d-4804-test-zone1f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:57.961 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:57.961 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.962 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:57.962 DEBUG:botocore.hooks:Event before-parameter-build.iam.GetRole: calling handler 2024-04-16T18:15:57.962 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.962 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:57.963 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.963 DEBUG:botocore.hooks:Event before-call.iam.GetRole: calling handler 2024-04-16T18:15:57.963 DEBUG:botocore.endpoint:Making request for OperationModel(name=GetRole) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'GetRole', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.963 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler > 2024-04-16T18:15:57.963 DEBUG:botocore.hooks:Event choose-signer.iam.GetRole: calling handler 2024-04-16T18:15:57.964 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.964 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 02e2c37e0f639aed605a56f302249a2397fee3f10d1ef5436cec797a2864aea1 2024-04-16T18:15:57.964 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request 7a72c0b71ae8108e030b2bb09fc21ef87bb78b22083238ed66bd5ef2fc28daab 2024-04-16T18:15:57.964 DEBUG:botocore.auth:Signature: 43bccf3b92034b9ab42fe9134a96d6f288b571c55ed33decb71e0aa5d6241cee 2024-04-16T18:15:57.964 DEBUG:botocore.hooks:Event request-created.iam.GetRole: calling handler 2024-04-16T18:15:57.965 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.967 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000d0552ca0ce78afb9-00661ec05d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '773', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.968 DEBUG:botocore.parsers:Response body: b'tx00000d0552ca0ce78afb9-00661ec05d-5397-test-zone2f5164008-448a-49cc-84d4-42cd815675f7roles-lybtug-2/arn:aws:iam::RGW11111111111111111:role/roles-lybtug-22024-04-16T18:15:35.352Z3600{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "arn:aws:iam:::user/testuser"}, "Action": ["sts:AssumeRole"]}]}' 2024-04-16T18:15:57.968 DEBUG:botocore.hooks:Event needs-retry.iam.GetRole: calling handler 2024-04-16T18:15:57.968 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.968 DEBUG:botocore.hooks:Event after-call.iam.GetRole: calling handler 2024-04-16T18:15:57.969 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.969 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.969 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.969 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.970 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.970 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.970 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:57.970 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.971 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.971 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date a1d084b46fad81a075761bd5542a6a0e50950ec6311a68cd737f0f1de5b600c0 2024-04-16T18:15:57.971 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request ea19ef0347dc9ac71ce05c4de4b47742d0a25504ae7855ec49d1ba9de51b7c3a 2024-04-16T18:15:57.971 DEBUG:botocore.auth:Signature: a4db3f27784f6e598fe0b0cbc14ee0bcc63eb83515d5984bb75cd5352da662a4 2024-04-16T18:15:57.971 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.971 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.974 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx00000c61fe28616256872-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.975 DEBUG:botocore.parsers:Response body: b'tx00000c61fe28616256872-00661ec05d-4804-test-zone1' 2024-04-16T18:15:57.975 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.975 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.975 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.975 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.975 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.976 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:57.976 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.976 DEBUG:botocore.hooks:Event before-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.976 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.976 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler > 2024-04-16T18:15:57.976 DEBUG:botocore.hooks:Event choose-signer.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.977 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.977 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date a1d084b46fad81a075761bd5542a6a0e50950ec6311a68cd737f0f1de5b600c0 2024-04-16T18:15:57.977 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request da5d7e59c937b6d41bed814ba559dbf31f140d48194a0e683cf3b8297405a02f 2024-04-16T18:15:57.977 DEBUG:botocore.auth:Signature: 96e5d7694b3ea668e4bfda442b79508b099afc882ed7ad271afe814fb8ac57d7 2024-04-16T18:15:57.977 DEBUG:botocore.hooks:Event request-created.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.978 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:57.981 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000082265650bd4275cf-00661ec05d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '239', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:57.981 DEBUG:botocore.parsers:Response body: b'tx0000082265650bd4275cf-00661ec05d-5397-test-zone2' 2024-04-16T18:15:57.981 DEBUG:botocore.hooks:Event needs-retry.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.981 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:57.981 DEBUG:botocore.hooks:Event after-call.iam.ListRolePolicies: calling handler 2024-04-16T18:15:57.982 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.982 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:57.982 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:57.982 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.982 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.983 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:57.983 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:57.983 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.983 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:57.983 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181557Z content-type;host;x-amz-date 84b65f4bc9cead3a6678f4d4a45b599660046d1017f83ffab2a97c93c704c904 2024-04-16T18:15:57.984 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181557Z 20240416/test-zonegroup/iam/aws4_request ba7f520a336c9a223b875d47b10c850bdf0038e6b25707e484e3754a1eaf05f4 2024-04-16T18:15:57.984 DEBUG:botocore.auth:Signature: b5468d38d3d3d52a2d8a175c0aca27888fd9294d7031fedddd826a8af89de3cc 2024-04-16T18:15:57.984 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:57.984 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.030 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000000de42301c1dc32d4-00661ec05d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:57 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.031 DEBUG:botocore.parsers:Response body: b'tx000000de42301c1dc32d4-00661ec05d-4804-test-zone1' 2024-04-16T18:15:58.031 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.031 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.031 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.032 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.032 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.032 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:58.032 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.032 DEBUG:botocore.hooks:Event before-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.033 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListAttachedRolePolicies) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListAttachedRolePolicies', 'Version': '2010-05-08', 'RoleName': 'roles-lybtug-2'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.033 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler > 2024-04-16T18:15:58.033 DEBUG:botocore.hooks:Event choose-signer.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.033 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.034 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date 84b65f4bc9cead3a6678f4d4a45b599660046d1017f83ffab2a97c93c704c904 2024-04-16T18:15:58.034 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request e6c111712eaecee04a4d2c44bb12018f3b0da1000a5f485006a03f9132f4b30c 2024-04-16T18:15:58.034 DEBUG:botocore.auth:Signature: c2c9a8ffb8f81f1e717e75fa98567f38470026920e8c23d79184a034bef99641 2024-04-16T18:15:58.034 DEBUG:botocore.hooks:Event request-created.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.034 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.037 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000018da145d477616ab-00661ec05e-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '331', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.037 DEBUG:botocore.parsers:Response body: b'tx0000018da145d477616ab-00661ec05e-5397-test-zone2' 2024-04-16T18:15:58.038 DEBUG:botocore.hooks:Event needs-retry.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.038 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.038 DEBUG:botocore.hooks:Event after-call.iam.ListAttachedRolePolicies: calling handler 2024-04-16T18:15:58.038 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUsers: calling handler 2024-04-16T18:15:58.039 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.039 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.039 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:58.039 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:58.039 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUsers) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUsers', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.039 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler > 2024-04-16T18:15:58.040 DEBUG:botocore.hooks:Event choose-signer.iam.ListUsers: calling handler 2024-04-16T18:15:58.040 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.040 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date b6359072c78d70ebee1e81adcbab4f01bf2c23245fa365ef83fe8f1f955085e2 2024-04-16T18:15:58.040 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request 6d72736eb127d62339695c16633bc5ad1bd51fc936be000e304383f7052e929a 2024-04-16T18:15:58.040 DEBUG:botocore.auth:Signature: fc0541e77cd897bba44d3228893daa9384b565d4970dc0899a7ca5dca2f2d234 2024-04-16T18:15:58.041 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler 2024-04-16T18:15:58.041 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.045 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx000005b5f337c20defbcf-00661ec05e-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.045 DEBUG:botocore.parsers:Response body: b'false' 2024-04-16T18:15:58.045 DEBUG:botocore.hooks:Event needs-retry.iam.ListUsers: calling handler 2024-04-16T18:15:58.046 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.046 DEBUG:botocore.hooks:Event after-call.iam.ListUsers: calling handler 2024-04-16T18:15:58.046 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListUsers: calling handler 2024-04-16T18:15:58.046 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.046 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:58.046 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:58.047 DEBUG:botocore.hooks:Event before-call.iam.ListUsers: calling handler 2024-04-16T18:15:58.047 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListUsers) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListUsers', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.047 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler > 2024-04-16T18:15:58.047 DEBUG:botocore.hooks:Event choose-signer.iam.ListUsers: calling handler 2024-04-16T18:15:58.047 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.048 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date b6359072c78d70ebee1e81adcbab4f01bf2c23245fa365ef83fe8f1f955085e2 2024-04-16T18:15:58.048 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request 6649c81787d20cfea19f33087fcd7046b417e1f4026ff2eefc90017468cb673c 2024-04-16T18:15:58.048 DEBUG:botocore.auth:Signature: a8c8f1e60e64a93ba893d4258cd6fdd8fe1c8e964523180e19ab96d4a0626f73 2024-04-16T18:15:58.048 DEBUG:botocore.hooks:Event request-created.iam.ListUsers: calling handler 2024-04-16T18:15:58.048 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.053 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx000006fe7e7f984bf75a6-00661ec05e-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.053 DEBUG:botocore.parsers:Response body: b'false' 2024-04-16T18:15:58.053 DEBUG:botocore.hooks:Event needs-retry.iam.ListUsers: calling handler 2024-04-16T18:15:58.054 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.054 DEBUG:botocore.hooks:Event after-call.iam.ListUsers: calling handler 2024-04-16T18:15:58.054 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroups: calling handler 2024-04-16T18:15:58.054 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.054 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.055 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:58.055 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:58.055 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroups) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroups', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.055 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler > 2024-04-16T18:15:58.055 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroups: calling handler 2024-04-16T18:15:58.056 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.056 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date 5f776d91509b9c99b8cb5eb5d6d4a787a33ae41c8cd6e7b69effca69080e1e1f 2024-04-16T18:15:58.056 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request 06adf6a25c5810e088808ef48c72bfd8232b8787ef8e27e8e6cae67f690958ab 2024-04-16T18:15:58.056 DEBUG:botocore.auth:Signature: 56073bc9abd2936c3555e473fa3e006ae694498c1cd29dd41a430cb70d0a3734 2024-04-16T18:15:58.056 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler 2024-04-16T18:15:58.056 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.061 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000dfbe380983acc6e2-00661ec05e-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.061 DEBUG:botocore.parsers:Response body: b'false' 2024-04-16T18:15:58.061 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroups: calling handler 2024-04-16T18:15:58.061 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.061 DEBUG:botocore.hooks:Event after-call.iam.ListGroups: calling handler 2024-04-16T18:15:58.062 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListGroups: calling handler 2024-04-16T18:15:58.062 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.062 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:58.062 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:58.062 DEBUG:botocore.hooks:Event before-call.iam.ListGroups: calling handler 2024-04-16T18:15:58.063 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListGroups) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListGroups', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.063 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler > 2024-04-16T18:15:58.063 DEBUG:botocore.hooks:Event choose-signer.iam.ListGroups: calling handler 2024-04-16T18:15:58.063 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.063 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date 5f776d91509b9c99b8cb5eb5d6d4a787a33ae41c8cd6e7b69effca69080e1e1f 2024-04-16T18:15:58.064 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request 0fe798f36dfe097beeb189841443468c3343f9616464c9bc69024666a958d1e6 2024-04-16T18:15:58.064 DEBUG:botocore.auth:Signature: 2d5e6edc683c7ad44dee2c35760b48bfee61447d5ba268c1e486a6bea06b5d7e 2024-04-16T18:15:58.064 DEBUG:botocore.hooks:Event request-created.iam.ListGroups: calling handler 2024-04-16T18:15:58.064 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.110 DEBUG:botocore.parsers:Response headers: {'Transfer-Encoding': 'chunked', 'x-amz-request-id': 'tx00000a1fd190cd6fd5675-00661ec05e-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.111 DEBUG:botocore.parsers:Response body: b'false' 2024-04-16T18:15:58.111 DEBUG:botocore.hooks:Event needs-retry.iam.ListGroups: calling handler 2024-04-16T18:15:58.111 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.111 DEBUG:botocore.hooks:Event after-call.iam.ListGroups: calling handler 2024-04-16T18:15:58.112 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.112 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.112 DEBUG:botocore.regions:Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.112 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.112 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.112 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListOpenIDConnectProviders) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListOpenIDConnectProviders', 'Version': '2010-05-08'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.113 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler > 2024-04-16T18:15:58.113 DEBUG:botocore.hooks:Event choose-signer.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.113 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.113 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi071.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date 6be253d5bef7fe9cb4ed0d00d8a18351f59593888e86906ef784f147279adb10 2024-04-16T18:15:58.113 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request 8a701913696e79b9f9ae6fd4494869b660d9303225bf05566b33c885d47946a3 2024-04-16T18:15:58.114 DEBUG:botocore.auth:Signature: 9a486655e1df64837e7cc28416531f2fa99278605b639f4873f6d82242517ac5 2024-04-16T18:15:58.114 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.114 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.143 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx0000039ab451f4fe1d80e-00661ec05e-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '357', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.143 DEBUG:botocore.parsers:Response body: b'tx0000039ab451f4fe1d80e-00661ec05e-4804-test-zone1' 2024-04-16T18:15:58.143 DEBUG:botocore.hooks:Event needs-retry.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.143 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.144 DEBUG:botocore.hooks:Event after-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.144 DEBUG:botocore.hooks:Event before-parameter-build.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.144 DEBUG:botocore.regions:Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T18:15:58.144 DEBUG:botocore.regions:Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:15:58.144 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.144 DEBUG:botocore.hooks:Event before-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.145 DEBUG:botocore.endpoint:Making request for OperationModel(name=ListOpenIDConnectProviders) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListOpenIDConnectProviders', 'Version': '2010-05-08'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T18:15:58.145 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler > 2024-04-16T18:15:58.145 DEBUG:botocore.hooks:Event choose-signer.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.145 DEBUG:botocore.auth:Calculating signature using v4 auth. 2024-04-16T18:15:58.146 DEBUG:botocore.auth:CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:smithi122.front.sepia.ceph.com:8000 x-amz-date:20240416T181558Z content-type;host;x-amz-date 6be253d5bef7fe9cb4ed0d00d8a18351f59593888e86906ef784f147279adb10 2024-04-16T18:15:58.146 DEBUG:botocore.auth:StringToSign: AWS4-HMAC-SHA256 20240416T181558Z 20240416/test-zonegroup/iam/aws4_request 2ae85632680899d1a5db922a600f784d79fb398b32a1cdb596f5a6eaec9915b9 2024-04-16T18:15:58.146 DEBUG:botocore.auth:Signature: a394394ed0ab485ed0b83d02940743a079dd7614f1356ea3a0c07aa4d0307d73 2024-04-16T18:15:58.146 DEBUG:botocore.hooks:Event request-created.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.146 DEBUG:botocore.endpoint:Sending http request: 2024-04-16T18:15:58.177 DEBUG:botocore.parsers:Response headers: {'x-amz-request-id': 'tx000006aea6bc55ceaed80-00661ec05e-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '357', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T18:15:58.177 DEBUG:botocore.parsers:Response body: b'tx000006aea6bc55ceaed80-00661ec05e-5397-test-zone2' 2024-04-16T18:15:58.177 DEBUG:botocore.hooks:Event needs-retry.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.177 DEBUG:botocore.retryhandler:No retry needed. 2024-04-16T18:15:58.177 DEBUG:botocore.hooks:Event after-call.iam.ListOpenIDConnectProviders: calling handler 2024-04-16T18:15:58.178 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_account_metadata_sync ... ok 2024-04-16T18:15:58.179 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=lybtug-50 2024-04-16T18:15:58.179 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:15:58.179 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:15:58.180 DEBUG:boto:Method: PUT 2024-04-16T18:15:58.180 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:15:58.180 DEBUG:boto:Data: 2024-04-16T18:15:58.180 DEBUG:boto:Headers: {} 2024-04-16T18:15:58.180 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.180 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.180 DEBUG:boto:Params: {} 2024-04-16T18:15:58.181 DEBUG:boto:Token: None 2024-04-16T18:15:58.181 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/ 2024-04-16T18:15:58.181 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uvhsFsGYw4ICde9McLjlrHMnkdg= 2024-04-16T18:15:58.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uvhsFsGYw4ICde9McLjlrHMnkdg=', 'Content-Length': '0'} 2024-04-16T18:15:58.217 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000615ed1d6aacb179f-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.217 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:15:58.217 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:15:58.217 DEBUG:boto:Method: HEAD 2024-04-16T18:15:58.217 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:15:58.218 DEBUG:boto:Data: 2024-04-16T18:15:58.218 DEBUG:boto:Headers: {} 2024-04-16T18:15:58.218 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.218 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.218 DEBUG:boto:Params: {} 2024-04-16T18:15:58.218 DEBUG:boto:Token: None 2024-04-16T18:15:58.218 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/ 2024-04-16T18:15:58.219 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T18:15:58.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T18:15:58.223 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b23e300758907cb0-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.223 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:15:58.223 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:15:58.224 DEBUG:boto:Method: PUT 2024-04-16T18:15:58.224 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:15:58.224 DEBUG:boto:Data: 2024-04-16T18:15:58.224 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:15:58.224 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.224 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.224 DEBUG:boto:Params: {} 2024-04-16T18:15:58.224 DEBUG:boto:Token: None 2024-04-16T18:15:58.225 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/a 2024-04-16T18:15:58.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5thL707kPxD7kJii1Vpgz2cqPyE= 2024-04-16T18:15:58.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5thL707kPxD7kJii1Vpgz2cqPyE='} 2024-04-16T18:15:58.275 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d4f0a17730323df4-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.275 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:15:58.275 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:15:58.275 DEBUG:boto:Method: HEAD 2024-04-16T18:15:58.275 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:15:58.275 DEBUG:boto:Data: 2024-04-16T18:15:58.276 DEBUG:boto:Headers: {} 2024-04-16T18:15:58.276 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.276 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.276 DEBUG:boto:Params: {} 2024-04-16T18:15:58.276 DEBUG:boto:Token: None 2024-04-16T18:15:58.276 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/ 2024-04-16T18:15:58.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T18:15:58.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T18:15:58.280 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007a223f0b542ae158-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.281 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:15:58.281 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:15:58.281 DEBUG:boto:Method: PUT 2024-04-16T18:15:58.281 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:15:58.281 DEBUG:boto:Data: 2024-04-16T18:15:58.281 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:15:58.281 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.282 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.282 DEBUG:boto:Params: {} 2024-04-16T18:15:58.282 DEBUG:boto:Token: None 2024-04-16T18:15:58.282 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/b 2024-04-16T18:15:58.282 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9syNMp1yVQR0SM+A4Ezlh3in78c= 2024-04-16T18:15:58.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9syNMp1yVQR0SM+A4Ezlh3in78c='} 2024-04-16T18:15:58.296 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001dde57b76928a1d4-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.297 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:15:58.297 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:15:58.297 DEBUG:boto:Method: HEAD 2024-04-16T18:15:58.297 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:15:58.297 DEBUG:boto:Data: 2024-04-16T18:15:58.297 DEBUG:boto:Headers: {} 2024-04-16T18:15:58.297 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.297 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.298 DEBUG:boto:Params: {} 2024-04-16T18:15:58.298 DEBUG:boto:Token: None 2024-04-16T18:15:58.298 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/ 2024-04-16T18:15:58.298 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T18:15:58.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T18:15:58.302 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009ac16bb7360a73d3-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.302 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:15:58.303 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:15:58.303 DEBUG:boto:Method: PUT 2024-04-16T18:15:58.303 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:15:58.303 DEBUG:boto:Data: 2024-04-16T18:15:58.303 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:15:58.303 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.303 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.304 DEBUG:boto:Params: {} 2024-04-16T18:15:58.304 DEBUG:boto:Token: None 2024-04-16T18:15:58.304 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/c 2024-04-16T18:15:58.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vbCO5WoKisUgAkk0p8aTdcTGTFs= 2024-04-16T18:15:58.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vbCO5WoKisUgAkk0p8aTdcTGTFs='} 2024-04-16T18:15:58.347 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b1f0ed1038f6bbfb-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.347 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:15:58.347 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:15:58.347 DEBUG:boto:Method: HEAD 2024-04-16T18:15:58.347 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:15:58.347 DEBUG:boto:Data: 2024-04-16T18:15:58.347 DEBUG:boto:Headers: {} 2024-04-16T18:15:58.348 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.348 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.348 DEBUG:boto:Params: {} 2024-04-16T18:15:58.348 DEBUG:boto:Token: None 2024-04-16T18:15:58.348 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/ 2024-04-16T18:15:58.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T18:15:58.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T18:15:58.352 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ecaa10b96fbb8703-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.353 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:15:58.353 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:15:58.353 DEBUG:boto:Method: PUT 2024-04-16T18:15:58.353 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:15:58.353 DEBUG:boto:Data: 2024-04-16T18:15:58.353 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:15:58.354 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:15:58.354 DEBUG:boto:Port: 8000 2024-04-16T18:15:58.354 DEBUG:boto:Params: {} 2024-04-16T18:15:58.354 DEBUG:boto:Token: None 2024-04-16T18:15:58.354 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:15:58 GMT /lybtug-50/d 2024-04-16T18:15:58.354 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kHxBMgSUbPDcTq6g5dA/pts8Kt0= 2024-04-16T18:15:58.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kHxBMgSUbPDcTq6g5dA/pts8Kt0='} 2024-04-16T18:15:58.367 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001c3bcc8d42f5e803-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:15:58.368 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:58.368 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:58.368 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:58.409 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:58.409 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:58.479 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:15:58.511 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291358.216212_532.1","last_update":"2024-04-16T18:15:58.216212Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:15:58.512 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291358.216212_532.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:58.512 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:15:58.512 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:58.512 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:15:58.512 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:15:58.553 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:58.554 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:58.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:58.623+0000 7fd3501be980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:58.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:58.623+0000 7fd3501be980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:15:58.654 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291358.216212_532.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:58.216212Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:58Z"} 2024-04-16T18:15:58.654 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291358.216212_532.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:58.216212Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:58Z"} 2024-04-16T18:15:58.654 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291358.216212_532.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:58.216212Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:15:58.655 DEBUG:rgw_multi.tests:log_status={0: '1_1713291358.216212_532.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:58.655 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291358.216212_532.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:15:58.655 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:15:58.655 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-50 2024-04-16T18:15:58.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:15:58.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:15:58.656 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:15:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:15:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:15:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:15:58.827+0000 7fad9dacb980 1 waiting to reach incremental sync.. 2024-04-16T18:16:03.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:03.830+0000 7fad9dacb980 1 bucket sync caught up with source: 2024-04-16T18:16:03.830 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, 00000000001.55.6, , , , 00000000001.67.6, , , 00000000001.46.6, , 00000000001.53.6] 2024-04-16T18:16:03.830 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, 00000000001.55.6, , , , 00000000001.67.6, , , 00000000001.46.6, , 00000000001.53.6] 2024-04-16T18:16:03.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:03.830+0000 7fad9dacb980 0 bucket checkpoint complete 2024-04-16T18:16:03.873 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-50 2024-04-16T18:16:03.873 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:16:03.874 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:16:03.874 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:16:03.925 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:03.925 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:04.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:16:04.006+0000 7fb7e31d9980 1 waiting to reach incremental sync.. 2024-04-16T18:16:09.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:16:09.006+0000 7fb7e31d9980 1 waiting to reach incremental sync.. 2024-04-16T18:16:14.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:16:14.006+0000 7fb7e31d9980 1 bucket sync caught up with source: 2024-04-16T18:16:14.005 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.901.6, , , , 00000000001.1167.6, , , 00000000001.306.6, , 00000000001.407.6] 2024-04-16T18:16:14.005 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.901.6, , , , 00000000001.1167.6, , , 00000000001.306.6, , 00000000001.407.6] 2024-04-16T18:16:14.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:16:14.006+0000 7fb7e31d9980 0 bucket checkpoint complete 2024-04-16T18:16:14.051 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T18:16:14.051 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.051 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.051 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.051 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:16:14.052 DEBUG:boto:Data: 2024-04-16T18:16:14.052 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.052 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.052 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.052 DEBUG:boto:Params: {} 2024-04-16T18:16:14.052 DEBUG:boto:Token: None 2024-04-16T18:16:14.052 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0= 2024-04-16T18:16:14.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0=', 'Content-Length': '0'} 2024-04-16T18:16:14.056 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000250fba0e42c19b5b-00661ec06e-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.057 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.057 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.057 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.057 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:16:14.057 DEBUG:boto:Data: 2024-04-16T18:16:14.057 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.057 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.057 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.057 DEBUG:boto:Params: {} 2024-04-16T18:16:14.057 DEBUG:boto:Token: None 2024-04-16T18:16:14.057 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.058 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0= 2024-04-16T18:16:14.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0=', 'Content-Length': '0'} 2024-04-16T18:16:14.061 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ccd602ab393da906-00661ec06e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.061 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:16:14.061 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.062 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.062 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.062 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.062 DEBUG:boto:Method: GET 2024-04-16T18:16:14.062 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.062 DEBUG:boto:Data: 2024-04-16T18:16:14.062 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.062 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.062 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.062 DEBUG:boto:Params: {} 2024-04-16T18:16:14.062 DEBUG:boto:Token: None 2024-04-16T18:16:14.063 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/?versions 2024-04-16T18:16:14.063 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T18:16:14.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T18:16:14.067 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000489c48a5765b016a-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.067 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.068 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:16:14.068 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:16:14.068 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:16:14.068 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:16:14.068 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:16:14.068 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.069 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.069 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.069 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.069 DEBUG:boto:Method: GET 2024-04-16T18:16:14.069 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.069 DEBUG:boto:Data: 2024-04-16T18:16:14.069 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.069 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.069 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.069 DEBUG:boto:Params: {} 2024-04-16T18:16:14.069 DEBUG:boto:Token: None 2024-04-16T18:16:14.070 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/?versions 2024-04-16T18:16:14.070 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T18:16:14.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T18:16:14.073 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067e9cd392d9b304e-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.074 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.074 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:16:14.074 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:16:14.074 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:16:14.074 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:16:14.074 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.074 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.075 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.075 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.075 DEBUG:boto:Method: GET 2024-04-16T18:16:14.075 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.075 DEBUG:boto:Data: 2024-04-16T18:16:14.075 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.075 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.075 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.075 DEBUG:boto:Params: {} 2024-04-16T18:16:14.075 DEBUG:boto:Token: None 2024-04-16T18:16:14.075 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/?versions 2024-04-16T18:16:14.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T18:16:14.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T18:16:14.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a76fa31306ba956d-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.080 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.080 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.081 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.081 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.081 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.081 DEBUG:boto:Method: GET 2024-04-16T18:16:14.081 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:16:14.081 DEBUG:boto:Data: 2024-04-16T18:16:14.081 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.081 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.081 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.081 DEBUG:boto:Params: {} 2024-04-16T18:16:14.081 DEBUG:boto:Token: None 2024-04-16T18:16:14.081 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/?versions 2024-04-16T18:16:14.082 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T18:16:14.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T18:16:14.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6174e676fde57f9-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.086 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.086 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:16:14.086 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.086 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.086 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.086 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.087 DEBUG:boto:Method: GET 2024-04-16T18:16:14.087 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:16:14.087 DEBUG:boto:Data: 2024-04-16T18:16:14.087 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.087 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.087 DEBUG:boto:Params: {} 2024-04-16T18:16:14.087 DEBUG:boto:Token: None 2024-04-16T18:16:14.087 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a?versionId=null 2024-04-16T18:16:14.087 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T18:16:14.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T18:16:14.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a49399ff1a7ab6b6-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.134 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.134 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.135 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.135 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.135 DEBUG:boto:Method: GET 2024-04-16T18:16:14.135 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:16:14.135 DEBUG:boto:Data: 2024-04-16T18:16:14.135 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.135 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.135 DEBUG:boto:Params: {} 2024-04-16T18:16:14.135 DEBUG:boto:Token: None 2024-04-16T18:16:14.135 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a?versionId=null 2024-04-16T18:16:14.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T18:16:14.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T18:16:14.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013d5812e7cd278ef-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.139 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.139 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.139 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.139 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.139 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.139 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:16:14.140 DEBUG:boto:Data: 2024-04-16T18:16:14.140 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.140 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.140 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.140 DEBUG:boto:Params: {} 2024-04-16T18:16:14.140 DEBUG:boto:Token: None 2024-04-16T18:16:14.140 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a?versionId=null 2024-04-16T18:16:14.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY= 2024-04-16T18:16:14.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY=', 'Content-Length': '0'} 2024-04-16T18:16:14.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000637dd4dd264e4d2d-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.143 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.143 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.144 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.144 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.144 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.144 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:16:14.144 DEBUG:boto:Data: 2024-04-16T18:16:14.144 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.144 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.144 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.144 DEBUG:boto:Params: {} 2024-04-16T18:16:14.144 DEBUG:boto:Token: None 2024-04-16T18:16:14.145 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a?versionId=null 2024-04-16T18:16:14.145 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY= 2024-04-16T18:16:14.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY=', 'Content-Length': '0'} 2024-04-16T18:16:14.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004820bd4c9a562e31-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.148 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:16:14.148 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.148 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.148 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.148 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.148 DEBUG:boto:Method: GET 2024-04-16T18:16:14.148 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:16:14.148 DEBUG:boto:Data: 2024-04-16T18:16:14.148 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.148 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.149 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.149 DEBUG:boto:Params: {} 2024-04-16T18:16:14.149 DEBUG:boto:Token: None 2024-04-16T18:16:14.149 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a?versionId=null 2024-04-16T18:16:14.149 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T18:16:14.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T18:16:14.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004982d41bb25cb472-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.194 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.194 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.195 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.195 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:16:14.195 DEBUG:boto:Method: GET 2024-04-16T18:16:14.195 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:16:14.195 DEBUG:boto:Data: 2024-04-16T18:16:14.195 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.195 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.195 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.195 DEBUG:boto:Params: {} 2024-04-16T18:16:14.195 DEBUG:boto:Token: None 2024-04-16T18:16:14.196 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a?versionId=null 2024-04-16T18:16:14.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T18:16:14.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T18:16:14.198 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df21cbd0cb1452c7-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.199 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.199 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.199 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.199 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.199 DEBUG:boto:Method: GET 2024-04-16T18:16:14.199 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:16:14.199 DEBUG:boto:Data: 2024-04-16T18:16:14.199 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.199 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.199 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.200 DEBUG:boto:Params: {} 2024-04-16T18:16:14.200 DEBUG:boto:Token: None 2024-04-16T18:16:14.200 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d4ff0da215c30d9-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.246 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.247 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.247 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.247 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.247 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.247 DEBUG:boto:Method: GET 2024-04-16T18:16:14.247 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:16:14.247 DEBUG:boto:Data: 2024-04-16T18:16:14.247 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.247 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.247 DEBUG:boto:Params: {} 2024-04-16T18:16:14.248 DEBUG:boto:Token: None 2024-04-16T18:16:14.248 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.252 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ded40dd8ec6580c7-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.252 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.253 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.253 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.253 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.253 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:16:14.253 DEBUG:boto:Data: 2024-04-16T18:16:14.253 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.253 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.253 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.253 DEBUG:boto:Params: {} 2024-04-16T18:16:14.254 DEBUG:boto:Token: None 2024-04-16T18:16:14.254 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a 2024-04-16T18:16:14.254 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE= 2024-04-16T18:16:14.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE=', 'Content-Length': '0'} 2024-04-16T18:16:14.257 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b63a7e689d61f9-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.257 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.257 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.257 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.257 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:16:14.257 DEBUG:boto:Data: 2024-04-16T18:16:14.257 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.257 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.258 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.258 DEBUG:boto:Params: {} 2024-04-16T18:16:14.258 DEBUG:boto:Token: None 2024-04-16T18:16:14.258 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a 2024-04-16T18:16:14.258 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE= 2024-04-16T18:16:14.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE=', 'Content-Length': '0'} 2024-04-16T18:16:14.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5db143facb468d2-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.261 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:16:14.261 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.261 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.262 DEBUG:boto:Method: GET 2024-04-16T18:16:14.262 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:16:14.262 DEBUG:boto:Data: 2024-04-16T18:16:14.262 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.262 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.262 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.262 DEBUG:boto:Params: {} 2024-04-16T18:16:14.262 DEBUG:boto:Token: None 2024-04-16T18:16:14.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a 2024-04-16T18:16:14.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I= 2024-04-16T18:16:14.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I=', 'Content-Length': '0'} 2024-04-16T18:16:14.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a3ed1953b6978e1-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.306 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:16:14.306 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:16:14.306 DEBUG:boto:Method: GET 2024-04-16T18:16:14.307 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:16:14.307 DEBUG:boto:Data: 2024-04-16T18:16:14.307 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.307 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.307 DEBUG:boto:Params: {} 2024-04-16T18:16:14.307 DEBUG:boto:Token: None 2024-04-16T18:16:14.307 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/a 2024-04-16T18:16:14.307 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I= 2024-04-16T18:16:14.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I=', 'Content-Length': '0'} 2024-04-16T18:16:14.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d72aca4917f6b16-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.311 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.311 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.311 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.311 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.311 DEBUG:boto:Method: GET 2024-04-16T18:16:14.311 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:16:14.311 DEBUG:boto:Data: 2024-04-16T18:16:14.311 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.311 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.312 DEBUG:boto:Params: {} 2024-04-16T18:16:14.312 DEBUG:boto:Token: None 2024-04-16T18:16:14.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.316 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001229000d3ae140ae-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.316 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.316 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.316 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.316 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.317 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:16:14.317 DEBUG:boto:Method: GET 2024-04-16T18:16:14.317 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:16:14.317 DEBUG:boto:Data: 2024-04-16T18:16:14.317 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.317 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.317 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.317 DEBUG:boto:Params: {} 2024-04-16T18:16:14.317 DEBUG:boto:Token: None 2024-04-16T18:16:14.317 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.318 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.321 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab16a75e8c34797f-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.322 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.322 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:16:14.322 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.322 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.322 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.322 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.322 DEBUG:boto:Method: GET 2024-04-16T18:16:14.322 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:16:14.323 DEBUG:boto:Data: 2024-04-16T18:16:14.323 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.323 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.323 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.323 DEBUG:boto:Params: {} 2024-04-16T18:16:14.323 DEBUG:boto:Token: None 2024-04-16T18:16:14.323 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b?versionId=null 2024-04-16T18:16:14.323 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T18:16:14.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T18:16:14.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001000939febb5de50-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.370 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.370 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.371 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.371 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.371 DEBUG:boto:Method: GET 2024-04-16T18:16:14.371 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:16:14.371 DEBUG:boto:Data: 2024-04-16T18:16:14.371 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.371 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.371 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.371 DEBUG:boto:Params: {} 2024-04-16T18:16:14.371 DEBUG:boto:Token: None 2024-04-16T18:16:14.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b?versionId=null 2024-04-16T18:16:14.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T18:16:14.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T18:16:14.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0576bc2956d65f9-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.375 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.375 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.375 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.375 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.375 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.375 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:16:14.375 DEBUG:boto:Data: 2024-04-16T18:16:14.376 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.376 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.376 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.376 DEBUG:boto:Params: {} 2024-04-16T18:16:14.376 DEBUG:boto:Token: None 2024-04-16T18:16:14.376 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b?versionId=null 2024-04-16T18:16:14.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg= 2024-04-16T18:16:14.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg=', 'Content-Length': '0'} 2024-04-16T18:16:14.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bf4fc1e91d52cb3-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.379 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.380 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.380 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.380 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.380 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.380 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:16:14.380 DEBUG:boto:Data: 2024-04-16T18:16:14.380 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.380 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.380 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.380 DEBUG:boto:Params: {} 2024-04-16T18:16:14.380 DEBUG:boto:Token: None 2024-04-16T18:16:14.381 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b?versionId=null 2024-04-16T18:16:14.381 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg= 2024-04-16T18:16:14.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg=', 'Content-Length': '0'} 2024-04-16T18:16:14.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd8eec98ee657eda-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.384 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:16:14.384 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.384 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.384 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.384 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.384 DEBUG:boto:Method: GET 2024-04-16T18:16:14.384 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:16:14.384 DEBUG:boto:Data: 2024-04-16T18:16:14.384 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.384 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.384 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.384 DEBUG:boto:Params: {} 2024-04-16T18:16:14.385 DEBUG:boto:Token: None 2024-04-16T18:16:14.385 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b?versionId=null 2024-04-16T18:16:14.385 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T18:16:14.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T18:16:14.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ccff8d860766ff0-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.430 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.430 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.431 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.431 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:16:14.431 DEBUG:boto:Method: GET 2024-04-16T18:16:14.431 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:16:14.431 DEBUG:boto:Data: 2024-04-16T18:16:14.431 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.431 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.431 DEBUG:boto:Params: {} 2024-04-16T18:16:14.431 DEBUG:boto:Token: None 2024-04-16T18:16:14.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b?versionId=null 2024-04-16T18:16:14.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T18:16:14.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T18:16:14.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071ea13b6b5c1d357-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.435 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.435 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.435 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.435 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.435 DEBUG:boto:Method: GET 2024-04-16T18:16:14.435 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:16:14.435 DEBUG:boto:Data: 2024-04-16T18:16:14.435 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.436 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.436 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.436 DEBUG:boto:Params: {} 2024-04-16T18:16:14.436 DEBUG:boto:Token: None 2024-04-16T18:16:14.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000492702019b4eaa90-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.482 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.483 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.483 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.483 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.483 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.483 DEBUG:boto:Method: GET 2024-04-16T18:16:14.483 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:16:14.483 DEBUG:boto:Data: 2024-04-16T18:16:14.483 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.483 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.484 DEBUG:boto:Params: {} 2024-04-16T18:16:14.484 DEBUG:boto:Token: None 2024-04-16T18:16:14.484 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5cb1932fa9d2967-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.488 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.488 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.489 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.489 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.489 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:16:14.489 DEBUG:boto:Data: 2024-04-16T18:16:14.489 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.489 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.489 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.489 DEBUG:boto:Params: {} 2024-04-16T18:16:14.489 DEBUG:boto:Token: None 2024-04-16T18:16:14.489 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b 2024-04-16T18:16:14.490 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ= 2024-04-16T18:16:14.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ=', 'Content-Length': '0'} 2024-04-16T18:16:14.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001dc45b379570665f-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.493 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.493 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.493 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.493 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:16:14.493 DEBUG:boto:Data: 2024-04-16T18:16:14.493 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.493 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.493 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.493 DEBUG:boto:Params: {} 2024-04-16T18:16:14.493 DEBUG:boto:Token: None 2024-04-16T18:16:14.494 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b 2024-04-16T18:16:14.494 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ= 2024-04-16T18:16:14.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ=', 'Content-Length': '0'} 2024-04-16T18:16:14.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d640d94f9df5ebbc-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.497 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:16:14.497 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.497 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.497 DEBUG:boto:Method: GET 2024-04-16T18:16:14.497 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:16:14.497 DEBUG:boto:Data: 2024-04-16T18:16:14.497 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.498 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.498 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.498 DEBUG:boto:Params: {} 2024-04-16T18:16:14.498 DEBUG:boto:Token: None 2024-04-16T18:16:14.498 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b 2024-04-16T18:16:14.498 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI= 2024-04-16T18:16:14.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI=', 'Content-Length': '0'} 2024-04-16T18:16:14.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2894c2a5dac881b-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.542 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:16:14.542 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:16:14.543 DEBUG:boto:Method: GET 2024-04-16T18:16:14.543 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:16:14.543 DEBUG:boto:Data: 2024-04-16T18:16:14.543 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.543 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.543 DEBUG:boto:Params: {} 2024-04-16T18:16:14.543 DEBUG:boto:Token: None 2024-04-16T18:16:14.543 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/b 2024-04-16T18:16:14.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI= 2024-04-16T18:16:14.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI=', 'Content-Length': '0'} 2024-04-16T18:16:14.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008aac2e293166aca2-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.547 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.547 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.547 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.547 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.547 DEBUG:boto:Method: GET 2024-04-16T18:16:14.547 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:16:14.547 DEBUG:boto:Data: 2024-04-16T18:16:14.547 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.547 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.547 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.547 DEBUG:boto:Params: {} 2024-04-16T18:16:14.548 DEBUG:boto:Token: None 2024-04-16T18:16:14.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.551 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f406332fb602772-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.552 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.552 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.552 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.552 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.552 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:16:14.552 DEBUG:boto:Method: GET 2024-04-16T18:16:14.553 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:16:14.553 DEBUG:boto:Data: 2024-04-16T18:16:14.553 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.553 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.553 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.553 DEBUG:boto:Params: {} 2024-04-16T18:16:14.553 DEBUG:boto:Token: None 2024-04-16T18:16:14.553 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.557 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a8eb04d9177f1f9-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.558 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.558 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:16:14.558 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.558 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.558 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.558 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.558 DEBUG:boto:Method: GET 2024-04-16T18:16:14.559 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:16:14.559 DEBUG:boto:Data: 2024-04-16T18:16:14.559 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.559 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.559 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.559 DEBUG:boto:Params: {} 2024-04-16T18:16:14.559 DEBUG:boto:Token: None 2024-04-16T18:16:14.559 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c?versionId=null 2024-04-16T18:16:14.559 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T18:16:14.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T18:16:14.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f4d54b91eb7d911-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.606 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.606 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.606 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.607 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.607 DEBUG:boto:Method: GET 2024-04-16T18:16:14.607 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:16:14.607 DEBUG:boto:Data: 2024-04-16T18:16:14.607 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.607 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.607 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.607 DEBUG:boto:Params: {} 2024-04-16T18:16:14.607 DEBUG:boto:Token: None 2024-04-16T18:16:14.607 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c?versionId=null 2024-04-16T18:16:14.607 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T18:16:14.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T18:16:14.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d44ff7844ace201-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.611 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.611 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.611 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.611 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.611 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.611 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:16:14.611 DEBUG:boto:Data: 2024-04-16T18:16:14.611 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.611 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.611 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.611 DEBUG:boto:Params: {} 2024-04-16T18:16:14.611 DEBUG:boto:Token: None 2024-04-16T18:16:14.612 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c?versionId=null 2024-04-16T18:16:14.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM= 2024-04-16T18:16:14.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM=', 'Content-Length': '0'} 2024-04-16T18:16:14.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbd3ddefb5d3cbd2-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.615 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.615 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.615 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.615 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.615 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.615 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:16:14.615 DEBUG:boto:Data: 2024-04-16T18:16:14.615 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.615 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.616 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.616 DEBUG:boto:Params: {} 2024-04-16T18:16:14.616 DEBUG:boto:Token: None 2024-04-16T18:16:14.616 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c?versionId=null 2024-04-16T18:16:14.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM= 2024-04-16T18:16:14.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM=', 'Content-Length': '0'} 2024-04-16T18:16:14.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a56d300931273479-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:16:14.619 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.619 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.619 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.619 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.619 DEBUG:boto:Method: GET 2024-04-16T18:16:14.619 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:16:14.619 DEBUG:boto:Data: 2024-04-16T18:16:14.620 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.620 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.620 DEBUG:boto:Params: {} 2024-04-16T18:16:14.620 DEBUG:boto:Token: None 2024-04-16T18:16:14.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c?versionId=null 2024-04-16T18:16:14.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T18:16:14.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T18:16:14.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001813b39a0a9a12dc-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.666 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.666 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.666 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.667 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:16:14.667 DEBUG:boto:Method: GET 2024-04-16T18:16:14.667 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:16:14.667 DEBUG:boto:Data: 2024-04-16T18:16:14.667 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.667 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.667 DEBUG:boto:Params: {} 2024-04-16T18:16:14.667 DEBUG:boto:Token: None 2024-04-16T18:16:14.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c?versionId=null 2024-04-16T18:16:14.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T18:16:14.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T18:16:14.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e05a3b25865d019a-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.671 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.671 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.671 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.671 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.671 DEBUG:boto:Method: GET 2024-04-16T18:16:14.671 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:16:14.671 DEBUG:boto:Data: 2024-04-16T18:16:14.671 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.672 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.672 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.672 DEBUG:boto:Params: {} 2024-04-16T18:16:14.672 DEBUG:boto:Token: None 2024-04-16T18:16:14.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4db10df60f805ce-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.718 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.719 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.719 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.719 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.719 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.719 DEBUG:boto:Method: GET 2024-04-16T18:16:14.719 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:16:14.719 DEBUG:boto:Data: 2024-04-16T18:16:14.719 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.719 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.719 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.719 DEBUG:boto:Params: {} 2024-04-16T18:16:14.720 DEBUG:boto:Token: None 2024-04-16T18:16:14.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079f217cea22b942a-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.724 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.724 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.724 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.724 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.724 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:16:14.724 DEBUG:boto:Data: 2024-04-16T18:16:14.724 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.724 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.724 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.725 DEBUG:boto:Params: {} 2024-04-16T18:16:14.725 DEBUG:boto:Token: None 2024-04-16T18:16:14.725 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c 2024-04-16T18:16:14.725 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40= 2024-04-16T18:16:14.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40=', 'Content-Length': '0'} 2024-04-16T18:16:14.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c9085e09649e986-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.728 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.729 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.729 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.729 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:16:14.729 DEBUG:boto:Data: 2024-04-16T18:16:14.729 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.729 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.729 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.729 DEBUG:boto:Params: {} 2024-04-16T18:16:14.729 DEBUG:boto:Token: None 2024-04-16T18:16:14.729 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c 2024-04-16T18:16:14.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40= 2024-04-16T18:16:14.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40=', 'Content-Length': '0'} 2024-04-16T18:16:14.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b5eacee7bddbed1-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.732 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:16:14.733 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.733 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.733 DEBUG:boto:Method: GET 2024-04-16T18:16:14.733 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:16:14.733 DEBUG:boto:Data: 2024-04-16T18:16:14.733 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.733 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.733 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.733 DEBUG:boto:Params: {} 2024-04-16T18:16:14.733 DEBUG:boto:Token: None 2024-04-16T18:16:14.733 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c 2024-04-16T18:16:14.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E= 2024-04-16T18:16:14.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E=', 'Content-Length': '0'} 2024-04-16T18:16:14.737 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013ce4732480fc210-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.778 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:16:14.778 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:16:14.779 DEBUG:boto:Method: GET 2024-04-16T18:16:14.779 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:16:14.779 DEBUG:boto:Data: 2024-04-16T18:16:14.779 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.779 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.779 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.779 DEBUG:boto:Params: {} 2024-04-16T18:16:14.779 DEBUG:boto:Token: None 2024-04-16T18:16:14.779 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/c 2024-04-16T18:16:14.779 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E= 2024-04-16T18:16:14.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E=', 'Content-Length': '0'} 2024-04-16T18:16:14.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002ba4a669dec3993-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.783 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.783 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.783 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.783 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.784 DEBUG:boto:Method: GET 2024-04-16T18:16:14.784 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:16:14.784 DEBUG:boto:Data: 2024-04-16T18:16:14.784 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.784 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.784 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.784 DEBUG:boto:Params: {} 2024-04-16T18:16:14.784 DEBUG:boto:Token: None 2024-04-16T18:16:14.784 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.785 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.788 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0e55afa5fddf315-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.830 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.831 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.831 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.831 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.831 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:16:14.831 DEBUG:boto:Method: GET 2024-04-16T18:16:14.831 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:16:14.831 DEBUG:boto:Data: 2024-04-16T18:16:14.831 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.831 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.831 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.832 DEBUG:boto:Params: {} 2024-04-16T18:16:14.832 DEBUG:boto:Token: None 2024-04-16T18:16:14.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3d1ded25fb2e5fc-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.836 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.836 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:16:14.837 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.837 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.837 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.837 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.837 DEBUG:boto:Method: GET 2024-04-16T18:16:14.837 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:16:14.837 DEBUG:boto:Data: 2024-04-16T18:16:14.837 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.837 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.837 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.837 DEBUG:boto:Params: {} 2024-04-16T18:16:14.837 DEBUG:boto:Token: None 2024-04-16T18:16:14.838 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d?versionId=null 2024-04-16T18:16:14.838 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T18:16:14.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T18:16:14.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000839e19fcf3024cae-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.882 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.882 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.882 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.883 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.883 DEBUG:boto:Method: GET 2024-04-16T18:16:14.883 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:16:14.883 DEBUG:boto:Data: 2024-04-16T18:16:14.883 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.883 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.883 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.883 DEBUG:boto:Params: {} 2024-04-16T18:16:14.883 DEBUG:boto:Token: None 2024-04-16T18:16:14.883 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d?versionId=null 2024-04-16T18:16:14.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T18:16:14.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T18:16:14.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007267426868d6694c-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.887 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.887 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.887 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.887 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.887 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.887 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:16:14.887 DEBUG:boto:Data: 2024-04-16T18:16:14.887 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.887 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.887 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.887 DEBUG:boto:Params: {} 2024-04-16T18:16:14.888 DEBUG:boto:Token: None 2024-04-16T18:16:14.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d?versionId=null 2024-04-16T18:16:14.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI= 2024-04-16T18:16:14.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI=', 'Content-Length': '0'} 2024-04-16T18:16:14.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef8d4e310ddf7504-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.891 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.891 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.891 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.891 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.891 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.892 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:16:14.892 DEBUG:boto:Data: 2024-04-16T18:16:14.892 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.892 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.892 DEBUG:boto:Params: {} 2024-04-16T18:16:14.892 DEBUG:boto:Token: None 2024-04-16T18:16:14.892 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d?versionId=null 2024-04-16T18:16:14.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI= 2024-04-16T18:16:14.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI=', 'Content-Length': '0'} 2024-04-16T18:16:14.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef595552e67feaa3-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.895 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:16:14.896 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.896 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.896 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.896 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.896 DEBUG:boto:Method: GET 2024-04-16T18:16:14.896 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:16:14.896 DEBUG:boto:Data: 2024-04-16T18:16:14.896 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.896 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.896 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.896 DEBUG:boto:Params: {} 2024-04-16T18:16:14.896 DEBUG:boto:Token: None 2024-04-16T18:16:14.897 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d?versionId=null 2024-04-16T18:16:14.897 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T18:16:14.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T18:16:14.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f32c21b3e01f92a-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.942 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.942 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.942 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.943 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:16:14.943 DEBUG:boto:Method: GET 2024-04-16T18:16:14.943 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:16:14.943 DEBUG:boto:Data: 2024-04-16T18:16:14.943 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.943 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.943 DEBUG:boto:Params: {} 2024-04-16T18:16:14.943 DEBUG:boto:Token: None 2024-04-16T18:16:14.943 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d?versionId=null 2024-04-16T18:16:14.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T18:16:14.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T18:16:14.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baba412cc933aabb-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.947 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.947 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.947 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:16:14.947 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:16:14.947 DEBUG:boto:Method: GET 2024-04-16T18:16:14.947 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:16:14.947 DEBUG:boto:Data: 2024-04-16T18:16:14.947 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.948 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.948 DEBUG:boto:Params: {} 2024-04-16T18:16:14.948 DEBUG:boto:Token: None 2024-04-16T18:16:14.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd39172da960b0b4-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.952 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.952 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:14.952 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:14.952 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:16:14.952 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:16:14.953 DEBUG:boto:Method: GET 2024-04-16T18:16:14.953 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:16:14.953 DEBUG:boto:Data: 2024-04-16T18:16:14.953 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:14.953 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.953 DEBUG:boto:Params: {} 2024-04-16T18:16:14.953 DEBUG:boto:Token: None 2024-04-16T18:16:14.953 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/ 2024-04-16T18:16:14.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T18:16:14.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T18:16:14.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008be175c3ce717008-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:14.998 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:14.998 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:14.999 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:14.999 DEBUG:boto:Method: HEAD 2024-04-16T18:16:14.999 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:16:14.999 DEBUG:boto:Data: 2024-04-16T18:16:14.999 DEBUG:boto:Headers: {} 2024-04-16T18:16:14.999 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:14.999 DEBUG:boto:Port: 8000 2024-04-16T18:16:14.999 DEBUG:boto:Params: {} 2024-04-16T18:16:14.999 DEBUG:boto:Token: None 2024-04-16T18:16:14.999 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:14 GMT /lybtug-50/d 2024-04-16T18:16:15.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wdl6mP8v9hYo9lyKZtVtP+FfDoY= 2024-04-16T18:16:15.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wdl6mP8v9hYo9lyKZtVtP+FfDoY=', 'Content-Length': '0'} 2024-04-16T18:16:15.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e68fdaa4c27b1fae-00661ec06f-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:15.002 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:15.003 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:15.003 DEBUG:boto:Method: HEAD 2024-04-16T18:16:15.003 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:16:15.003 DEBUG:boto:Data: 2024-04-16T18:16:15.003 DEBUG:boto:Headers: {} 2024-04-16T18:16:15.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:15.003 DEBUG:boto:Port: 8000 2024-04-16T18:16:15.003 DEBUG:boto:Params: {} 2024-04-16T18:16:15.003 DEBUG:boto:Token: None 2024-04-16T18:16:15.003 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:15 GMT /lybtug-50/d 2024-04-16T18:16:15.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u1lpxb5EdR/jqPlCQ+hva1SLtW8= 2024-04-16T18:16:15.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u1lpxb5EdR/jqPlCQ+hva1SLtW8=', 'Content-Length': '0'} 2024-04-16T18:16:15.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009007d94e267a19d3-00661ec06f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:15.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:16:15.006 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:15.007 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:15.007 DEBUG:boto:Method: GET 2024-04-16T18:16:15.007 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:16:15.007 DEBUG:boto:Data: 2024-04-16T18:16:15.007 DEBUG:boto:Headers: {} 2024-04-16T18:16:15.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:15.007 DEBUG:boto:Port: 8000 2024-04-16T18:16:15.007 DEBUG:boto:Params: {} 2024-04-16T18:16:15.007 DEBUG:boto:Token: None 2024-04-16T18:16:15.007 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:15 GMT /lybtug-50/d 2024-04-16T18:16:15.007 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM= 2024-04-16T18:16:15.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM=', 'Content-Length': '0'} 2024-04-16T18:16:15.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000836c1340cb19e47d-00661ec06f-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:15.050 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:16:15.050 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:16:15.051 DEBUG:boto:Method: GET 2024-04-16T18:16:15.051 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:16:15.051 DEBUG:boto:Data: 2024-04-16T18:16:15.051 DEBUG:boto:Headers: {} 2024-04-16T18:16:15.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:15.051 DEBUG:boto:Port: 8000 2024-04-16T18:16:15.051 DEBUG:boto:Params: {} 2024-04-16T18:16:15.051 DEBUG:boto:Token: None 2024-04-16T18:16:15.051 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:15 GMT /lybtug-50/d 2024-04-16T18:16:15.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM= 2024-04-16T18:16:15.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM=', 'Content-Length': '0'} 2024-04-16T18:16:15.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000659d1c1dc4cee97f-00661ec06f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:15.055 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:15.055 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:15.055 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:16:15.055 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:16:15.055 DEBUG:boto:Method: GET 2024-04-16T18:16:15.055 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:16:15.056 DEBUG:boto:Data: 2024-04-16T18:16:15.056 DEBUG:boto:Headers: {} 2024-04-16T18:16:15.056 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:16:15.056 DEBUG:boto:Port: 8000 2024-04-16T18:16:15.056 DEBUG:boto:Params: {} 2024-04-16T18:16:15.056 DEBUG:boto:Token: None 2024-04-16T18:16:15.056 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:15 GMT /lybtug-50/ 2024-04-16T18:16:15.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY= 2024-04-16T18:16:15.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY=', 'Content-Length': '0'} 2024-04-16T18:16:15.060 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f55706da73ad987-00661ec06f-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:15.102 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:15.103 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:15.103 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:15.103 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:16:15.103 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:16:15.103 DEBUG:boto:Method: GET 2024-04-16T18:16:15.103 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:16:15.103 DEBUG:boto:Data: 2024-04-16T18:16:15.103 DEBUG:boto:Headers: {} 2024-04-16T18:16:15.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:15.103 DEBUG:boto:Port: 8000 2024-04-16T18:16:15.104 DEBUG:boto:Params: {} 2024-04-16T18:16:15.104 DEBUG:boto:Token: None 2024-04-16T18:16:15.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:16:15 GMT /lybtug-50/ 2024-04-16T18:16:15.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY= 2024-04-16T18:16:15.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY=', 'Content-Length': '0'} 2024-04-16T18:16:15.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fcc88e79b2f0fcd-00661ec06f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:15.108 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:16:15.109 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T18:16:15.109 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:16:15.109 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:16:15.109 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:16:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:15.248 INFO:teuthology.orchestra.run.smithi071.stdout:{"layout":{"resharding":"None","current_index":{"gen":0,"layout":{"type":"Normal","normal":{"num_shards":11,"hash_type":"Mod"}}},"logs":[{"gen":0,"layout":{"type":"InIndex","in_index":{"gen":0,"layout":{"num_shards":11,"hash_type":"Mod"}}}}]}} 2024-04-16T18:16:15.249 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-50', '--gen', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:16:15.249 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-50', '--gen', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:16:15.249 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-50 --gen 0 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:16:15.325 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:15.326 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:15.424 INFO:teuthology.orchestra.run.smithi071.stdout:[{"op_id":"1#00000000001.55.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15343941463733059060","op":"write","object":"a","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.232829449Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]},{"op_id":"5#00000000001.67.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2034444563773777923","op":"write","object":"d","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.362254267Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]},{"op_id":"8#00000000001.46.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12822008793054755835","op":"write","object":"c","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.312027972Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]},{"op_id":"10#00000000001.53.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2152254117184578004","op":"write","object":"b","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.290803125Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]}] 2024-04-16T18:16:15.425 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:16:15.425 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:16:15.425 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket lybtug-50 --num-shards 13 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:16:15.500 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:15.500 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:15.592 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:16:15.593 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-50 2024-04-16T18:16:15.601 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 4 2024-04-16T18:16:15.691 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:16:15.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:16:15.692 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:16:15.771 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:15.771 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:{"layout":{"resharding":"None","current_index":{"gen":1,"layout":{"type":"Normal","normal":{"num_shards":13,"hash_type":"Mod"}}},"logs":[{"gen":0,"layout":{"type":"InIndex","in_index":{"gen":0,"layout":{"num_shards":11,"hash_type":"Mod"}}}},{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}}]}} 2024-04-16T18:16:15.866 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-50', '--gen', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:16:15.866 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-50', '--gen', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:16:15.866 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-50 --gen 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:16:15.948 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:15.949 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:16.041 INFO:teuthology.orchestra.run.smithi071.stdout:[] 2024-04-16T18:16:16.041 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:16.041 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:16.042 DEBUG:boto:Method: HEAD 2024-04-16T18:16:16.042 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:16:16.042 DEBUG:boto:Data: 2024-04-16T18:16:16.042 DEBUG:boto:Headers: {} 2024-04-16T18:16:16.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.042 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.042 DEBUG:boto:Params: {} 2024-04-16T18:16:16.042 DEBUG:boto:Token: None 2024-04-16T18:16:16.042 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/ 2024-04-16T18:16:16.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T18:16:16.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T18:16:16.046 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008cbf16b4f7dd8b3b-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.046 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:16:16.046 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:16:16.046 DEBUG:boto:Method: PUT 2024-04-16T18:16:16.047 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:16:16.047 DEBUG:boto:Data: 2024-04-16T18:16:16.047 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:16:16.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.047 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.047 DEBUG:boto:Params: {} 2024-04-16T18:16:16.047 DEBUG:boto:Token: None 2024-04-16T18:16:16.047 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/e 2024-04-16T18:16:16.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ruYWUxSP8YhsmFw1G/B34FCsbQg= 2024-04-16T18:16:16.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ruYWUxSP8YhsmFw1G/B34FCsbQg='} 2024-04-16T18:16:16.055 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d6f20914fc9fcc09-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.055 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:16.056 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:16.056 DEBUG:boto:Method: HEAD 2024-04-16T18:16:16.056 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:16:16.056 DEBUG:boto:Data: 2024-04-16T18:16:16.056 DEBUG:boto:Headers: {} 2024-04-16T18:16:16.056 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.056 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.056 DEBUG:boto:Params: {} 2024-04-16T18:16:16.056 DEBUG:boto:Token: None 2024-04-16T18:16:16.056 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/ 2024-04-16T18:16:16.057 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T18:16:16.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T18:16:16.060 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000548abd2aeee90cbe-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.060 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:16:16.060 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:16:16.060 DEBUG:boto:Method: PUT 2024-04-16T18:16:16.060 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:16:16.060 DEBUG:boto:Data: 2024-04-16T18:16:16.060 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:16:16.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.061 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.061 DEBUG:boto:Params: {} 2024-04-16T18:16:16.061 DEBUG:boto:Token: None 2024-04-16T18:16:16.061 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/f 2024-04-16T18:16:16.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l5a9P60fXC4jP9HpVd833oxmAwY= 2024-04-16T18:16:16.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l5a9P60fXC4jP9HpVd833oxmAwY='} 2024-04-16T18:16:16.102 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003e81fcd115f30c79-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.103 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:16.103 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:16.103 DEBUG:boto:Method: HEAD 2024-04-16T18:16:16.103 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:16:16.103 DEBUG:boto:Data: 2024-04-16T18:16:16.103 DEBUG:boto:Headers: {} 2024-04-16T18:16:16.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.103 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.103 DEBUG:boto:Params: {} 2024-04-16T18:16:16.104 DEBUG:boto:Token: None 2024-04-16T18:16:16.104 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/ 2024-04-16T18:16:16.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T18:16:16.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T18:16:16.107 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000040dc6d300b986900-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.107 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:16:16.107 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:16:16.108 DEBUG:boto:Method: PUT 2024-04-16T18:16:16.108 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:16:16.108 DEBUG:boto:Data: 2024-04-16T18:16:16.108 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:16:16.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.108 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.108 DEBUG:boto:Params: {} 2024-04-16T18:16:16.108 DEBUG:boto:Token: None 2024-04-16T18:16:16.108 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/g 2024-04-16T18:16:16.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Aw1cLLxzhiGUw1Jt3csiSZmZeVI= 2024-04-16T18:16:16.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Aw1cLLxzhiGUw1Jt3csiSZmZeVI='} 2024-04-16T18:16:16.119 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dad40ab842caf3c8-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.119 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:16:16.119 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:16:16.119 DEBUG:boto:Method: HEAD 2024-04-16T18:16:16.119 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:16:16.119 DEBUG:boto:Data: 2024-04-16T18:16:16.119 DEBUG:boto:Headers: {} 2024-04-16T18:16:16.119 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.119 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.120 DEBUG:boto:Params: {} 2024-04-16T18:16:16.120 DEBUG:boto:Token: None 2024-04-16T18:16:16.120 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/ 2024-04-16T18:16:16.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T18:16:16.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T18:16:16.123 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000496ae517ec22d2d7-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.123 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:16:16.124 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:16:16.124 DEBUG:boto:Method: PUT 2024-04-16T18:16:16.124 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:16:16.124 DEBUG:boto:Data: 2024-04-16T18:16:16.124 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:16:16.124 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:16:16.124 DEBUG:boto:Port: 8000 2024-04-16T18:16:16.124 DEBUG:boto:Params: {} 2024-04-16T18:16:16.124 DEBUG:boto:Token: None 2024-04-16T18:16:16.124 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:16:16 GMT /lybtug-50/h 2024-04-16T18:16:16.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B8nHYSG6uRZvjFXauDZowtv9Qsw= 2024-04-16T18:16:16.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B8nHYSG6uRZvjFXauDZowtv9Qsw='} 2024-04-16T18:16:16.166 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000689ebcc5787d5525-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:16:16.167 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-50 2024-04-16T18:16:16.167 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:16:16.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:16:16.167 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:16:16.218 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:16:16.218 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:16:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:16.302+0000 7fb9de2a8980 1 waiting to reach latest gen 1, on 0.. 2024-04-16T18:16:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:21.306+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:21.306 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:21.306 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:26.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:26.310+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:26.308 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:26.308 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:31.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:31.310+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:31.310 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:31.310 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:36.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:36.314+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:36.312 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:36.312 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:41.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:41.318+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:41.314 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:41.314 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:46.318+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:46.316 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:46.316 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:51.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:51.317+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:51.318 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:51.318 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:16:56.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:16:56.321+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:16:56.320 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:16:56.320 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:17:01.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:01.321+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:17:01.322 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , ] 2024-04-16T18:17:01.322 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:17:06.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:06.325+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T18:17:06.325 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , 00000000002.36.6, , , , 00000000001.32.6, ] 2024-04-16T18:17:06.325 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:17:11.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:11.325+0000 7fb9de2a8980 1 bucket sync caught up with source: 2024-04-16T18:17:11.326 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:17:11.326 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T18:17:11.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:11.325+0000 7fb9de2a8980 0 bucket checkpoint complete 2024-04-16T18:17:11.373 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-50 2024-04-16T18:17:11.373 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:11.373 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:11.374 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:17:11.426 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:11.426 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:11.505+0000 7fdd6454c980 1 waiting for incremental sync to catch up: 2024-04-16T18:17:11.503 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.905.6, , , , 00000000001.1167.6, , , 00000000001.306.6, , 00000000002.413.6] 2024-04-16T18:17:11.503 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T18:17:16.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:16.505+0000 7fdd6454c980 1 bucket sync caught up with source: 2024-04-16T18:17:16.506 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T18:17:16.506 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T18:17:16.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:16.505+0000 7fdd6454c980 0 bucket checkpoint complete 2024-04-16T18:17:16.555 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T18:17:16.555 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.555 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.556 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.556 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:17:16.556 DEBUG:boto:Data: 2024-04-16T18:17:16.556 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.556 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.556 DEBUG:boto:Params: {} 2024-04-16T18:17:16.557 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:17:16.557 DEBUG:boto:Token: None 2024-04-16T18:17:16.557 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.557 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM= 2024-04-16T18:17:16.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM=', 'Content-Length': '0'} 2024-04-16T18:17:16.565 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000010dcc596d8edb598-00661ec0ac-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.565 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.565 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.565 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.565 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:17:16.565 DEBUG:boto:Data: 2024-04-16T18:17:16.565 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.565 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.565 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.565 DEBUG:boto:Params: {} 2024-04-16T18:17:16.566 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:17:16.566 DEBUG:boto:Token: None 2024-04-16T18:17:16.566 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.566 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM= 2024-04-16T18:17:16.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM=', 'Content-Length': '0'} 2024-04-16T18:17:16.572 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000024f6b9107005997b-00661ec0ac-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.572 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:17:16.572 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.572 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.572 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.572 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.572 DEBUG:boto:Method: GET 2024-04-16T18:17:16.572 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.573 DEBUG:boto:Data: 2024-04-16T18:17:16.573 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.573 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.573 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.573 DEBUG:boto:Params: {} 2024-04-16T18:17:16.573 DEBUG:boto:Token: None 2024-04-16T18:17:16.573 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/?versions 2024-04-16T18:17:16.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T18:17:16.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T18:17:16.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ee9633d3d403e2b-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.578 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.579 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:17:16.579 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:17:16.579 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:17:16.579 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:17:16.579 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:17:16.579 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:17:16.580 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:17:16.580 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:17:16.580 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:17:16.580 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.580 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.580 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.580 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.580 DEBUG:boto:Method: GET 2024-04-16T18:17:16.580 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.580 DEBUG:boto:Data: 2024-04-16T18:17:16.581 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.581 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.581 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.581 DEBUG:boto:Params: {} 2024-04-16T18:17:16.581 DEBUG:boto:Token: None 2024-04-16T18:17:16.581 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/?versions 2024-04-16T18:17:16.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T18:17:16.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T18:17:16.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005205633bf757be12-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.585 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.585 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:17:16.585 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:17:16.586 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:17:16.586 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:17:16.586 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:17:16.586 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:17:16.586 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:17:16.586 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:17:16.586 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.586 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.586 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.586 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.586 DEBUG:boto:Method: GET 2024-04-16T18:17:16.587 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.587 DEBUG:boto:Data: 2024-04-16T18:17:16.587 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.587 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.587 DEBUG:boto:Params: {} 2024-04-16T18:17:16.587 DEBUG:boto:Token: None 2024-04-16T18:17:16.587 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/?versions 2024-04-16T18:17:16.587 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T18:17:16.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T18:17:16.593 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067061c63e84d12d3-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.593 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.594 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.594 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.594 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.594 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.594 DEBUG:boto:Method: GET 2024-04-16T18:17:16.594 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:16.594 DEBUG:boto:Data: 2024-04-16T18:17:16.595 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.595 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.595 DEBUG:boto:Params: {} 2024-04-16T18:17:16.595 DEBUG:boto:Token: None 2024-04-16T18:17:16.595 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/?versions 2024-04-16T18:17:16.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T18:17:16.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T18:17:16.599 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c46d528db8db9f5f-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.599 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.600 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:17:16.600 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.600 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.600 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.601 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.601 DEBUG:boto:Method: GET 2024-04-16T18:17:16.601 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:16.601 DEBUG:boto:Data: 2024-04-16T18:17:16.601 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.601 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.601 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.601 DEBUG:boto:Params: {} 2024-04-16T18:17:16.601 DEBUG:boto:Token: None 2024-04-16T18:17:16.601 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:16.602 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T18:17:16.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T18:17:16.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4f97346761b2cb3-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.646 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.646 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.647 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.647 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.647 DEBUG:boto:Method: GET 2024-04-16T18:17:16.647 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:16.647 DEBUG:boto:Data: 2024-04-16T18:17:16.647 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.647 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.647 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.647 DEBUG:boto:Params: {} 2024-04-16T18:17:16.647 DEBUG:boto:Token: None 2024-04-16T18:17:16.648 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:16.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T18:17:16.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T18:17:16.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000417f45a78f7022b2-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.651 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.651 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.651 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.651 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.651 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.652 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:16.652 DEBUG:boto:Data: 2024-04-16T18:17:16.652 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.652 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.652 DEBUG:boto:Params: {} 2024-04-16T18:17:16.652 DEBUG:boto:Token: None 2024-04-16T18:17:16.652 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:16.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U= 2024-04-16T18:17:16.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U=', 'Content-Length': '0'} 2024-04-16T18:17:16.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003e407f64ffd74bf-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.655 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.656 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.656 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.656 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.656 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.656 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:16.656 DEBUG:boto:Data: 2024-04-16T18:17:16.656 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.656 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.656 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.656 DEBUG:boto:Params: {} 2024-04-16T18:17:16.656 DEBUG:boto:Token: None 2024-04-16T18:17:16.657 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:16.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U= 2024-04-16T18:17:16.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U=', 'Content-Length': '0'} 2024-04-16T18:17:16.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dd39912c6800d45-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.660 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:17:16.660 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.660 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.660 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.660 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.661 DEBUG:boto:Method: GET 2024-04-16T18:17:16.661 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:16.661 DEBUG:boto:Data: 2024-04-16T18:17:16.661 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.661 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.661 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.661 DEBUG:boto:Params: {} 2024-04-16T18:17:16.661 DEBUG:boto:Token: None 2024-04-16T18:17:16.661 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:16.661 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T18:17:16.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T18:17:16.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001127a8c240d2a2f8-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.706 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.707 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.707 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.707 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:16.707 DEBUG:boto:Method: GET 2024-04-16T18:17:16.707 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:16.707 DEBUG:boto:Data: 2024-04-16T18:17:16.707 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.707 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.707 DEBUG:boto:Params: {} 2024-04-16T18:17:16.708 DEBUG:boto:Token: None 2024-04-16T18:17:16.708 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:16.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T18:17:16.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T18:17:16.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f6993ac9b179f79-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.711 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.711 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.712 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.712 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.712 DEBUG:boto:Method: GET 2024-04-16T18:17:16.712 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:16.712 DEBUG:boto:Data: 2024-04-16T18:17:16.712 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.712 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.712 DEBUG:boto:Params: {} 2024-04-16T18:17:16.712 DEBUG:boto:Token: None 2024-04-16T18:17:16.713 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T18:17:16.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T18:17:16.717 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6d6161c07361489-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.758 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.759 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.759 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.759 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.759 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.759 DEBUG:boto:Method: GET 2024-04-16T18:17:16.759 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:16.759 DEBUG:boto:Data: 2024-04-16T18:17:16.759 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.760 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.760 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.760 DEBUG:boto:Params: {} 2024-04-16T18:17:16.760 DEBUG:boto:Token: None 2024-04-16T18:17:16.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T18:17:16.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T18:17:16.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b983e0239bd401e3-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.765 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.765 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.765 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.765 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.765 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:16.765 DEBUG:boto:Data: 2024-04-16T18:17:16.765 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.766 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.766 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.766 DEBUG:boto:Params: {} 2024-04-16T18:17:16.766 DEBUG:boto:Token: None 2024-04-16T18:17:16.766 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a 2024-04-16T18:17:16.766 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4= 2024-04-16T18:17:16.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4=', 'Content-Length': '0'} 2024-04-16T18:17:16.769 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4129ecfb4a97016-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.770 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.770 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.770 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.770 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:16.770 DEBUG:boto:Data: 2024-04-16T18:17:16.770 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.770 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.771 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.771 DEBUG:boto:Params: {} 2024-04-16T18:17:16.771 DEBUG:boto:Token: None 2024-04-16T18:17:16.771 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a 2024-04-16T18:17:16.771 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4= 2024-04-16T18:17:16.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4=', 'Content-Length': '0'} 2024-04-16T18:17:16.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5ab46c088b9ac44-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.774 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:17:16.774 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.775 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.775 DEBUG:boto:Method: GET 2024-04-16T18:17:16.775 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:16.775 DEBUG:boto:Data: 2024-04-16T18:17:16.775 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.775 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.775 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.775 DEBUG:boto:Params: {} 2024-04-16T18:17:16.775 DEBUG:boto:Token: None 2024-04-16T18:17:16.776 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a 2024-04-16T18:17:16.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs= 2024-04-16T18:17:16.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs=', 'Content-Length': '0'} 2024-04-16T18:17:16.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d50d4f6565a1fa9-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.822 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:16.822 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:16.823 DEBUG:boto:Method: GET 2024-04-16T18:17:16.823 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:16.823 DEBUG:boto:Data: 2024-04-16T18:17:16.823 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.823 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.823 DEBUG:boto:Params: {} 2024-04-16T18:17:16.823 DEBUG:boto:Token: None 2024-04-16T18:17:16.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/a 2024-04-16T18:17:16.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs= 2024-04-16T18:17:16.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs=', 'Content-Length': '0'} 2024-04-16T18:17:16.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000571bfba6ffde9338-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.827 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.827 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.827 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.827 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.828 DEBUG:boto:Method: GET 2024-04-16T18:17:16.828 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:16.828 DEBUG:boto:Data: 2024-04-16T18:17:16.828 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.828 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.828 DEBUG:boto:Params: {} 2024-04-16T18:17:16.828 DEBUG:boto:Token: None 2024-04-16T18:17:16.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T18:17:16.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T18:17:16.832 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9d313c37a2430ee-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.833 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.833 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.833 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.833 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.834 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:16.834 DEBUG:boto:Method: GET 2024-04-16T18:17:16.834 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:16.834 DEBUG:boto:Data: 2024-04-16T18:17:16.834 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.834 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.834 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.834 DEBUG:boto:Params: {} 2024-04-16T18:17:16.834 DEBUG:boto:Token: None 2024-04-16T18:17:16.835 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T18:17:16.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T18:17:16.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3103312ecd3ef4b-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.839 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:17:16.839 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.840 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.840 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.840 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.840 DEBUG:boto:Method: GET 2024-04-16T18:17:16.840 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:16.840 DEBUG:boto:Data: 2024-04-16T18:17:16.840 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.840 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.840 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.840 DEBUG:boto:Params: {} 2024-04-16T18:17:16.841 DEBUG:boto:Token: None 2024-04-16T18:17:16.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:16.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T18:17:16.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T18:17:16.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f863d55ce30e14cd-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.886 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.887 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.887 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.887 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.887 DEBUG:boto:Method: GET 2024-04-16T18:17:16.887 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:16.887 DEBUG:boto:Data: 2024-04-16T18:17:16.887 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.887 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.887 DEBUG:boto:Params: {} 2024-04-16T18:17:16.888 DEBUG:boto:Token: None 2024-04-16T18:17:16.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:16.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T18:17:16.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T18:17:16.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ffba2674ad80d544-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.892 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.892 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.892 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.892 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.892 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.893 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:16.893 DEBUG:boto:Data: 2024-04-16T18:17:16.893 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.893 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.893 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.893 DEBUG:boto:Params: {} 2024-04-16T18:17:16.893 DEBUG:boto:Token: None 2024-04-16T18:17:16.893 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:16.894 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo= 2024-04-16T18:17:16.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo=', 'Content-Length': '0'} 2024-04-16T18:17:16.897 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002baa3a5dbb13679c-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.897 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.897 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.897 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.897 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.897 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.898 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:16.898 DEBUG:boto:Data: 2024-04-16T18:17:16.898 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.898 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.898 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.898 DEBUG:boto:Params: {} 2024-04-16T18:17:16.898 DEBUG:boto:Token: None 2024-04-16T18:17:16.898 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:16.899 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo= 2024-04-16T18:17:16.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo=', 'Content-Length': '0'} 2024-04-16T18:17:16.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb93e3f7666a59a3-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.902 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:17:16.902 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.902 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.902 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.902 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.902 DEBUG:boto:Method: GET 2024-04-16T18:17:16.902 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:16.902 DEBUG:boto:Data: 2024-04-16T18:17:16.903 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.903 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.903 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.903 DEBUG:boto:Params: {} 2024-04-16T18:17:16.903 DEBUG:boto:Token: None 2024-04-16T18:17:16.903 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:16.903 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T18:17:16.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T18:17:16.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057f6b799ad464157-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.946 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.947 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.947 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.947 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:16.947 DEBUG:boto:Method: GET 2024-04-16T18:17:16.947 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:16.947 DEBUG:boto:Data: 2024-04-16T18:17:16.947 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.947 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.948 DEBUG:boto:Params: {} 2024-04-16T18:17:16.948 DEBUG:boto:Token: None 2024-04-16T18:17:16.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:16.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T18:17:16.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T18:17:16.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7133b53c2edeb42-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.952 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.952 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.952 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:16.952 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:16.952 DEBUG:boto:Method: GET 2024-04-16T18:17:16.952 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:16.952 DEBUG:boto:Data: 2024-04-16T18:17:16.952 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.952 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.952 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.953 DEBUG:boto:Params: {} 2024-04-16T18:17:16.953 DEBUG:boto:Token: None 2024-04-16T18:17:16.953 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T18:17:16.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T18:17:16.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000fa7687430d7df3c-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.957 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.957 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:16.958 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:16.958 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:16.958 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:16.958 DEBUG:boto:Method: GET 2024-04-16T18:17:16.958 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:16.958 DEBUG:boto:Data: 2024-04-16T18:17:16.958 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.958 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.958 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.959 DEBUG:boto:Params: {} 2024-04-16T18:17:16.959 DEBUG:boto:Token: None 2024-04-16T18:17:16.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/ 2024-04-16T18:17:16.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T18:17:16.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T18:17:16.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056d3ea1b8a0f1e1b-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.963 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:16.964 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.964 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.964 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.964 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:16.964 DEBUG:boto:Data: 2024-04-16T18:17:16.964 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.964 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.965 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.965 DEBUG:boto:Params: {} 2024-04-16T18:17:16.965 DEBUG:boto:Token: None 2024-04-16T18:17:16.965 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b 2024-04-16T18:17:16.965 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc= 2024-04-16T18:17:16.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc=', 'Content-Length': '0'} 2024-04-16T18:17:16.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004572b6a5c6a10bd-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.968 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.969 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.969 DEBUG:boto:Method: HEAD 2024-04-16T18:17:16.969 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:16.969 DEBUG:boto:Data: 2024-04-16T18:17:16.969 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.969 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:16.969 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.969 DEBUG:boto:Params: {} 2024-04-16T18:17:16.969 DEBUG:boto:Token: None 2024-04-16T18:17:16.970 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b 2024-04-16T18:17:16.970 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc= 2024-04-16T18:17:16.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc=', 'Content-Length': '0'} 2024-04-16T18:17:16.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae7865510aa87c90-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:16.973 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:17:16.973 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:16.973 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:16.973 DEBUG:boto:Method: GET 2024-04-16T18:17:16.973 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:16.973 DEBUG:boto:Data: 2024-04-16T18:17:16.974 DEBUG:boto:Headers: {} 2024-04-16T18:17:16.974 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:16.974 DEBUG:boto:Port: 8000 2024-04-16T18:17:16.974 DEBUG:boto:Params: {} 2024-04-16T18:17:16.974 DEBUG:boto:Token: None 2024-04-16T18:17:16.974 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:16 GMT /lybtug-50/b 2024-04-16T18:17:16.974 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QhfYwKKsEuB1tmy+SZ6IZuxYDgw= 2024-04-16T18:17:16.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QhfYwKKsEuB1tmy+SZ6IZuxYDgw=', 'Content-Length': '0'} 2024-04-16T18:17:16.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8c6c26a3ef3cf12-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.018 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:17.019 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:17.019 DEBUG:boto:Method: GET 2024-04-16T18:17:17.019 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:17.019 DEBUG:boto:Data: 2024-04-16T18:17:17.019 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.019 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.019 DEBUG:boto:Params: {} 2024-04-16T18:17:17.020 DEBUG:boto:Token: None 2024-04-16T18:17:17.020 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/b 2024-04-16T18:17:17.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IxfhkbeBvpbvkTLoMQnR5CHiq9Q= 2024-04-16T18:17:17.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IxfhkbeBvpbvkTLoMQnR5CHiq9Q=', 'Content-Length': '0'} 2024-04-16T18:17:17.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079d16a625ffaed2b-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.023 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.023 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.023 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:17.024 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:17.024 DEBUG:boto:Method: GET 2024-04-16T18:17:17.024 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:17.024 DEBUG:boto:Data: 2024-04-16T18:17:17.024 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.024 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.024 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.024 DEBUG:boto:Params: {} 2024-04-16T18:17:17.024 DEBUG:boto:Token: None 2024-04-16T18:17:17.025 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.028 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000792eb5b3ee7d6693-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.029 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.029 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.029 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.029 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:17.029 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:17.030 DEBUG:boto:Method: GET 2024-04-16T18:17:17.030 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:17.030 DEBUG:boto:Data: 2024-04-16T18:17:17.030 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.030 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.030 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.030 DEBUG:boto:Params: {} 2024-04-16T18:17:17.030 DEBUG:boto:Token: None 2024-04-16T18:17:17.030 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.031 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.035 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9058d4db2ec3b76-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.078 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.079 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:17:17.079 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.079 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.079 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.079 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.080 DEBUG:boto:Method: GET 2024-04-16T18:17:17.080 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:17.080 DEBUG:boto:Data: 2024-04-16T18:17:17.080 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.080 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.080 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.080 DEBUG:boto:Params: {} 2024-04-16T18:17:17.080 DEBUG:boto:Token: None 2024-04-16T18:17:17.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:17.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T18:17:17.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T18:17:17.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cca416191edd493-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.126 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.127 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.127 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.127 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.127 DEBUG:boto:Method: GET 2024-04-16T18:17:17.127 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:17.127 DEBUG:boto:Data: 2024-04-16T18:17:17.127 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.127 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.128 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.128 DEBUG:boto:Params: {} 2024-04-16T18:17:17.128 DEBUG:boto:Token: None 2024-04-16T18:17:17.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:17.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T18:17:17.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T18:17:17.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d161c5038e2c6d9-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.132 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.132 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.132 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.132 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.132 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.132 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:17.133 DEBUG:boto:Data: 2024-04-16T18:17:17.133 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.133 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.133 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.133 DEBUG:boto:Params: {} 2024-04-16T18:17:17.133 DEBUG:boto:Token: None 2024-04-16T18:17:17.133 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:17.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE= 2024-04-16T18:17:17.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE=', 'Content-Length': '0'} 2024-04-16T18:17:17.137 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009da4d00d681d79a5-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.137 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.137 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.137 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.137 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.137 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.138 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:17.138 DEBUG:boto:Data: 2024-04-16T18:17:17.138 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.138 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.138 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.138 DEBUG:boto:Params: {} 2024-04-16T18:17:17.139 DEBUG:boto:Token: None 2024-04-16T18:17:17.139 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:17.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE= 2024-04-16T18:17:17.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE=', 'Content-Length': '0'} 2024-04-16T18:17:17.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc9b8a889cddfbd9-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.142 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:17:17.143 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.143 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.143 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.143 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.143 DEBUG:boto:Method: GET 2024-04-16T18:17:17.143 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:17.143 DEBUG:boto:Data: 2024-04-16T18:17:17.144 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.144 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.144 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.144 DEBUG:boto:Params: {} 2024-04-16T18:17:17.144 DEBUG:boto:Token: None 2024-04-16T18:17:17.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:17.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T18:17:17.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T18:17:17.148 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007848327eb6b75955-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.190 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.191 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.191 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.191 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:17.191 DEBUG:boto:Method: GET 2024-04-16T18:17:17.191 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:17.191 DEBUG:boto:Data: 2024-04-16T18:17:17.191 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.191 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.192 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.192 DEBUG:boto:Params: {} 2024-04-16T18:17:17.192 DEBUG:boto:Token: None 2024-04-16T18:17:17.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:17.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T18:17:17.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T18:17:17.196 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fdefc191176b0af-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.196 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.197 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.197 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.197 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.197 DEBUG:boto:Method: GET 2024-04-16T18:17:17.197 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:17.197 DEBUG:boto:Data: 2024-04-16T18:17:17.197 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.198 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.198 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.198 DEBUG:boto:Params: {} 2024-04-16T18:17:17.198 DEBUG:boto:Token: None 2024-04-16T18:17:17.198 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.198 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012d96947b3b85a26-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.203 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.203 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.203 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.204 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.204 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.204 DEBUG:boto:Method: GET 2024-04-16T18:17:17.204 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:17.204 DEBUG:boto:Data: 2024-04-16T18:17:17.204 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.204 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.204 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.204 DEBUG:boto:Params: {} 2024-04-16T18:17:17.205 DEBUG:boto:Token: None 2024-04-16T18:17:17.205 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.205 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.209 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036abfc28cfc09a3a-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.209 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.209 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.210 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.210 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.210 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:17.210 DEBUG:boto:Data: 2024-04-16T18:17:17.210 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.210 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.210 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.211 DEBUG:boto:Params: {} 2024-04-16T18:17:17.211 DEBUG:boto:Token: None 2024-04-16T18:17:17.211 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c 2024-04-16T18:17:17.211 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0= 2024-04-16T18:17:17.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0=', 'Content-Length': '0'} 2024-04-16T18:17:17.214 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af4b4cb8b40957d3-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.215 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.215 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.215 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.215 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:17.215 DEBUG:boto:Data: 2024-04-16T18:17:17.215 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.215 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.215 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.216 DEBUG:boto:Params: {} 2024-04-16T18:17:17.216 DEBUG:boto:Token: None 2024-04-16T18:17:17.216 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c 2024-04-16T18:17:17.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0= 2024-04-16T18:17:17.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0=', 'Content-Length': '0'} 2024-04-16T18:17:17.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000976a5c5eb205b751-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.219 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:17:17.219 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.220 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.220 DEBUG:boto:Method: GET 2024-04-16T18:17:17.220 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:17.220 DEBUG:boto:Data: 2024-04-16T18:17:17.220 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.220 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.220 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.220 DEBUG:boto:Params: {} 2024-04-16T18:17:17.221 DEBUG:boto:Token: None 2024-04-16T18:17:17.221 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c 2024-04-16T18:17:17.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4= 2024-04-16T18:17:17.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4=', 'Content-Length': '0'} 2024-04-16T18:17:17.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0b33ce54bfbcd47-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.266 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:17.267 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:17.267 DEBUG:boto:Method: GET 2024-04-16T18:17:17.267 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:17.267 DEBUG:boto:Data: 2024-04-16T18:17:17.267 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.267 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.267 DEBUG:boto:Params: {} 2024-04-16T18:17:17.268 DEBUG:boto:Token: None 2024-04-16T18:17:17.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/c 2024-04-16T18:17:17.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4= 2024-04-16T18:17:17.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4=', 'Content-Length': '0'} 2024-04-16T18:17:17.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b0ccda0beeeec10-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.272 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.272 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.272 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.272 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.272 DEBUG:boto:Method: GET 2024-04-16T18:17:17.273 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:17.273 DEBUG:boto:Data: 2024-04-16T18:17:17.273 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.273 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.273 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.273 DEBUG:boto:Params: {} 2024-04-16T18:17:17.273 DEBUG:boto:Token: None 2024-04-16T18:17:17.274 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.274 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.278 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c3272cb9755088c-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.318 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.319 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.319 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.319 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.319 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:17.319 DEBUG:boto:Method: GET 2024-04-16T18:17:17.319 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:17.320 DEBUG:boto:Data: 2024-04-16T18:17:17.320 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.320 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.320 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.320 DEBUG:boto:Params: {} 2024-04-16T18:17:17.320 DEBUG:boto:Token: None 2024-04-16T18:17:17.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.324 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a91e3ff25d7a4e42-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.325 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:17:17.325 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.325 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.326 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.326 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.326 DEBUG:boto:Method: GET 2024-04-16T18:17:17.326 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:17.326 DEBUG:boto:Data: 2024-04-16T18:17:17.326 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.326 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.326 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.326 DEBUG:boto:Params: {} 2024-04-16T18:17:17.327 DEBUG:boto:Token: None 2024-04-16T18:17:17.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:17.327 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T18:17:17.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T18:17:17.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000691bce239c8774d1-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.370 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.371 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.371 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.371 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.371 DEBUG:boto:Method: GET 2024-04-16T18:17:17.371 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:17.371 DEBUG:boto:Data: 2024-04-16T18:17:17.371 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.371 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.372 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.372 DEBUG:boto:Params: {} 2024-04-16T18:17:17.372 DEBUG:boto:Token: None 2024-04-16T18:17:17.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:17.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T18:17:17.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T18:17:17.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b9137d94790dc7c-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.376 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.376 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.376 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.376 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.376 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.376 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:17.377 DEBUG:boto:Data: 2024-04-16T18:17:17.377 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.377 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.377 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.377 DEBUG:boto:Params: {} 2024-04-16T18:17:17.377 DEBUG:boto:Token: None 2024-04-16T18:17:17.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:17.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c= 2024-04-16T18:17:17.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c=', 'Content-Length': '0'} 2024-04-16T18:17:17.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be03ad3eceba214e-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.381 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.381 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.381 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.381 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.382 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.382 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:17.382 DEBUG:boto:Data: 2024-04-16T18:17:17.382 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.382 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.382 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.382 DEBUG:boto:Params: {} 2024-04-16T18:17:17.382 DEBUG:boto:Token: None 2024-04-16T18:17:17.383 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:17.383 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c= 2024-04-16T18:17:17.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c=', 'Content-Length': '0'} 2024-04-16T18:17:17.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bbb22c0c6e33a12-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.386 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:17:17.386 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.386 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.387 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.387 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.387 DEBUG:boto:Method: GET 2024-04-16T18:17:17.387 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:17.387 DEBUG:boto:Data: 2024-04-16T18:17:17.387 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.387 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.387 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.387 DEBUG:boto:Params: {} 2024-04-16T18:17:17.387 DEBUG:boto:Token: None 2024-04-16T18:17:17.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:17.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T18:17:17.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T18:17:17.391 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bab675b3d6fbeebd-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.434 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.435 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.435 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.435 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:17.435 DEBUG:boto:Method: GET 2024-04-16T18:17:17.435 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:17.435 DEBUG:boto:Data: 2024-04-16T18:17:17.435 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.436 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.436 DEBUG:boto:Params: {} 2024-04-16T18:17:17.436 DEBUG:boto:Token: None 2024-04-16T18:17:17.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:17.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T18:17:17.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T18:17:17.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e49a464506915b91-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.440 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.440 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.440 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.440 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.440 DEBUG:boto:Method: GET 2024-04-16T18:17:17.440 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:17.440 DEBUG:boto:Data: 2024-04-16T18:17:17.441 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.441 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.441 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.441 DEBUG:boto:Params: {} 2024-04-16T18:17:17.441 DEBUG:boto:Token: None 2024-04-16T18:17:17.441 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.441 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000040c642602d4aac9-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.446 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.446 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.446 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.447 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.447 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.447 DEBUG:boto:Method: GET 2024-04-16T18:17:17.447 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:17.447 DEBUG:boto:Data: 2024-04-16T18:17:17.447 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.447 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.447 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.447 DEBUG:boto:Params: {} 2024-04-16T18:17:17.448 DEBUG:boto:Token: None 2024-04-16T18:17:17.448 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7ae7eddc35ebdb6-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.452 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.453 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.453 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.453 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.453 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:17.454 DEBUG:boto:Data: 2024-04-16T18:17:17.454 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.454 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.454 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.454 DEBUG:boto:Params: {} 2024-04-16T18:17:17.454 DEBUG:boto:Token: None 2024-04-16T18:17:17.454 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d 2024-04-16T18:17:17.455 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc= 2024-04-16T18:17:17.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc=', 'Content-Length': '0'} 2024-04-16T18:17:17.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4974f16e1390c33-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.458 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.458 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.459 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.459 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:17.459 DEBUG:boto:Data: 2024-04-16T18:17:17.459 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.459 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.459 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.459 DEBUG:boto:Params: {} 2024-04-16T18:17:17.459 DEBUG:boto:Token: None 2024-04-16T18:17:17.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d 2024-04-16T18:17:17.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc= 2024-04-16T18:17:17.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc=', 'Content-Length': '0'} 2024-04-16T18:17:17.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000957fae678d29c54e-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.463 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:17:17.463 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.464 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.464 DEBUG:boto:Method: GET 2024-04-16T18:17:17.464 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:17.464 DEBUG:boto:Data: 2024-04-16T18:17:17.464 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.464 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.464 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.465 DEBUG:boto:Params: {} 2024-04-16T18:17:17.465 DEBUG:boto:Token: None 2024-04-16T18:17:17.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d 2024-04-16T18:17:17.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8= 2024-04-16T18:17:17.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8=', 'Content-Length': '0'} 2024-04-16T18:17:17.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cade9a280d3e6824-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.511 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:17.511 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:17.511 DEBUG:boto:Method: GET 2024-04-16T18:17:17.511 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:17.511 DEBUG:boto:Data: 2024-04-16T18:17:17.511 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.511 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.511 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.512 DEBUG:boto:Params: {} 2024-04-16T18:17:17.512 DEBUG:boto:Token: None 2024-04-16T18:17:17.512 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/d 2024-04-16T18:17:17.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8= 2024-04-16T18:17:17.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8=', 'Content-Length': '0'} 2024-04-16T18:17:17.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f585a00f598c00f5-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.516 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.516 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.516 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.516 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.516 DEBUG:boto:Method: GET 2024-04-16T18:17:17.516 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:17.517 DEBUG:boto:Data: 2024-04-16T18:17:17.517 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.517 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.517 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.517 DEBUG:boto:Params: {} 2024-04-16T18:17:17.517 DEBUG:boto:Token: None 2024-04-16T18:17:17.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.522 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b808f2b546cbd2ed-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.562 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.563 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.563 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.563 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.563 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:17.563 DEBUG:boto:Method: GET 2024-04-16T18:17:17.563 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:17.564 DEBUG:boto:Data: 2024-04-16T18:17:17.564 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.564 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.564 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.564 DEBUG:boto:Params: {} 2024-04-16T18:17:17.564 DEBUG:boto:Token: None 2024-04-16T18:17:17.564 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063fd38737b171ab1-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.569 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.569 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:17:17.569 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.569 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.569 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.570 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.570 DEBUG:boto:Method: GET 2024-04-16T18:17:17.570 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:17.570 DEBUG:boto:Data: 2024-04-16T18:17:17.570 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.570 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.570 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.570 DEBUG:boto:Params: {} 2024-04-16T18:17:17.571 DEBUG:boto:Token: None 2024-04-16T18:17:17.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:17.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T18:17:17.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T18:17:17.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4296b5745e993c8-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.618 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.619 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.619 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.619 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.619 DEBUG:boto:Method: GET 2024-04-16T18:17:17.619 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:17.619 DEBUG:boto:Data: 2024-04-16T18:17:17.619 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.620 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.620 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.620 DEBUG:boto:Params: {} 2024-04-16T18:17:17.620 DEBUG:boto:Token: None 2024-04-16T18:17:17.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:17.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T18:17:17.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T18:17:17.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f832eb25cd103bb5-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.624 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.625 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.625 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.625 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.625 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.625 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:17.625 DEBUG:boto:Data: 2024-04-16T18:17:17.625 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.625 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.626 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.626 DEBUG:boto:Params: {} 2024-04-16T18:17:17.626 DEBUG:boto:Token: None 2024-04-16T18:17:17.626 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:17.626 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM= 2024-04-16T18:17:17.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM=', 'Content-Length': '0'} 2024-04-16T18:17:17.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000581f7cb195ec8fa-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.630 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.630 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.630 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.630 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.631 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.631 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:17.631 DEBUG:boto:Data: 2024-04-16T18:17:17.631 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.631 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.631 DEBUG:boto:Params: {} 2024-04-16T18:17:17.631 DEBUG:boto:Token: None 2024-04-16T18:17:17.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:17.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM= 2024-04-16T18:17:17.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM=', 'Content-Length': '0'} 2024-04-16T18:17:17.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b68e0852735367ca-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.636 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:17:17.636 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.636 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.636 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.636 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.636 DEBUG:boto:Method: GET 2024-04-16T18:17:17.637 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:17.637 DEBUG:boto:Data: 2024-04-16T18:17:17.637 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.637 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.637 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.637 DEBUG:boto:Params: {} 2024-04-16T18:17:17.637 DEBUG:boto:Token: None 2024-04-16T18:17:17.638 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:17.638 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T18:17:17.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T18:17:17.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000441e1c83194c9355-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.682 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.683 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.683 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.683 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:17.683 DEBUG:boto:Method: GET 2024-04-16T18:17:17.683 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:17.683 DEBUG:boto:Data: 2024-04-16T18:17:17.684 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.684 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.684 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.684 DEBUG:boto:Params: {} 2024-04-16T18:17:17.684 DEBUG:boto:Token: None 2024-04-16T18:17:17.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:17.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T18:17:17.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T18:17:17.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089959a83d3524d2d-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.688 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.689 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.689 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.689 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.689 DEBUG:boto:Method: GET 2024-04-16T18:17:17.689 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:17.689 DEBUG:boto:Data: 2024-04-16T18:17:17.689 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.690 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.690 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.690 DEBUG:boto:Params: {} 2024-04-16T18:17:17.690 DEBUG:boto:Token: None 2024-04-16T18:17:17.690 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.690 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.695 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018e297a8a51bad07-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.738 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.739 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.739 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.739 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.739 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.740 DEBUG:boto:Method: GET 2024-04-16T18:17:17.740 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:17.740 DEBUG:boto:Data: 2024-04-16T18:17:17.740 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.740 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.740 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.740 DEBUG:boto:Params: {} 2024-04-16T18:17:17.740 DEBUG:boto:Token: None 2024-04-16T18:17:17.741 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.741 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.745 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009da02acc863571d6-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.745 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.746 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.746 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.746 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.746 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:17.746 DEBUG:boto:Data: 2024-04-16T18:17:17.746 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.747 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.747 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.747 DEBUG:boto:Params: {} 2024-04-16T18:17:17.747 DEBUG:boto:Token: None 2024-04-16T18:17:17.747 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e 2024-04-16T18:17:17.747 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM= 2024-04-16T18:17:17.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM=', 'Content-Length': '0'} 2024-04-16T18:17:17.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf183f5ec5735360-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.751 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.751 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.751 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.751 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:17.751 DEBUG:boto:Data: 2024-04-16T18:17:17.752 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.752 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.752 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.752 DEBUG:boto:Params: {} 2024-04-16T18:17:17.752 DEBUG:boto:Token: None 2024-04-16T18:17:17.752 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e 2024-04-16T18:17:17.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM= 2024-04-16T18:17:17.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM=', 'Content-Length': '0'} 2024-04-16T18:17:17.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b23fff7046528f3-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.756 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:17:17.756 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.756 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.756 DEBUG:boto:Method: GET 2024-04-16T18:17:17.757 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:17.757 DEBUG:boto:Data: 2024-04-16T18:17:17.757 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.757 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.757 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.757 DEBUG:boto:Params: {} 2024-04-16T18:17:17.757 DEBUG:boto:Token: None 2024-04-16T18:17:17.758 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e 2024-04-16T18:17:17.758 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI= 2024-04-16T18:17:17.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI=', 'Content-Length': '0'} 2024-04-16T18:17:17.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002627e91df3d4671b-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.802 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:17.803 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:17.803 DEBUG:boto:Method: GET 2024-04-16T18:17:17.803 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:17.803 DEBUG:boto:Data: 2024-04-16T18:17:17.803 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.803 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.803 DEBUG:boto:Params: {} 2024-04-16T18:17:17.804 DEBUG:boto:Token: None 2024-04-16T18:17:17.804 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/e 2024-04-16T18:17:17.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI= 2024-04-16T18:17:17.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI=', 'Content-Length': '0'} 2024-04-16T18:17:17.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b423da786b2b964c-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.808 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.808 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.808 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.808 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.808 DEBUG:boto:Method: GET 2024-04-16T18:17:17.809 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:17.809 DEBUG:boto:Data: 2024-04-16T18:17:17.809 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.809 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.809 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.809 DEBUG:boto:Params: {} 2024-04-16T18:17:17.809 DEBUG:boto:Token: None 2024-04-16T18:17:17.810 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.810 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dac45b11b75ffa17-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.854 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.855 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.855 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.855 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.856 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:17.856 DEBUG:boto:Method: GET 2024-04-16T18:17:17.856 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:17.856 DEBUG:boto:Data: 2024-04-16T18:17:17.856 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.856 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.856 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.856 DEBUG:boto:Params: {} 2024-04-16T18:17:17.857 DEBUG:boto:Token: None 2024-04-16T18:17:17.857 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.861 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a7c42fe1e33a3d3-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.861 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:17.862 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:17:17.862 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:17.862 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:17.862 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.862 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.862 DEBUG:boto:Method: GET 2024-04-16T18:17:17.863 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:17.863 DEBUG:boto:Data: 2024-04-16T18:17:17.863 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.863 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.863 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.863 DEBUG:boto:Params: {} 2024-04-16T18:17:17.863 DEBUG:boto:Token: None 2024-04-16T18:17:17.863 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:17.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T18:17:17.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T18:17:17.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000647b14d90740c109-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.910 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:17.911 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:17.911 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.911 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.911 DEBUG:boto:Method: GET 2024-04-16T18:17:17.911 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:17.911 DEBUG:boto:Data: 2024-04-16T18:17:17.911 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.912 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.912 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.912 DEBUG:boto:Params: {} 2024-04-16T18:17:17.912 DEBUG:boto:Token: None 2024-04-16T18:17:17.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:17.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T18:17:17.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T18:17:17.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fef2035d89b1ead1-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.916 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:17.916 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:17.916 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.917 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.917 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.917 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:17.917 DEBUG:boto:Data: 2024-04-16T18:17:17.917 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.917 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.917 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.917 DEBUG:boto:Params: {} 2024-04-16T18:17:17.918 DEBUG:boto:Token: None 2024-04-16T18:17:17.918 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:17.918 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s= 2024-04-16T18:17:17.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s=', 'Content-Length': '0'} 2024-04-16T18:17:17.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038912aed22eeae79-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.921 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:17.922 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:17.922 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.922 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.922 DEBUG:boto:Method: HEAD 2024-04-16T18:17:17.922 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:17.922 DEBUG:boto:Data: 2024-04-16T18:17:17.922 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.923 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.923 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.923 DEBUG:boto:Params: {} 2024-04-16T18:17:17.923 DEBUG:boto:Token: None 2024-04-16T18:17:17.923 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:17.923 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s= 2024-04-16T18:17:17.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s=', 'Content-Length': '0'} 2024-04-16T18:17:17.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a12c70c254b58b2-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:17:17.927 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:17.927 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:17.927 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.927 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.928 DEBUG:boto:Method: GET 2024-04-16T18:17:17.928 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:17.928 DEBUG:boto:Data: 2024-04-16T18:17:17.928 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.928 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.928 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.928 DEBUG:boto:Params: {} 2024-04-16T18:17:17.928 DEBUG:boto:Token: None 2024-04-16T18:17:17.929 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:17.929 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T18:17:17.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T18:17:17.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008262f880d5c7e7ed-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.974 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:17.975 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:17.975 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.975 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:17.975 DEBUG:boto:Method: GET 2024-04-16T18:17:17.975 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:17.975 DEBUG:boto:Data: 2024-04-16T18:17:17.975 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.975 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:17.975 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.975 DEBUG:boto:Params: {} 2024-04-16T18:17:17.976 DEBUG:boto:Token: None 2024-04-16T18:17:17.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:17.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T18:17:17.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T18:17:17.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020696a23cfc97d4d-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:17.980 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:17.980 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:17.980 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:17.980 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:17.980 DEBUG:boto:Method: GET 2024-04-16T18:17:17.980 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:17.980 DEBUG:boto:Data: 2024-04-16T18:17:17.981 DEBUG:boto:Headers: {} 2024-04-16T18:17:17.981 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:17.981 DEBUG:boto:Port: 8000 2024-04-16T18:17:17.981 DEBUG:boto:Params: {} 2024-04-16T18:17:17.981 DEBUG:boto:Token: None 2024-04-16T18:17:17.981 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:17 GMT /lybtug-50/ 2024-04-16T18:17:17.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T18:17:17.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T18:17:17.985 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007db1a0703a73bb31-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.026 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.027 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.027 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.027 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:18.027 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:18.027 DEBUG:boto:Method: GET 2024-04-16T18:17:18.027 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:18.027 DEBUG:boto:Data: 2024-04-16T18:17:18.028 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.028 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.028 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.028 DEBUG:boto:Params: {} 2024-04-16T18:17:18.028 DEBUG:boto:Token: None 2024-04-16T18:17:18.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0cfd0f7f40518d5-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.032 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.033 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:18.033 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:18.033 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.033 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:18.033 DEBUG:boto:Data: 2024-04-16T18:17:18.033 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.033 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.033 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.034 DEBUG:boto:Params: {} 2024-04-16T18:17:18.034 DEBUG:boto:Token: None 2024-04-16T18:17:18.034 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/f 2024-04-16T18:17:18.034 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU= 2024-04-16T18:17:18.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU=', 'Content-Length': '0'} 2024-04-16T18:17:18.037 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d4a18f582318937-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.037 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:18.037 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:18.038 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.038 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:18.038 DEBUG:boto:Data: 2024-04-16T18:17:18.038 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.038 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.038 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.038 DEBUG:boto:Params: {} 2024-04-16T18:17:18.038 DEBUG:boto:Token: None 2024-04-16T18:17:18.038 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/f 2024-04-16T18:17:18.039 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU= 2024-04-16T18:17:18.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU=', 'Content-Length': '0'} 2024-04-16T18:17:18.042 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2396b9ee6ae3cf6-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.042 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:17:18.042 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:18.042 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:18.042 DEBUG:boto:Method: GET 2024-04-16T18:17:18.042 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:18.042 DEBUG:boto:Data: 2024-04-16T18:17:18.043 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.043 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.043 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.043 DEBUG:boto:Params: {} 2024-04-16T18:17:18.043 DEBUG:boto:Token: None 2024-04-16T18:17:18.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/f 2024-04-16T18:17:18.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw= 2024-04-16T18:17:18.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw=', 'Content-Length': '0'} 2024-04-16T18:17:18.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4ceb83cac0b9159-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.086 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:18.086 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:18.087 DEBUG:boto:Method: GET 2024-04-16T18:17:18.087 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:18.087 DEBUG:boto:Data: 2024-04-16T18:17:18.087 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.087 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.087 DEBUG:boto:Params: {} 2024-04-16T18:17:18.087 DEBUG:boto:Token: None 2024-04-16T18:17:18.088 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/f 2024-04-16T18:17:18.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw= 2024-04-16T18:17:18.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw=', 'Content-Length': '0'} 2024-04-16T18:17:18.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021f96a1e7366acd0-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.091 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.091 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.091 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:18.091 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:18.092 DEBUG:boto:Method: GET 2024-04-16T18:17:18.092 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:18.092 DEBUG:boto:Data: 2024-04-16T18:17:18.092 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.092 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.092 DEBUG:boto:Params: {} 2024-04-16T18:17:18.092 DEBUG:boto:Token: None 2024-04-16T18:17:18.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.093 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.097 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000749c8b43d085cd83-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.097 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.097 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.098 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.098 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:18.098 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:18.098 DEBUG:boto:Method: GET 2024-04-16T18:17:18.098 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:18.098 DEBUG:boto:Data: 2024-04-16T18:17:18.098 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.098 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.098 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.099 DEBUG:boto:Params: {} 2024-04-16T18:17:18.099 DEBUG:boto:Token: None 2024-04-16T18:17:18.099 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.099 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.103 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b53210aeff2990b-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.103 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.103 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:17:18.104 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.104 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.104 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.104 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.104 DEBUG:boto:Method: GET 2024-04-16T18:17:18.104 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:18.104 DEBUG:boto:Data: 2024-04-16T18:17:18.104 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.104 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.105 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.105 DEBUG:boto:Params: {} 2024-04-16T18:17:18.105 DEBUG:boto:Token: None 2024-04-16T18:17:18.105 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:18.105 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T18:17:18.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T18:17:18.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b4b802de66cf1b8-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.150 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.151 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.151 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.151 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.151 DEBUG:boto:Method: GET 2024-04-16T18:17:18.151 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:18.151 DEBUG:boto:Data: 2024-04-16T18:17:18.151 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.151 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.151 DEBUG:boto:Params: {} 2024-04-16T18:17:18.152 DEBUG:boto:Token: None 2024-04-16T18:17:18.152 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:18.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T18:17:18.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T18:17:18.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe6a3540e3f77ee3-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.155 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.155 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.156 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.156 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.156 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.156 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:18.156 DEBUG:boto:Data: 2024-04-16T18:17:18.156 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.156 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.156 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.156 DEBUG:boto:Params: {} 2024-04-16T18:17:18.156 DEBUG:boto:Token: None 2024-04-16T18:17:18.157 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:18.157 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg= 2024-04-16T18:17:18.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg=', 'Content-Length': '0'} 2024-04-16T18:17:18.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2df72848c33ac77-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.160 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.160 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.160 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.160 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.160 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.160 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:18.160 DEBUG:boto:Data: 2024-04-16T18:17:18.160 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.160 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.161 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.161 DEBUG:boto:Params: {} 2024-04-16T18:17:18.161 DEBUG:boto:Token: None 2024-04-16T18:17:18.161 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:18.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg= 2024-04-16T18:17:18.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg=', 'Content-Length': '0'} 2024-04-16T18:17:18.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002bf4138a9a9084c-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.164 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:17:18.164 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.165 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.165 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.165 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.165 DEBUG:boto:Method: GET 2024-04-16T18:17:18.165 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:18.165 DEBUG:boto:Data: 2024-04-16T18:17:18.165 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.165 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.165 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.165 DEBUG:boto:Params: {} 2024-04-16T18:17:18.165 DEBUG:boto:Token: None 2024-04-16T18:17:18.166 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:18.166 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T18:17:18.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T18:17:18.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055fc90d799218396-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.210 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.211 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.211 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.211 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:18.211 DEBUG:boto:Method: GET 2024-04-16T18:17:18.211 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:18.211 DEBUG:boto:Data: 2024-04-16T18:17:18.211 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.211 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.211 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.211 DEBUG:boto:Params: {} 2024-04-16T18:17:18.211 DEBUG:boto:Token: None 2024-04-16T18:17:18.212 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:18.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T18:17:18.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T18:17:18.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baa67b5fa5449665-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.215 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.215 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.215 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.215 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.216 DEBUG:boto:Method: GET 2024-04-16T18:17:18.216 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:18.216 DEBUG:boto:Data: 2024-04-16T18:17:18.216 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.216 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.216 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.216 DEBUG:boto:Params: {} 2024-04-16T18:17:18.216 DEBUG:boto:Token: None 2024-04-16T18:17:18.216 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.220 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b30700ca55709edc-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.262 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.263 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.263 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.263 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.263 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.263 DEBUG:boto:Method: GET 2024-04-16T18:17:18.263 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:18.263 DEBUG:boto:Data: 2024-04-16T18:17:18.263 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.263 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.263 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.264 DEBUG:boto:Params: {} 2024-04-16T18:17:18.264 DEBUG:boto:Token: None 2024-04-16T18:17:18.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001743687a37f33c6c-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.268 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.268 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.268 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.268 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.268 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:18.269 DEBUG:boto:Data: 2024-04-16T18:17:18.269 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.269 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.269 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.269 DEBUG:boto:Params: {} 2024-04-16T18:17:18.269 DEBUG:boto:Token: None 2024-04-16T18:17:18.269 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g 2024-04-16T18:17:18.269 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM= 2024-04-16T18:17:18.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM=', 'Content-Length': '0'} 2024-04-16T18:17:18.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e6d3636ae51385a-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.272 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.272 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.272 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.273 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:18.273 DEBUG:boto:Data: 2024-04-16T18:17:18.273 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.273 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.273 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.273 DEBUG:boto:Params: {} 2024-04-16T18:17:18.273 DEBUG:boto:Token: None 2024-04-16T18:17:18.273 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g 2024-04-16T18:17:18.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM= 2024-04-16T18:17:18.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM=', 'Content-Length': '0'} 2024-04-16T18:17:18.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5a0c03b02d92dab-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.276 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:17:18.277 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.277 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.277 DEBUG:boto:Method: GET 2024-04-16T18:17:18.277 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:18.277 DEBUG:boto:Data: 2024-04-16T18:17:18.277 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.277 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.277 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.277 DEBUG:boto:Params: {} 2024-04-16T18:17:18.277 DEBUG:boto:Token: None 2024-04-16T18:17:18.278 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g 2024-04-16T18:17:18.278 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8= 2024-04-16T18:17:18.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8=', 'Content-Length': '0'} 2024-04-16T18:17:18.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000309cfab461ab8ad6-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.322 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:18.322 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:18.323 DEBUG:boto:Method: GET 2024-04-16T18:17:18.323 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:18.323 DEBUG:boto:Data: 2024-04-16T18:17:18.323 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.323 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.323 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.323 DEBUG:boto:Params: {} 2024-04-16T18:17:18.323 DEBUG:boto:Token: None 2024-04-16T18:17:18.323 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/g 2024-04-16T18:17:18.323 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8= 2024-04-16T18:17:18.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8=', 'Content-Length': '0'} 2024-04-16T18:17:18.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090cbf17f378a2bc6-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.327 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.327 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.327 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.327 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.327 DEBUG:boto:Method: GET 2024-04-16T18:17:18.327 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:18.327 DEBUG:boto:Data: 2024-04-16T18:17:18.327 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.327 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.327 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.327 DEBUG:boto:Params: {} 2024-04-16T18:17:18.328 DEBUG:boto:Token: None 2024-04-16T18:17:18.328 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.332 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006435e0da09a43c0c-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.374 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.375 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.375 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.375 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.375 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:18.375 DEBUG:boto:Method: GET 2024-04-16T18:17:18.375 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:18.375 DEBUG:boto:Data: 2024-04-16T18:17:18.375 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.375 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.375 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.376 DEBUG:boto:Params: {} 2024-04-16T18:17:18.376 DEBUG:boto:Token: None 2024-04-16T18:17:18.376 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf1fe7ec92ea22be-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.380 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:17:18.380 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.381 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.381 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.381 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.381 DEBUG:boto:Method: GET 2024-04-16T18:17:18.381 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:18.381 DEBUG:boto:Data: 2024-04-16T18:17:18.381 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.381 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.381 DEBUG:boto:Params: {} 2024-04-16T18:17:18.381 DEBUG:boto:Token: None 2024-04-16T18:17:18.382 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:18.382 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T18:17:18.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T18:17:18.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2b2f70ebc2be26b-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.426 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.426 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.427 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.427 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.427 DEBUG:boto:Method: GET 2024-04-16T18:17:18.427 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:18.427 DEBUG:boto:Data: 2024-04-16T18:17:18.427 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.427 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.427 DEBUG:boto:Params: {} 2024-04-16T18:17:18.427 DEBUG:boto:Token: None 2024-04-16T18:17:18.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:18.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T18:17:18.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T18:17:18.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5d8333032b3653c-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.431 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.431 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.432 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.432 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.432 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.432 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:18.432 DEBUG:boto:Data: 2024-04-16T18:17:18.432 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.432 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.432 DEBUG:boto:Params: {} 2024-04-16T18:17:18.432 DEBUG:boto:Token: None 2024-04-16T18:17:18.433 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:18.433 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q= 2024-04-16T18:17:18.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q=', 'Content-Length': '0'} 2024-04-16T18:17:18.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca060d8c7eb22fd3-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.436 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.436 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.436 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.436 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.436 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.436 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:18.437 DEBUG:boto:Data: 2024-04-16T18:17:18.437 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.437 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.437 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.437 DEBUG:boto:Params: {} 2024-04-16T18:17:18.437 DEBUG:boto:Token: None 2024-04-16T18:17:18.437 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:18.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q= 2024-04-16T18:17:18.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q=', 'Content-Length': '0'} 2024-04-16T18:17:18.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c03b698e783382b6-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.440 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:17:18.441 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.441 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.441 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.441 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.441 DEBUG:boto:Method: GET 2024-04-16T18:17:18.441 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:18.441 DEBUG:boto:Data: 2024-04-16T18:17:18.441 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.441 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.441 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.442 DEBUG:boto:Params: {} 2024-04-16T18:17:18.442 DEBUG:boto:Token: None 2024-04-16T18:17:18.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:18.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T18:17:18.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T18:17:18.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1ae2e2cddfcd164-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.486 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.486 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.487 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.487 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:18.487 DEBUG:boto:Method: GET 2024-04-16T18:17:18.487 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:18.487 DEBUG:boto:Data: 2024-04-16T18:17:18.487 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.487 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.487 DEBUG:boto:Params: {} 2024-04-16T18:17:18.487 DEBUG:boto:Token: None 2024-04-16T18:17:18.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:18.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T18:17:18.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T18:17:18.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ec3a85523b3e16d-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.491 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.491 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.491 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.491 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.491 DEBUG:boto:Method: GET 2024-04-16T18:17:18.492 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:18.492 DEBUG:boto:Data: 2024-04-16T18:17:18.492 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.492 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.492 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.492 DEBUG:boto:Params: {} 2024-04-16T18:17:18.492 DEBUG:boto:Token: None 2024-04-16T18:17:18.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.496 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1a85348bd49d5a5-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.496 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.497 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.497 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.497 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.497 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.497 DEBUG:boto:Method: GET 2024-04-16T18:17:18.497 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:18.497 DEBUG:boto:Data: 2024-04-16T18:17:18.497 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.497 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.497 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.498 DEBUG:boto:Params: {} 2024-04-16T18:17:18.498 DEBUG:boto:Token: None 2024-04-16T18:17:18.498 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.498 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.502 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed6e2ed003d1bdcd-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.546 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.547 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.547 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.547 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.547 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:18.547 DEBUG:boto:Data: 2024-04-16T18:17:18.547 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.547 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.547 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.547 DEBUG:boto:Params: {} 2024-04-16T18:17:18.548 DEBUG:boto:Token: None 2024-04-16T18:17:18.548 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h 2024-04-16T18:17:18.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg= 2024-04-16T18:17:18.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg=', 'Content-Length': '0'} 2024-04-16T18:17:18.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004328659a15ac77ff-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.551 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.552 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.552 DEBUG:boto:Method: HEAD 2024-04-16T18:17:18.552 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:18.552 DEBUG:boto:Data: 2024-04-16T18:17:18.552 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.552 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.552 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.552 DEBUG:boto:Params: {} 2024-04-16T18:17:18.552 DEBUG:boto:Token: None 2024-04-16T18:17:18.552 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h 2024-04-16T18:17:18.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg= 2024-04-16T18:17:18.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg=', 'Content-Length': '0'} 2024-04-16T18:17:18.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000aeec3e0d7410ee7-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.556 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:17:18.556 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.556 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.556 DEBUG:boto:Method: GET 2024-04-16T18:17:18.556 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:18.556 DEBUG:boto:Data: 2024-04-16T18:17:18.556 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.557 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.557 DEBUG:boto:Params: {} 2024-04-16T18:17:18.557 DEBUG:boto:Token: None 2024-04-16T18:17:18.557 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h 2024-04-16T18:17:18.557 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY= 2024-04-16T18:17:18.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY=', 'Content-Length': '0'} 2024-04-16T18:17:18.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081bf062ac161d869-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.602 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:18.602 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:18.603 DEBUG:boto:Method: GET 2024-04-16T18:17:18.603 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:18.603 DEBUG:boto:Data: 2024-04-16T18:17:18.603 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.603 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.603 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.603 DEBUG:boto:Params: {} 2024-04-16T18:17:18.603 DEBUG:boto:Token: None 2024-04-16T18:17:18.603 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/h 2024-04-16T18:17:18.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY= 2024-04-16T18:17:18.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY=', 'Content-Length': '0'} 2024-04-16T18:17:18.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000415fbbf2e2dab53f-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.607 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.607 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.607 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.607 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.607 DEBUG:boto:Method: GET 2024-04-16T18:17:18.607 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:18.607 DEBUG:boto:Data: 2024-04-16T18:17:18.608 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.608 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:18.608 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.608 DEBUG:boto:Params: {} 2024-04-16T18:17:18.608 DEBUG:boto:Token: None 2024-04-16T18:17:18.608 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.612 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058a3676c3d29cc61-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.654 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.655 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:18.655 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:18.655 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.655 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:18.655 DEBUG:boto:Method: GET 2024-04-16T18:17:18.655 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:18.655 DEBUG:boto:Data: 2024-04-16T18:17:18.655 DEBUG:boto:Headers: {} 2024-04-16T18:17:18.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:18.655 DEBUG:boto:Port: 8000 2024-04-16T18:17:18.656 DEBUG:boto:Params: {} 2024-04-16T18:17:18.656 DEBUG:boto:Token: None 2024-04-16T18:17:18.656 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:18 GMT /lybtug-50/ 2024-04-16T18:17:18.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T18:17:18.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T18:17:18.660 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066eb2b689732b784-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:18.660 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:18.660 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T18:17:18.661 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:18.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:18.661 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket lybtug-50 --num-shards 15 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:18.702 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:18.703 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:18.794 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:18.794 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-50 2024-04-16T18:17:18.804 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 8 2024-04-16T18:17:18.908 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:18.909 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:18.909 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:18.986 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:18.987 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:19.087 INFO:teuthology.orchestra.run.smithi071.stdout:{"layout":{"resharding":"None","current_index":{"gen":2,"layout":{"type":"Normal","normal":{"num_shards":15,"hash_type":"Mod"}}},"logs":[{"gen":0,"layout":{"type":"InIndex","in_index":{"gen":0,"layout":{"num_shards":11,"hash_type":"Mod"}}}},{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}},{"gen":2,"layout":{"type":"InIndex","in_index":{"gen":2,"layout":{"num_shards":15,"hash_type":"Mod"}}}}]}} 2024-04-16T18:17:19.087 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-50 2024-04-16T18:17:19.087 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:19.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:19.087 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:17:19.138 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:19.139 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:19.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:19.221+0000 7fde8b1ef980 1 waiting to reach latest gen 2, on 1.. 2024-04-16T18:17:24.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:24.229+0000 7fde8b1ef980 1 bucket sync caught up with empty source 2024-04-16T18:17:24.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:24.229+0000 7fde8b1ef980 0 bucket checkpoint complete 2024-04-16T18:17:24.270 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-50 2024-04-16T18:17:24.271 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:24.271 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:24.271 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:17:24.322 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:24.322 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:24.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:24.405+0000 7f72223d0980 1 bucket sync caught up with source: 2024-04-16T18:17:24.402 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T18:17:24.402 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T18:17:24.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:24.405+0000 7f72223d0980 0 bucket checkpoint complete 2024-04-16T18:17:24.433 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T18:17:24.434 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.434 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.434 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.434 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:17:24.434 DEBUG:boto:Data: 2024-04-16T18:17:24.434 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.434 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.434 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.434 DEBUG:boto:Params: {} 2024-04-16T18:17:24.434 DEBUG:boto:Token: None 2024-04-16T18:17:24.435 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.435 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM= 2024-04-16T18:17:24.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM=', 'Content-Length': '0'} 2024-04-16T18:17:24.439 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0439cf64ce0e8de-00661ec0b4-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.439 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.439 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.439 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.439 DEBUG:boto:Path: /lybtug-50/ 2024-04-16T18:17:24.439 DEBUG:boto:Data: 2024-04-16T18:17:24.439 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.439 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.439 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.440 DEBUG:boto:Params: {} 2024-04-16T18:17:24.440 DEBUG:boto:Token: None 2024-04-16T18:17:24.440 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM= 2024-04-16T18:17:24.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM=', 'Content-Length': '0'} 2024-04-16T18:17:24.443 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001b2541f1698401cd-00661ec0b4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.443 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:17:24.443 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.443 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.443 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.443 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.443 DEBUG:boto:Method: GET 2024-04-16T18:17:24.443 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.443 DEBUG:boto:Data: 2024-04-16T18:17:24.443 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.444 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.444 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.444 DEBUG:boto:Params: {} 2024-04-16T18:17:24.444 DEBUG:boto:Token: None 2024-04-16T18:17:24.444 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/?versions 2024-04-16T18:17:24.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T18:17:24.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T18:17:24.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db991537d01f3bed-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.448 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:17:24.449 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:17:24.450 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:17:24.450 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:17:24.450 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.450 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.450 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.450 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.450 DEBUG:boto:Method: GET 2024-04-16T18:17:24.450 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.450 DEBUG:boto:Data: 2024-04-16T18:17:24.450 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.451 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.451 DEBUG:boto:Params: {} 2024-04-16T18:17:24.451 DEBUG:boto:Token: None 2024-04-16T18:17:24.451 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/?versions 2024-04-16T18:17:24.451 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T18:17:24.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T18:17:24.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c73e0eefd9983c96-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.454 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=a 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=b 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=c 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=d 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=e 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=f 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=g 2024-04-16T18:17:24.455 DEBUG:tasks.rgw_multi.zone_rados:o=h 2024-04-16T18:17:24.455 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.455 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.456 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.456 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.456 DEBUG:boto:Method: GET 2024-04-16T18:17:24.456 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.456 DEBUG:boto:Data: 2024-04-16T18:17:24.456 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.456 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.456 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.456 DEBUG:boto:Params: {} 2024-04-16T18:17:24.456 DEBUG:boto:Token: None 2024-04-16T18:17:24.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/?versions 2024-04-16T18:17:24.457 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T18:17:24.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T18:17:24.460 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055ee618352cfd940-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.460 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.461 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.461 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.461 DEBUG:boto:path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.461 DEBUG:boto:auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.461 DEBUG:boto:Method: GET 2024-04-16T18:17:24.461 DEBUG:boto:Path: /lybtug-50/?versions&max-keys=999 2024-04-16T18:17:24.462 DEBUG:boto:Data: 2024-04-16T18:17:24.462 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.462 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.462 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.462 DEBUG:boto:Params: {} 2024-04-16T18:17:24.462 DEBUG:boto:Token: None 2024-04-16T18:17:24.462 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/?versions 2024-04-16T18:17:24.462 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T18:17:24.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T18:17:24.465 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe2c690732990985-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.465 DEBUG:boto:b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:17:24.466 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.466 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.466 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.466 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.466 DEBUG:boto:Method: GET 2024-04-16T18:17:24.467 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:24.467 DEBUG:boto:Data: 2024-04-16T18:17:24.467 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.467 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.467 DEBUG:boto:Params: {} 2024-04-16T18:17:24.467 DEBUG:boto:Token: None 2024-04-16T18:17:24.467 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:24.467 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T18:17:24.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T18:17:24.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000624f87421d358c29-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.515 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.515 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.515 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.515 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.515 DEBUG:boto:Method: GET 2024-04-16T18:17:24.515 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:24.515 DEBUG:boto:Data: 2024-04-16T18:17:24.515 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.515 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.515 DEBUG:boto:Params: {} 2024-04-16T18:17:24.515 DEBUG:boto:Token: None 2024-04-16T18:17:24.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:24.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T18:17:24.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T18:17:24.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5fb2b8ec1e7de41-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.519 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.519 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.519 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.519 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.519 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.519 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:24.519 DEBUG:boto:Data: 2024-04-16T18:17:24.519 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.520 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.520 DEBUG:boto:Params: {} 2024-04-16T18:17:24.520 DEBUG:boto:Token: None 2024-04-16T18:17:24.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:24.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ= 2024-04-16T18:17:24.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ=', 'Content-Length': '0'} 2024-04-16T18:17:24.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fa4af6a4b5e85fb-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.523 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.523 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.524 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.524 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.524 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.524 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:24.524 DEBUG:boto:Data: 2024-04-16T18:17:24.524 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.524 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.524 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.524 DEBUG:boto:Params: {} 2024-04-16T18:17:24.524 DEBUG:boto:Token: None 2024-04-16T18:17:24.524 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:24.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ= 2024-04-16T18:17:24.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ=', 'Content-Length': '0'} 2024-04-16T18:17:24.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2d0f1719bf6aa82-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:17:24.527 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.527 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.528 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.528 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.528 DEBUG:boto:Method: GET 2024-04-16T18:17:24.528 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:24.528 DEBUG:boto:Data: 2024-04-16T18:17:24.528 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.528 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.528 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.528 DEBUG:boto:Params: {} 2024-04-16T18:17:24.528 DEBUG:boto:Token: None 2024-04-16T18:17:24.529 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:24.529 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T18:17:24.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T18:17:24.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5998a2f701bfde0-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.574 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.575 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.575 DEBUG:boto:path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.575 DEBUG:boto:auth_path=/lybtug-50/a?versionId=null 2024-04-16T18:17:24.575 DEBUG:boto:Method: GET 2024-04-16T18:17:24.575 DEBUG:boto:Path: /lybtug-50/a?versionId=null 2024-04-16T18:17:24.575 DEBUG:boto:Data: 2024-04-16T18:17:24.575 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.575 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.575 DEBUG:boto:Params: {} 2024-04-16T18:17:24.576 DEBUG:boto:Token: None 2024-04-16T18:17:24.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a?versionId=null 2024-04-16T18:17:24.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T18:17:24.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T18:17:24.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd8282fc10c4e8a9-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.579 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.579 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.579 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.579 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.579 DEBUG:boto:Method: GET 2024-04-16T18:17:24.579 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:24.580 DEBUG:boto:Data: 2024-04-16T18:17:24.580 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.580 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.580 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.580 DEBUG:boto:Params: {} 2024-04-16T18:17:24.580 DEBUG:boto:Token: None 2024-04-16T18:17:24.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073cd4418032fb5f6-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.626 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.627 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.627 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.627 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.627 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.627 DEBUG:boto:Method: GET 2024-04-16T18:17:24.627 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:24.627 DEBUG:boto:Data: 2024-04-16T18:17:24.627 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.627 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.627 DEBUG:boto:Params: {} 2024-04-16T18:17:24.628 DEBUG:boto:Token: None 2024-04-16T18:17:24.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8abead73ae0a159-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.632 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.633 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.633 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.633 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.633 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:24.633 DEBUG:boto:Data: 2024-04-16T18:17:24.633 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.633 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.633 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.633 DEBUG:boto:Params: {} 2024-04-16T18:17:24.633 DEBUG:boto:Token: None 2024-04-16T18:17:24.633 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a 2024-04-16T18:17:24.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU= 2024-04-16T18:17:24.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU=', 'Content-Length': '0'} 2024-04-16T18:17:24.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f32eb342b13ea44c-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.637 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.637 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.637 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.637 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:24.637 DEBUG:boto:Data: 2024-04-16T18:17:24.637 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.637 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.637 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.637 DEBUG:boto:Params: {} 2024-04-16T18:17:24.637 DEBUG:boto:Token: None 2024-04-16T18:17:24.637 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a 2024-04-16T18:17:24.638 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU= 2024-04-16T18:17:24.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU=', 'Content-Length': '0'} 2024-04-16T18:17:24.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004733b289894f47e0-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2024-04-16T18:17:24.641 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.641 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.641 DEBUG:boto:Method: GET 2024-04-16T18:17:24.641 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:24.641 DEBUG:boto:Data: 2024-04-16T18:17:24.641 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.641 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.641 DEBUG:boto:Params: {} 2024-04-16T18:17:24.642 DEBUG:boto:Token: None 2024-04-16T18:17:24.642 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a 2024-04-16T18:17:24.642 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU= 2024-04-16T18:17:24.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU=', 'Content-Length': '0'} 2024-04-16T18:17:24.645 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7a0cdb71dc38990-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.686 DEBUG:boto:path=/lybtug-50/a 2024-04-16T18:17:24.686 DEBUG:boto:auth_path=/lybtug-50/a 2024-04-16T18:17:24.687 DEBUG:boto:Method: GET 2024-04-16T18:17:24.687 DEBUG:boto:Path: /lybtug-50/a 2024-04-16T18:17:24.687 DEBUG:boto:Data: 2024-04-16T18:17:24.687 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.687 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.687 DEBUG:boto:Params: {} 2024-04-16T18:17:24.687 DEBUG:boto:Token: None 2024-04-16T18:17:24.687 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/a 2024-04-16T18:17:24.687 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU= 2024-04-16T18:17:24.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU=', 'Content-Length': '0'} 2024-04-16T18:17:24.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006eb8018829fd954-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.690 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.690 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.691 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.691 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.691 DEBUG:boto:Method: GET 2024-04-16T18:17:24.691 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:24.691 DEBUG:boto:Data: 2024-04-16T18:17:24.691 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.691 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.691 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.691 DEBUG:boto:Params: {} 2024-04-16T18:17:24.691 DEBUG:boto:Token: None 2024-04-16T18:17:24.691 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c1df529351ee20d-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.738 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.738 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.739 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.739 DEBUG:boto:path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.739 DEBUG:boto:auth_path=/lybtug-50/?prefix=a 2024-04-16T18:17:24.739 DEBUG:boto:Method: GET 2024-04-16T18:17:24.739 DEBUG:boto:Path: /lybtug-50/?prefix=a 2024-04-16T18:17:24.739 DEBUG:boto:Data: 2024-04-16T18:17:24.739 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.739 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.739 DEBUG:boto:Params: {} 2024-04-16T18:17:24.739 DEBUG:boto:Token: None 2024-04-16T18:17:24.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000697fa6f751585cda-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.744 DEBUG:boto:b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:17:24.744 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.745 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.745 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.745 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.745 DEBUG:boto:Method: GET 2024-04-16T18:17:24.745 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:24.745 DEBUG:boto:Data: 2024-04-16T18:17:24.745 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.745 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.745 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.745 DEBUG:boto:Params: {} 2024-04-16T18:17:24.745 DEBUG:boto:Token: None 2024-04-16T18:17:24.745 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:24.746 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T18:17:24.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T18:17:24.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efa4836074cc2357-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.790 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.790 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.790 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.791 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.791 DEBUG:boto:Method: GET 2024-04-16T18:17:24.791 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:24.791 DEBUG:boto:Data: 2024-04-16T18:17:24.791 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.791 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.791 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.791 DEBUG:boto:Params: {} 2024-04-16T18:17:24.791 DEBUG:boto:Token: None 2024-04-16T18:17:24.791 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:24.791 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T18:17:24.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T18:17:24.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083294fb68dbdc055-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.795 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.795 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.795 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.795 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.795 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.796 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:24.796 DEBUG:boto:Data: 2024-04-16T18:17:24.796 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.796 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.796 DEBUG:boto:Params: {} 2024-04-16T18:17:24.796 DEBUG:boto:Token: None 2024-04-16T18:17:24.796 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:24.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M= 2024-04-16T18:17:24.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M=', 'Content-Length': '0'} 2024-04-16T18:17:24.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e823fa3f03785376-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.800 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.800 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.800 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.800 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.800 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.800 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:24.800 DEBUG:boto:Data: 2024-04-16T18:17:24.801 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.801 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.801 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.801 DEBUG:boto:Params: {} 2024-04-16T18:17:24.801 DEBUG:boto:Token: None 2024-04-16T18:17:24.801 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:24.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M= 2024-04-16T18:17:24.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M=', 'Content-Length': '0'} 2024-04-16T18:17:24.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005226a851074f7546-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.804 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:17:24.804 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.805 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.805 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.805 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.805 DEBUG:boto:Method: GET 2024-04-16T18:17:24.805 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:24.805 DEBUG:boto:Data: 2024-04-16T18:17:24.805 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.805 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.805 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.805 DEBUG:boto:Params: {} 2024-04-16T18:17:24.805 DEBUG:boto:Token: None 2024-04-16T18:17:24.805 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:24.806 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T18:17:24.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T18:17:24.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2f1120c55ef05f9-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.850 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.850 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.851 DEBUG:boto:path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.851 DEBUG:boto:auth_path=/lybtug-50/b?versionId=null 2024-04-16T18:17:24.851 DEBUG:boto:Method: GET 2024-04-16T18:17:24.851 DEBUG:boto:Path: /lybtug-50/b?versionId=null 2024-04-16T18:17:24.851 DEBUG:boto:Data: 2024-04-16T18:17:24.851 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.851 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.851 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.851 DEBUG:boto:Params: {} 2024-04-16T18:17:24.851 DEBUG:boto:Token: None 2024-04-16T18:17:24.851 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b?versionId=null 2024-04-16T18:17:24.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T18:17:24.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T18:17:24.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3f046c4f7ae09b4-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.855 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.855 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.855 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:24.855 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:24.856 DEBUG:boto:Method: GET 2024-04-16T18:17:24.856 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:24.856 DEBUG:boto:Data: 2024-04-16T18:17:24.856 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.856 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.856 DEBUG:boto:Params: {} 2024-04-16T18:17:24.856 DEBUG:boto:Token: None 2024-04-16T18:17:24.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b7694677e39306b-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.861 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.861 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.861 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.861 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:24.861 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:24.861 DEBUG:boto:Method: GET 2024-04-16T18:17:24.862 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:24.862 DEBUG:boto:Data: 2024-04-16T18:17:24.862 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.862 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.862 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.862 DEBUG:boto:Params: {} 2024-04-16T18:17:24.862 DEBUG:boto:Token: None 2024-04-16T18:17:24.863 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.867 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d12053607e05e264-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.910 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:24.911 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.911 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.911 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.911 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:24.911 DEBUG:boto:Data: 2024-04-16T18:17:24.911 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.911 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.911 DEBUG:boto:Params: {} 2024-04-16T18:17:24.911 DEBUG:boto:Token: None 2024-04-16T18:17:24.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b 2024-04-16T18:17:24.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto= 2024-04-16T18:17:24.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto=', 'Content-Length': '0'} 2024-04-16T18:17:24.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a30c9f0d970c96d6-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.915 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.915 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.915 DEBUG:boto:Method: HEAD 2024-04-16T18:17:24.915 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:24.915 DEBUG:boto:Data: 2024-04-16T18:17:24.915 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.915 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.915 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.915 DEBUG:boto:Params: {} 2024-04-16T18:17:24.916 DEBUG:boto:Token: None 2024-04-16T18:17:24.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b 2024-04-16T18:17:24.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto= 2024-04-16T18:17:24.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto=', 'Content-Length': '0'} 2024-04-16T18:17:24.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007786b506fb94abde-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2024-04-16T18:17:24.920 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.920 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.920 DEBUG:boto:Method: GET 2024-04-16T18:17:24.920 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:24.920 DEBUG:boto:Data: 2024-04-16T18:17:24.920 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.920 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.920 DEBUG:boto:Params: {} 2024-04-16T18:17:24.920 DEBUG:boto:Token: None 2024-04-16T18:17:24.920 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b 2024-04-16T18:17:24.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ= 2024-04-16T18:17:24.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ=', 'Content-Length': '0'} 2024-04-16T18:17:24.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b8d76cb1ab0404d-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.966 DEBUG:boto:path=/lybtug-50/b 2024-04-16T18:17:24.967 DEBUG:boto:auth_path=/lybtug-50/b 2024-04-16T18:17:24.967 DEBUG:boto:Method: GET 2024-04-16T18:17:24.967 DEBUG:boto:Path: /lybtug-50/b 2024-04-16T18:17:24.967 DEBUG:boto:Data: 2024-04-16T18:17:24.967 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:24.967 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.967 DEBUG:boto:Params: {} 2024-04-16T18:17:24.967 DEBUG:boto:Token: None 2024-04-16T18:17:24.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/b 2024-04-16T18:17:24.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ= 2024-04-16T18:17:24.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ=', 'Content-Length': '0'} 2024-04-16T18:17:24.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000acbf5d77dc2a384-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:24.971 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:24.971 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:24.971 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:24.971 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:24.971 DEBUG:boto:Method: GET 2024-04-16T18:17:24.971 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:24.971 DEBUG:boto:Data: 2024-04-16T18:17:24.972 DEBUG:boto:Headers: {} 2024-04-16T18:17:24.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:24.972 DEBUG:boto:Port: 8000 2024-04-16T18:17:24.972 DEBUG:boto:Params: {} 2024-04-16T18:17:24.972 DEBUG:boto:Token: None 2024-04-16T18:17:24.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:24 GMT /lybtug-50/ 2024-04-16T18:17:24.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T18:17:24.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T18:17:24.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068acff4f48a8ed72-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.018 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.019 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.019 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.019 DEBUG:boto:path=/lybtug-50/?prefix=b 2024-04-16T18:17:25.019 DEBUG:boto:auth_path=/lybtug-50/?prefix=b 2024-04-16T18:17:25.019 DEBUG:boto:Method: GET 2024-04-16T18:17:25.019 DEBUG:boto:Path: /lybtug-50/?prefix=b 2024-04-16T18:17:25.019 DEBUG:boto:Data: 2024-04-16T18:17:25.019 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.019 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.019 DEBUG:boto:Params: {} 2024-04-16T18:17:25.020 DEBUG:boto:Token: None 2024-04-16T18:17:25.020 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9d378196d988fab-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.025 DEBUG:boto:b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.025 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:17:25.025 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.025 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.025 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.025 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.025 DEBUG:boto:Method: GET 2024-04-16T18:17:25.025 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:25.026 DEBUG:boto:Data: 2024-04-16T18:17:25.026 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.026 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.026 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.026 DEBUG:boto:Params: {} 2024-04-16T18:17:25.026 DEBUG:boto:Token: None 2024-04-16T18:17:25.026 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:25.026 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T18:17:25.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T18:17:25.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f271b74245aff31-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.070 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.070 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.071 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.071 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.071 DEBUG:boto:Method: GET 2024-04-16T18:17:25.071 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:25.071 DEBUG:boto:Data: 2024-04-16T18:17:25.071 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.071 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.071 DEBUG:boto:Params: {} 2024-04-16T18:17:25.071 DEBUG:boto:Token: None 2024-04-16T18:17:25.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:25.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T18:17:25.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T18:17:25.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4851165a660adc8-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.075 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.075 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.075 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.075 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.076 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.076 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:25.076 DEBUG:boto:Data: 2024-04-16T18:17:25.076 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.076 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.076 DEBUG:boto:Params: {} 2024-04-16T18:17:25.076 DEBUG:boto:Token: None 2024-04-16T18:17:25.076 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:25.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE= 2024-04-16T18:17:25.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE=', 'Content-Length': '0'} 2024-04-16T18:17:25.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fc56d2515ebcd85-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.080 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.080 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.080 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.080 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.080 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.080 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:25.080 DEBUG:boto:Data: 2024-04-16T18:17:25.080 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.081 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.081 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.081 DEBUG:boto:Params: {} 2024-04-16T18:17:25.081 DEBUG:boto:Token: None 2024-04-16T18:17:25.081 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:25.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE= 2024-04-16T18:17:25.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE=', 'Content-Length': '0'} 2024-04-16T18:17:25.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000057a86c0c85fbc69-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:17:25.085 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.085 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.085 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.085 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.085 DEBUG:boto:Method: GET 2024-04-16T18:17:25.085 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:25.085 DEBUG:boto:Data: 2024-04-16T18:17:25.085 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.085 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.085 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.085 DEBUG:boto:Params: {} 2024-04-16T18:17:25.086 DEBUG:boto:Token: None 2024-04-16T18:17:25.086 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:25.086 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T18:17:25.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T18:17:25.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f31e2142b1d5014-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.130 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.130 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.130 DEBUG:boto:path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.131 DEBUG:boto:auth_path=/lybtug-50/c?versionId=null 2024-04-16T18:17:25.131 DEBUG:boto:Method: GET 2024-04-16T18:17:25.131 DEBUG:boto:Path: /lybtug-50/c?versionId=null 2024-04-16T18:17:25.131 DEBUG:boto:Data: 2024-04-16T18:17:25.131 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.131 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.131 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.131 DEBUG:boto:Params: {} 2024-04-16T18:17:25.131 DEBUG:boto:Token: None 2024-04-16T18:17:25.131 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c?versionId=null 2024-04-16T18:17:25.131 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T18:17:25.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T18:17:25.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d0e2f69085cbe64-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.135 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.135 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.135 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.135 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.136 DEBUG:boto:Method: GET 2024-04-16T18:17:25.136 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:25.136 DEBUG:boto:Data: 2024-04-16T18:17:25.136 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.136 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.136 DEBUG:boto:Params: {} 2024-04-16T18:17:25.136 DEBUG:boto:Token: None 2024-04-16T18:17:25.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ca3c7da969f4285-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.140 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.141 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.141 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.141 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.141 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.141 DEBUG:boto:Method: GET 2024-04-16T18:17:25.141 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:25.141 DEBUG:boto:Data: 2024-04-16T18:17:25.141 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.141 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.141 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.141 DEBUG:boto:Params: {} 2024-04-16T18:17:25.142 DEBUG:boto:Token: None 2024-04-16T18:17:25.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.142 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.146 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0f26b4892fcd554-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.190 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.191 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.191 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.191 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.191 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:25.191 DEBUG:boto:Data: 2024-04-16T18:17:25.191 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.191 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.191 DEBUG:boto:Params: {} 2024-04-16T18:17:25.191 DEBUG:boto:Token: None 2024-04-16T18:17:25.191 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c 2024-04-16T18:17:25.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8= 2024-04-16T18:17:25.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8=', 'Content-Length': '0'} 2024-04-16T18:17:25.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa876ee9b8eba0c1-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.195 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.195 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.195 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.195 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:25.195 DEBUG:boto:Data: 2024-04-16T18:17:25.195 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.196 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.196 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.196 DEBUG:boto:Params: {} 2024-04-16T18:17:25.196 DEBUG:boto:Token: None 2024-04-16T18:17:25.196 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c 2024-04-16T18:17:25.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8= 2024-04-16T18:17:25.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8=', 'Content-Length': '0'} 2024-04-16T18:17:25.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008233ac211406f5a2-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.199 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2024-04-16T18:17:25.199 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.200 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.200 DEBUG:boto:Method: GET 2024-04-16T18:17:25.200 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:25.200 DEBUG:boto:Data: 2024-04-16T18:17:25.200 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.200 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.200 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.200 DEBUG:boto:Params: {} 2024-04-16T18:17:25.200 DEBUG:boto:Token: None 2024-04-16T18:17:25.200 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c 2024-04-16T18:17:25.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo= 2024-04-16T18:17:25.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo=', 'Content-Length': '0'} 2024-04-16T18:17:25.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000304c37ae768718c7-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.246 DEBUG:boto:path=/lybtug-50/c 2024-04-16T18:17:25.246 DEBUG:boto:auth_path=/lybtug-50/c 2024-04-16T18:17:25.247 DEBUG:boto:Method: GET 2024-04-16T18:17:25.247 DEBUG:boto:Path: /lybtug-50/c 2024-04-16T18:17:25.247 DEBUG:boto:Data: 2024-04-16T18:17:25.247 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.247 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.247 DEBUG:boto:Params: {} 2024-04-16T18:17:25.247 DEBUG:boto:Token: None 2024-04-16T18:17:25.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/c 2024-04-16T18:17:25.247 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo= 2024-04-16T18:17:25.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo=', 'Content-Length': '0'} 2024-04-16T18:17:25.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077f43b9e271d1a9a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.250 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.251 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.251 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.251 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.251 DEBUG:boto:Method: GET 2024-04-16T18:17:25.251 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:25.251 DEBUG:boto:Data: 2024-04-16T18:17:25.251 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.251 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.251 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.251 DEBUG:boto:Params: {} 2024-04-16T18:17:25.251 DEBUG:boto:Token: None 2024-04-16T18:17:25.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be2056ea81d5ef7e-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.298 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.299 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.299 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.299 DEBUG:boto:path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.299 DEBUG:boto:auth_path=/lybtug-50/?prefix=c 2024-04-16T18:17:25.299 DEBUG:boto:Method: GET 2024-04-16T18:17:25.299 DEBUG:boto:Path: /lybtug-50/?prefix=c 2024-04-16T18:17:25.299 DEBUG:boto:Data: 2024-04-16T18:17:25.299 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.299 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.299 DEBUG:boto:Params: {} 2024-04-16T18:17:25.300 DEBUG:boto:Token: None 2024-04-16T18:17:25.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.304 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000979a62a2531c1185-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.304 DEBUG:boto:b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.305 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:17:25.305 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.305 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.305 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.305 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.305 DEBUG:boto:Method: GET 2024-04-16T18:17:25.305 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:25.305 DEBUG:boto:Data: 2024-04-16T18:17:25.305 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.305 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.306 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.306 DEBUG:boto:Params: {} 2024-04-16T18:17:25.306 DEBUG:boto:Token: None 2024-04-16T18:17:25.306 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:25.306 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T18:17:25.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T18:17:25.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069c3ee23558073d8-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.350 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.350 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.350 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.351 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.351 DEBUG:boto:Method: GET 2024-04-16T18:17:25.351 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:25.351 DEBUG:boto:Data: 2024-04-16T18:17:25.351 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.351 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.351 DEBUG:boto:Params: {} 2024-04-16T18:17:25.351 DEBUG:boto:Token: None 2024-04-16T18:17:25.351 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:25.351 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T18:17:25.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T18:17:25.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a44123dad02fe254-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.355 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.355 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.355 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.355 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.355 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.355 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:25.355 DEBUG:boto:Data: 2024-04-16T18:17:25.355 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.356 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.356 DEBUG:boto:Params: {} 2024-04-16T18:17:25.356 DEBUG:boto:Token: None 2024-04-16T18:17:25.356 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:25.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0= 2024-04-16T18:17:25.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0=', 'Content-Length': '0'} 2024-04-16T18:17:25.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb8bdd2ec89fca26-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.359 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.359 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.359 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.359 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.359 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.359 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:25.360 DEBUG:boto:Data: 2024-04-16T18:17:25.360 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.360 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.360 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.360 DEBUG:boto:Params: {} 2024-04-16T18:17:25.360 DEBUG:boto:Token: None 2024-04-16T18:17:25.360 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:25.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0= 2024-04-16T18:17:25.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0=', 'Content-Length': '0'} 2024-04-16T18:17:25.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095e35904aa47afbc-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.364 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:17:25.364 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.364 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.364 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.364 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.364 DEBUG:boto:Method: GET 2024-04-16T18:17:25.364 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:25.364 DEBUG:boto:Data: 2024-04-16T18:17:25.364 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.364 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.364 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.365 DEBUG:boto:Params: {} 2024-04-16T18:17:25.365 DEBUG:boto:Token: None 2024-04-16T18:17:25.365 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:25.365 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T18:17:25.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T18:17:25.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040ee4b293558fe0c-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.410 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.410 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.411 DEBUG:boto:path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.411 DEBUG:boto:auth_path=/lybtug-50/d?versionId=null 2024-04-16T18:17:25.411 DEBUG:boto:Method: GET 2024-04-16T18:17:25.411 DEBUG:boto:Path: /lybtug-50/d?versionId=null 2024-04-16T18:17:25.411 DEBUG:boto:Data: 2024-04-16T18:17:25.411 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.411 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.411 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.411 DEBUG:boto:Params: {} 2024-04-16T18:17:25.411 DEBUG:boto:Token: None 2024-04-16T18:17:25.411 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d?versionId=null 2024-04-16T18:17:25.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T18:17:25.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T18:17:25.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004570202c72e2252-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.415 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.415 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.415 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.415 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.415 DEBUG:boto:Method: GET 2024-04-16T18:17:25.415 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:25.415 DEBUG:boto:Data: 2024-04-16T18:17:25.415 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.416 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.416 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.416 DEBUG:boto:Params: {} 2024-04-16T18:17:25.416 DEBUG:boto:Token: None 2024-04-16T18:17:25.416 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.420 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bbc1a4ae5d4aad6-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.420 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.421 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.421 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.421 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.421 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.421 DEBUG:boto:Method: GET 2024-04-16T18:17:25.421 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:25.421 DEBUG:boto:Data: 2024-04-16T18:17:25.421 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.421 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.421 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.421 DEBUG:boto:Params: {} 2024-04-16T18:17:25.422 DEBUG:boto:Token: None 2024-04-16T18:17:25.422 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.422 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.426 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083e1de40b40792f6-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.466 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.467 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.467 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.467 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.467 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:25.467 DEBUG:boto:Data: 2024-04-16T18:17:25.467 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.467 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.467 DEBUG:boto:Params: {} 2024-04-16T18:17:25.468 DEBUG:boto:Token: None 2024-04-16T18:17:25.468 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d 2024-04-16T18:17:25.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A= 2024-04-16T18:17:25.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A=', 'Content-Length': '0'} 2024-04-16T18:17:25.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009523773fc2489d05-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.471 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.471 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.471 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.471 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:25.471 DEBUG:boto:Data: 2024-04-16T18:17:25.471 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.471 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.471 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.471 DEBUG:boto:Params: {} 2024-04-16T18:17:25.471 DEBUG:boto:Token: None 2024-04-16T18:17:25.472 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d 2024-04-16T18:17:25.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A= 2024-04-16T18:17:25.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A=', 'Content-Length': '0'} 2024-04-16T18:17:25.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030756fa89a163535-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2024-04-16T18:17:25.475 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.475 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.475 DEBUG:boto:Method: GET 2024-04-16T18:17:25.475 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:25.475 DEBUG:boto:Data: 2024-04-16T18:17:25.475 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.475 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.475 DEBUG:boto:Params: {} 2024-04-16T18:17:25.475 DEBUG:boto:Token: None 2024-04-16T18:17:25.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d 2024-04-16T18:17:25.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI= 2024-04-16T18:17:25.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI=', 'Content-Length': '0'} 2024-04-16T18:17:25.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026315591c90cdb07-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.522 DEBUG:boto:path=/lybtug-50/d 2024-04-16T18:17:25.522 DEBUG:boto:auth_path=/lybtug-50/d 2024-04-16T18:17:25.523 DEBUG:boto:Method: GET 2024-04-16T18:17:25.523 DEBUG:boto:Path: /lybtug-50/d 2024-04-16T18:17:25.523 DEBUG:boto:Data: 2024-04-16T18:17:25.523 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.523 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.523 DEBUG:boto:Params: {} 2024-04-16T18:17:25.523 DEBUG:boto:Token: None 2024-04-16T18:17:25.523 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/d 2024-04-16T18:17:25.523 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI= 2024-04-16T18:17:25.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI=', 'Content-Length': '0'} 2024-04-16T18:17:25.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1b898e2359a8a66-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.527 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.527 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.527 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.527 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.527 DEBUG:boto:Method: GET 2024-04-16T18:17:25.527 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:25.527 DEBUG:boto:Data: 2024-04-16T18:17:25.527 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.527 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.527 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.527 DEBUG:boto:Params: {} 2024-04-16T18:17:25.527 DEBUG:boto:Token: None 2024-04-16T18:17:25.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075b2e603916c087b-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.574 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.575 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.575 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.575 DEBUG:boto:path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.575 DEBUG:boto:auth_path=/lybtug-50/?prefix=d 2024-04-16T18:17:25.575 DEBUG:boto:Method: GET 2024-04-16T18:17:25.575 DEBUG:boto:Path: /lybtug-50/?prefix=d 2024-04-16T18:17:25.575 DEBUG:boto:Data: 2024-04-16T18:17:25.575 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.575 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.575 DEBUG:boto:Params: {} 2024-04-16T18:17:25.575 DEBUG:boto:Token: None 2024-04-16T18:17:25.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f3a6df08d27935a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.581 DEBUG:boto:b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.581 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:17:25.581 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.581 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.581 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.582 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.582 DEBUG:boto:Method: GET 2024-04-16T18:17:25.582 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:25.582 DEBUG:boto:Data: 2024-04-16T18:17:25.582 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.582 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.582 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.582 DEBUG:boto:Params: {} 2024-04-16T18:17:25.582 DEBUG:boto:Token: None 2024-04-16T18:17:25.582 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:25.582 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T18:17:25.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T18:17:25.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b787651555ea85b-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.626 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.626 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.626 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.627 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.627 DEBUG:boto:Method: GET 2024-04-16T18:17:25.627 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:25.627 DEBUG:boto:Data: 2024-04-16T18:17:25.627 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.627 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.627 DEBUG:boto:Params: {} 2024-04-16T18:17:25.627 DEBUG:boto:Token: None 2024-04-16T18:17:25.627 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:25.627 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T18:17:25.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T18:17:25.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba99448b5e63ff6a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.631 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.631 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.631 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.631 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.631 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.631 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:25.631 DEBUG:boto:Data: 2024-04-16T18:17:25.631 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.631 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.632 DEBUG:boto:Params: {} 2024-04-16T18:17:25.632 DEBUG:boto:Token: None 2024-04-16T18:17:25.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:25.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I= 2024-04-16T18:17:25.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I=', 'Content-Length': '0'} 2024-04-16T18:17:25.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fde0f8dd82651f13-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.635 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.635 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.635 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.635 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.635 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.635 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:25.635 DEBUG:boto:Data: 2024-04-16T18:17:25.636 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.636 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.636 DEBUG:boto:Params: {} 2024-04-16T18:17:25.636 DEBUG:boto:Token: None 2024-04-16T18:17:25.636 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:25.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I= 2024-04-16T18:17:25.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I=', 'Content-Length': '0'} 2024-04-16T18:17:25.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab3a0c37330a69b8-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.639 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:17:25.640 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.640 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.640 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.640 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.640 DEBUG:boto:Method: GET 2024-04-16T18:17:25.640 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:25.640 DEBUG:boto:Data: 2024-04-16T18:17:25.640 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.640 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.640 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.640 DEBUG:boto:Params: {} 2024-04-16T18:17:25.641 DEBUG:boto:Token: None 2024-04-16T18:17:25.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:25.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T18:17:25.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T18:17:25.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b57bec5ce1e6ce-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.686 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.686 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.687 DEBUG:boto:path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.687 DEBUG:boto:auth_path=/lybtug-50/e?versionId=null 2024-04-16T18:17:25.687 DEBUG:boto:Method: GET 2024-04-16T18:17:25.687 DEBUG:boto:Path: /lybtug-50/e?versionId=null 2024-04-16T18:17:25.687 DEBUG:boto:Data: 2024-04-16T18:17:25.687 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.687 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.687 DEBUG:boto:Params: {} 2024-04-16T18:17:25.687 DEBUG:boto:Token: None 2024-04-16T18:17:25.687 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e?versionId=null 2024-04-16T18:17:25.687 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T18:17:25.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T18:17:25.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b38fb2116fab32ee-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.691 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.691 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.691 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.691 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.691 DEBUG:boto:Method: GET 2024-04-16T18:17:25.691 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:25.691 DEBUG:boto:Data: 2024-04-16T18:17:25.691 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.691 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.691 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.692 DEBUG:boto:Params: {} 2024-04-16T18:17:25.692 DEBUG:boto:Token: None 2024-04-16T18:17:25.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.695 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d118acbc11996f8-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.695 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.696 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.696 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.696 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.696 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.696 DEBUG:boto:Method: GET 2024-04-16T18:17:25.696 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:25.696 DEBUG:boto:Data: 2024-04-16T18:17:25.696 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.696 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.696 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.697 DEBUG:boto:Params: {} 2024-04-16T18:17:25.697 DEBUG:boto:Token: None 2024-04-16T18:17:25.697 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.697 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014fa6998b46c2865-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.742 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.743 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.743 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.743 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.743 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:25.743 DEBUG:boto:Data: 2024-04-16T18:17:25.743 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.743 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.743 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.743 DEBUG:boto:Params: {} 2024-04-16T18:17:25.743 DEBUG:boto:Token: None 2024-04-16T18:17:25.744 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e 2024-04-16T18:17:25.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg= 2024-04-16T18:17:25.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg=', 'Content-Length': '0'} 2024-04-16T18:17:25.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072fff4d3b846ff7c-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.747 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.747 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.747 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.748 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:25.748 DEBUG:boto:Data: 2024-04-16T18:17:25.748 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.748 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.748 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.748 DEBUG:boto:Params: {} 2024-04-16T18:17:25.748 DEBUG:boto:Token: None 2024-04-16T18:17:25.748 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e 2024-04-16T18:17:25.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg= 2024-04-16T18:17:25.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg=', 'Content-Length': '0'} 2024-04-16T18:17:25.752 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000654242232d00b739-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2024-04-16T18:17:25.752 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.752 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.752 DEBUG:boto:Method: GET 2024-04-16T18:17:25.752 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:25.752 DEBUG:boto:Data: 2024-04-16T18:17:25.752 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.752 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.752 DEBUG:boto:Params: {} 2024-04-16T18:17:25.753 DEBUG:boto:Token: None 2024-04-16T18:17:25.753 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e 2024-04-16T18:17:25.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk= 2024-04-16T18:17:25.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk=', 'Content-Length': '0'} 2024-04-16T18:17:25.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fc49cf56b28ba5a-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.802 DEBUG:boto:path=/lybtug-50/e 2024-04-16T18:17:25.802 DEBUG:boto:auth_path=/lybtug-50/e 2024-04-16T18:17:25.803 DEBUG:boto:Method: GET 2024-04-16T18:17:25.803 DEBUG:boto:Path: /lybtug-50/e 2024-04-16T18:17:25.803 DEBUG:boto:Data: 2024-04-16T18:17:25.803 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.803 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.803 DEBUG:boto:Params: {} 2024-04-16T18:17:25.803 DEBUG:boto:Token: None 2024-04-16T18:17:25.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/e 2024-04-16T18:17:25.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk= 2024-04-16T18:17:25.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk=', 'Content-Length': '0'} 2024-04-16T18:17:25.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab6c17a25d59b97f-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.807 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.807 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.807 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.807 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.807 DEBUG:boto:Method: GET 2024-04-16T18:17:25.807 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:25.807 DEBUG:boto:Data: 2024-04-16T18:17:25.807 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.808 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.808 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.808 DEBUG:boto:Params: {} 2024-04-16T18:17:25.808 DEBUG:boto:Token: None 2024-04-16T18:17:25.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000008398197cdf0466-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.854 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.854 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.855 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.855 DEBUG:boto:path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.855 DEBUG:boto:auth_path=/lybtug-50/?prefix=e 2024-04-16T18:17:25.855 DEBUG:boto:Method: GET 2024-04-16T18:17:25.855 DEBUG:boto:Path: /lybtug-50/?prefix=e 2024-04-16T18:17:25.855 DEBUG:boto:Data: 2024-04-16T18:17:25.855 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.855 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.855 DEBUG:boto:Params: {} 2024-04-16T18:17:25.855 DEBUG:boto:Token: None 2024-04-16T18:17:25.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001875c40ad55ca6eb-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.860 DEBUG:boto:b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.860 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:17:25.860 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:25.860 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:25.861 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.861 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.861 DEBUG:boto:Method: GET 2024-04-16T18:17:25.861 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:25.861 DEBUG:boto:Data: 2024-04-16T18:17:25.861 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.861 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.861 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.861 DEBUG:boto:Params: {} 2024-04-16T18:17:25.861 DEBUG:boto:Token: None 2024-04-16T18:17:25.861 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:25.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T18:17:25.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T18:17:25.865 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb01df5223d4e46c-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.906 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:25.906 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:25.906 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.907 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.907 DEBUG:boto:Method: GET 2024-04-16T18:17:25.907 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:25.907 DEBUG:boto:Data: 2024-04-16T18:17:25.907 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.907 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.907 DEBUG:boto:Params: {} 2024-04-16T18:17:25.907 DEBUG:boto:Token: None 2024-04-16T18:17:25.907 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:25.907 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T18:17:25.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T18:17:25.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080cd6a32d8bad9c1-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.911 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:25.911 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:25.911 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.911 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.911 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.911 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:25.911 DEBUG:boto:Data: 2024-04-16T18:17:25.912 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.912 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.912 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.912 DEBUG:boto:Params: {} 2024-04-16T18:17:25.912 DEBUG:boto:Token: None 2024-04-16T18:17:25.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:25.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8= 2024-04-16T18:17:25.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8=', 'Content-Length': '0'} 2024-04-16T18:17:25.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc14391bd71537ea-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.915 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:25.915 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:25.915 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.915 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.915 DEBUG:boto:Method: HEAD 2024-04-16T18:17:25.916 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:25.916 DEBUG:boto:Data: 2024-04-16T18:17:25.916 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.916 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.916 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.916 DEBUG:boto:Params: {} 2024-04-16T18:17:25.916 DEBUG:boto:Token: None 2024-04-16T18:17:25.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:25.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8= 2024-04-16T18:17:25.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8=', 'Content-Length': '0'} 2024-04-16T18:17:25.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030f1738433e8a5a7-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:17:25.919 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:25.920 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:25.920 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.920 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.920 DEBUG:boto:Method: GET 2024-04-16T18:17:25.920 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:25.920 DEBUG:boto:Data: 2024-04-16T18:17:25.920 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.920 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.920 DEBUG:boto:Params: {} 2024-04-16T18:17:25.920 DEBUG:boto:Token: None 2024-04-16T18:17:25.920 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:25.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T18:17:25.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T18:17:25.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026680b0457b6e5d8-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.966 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:25.966 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:25.967 DEBUG:boto:path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.967 DEBUG:boto:auth_path=/lybtug-50/f?versionId=null 2024-04-16T18:17:25.967 DEBUG:boto:Method: GET 2024-04-16T18:17:25.967 DEBUG:boto:Path: /lybtug-50/f?versionId=null 2024-04-16T18:17:25.967 DEBUG:boto:Data: 2024-04-16T18:17:25.967 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.967 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.967 DEBUG:boto:Params: {} 2024-04-16T18:17:25.967 DEBUG:boto:Token: None 2024-04-16T18:17:25.967 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/f?versionId=null 2024-04-16T18:17:25.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T18:17:25.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T18:17:25.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021eb30cb27f4a1c4-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.971 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.971 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.971 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:25.971 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:25.971 DEBUG:boto:Method: GET 2024-04-16T18:17:25.971 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:25.971 DEBUG:boto:Data: 2024-04-16T18:17:25.971 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.971 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:25.972 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.972 DEBUG:boto:Params: {} 2024-04-16T18:17:25.972 DEBUG:boto:Token: None 2024-04-16T18:17:25.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b33c61bdb57b3ff6-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:25.976 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:25.976 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:25.976 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:25.977 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:25.977 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:25.977 DEBUG:boto:Method: GET 2024-04-16T18:17:25.977 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:25.977 DEBUG:boto:Data: 2024-04-16T18:17:25.977 DEBUG:boto:Headers: {} 2024-04-16T18:17:25.977 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:25.977 DEBUG:boto:Port: 8000 2024-04-16T18:17:25.977 DEBUG:boto:Params: {} 2024-04-16T18:17:25.977 DEBUG:boto:Token: None 2024-04-16T18:17:25.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:25 GMT /lybtug-50/ 2024-04-16T18:17:25.978 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T18:17:25.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T18:17:25.982 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2dec0afd100d99a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.026 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.027 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:26.027 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:26.027 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.027 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:26.027 DEBUG:boto:Data: 2024-04-16T18:17:26.027 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.027 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.027 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.027 DEBUG:boto:Params: {} 2024-04-16T18:17:26.027 DEBUG:boto:Token: None 2024-04-16T18:17:26.027 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/f 2024-04-16T18:17:26.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE= 2024-04-16T18:17:26.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE=', 'Content-Length': '0'} 2024-04-16T18:17:26.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a0904fa64285ebc-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.031 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:26.031 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:26.031 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.031 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:26.031 DEBUG:boto:Data: 2024-04-16T18:17:26.031 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.032 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.032 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.032 DEBUG:boto:Params: {} 2024-04-16T18:17:26.032 DEBUG:boto:Token: None 2024-04-16T18:17:26.032 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/f 2024-04-16T18:17:26.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE= 2024-04-16T18:17:26.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE=', 'Content-Length': '0'} 2024-04-16T18:17:26.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000874bfe9c7f44050f-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.035 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2024-04-16T18:17:26.035 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:26.035 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:26.036 DEBUG:boto:Method: GET 2024-04-16T18:17:26.036 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:26.036 DEBUG:boto:Data: 2024-04-16T18:17:26.036 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.036 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.036 DEBUG:boto:Params: {} 2024-04-16T18:17:26.036 DEBUG:boto:Token: None 2024-04-16T18:17:26.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/f 2024-04-16T18:17:26.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U= 2024-04-16T18:17:26.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U=', 'Content-Length': '0'} 2024-04-16T18:17:26.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd4b2457cb54ec81-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.082 DEBUG:boto:path=/lybtug-50/f 2024-04-16T18:17:26.082 DEBUG:boto:auth_path=/lybtug-50/f 2024-04-16T18:17:26.083 DEBUG:boto:Method: GET 2024-04-16T18:17:26.083 DEBUG:boto:Path: /lybtug-50/f 2024-04-16T18:17:26.083 DEBUG:boto:Data: 2024-04-16T18:17:26.083 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.083 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.083 DEBUG:boto:Params: {} 2024-04-16T18:17:26.083 DEBUG:boto:Token: None 2024-04-16T18:17:26.083 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/f 2024-04-16T18:17:26.083 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U= 2024-04-16T18:17:26.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U=', 'Content-Length': '0'} 2024-04-16T18:17:26.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000523c42661b66be6e-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.086 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.087 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.087 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:26.087 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:26.087 DEBUG:boto:Method: GET 2024-04-16T18:17:26.087 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:26.087 DEBUG:boto:Data: 2024-04-16T18:17:26.087 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.087 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.087 DEBUG:boto:Params: {} 2024-04-16T18:17:26.087 DEBUG:boto:Token: None 2024-04-16T18:17:26.088 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.091 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1ac1cd86e85e384-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.134 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.135 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.135 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.135 DEBUG:boto:path=/lybtug-50/?prefix=f 2024-04-16T18:17:26.135 DEBUG:boto:auth_path=/lybtug-50/?prefix=f 2024-04-16T18:17:26.135 DEBUG:boto:Method: GET 2024-04-16T18:17:26.135 DEBUG:boto:Path: /lybtug-50/?prefix=f 2024-04-16T18:17:26.135 DEBUG:boto:Data: 2024-04-16T18:17:26.135 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.135 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.136 DEBUG:boto:Params: {} 2024-04-16T18:17:26.136 DEBUG:boto:Token: None 2024-04-16T18:17:26.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.141 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f09985119b7c347c-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.141 DEBUG:boto:b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.141 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:17:26.141 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.141 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.141 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.141 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.142 DEBUG:boto:Method: GET 2024-04-16T18:17:26.142 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:26.142 DEBUG:boto:Data: 2024-04-16T18:17:26.142 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.142 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.142 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.142 DEBUG:boto:Params: {} 2024-04-16T18:17:26.142 DEBUG:boto:Token: None 2024-04-16T18:17:26.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:26.142 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T18:17:26.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T18:17:26.145 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d127041d6c8a5bc-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.186 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.186 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.187 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.187 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.189 DEBUG:boto:Method: GET 2024-04-16T18:17:26.189 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:26.189 DEBUG:boto:Data: 2024-04-16T18:17:26.189 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.189 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.190 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.190 DEBUG:boto:Params: {} 2024-04-16T18:17:26.190 DEBUG:boto:Token: None 2024-04-16T18:17:26.190 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:26.190 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T18:17:26.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T18:17:26.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1e02e17e26702b4-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.193 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.194 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.194 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.194 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.194 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.194 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:26.194 DEBUG:boto:Data: 2024-04-16T18:17:26.194 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.194 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.194 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.194 DEBUG:boto:Params: {} 2024-04-16T18:17:26.194 DEBUG:boto:Token: None 2024-04-16T18:17:26.194 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:26.195 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk= 2024-04-16T18:17:26.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk=', 'Content-Length': '0'} 2024-04-16T18:17:26.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f5034db783df87e-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.198 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.198 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.198 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.198 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.198 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.198 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:26.198 DEBUG:boto:Data: 2024-04-16T18:17:26.198 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.198 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.199 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.199 DEBUG:boto:Params: {} 2024-04-16T18:17:26.199 DEBUG:boto:Token: None 2024-04-16T18:17:26.199 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:26.199 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk= 2024-04-16T18:17:26.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk=', 'Content-Length': '0'} 2024-04-16T18:17:26.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a54c2b74c243e3ea-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.202 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:17:26.202 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.203 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.203 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.203 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.203 DEBUG:boto:Method: GET 2024-04-16T18:17:26.203 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:26.203 DEBUG:boto:Data: 2024-04-16T18:17:26.203 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.204 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.204 DEBUG:boto:Params: {} 2024-04-16T18:17:26.204 DEBUG:boto:Token: None 2024-04-16T18:17:26.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:26.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T18:17:26.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T18:17:26.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b35cffa06fbbbe2-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.250 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.251 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.251 DEBUG:boto:path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.251 DEBUG:boto:auth_path=/lybtug-50/g?versionId=null 2024-04-16T18:17:26.251 DEBUG:boto:Method: GET 2024-04-16T18:17:26.251 DEBUG:boto:Path: /lybtug-50/g?versionId=null 2024-04-16T18:17:26.251 DEBUG:boto:Data: 2024-04-16T18:17:26.251 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.251 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.251 DEBUG:boto:Params: {} 2024-04-16T18:17:26.251 DEBUG:boto:Token: None 2024-04-16T18:17:26.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g?versionId=null 2024-04-16T18:17:26.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T18:17:26.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T18:17:26.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a4b78bc389bc5cc-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.255 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.255 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.255 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.255 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.255 DEBUG:boto:Method: GET 2024-04-16T18:17:26.256 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:26.256 DEBUG:boto:Data: 2024-04-16T18:17:26.256 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.256 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.256 DEBUG:boto:Params: {} 2024-04-16T18:17:26.256 DEBUG:boto:Token: None 2024-04-16T18:17:26.256 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037f8ea3b0dd0a2c3-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.302 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.303 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.303 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.303 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.303 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.303 DEBUG:boto:Method: GET 2024-04-16T18:17:26.303 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:26.303 DEBUG:boto:Data: 2024-04-16T18:17:26.303 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.303 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.304 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.304 DEBUG:boto:Params: {} 2024-04-16T18:17:26.304 DEBUG:boto:Token: None 2024-04-16T18:17:26.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057f8d4cd42d40cbc-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.309 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.309 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.309 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.309 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.309 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:26.309 DEBUG:boto:Data: 2024-04-16T18:17:26.309 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.309 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.309 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.310 DEBUG:boto:Params: {} 2024-04-16T18:17:26.310 DEBUG:boto:Token: None 2024-04-16T18:17:26.310 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g 2024-04-16T18:17:26.310 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M= 2024-04-16T18:17:26.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M=', 'Content-Length': '0'} 2024-04-16T18:17:26.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000252ef69447c1ac37-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.313 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.313 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.313 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.313 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:26.313 DEBUG:boto:Data: 2024-04-16T18:17:26.313 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.314 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.314 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.314 DEBUG:boto:Params: {} 2024-04-16T18:17:26.314 DEBUG:boto:Token: None 2024-04-16T18:17:26.314 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g 2024-04-16T18:17:26.314 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M= 2024-04-16T18:17:26.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M=', 'Content-Length': '0'} 2024-04-16T18:17:26.317 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fae2ddab1d06a98e-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.317 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2024-04-16T18:17:26.317 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.317 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.318 DEBUG:boto:Method: GET 2024-04-16T18:17:26.318 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:26.318 DEBUG:boto:Data: 2024-04-16T18:17:26.318 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.318 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.318 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.318 DEBUG:boto:Params: {} 2024-04-16T18:17:26.318 DEBUG:boto:Token: None 2024-04-16T18:17:26.318 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g 2024-04-16T18:17:26.318 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw= 2024-04-16T18:17:26.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw=', 'Content-Length': '0'} 2024-04-16T18:17:26.321 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d1b7b645a1874d5-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.362 DEBUG:boto:path=/lybtug-50/g 2024-04-16T18:17:26.362 DEBUG:boto:auth_path=/lybtug-50/g 2024-04-16T18:17:26.362 DEBUG:boto:Method: GET 2024-04-16T18:17:26.363 DEBUG:boto:Path: /lybtug-50/g 2024-04-16T18:17:26.363 DEBUG:boto:Data: 2024-04-16T18:17:26.363 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.363 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.363 DEBUG:boto:Params: {} 2024-04-16T18:17:26.363 DEBUG:boto:Token: None 2024-04-16T18:17:26.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/g 2024-04-16T18:17:26.363 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw= 2024-04-16T18:17:26.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw=', 'Content-Length': '0'} 2024-04-16T18:17:26.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001416c27874183696-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.366 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.366 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.367 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.367 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.367 DEBUG:boto:Method: GET 2024-04-16T18:17:26.367 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:26.367 DEBUG:boto:Data: 2024-04-16T18:17:26.367 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.367 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.367 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.367 DEBUG:boto:Params: {} 2024-04-16T18:17:26.367 DEBUG:boto:Token: None 2024-04-16T18:17:26.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.372 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d33986b67e93bd67-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.372 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.372 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.372 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.372 DEBUG:boto:path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.372 DEBUG:boto:auth_path=/lybtug-50/?prefix=g 2024-04-16T18:17:26.372 DEBUG:boto:Method: GET 2024-04-16T18:17:26.372 DEBUG:boto:Path: /lybtug-50/?prefix=g 2024-04-16T18:17:26.373 DEBUG:boto:Data: 2024-04-16T18:17:26.373 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.373 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.373 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.373 DEBUG:boto:Params: {} 2024-04-16T18:17:26.373 DEBUG:boto:Token: None 2024-04-16T18:17:26.373 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.373 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.378 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b94ff15a799fdeee-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.418 DEBUG:boto:b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.419 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:17:26.419 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.419 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.419 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.419 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.419 DEBUG:boto:Method: GET 2024-04-16T18:17:26.419 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:26.419 DEBUG:boto:Data: 2024-04-16T18:17:26.419 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.420 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.420 DEBUG:boto:Params: {} 2024-04-16T18:17:26.420 DEBUG:boto:Token: None 2024-04-16T18:17:26.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:26.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T18:17:26.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T18:17:26.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033f59013adb658ba-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.466 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.466 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.467 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.467 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.467 DEBUG:boto:Method: GET 2024-04-16T18:17:26.467 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:26.467 DEBUG:boto:Data: 2024-04-16T18:17:26.467 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.467 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.467 DEBUG:boto:Params: {} 2024-04-16T18:17:26.467 DEBUG:boto:Token: None 2024-04-16T18:17:26.467 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:26.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T18:17:26.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T18:17:26.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e39e41e4c77ab529-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.471 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.471 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.471 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.471 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.471 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.471 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:26.471 DEBUG:boto:Data: 2024-04-16T18:17:26.471 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.471 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.472 DEBUG:boto:Params: {} 2024-04-16T18:17:26.472 DEBUG:boto:Token: None 2024-04-16T18:17:26.472 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:26.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E= 2024-04-16T18:17:26.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E=', 'Content-Length': '0'} 2024-04-16T18:17:26.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000505518a94439367e-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.475 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.475 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.475 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.476 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.476 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.476 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:26.476 DEBUG:boto:Data: 2024-04-16T18:17:26.476 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.476 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.476 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.476 DEBUG:boto:Params: {} 2024-04-16T18:17:26.476 DEBUG:boto:Token: None 2024-04-16T18:17:26.476 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:26.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E= 2024-04-16T18:17:26.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E=', 'Content-Length': '0'} 2024-04-16T18:17:26.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1ffb9aecff403f3-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:17:26.479 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.479 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.480 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.480 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.480 DEBUG:boto:Method: GET 2024-04-16T18:17:26.480 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:26.480 DEBUG:boto:Data: 2024-04-16T18:17:26.480 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.480 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.480 DEBUG:boto:Params: {} 2024-04-16T18:17:26.480 DEBUG:boto:Token: None 2024-04-16T18:17:26.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:26.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T18:17:26.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T18:17:26.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d31c4b08809de2c-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.526 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.526 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.527 DEBUG:boto:path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.527 DEBUG:boto:auth_path=/lybtug-50/h?versionId=null 2024-04-16T18:17:26.527 DEBUG:boto:Method: GET 2024-04-16T18:17:26.527 DEBUG:boto:Path: /lybtug-50/h?versionId=null 2024-04-16T18:17:26.527 DEBUG:boto:Data: 2024-04-16T18:17:26.527 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.527 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.527 DEBUG:boto:Params: {} 2024-04-16T18:17:26.527 DEBUG:boto:Token: None 2024-04-16T18:17:26.527 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h?versionId=null 2024-04-16T18:17:26.527 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T18:17:26.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T18:17:26.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca677ea7f7bdf4aa-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.531 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.531 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.531 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.531 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.531 DEBUG:boto:Method: GET 2024-04-16T18:17:26.531 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:26.531 DEBUG:boto:Data: 2024-04-16T18:17:26.532 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.532 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.532 DEBUG:boto:Params: {} 2024-04-16T18:17:26.532 DEBUG:boto:Token: None 2024-04-16T18:17:26.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045be7cfea307ab33-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.536 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.536 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.537 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.537 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.537 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.537 DEBUG:boto:Method: GET 2024-04-16T18:17:26.537 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:26.537 DEBUG:boto:Data: 2024-04-16T18:17:26.537 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.537 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.537 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.537 DEBUG:boto:Params: {} 2024-04-16T18:17:26.537 DEBUG:boto:Token: None 2024-04-16T18:17:26.537 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.538 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac4ff7ed3afeaf71-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.542 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.542 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.542 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.542 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.543 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:26.543 DEBUG:boto:Data: 2024-04-16T18:17:26.543 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.543 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.543 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.543 DEBUG:boto:Params: {} 2024-04-16T18:17:26.543 DEBUG:boto:Token: None 2024-04-16T18:17:26.543 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h 2024-04-16T18:17:26.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk= 2024-04-16T18:17:26.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk=', 'Content-Length': '0'} 2024-04-16T18:17:26.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063cf2fce5f5bea2a-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.546 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.546 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.547 DEBUG:boto:Method: HEAD 2024-04-16T18:17:26.547 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:26.547 DEBUG:boto:Data: 2024-04-16T18:17:26.547 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.547 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.547 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.547 DEBUG:boto:Params: {} 2024-04-16T18:17:26.547 DEBUG:boto:Token: None 2024-04-16T18:17:26.547 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h 2024-04-16T18:17:26.547 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk= 2024-04-16T18:17:26.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk=', 'Content-Length': '0'} 2024-04-16T18:17:26.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abb3849dfec3ecfb-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.550 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2024-04-16T18:17:26.551 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.551 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.551 DEBUG:boto:Method: GET 2024-04-16T18:17:26.551 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:26.551 DEBUG:boto:Data: 2024-04-16T18:17:26.551 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.551 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.551 DEBUG:boto:Params: {} 2024-04-16T18:17:26.551 DEBUG:boto:Token: None 2024-04-16T18:17:26.551 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h 2024-04-16T18:17:26.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8= 2024-04-16T18:17:26.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8=', 'Content-Length': '0'} 2024-04-16T18:17:26.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b64a7540cba10ae-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.599 DEBUG:boto:path=/lybtug-50/h 2024-04-16T18:17:26.599 DEBUG:boto:auth_path=/lybtug-50/h 2024-04-16T18:17:26.599 DEBUG:boto:Method: GET 2024-04-16T18:17:26.599 DEBUG:boto:Path: /lybtug-50/h 2024-04-16T18:17:26.599 DEBUG:boto:Data: 2024-04-16T18:17:26.599 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.599 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.599 DEBUG:boto:Params: {} 2024-04-16T18:17:26.599 DEBUG:boto:Token: None 2024-04-16T18:17:26.600 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/h 2024-04-16T18:17:26.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8= 2024-04-16T18:17:26.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8=', 'Content-Length': '0'} 2024-04-16T18:17:26.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005394276f7902d10e-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.603 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.603 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.603 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.603 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.603 DEBUG:boto:Method: GET 2024-04-16T18:17:26.603 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:26.603 DEBUG:boto:Data: 2024-04-16T18:17:26.604 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:26.604 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.604 DEBUG:boto:Params: {} 2024-04-16T18:17:26.604 DEBUG:boto:Token: None 2024-04-16T18:17:26.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d31771ca0404ab67-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.650 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.651 DEBUG:boto:path=/lybtug-50/ 2024-04-16T18:17:26.651 DEBUG:boto:auth_path=/lybtug-50/ 2024-04-16T18:17:26.651 DEBUG:boto:path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.651 DEBUG:boto:auth_path=/lybtug-50/?prefix=h 2024-04-16T18:17:26.651 DEBUG:boto:Method: GET 2024-04-16T18:17:26.651 DEBUG:boto:Path: /lybtug-50/?prefix=h 2024-04-16T18:17:26.651 DEBUG:boto:Data: 2024-04-16T18:17:26.651 DEBUG:boto:Headers: {} 2024-04-16T18:17:26.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:26.651 DEBUG:boto:Port: 8000 2024-04-16T18:17:26.651 DEBUG:boto:Params: {} 2024-04-16T18:17:26.651 DEBUG:boto:Token: None 2024-04-16T18:17:26.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:26 GMT /lybtug-50/ 2024-04-16T18:17:26.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T18:17:26.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T18:17:26.656 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084d5a1d7b7946296-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:26.656 DEBUG:boto:b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:26.657 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T18:17:26.657 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:26.657 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:26.657 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:26.693 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:27.048 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.053+0000 7fbdc43fc640 -1 req 18081045207925834297 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-43:054102a3-8d94-4596-8b3b-6341eab312e1.4804.12,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-43:054102a3-8d94-4596-8b3b-6341eab312e1.4804.12,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.061 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.065+0000 7fbdccc0d640 -1 req 14660751659862165158 0.003999925s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-25:054102a3-8d94-4596-8b3b-6341eab312e1.4353.29,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-25:054102a3-8d94-4596-8b3b-6341eab312e1.4353.29,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.144 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.149+0000 7fbdeb44a640 -1 req 26479994857397418 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-34:054102a3-8d94-4596-8b3b-6341eab312e1.4804.3,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-34:054102a3-8d94-4596-8b3b-6341eab312e1.4804.3,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.144 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.149+0000 7fbde1c37640 -1 req 4080226723044830306 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-24:054102a3-8d94-4596-8b3b-6341eab312e1.4353.28,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-24:054102a3-8d94-4596-8b3b-6341eab312e1.4353.28,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.145 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.149+0000 7fbdf6460640 -1 req 7259517271128369061 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.166 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.173+0000 7fbe07c83640 -1 req 13089633187912110970 0.007999849s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-21:054102a3-8d94-4596-8b3b-6341eab312e1.4353.25,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-21:054102a3-8d94-4596-8b3b-6341eab312e1.4353.25,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.196 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.201+0000 7fbe0e490640 -1 req 1862940798833270506 0.007999849s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-33:054102a3-8d94-4596-8b3b-6341eab312e1.4804.2,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-33:054102a3-8d94-4596-8b3b-6341eab312e1.4804.2,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.305 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:27.305 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:27.305 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:27.488 INFO:teuthology.orchestra.run.smithi071.stdout:{"layout":{"resharding":"None","current_index":{"gen":2,"layout":{"type":"Normal","normal":{"num_shards":15,"hash_type":"Mod"}}},"logs":[{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}},{"gen":2,"layout":{"type":"InIndex","in_index":{"gen":2,"layout":{"num_shards":15,"hash_type":"Mod"}}}}]}} 2024-04-16T18:17:27.488 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:27.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:27.488 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:27.560 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:27.560 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:27.854 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.857+0000 7fbe22cb9640 -1 req 5988557027030468213 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.855 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.861+0000 7fbe454fe640 -1 req 6899803751102318845 0.007999849s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-26:054102a3-8d94-4596-8b3b-6341eab312e1.4353.30,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-26:054102a3-8d94-4596-8b3b-6341eab312e1.4353.30,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.869 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.873+0000 7fbe68544640 -1 req 572665038004721884 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-32:054102a3-8d94-4596-8b3b-6341eab312e1.4804.1,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-32:054102a3-8d94-4596-8b3b-6341eab312e1.4804.1,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.920 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.925+0000 7fbd7e370640 -1 req 15925822944464962007 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-35:054102a3-8d94-4596-8b3b-6341eab312e1.4804.4,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-35:054102a3-8d94-4596-8b3b-6341eab312e1.4804.4,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:27.980 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:27.985+0000 7fbd8337a640 -1 req 14826523632275738640 0.003999925s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-23:054102a3-8d94-4596-8b3b-6341eab312e1.4353.27,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-23:054102a3-8d94-4596-8b3b-6341eab312e1.4353.27,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T18:17:28.086 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:28.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:28.086 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:28.272 INFO:teuthology.orchestra.run.smithi071.stdout:{"layout":{"resharding":"None","current_index":{"gen":2,"layout":{"type":"Normal","normal":{"num_shards":15,"hash_type":"Mod"}}},"logs":[{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}},{"gen":2,"layout":{"type":"InIndex","in_index":{"gen":2,"layout":{"num_shards":15,"hash_type":"Mod"}}}}]}} 2024-04-16T18:17:33.277 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_reshard_index_log_trim ... FAIL 2024-04-16T18:17:33.282 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.282 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.282 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.282 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.282 DEBUG:boto:Data: 2024-04-16T18:17:33.282 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.282 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.283 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.283 DEBUG:boto:Params: {} 2024-04-16T18:17:33.283 DEBUG:boto:Token: None 2024-04-16T18:17:33.283 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h3xuMGSJt2cjhvftWosQs8XRHz4= 2024-04-16T18:17:33.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h3xuMGSJt2cjhvftWosQs8XRHz4=', 'Content-Length': '0'} 2024-04-16T18:17:33.321 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fd2cee012ccb66c3-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.321 DEBUG:rgw_multi.tests:created bucket=lybtug-51 2024-04-16T18:17:33.321 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:33.321 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:33.322 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:33.437 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:17:33.471 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291453.320908_569.1","last_update":"2024-04-16T18:17:33.320908Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:17:33.471 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:17:33.471 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:17:33.471 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:33.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:33.472 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:33.518 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:33.518 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:33.593+0000 7fa25c729980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:17:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:33.593+0000 7fa25c729980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:17:33.621 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:17:33Z"} 2024-04-16T18:17:33.621 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:17:33Z"} 2024-04-16T18:17:33.621 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:17:33.622 DEBUG:rgw_multi.tests:log_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:17:33.622 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:17:33.622 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:17:33.622 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:33.622 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:33.622 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:33.667 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:33.667 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:33.767 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000001","obj00000000000000000005","obj00000000000000000012","obj00000000000000000004","obj00000000000000000011","obj00000000000000000000","obj00000000000000000010","obj00000000000000000003","obj00000000000000000013","obj00000000000000000002","obj00000000000000000006"]} 2024-04-16T18:17:33.768 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.768 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.768 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.768 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.768 DEBUG:boto:Data: 2024-04-16T18:17:33.768 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.769 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.769 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.769 DEBUG:boto:Params: {} 2024-04-16T18:17:33.769 DEBUG:boto:Token: None 2024-04-16T18:17:33.769 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.769 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.772 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a13151945503034f-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.773 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:33.773 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:33.773 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.773 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:33.773 DEBUG:boto:Data: 2024-04-16T18:17:33.773 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.773 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.773 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.773 DEBUG:boto:Params: {} 2024-04-16T18:17:33.773 DEBUG:boto:Token: None 2024-04-16T18:17:33.774 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:33.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G4EqN89QDFUJkv/Gp9O25HVKLW0= 2024-04-16T18:17:33.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G4EqN89QDFUJkv/Gp9O25HVKLW0='} 2024-04-16T18:17:33.818 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000650326694f928534-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.819 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.819 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.819 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.819 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.819 DEBUG:boto:Data: 2024-04-16T18:17:33.819 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.819 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.819 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.819 DEBUG:boto:Params: {} 2024-04-16T18:17:33.819 DEBUG:boto:Token: None 2024-04-16T18:17:33.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.823 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007a8c617142730541-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.823 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:33.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:33.823 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.823 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:33.824 DEBUG:boto:Data: 2024-04-16T18:17:33.824 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.824 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.824 DEBUG:boto:Params: {} 2024-04-16T18:17:33.824 DEBUG:boto:Token: None 2024-04-16T18:17:33.824 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:33.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UAo2xJa81qja2sO7R3DQZs6QVsg= 2024-04-16T18:17:33.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UAo2xJa81qja2sO7R3DQZs6QVsg='} 2024-04-16T18:17:33.836 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000abee6eb535955cdd-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.836 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.836 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.836 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.836 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.836 DEBUG:boto:Data: 2024-04-16T18:17:33.836 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.837 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.837 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.837 DEBUG:boto:Params: {} 2024-04-16T18:17:33.837 DEBUG:boto:Token: None 2024-04-16T18:17:33.837 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.840 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007f1f1982290cf1dc-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.840 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:33.841 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:33.841 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.841 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:33.841 DEBUG:boto:Data: 2024-04-16T18:17:33.841 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.841 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.841 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.841 DEBUG:boto:Params: {} 2024-04-16T18:17:33.841 DEBUG:boto:Token: None 2024-04-16T18:17:33.842 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:33.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r/Xu7Ks9XHSVCj1s0VyXMgrsqKs= 2024-04-16T18:17:33.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r/Xu7Ks9XHSVCj1s0VyXMgrsqKs='} 2024-04-16T18:17:33.886 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d0699891479bf4d3-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.887 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.887 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.887 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.887 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.887 DEBUG:boto:Data: 2024-04-16T18:17:33.887 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.887 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.887 DEBUG:boto:Params: {} 2024-04-16T18:17:33.887 DEBUG:boto:Token: None 2024-04-16T18:17:33.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.891 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a639c9189e9e6246-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.891 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:33.891 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:33.891 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.891 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:33.892 DEBUG:boto:Data: 2024-04-16T18:17:33.892 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.892 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.892 DEBUG:boto:Params: {} 2024-04-16T18:17:33.892 DEBUG:boto:Token: None 2024-04-16T18:17:33.892 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:33.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bZhTzcxx10lnEObm7kjZtyBEW2g= 2024-04-16T18:17:33.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bZhTzcxx10lnEObm7kjZtyBEW2g='} 2024-04-16T18:17:33.904 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a33c66f818296050-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.904 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.905 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.905 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.905 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.905 DEBUG:boto:Data: 2024-04-16T18:17:33.905 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.905 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.905 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.905 DEBUG:boto:Params: {} 2024-04-16T18:17:33.905 DEBUG:boto:Token: None 2024-04-16T18:17:33.905 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.905 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.909 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dd215836fbcb1835-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.910 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:33.910 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:33.910 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.910 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:33.910 DEBUG:boto:Data: 2024-04-16T18:17:33.910 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.910 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.910 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.910 DEBUG:boto:Params: {} 2024-04-16T18:17:33.910 DEBUG:boto:Token: None 2024-04-16T18:17:33.911 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:33.911 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rjq6z5+bP7ruNacMtEZYikt8o50= 2024-04-16T18:17:33.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rjq6z5+bP7ruNacMtEZYikt8o50='} 2024-04-16T18:17:33.954 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004a0f8a9cdc6dbaf2-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.955 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.955 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.955 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.955 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.955 DEBUG:boto:Data: 2024-04-16T18:17:33.955 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.955 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.955 DEBUG:boto:Params: {} 2024-04-16T18:17:33.955 DEBUG:boto:Token: None 2024-04-16T18:17:33.956 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.959 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eeea9f809f74a3d8-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.959 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:33.960 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:33.960 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.960 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:33.960 DEBUG:boto:Data: 2024-04-16T18:17:33.960 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.960 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.960 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.960 DEBUG:boto:Params: {} 2024-04-16T18:17:33.960 DEBUG:boto:Token: None 2024-04-16T18:17:33.961 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:33.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GXsNbtNEU8dOISnFl24/7aw0fLM= 2024-04-16T18:17:33.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GXsNbtNEU8dOISnFl24/7aw0fLM='} 2024-04-16T18:17:33.973 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d44c0d4448c94d8d-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.973 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:33.973 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:33.973 DEBUG:boto:Method: HEAD 2024-04-16T18:17:33.974 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:33.974 DEBUG:boto:Data: 2024-04-16T18:17:33.974 DEBUG:boto:Headers: {} 2024-04-16T18:17:33.974 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.974 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.974 DEBUG:boto:Params: {} 2024-04-16T18:17:33.974 DEBUG:boto:Token: None 2024-04-16T18:17:33.974 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/ 2024-04-16T18:17:33.974 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk= 2024-04-16T18:17:33.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3HwQvckH/huZ2Z0RqXR7D6k6ANk=', 'Content-Length': '0'} 2024-04-16T18:17:33.978 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000032d50882b4f0364e-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:33.978 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:33.978 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:33.978 DEBUG:boto:Method: PUT 2024-04-16T18:17:33.978 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:33.978 DEBUG:boto:Data: 2024-04-16T18:17:33.978 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:33.979 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:33.979 DEBUG:boto:Port: 8000 2024-04-16T18:17:33.979 DEBUG:boto:Params: {} 2024-04-16T18:17:33.979 DEBUG:boto:Token: None 2024-04-16T18:17:33.979 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:33 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:33.979 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U6g7RTQpsPklNj7kIJBazj1QjDg= 2024-04-16T18:17:33.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:33 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U6g7RTQpsPklNj7kIJBazj1QjDg='} 2024-04-16T18:17:34.022 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000c636f4076ac7880-00661ec0bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.023 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:34.023 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:34.023 DEBUG:boto:Method: HEAD 2024-04-16T18:17:34.023 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:34.023 DEBUG:boto:Data: 2024-04-16T18:17:34.023 DEBUG:boto:Headers: {} 2024-04-16T18:17:34.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:34.023 DEBUG:boto:Port: 8000 2024-04-16T18:17:34.023 DEBUG:boto:Params: {} 2024-04-16T18:17:34.024 DEBUG:boto:Token: None 2024-04-16T18:17:34.024 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:34 GMT /lybtug-51/ 2024-04-16T18:17:34.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qJ2HyBxGuoT7vmmU6sTXiKB/evY= 2024-04-16T18:17:34.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qJ2HyBxGuoT7vmmU6sTXiKB/evY=', 'Content-Length': '0'} 2024-04-16T18:17:34.027 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007b52df138e1d3ce0-00661ec0be-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.028 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:34.028 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:34.028 DEBUG:boto:Method: PUT 2024-04-16T18:17:34.028 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:34.028 DEBUG:boto:Data: 2024-04-16T18:17:34.028 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:34.028 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:34.028 DEBUG:boto:Port: 8000 2024-04-16T18:17:34.028 DEBUG:boto:Params: {} 2024-04-16T18:17:34.029 DEBUG:boto:Token: None 2024-04-16T18:17:34.029 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:34 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:34.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m0pf9VvaNqUzH4SKKoumhfp/iQk= 2024-04-16T18:17:34.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m0pf9VvaNqUzH4SKKoumhfp/iQk='} 2024-04-16T18:17:34.041 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fc3185c23dc7068b-00661ec0be-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.041 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:34.041 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:34.041 DEBUG:boto:Method: HEAD 2024-04-16T18:17:34.041 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:34.041 DEBUG:boto:Data: 2024-04-16T18:17:34.041 DEBUG:boto:Headers: {} 2024-04-16T18:17:34.041 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:34.042 DEBUG:boto:Port: 8000 2024-04-16T18:17:34.042 DEBUG:boto:Params: {} 2024-04-16T18:17:34.042 DEBUG:boto:Token: None 2024-04-16T18:17:34.042 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:34 GMT /lybtug-51/ 2024-04-16T18:17:34.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qJ2HyBxGuoT7vmmU6sTXiKB/evY= 2024-04-16T18:17:34.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qJ2HyBxGuoT7vmmU6sTXiKB/evY=', 'Content-Length': '0'} 2024-04-16T18:17:34.046 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000560b23a8927bb93-00661ec0be-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.046 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:34.046 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:34.046 DEBUG:boto:Method: PUT 2024-04-16T18:17:34.046 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:34.046 DEBUG:boto:Data: 2024-04-16T18:17:34.046 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:34.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:34.047 DEBUG:boto:Port: 8000 2024-04-16T18:17:34.047 DEBUG:boto:Params: {} 2024-04-16T18:17:34.047 DEBUG:boto:Token: None 2024-04-16T18:17:34.047 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:34 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:34.047 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O4vmEd7OIFwNqO4hy8ZJXWeFOK4= 2024-04-16T18:17:34.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O4vmEd7OIFwNqO4hy8ZJXWeFOK4='} 2024-04-16T18:17:34.090 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000026ece2d2788bf392-00661ec0be-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.091 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:34.091 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:34.091 DEBUG:boto:Method: HEAD 2024-04-16T18:17:34.091 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:34.091 DEBUG:boto:Data: 2024-04-16T18:17:34.091 DEBUG:boto:Headers: {} 2024-04-16T18:17:34.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:34.091 DEBUG:boto:Port: 8000 2024-04-16T18:17:34.091 DEBUG:boto:Params: {} 2024-04-16T18:17:34.091 DEBUG:boto:Token: None 2024-04-16T18:17:34.092 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:34 GMT /lybtug-51/ 2024-04-16T18:17:34.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qJ2HyBxGuoT7vmmU6sTXiKB/evY= 2024-04-16T18:17:34.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qJ2HyBxGuoT7vmmU6sTXiKB/evY=', 'Content-Length': '0'} 2024-04-16T18:17:34.095 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '27'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002959355d9a18f33c-00661ec0be-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.095 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:34.095 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:34.095 DEBUG:boto:Method: PUT 2024-04-16T18:17:34.096 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:34.096 DEBUG:boto:Data: 2024-04-16T18:17:34.096 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:34.096 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:34.096 DEBUG:boto:Port: 8000 2024-04-16T18:17:34.096 DEBUG:boto:Params: {} 2024-04-16T18:17:34.096 DEBUG:boto:Token: None 2024-04-16T18:17:34.096 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:34 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:34.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zawdymm93Q5nPvf9zU/LguwiJXY= 2024-04-16T18:17:34.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:34 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zawdymm93Q5nPvf9zU/LguwiJXY='} 2024-04-16T18:17:34.109 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002102f1170c391ace-00661ec0be-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:34.109 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-51 2024-04-16T18:17:34.109 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:34.110 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:34.110 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-51 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:17:34.158 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:34.158 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:34.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:34.241+0000 7f8a18653980 1 waiting to reach incremental sync.. 2024-04-16T18:17:39.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:39.245+0000 7f8a18653980 1 bucket sync caught up with source: 2024-04-16T18:17:39.241 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.121.6, 00000000001.54.6, 00000000001.39.6, , 00000000001.28.6, 00000000001.53.6, 00000000001.48.6, 00000000001.71.6, 00000000001.38.6, 00000000001.64.6, 00000000001.30.6] 2024-04-16T18:17:39.241 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.121.6, 00000000001.54.6, 00000000001.39.6, , 00000000001.28.6, 00000000001.53.6, 00000000001.48.6, 00000000001.71.6, 00000000001.38.6, 00000000001.64.6, 00000000001.30.6] 2024-04-16T18:17:39.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:17:39.245+0000 7f8a18653980 0 bucket checkpoint complete 2024-04-16T18:17:39.292 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-51 2024-04-16T18:17:39.292 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:39.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:17:39.293 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-51 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:17:39.345 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:39.345 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:39.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:39.428+0000 7fdc699b7980 1 waiting for incremental sync to catch up: 2024-04-16T18:17:39.426 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.414.6, 00000000001.935.6, , 00000000001.624.6, 00000000001.316.6, 00000000001.911.6, 00000000001.1312.6, 00000000001.934.6, 00000000001.422.6, 00000000001.625.6] 2024-04-16T18:17:39.426 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000001.896.6, 00000000001.414.6, 00000000001.935.6, , 00000000001.624.6, 00000000001.316.6, 00000000001.911.6, 00000000001.1312.6, 00000000001.934.6, 00000000001.422.6, 00000000001.625.6] 2024-04-16T18:17:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:44.428+0000 7fdc699b7980 1 bucket sync caught up with source: 2024-04-16T18:17:44.429 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.896.6, 00000000001.414.6, 00000000001.935.6, , 00000000001.624.6, 00000000001.316.6, 00000000001.911.6, 00000000001.1312.6, 00000000001.934.6, 00000000001.422.6, 00000000001.625.6] 2024-04-16T18:17:44.429 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000001.896.6, 00000000001.414.6, 00000000001.935.6, , 00000000001.624.6, 00000000001.316.6, 00000000001.911.6, 00000000001.1312.6, 00000000001.934.6, 00000000001.422.6, 00000000001.625.6] 2024-04-16T18:17:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:17:44.428+0000 7fdc699b7980 0 bucket checkpoint complete 2024-04-16T18:17:44.477 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-51 zones={test-zone2, test-zone1} 2024-04-16T18:17:44.477 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.477 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.477 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.477 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:44.477 DEBUG:boto:Data: 2024-04-16T18:17:44.477 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.477 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.477 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.478 DEBUG:boto:Params: {} 2024-04-16T18:17:44.478 DEBUG:boto:Token: None 2024-04-16T18:17:44.478 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.478 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr16f/Bf9+K2yavX6vYPc0pra1Q= 2024-04-16T18:17:44.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr16f/Bf9+K2yavX6vYPc0pra1Q=', 'Content-Length': '0'} 2024-04-16T18:17:44.482 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000234253a5a1fa85cd-00661ec0c8-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.482 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.483 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.483 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.483 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:44.483 DEBUG:boto:Data: 2024-04-16T18:17:44.483 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.483 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.483 DEBUG:boto:Params: {} 2024-04-16T18:17:44.483 DEBUG:boto:Token: None 2024-04-16T18:17:44.483 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr16f/Bf9+K2yavX6vYPc0pra1Q= 2024-04-16T18:17:44.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr16f/Bf9+K2yavX6vYPc0pra1Q=', 'Content-Length': '0'} 2024-04-16T18:17:44.487 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009726e5299884e114-00661ec0c8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.487 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:17:44.487 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.487 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.487 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.487 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.488 DEBUG:boto:Method: GET 2024-04-16T18:17:44.488 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.488 DEBUG:boto:Data: 2024-04-16T18:17:44.488 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.488 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.488 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.488 DEBUG:boto:Params: {} 2024-04-16T18:17:44.488 DEBUG:boto:Token: None 2024-04-16T18:17:44.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/?versions 2024-04-16T18:17:44.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk= 2024-04-16T18:17:44.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk=', 'Content-Length': '0'} 2024-04-16T18:17:44.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057010c772fb020e2-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.493 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.494 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000000 2024-04-16T18:17:44.494 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000001 2024-04-16T18:17:44.494 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000002 2024-04-16T18:17:44.494 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000003 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000005 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000006 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000010 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000011 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000012 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000013 2024-04-16T18:17:44.495 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:17:44.495 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.495 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.495 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.495 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.496 DEBUG:boto:Method: GET 2024-04-16T18:17:44.496 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.496 DEBUG:boto:Data: 2024-04-16T18:17:44.496 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.496 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.496 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.496 DEBUG:boto:Params: {} 2024-04-16T18:17:44.496 DEBUG:boto:Token: None 2024-04-16T18:17:44.496 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/?versions 2024-04-16T18:17:44.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk= 2024-04-16T18:17:44.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk=', 'Content-Length': '0'} 2024-04-16T18:17:44.500 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062d0a5ed5539eb1b-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.500 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000000 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000001 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000002 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000003 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000005 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000006 2024-04-16T18:17:44.501 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000010 2024-04-16T18:17:44.502 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000011 2024-04-16T18:17:44.502 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000012 2024-04-16T18:17:44.502 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000013 2024-04-16T18:17:44.502 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.502 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.502 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.502 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.502 DEBUG:boto:Method: GET 2024-04-16T18:17:44.503 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.503 DEBUG:boto:Data: 2024-04-16T18:17:44.503 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.503 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.503 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.503 DEBUG:boto:Params: {} 2024-04-16T18:17:44.503 DEBUG:boto:Token: None 2024-04-16T18:17:44.503 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/?versions 2024-04-16T18:17:44.503 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk= 2024-04-16T18:17:44.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk=', 'Content-Length': '0'} 2024-04-16T18:17:44.508 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a54635a9b3c3f636-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.508 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.509 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.509 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.509 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.509 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.509 DEBUG:boto:Method: GET 2024-04-16T18:17:44.509 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:17:44.509 DEBUG:boto:Data: 2024-04-16T18:17:44.509 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.509 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.510 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.510 DEBUG:boto:Params: {} 2024-04-16T18:17:44.510 DEBUG:boto:Token: None 2024-04-16T18:17:44.510 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/?versions 2024-04-16T18:17:44.510 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk= 2024-04-16T18:17:44.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NkRSyq9PMIcTtaYabo+KZTBM5fk=', 'Content-Length': '0'} 2024-04-16T18:17:44.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d34da0dbcdc47cd3-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.514 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000000 2024-04-16T18:17:44.515 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.515 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.515 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.515 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.516 DEBUG:boto:Method: GET 2024-04-16T18:17:44.516 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.516 DEBUG:boto:Data: 2024-04-16T18:17:44.516 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.516 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.516 DEBUG:boto:Params: {} 2024-04-16T18:17:44.516 DEBUG:boto:Token: None 2024-04-16T18:17:44.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio= 2024-04-16T18:17:44.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio=', 'Content-Length': '0'} 2024-04-16T18:17:44.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000045948f2f223b55a-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.520 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.520 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.520 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.520 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.520 DEBUG:boto:Method: GET 2024-04-16T18:17:44.520 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.521 DEBUG:boto:Data: 2024-04-16T18:17:44.521 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.521 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.521 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.521 DEBUG:boto:Params: {} 2024-04-16T18:17:44.521 DEBUG:boto:Token: None 2024-04-16T18:17:44.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio= 2024-04-16T18:17:44.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio=', 'Content-Length': '0'} 2024-04-16T18:17:44.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029fbd674018a07e4-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.566 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.566 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.567 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.567 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.567 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.567 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.567 DEBUG:boto:Data: 2024-04-16T18:17:44.567 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.567 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.567 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.567 DEBUG:boto:Params: {} 2024-04-16T18:17:44.567 DEBUG:boto:Token: None 2024-04-16T18:17:44.568 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RYbJB13KJdeWDkaUlU88ZpxzibA= 2024-04-16T18:17:44.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RYbJB13KJdeWDkaUlU88ZpxzibA=', 'Content-Length': '0'} 2024-04-16T18:17:44.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d6ef68e2a3f4511-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.571 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.571 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.571 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.571 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.571 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.571 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.571 DEBUG:boto:Data: 2024-04-16T18:17:44.571 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.572 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.572 DEBUG:boto:Params: {} 2024-04-16T18:17:44.572 DEBUG:boto:Token: None 2024-04-16T18:17:44.572 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RYbJB13KJdeWDkaUlU88ZpxzibA= 2024-04-16T18:17:44.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RYbJB13KJdeWDkaUlU88ZpxzibA=', 'Content-Length': '0'} 2024-04-16T18:17:44.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f536fb27046d5bef-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.575 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000000 2024-04-16T18:17:44.575 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.575 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.576 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.576 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.576 DEBUG:boto:Method: GET 2024-04-16T18:17:44.576 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.576 DEBUG:boto:Data: 2024-04-16T18:17:44.576 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.576 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.576 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.576 DEBUG:boto:Params: {} 2024-04-16T18:17:44.576 DEBUG:boto:Token: None 2024-04-16T18:17:44.577 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio= 2024-04-16T18:17:44.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio=', 'Content-Length': '0'} 2024-04-16T18:17:44.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cf9122bf44d7cbd-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.622 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.623 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.623 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.623 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.623 DEBUG:boto:Method: GET 2024-04-16T18:17:44.623 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.623 DEBUG:boto:Data: 2024-04-16T18:17:44.623 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.623 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.623 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.623 DEBUG:boto:Params: {} 2024-04-16T18:17:44.624 DEBUG:boto:Token: None 2024-04-16T18:17:44.624 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:17:44.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio= 2024-04-16T18:17:44.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xbdh5vTcYSILwtMrqqRghlX0aio=', 'Content-Length': '0'} 2024-04-16T18:17:44.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000618154e8ddd730f6-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.627 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.627 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.627 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.628 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.628 DEBUG:boto:Method: GET 2024-04-16T18:17:44.628 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.628 DEBUG:boto:Data: 2024-04-16T18:17:44.628 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.628 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.628 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.628 DEBUG:boto:Params: {} 2024-04-16T18:17:44.628 DEBUG:boto:Token: None 2024-04-16T18:17:44.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.629 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a7c5dadabe5363e-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.633 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.634 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.634 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.634 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.634 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.634 DEBUG:boto:Method: GET 2024-04-16T18:17:44.634 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.634 DEBUG:boto:Data: 2024-04-16T18:17:44.635 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.635 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.635 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.635 DEBUG:boto:Params: {} 2024-04-16T18:17:44.635 DEBUG:boto:Token: None 2024-04-16T18:17:44.635 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.635 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063abab8ba869a74d-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.640 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.640 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.640 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.640 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.641 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.641 DEBUG:boto:Data: 2024-04-16T18:17:44.641 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.641 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.641 DEBUG:boto:Params: {} 2024-04-16T18:17:44.641 DEBUG:boto:Token: None 2024-04-16T18:17:44.641 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fJS9dfrQY4E+LyHTkLHml0VIqOw= 2024-04-16T18:17:44.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fJS9dfrQY4E+LyHTkLHml0VIqOw=', 'Content-Length': '0'} 2024-04-16T18:17:44.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a01bd6068693e2cc-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.645 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.645 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.645 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.645 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.645 DEBUG:boto:Data: 2024-04-16T18:17:44.645 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.645 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.645 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.645 DEBUG:boto:Params: {} 2024-04-16T18:17:44.646 DEBUG:boto:Token: None 2024-04-16T18:17:44.646 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.646 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fJS9dfrQY4E+LyHTkLHml0VIqOw= 2024-04-16T18:17:44.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fJS9dfrQY4E+LyHTkLHml0VIqOw=', 'Content-Length': '0'} 2024-04-16T18:17:44.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008870558aec4132ac-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.649 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000000 2024-04-16T18:17:44.649 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.650 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.650 DEBUG:boto:Method: GET 2024-04-16T18:17:44.650 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.650 DEBUG:boto:Data: 2024-04-16T18:17:44.650 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.650 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.650 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.650 DEBUG:boto:Params: {} 2024-04-16T18:17:44.650 DEBUG:boto:Token: None 2024-04-16T18:17:44.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.651 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:glca9yujcaDvG/nAHDGVn8Llz9Q= 2024-04-16T18:17:44.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:glca9yujcaDvG/nAHDGVn8Llz9Q=', 'Content-Length': '0'} 2024-04-16T18:17:44.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000683550b567b7e819-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.694 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.694 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.695 DEBUG:boto:Method: GET 2024-04-16T18:17:44.695 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.695 DEBUG:boto:Data: 2024-04-16T18:17:44.695 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.695 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.695 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.695 DEBUG:boto:Params: {} 2024-04-16T18:17:44.695 DEBUG:boto:Token: None 2024-04-16T18:17:44.695 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:44.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:glca9yujcaDvG/nAHDGVn8Llz9Q= 2024-04-16T18:17:44.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:glca9yujcaDvG/nAHDGVn8Llz9Q=', 'Content-Length': '0'} 2024-04-16T18:17:44.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f11ea61461e5843-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.699 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.699 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.699 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.699 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.699 DEBUG:boto:Method: GET 2024-04-16T18:17:44.699 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.699 DEBUG:boto:Data: 2024-04-16T18:17:44.699 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.700 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.700 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.700 DEBUG:boto:Params: {} 2024-04-16T18:17:44.700 DEBUG:boto:Token: None 2024-04-16T18:17:44.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa9d0b44ce2d6425-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.704 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.705 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.705 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.705 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.705 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.705 DEBUG:boto:Method: GET 2024-04-16T18:17:44.705 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:17:44.705 DEBUG:boto:Data: 2024-04-16T18:17:44.705 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.706 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.706 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.706 DEBUG:boto:Params: {} 2024-04-16T18:17:44.706 DEBUG:boto:Token: None 2024-04-16T18:17:44.706 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.706 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.710 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0498cb0584462a8-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.710 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:17:33.849Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.711 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000001 2024-04-16T18:17:44.711 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.711 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.711 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.711 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.711 DEBUG:boto:Method: GET 2024-04-16T18:17:44.711 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.711 DEBUG:boto:Data: 2024-04-16T18:17:44.711 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.712 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.712 DEBUG:boto:Params: {} 2024-04-16T18:17:44.712 DEBUG:boto:Token: None 2024-04-16T18:17:44.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY= 2024-04-16T18:17:44.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY=', 'Content-Length': '0'} 2024-04-16T18:17:44.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a731eb36dfd06c1-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.758 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.761 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.761 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.761 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.761 DEBUG:boto:Method: GET 2024-04-16T18:17:44.761 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.761 DEBUG:boto:Data: 2024-04-16T18:17:44.762 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.762 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.762 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.762 DEBUG:boto:Params: {} 2024-04-16T18:17:44.762 DEBUG:boto:Token: None 2024-04-16T18:17:44.762 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY= 2024-04-16T18:17:44.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY=', 'Content-Length': '0'} 2024-04-16T18:17:44.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f057e67b745d9add-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.766 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.766 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.766 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.767 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.767 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.767 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.767 DEBUG:boto:Data: 2024-04-16T18:17:44.767 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.767 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.767 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.767 DEBUG:boto:Params: {} 2024-04-16T18:17:44.767 DEBUG:boto:Token: None 2024-04-16T18:17:44.767 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.767 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JL9PGZ6IYJ1WHL9zlhpF3DqQEng= 2024-04-16T18:17:44.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JL9PGZ6IYJ1WHL9zlhpF3DqQEng=', 'Content-Length': '0'} 2024-04-16T18:17:44.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000194dd6b2a55e0459-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.771 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.771 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.771 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.771 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.771 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.771 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.771 DEBUG:boto:Data: 2024-04-16T18:17:44.771 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.772 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.772 DEBUG:boto:Params: {} 2024-04-16T18:17:44.772 DEBUG:boto:Token: None 2024-04-16T18:17:44.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JL9PGZ6IYJ1WHL9zlhpF3DqQEng= 2024-04-16T18:17:44.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JL9PGZ6IYJ1WHL9zlhpF3DqQEng=', 'Content-Length': '0'} 2024-04-16T18:17:44.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b3c6061a4c81b0b-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000001 2024-04-16T18:17:44.775 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.775 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.775 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.775 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.775 DEBUG:boto:Method: GET 2024-04-16T18:17:44.776 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.776 DEBUG:boto:Data: 2024-04-16T18:17:44.776 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.776 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.776 DEBUG:boto:Params: {} 2024-04-16T18:17:44.776 DEBUG:boto:Token: None 2024-04-16T18:17:44.776 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY= 2024-04-16T18:17:44.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY=', 'Content-Length': '0'} 2024-04-16T18:17:44.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e567ea2c7ce233cf-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.822 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.823 DEBUG:boto:Method: GET 2024-04-16T18:17:44.823 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.823 DEBUG:boto:Data: 2024-04-16T18:17:44.823 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.823 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.823 DEBUG:boto:Params: {} 2024-04-16T18:17:44.823 DEBUG:boto:Token: None 2024-04-16T18:17:44.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:17:44.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY= 2024-04-16T18:17:44.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MReTLWKGhTAu8ZjxHp1QgC1IirY=', 'Content-Length': '0'} 2024-04-16T18:17:44.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000feab5f424f4c6815-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.827 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.827 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.827 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.827 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.827 DEBUG:boto:Method: GET 2024-04-16T18:17:44.827 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.828 DEBUG:boto:Data: 2024-04-16T18:17:44.828 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.828 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.828 DEBUG:boto:Params: {} 2024-04-16T18:17:44.828 DEBUG:boto:Token: None 2024-04-16T18:17:44.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.832 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009787052ae4aee666-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.832 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.833 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.833 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.833 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.833 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.833 DEBUG:boto:Method: GET 2024-04-16T18:17:44.833 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.833 DEBUG:boto:Data: 2024-04-16T18:17:44.833 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.833 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.834 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.834 DEBUG:boto:Params: {} 2024-04-16T18:17:44.834 DEBUG:boto:Token: None 2024-04-16T18:17:44.834 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.834 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4863e7940c8afec-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.838 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.838 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.839 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.839 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.839 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.839 DEBUG:boto:Data: 2024-04-16T18:17:44.839 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.839 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.839 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.839 DEBUG:boto:Params: {} 2024-04-16T18:17:44.839 DEBUG:boto:Token: None 2024-04-16T18:17:44.839 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.839 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SBgBf/32ODIOzhXK+A3mOZbOZqw= 2024-04-16T18:17:44.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SBgBf/32ODIOzhXK+A3mOZbOZqw=', 'Content-Length': '0'} 2024-04-16T18:17:44.842 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a23d8a2c639191d-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.843 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.843 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.843 DEBUG:boto:Method: HEAD 2024-04-16T18:17:44.843 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.843 DEBUG:boto:Data: 2024-04-16T18:17:44.843 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.843 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.843 DEBUG:boto:Params: {} 2024-04-16T18:17:44.843 DEBUG:boto:Token: None 2024-04-16T18:17:44.843 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SBgBf/32ODIOzhXK+A3mOZbOZqw= 2024-04-16T18:17:44.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SBgBf/32ODIOzhXK+A3mOZbOZqw=', 'Content-Length': '0'} 2024-04-16T18:17:44.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aac4ad2c017a4f12-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.847 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000001 2024-04-16T18:17:44.847 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.847 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.847 DEBUG:boto:Method: GET 2024-04-16T18:17:44.847 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.847 DEBUG:boto:Data: 2024-04-16T18:17:44.847 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.847 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.847 DEBUG:boto:Params: {} 2024-04-16T18:17:44.848 DEBUG:boto:Token: None 2024-04-16T18:17:44.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1tiFt4UlKEl/juytGU5A+SX5Tec= 2024-04-16T18:17:44.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1tiFt4UlKEl/juytGU5A+SX5Tec=', 'Content-Length': '0'} 2024-04-16T18:17:44.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f268de7c3c2dd046-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.894 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.894 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.895 DEBUG:boto:Method: GET 2024-04-16T18:17:44.895 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.895 DEBUG:boto:Data: 2024-04-16T18:17:44.895 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.895 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.895 DEBUG:boto:Params: {} 2024-04-16T18:17:44.895 DEBUG:boto:Token: None 2024-04-16T18:17:44.895 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:44.895 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1tiFt4UlKEl/juytGU5A+SX5Tec= 2024-04-16T18:17:44.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1tiFt4UlKEl/juytGU5A+SX5Tec=', 'Content-Length': '0'} 2024-04-16T18:17:44.898 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006eb097f6dcd10c2e-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.898 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.898 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.899 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.899 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.899 DEBUG:boto:Method: GET 2024-04-16T18:17:44.899 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.899 DEBUG:boto:Data: 2024-04-16T18:17:44.899 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.899 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.899 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.899 DEBUG:boto:Params: {} 2024-04-16T18:17:44.899 DEBUG:boto:Token: None 2024-04-16T18:17:44.899 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.904 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7cf4750136c4d09-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.904 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.904 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:44.905 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:44.905 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.905 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.905 DEBUG:boto:Method: GET 2024-04-16T18:17:44.905 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:17:44.905 DEBUG:boto:Data: 2024-04-16T18:17:44.905 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.905 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.905 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.905 DEBUG:boto:Params: {} 2024-04-16T18:17:44.905 DEBUG:boto:Token: None 2024-04-16T18:17:44.906 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/ 2024-04-16T18:17:44.906 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY= 2024-04-16T18:17:44.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:haqjJZfHtBizE4kbDH47seIcINY=', 'Content-Length': '0'} 2024-04-16T18:17:44.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042f0636fdc6017fc-00661ec0c8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.950 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:17:33.968Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:44.951 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000002 2024-04-16T18:17:44.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:44.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:44.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.951 DEBUG:boto:Method: GET 2024-04-16T18:17:44.951 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.951 DEBUG:boto:Data: 2024-04-16T18:17:44.951 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.951 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:44.951 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.952 DEBUG:boto:Params: {} 2024-04-16T18:17:44.952 DEBUG:boto:Token: None 2024-04-16T18:17:44.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XGbQQD6VkCjp2JeR3kUdJIiPOB0= 2024-04-16T18:17:44.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XGbQQD6VkCjp2JeR3kUdJIiPOB0=', 'Content-Length': '0'} 2024-04-16T18:17:44.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d8229c082fde2a6-00661ec0c8-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:44.998 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:44.998 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:44.999 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.999 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.999 DEBUG:boto:Method: GET 2024-04-16T18:17:44.999 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:44.999 DEBUG:boto:Data: 2024-04-16T18:17:44.999 DEBUG:boto:Headers: {} 2024-04-16T18:17:44.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:44.999 DEBUG:boto:Port: 8000 2024-04-16T18:17:44.999 DEBUG:boto:Params: {} 2024-04-16T18:17:44.999 DEBUG:boto:Token: None 2024-04-16T18:17:44.999 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:44 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XGbQQD6VkCjp2JeR3kUdJIiPOB0= 2024-04-16T18:17:45.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XGbQQD6VkCjp2JeR3kUdJIiPOB0=', 'Content-Length': '0'} 2024-04-16T18:17:45.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0ec8713cd815c83-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.003 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.003 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.003 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.003 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.003 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.003 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.003 DEBUG:boto:Data: 2024-04-16T18:17:45.004 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.004 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.004 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.004 DEBUG:boto:Params: {} 2024-04-16T18:17:45.004 DEBUG:boto:Token: None 2024-04-16T18:17:45.004 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sumCQizigd+78LP2CPKeHBVbrkE= 2024-04-16T18:17:45.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sumCQizigd+78LP2CPKeHBVbrkE=', 'Content-Length': '0'} 2024-04-16T18:17:45.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff57c952bb69830b-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.007 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.007 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.007 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.008 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.008 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.008 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.008 DEBUG:boto:Data: 2024-04-16T18:17:45.008 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.008 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.008 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.008 DEBUG:boto:Params: {} 2024-04-16T18:17:45.008 DEBUG:boto:Token: None 2024-04-16T18:17:45.008 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sumCQizigd+78LP2CPKeHBVbrkE= 2024-04-16T18:17:45.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sumCQizigd+78LP2CPKeHBVbrkE=', 'Content-Length': '0'} 2024-04-16T18:17:45.011 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000750802b46db4e5f6-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.012 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000002 2024-04-16T18:17:45.012 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.012 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.012 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.012 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.012 DEBUG:boto:Method: GET 2024-04-16T18:17:45.012 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.012 DEBUG:boto:Data: 2024-04-16T18:17:45.012 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.012 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.012 DEBUG:boto:Params: {} 2024-04-16T18:17:45.013 DEBUG:boto:Token: None 2024-04-16T18:17:45.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u23S2AlPhR1jZWvinmep8dWwal8= 2024-04-16T18:17:45.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u23S2AlPhR1jZWvinmep8dWwal8=', 'Content-Length': '0'} 2024-04-16T18:17:45.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005519aa9fbbbc9023-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.058 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.058 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.059 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.059 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.059 DEBUG:boto:Method: GET 2024-04-16T18:17:45.059 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.059 DEBUG:boto:Data: 2024-04-16T18:17:45.059 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.059 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.059 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.059 DEBUG:boto:Params: {} 2024-04-16T18:17:45.059 DEBUG:boto:Token: None 2024-04-16T18:17:45.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:17:45.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u23S2AlPhR1jZWvinmep8dWwal8= 2024-04-16T18:17:45.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u23S2AlPhR1jZWvinmep8dWwal8=', 'Content-Length': '0'} 2024-04-16T18:17:45.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035119f7c5bb58938-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.063 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.063 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.063 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.063 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.063 DEBUG:boto:Method: GET 2024-04-16T18:17:45.064 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.064 DEBUG:boto:Data: 2024-04-16T18:17:45.064 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.064 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.064 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.064 DEBUG:boto:Params: {} 2024-04-16T18:17:45.064 DEBUG:boto:Token: None 2024-04-16T18:17:45.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9108cee16e9e586-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.069 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.069 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.070 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.070 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.070 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.070 DEBUG:boto:Method: GET 2024-04-16T18:17:45.070 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.070 DEBUG:boto:Data: 2024-04-16T18:17:45.070 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.070 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.070 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.070 DEBUG:boto:Params: {} 2024-04-16T18:17:45.070 DEBUG:boto:Token: None 2024-04-16T18:17:45.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001449982906234d4f-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.075 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.075 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.075 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.075 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.075 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.075 DEBUG:boto:Data: 2024-04-16T18:17:45.075 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.076 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.076 DEBUG:boto:Params: {} 2024-04-16T18:17:45.076 DEBUG:boto:Token: None 2024-04-16T18:17:45.076 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sahIq2ve+R9+RWaTrzR+8jUSO3k= 2024-04-16T18:17:45.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sahIq2ve+R9+RWaTrzR+8jUSO3k=', 'Content-Length': '0'} 2024-04-16T18:17:45.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1228dbd828847df-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.079 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.080 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.080 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.080 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.080 DEBUG:boto:Data: 2024-04-16T18:17:45.080 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.080 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.080 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.080 DEBUG:boto:Params: {} 2024-04-16T18:17:45.080 DEBUG:boto:Token: None 2024-04-16T18:17:45.080 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sahIq2ve+R9+RWaTrzR+8jUSO3k= 2024-04-16T18:17:45.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sahIq2ve+R9+RWaTrzR+8jUSO3k=', 'Content-Length': '0'} 2024-04-16T18:17:45.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ae36a2c8511c32e-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000002 2024-04-16T18:17:45.084 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.084 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.084 DEBUG:boto:Method: GET 2024-04-16T18:17:45.084 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.084 DEBUG:boto:Data: 2024-04-16T18:17:45.084 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.085 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.085 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.085 DEBUG:boto:Params: {} 2024-04-16T18:17:45.085 DEBUG:boto:Token: None 2024-04-16T18:17:45.085 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.085 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AV1SxsX5W9o9ftugwywBDVaadj0= 2024-04-16T18:17:45.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AV1SxsX5W9o9ftugwywBDVaadj0=', 'Content-Length': '0'} 2024-04-16T18:17:45.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026beb9b54c29f85b-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.130 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.130 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.131 DEBUG:boto:Method: GET 2024-04-16T18:17:45.131 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.131 DEBUG:boto:Data: 2024-04-16T18:17:45.131 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.131 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.131 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.131 DEBUG:boto:Params: {} 2024-04-16T18:17:45.131 DEBUG:boto:Token: None 2024-04-16T18:17:45.131 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:45.131 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AV1SxsX5W9o9ftugwywBDVaadj0= 2024-04-16T18:17:45.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AV1SxsX5W9o9ftugwywBDVaadj0=', 'Content-Length': '0'} 2024-04-16T18:17:45.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000071d8e6fae6e83ad-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.135 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.135 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.136 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.136 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.136 DEBUG:boto:Method: GET 2024-04-16T18:17:45.136 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.136 DEBUG:boto:Data: 2024-04-16T18:17:45.136 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.136 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.136 DEBUG:boto:Params: {} 2024-04-16T18:17:45.136 DEBUG:boto:Token: None 2024-04-16T18:17:45.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000872f9b0c701c8d5-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.140 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.141 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.141 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.141 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.141 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.141 DEBUG:boto:Method: GET 2024-04-16T18:17:45.141 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:17:45.141 DEBUG:boto:Data: 2024-04-16T18:17:45.141 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.141 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.141 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.141 DEBUG:boto:Params: {} 2024-04-16T18:17:45.142 DEBUG:boto:Token: None 2024-04-16T18:17:45.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.142 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.145 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e95c8a2fcf8cfbb-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.146 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:17:33.986Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.146 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000003 2024-04-16T18:17:45.146 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.146 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.146 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.146 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.146 DEBUG:boto:Method: GET 2024-04-16T18:17:45.147 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.147 DEBUG:boto:Data: 2024-04-16T18:17:45.147 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.147 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.147 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.147 DEBUG:boto:Params: {} 2024-04-16T18:17:45.147 DEBUG:boto:Token: None 2024-04-16T18:17:45.147 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8= 2024-04-16T18:17:45.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8=', 'Content-Length': '0'} 2024-04-16T18:17:45.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005a34d5cb1caf9ab-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.190 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.190 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.191 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.191 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.191 DEBUG:boto:Method: GET 2024-04-16T18:17:45.191 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.191 DEBUG:boto:Data: 2024-04-16T18:17:45.191 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.191 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.191 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.191 DEBUG:boto:Params: {} 2024-04-16T18:17:45.191 DEBUG:boto:Token: None 2024-04-16T18:17:45.191 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.191 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8= 2024-04-16T18:17:45.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8=', 'Content-Length': '0'} 2024-04-16T18:17:45.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cf4e8c311e866e6-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.195 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.196 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.196 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.196 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.196 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.196 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.196 DEBUG:boto:Data: 2024-04-16T18:17:45.196 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.196 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.196 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.196 DEBUG:boto:Params: {} 2024-04-16T18:17:45.196 DEBUG:boto:Token: None 2024-04-16T18:17:45.197 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aat8ooXmT/5n/eH6GBcLp3PMFNQ= 2024-04-16T18:17:45.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aat8ooXmT/5n/eH6GBcLp3PMFNQ=', 'Content-Length': '0'} 2024-04-16T18:17:45.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056a775fb3dc92cae-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.200 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.200 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.200 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.200 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.200 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.200 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.200 DEBUG:boto:Data: 2024-04-16T18:17:45.200 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.201 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.201 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.201 DEBUG:boto:Params: {} 2024-04-16T18:17:45.201 DEBUG:boto:Token: None 2024-04-16T18:17:45.201 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.201 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aat8ooXmT/5n/eH6GBcLp3PMFNQ= 2024-04-16T18:17:45.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aat8ooXmT/5n/eH6GBcLp3PMFNQ=', 'Content-Length': '0'} 2024-04-16T18:17:45.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089c705c42b7d4f85-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.204 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000003 2024-04-16T18:17:45.204 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.205 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.205 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.205 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.205 DEBUG:boto:Method: GET 2024-04-16T18:17:45.205 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.205 DEBUG:boto:Data: 2024-04-16T18:17:45.205 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.205 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.205 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.205 DEBUG:boto:Params: {} 2024-04-16T18:17:45.205 DEBUG:boto:Token: None 2024-04-16T18:17:45.205 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.206 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8= 2024-04-16T18:17:45.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8=', 'Content-Length': '0'} 2024-04-16T18:17:45.208 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e6a144c530ecd86-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.250 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.250 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.250 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.251 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.251 DEBUG:boto:Method: GET 2024-04-16T18:17:45.251 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.251 DEBUG:boto:Data: 2024-04-16T18:17:45.251 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.251 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.251 DEBUG:boto:Params: {} 2024-04-16T18:17:45.251 DEBUG:boto:Token: None 2024-04-16T18:17:45.251 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:17:45.251 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8= 2024-04-16T18:17:45.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hgftJkxZ0eM8bryy+IKDEwcNgx8=', 'Content-Length': '0'} 2024-04-16T18:17:45.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f264c0db9fd3dcb7-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.255 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.255 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.255 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.255 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.255 DEBUG:boto:Method: GET 2024-04-16T18:17:45.255 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.255 DEBUG:boto:Data: 2024-04-16T18:17:45.256 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.256 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.256 DEBUG:boto:Params: {} 2024-04-16T18:17:45.256 DEBUG:boto:Token: None 2024-04-16T18:17:45.256 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002de417fa7c34d047-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.260 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.260 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.261 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.261 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.261 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.261 DEBUG:boto:Method: GET 2024-04-16T18:17:45.261 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.261 DEBUG:boto:Data: 2024-04-16T18:17:45.261 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.261 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.261 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.261 DEBUG:boto:Params: {} 2024-04-16T18:17:45.261 DEBUG:boto:Token: None 2024-04-16T18:17:45.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.265 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a07b762e112c6519-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.306 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.307 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.307 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.307 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.307 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.307 DEBUG:boto:Data: 2024-04-16T18:17:45.307 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.307 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.307 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.307 DEBUG:boto:Params: {} 2024-04-16T18:17:45.307 DEBUG:boto:Token: None 2024-04-16T18:17:45.307 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EgPTmKqgVKhe47/rEDjifno7X8s= 2024-04-16T18:17:45.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EgPTmKqgVKhe47/rEDjifno7X8s=', 'Content-Length': '0'} 2024-04-16T18:17:45.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa9a2030af08dbb3-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.311 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.311 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.311 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.311 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.311 DEBUG:boto:Data: 2024-04-16T18:17:45.311 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.311 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.311 DEBUG:boto:Params: {} 2024-04-16T18:17:45.312 DEBUG:boto:Token: None 2024-04-16T18:17:45.312 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EgPTmKqgVKhe47/rEDjifno7X8s= 2024-04-16T18:17:45.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EgPTmKqgVKhe47/rEDjifno7X8s=', 'Content-Length': '0'} 2024-04-16T18:17:45.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5e7056f4c926881-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.315 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000003 2024-04-16T18:17:45.315 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.315 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.315 DEBUG:boto:Method: GET 2024-04-16T18:17:45.315 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.316 DEBUG:boto:Data: 2024-04-16T18:17:45.316 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.316 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.316 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.316 DEBUG:boto:Params: {} 2024-04-16T18:17:45.316 DEBUG:boto:Token: None 2024-04-16T18:17:45.316 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s0fhFr5sHQ2eeHjw0VK3UXJ04qs= 2024-04-16T18:17:45.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s0fhFr5sHQ2eeHjw0VK3UXJ04qs=', 'Content-Length': '0'} 2024-04-16T18:17:45.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089fe5f898e6d5355-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.362 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.363 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.363 DEBUG:boto:Method: GET 2024-04-16T18:17:45.363 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.363 DEBUG:boto:Data: 2024-04-16T18:17:45.363 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.363 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.363 DEBUG:boto:Params: {} 2024-04-16T18:17:45.363 DEBUG:boto:Token: None 2024-04-16T18:17:45.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:45.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s0fhFr5sHQ2eeHjw0VK3UXJ04qs= 2024-04-16T18:17:45.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s0fhFr5sHQ2eeHjw0VK3UXJ04qs=', 'Content-Length': '0'} 2024-04-16T18:17:45.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bbaa8c388533e4a-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.367 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.368 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.368 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.368 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.368 DEBUG:boto:Method: GET 2024-04-16T18:17:45.368 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.368 DEBUG:boto:Data: 2024-04-16T18:17:45.368 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.368 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.368 DEBUG:boto:Params: {} 2024-04-16T18:17:45.368 DEBUG:boto:Token: None 2024-04-16T18:17:45.368 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.373 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c416d9105fc33ba-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.414 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.415 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.415 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.415 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.415 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.415 DEBUG:boto:Method: GET 2024-04-16T18:17:45.415 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:17:45.415 DEBUG:boto:Data: 2024-04-16T18:17:45.415 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.415 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.415 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.415 DEBUG:boto:Params: {} 2024-04-16T18:17:45.415 DEBUG:boto:Token: None 2024-04-16T18:17:45.416 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.419 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e1cf3b9451829b9-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.420 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:17:34.104Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000005 2024-04-16T18:17:45.420 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.420 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.420 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.420 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.420 DEBUG:boto:Method: GET 2024-04-16T18:17:45.420 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.421 DEBUG:boto:Data: 2024-04-16T18:17:45.421 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.421 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.421 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.421 DEBUG:boto:Params: {} 2024-04-16T18:17:45.421 DEBUG:boto:Token: None 2024-04-16T18:17:45.421 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.421 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4= 2024-04-16T18:17:45.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4=', 'Content-Length': '0'} 2024-04-16T18:17:45.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000373a7b0fe3e17e92-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.466 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.466 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.466 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.467 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.467 DEBUG:boto:Method: GET 2024-04-16T18:17:45.467 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.467 DEBUG:boto:Data: 2024-04-16T18:17:45.467 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.467 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.467 DEBUG:boto:Params: {} 2024-04-16T18:17:45.467 DEBUG:boto:Token: None 2024-04-16T18:17:45.467 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4= 2024-04-16T18:17:45.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4=', 'Content-Length': '0'} 2024-04-16T18:17:45.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2f6876fa2949d9c-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.471 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.471 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.471 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.471 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.471 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.471 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.472 DEBUG:boto:Data: 2024-04-16T18:17:45.472 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.472 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.472 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.472 DEBUG:boto:Params: {} 2024-04-16T18:17:45.472 DEBUG:boto:Token: None 2024-04-16T18:17:45.472 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BVS70YSZg5vpoSuHPxA8sOlRz04= 2024-04-16T18:17:45.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BVS70YSZg5vpoSuHPxA8sOlRz04=', 'Content-Length': '0'} 2024-04-16T18:17:45.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000761cbf465a321f46-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.475 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.476 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.476 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.476 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.476 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.476 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.476 DEBUG:boto:Data: 2024-04-16T18:17:45.476 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.476 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.476 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.476 DEBUG:boto:Params: {} 2024-04-16T18:17:45.476 DEBUG:boto:Token: None 2024-04-16T18:17:45.477 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.477 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BVS70YSZg5vpoSuHPxA8sOlRz04= 2024-04-16T18:17:45.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BVS70YSZg5vpoSuHPxA8sOlRz04=', 'Content-Length': '0'} 2024-04-16T18:17:45.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e682a9f97ec9317-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.480 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000005 2024-04-16T18:17:45.480 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.480 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.480 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.480 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.480 DEBUG:boto:Method: GET 2024-04-16T18:17:45.480 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.481 DEBUG:boto:Data: 2024-04-16T18:17:45.481 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.481 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.481 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.481 DEBUG:boto:Params: {} 2024-04-16T18:17:45.481 DEBUG:boto:Token: None 2024-04-16T18:17:45.481 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.481 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4= 2024-04-16T18:17:45.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4=', 'Content-Length': '0'} 2024-04-16T18:17:45.484 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033a8f481e3f5ebdd-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.526 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.526 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.526 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.527 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.527 DEBUG:boto:Method: GET 2024-04-16T18:17:45.527 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.527 DEBUG:boto:Data: 2024-04-16T18:17:45.527 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.527 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.527 DEBUG:boto:Params: {} 2024-04-16T18:17:45.527 DEBUG:boto:Token: None 2024-04-16T18:17:45.527 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:17:45.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4= 2024-04-16T18:17:45.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YVRVgSn/GtbXXYz1xvlrVKgwsv4=', 'Content-Length': '0'} 2024-04-16T18:17:45.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025bfb0d200e53cb8-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.531 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.531 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.531 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.531 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.531 DEBUG:boto:Method: GET 2024-04-16T18:17:45.531 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.532 DEBUG:boto:Data: 2024-04-16T18:17:45.532 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.532 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.532 DEBUG:boto:Params: {} 2024-04-16T18:17:45.532 DEBUG:boto:Token: None 2024-04-16T18:17:45.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008275d0d7ffef37cc-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.536 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.536 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.537 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.537 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.537 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.537 DEBUG:boto:Method: GET 2024-04-16T18:17:45.537 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.537 DEBUG:boto:Data: 2024-04-16T18:17:45.537 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.537 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.537 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.537 DEBUG:boto:Params: {} 2024-04-16T18:17:45.537 DEBUG:boto:Token: None 2024-04-16T18:17:45.538 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.538 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.541 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af3ff13060ea4a78-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.582 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.582 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.583 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.583 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.583 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.583 DEBUG:boto:Data: 2024-04-16T18:17:45.583 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.583 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.583 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.583 DEBUG:boto:Params: {} 2024-04-16T18:17:45.583 DEBUG:boto:Token: None 2024-04-16T18:17:45.583 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n6hnjt8Kc/qhJ4w8o5pRdAdVjbk= 2024-04-16T18:17:45.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n6hnjt8Kc/qhJ4w8o5pRdAdVjbk=', 'Content-Length': '0'} 2024-04-16T18:17:45.587 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2bcba0c3dc5f83c-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.587 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.587 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.587 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.587 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.588 DEBUG:boto:Data: 2024-04-16T18:17:45.588 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.588 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.588 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.588 DEBUG:boto:Params: {} 2024-04-16T18:17:45.588 DEBUG:boto:Token: None 2024-04-16T18:17:45.588 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n6hnjt8Kc/qhJ4w8o5pRdAdVjbk= 2024-04-16T18:17:45.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n6hnjt8Kc/qhJ4w8o5pRdAdVjbk=', 'Content-Length': '0'} 2024-04-16T18:17:45.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cb564449ebfb0a3-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.591 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000005 2024-04-16T18:17:45.591 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.592 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.592 DEBUG:boto:Method: GET 2024-04-16T18:17:45.592 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.592 DEBUG:boto:Data: 2024-04-16T18:17:45.592 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.592 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.592 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.592 DEBUG:boto:Params: {} 2024-04-16T18:17:45.592 DEBUG:boto:Token: None 2024-04-16T18:17:45.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5slRMOIlZwUGrpVeYGdDJB6wo+s= 2024-04-16T18:17:45.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5slRMOIlZwUGrpVeYGdDJB6wo+s=', 'Content-Length': '0'} 2024-04-16T18:17:45.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038983dc4c39e674b-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.638 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.638 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.639 DEBUG:boto:Method: GET 2024-04-16T18:17:45.639 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.639 DEBUG:boto:Data: 2024-04-16T18:17:45.639 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.639 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.639 DEBUG:boto:Params: {} 2024-04-16T18:17:45.639 DEBUG:boto:Token: None 2024-04-16T18:17:45.639 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:45.639 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5slRMOIlZwUGrpVeYGdDJB6wo+s= 2024-04-16T18:17:45.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5slRMOIlZwUGrpVeYGdDJB6wo+s=', 'Content-Length': '0'} 2024-04-16T18:17:45.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c99e49464756598-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.643 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.643 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.643 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.643 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.643 DEBUG:boto:Method: GET 2024-04-16T18:17:45.643 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.643 DEBUG:boto:Data: 2024-04-16T18:17:45.643 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.643 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.644 DEBUG:boto:Params: {} 2024-04-16T18:17:45.644 DEBUG:boto:Token: None 2024-04-16T18:17:45.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.648 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd9831f7af93295a-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.690 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.691 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.691 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.691 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.691 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.691 DEBUG:boto:Method: GET 2024-04-16T18:17:45.691 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:17:45.691 DEBUG:boto:Data: 2024-04-16T18:17:45.691 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.691 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.691 DEBUG:boto:Params: {} 2024-04-16T18:17:45.692 DEBUG:boto:Token: None 2024-04-16T18:17:45.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091f9447e401ce957-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.696 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:33.781Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.696 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000006 2024-04-16T18:17:45.696 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.696 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.697 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.697 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.697 DEBUG:boto:Method: GET 2024-04-16T18:17:45.697 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.697 DEBUG:boto:Data: 2024-04-16T18:17:45.697 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.697 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.697 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.697 DEBUG:boto:Params: {} 2024-04-16T18:17:45.697 DEBUG:boto:Token: None 2024-04-16T18:17:45.698 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.698 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI= 2024-04-16T18:17:45.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI=', 'Content-Length': '0'} 2024-04-16T18:17:45.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076b9b65059fba6cb-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.742 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.742 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.742 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.743 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.743 DEBUG:boto:Method: GET 2024-04-16T18:17:45.743 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.743 DEBUG:boto:Data: 2024-04-16T18:17:45.743 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.743 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.743 DEBUG:boto:Params: {} 2024-04-16T18:17:45.743 DEBUG:boto:Token: None 2024-04-16T18:17:45.743 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.743 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI= 2024-04-16T18:17:45.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI=', 'Content-Length': '0'} 2024-04-16T18:17:45.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aef29eae3957a8ef-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.747 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.747 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.747 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.747 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.748 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.748 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.748 DEBUG:boto:Data: 2024-04-16T18:17:45.748 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.748 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.748 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.748 DEBUG:boto:Params: {} 2024-04-16T18:17:45.748 DEBUG:boto:Token: None 2024-04-16T18:17:45.748 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7B6jeC/8COSRo7aXTXkpFuEtfYg= 2024-04-16T18:17:45.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7B6jeC/8COSRo7aXTXkpFuEtfYg=', 'Content-Length': '0'} 2024-04-16T18:17:45.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000351dc7bb17d126d5-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.752 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.752 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.752 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.752 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.752 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.752 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.752 DEBUG:boto:Data: 2024-04-16T18:17:45.752 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.752 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.752 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.753 DEBUG:boto:Params: {} 2024-04-16T18:17:45.753 DEBUG:boto:Token: None 2024-04-16T18:17:45.753 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7B6jeC/8COSRo7aXTXkpFuEtfYg= 2024-04-16T18:17:45.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7B6jeC/8COSRo7aXTXkpFuEtfYg=', 'Content-Length': '0'} 2024-04-16T18:17:45.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f907a912e7c54ff-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.756 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000006 2024-04-16T18:17:45.756 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.756 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.757 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.757 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.757 DEBUG:boto:Method: GET 2024-04-16T18:17:45.757 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.757 DEBUG:boto:Data: 2024-04-16T18:17:45.757 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.757 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.757 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.757 DEBUG:boto:Params: {} 2024-04-16T18:17:45.757 DEBUG:boto:Token: None 2024-04-16T18:17:45.757 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI= 2024-04-16T18:17:45.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI=', 'Content-Length': '0'} 2024-04-16T18:17:45.761 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a161621ffb5f6ff9-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.802 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.802 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.803 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.803 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.803 DEBUG:boto:Method: GET 2024-04-16T18:17:45.803 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.803 DEBUG:boto:Data: 2024-04-16T18:17:45.803 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.803 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.803 DEBUG:boto:Params: {} 2024-04-16T18:17:45.803 DEBUG:boto:Token: None 2024-04-16T18:17:45.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:17:45.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI= 2024-04-16T18:17:45.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OgQv64ZNwiPSArmX3Hd7Al0hbRI=', 'Content-Length': '0'} 2024-04-16T18:17:45.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5d64a1eb7e31843-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.807 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.807 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.807 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.807 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.807 DEBUG:boto:Method: GET 2024-04-16T18:17:45.807 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.808 DEBUG:boto:Data: 2024-04-16T18:17:45.808 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.808 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.808 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.808 DEBUG:boto:Params: {} 2024-04-16T18:17:45.808 DEBUG:boto:Token: None 2024-04-16T18:17:45.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e40aa6d95e7901c5-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.813 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.813 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.813 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.813 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.813 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.813 DEBUG:boto:Method: GET 2024-04-16T18:17:45.813 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.814 DEBUG:boto:Data: 2024-04-16T18:17:45.814 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.814 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.814 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.814 DEBUG:boto:Params: {} 2024-04-16T18:17:45.814 DEBUG:boto:Token: None 2024-04-16T18:17:45.814 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.818 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7eb84bc6f0441f4-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.862 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.863 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.863 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.863 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.863 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.863 DEBUG:boto:Data: 2024-04-16T18:17:45.863 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.863 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.863 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.863 DEBUG:boto:Params: {} 2024-04-16T18:17:45.864 DEBUG:boto:Token: None 2024-04-16T18:17:45.864 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:leC4g3ldzTzSNHWX/SL+6Tq93kM= 2024-04-16T18:17:45.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:leC4g3ldzTzSNHWX/SL+6Tq93kM=', 'Content-Length': '0'} 2024-04-16T18:17:45.866 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a053ac93a9820ec7-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.867 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.867 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.867 DEBUG:boto:Method: HEAD 2024-04-16T18:17:45.867 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.867 DEBUG:boto:Data: 2024-04-16T18:17:45.867 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.867 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.867 DEBUG:boto:Params: {} 2024-04-16T18:17:45.868 DEBUG:boto:Token: None 2024-04-16T18:17:45.868 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:leC4g3ldzTzSNHWX/SL+6Tq93kM= 2024-04-16T18:17:45.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:leC4g3ldzTzSNHWX/SL+6Tq93kM=', 'Content-Length': '0'} 2024-04-16T18:17:45.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c2865dacfceee06-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.871 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000006 2024-04-16T18:17:45.871 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.871 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.871 DEBUG:boto:Method: GET 2024-04-16T18:17:45.871 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.871 DEBUG:boto:Data: 2024-04-16T18:17:45.871 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.871 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.872 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.872 DEBUG:boto:Params: {} 2024-04-16T18:17:45.872 DEBUG:boto:Token: None 2024-04-16T18:17:45.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5KVfW+QMS17p8b+ehZz6Q4E6Cbo= 2024-04-16T18:17:45.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5KVfW+QMS17p8b+ehZz6Q4E6Cbo=', 'Content-Length': '0'} 2024-04-16T18:17:45.875 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b89827a53847f0e0-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.918 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.918 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.919 DEBUG:boto:Method: GET 2024-04-16T18:17:45.919 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.919 DEBUG:boto:Data: 2024-04-16T18:17:45.919 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.919 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.919 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.919 DEBUG:boto:Params: {} 2024-04-16T18:17:45.919 DEBUG:boto:Token: None 2024-04-16T18:17:45.919 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:45.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5KVfW+QMS17p8b+ehZz6Q4E6Cbo= 2024-04-16T18:17:45.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5KVfW+QMS17p8b+ehZz6Q4E6Cbo=', 'Content-Length': '0'} 2024-04-16T18:17:45.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031a3ea62ba531fd5-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.923 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.923 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.923 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.923 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.923 DEBUG:boto:Method: GET 2024-04-16T18:17:45.923 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.923 DEBUG:boto:Data: 2024-04-16T18:17:45.924 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.924 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.924 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.924 DEBUG:boto:Params: {} 2024-04-16T18:17:45.924 DEBUG:boto:Token: None 2024-04-16T18:17:45.924 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.928 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000578eae6f5282db23-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.970 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.971 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:45.971 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:45.971 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.971 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.971 DEBUG:boto:Method: GET 2024-04-16T18:17:45.971 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:17:45.971 DEBUG:boto:Data: 2024-04-16T18:17:45.971 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:45.971 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.971 DEBUG:boto:Params: {} 2024-04-16T18:17:45.971 DEBUG:boto:Token: None 2024-04-16T18:17:45.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/ 2024-04-16T18:17:45.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk= 2024-04-16T18:17:45.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/JgnRK+NR+WBaAU+q9AOsZlFjAk=', 'Content-Length': '0'} 2024-04-16T18:17:45.975 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041999a0d903cba49-00661ec0c9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:45.976 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:17:33.918Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:45.976 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000010 2024-04-16T18:17:45.976 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:45.976 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:45.976 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:45.976 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:45.977 DEBUG:boto:Method: GET 2024-04-16T18:17:45.977 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:45.977 DEBUG:boto:Data: 2024-04-16T18:17:45.977 DEBUG:boto:Headers: {} 2024-04-16T18:17:45.977 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:45.977 DEBUG:boto:Port: 8000 2024-04-16T18:17:45.977 DEBUG:boto:Params: {} 2024-04-16T18:17:45.977 DEBUG:boto:Token: None 2024-04-16T18:17:45.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:45 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:45.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hiQ35mcLfcXifHp/umgLO2vUk+Q= 2024-04-16T18:17:45.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hiQ35mcLfcXifHp/umgLO2vUk+Q=', 'Content-Length': '0'} 2024-04-16T18:17:45.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac210c74aaeff4ec-00661ec0c9-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.022 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.022 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.023 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.023 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.023 DEBUG:boto:Method: GET 2024-04-16T18:17:46.023 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.023 DEBUG:boto:Data: 2024-04-16T18:17:46.023 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.023 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.023 DEBUG:boto:Params: {} 2024-04-16T18:17:46.023 DEBUG:boto:Token: None 2024-04-16T18:17:46.023 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NlDWR8W7htUZRhy8qJSFOwOy1E8= 2024-04-16T18:17:46.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NlDWR8W7htUZRhy8qJSFOwOy1E8=', 'Content-Length': '0'} 2024-04-16T18:17:46.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000623ed42b0cbd0272-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.027 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.027 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.027 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.027 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.027 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.028 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.028 DEBUG:boto:Data: 2024-04-16T18:17:46.028 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.028 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.028 DEBUG:boto:Params: {} 2024-04-16T18:17:46.028 DEBUG:boto:Token: None 2024-04-16T18:17:46.028 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j3cINbKt24ZjEAdHxorLV4oWCTU= 2024-04-16T18:17:46.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j3cINbKt24ZjEAdHxorLV4oWCTU=', 'Content-Length': '0'} 2024-04-16T18:17:46.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f355cd133ec20ddf-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.031 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.032 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.032 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.032 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.032 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.032 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.032 DEBUG:boto:Data: 2024-04-16T18:17:46.032 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.032 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.032 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.032 DEBUG:boto:Params: {} 2024-04-16T18:17:46.032 DEBUG:boto:Token: None 2024-04-16T18:17:46.033 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.033 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j3cINbKt24ZjEAdHxorLV4oWCTU= 2024-04-16T18:17:46.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j3cINbKt24ZjEAdHxorLV4oWCTU=', 'Content-Length': '0'} 2024-04-16T18:17:46.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cfae63f24591a2c-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.036 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000010 2024-04-16T18:17:46.036 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.036 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.036 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.036 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.036 DEBUG:boto:Method: GET 2024-04-16T18:17:46.036 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.036 DEBUG:boto:Data: 2024-04-16T18:17:46.036 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.036 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.036 DEBUG:boto:Params: {} 2024-04-16T18:17:46.037 DEBUG:boto:Token: None 2024-04-16T18:17:46.037 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NlDWR8W7htUZRhy8qJSFOwOy1E8= 2024-04-16T18:17:46.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NlDWR8W7htUZRhy8qJSFOwOy1E8=', 'Content-Length': '0'} 2024-04-16T18:17:46.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003904c2ad6affcc6f-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.086 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.086 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.087 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.087 DEBUG:boto:Method: GET 2024-04-16T18:17:46.087 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.087 DEBUG:boto:Data: 2024-04-16T18:17:46.087 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.087 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.087 DEBUG:boto:Params: {} 2024-04-16T18:17:46.087 DEBUG:boto:Token: None 2024-04-16T18:17:46.087 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:17:46.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NlDWR8W7htUZRhy8qJSFOwOy1E8= 2024-04-16T18:17:46.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NlDWR8W7htUZRhy8qJSFOwOy1E8=', 'Content-Length': '0'} 2024-04-16T18:17:46.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3b4ac42a6633c5d-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.091 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.091 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.091 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.091 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.091 DEBUG:boto:Method: GET 2024-04-16T18:17:46.091 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.091 DEBUG:boto:Data: 2024-04-16T18:17:46.091 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.091 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.091 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.092 DEBUG:boto:Params: {} 2024-04-16T18:17:46.092 DEBUG:boto:Token: None 2024-04-16T18:17:46.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.096 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea29a3d4fee31ce5-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.096 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.096 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.096 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.097 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.097 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.097 DEBUG:boto:Method: GET 2024-04-16T18:17:46.097 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.097 DEBUG:boto:Data: 2024-04-16T18:17:46.097 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.097 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.097 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.097 DEBUG:boto:Params: {} 2024-04-16T18:17:46.097 DEBUG:boto:Token: None 2024-04-16T18:17:46.097 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.098 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.101 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cae86623937af74-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.142 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.143 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.143 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.143 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.143 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.143 DEBUG:boto:Data: 2024-04-16T18:17:46.143 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.143 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.143 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.143 DEBUG:boto:Params: {} 2024-04-16T18:17:46.143 DEBUG:boto:Token: None 2024-04-16T18:17:46.143 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XcnO8ghbos3IU3beFtK9xQ1/kuo= 2024-04-16T18:17:46.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XcnO8ghbos3IU3beFtK9xQ1/kuo=', 'Content-Length': '0'} 2024-04-16T18:17:46.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005db72375ce1708d5-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.146 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.146 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.146 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.146 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.147 DEBUG:boto:Data: 2024-04-16T18:17:46.147 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.147 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.147 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.147 DEBUG:boto:Params: {} 2024-04-16T18:17:46.147 DEBUG:boto:Token: None 2024-04-16T18:17:46.147 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XcnO8ghbos3IU3beFtK9xQ1/kuo= 2024-04-16T18:17:46.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XcnO8ghbos3IU3beFtK9xQ1/kuo=', 'Content-Length': '0'} 2024-04-16T18:17:46.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006388d04c4824c84e-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.150 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000010 2024-04-16T18:17:46.150 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.150 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.151 DEBUG:boto:Method: GET 2024-04-16T18:17:46.151 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.151 DEBUG:boto:Data: 2024-04-16T18:17:46.151 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.151 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.151 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.151 DEBUG:boto:Params: {} 2024-04-16T18:17:46.151 DEBUG:boto:Token: None 2024-04-16T18:17:46.151 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.151 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Gv3zQFY/yxwTha58o1yaizgPe4= 2024-04-16T18:17:46.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Gv3zQFY/yxwTha58o1yaizgPe4=', 'Content-Length': '0'} 2024-04-16T18:17:46.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022db9ffc765b1b8a-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.199 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.199 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.199 DEBUG:boto:Method: GET 2024-04-16T18:17:46.199 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.199 DEBUG:boto:Data: 2024-04-16T18:17:46.199 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.199 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.199 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.199 DEBUG:boto:Params: {} 2024-04-16T18:17:46.200 DEBUG:boto:Token: None 2024-04-16T18:17:46.200 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:46.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Gv3zQFY/yxwTha58o1yaizgPe4= 2024-04-16T18:17:46.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Gv3zQFY/yxwTha58o1yaizgPe4=', 'Content-Length': '0'} 2024-04-16T18:17:46.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002982ef4180485b4f-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.203 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.203 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.203 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.204 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.204 DEBUG:boto:Method: GET 2024-04-16T18:17:46.204 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.204 DEBUG:boto:Data: 2024-04-16T18:17:46.204 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.204 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.204 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.204 DEBUG:boto:Params: {} 2024-04-16T18:17:46.204 DEBUG:boto:Token: None 2024-04-16T18:17:46.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.209 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008062daf2329f6f2f-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.209 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.209 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.209 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.209 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.209 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.210 DEBUG:boto:Method: GET 2024-04-16T18:17:46.210 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:17:46.210 DEBUG:boto:Data: 2024-04-16T18:17:46.210 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.210 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.210 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.210 DEBUG:boto:Params: {} 2024-04-16T18:17:46.210 DEBUG:boto:Token: None 2024-04-16T18:17:46.210 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.210 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.214 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cfa0cc774d38031-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.215 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:17:34.055Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.215 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000011 2024-04-16T18:17:46.215 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.215 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.215 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.215 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.215 DEBUG:boto:Method: GET 2024-04-16T18:17:46.215 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.215 DEBUG:boto:Data: 2024-04-16T18:17:46.216 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.216 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.216 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.216 DEBUG:boto:Params: {} 2024-04-16T18:17:46.216 DEBUG:boto:Token: None 2024-04-16T18:17:46.216 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0= 2024-04-16T18:17:46.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0=', 'Content-Length': '0'} 2024-04-16T18:17:46.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1f8eff9d4c6169a-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.262 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.262 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.263 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.263 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.263 DEBUG:boto:Method: GET 2024-04-16T18:17:46.263 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.263 DEBUG:boto:Data: 2024-04-16T18:17:46.263 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.263 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.263 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.263 DEBUG:boto:Params: {} 2024-04-16T18:17:46.263 DEBUG:boto:Token: None 2024-04-16T18:17:46.263 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0= 2024-04-16T18:17:46.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0=', 'Content-Length': '0'} 2024-04-16T18:17:46.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034f5c4169262eb9f-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.267 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.267 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.267 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.267 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.267 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.267 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.267 DEBUG:boto:Data: 2024-04-16T18:17:46.267 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.267 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.268 DEBUG:boto:Params: {} 2024-04-16T18:17:46.268 DEBUG:boto:Token: None 2024-04-16T18:17:46.268 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KIoyUfyc/+RUZiK5eFXp4EatefA= 2024-04-16T18:17:46.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KIoyUfyc/+RUZiK5eFXp4EatefA=', 'Content-Length': '0'} 2024-04-16T18:17:46.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096b0bc17ff590d0c-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.272 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.272 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.272 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.272 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.272 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.272 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.272 DEBUG:boto:Data: 2024-04-16T18:17:46.272 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.272 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.272 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.272 DEBUG:boto:Params: {} 2024-04-16T18:17:46.273 DEBUG:boto:Token: None 2024-04-16T18:17:46.273 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KIoyUfyc/+RUZiK5eFXp4EatefA= 2024-04-16T18:17:46.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KIoyUfyc/+RUZiK5eFXp4EatefA=', 'Content-Length': '0'} 2024-04-16T18:17:46.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5a348336f27f07a-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.276 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000011 2024-04-16T18:17:46.276 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.276 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.276 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.276 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.276 DEBUG:boto:Method: GET 2024-04-16T18:17:46.277 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.277 DEBUG:boto:Data: 2024-04-16T18:17:46.277 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.277 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.277 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.277 DEBUG:boto:Params: {} 2024-04-16T18:17:46.277 DEBUG:boto:Token: None 2024-04-16T18:17:46.277 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0= 2024-04-16T18:17:46.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0=', 'Content-Length': '0'} 2024-04-16T18:17:46.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a79c66897bb0b95c-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.322 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.322 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.322 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.323 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.323 DEBUG:boto:Method: GET 2024-04-16T18:17:46.323 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.323 DEBUG:boto:Data: 2024-04-16T18:17:46.323 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.323 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.323 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.323 DEBUG:boto:Params: {} 2024-04-16T18:17:46.323 DEBUG:boto:Token: None 2024-04-16T18:17:46.323 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:17:46.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0= 2024-04-16T18:17:46.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wEXedaO+3GXT3FSsSn1ffPOThW0=', 'Content-Length': '0'} 2024-04-16T18:17:46.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9e8dbdd05d5047a-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.327 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.327 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.327 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.327 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.327 DEBUG:boto:Method: GET 2024-04-16T18:17:46.327 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.327 DEBUG:boto:Data: 2024-04-16T18:17:46.327 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.327 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.328 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.328 DEBUG:boto:Params: {} 2024-04-16T18:17:46.328 DEBUG:boto:Token: None 2024-04-16T18:17:46.328 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.332 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029780522e75312f3-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.374 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.375 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.375 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.375 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.375 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.375 DEBUG:boto:Method: GET 2024-04-16T18:17:46.375 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.375 DEBUG:boto:Data: 2024-04-16T18:17:46.375 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.375 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.375 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.376 DEBUG:boto:Params: {} 2024-04-16T18:17:46.376 DEBUG:boto:Token: None 2024-04-16T18:17:46.376 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000499cabe6f34237e0-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.380 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.380 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.380 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.380 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.380 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.380 DEBUG:boto:Data: 2024-04-16T18:17:46.381 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.381 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.381 DEBUG:boto:Params: {} 2024-04-16T18:17:46.381 DEBUG:boto:Token: None 2024-04-16T18:17:46.381 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.381 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qNY1MxbqqzdpYG807e8x6lKpvCU= 2024-04-16T18:17:46.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qNY1MxbqqzdpYG807e8x6lKpvCU=', 'Content-Length': '0'} 2024-04-16T18:17:46.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afe0581123ece981-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.385 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.385 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.385 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.385 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.385 DEBUG:boto:Data: 2024-04-16T18:17:46.385 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.385 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.385 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.385 DEBUG:boto:Params: {} 2024-04-16T18:17:46.386 DEBUG:boto:Token: None 2024-04-16T18:17:46.386 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.386 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qNY1MxbqqzdpYG807e8x6lKpvCU= 2024-04-16T18:17:46.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qNY1MxbqqzdpYG807e8x6lKpvCU=', 'Content-Length': '0'} 2024-04-16T18:17:46.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cff59de24ccdf0c6-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.389 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000011 2024-04-16T18:17:46.389 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.389 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.389 DEBUG:boto:Method: GET 2024-04-16T18:17:46.389 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.389 DEBUG:boto:Data: 2024-04-16T18:17:46.389 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.390 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.390 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.390 DEBUG:boto:Params: {} 2024-04-16T18:17:46.390 DEBUG:boto:Token: None 2024-04-16T18:17:46.390 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.390 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0w3GLHqcSyQJWlUoD+bGbY6tI8A= 2024-04-16T18:17:46.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0w3GLHqcSyQJWlUoD+bGbY6tI8A=', 'Content-Length': '0'} 2024-04-16T18:17:46.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000924928f9391fe637-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.434 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.435 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.435 DEBUG:boto:Method: GET 2024-04-16T18:17:46.435 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.435 DEBUG:boto:Data: 2024-04-16T18:17:46.435 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.435 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.435 DEBUG:boto:Params: {} 2024-04-16T18:17:46.435 DEBUG:boto:Token: None 2024-04-16T18:17:46.435 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:46.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0w3GLHqcSyQJWlUoD+bGbY6tI8A= 2024-04-16T18:17:46.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0w3GLHqcSyQJWlUoD+bGbY6tI8A=', 'Content-Length': '0'} 2024-04-16T18:17:46.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040a98839f1b8160a-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.439 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.439 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.439 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.439 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.439 DEBUG:boto:Method: GET 2024-04-16T18:17:46.439 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.439 DEBUG:boto:Data: 2024-04-16T18:17:46.440 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.440 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.440 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.440 DEBUG:boto:Params: {} 2024-04-16T18:17:46.440 DEBUG:boto:Token: None 2024-04-16T18:17:46.440 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093756518b2ad6706-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.486 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.487 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.487 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.487 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.487 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.487 DEBUG:boto:Method: GET 2024-04-16T18:17:46.487 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:17:46.487 DEBUG:boto:Data: 2024-04-16T18:17:46.487 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.487 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.487 DEBUG:boto:Params: {} 2024-04-16T18:17:46.488 DEBUG:boto:Token: None 2024-04-16T18:17:46.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.492 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025aac476c5e617af-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.492 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:17:33.900Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.493 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000012 2024-04-16T18:17:46.493 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.493 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.493 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.493 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.493 DEBUG:boto:Method: GET 2024-04-16T18:17:46.493 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.493 DEBUG:boto:Data: 2024-04-16T18:17:46.493 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.493 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.493 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.493 DEBUG:boto:Params: {} 2024-04-16T18:17:46.494 DEBUG:boto:Token: None 2024-04-16T18:17:46.494 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.494 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w= 2024-04-16T18:17:46.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w=', 'Content-Length': '0'} 2024-04-16T18:17:46.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fe78258f09ffd92-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.538 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.538 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.539 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.539 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.539 DEBUG:boto:Method: GET 2024-04-16T18:17:46.539 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.539 DEBUG:boto:Data: 2024-04-16T18:17:46.539 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.539 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.539 DEBUG:boto:Params: {} 2024-04-16T18:17:46.539 DEBUG:boto:Token: None 2024-04-16T18:17:46.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w= 2024-04-16T18:17:46.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w=', 'Content-Length': '0'} 2024-04-16T18:17:46.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d24bfbccabd3d6cf-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.543 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.543 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.543 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.543 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.543 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.543 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.544 DEBUG:boto:Data: 2024-04-16T18:17:46.544 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.544 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.544 DEBUG:boto:Params: {} 2024-04-16T18:17:46.544 DEBUG:boto:Token: None 2024-04-16T18:17:46.544 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GTAuaGAfwJtkVeelY4kllWa/ZRE= 2024-04-16T18:17:46.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GTAuaGAfwJtkVeelY4kllWa/ZRE=', 'Content-Length': '0'} 2024-04-16T18:17:46.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b057c9e1e5b1c366-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.548 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.548 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.548 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.548 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.548 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.548 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.548 DEBUG:boto:Data: 2024-04-16T18:17:46.548 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.548 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.548 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.549 DEBUG:boto:Params: {} 2024-04-16T18:17:46.549 DEBUG:boto:Token: None 2024-04-16T18:17:46.549 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GTAuaGAfwJtkVeelY4kllWa/ZRE= 2024-04-16T18:17:46.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GTAuaGAfwJtkVeelY4kllWa/ZRE=', 'Content-Length': '0'} 2024-04-16T18:17:46.552 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7cb56fd4e4df98e-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.552 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000012 2024-04-16T18:17:46.552 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.552 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.552 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.552 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.552 DEBUG:boto:Method: GET 2024-04-16T18:17:46.553 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.553 DEBUG:boto:Data: 2024-04-16T18:17:46.553 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.553 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.553 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.553 DEBUG:boto:Params: {} 2024-04-16T18:17:46.553 DEBUG:boto:Token: None 2024-04-16T18:17:46.553 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w= 2024-04-16T18:17:46.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w=', 'Content-Length': '0'} 2024-04-16T18:17:46.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009516dcab4b89f08b-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.598 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.598 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.599 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.599 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.599 DEBUG:boto:Method: GET 2024-04-16T18:17:46.599 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.599 DEBUG:boto:Data: 2024-04-16T18:17:46.599 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.599 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.599 DEBUG:boto:Params: {} 2024-04-16T18:17:46.599 DEBUG:boto:Token: None 2024-04-16T18:17:46.600 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:17:46.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w= 2024-04-16T18:17:46.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zZj2rEt9lmi2n9PdkZsw5gF7T/w=', 'Content-Length': '0'} 2024-04-16T18:17:46.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e97c18b3fe91853-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.603 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.603 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.603 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.603 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.603 DEBUG:boto:Method: GET 2024-04-16T18:17:46.603 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.604 DEBUG:boto:Data: 2024-04-16T18:17:46.604 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.604 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.604 DEBUG:boto:Params: {} 2024-04-16T18:17:46.604 DEBUG:boto:Token: None 2024-04-16T18:17:46.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004885430a27d4cfdf-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.608 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.609 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.609 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.609 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.609 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.609 DEBUG:boto:Method: GET 2024-04-16T18:17:46.609 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.609 DEBUG:boto:Data: 2024-04-16T18:17:46.609 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.610 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.610 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.610 DEBUG:boto:Params: {} 2024-04-16T18:17:46.610 DEBUG:boto:Token: None 2024-04-16T18:17:46.610 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.610 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.614 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005291e30d855ce5b1-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.654 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.655 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.655 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.655 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.655 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.655 DEBUG:boto:Data: 2024-04-16T18:17:46.655 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.655 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.655 DEBUG:boto:Params: {} 2024-04-16T18:17:46.655 DEBUG:boto:Token: None 2024-04-16T18:17:46.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fcZZOVTafbrpnJpYvAVkUN//tvs= 2024-04-16T18:17:46.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fcZZOVTafbrpnJpYvAVkUN//tvs=', 'Content-Length': '0'} 2024-04-16T18:17:46.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e23ca5f6ee78d6f2-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.659 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.659 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.659 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.659 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.659 DEBUG:boto:Data: 2024-04-16T18:17:46.660 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.660 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.660 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.660 DEBUG:boto:Params: {} 2024-04-16T18:17:46.660 DEBUG:boto:Token: None 2024-04-16T18:17:46.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fcZZOVTafbrpnJpYvAVkUN//tvs= 2024-04-16T18:17:46.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fcZZOVTafbrpnJpYvAVkUN//tvs=', 'Content-Length': '0'} 2024-04-16T18:17:46.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059aa36ad052fb793-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000012 2024-04-16T18:17:46.664 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.664 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.664 DEBUG:boto:Method: GET 2024-04-16T18:17:46.664 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.664 DEBUG:boto:Data: 2024-04-16T18:17:46.664 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.664 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.664 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.664 DEBUG:boto:Params: {} 2024-04-16T18:17:46.664 DEBUG:boto:Token: None 2024-04-16T18:17:46.664 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.665 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G3EAhoCQ7vr6Fwv76JXcK+76OEM= 2024-04-16T18:17:46.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G3EAhoCQ7vr6Fwv76JXcK+76OEM=', 'Content-Length': '0'} 2024-04-16T18:17:46.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d405f11cd41cf35a-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.710 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.710 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.710 DEBUG:boto:Method: GET 2024-04-16T18:17:46.711 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.711 DEBUG:boto:Data: 2024-04-16T18:17:46.711 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.711 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.711 DEBUG:boto:Params: {} 2024-04-16T18:17:46.711 DEBUG:boto:Token: None 2024-04-16T18:17:46.711 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:46.711 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G3EAhoCQ7vr6Fwv76JXcK+76OEM= 2024-04-16T18:17:46.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G3EAhoCQ7vr6Fwv76JXcK+76OEM=', 'Content-Length': '0'} 2024-04-16T18:17:46.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:33 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2482de5b85d5eab-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.714 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.714 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.715 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.715 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.715 DEBUG:boto:Method: GET 2024-04-16T18:17:46.715 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.715 DEBUG:boto:Data: 2024-04-16T18:17:46.715 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.715 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.715 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.715 DEBUG:boto:Params: {} 2024-04-16T18:17:46.715 DEBUG:boto:Token: None 2024-04-16T18:17:46.715 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d8b8729f07ff018-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.762 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.763 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.763 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.763 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.763 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.763 DEBUG:boto:Method: GET 2024-04-16T18:17:46.763 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:17:46.763 DEBUG:boto:Data: 2024-04-16T18:17:46.763 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.763 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.763 DEBUG:boto:Params: {} 2024-04-16T18:17:46.763 DEBUG:boto:Token: None 2024-04-16T18:17:46.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000785ef30861d15dab-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.768 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:17:33.831Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.768 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000013 2024-04-16T18:17:46.768 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.768 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.768 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.768 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.769 DEBUG:boto:Method: GET 2024-04-16T18:17:46.769 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.769 DEBUG:boto:Data: 2024-04-16T18:17:46.769 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.769 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.769 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.769 DEBUG:boto:Params: {} 2024-04-16T18:17:46.769 DEBUG:boto:Token: None 2024-04-16T18:17:46.769 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.769 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA= 2024-04-16T18:17:46.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA=', 'Content-Length': '0'} 2024-04-16T18:17:46.772 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a53bbcdd54ec658e-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.814 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.814 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.814 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.815 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.815 DEBUG:boto:Method: GET 2024-04-16T18:17:46.815 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.815 DEBUG:boto:Data: 2024-04-16T18:17:46.815 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.815 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.815 DEBUG:boto:Params: {} 2024-04-16T18:17:46.815 DEBUG:boto:Token: None 2024-04-16T18:17:46.815 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA= 2024-04-16T18:17:46.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA=', 'Content-Length': '0'} 2024-04-16T18:17:46.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f6f0bb4e9176336-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.819 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.819 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.819 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.819 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.819 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.819 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.819 DEBUG:boto:Data: 2024-04-16T18:17:46.819 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.819 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.820 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.820 DEBUG:boto:Params: {} 2024-04-16T18:17:46.820 DEBUG:boto:Token: None 2024-04-16T18:17:46.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rb61ZyK4npy/8/TijAmCwLxEHx0= 2024-04-16T18:17:46.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rb61ZyK4npy/8/TijAmCwLxEHx0=', 'Content-Length': '0'} 2024-04-16T18:17:46.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b08645393c3eefbb-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.823 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.823 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.823 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.823 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.823 DEBUG:boto:Data: 2024-04-16T18:17:46.824 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.824 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.824 DEBUG:boto:Params: {} 2024-04-16T18:17:46.824 DEBUG:boto:Token: None 2024-04-16T18:17:46.824 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rb61ZyK4npy/8/TijAmCwLxEHx0= 2024-04-16T18:17:46.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rb61ZyK4npy/8/TijAmCwLxEHx0=', 'Content-Length': '0'} 2024-04-16T18:17:46.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ef2c98c8f527fb0-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.827 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000013 2024-04-16T18:17:46.827 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.828 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.828 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.828 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.828 DEBUG:boto:Method: GET 2024-04-16T18:17:46.828 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.828 DEBUG:boto:Data: 2024-04-16T18:17:46.828 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.828 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.828 DEBUG:boto:Params: {} 2024-04-16T18:17:46.828 DEBUG:boto:Token: None 2024-04-16T18:17:46.829 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA= 2024-04-16T18:17:46.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA=', 'Content-Length': '0'} 2024-04-16T18:17:46.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce31a72482e5620a-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.874 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.874 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.875 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.875 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.875 DEBUG:boto:Method: GET 2024-04-16T18:17:46.875 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.875 DEBUG:boto:Data: 2024-04-16T18:17:46.875 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.875 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.875 DEBUG:boto:Params: {} 2024-04-16T18:17:46.875 DEBUG:boto:Token: None 2024-04-16T18:17:46.875 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:17:46.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA= 2024-04-16T18:17:46.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9+Y+PKySv77k29jYc8Gd5vCY3qA=', 'Content-Length': '0'} 2024-04-16T18:17:46.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce340e14be43f36c-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.879 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.879 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.879 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.879 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.879 DEBUG:boto:Method: GET 2024-04-16T18:17:46.879 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.879 DEBUG:boto:Data: 2024-04-16T18:17:46.879 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.879 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.879 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.879 DEBUG:boto:Params: {} 2024-04-16T18:17:46.880 DEBUG:boto:Token: None 2024-04-16T18:17:46.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065c1f8032bfcea7c-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.884 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.884 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.885 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.885 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.885 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.885 DEBUG:boto:Method: GET 2024-04-16T18:17:46.885 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.885 DEBUG:boto:Data: 2024-04-16T18:17:46.885 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.885 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.885 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.885 DEBUG:boto:Params: {} 2024-04-16T18:17:46.885 DEBUG:boto:Token: None 2024-04-16T18:17:46.885 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.889 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c031319c3a9e80b8-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.930 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:46.931 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.931 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.931 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.931 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.931 DEBUG:boto:Data: 2024-04-16T18:17:46.931 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.931 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.931 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.931 DEBUG:boto:Params: {} 2024-04-16T18:17:46.931 DEBUG:boto:Token: None 2024-04-16T18:17:46.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sr2piBPkYCoqBX7MRC8qYgspi8o= 2024-04-16T18:17:46.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sr2piBPkYCoqBX7MRC8qYgspi8o=', 'Content-Length': '0'} 2024-04-16T18:17:46.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ffe97ef87581dd8-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.935 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.935 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.935 DEBUG:boto:Method: HEAD 2024-04-16T18:17:46.935 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.935 DEBUG:boto:Data: 2024-04-16T18:17:46.935 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.935 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.935 DEBUG:boto:Params: {} 2024-04-16T18:17:46.935 DEBUG:boto:Token: None 2024-04-16T18:17:46.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sr2piBPkYCoqBX7MRC8qYgspi8o= 2024-04-16T18:17:46.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sr2piBPkYCoqBX7MRC8qYgspi8o=', 'Content-Length': '0'} 2024-04-16T18:17:46.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002aa7a6861ead63c8-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.939 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000013 2024-04-16T18:17:46.939 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.939 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.939 DEBUG:boto:Method: GET 2024-04-16T18:17:46.939 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.939 DEBUG:boto:Data: 2024-04-16T18:17:46.939 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.939 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.940 DEBUG:boto:Params: {} 2024-04-16T18:17:46.940 DEBUG:boto:Token: None 2024-04-16T18:17:46.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7YF70UcgTIbcwHt4qOxAGlDzYc= 2024-04-16T18:17:46.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7YF70UcgTIbcwHt4qOxAGlDzYc=', 'Content-Length': '0'} 2024-04-16T18:17:46.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4485965a2ac948b-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.986 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.986 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.987 DEBUG:boto:Method: GET 2024-04-16T18:17:46.987 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.987 DEBUG:boto:Data: 2024-04-16T18:17:46.987 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.987 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:46.987 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.987 DEBUG:boto:Params: {} 2024-04-16T18:17:46.987 DEBUG:boto:Token: None 2024-04-16T18:17:46.987 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:46.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7YF70UcgTIbcwHt4qOxAGlDzYc= 2024-04-16T18:17:46.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7YF70UcgTIbcwHt4qOxAGlDzYc=', 'Content-Length': '0'} 2024-04-16T18:17:46.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:34 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb1c5ef6facf9bb2-00661ec0ca-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:46.990 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:46.990 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:46.990 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.991 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.991 DEBUG:boto:Method: GET 2024-04-16T18:17:46.991 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:46.991 DEBUG:boto:Data: 2024-04-16T18:17:46.991 DEBUG:boto:Headers: {} 2024-04-16T18:17:46.991 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:17:46.991 DEBUG:boto:Port: 8000 2024-04-16T18:17:46.991 DEBUG:boto:Params: {} 2024-04-16T18:17:46.991 DEBUG:boto:Token: None 2024-04-16T18:17:46.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:46 GMT /lybtug-51/ 2024-04-16T18:17:46.991 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c= 2024-04-16T18:17:46.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HaDG2EtxSO4yV0Ozk50rGnoxA+c=', 'Content-Length': '0'} 2024-04-16T18:17:46.995 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf81735d39775537-00661ec0ca-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:47.038 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:47.039 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:47.082 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:47.082 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:47.082 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:47.082 DEBUG:boto:Method: GET 2024-04-16T18:17:47.082 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:17:47.082 DEBUG:boto:Data: 2024-04-16T18:17:47.082 DEBUG:boto:Headers: {} 2024-04-16T18:17:47.082 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:47.083 DEBUG:boto:Port: 8000 2024-04-16T18:17:47.083 DEBUG:boto:Params: {} 2024-04-16T18:17:47.083 DEBUG:boto:Token: None 2024-04-16T18:17:47.083 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:17:47 GMT /lybtug-51/ 2024-04-16T18:17:47.083 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZrymZFMRD4fJ8MWKZ+c6o+kfKGM= 2024-04-16T18:17:47.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZrymZFMRD4fJ8MWKZ+c6o+kfKGM=', 'Content-Length': '0'} 2024-04-16T18:17:47.087 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038f437066ec0f918-00661ec0cb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:47.087 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:34.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:17:47.087 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-51 zones={test-zone2, test-zone1} 2024-04-16T18:17:47.087 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2024-04-16T18:17:47.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T18:17:47.089 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:47.092+0000 7fbe7b56a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 119704) UID: 0 2024-04-16T18:17:47.089 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:17:47.092+0000 7fbe7edb2a40 -1 shutting down 2024-04-16T18:17:52.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T18:17:53.188 INFO:tasks.rgw.c2.client.0:Stopped 2024-04-16T18:17:53.188 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '17', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:53.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '17', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:53.189 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --num-shards 17 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:53.238 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:53.339 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:53.339 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-51 2024-04-16T18:17:53.348 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 10 2024-04-16T18:17:53.433 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:53.433 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:53.434 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:53.573 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000007","obj00000000000000000016","obj00000000000000000024","obj00000000000000000020","obj00000000000000000028","obj00000000000000000000","obj00000000000000000032","obj00000000000000000001","obj00000000000000000010","obj00000000000000000019","obj00000000000000000013","obj00000000000000000021","obj00000000000000000030","obj00000000000000000026","obj00000000000000000033","obj00000000000000000004","obj00000000000000000076"]} 2024-04-16T18:17:53.573 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.573 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.574 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.574 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.574 DEBUG:boto:Data: 2024-04-16T18:17:53.574 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.574 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.574 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.574 DEBUG:boto:Params: {} 2024-04-16T18:17:53.574 DEBUG:boto:Token: None 2024-04-16T18:17:53.574 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.575 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.578 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000051dd97ec8ecd4a6e-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.579 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:53.579 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:53.579 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.579 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:53.579 DEBUG:boto:Data: 2024-04-16T18:17:53.579 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.579 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.579 DEBUG:boto:Params: {} 2024-04-16T18:17:53.579 DEBUG:boto:Token: None 2024-04-16T18:17:53.580 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:53.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dvn3k44y6g6GTqUpKHijmelwvOw= 2024-04-16T18:17:53.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dvn3k44y6g6GTqUpKHijmelwvOw='} 2024-04-16T18:17:53.592 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e206164af32e4b02-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.592 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.592 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.592 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.592 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.592 DEBUG:boto:Data: 2024-04-16T18:17:53.592 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.592 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.593 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.593 DEBUG:boto:Params: {} 2024-04-16T18:17:53.593 DEBUG:boto:Token: None 2024-04-16T18:17:53.593 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.593 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.597 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000086a9f9e5e8054428-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.597 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:17:53.597 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:17:53.597 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.597 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:17:53.597 DEBUG:boto:Data: 2024-04-16T18:17:53.597 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.597 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.597 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.597 DEBUG:boto:Params: {} 2024-04-16T18:17:53.598 DEBUG:boto:Token: None 2024-04-16T18:17:53.598 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:17:53.598 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g2pUkyZF73qitEEZDRGD7Nm2uAQ= 2024-04-16T18:17:53.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g2pUkyZF73qitEEZDRGD7Nm2uAQ='} 2024-04-16T18:17:53.642 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000007ea755243cf2978-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.643 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.643 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.643 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.643 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.643 DEBUG:boto:Data: 2024-04-16T18:17:53.643 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.643 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.643 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.643 DEBUG:boto:Params: {} 2024-04-16T18:17:53.644 DEBUG:boto:Token: None 2024-04-16T18:17:53.644 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.647 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '33'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001f380327afcec33c-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.647 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:17:53.648 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:17:53.648 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.648 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:17:53.648 DEBUG:boto:Data: 2024-04-16T18:17:53.648 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.648 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.648 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.648 DEBUG:boto:Params: {} 2024-04-16T18:17:53.648 DEBUG:boto:Token: None 2024-04-16T18:17:53.649 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:17:53.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EN+3OKbjXUDQvGKfo87w0UBpZdM= 2024-04-16T18:17:53.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EN+3OKbjXUDQvGKfo87w0UBpZdM='} 2024-04-16T18:17:53.659 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ba20c9ec94d12ff8-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.659 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.659 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.659 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.659 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.659 DEBUG:boto:Data: 2024-04-16T18:17:53.659 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.659 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.659 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.660 DEBUG:boto:Params: {} 2024-04-16T18:17:53.660 DEBUG:boto:Token: None 2024-04-16T18:17:53.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.663 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002fda6af719e004c7-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.664 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:17:53.664 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:17:53.664 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.664 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076 2024-04-16T18:17:53.664 DEBUG:boto:Data: 2024-04-16T18:17:53.664 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.664 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.664 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.664 DEBUG:boto:Params: {} 2024-04-16T18:17:53.664 DEBUG:boto:Token: None 2024-04-16T18:17:53.665 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000076 2024-04-16T18:17:53.665 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3NSca740LfbZgUXV2HCon1N4vTc= 2024-04-16T18:17:53.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3NSca740LfbZgUXV2HCon1N4vTc='} 2024-04-16T18:17:53.706 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000060960b46b3260131-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.707 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.707 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.707 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.707 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.707 DEBUG:boto:Data: 2024-04-16T18:17:53.707 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.707 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.707 DEBUG:boto:Params: {} 2024-04-16T18:17:53.707 DEBUG:boto:Token: None 2024-04-16T18:17:53.708 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.711 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '39'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000077f5840d4f8cd7b-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.712 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:53.712 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:53.712 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.712 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:17:53.712 DEBUG:boto:Data: 2024-04-16T18:17:53.712 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.712 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.712 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.712 DEBUG:boto:Params: {} 2024-04-16T18:17:53.712 DEBUG:boto:Token: None 2024-04-16T18:17:53.713 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:17:53.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UkLVOizxDCusckl1RvWJJaqacgY= 2024-04-16T18:17:53.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UkLVOizxDCusckl1RvWJJaqacgY='} 2024-04-16T18:17:53.725 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006a998c4de3302e22-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.725 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.725 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.726 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.726 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.726 DEBUG:boto:Data: 2024-04-16T18:17:53.726 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.726 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.726 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.726 DEBUG:boto:Params: {} 2024-04-16T18:17:53.726 DEBUG:boto:Token: None 2024-04-16T18:17:53.726 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.726 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.730 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006f7ceaa2262e736d-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.730 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:53.730 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:53.731 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.731 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:53.731 DEBUG:boto:Data: 2024-04-16T18:17:53.731 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.731 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.731 DEBUG:boto:Params: {} 2024-04-16T18:17:53.731 DEBUG:boto:Token: None 2024-04-16T18:17:53.731 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:53.731 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1zxyorcdmxa1bSOZ434xbPgjiQI= 2024-04-16T18:17:53.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1zxyorcdmxa1bSOZ434xbPgjiQI='} 2024-04-16T18:17:53.774 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000aeaeec4381799661-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.775 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.775 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.775 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.775 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.775 DEBUG:boto:Data: 2024-04-16T18:17:53.775 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.775 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.775 DEBUG:boto:Params: {} 2024-04-16T18:17:53.775 DEBUG:boto:Token: None 2024-04-16T18:17:53.776 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.780 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006b4ed27fda820f50-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.780 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:53.780 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:53.780 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.780 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:17:53.780 DEBUG:boto:Data: 2024-04-16T18:17:53.781 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.781 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.781 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.781 DEBUG:boto:Params: {} 2024-04-16T18:17:53.781 DEBUG:boto:Token: None 2024-04-16T18:17:53.781 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:17:53.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gvdCDT/jc1P03BGymxs4Da40G4E= 2024-04-16T18:17:53.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gvdCDT/jc1P03BGymxs4Da40G4E='} 2024-04-16T18:17:53.793 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000da65e87c477059ff-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.793 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.793 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.793 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.793 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.793 DEBUG:boto:Data: 2024-04-16T18:17:53.793 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.794 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.794 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.794 DEBUG:boto:Params: {} 2024-04-16T18:17:53.794 DEBUG:boto:Token: None 2024-04-16T18:17:53.794 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.794 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.798 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000afa04899c97f927-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.798 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:17:53.798 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:17:53.798 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.798 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:17:53.798 DEBUG:boto:Data: 2024-04-16T18:17:53.798 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.798 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.799 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.799 DEBUG:boto:Params: {} 2024-04-16T18:17:53.799 DEBUG:boto:Token: None 2024-04-16T18:17:53.799 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:17:53.799 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Sa9nukiOsv9wgi0U+iTloXxYS1I= 2024-04-16T18:17:53.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sa9nukiOsv9wgi0U+iTloXxYS1I='} 2024-04-16T18:17:53.842 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000026ae4755dce20a2f-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.843 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.843 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.843 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.843 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.843 DEBUG:boto:Data: 2024-04-16T18:17:53.843 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.843 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.843 DEBUG:boto:Params: {} 2024-04-16T18:17:53.844 DEBUG:boto:Token: None 2024-04-16T18:17:53.844 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.848 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000022344b55a5ab2d98-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.848 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:53.848 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:53.848 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.848 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:17:53.848 DEBUG:boto:Data: 2024-04-16T18:17:53.848 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.849 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.849 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.849 DEBUG:boto:Params: {} 2024-04-16T18:17:53.849 DEBUG:boto:Token: None 2024-04-16T18:17:53.849 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:17:53.849 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LjCvOcRlM9iw+zcxthckVu7eOyo= 2024-04-16T18:17:53.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LjCvOcRlM9iw+zcxthckVu7eOyo='} 2024-04-16T18:17:53.862 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000aba3f68f04932f95-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.862 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.862 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.862 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.862 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.863 DEBUG:boto:Data: 2024-04-16T18:17:53.863 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.863 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.863 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.863 DEBUG:boto:Params: {} 2024-04-16T18:17:53.863 DEBUG:boto:Token: None 2024-04-16T18:17:53.863 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.867 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '51'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000850edc0b10e79034-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.868 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:17:53.868 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:17:53.868 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.868 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032 2024-04-16T18:17:53.868 DEBUG:boto:Data: 2024-04-16T18:17:53.868 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.868 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.868 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.868 DEBUG:boto:Params: {} 2024-04-16T18:17:53.868 DEBUG:boto:Token: None 2024-04-16T18:17:53.869 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000032 2024-04-16T18:17:53.869 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C/0OMUUKJ6JQlP+xQoRaL2qB6Ag= 2024-04-16T18:17:53.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C/0OMUUKJ6JQlP+xQoRaL2qB6Ag='} 2024-04-16T18:17:53.910 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000612656e9b39047e3-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.911 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.911 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.911 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.911 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.911 DEBUG:boto:Data: 2024-04-16T18:17:53.911 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.911 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.911 DEBUG:boto:Params: {} 2024-04-16T18:17:53.912 DEBUG:boto:Token: None 2024-04-16T18:17:53.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.916 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000daa0e4480756458e-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.916 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:17:53.916 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:17:53.916 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.916 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028 2024-04-16T18:17:53.916 DEBUG:boto:Data: 2024-04-16T18:17:53.916 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.917 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.917 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.917 DEBUG:boto:Params: {} 2024-04-16T18:17:53.917 DEBUG:boto:Token: None 2024-04-16T18:17:53.917 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000028 2024-04-16T18:17:53.917 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LJRLQ0gHFBYX59DSbeXVzOPf8IE= 2024-04-16T18:17:53.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LJRLQ0gHFBYX59DSbeXVzOPf8IE='} 2024-04-16T18:17:53.930 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001764533a9997580d-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.930 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.931 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.931 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.931 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.931 DEBUG:boto:Data: 2024-04-16T18:17:53.931 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.931 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.931 DEBUG:boto:Params: {} 2024-04-16T18:17:53.931 DEBUG:boto:Token: None 2024-04-16T18:17:53.931 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.936 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '57'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000af80a73710c43c1b-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.936 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:17:53.936 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:17:53.936 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.936 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:17:53.936 DEBUG:boto:Data: 2024-04-16T18:17:53.936 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.937 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.937 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.937 DEBUG:boto:Params: {} 2024-04-16T18:17:53.937 DEBUG:boto:Token: None 2024-04-16T18:17:53.937 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:17:53.937 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TZQD+Y1yg9kmkTHcO1WdcggSgGU= 2024-04-16T18:17:53.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TZQD+Y1yg9kmkTHcO1WdcggSgGU='} 2024-04-16T18:17:53.982 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000099893c4d9b8d170-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.983 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:53.983 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:53.983 DEBUG:boto:Method: HEAD 2024-04-16T18:17:53.983 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:53.983 DEBUG:boto:Data: 2024-04-16T18:17:53.983 DEBUG:boto:Headers: {} 2024-04-16T18:17:53.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.983 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.983 DEBUG:boto:Params: {} 2024-04-16T18:17:53.984 DEBUG:boto:Token: None 2024-04-16T18:17:53.984 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/ 2024-04-16T18:17:53.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s= 2024-04-16T18:17:53.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvYCd3D6rFcAhmWPoI8r7uLZ7s=', 'Content-Length': '0'} 2024-04-16T18:17:53.988 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004c633ecbddd3dfeb-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:53.988 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:53.988 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:53.988 DEBUG:boto:Method: PUT 2024-04-16T18:17:53.988 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:53.988 DEBUG:boto:Data: 2024-04-16T18:17:53.988 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:53.988 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:53.989 DEBUG:boto:Port: 8000 2024-04-16T18:17:53.989 DEBUG:boto:Params: {} 2024-04-16T18:17:53.989 DEBUG:boto:Token: None 2024-04-16T18:17:53.989 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:53 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:53.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kr+1zZnmoxbuvK43FvHSDOaKPYg= 2024-04-16T18:17:53.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr+1zZnmoxbuvK43FvHSDOaKPYg='} 2024-04-16T18:17:54.006 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bc31da68692f69ef-00661ec0d1-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.006 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.006 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.007 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.007 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.007 DEBUG:boto:Data: 2024-04-16T18:17:54.007 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.007 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.007 DEBUG:boto:Params: {} 2024-04-16T18:17:54.007 DEBUG:boto:Token: None 2024-04-16T18:17:54.007 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.007 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.011 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e6843920cb77e37a-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.011 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:17:54.011 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:17:54.012 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.012 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024 2024-04-16T18:17:54.012 DEBUG:boto:Data: 2024-04-16T18:17:54.012 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.012 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.012 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.012 DEBUG:boto:Params: {} 2024-04-16T18:17:54.012 DEBUG:boto:Token: None 2024-04-16T18:17:54.012 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000024 2024-04-16T18:17:54.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BnOmNJPorp5hkhYnHQTRHrbiKt4= 2024-04-16T18:17:54.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BnOmNJPorp5hkhYnHQTRHrbiKt4='} 2024-04-16T18:17:54.054 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ff6898b75a65e3a9-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.055 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.055 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.055 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.055 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.055 DEBUG:boto:Data: 2024-04-16T18:17:54.055 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.055 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.055 DEBUG:boto:Params: {} 2024-04-16T18:17:54.055 DEBUG:boto:Token: None 2024-04-16T18:17:54.056 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.059 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '63'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ed2476743535e27-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.060 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:54.060 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:54.060 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.060 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:54.060 DEBUG:boto:Data: 2024-04-16T18:17:54.060 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.060 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.060 DEBUG:boto:Params: {} 2024-04-16T18:17:54.061 DEBUG:boto:Token: None 2024-04-16T18:17:54.061 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:54.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T4FYaKb/6V10W3cqhXGyH2Hyics= 2024-04-16T18:17:54.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T4FYaKb/6V10W3cqhXGyH2Hyics='} 2024-04-16T18:17:54.077 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004846a8fa8ce658a4-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.077 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.078 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.078 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.078 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.078 DEBUG:boto:Data: 2024-04-16T18:17:54.078 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.078 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.078 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.078 DEBUG:boto:Params: {} 2024-04-16T18:17:54.078 DEBUG:boto:Token: None 2024-04-16T18:17:54.078 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.079 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.082 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '63'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fe374a0dc6e468f1-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.083 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.083 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.083 DEBUG:boto:Data: 2024-04-16T18:17:54.083 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.083 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.083 DEBUG:boto:Params: {} 2024-04-16T18:17:54.083 DEBUG:boto:Token: None 2024-04-16T18:17:54.084 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0S5PIsXgwjxqjXQsuY9CkwDhjBE= 2024-04-16T18:17:54.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0S5PIsXgwjxqjXQsuY9CkwDhjBE='} 2024-04-16T18:17:54.126 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007691942144e3b80c-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.127 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '19', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:54.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '19', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:54.127 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --num-shards 19 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:54.173 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:54.173 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:54.270 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:54.270 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-51 2024-04-16T18:17:54.285 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 22 2024-04-16T18:17:54.413 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:54.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:54.414 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:54.500 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:54.500 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:54.598 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000005","obj00000000000000000010","obj00000000000000000016","obj00000000000000000021","obj00000000000000000004","obj00000000000000000025","obj00000000000000000030","obj00000000000000000015","obj00000000000000000019","obj00000000000000000003","obj00000000000000000009","obj00000000000000000014","obj00000000000000000012","obj00000000000000000002","obj00000000000000000000","obj00000000000000000006","obj00000000000000000011","obj00000000000000000001","obj00000000000000000007"]} 2024-04-16T18:17:54.599 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.599 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.599 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.599 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.599 DEBUG:boto:Data: 2024-04-16T18:17:54.599 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.599 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.599 DEBUG:boto:Params: {} 2024-04-16T18:17:54.599 DEBUG:boto:Token: None 2024-04-16T18:17:54.600 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.603 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000203b2b06981f67bc-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.604 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:17:54.604 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:17:54.604 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.604 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:17:54.604 DEBUG:boto:Data: 2024-04-16T18:17:54.604 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.604 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.604 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.604 DEBUG:boto:Params: {} 2024-04-16T18:17:54.605 DEBUG:boto:Token: None 2024-04-16T18:17:54.605 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:17:54.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RMREe+/9/Mh8HNpF1KedDyYvX2s= 2024-04-16T18:17:54.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RMREe+/9/Mh8HNpF1KedDyYvX2s='} 2024-04-16T18:17:54.614 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fffe2bef0ff81a73-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.615 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.615 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.615 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.615 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.615 DEBUG:boto:Data: 2024-04-16T18:17:54.615 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.615 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.615 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.615 DEBUG:boto:Params: {} 2024-04-16T18:17:54.615 DEBUG:boto:Token: None 2024-04-16T18:17:54.616 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.619 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009a2825ac9ae4d4ad-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.619 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:54.619 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:54.619 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.619 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:17:54.619 DEBUG:boto:Data: 2024-04-16T18:17:54.619 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.619 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.620 DEBUG:boto:Params: {} 2024-04-16T18:17:54.620 DEBUG:boto:Token: None 2024-04-16T18:17:54.620 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:17:54.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NYtBBflSQwmXxAbjZSSnucttYSs= 2024-04-16T18:17:54.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NYtBBflSQwmXxAbjZSSnucttYSs='} 2024-04-16T18:17:54.662 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000031991d5430452c73-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.662 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.663 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.663 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.663 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.663 DEBUG:boto:Data: 2024-04-16T18:17:54.663 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.663 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.663 DEBUG:boto:Params: {} 2024-04-16T18:17:54.663 DEBUG:boto:Token: None 2024-04-16T18:17:54.663 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.668 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000908a686b9b739160-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.668 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:54.668 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:54.668 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.668 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:54.668 DEBUG:boto:Data: 2024-04-16T18:17:54.668 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.668 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.669 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.669 DEBUG:boto:Params: {} 2024-04-16T18:17:54.669 DEBUG:boto:Token: None 2024-04-16T18:17:54.669 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:54.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PajAIi6gQbnmt8yKMnOiwDGu/J0= 2024-04-16T18:17:54.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PajAIi6gQbnmt8yKMnOiwDGu/J0='} 2024-04-16T18:17:54.683 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d0b4354a0b5fabf4-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.684 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.684 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.684 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.684 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.684 DEBUG:boto:Data: 2024-04-16T18:17:54.684 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.684 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.684 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.684 DEBUG:boto:Params: {} 2024-04-16T18:17:54.684 DEBUG:boto:Token: None 2024-04-16T18:17:54.685 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.688 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001f8c95be8f446839-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.689 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:54.689 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:54.689 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.689 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:54.689 DEBUG:boto:Data: 2024-04-16T18:17:54.689 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.689 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.689 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.689 DEBUG:boto:Params: {} 2024-04-16T18:17:54.689 DEBUG:boto:Token: None 2024-04-16T18:17:54.690 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:54.690 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OHMeMP9edyvfeQtupzIYw7NKHJE= 2024-04-16T18:17:54.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OHMeMP9edyvfeQtupzIYw7NKHJE='} 2024-04-16T18:17:54.734 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004420431bb3abc4ec-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.734 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.735 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.735 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.735 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.735 DEBUG:boto:Data: 2024-04-16T18:17:54.735 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.735 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.735 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.735 DEBUG:boto:Params: {} 2024-04-16T18:17:54.735 DEBUG:boto:Token: None 2024-04-16T18:17:54.736 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.740 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ace2705ad8f5c9d4-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.740 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:54.741 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:54.741 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.741 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:54.741 DEBUG:boto:Data: 2024-04-16T18:17:54.741 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.741 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.741 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.741 DEBUG:boto:Params: {} 2024-04-16T18:17:54.741 DEBUG:boto:Token: None 2024-04-16T18:17:54.742 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:54.742 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DVW5xvepujw345sUO3n53COJkiU= 2024-04-16T18:17:54.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DVW5xvepujw345sUO3n53COJkiU='} 2024-04-16T18:17:54.757 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009332abb67a8668c8-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.757 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.758 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.758 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.758 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.758 DEBUG:boto:Data: 2024-04-16T18:17:54.758 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.758 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.758 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.758 DEBUG:boto:Params: {} 2024-04-16T18:17:54.758 DEBUG:boto:Token: None 2024-04-16T18:17:54.758 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.759 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.763 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000c5d5a3f582ae330-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.764 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.764 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.764 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.764 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.764 DEBUG:boto:Data: 2024-04-16T18:17:54.764 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.764 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.765 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.765 DEBUG:boto:Params: {} 2024-04-16T18:17:54.765 DEBUG:boto:Token: None 2024-04-16T18:17:54.765 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:17:54.765 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0S5PIsXgwjxqjXQsuY9CkwDhjBE= 2024-04-16T18:17:54.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0S5PIsXgwjxqjXQsuY9CkwDhjBE='} 2024-04-16T18:17:54.810 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000af2a54314b546bac-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.811 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.811 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.811 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.811 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.811 DEBUG:boto:Data: 2024-04-16T18:17:54.811 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.811 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.811 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.811 DEBUG:boto:Params: {} 2024-04-16T18:17:54.811 DEBUG:boto:Token: None 2024-04-16T18:17:54.812 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.816 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001043ef94f73cd267-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.816 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:54.817 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:54.817 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.817 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:54.817 DEBUG:boto:Data: 2024-04-16T18:17:54.817 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.817 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.817 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.817 DEBUG:boto:Params: {} 2024-04-16T18:17:54.817 DEBUG:boto:Token: None 2024-04-16T18:17:54.818 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:54.818 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LJiGsFhsTtRv9WoR9+3eCMmBK4U= 2024-04-16T18:17:54.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LJiGsFhsTtRv9WoR9+3eCMmBK4U='} 2024-04-16T18:17:54.834 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003682c5112ca5a6c5-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.834 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.834 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.835 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.835 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.835 DEBUG:boto:Data: 2024-04-16T18:17:54.835 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.835 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.835 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.835 DEBUG:boto:Params: {} 2024-04-16T18:17:54.835 DEBUG:boto:Token: None 2024-04-16T18:17:54.835 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.840 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001bc05926f3391863-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.840 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:17:54.840 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:17:54.841 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.841 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:17:54.841 DEBUG:boto:Data: 2024-04-16T18:17:54.841 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.841 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.841 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.841 DEBUG:boto:Params: {} 2024-04-16T18:17:54.841 DEBUG:boto:Token: None 2024-04-16T18:17:54.841 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:17:54.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MXXGWhrB/FnJtGxEXQm4BPovMyM= 2024-04-16T18:17:54.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MXXGWhrB/FnJtGxEXQm4BPovMyM='} 2024-04-16T18:17:54.886 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000601e679a57cf6b59-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.887 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.887 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.887 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.887 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.887 DEBUG:boto:Data: 2024-04-16T18:17:54.887 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.887 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.887 DEBUG:boto:Params: {} 2024-04-16T18:17:54.887 DEBUG:boto:Token: None 2024-04-16T18:17:54.887 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.892 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004547af86358b9c64-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.892 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:54.892 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:54.893 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.893 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:54.893 DEBUG:boto:Data: 2024-04-16T18:17:54.893 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.893 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.893 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.893 DEBUG:boto:Params: {} 2024-04-16T18:17:54.893 DEBUG:boto:Token: None 2024-04-16T18:17:54.893 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:54.893 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ecu2sLl+s0ru677sQqkgzEngwZw= 2024-04-16T18:17:54.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ecu2sLl+s0ru677sQqkgzEngwZw='} 2024-04-16T18:17:54.911 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000052e3f1ea1ab00742-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.911 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.911 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.912 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.912 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.912 DEBUG:boto:Data: 2024-04-16T18:17:54.912 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.912 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.912 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.912 DEBUG:boto:Params: {} 2024-04-16T18:17:54.912 DEBUG:boto:Token: None 2024-04-16T18:17:54.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.917 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ed2a08ea90ba58b0-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.917 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:54.917 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:54.918 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.918 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:17:54.918 DEBUG:boto:Data: 2024-04-16T18:17:54.918 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.918 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.918 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.918 DEBUG:boto:Params: {} 2024-04-16T18:17:54.918 DEBUG:boto:Token: None 2024-04-16T18:17:54.918 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:17:54.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gXm5htJuSnkkPD7xwUnHXeQcgyk= 2024-04-16T18:17:54.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gXm5htJuSnkkPD7xwUnHXeQcgyk='} 2024-04-16T18:17:54.962 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003cf1eca86a096404-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.963 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.963 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.963 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.963 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.963 DEBUG:boto:Data: 2024-04-16T18:17:54.963 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.963 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.963 DEBUG:boto:Params: {} 2024-04-16T18:17:54.963 DEBUG:boto:Token: None 2024-04-16T18:17:54.963 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.968 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000068bf5a86b2a4b210-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.968 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:17:54.968 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:17:54.969 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.969 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:17:54.969 DEBUG:boto:Data: 2024-04-16T18:17:54.969 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.969 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.969 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.969 DEBUG:boto:Params: {} 2024-04-16T18:17:54.969 DEBUG:boto:Token: None 2024-04-16T18:17:54.969 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:17:54.969 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RS8N5WJli9/XsCokLisGELKp/lo= 2024-04-16T18:17:54.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RS8N5WJli9/XsCokLisGELKp/lo='} 2024-04-16T18:17:54.983 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002d327863643268eb-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.983 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:54.983 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:54.983 DEBUG:boto:Method: HEAD 2024-04-16T18:17:54.983 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:54.983 DEBUG:boto:Data: 2024-04-16T18:17:54.983 DEBUG:boto:Headers: {} 2024-04-16T18:17:54.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.983 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.983 DEBUG:boto:Params: {} 2024-04-16T18:17:54.983 DEBUG:boto:Token: None 2024-04-16T18:17:54.984 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/ 2024-04-16T18:17:54.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4= 2024-04-16T18:17:54.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:67I3/R0HoRwgEWUwiWvyeojaRM4=', 'Content-Length': '0'} 2024-04-16T18:17:54.988 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007a27bc7b24f173be-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:54 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:54.988 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:54.988 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:54.988 DEBUG:boto:Method: PUT 2024-04-16T18:17:54.989 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:54.989 DEBUG:boto:Data: 2024-04-16T18:17:54.989 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:54.989 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:54.989 DEBUG:boto:Port: 8000 2024-04-16T18:17:54.989 DEBUG:boto:Params: {} 2024-04-16T18:17:54.989 DEBUG:boto:Token: None 2024-04-16T18:17:54.989 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:54 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:54.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SVOMfGozkjuV/Y0T2FQCTsmxBSQ= 2024-04-16T18:17:54.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:54 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SVOMfGozkjuV/Y0T2FQCTsmxBSQ='} 2024-04-16T18:17:55.034 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f1c0c5ce86f5a34d-00661ec0d2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.035 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.035 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.035 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.035 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.035 DEBUG:boto:Data: 2024-04-16T18:17:55.035 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.035 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.035 DEBUG:boto:Params: {} 2024-04-16T18:17:55.035 DEBUG:boto:Token: None 2024-04-16T18:17:55.036 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.041 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001921e8609db72a78-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.041 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:55.041 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:55.042 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.042 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:17:55.042 DEBUG:boto:Data: 2024-04-16T18:17:55.042 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.042 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.042 DEBUG:boto:Params: {} 2024-04-16T18:17:55.042 DEBUG:boto:Token: None 2024-04-16T18:17:55.042 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:17:55.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PUkBspnjJaJyGEbqQxTgRR4O4yY= 2024-04-16T18:17:55.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PUkBspnjJaJyGEbqQxTgRR4O4yY='} 2024-04-16T18:17:55.054 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000291979f0eac4401e-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.054 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.054 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.054 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.055 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.055 DEBUG:boto:Data: 2024-04-16T18:17:55.055 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.055 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.055 DEBUG:boto:Params: {} 2024-04-16T18:17:55.055 DEBUG:boto:Token: None 2024-04-16T18:17:55.055 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.055 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.060 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000248efc0ebc50f07c-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.060 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.060 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.060 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.060 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.060 DEBUG:boto:Data: 2024-04-16T18:17:55.060 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.061 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.061 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.061 DEBUG:boto:Params: {} 2024-04-16T18:17:55.061 DEBUG:boto:Token: None 2024-04-16T18:17:55.061 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MM17a4GdSW2UTZJiK+gcvZ2qhR8= 2024-04-16T18:17:55.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MM17a4GdSW2UTZJiK+gcvZ2qhR8='} 2024-04-16T18:17:55.110 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000084e5ae6172a2a08b-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.111 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.111 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.111 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.111 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.111 DEBUG:boto:Data: 2024-04-16T18:17:55.111 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.111 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.111 DEBUG:boto:Params: {} 2024-04-16T18:17:55.111 DEBUG:boto:Token: None 2024-04-16T18:17:55.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.116 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002ac9b88017b9842f-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.116 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:55.117 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:55.117 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.117 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:55.117 DEBUG:boto:Data: 2024-04-16T18:17:55.117 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.117 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.117 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.117 DEBUG:boto:Params: {} 2024-04-16T18:17:55.117 DEBUG:boto:Token: None 2024-04-16T18:17:55.117 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:55.118 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p9f5T7iACaZaTH/lIRIeAy7mwH0= 2024-04-16T18:17:55.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p9f5T7iACaZaTH/lIRIeAy7mwH0='} 2024-04-16T18:17:55.134 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002286859bd141d828-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.134 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.134 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.134 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.135 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.135 DEBUG:boto:Data: 2024-04-16T18:17:55.135 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.135 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.135 DEBUG:boto:Params: {} 2024-04-16T18:17:55.135 DEBUG:boto:Token: None 2024-04-16T18:17:55.135 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.140 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006f567fd7f26e36e2-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.140 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:55.140 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:55.140 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.140 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:17:55.141 DEBUG:boto:Data: 2024-04-16T18:17:55.141 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.141 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.141 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.141 DEBUG:boto:Params: {} 2024-04-16T18:17:55.141 DEBUG:boto:Token: None 2024-04-16T18:17:55.141 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:17:55.141 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bf54y2BA/GqpdFG+TBa8/8wF3ls= 2024-04-16T18:17:55.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bf54y2BA/GqpdFG+TBa8/8wF3ls='} 2024-04-16T18:17:55.186 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ff4ed19b220f0177-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.186 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.187 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.187 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.187 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.187 DEBUG:boto:Data: 2024-04-16T18:17:55.187 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.187 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.187 DEBUG:boto:Params: {} 2024-04-16T18:17:55.187 DEBUG:boto:Token: None 2024-04-16T18:17:55.187 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.192 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000d0afff9e62b3b4b-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.192 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:17:55.192 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:17:55.193 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.193 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:17:55.193 DEBUG:boto:Data: 2024-04-16T18:17:55.193 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.193 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.193 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.193 DEBUG:boto:Params: {} 2024-04-16T18:17:55.193 DEBUG:boto:Token: None 2024-04-16T18:17:55.193 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:17:55.194 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4aRhn3IJdPH2l4x9VBZlpkkGHyY= 2024-04-16T18:17:55.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4aRhn3IJdPH2l4x9VBZlpkkGHyY='} 2024-04-16T18:17:55.210 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000054806b9a243045c9-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.210 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.210 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.210 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.210 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.211 DEBUG:boto:Data: 2024-04-16T18:17:55.211 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.211 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.211 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.211 DEBUG:boto:Params: {} 2024-04-16T18:17:55.211 DEBUG:boto:Token: None 2024-04-16T18:17:55.211 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.211 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.216 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007ce266eb741e0aea-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.216 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:17:55.216 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:17:55.216 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.216 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025 2024-04-16T18:17:55.216 DEBUG:boto:Data: 2024-04-16T18:17:55.216 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.217 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.217 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.217 DEBUG:boto:Params: {} 2024-04-16T18:17:55.217 DEBUG:boto:Token: None 2024-04-16T18:17:55.217 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000025 2024-04-16T18:17:55.217 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sMV6AJge1PRuNAelWZggexf1RBg= 2024-04-16T18:17:55.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sMV6AJge1PRuNAelWZggexf1RBg='} 2024-04-16T18:17:55.262 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009936c074ce51e985-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.263 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '23', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:55.263 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '23', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:55.263 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --num-shards 23 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:55.308 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:55.308 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:55.409 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:55.409 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-51 2024-04-16T18:17:55.423 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 27 2024-04-16T18:17:55.540 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '23', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:55.540 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '23', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:55.540 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 23 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:55.721 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000058","obj00000000000000000026","obj00000000000000000011","obj00000000000000000046","obj00000000000000000014","obj00000000000000000050","obj00000000000000000032","obj00000000000000000002","obj00000000000000000039","obj00000000000000000007","obj00000000000000000006","obj00000000000000000040","obj00000000000000000009","obj00000000000000000045","obj00000000000000000013","obj00000000000000000012","obj00000000000000000033","obj00000000000000000001","obj00000000000000000000","obj00000000000000000021","obj00000000000000000003","obj00000000000000000055","obj00000000000000000008"]} 2024-04-16T18:17:55.721 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.721 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.722 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.722 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.722 DEBUG:boto:Data: 2024-04-16T18:17:55.722 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.722 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.722 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.722 DEBUG:boto:Params: {} 2024-04-16T18:17:55.722 DEBUG:boto:Token: None 2024-04-16T18:17:55.722 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.722 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.726 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '81'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b0c7fdeaf0c16973-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.726 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:17:55.727 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:17:55.727 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.727 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:17:55.727 DEBUG:boto:Data: 2024-04-16T18:17:55.727 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.727 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.727 DEBUG:boto:Params: {} 2024-04-16T18:17:55.727 DEBUG:boto:Token: None 2024-04-16T18:17:55.727 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:17:55.727 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZNdOtQVHX4i3nL3Du/9Buad7+YA= 2024-04-16T18:17:55.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZNdOtQVHX4i3nL3Du/9Buad7+YA='} 2024-04-16T18:17:55.739 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003847add6f2ae022c-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.739 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.739 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.739 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.739 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.739 DEBUG:boto:Data: 2024-04-16T18:17:55.739 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.739 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.739 DEBUG:boto:Params: {} 2024-04-16T18:17:55.740 DEBUG:boto:Token: None 2024-04-16T18:17:55.740 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.743 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '81'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a611abaf8bf926c3-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.744 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:17:55.744 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:17:55.744 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.744 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055 2024-04-16T18:17:55.744 DEBUG:boto:Data: 2024-04-16T18:17:55.744 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.744 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.744 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.744 DEBUG:boto:Params: {} 2024-04-16T18:17:55.745 DEBUG:boto:Token: None 2024-04-16T18:17:55.745 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000055 2024-04-16T18:17:55.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EsHRg6iuliQ3LUsh2vbhWmBwQKE= 2024-04-16T18:17:55.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EsHRg6iuliQ3LUsh2vbhWmBwQKE='} 2024-04-16T18:17:55.755 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f54090feeee03469-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.755 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.755 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.755 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.755 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.755 DEBUG:boto:Data: 2024-04-16T18:17:55.756 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.756 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.756 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.756 DEBUG:boto:Params: {} 2024-04-16T18:17:55.756 DEBUG:boto:Token: None 2024-04-16T18:17:55.756 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.759 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d96c3da7bcda3a12-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.760 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:17:55.760 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:17:55.760 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.760 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:17:55.760 DEBUG:boto:Data: 2024-04-16T18:17:55.760 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.760 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.760 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.760 DEBUG:boto:Params: {} 2024-04-16T18:17:55.760 DEBUG:boto:Token: None 2024-04-16T18:17:55.761 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:17:55.761 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U6w4XR11s6kv2tWqCst9FxXdtIM= 2024-04-16T18:17:55.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U6w4XR11s6kv2tWqCst9FxXdtIM='} 2024-04-16T18:17:55.803 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007311aa3dcd3aae32-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.803 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.803 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.803 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.803 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.803 DEBUG:boto:Data: 2024-04-16T18:17:55.803 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.803 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.804 DEBUG:boto:Params: {} 2024-04-16T18:17:55.804 DEBUG:boto:Token: None 2024-04-16T18:17:55.804 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.809 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000253e9967cc98f7ea-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.809 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:55.809 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:55.809 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.809 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:55.809 DEBUG:boto:Data: 2024-04-16T18:17:55.809 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.809 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.809 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.809 DEBUG:boto:Params: {} 2024-04-16T18:17:55.810 DEBUG:boto:Token: None 2024-04-16T18:17:55.810 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:55.810 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LAd9m9UFe82/Q0QQ/EDwAiBdNY8= 2024-04-16T18:17:55.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LAd9m9UFe82/Q0QQ/EDwAiBdNY8='} 2024-04-16T18:17:55.824 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c3880a092278daa7-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.824 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.824 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.824 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.825 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.825 DEBUG:boto:Data: 2024-04-16T18:17:55.825 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.825 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.825 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.825 DEBUG:boto:Params: {} 2024-04-16T18:17:55.825 DEBUG:boto:Token: None 2024-04-16T18:17:55.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.830 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f6f45ec76f274b9d-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.830 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:55.830 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:55.830 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.830 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:55.830 DEBUG:boto:Data: 2024-04-16T18:17:55.830 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.830 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.830 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.831 DEBUG:boto:Params: {} 2024-04-16T18:17:55.831 DEBUG:boto:Token: None 2024-04-16T18:17:55.831 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:55.831 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iiTD49C4sTfC0bFGrclFXv4joaQ= 2024-04-16T18:17:55.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iiTD49C4sTfC0bFGrclFXv4joaQ='} 2024-04-16T18:17:55.874 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f1ccf7c0480ae94e-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.874 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.875 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.875 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.875 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.875 DEBUG:boto:Data: 2024-04-16T18:17:55.875 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.875 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.875 DEBUG:boto:Params: {} 2024-04-16T18:17:55.875 DEBUG:boto:Token: None 2024-04-16T18:17:55.875 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.879 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000053b54f8a7a1e39fe-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.879 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.880 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.880 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.880 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.880 DEBUG:boto:Data: 2024-04-16T18:17:55.880 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.880 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.880 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.880 DEBUG:boto:Params: {} 2024-04-16T18:17:55.880 DEBUG:boto:Token: None 2024-04-16T18:17:55.880 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:55.881 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MM17a4GdSW2UTZJiK+gcvZ2qhR8= 2024-04-16T18:17:55.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MM17a4GdSW2UTZJiK+gcvZ2qhR8='} 2024-04-16T18:17:55.894 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d8df41c2ef333a4f-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.895 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.895 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.895 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.895 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.895 DEBUG:boto:Data: 2024-04-16T18:17:55.895 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.895 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.895 DEBUG:boto:Params: {} 2024-04-16T18:17:55.895 DEBUG:boto:Token: None 2024-04-16T18:17:55.895 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.900 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000908251fef5a3907b-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.900 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:17:55.900 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:17:55.901 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.901 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:17:55.901 DEBUG:boto:Data: 2024-04-16T18:17:55.901 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.901 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.901 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.901 DEBUG:boto:Params: {} 2024-04-16T18:17:55.901 DEBUG:boto:Token: None 2024-04-16T18:17:55.901 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:17:55.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vmOrb/yjE7E0+32i8xmhaEkJAx0= 2024-04-16T18:17:55.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vmOrb/yjE7E0+32i8xmhaEkJAx0='} 2024-04-16T18:17:55.946 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007fdd78b94acd9d85-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.947 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.947 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.947 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.947 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.947 DEBUG:boto:Data: 2024-04-16T18:17:55.947 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.947 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.947 DEBUG:boto:Params: {} 2024-04-16T18:17:55.947 DEBUG:boto:Token: None 2024-04-16T18:17:55.948 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.952 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f9775dd23187a85f-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.952 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:17:55.952 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:17:55.953 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.953 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050 2024-04-16T18:17:55.953 DEBUG:boto:Data: 2024-04-16T18:17:55.953 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.953 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.953 DEBUG:boto:Params: {} 2024-04-16T18:17:55.953 DEBUG:boto:Token: None 2024-04-16T18:17:55.953 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000050 2024-04-16T18:17:55.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u+vhqIoRLpq81xaR3mDeFEFXU5U= 2024-04-16T18:17:55.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u+vhqIoRLpq81xaR3mDeFEFXU5U='} 2024-04-16T18:17:55.966 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c44da2d678c82f59-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.966 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:55.966 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:55.966 DEBUG:boto:Method: HEAD 2024-04-16T18:17:55.966 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:55.966 DEBUG:boto:Data: 2024-04-16T18:17:55.967 DEBUG:boto:Headers: {} 2024-04-16T18:17:55.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.967 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.967 DEBUG:boto:Params: {} 2024-04-16T18:17:55.967 DEBUG:boto:Token: None 2024-04-16T18:17:55.967 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/ 2024-04-16T18:17:55.967 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA= 2024-04-16T18:17:55.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vWZO/FaLSgUebBlMVbAThbqLYEA=', 'Content-Length': '0'} 2024-04-16T18:17:55.971 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '87'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006d36ca839b798cc7-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:55.971 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:55.972 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:55.972 DEBUG:boto:Method: PUT 2024-04-16T18:17:55.972 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:17:55.972 DEBUG:boto:Data: 2024-04-16T18:17:55.972 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:55.972 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:55.972 DEBUG:boto:Port: 8000 2024-04-16T18:17:55.972 DEBUG:boto:Params: {} 2024-04-16T18:17:55.972 DEBUG:boto:Token: None 2024-04-16T18:17:55.972 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:55 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:17:55.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ed2cgxoQgqyFTPi4EvFzV5b50HI= 2024-04-16T18:17:55.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:55 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ed2cgxoQgqyFTPi4EvFzV5b50HI='} 2024-04-16T18:17:56.014 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005f1ce5c147edd697-00661ec0d3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.015 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.015 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.015 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.015 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.015 DEBUG:boto:Data: 2024-04-16T18:17:56.015 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.015 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.015 DEBUG:boto:Params: {} 2024-04-16T18:17:56.015 DEBUG:boto:Token: None 2024-04-16T18:17:56.015 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.021 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '87'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000060bce2dc9eb214b5-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.021 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:17:56.021 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:17:56.021 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.021 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:17:56.021 DEBUG:boto:Data: 2024-04-16T18:17:56.021 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.021 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.022 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.022 DEBUG:boto:Params: {} 2024-04-16T18:17:56.022 DEBUG:boto:Token: None 2024-04-16T18:17:56.022 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:17:56.022 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6YTLC2rJ1iFWl5v7VAx7ws8qXVk= 2024-04-16T18:17:56.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6YTLC2rJ1iFWl5v7VAx7ws8qXVk='} 2024-04-16T18:17:56.035 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000461fddd493bb2b73-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.035 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.035 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.035 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.036 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.036 DEBUG:boto:Data: 2024-04-16T18:17:56.036 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.036 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.036 DEBUG:boto:Params: {} 2024-04-16T18:17:56.036 DEBUG:boto:Token: None 2024-04-16T18:17:56.036 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.041 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009d8cd50d9bbe2d32-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.042 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:56.042 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:56.042 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.042 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:56.042 DEBUG:boto:Data: 2024-04-16T18:17:56.042 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.042 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.042 DEBUG:boto:Params: {} 2024-04-16T18:17:56.042 DEBUG:boto:Token: None 2024-04-16T18:17:56.043 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:56.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rkhLEVLjrewyTovO18HGdAP/qsk= 2024-04-16T18:17:56.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rkhLEVLjrewyTovO18HGdAP/qsk='} 2024-04-16T18:17:56.086 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000169cb38a392c1e61-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.087 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.087 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.087 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.087 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.087 DEBUG:boto:Data: 2024-04-16T18:17:56.087 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.087 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.087 DEBUG:boto:Params: {} 2024-04-16T18:17:56.087 DEBUG:boto:Token: None 2024-04-16T18:17:56.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.092 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cb8f850fc896c644-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.093 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:17:56.093 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:17:56.093 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.093 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046 2024-04-16T18:17:56.093 DEBUG:boto:Data: 2024-04-16T18:17:56.093 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.093 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.093 DEBUG:boto:Params: {} 2024-04-16T18:17:56.093 DEBUG:boto:Token: None 2024-04-16T18:17:56.094 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000046 2024-04-16T18:17:56.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A31qjCZiUx6vOYk5bhIjEuobiNI= 2024-04-16T18:17:56.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A31qjCZiUx6vOYk5bhIjEuobiNI='} 2024-04-16T18:17:56.106 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ac84915eb4b81ef6-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.106 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.106 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.106 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.107 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.107 DEBUG:boto:Data: 2024-04-16T18:17:56.107 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.107 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.107 DEBUG:boto:Params: {} 2024-04-16T18:17:56.107 DEBUG:boto:Token: None 2024-04-16T18:17:56.107 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.107 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.112 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '93'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aa1148c90140a0bc-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.112 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:56.112 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:56.112 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.112 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:17:56.113 DEBUG:boto:Data: 2024-04-16T18:17:56.113 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.113 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.113 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.113 DEBUG:boto:Params: {} 2024-04-16T18:17:56.113 DEBUG:boto:Token: None 2024-04-16T18:17:56.113 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:17:56.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JDjt/irD8QyMVSEWzDn/8UOFHzQ= 2024-04-16T18:17:56.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JDjt/irD8QyMVSEWzDn/8UOFHzQ='} 2024-04-16T18:17:56.158 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fa35551e9dfd4abf-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.159 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.159 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.159 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.159 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.159 DEBUG:boto:Data: 2024-04-16T18:17:56.159 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.159 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.159 DEBUG:boto:Params: {} 2024-04-16T18:17:56.159 DEBUG:boto:Token: None 2024-04-16T18:17:56.160 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.164 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '93'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a9b52b94c3ae51b0-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.164 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:56.164 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:56.165 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.165 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:56.165 DEBUG:boto:Data: 2024-04-16T18:17:56.165 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.165 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.165 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.165 DEBUG:boto:Params: {} 2024-04-16T18:17:56.165 DEBUG:boto:Token: None 2024-04-16T18:17:56.165 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:56.165 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mY+sZWwvNIllnloxJ9fpAVuQncY= 2024-04-16T18:17:56.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mY+sZWwvNIllnloxJ9fpAVuQncY='} 2024-04-16T18:17:56.182 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005ced7e666456630b-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.182 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.182 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.182 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.183 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.183 DEBUG:boto:Data: 2024-04-16T18:17:56.183 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.183 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.183 DEBUG:boto:Params: {} 2024-04-16T18:17:56.183 DEBUG:boto:Token: None 2024-04-16T18:17:56.183 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.183 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.188 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '93'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e708655c144f1fb5-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.188 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:17:56.188 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:17:56.188 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.188 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045 2024-04-16T18:17:56.188 DEBUG:boto:Data: 2024-04-16T18:17:56.189 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.189 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.189 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.189 DEBUG:boto:Params: {} 2024-04-16T18:17:56.189 DEBUG:boto:Token: None 2024-04-16T18:17:56.189 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000045 2024-04-16T18:17:56.189 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ha7zf1bQh8ayxPIk02LeSocley8= 2024-04-16T18:17:56.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ha7zf1bQh8ayxPIk02LeSocley8='} 2024-04-16T18:17:56.235 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b4c0f951ff5fce4c-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.235 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.235 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.235 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.235 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.235 DEBUG:boto:Data: 2024-04-16T18:17:56.235 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.235 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.236 DEBUG:boto:Params: {} 2024-04-16T18:17:56.236 DEBUG:boto:Token: None 2024-04-16T18:17:56.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.240 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f6d86f1a1b6e3ad9-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.240 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:56.240 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:56.240 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.241 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:56.241 DEBUG:boto:Data: 2024-04-16T18:17:56.241 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.241 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.241 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.241 DEBUG:boto:Params: {} 2024-04-16T18:17:56.241 DEBUG:boto:Token: None 2024-04-16T18:17:56.241 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:56.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+xNaw/2DsbBocgDYze+UZ6YqBpw= 2024-04-16T18:17:56.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+xNaw/2DsbBocgDYze+UZ6YqBpw='} 2024-04-16T18:17:56.258 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000caa0181dfea1b300-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.258 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.258 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.258 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.259 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.259 DEBUG:boto:Data: 2024-04-16T18:17:56.259 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.259 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.259 DEBUG:boto:Params: {} 2024-04-16T18:17:56.259 DEBUG:boto:Token: None 2024-04-16T18:17:56.259 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.259 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.264 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d466a97f6e6ca7b6-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.264 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:17:56.264 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:17:56.264 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.264 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039 2024-04-16T18:17:56.264 DEBUG:boto:Data: 2024-04-16T18:17:56.264 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.265 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.265 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.265 DEBUG:boto:Params: {} 2024-04-16T18:17:56.265 DEBUG:boto:Token: None 2024-04-16T18:17:56.265 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000039 2024-04-16T18:17:56.265 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UEBkjHln5C8lkUg+IW4R4m5rpo0= 2024-04-16T18:17:56.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UEBkjHln5C8lkUg+IW4R4m5rpo0='} 2024-04-16T18:17:56.310 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a1e001166b514596-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.311 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.311 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.311 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.311 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.311 DEBUG:boto:Data: 2024-04-16T18:17:56.311 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.311 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.311 DEBUG:boto:Params: {} 2024-04-16T18:17:56.311 DEBUG:boto:Token: None 2024-04-16T18:17:56.311 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.316 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '99'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a21951fd5048dd97-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.317 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:56.317 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:56.317 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.317 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:17:56.317 DEBUG:boto:Data: 2024-04-16T18:17:56.317 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.317 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.317 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.317 DEBUG:boto:Params: {} 2024-04-16T18:17:56.317 DEBUG:boto:Token: None 2024-04-16T18:17:56.318 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:17:56.318 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2npvozvb5qVJ6sRJLXsDO0iTMw0= 2024-04-16T18:17:56.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2npvozvb5qVJ6sRJLXsDO0iTMw0='} 2024-04-16T18:17:56.334 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000068e2c593dca98c9c-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.334 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.334 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.334 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.334 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.334 DEBUG:boto:Data: 2024-04-16T18:17:56.334 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.335 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.335 DEBUG:boto:Params: {} 2024-04-16T18:17:56.335 DEBUG:boto:Token: None 2024-04-16T18:17:56.335 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.340 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '99'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009091574ba9b13e27-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.340 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:17:56.340 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:17:56.340 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.340 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058 2024-04-16T18:17:56.340 DEBUG:boto:Data: 2024-04-16T18:17:56.340 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.340 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.341 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.341 DEBUG:boto:Params: {} 2024-04-16T18:17:56.341 DEBUG:boto:Token: None 2024-04-16T18:17:56.341 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000058 2024-04-16T18:17:56.341 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:My3TYVgR6Guc4Wn92r/EkPoqCno= 2024-04-16T18:17:56.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:My3TYVgR6Guc4Wn92r/EkPoqCno='} 2024-04-16T18:17:56.386 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000acebdaa2ad111908-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.387 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.387 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.387 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.387 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.387 DEBUG:boto:Data: 2024-04-16T18:17:56.387 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.387 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.387 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.387 DEBUG:boto:Params: {} 2024-04-16T18:17:56.387 DEBUG:boto:Token: None 2024-04-16T18:17:56.388 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.392 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005e863757811f2d58-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.392 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:17:56.392 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:17:56.393 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.393 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040 2024-04-16T18:17:56.393 DEBUG:boto:Data: 2024-04-16T18:17:56.393 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.393 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.393 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.393 DEBUG:boto:Params: {} 2024-04-16T18:17:56.393 DEBUG:boto:Token: None 2024-04-16T18:17:56.393 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000040 2024-04-16T18:17:56.393 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RpVftX7s6Mwhn8S2Ec/jv4+Dml8= 2024-04-16T18:17:56.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RpVftX7s6Mwhn8S2Ec/jv4+Dml8='} 2024-04-16T18:17:56.405 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ae62fa748ec8c33d-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.405 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.405 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.406 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.406 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.406 DEBUG:boto:Data: 2024-04-16T18:17:56.406 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.406 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.406 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.406 DEBUG:boto:Params: {} 2024-04-16T18:17:56.406 DEBUG:boto:Token: None 2024-04-16T18:17:56.406 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.407 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.411 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000630ebbbdac55eb20-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.412 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:56.412 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:56.412 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.412 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:56.412 DEBUG:boto:Data: 2024-04-16T18:17:56.412 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.412 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.412 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.412 DEBUG:boto:Params: {} 2024-04-16T18:17:56.412 DEBUG:boto:Token: None 2024-04-16T18:17:56.413 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:56.413 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eJgQebvoaaBY5cvTIzwSqaq/Hq0= 2024-04-16T18:17:56.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eJgQebvoaaBY5cvTIzwSqaq/Hq0='} 2024-04-16T18:17:56.458 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003e4d84362d10de75-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.459 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '29', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:56.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '29', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:56.459 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --num-shards 29 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:56.505 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:56.505 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:56.610 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:56.610 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-51 2024-04-16T18:17:56.632 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 35 2024-04-16T18:17:56.777 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '29', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:56.777 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '29', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:56.777 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 29 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:56.859 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:56.859 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:56.958 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000006","obj00000000000000000059","obj00000000000000000027","obj00000000000000000004","obj00000000000000000011","obj00000000000000000002","obj00000000000000000019","obj00000000000000000049","obj00000000000000000017","obj00000000000000000008","obj00000000000000000015","obj00000000000000000031","obj00000000000000000022","obj00000000000000000013","obj00000000000000000020","obj00000000000000000037","obj00000000000000000005","obj00000000000000000012","obj00000000000000000003","obj00000000000000000042","obj00000000000000000010","obj00000000000000000001","obj00000000000000000009","obj00000000000000000025","obj00000000000000000016","obj00000000000000000000","obj00000000000000000014","obj00000000000000000030","obj00000000000000000021"]} 2024-04-16T18:17:56.958 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.958 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.958 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.958 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.959 DEBUG:boto:Data: 2024-04-16T18:17:56.959 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.959 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.959 DEBUG:boto:Params: {} 2024-04-16T18:17:56.959 DEBUG:boto:Token: None 2024-04-16T18:17:56.959 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.964 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007606838a7333a6d7-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.965 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:17:56.965 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:17:56.965 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.965 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:17:56.965 DEBUG:boto:Data: 2024-04-16T18:17:56.965 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.965 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.965 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.965 DEBUG:boto:Params: {} 2024-04-16T18:17:56.965 DEBUG:boto:Token: None 2024-04-16T18:17:56.965 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:17:56.966 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0dsl2LmOZhzpwdulYNrreLw9+Zo= 2024-04-16T18:17:56.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0dsl2LmOZhzpwdulYNrreLw9+Zo='} 2024-04-16T18:17:56.977 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000048c22650e1bbd7a4-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.978 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:56.978 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:56.978 DEBUG:boto:Method: HEAD 2024-04-16T18:17:56.978 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:56.978 DEBUG:boto:Data: 2024-04-16T18:17:56.978 DEBUG:boto:Headers: {} 2024-04-16T18:17:56.978 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.978 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.978 DEBUG:boto:Params: {} 2024-04-16T18:17:56.978 DEBUG:boto:Token: None 2024-04-16T18:17:56.978 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/ 2024-04-16T18:17:56.979 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8= 2024-04-16T18:17:56.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DmpY3S0F4yu4sPsQQxai5XM2+N8=', 'Content-Length': '0'} 2024-04-16T18:17:56.983 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000777e847048599cd5-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:56.984 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:17:56.984 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:17:56.984 DEBUG:boto:Method: PUT 2024-04-16T18:17:56.984 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017 2024-04-16T18:17:56.984 DEBUG:boto:Data: 2024-04-16T18:17:56.984 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:56.984 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:56.984 DEBUG:boto:Port: 8000 2024-04-16T18:17:56.984 DEBUG:boto:Params: {} 2024-04-16T18:17:56.984 DEBUG:boto:Token: None 2024-04-16T18:17:56.985 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:56 GMT /lybtug-51/obj00000000000000000017 2024-04-16T18:17:56.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bNWG6ReB/abOX5c/HHgM7pdTItU= 2024-04-16T18:17:56.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:56 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bNWG6ReB/abOX5c/HHgM7pdTItU='} 2024-04-16T18:17:57.026 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004ceff5a788130e6e-00661ec0d4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.027 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.027 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.027 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.027 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.027 DEBUG:boto:Data: 2024-04-16T18:17:57.027 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.027 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.027 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.027 DEBUG:boto:Params: {} 2024-04-16T18:17:57.027 DEBUG:boto:Token: None 2024-04-16T18:17:57.028 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.033 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '36'), ('X-RGW-Bytes-Used', '108'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004725700339253970-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.033 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:57.033 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:57.033 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.033 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:57.033 DEBUG:boto:Data: 2024-04-16T18:17:57.033 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.033 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.033 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.034 DEBUG:boto:Params: {} 2024-04-16T18:17:57.034 DEBUG:boto:Token: None 2024-04-16T18:17:57.034 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:57.034 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OCYtNvtBP1rtZDbIT4zydJeyVgk= 2024-04-16T18:17:57.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OCYtNvtBP1rtZDbIT4zydJeyVgk='} 2024-04-16T18:17:57.050 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a47c66d03494de08-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.050 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.050 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.050 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.050 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.050 DEBUG:boto:Data: 2024-04-16T18:17:57.050 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.051 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.051 DEBUG:boto:Params: {} 2024-04-16T18:17:57.051 DEBUG:boto:Token: None 2024-04-16T18:17:57.051 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.056 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '36'), ('X-RGW-Bytes-Used', '108'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001aa33e0ebbf1fa4f-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.056 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:17:57.056 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:17:57.056 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.057 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031 2024-04-16T18:17:57.057 DEBUG:boto:Data: 2024-04-16T18:17:57.057 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.057 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.057 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.057 DEBUG:boto:Params: {} 2024-04-16T18:17:57.057 DEBUG:boto:Token: None 2024-04-16T18:17:57.057 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000031 2024-04-16T18:17:57.057 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8LQg2cHR8tGzCrfxt+qM0NozrYU= 2024-04-16T18:17:57.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8LQg2cHR8tGzCrfxt+qM0NozrYU='} 2024-04-16T18:17:57.102 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b1ddbb5960f9b801-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.103 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.103 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.103 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.103 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.103 DEBUG:boto:Data: 2024-04-16T18:17:57.103 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.103 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.103 DEBUG:boto:Params: {} 2024-04-16T18:17:57.103 DEBUG:boto:Token: None 2024-04-16T18:17:57.104 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.109 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '111'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000032030cc3846a719b-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.109 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:57.109 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:57.110 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.110 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:57.110 DEBUG:boto:Data: 2024-04-16T18:17:57.110 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.110 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.110 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.110 DEBUG:boto:Params: {} 2024-04-16T18:17:57.110 DEBUG:boto:Token: None 2024-04-16T18:17:57.110 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:57.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sK4mdzo4HQ2w7Y66yp6if8RpGik= 2024-04-16T18:17:57.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sK4mdzo4HQ2w7Y66yp6if8RpGik='} 2024-04-16T18:17:57.127 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000060702542c37d77e2-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.128 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.128 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.128 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.128 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.128 DEBUG:boto:Data: 2024-04-16T18:17:57.128 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.128 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.128 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.128 DEBUG:boto:Params: {} 2024-04-16T18:17:57.128 DEBUG:boto:Token: None 2024-04-16T18:17:57.128 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.129 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.134 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '111'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000074efcb0802378880-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.134 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:57.134 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:57.134 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.135 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:57.135 DEBUG:boto:Data: 2024-04-16T18:17:57.135 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.135 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.135 DEBUG:boto:Params: {} 2024-04-16T18:17:57.135 DEBUG:boto:Token: None 2024-04-16T18:17:57.135 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:57.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L1R8UfV9KbIhr0wJDjOPkmwEHAg= 2024-04-16T18:17:57.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L1R8UfV9KbIhr0wJDjOPkmwEHAg='} 2024-04-16T18:17:57.178 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e432b956eb8492b5-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.179 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.179 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.179 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.179 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.179 DEBUG:boto:Data: 2024-04-16T18:17:57.179 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.179 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.179 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.179 DEBUG:boto:Params: {} 2024-04-16T18:17:57.179 DEBUG:boto:Token: None 2024-04-16T18:17:57.180 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.185 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '111'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c0c97417d0a6eb87-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.186 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:17:57.186 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:17:57.186 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.186 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037 2024-04-16T18:17:57.186 DEBUG:boto:Data: 2024-04-16T18:17:57.186 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.186 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.187 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.187 DEBUG:boto:Params: {} 2024-04-16T18:17:57.187 DEBUG:boto:Token: None 2024-04-16T18:17:57.187 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000037 2024-04-16T18:17:57.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bETNXHQDkXBrGtNXzhzuGV9wtm8= 2024-04-16T18:17:57.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bETNXHQDkXBrGtNXzhzuGV9wtm8='} 2024-04-16T18:17:57.199 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c14808dabb5db567-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.199 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.199 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.199 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.199 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.199 DEBUG:boto:Data: 2024-04-16T18:17:57.199 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.199 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.199 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.200 DEBUG:boto:Params: {} 2024-04-16T18:17:57.200 DEBUG:boto:Token: None 2024-04-16T18:17:57.200 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.205 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066108c297883716e-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.206 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:57.206 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:17:57.206 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.206 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:17:57.206 DEBUG:boto:Data: 2024-04-16T18:17:57.206 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.206 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.206 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.206 DEBUG:boto:Params: {} 2024-04-16T18:17:57.206 DEBUG:boto:Token: None 2024-04-16T18:17:57.207 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:17:57.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1yUDq340cSTf0yZf5V09FaMIXx8= 2024-04-16T18:17:57.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1yUDq340cSTf0yZf5V09FaMIXx8='} 2024-04-16T18:17:57.250 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008ed3019542d0121c-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.250 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.251 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.251 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.251 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.251 DEBUG:boto:Data: 2024-04-16T18:17:57.251 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.251 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.251 DEBUG:boto:Params: {} 2024-04-16T18:17:57.251 DEBUG:boto:Token: None 2024-04-16T18:17:57.252 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.257 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000973053616ea6ef3-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.257 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:17:57.257 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:17:57.257 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.257 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:17:57.257 DEBUG:boto:Data: 2024-04-16T18:17:57.257 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.257 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.258 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.258 DEBUG:boto:Params: {} 2024-04-16T18:17:57.258 DEBUG:boto:Token: None 2024-04-16T18:17:57.258 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:17:57.258 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W4UnyEenasN28WjNyxVzbTGrCws= 2024-04-16T18:17:57.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W4UnyEenasN28WjNyxVzbTGrCws='} 2024-04-16T18:17:57.275 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000048f3da11df1590a1-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.275 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.276 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.276 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.276 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.276 DEBUG:boto:Data: 2024-04-16T18:17:57.276 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.276 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.276 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.276 DEBUG:boto:Params: {} 2024-04-16T18:17:57.276 DEBUG:boto:Token: None 2024-04-16T18:17:57.276 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.282 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000023fabdb9b5916c52-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.282 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:57.282 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:57.282 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.282 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:57.283 DEBUG:boto:Data: 2024-04-16T18:17:57.283 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.283 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.283 DEBUG:boto:Params: {} 2024-04-16T18:17:57.283 DEBUG:boto:Token: None 2024-04-16T18:17:57.283 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:57.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yi+fp/iVubkQ48ztKkqiM773Tzs= 2024-04-16T18:17:57.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yi+fp/iVubkQ48ztKkqiM773Tzs='} 2024-04-16T18:17:57.326 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b6e22488961331bc-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.327 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.327 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.327 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.327 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.327 DEBUG:boto:Data: 2024-04-16T18:17:57.327 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.327 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.327 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.327 DEBUG:boto:Params: {} 2024-04-16T18:17:57.327 DEBUG:boto:Token: None 2024-04-16T18:17:57.328 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.334 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c2a539fb50718bb2-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.334 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:57.334 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:57.334 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.334 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:57.334 DEBUG:boto:Data: 2024-04-16T18:17:57.334 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.334 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.335 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.335 DEBUG:boto:Params: {} 2024-04-16T18:17:57.335 DEBUG:boto:Token: None 2024-04-16T18:17:57.335 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:57.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TwedZAbiWx2aouV/LClZNRgqHP4= 2024-04-16T18:17:57.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TwedZAbiWx2aouV/LClZNRgqHP4='} 2024-04-16T18:17:57.352 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009615b80bff77bcf5-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.352 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.352 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.352 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.353 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.353 DEBUG:boto:Data: 2024-04-16T18:17:57.353 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.353 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.353 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.353 DEBUG:boto:Params: {} 2024-04-16T18:17:57.353 DEBUG:boto:Token: None 2024-04-16T18:17:57.353 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.358 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006dd3cf87226dcaeb-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.358 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:57.359 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:57.359 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.359 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:57.359 DEBUG:boto:Data: 2024-04-16T18:17:57.359 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.359 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.359 DEBUG:boto:Params: {} 2024-04-16T18:17:57.359 DEBUG:boto:Token: None 2024-04-16T18:17:57.360 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:57.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCuQO10ay95DwEfaIiHSZzBL1s8= 2024-04-16T18:17:57.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCuQO10ay95DwEfaIiHSZzBL1s8='} 2024-04-16T18:17:57.402 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bdf0f2fc34004282-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.403 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.403 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.403 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.403 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.403 DEBUG:boto:Data: 2024-04-16T18:17:57.403 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.403 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.403 DEBUG:boto:Params: {} 2024-04-16T18:17:57.403 DEBUG:boto:Token: None 2024-04-16T18:17:57.404 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.410 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cee2c95e5a9cf584-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.410 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:57.410 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:17:57.410 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.410 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:17:57.410 DEBUG:boto:Data: 2024-04-16T18:17:57.410 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.410 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.411 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.411 DEBUG:boto:Params: {} 2024-04-16T18:17:57.411 DEBUG:boto:Token: None 2024-04-16T18:17:57.411 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:17:57.411 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1iyb7YtpAYjmJnapsX4B2rtP4iU= 2024-04-16T18:17:57.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1iyb7YtpAYjmJnapsX4B2rtP4iU='} 2024-04-16T18:17:57.427 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000154e4b83b946b7a9-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.427 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.428 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.428 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.428 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.428 DEBUG:boto:Data: 2024-04-16T18:17:57.428 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.428 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.428 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.428 DEBUG:boto:Params: {} 2024-04-16T18:17:57.428 DEBUG:boto:Token: None 2024-04-16T18:17:57.428 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.434 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000257eb8782f076471-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.434 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:57.434 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:57.434 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.435 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:17:57.435 DEBUG:boto:Data: 2024-04-16T18:17:57.435 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.435 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.435 DEBUG:boto:Params: {} 2024-04-16T18:17:57.435 DEBUG:boto:Token: None 2024-04-16T18:17:57.435 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:17:57.435 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ucEo0W6/RqSy7vdSg2U2LXyniHo= 2024-04-16T18:17:57.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ucEo0W6/RqSy7vdSg2U2LXyniHo='} 2024-04-16T18:17:57.478 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009591286283a278ee-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.479 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.479 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.479 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.479 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.479 DEBUG:boto:Data: 2024-04-16T18:17:57.479 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.479 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.479 DEBUG:boto:Params: {} 2024-04-16T18:17:57.479 DEBUG:boto:Token: None 2024-04-16T18:17:57.480 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.486 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a54704c31df9e016-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.486 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:17:57.486 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:17:57.487 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.487 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:17:57.487 DEBUG:boto:Data: 2024-04-16T18:17:57.487 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.487 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.487 DEBUG:boto:Params: {} 2024-04-16T18:17:57.487 DEBUG:boto:Token: None 2024-04-16T18:17:57.487 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:17:57.487 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MKnQgts7+9r05vMQvw52dZdZL40= 2024-04-16T18:17:57.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MKnQgts7+9r05vMQvw52dZdZL40='} 2024-04-16T18:17:57.504 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000220ebf5f643194b0-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.504 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.505 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.505 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.505 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.505 DEBUG:boto:Data: 2024-04-16T18:17:57.505 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.505 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.505 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.505 DEBUG:boto:Params: {} 2024-04-16T18:17:57.505 DEBUG:boto:Token: None 2024-04-16T18:17:57.505 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.506 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.511 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000056bace9b29baab02-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.511 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:17:57.512 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:17:57.512 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.512 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025 2024-04-16T18:17:57.512 DEBUG:boto:Data: 2024-04-16T18:17:57.512 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.512 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.512 DEBUG:boto:Params: {} 2024-04-16T18:17:57.512 DEBUG:boto:Token: None 2024-04-16T18:17:57.513 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000025 2024-04-16T18:17:57.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUXCAP5tyFu64ZwvZl0zOOufdPA= 2024-04-16T18:17:57.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUXCAP5tyFu64ZwvZl0zOOufdPA='} 2024-04-16T18:17:57.554 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ed1dc508c672134f-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.555 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.555 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.555 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.555 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.555 DEBUG:boto:Data: 2024-04-16T18:17:57.555 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.555 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.555 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.555 DEBUG:boto:Params: {} 2024-04-16T18:17:57.555 DEBUG:boto:Token: None 2024-04-16T18:17:57.555 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.561 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ca13774edfb1499a-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.561 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:57.562 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:57.562 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.562 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:57.562 DEBUG:boto:Data: 2024-04-16T18:17:57.562 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.562 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.562 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.562 DEBUG:boto:Params: {} 2024-04-16T18:17:57.562 DEBUG:boto:Token: None 2024-04-16T18:17:57.562 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:57.563 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oKSFMbxr3esQm/lOZx8LSfc1/mg= 2024-04-16T18:17:57.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oKSFMbxr3esQm/lOZx8LSfc1/mg='} 2024-04-16T18:17:57.578 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000602b0a8452d4410a-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.579 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.579 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.579 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.579 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.579 DEBUG:boto:Data: 2024-04-16T18:17:57.579 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.579 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.579 DEBUG:boto:Params: {} 2024-04-16T18:17:57.579 DEBUG:boto:Token: None 2024-04-16T18:17:57.580 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.586 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007d3cb2b7c30199e4-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.586 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:57.586 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:57.586 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.586 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:17:57.586 DEBUG:boto:Data: 2024-04-16T18:17:57.586 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.587 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.587 DEBUG:boto:Params: {} 2024-04-16T18:17:57.587 DEBUG:boto:Token: None 2024-04-16T18:17:57.587 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:17:57.587 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gm3SaZD9IvBrhCNOtf/3PtpfTDc= 2024-04-16T18:17:57.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gm3SaZD9IvBrhCNOtf/3PtpfTDc='} 2024-04-16T18:17:57.630 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a07740596868d682-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.631 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.631 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.631 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.631 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.631 DEBUG:boto:Data: 2024-04-16T18:17:57.631 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.631 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.631 DEBUG:boto:Params: {} 2024-04-16T18:17:57.631 DEBUG:boto:Token: None 2024-04-16T18:17:57.631 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.637 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000849a6d905ab6af2e-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.638 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:17:57.638 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:17:57.638 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.638 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:17:57.638 DEBUG:boto:Data: 2024-04-16T18:17:57.638 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.638 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.638 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.638 DEBUG:boto:Params: {} 2024-04-16T18:17:57.638 DEBUG:boto:Token: None 2024-04-16T18:17:57.639 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:17:57.639 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vdPxBagTTBCZaVQ71w8bRFYKd0Q= 2024-04-16T18:17:57.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vdPxBagTTBCZaVQ71w8bRFYKd0Q='} 2024-04-16T18:17:57.655 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000715bd6dbc3806c1d-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.655 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.655 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.655 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.655 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.655 DEBUG:boto:Data: 2024-04-16T18:17:57.655 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.656 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.656 DEBUG:boto:Params: {} 2024-04-16T18:17:57.656 DEBUG:boto:Token: None 2024-04-16T18:17:57.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.662 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ee5a566da05acac6-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.662 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:17:57.662 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:17:57.662 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.663 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:17:57.663 DEBUG:boto:Data: 2024-04-16T18:17:57.663 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.663 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.663 DEBUG:boto:Params: {} 2024-04-16T18:17:57.663 DEBUG:boto:Token: None 2024-04-16T18:17:57.663 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:17:57.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0Sy3bJY86L7Lzy8ELCIlvhGPJzs= 2024-04-16T18:17:57.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0Sy3bJY86L7Lzy8ELCIlvhGPJzs='} 2024-04-16T18:17:57.706 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ef24bb01ec53dc03-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.707 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.707 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.707 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.707 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.707 DEBUG:boto:Data: 2024-04-16T18:17:57.707 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.707 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.707 DEBUG:boto:Params: {} 2024-04-16T18:17:57.707 DEBUG:boto:Token: None 2024-04-16T18:17:57.707 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.713 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000606f3889ca165f67-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.713 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:17:57.713 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:17:57.713 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.713 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042 2024-04-16T18:17:57.714 DEBUG:boto:Data: 2024-04-16T18:17:57.714 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.714 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.714 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.714 DEBUG:boto:Params: {} 2024-04-16T18:17:57.714 DEBUG:boto:Token: None 2024-04-16T18:17:57.714 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000042 2024-04-16T18:17:57.714 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O/+cKXzuVZiXDhYHQ2gNgvMNXfk= 2024-04-16T18:17:57.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O/+cKXzuVZiXDhYHQ2gNgvMNXfk='} 2024-04-16T18:17:57.726 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000acb66e17471455fe-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.726 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.726 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.727 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.727 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.727 DEBUG:boto:Data: 2024-04-16T18:17:57.727 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.727 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.727 DEBUG:boto:Params: {} 2024-04-16T18:17:57.727 DEBUG:boto:Token: None 2024-04-16T18:17:57.727 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.727 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.733 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '39'), ('X-RGW-Bytes-Used', '117'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c9d6575e06adbab1-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.733 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:17:57.733 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:17:57.733 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.733 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022 2024-04-16T18:17:57.733 DEBUG:boto:Data: 2024-04-16T18:17:57.733 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.734 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.734 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.734 DEBUG:boto:Params: {} 2024-04-16T18:17:57.734 DEBUG:boto:Token: None 2024-04-16T18:17:57.734 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000022 2024-04-16T18:17:57.734 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H211vbmdwVwOCED8IIjlreJsuow= 2024-04-16T18:17:57.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H211vbmdwVwOCED8IIjlreJsuow='} 2024-04-16T18:17:57.778 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000023c1c7717f136348-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.779 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.779 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.779 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.779 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.779 DEBUG:boto:Data: 2024-04-16T18:17:57.779 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.779 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.779 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.779 DEBUG:boto:Params: {} 2024-04-16T18:17:57.779 DEBUG:boto:Token: None 2024-04-16T18:17:57.780 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.785 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000002da6fe61a674f0f-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.786 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:17:57.786 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:17:57.786 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.786 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:17:57.786 DEBUG:boto:Data: 2024-04-16T18:17:57.786 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.786 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.786 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.786 DEBUG:boto:Params: {} 2024-04-16T18:17:57.787 DEBUG:boto:Token: None 2024-04-16T18:17:57.787 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:17:57.787 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qs8d3UG9SVhGMh41TsXEWPUlhLU= 2024-04-16T18:17:57.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qs8d3UG9SVhGMh41TsXEWPUlhLU='} 2024-04-16T18:17:57.803 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f9043a02a4ec1f6e-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.803 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.804 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.804 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.804 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.804 DEBUG:boto:Data: 2024-04-16T18:17:57.804 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.804 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.804 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.804 DEBUG:boto:Params: {} 2024-04-16T18:17:57.804 DEBUG:boto:Token: None 2024-04-16T18:17:57.804 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.810 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000070b8222f4eb9ec18-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.810 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:57.810 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:57.810 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.810 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:17:57.810 DEBUG:boto:Data: 2024-04-16T18:17:57.810 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.810 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.811 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.811 DEBUG:boto:Params: {} 2024-04-16T18:17:57.811 DEBUG:boto:Token: None 2024-04-16T18:17:57.811 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:17:57.811 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:okPC1HQXDoUpyZ6ai9EroohGVtM= 2024-04-16T18:17:57.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:okPC1HQXDoUpyZ6ai9EroohGVtM='} 2024-04-16T18:17:57.854 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000065e9cd34199962c-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.855 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.855 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.855 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.855 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.855 DEBUG:boto:Data: 2024-04-16T18:17:57.855 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.855 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.855 DEBUG:boto:Params: {} 2024-04-16T18:17:57.855 DEBUG:boto:Token: None 2024-04-16T18:17:57.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.861 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000974a797457b01fe7-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.861 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:57.861 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:57.861 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.862 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:17:57.862 DEBUG:boto:Data: 2024-04-16T18:17:57.862 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.862 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.862 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.862 DEBUG:boto:Params: {} 2024-04-16T18:17:57.862 DEBUG:boto:Token: None 2024-04-16T18:17:57.862 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:17:57.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RcsrY6KAxsGC7wvflAce3Uc8rJI= 2024-04-16T18:17:57.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RcsrY6KAxsGC7wvflAce3Uc8rJI='} 2024-04-16T18:17:57.879 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000050caa8b18ee8f146-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.880 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.880 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.880 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.880 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.880 DEBUG:boto:Data: 2024-04-16T18:17:57.880 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.880 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.880 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.880 DEBUG:boto:Params: {} 2024-04-16T18:17:57.880 DEBUG:boto:Token: None 2024-04-16T18:17:57.880 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.881 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.886 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009557df5022aee1da-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.887 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:17:57.887 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:17:57.887 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.887 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027 2024-04-16T18:17:57.887 DEBUG:boto:Data: 2024-04-16T18:17:57.887 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.887 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.887 DEBUG:boto:Params: {} 2024-04-16T18:17:57.887 DEBUG:boto:Token: None 2024-04-16T18:17:57.888 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000027 2024-04-16T18:17:57.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LuFyf510JaHYqA8PoSF7XZMKHYg= 2024-04-16T18:17:57.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LuFyf510JaHYqA8PoSF7XZMKHYg='} 2024-04-16T18:17:57.930 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b1b7074f87a88ef5-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.931 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:57.931 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:57.931 DEBUG:boto:Method: HEAD 2024-04-16T18:17:57.931 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:57.931 DEBUG:boto:Data: 2024-04-16T18:17:57.931 DEBUG:boto:Headers: {} 2024-04-16T18:17:57.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.931 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.931 DEBUG:boto:Params: {} 2024-04-16T18:17:57.931 DEBUG:boto:Token: None 2024-04-16T18:17:57.931 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/ 2024-04-16T18:17:57.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk= 2024-04-16T18:17:57.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PNmz/RjeTglVeXHzhHufHfqgJUk=', 'Content-Length': '0'} 2024-04-16T18:17:57.937 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '41'), ('X-RGW-Bytes-Used', '123'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000165941c6227672c-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.937 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:17:57.937 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:17:57.937 DEBUG:boto:Method: PUT 2024-04-16T18:17:57.937 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049 2024-04-16T18:17:57.937 DEBUG:boto:Data: 2024-04-16T18:17:57.938 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:57.938 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:57.938 DEBUG:boto:Port: 8000 2024-04-16T18:17:57.938 DEBUG:boto:Params: {} 2024-04-16T18:17:57.938 DEBUG:boto:Token: None 2024-04-16T18:17:57.938 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:57 GMT /lybtug-51/obj00000000000000000049 2024-04-16T18:17:57.938 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H8EldNrM4sm8QzNVg+Qg1OHa2DQ= 2024-04-16T18:17:57.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:57 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H8EldNrM4sm8QzNVg+Qg1OHa2DQ='} 2024-04-16T18:17:57.950 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c64caeda8fa7b68d-00661ec0d5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:57.951 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '31', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:57.951 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '31', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:57.951 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --num-shards 31 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:57.996 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:57.996 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:58.097 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:58.097 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-51 2024-04-16T18:17:58.123 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 42 2024-04-16T18:17:58.275 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '31', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:58.276 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '31', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:58.276 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 31 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:58.361 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:58.361 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:58.460 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000019","obj00000000000000000006","obj00000000000000000467","obj00000000000000000005","obj00000000000000000056","obj00000000000000000004","obj00000000000000000023","obj00000000000000000317","obj00000000000000000022","obj00000000000000000000","obj00000000000000000021","obj00000000000000000009","obj00000000000000000271","obj00000000000000000008","obj00000000000000000018","obj00000000000000000007","obj00000000000000000017","obj00000000000000000162","obj00000000000000000016","obj00000000000000000003","obj00000000000000000423","obj00000000000000000002","obj00000000000000000012","obj00000000000000000001","obj00000000000000000011","obj00000000000000000283","obj00000000000000000010","obj00000000000000000061","obj00000000000000000050","obj00000000000000000029","obj00000000000000000133"]} 2024-04-16T18:17:58.460 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.460 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.460 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.460 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.460 DEBUG:boto:Data: 2024-04-16T18:17:58.460 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.461 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.461 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.461 DEBUG:boto:Params: {} 2024-04-16T18:17:58.461 DEBUG:boto:Token: None 2024-04-16T18:17:58.461 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.465 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001747c9e2f70fde25-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.465 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:58.465 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:17:58.466 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.466 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:17:58.466 DEBUG:boto:Data: 2024-04-16T18:17:58.466 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.466 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.466 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.466 DEBUG:boto:Params: {} 2024-04-16T18:17:58.466 DEBUG:boto:Token: None 2024-04-16T18:17:58.466 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:17:58.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NZpntMY+laYTdTo95mnlLX6WVsA= 2024-04-16T18:17:58.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NZpntMY+laYTdTo95mnlLX6WVsA='} 2024-04-16T18:17:58.510 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000064f156b82d24e8f8-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.511 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.511 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.511 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.511 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.511 DEBUG:boto:Data: 2024-04-16T18:17:58.511 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.511 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.511 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.511 DEBUG:boto:Params: {} 2024-04-16T18:17:58.511 DEBUG:boto:Token: None 2024-04-16T18:17:58.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.515 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c0c4445827b81fa7-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.516 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:58.516 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:17:58.516 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.516 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:17:58.516 DEBUG:boto:Data: 2024-04-16T18:17:58.516 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.516 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.516 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.516 DEBUG:boto:Params: {} 2024-04-16T18:17:58.516 DEBUG:boto:Token: None 2024-04-16T18:17:58.517 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:17:58.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0BbAuJbL9pg9MiulqHoKBhBP8Qg= 2024-04-16T18:17:58.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0BbAuJbL9pg9MiulqHoKBhBP8Qg='} 2024-04-16T18:17:58.533 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000065310034d3469cb6-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.533 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.533 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.533 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.533 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.533 DEBUG:boto:Data: 2024-04-16T18:17:58.533 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.533 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.533 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.533 DEBUG:boto:Params: {} 2024-04-16T18:17:58.534 DEBUG:boto:Token: None 2024-04-16T18:17:58.534 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.538 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c3472d045e9f67f2-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.538 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:17:58.538 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:17:58.539 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.539 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271 2024-04-16T18:17:58.539 DEBUG:boto:Data: 2024-04-16T18:17:58.539 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.539 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.539 DEBUG:boto:Params: {} 2024-04-16T18:17:58.539 DEBUG:boto:Token: None 2024-04-16T18:17:58.539 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000271 2024-04-16T18:17:58.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h75A5UCiSV9WGoFxnIRpukt85cw= 2024-04-16T18:17:58.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h75A5UCiSV9WGoFxnIRpukt85cw='} 2024-04-16T18:17:58.582 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008b5a0fbade179789-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.583 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.583 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.583 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.583 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.583 DEBUG:boto:Data: 2024-04-16T18:17:58.583 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.583 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.583 DEBUG:boto:Params: {} 2024-04-16T18:17:58.583 DEBUG:boto:Token: None 2024-04-16T18:17:58.583 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.589 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '43'), ('X-RGW-Bytes-Used', '129'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000276b66085a42420a-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.589 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:17:58.589 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:17:58.590 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.590 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162 2024-04-16T18:17:58.590 DEBUG:boto:Data: 2024-04-16T18:17:58.590 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.590 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.590 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.590 DEBUG:boto:Params: {} 2024-04-16T18:17:58.590 DEBUG:boto:Token: None 2024-04-16T18:17:58.590 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000162 2024-04-16T18:17:58.591 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9gT+c6uIIc9pOGiW91lY8HtYjNw= 2024-04-16T18:17:58.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9gT+c6uIIc9pOGiW91lY8HtYjNw='} 2024-04-16T18:17:58.602 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005b507ed9962f9d47-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.603 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.603 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.603 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.603 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.603 DEBUG:boto:Data: 2024-04-16T18:17:58.603 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.603 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.603 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.603 DEBUG:boto:Params: {} 2024-04-16T18:17:58.603 DEBUG:boto:Token: None 2024-04-16T18:17:58.603 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.608 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '44'), ('X-RGW-Bytes-Used', '132'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009a7b600dc9d3580c-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.609 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:17:58.609 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:17:58.609 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.609 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018 2024-04-16T18:17:58.609 DEBUG:boto:Data: 2024-04-16T18:17:58.609 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.609 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.609 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.609 DEBUG:boto:Params: {} 2024-04-16T18:17:58.609 DEBUG:boto:Token: None 2024-04-16T18:17:58.610 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000018 2024-04-16T18:17:58.610 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3ItIAUNz8S58KOcMnBQVb9egmg= 2024-04-16T18:17:58.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3ItIAUNz8S58KOcMnBQVb9egmg='} 2024-04-16T18:17:58.658 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b02c04dd7339826c-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.659 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.659 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.659 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.659 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.659 DEBUG:boto:Data: 2024-04-16T18:17:58.659 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.659 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.659 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.659 DEBUG:boto:Params: {} 2024-04-16T18:17:58.660 DEBUG:boto:Token: None 2024-04-16T18:17:58.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.665 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '135'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fee15a05fe38ef4f-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.666 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:58.666 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:17:58.666 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.666 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:17:58.666 DEBUG:boto:Data: 2024-04-16T18:17:58.666 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.666 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.666 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.666 DEBUG:boto:Params: {} 2024-04-16T18:17:58.666 DEBUG:boto:Token: None 2024-04-16T18:17:58.667 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:17:58.667 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gLltoUWV9uX5JGZ6KCnp8TiKquM= 2024-04-16T18:17:58.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gLltoUWV9uX5JGZ6KCnp8TiKquM='} 2024-04-16T18:17:58.683 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000029ac7a004e49f20a-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.683 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.683 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.683 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.683 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.683 DEBUG:boto:Data: 2024-04-16T18:17:58.683 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.683 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.683 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.684 DEBUG:boto:Params: {} 2024-04-16T18:17:58.684 DEBUG:boto:Token: None 2024-04-16T18:17:58.684 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.690 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '135'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000089317cbbfb249a9c-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.690 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:58.690 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:17:58.690 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.690 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:17:58.690 DEBUG:boto:Data: 2024-04-16T18:17:58.690 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.691 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.691 DEBUG:boto:Params: {} 2024-04-16T18:17:58.691 DEBUG:boto:Token: None 2024-04-16T18:17:58.691 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:17:58.691 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:q6xXdaXoAzxVo7YT5aaysQPXYnU= 2024-04-16T18:17:58.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:q6xXdaXoAzxVo7YT5aaysQPXYnU='} 2024-04-16T18:17:58.734 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000016fa246ca4f2a7ba-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.735 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.735 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.735 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.735 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.735 DEBUG:boto:Data: 2024-04-16T18:17:58.735 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.735 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.735 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.735 DEBUG:boto:Params: {} 2024-04-16T18:17:58.735 DEBUG:boto:Token: None 2024-04-16T18:17:58.736 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.742 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '135'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008974c21a7b1603b2-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.742 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:17:58.742 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:17:58.742 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.742 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317 2024-04-16T18:17:58.743 DEBUG:boto:Data: 2024-04-16T18:17:58.743 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.743 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.743 DEBUG:boto:Params: {} 2024-04-16T18:17:58.743 DEBUG:boto:Token: None 2024-04-16T18:17:58.743 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000317 2024-04-16T18:17:58.743 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:36sQkXxSL/DLFhGcTudbTgYJ2Bo= 2024-04-16T18:17:58.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:36sQkXxSL/DLFhGcTudbTgYJ2Bo='} 2024-04-16T18:17:58.757 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000e89b208a9e82084-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.757 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.757 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.757 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.757 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.757 DEBUG:boto:Data: 2024-04-16T18:17:58.757 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.757 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.758 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.758 DEBUG:boto:Params: {} 2024-04-16T18:17:58.758 DEBUG:boto:Token: None 2024-04-16T18:17:58.758 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.758 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.764 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '46'), ('X-RGW-Bytes-Used', '138'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bc9efe80e23462f4-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.764 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:17:58.764 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:17:58.764 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.764 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467 2024-04-16T18:17:58.765 DEBUG:boto:Data: 2024-04-16T18:17:58.765 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.765 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.765 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.765 DEBUG:boto:Params: {} 2024-04-16T18:17:58.765 DEBUG:boto:Token: None 2024-04-16T18:17:58.765 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000467 2024-04-16T18:17:58.765 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9z7hQuo7rrt9kO+miulTX8bttWE= 2024-04-16T18:17:58.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9z7hQuo7rrt9kO+miulTX8bttWE='} 2024-04-16T18:17:58.810 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b3abf7706f56dd81-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.811 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.811 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.811 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.811 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.811 DEBUG:boto:Data: 2024-04-16T18:17:58.811 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.811 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.811 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.811 DEBUG:boto:Params: {} 2024-04-16T18:17:58.812 DEBUG:boto:Token: None 2024-04-16T18:17:58.812 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.818 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '47'), ('X-RGW-Bytes-Used', '141'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006b3e1f934a1f0fbc-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.818 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:17:58.818 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:17:58.819 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.819 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133 2024-04-16T18:17:58.819 DEBUG:boto:Data: 2024-04-16T18:17:58.819 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.819 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.819 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.819 DEBUG:boto:Params: {} 2024-04-16T18:17:58.819 DEBUG:boto:Token: None 2024-04-16T18:17:58.819 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000133 2024-04-16T18:17:58.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QCSJ9PkooR58qRshbr2rad/beeo= 2024-04-16T18:17:58.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QCSJ9PkooR58qRshbr2rad/beeo='} 2024-04-16T18:17:58.832 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000022082b78049b9ee4-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.832 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.832 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.832 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.832 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.832 DEBUG:boto:Data: 2024-04-16T18:17:58.832 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.833 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.833 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.833 DEBUG:boto:Params: {} 2024-04-16T18:17:58.833 DEBUG:boto:Token: None 2024-04-16T18:17:58.833 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.839 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '48'), ('X-RGW-Bytes-Used', '144'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000076e15eaeb09aadc5-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.839 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:17:58.839 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:17:58.840 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.840 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023 2024-04-16T18:17:58.840 DEBUG:boto:Data: 2024-04-16T18:17:58.840 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.840 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.840 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.840 DEBUG:boto:Params: {} 2024-04-16T18:17:58.840 DEBUG:boto:Token: None 2024-04-16T18:17:58.841 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000023 2024-04-16T18:17:58.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2MjVs2zMMJVfJr8MZWCIxjoGnoA= 2024-04-16T18:17:58.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2MjVs2zMMJVfJr8MZWCIxjoGnoA='} 2024-04-16T18:17:58.887 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000703b6d54da21d4ef-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.887 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.887 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.887 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.887 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.887 DEBUG:boto:Data: 2024-04-16T18:17:58.887 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.887 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.888 DEBUG:boto:Params: {} 2024-04-16T18:17:58.888 DEBUG:boto:Token: None 2024-04-16T18:17:58.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.894 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '49'), ('X-RGW-Bytes-Used', '147'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000626eb8219c3c0b64-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.894 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:58.894 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:17:58.894 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.895 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:17:58.895 DEBUG:boto:Data: 2024-04-16T18:17:58.895 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.895 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.895 DEBUG:boto:Params: {} 2024-04-16T18:17:58.895 DEBUG:boto:Token: None 2024-04-16T18:17:58.895 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:17:58.895 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VH1Hl45ohgD4SFADREQodrXoDFs= 2024-04-16T18:17:58.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VH1Hl45ohgD4SFADREQodrXoDFs='} 2024-04-16T18:17:58.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2024-04-16T18:17:58.911 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002adf13188b41bdda-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.911 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.911 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.912 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.912 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.912 DEBUG:boto:Data: 2024-04-16T18:17:58.912 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.912 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.912 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.912 DEBUG:boto:Params: {} 2024-04-16T18:17:58.912 DEBUG:boto:Token: None 2024-04-16T18:17:58.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.918 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '49'), ('X-RGW-Bytes-Used', '147'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001fc67b54407c462b-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.919 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:17:58.919 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:17:58.919 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.919 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:17:58.919 DEBUG:boto:Data: 2024-04-16T18:17:58.919 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.919 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.919 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.919 DEBUG:boto:Params: {} 2024-04-16T18:17:58.919 DEBUG:boto:Token: None 2024-04-16T18:17:58.919 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:17:58.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:S+9TIOjsh8qv94LEuxRl6hqFsFo= 2024-04-16T18:17:58.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:S+9TIOjsh8qv94LEuxRl6hqFsFo='} 2024-04-16T18:17:58.962 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003f0b9ddb2b23b7ea-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.963 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.963 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.963 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.963 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.963 DEBUG:boto:Data: 2024-04-16T18:17:58.963 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.963 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.963 DEBUG:boto:Params: {} 2024-04-16T18:17:58.963 DEBUG:boto:Token: None 2024-04-16T18:17:58.964 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.969 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '49'), ('X-RGW-Bytes-Used', '147'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004c1baa70a8382aae-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.970 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:17:58.970 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:17:58.970 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.970 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423 2024-04-16T18:17:58.970 DEBUG:boto:Data: 2024-04-16T18:17:58.970 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.970 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.970 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.971 DEBUG:boto:Params: {} 2024-04-16T18:17:58.971 DEBUG:boto:Token: None 2024-04-16T18:17:58.971 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000423 2024-04-16T18:17:58.971 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cUY6usVUrkOpvV5koiPfWf6rhpU= 2024-04-16T18:17:58.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cUY6usVUrkOpvV5koiPfWf6rhpU='} 2024-04-16T18:17:58.984 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000058ebe2a2692bd7d-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.984 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:58.984 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:58.984 DEBUG:boto:Method: HEAD 2024-04-16T18:17:58.984 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:58.984 DEBUG:boto:Data: 2024-04-16T18:17:58.984 DEBUG:boto:Headers: {} 2024-04-16T18:17:58.984 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.984 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.984 DEBUG:boto:Params: {} 2024-04-16T18:17:58.984 DEBUG:boto:Token: None 2024-04-16T18:17:58.985 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/ 2024-04-16T18:17:58.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk= 2024-04-16T18:17:58.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Uy7Xq+F2DI5nilRn7fv86SP0jGk=', 'Content-Length': '0'} 2024-04-16T18:17:58.990 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '50'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e17b80eae0aefcb4-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:58.991 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:58.991 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:17:58.991 DEBUG:boto:Method: PUT 2024-04-16T18:17:58.991 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:17:58.991 DEBUG:boto:Data: 2024-04-16T18:17:58.991 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:58.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:58.991 DEBUG:boto:Port: 8000 2024-04-16T18:17:58.991 DEBUG:boto:Params: {} 2024-04-16T18:17:58.991 DEBUG:boto:Token: None 2024-04-16T18:17:58.992 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:58 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:17:58.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5IK+0JmXenKmqQg977sZTZrd0PQ= 2024-04-16T18:17:58.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5IK+0JmXenKmqQg977sZTZrd0PQ='} 2024-04-16T18:17:59.034 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d32ba6c8e0fa7a32-00661ec0d6-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.035 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.035 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.035 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.035 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.035 DEBUG:boto:Data: 2024-04-16T18:17:59.035 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.035 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.035 DEBUG:boto:Params: {} 2024-04-16T18:17:59.035 DEBUG:boto:Token: None 2024-04-16T18:17:59.035 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.041 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '50'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cce47d5526321c39-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.041 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:59.042 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:17:59.042 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.042 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:17:59.042 DEBUG:boto:Data: 2024-04-16T18:17:59.042 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.042 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.042 DEBUG:boto:Params: {} 2024-04-16T18:17:59.042 DEBUG:boto:Token: None 2024-04-16T18:17:59.042 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:17:59.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b51k6/dw86E9+fqC+kkL2xzV61A= 2024-04-16T18:17:59.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b51k6/dw86E9+fqC+kkL2xzV61A='} 2024-04-16T18:17:59.059 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ae2515ecd0f35d53-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.059 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.060 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.060 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.060 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.060 DEBUG:boto:Data: 2024-04-16T18:17:59.060 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.060 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.060 DEBUG:boto:Params: {} 2024-04-16T18:17:59.060 DEBUG:boto:Token: None 2024-04-16T18:17:59.061 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.066 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '50'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007abbb89f89bec2bd-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.066 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:17:59.067 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:17:59.067 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.067 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029 2024-04-16T18:17:59.067 DEBUG:boto:Data: 2024-04-16T18:17:59.067 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.067 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.067 DEBUG:boto:Params: {} 2024-04-16T18:17:59.067 DEBUG:boto:Token: None 2024-04-16T18:17:59.067 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000029 2024-04-16T18:17:59.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TnaXzYGkZoiQwPx3PWuo9qM18IU= 2024-04-16T18:17:59.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TnaXzYGkZoiQwPx3PWuo9qM18IU='} 2024-04-16T18:17:59.110 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002344cf55e7fd56af-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.111 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.111 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.111 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.111 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.111 DEBUG:boto:Data: 2024-04-16T18:17:59.111 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.111 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.111 DEBUG:boto:Params: {} 2024-04-16T18:17:59.111 DEBUG:boto:Token: None 2024-04-16T18:17:59.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.118 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b8e1339c2e84cb91-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.118 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:17:59.118 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:17:59.118 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.118 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050 2024-04-16T18:17:59.119 DEBUG:boto:Data: 2024-04-16T18:17:59.119 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.119 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.119 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.119 DEBUG:boto:Params: {} 2024-04-16T18:17:59.119 DEBUG:boto:Token: None 2024-04-16T18:17:59.119 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000050 2024-04-16T18:17:59.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1yjG7F1Vrdb+qV7gqEh0r2eP5is= 2024-04-16T18:17:59.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1yjG7F1Vrdb+qV7gqEh0r2eP5is='} 2024-04-16T18:17:59.135 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005f3e74cffdd59c6a-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.136 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.136 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.136 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.136 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.136 DEBUG:boto:Data: 2024-04-16T18:17:59.136 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.136 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.136 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.136 DEBUG:boto:Params: {} 2024-04-16T18:17:59.136 DEBUG:boto:Token: None 2024-04-16T18:17:59.137 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.142 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a8a120d23261acdc-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.143 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:59.143 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:17:59.143 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.143 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:17:59.143 DEBUG:boto:Data: 2024-04-16T18:17:59.143 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.143 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.143 DEBUG:boto:Params: {} 2024-04-16T18:17:59.144 DEBUG:boto:Token: None 2024-04-16T18:17:59.144 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:17:59.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yxiprvVDc3kQ2VwxMaU0lghKXY8= 2024-04-16T18:17:59.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yxiprvVDc3kQ2VwxMaU0lghKXY8='} 2024-04-16T18:17:59.186 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e22865c98bc742dc-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.187 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.187 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.187 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.187 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.187 DEBUG:boto:Data: 2024-04-16T18:17:59.187 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.187 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.187 DEBUG:boto:Params: {} 2024-04-16T18:17:59.187 DEBUG:boto:Token: None 2024-04-16T18:17:59.188 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.193 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000012e9e65ca87e4149-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.194 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:17:59.194 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:17:59.194 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.194 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022 2024-04-16T18:17:59.194 DEBUG:boto:Data: 2024-04-16T18:17:59.194 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.194 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.194 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.194 DEBUG:boto:Params: {} 2024-04-16T18:17:59.194 DEBUG:boto:Token: None 2024-04-16T18:17:59.195 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000022 2024-04-16T18:17:59.195 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0paQxWiNMP3/p6dF3mL+5jXL++8= 2024-04-16T18:17:59.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0paQxWiNMP3/p6dF3mL+5jXL++8='} 2024-04-16T18:17:59.213 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000009a7d923aea4478-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.213 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.213 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.213 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.213 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.213 DEBUG:boto:Data: 2024-04-16T18:17:59.213 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.213 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.213 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.213 DEBUG:boto:Params: {} 2024-04-16T18:17:59.214 DEBUG:boto:Token: None 2024-04-16T18:17:59.214 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.214 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.220 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000021c9d0e40a3d3dc-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.220 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:59.220 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:17:59.220 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.221 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:17:59.221 DEBUG:boto:Data: 2024-04-16T18:17:59.221 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.221 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.221 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.221 DEBUG:boto:Params: {} 2024-04-16T18:17:59.221 DEBUG:boto:Token: None 2024-04-16T18:17:59.221 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:17:59.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fGEFXtYV06OmswaPNrIVzT1I1X4= 2024-04-16T18:17:59.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fGEFXtYV06OmswaPNrIVzT1I1X4='} 2024-04-16T18:17:59.266 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000094e87997016dceee-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.267 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.267 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.267 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.267 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.267 DEBUG:boto:Data: 2024-04-16T18:17:59.267 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.267 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.267 DEBUG:boto:Params: {} 2024-04-16T18:17:59.268 DEBUG:boto:Token: None 2024-04-16T18:17:59.268 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.273 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000786c81ebd5db15fc-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.274 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:59.274 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:17:59.274 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.274 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:17:59.274 DEBUG:boto:Data: 2024-04-16T18:17:59.274 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.274 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.274 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.274 DEBUG:boto:Params: {} 2024-04-16T18:17:59.274 DEBUG:boto:Token: None 2024-04-16T18:17:59.275 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:17:59.275 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tvc1WFzlB3qH0bByLJQdU7QPfbU= 2024-04-16T18:17:59.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tvc1WFzlB3qH0bByLJQdU7QPfbU='} 2024-04-16T18:17:59.291 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009e452470a1a4c98d-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.291 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.292 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.292 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.292 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.292 DEBUG:boto:Data: 2024-04-16T18:17:59.292 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.292 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.292 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.292 DEBUG:boto:Params: {} 2024-04-16T18:17:59.292 DEBUG:boto:Token: None 2024-04-16T18:17:59.292 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.298 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c0a4c20a7eeecca0-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.298 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:59.298 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:17:59.298 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.298 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:17:59.299 DEBUG:boto:Data: 2024-04-16T18:17:59.299 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.299 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.299 DEBUG:boto:Params: {} 2024-04-16T18:17:59.299 DEBUG:boto:Token: None 2024-04-16T18:17:59.299 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:17:59.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YY1E+kkixCudiZHx5D1OM9OmFaU= 2024-04-16T18:17:59.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YY1E+kkixCudiZHx5D1OM9OmFaU='} 2024-04-16T18:17:59.342 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a78bd6effd6baa6c-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.343 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.343 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.343 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.343 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.343 DEBUG:boto:Data: 2024-04-16T18:17:59.343 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.343 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.343 DEBUG:boto:Params: {} 2024-04-16T18:17:59.343 DEBUG:boto:Token: None 2024-04-16T18:17:59.343 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.350 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b1d2f1ce58b954c8-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.350 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:59.350 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:17:59.350 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.350 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:17:59.350 DEBUG:boto:Data: 2024-04-16T18:17:59.350 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.351 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.351 DEBUG:boto:Params: {} 2024-04-16T18:17:59.351 DEBUG:boto:Token: None 2024-04-16T18:17:59.351 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:17:59.351 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P0kwSO8+BV9LYHcTlnFidBj7/uw= 2024-04-16T18:17:59.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P0kwSO8+BV9LYHcTlnFidBj7/uw='} 2024-04-16T18:17:59.368 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000036b1029c63819222-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.368 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.368 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.368 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.368 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.369 DEBUG:boto:Data: 2024-04-16T18:17:59.369 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.369 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.369 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.369 DEBUG:boto:Params: {} 2024-04-16T18:17:59.369 DEBUG:boto:Token: None 2024-04-16T18:17:59.369 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.376 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000097e943d7c7278bcd-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.376 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:59.376 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:17:59.377 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.377 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:17:59.377 DEBUG:boto:Data: 2024-04-16T18:17:59.377 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.377 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.377 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.377 DEBUG:boto:Params: {} 2024-04-16T18:17:59.377 DEBUG:boto:Token: None 2024-04-16T18:17:59.377 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:17:59.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aW+7WS9sFdtNt9sU8eww1PKSDwM= 2024-04-16T18:17:59.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aW+7WS9sFdtNt9sU8eww1PKSDwM='} 2024-04-16T18:17:59.422 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008de861d6ef3faa40-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.423 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.423 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.423 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.423 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.423 DEBUG:boto:Data: 2024-04-16T18:17:59.423 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.423 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.423 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.423 DEBUG:boto:Params: {} 2024-04-16T18:17:59.424 DEBUG:boto:Token: None 2024-04-16T18:17:59.424 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.429 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cfebd49fc7fcbc03-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.429 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:17:59.429 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:17:59.429 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.429 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:17:59.429 DEBUG:boto:Data: 2024-04-16T18:17:59.430 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.430 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.430 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.430 DEBUG:boto:Params: {} 2024-04-16T18:17:59.430 DEBUG:boto:Token: None 2024-04-16T18:17:59.430 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:17:59.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LQfBrixcRY3sn2Lf/49pvG2Fxs8= 2024-04-16T18:17:59.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LQfBrixcRY3sn2Lf/49pvG2Fxs8='} 2024-04-16T18:17:59.447 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000da7c4b3e42722c2a-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.447 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.448 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.448 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.448 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.448 DEBUG:boto:Data: 2024-04-16T18:17:59.448 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.448 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.448 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.448 DEBUG:boto:Params: {} 2024-04-16T18:17:59.448 DEBUG:boto:Token: None 2024-04-16T18:17:59.449 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.449 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.454 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '51'), ('X-RGW-Bytes-Used', '153'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002bb10831b85e62eb-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.455 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:17:59.455 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:17:59.455 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.455 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056 2024-04-16T18:17:59.455 DEBUG:boto:Data: 2024-04-16T18:17:59.455 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.455 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.455 DEBUG:boto:Params: {} 2024-04-16T18:17:59.455 DEBUG:boto:Token: None 2024-04-16T18:17:59.456 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000056 2024-04-16T18:17:59.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sE112AB2wMLEiDA7X5yYDDmN4ag= 2024-04-16T18:17:59.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sE112AB2wMLEiDA7X5yYDDmN4ag='} 2024-04-16T18:17:59.498 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000e1103fc45f56aeb-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.499 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:17:59.499 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:17:59.499 DEBUG:boto:Method: HEAD 2024-04-16T18:17:59.499 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:17:59.499 DEBUG:boto:Data: 2024-04-16T18:17:59.499 DEBUG:boto:Headers: {} 2024-04-16T18:17:59.499 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.499 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.499 DEBUG:boto:Params: {} 2024-04-16T18:17:59.499 DEBUG:boto:Token: None 2024-04-16T18:17:59.500 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/ 2024-04-16T18:17:59.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU= 2024-04-16T18:17:59.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kJV1N04fENw77oVOEx3l108DDU=', 'Content-Length': '0'} 2024-04-16T18:17:59.506 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '52'), ('X-RGW-Bytes-Used', '156'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d2f3f52a50fd06a8-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.506 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:17:59.506 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:17:59.506 DEBUG:boto:Method: PUT 2024-04-16T18:17:59.506 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283 2024-04-16T18:17:59.506 DEBUG:boto:Data: 2024-04-16T18:17:59.506 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:17:59.506 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:17:59.506 DEBUG:boto:Port: 8000 2024-04-16T18:17:59.507 DEBUG:boto:Params: {} 2024-04-16T18:17:59.507 DEBUG:boto:Token: None 2024-04-16T18:17:59.507 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:17:59 GMT /lybtug-51/obj00000000000000000283 2024-04-16T18:17:59.507 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:upo/V8CB0y8UouatLOSnIe1OF5A= 2024-04-16T18:17:59.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:17:59 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:upo/V8CB0y8UouatLOSnIe1OF5A='} 2024-04-16T18:17:59.519 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000030068d293c24913-00661ec0d7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:17:59.519 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '37', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:59.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--num-shards', '37', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:17:59.519 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --num-shards 37 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:17:59.564 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:59.564 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:17:59.671 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:17:59.671 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-51 2024-04-16T18:17:59.695 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 53 2024-04-16T18:17:59.852 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '37', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:59.852 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '37', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:17:59.852 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 37 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:17:59.937 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:17:59.937 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:00.032 INFO:teuthology.orchestra.run.smithi071.stdout:{"objs":["obj00000000000000000060","obj00000000000000000000","obj00000000000000000011","obj00000000000000000001","obj00000000000000000033","obj00000000000000000065","obj00000000000000000005","obj00000000000000000016","obj00000000000000000048","obj00000000000000000015","obj00000000000000000026","obj00000000000000000030","obj00000000000000000062","obj00000000000000000009","obj00000000000000000040","obj00000000000000000024","obj00000000000000000012","obj00000000000000000002","obj00000000000000000013","obj00000000000000000045","obj00000000000000000006","obj00000000000000000017","obj00000000000000000007","obj00000000000000000039","obj00000000000000000027","obj00000000000000000010","obj00000000000000000021","obj00000000000000000053","obj00000000000000000020","obj00000000000000000004","obj00000000000000000036","obj00000000000000000003","obj00000000000000000014","obj00000000000000000019","obj00000000000000000057","obj00000000000000000018","obj00000000000000000008"]} 2024-04-16T18:18:00.033 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.033 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.033 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.033 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.033 DEBUG:boto:Data: 2024-04-16T18:18:00.033 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.033 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.033 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.033 DEBUG:boto:Params: {} 2024-04-16T18:18:00.033 DEBUG:boto:Token: None 2024-04-16T18:18:00.034 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.034 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.038 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '53'), ('X-RGW-Bytes-Used', '159'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002402a5c04395d1bb-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.039 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:00.039 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:00.039 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.039 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045 2024-04-16T18:18:00.039 DEBUG:boto:Data: 2024-04-16T18:18:00.039 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.039 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.039 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.040 DEBUG:boto:Params: {} 2024-04-16T18:18:00.040 DEBUG:boto:Token: None 2024-04-16T18:18:00.040 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000045 2024-04-16T18:18:00.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IJHevIRB6fL9Ps1U2HLQnqtw8o8= 2024-04-16T18:18:00.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IJHevIRB6fL9Ps1U2HLQnqtw8o8='} 2024-04-16T18:18:00.086 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000942e3f587fcafc9e-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.087 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.087 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.087 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.087 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.087 DEBUG:boto:Data: 2024-04-16T18:18:00.087 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.087 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.087 DEBUG:boto:Params: {} 2024-04-16T18:18:00.087 DEBUG:boto:Token: None 2024-04-16T18:18:00.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.093 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '53'), ('X-RGW-Bytes-Used', '159'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000433c4e7f4b9af945-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.093 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:00.093 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:00.093 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.093 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:18:00.093 DEBUG:boto:Data: 2024-04-16T18:18:00.093 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.094 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.094 DEBUG:boto:Params: {} 2024-04-16T18:18:00.094 DEBUG:boto:Token: None 2024-04-16T18:18:00.094 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:18:00.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:thsuwr0aho7lsCBGtoAfpeCDFaQ= 2024-04-16T18:18:00.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:thsuwr0aho7lsCBGtoAfpeCDFaQ='} 2024-04-16T18:18:00.108 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fa8b07c8112f6af0-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.108 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.108 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.108 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.108 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.108 DEBUG:boto:Data: 2024-04-16T18:18:00.108 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.108 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.109 DEBUG:boto:Params: {} 2024-04-16T18:18:00.109 DEBUG:boto:Token: None 2024-04-16T18:18:00.109 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.109 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.114 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '53'), ('X-RGW-Bytes-Used', '159'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008fc7a2bf37ff273b-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.114 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:00.114 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:00.114 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.114 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:18:00.114 DEBUG:boto:Data: 2024-04-16T18:18:00.115 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.115 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.115 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.115 DEBUG:boto:Params: {} 2024-04-16T18:18:00.115 DEBUG:boto:Token: None 2024-04-16T18:18:00.115 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:18:00.115 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cWJpaZKysMBhJLkZUE9L8Mp8ORU= 2024-04-16T18:18:00.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cWJpaZKysMBhJLkZUE9L8Mp8ORU='} 2024-04-16T18:18:00.162 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f000848aa3fcf7d9-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.163 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.163 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.163 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.163 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.163 DEBUG:boto:Data: 2024-04-16T18:18:00.163 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.163 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.163 DEBUG:boto:Params: {} 2024-04-16T18:18:00.163 DEBUG:boto:Token: None 2024-04-16T18:18:00.163 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.169 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '53'), ('X-RGW-Bytes-Used', '159'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004278cec608bc6203-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.169 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:00.169 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:00.170 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.170 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017 2024-04-16T18:18:00.170 DEBUG:boto:Data: 2024-04-16T18:18:00.170 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.170 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.170 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.170 DEBUG:boto:Params: {} 2024-04-16T18:18:00.170 DEBUG:boto:Token: None 2024-04-16T18:18:00.171 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000017 2024-04-16T18:18:00.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NT5idH44vPll7nM8+go0A1YZBTU= 2024-04-16T18:18:00.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NT5idH44vPll7nM8+go0A1YZBTU='} 2024-04-16T18:18:00.188 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a4544fa22521a093-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.189 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.189 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.189 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.189 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.189 DEBUG:boto:Data: 2024-04-16T18:18:00.189 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.189 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.189 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.189 DEBUG:boto:Params: {} 2024-04-16T18:18:00.189 DEBUG:boto:Token: None 2024-04-16T18:18:00.190 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.190 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.196 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '53'), ('X-RGW-Bytes-Used', '159'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000060bffdbcb6587978-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.196 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:00.196 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:00.196 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.196 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036 2024-04-16T18:18:00.196 DEBUG:boto:Data: 2024-04-16T18:18:00.196 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.196 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.196 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.197 DEBUG:boto:Params: {} 2024-04-16T18:18:00.197 DEBUG:boto:Token: None 2024-04-16T18:18:00.197 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000036 2024-04-16T18:18:00.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xaqrnkieDyoybin1QMhzdlNssvQ= 2024-04-16T18:18:00.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xaqrnkieDyoybin1QMhzdlNssvQ='} 2024-04-16T18:18:00.242 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ddff5895d627d18f-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.243 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.243 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.243 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.243 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.243 DEBUG:boto:Data: 2024-04-16T18:18:00.243 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.243 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.243 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.243 DEBUG:boto:Params: {} 2024-04-16T18:18:00.244 DEBUG:boto:Token: None 2024-04-16T18:18:00.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.249 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000028af97619d3953b6-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.250 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:00.250 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:00.250 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.250 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040 2024-04-16T18:18:00.250 DEBUG:boto:Data: 2024-04-16T18:18:00.250 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.250 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.250 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.251 DEBUG:boto:Params: {} 2024-04-16T18:18:00.251 DEBUG:boto:Token: None 2024-04-16T18:18:00.251 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000040 2024-04-16T18:18:00.251 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HQU0SFI0P71BYMI8fPeERASKOGc= 2024-04-16T18:18:00.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HQU0SFI0P71BYMI8fPeERASKOGc='} 2024-04-16T18:18:00.268 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a6a9a6677d33c88f-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.268 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.268 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.268 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.268 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.268 DEBUG:boto:Data: 2024-04-16T18:18:00.268 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.268 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.268 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.268 DEBUG:boto:Params: {} 2024-04-16T18:18:00.269 DEBUG:boto:Token: None 2024-04-16T18:18:00.269 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.269 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.274 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000edf2a092f05169b3-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.277 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:00.277 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:00.277 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.277 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:18:00.277 DEBUG:boto:Data: 2024-04-16T18:18:00.277 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.277 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.277 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.277 DEBUG:boto:Params: {} 2024-04-16T18:18:00.277 DEBUG:boto:Token: None 2024-04-16T18:18:00.278 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:18:00.278 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTATNwLSkTCPlpO/blDXT9N9vQ8= 2024-04-16T18:18:00.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTATNwLSkTCPlpO/blDXT9N9vQ8='} 2024-04-16T18:18:00.322 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000081cdd8b25b660a23-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.322 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.323 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.323 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.323 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.323 DEBUG:boto:Data: 2024-04-16T18:18:00.323 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.323 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.323 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.323 DEBUG:boto:Params: {} 2024-04-16T18:18:00.323 DEBUG:boto:Token: None 2024-04-16T18:18:00.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.329 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001cd0e42739228920-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.329 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:00.329 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:00.329 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.329 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:18:00.330 DEBUG:boto:Data: 2024-04-16T18:18:00.330 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.330 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.330 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.330 DEBUG:boto:Params: {} 2024-04-16T18:18:00.330 DEBUG:boto:Token: None 2024-04-16T18:18:00.330 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:18:00.330 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RZm8hlUNKllAZzjX+tMMG3WSa+M= 2024-04-16T18:18:00.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RZm8hlUNKllAZzjX+tMMG3WSa+M='} 2024-04-16T18:18:00.346 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001dc7f384ed69d3a4-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.346 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.346 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.347 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.347 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.347 DEBUG:boto:Data: 2024-04-16T18:18:00.347 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.347 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.347 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.347 DEBUG:boto:Params: {} 2024-04-16T18:18:00.347 DEBUG:boto:Token: None 2024-04-16T18:18:00.347 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.347 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.353 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000da743a659f7e840d-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.353 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:00.353 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:00.353 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.353 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:18:00.353 DEBUG:boto:Data: 2024-04-16T18:18:00.354 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.354 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.354 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.354 DEBUG:boto:Params: {} 2024-04-16T18:18:00.354 DEBUG:boto:Token: None 2024-04-16T18:18:00.354 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:18:00.354 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QOdlvb9YrOJt6hc4wrr9vxGQwkg= 2024-04-16T18:18:00.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QOdlvb9YrOJt6hc4wrr9vxGQwkg='} 2024-04-16T18:18:00.398 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001f7ec4bef952fe2a-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.398 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.399 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.399 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.399 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.399 DEBUG:boto:Data: 2024-04-16T18:18:00.399 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.399 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.399 DEBUG:boto:Params: {} 2024-04-16T18:18:00.399 DEBUG:boto:Token: None 2024-04-16T18:18:00.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.405 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000857098d830c50ada-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.406 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:00.406 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:00.406 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.406 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:18:00.406 DEBUG:boto:Data: 2024-04-16T18:18:00.406 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.406 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.407 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.407 DEBUG:boto:Params: {} 2024-04-16T18:18:00.407 DEBUG:boto:Token: None 2024-04-16T18:18:00.407 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:18:00.407 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8qtdjSzHwGq/2nio8BIcXJHWfpU= 2024-04-16T18:18:00.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8qtdjSzHwGq/2nio8BIcXJHWfpU='} 2024-04-16T18:18:00.425 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000675390bca139fe37-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.425 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.425 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.426 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.426 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.426 DEBUG:boto:Data: 2024-04-16T18:18:00.426 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.426 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.426 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.426 DEBUG:boto:Params: {} 2024-04-16T18:18:00.426 DEBUG:boto:Token: None 2024-04-16T18:18:00.426 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.427 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.432 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000643af64fa5179088-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.432 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:00.432 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:00.432 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.432 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:18:00.433 DEBUG:boto:Data: 2024-04-16T18:18:00.433 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.433 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.433 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.433 DEBUG:boto:Params: {} 2024-04-16T18:18:00.433 DEBUG:boto:Token: None 2024-04-16T18:18:00.433 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:18:00.433 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x93SbKuqAVlXZgfBdLHQioAHbpA= 2024-04-16T18:18:00.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x93SbKuqAVlXZgfBdLHQioAHbpA='} 2024-04-16T18:18:00.478 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c564f9af3af8148c-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.481 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.481 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.481 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.481 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.481 DEBUG:boto:Data: 2024-04-16T18:18:00.481 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.482 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.482 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.482 DEBUG:boto:Params: {} 2024-04-16T18:18:00.482 DEBUG:boto:Token: None 2024-04-16T18:18:00.482 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.482 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.482 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.489 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007d8a83ff2b97af5f-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.489 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:00.490 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:00.490 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.490 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:18:00.490 DEBUG:boto:Data: 2024-04-16T18:18:00.490 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.490 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.490 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.490 DEBUG:boto:Params: {} 2024-04-16T18:18:00.490 DEBUG:boto:Token: None 2024-04-16T18:18:00.490 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:18:00.491 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jeWP72L3aF9aMhHCBpOp50nMRsk= 2024-04-16T18:18:00.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jeWP72L3aF9aMhHCBpOp50nMRsk='} 2024-04-16T18:18:00.506 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a9783ab382cd8d68-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.507 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.507 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.507 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.507 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.507 DEBUG:boto:Data: 2024-04-16T18:18:00.507 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.507 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.507 DEBUG:boto:Params: {} 2024-04-16T18:18:00.507 DEBUG:boto:Token: None 2024-04-16T18:18:00.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.513 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000027cfcfb6d95d1a62-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.513 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:00.514 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:00.514 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.514 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:18:00.514 DEBUG:boto:Data: 2024-04-16T18:18:00.514 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.514 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.514 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.514 DEBUG:boto:Params: {} 2024-04-16T18:18:00.514 DEBUG:boto:Token: None 2024-04-16T18:18:00.515 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:18:00.515 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jzFe3MPuqDFiclhu7Zb3DksOIN8= 2024-04-16T18:18:00.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jzFe3MPuqDFiclhu7Zb3DksOIN8='} 2024-04-16T18:18:00.558 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000061f094d9ebb80012-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.559 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.559 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.559 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.559 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.559 DEBUG:boto:Data: 2024-04-16T18:18:00.559 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.559 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.559 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.559 DEBUG:boto:Params: {} 2024-04-16T18:18:00.559 DEBUG:boto:Token: None 2024-04-16T18:18:00.559 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.565 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '54'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008b78f7523788ce4e-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.565 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:00.566 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:00.566 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.566 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048 2024-04-16T18:18:00.566 DEBUG:boto:Data: 2024-04-16T18:18:00.566 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.566 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.566 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.566 DEBUG:boto:Params: {} 2024-04-16T18:18:00.566 DEBUG:boto:Token: None 2024-04-16T18:18:00.566 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000048 2024-04-16T18:18:00.567 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3yBeYxVtDWK0IN0ddZKeRPGc9TE= 2024-04-16T18:18:00.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3yBeYxVtDWK0IN0ddZKeRPGc9TE='} 2024-04-16T18:18:00.579 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000026a2dbf881a7bfb-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.579 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.579 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.579 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.579 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.580 DEBUG:boto:Data: 2024-04-16T18:18:00.580 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.580 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.580 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.580 DEBUG:boto:Params: {} 2024-04-16T18:18:00.580 DEBUG:boto:Token: None 2024-04-16T18:18:00.580 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.586 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '55'), ('X-RGW-Bytes-Used', '165'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000033880fa18554e156-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.586 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:00.586 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:00.586 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.586 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065 2024-04-16T18:18:00.586 DEBUG:boto:Data: 2024-04-16T18:18:00.586 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.587 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.587 DEBUG:boto:Params: {} 2024-04-16T18:18:00.587 DEBUG:boto:Token: None 2024-04-16T18:18:00.587 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000065 2024-04-16T18:18:00.587 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z0QvkFytYOIuiYflO2tQUQKU7A0= 2024-04-16T18:18:00.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z0QvkFytYOIuiYflO2tQUQKU7A0='} 2024-04-16T18:18:00.630 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c304c5a9d08a3805-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.631 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.631 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.631 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.631 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.631 DEBUG:boto:Data: 2024-04-16T18:18:00.631 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.631 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.631 DEBUG:boto:Params: {} 2024-04-16T18:18:00.631 DEBUG:boto:Token: None 2024-04-16T18:18:00.631 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.637 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '56'), ('X-RGW-Bytes-Used', '168'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000826e6824b3371961-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.638 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:00.638 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:00.638 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.638 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027 2024-04-16T18:18:00.638 DEBUG:boto:Data: 2024-04-16T18:18:00.638 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.638 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.638 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.638 DEBUG:boto:Params: {} 2024-04-16T18:18:00.638 DEBUG:boto:Token: None 2024-04-16T18:18:00.639 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000027 2024-04-16T18:18:00.639 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c1Ho4qzqV+QFRUcIC8FFxKa9I18= 2024-04-16T18:18:00.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c1Ho4qzqV+QFRUcIC8FFxKa9I18='} 2024-04-16T18:18:00.654 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b5a8f133f6461c8e-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.655 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.655 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.655 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.655 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.655 DEBUG:boto:Data: 2024-04-16T18:18:00.655 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.655 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.655 DEBUG:boto:Params: {} 2024-04-16T18:18:00.655 DEBUG:boto:Token: None 2024-04-16T18:18:00.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.662 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '56'), ('X-RGW-Bytes-Used', '168'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000369a0559994313ac-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.662 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:00.662 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:00.662 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.662 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057 2024-04-16T18:18:00.662 DEBUG:boto:Data: 2024-04-16T18:18:00.662 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.662 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.662 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.663 DEBUG:boto:Params: {} 2024-04-16T18:18:00.663 DEBUG:boto:Token: None 2024-04-16T18:18:00.663 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000057 2024-04-16T18:18:00.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:axQsGC7odIwVdnx9APHxxynr4Y0= 2024-04-16T18:18:00.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:axQsGC7odIwVdnx9APHxxynr4Y0='} 2024-04-16T18:18:00.706 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006ef534b60cdc18aa-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.707 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.707 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.707 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.707 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.707 DEBUG:boto:Data: 2024-04-16T18:18:00.707 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.707 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.707 DEBUG:boto:Params: {} 2024-04-16T18:18:00.707 DEBUG:boto:Token: None 2024-04-16T18:18:00.708 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.713 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '57'), ('X-RGW-Bytes-Used', '171'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a02e1c62e5dd4d76-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.713 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:00.713 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:00.714 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.714 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053 2024-04-16T18:18:00.714 DEBUG:boto:Data: 2024-04-16T18:18:00.714 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.714 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.714 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.714 DEBUG:boto:Params: {} 2024-04-16T18:18:00.714 DEBUG:boto:Token: None 2024-04-16T18:18:00.714 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000053 2024-04-16T18:18:00.714 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14yUuexpoSeTxkvd6biYNFwzBQE= 2024-04-16T18:18:00.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14yUuexpoSeTxkvd6biYNFwzBQE='} 2024-04-16T18:18:00.726 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004f9c69918d3d50d0-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.727 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.727 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.727 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.727 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.727 DEBUG:boto:Data: 2024-04-16T18:18:00.727 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.727 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.727 DEBUG:boto:Params: {} 2024-04-16T18:18:00.727 DEBUG:boto:Token: None 2024-04-16T18:18:00.728 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.733 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002e4040c73fb94142-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.733 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:00.733 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:00.733 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.733 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039 2024-04-16T18:18:00.734 DEBUG:boto:Data: 2024-04-16T18:18:00.734 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.734 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.734 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.734 DEBUG:boto:Params: {} 2024-04-16T18:18:00.734 DEBUG:boto:Token: None 2024-04-16T18:18:00.734 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000039 2024-04-16T18:18:00.734 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:APbAmbgmhPbzKEy8JjTWSqPEwDc= 2024-04-16T18:18:00.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:APbAmbgmhPbzKEy8JjTWSqPEwDc='} 2024-04-16T18:18:00.778 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000eedd5c9552610add-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.779 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.779 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.779 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.779 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.779 DEBUG:boto:Data: 2024-04-16T18:18:00.779 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.779 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.779 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.779 DEBUG:boto:Params: {} 2024-04-16T18:18:00.779 DEBUG:boto:Token: None 2024-04-16T18:18:00.779 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.785 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000ec9ef4c9bd3a26e-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.786 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:00.786 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:00.786 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.786 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:18:00.786 DEBUG:boto:Data: 2024-04-16T18:18:00.786 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.786 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.787 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.787 DEBUG:boto:Params: {} 2024-04-16T18:18:00.787 DEBUG:boto:Token: None 2024-04-16T18:18:00.787 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:18:00.787 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8GDz1rydZ7hwUVO2vOK+Gk8fAa8= 2024-04-16T18:18:00.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8GDz1rydZ7hwUVO2vOK+Gk8fAa8='} 2024-04-16T18:18:00.803 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000037badb40ba366040-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.803 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.803 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.804 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.804 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.804 DEBUG:boto:Data: 2024-04-16T18:18:00.804 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.804 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.804 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.804 DEBUG:boto:Params: {} 2024-04-16T18:18:00.804 DEBUG:boto:Token: None 2024-04-16T18:18:00.804 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.810 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000027989a1888f7c07e-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.810 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:00.810 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:00.810 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.810 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:18:00.810 DEBUG:boto:Data: 2024-04-16T18:18:00.810 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.810 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.810 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.810 DEBUG:boto:Params: {} 2024-04-16T18:18:00.811 DEBUG:boto:Token: None 2024-04-16T18:18:00.811 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:18:00.811 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SQD4DbIjMz10Y3+XPA9Os1LzT+E= 2024-04-16T18:18:00.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SQD4DbIjMz10Y3+XPA9Os1LzT+E='} 2024-04-16T18:18:00.854 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000005c15320176baecd-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.855 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.855 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.855 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.855 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.855 DEBUG:boto:Data: 2024-04-16T18:18:00.855 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.855 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.855 DEBUG:boto:Params: {} 2024-04-16T18:18:00.855 DEBUG:boto:Token: None 2024-04-16T18:18:00.855 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.862 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000037beccd33ff82c1d-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.862 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:00.862 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:00.862 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.862 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:18:00.862 DEBUG:boto:Data: 2024-04-16T18:18:00.862 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.862 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.863 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.863 DEBUG:boto:Params: {} 2024-04-16T18:18:00.863 DEBUG:boto:Token: None 2024-04-16T18:18:00.863 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:18:00.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bHyw13ItrGAZuSsaLuDaBvu1jMg= 2024-04-16T18:18:00.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bHyw13ItrGAZuSsaLuDaBvu1jMg='} 2024-04-16T18:18:00.879 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000fb49246f10cef48-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.879 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.879 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.879 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.879 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.879 DEBUG:boto:Data: 2024-04-16T18:18:00.879 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.879 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.879 DEBUG:boto:Params: {} 2024-04-16T18:18:00.880 DEBUG:boto:Token: None 2024-04-16T18:18:00.880 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.886 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b3bae50a973d5a14-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.886 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:00.886 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:00.886 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.886 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:18:00.886 DEBUG:boto:Data: 2024-04-16T18:18:00.886 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.886 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.886 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.887 DEBUG:boto:Params: {} 2024-04-16T18:18:00.887 DEBUG:boto:Token: None 2024-04-16T18:18:00.887 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:18:00.887 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uu57Dx+CaX6U2qHyWnX9mwow5N4= 2024-04-16T18:18:00.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uu57Dx+CaX6U2qHyWnX9mwow5N4='} 2024-04-16T18:18:00.930 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008801b82fb1c1e347-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.931 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.931 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.931 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.931 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.931 DEBUG:boto:Data: 2024-04-16T18:18:00.931 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.931 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.931 DEBUG:boto:Params: {} 2024-04-16T18:18:00.931 DEBUG:boto:Token: None 2024-04-16T18:18:00.931 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.937 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006b902940dcc9e65d-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.937 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:00.938 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:00.938 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.938 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:18:00.938 DEBUG:boto:Data: 2024-04-16T18:18:00.938 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.938 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.938 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.938 DEBUG:boto:Params: {} 2024-04-16T18:18:00.938 DEBUG:boto:Token: None 2024-04-16T18:18:00.938 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:18:00.939 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gvdZPT+5HNSJ2zktmsdaxHqCbHs= 2024-04-16T18:18:00.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gvdZPT+5HNSJ2zktmsdaxHqCbHs='} 2024-04-16T18:18:00.956 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a3a42a4f4f1d5ce0-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.956 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:00.956 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:00.957 DEBUG:boto:Method: HEAD 2024-04-16T18:18:00.957 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:00.957 DEBUG:boto:Data: 2024-04-16T18:18:00.957 DEBUG:boto:Headers: {} 2024-04-16T18:18:00.957 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.957 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.957 DEBUG:boto:Params: {} 2024-04-16T18:18:00.957 DEBUG:boto:Token: None 2024-04-16T18:18:00.957 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/ 2024-04-16T18:18:00.957 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U= 2024-04-16T18:18:00.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EBXLWWA4QjAsB/FFrC+asqCAd+U=', 'Content-Length': '0'} 2024-04-16T18:18:00.963 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008970fe260cb94181-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:00.963 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:00.964 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:00.964 DEBUG:boto:Method: PUT 2024-04-16T18:18:00.964 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:18:00.964 DEBUG:boto:Data: 2024-04-16T18:18:00.964 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:00.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:00.964 DEBUG:boto:Port: 8000 2024-04-16T18:18:00.964 DEBUG:boto:Params: {} 2024-04-16T18:18:00.964 DEBUG:boto:Token: None 2024-04-16T18:18:00.964 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:00 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:18:00.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+R4OME95hgakuxnK2f/eMkANMOs= 2024-04-16T18:18:00.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+R4OME95hgakuxnK2f/eMkANMOs='} 2024-04-16T18:18:01.006 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b2c5b98edb1e94e6-00661ec0d8-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.007 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.007 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.007 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.007 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.007 DEBUG:boto:Data: 2024-04-16T18:18:01.007 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.007 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.007 DEBUG:boto:Params: {} 2024-04-16T18:18:01.007 DEBUG:boto:Token: None 2024-04-16T18:18:01.007 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.013 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '58'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000087a373a3596e2cc9-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.014 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:01.014 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:01.014 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.014 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060 2024-04-16T18:18:01.014 DEBUG:boto:Data: 2024-04-16T18:18:01.014 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.014 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.014 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.014 DEBUG:boto:Params: {} 2024-04-16T18:18:01.014 DEBUG:boto:Token: None 2024-04-16T18:18:01.015 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000060 2024-04-16T18:18:01.015 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DogNV/jA/vka6Hrb62gFi+vEPlI= 2024-04-16T18:18:01.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DogNV/jA/vka6Hrb62gFi+vEPlI='} 2024-04-16T18:18:01.028 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ee49119dc5267694-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.028 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.029 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.029 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.029 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.029 DEBUG:boto:Data: 2024-04-16T18:18:01.029 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.029 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.029 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.029 DEBUG:boto:Params: {} 2024-04-16T18:18:01.029 DEBUG:boto:Token: None 2024-04-16T18:18:01.029 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.030 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.035 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '59'), ('X-RGW-Bytes-Used', '177'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e124b03b5accdb20-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.036 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:01.036 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:01.036 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.036 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:18:01.036 DEBUG:boto:Data: 2024-04-16T18:18:01.036 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.036 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.036 DEBUG:boto:Params: {} 2024-04-16T18:18:01.036 DEBUG:boto:Token: None 2024-04-16T18:18:01.037 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:18:01.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PN+2DHKHQa7IF4h+5TwO66zP590= 2024-04-16T18:18:01.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PN+2DHKHQa7IF4h+5TwO66zP590='} 2024-04-16T18:18:01.078 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000014698db36a0626f9-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.079 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.079 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.079 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.079 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.079 DEBUG:boto:Data: 2024-04-16T18:18:01.079 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.079 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.079 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.079 DEBUG:boto:Params: {} 2024-04-16T18:18:01.080 DEBUG:boto:Token: None 2024-04-16T18:18:01.080 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.086 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '59'), ('X-RGW-Bytes-Used', '177'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007f03c07da297e8f3-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.086 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:01.086 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:01.086 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.086 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062 2024-04-16T18:18:01.086 DEBUG:boto:Data: 2024-04-16T18:18:01.086 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.086 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.086 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.086 DEBUG:boto:Params: {} 2024-04-16T18:18:01.087 DEBUG:boto:Token: None 2024-04-16T18:18:01.087 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000062 2024-04-16T18:18:01.087 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AkgKimXdTlcewukHxeNQzNLAAY8= 2024-04-16T18:18:01.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AkgKimXdTlcewukHxeNQzNLAAY8='} 2024-04-16T18:18:01.099 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ae9e875b1919719b-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.100 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.100 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.100 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.100 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.100 DEBUG:boto:Data: 2024-04-16T18:18:01.100 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.100 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.100 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.100 DEBUG:boto:Params: {} 2024-04-16T18:18:01.101 DEBUG:boto:Token: None 2024-04-16T18:18:01.101 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.101 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.107 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ef51e3e0154b465d-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.107 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:01.107 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:01.107 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.107 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:18:01.108 DEBUG:boto:Data: 2024-04-16T18:18:01.108 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.108 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.108 DEBUG:boto:Params: {} 2024-04-16T18:18:01.108 DEBUG:boto:Token: None 2024-04-16T18:18:01.108 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:18:01.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yfe2SrN/UsZ6neUyOHKMa1ByXD8= 2024-04-16T18:18:01.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yfe2SrN/UsZ6neUyOHKMa1ByXD8='} 2024-04-16T18:18:01.150 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cdde3e533c25e0a5-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.151 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.151 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.151 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.151 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.151 DEBUG:boto:Data: 2024-04-16T18:18:01.151 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.151 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.151 DEBUG:boto:Params: {} 2024-04-16T18:18:01.151 DEBUG:boto:Token: None 2024-04-16T18:18:01.151 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.157 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000061b3838d9a371bdc-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.158 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:01.158 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:01.158 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.158 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024 2024-04-16T18:18:01.158 DEBUG:boto:Data: 2024-04-16T18:18:01.158 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.158 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.158 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.158 DEBUG:boto:Params: {} 2024-04-16T18:18:01.158 DEBUG:boto:Token: None 2024-04-16T18:18:01.159 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000024 2024-04-16T18:18:01.159 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OjYf6Yk8xOSu5Cojlu0WRAZSoJw= 2024-04-16T18:18:01.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OjYf6Yk8xOSu5Cojlu0WRAZSoJw='} 2024-04-16T18:18:01.177 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000266483c0887a41d2-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.177 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.177 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.177 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.178 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.178 DEBUG:boto:Data: 2024-04-16T18:18:01.178 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.178 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.178 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.178 DEBUG:boto:Params: {} 2024-04-16T18:18:01.178 DEBUG:boto:Token: None 2024-04-16T18:18:01.178 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.178 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.184 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000091b533c5d7e1aa3c-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.184 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:01.184 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:01.185 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.185 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:18:01.185 DEBUG:boto:Data: 2024-04-16T18:18:01.185 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.185 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.185 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.185 DEBUG:boto:Params: {} 2024-04-16T18:18:01.185 DEBUG:boto:Token: None 2024-04-16T18:18:01.185 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:18:01.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Mgmh+IDu0TE+PB+Pd63S8Ah/zE= 2024-04-16T18:18:01.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Mgmh+IDu0TE+PB+Pd63S8Ah/zE='} 2024-04-16T18:18:01.230 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000aac8ef063067e61a-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.231 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.231 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.231 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.231 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.231 DEBUG:boto:Data: 2024-04-16T18:18:01.231 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.231 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.231 DEBUG:boto:Params: {} 2024-04-16T18:18:01.231 DEBUG:boto:Token: None 2024-04-16T18:18:01.231 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.237 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007e9b4fe712ca5987-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.237 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:01.238 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:01.238 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.238 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:18:01.238 DEBUG:boto:Data: 2024-04-16T18:18:01.238 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.238 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.238 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.238 DEBUG:boto:Params: {} 2024-04-16T18:18:01.238 DEBUG:boto:Token: None 2024-04-16T18:18:01.239 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:18:01.239 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/ShPDnMKodys1NOTI22861WpzVE= 2024-04-16T18:18:01.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/ShPDnMKodys1NOTI22861WpzVE='} 2024-04-16T18:18:01.255 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008f7af36ce41d8339-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.255 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.255 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.255 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.256 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.256 DEBUG:boto:Data: 2024-04-16T18:18:01.256 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.256 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.256 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.256 DEBUG:boto:Params: {} 2024-04-16T18:18:01.256 DEBUG:boto:Token: None 2024-04-16T18:18:01.256 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.262 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002fe7e97b058422fd-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.262 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:01.262 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:01.262 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.262 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:18:01.262 DEBUG:boto:Data: 2024-04-16T18:18:01.262 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.262 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.262 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.263 DEBUG:boto:Params: {} 2024-04-16T18:18:01.263 DEBUG:boto:Token: None 2024-04-16T18:18:01.263 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:18:01.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n/HqFM+7w1MOu9g3i54pz/Rmivs= 2024-04-16T18:18:01.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n/HqFM+7w1MOu9g3i54pz/Rmivs='} 2024-04-16T18:18:01.306 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004f6adc2ba38f3f21-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.306 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:01.307 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:01.307 DEBUG:boto:Method: HEAD 2024-04-16T18:18:01.307 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:01.307 DEBUG:boto:Data: 2024-04-16T18:18:01.307 DEBUG:boto:Headers: {} 2024-04-16T18:18:01.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.307 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.307 DEBUG:boto:Params: {} 2024-04-16T18:18:01.307 DEBUG:boto:Token: None 2024-04-16T18:18:01.307 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/ 2024-04-16T18:18:01.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo= 2024-04-16T18:18:01.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JbC278xoU5MqiKZDVt7QpAqF2Bo=', 'Content-Length': '0'} 2024-04-16T18:18:01.313 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005436e4666ebfd61b-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.314 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:01.314 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:01.314 DEBUG:boto:Method: PUT 2024-04-16T18:18:01.314 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018 2024-04-16T18:18:01.314 DEBUG:boto:Data: 2024-04-16T18:18:01.314 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:18:01.314 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:01.314 DEBUG:boto:Port: 8000 2024-04-16T18:18:01.314 DEBUG:boto:Params: {} 2024-04-16T18:18:01.315 DEBUG:boto:Token: None 2024-04-16T18:18:01.315 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:18:01 GMT /lybtug-51/obj00000000000000000018 2024-04-16T18:18:01.315 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CW/HgLGNZx8uAx4MtpSjvkpyX2k= 2024-04-16T18:18:01.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:18:01 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CW/HgLGNZx8uAx4MtpSjvkpyX2k='} 2024-04-16T18:18:01.332 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001cf2dbe424b9dce5-00661ec0d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:01.332 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'run', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:18:01.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'run', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:18:01.332 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync run --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:18:01.379 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:01.379 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:01.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 2 2024-04-16T18:18:01.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000001.28.6 2024-04-16T18:18:01.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 2 2024-04-16T18:18:01.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position=00000000001.54.6 2024-04-16T18:18:01.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 2 2024-04-16T18:18:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position=00000000001.48.6 2024-04-16T18:18:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 2 2024-04-16T18:18:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000001.71.6 2024-04-16T18:18:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 2 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000001.53.6 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 2 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 2 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000001.39.6 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000001.38.6 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 2 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 2 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000001.121.6 2024-04-16T18:18:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 2 2024-04-16T18:18:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000001.64.6 2024-04-16T18:18:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 2 2024-04-16T18:18:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000001.30.6 2024-04-16T18:18:01.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:01.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:01.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 0 2024-04-16T18:18:01.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.656+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 0 2024-04-16T18:18:01.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:01.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000001.125.6 iter->object=obj00000000000000000024 iter->instance= name=obj00000000000000000024 instance= ns= 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000024 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:17:54.020406+0000 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000024]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.125.6 op=0 op_state=1 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000024]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000024[0] 2024-04-16T18:18:01.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000001.125.6 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000001.76.6 iter->object=obj00000000000000000020 iter->instance= name=obj00000000000000000020 instance= ns= 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000020 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:53.945282+0000 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.76.6 op=0 op_state=1 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: : injecting data sync error on key=obj00000000000000000020 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000001.76.6 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000020 2024-04-16T18:18:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.660+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: parsed entry: id=00000000002.76.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5/obj00000000000000000000 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: entry->timestamp=2024-04-16T18:17:53.997404+0000 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.76.6 op=0 op_state=1 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000000]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000000[0] 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000002.76.6 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: parsed entry: id=00000000001.31.6 iter->object=obj00000000000000000016 iter->instance= name=obj00000000000000000016 instance= ns= 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1/obj00000000000000000016 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: entry->timestamp=2024-04-16T18:17:54.091722+0000 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000016]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.31.6 op=0 op_state=1 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000016]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000016[0] 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position=00000000001.31.6 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket shard: 0 2024-04-16T18:18:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.664+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket shard: 0 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 0 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000002.52.6 iter->object=obj00000000000000000010 iter->instance= name=obj00000000000000000010 instance= ns= 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000010 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:17:53.587356+0000 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000010]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.52.6 op=0 op_state=1 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000010]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000010[0] 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000002.52.6 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket shard: 0 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket shard: 0 2024-04-16T18:18:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: parsed entry: id=00000000001.36.6 iter->object=obj00000000000000000021 iter->instance= name=obj00000000000000000021 instance= ns= 2024-04-16T18:18:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11/obj00000000000000000021 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: entry->timestamp=2024-04-16T18:17:53.789358+0000 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000021]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.36.6 op=0 op_state=1 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000021]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000021[0] 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position=00000000001.36.6 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000001.54.6 iter->object=obj00000000000000000030 iter->instance= name=obj00000000000000000030 instance= ns= 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000030 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:53.807106+0000 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000030]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.54.6 op=0 op_state=1 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000030]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000030[0] 2024-04-16T18:18:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000001.54.6 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000001.86.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000019 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:53.857432+0000 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.86.6 op=0 op_state=1 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: : injecting data sync error on key=obj00000000000000000019 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000001.86.6 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.672+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000019 2024-04-16T18:18:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: parsed entry: id=00000000001.44.6 iter->object=obj00000000000000000007 iter->instance= name=obj00000000000000000007 instance= ns= 2024-04-16T18:18:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0/obj00000000000000000007 2024-04-16T18:18:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: entry->timestamp=2024-04-16T18:17:53.720854+0000 2024-04-16T18:18:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000007]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.44.6 op=0 op_state=1 2024-04-16T18:18:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000007]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000007[0] 2024-04-16T18:18:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000001.44.6 2024-04-16T18:18:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000001.57.6 iter->object=obj00000000000000000028 iter->instance= name=obj00000000000000000028 instance= ns= 2024-04-16T18:18:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000028 2024-04-16T18:18:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:17:53.925661+0000 2024-04-16T18:18:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000028]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.57.6 op=0 op_state=1 2024-04-16T18:18:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000028]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000028[0] 2024-04-16T18:18:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000001.57.6 2024-04-16T18:18:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: a sync operation returned error 2024-04-16T18:18:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: backing out with sync_status=-5 2024-04-16T18:18:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.680+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.684+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000000]: success 2024-04-16T18:18:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5:1 marker=00000000002.76.6 timestamp=2024-04-16T18:17:53.997404+0000 2024-04-16T18:18:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000016]: success 2024-04-16T18:18:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1:1 marker=00000000001.31.6 timestamp=2024-04-16T18:17:54.091722+0000 2024-04-16T18:18:01.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000007]: success 2024-04-16T18:18:01.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0:1 marker=00000000001.44.6 timestamp=2024-04-16T18:17:53.720854+0000 2024-04-16T18:18:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000024]: success 2024-04-16T18:18:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2:1 marker=00000000001.125.6 timestamp=2024-04-16T18:17:54.020406+0000 2024-04-16T18:18:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000030]: success 2024-04-16T18:18:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.688+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12:1 marker=00000000001.54.6 timestamp=2024-04-16T18:17:53.807106+0000 2024-04-16T18:18:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.692+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000010]: success 2024-04-16T18:18:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.692+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8:1 marker=00000000002.52.6 timestamp=2024-04-16T18:17:53.587356+0000 2024-04-16T18:18:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: parsed entry: id=00000000001.21.6 iter->object=obj00000000000000000032 iter->instance= name=obj00000000000000000032 instance= ns= 2024-04-16T18:18:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6/obj00000000000000000032 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: entry->timestamp=2024-04-16T18:17:53.876796+0000 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000032]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.21.6 op=0 op_state=1 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000032]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000032[0] 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position=00000000001.21.6 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000001 iter->instance= name=obj00000000000000000001 instance= ns= 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000001 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:17:53.739624+0000 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: : injecting data sync error on key=obj00000000000000000001 2024-04-16T18:18:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.708+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000001 2024-04-16T18:18:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: a sync operation returned error 2024-04-16T18:18:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: backing out with sync_status=-5 2024-04-16T18:18:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.712+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: parsed entry: id=00000000001.83.6 iter->object=obj00000000000000000033 iter->instance= name=obj00000000000000000033 instance= ns= 2024-04-16T18:18:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14/obj00000000000000000033 2024-04-16T18:18:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: entry->timestamp=2024-04-16T18:17:53.656480+0000 2024-04-16T18:18:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000033]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.83.6 op=0 op_state=1 2024-04-16T18:18:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000033]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000033[0] 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position=00000000001.83.6 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.63.6 iter->object=obj00000000000000000013 iter->instance= name=obj00000000000000000013 instance= ns= 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000013 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:17:54.069137+0000 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.63.6 op=0 op_state=1 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: : injecting data sync error on key=obj00000000000000000013 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.63.6 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000013 2024-04-16T18:18:01.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000032]: success 2024-04-16T18:18:01.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.716+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6:1 marker=00000000001.21.6 timestamp=2024-04-16T18:17:53.876796+0000 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: parsed entry: id=00000000001.58.6 iter->object=obj00000000000000000026 iter->instance= name=obj00000000000000000026 instance= ns= 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13/obj00000000000000000026 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: entry->timestamp=2024-04-16T18:17:53.605547+0000 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000026]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.58.6 op=0 op_state=1 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000026]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000026[0] 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position=00000000001.58.6 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: parsed entry: id=00000000001.42.6 iter->object=obj00000000000000000076 iter->instance= name=obj00000000000000000076 instance= ns= 2024-04-16T18:18:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16/obj00000000000000000076 2024-04-16T18:18:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: entry->timestamp=2024-04-16T18:17:53.672376+0000 2024-04-16T18:18:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000076]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.42.6 op=0 op_state=1 2024-04-16T18:18:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000076]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000076[0] 2024-04-16T18:18:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.720+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position=00000000001.42.6 2024-04-16T18:18:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.724+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000021]: success 2024-04-16T18:18:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.724+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11:1 marker=00000000001.36.6 timestamp=2024-04-16T18:17:53.789358+0000 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000001.86.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000019 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:53.857432+0000 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.86.6 op=0 op_state=1 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: : injecting data sync error on key=obj00000000000000000019 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000001.86.6 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000019 2024-04-16T18:18:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000028]: success 2024-04-16T18:18:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.728+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4:1 marker=00000000001.57.6 timestamp=2024-04-16T18:17:53.925661+0000 2024-04-16T18:18:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.732+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000033]: success 2024-04-16T18:18:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.732+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14:1 marker=00000000001.83.6 timestamp=2024-04-16T18:17:53.656480+0000 2024-04-16T18:18:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.732+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000076]: success 2024-04-16T18:18:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.732+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16:1 marker=00000000001.42.6 timestamp=2024-04-16T18:17:53.672376+0000 2024-04-16T18:18:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: a sync operation returned error 2024-04-16T18:18:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: backing out with sync_status=-5 2024-04-16T18:18:01.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:01.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:01.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:01.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.752+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: a sync operation returned error 2024-04-16T18:18:01.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000001.76.6 iter->object=obj00000000000000000020 iter->instance= name=obj00000000000000000020 instance= ns= 2024-04-16T18:18:01.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000020 2024-04-16T18:18:01.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:53.945282+0000 2024-04-16T18:18:01.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.76.6 op=0 op_state=1 2024-04-16T18:18:01.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000020[0] 2024-04-16T18:18:01.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000001.76.6 2024-04-16T18:18:01.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: backing out with sync_status=-5 2024-04-16T18:18:01.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:01.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:01.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:01.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.760+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: a sync operation returned error 2024-04-16T18:18:01.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: backing out with sync_status=-5 2024-04-16T18:18:01.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:01.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:01.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000020]: success 2024-04-16T18:18:01.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3:1 marker=00000000001.76.6 timestamp=2024-04-16T18:17:53.945282+0000 2024-04-16T18:18:01.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:01.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.772+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.780+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000026]: success 2024-04-16T18:18:01.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.780+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13:1 marker=00000000001.58.6 timestamp=2024-04-16T18:17:53.605547+0000 2024-04-16T18:18:01.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.800+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000001 iter->instance= name=obj00000000000000000001 instance= ns= 2024-04-16T18:18:01.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.800+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000001 2024-04-16T18:18:01.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.800+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:17:53.739624+0000 2024-04-16T18:18:01.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.800+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:01.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.800+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000001[0] 2024-04-16T18:18:01.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.800+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.808+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.63.6 iter->object=obj00000000000000000013 iter->instance= name=obj00000000000000000013 instance= ns= 2024-04-16T18:18:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.808+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000013 2024-04-16T18:18:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.808+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:17:54.069137+0000 2024-04-16T18:18:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.808+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.63.6 op=0 op_state=1 2024-04-16T18:18:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.808+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000013[0] 2024-04-16T18:18:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.808+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.63.6 2024-04-16T18:18:01.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.812+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000001]: success 2024-04-16T18:18:01.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.812+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7:1 marker=00000000002.44.6 timestamp=2024-04-16T18:17:53.739624+0000 2024-04-16T18:18:01.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.816+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000013]: success 2024-04-16T18:18:01.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10:1 marker=00000000002.63.6 timestamp=2024-04-16T18:17:54.069137+0000 2024-04-16T18:18:01.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000001.86.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:01.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000019 2024-04-16T18:18:01.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:53.857432+0000 2024-04-16T18:18:01.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.86.6 op=0 op_state=1 2024-04-16T18:18:01.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000019[0] 2024-04-16T18:18:01.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.820+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000001.86.6 2024-04-16T18:18:01.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.828+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000019]: success 2024-04-16T18:18:01.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.828+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9:1 marker=00000000001.86.6 timestamp=2024-04-16T18:17:53.857432+0000 2024-04-16T18:18:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.872+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.872+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.872+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 0 2024-04-16T18:18:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.872+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 0 2024-04-16T18:18:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.876+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 0 2024-04-16T18:18:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket shard: 0 2024-04-16T18:18:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000002.57.6 iter->object=obj00000000000000000016 iter->instance= name=obj00000000000000000016 instance= ns= 2024-04-16T18:18:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000016 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:17:54.773229+0000 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000016]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.57.6 op=0 op_state=1 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000002.63.6 iter->object=obj00000000000000000021 iter->instance= name=obj00000000000000000021 instance= ns= 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000021 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:54.627402+0000 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000021]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.63.6 op=0 op_state=1 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000016]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000016[0] 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000002.57.6 2024-04-16T18:18:01.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000021]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000021[0] 2024-04-16T18:18:01.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000002.63.6 2024-04-16T18:18:01.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.880+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket shard: 0 2024-04-16T18:18:01.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket shard: 0 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000002.62.6 iter->object=obj00000000000000000003 iter->instance= name=obj00000000000000000003 instance= ns= 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000003 2024-04-16T18:18:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:54.826002+0000 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000003]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.62.6 op=0 op_state=1 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000003]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000003[0] 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000002.62.6 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000002.68.6 iter->object=obj00000000000000000015 iter->instance= name=obj00000000000000000015 instance= ns= 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000015 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:17:54.849800+0000 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000015]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.68.6 op=0 op_state=1 2024-04-16T18:18:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000002.70.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000019 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:17:55.149777+0000 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.70.6 op=0 op_state=1 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000015]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000015[0] 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000002.68.6 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000019]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000019[0] 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000002.70.6 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: parsed entry: id=00000000001.72.6 iter->object=obj00000000000000000025 iter->instance= name=obj00000000000000000025 instance= ns= 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5/obj00000000000000000025 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: entry->timestamp=2024-04-16T18:17:55.225460+0000 2024-04-16T18:18:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000025]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.72.6 op=0 op_state=1 2024-04-16T18:18:01.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000025]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000025[0] 2024-04-16T18:18:01.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000001.72.6 2024-04-16T18:18:01.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket shard: 0 2024-04-16T18:18:01.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket shard: 0 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000016]: success 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2:2 marker=00000000002.57.6 timestamp=2024-04-16T18:17:54.773229+0000 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket shard: 0 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position= 2024-04-16T18:18:01.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000021]: success 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.888+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3:2 marker=00000000002.63.6 timestamp=2024-04-16T18:17:54.627402+0000 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.892+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: parsed entry: id=00000000002.46.6 iter->object=obj00000000000000000011 iter->instance= name=obj00000000000000000011 instance= ns= 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.892+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16/obj00000000000000000011 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.892+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: entry->timestamp=2024-04-16T18:17:54.901542+0000 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.892+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000011]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.46.6 op=0 op_state=1 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.892+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000011]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000011[0] 2024-04-16T18:18:01.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.892+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position=00000000002.46.6 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000001 iter->instance= name=obj00000000000000000001 instance= ns= 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17/obj00000000000000000001 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: entry->timestamp=2024-04-16T18:17:54.697847+0000 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000001]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000001]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000001[0] 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000019]: success 2024-04-16T18:18:01.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8:2 marker=00000000002.70.6 timestamp=2024-04-16T18:17:55.149777+0000 2024-04-16T18:18:01.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: parsed entry: id=00000000002.59.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:01.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14/obj00000000000000000000 2024-04-16T18:18:01.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: entry->timestamp=2024-04-16T18:17:55.069210+0000 2024-04-16T18:18:01.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.59.6 op=0 op_state=1 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: parsed entry: id=00000000002.49.6 iter->object=obj00000000000000000007 iter->instance= name=obj00000000000000000007 instance= ns= 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18/obj00000000000000000007 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: entry->timestamp=2024-04-16T18:17:54.926758+0000 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000007]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.49.6 op=0 op_state=1 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000000]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000000[0] 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position=00000000002.59.6 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000007]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000007[0] 2024-04-16T18:18:01.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position=00000000002.49.6 2024-04-16T18:18:01.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.900+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000001]: success 2024-04-16T18:18:01.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.900+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17:2 marker=00000000002.44.6 timestamp=2024-04-16T18:17:54.697847+0000 2024-04-16T18:18:01.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.900+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000000]: success 2024-04-16T18:18:01.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.900+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14:2 marker=00000000002.59.6 timestamp=2024-04-16T18:17:55.069210+0000 2024-04-16T18:18:01.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.900+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000007]: success 2024-04-16T18:18:01.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.900+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18:2 marker=00000000002.49.6 timestamp=2024-04-16T18:17:54.926758+0000 2024-04-16T18:18:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.908+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000003]: success 2024-04-16T18:18:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.908+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9:2 marker=00000000002.62.6 timestamp=2024-04-16T18:17:54.826002+0000 2024-04-16T18:18:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.908+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000025]: success 2024-04-16T18:18:01.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.908+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5:2 marker=00000000001.72.6 timestamp=2024-04-16T18:17:55.225460+0000 2024-04-16T18:18:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.912+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000011]: success 2024-04-16T18:18:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.912+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16:2 marker=00000000002.46.6 timestamp=2024-04-16T18:17:54.901542+0000 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: parsed entry: id=00000000002.72.6 iter->object=obj00000000000000000005 iter->instance= name=obj00000000000000000005 instance= ns= 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0/obj00000000000000000005 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: entry->timestamp=2024-04-16T18:17:55.125695+0000 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000005]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.72.6 op=0 op_state=1 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000005]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000005[0] 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000002.72.6 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000001.58.6 iter->object=obj00000000000000000004 iter->instance= name=obj00000000000000000004 instance= ns= 2024-04-16T18:18:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000004 2024-04-16T18:18:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:17:54.977970+0000 2024-04-16T18:18:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000004]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.58.6 op=0 op_state=1 2024-04-16T18:18:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000004]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000004[0] 2024-04-16T18:18:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.920+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000001.58.6 2024-04-16T18:18:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.924+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: parsed entry: id=00000000002.26.6 iter->object=obj00000000000000000010 iter->instance= name=obj00000000000000000010 instance= ns= 2024-04-16T18:18:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.924+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1/obj00000000000000000010 2024-04-16T18:18:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.924+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: entry->timestamp=2024-04-16T18:17:54.997374+0000 2024-04-16T18:18:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.924+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000010]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.26.6 op=0 op_state=1 2024-04-16T18:18:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.924+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000010]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000010[0] 2024-04-16T18:18:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.924+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position=00000000002.26.6 2024-04-16T18:18:01.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: parsed entry: id=00000000001.57.6 iter->object=obj00000000000000000014 iter->instance= name=obj00000000000000000014 instance= ns= 2024-04-16T18:18:01.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11/obj00000000000000000014 2024-04-16T18:18:01.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: entry->timestamp=2024-04-16T18:17:54.612347+0000 2024-04-16T18:18:01.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000014]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.57.6 op=0 op_state=1 2024-04-16T18:18:01.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000014]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000014[0] 2024-04-16T18:18:01.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position=00000000001.57.6 2024-04-16T18:18:01.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000010]: success 2024-04-16T18:18:01.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.928+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1:2 marker=00000000002.26.6 timestamp=2024-04-16T18:17:54.997374+0000 2024-04-16T18:18:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: parsed entry: id=00000000002.81.6 iter->object=obj00000000000000000002 iter->instance= name=obj00000000000000000002 instance= ns= 2024-04-16T18:18:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13/obj00000000000000000002 2024-04-16T18:18:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: entry->timestamp=2024-04-16T18:17:54.749811+0000 2024-04-16T18:18:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000002]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.81.6 op=0 op_state=1 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000002]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000002[0] 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position=00000000002.81.6 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000002.60.6 iter->object=obj00000000000000000012 iter->instance= name=obj00000000000000000012 instance= ns= 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000012 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:54.677437+0000 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000012]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.60.6 op=0 op_state=1 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: parsed entry: id=00000000002.88.6 iter->object=obj00000000000000000030 iter->instance= name=obj00000000000000000030 instance= ns= 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6/obj00000000000000000030 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: entry->timestamp=2024-04-16T18:17:55.201706+0000 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000030]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.88.6 op=0 op_state=1 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000012]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000012[0] 2024-04-16T18:18:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000002.60.6 2024-04-16T18:18:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000030]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000030[0] 2024-04-16T18:18:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position=00000000002.88.6 2024-04-16T18:18:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000004]: success 2024-04-16T18:18:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.932+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4:2 marker=00000000001.58.6 timestamp=2024-04-16T18:17:54.977970+0000 2024-04-16T18:18:01.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000001.46.6 iter->object=obj00000000000000000009 iter->instance= name=obj00000000000000000009 instance= ns= 2024-04-16T18:18:01.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000009 2024-04-16T18:18:01.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:17:55.050391+0000 2024-04-16T18:18:01.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000009]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.46.6 op=0 op_state=1 2024-04-16T18:18:01.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000009]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000009[0] 2024-04-16T18:18:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000001.46.6 2024-04-16T18:18:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000005]: success 2024-04-16T18:18:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0:2 marker=00000000002.72.6 timestamp=2024-04-16T18:17:55.125695+0000 2024-04-16T18:18:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000030]: success 2024-04-16T18:18:01.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.936+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6:2 marker=00000000002.88.6 timestamp=2024-04-16T18:17:55.201706+0000 2024-04-16T18:18:01.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.940+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000014]: success 2024-04-16T18:18:01.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.940+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11:2 marker=00000000001.57.6 timestamp=2024-04-16T18:17:54.612347+0000 2024-04-16T18:18:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.944+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000015]: success 2024-04-16T18:18:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7:2 marker=00000000002.68.6 timestamp=2024-04-16T18:17:54.849800+0000 2024-04-16T18:18:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.948+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000002]: success 2024-04-16T18:18:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13:2 marker=00000000002.81.6 timestamp=2024-04-16T18:17:54.749811+0000 2024-04-16T18:18:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.948+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000012]: success 2024-04-16T18:18:01.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12:2 marker=00000000002.60.6 timestamp=2024-04-16T18:17:54.677437+0000 2024-04-16T18:18:01.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.952+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000009]: success 2024-04-16T18:18:01.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:01.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10:2 marker=00000000001.46.6 timestamp=2024-04-16T18:17:55.050391+0000 2024-04-16T18:18:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 0 2024-04-16T18:18:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 0 2024-04-16T18:18:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 0 2024-04-16T18:18:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: parsed entry: id=00000000001.71.6 iter->object=obj00000000000000000058 iter->instance= name=obj00000000000000000058 instance= ns= 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0/obj00000000000000000058 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: entry->timestamp=2024-04-16T18:17:56.349217+0000 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000058]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.71.6 op=0 op_state=1 2024-04-16T18:18:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000058]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000058[0] 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000001.71.6 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: parsed entry: id=00000000002.81.6 iter->object=obj00000000000000000026 iter->instance= name=obj00000000000000000026 instance= ns= 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1/obj00000000000000000026 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: entry->timestamp=2024-04-16T18:17:55.909743+0000 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000026]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.81.6 op=0 op_state=1 2024-04-16T18:18:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000026]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000026[0] 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position=00000000002.81.6 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket shard: 0 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket shard: 0 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket shard: 0 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket shard: 0 2024-04-16T18:18:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket shard: 0 2024-04-16T18:18:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket shard: 0 2024-04-16T18:18:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket shard: 0 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket shard: 0 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket shard: 0 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket shard: 0 2024-04-16T18:18:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.016+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000026]: success 2024-04-16T18:18:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1:3 marker=00000000002.81.6 timestamp=2024-04-16T18:17:55.909743+0000 2024-04-16T18:18:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.020+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000058]: success 2024-04-16T18:18:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0:3 marker=00000000001.71.6 timestamp=2024-04-16T18:17:56.349217+0000 2024-04-16T18:18:02.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.45.6 iter->object=obj00000000000000000006 iter->instance= name=obj00000000000000000006 instance= ns= 2024-04-16T18:18:02.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000006 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:17:56.173541+0000 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000006]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.45.6 op=0 op_state=1 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000002.47.6 iter->object=obj00000000000000000009 iter->instance= name=obj00000000000000000009 instance= ns= 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000009 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:56.325734+0000 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000009]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.47.6 op=0 op_state=1 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: parsed entry: id=00000000001.64.6 iter->object=obj00000000000000000040 iter->instance= name=obj00000000000000000040 instance= ns= 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11/obj00000000000000000040 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: entry->timestamp=2024-04-16T18:17:56.401494+0000 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000040]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.64.6 op=0 op_state=1 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000013 iter->instance= name=obj00000000000000000013 instance= ns= 2024-04-16T18:18:02.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14/obj00000000000000000013 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: entry->timestamp=2024-04-16T18:17:55.817940+0000 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000013]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: parsed entry: id=00000000002.46.6 iter->object=obj00000000000000000012 iter->instance= name=obj00000000000000000012 instance= ns= 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15/obj00000000000000000012 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: entry->timestamp=2024-04-16T18:17:56.050833+0000 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000012]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.46.6 op=0 op_state=1 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: parsed entry: id=00000000001.83.6 iter->object=obj00000000000000000045 iter->instance= name=obj00000000000000000045 instance= ns= 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13/obj00000000000000000045 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: entry->timestamp=2024-04-16T18:17:56.197329+0000 2024-04-16T18:18:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000045]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.83.6 op=0 op_state=1 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: parsed entry: id=00000000002.69.6 iter->object=obj00000000000000000033 iter->instance= name=obj00000000000000000033 instance= ns= 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16/obj00000000000000000033 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: entry->timestamp=2024-04-16T18:17:55.734983+0000 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000033]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.69.6 op=0 op_state=1 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: parsed entry: id=00000000002.40.6 iter->object=obj00000000000000000003 iter->instance= name=obj00000000000000000003 instance= ns= 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20/obj00000000000000000003 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: entry->timestamp=2024-04-16T18:17:56.421157+0000 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000003]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.40.6 op=0 op_state=1 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000006]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000006[0] 2024-04-16T18:18:02.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.45.6 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: parsed entry: id=00000000002.54.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18/obj00000000000000000000 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: entry->timestamp=2024-04-16T18:17:55.888684+0000 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.54.6 op=0 op_state=1 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000009]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000009[0] 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000002.47.6 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000040]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000040[0] 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position=00000000001.64.6 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000013]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000013[0] 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:02.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000012]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000012[0] 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position=00000000002.46.6 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000045]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000045[0] 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position=00000000001.83.6 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000033]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000033[0] 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position=00000000002.69.6 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000003]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000003[0] 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position=00000000002.40.6 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000000]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000000[0] 2024-04-16T18:18:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position=00000000002.54.6 2024-04-16T18:18:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.036+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000001.46.6 iter->object=obj00000000000000000046 iter->instance= name=obj00000000000000000046 instance= ns= 2024-04-16T18:18:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.036+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000046 2024-04-16T18:18:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.036+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:56.102179+0000 2024-04-16T18:18:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.036+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000046]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.46.6 op=0 op_state=1 2024-04-16T18:18:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.036+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000046]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000046[0] 2024-04-16T18:18:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.036+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000001.46.6 2024-04-16T18:18:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.040+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000009]: success 2024-04-16T18:18:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.040+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12:3 marker=00000000002.47.6 timestamp=2024-04-16T18:17:56.325734+0000 2024-04-16T18:18:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.040+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000003]: success 2024-04-16T18:18:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.040+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20:3 marker=00000000002.40.6 timestamp=2024-04-16T18:17:56.421157+0000 2024-04-16T18:18:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000013]: success 2024-04-16T18:18:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14:3 marker=00000000002.44.6 timestamp=2024-04-16T18:17:55.817940+0000 2024-04-16T18:18:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000033]: success 2024-04-16T18:18:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16:3 marker=00000000002.69.6 timestamp=2024-04-16T18:17:55.734983+0000 2024-04-16T18:18:02.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000000]: success 2024-04-16T18:18:02.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18:3 marker=00000000002.54.6 timestamp=2024-04-16T18:17:55.888684+0000 2024-04-16T18:18:02.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000012]: success 2024-04-16T18:18:02.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.044+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15:3 marker=00000000002.46.6 timestamp=2024-04-16T18:17:56.050833+0000 2024-04-16T18:18:02.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: parsed entry: id=00000000001.67.6 iter->object=obj00000000000000000050 iter->instance= name=obj00000000000000000050 instance= ns= 2024-04-16T18:18:02.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5/obj00000000000000000050 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: entry->timestamp=2024-04-16T18:17:55.961752+0000 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000050]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.67.6 op=0 op_state=1 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000050]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000050[0] 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000001.67.6 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000011 iter->instance= name=obj00000000000000000011 instance= ns= 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000011 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:17:55.838469+0000 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000011]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000002.31.6 iter->object=obj00000000000000000014 iter->instance= name=obj00000000000000000014 instance= ns= 2024-04-16T18:18:02.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000014 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:17:55.768642+0000 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000014]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.31.6 op=0 op_state=1 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000011]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000011[0] 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000014]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000014[0] 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000002.31.6 2024-04-16T18:18:02.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000046]: success 2024-04-16T18:18:02.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.052+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3:3 marker=00000000001.46.6 timestamp=2024-04-16T18:17:56.102179+0000 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000001.40.6 iter->object=obj00000000000000000039 iter->instance= name=obj00000000000000000039 instance= ns= 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000039 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:17:56.273081+0000 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000039]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.40.6 op=0 op_state=1 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000039]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000039[0] 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000001.40.6 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000002.47.6 iter->object=obj00000000000000000002 iter->instance= name=obj00000000000000000002 instance= ns= 2024-04-16T18:18:02.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000002 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:17:56.249334+0000 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000002]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.47.6 op=0 op_state=1 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000002.38.6 iter->object=obj00000000000000000007 iter->instance= name=obj00000000000000000007 instance= ns= 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000007 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:56.121639+0000 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000007]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.38.6 op=0 op_state=1 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000002]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000002[0] 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000002.47.6 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000007]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000007[0] 2024-04-16T18:18:02.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000002.38.6 2024-04-16T18:18:02.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000006]: success 2024-04-16T18:18:02.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10:3 marker=00000000002.45.6 timestamp=2024-04-16T18:17:56.173541+0000 2024-04-16T18:18:02.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000045]: success 2024-04-16T18:18:02.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13:3 marker=00000000001.83.6 timestamp=2024-04-16T18:17:56.197329+0000 2024-04-16T18:18:02.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000011]: success 2024-04-16T18:18:02.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2:3 marker=00000000002.44.6 timestamp=2024-04-16T18:17:55.838469+0000 2024-04-16T18:18:02.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000014]: success 2024-04-16T18:18:02.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.056+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4:3 marker=00000000002.31.6 timestamp=2024-04-16T18:17:55.768642+0000 2024-04-16T18:18:02.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.060+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000002]: success 2024-04-16T18:18:02.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.060+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7:3 marker=00000000002.47.6 timestamp=2024-04-16T18:17:56.249334+0000 2024-04-16T18:18:02.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.060+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000007]: success 2024-04-16T18:18:02.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.060+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9:3 marker=00000000002.38.6 timestamp=2024-04-16T18:17:56.121639+0000 2024-04-16T18:18:02.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.064+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000050]: success 2024-04-16T18:18:02.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.064+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5:3 marker=00000000001.67.6 timestamp=2024-04-16T18:17:55.961752+0000 2024-04-16T18:18:02.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.068+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000039]: success 2024-04-16T18:18:02.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.068+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8:3 marker=00000000001.40.6 timestamp=2024-04-16T18:17:56.273081+0000 2024-04-16T18:18:02.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.076+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: parsed entry: id=00000000001.62.6 iter->object=obj00000000000000000055 iter->instance= name=obj00000000000000000055 instance= ns= 2024-04-16T18:18:02.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.076+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21/obj00000000000000000055 2024-04-16T18:18:02.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.076+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: entry->timestamp=2024-04-16T18:17:55.752417+0000 2024-04-16T18:18:02.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.076+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000055]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.62.6 op=0 op_state=1 2024-04-16T18:18:02.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.076+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000055]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000055[0] 2024-04-16T18:18:02.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.076+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position=00000000001.62.6 2024-04-16T18:18:02.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: parsed entry: id=00000000002.81.6 iter->object=obj00000000000000000021 iter->instance= name=obj00000000000000000021 instance= ns= 2024-04-16T18:18:02.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19/obj00000000000000000021 2024-04-16T18:18:02.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: entry->timestamp=2024-04-16T18:17:55.980496+0000 2024-04-16T18:18:02.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000021]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.81.6 op=0 op_state=1 2024-04-16T18:18:02.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000021]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000021[0] 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position=00000000002.81.6 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: parsed entry: id=00000000002.83.6 iter->object=obj00000000000000000008 iter->instance= name=obj00000000000000000008 instance= ns= 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22/obj00000000000000000008 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: entry->timestamp=2024-04-16T18:17:56.030520+0000 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000008]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.83.6 op=0 op_state=1 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000008]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000008[0] 2024-04-16T18:18:02.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.080+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position=00000000002.83.6 2024-04-16T18:18:02.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.084+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000021]: success 2024-04-16T18:18:02.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.084+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19:3 marker=00000000002.81.6 timestamp=2024-04-16T18:17:55.980496+0000 2024-04-16T18:18:02.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.088+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000055]: success 2024-04-16T18:18:02.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.088+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21:3 marker=00000000001.62.6 timestamp=2024-04-16T18:17:55.752417+0000 2024-04-16T18:18:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.092+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000008]: success 2024-04-16T18:18:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.092+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22:3 marker=00000000002.83.6 timestamp=2024-04-16T18:17:56.030520+0000 2024-04-16T18:18:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.092+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000040]: success 2024-04-16T18:18:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.092+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11:3 marker=00000000001.64.6 timestamp=2024-04-16T18:17:56.401494+0000 2024-04-16T18:18:02.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.228+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.228+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.228+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 0 2024-04-16T18:18:02.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 0 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:02.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.232+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket shard: 0 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: parsed entry: id=00000000002.67.6 iter->object=obj00000000000000000006 iter->instance= name=obj00000000000000000006 instance= ns= 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0/obj00000000000000000006 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: entry->timestamp=2024-04-16T18:17:57.570781+0000 2024-04-16T18:18:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000006]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.67.6 op=0 op_state=1 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000006]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000006[0] 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000002.67.6 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000001.65.6 iter->object=obj00000000000000000027 iter->instance= name=obj00000000000000000027 instance= ns= 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000027 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:17:57.896154+0000 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.65.6 op=0 op_state=1 2024-04-16T18:18:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: : injecting data sync error on key=obj00000000000000000027 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000001.65.6 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000027 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000002.60.6 iter->object=obj00000000000000000011 iter->instance= name=obj00000000000000000011 instance= ns= 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000011 2024-04-16T18:18:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:17:57.041803+0000 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.236+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000011]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.60.6 op=0 op_state=1 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: parsed entry: id=00000000002.42.6 iter->object=obj00000000000000000002 iter->instance= name=obj00000000000000000002 instance= ns= 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5/obj00000000000000000002 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: entry->timestamp=2024-04-16T18:17:57.291462+0000 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000002]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.42.6 op=0 op_state=1 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000011]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000011[0] 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000002.60.6 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket shard: 0 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000002]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000002[0] 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000002.42.6 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: parsed entry: id=00000000002.65.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6/obj00000000000000000019 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: entry->timestamp=2024-04-16T18:17:57.819184+0000 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.65.6 op=0 op_state=1 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: : injecting data sync error on key=obj00000000000000000019 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position=00000000002.65.6 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6/obj00000000000000000019 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket shard: 0 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket shard: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket shard: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: sync status for source bucket shard: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.240+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket shard: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000006]: success 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0:4 marker=00000000002.67.6 timestamp=2024-04-16T18:17:57.570781+0000 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket shard: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket shard: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: sync status for source bucket shard: 0 2024-04-16T18:18:02.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket shard: 0 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket shard: 0 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket shard: 0 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: sync status for source bucket shard: 0 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.244+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: sync status for source bucket shard: 0 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000011]: success 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4:4 marker=00000000002.60.6 timestamp=2024-04-16T18:17:57.041803+0000 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: sync status for source bucket shard: 0 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: parsed entry: id=00000000001.48.6 iter->object=obj00000000000000000037 iter->instance= name=obj00000000000000000037 instance= ns= 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15/obj00000000000000000037 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: entry->timestamp=2024-04-16T18:17:57.195073+0000 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000037]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.48.6 op=0 op_state=1 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000002.61.6 iter->object=obj00000000000000000022 iter->instance= name=obj00000000000000000022 instance= ns= 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000022 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:57.742156+0000 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.61.6 op=0 op_state=1 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000037]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000037[0] 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position=00000000001.48.6 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000002]: success 2024-04-16T18:18:02.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5:4 marker=00000000002.42.6 timestamp=2024-04-16T18:17:57.291462+0000 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: : injecting data sync error on key=obj00000000000000000022 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000002.61.6 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: a sync operation returned error 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000022 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: backing out with sync_status=-5 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: parsed entry: id=00000000002.88.6 iter->object=obj00000000000000000003 iter->instance= name=obj00000000000000000003 instance= ns= 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18/obj00000000000000000003 2024-04-16T18:18:02.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: entry->timestamp=2024-04-16T18:17:57.143426+0000 2024-04-16T18:18:02.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000003]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.88.6 op=0 op_state=1 2024-04-16T18:18:02.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000003]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000003[0] 2024-04-16T18:18:02.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.248+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position=00000000002.88.6 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: parsed entry: id=00000000002.65.6 iter->object=obj00000000000000000025 iter->instance= name=obj00000000000000000025 instance= ns= 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23/obj00000000000000000025 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: entry->timestamp=2024-04-16T18:17:57.520807+0000 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000025]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.65.6 op=0 op_state=1 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000025]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000025[0] 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position=00000000002.65.6 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: parsed entry: id=00000000002.48.6 iter->object=obj00000000000000000016 iter->instance= name=obj00000000000000000016 instance= ns= 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24/obj00000000000000000016 2024-04-16T18:18:02.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: entry->timestamp=2024-04-16T18:17:57.595116+0000 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000016]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.48.6 op=0 op_state=1 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000016]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000016[0] 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: listing bilog for incremental sync; position=00000000002.48.6 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: parsed entry: id=00000000002.46.6 iter->object=obj00000000000000000001 iter->instance= name=obj00000000000000000001 instance= ns= 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21/obj00000000000000000001 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: entry->timestamp=2024-04-16T18:17:57.343179+0000 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000001]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.46.6 op=0 op_state=1 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: parsed entry: id=00000000002.53.6 iter->object=obj00000000000000000010 iter->instance= name=obj00000000000000000010 instance= ns= 2024-04-16T18:18:02.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20/obj00000000000000000010 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: entry->timestamp=2024-04-16T18:17:57.419104+0000 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000010]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.53.6 op=0 op_state=1 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: parsed entry: id=00000000002.56.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25/obj00000000000000000000 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: entry->timestamp=2024-04-16T18:17:57.118598+0000 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.56.6 op=0 op_state=1 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000001]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000001[0] 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position=00000000002.46.6 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000002.91.6 iter->object=obj00000000000000000004 iter->instance= name=obj00000000000000000004 instance= ns= 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000004 2024-04-16T18:18:02.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:57.646978+0000 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.91.6 op=0 op_state=1 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: parsed entry: id=00000000002.52.6 iter->object=obj00000000000000000021 iter->instance= name=obj00000000000000000021 instance= ns= 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28/obj00000000000000000021 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: entry->timestamp=2024-04-16T18:17:57.443399+0000 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000021]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.52.6 op=0 op_state=1 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000010]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000010[0] 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position=00000000002.53.6 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000001.67.6 iter->object=obj00000000000000000049 iter->instance= name=obj00000000000000000049 instance= ns= 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000049 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:17:57.946358+0000 2024-04-16T18:18:02.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000049]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.67.6 op=0 op_state=1 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000000]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000000[0] 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: listing bilog for incremental sync; position=00000000002.56.6 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000020 iter->instance= name=obj00000000000000000020 instance= ns= 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14/obj00000000000000000020 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: entry->timestamp=2024-04-16T18:17:57.495596+0000 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000020]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: : injecting data sync error on key=obj00000000000000000004 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000002.91.6 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000021]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000021[0] 2024-04-16T18:18:02.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: listing bilog for incremental sync; position=00000000002.52.6 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000049]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000049[0] 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000001.67.6 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000020]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000020[0] 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000004 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.256+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: a sync operation returned error 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: backing out with sync_status=-5 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000003]: success 2024-04-16T18:18:02.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18:4 marker=00000000002.88.6 timestamp=2024-04-16T18:17:57.143426+0000 2024-04-16T18:18:02.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:02.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.260+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000025]: success 2024-04-16T18:18:02.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23:4 marker=00000000002.65.6 timestamp=2024-04-16T18:17:57.520807+0000 2024-04-16T18:18:02.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000021]: success 2024-04-16T18:18:02.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28:4 marker=00000000002.52.6 timestamp=2024-04-16T18:17:57.443399+0000 2024-04-16T18:18:02.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000010]: success 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20:4 marker=00000000002.53.6 timestamp=2024-04-16T18:17:57.419104+0000 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000020]: success 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14:4 marker=00000000002.44.6 timestamp=2024-04-16T18:17:57.495596+0000 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.264+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.268+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000016]: success 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.268+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24:4 marker=00000000002.48.6 timestamp=2024-04-16T18:17:57.595116+0000 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.268+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000001]: success 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.268+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21:4 marker=00000000002.46.6 timestamp=2024-04-16T18:17:57.343179+0000 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.268+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000000]: success 2024-04-16T18:18:02.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.268+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25:4 marker=00000000002.56.6 timestamp=2024-04-16T18:17:57.118598+0000 2024-04-16T18:18:02.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.276+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000049]: success 2024-04-16T18:18:02.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.276+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7:4 marker=00000000001.67.6 timestamp=2024-04-16T18:17:57.946358+0000 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000001.54.6 iter->object=obj00000000000000000017 iter->instance= name=obj00000000000000000017 instance= ns= 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000017 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:17:56.992191+0000 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.54.6 op=0 op_state=1 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: : injecting data sync error on key=obj00000000000000000017 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000001.54.6 2024-04-16T18:18:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.280+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000017 2024-04-16T18:18:02.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000002.62.6 iter->object=obj00000000000000000008 iter->instance= name=obj00000000000000000008 instance= ns= 2024-04-16T18:18:02.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000008 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:57.794921+0000 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000008]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.62.6 op=0 op_state=1 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000008]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000008[0] 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000002.62.6 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.42.6 iter->object=obj00000000000000000015 iter->instance= name=obj00000000000000000015 instance= ns= 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000015 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:17:56.973168+0000 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000015]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.42.6 op=0 op_state=1 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: parsed entry: id=00000000001.72.6 iter->object=obj00000000000000000031 iter->instance= name=obj00000000000000000031 instance= ns= 2024-04-16T18:18:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11/obj00000000000000000031 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: entry->timestamp=2024-04-16T18:17:57.064923+0000 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000031]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.72.6 op=0 op_state=1 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000015]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000015[0] 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.42.6 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000031]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000031[0] 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position=00000000001.72.6 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: a sync operation returned error 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: backing out with sync_status=-5 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.284+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.288+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:02.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.288+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.288+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000015]: success 2024-04-16T18:18:02.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.288+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10:4 marker=00000000002.42.6 timestamp=2024-04-16T18:17:56.973168+0000 2024-04-16T18:18:02.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.288+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000008]: success 2024-04-16T18:18:02.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.288+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9:4 marker=00000000002.62.6 timestamp=2024-04-16T18:17:57.794921+0000 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: parsed entry: id=00000000002.63.6 iter->object=obj00000000000000000009 iter->instance= name=obj00000000000000000009 instance= ns= 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22/obj00000000000000000009 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: entry->timestamp=2024-04-16T18:17:57.870663+0000 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000009]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.63.6 op=0 op_state=1 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000009]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000009[0] 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position=00000000002.63.6 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: parsed entry: id=00000000002.90.6 iter->object=obj00000000000000000013 iter->instance= name=obj00000000000000000013 instance= ns= 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13/obj00000000000000000013 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: entry->timestamp=2024-04-16T18:17:57.214824+0000 2024-04-16T18:18:02.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000013]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.90.6 op=0 op_state=1 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: parsed entry: id=00000000001.62.6 iter->object=obj00000000000000000042 iter->instance= name=obj00000000000000000042 instance= ns= 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19/obj00000000000000000042 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: entry->timestamp=2024-04-16T18:17:57.722493+0000 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000042]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.62.6 op=0 op_state=1 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000013]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000013[0] 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position=00000000002.90.6 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000042]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000042[0] 2024-04-16T18:18:02.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.292+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position=00000000001.62.6 2024-04-16T18:18:02.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: parsed entry: id=00000000002.130.6 iter->object=obj00000000000000000005 iter->instance= name=obj00000000000000000005 instance= ns= 2024-04-16T18:18:02.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16/obj00000000000000000005 2024-04-16T18:18:02.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: entry->timestamp=2024-04-16T18:17:57.367732+0000 2024-04-16T18:18:02.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000005]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.130.6 op=0 op_state=1 2024-04-16T18:18:02.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000005]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000005[0] 2024-04-16T18:18:02.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position=00000000002.130.6 2024-04-16T18:18:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: parsed entry: id=00000000002.44.6 iter->object=obj00000000000000000030 iter->instance= name=obj00000000000000000030 instance= ns= 2024-04-16T18:18:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27/obj00000000000000000030 2024-04-16T18:18:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: entry->timestamp=2024-04-16T18:17:57.266935+0000 2024-04-16T18:18:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:entry[obj00000000000000000030]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.44.6 op=0 op_state=1 2024-04-16T18:18:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:entry[obj00000000000000000030]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000030[0] 2024-04-16T18:18:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: listing bilog for incremental sync; position=00000000002.44.6 2024-04-16T18:18:02.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000031]: success 2024-04-16T18:18:02.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11:4 marker=00000000001.72.6 timestamp=2024-04-16T18:17:57.064923+0000 2024-04-16T18:18:02.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000009]: success 2024-04-16T18:18:02.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22:4 marker=00000000002.63.6 timestamp=2024-04-16T18:17:57.870663+0000 2024-04-16T18:18:02.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000013]: success 2024-04-16T18:18:02.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.296+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13:4 marker=00000000002.90.6 timestamp=2024-04-16T18:17:57.214824+0000 2024-04-16T18:18:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: parsed entry: id=00000000002.59.6 iter->object=obj00000000000000000014 iter->instance= name=obj00000000000000000014 instance= ns= 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26/obj00000000000000000014 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: entry->timestamp=2024-04-16T18:17:57.671549+0000 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:entry[obj00000000000000000014]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.59.6 op=0 op_state=1 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:entry[obj00000000000000000014]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000014[0] 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: listing bilog for incremental sync; position=00000000002.59.6 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:entry[obj00000000000000000030]: success 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27:4 marker=00000000002.44.6 timestamp=2024-04-16T18:17:57.266935+0000 2024-04-16T18:18:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000005]: success 2024-04-16T18:18:02.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.300+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16:4 marker=00000000002.130.6 timestamp=2024-04-16T18:17:57.367732+0000 2024-04-16T18:18:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.304+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:entry[obj00000000000000000014]: success 2024-04-16T18:18:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.304+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26:4 marker=00000000002.59.6 timestamp=2024-04-16T18:17:57.671549+0000 2024-04-16T18:18:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.304+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000042]: success 2024-04-16T18:18:02.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.304+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19:4 marker=00000000001.62.6 timestamp=2024-04-16T18:17:57.722493+0000 2024-04-16T18:18:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.308+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000037]: success 2024-04-16T18:18:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.308+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15:4 marker=00000000001.48.6 timestamp=2024-04-16T18:17:57.195073+0000 2024-04-16T18:18:02.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: parsed entry: id=00000000002.65.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:02.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6/obj00000000000000000019 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: entry->timestamp=2024-04-16T18:17:57.819184+0000 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.65.6 op=0 op_state=1 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: a sync operation returned error 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000019[0] 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position=00000000002.65.6 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: backing out with sync_status=-5 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.312+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000002.61.6 iter->object=obj00000000000000000022 iter->instance= name=obj00000000000000000022 instance= ns= 2024-04-16T18:18:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000022 2024-04-16T18:18:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:57.742156+0000 2024-04-16T18:18:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.61.6 op=0 op_state=1 2024-04-16T18:18:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000022[0] 2024-04-16T18:18:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000002.61.6 2024-04-16T18:18:02.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000019]: success 2024-04-16T18:18:02.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.316+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6:4 marker=00000000002.65.6 timestamp=2024-04-16T18:17:57.819184+0000 2024-04-16T18:18:02.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.320+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: a sync operation returned error 2024-04-16T18:18:02.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: backing out with sync_status=-5 2024-04-16T18:18:02.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:02.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000022]: success 2024-04-16T18:18:02.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.324+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12:4 marker=00000000002.61.6 timestamp=2024-04-16T18:17:57.742156+0000 2024-04-16T18:18:02.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.328+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.328+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:02.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.328+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.332+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000001.65.6 iter->object=obj00000000000000000027 iter->instance= name=obj00000000000000000027 instance= ns= 2024-04-16T18:18:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.332+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000027 2024-04-16T18:18:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.332+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:17:57.896154+0000 2024-04-16T18:18:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.332+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.65.6 op=0 op_state=1 2024-04-16T18:18:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.332+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000027[0] 2024-04-16T18:18:02.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.332+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000001.65.6 2024-04-16T18:18:02.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.368+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000002.91.6 iter->object=obj00000000000000000004 iter->instance= name=obj00000000000000000004 instance= ns= 2024-04-16T18:18:02.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.368+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000004 2024-04-16T18:18:02.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.368+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:57.646978+0000 2024-04-16T18:18:02.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.368+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.91.6 op=0 op_state=1 2024-04-16T18:18:02.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.368+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000004[0] 2024-04-16T18:18:02.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.368+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000002.91.6 2024-04-16T18:18:02.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.372+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000004]: success 2024-04-16T18:18:02.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.372+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3:4 marker=00000000002.91.6 timestamp=2024-04-16T18:17:57.646978+0000 2024-04-16T18:18:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.376+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000001.54.6 iter->object=obj00000000000000000017 iter->instance= name=obj00000000000000000017 instance= ns= 2024-04-16T18:18:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.376+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000017 2024-04-16T18:18:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.376+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:17:56.992191+0000 2024-04-16T18:18:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.376+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.54.6 op=0 op_state=1 2024-04-16T18:18:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.376+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000017[0] 2024-04-16T18:18:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.376+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000001.54.6 2024-04-16T18:18:02.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.384+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000027]: success 2024-04-16T18:18:02.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.384+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2:4 marker=00000000001.65.6 timestamp=2024-04-16T18:17:57.896154+0000 2024-04-16T18:18:02.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.384+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000017]: success 2024-04-16T18:18:02.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.384+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8:4 marker=00000000001.54.6 timestamp=2024-04-16T18:17:56.992191+0000 2024-04-16T18:18:02.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 0 2024-04-16T18:18:02.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.540+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 0 2024-04-16T18:18:02.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:02.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: parsed entry: id=00000000002.50.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0/obj00000000000000000019 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: entry->timestamp=2024-04-16T18:17:58.675067+0000 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.50.6 op=0 op_state=1 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000019]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000019[0] 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000002.50.6 2024-04-16T18:18:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.544+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket shard: 0 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: parsed entry: id=00000000002.90.6 iter->object=obj00000000000000000006 iter->instance= name=obj00000000000000000006 instance= ns= 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1/obj00000000000000000006 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: entry->timestamp=2024-04-16T18:17:59.282815+0000 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000006]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.90.6 op=0 op_state=1 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000006]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000006[0] 2024-04-16T18:18:02.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.548+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position=00000000002.90.6 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket shard: 0 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000001.45.6 iter->object=obj00000000000000000467 iter->instance= name=obj00000000000000000467 instance= ns= 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000467 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:17:58.773708+0000 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000467]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.45.6 op=0 op_state=1 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket shard: 0 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000467]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000467[0] 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000001.45.6 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000001.61.6 iter->object=obj00000000000000000056 iter->instance= name=obj00000000000000000056 instance= ns= 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000056 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:17:59.463830+0000 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.61.6 op=0 op_state=1 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000002.78.6 iter->object=obj00000000000000000005 iter->instance= name=obj00000000000000000005 instance= ns= 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000005 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:17:59.359305+0000 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000005]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.78.6 op=0 op_state=1 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: : injecting data sync error on key=obj00000000000000000056 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000001.61.6 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000001.72.6 iter->object=obj00000000000000000317 iter->instance= name=obj00000000000000000317 instance= ns= 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000317 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:17:58.751497+0000 2024-04-16T18:18:02.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000317]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.72.6 op=0 op_state=1 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000005]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000005[0] 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000002.78.6 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000056 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000317]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000317[0] 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000001.72.6 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: parsed entry: id=00000000002.50.6 iter->object=obj00000000000000000023 iter->instance= name=obj00000000000000000023 instance= ns= 2024-04-16T18:18:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6/obj00000000000000000023 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: entry->timestamp=2024-04-16T18:17:58.848804+0000 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000023]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.50.6 op=0 op_state=1 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000023]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000023[0] 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position=00000000002.50.6 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket shard: 0 2024-04-16T18:18:02.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.552+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket shard: 0 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000019]: success 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0:5 marker=00000000002.50.6 timestamp=2024-04-16T18:17:58.675067+0000 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 0 2024-04-16T18:18:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket shard: 0 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket shard: 0 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket shard: 0 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket shard: 0 2024-04-16T18:18:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket shard: 0 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: parsed entry: id=00000000002.74.6 iter->object=obj00000000000000000009 iter->instance= name=obj00000000000000000009 instance= ns= 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11/obj00000000000000000009 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: entry->timestamp=2024-04-16T18:17:58.699548+0000 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000009]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.74.6 op=0 op_state=1 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket shard: 0 2024-04-16T18:18:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000009]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000009[0] 2024-04-16T18:18:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.556+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position=00000000002.74.6 2024-04-16T18:18:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: sync status for source bucket shard: 0 2024-04-16T18:18:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: sync status for source bucket shard: 0 2024-04-16T18:18:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000006]: success 2024-04-16T18:18:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1:5 marker=00000000002.90.6 timestamp=2024-04-16T18:17:59.282815+0000 2024-04-16T18:18:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: sync status for source bucket shard: 0 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: a sync operation returned error 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: sync status for source bucket shard: 0 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: sync status for source bucket shard: 0 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: sync status for source bucket shard: 0 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.560+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: backing out with sync_status=-5 2024-04-16T18:18:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: parsed entry: id=00000000002.59.6 iter->object=obj00000000000000000007 iter->instance= name=obj00000000000000000007 instance= ns= 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15/obj00000000000000000007 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: entry->timestamp=2024-04-16T18:17:58.474169+0000 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.59.6 op=0 op_state=1 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000005]: success 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3:5 marker=00000000002.78.6 timestamp=2024-04-16T18:17:59.359305+0000 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: sync status for source bucket shard: 0 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: : injecting data sync error on key=obj00000000000000000007 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position=00000000002.59.6 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15/obj00000000000000000007 2024-04-16T18:18:02.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: parsed entry: id=00000000001.67.6 iter->object=obj00000000000000000423 iter->instance= name=obj00000000000000000423 instance= ns= 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20/obj00000000000000000423 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: entry->timestamp=2024-04-16T18:17:58.978960+0000 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.67.6 op=0 op_state=1 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: : injecting data sync error on key=obj00000000000000000423 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position=00000000001.67.6 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.564+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20/obj00000000000000000423 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000467]: success 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2:5 marker=00000000001.45.6 timestamp=2024-04-16T18:17:58.773708+0000 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: parsed entry: id=00000000001.73.6 iter->object=obj00000000000000000162 iter->instance= name=obj00000000000000000162 instance= ns= 2024-04-16T18:18:02.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17/obj00000000000000000162 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: entry->timestamp=2024-04-16T18:17:58.598685+0000 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000162]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.73.6 op=0 op_state=1 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000162]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000162[0] 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position=00000000001.73.6 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: parsed entry: id=00000000002.51.6 iter->object=obj00000000000000000012 iter->instance= name=obj00000000000000000012 instance= ns= 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22/obj00000000000000000012 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: entry->timestamp=2024-04-16T18:17:59.307488+0000 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.51.6 op=0 op_state=1 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: parsed entry: id=00000000002.54.6 iter->object=obj00000000000000000001 iter->instance= name=obj00000000000000000001 instance= ns= 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23/obj00000000000000000001 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: entry->timestamp=2024-04-16T18:17:59.229522+0000 2024-04-16T18:18:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000001]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.54.6 op=0 op_state=1 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: : injecting data sync error on key=obj00000000000000000012 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position=00000000002.51.6 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000001]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000001[0] 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position=00000000002.54.6 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000009]: success 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11:5 marker=00000000002.74.6 timestamp=2024-04-16T18:17:58.699548+0000 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.568+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22/obj00000000000000000012 2024-04-16T18:18:02.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: parsed entry: id=00000000002.75.6 iter->object=obj00000000000000000002 iter->instance= name=obj00000000000000000002 instance= ns= 2024-04-16T18:18:02.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21/obj00000000000000000002 2024-04-16T18:18:02.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: entry->timestamp=2024-04-16T18:17:59.050811+0000 2024-04-16T18:18:02.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000002]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.75.6 op=0 op_state=1 2024-04-16T18:18:02.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: parsed entry: id=00000000001.39.6 iter->object=obj00000000000000000133 iter->instance= name=obj00000000000000000133 instance= ns= 2024-04-16T18:18:02.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30/obj00000000000000000133 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: entry->timestamp=2024-04-16T18:17:58.827838+0000 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.39.6 op=0 op_state=1 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000002]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000002[0] 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position=00000000002.75.6 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: : injecting data sync error on key=obj00000000000000000133 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: listing bilog for incremental sync; position=00000000001.39.6 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30/obj00000000000000000133 2024-04-16T18:18:02.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: parsed entry: id=00000000002.70.6 iter->object=obj00000000000000000011 iter->instance= name=obj00000000000000000011 instance= ns= 2024-04-16T18:18:02.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24/obj00000000000000000011 2024-04-16T18:18:02.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: entry->timestamp=2024-04-16T18:17:58.524513+0000 2024-04-16T18:18:02.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000011]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.70.6 op=0 op_state=1 2024-04-16T18:18:02.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000011]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000011[0] 2024-04-16T18:18:02.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.572+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: listing bilog for incremental sync; position=00000000002.70.6 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: parsed entry: id=00000000001.60.6 iter->object=obj00000000000000000283 iter->instance= name=obj00000000000000000283 instance= ns= 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25/obj00000000000000000283 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: entry->timestamp=2024-04-16T18:17:59.515055+0000 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000283]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.60.6 op=0 op_state=1 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: parsed entry: id=00000000001.47.6 iter->object=obj00000000000000000029 iter->instance= name=obj00000000000000000029 instance= ns= 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29/obj00000000000000000029 2024-04-16T18:18:02.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: entry->timestamp=2024-04-16T18:17:59.075578+0000 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.47.6 op=0 op_state=1 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: parsed entry: id=00000000002.76.6 iter->object=obj00000000000000000050 iter->instance= name=obj00000000000000000050 instance= ns= 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28/obj00000000000000000050 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: entry->timestamp=2024-04-16T18:17:59.127593+0000 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000050]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.76.6 op=0 op_state=1 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000283]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000283[0] 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: listing bilog for incremental sync; position=00000000001.60.6 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: : injecting data sync error on key=obj00000000000000000029 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: listing bilog for incremental sync; position=00000000001.47.6 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000050]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000050[0] 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: listing bilog for incremental sync; position=00000000002.76.6 2024-04-16T18:18:02.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29/obj00000000000000000029 2024-04-16T18:18:02.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:entry[obj00000000000000000023]: success 2024-04-16T18:18:02.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.576+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6:5 marker=00000000002.50.6 timestamp=2024-04-16T18:17:58.848804+0000 2024-04-16T18:18:02.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.580+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000002]: success 2024-04-16T18:18:02.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.580+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21:5 marker=00000000002.75.6 timestamp=2024-04-16T18:17:59.050811+0000 2024-04-16T18:18:02.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.580+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000001]: success 2024-04-16T18:18:02.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.580+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23:5 marker=00000000002.54.6 timestamp=2024-04-16T18:17:59.229522+0000 2024-04-16T18:18:02.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.580+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000011]: success 2024-04-16T18:18:02.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.580+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24:5 marker=00000000002.70.6 timestamp=2024-04-16T18:17:58.524513+0000 2024-04-16T18:18:02.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.584+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000050]: success 2024-04-16T18:18:02.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.584+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28:5 marker=00000000002.76.6 timestamp=2024-04-16T18:17:59.127593+0000 2024-04-16T18:18:02.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000283]: success 2024-04-16T18:18:02.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25:5 marker=00000000001.60.6 timestamp=2024-04-16T18:17:59.515055+0000 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000002.54.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000000 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:58.903499+0000 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.54.6 op=0 op_state=1 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: : injecting data sync error on key=obj00000000000000000000 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000002.54.6 2024-04-16T18:18:02.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.592+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000000 2024-04-16T18:18:02.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.58.6 iter->object=obj00000000000000000021 iter->instance= name=obj00000000000000000021 instance= ns= 2024-04-16T18:18:02.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000021 2024-04-16T18:18:02.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:17:59.385782+0000 2024-04-16T18:18:02.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000021]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.58.6 op=0 op_state=1 2024-04-16T18:18:02.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000001.37.6 iter->object=obj00000000000000000271 iter->instance= name=obj00000000000000000271 instance= ns= 2024-04-16T18:18:02.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:58.547717+0000 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.37.6 op=0 op_state=1 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000021]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000021[0] 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.58.6 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: : injecting data sync error on key=obj00000000000000000271 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000001.37.6 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.596+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.600+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000021]: success 2024-04-16T18:18:02.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.600+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10:5 marker=00000000002.58.6 timestamp=2024-04-16T18:17:59.385782+0000 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: parsed entry: id=00000000002.37.6 iter->object=obj00000000000000000008 iter->instance= name=obj00000000000000000008 instance= ns= 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13/obj00000000000000000008 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: entry->timestamp=2024-04-16T18:17:59.438467+0000 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000008]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.37.6 op=0 op_state=1 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000008]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000008[0] 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position=00000000002.37.6 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: parsed entry: id=00000000002.95.6 iter->object=obj00000000000000000018 iter->instance= name=obj00000000000000000018 instance= ns= 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14/obj00000000000000000018 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: entry->timestamp=2024-04-16T18:17:58.617989+0000 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000018]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.95.6 op=0 op_state=1 2024-04-16T18:18:02.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: parsed entry: id=00000000002.97.6 iter->object=obj00000000000000000016 iter->instance= name=obj00000000000000000016 instance= ns= 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18/obj00000000000000000016 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: entry->timestamp=2024-04-16T18:17:59.000005+0000 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000016]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.97.6 op=0 op_state=1 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: parsed entry: id=00000000002.53.6 iter->object=obj00000000000000000003 iter->instance= name=obj00000000000000000003 instance= ns= 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19/obj00000000000000000003 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: entry->timestamp=2024-04-16T18:17:59.152227+0000 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000003]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.53.6 op=0 op_state=1 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000018]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000018[0] 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position=00000000002.95.6 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000016]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000016[0] 2024-04-16T18:18:02.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position=00000000002.97.6 2024-04-16T18:18:02.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000003]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000003[0] 2024-04-16T18:18:02.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position=00000000002.53.6 2024-04-16T18:18:02.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000317]: success 2024-04-16T18:18:02.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.608+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7:5 marker=00000000001.72.6 timestamp=2024-04-16T18:17:58.751497+0000 2024-04-16T18:18:02.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.612+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000003]: success 2024-04-16T18:18:02.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.612+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19:5 marker=00000000002.53.6 timestamp=2024-04-16T18:17:59.152227+0000 2024-04-16T18:18:02.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.612+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000008]: success 2024-04-16T18:18:02.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.612+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13:5 marker=00000000002.37.6 timestamp=2024-04-16T18:17:59.438467+0000 2024-04-16T18:18:02.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.612+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:entry[obj00000000000000000016]: success 2024-04-16T18:18:02.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.612+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18:5 marker=00000000002.97.6 timestamp=2024-04-16T18:17:59.000005+0000 2024-04-16T18:18:02.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000002.52.6 iter->object=obj00000000000000000022 iter->instance= name=obj00000000000000000022 instance= ns= 2024-04-16T18:18:02.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000022 2024-04-16T18:18:02.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:17:59.202824+0000 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000022]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.52.6 op=0 op_state=1 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000022]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000022[0] 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000002.52.6 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: parsed entry: id=00000000002.90.6 iter->object=obj00000000000000000004 iter->instance= name=obj00000000000000000004 instance= ns= 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5/obj00000000000000000004 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: entry->timestamp=2024-04-16T18:17:58.927755+0000 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000004]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.90.6 op=0 op_state=1 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000004]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000004[0] 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000002.90.6 2024-04-16T18:18:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: a sync operation returned error 2024-04-16T18:18:02.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: a sync operation returned error 2024-04-16T18:18:02.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: backing out with sync_status=-5 2024-04-16T18:18:02.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: backing out with sync_status=-5 2024-04-16T18:18:02.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.616+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: a sync operation returned error 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: backing out with sync_status=-5 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:02.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket shard: 0 2024-04-16T18:18:02.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000004]: success 2024-04-16T18:18:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5:5 marker=00000000002.90.6 timestamp=2024-04-16T18:17:58.927755+0000 2024-04-16T18:18:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000022]: success 2024-04-16T18:18:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8:5 marker=00000000002.52.6 timestamp=2024-04-16T18:17:59.202824+0000 2024-04-16T18:18:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000018]: success 2024-04-16T18:18:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.620+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14:5 marker=00000000002.95.6 timestamp=2024-04-16T18:17:58.617989+0000 2024-04-16T18:18:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000001.61.6 iter->object=obj00000000000000000056 iter->instance= name=obj00000000000000000056 instance= ns= 2024-04-16T18:18:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000056 2024-04-16T18:18:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:17:59.463830+0000 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.61.6 op=0 op_state=1 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: a sync operation returned error 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000056[0] 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000001.61.6 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: a sync operation returned error 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: backing out with sync_status=-5 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: backing out with sync_status=-5 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket shard: 0 2024-04-16T18:18:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000162]: success 2024-04-16T18:18:02.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.624+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17:5 marker=00000000001.73.6 timestamp=2024-04-16T18:17:58.598685+0000 2024-04-16T18:18:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.628+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.628+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.628+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: sync status for source bucket shard: 0 2024-04-16T18:18:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.628+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.628+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: sync status for source bucket shard: 0 2024-04-16T18:18:02.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.628+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.636+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000056]: success 2024-04-16T18:18:02.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.636+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4:5 marker=00000000001.61.6 timestamp=2024-04-16T18:17:59.463830+0000 2024-04-16T18:18:02.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.636+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: a sync operation returned error 2024-04-16T18:18:02.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.636+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: backing out with sync_status=-5 2024-04-16T18:18:02.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.636+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.636+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.640+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.640+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:02.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.640+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.644+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: a sync operation returned error 2024-04-16T18:18:02.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.644+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: backing out with sync_status=-5 2024-04-16T18:18:02.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.644+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.644+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.648+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.648+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.648+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: parsed entry: id=00000000002.59.6 iter->object=obj00000000000000000007 iter->instance= name=obj00000000000000000007 instance= ns= 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15/obj00000000000000000007 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: entry->timestamp=2024-04-16T18:17:58.474169+0000 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.59.6 op=0 op_state=1 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000007[0] 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position=00000000002.59.6 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: parsed entry: id=00000000001.67.6 iter->object=obj00000000000000000423 iter->instance= name=obj00000000000000000423 instance= ns= 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20/obj00000000000000000423 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: entry->timestamp=2024-04-16T18:17:58.978960+0000 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.67.6 op=0 op_state=1 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000423[0] 2024-04-16T18:18:02.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.668+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position=00000000001.67.6 2024-04-16T18:18:02.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: parsed entry: id=00000000002.51.6 iter->object=obj00000000000000000012 iter->instance= name=obj00000000000000000012 instance= ns= 2024-04-16T18:18:02.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22/obj00000000000000000012 2024-04-16T18:18:02.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: entry->timestamp=2024-04-16T18:17:59.307488+0000 2024-04-16T18:18:02.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.51.6 op=0 op_state=1 2024-04-16T18:18:02.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000012[0] 2024-04-16T18:18:02.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position=00000000002.51.6 2024-04-16T18:18:02.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000007]: success 2024-04-16T18:18:02.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.672+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15:5 marker=00000000002.59.6 timestamp=2024-04-16T18:17:58.474169+0000 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: parsed entry: id=00000000001.39.6 iter->object=obj00000000000000000133 iter->instance= name=obj00000000000000000133 instance= ns= 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30/obj00000000000000000133 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: entry->timestamp=2024-04-16T18:17:58.827838+0000 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.39.6 op=0 op_state=1 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000133[0] 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: listing bilog for incremental sync; position=00000000001.39.6 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: parsed entry: id=00000000001.47.6 iter->object=obj00000000000000000029 iter->instance= name=obj00000000000000000029 instance= ns= 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29/obj00000000000000000029 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: entry->timestamp=2024-04-16T18:17:59.075578+0000 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.47.6 op=0 op_state=1 2024-04-16T18:18:02.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000029[0] 2024-04-16T18:18:02.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: listing bilog for incremental sync; position=00000000001.47.6 2024-04-16T18:18:02.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000012]: success 2024-04-16T18:18:02.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.676+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22:5 marker=00000000002.51.6 timestamp=2024-04-16T18:17:59.307488+0000 2024-04-16T18:18:02.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.680+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000423]: success 2024-04-16T18:18:02.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.680+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20:5 marker=00000000001.67.6 timestamp=2024-04-16T18:17:58.978960+0000 2024-04-16T18:18:02.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000002.54.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:02.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000000 2024-04-16T18:18:02.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:17:58.903499+0000 2024-04-16T18:18:02.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.54.6 op=0 op_state=1 2024-04-16T18:18:02.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000000[0] 2024-04-16T18:18:02.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000002.54.6 2024-04-16T18:18:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000133]: success 2024-04-16T18:18:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.684+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30:5 marker=00000000001.39.6 timestamp=2024-04-16T18:17:58.827838+0000 2024-04-16T18:18:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.688+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000000]: success 2024-04-16T18:18:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.688+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9:5 marker=00000000002.54.6 timestamp=2024-04-16T18:17:58.903499+0000 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000001.37.6 iter->object=obj00000000000000000271 iter->instance= name=obj00000000000000000271 instance= ns= 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:58.547717+0000 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.37.6 op=0 op_state=1 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: : injecting data sync error on key=obj00000000000000000271 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000001.37.6 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.692+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.736+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000029]: success 2024-04-16T18:18:02.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.736+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29:5 marker=00000000001.47.6 timestamp=2024-04-16T18:17:59.075578+0000 2024-04-16T18:18:02.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.740+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: a sync operation returned error 2024-04-16T18:18:02.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.740+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: backing out with sync_status=-5 2024-04-16T18:18:02.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.740+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.740+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.740+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.744+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.744+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000001.37.6 iter->object=obj00000000000000000271 iter->instance= name=obj00000000000000000271 instance= ns= 2024-04-16T18:18:02.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:58.547717+0000 2024-04-16T18:18:02.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.37.6 op=0 op_state=1 2024-04-16T18:18:02.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: : injecting data sync error on key=obj00000000000000000271 2024-04-16T18:18:02.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000001.37.6 2024-04-16T18:18:02.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.788+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: a sync operation returned error 2024-04-16T18:18:02.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: backing out with sync_status=-5 2024-04-16T18:18:02.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.836+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000001.37.6 iter->object=obj00000000000000000271 iter->instance= name=obj00000000000000000271 instance= ns= 2024-04-16T18:18:02.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000271 2024-04-16T18:18:02.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:17:58.547717+0000 2024-04-16T18:18:02.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.37.6 op=0 op_state=1 2024-04-16T18:18:02.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.884+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000271[0] 2024-04-16T18:18:02.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.884+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000001.37.6 2024-04-16T18:18:02.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.896+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000271]: success 2024-04-16T18:18:02.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.896+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12:5 marker=00000000001.37.6 timestamp=2024-04-16T18:17:58.547717+0000 2024-04-16T18:18:02.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: sync status for source bucket shard: 0 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: sync status for source bucket shard: 0 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: sync status for source bucket shard: 0 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: sync status for source bucket shard: 0 2024-04-16T18:18:02.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: sync status for source bucket shard: 0 2024-04-16T18:18:02.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: sync status for source bucket shard: 0 2024-04-16T18:18:02.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:6]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.944+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: sync status for source bucket shard: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: sync status for source bucket shard: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: sync status for source bucket shard: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: sync status for source bucket shard: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: parsed entry: id=00000000002.59.6 iter->object=obj00000000000000000033 iter->instance= name=obj00000000000000000033 instance= ns= 2024-04-16T18:18:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4/obj00000000000000000033 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: entry->timestamp=2024-04-16T18:18:00.362529+0000 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000033]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.59.6 op=0 op_state=1 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: parsed entry: id=00000000002.101.6 iter->object=obj00000000000000000011 iter->instance= name=obj00000000000000000011 instance= ns= 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2/obj00000000000000000011 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: entry->timestamp=2024-04-16T18:18:00.946708+0000 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000011]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.101.6 op=0 op_state=1 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000033]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000033[0] 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.948+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: listing bilog for incremental sync; position=00000000002.59.6 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: parsed entry: id=00000000002.139.6 iter->object=obj00000000000000000000 iter->instance= name=obj00000000000000000000 instance= ns= 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1/obj00000000000000000000 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: entry->timestamp=2024-04-16T18:18:01.193434+0000 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000000]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.139.6 op=0 op_state=1 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000011]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000011[0] 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: listing bilog for incremental sync; position=00000000002.101.6 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000000]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000000[0] 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: listing bilog for incremental sync; position=00000000002.139.6 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: parsed entry: id=00000000002.100.6 iter->object=obj00000000000000000001 iter->instance= name=obj00000000000000000001 instance= ns= 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3/obj00000000000000000001 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: entry->timestamp=2024-04-16T18:18:00.871140+0000 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000001]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.100.6 op=0 op_state=1 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000001]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000001[0] 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: listing bilog for incremental sync; position=00000000002.100.6 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: sync status for source bucket shard: 0 2024-04-16T18:18:02.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: sync status for source bucket shard: 0 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: parsed entry: id=00000000001.61.6 iter->object=obj00000000000000000065 iter->instance= name=obj00000000000000000065 instance= ns= 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5/obj00000000000000000065 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: entry->timestamp=2024-04-16T18:18:00.595634+0000 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000065]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.61.6 op=0 op_state=1 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000065]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000065[0] 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: listing bilog for incremental sync; position=00000000001.61.6 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: sync status for source bucket shard: 0 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.952+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: parsed entry: id=00000000002.67.6 iter->object=obj00000000000000000016 iter->instance= name=obj00000000000000000016 instance= ns= 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7/obj00000000000000000016 2024-04-16T18:18:02.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: entry->timestamp=2024-04-16T18:18:00.286171+0000 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000016]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.67.6 op=0 op_state=1 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: sync status for source bucket shard: 0 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000016]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000016[0] 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: listing bilog for incremental sync; position=00000000002.67.6 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: sync status for source bucket shard: 0 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: sync status for source bucket shard: 0 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: sync status for source bucket shard: 0 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: sync status for source bucket shard: 0 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: sync status for source bucket shard: 0 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:18]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: sync status for source bucket shard: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:32]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.956+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: sync status for source bucket shard: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: sync status for source bucket shard: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: sync status for source bucket shard: 0 2024-04-16T18:18:02.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket shard: 0 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: sync status for source bucket shard: 0 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: sync status for source bucket shard: 0 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: sync status for source bucket shard: 0 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: sync status for source bucket shard: 0 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:entry[obj00000000000000000033]: success 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:4:6 marker=00000000002.59.6 timestamp=2024-04-16T18:18:00.362529+0000 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.960+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:entry[obj00000000000000000001]: success 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:3:6 marker=00000000002.100.6 timestamp=2024-04-16T18:18:00.871140+0000 2024-04-16T18:18:02.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: sync status for source bucket shard: 0 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: sync status for source bucket shard: 0 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:32]: sync status for source bucket shard: 0 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: sync status for source bucket shard: 0 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:32]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:32]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: sync status for source bucket shard: 0 2024-04-16T18:18:02.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: parsed entry: id=00000000002.98.6 iter->object=obj00000000000000000040 iter->instance= name=obj00000000000000000040 instance= ns= 2024-04-16T18:18:02.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14/obj00000000000000000040 2024-04-16T18:18:02.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: entry->timestamp=2024-04-16T18:18:00.259038+0000 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000040]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.98.6 op=0 op_state=1 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000040]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000040[0] 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: listing bilog for incremental sync; position=00000000002.98.6 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: sync status for source bucket shard: 0 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: sync status for source bucket shard: 0 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: sync status for source bucket shard: 0 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:entry[obj00000000000000000016]: success 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.964+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:7:6 marker=00000000002.67.6 timestamp=2024-04-16T18:18:00.286171+0000 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:entry[obj00000000000000000011]: success 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:2:6 marker=00000000002.101.6 timestamp=2024-04-16T18:18:00.946708+0000 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:entry[obj00000000000000000000]: success 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:1:6 marker=00000000002.139.6 timestamp=2024-04-16T18:18:01.193434+0000 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: parsed entry: id=00000000002.81.6 iter->object=obj00000000000000000024 iter->instance= name=obj00000000000000000024 instance= ns= 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15/obj00000000000000000024 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: entry->timestamp=2024-04-16T18:18:01.167048+0000 2024-04-16T18:18:02.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000024]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.81.6 op=0 op_state=1 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: parsed entry: id=00000000002.78.6 iter->object=obj00000000000000000045 iter->instance= name=obj00000000000000000045 instance= ns= 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19/obj00000000000000000045 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: entry->timestamp=2024-04-16T18:18:00.047665+0000 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000045]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.78.6 op=0 op_state=1 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: parsed entry: id=00000000002.76.6 iter->object=obj00000000000000000002 iter->instance= name=obj00000000000000000002 instance= ns= 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17/obj00000000000000000002 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: entry->timestamp=2024-04-16T18:18:01.116481+0000 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000002]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.76.6 op=0 op_state=1 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000024]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000024[0] 2024-04-16T18:18:02.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: listing bilog for incremental sync; position=00000000002.81.6 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: parsed entry: id=00000000002.73.6 iter->object=obj00000000000000000030 iter->instance= name=obj00000000000000000030 instance= ns= 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11/obj00000000000000000030 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: entry->timestamp=2024-04-16T18:18:00.795226+0000 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000030]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.73.6 op=0 op_state=1 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000045]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000045[0] 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: listing bilog for incremental sync; position=00000000002.78.6 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: parsed entry: id=00000000002.69.6 iter->object=obj00000000000000000009 iter->instance= name=obj00000000000000000009 instance= ns= 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13/obj00000000000000000009 2024-04-16T18:18:02.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: entry->timestamp=2024-04-16T18:18:00.498902+0000 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000009]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.69.6 op=0 op_state=1 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000002]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000002[0] 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.968+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: listing bilog for incremental sync; position=00000000002.76.6 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.972+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000030]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000030[0] 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.972+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: listing bilog for incremental sync; position=00000000002.73.6 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.972+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000009]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000009[0] 2024-04-16T18:18:02.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.972+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: listing bilog for incremental sync; position=00000000002.69.6 2024-04-16T18:18:02.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: parsed entry: id=00000000002.100.6 iter->object=obj00000000000000000027 iter->instance= name=obj00000000000000000027 instance= ns= 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24/obj00000000000000000027 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: entry->timestamp=2024-04-16T18:18:00.646787+0000 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000027]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.100.6 op=0 op_state=1 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: parsed entry: id=00000000002.57.6 iter->object=obj00000000000000000021 iter->instance= name=obj00000000000000000021 instance= ns= 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26/obj00000000000000000021 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: entry->timestamp=2024-04-16T18:18:00.441439+0000 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:entry[obj00000000000000000021]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.57.6 op=0 op_state=1 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: parsed entry: id=00000000001.102.6 iter->object=obj00000000000000000062 iter->instance= name=obj00000000000000000062 instance= ns= 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12/obj00000000000000000062 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: entry->timestamp=2024-04-16T18:18:01.094956+0000 2024-04-16T18:18:02.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000062]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.102.6 op=0 op_state=1 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000027]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000027[0] 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: listing bilog for incremental sync; position=00000000002.100.6 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: parsed entry: id=00000000002.87.6 iter->object=obj00000000000000000039 iter->instance= name=obj00000000000000000039 instance= ns= 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23/obj00000000000000000039 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: entry->timestamp=2024-04-16T18:18:00.742403+0000 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.87.6 op=0 op_state=1 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: parsed entry: id=00000000002.79.6 iter->object=obj00000000000000000010 iter->instance= name=obj00000000000000000010 instance= ns= 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25/obj00000000000000000010 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: entry->timestamp=2024-04-16T18:18:00.895102+0000 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000010]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.79.6 op=0 op_state=1 2024-04-16T18:18:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: parsed entry: id=00000000002.79.6 iter->object=obj00000000000000000012 iter->instance= name=obj00000000000000000012 instance= ns= 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16/obj00000000000000000012 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: entry->timestamp=2024-04-16T18:18:00.123094+0000 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000012]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.79.6 op=0 op_state=1 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:entry[obj00000000000000000021]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000021[0] 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: listing bilog for incremental sync; position=00000000002.57.6 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000062]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000062[0] 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: listing bilog for incremental sync; position=00000000001.102.6 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: : injecting data sync error on key=obj00000000000000000039 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position=00000000002.87.6 2024-04-16T18:18:02.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000010]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000010[0] 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: listing bilog for incremental sync; position=00000000002.79.6 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000012]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000012[0] 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: listing bilog for incremental sync; position=00000000002.79.6 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23/obj00000000000000000039 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: parsed entry: id=00000000002.85.6 iter->object=obj00000000000000000053 iter->instance= name=obj00000000000000000053 instance= ns= 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27/obj00000000000000000053 2024-04-16T18:18:02.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: entry->timestamp=2024-04-16T18:18:00.722581+0000 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:entry[obj00000000000000000053]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.85.6 op=0 op_state=1 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: parsed entry: id=00000000002.99.6 iter->object=obj00000000000000000019 iter->instance= name=obj00000000000000000019 instance= ns= 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33/obj00000000000000000019 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: entry->timestamp=2024-04-16T18:18:00.101781+0000 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:entry[obj00000000000000000019]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.99.6 op=0 op_state=1 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:entry[obj00000000000000000053]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000053[0] 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: listing bilog for incremental sync; position=00000000002.85.6 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:entry[obj00000000000000000065]: success 2024-04-16T18:18:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.976+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:5:6 marker=00000000001.61.6 timestamp=2024-04-16T18:18:00.595634+0000 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:entry[obj00000000000000000040]: success 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:14:6 marker=00000000002.98.6 timestamp=2024-04-16T18:18:00.259038+0000 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:entry[obj00000000000000000019]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000019[0] 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: listing bilog for incremental sync; position=00000000002.99.6 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:entry[obj00000000000000000045]: success 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:19:6 marker=00000000002.78.6 timestamp=2024-04-16T18:18:00.047665+0000 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:entry[obj00000000000000000024]: success 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:15:6 marker=00000000002.81.6 timestamp=2024-04-16T18:18:01.167048+0000 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:entry[obj00000000000000000002]: success 2024-04-16T18:18:02.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:17:6 marker=00000000002.76.6 timestamp=2024-04-16T18:18:01.116481+0000 2024-04-16T18:18:02.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:entry[obj00000000000000000009]: success 2024-04-16T18:18:02.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:13:6 marker=00000000002.69.6 timestamp=2024-04-16T18:18:00.498902+0000 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: a sync operation returned error 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: backing out with sync_status=-5 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.980+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:entry[obj00000000000000000030]: success 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:11:6 marker=00000000002.73.6 timestamp=2024-04-16T18:18:00.795226+0000 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: sync status for source bucket shard: 0 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position= 2024-04-16T18:18:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:entry[obj00000000000000000012]: success 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:16:6 marker=00000000002.79.6 timestamp=2024-04-16T18:18:00.123094+0000 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:entry[obj00000000000000000010]: success 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:25:6 marker=00000000002.79.6 timestamp=2024-04-16T18:18:00.895102+0000 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:entry[obj00000000000000000027]: success 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.984+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:24:6 marker=00000000002.100.6 timestamp=2024-04-16T18:18:00.646787+0000 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.988+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:entry[obj00000000000000000019]: success 2024-04-16T18:18:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.988+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:33:6 marker=00000000002.99.6 timestamp=2024-04-16T18:18:00.101781+0000 2024-04-16T18:18:02.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.988+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:entry[obj00000000000000000021]: success 2024-04-16T18:18:02.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.988+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:26:6 marker=00000000002.57.6 timestamp=2024-04-16T18:18:00.441439+0000 2024-04-16T18:18:02.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: parsed entry: id=00000000001.80.6 iter->object=obj00000000000000000048 iter->instance= name=obj00000000000000000048 instance= ns= 2024-04-16T18:18:02.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8/obj00000000000000000048 2024-04-16T18:18:02.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: entry->timestamp=2024-04-16T18:18:00.574989+0000 2024-04-16T18:18:02.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000048]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.80.6 op=0 op_state=1 2024-04-16T18:18:02.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000048]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000048[0] 2024-04-16T18:18:02.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: listing bilog for incremental sync; position=00000000001.80.6 2024-04-16T18:18:02.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:entry[obj00000000000000000062]: success 2024-04-16T18:18:02.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.992+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:12:6 marker=00000000001.102.6 timestamp=2024-04-16T18:18:01.094956+0000 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: parsed entry: id=00000000001.102.6 iter->object=obj00000000000000000060 iter->instance= name=obj00000000000000000060 instance= ns= 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0/obj00000000000000000060 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: entry->timestamp=2024-04-16T18:18:01.022952+0000 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000060]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.102.6 op=0 op_state=1 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000060]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000060[0] 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: listing bilog for incremental sync; position=00000000001.102.6 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: parsed entry: id=00000000002.67.6 iter->object=obj00000000000000000015 iter->instance= name=obj00000000000000000015 instance= ns= 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9/obj00000000000000000015 2024-04-16T18:18:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: entry->timestamp=2024-04-16T18:18:01.246790+0000 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000015]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.67.6 op=0 op_state=1 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.77.6 iter->object=obj00000000000000000026 iter->instance= name=obj00000000000000000026 instance= ns= 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000026 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:18:00.338378+0000 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.77.6 op=0 op_state=1 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000015]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000015[0] 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: listing bilog for incremental sync; position=00000000002.67.6 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: : injecting data sync error on key=obj00000000000000000026 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.77.6 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:18:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 0 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: ERROR: failed to sync object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000026 2024-04-16T18:18:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:entry[obj00000000000000000053]: success 2024-04-16T18:18:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:02.996+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:27:6 marker=00000000002.85.6 timestamp=2024-04-16T18:18:00.722581+0000 2024-04-16T18:18:02.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.000+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:entry[obj00000000000000000015]: success 2024-04-16T18:18:02.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.000+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:9:6 marker=00000000002.67.6 timestamp=2024-04-16T18:18:01.246790+0000 2024-04-16T18:18:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.004+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:entry[obj00000000000000000048]: success 2024-04-16T18:18:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.004+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:8:6 marker=00000000001.80.6 timestamp=2024-04-16T18:18:00.574989+0000 2024-04-16T18:18:03.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.008+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:entry[obj00000000000000000060]: success 2024-04-16T18:18:03.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.008+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:0:6 marker=00000000001.102.6 timestamp=2024-04-16T18:18:01.022952+0000 2024-04-16T18:18:03.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: parsed entry: id=00000000002.59.6 iter->object=obj00000000000000000006 iter->instance= name=obj00000000000000000006 instance= ns= 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20/obj00000000000000000006 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: entry->timestamp=2024-04-16T18:18:00.972809+0000 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000006]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.59.6 op=0 op_state=1 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000006]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000006[0] 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: listing bilog for incremental sync; position=00000000002.59.6 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: parsed entry: id=00000000002.86.6 iter->object=obj00000000000000000020 iter->instance= name=obj00000000000000000020 instance= ns= 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28/obj00000000000000000020 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: entry->timestamp=2024-04-16T18:18:00.819088+0000 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000020]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.86.6 op=0 op_state=1 2024-04-16T18:18:03.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000020]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000020[0] 2024-04-16T18:18:03.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.012+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: listing bilog for incremental sync; position=00000000002.86.6 2024-04-16T18:18:03.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: parsed entry: id=00000000002.141.6 iter->object=obj00000000000000000007 iter->instance= name=obj00000000000000000007 instance= ns= 2024-04-16T18:18:03.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22/obj00000000000000000007 2024-04-16T18:18:03.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: entry->timestamp=2024-04-16T18:18:01.270969+0000 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000007]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.141.6 op=0 op_state=1 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000007]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000007[0] 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: listing bilog for incremental sync; position=00000000002.141.6 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: parsed entry: id=00000000002.85.6 iter->object=obj00000000000000000017 iter->instance= name=obj00000000000000000017 instance= ns= 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21/obj00000000000000000017 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: entry->timestamp=2024-04-16T18:18:00.178964+0000 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000017]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.85.6 op=0 op_state=1 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: parsed entry: id=00000000002.61.6 iter->object=obj00000000000000000004 iter->instance= name=obj00000000000000000004 instance= ns= 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29/obj00000000000000000004 2024-04-16T18:18:03.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: entry->timestamp=2024-04-16T18:18:01.044770+0000 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000004]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.61.6 op=0 op_state=1 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: parsed entry: id=00000000001.75.6 iter->object=obj00000000000000000036 iter->instance= name=obj00000000000000000036 instance= ns= 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30/obj00000000000000000036 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: entry->timestamp=2024-04-16T18:18:00.205241+0000 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000036]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.75.6 op=0 op_state=1 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: parsed entry: id=00000000002.56.6 iter->object=obj00000000000000000008 iter->instance= name=obj00000000000000000008 instance= ns= 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36/obj00000000000000000008 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: entry->timestamp=2024-04-16T18:18:00.522761+0000 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:entry[obj00000000000000000008]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.56.6 op=0 op_state=1 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000017]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000017[0] 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: listing bilog for incremental sync; position=00000000002.85.6 2024-04-16T18:18:03.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000004]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000004[0] 2024-04-16T18:18:03.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: listing bilog for incremental sync; position=00000000002.61.6 2024-04-16T18:18:03.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000036]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000036[0] 2024-04-16T18:18:03.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: listing bilog for incremental sync; position=00000000001.75.6 2024-04-16T18:18:03.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:entry[obj00000000000000000008]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000008[0] 2024-04-16T18:18:03.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: listing bilog for incremental sync; position=00000000002.56.6 2024-04-16T18:18:03.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:entry[obj00000000000000000006]: success 2024-04-16T18:18:03.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:20:6 marker=00000000002.59.6 timestamp=2024-04-16T18:18:00.972809+0000 2024-04-16T18:18:03.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:entry[obj00000000000000000020]: success 2024-04-16T18:18:03.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.016+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:28:6 marker=00000000002.86.6 timestamp=2024-04-16T18:18:00.819088+0000 2024-04-16T18:18:03.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: parsed entry: id=00000000002.79.6 iter->object=obj00000000000000000003 iter->instance= name=obj00000000000000000003 instance= ns= 2024-04-16T18:18:03.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31/obj00000000000000000003 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: entry->timestamp=2024-04-16T18:18:00.415588+0000 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:entry[obj00000000000000000003]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.79.6 op=0 op_state=1 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:entry[obj00000000000000000003]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000003[0] 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: listing bilog for incremental sync; position=00000000002.79.6 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: parsed entry: id=00000000001.71.6 iter->object=obj00000000000000000057 iter->instance= name=obj00000000000000000057 instance= ns= 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34/obj00000000000000000057 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: entry->timestamp=2024-04-16T18:18:00.671115+0000 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:entry[obj00000000000000000057]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000001.71.6 op=0 op_state=1 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: parsed entry: id=00000000002.81.6 iter->object=obj00000000000000000018 iter->instance= name=obj00000000000000000018 instance= ns= 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35/obj00000000000000000018 2024-04-16T18:18:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: entry->timestamp=2024-04-16T18:18:01.323643+0000 2024-04-16T18:18:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:entry[obj00000000000000000018]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.81.6 op=0 op_state=1 2024-04-16T18:18:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:entry[obj00000000000000000057]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000057[0] 2024-04-16T18:18:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: listing bilog for incremental sync; position=00000000001.71.6 2024-04-16T18:18:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:entry[obj00000000000000000018]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000018[0] 2024-04-16T18:18:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.020+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: listing bilog for incremental sync; position=00000000002.81.6 2024-04-16T18:18:03.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:entry[obj00000000000000000004]: success 2024-04-16T18:18:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:29:6 marker=00000000002.61.6 timestamp=2024-04-16T18:18:01.044770+0000 2024-04-16T18:18:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:entry[obj00000000000000000017]: success 2024-04-16T18:18:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:21:6 marker=00000000002.85.6 timestamp=2024-04-16T18:18:00.178964+0000 2024-04-16T18:18:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:entry[obj00000000000000000007]: success 2024-04-16T18:18:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:22:6 marker=00000000002.141.6 timestamp=2024-04-16T18:18:01.270969+0000 2024-04-16T18:18:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:entry[obj00000000000000000003]: success 2024-04-16T18:18:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:31:6 marker=00000000002.79.6 timestamp=2024-04-16T18:18:00.415588+0000 2024-04-16T18:18:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:entry[obj00000000000000000008]: success 2024-04-16T18:18:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.024+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:36:6 marker=00000000002.56.6 timestamp=2024-04-16T18:18:00.522761+0000 2024-04-16T18:18:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.028+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:entry[obj00000000000000000018]: success 2024-04-16T18:18:03.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.028+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:35:6 marker=00000000002.81.6 timestamp=2024-04-16T18:18:01.323643+0000 2024-04-16T18:18:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: parsed entry: id=00000000002.87.6 iter->object=obj00000000000000000039 iter->instance= name=obj00000000000000000039 instance= ns= 2024-04-16T18:18:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23/obj00000000000000000039 2024-04-16T18:18:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: entry->timestamp=2024-04-16T18:18:00.742403+0000 2024-04-16T18:18:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.87.6 op=0 op_state=1 2024-04-16T18:18:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000039[0] 2024-04-16T18:18:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: listing bilog for incremental sync; position=00000000002.87.6 2024-04-16T18:18:03.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:entry[obj00000000000000000036]: success 2024-04-16T18:18:03.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:30:6 marker=00000000001.75.6 timestamp=2024-04-16T18:18:00.205241+0000 2024-04-16T18:18:03.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:entry[obj00000000000000000057]: success 2024-04-16T18:18:03.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.032+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:34:6 marker=00000000001.71.6 timestamp=2024-04-16T18:18:00.671115+0000 2024-04-16T18:18:03.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.036+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:entry[obj00000000000000000039]: success 2024-04-16T18:18:03.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.036+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:23:6 marker=00000000002.87.6 timestamp=2024-04-16T18:18:00.742403+0000 2024-04-16T18:18:03.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: a sync operation returned error 2024-04-16T18:18:03.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: backing out with sync_status=-5 2024-04-16T18:18:03.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: incremental sync on bucket failed, retcode=-5 2024-04-16T18:18:03.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: ERROR: incremental sync failed. error: -5 2024-04-16T18:18:03.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:18:03.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: sync status for source bucket shard: 0 2024-04-16T18:18:03.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.048+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position= 2024-04-16T18:18:03.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.096+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: parsed entry: id=00000000002.77.6 iter->object=obj00000000000000000026 iter->instance= name=obj00000000000000000026 instance= ns= 2024-04-16T18:18:03.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.096+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: syncing object: lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10/obj00000000000000000026 2024-04-16T18:18:03.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.096+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: entry->timestamp=2024-04-16T18:18:00.338378+0000 2024-04-16T18:18:03.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.096+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=00000000002.77.6 op=0 op_state=1 2024-04-16T18:18:03.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.096+0000 7f80bd517980 5 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-51[054102a3-8d94-4596-8b3b-6341eab312e1.4804.20])/obj00000000000000000026[0] 2024-04-16T18:18:03.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.096+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: listing bilog for incremental sync; position=00000000002.77.6 2024-04-16T18:18:03.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.100+0000 7f80bd517980 10 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:entry[obj00000000000000000026]: success 2024-04-16T18:18:03.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:03.100+0000 7f80bd517980 20 RGW-SYNC:bucket[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20<-lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]:inc_sync[lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10]: updating marker marker_oid=bucket.sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20:10:6 marker=00000000002.77.6 timestamp=2024-04-16T18:18:00.338378+0000 2024-04-16T18:18:03.201 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'list', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--max-entries', '10000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:03.201 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'list', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--max-entries', '10000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:03.201 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket list --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --max-entries 10000 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:03.249 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:03.250 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:03.358 INFO:teuthology.orchestra.run.smithi071.stdout:[{"name":"obj00000000000000000000","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:01.193434Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12306348791650838042","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000001","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.871140Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1131690239766359880","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000002","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:01.116481Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.14834362749817315493","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000003","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.415588Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.7445453748806745655","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000004","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:01.044770Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1470862555030628089","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000005","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:59.359305Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.3940934019633484322","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000006","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.972809Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12881906332375880934","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000007","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.270969Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.5722628356513414945","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000008","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.522761Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.7057304279773085714","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000009","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.498902Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12211574932283166056","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000010","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.895102Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.9800316779120157511","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000011","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.946708Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.11791596244480908512","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000012","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.123094Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17293968300094322649","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000013","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:57.214824Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10291571314054402588","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000014","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:57.671549Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17232103691160640515","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000015","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:01.246790Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10338843543591551801","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000016","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.286171Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.9353370261667580451","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000017","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.178964Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.11841176878090133651","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000018","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.323643Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2085971350341147877","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000019","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.101781Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.18053532087107939056","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000020","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.819088Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.414704037991788237","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000021","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.441439Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.14223768054146798732","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000022","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.202824Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.43485213420569720","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000023","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:58.848804Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.8087177767078188271","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000024","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.167048Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2766480934059000274","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000025","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:57.520807Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17086029202793304911","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000026","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.338378Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2145951499709698980","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000027","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.646787Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.13089977522429762702","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000028","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:53.925661Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1685563671693449229","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000029","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:59.075578Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2541384057615111855","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000030","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.795226Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.4015763088802734144","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000031","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:57.064923Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12816606107119761409","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000032","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:53.876796Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.7000378232533960675","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000033","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.362529Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2269467586747104810","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000036","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.205241Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15996601802009596303","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000037","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:57.195073Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.13927391583182632295","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000039","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.742403Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17212015147281877725","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000040","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.259038Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12009312844772526223","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000042","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:57.722493Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12445255666540762622","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000045","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.047665Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10677541415875706014","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000046","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:56.102179Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12431220707394330358","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000048","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.574989Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.174001836252625915","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000049","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:57.946358Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.14288987971475977869","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000050","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:59.127593Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6863051318824901738","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000053","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.722581Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.5736576099223949520","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000055","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:55.752417Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17672284362405590121","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000056","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.463830Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1013595773175425771","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000057","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.671115Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.7995354669970626730","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000058","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:56.349217Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12460293186273745160","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000060","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.022952Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17170274423641896596","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000062","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.094956Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12582643234252616091","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000065","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.595634Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.14052574070441654277","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000076","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:53.672376Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6959762672428908849","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000133","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:58.827838Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2452257791576612580","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000162","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:58.598685Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6579898528581000519","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000271","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:58.547717Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10041355614440363913","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000283","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.515055Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.216288035745188115","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000317","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:58.751497Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1047564138620395652","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000423","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:58.978960Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.400466505127607677","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000467","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:58.773708Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12946713616082984321","flags":0,"pending_map":[],"versioned_epoch":0}] 2024-04-16T18:18:03.359 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:03.359 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-51', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:03.359 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket list --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-51 --max-entries 10000 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:03.404 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:03.404 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:[{"name":"obj00000000000000000000","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.193434Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_iyMydOlaaSxoeSkOw3hwHV_h81-gyGI","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000001","instance":"","ver":{"pool":9,"epoch":23},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.871140Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_UJWleXlQfCGq0XjsVsiHRWLSmOrycpg","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000002","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:01.116481Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_9pDWm4JliYCZQ_X-pMr4lfuLQNSU1qc","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000003","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.415588Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lanhAA22BMe4g0u4E6Fh_eQO1Vh_xZA","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000004","instance":"","ver":{"pool":9,"epoch":1},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:01.044770Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_q795efKTqbX7ZnwaEkDnVvCb0a99Ygk","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000005","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.359305Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_A9J7h5eM3AHhfd_nqqbLQAccK9HIY-2","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000006","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.972809Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_aL0Nacs2k7Oy_7QayunT4_DXpbqOQRC","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000007","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:01.270969Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_v3pBkIRKeHFhreXP-y6b7F21ZjU77b3","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000008","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.522761Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_9bCB77apCHGrK48yzSH6_LZcw5BjxDW","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000009","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.498902Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_-bnBhwwHWXXddX3hQ0Hi50Ybue0o4r_","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000010","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.895102Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_7ifr-OBkGspCOn8nieUResLL8TvSQou","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000011","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.946708Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_jrNBwPqxgEspX6HoKXdY8mTkJdXBTep","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000012","instance":"","ver":{"pool":9,"epoch":14},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.123094Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_3Dn8h0YlQGVdGR-XHBb263J7MpaY1w5","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000013","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:57.214824Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lrZFqrzcNskE6s2a63OC7_LZY68wwBw","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000014","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:57.671549Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_pto81LSs2HXkWS5CVcV1KtCKOoUs-Zc","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000015","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:01.246790Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_7013h4mfdJDOsb7PaPumg7ZolBtB_Dm","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000016","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.286171Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_cGuV6kUU7Jh6v1L9Gk9j-IZ94yY2X8-","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000017","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.178964Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_vUXvDQoHFKRn_A4eC22jQFefxTUQH-j","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000018","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:01.323643Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_sMV572wRqco4GP7JuEFH__ag6HzObyT","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000019","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.101781Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_2fA7-qaMFM6apKbKMPb3L1hJ4Xe2Nlb","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000020","instance":"","ver":{"pool":9,"epoch":2},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.819088Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_tAmmybl_uwxThcQpLwRN5OjMVhazBLx","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000021","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.441439Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_ZovukViYXG27fTWVnCbvtL_5-vYLbVx","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000022","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:59.202824Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lsEbXEVkKxClPlxB-3VDxlV8CNAK2Oi","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000023","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:58.848804Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_s3T1iZbNCqxw1qatObYDsp8NCBPgnMX","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000024","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.167048Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_yofaPdo_7ZsB-wMs-Xg0q2-4Dathj_o","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000025","instance":"","ver":{"pool":9,"epoch":37},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:57.520807Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_hDIotzIoSnNGLCQULd8jZ5lbfqDbd37","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000026","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.338378Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_EzFKKrLoe_fEzpD6xQIVVQbq1Re1gI1","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000027","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.646787Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lEXMKYT7WMdjIjaVNfpTVXSQhhenxQa","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000028","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:53.925661Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_SGTQop-uWDJtnnl6SatCAtqqO4F8DUz","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000029","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:59.075578Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_RVRjvC4a8psBINWwA3gwckHyJrjpwdi","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000030","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.795226Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_3bJ0QJ4fGswFGKk9_U5mZcPrVxCsmAr","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000031","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:57.064923Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_uvUy4Dq4RZCOpSieRJlnow-ukR8RVfo","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000032","instance":"","ver":{"pool":9,"epoch":12},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:53.876796Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_A5Vd776gPDarJ3apVT6V9bykTSfNmSP","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000033","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.362529Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lEiJOnzjjIA9Bru4fyi9iWkJmUy6Yxw","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000036","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:18:00.205241Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_YJ_1HVpItf8V2JZsrgjROgrCpasaw58","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000037","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:57.195073Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_Wryg4pQP7cTnZ0YhfQL3Infu204xxY3","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000039","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.742403Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_wfmLw-2B_NDsg5_qS0FezIC97h-x2HE","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000040","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.259038Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lAw1dE26t2oToFioy_Ml2XngRFwO-Z0","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000042","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:57.722493Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_B_iWpOQ-Up1stqKIF3ptvrE_7DoeZTA","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000045","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.047665Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_FPxb9YGER_MHOX2BLI1PV5_MVkFTmZy","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000046","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:56.102179Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_YLyGOXuwnnIL7_Pvl0BrGwlWtBt65gi","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000048","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:18:00.574989Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_mK2vzofqcA9u6gdpbTcGUlpnTzXLE34","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000049","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:57.946358Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_x9qghaAdgr_JmQ_0vTv376nMoGE9gTQ","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000050","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.127593Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_2c28pzxtEaQofQ1iUHSkxzHn4xw8IHm","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000053","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.722581Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_0xB_RXEPcObQKbKmExd4rzX9FxTni3M","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000055","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:55.752417Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_zq5kZ3w1ZZIOO8Muaf1_JLUIDAsMwSM","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000056","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.463830Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_O6IVzCmizQkabuaNGAT0NlezIwDlbyr","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000057","instance":"","ver":{"pool":9,"epoch":1},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:00.671115Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_LrulgPSN0PCacVRXH5zrQzL6tu2ySvX","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000058","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:56.349217Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_ET9cbkInzKRSG1n9rjdn3hZD8NNaE1H","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000060","instance":"","ver":{"pool":9,"epoch":1},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.022952Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_FhIl7JHOxaWgtSDHAYsEOEfmeTsKSCq","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000062","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:18:01.094956Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_3mYF4TdtQ7A-JFraZofoz58oOHAFjnj","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000065","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:18:00.595634Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_ORQyzBn5urhYa-GsuAT0hPK6yJxRih8","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000076","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:17:53.672376Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_3BFyJUwLuXiBFHGwk5ld7zuoVT99bJR","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000133","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:58.827838Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_cbJc4eZcQS-9yoRVdOplrAqYHrJBgGc","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000162","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:58.598685Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_s5fgH8YerFj_EVTyQjsEbyNFgdiZhuR","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000271","instance":"","ver":{"pool":9,"epoch":24},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:58.547717Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_YsCh5x1HB1Z3IKeQZfnq5VvjrLVe422","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000283","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:59.515055Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_nZSbgfCXkC1SCvRlvPdHyVmY7bhnCUv","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000317","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:17:58.751497Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_yhg2tKGLX4Ku4PghwVOnPiYgRosEy7k","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000423","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:17:58.978960Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_sRJ_3_MofJIYycYFSGpKVPGkk1qh3qe","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj00000000000000000467","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:17:58.773708Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_OpnVWGHwEIK2WrOL7ndMm_qFa1bYKN9","flags":0,"pending_map":[],"versioned_epoch":0}] 2024-04-16T18:18:03.511 INFO:tasks.rgw.c2.client.0:Restarting daemon 2024-04-16T18:18:03.511 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T18:18:03.556 INFO:tasks.rgw.c2.client.0:Started 2024-04-16T18:18:03.556 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:03.566 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T18:18:03.566 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:18:03.566 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:18:03.566 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T18:18:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:curl: (7) Failed to connect to smithi122.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T18:18:03.818 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:18:03.824+0000 7fd8a37dba40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T18:18:04.569 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:04.581 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:18:04.581 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:18:04.584 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 65361 0 --:--:-- --:--:-- --:--:-- 93500 2024-04-16T18:18:04.585 INFO:teuthology.orchestra.run.smithi122.stdout:anonymous 2024-04-16T18:18:04.585 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-51 2024-04-16T18:18:04.586 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:18:04.586 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:18:04.586 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-51 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:18:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:04.764+0000 7faa35be5980 1 bucket sync caught up with source: 2024-04-16T18:18:04.762 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.102.6, 00000000002.139.6, 00000000002.101.6, 00000000002.100.6, 00000000002.59.6, 00000000001.61.6, , 00000000002.67.6, 00000000001.80.6, 00000000002.67.6, 00000000002.77.6, 00000000002.73.6, 00000000001.102.6, 00000000002.69.6, 00000000002.98.6, 00000000002.81.6, 00000000002.79.6, 00000000002.76.6, , 00000000002.78.6, 00000000002.59.6, 00000000002.85.6, 00000000002.141.6, 00000000002.87.6, 00000000002.100.6, 00000000002.79.6, 00000000002.57.6, 00000000002.85.6, 00000000002.86.6, 00000000002.61.6, 00000000001.75.6, 00000000002.79.6, , 00000000002.99.6, 00000000001.71.6, 00000000002.81.6, 00000000002.56.6] 2024-04-16T18:18:04.762 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.102.6, 00000000002.139.6, 00000000002.101.6, 00000000002.100.6, 00000000002.59.6, 00000000001.61.6, , 00000000002.67.6, 00000000001.80.6, 00000000002.67.6, 00000000002.77.6, 00000000002.73.6, 00000000001.102.6, 00000000002.69.6, 00000000002.98.6, 00000000002.81.6, 00000000002.79.6, 00000000002.76.6, , 00000000002.78.6, 00000000002.59.6, 00000000002.85.6, 00000000002.141.6, 00000000002.87.6, 00000000002.100.6, 00000000002.79.6, 00000000002.57.6, 00000000002.85.6, 00000000002.86.6, 00000000002.61.6, 00000000001.75.6, 00000000002.79.6, , 00000000002.99.6, 00000000001.71.6, 00000000002.81.6, 00000000002.56.6] 2024-04-16T18:18:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:04.764+0000 7faa35be5980 0 bucket checkpoint complete 2024-04-16T18:18:04.792 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-51 2024-04-16T18:18:04.792 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:18:04.793 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-51', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:18:04.793 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-51 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:18:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:04.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:18:04.928+0000 7f44fbee6980 1 waiting for incremental sync to catch up: 2024-04-16T18:18:04.929 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000001.896.6, 00000000007.426.6, 00000000006.945.6, 00000000008.1036.6, 00000000001.624.6, 00000000008.330.6, 00000000007.923.6, 00000000006.1322.6, 00000000001.934.6, 00000000007.434.6, 00000000004.637.6] 2024-04-16T18:18:04.929 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000005.904.6, 00000000007.426.6, 00000000006.945.6, 00000000008.1036.6, 00000000005.639.6, 00000000008.330.6, 00000000007.923.6, 00000000006.1322.6, 00000000007.946.6, 00000000007.434.6, 00000000004.637.6] 2024-04-16T18:18:09.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:18:09.932+0000 7f44fbee6980 1 bucket sync caught up with source: 2024-04-16T18:18:09.932 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [00000000005.904.6, 00000000007.426.6, 00000000006.945.6, 00000000008.1036.6, 00000000005.639.6, 00000000008.330.6, 00000000007.923.6, 00000000006.1322.6, 00000000007.946.6, 00000000007.434.6, 00000000004.637.6] 2024-04-16T18:18:09.933 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [00000000005.904.6, 00000000007.426.6, 00000000006.945.6, 00000000008.1036.6, 00000000005.639.6, 00000000008.330.6, 00000000007.923.6, 00000000006.1322.6, 00000000007.946.6, 00000000007.434.6, 00000000004.637.6] 2024-04-16T18:18:09.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:18:09.932+0000 7f44fbee6980 0 bucket checkpoint complete 2024-04-16T18:18:09.983 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-51 zones={test-zone2, test-zone1} 2024-04-16T18:18:09.983 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:09.983 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:09.983 DEBUG:boto:Method: HEAD 2024-04-16T18:18:09.983 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:09.983 DEBUG:boto:Data: 2024-04-16T18:18:09.984 DEBUG:boto:Headers: {} 2024-04-16T18:18:09.984 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:09.984 DEBUG:boto:Port: 8000 2024-04-16T18:18:09.984 DEBUG:boto:Params: {} 2024-04-16T18:18:09.984 DEBUG:boto:Token: None 2024-04-16T18:18:09.985 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:09 GMT /lybtug-51/ 2024-04-16T18:18:09.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6gu7Yu7Dc4HgKD6z0fGaIb1qKbg= 2024-04-16T18:18:09.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6gu7Yu7Dc4HgKD6z0fGaIb1qKbg=', 'Content-Length': '0'} 2024-04-16T18:18:09.985 DEBUG:boto:encountered RemoteDisconnected exception, reconnecting 2024-04-16T18:18:09.985 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:18:10.448 DEBUG:boto:Token: None 2024-04-16T18:18:10.448 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:09 GMT /lybtug-51/ 2024-04-16T18:18:10.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6gu7Yu7Dc4HgKD6z0fGaIb1qKbg= 2024-04-16T18:18:10.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:09 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6gu7Yu7Dc4HgKD6z0fGaIb1qKbg=', 'Content-Length': '0'} 2024-04-16T18:18:10.460 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f53b5241be415ac2-00661ec0e2-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.460 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.460 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.460 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.460 DEBUG:boto:Path: /lybtug-51/ 2024-04-16T18:18:10.460 DEBUG:boto:Data: 2024-04-16T18:18:10.460 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.460 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.461 DEBUG:boto:Params: {} 2024-04-16T18:18:10.461 DEBUG:boto:Token: None 2024-04-16T18:18:10.461 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QbPPuXOtTFfpLpvdmzMNaaHi5mU= 2024-04-16T18:18:10.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QbPPuXOtTFfpLpvdmzMNaaHi5mU=', 'Content-Length': '0'} 2024-04-16T18:18:10.467 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '60'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006b7e290fa17c837b-00661ec0e2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.467 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:18:10.467 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.468 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.468 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.468 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.468 DEBUG:boto:Method: GET 2024-04-16T18:18:10.468 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.468 DEBUG:boto:Data: 2024-04-16T18:18:10.468 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.468 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.468 DEBUG:boto:Params: {} 2024-04-16T18:18:10.468 DEBUG:boto:Token: None 2024-04-16T18:18:10.469 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/?versions 2024-04-16T18:18:10.469 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY= 2024-04-16T18:18:10.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY=', 'Content-Length': '0'} 2024-04-16T18:18:10.475 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f63b2d7c7565aa31-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.476 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000004nulltrue2024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000007nulltrue2024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000008nulltrue2024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000009nulltrue2024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000014nulltrue2024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000015nulltrue2024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000016nulltrue2024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000017nulltrue2024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000018nulltrue2024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000019nulltrue2024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000020nulltrue2024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000021nulltrue2024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000022nulltrue2024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000023nulltrue2024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000024nulltrue2024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000025nulltrue2024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000026nulltrue2024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000027nulltrue2024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000028nulltrue2024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000029nulltrue2024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000030nulltrue2024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000031nulltrue2024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000032nulltrue2024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000033nulltrue2024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000036nulltrue2024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000037nulltrue2024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000039nulltrue2024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000040nulltrue2024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000042nulltrue2024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000045nulltrue2024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000046nulltrue2024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000048nulltrue2024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000049nulltrue2024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000050nulltrue2024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000053nulltrue2024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000055nulltrue2024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000056nulltrue2024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000057nulltrue2024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000058nulltrue2024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000060nulltrue2024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000062nulltrue2024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000065nulltrue2024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000076nulltrue2024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000133nulltrue2024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000162nulltrue2024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000271nulltrue2024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000283nulltrue2024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000317nulltrue2024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000423nulltrue2024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000467nulltrue2024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.478 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000000 2024-04-16T18:18:10.478 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000001 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000002 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000003 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000004 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000005 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000006 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000007 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000008 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000009 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000010 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000011 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000012 2024-04-16T18:18:10.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000013 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000014 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000015 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000016 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000017 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000018 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000019 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000020 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000021 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000022 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000023 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000024 2024-04-16T18:18:10.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000025 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000026 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000027 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000028 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000029 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000030 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000031 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000032 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000033 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000036 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000037 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000039 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000040 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000042 2024-04-16T18:18:10.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000045 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000046 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000048 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000049 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000050 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000053 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000055 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000056 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000057 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000058 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000060 2024-04-16T18:18:10.482 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000062 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000065 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000076 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000133 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000162 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000271 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000283 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000317 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000423 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000467 2024-04-16T18:18:10.483 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:18:10.483 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.484 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.484 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.484 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.484 DEBUG:boto:Method: GET 2024-04-16T18:18:10.484 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.484 DEBUG:boto:Data: 2024-04-16T18:18:10.484 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.484 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.484 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.484 DEBUG:boto:Params: {} 2024-04-16T18:18:10.484 DEBUG:boto:Token: None 2024-04-16T18:18:10.485 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/?versions 2024-04-16T18:18:10.485 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY= 2024-04-16T18:18:10.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY=', 'Content-Length': '0'} 2024-04-16T18:18:10.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e772ee3d3725d42c-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.492 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000004nulltrue2024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000007nulltrue2024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000008nulltrue2024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000009nulltrue2024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000014nulltrue2024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000015nulltrue2024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000016nulltrue2024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000017nulltrue2024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000018nulltrue2024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000019nulltrue2024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000020nulltrue2024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000021nulltrue2024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000022nulltrue2024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000023nulltrue2024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000024nulltrue2024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000025nulltrue2024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000026nulltrue2024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000027nulltrue2024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000028nulltrue2024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000029nulltrue2024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000030nulltrue2024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000031nulltrue2024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000032nulltrue2024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000033nulltrue2024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000036nulltrue2024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000037nulltrue2024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000039nulltrue2024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000040nulltrue2024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000042nulltrue2024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000045nulltrue2024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000046nulltrue2024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000048nulltrue2024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000049nulltrue2024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000050nulltrue2024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000053nulltrue2024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000055nulltrue2024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000056nulltrue2024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000057nulltrue2024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000058nulltrue2024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000060nulltrue2024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000062nulltrue2024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000065nulltrue2024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000076nulltrue2024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000133nulltrue2024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000162nulltrue2024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000271nulltrue2024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000283nulltrue2024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000317nulltrue2024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000423nulltrue2024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000467nulltrue2024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.494 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000000 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000001 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000002 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000003 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000004 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000005 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000006 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000007 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000008 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000009 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000010 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000011 2024-04-16T18:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000012 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000013 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000014 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000015 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000016 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000017 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000018 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000019 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000020 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000021 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000022 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000023 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000024 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000025 2024-04-16T18:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000026 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000027 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000028 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000029 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000030 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000031 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000032 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000033 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000036 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000037 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000039 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000040 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000042 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000045 2024-04-16T18:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000046 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000048 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000049 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000050 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000053 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000055 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000056 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000057 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000058 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000060 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000062 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000065 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000076 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000133 2024-04-16T18:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000162 2024-04-16T18:18:10.499 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000271 2024-04-16T18:18:10.499 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000283 2024-04-16T18:18:10.499 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000317 2024-04-16T18:18:10.499 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000423 2024-04-16T18:18:10.499 DEBUG:tasks.rgw_multi.zone_rados:o=obj00000000000000000467 2024-04-16T18:18:10.499 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.499 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.499 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.499 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.499 DEBUG:boto:Method: GET 2024-04-16T18:18:10.499 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.499 DEBUG:boto:Data: 2024-04-16T18:18:10.499 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.500 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.500 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.500 DEBUG:boto:Params: {} 2024-04-16T18:18:10.500 DEBUG:boto:Token: None 2024-04-16T18:18:10.500 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/?versions 2024-04-16T18:18:10.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY= 2024-04-16T18:18:10.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY=', 'Content-Length': '0'} 2024-04-16T18:18:10.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa31f311c1d4f352-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.506 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000004nulltrue2024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000007nulltrue2024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000008nulltrue2024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000009nulltrue2024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000014nulltrue2024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000015nulltrue2024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000016nulltrue2024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000017nulltrue2024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000018nulltrue2024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000019nulltrue2024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000020nulltrue2024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000021nulltrue2024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000022nulltrue2024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000023nulltrue2024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000024nulltrue2024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000025nulltrue2024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000026nulltrue2024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000027nulltrue2024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000028nulltrue2024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000029nulltrue2024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000030nulltrue2024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000031nulltrue2024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000032nulltrue2024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000033nulltrue2024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000036nulltrue2024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000037nulltrue2024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000039nulltrue2024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000040nulltrue2024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000042nulltrue2024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000045nulltrue2024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000046nulltrue2024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000048nulltrue2024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000049nulltrue2024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000050nulltrue2024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000053nulltrue2024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000055nulltrue2024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000056nulltrue2024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000057nulltrue2024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000058nulltrue2024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000060nulltrue2024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000062nulltrue2024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000065nulltrue2024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000076nulltrue2024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000133nulltrue2024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000162nulltrue2024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000271nulltrue2024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000283nulltrue2024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000317nulltrue2024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000423nulltrue2024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000467nulltrue2024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.508 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.508 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.509 DEBUG:boto:path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.509 DEBUG:boto:auth_path=/lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.509 DEBUG:boto:Method: GET 2024-04-16T18:18:10.509 DEBUG:boto:Path: /lybtug-51/?versions&max-keys=999 2024-04-16T18:18:10.509 DEBUG:boto:Data: 2024-04-16T18:18:10.509 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.509 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.509 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.509 DEBUG:boto:Params: {} 2024-04-16T18:18:10.509 DEBUG:boto:Token: None 2024-04-16T18:18:10.509 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/?versions 2024-04-16T18:18:10.510 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY= 2024-04-16T18:18:10.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CLPtdTxInRrF8N8xBYunTP6+bSY=', 'Content-Length': '0'} 2024-04-16T18:18:10.516 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000595ff0926dd7e41e-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.517 DEBUG:boto:b'lybtug-51999falseobj00000000000000000000nulltrue2024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000001nulltrue2024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000002nulltrue2024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000003nulltrue2024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000004nulltrue2024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000005nulltrue2024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000006nulltrue2024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000007nulltrue2024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000008nulltrue2024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000009nulltrue2024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000010nulltrue2024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000011nulltrue2024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000012nulltrue2024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000013nulltrue2024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000014nulltrue2024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000015nulltrue2024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000016nulltrue2024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000017nulltrue2024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000018nulltrue2024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000019nulltrue2024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000020nulltrue2024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000021nulltrue2024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000022nulltrue2024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000023nulltrue2024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000024nulltrue2024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000025nulltrue2024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000026nulltrue2024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000027nulltrue2024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000028nulltrue2024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000029nulltrue2024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000030nulltrue2024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000031nulltrue2024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000032nulltrue2024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000033nulltrue2024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000036nulltrue2024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000037nulltrue2024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000039nulltrue2024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000040nulltrue2024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000042nulltrue2024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000045nulltrue2024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000046nulltrue2024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000048nulltrue2024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000049nulltrue2024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000050nulltrue2024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000053nulltrue2024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000055nulltrue2024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000056nulltrue2024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000057nulltrue2024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000058nulltrue2024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000060nulltrue2024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000062nulltrue2024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000065nulltrue2024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000076nulltrue2024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000133nulltrue2024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000162nulltrue2024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000271nulltrue2024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000283nulltrue2024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000317nulltrue2024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000423nulltrue2024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj00000000000000000467nulltrue2024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000000 2024-04-16T18:18:10.520 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.521 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.521 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.521 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.521 DEBUG:boto:Method: GET 2024-04-16T18:18:10.521 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.521 DEBUG:boto:Data: 2024-04-16T18:18:10.521 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.521 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.521 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.521 DEBUG:boto:Params: {} 2024-04-16T18:18:10.521 DEBUG:boto:Token: None 2024-04-16T18:18:10.522 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE= 2024-04-16T18:18:10.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE=', 'Content-Length': '0'} 2024-04-16T18:18:10.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2272c214aeb4e3c-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.570 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.570 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.571 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.571 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.571 DEBUG:boto:Method: GET 2024-04-16T18:18:10.571 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.571 DEBUG:boto:Data: 2024-04-16T18:18:10.571 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.571 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.571 DEBUG:boto:Params: {} 2024-04-16T18:18:10.571 DEBUG:boto:Token: None 2024-04-16T18:18:10.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE= 2024-04-16T18:18:10.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE=', 'Content-Length': '0'} 2024-04-16T18:18:10.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066ed6d633f9cf96f-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.618 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.619 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.619 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.619 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.619 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.619 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.619 DEBUG:boto:Data: 2024-04-16T18:18:10.619 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.619 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.619 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.619 DEBUG:boto:Params: {} 2024-04-16T18:18:10.620 DEBUG:boto:Token: None 2024-04-16T18:18:10.620 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CT1qxMwqeNvXzs2kb0mI7bjm2uk= 2024-04-16T18:18:10.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CT1qxMwqeNvXzs2kb0mI7bjm2uk=', 'Content-Length': '0'} 2024-04-16T18:18:10.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e3b9935bca35975-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.623 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.623 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.623 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.623 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.623 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.623 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.624 DEBUG:boto:Data: 2024-04-16T18:18:10.624 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.624 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.624 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.624 DEBUG:boto:Params: {} 2024-04-16T18:18:10.624 DEBUG:boto:Token: None 2024-04-16T18:18:10.624 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CT1qxMwqeNvXzs2kb0mI7bjm2uk= 2024-04-16T18:18:10.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CT1qxMwqeNvXzs2kb0mI7bjm2uk=', 'Content-Length': '0'} 2024-04-16T18:18:10.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd45c138280d374a-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.628 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000000 2024-04-16T18:18:10.628 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.628 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.628 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.628 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.628 DEBUG:boto:Method: GET 2024-04-16T18:18:10.628 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.628 DEBUG:boto:Data: 2024-04-16T18:18:10.628 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.629 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.629 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.629 DEBUG:boto:Params: {} 2024-04-16T18:18:10.629 DEBUG:boto:Token: None 2024-04-16T18:18:10.629 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.629 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE= 2024-04-16T18:18:10.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE=', 'Content-Length': '0'} 2024-04-16T18:18:10.632 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2aa04a8c58aea5e-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.632 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.633 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.633 DEBUG:boto:path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.633 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.633 DEBUG:boto:Method: GET 2024-04-16T18:18:10.633 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.633 DEBUG:boto:Data: 2024-04-16T18:18:10.633 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.633 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.633 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.633 DEBUG:boto:Params: {} 2024-04-16T18:18:10.633 DEBUG:boto:Token: None 2024-04-16T18:18:10.634 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000?versionId=null 2024-04-16T18:18:10.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE= 2024-04-16T18:18:10.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vAZItTHsfd5akjzIYGXC2hCcowE=', 'Content-Length': '0'} 2024-04-16T18:18:10.637 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093dd4667f8fff546-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.637 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.637 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.637 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.637 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.637 DEBUG:boto:Method: GET 2024-04-16T18:18:10.638 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.638 DEBUG:boto:Data: 2024-04-16T18:18:10.638 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.638 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.638 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.638 DEBUG:boto:Params: {} 2024-04-16T18:18:10.638 DEBUG:boto:Token: None 2024-04-16T18:18:10.638 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.638 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.643 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002227c7fe3ad76fc0-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.643 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.644 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.644 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.644 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.644 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.644 DEBUG:boto:Method: GET 2024-04-16T18:18:10.644 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.644 DEBUG:boto:Data: 2024-04-16T18:18:10.644 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.644 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.645 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.645 DEBUG:boto:Params: {} 2024-04-16T18:18:10.645 DEBUG:boto:Token: None 2024-04-16T18:18:10.645 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.651 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3a2850b37126f56-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.651 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.652 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.652 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.652 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.652 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.652 DEBUG:boto:Data: 2024-04-16T18:18:10.652 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.652 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.652 DEBUG:boto:Params: {} 2024-04-16T18:18:10.652 DEBUG:boto:Token: None 2024-04-16T18:18:10.653 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.653 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14zHI3uymLqYs2M9207EkeFsnFc= 2024-04-16T18:18:10.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14zHI3uymLqYs2M9207EkeFsnFc=', 'Content-Length': '0'} 2024-04-16T18:18:10.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d60fb2e1691685be-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.656 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.656 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.656 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.656 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.656 DEBUG:boto:Data: 2024-04-16T18:18:10.656 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.656 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.656 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.656 DEBUG:boto:Params: {} 2024-04-16T18:18:10.656 DEBUG:boto:Token: None 2024-04-16T18:18:10.657 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:14zHI3uymLqYs2M9207EkeFsnFc= 2024-04-16T18:18:10.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:14zHI3uymLqYs2M9207EkeFsnFc=', 'Content-Length': '0'} 2024-04-16T18:18:10.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027fc255916000309-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.660 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000000 2024-04-16T18:18:10.660 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.660 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.660 DEBUG:boto:Method: GET 2024-04-16T18:18:10.660 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.661 DEBUG:boto:Data: 2024-04-16T18:18:10.661 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.661 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.661 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.661 DEBUG:boto:Params: {} 2024-04-16T18:18:10.661 DEBUG:boto:Token: None 2024-04-16T18:18:10.661 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.661 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WbfQlJJNVAoN2qwLcy5yR+2o6nw= 2024-04-16T18:18:10.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WbfQlJJNVAoN2qwLcy5yR+2o6nw=', 'Content-Length': '0'} 2024-04-16T18:18:10.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a91c20e346f86544-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.706 DEBUG:boto:path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.706 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.707 DEBUG:boto:Method: GET 2024-04-16T18:18:10.707 DEBUG:boto:Path: /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.707 DEBUG:boto:Data: 2024-04-16T18:18:10.707 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.707 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.707 DEBUG:boto:Params: {} 2024-04-16T18:18:10.707 DEBUG:boto:Token: None 2024-04-16T18:18:10.707 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000000 2024-04-16T18:18:10.707 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WbfQlJJNVAoN2qwLcy5yR+2o6nw= 2024-04-16T18:18:10.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WbfQlJJNVAoN2qwLcy5yR+2o6nw=', 'Content-Length': '0'} 2024-04-16T18:18:10.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000946475b94d4a57e0-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.711 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.711 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.711 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.711 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.711 DEBUG:boto:Method: GET 2024-04-16T18:18:10.711 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.711 DEBUG:boto:Data: 2024-04-16T18:18:10.711 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.711 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.711 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.711 DEBUG:boto:Params: {} 2024-04-16T18:18:10.712 DEBUG:boto:Token: None 2024-04-16T18:18:10.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b2e1a7a52e26f91-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.716 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.717 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.717 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.717 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.717 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.717 DEBUG:boto:Method: GET 2024-04-16T18:18:10.717 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000000 2024-04-16T18:18:10.717 DEBUG:boto:Data: 2024-04-16T18:18:10.717 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.718 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.718 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.718 DEBUG:boto:Params: {} 2024-04-16T18:18:10.718 DEBUG:boto:Token: None 2024-04-16T18:18:10.718 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.724 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b06eb315a5bb4c6-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.724 DEBUG:boto:b'lybtug-51obj000000000000000000001000falseobj000000000000000000002024-04-16T18:18:01.193Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.724 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000001 2024-04-16T18:18:10.724 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.724 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.724 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.724 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.725 DEBUG:boto:Method: GET 2024-04-16T18:18:10.725 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.725 DEBUG:boto:Data: 2024-04-16T18:18:10.725 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.725 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.725 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.725 DEBUG:boto:Params: {} 2024-04-16T18:18:10.725 DEBUG:boto:Token: None 2024-04-16T18:18:10.725 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.725 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY= 2024-04-16T18:18:10.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY=', 'Content-Length': '0'} 2024-04-16T18:18:10.729 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007276908f4240f5a2-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.729 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.729 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.729 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.729 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.729 DEBUG:boto:Method: GET 2024-04-16T18:18:10.729 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.729 DEBUG:boto:Data: 2024-04-16T18:18:10.730 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.730 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.730 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.730 DEBUG:boto:Params: {} 2024-04-16T18:18:10.730 DEBUG:boto:Token: None 2024-04-16T18:18:10.730 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.730 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY= 2024-04-16T18:18:10.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY=', 'Content-Length': '0'} 2024-04-16T18:18:10.733 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ca90795c7be5268-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.733 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.734 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.734 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.734 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.734 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.734 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.734 DEBUG:boto:Data: 2024-04-16T18:18:10.734 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.734 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.734 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.734 DEBUG:boto:Params: {} 2024-04-16T18:18:10.734 DEBUG:boto:Token: None 2024-04-16T18:18:10.735 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.735 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LlmafnrLAgbasi5AD6n6bEpvSXw= 2024-04-16T18:18:10.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LlmafnrLAgbasi5AD6n6bEpvSXw=', 'Content-Length': '0'} 2024-04-16T18:18:10.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea4b1fdbab084a68-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.739 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.739 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.739 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.739 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.739 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.740 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.740 DEBUG:boto:Data: 2024-04-16T18:18:10.740 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.740 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.740 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.740 DEBUG:boto:Params: {} 2024-04-16T18:18:10.740 DEBUG:boto:Token: None 2024-04-16T18:18:10.740 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LlmafnrLAgbasi5AD6n6bEpvSXw= 2024-04-16T18:18:10.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LlmafnrLAgbasi5AD6n6bEpvSXw=', 'Content-Length': '0'} 2024-04-16T18:18:10.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd5c120f59ddf667-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.743 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000001 2024-04-16T18:18:10.743 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.743 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.744 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.744 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.744 DEBUG:boto:Method: GET 2024-04-16T18:18:10.744 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.744 DEBUG:boto:Data: 2024-04-16T18:18:10.744 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.744 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.744 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.744 DEBUG:boto:Params: {} 2024-04-16T18:18:10.744 DEBUG:boto:Token: None 2024-04-16T18:18:10.744 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY= 2024-04-16T18:18:10.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY=', 'Content-Length': '0'} 2024-04-16T18:18:10.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a89113d6f7329cc-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.790 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.791 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.791 DEBUG:boto:path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.791 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.791 DEBUG:boto:Method: GET 2024-04-16T18:18:10.791 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.791 DEBUG:boto:Data: 2024-04-16T18:18:10.791 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.791 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.791 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.791 DEBUG:boto:Params: {} 2024-04-16T18:18:10.792 DEBUG:boto:Token: None 2024-04-16T18:18:10.792 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001?versionId=null 2024-04-16T18:18:10.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY= 2024-04-16T18:18:10.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RRgjxuuuP5sUZqX5CDHvdvvrqtY=', 'Content-Length': '0'} 2024-04-16T18:18:10.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000046928691613af86-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.795 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.795 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.795 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.795 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.796 DEBUG:boto:Method: GET 2024-04-16T18:18:10.796 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.796 DEBUG:boto:Data: 2024-04-16T18:18:10.796 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.796 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.796 DEBUG:boto:Params: {} 2024-04-16T18:18:10.796 DEBUG:boto:Token: None 2024-04-16T18:18:10.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010066cf26b9be7af-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.801 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.801 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.802 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.802 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.802 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.802 DEBUG:boto:Method: GET 2024-04-16T18:18:10.802 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.802 DEBUG:boto:Data: 2024-04-16T18:18:10.802 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.802 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.802 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.802 DEBUG:boto:Params: {} 2024-04-16T18:18:10.803 DEBUG:boto:Token: None 2024-04-16T18:18:10.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.809 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000864e5adf6b28f869-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.850 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.851 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.851 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.851 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.851 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.851 DEBUG:boto:Data: 2024-04-16T18:18:10.851 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.851 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.851 DEBUG:boto:Params: {} 2024-04-16T18:18:10.851 DEBUG:boto:Token: None 2024-04-16T18:18:10.852 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BmxtIp0FLCig95yu9XlCmvsJHkk= 2024-04-16T18:18:10.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BmxtIp0FLCig95yu9XlCmvsJHkk=', 'Content-Length': '0'} 2024-04-16T18:18:10.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c812cbc573bc2e0-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.855 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.855 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.855 DEBUG:boto:Method: HEAD 2024-04-16T18:18:10.855 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.855 DEBUG:boto:Data: 2024-04-16T18:18:10.855 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.855 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.855 DEBUG:boto:Params: {} 2024-04-16T18:18:10.855 DEBUG:boto:Token: None 2024-04-16T18:18:10.856 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BmxtIp0FLCig95yu9XlCmvsJHkk= 2024-04-16T18:18:10.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BmxtIp0FLCig95yu9XlCmvsJHkk=', 'Content-Length': '0'} 2024-04-16T18:18:10.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073d3efe1ef05bc61-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.859 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000001 2024-04-16T18:18:10.859 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.859 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.859 DEBUG:boto:Method: GET 2024-04-16T18:18:10.859 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.859 DEBUG:boto:Data: 2024-04-16T18:18:10.859 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.860 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.860 DEBUG:boto:Params: {} 2024-04-16T18:18:10.860 DEBUG:boto:Token: None 2024-04-16T18:18:10.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZQeBaV61jELPcisf5C3d5FtIqyc= 2024-04-16T18:18:10.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZQeBaV61jELPcisf5C3d5FtIqyc=', 'Content-Length': '0'} 2024-04-16T18:18:10.863 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e4e8d781519ff7b-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.906 DEBUG:boto:path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.906 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.907 DEBUG:boto:Method: GET 2024-04-16T18:18:10.907 DEBUG:boto:Path: /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.907 DEBUG:boto:Data: 2024-04-16T18:18:10.907 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.907 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.907 DEBUG:boto:Params: {} 2024-04-16T18:18:10.907 DEBUG:boto:Token: None 2024-04-16T18:18:10.907 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000001 2024-04-16T18:18:10.907 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZQeBaV61jELPcisf5C3d5FtIqyc= 2024-04-16T18:18:10.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZQeBaV61jELPcisf5C3d5FtIqyc=', 'Content-Length': '0'} 2024-04-16T18:18:10.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9e72e4889864ced-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.910 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.911 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.911 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.911 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.911 DEBUG:boto:Method: GET 2024-04-16T18:18:10.911 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.911 DEBUG:boto:Data: 2024-04-16T18:18:10.911 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.911 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.911 DEBUG:boto:Params: {} 2024-04-16T18:18:10.911 DEBUG:boto:Token: None 2024-04-16T18:18:10.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2da4f0c2991663c-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.958 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.959 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:10.959 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:10.959 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.959 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.959 DEBUG:boto:Method: GET 2024-04-16T18:18:10.959 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000001 2024-04-16T18:18:10.959 DEBUG:boto:Data: 2024-04-16T18:18:10.959 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:10.959 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.960 DEBUG:boto:Params: {} 2024-04-16T18:18:10.960 DEBUG:boto:Token: None 2024-04-16T18:18:10.960 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/ 2024-04-16T18:18:10.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec= 2024-04-16T18:18:10.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/6ifcduUn3H9S0vv/VoGEwQtec=', 'Content-Length': '0'} 2024-04-16T18:18:10.966 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae1b5121ce6c999b-00661ec0e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:10.966 DEBUG:boto:b'lybtug-51obj000000000000000000011000falseobj000000000000000000012024-04-16T18:18:00.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:10.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000002 2024-04-16T18:18:10.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:10.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:10.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:10.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:10.967 DEBUG:boto:Method: GET 2024-04-16T18:18:10.967 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:10.967 DEBUG:boto:Data: 2024-04-16T18:18:10.967 DEBUG:boto:Headers: {} 2024-04-16T18:18:10.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:10.967 DEBUG:boto:Port: 8000 2024-04-16T18:18:10.968 DEBUG:boto:Params: {} 2024-04-16T18:18:10.968 DEBUG:boto:Token: None 2024-04-16T18:18:10.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:10 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:10.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:trADVbFS7oS3piNOuZd0QiKbsCg= 2024-04-16T18:18:10.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:trADVbFS7oS3piNOuZd0QiKbsCg=', 'Content-Length': '0'} 2024-04-16T18:18:10.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da8a18b1d691aa1b-00661ec0e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.014 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.014 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.014 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.015 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.015 DEBUG:boto:Method: GET 2024-04-16T18:18:11.015 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.015 DEBUG:boto:Data: 2024-04-16T18:18:11.015 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.015 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.015 DEBUG:boto:Params: {} 2024-04-16T18:18:11.015 DEBUG:boto:Token: None 2024-04-16T18:18:11.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.015 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4Ei0DqIjfMmgQ5/kj3P5WqYKp7Q= 2024-04-16T18:18:11.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4Ei0DqIjfMmgQ5/kj3P5WqYKp7Q=', 'Content-Length': '0'} 2024-04-16T18:18:11.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f53a80fbd2fdaf1-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.019 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.019 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.019 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.019 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.020 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.020 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.020 DEBUG:boto:Data: 2024-04-16T18:18:11.020 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.020 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.020 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.020 DEBUG:boto:Params: {} 2024-04-16T18:18:11.020 DEBUG:boto:Token: None 2024-04-16T18:18:11.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O7JkayH9TT+3F2DwjPyWKVZUXqs= 2024-04-16T18:18:11.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O7JkayH9TT+3F2DwjPyWKVZUXqs=', 'Content-Length': '0'} 2024-04-16T18:18:11.024 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003ff0a40a9f83de4-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.024 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.024 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.024 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.024 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.024 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.024 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.024 DEBUG:boto:Data: 2024-04-16T18:18:11.024 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.024 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.025 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.025 DEBUG:boto:Params: {} 2024-04-16T18:18:11.025 DEBUG:boto:Token: None 2024-04-16T18:18:11.025 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.025 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O7JkayH9TT+3F2DwjPyWKVZUXqs= 2024-04-16T18:18:11.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O7JkayH9TT+3F2DwjPyWKVZUXqs=', 'Content-Length': '0'} 2024-04-16T18:18:11.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf897a1c573d471b-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.028 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000002 2024-04-16T18:18:11.028 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.029 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.029 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.029 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.029 DEBUG:boto:Method: GET 2024-04-16T18:18:11.029 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.029 DEBUG:boto:Data: 2024-04-16T18:18:11.029 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.029 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.029 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.029 DEBUG:boto:Params: {} 2024-04-16T18:18:11.029 DEBUG:boto:Token: None 2024-04-16T18:18:11.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.030 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4Ei0DqIjfMmgQ5/kj3P5WqYKp7Q= 2024-04-16T18:18:11.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4Ei0DqIjfMmgQ5/kj3P5WqYKp7Q=', 'Content-Length': '0'} 2024-04-16T18:18:11.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5d0039cdffdde99-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.074 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.074 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.075 DEBUG:boto:path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.075 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.075 DEBUG:boto:Method: GET 2024-04-16T18:18:11.075 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.075 DEBUG:boto:Data: 2024-04-16T18:18:11.075 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.075 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.075 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.075 DEBUG:boto:Params: {} 2024-04-16T18:18:11.075 DEBUG:boto:Token: None 2024-04-16T18:18:11.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002?versionId=null 2024-04-16T18:18:11.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4Ei0DqIjfMmgQ5/kj3P5WqYKp7Q= 2024-04-16T18:18:11.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4Ei0DqIjfMmgQ5/kj3P5WqYKp7Q=', 'Content-Length': '0'} 2024-04-16T18:18:11.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9024b60873ee5f1-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.079 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.079 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.080 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.080 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.080 DEBUG:boto:Method: GET 2024-04-16T18:18:11.080 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.080 DEBUG:boto:Data: 2024-04-16T18:18:11.080 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.080 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.080 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.080 DEBUG:boto:Params: {} 2024-04-16T18:18:11.080 DEBUG:boto:Token: None 2024-04-16T18:18:11.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad9c3c1121e8cf2e-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.085 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.085 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.085 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.085 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.086 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.086 DEBUG:boto:Method: GET 2024-04-16T18:18:11.086 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.086 DEBUG:boto:Data: 2024-04-16T18:18:11.086 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.086 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.086 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.086 DEBUG:boto:Params: {} 2024-04-16T18:18:11.086 DEBUG:boto:Token: None 2024-04-16T18:18:11.086 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.086 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.092 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027510d0ad71ad028-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.134 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.135 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.135 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.135 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.135 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.135 DEBUG:boto:Data: 2024-04-16T18:18:11.135 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.135 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.135 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.135 DEBUG:boto:Params: {} 2024-04-16T18:18:11.135 DEBUG:boto:Token: None 2024-04-16T18:18:11.136 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7RqREuOAQYsA8AU3wYstsL5Ou3Y= 2024-04-16T18:18:11.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7RqREuOAQYsA8AU3wYstsL5Ou3Y=', 'Content-Length': '0'} 2024-04-16T18:18:11.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082f630d65629f1bb-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.139 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.139 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.139 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.139 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.139 DEBUG:boto:Data: 2024-04-16T18:18:11.139 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.140 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.140 DEBUG:boto:Params: {} 2024-04-16T18:18:11.140 DEBUG:boto:Token: None 2024-04-16T18:18:11.140 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7RqREuOAQYsA8AU3wYstsL5Ou3Y= 2024-04-16T18:18:11.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7RqREuOAQYsA8AU3wYstsL5Ou3Y=', 'Content-Length': '0'} 2024-04-16T18:18:11.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a20f0c328b9a1184-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.143 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000002 2024-04-16T18:18:11.143 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.143 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.143 DEBUG:boto:Method: GET 2024-04-16T18:18:11.144 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.144 DEBUG:boto:Data: 2024-04-16T18:18:11.144 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.144 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.144 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.144 DEBUG:boto:Params: {} 2024-04-16T18:18:11.144 DEBUG:boto:Token: None 2024-04-16T18:18:11.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8b21BrTFA6HpR77KvAwDpJQTpDc= 2024-04-16T18:18:11.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8b21BrTFA6HpR77KvAwDpJQTpDc=', 'Content-Length': '0'} 2024-04-16T18:18:11.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d53daa843238911-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.190 DEBUG:boto:path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.190 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.191 DEBUG:boto:Method: GET 2024-04-16T18:18:11.191 DEBUG:boto:Path: /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.191 DEBUG:boto:Data: 2024-04-16T18:18:11.191 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.191 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.191 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.191 DEBUG:boto:Params: {} 2024-04-16T18:18:11.191 DEBUG:boto:Token: None 2024-04-16T18:18:11.191 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000002 2024-04-16T18:18:11.191 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8b21BrTFA6HpR77KvAwDpJQTpDc= 2024-04-16T18:18:11.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8b21BrTFA6HpR77KvAwDpJQTpDc=', 'Content-Length': '0'} 2024-04-16T18:18:11.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007602d39d8f1c8551-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.195 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.195 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.195 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.195 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.195 DEBUG:boto:Method: GET 2024-04-16T18:18:11.195 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.196 DEBUG:boto:Data: 2024-04-16T18:18:11.196 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.196 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.196 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.196 DEBUG:boto:Params: {} 2024-04-16T18:18:11.196 DEBUG:boto:Token: None 2024-04-16T18:18:11.196 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.200 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d89e35c99345176d-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.242 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.243 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.243 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.243 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.243 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.243 DEBUG:boto:Method: GET 2024-04-16T18:18:11.243 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000002 2024-04-16T18:18:11.243 DEBUG:boto:Data: 2024-04-16T18:18:11.243 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.243 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.244 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.244 DEBUG:boto:Params: {} 2024-04-16T18:18:11.244 DEBUG:boto:Token: None 2024-04-16T18:18:11.244 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038bc975367f43621-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.251 DEBUG:boto:b'lybtug-51obj000000000000000000021000falseobj000000000000000000022024-04-16T18:18:01.116Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000003 2024-04-16T18:18:11.251 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.251 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.251 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.251 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.251 DEBUG:boto:Method: GET 2024-04-16T18:18:11.252 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.252 DEBUG:boto:Data: 2024-04-16T18:18:11.252 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.252 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.252 DEBUG:boto:Params: {} 2024-04-16T18:18:11.252 DEBUG:boto:Token: None 2024-04-16T18:18:11.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M= 2024-04-16T18:18:11.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M=', 'Content-Length': '0'} 2024-04-16T18:18:11.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001650b5c0c056a554-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.298 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.298 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.298 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.299 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.299 DEBUG:boto:Method: GET 2024-04-16T18:18:11.299 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.299 DEBUG:boto:Data: 2024-04-16T18:18:11.299 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.299 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.299 DEBUG:boto:Params: {} 2024-04-16T18:18:11.299 DEBUG:boto:Token: None 2024-04-16T18:18:11.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M= 2024-04-16T18:18:11.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M=', 'Content-Length': '0'} 2024-04-16T18:18:11.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9a2a390e2ba4d1c-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.303 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.303 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.303 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.303 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.303 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.303 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.303 DEBUG:boto:Data: 2024-04-16T18:18:11.304 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.304 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.304 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.304 DEBUG:boto:Params: {} 2024-04-16T18:18:11.304 DEBUG:boto:Token: None 2024-04-16T18:18:11.304 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T6E30bICJeqMx5SzFXDN05zpn6Y= 2024-04-16T18:18:11.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T6E30bICJeqMx5SzFXDN05zpn6Y=', 'Content-Length': '0'} 2024-04-16T18:18:11.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9a7427941602f3e-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.307 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.307 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.307 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.308 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.308 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.308 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.308 DEBUG:boto:Data: 2024-04-16T18:18:11.308 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.308 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.308 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.308 DEBUG:boto:Params: {} 2024-04-16T18:18:11.308 DEBUG:boto:Token: None 2024-04-16T18:18:11.308 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T6E30bICJeqMx5SzFXDN05zpn6Y= 2024-04-16T18:18:11.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T6E30bICJeqMx5SzFXDN05zpn6Y=', 'Content-Length': '0'} 2024-04-16T18:18:11.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027c2105ccc78bc50-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.311 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000003 2024-04-16T18:18:11.311 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.311 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.312 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.312 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.312 DEBUG:boto:Method: GET 2024-04-16T18:18:11.312 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.312 DEBUG:boto:Data: 2024-04-16T18:18:11.312 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.312 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.312 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.312 DEBUG:boto:Params: {} 2024-04-16T18:18:11.312 DEBUG:boto:Token: None 2024-04-16T18:18:11.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.313 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M= 2024-04-16T18:18:11.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M=', 'Content-Length': '0'} 2024-04-16T18:18:11.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074e2bc59cfee4027-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.358 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.358 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.359 DEBUG:boto:path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.359 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.359 DEBUG:boto:Method: GET 2024-04-16T18:18:11.359 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.359 DEBUG:boto:Data: 2024-04-16T18:18:11.359 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.359 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.359 DEBUG:boto:Params: {} 2024-04-16T18:18:11.359 DEBUG:boto:Token: None 2024-04-16T18:18:11.359 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003?versionId=null 2024-04-16T18:18:11.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M= 2024-04-16T18:18:11.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3pR+xyQ6UmJIcfCUQxkvxX2jf6M=', 'Content-Length': '0'} 2024-04-16T18:18:11.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038933090ac66fa3e-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.363 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.363 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.363 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.363 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.363 DEBUG:boto:Method: GET 2024-04-16T18:18:11.363 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.363 DEBUG:boto:Data: 2024-04-16T18:18:11.363 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.364 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.364 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.364 DEBUG:boto:Params: {} 2024-04-16T18:18:11.364 DEBUG:boto:Token: None 2024-04-16T18:18:11.364 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbeea7a3f2c82f5d-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.368 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.369 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.369 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.369 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.369 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.369 DEBUG:boto:Method: GET 2024-04-16T18:18:11.369 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.369 DEBUG:boto:Data: 2024-04-16T18:18:11.369 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.369 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.369 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.370 DEBUG:boto:Params: {} 2024-04-16T18:18:11.370 DEBUG:boto:Token: None 2024-04-16T18:18:11.370 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.370 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a45cbcb9f764718c-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.418 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.419 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.419 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.419 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.419 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.419 DEBUG:boto:Data: 2024-04-16T18:18:11.419 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.419 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.419 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.419 DEBUG:boto:Params: {} 2024-04-16T18:18:11.419 DEBUG:boto:Token: None 2024-04-16T18:18:11.420 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2AXB+TNSDA8QfteIdynG2lgpbk= 2024-04-16T18:18:11.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2AXB+TNSDA8QfteIdynG2lgpbk=', 'Content-Length': '0'} 2024-04-16T18:18:11.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008de9a14f13166af1-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.423 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.423 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.423 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.423 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.424 DEBUG:boto:Data: 2024-04-16T18:18:11.424 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.424 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.424 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.424 DEBUG:boto:Params: {} 2024-04-16T18:18:11.424 DEBUG:boto:Token: None 2024-04-16T18:18:11.424 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H2AXB+TNSDA8QfteIdynG2lgpbk= 2024-04-16T18:18:11.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H2AXB+TNSDA8QfteIdynG2lgpbk=', 'Content-Length': '0'} 2024-04-16T18:18:11.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbb47377194254c5-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.428 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000003 2024-04-16T18:18:11.428 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.428 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.428 DEBUG:boto:Method: GET 2024-04-16T18:18:11.428 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.428 DEBUG:boto:Data: 2024-04-16T18:18:11.428 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.428 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.428 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.428 DEBUG:boto:Params: {} 2024-04-16T18:18:11.429 DEBUG:boto:Token: None 2024-04-16T18:18:11.429 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.429 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ja9QNGgVoF3g4O1x+2alVo/OBio= 2024-04-16T18:18:11.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ja9QNGgVoF3g4O1x+2alVo/OBio=', 'Content-Length': '0'} 2024-04-16T18:18:11.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003850b56194260fbe-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.474 DEBUG:boto:path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.474 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.475 DEBUG:boto:Method: GET 2024-04-16T18:18:11.475 DEBUG:boto:Path: /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.475 DEBUG:boto:Data: 2024-04-16T18:18:11.475 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.475 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.475 DEBUG:boto:Params: {} 2024-04-16T18:18:11.475 DEBUG:boto:Token: None 2024-04-16T18:18:11.475 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000003 2024-04-16T18:18:11.475 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ja9QNGgVoF3g4O1x+2alVo/OBio= 2024-04-16T18:18:11.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ja9QNGgVoF3g4O1x+2alVo/OBio=', 'Content-Length': '0'} 2024-04-16T18:18:11.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe0cee668dbf7580-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.479 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.479 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.479 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.479 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.479 DEBUG:boto:Method: GET 2024-04-16T18:18:11.479 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.479 DEBUG:boto:Data: 2024-04-16T18:18:11.479 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.479 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.480 DEBUG:boto:Params: {} 2024-04-16T18:18:11.480 DEBUG:boto:Token: None 2024-04-16T18:18:11.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0458286e5e53ff6-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.526 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.527 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.527 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.527 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.527 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.527 DEBUG:boto:Method: GET 2024-04-16T18:18:11.527 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000003 2024-04-16T18:18:11.527 DEBUG:boto:Data: 2024-04-16T18:18:11.527 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.527 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.527 DEBUG:boto:Params: {} 2024-04-16T18:18:11.528 DEBUG:boto:Token: None 2024-04-16T18:18:11.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e83bc0a234dc552c-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.534 DEBUG:boto:b'lybtug-51obj000000000000000000031000falseobj000000000000000000032024-04-16T18:18:00.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.534 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000004 2024-04-16T18:18:11.535 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.535 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.535 DEBUG:boto:path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.535 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.535 DEBUG:boto:Method: GET 2024-04-16T18:18:11.535 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.535 DEBUG:boto:Data: 2024-04-16T18:18:11.535 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.535 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.535 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.535 DEBUG:boto:Params: {} 2024-04-16T18:18:11.536 DEBUG:boto:Token: None 2024-04-16T18:18:11.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA= 2024-04-16T18:18:11.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA=', 'Content-Length': '0'} 2024-04-16T18:18:11.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000889bdd06bf2a8f0b-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.582 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.582 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.583 DEBUG:boto:path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.583 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.583 DEBUG:boto:Method: GET 2024-04-16T18:18:11.583 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.583 DEBUG:boto:Data: 2024-04-16T18:18:11.583 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.583 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.583 DEBUG:boto:Params: {} 2024-04-16T18:18:11.583 DEBUG:boto:Token: None 2024-04-16T18:18:11.583 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA= 2024-04-16T18:18:11.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA=', 'Content-Length': '0'} 2024-04-16T18:18:11.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad2ee6de0fb51500-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.589 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.590 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.590 DEBUG:boto:path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.590 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.590 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.590 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.590 DEBUG:boto:Data: 2024-04-16T18:18:11.590 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.590 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.590 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.590 DEBUG:boto:Params: {} 2024-04-16T18:18:11.591 DEBUG:boto:Token: None 2024-04-16T18:18:11.591 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.591 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yirXxU0lyXeUFKlOWgTzFx7KIFg= 2024-04-16T18:18:11.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yirXxU0lyXeUFKlOWgTzFx7KIFg=', 'Content-Length': '0'} 2024-04-16T18:18:11.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebdb8f6f07e10726-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.594 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.594 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.594 DEBUG:boto:path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.594 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.594 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.594 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.594 DEBUG:boto:Data: 2024-04-16T18:18:11.595 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.595 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.595 DEBUG:boto:Params: {} 2024-04-16T18:18:11.595 DEBUG:boto:Token: None 2024-04-16T18:18:11.595 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yirXxU0lyXeUFKlOWgTzFx7KIFg= 2024-04-16T18:18:11.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yirXxU0lyXeUFKlOWgTzFx7KIFg=', 'Content-Length': '0'} 2024-04-16T18:18:11.598 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dcce46d561f668d-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.598 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000004 2024-04-16T18:18:11.598 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.598 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.599 DEBUG:boto:path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.599 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.599 DEBUG:boto:Method: GET 2024-04-16T18:18:11.599 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.599 DEBUG:boto:Data: 2024-04-16T18:18:11.599 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.599 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.599 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.599 DEBUG:boto:Params: {} 2024-04-16T18:18:11.599 DEBUG:boto:Token: None 2024-04-16T18:18:11.599 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA= 2024-04-16T18:18:11.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA=', 'Content-Length': '0'} 2024-04-16T18:18:11.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd352ce8e75e0c36-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.646 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.646 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.647 DEBUG:boto:path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.647 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.647 DEBUG:boto:Method: GET 2024-04-16T18:18:11.647 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.647 DEBUG:boto:Data: 2024-04-16T18:18:11.647 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.647 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.647 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.647 DEBUG:boto:Params: {} 2024-04-16T18:18:11.647 DEBUG:boto:Token: None 2024-04-16T18:18:11.648 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004?versionId=null 2024-04-16T18:18:11.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA= 2024-04-16T18:18:11.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uHrJgz5gN1hjiqXKmLImS9mHycA=', 'Content-Length': '0'} 2024-04-16T18:18:11.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000795e2efbc7464d94-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.651 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.651 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.651 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.651 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.651 DEBUG:boto:Method: GET 2024-04-16T18:18:11.651 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.651 DEBUG:boto:Data: 2024-04-16T18:18:11.652 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.652 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.652 DEBUG:boto:Params: {} 2024-04-16T18:18:11.652 DEBUG:boto:Token: None 2024-04-16T18:18:11.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.657 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075c8a496b83a99b5-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.698 DEBUG:boto:b'lybtug-51obj000000000000000000041000falseobj000000000000000000042024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.699 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.699 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.699 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.699 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.699 DEBUG:boto:Method: GET 2024-04-16T18:18:11.699 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.699 DEBUG:boto:Data: 2024-04-16T18:18:11.699 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.699 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.699 DEBUG:boto:Params: {} 2024-04-16T18:18:11.700 DEBUG:boto:Token: None 2024-04-16T18:18:11.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.706 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e3be14e619bb372-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.706 DEBUG:boto:b'lybtug-51obj000000000000000000041000falseobj000000000000000000042024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.706 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.706 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.706 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.707 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.707 DEBUG:boto:Data: 2024-04-16T18:18:11.707 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.707 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.707 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.707 DEBUG:boto:Params: {} 2024-04-16T18:18:11.707 DEBUG:boto:Token: None 2024-04-16T18:18:11.707 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.707 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rULAUqmFxLCweyjAb4RWKREPD9Q= 2024-04-16T18:18:11.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rULAUqmFxLCweyjAb4RWKREPD9Q=', 'Content-Length': '0'} 2024-04-16T18:18:11.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000484451c35997bc16-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.711 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.711 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.711 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.711 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.711 DEBUG:boto:Data: 2024-04-16T18:18:11.711 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.711 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.711 DEBUG:boto:Params: {} 2024-04-16T18:18:11.711 DEBUG:boto:Token: None 2024-04-16T18:18:11.712 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rULAUqmFxLCweyjAb4RWKREPD9Q= 2024-04-16T18:18:11.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rULAUqmFxLCweyjAb4RWKREPD9Q=', 'Content-Length': '0'} 2024-04-16T18:18:11.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051243faa8110009c-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000004 2024-04-16T18:18:11.715 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.715 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.715 DEBUG:boto:Method: GET 2024-04-16T18:18:11.715 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.715 DEBUG:boto:Data: 2024-04-16T18:18:11.716 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.716 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.716 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.716 DEBUG:boto:Params: {} 2024-04-16T18:18:11.716 DEBUG:boto:Token: None 2024-04-16T18:18:11.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MT/lyFDr9ZnszsRHz55l9g/SSvQ= 2024-04-16T18:18:11.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MT/lyFDr9ZnszsRHz55l9g/SSvQ=', 'Content-Length': '0'} 2024-04-16T18:18:11.719 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007907c48675d166c4-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.762 DEBUG:boto:path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.763 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.763 DEBUG:boto:Method: GET 2024-04-16T18:18:11.763 DEBUG:boto:Path: /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.763 DEBUG:boto:Data: 2024-04-16T18:18:11.763 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.763 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.763 DEBUG:boto:Params: {} 2024-04-16T18:18:11.763 DEBUG:boto:Token: None 2024-04-16T18:18:11.763 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000004 2024-04-16T18:18:11.763 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MT/lyFDr9ZnszsRHz55l9g/SSvQ= 2024-04-16T18:18:11.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MT/lyFDr9ZnszsRHz55l9g/SSvQ=', 'Content-Length': '0'} 2024-04-16T18:18:11.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cd37ab4cceddf9a-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.767 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.767 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.767 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.767 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.767 DEBUG:boto:Method: GET 2024-04-16T18:18:11.767 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.767 DEBUG:boto:Data: 2024-04-16T18:18:11.767 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.767 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.768 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.768 DEBUG:boto:Params: {} 2024-04-16T18:18:11.768 DEBUG:boto:Token: None 2024-04-16T18:18:11.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd101d230594b7f2-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.814 DEBUG:boto:b'lybtug-51obj000000000000000000041000falseobj000000000000000000042024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.815 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.815 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.815 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.815 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.815 DEBUG:boto:Method: GET 2024-04-16T18:18:11.815 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000004 2024-04-16T18:18:11.815 DEBUG:boto:Data: 2024-04-16T18:18:11.815 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.815 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.815 DEBUG:boto:Params: {} 2024-04-16T18:18:11.815 DEBUG:boto:Token: None 2024-04-16T18:18:11.816 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b216c9a2a3c73455-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.822 DEBUG:boto:b'lybtug-51obj000000000000000000041000falseobj000000000000000000042024-04-16T18:18:01.044Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.823 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000005 2024-04-16T18:18:11.823 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.823 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.823 DEBUG:boto:Method: GET 2024-04-16T18:18:11.823 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.823 DEBUG:boto:Data: 2024-04-16T18:18:11.823 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.823 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.824 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.824 DEBUG:boto:Params: {} 2024-04-16T18:18:11.824 DEBUG:boto:Token: None 2024-04-16T18:18:11.824 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4= 2024-04-16T18:18:11.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4=', 'Content-Length': '0'} 2024-04-16T18:18:11.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bf0fc8f64e8db7a-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.870 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.870 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.871 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.871 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.871 DEBUG:boto:Method: GET 2024-04-16T18:18:11.871 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.871 DEBUG:boto:Data: 2024-04-16T18:18:11.871 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.871 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.871 DEBUG:boto:Params: {} 2024-04-16T18:18:11.871 DEBUG:boto:Token: None 2024-04-16T18:18:11.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4= 2024-04-16T18:18:11.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4=', 'Content-Length': '0'} 2024-04-16T18:18:11.875 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdf8b931d0ad8237-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.875 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.875 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.876 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.876 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.876 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.876 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.876 DEBUG:boto:Data: 2024-04-16T18:18:11.876 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.876 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.876 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.876 DEBUG:boto:Params: {} 2024-04-16T18:18:11.876 DEBUG:boto:Token: None 2024-04-16T18:18:11.877 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.877 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ke7gcehTbFEHhFIFoADBW1zjW5A= 2024-04-16T18:18:11.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ke7gcehTbFEHhFIFoADBW1zjW5A=', 'Content-Length': '0'} 2024-04-16T18:18:11.880 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077f45d6b3872c6dc-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.880 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.880 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.880 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.880 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.880 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.880 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.881 DEBUG:boto:Data: 2024-04-16T18:18:11.881 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.881 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.881 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.881 DEBUG:boto:Params: {} 2024-04-16T18:18:11.881 DEBUG:boto:Token: None 2024-04-16T18:18:11.881 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.881 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ke7gcehTbFEHhFIFoADBW1zjW5A= 2024-04-16T18:18:11.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ke7gcehTbFEHhFIFoADBW1zjW5A=', 'Content-Length': '0'} 2024-04-16T18:18:11.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032e4506f0b28c495-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.885 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000005 2024-04-16T18:18:11.885 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.885 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.885 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.885 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.885 DEBUG:boto:Method: GET 2024-04-16T18:18:11.885 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.885 DEBUG:boto:Data: 2024-04-16T18:18:11.885 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.885 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.886 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.886 DEBUG:boto:Params: {} 2024-04-16T18:18:11.886 DEBUG:boto:Token: None 2024-04-16T18:18:11.886 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4= 2024-04-16T18:18:11.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4=', 'Content-Length': '0'} 2024-04-16T18:18:11.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005078e0d91e0162dd-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.930 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.931 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.931 DEBUG:boto:path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.931 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.931 DEBUG:boto:Method: GET 2024-04-16T18:18:11.931 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.931 DEBUG:boto:Data: 2024-04-16T18:18:11.931 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.931 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.931 DEBUG:boto:Params: {} 2024-04-16T18:18:11.932 DEBUG:boto:Token: None 2024-04-16T18:18:11.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005?versionId=null 2024-04-16T18:18:11.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4= 2024-04-16T18:18:11.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OK9nlpBFrxBUoDqEUMvT0iKhOD4=', 'Content-Length': '0'} 2024-04-16T18:18:11.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7fcead29b4968da-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.935 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.935 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.936 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:11.936 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:11.936 DEBUG:boto:Method: GET 2024-04-16T18:18:11.936 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:11.936 DEBUG:boto:Data: 2024-04-16T18:18:11.936 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.936 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.936 DEBUG:boto:Params: {} 2024-04-16T18:18:11.936 DEBUG:boto:Token: None 2024-04-16T18:18:11.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.941 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1c51a557f76c4df-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.941 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.941 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:11.941 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:11.942 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:11.942 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:11.942 DEBUG:boto:Method: GET 2024-04-16T18:18:11.942 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:11.942 DEBUG:boto:Data: 2024-04-16T18:18:11.942 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.942 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.942 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.942 DEBUG:boto:Params: {} 2024-04-16T18:18:11.942 DEBUG:boto:Token: None 2024-04-16T18:18:11.943 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/ 2024-04-16T18:18:11.943 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w= 2024-04-16T18:18:11.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THevJotlROUgE8YMOXZjWAFdT2w=', 'Content-Length': '0'} 2024-04-16T18:18:11.948 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff2e63c7007b8468-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.948 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:11.949 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.949 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.949 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.949 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.949 DEBUG:boto:Data: 2024-04-16T18:18:11.949 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.949 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.949 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.949 DEBUG:boto:Params: {} 2024-04-16T18:18:11.950 DEBUG:boto:Token: None 2024-04-16T18:18:11.950 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.950 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DnOlZA1k/WiXiteLMulKbyJ8iX4= 2024-04-16T18:18:11.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DnOlZA1k/WiXiteLMulKbyJ8iX4=', 'Content-Length': '0'} 2024-04-16T18:18:11.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010f32180d06d0553-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.954 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.954 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.954 DEBUG:boto:Method: HEAD 2024-04-16T18:18:11.954 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.954 DEBUG:boto:Data: 2024-04-16T18:18:11.954 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.954 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:11.954 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.954 DEBUG:boto:Params: {} 2024-04-16T18:18:11.954 DEBUG:boto:Token: None 2024-04-16T18:18:11.954 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.955 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DnOlZA1k/WiXiteLMulKbyJ8iX4= 2024-04-16T18:18:11.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DnOlZA1k/WiXiteLMulKbyJ8iX4=', 'Content-Length': '0'} 2024-04-16T18:18:11.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079979f63bc6603df-00661ec0e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:11.958 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000005 2024-04-16T18:18:11.958 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.958 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.958 DEBUG:boto:Method: GET 2024-04-16T18:18:11.958 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.958 DEBUG:boto:Data: 2024-04-16T18:18:11.958 DEBUG:boto:Headers: {} 2024-04-16T18:18:11.958 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:11.958 DEBUG:boto:Port: 8000 2024-04-16T18:18:11.959 DEBUG:boto:Params: {} 2024-04-16T18:18:11.959 DEBUG:boto:Token: None 2024-04-16T18:18:11.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:11 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:18:11.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xzWxC7MKCgFV+qLT706mChJWiTQ= 2024-04-16T18:18:11.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xzWxC7MKCgFV+qLT706mChJWiTQ=', 'Content-Length': '0'} 2024-04-16T18:18:11.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc5540e290dbb8fd-00661ec0e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.006 DEBUG:boto:path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:12.006 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000005 2024-04-16T18:18:12.007 DEBUG:boto:Method: GET 2024-04-16T18:18:12.007 DEBUG:boto:Path: /lybtug-51/obj00000000000000000005 2024-04-16T18:18:12.007 DEBUG:boto:Data: 2024-04-16T18:18:12.007 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.007 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.007 DEBUG:boto:Params: {} 2024-04-16T18:18:12.007 DEBUG:boto:Token: None 2024-04-16T18:18:12.007 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000005 2024-04-16T18:18:12.007 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JGzSHkZTAuaDU5vaePDxplUXxKs= 2024-04-16T18:18:12.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JGzSHkZTAuaDU5vaePDxplUXxKs=', 'Content-Length': '0'} 2024-04-16T18:18:12.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000491c148e80a3eca-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.011 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.011 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.011 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:12.011 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:12.011 DEBUG:boto:Method: GET 2024-04-16T18:18:12.011 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:12.011 DEBUG:boto:Data: 2024-04-16T18:18:12.011 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.011 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.012 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.012 DEBUG:boto:Params: {} 2024-04-16T18:18:12.012 DEBUG:boto:Token: None 2024-04-16T18:18:12.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fd3512bd5a16a33-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.058 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.059 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.059 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.059 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:12.059 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:12.059 DEBUG:boto:Method: GET 2024-04-16T18:18:12.059 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000005 2024-04-16T18:18:12.059 DEBUG:boto:Data: 2024-04-16T18:18:12.059 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.059 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.059 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.060 DEBUG:boto:Params: {} 2024-04-16T18:18:12.060 DEBUG:boto:Token: None 2024-04-16T18:18:12.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.066 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ffe4df8e16b64a1-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.066 DEBUG:boto:b'lybtug-51obj000000000000000000051000falseobj000000000000000000052024-04-16T18:17:59.359Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.066 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000006 2024-04-16T18:18:12.066 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.067 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.067 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.067 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.067 DEBUG:boto:Method: GET 2024-04-16T18:18:12.067 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.067 DEBUG:boto:Data: 2024-04-16T18:18:12.067 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.067 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.067 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.067 DEBUG:boto:Params: {} 2024-04-16T18:18:12.067 DEBUG:boto:Token: None 2024-04-16T18:18:12.067 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA= 2024-04-16T18:18:12.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA=', 'Content-Length': '0'} 2024-04-16T18:18:12.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007643ceabe25d8345-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.114 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.114 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.114 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.115 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.115 DEBUG:boto:Method: GET 2024-04-16T18:18:12.115 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.115 DEBUG:boto:Data: 2024-04-16T18:18:12.115 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.115 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.115 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.115 DEBUG:boto:Params: {} 2024-04-16T18:18:12.115 DEBUG:boto:Token: None 2024-04-16T18:18:12.115 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA= 2024-04-16T18:18:12.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA=', 'Content-Length': '0'} 2024-04-16T18:18:12.118 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013fafa2ab490a793-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.119 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.119 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.119 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.119 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.119 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.119 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.119 DEBUG:boto:Data: 2024-04-16T18:18:12.119 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.120 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.120 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.120 DEBUG:boto:Params: {} 2024-04-16T18:18:12.120 DEBUG:boto:Token: None 2024-04-16T18:18:12.120 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kxbNBD/ZJMrGF63eGP8FvC9ZOBc= 2024-04-16T18:18:12.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kxbNBD/ZJMrGF63eGP8FvC9ZOBc=', 'Content-Length': '0'} 2024-04-16T18:18:12.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082d155096f088153-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.123 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.124 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.124 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.124 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.124 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.124 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.124 DEBUG:boto:Data: 2024-04-16T18:18:12.124 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.124 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.124 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.124 DEBUG:boto:Params: {} 2024-04-16T18:18:12.124 DEBUG:boto:Token: None 2024-04-16T18:18:12.125 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kxbNBD/ZJMrGF63eGP8FvC9ZOBc= 2024-04-16T18:18:12.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kxbNBD/ZJMrGF63eGP8FvC9ZOBc=', 'Content-Length': '0'} 2024-04-16T18:18:12.128 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000659912bb32d6cb82-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.128 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000006 2024-04-16T18:18:12.128 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.128 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.128 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.128 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.128 DEBUG:boto:Method: GET 2024-04-16T18:18:12.129 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.129 DEBUG:boto:Data: 2024-04-16T18:18:12.129 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.129 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.129 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.129 DEBUG:boto:Params: {} 2024-04-16T18:18:12.129 DEBUG:boto:Token: None 2024-04-16T18:18:12.129 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.129 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA= 2024-04-16T18:18:12.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA=', 'Content-Length': '0'} 2024-04-16T18:18:12.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000282873304ea71f7-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.178 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.178 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.179 DEBUG:boto:path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.179 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.179 DEBUG:boto:Method: GET 2024-04-16T18:18:12.179 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.179 DEBUG:boto:Data: 2024-04-16T18:18:12.179 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.179 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.179 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.179 DEBUG:boto:Params: {} 2024-04-16T18:18:12.179 DEBUG:boto:Token: None 2024-04-16T18:18:12.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006?versionId=null 2024-04-16T18:18:12.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA= 2024-04-16T18:18:12.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YCXpQqtX1mYQeRtNc1W0BBrtZgA=', 'Content-Length': '0'} 2024-04-16T18:18:12.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9fca2b10a4b6cb8-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.183 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.183 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.183 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.183 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.183 DEBUG:boto:Method: GET 2024-04-16T18:18:12.184 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.184 DEBUG:boto:Data: 2024-04-16T18:18:12.184 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.184 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.184 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.184 DEBUG:boto:Params: {} 2024-04-16T18:18:12.184 DEBUG:boto:Token: None 2024-04-16T18:18:12.184 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c4f961488cf96c4-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.189 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.189 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.189 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.189 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.189 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.189 DEBUG:boto:Method: GET 2024-04-16T18:18:12.189 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.189 DEBUG:boto:Data: 2024-04-16T18:18:12.189 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.189 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.190 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.190 DEBUG:boto:Params: {} 2024-04-16T18:18:12.190 DEBUG:boto:Token: None 2024-04-16T18:18:12.190 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.190 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df935d8a897aa100-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.196 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.197 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.197 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.197 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.197 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.197 DEBUG:boto:Data: 2024-04-16T18:18:12.197 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.197 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.197 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.197 DEBUG:boto:Params: {} 2024-04-16T18:18:12.197 DEBUG:boto:Token: None 2024-04-16T18:18:12.198 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.198 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a7RpTsXt4h88SKt1vzGo5/TIm3w= 2024-04-16T18:18:12.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a7RpTsXt4h88SKt1vzGo5/TIm3w=', 'Content-Length': '0'} 2024-04-16T18:18:12.201 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077e6c26a3e327584-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.201 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.201 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.201 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.201 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.201 DEBUG:boto:Data: 2024-04-16T18:18:12.201 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.201 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.201 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.201 DEBUG:boto:Params: {} 2024-04-16T18:18:12.202 DEBUG:boto:Token: None 2024-04-16T18:18:12.202 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.202 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a7RpTsXt4h88SKt1vzGo5/TIm3w= 2024-04-16T18:18:12.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a7RpTsXt4h88SKt1vzGo5/TIm3w=', 'Content-Length': '0'} 2024-04-16T18:18:12.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000027e11863b68e6f0-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.205 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000006 2024-04-16T18:18:12.205 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.205 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.205 DEBUG:boto:Method: GET 2024-04-16T18:18:12.205 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.205 DEBUG:boto:Data: 2024-04-16T18:18:12.205 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.205 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.206 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.206 DEBUG:boto:Params: {} 2024-04-16T18:18:12.206 DEBUG:boto:Token: None 2024-04-16T18:18:12.206 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.206 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QbbiJY9VJBDMZdgMBxe7xR+RSBQ= 2024-04-16T18:18:12.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QbbiJY9VJBDMZdgMBxe7xR+RSBQ=', 'Content-Length': '0'} 2024-04-16T18:18:12.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096d5fa2418dd20aa-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.250 DEBUG:boto:path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.251 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.251 DEBUG:boto:Method: GET 2024-04-16T18:18:12.251 DEBUG:boto:Path: /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.251 DEBUG:boto:Data: 2024-04-16T18:18:12.251 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.251 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.251 DEBUG:boto:Params: {} 2024-04-16T18:18:12.251 DEBUG:boto:Token: None 2024-04-16T18:18:12.251 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000006 2024-04-16T18:18:12.251 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QbbiJY9VJBDMZdgMBxe7xR+RSBQ= 2024-04-16T18:18:12.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QbbiJY9VJBDMZdgMBxe7xR+RSBQ=', 'Content-Length': '0'} 2024-04-16T18:18:12.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d78c2ef70a9165fb-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.255 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.255 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.255 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.255 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.255 DEBUG:boto:Method: GET 2024-04-16T18:18:12.255 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.255 DEBUG:boto:Data: 2024-04-16T18:18:12.255 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.255 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.255 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.255 DEBUG:boto:Params: {} 2024-04-16T18:18:12.256 DEBUG:boto:Token: None 2024-04-16T18:18:12.256 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0feddf8f54505a8-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.302 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.303 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.303 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.303 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.303 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.303 DEBUG:boto:Method: GET 2024-04-16T18:18:12.303 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000006 2024-04-16T18:18:12.303 DEBUG:boto:Data: 2024-04-16T18:18:12.303 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.303 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.303 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.304 DEBUG:boto:Params: {} 2024-04-16T18:18:12.304 DEBUG:boto:Token: None 2024-04-16T18:18:12.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035c6fb3ade148a83-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.310 DEBUG:boto:b'lybtug-51obj000000000000000000061000falseobj000000000000000000062024-04-16T18:18:00.972Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.311 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000007 2024-04-16T18:18:12.311 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.311 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.311 DEBUG:boto:path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.311 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.311 DEBUG:boto:Method: GET 2024-04-16T18:18:12.311 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.311 DEBUG:boto:Data: 2024-04-16T18:18:12.311 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.311 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.312 DEBUG:boto:Params: {} 2024-04-16T18:18:12.312 DEBUG:boto:Token: None 2024-04-16T18:18:12.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo= 2024-04-16T18:18:12.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo=', 'Content-Length': '0'} 2024-04-16T18:18:12.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cf1ab0568619c34-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.358 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.358 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.359 DEBUG:boto:path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.359 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.359 DEBUG:boto:Method: GET 2024-04-16T18:18:12.359 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.359 DEBUG:boto:Data: 2024-04-16T18:18:12.359 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.359 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.359 DEBUG:boto:Params: {} 2024-04-16T18:18:12.359 DEBUG:boto:Token: None 2024-04-16T18:18:12.359 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo= 2024-04-16T18:18:12.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo=', 'Content-Length': '0'} 2024-04-16T18:18:12.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa15b5a5e63e46a4-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.364 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.364 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.364 DEBUG:boto:path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.364 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.364 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.364 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.364 DEBUG:boto:Data: 2024-04-16T18:18:12.364 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.364 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.364 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.364 DEBUG:boto:Params: {} 2024-04-16T18:18:12.365 DEBUG:boto:Token: None 2024-04-16T18:18:12.365 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.365 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CSf6ZvAfKiC99Jdd6iI6He8FiK8= 2024-04-16T18:18:12.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CSf6ZvAfKiC99Jdd6iI6He8FiK8=', 'Content-Length': '0'} 2024-04-16T18:18:12.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007518c49a85c5a483-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.368 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.368 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.368 DEBUG:boto:path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.368 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.368 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.368 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.369 DEBUG:boto:Data: 2024-04-16T18:18:12.369 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.369 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.369 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.369 DEBUG:boto:Params: {} 2024-04-16T18:18:12.369 DEBUG:boto:Token: None 2024-04-16T18:18:12.369 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CSf6ZvAfKiC99Jdd6iI6He8FiK8= 2024-04-16T18:18:12.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CSf6ZvAfKiC99Jdd6iI6He8FiK8=', 'Content-Length': '0'} 2024-04-16T18:18:12.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e459944dd8cc4343-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000007 2024-04-16T18:18:12.372 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.373 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.373 DEBUG:boto:path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.373 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.373 DEBUG:boto:Method: GET 2024-04-16T18:18:12.373 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.373 DEBUG:boto:Data: 2024-04-16T18:18:12.373 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.373 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.373 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.373 DEBUG:boto:Params: {} 2024-04-16T18:18:12.373 DEBUG:boto:Token: None 2024-04-16T18:18:12.373 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.374 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo= 2024-04-16T18:18:12.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo=', 'Content-Length': '0'} 2024-04-16T18:18:12.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3dbdb0bdea6e916-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.418 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.418 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.419 DEBUG:boto:path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.419 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.419 DEBUG:boto:Method: GET 2024-04-16T18:18:12.419 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.419 DEBUG:boto:Data: 2024-04-16T18:18:12.419 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.419 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.419 DEBUG:boto:Params: {} 2024-04-16T18:18:12.419 DEBUG:boto:Token: None 2024-04-16T18:18:12.419 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007?versionId=null 2024-04-16T18:18:12.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo= 2024-04-16T18:18:12.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6QPGfS+qgkB+dFhnU9fc+xe7mgo=', 'Content-Length': '0'} 2024-04-16T18:18:12.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b09d604595e955f-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.423 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.423 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.423 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.423 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.423 DEBUG:boto:Method: GET 2024-04-16T18:18:12.423 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.423 DEBUG:boto:Data: 2024-04-16T18:18:12.423 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.423 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.424 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.424 DEBUG:boto:Params: {} 2024-04-16T18:18:12.424 DEBUG:boto:Token: None 2024-04-16T18:18:12.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.428 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f2b51976d46e062-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.429 DEBUG:boto:b'lybtug-51obj000000000000000000071000falseobj000000000000000000072024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.429 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.429 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.429 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.429 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.429 DEBUG:boto:Method: GET 2024-04-16T18:18:12.429 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.429 DEBUG:boto:Data: 2024-04-16T18:18:12.430 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.430 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.430 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.430 DEBUG:boto:Params: {} 2024-04-16T18:18:12.430 DEBUG:boto:Token: None 2024-04-16T18:18:12.430 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8b3063efff5dc72-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.437 DEBUG:boto:b'lybtug-51obj000000000000000000071000falseobj000000000000000000072024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.437 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.437 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.437 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.437 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.437 DEBUG:boto:Data: 2024-04-16T18:18:12.438 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.438 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.438 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.438 DEBUG:boto:Params: {} 2024-04-16T18:18:12.438 DEBUG:boto:Token: None 2024-04-16T18:18:12.438 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.438 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:exVSR2uthOs6ppJ1gvkyeTmMPGE= 2024-04-16T18:18:12.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:exVSR2uthOs6ppJ1gvkyeTmMPGE=', 'Content-Length': '0'} 2024-04-16T18:18:12.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4166b26a7530640-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.441 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.441 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.442 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.442 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.442 DEBUG:boto:Data: 2024-04-16T18:18:12.442 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.442 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.442 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.442 DEBUG:boto:Params: {} 2024-04-16T18:18:12.442 DEBUG:boto:Token: None 2024-04-16T18:18:12.442 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:exVSR2uthOs6ppJ1gvkyeTmMPGE= 2024-04-16T18:18:12.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:exVSR2uthOs6ppJ1gvkyeTmMPGE=', 'Content-Length': '0'} 2024-04-16T18:18:12.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f6193b080c8645b-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.446 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000007 2024-04-16T18:18:12.446 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.446 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.446 DEBUG:boto:Method: GET 2024-04-16T18:18:12.446 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.446 DEBUG:boto:Data: 2024-04-16T18:18:12.446 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.446 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.446 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.446 DEBUG:boto:Params: {} 2024-04-16T18:18:12.447 DEBUG:boto:Token: None 2024-04-16T18:18:12.447 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.447 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xfol6c6CRODiHeO+dhNl0r8lXI8= 2024-04-16T18:18:12.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xfol6c6CRODiHeO+dhNl0r8lXI8=', 'Content-Length': '0'} 2024-04-16T18:18:12.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e55e69559d3da8a-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.490 DEBUG:boto:path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.490 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.491 DEBUG:boto:Method: GET 2024-04-16T18:18:12.491 DEBUG:boto:Path: /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.491 DEBUG:boto:Data: 2024-04-16T18:18:12.491 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.491 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.491 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.491 DEBUG:boto:Params: {} 2024-04-16T18:18:12.491 DEBUG:boto:Token: None 2024-04-16T18:18:12.491 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000007 2024-04-16T18:18:12.491 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xfol6c6CRODiHeO+dhNl0r8lXI8= 2024-04-16T18:18:12.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xfol6c6CRODiHeO+dhNl0r8lXI8=', 'Content-Length': '0'} 2024-04-16T18:18:12.494 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ea8a4d0e9186193-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.494 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.495 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.495 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.495 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.495 DEBUG:boto:Method: GET 2024-04-16T18:18:12.495 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.495 DEBUG:boto:Data: 2024-04-16T18:18:12.495 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.495 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.495 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.496 DEBUG:boto:Params: {} 2024-04-16T18:18:12.496 DEBUG:boto:Token: None 2024-04-16T18:18:12.496 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.500 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d1824f2c3e366bc-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.542 DEBUG:boto:b'lybtug-51obj000000000000000000071000falseobj000000000000000000072024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.543 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.543 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.543 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.543 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.543 DEBUG:boto:Method: GET 2024-04-16T18:18:12.543 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000007 2024-04-16T18:18:12.543 DEBUG:boto:Data: 2024-04-16T18:18:12.543 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.543 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.543 DEBUG:boto:Params: {} 2024-04-16T18:18:12.544 DEBUG:boto:Token: None 2024-04-16T18:18:12.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.550 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ec095c55051b98f-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.550 DEBUG:boto:b'lybtug-51obj000000000000000000071000falseobj000000000000000000072024-04-16T18:18:01.270Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.550 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000008 2024-04-16T18:18:12.551 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.551 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.551 DEBUG:boto:path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.551 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.551 DEBUG:boto:Method: GET 2024-04-16T18:18:12.551 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.551 DEBUG:boto:Data: 2024-04-16T18:18:12.551 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.551 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.551 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.551 DEBUG:boto:Params: {} 2024-04-16T18:18:12.551 DEBUG:boto:Token: None 2024-04-16T18:18:12.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI= 2024-04-16T18:18:12.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI=', 'Content-Length': '0'} 2024-04-16T18:18:12.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d7865bda0e00ffd-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.598 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.598 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.599 DEBUG:boto:path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.599 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.599 DEBUG:boto:Method: GET 2024-04-16T18:18:12.599 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.599 DEBUG:boto:Data: 2024-04-16T18:18:12.599 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.599 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.599 DEBUG:boto:Params: {} 2024-04-16T18:18:12.599 DEBUG:boto:Token: None 2024-04-16T18:18:12.599 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI= 2024-04-16T18:18:12.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI=', 'Content-Length': '0'} 2024-04-16T18:18:12.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f84ef97930811a75-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.603 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.603 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.603 DEBUG:boto:path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.604 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.604 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.604 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.604 DEBUG:boto:Data: 2024-04-16T18:18:12.604 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.604 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.604 DEBUG:boto:Params: {} 2024-04-16T18:18:12.604 DEBUG:boto:Token: None 2024-04-16T18:18:12.604 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CKKZB0Lhv5ucgCr7rZxImaZ4jrM= 2024-04-16T18:18:12.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CKKZB0Lhv5ucgCr7rZxImaZ4jrM=', 'Content-Length': '0'} 2024-04-16T18:18:12.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000228da31e776a690e-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.608 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.608 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.608 DEBUG:boto:path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.608 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.608 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.609 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.609 DEBUG:boto:Data: 2024-04-16T18:18:12.609 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.609 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.609 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.609 DEBUG:boto:Params: {} 2024-04-16T18:18:12.609 DEBUG:boto:Token: None 2024-04-16T18:18:12.609 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.609 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CKKZB0Lhv5ucgCr7rZxImaZ4jrM= 2024-04-16T18:18:12.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CKKZB0Lhv5ucgCr7rZxImaZ4jrM=', 'Content-Length': '0'} 2024-04-16T18:18:12.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab154820d8127937-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.612 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000008 2024-04-16T18:18:12.613 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.613 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.613 DEBUG:boto:path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.613 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.613 DEBUG:boto:Method: GET 2024-04-16T18:18:12.613 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.613 DEBUG:boto:Data: 2024-04-16T18:18:12.613 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.613 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.613 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.613 DEBUG:boto:Params: {} 2024-04-16T18:18:12.614 DEBUG:boto:Token: None 2024-04-16T18:18:12.614 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.614 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI= 2024-04-16T18:18:12.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI=', 'Content-Length': '0'} 2024-04-16T18:18:12.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008870e37ab50bc867-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.658 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.658 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.658 DEBUG:boto:path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.659 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.659 DEBUG:boto:Method: GET 2024-04-16T18:18:12.659 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.659 DEBUG:boto:Data: 2024-04-16T18:18:12.659 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.659 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.659 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.659 DEBUG:boto:Params: {} 2024-04-16T18:18:12.659 DEBUG:boto:Token: None 2024-04-16T18:18:12.659 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008?versionId=null 2024-04-16T18:18:12.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI= 2024-04-16T18:18:12.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9+fAc0n/KGwQ9cH6M9gDx2+YeI=', 'Content-Length': '0'} 2024-04-16T18:18:12.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007156fdb646fb76c1-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.663 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.663 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.663 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.663 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.663 DEBUG:boto:Method: GET 2024-04-16T18:18:12.663 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.663 DEBUG:boto:Data: 2024-04-16T18:18:12.663 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.664 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.664 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.664 DEBUG:boto:Params: {} 2024-04-16T18:18:12.664 DEBUG:boto:Token: None 2024-04-16T18:18:12.664 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.668 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f28706d9339dff1-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.668 DEBUG:boto:b'lybtug-51obj000000000000000000081000falseobj000000000000000000082024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.669 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.669 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.669 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.669 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.669 DEBUG:boto:Method: GET 2024-04-16T18:18:12.669 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.669 DEBUG:boto:Data: 2024-04-16T18:18:12.669 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.669 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.670 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.670 DEBUG:boto:Params: {} 2024-04-16T18:18:12.670 DEBUG:boto:Token: None 2024-04-16T18:18:12.670 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.670 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000342b6018c6b74b09-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.676 DEBUG:boto:b'lybtug-51obj000000000000000000081000falseobj000000000000000000082024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.676 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.676 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.677 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.677 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.677 DEBUG:boto:Data: 2024-04-16T18:18:12.677 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.677 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.677 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.677 DEBUG:boto:Params: {} 2024-04-16T18:18:12.677 DEBUG:boto:Token: None 2024-04-16T18:18:12.677 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.677 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pSYcH3qLn41P7oWmJYtBCfpqIKU= 2024-04-16T18:18:12.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pSYcH3qLn41P7oWmJYtBCfpqIKU=', 'Content-Length': '0'} 2024-04-16T18:18:12.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b4e87e0543be886-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.681 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.681 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.681 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.681 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.681 DEBUG:boto:Data: 2024-04-16T18:18:12.681 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.681 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.681 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.682 DEBUG:boto:Params: {} 2024-04-16T18:18:12.682 DEBUG:boto:Token: None 2024-04-16T18:18:12.682 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.682 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pSYcH3qLn41P7oWmJYtBCfpqIKU= 2024-04-16T18:18:12.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pSYcH3qLn41P7oWmJYtBCfpqIKU=', 'Content-Length': '0'} 2024-04-16T18:18:12.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd518872f2682037-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.685 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000008 2024-04-16T18:18:12.686 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.686 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.686 DEBUG:boto:Method: GET 2024-04-16T18:18:12.686 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.686 DEBUG:boto:Data: 2024-04-16T18:18:12.686 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.686 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.686 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.686 DEBUG:boto:Params: {} 2024-04-16T18:18:12.686 DEBUG:boto:Token: None 2024-04-16T18:18:12.687 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.687 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ymiVCdmUA5GgoVecDLqYNbcjk2o= 2024-04-16T18:18:12.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ymiVCdmUA5GgoVecDLqYNbcjk2o=', 'Content-Length': '0'} 2024-04-16T18:18:12.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031c47ed46fb41d45-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.730 DEBUG:boto:path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.731 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.731 DEBUG:boto:Method: GET 2024-04-16T18:18:12.731 DEBUG:boto:Path: /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.731 DEBUG:boto:Data: 2024-04-16T18:18:12.731 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.731 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.731 DEBUG:boto:Params: {} 2024-04-16T18:18:12.731 DEBUG:boto:Token: None 2024-04-16T18:18:12.731 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000008 2024-04-16T18:18:12.731 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ymiVCdmUA5GgoVecDLqYNbcjk2o= 2024-04-16T18:18:12.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ymiVCdmUA5GgoVecDLqYNbcjk2o=', 'Content-Length': '0'} 2024-04-16T18:18:12.735 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a46d4c6775b3c244-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.735 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.735 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.735 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.735 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.736 DEBUG:boto:Method: GET 2024-04-16T18:18:12.736 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.736 DEBUG:boto:Data: 2024-04-16T18:18:12.736 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.736 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.736 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.736 DEBUG:boto:Params: {} 2024-04-16T18:18:12.736 DEBUG:boto:Token: None 2024-04-16T18:18:12.736 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.741 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000384fbabc139aeb66-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.741 DEBUG:boto:b'lybtug-51obj000000000000000000081000falseobj000000000000000000082024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.742 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.742 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.742 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.742 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.742 DEBUG:boto:Method: GET 2024-04-16T18:18:12.742 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000008 2024-04-16T18:18:12.742 DEBUG:boto:Data: 2024-04-16T18:18:12.742 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.742 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.742 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.742 DEBUG:boto:Params: {} 2024-04-16T18:18:12.743 DEBUG:boto:Token: None 2024-04-16T18:18:12.743 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.743 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.748 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7e51f736755fbb4-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.749 DEBUG:boto:b'lybtug-51obj000000000000000000081000falseobj000000000000000000082024-04-16T18:18:00.522Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.749 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000009 2024-04-16T18:18:12.749 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.749 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.749 DEBUG:boto:path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.749 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.749 DEBUG:boto:Method: GET 2024-04-16T18:18:12.749 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.750 DEBUG:boto:Data: 2024-04-16T18:18:12.750 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.750 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.750 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.750 DEBUG:boto:Params: {} 2024-04-16T18:18:12.750 DEBUG:boto:Token: None 2024-04-16T18:18:12.750 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.750 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg= 2024-04-16T18:18:12.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg=', 'Content-Length': '0'} 2024-04-16T18:18:12.754 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069d142f42bc1eb50-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.794 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.795 DEBUG:boto:path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.795 DEBUG:boto:Method: GET 2024-04-16T18:18:12.795 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.795 DEBUG:boto:Data: 2024-04-16T18:18:12.795 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.795 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.795 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.795 DEBUG:boto:Params: {} 2024-04-16T18:18:12.795 DEBUG:boto:Token: None 2024-04-16T18:18:12.795 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg= 2024-04-16T18:18:12.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg=', 'Content-Length': '0'} 2024-04-16T18:18:12.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee5bdeff69aefafc-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.799 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.799 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.799 DEBUG:boto:path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.799 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.799 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.800 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.800 DEBUG:boto:Data: 2024-04-16T18:18:12.800 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.800 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.800 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.800 DEBUG:boto:Params: {} 2024-04-16T18:18:12.800 DEBUG:boto:Token: None 2024-04-16T18:18:12.800 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5yiwZRWvVwgkBc5TEc/W8w1IEek= 2024-04-16T18:18:12.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5yiwZRWvVwgkBc5TEc/W8w1IEek=', 'Content-Length': '0'} 2024-04-16T18:18:12.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088ccf445ec40c619-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.804 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.804 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.804 DEBUG:boto:path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.804 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.804 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.804 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.804 DEBUG:boto:Data: 2024-04-16T18:18:12.804 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.804 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.804 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.804 DEBUG:boto:Params: {} 2024-04-16T18:18:12.805 DEBUG:boto:Token: None 2024-04-16T18:18:12.805 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5yiwZRWvVwgkBc5TEc/W8w1IEek= 2024-04-16T18:18:12.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5yiwZRWvVwgkBc5TEc/W8w1IEek=', 'Content-Length': '0'} 2024-04-16T18:18:12.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1c3709cc52155da-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000009 2024-04-16T18:18:12.808 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.808 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.808 DEBUG:boto:path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.808 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.809 DEBUG:boto:Method: GET 2024-04-16T18:18:12.809 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.809 DEBUG:boto:Data: 2024-04-16T18:18:12.809 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.809 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.809 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.809 DEBUG:boto:Params: {} 2024-04-16T18:18:12.809 DEBUG:boto:Token: None 2024-04-16T18:18:12.809 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg= 2024-04-16T18:18:12.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg=', 'Content-Length': '0'} 2024-04-16T18:18:12.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4bccf87dcca627c-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.854 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.854 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.854 DEBUG:boto:path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.855 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.855 DEBUG:boto:Method: GET 2024-04-16T18:18:12.855 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.855 DEBUG:boto:Data: 2024-04-16T18:18:12.855 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.855 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.855 DEBUG:boto:Params: {} 2024-04-16T18:18:12.855 DEBUG:boto:Token: None 2024-04-16T18:18:12.855 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009?versionId=null 2024-04-16T18:18:12.855 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg= 2024-04-16T18:18:12.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2CwoG0po3oWUTvGGNhDhUENLPjg=', 'Content-Length': '0'} 2024-04-16T18:18:12.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9677618895a7e8a-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.859 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.859 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.859 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.859 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.859 DEBUG:boto:Method: GET 2024-04-16T18:18:12.859 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.859 DEBUG:boto:Data: 2024-04-16T18:18:12.859 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.860 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.860 DEBUG:boto:Params: {} 2024-04-16T18:18:12.860 DEBUG:boto:Token: None 2024-04-16T18:18:12.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fecab2ce601b32ad-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.865 DEBUG:boto:b'lybtug-51obj000000000000000000091000falseobj000000000000000000092024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.865 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.865 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.865 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.865 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.865 DEBUG:boto:Method: GET 2024-04-16T18:18:12.866 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.866 DEBUG:boto:Data: 2024-04-16T18:18:12.866 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.866 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.866 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.866 DEBUG:boto:Params: {} 2024-04-16T18:18:12.866 DEBUG:boto:Token: None 2024-04-16T18:18:12.866 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.866 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000210b70d3680b3b2e-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.914 DEBUG:boto:b'lybtug-51obj000000000000000000091000falseobj000000000000000000092024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:12.915 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.915 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.915 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.915 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.915 DEBUG:boto:Data: 2024-04-16T18:18:12.915 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.915 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.915 DEBUG:boto:Params: {} 2024-04-16T18:18:12.916 DEBUG:boto:Token: None 2024-04-16T18:18:12.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iTTt1Jh7TNezu2+JzpBuXaHi8Hs= 2024-04-16T18:18:12.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iTTt1Jh7TNezu2+JzpBuXaHi8Hs=', 'Content-Length': '0'} 2024-04-16T18:18:12.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baf3bc6dc3e38ec0-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.920 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.920 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.920 DEBUG:boto:Method: HEAD 2024-04-16T18:18:12.920 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.920 DEBUG:boto:Data: 2024-04-16T18:18:12.920 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.920 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.920 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.920 DEBUG:boto:Params: {} 2024-04-16T18:18:12.920 DEBUG:boto:Token: None 2024-04-16T18:18:12.921 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iTTt1Jh7TNezu2+JzpBuXaHi8Hs= 2024-04-16T18:18:12.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iTTt1Jh7TNezu2+JzpBuXaHi8Hs=', 'Content-Length': '0'} 2024-04-16T18:18:12.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046ceb446be017f1a-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.924 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000009 2024-04-16T18:18:12.924 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.924 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.924 DEBUG:boto:Method: GET 2024-04-16T18:18:12.925 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.925 DEBUG:boto:Data: 2024-04-16T18:18:12.925 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.925 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.925 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.925 DEBUG:boto:Params: {} 2024-04-16T18:18:12.925 DEBUG:boto:Token: None 2024-04-16T18:18:12.925 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.925 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jv7megrazegedyneJpICg+2ED5E= 2024-04-16T18:18:12.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jv7megrazegedyneJpICg+2ED5E=', 'Content-Length': '0'} 2024-04-16T18:18:12.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d26ce5ea0d249bf8-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.970 DEBUG:boto:path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.970 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.971 DEBUG:boto:Method: GET 2024-04-16T18:18:12.971 DEBUG:boto:Path: /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.971 DEBUG:boto:Data: 2024-04-16T18:18:12.971 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:12.971 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.971 DEBUG:boto:Params: {} 2024-04-16T18:18:12.971 DEBUG:boto:Token: None 2024-04-16T18:18:12.971 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/obj00000000000000000009 2024-04-16T18:18:12.971 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jv7megrazegedyneJpICg+2ED5E= 2024-04-16T18:18:12.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jv7megrazegedyneJpICg+2ED5E=', 'Content-Length': '0'} 2024-04-16T18:18:12.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1966d9e8f9732c9-00661ec0e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:12.975 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:12.975 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:12.975 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.975 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.975 DEBUG:boto:Method: GET 2024-04-16T18:18:12.975 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:12.975 DEBUG:boto:Data: 2024-04-16T18:18:12.975 DEBUG:boto:Headers: {} 2024-04-16T18:18:12.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:12.975 DEBUG:boto:Port: 8000 2024-04-16T18:18:12.975 DEBUG:boto:Params: {} 2024-04-16T18:18:12.976 DEBUG:boto:Token: None 2024-04-16T18:18:12.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:12 GMT /lybtug-51/ 2024-04-16T18:18:12.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk= 2024-04-16T18:18:12.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MzNCyc2mrEUa8e7Skic8cpl/qUk=', 'Content-Length': '0'} 2024-04-16T18:18:12.979 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000232dc3381620bd91-00661ec0e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.022 DEBUG:boto:b'lybtug-51obj000000000000000000091000falseobj000000000000000000092024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.023 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.023 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.023 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:13.023 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:13.023 DEBUG:boto:Method: GET 2024-04-16T18:18:13.023 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000009 2024-04-16T18:18:13.023 DEBUG:boto:Data: 2024-04-16T18:18:13.023 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.023 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.023 DEBUG:boto:Params: {} 2024-04-16T18:18:13.023 DEBUG:boto:Token: None 2024-04-16T18:18:13.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef208cbb4770681e-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.031 DEBUG:boto:b'lybtug-51obj000000000000000000091000falseobj000000000000000000092024-04-16T18:18:00.498Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.031 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000010 2024-04-16T18:18:13.031 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.031 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.032 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.032 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.032 DEBUG:boto:Method: GET 2024-04-16T18:18:13.032 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.032 DEBUG:boto:Data: 2024-04-16T18:18:13.032 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.032 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.032 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.032 DEBUG:boto:Params: {} 2024-04-16T18:18:13.032 DEBUG:boto:Token: None 2024-04-16T18:18:13.033 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.033 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI= 2024-04-16T18:18:13.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI=', 'Content-Length': '0'} 2024-04-16T18:18:13.036 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072868593713ed4da-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.078 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.078 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.078 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.079 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.079 DEBUG:boto:Method: GET 2024-04-16T18:18:13.079 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.079 DEBUG:boto:Data: 2024-04-16T18:18:13.079 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.079 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.079 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.079 DEBUG:boto:Params: {} 2024-04-16T18:18:13.079 DEBUG:boto:Token: None 2024-04-16T18:18:13.079 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.079 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI= 2024-04-16T18:18:13.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI=', 'Content-Length': '0'} 2024-04-16T18:18:13.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3ee01000d52b3df-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.083 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.083 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.083 DEBUG:boto:Data: 2024-04-16T18:18:13.083 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.083 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.084 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.084 DEBUG:boto:Params: {} 2024-04-16T18:18:13.084 DEBUG:boto:Token: None 2024-04-16T18:18:13.084 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Blcg6zz8RuKKLfRwC+g59g4EKIM= 2024-04-16T18:18:13.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Blcg6zz8RuKKLfRwC+g59g4EKIM=', 'Content-Length': '0'} 2024-04-16T18:18:13.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000317bfea6c127c754-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.087 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.087 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.087 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.088 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.088 DEBUG:boto:Data: 2024-04-16T18:18:13.088 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.088 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.088 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.088 DEBUG:boto:Params: {} 2024-04-16T18:18:13.088 DEBUG:boto:Token: None 2024-04-16T18:18:13.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Blcg6zz8RuKKLfRwC+g59g4EKIM= 2024-04-16T18:18:13.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Blcg6zz8RuKKLfRwC+g59g4EKIM=', 'Content-Length': '0'} 2024-04-16T18:18:13.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035df182a3bdb7657-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000010 2024-04-16T18:18:13.092 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.092 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.092 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.092 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.092 DEBUG:boto:Method: GET 2024-04-16T18:18:13.092 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.092 DEBUG:boto:Data: 2024-04-16T18:18:13.092 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.092 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.092 DEBUG:boto:Params: {} 2024-04-16T18:18:13.093 DEBUG:boto:Token: None 2024-04-16T18:18:13.093 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.093 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI= 2024-04-16T18:18:13.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI=', 'Content-Length': '0'} 2024-04-16T18:18:13.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dc1b7dc1b17168a-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.138 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.139 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.139 DEBUG:boto:path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.139 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.139 DEBUG:boto:Method: GET 2024-04-16T18:18:13.139 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.139 DEBUG:boto:Data: 2024-04-16T18:18:13.139 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.139 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.139 DEBUG:boto:Params: {} 2024-04-16T18:18:13.139 DEBUG:boto:Token: None 2024-04-16T18:18:13.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010?versionId=null 2024-04-16T18:18:13.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI= 2024-04-16T18:18:13.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4sIVxrTXkDyyRBGGSpxBJZQBvWI=', 'Content-Length': '0'} 2024-04-16T18:18:13.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000702e29f48cd1f97c-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.143 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.143 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.143 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.143 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.144 DEBUG:boto:Method: GET 2024-04-16T18:18:13.144 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.144 DEBUG:boto:Data: 2024-04-16T18:18:13.144 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.144 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.144 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.144 DEBUG:boto:Params: {} 2024-04-16T18:18:13.144 DEBUG:boto:Token: None 2024-04-16T18:18:13.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f5a352d8949e8a2d-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.149 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.149 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.149 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.149 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.149 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.149 DEBUG:boto:Method: GET 2024-04-16T18:18:13.149 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.150 DEBUG:boto:Data: 2024-04-16T18:18:13.150 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.150 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.150 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.150 DEBUG:boto:Params: {} 2024-04-16T18:18:13.150 DEBUG:boto:Token: None 2024-04-16T18:18:13.150 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.150 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.156 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de271c00baa60fa6-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.198 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.199 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.199 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.199 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.199 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.199 DEBUG:boto:Data: 2024-04-16T18:18:13.199 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.199 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.199 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.199 DEBUG:boto:Params: {} 2024-04-16T18:18:13.200 DEBUG:boto:Token: None 2024-04-16T18:18:13.200 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:di+Q3Pb8CsD588dDyje1DHzBNYU= 2024-04-16T18:18:13.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:di+Q3Pb8CsD588dDyje1DHzBNYU=', 'Content-Length': '0'} 2024-04-16T18:18:13.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000408cd0733760630c-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.203 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.203 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.203 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.203 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.204 DEBUG:boto:Data: 2024-04-16T18:18:13.204 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.204 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.204 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.204 DEBUG:boto:Params: {} 2024-04-16T18:18:13.204 DEBUG:boto:Token: None 2024-04-16T18:18:13.204 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:di+Q3Pb8CsD588dDyje1DHzBNYU= 2024-04-16T18:18:13.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:di+Q3Pb8CsD588dDyje1DHzBNYU=', 'Content-Length': '0'} 2024-04-16T18:18:13.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f839897f3214c81-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.207 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000010 2024-04-16T18:18:13.207 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.208 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.208 DEBUG:boto:Method: GET 2024-04-16T18:18:13.208 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.208 DEBUG:boto:Data: 2024-04-16T18:18:13.208 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.208 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.208 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.208 DEBUG:boto:Params: {} 2024-04-16T18:18:13.208 DEBUG:boto:Token: None 2024-04-16T18:18:13.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zgA8Xjqod43DhVMPx7xfmjEQUqU= 2024-04-16T18:18:13.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zgA8Xjqod43DhVMPx7xfmjEQUqU=', 'Content-Length': '0'} 2024-04-16T18:18:13.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8ca8afc2fae7f4a-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.254 DEBUG:boto:path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.255 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.255 DEBUG:boto:Method: GET 2024-04-16T18:18:13.255 DEBUG:boto:Path: /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.255 DEBUG:boto:Data: 2024-04-16T18:18:13.255 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.255 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.255 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.255 DEBUG:boto:Params: {} 2024-04-16T18:18:13.255 DEBUG:boto:Token: None 2024-04-16T18:18:13.256 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000010 2024-04-16T18:18:13.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zgA8Xjqod43DhVMPx7xfmjEQUqU= 2024-04-16T18:18:13.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zgA8Xjqod43DhVMPx7xfmjEQUqU=', 'Content-Length': '0'} 2024-04-16T18:18:13.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3b1054d06e4b4c3-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.259 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.259 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.259 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.259 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.259 DEBUG:boto:Method: GET 2024-04-16T18:18:13.259 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.260 DEBUG:boto:Data: 2024-04-16T18:18:13.260 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.260 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.260 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.260 DEBUG:boto:Params: {} 2024-04-16T18:18:13.260 DEBUG:boto:Token: None 2024-04-16T18:18:13.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.265 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009066a126c223e9ef-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.306 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.307 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.307 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.307 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.307 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.307 DEBUG:boto:Method: GET 2024-04-16T18:18:13.307 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000010 2024-04-16T18:18:13.307 DEBUG:boto:Data: 2024-04-16T18:18:13.307 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.307 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.307 DEBUG:boto:Params: {} 2024-04-16T18:18:13.308 DEBUG:boto:Token: None 2024-04-16T18:18:13.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059a6587a095d63d1-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.314 DEBUG:boto:b'lybtug-51obj000000000000000000101000falseobj000000000000000000102024-04-16T18:18:00.895Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.314 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000011 2024-04-16T18:18:13.314 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.314 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.314 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.315 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.315 DEBUG:boto:Method: GET 2024-04-16T18:18:13.315 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.315 DEBUG:boto:Data: 2024-04-16T18:18:13.315 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.315 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.315 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.315 DEBUG:boto:Params: {} 2024-04-16T18:18:13.315 DEBUG:boto:Token: None 2024-04-16T18:18:13.315 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.315 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs= 2024-04-16T18:18:13.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs=', 'Content-Length': '0'} 2024-04-16T18:18:13.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec1522a98edc85ca-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.362 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.362 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.363 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.363 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.363 DEBUG:boto:Method: GET 2024-04-16T18:18:13.363 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.363 DEBUG:boto:Data: 2024-04-16T18:18:13.363 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.363 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.363 DEBUG:boto:Params: {} 2024-04-16T18:18:13.363 DEBUG:boto:Token: None 2024-04-16T18:18:13.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs= 2024-04-16T18:18:13.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs=', 'Content-Length': '0'} 2024-04-16T18:18:13.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000829249e01a194c39-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.367 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.368 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.368 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.368 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.368 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.368 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.368 DEBUG:boto:Data: 2024-04-16T18:18:13.368 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.368 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.368 DEBUG:boto:Params: {} 2024-04-16T18:18:13.369 DEBUG:boto:Token: None 2024-04-16T18:18:13.369 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DKdSCot5JYCUVrkanAVWA+jgwtw= 2024-04-16T18:18:13.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DKdSCot5JYCUVrkanAVWA+jgwtw=', 'Content-Length': '0'} 2024-04-16T18:18:13.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f71b1a7ccc676ebc-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.373 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.373 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.373 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.373 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.373 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.373 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.373 DEBUG:boto:Data: 2024-04-16T18:18:13.373 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.373 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.373 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.373 DEBUG:boto:Params: {} 2024-04-16T18:18:13.374 DEBUG:boto:Token: None 2024-04-16T18:18:13.374 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.374 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DKdSCot5JYCUVrkanAVWA+jgwtw= 2024-04-16T18:18:13.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DKdSCot5JYCUVrkanAVWA+jgwtw=', 'Content-Length': '0'} 2024-04-16T18:18:13.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ecbedb98198e06e9-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.377 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000011 2024-04-16T18:18:13.377 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.377 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.377 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.377 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.377 DEBUG:boto:Method: GET 2024-04-16T18:18:13.377 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.377 DEBUG:boto:Data: 2024-04-16T18:18:13.377 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.378 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.378 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.378 DEBUG:boto:Params: {} 2024-04-16T18:18:13.378 DEBUG:boto:Token: None 2024-04-16T18:18:13.378 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.378 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs= 2024-04-16T18:18:13.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs=', 'Content-Length': '0'} 2024-04-16T18:18:13.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044b89565d1cf6cec-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.422 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.422 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.423 DEBUG:boto:path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.423 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.423 DEBUG:boto:Method: GET 2024-04-16T18:18:13.423 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.423 DEBUG:boto:Data: 2024-04-16T18:18:13.423 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.423 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.423 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.423 DEBUG:boto:Params: {} 2024-04-16T18:18:13.423 DEBUG:boto:Token: None 2024-04-16T18:18:13.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011?versionId=null 2024-04-16T18:18:13.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs= 2024-04-16T18:18:13.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pK7+UWw15/08o/EVovQw1wxLbVs=', 'Content-Length': '0'} 2024-04-16T18:18:13.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024eb0cb573733621-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.427 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.427 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.427 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.427 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.427 DEBUG:boto:Method: GET 2024-04-16T18:18:13.427 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.428 DEBUG:boto:Data: 2024-04-16T18:18:13.428 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.428 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.428 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.428 DEBUG:boto:Params: {} 2024-04-16T18:18:13.428 DEBUG:boto:Token: None 2024-04-16T18:18:13.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.432 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cf274578fa01fc9-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.433 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.433 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.433 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.433 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.433 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.434 DEBUG:boto:Method: GET 2024-04-16T18:18:13.434 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.434 DEBUG:boto:Data: 2024-04-16T18:18:13.434 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.434 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.434 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.434 DEBUG:boto:Params: {} 2024-04-16T18:18:13.434 DEBUG:boto:Token: None 2024-04-16T18:18:13.434 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.434 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1381f755254a960-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.482 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.483 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.483 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.483 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.483 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.483 DEBUG:boto:Data: 2024-04-16T18:18:13.483 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.483 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.483 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.483 DEBUG:boto:Params: {} 2024-04-16T18:18:13.483 DEBUG:boto:Token: None 2024-04-16T18:18:13.484 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WsgISke4J4UAlw4g34TSqNq4NhA= 2024-04-16T18:18:13.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WsgISke4J4UAlw4g34TSqNq4NhA=', 'Content-Length': '0'} 2024-04-16T18:18:13.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000384f270e47de72bf-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.487 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.487 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.487 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.487 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.487 DEBUG:boto:Data: 2024-04-16T18:18:13.487 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.488 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.488 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.488 DEBUG:boto:Params: {} 2024-04-16T18:18:13.488 DEBUG:boto:Token: None 2024-04-16T18:18:13.488 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WsgISke4J4UAlw4g34TSqNq4NhA= 2024-04-16T18:18:13.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WsgISke4J4UAlw4g34TSqNq4NhA=', 'Content-Length': '0'} 2024-04-16T18:18:13.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf3d5cce2e3d508b-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.491 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000011 2024-04-16T18:18:13.491 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.492 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.492 DEBUG:boto:Method: GET 2024-04-16T18:18:13.492 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.492 DEBUG:boto:Data: 2024-04-16T18:18:13.492 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.492 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.492 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.492 DEBUG:boto:Params: {} 2024-04-16T18:18:13.492 DEBUG:boto:Token: None 2024-04-16T18:18:13.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bK9gk0PciA76RC19apNXqGPHbRE= 2024-04-16T18:18:13.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bK9gk0PciA76RC19apNXqGPHbRE=', 'Content-Length': '0'} 2024-04-16T18:18:13.496 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f7f3c2e1694476f-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.538 DEBUG:boto:path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.538 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.539 DEBUG:boto:Method: GET 2024-04-16T18:18:13.539 DEBUG:boto:Path: /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.539 DEBUG:boto:Data: 2024-04-16T18:18:13.539 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.539 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.539 DEBUG:boto:Params: {} 2024-04-16T18:18:13.539 DEBUG:boto:Token: None 2024-04-16T18:18:13.539 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000011 2024-04-16T18:18:13.539 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bK9gk0PciA76RC19apNXqGPHbRE= 2024-04-16T18:18:13.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bK9gk0PciA76RC19apNXqGPHbRE=', 'Content-Length': '0'} 2024-04-16T18:18:13.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b741307e54ab893a-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.543 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.543 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.543 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.543 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.544 DEBUG:boto:Method: GET 2024-04-16T18:18:13.544 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.544 DEBUG:boto:Data: 2024-04-16T18:18:13.544 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.544 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.544 DEBUG:boto:Params: {} 2024-04-16T18:18:13.544 DEBUG:boto:Token: None 2024-04-16T18:18:13.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc5c48a0abda2ff0-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.590 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.591 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.591 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.591 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.591 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.591 DEBUG:boto:Method: GET 2024-04-16T18:18:13.591 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000011 2024-04-16T18:18:13.591 DEBUG:boto:Data: 2024-04-16T18:18:13.591 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.591 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.591 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.592 DEBUG:boto:Params: {} 2024-04-16T18:18:13.592 DEBUG:boto:Token: None 2024-04-16T18:18:13.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.598 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000156003a64f9f9934-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.599 DEBUG:boto:b'lybtug-51obj000000000000000000111000falseobj000000000000000000112024-04-16T18:18:00.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.599 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000012 2024-04-16T18:18:13.599 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.599 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.599 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.599 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.599 DEBUG:boto:Method: GET 2024-04-16T18:18:13.599 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.599 DEBUG:boto:Data: 2024-04-16T18:18:13.599 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.600 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.600 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.600 DEBUG:boto:Params: {} 2024-04-16T18:18:13.600 DEBUG:boto:Token: None 2024-04-16T18:18:13.600 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY= 2024-04-16T18:18:13.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY=', 'Content-Length': '0'} 2024-04-16T18:18:13.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ae95af731014f4c-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.646 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.647 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.647 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.647 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.647 DEBUG:boto:Method: GET 2024-04-16T18:18:13.647 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.647 DEBUG:boto:Data: 2024-04-16T18:18:13.647 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.647 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.647 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.647 DEBUG:boto:Params: {} 2024-04-16T18:18:13.647 DEBUG:boto:Token: None 2024-04-16T18:18:13.648 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY= 2024-04-16T18:18:13.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY=', 'Content-Length': '0'} 2024-04-16T18:18:13.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3372eb6aade3cfb-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.651 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.652 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.652 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.652 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.652 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.652 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.652 DEBUG:boto:Data: 2024-04-16T18:18:13.652 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.652 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.652 DEBUG:boto:Params: {} 2024-04-16T18:18:13.652 DEBUG:boto:Token: None 2024-04-16T18:18:13.653 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.653 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eoIQQF6H6JaTQAFiG7+R5KFQuqc= 2024-04-16T18:18:13.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eoIQQF6H6JaTQAFiG7+R5KFQuqc=', 'Content-Length': '0'} 2024-04-16T18:18:13.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff7a56bc73c3602d-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.656 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.656 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.656 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.656 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.657 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.657 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.657 DEBUG:boto:Data: 2024-04-16T18:18:13.657 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.657 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.657 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.657 DEBUG:boto:Params: {} 2024-04-16T18:18:13.657 DEBUG:boto:Token: None 2024-04-16T18:18:13.657 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eoIQQF6H6JaTQAFiG7+R5KFQuqc= 2024-04-16T18:18:13.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eoIQQF6H6JaTQAFiG7+R5KFQuqc=', 'Content-Length': '0'} 2024-04-16T18:18:13.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024210d205b45d39a-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.660 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000012 2024-04-16T18:18:13.661 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.661 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.661 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.661 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.661 DEBUG:boto:Method: GET 2024-04-16T18:18:13.661 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.661 DEBUG:boto:Data: 2024-04-16T18:18:13.661 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.661 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.661 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.662 DEBUG:boto:Params: {} 2024-04-16T18:18:13.662 DEBUG:boto:Token: None 2024-04-16T18:18:13.662 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.662 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY= 2024-04-16T18:18:13.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY=', 'Content-Length': '0'} 2024-04-16T18:18:13.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003512320226da344f-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.706 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.707 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.707 DEBUG:boto:path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.707 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.707 DEBUG:boto:Method: GET 2024-04-16T18:18:13.707 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.707 DEBUG:boto:Data: 2024-04-16T18:18:13.707 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.707 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.707 DEBUG:boto:Params: {} 2024-04-16T18:18:13.707 DEBUG:boto:Token: None 2024-04-16T18:18:13.707 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012?versionId=null 2024-04-16T18:18:13.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY= 2024-04-16T18:18:13.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OeS5GoqIl65WNnnGTGC/7ihAGVY=', 'Content-Length': '0'} 2024-04-16T18:18:13.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000654a5e76d5d7a1e4-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.711 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.711 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.711 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.711 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.712 DEBUG:boto:Method: GET 2024-04-16T18:18:13.712 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.712 DEBUG:boto:Data: 2024-04-16T18:18:13.712 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.712 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.712 DEBUG:boto:Params: {} 2024-04-16T18:18:13.712 DEBUG:boto:Token: None 2024-04-16T18:18:13.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e19c100a13716e1-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.717 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.717 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.717 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.717 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.717 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.717 DEBUG:boto:Method: GET 2024-04-16T18:18:13.717 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.718 DEBUG:boto:Data: 2024-04-16T18:18:13.718 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.718 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.718 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.718 DEBUG:boto:Params: {} 2024-04-16T18:18:13.718 DEBUG:boto:Token: None 2024-04-16T18:18:13.718 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.724 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b530daa46024b97-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.766 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.767 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.767 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.767 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.767 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.767 DEBUG:boto:Data: 2024-04-16T18:18:13.767 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.767 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.767 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.767 DEBUG:boto:Params: {} 2024-04-16T18:18:13.767 DEBUG:boto:Token: None 2024-04-16T18:18:13.768 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ApO6ETw9xw/93+4eW5d9QKl5mO0= 2024-04-16T18:18:13.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ApO6ETw9xw/93+4eW5d9QKl5mO0=', 'Content-Length': '0'} 2024-04-16T18:18:13.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c5a76ac7ce0f218-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.771 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.771 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.771 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.772 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.772 DEBUG:boto:Data: 2024-04-16T18:18:13.772 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.772 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.772 DEBUG:boto:Params: {} 2024-04-16T18:18:13.772 DEBUG:boto:Token: None 2024-04-16T18:18:13.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ApO6ETw9xw/93+4eW5d9QKl5mO0= 2024-04-16T18:18:13.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ApO6ETw9xw/93+4eW5d9QKl5mO0=', 'Content-Length': '0'} 2024-04-16T18:18:13.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057236634cd8a0a4c-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000012 2024-04-16T18:18:13.775 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.776 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.776 DEBUG:boto:Method: GET 2024-04-16T18:18:13.776 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.776 DEBUG:boto:Data: 2024-04-16T18:18:13.776 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.776 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.776 DEBUG:boto:Params: {} 2024-04-16T18:18:13.776 DEBUG:boto:Token: None 2024-04-16T18:18:13.776 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ie49As/AKrN/jZNsxJba2ocpG1s= 2024-04-16T18:18:13.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ie49As/AKrN/jZNsxJba2ocpG1s=', 'Content-Length': '0'} 2024-04-16T18:18:13.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ed3fa1c4242f0f3-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.822 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.822 DEBUG:boto:Method: GET 2024-04-16T18:18:13.823 DEBUG:boto:Path: /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.823 DEBUG:boto:Data: 2024-04-16T18:18:13.823 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.823 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.823 DEBUG:boto:Params: {} 2024-04-16T18:18:13.823 DEBUG:boto:Token: None 2024-04-16T18:18:13.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000012 2024-04-16T18:18:13.823 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ie49As/AKrN/jZNsxJba2ocpG1s= 2024-04-16T18:18:13.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ie49As/AKrN/jZNsxJba2ocpG1s=', 'Content-Length': '0'} 2024-04-16T18:18:13.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006aabc1183976ce98-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.827 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.827 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.827 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.827 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.827 DEBUG:boto:Method: GET 2024-04-16T18:18:13.827 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.827 DEBUG:boto:Data: 2024-04-16T18:18:13.827 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.827 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.827 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.827 DEBUG:boto:Params: {} 2024-04-16T18:18:13.828 DEBUG:boto:Token: None 2024-04-16T18:18:13.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b90393ec4fc2722-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.874 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.875 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.875 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.875 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.875 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.875 DEBUG:boto:Method: GET 2024-04-16T18:18:13.875 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000012 2024-04-16T18:18:13.875 DEBUG:boto:Data: 2024-04-16T18:18:13.875 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.875 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.875 DEBUG:boto:Params: {} 2024-04-16T18:18:13.876 DEBUG:boto:Token: None 2024-04-16T18:18:13.876 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:13.882 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f92da640e3492384-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.882 DEBUG:boto:b'lybtug-51obj000000000000000000121000falseobj000000000000000000122024-04-16T18:18:00.123Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:13.883 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000013 2024-04-16T18:18:13.883 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.883 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.883 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.883 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.883 DEBUG:boto:Method: GET 2024-04-16T18:18:13.883 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.883 DEBUG:boto:Data: 2024-04-16T18:18:13.883 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.883 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.883 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.883 DEBUG:boto:Params: {} 2024-04-16T18:18:13.884 DEBUG:boto:Token: None 2024-04-16T18:18:13.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA= 2024-04-16T18:18:13.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA=', 'Content-Length': '0'} 2024-04-16T18:18:13.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002948fc59c83b6c03-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.930 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.930 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.931 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.931 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.931 DEBUG:boto:Method: GET 2024-04-16T18:18:13.931 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.931 DEBUG:boto:Data: 2024-04-16T18:18:13.931 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.931 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.931 DEBUG:boto:Params: {} 2024-04-16T18:18:13.931 DEBUG:boto:Token: None 2024-04-16T18:18:13.931 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA= 2024-04-16T18:18:13.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA=', 'Content-Length': '0'} 2024-04-16T18:18:13.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b90d53774fb633e-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.935 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.935 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.935 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.935 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.935 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.935 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.935 DEBUG:boto:Data: 2024-04-16T18:18:13.935 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.935 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.936 DEBUG:boto:Params: {} 2024-04-16T18:18:13.936 DEBUG:boto:Token: None 2024-04-16T18:18:13.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z6KIXmcSANiLGBDxKW02npClJp8= 2024-04-16T18:18:13.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z6KIXmcSANiLGBDxKW02npClJp8=', 'Content-Length': '0'} 2024-04-16T18:18:13.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034d00228353c12ad-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.939 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.939 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.939 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.939 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.939 DEBUG:boto:Method: HEAD 2024-04-16T18:18:13.940 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.940 DEBUG:boto:Data: 2024-04-16T18:18:13.940 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.940 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.940 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.940 DEBUG:boto:Params: {} 2024-04-16T18:18:13.940 DEBUG:boto:Token: None 2024-04-16T18:18:13.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z6KIXmcSANiLGBDxKW02npClJp8= 2024-04-16T18:18:13.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z6KIXmcSANiLGBDxKW02npClJp8=', 'Content-Length': '0'} 2024-04-16T18:18:13.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fca4813135be6ee-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.943 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000013 2024-04-16T18:18:13.943 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.944 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.944 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.944 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.944 DEBUG:boto:Method: GET 2024-04-16T18:18:13.944 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.944 DEBUG:boto:Data: 2024-04-16T18:18:13.944 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.944 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.944 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.944 DEBUG:boto:Params: {} 2024-04-16T18:18:13.945 DEBUG:boto:Token: None 2024-04-16T18:18:13.945 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA= 2024-04-16T18:18:13.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA=', 'Content-Length': '0'} 2024-04-16T18:18:13.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000754bed4f4264080a-00661ec0e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.990 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.990 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:13.990 DEBUG:boto:path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.991 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.991 DEBUG:boto:Method: GET 2024-04-16T18:18:13.991 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.991 DEBUG:boto:Data: 2024-04-16T18:18:13.991 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:13.991 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.991 DEBUG:boto:Params: {} 2024-04-16T18:18:13.991 DEBUG:boto:Token: None 2024-04-16T18:18:13.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/obj00000000000000000013?versionId=null 2024-04-16T18:18:13.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA= 2024-04-16T18:18:13.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aRwXTjNXK5DHaJ57KV6HPw0iqYA=', 'Content-Length': '0'} 2024-04-16T18:18:13.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e9df29f8cd0113f-00661ec0e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:13.995 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:13.995 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:13.995 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:13.995 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:13.995 DEBUG:boto:Method: GET 2024-04-16T18:18:13.995 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:13.995 DEBUG:boto:Data: 2024-04-16T18:18:13.995 DEBUG:boto:Headers: {} 2024-04-16T18:18:13.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:13.995 DEBUG:boto:Port: 8000 2024-04-16T18:18:13.995 DEBUG:boto:Params: {} 2024-04-16T18:18:13.996 DEBUG:boto:Token: None 2024-04-16T18:18:13.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:13 GMT /lybtug-51/ 2024-04-16T18:18:13.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o= 2024-04-16T18:18:13.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7OOA46a8nzvJ/fnap4GYpHRX/o=', 'Content-Length': '0'} 2024-04-16T18:18:14.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb756e77277c970d-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.000 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.001 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.001 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.001 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.001 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.001 DEBUG:boto:Method: GET 2024-04-16T18:18:14.001 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.001 DEBUG:boto:Data: 2024-04-16T18:18:14.001 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.001 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.001 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.001 DEBUG:boto:Params: {} 2024-04-16T18:18:14.002 DEBUG:boto:Token: None 2024-04-16T18:18:14.002 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.002 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.008 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aaba3492a2bce628-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.050 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.051 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.051 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.051 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.051 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.051 DEBUG:boto:Data: 2024-04-16T18:18:14.051 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.051 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.051 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.051 DEBUG:boto:Params: {} 2024-04-16T18:18:14.051 DEBUG:boto:Token: None 2024-04-16T18:18:14.052 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Zmjim/q/1m7Yx+hQ7nXKko7tVDU= 2024-04-16T18:18:14.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Zmjim/q/1m7Yx+hQ7nXKko7tVDU=', 'Content-Length': '0'} 2024-04-16T18:18:14.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aaf38f140ca15452-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.055 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.055 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.055 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.055 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.055 DEBUG:boto:Data: 2024-04-16T18:18:14.055 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.055 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.055 DEBUG:boto:Params: {} 2024-04-16T18:18:14.056 DEBUG:boto:Token: None 2024-04-16T18:18:14.056 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Zmjim/q/1m7Yx+hQ7nXKko7tVDU= 2024-04-16T18:18:14.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Zmjim/q/1m7Yx+hQ7nXKko7tVDU=', 'Content-Length': '0'} 2024-04-16T18:18:14.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002588d2425d2536df-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.059 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000013 2024-04-16T18:18:14.059 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.059 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.059 DEBUG:boto:Method: GET 2024-04-16T18:18:14.060 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.060 DEBUG:boto:Data: 2024-04-16T18:18:14.060 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.060 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.060 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.060 DEBUG:boto:Params: {} 2024-04-16T18:18:14.060 DEBUG:boto:Token: None 2024-04-16T18:18:14.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OTaWnLwpM5OO83nEZa4JriP2HNs= 2024-04-16T18:18:14.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OTaWnLwpM5OO83nEZa4JriP2HNs=', 'Content-Length': '0'} 2024-04-16T18:18:14.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6fba24568c9e598-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.106 DEBUG:boto:path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.106 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.107 DEBUG:boto:Method: GET 2024-04-16T18:18:14.107 DEBUG:boto:Path: /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.107 DEBUG:boto:Data: 2024-04-16T18:18:14.107 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.107 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.107 DEBUG:boto:Params: {} 2024-04-16T18:18:14.107 DEBUG:boto:Token: None 2024-04-16T18:18:14.107 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000013 2024-04-16T18:18:14.107 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OTaWnLwpM5OO83nEZa4JriP2HNs= 2024-04-16T18:18:14.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OTaWnLwpM5OO83nEZa4JriP2HNs=', 'Content-Length': '0'} 2024-04-16T18:18:14.110 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000099767eb3ff1966a-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.111 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.111 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.111 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.111 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.111 DEBUG:boto:Method: GET 2024-04-16T18:18:14.111 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.111 DEBUG:boto:Data: 2024-04-16T18:18:14.111 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.111 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.111 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.111 DEBUG:boto:Params: {} 2024-04-16T18:18:14.112 DEBUG:boto:Token: None 2024-04-16T18:18:14.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.116 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d4d01e5003c738d-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.158 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.159 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.159 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.159 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.159 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.159 DEBUG:boto:Method: GET 2024-04-16T18:18:14.159 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000013 2024-04-16T18:18:14.159 DEBUG:boto:Data: 2024-04-16T18:18:14.159 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.159 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.160 DEBUG:boto:Params: {} 2024-04-16T18:18:14.160 DEBUG:boto:Token: None 2024-04-16T18:18:14.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.165 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000915a6750c702b10d-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.166 DEBUG:boto:b'lybtug-51obj000000000000000000131000falseobj000000000000000000132024-04-16T18:17:57.214Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.166 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000014 2024-04-16T18:18:14.166 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.166 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.166 DEBUG:boto:path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.166 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.167 DEBUG:boto:Method: GET 2024-04-16T18:18:14.167 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.167 DEBUG:boto:Data: 2024-04-16T18:18:14.167 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.167 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.167 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.167 DEBUG:boto:Params: {} 2024-04-16T18:18:14.167 DEBUG:boto:Token: None 2024-04-16T18:18:14.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ= 2024-04-16T18:18:14.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ=', 'Content-Length': '0'} 2024-04-16T18:18:14.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055bd38211c6c7b14-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.214 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.214 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.215 DEBUG:boto:path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.215 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.215 DEBUG:boto:Method: GET 2024-04-16T18:18:14.215 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.215 DEBUG:boto:Data: 2024-04-16T18:18:14.215 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.215 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.215 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.215 DEBUG:boto:Params: {} 2024-04-16T18:18:14.215 DEBUG:boto:Token: None 2024-04-16T18:18:14.215 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.215 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ= 2024-04-16T18:18:14.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ=', 'Content-Length': '0'} 2024-04-16T18:18:14.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ffb5bfd9ebc5659-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.219 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.219 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.220 DEBUG:boto:path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.220 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.220 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.220 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.220 DEBUG:boto:Data: 2024-04-16T18:18:14.220 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.220 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.220 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.220 DEBUG:boto:Params: {} 2024-04-16T18:18:14.220 DEBUG:boto:Token: None 2024-04-16T18:18:14.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.221 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZ6DetApa2xBAEBIk0sxNabYVXw= 2024-04-16T18:18:14.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZ6DetApa2xBAEBIk0sxNabYVXw=', 'Content-Length': '0'} 2024-04-16T18:18:14.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2d96b9a94ec8283-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.224 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.224 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.224 DEBUG:boto:path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.224 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.225 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.225 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.225 DEBUG:boto:Data: 2024-04-16T18:18:14.225 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.225 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.225 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.225 DEBUG:boto:Params: {} 2024-04-16T18:18:14.225 DEBUG:boto:Token: None 2024-04-16T18:18:14.225 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nZ6DetApa2xBAEBIk0sxNabYVXw= 2024-04-16T18:18:14.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nZ6DetApa2xBAEBIk0sxNabYVXw=', 'Content-Length': '0'} 2024-04-16T18:18:14.228 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003aa88fab363f098a-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.229 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000014 2024-04-16T18:18:14.229 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.229 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.229 DEBUG:boto:path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.229 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.229 DEBUG:boto:Method: GET 2024-04-16T18:18:14.229 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.229 DEBUG:boto:Data: 2024-04-16T18:18:14.230 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.230 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.230 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.230 DEBUG:boto:Params: {} 2024-04-16T18:18:14.230 DEBUG:boto:Token: None 2024-04-16T18:18:14.230 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.230 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ= 2024-04-16T18:18:14.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ=', 'Content-Length': '0'} 2024-04-16T18:18:14.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c7436f6b1b9add4-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.274 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.275 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.275 DEBUG:boto:path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.275 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.275 DEBUG:boto:Method: GET 2024-04-16T18:18:14.275 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.275 DEBUG:boto:Data: 2024-04-16T18:18:14.275 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.275 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.275 DEBUG:boto:Params: {} 2024-04-16T18:18:14.275 DEBUG:boto:Token: None 2024-04-16T18:18:14.275 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014?versionId=null 2024-04-16T18:18:14.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ= 2024-04-16T18:18:14.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9VftsBxjNV/nNnllDtuH+QeQlTQ=', 'Content-Length': '0'} 2024-04-16T18:18:14.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8f7d92c6bc9eafe-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.279 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.279 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.279 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.279 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.279 DEBUG:boto:Method: GET 2024-04-16T18:18:14.279 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.280 DEBUG:boto:Data: 2024-04-16T18:18:14.280 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.280 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.280 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.280 DEBUG:boto:Params: {} 2024-04-16T18:18:14.280 DEBUG:boto:Token: None 2024-04-16T18:18:14.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b064c0168b8107c-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.284 DEBUG:boto:b'lybtug-51obj000000000000000000141000falseobj000000000000000000142024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.285 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.285 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.285 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.285 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.285 DEBUG:boto:Method: GET 2024-04-16T18:18:14.285 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.285 DEBUG:boto:Data: 2024-04-16T18:18:14.285 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.285 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.285 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.286 DEBUG:boto:Params: {} 2024-04-16T18:18:14.286 DEBUG:boto:Token: None 2024-04-16T18:18:14.286 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.286 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4b5d132c69e584b-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.334 DEBUG:boto:b'lybtug-51obj000000000000000000141000falseobj000000000000000000142024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.335 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.335 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.335 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.335 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.335 DEBUG:boto:Data: 2024-04-16T18:18:14.335 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.335 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.335 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.335 DEBUG:boto:Params: {} 2024-04-16T18:18:14.335 DEBUG:boto:Token: None 2024-04-16T18:18:14.335 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k63lX4O4nYmg5XyIa9YLaQ65pYw= 2024-04-16T18:18:14.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k63lX4O4nYmg5XyIa9YLaQ65pYw=', 'Content-Length': '0'} 2024-04-16T18:18:14.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074766e8144c9f5a3-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.339 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.339 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.339 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.339 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.339 DEBUG:boto:Data: 2024-04-16T18:18:14.339 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.340 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.340 DEBUG:boto:Params: {} 2024-04-16T18:18:14.340 DEBUG:boto:Token: None 2024-04-16T18:18:14.340 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k63lX4O4nYmg5XyIa9YLaQ65pYw= 2024-04-16T18:18:14.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k63lX4O4nYmg5XyIa9YLaQ65pYw=', 'Content-Length': '0'} 2024-04-16T18:18:14.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ca31375da1ab9c9-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.343 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000014 2024-04-16T18:18:14.343 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.343 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.344 DEBUG:boto:Method: GET 2024-04-16T18:18:14.344 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.344 DEBUG:boto:Data: 2024-04-16T18:18:14.344 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.344 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.344 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.344 DEBUG:boto:Params: {} 2024-04-16T18:18:14.344 DEBUG:boto:Token: None 2024-04-16T18:18:14.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUrkk0F9RpC/GXdbXexz/EJFS24= 2024-04-16T18:18:14.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUrkk0F9RpC/GXdbXexz/EJFS24=', 'Content-Length': '0'} 2024-04-16T18:18:14.348 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c24ead776d856131-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.390 DEBUG:boto:path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.390 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.391 DEBUG:boto:Method: GET 2024-04-16T18:18:14.391 DEBUG:boto:Path: /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.391 DEBUG:boto:Data: 2024-04-16T18:18:14.391 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.391 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.391 DEBUG:boto:Params: {} 2024-04-16T18:18:14.391 DEBUG:boto:Token: None 2024-04-16T18:18:14.391 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000014 2024-04-16T18:18:14.391 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUrkk0F9RpC/GXdbXexz/EJFS24= 2024-04-16T18:18:14.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUrkk0F9RpC/GXdbXexz/EJFS24=', 'Content-Length': '0'} 2024-04-16T18:18:14.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff59cb1cc2e86571-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.395 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.395 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.395 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.395 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.395 DEBUG:boto:Method: GET 2024-04-16T18:18:14.395 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.396 DEBUG:boto:Data: 2024-04-16T18:18:14.396 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.396 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.396 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.396 DEBUG:boto:Params: {} 2024-04-16T18:18:14.396 DEBUG:boto:Token: None 2024-04-16T18:18:14.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.401 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fbd5ef39456bded-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.442 DEBUG:boto:b'lybtug-51obj000000000000000000141000falseobj000000000000000000142024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.443 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.443 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.443 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.443 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.443 DEBUG:boto:Method: GET 2024-04-16T18:18:14.443 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000014 2024-04-16T18:18:14.443 DEBUG:boto:Data: 2024-04-16T18:18:14.443 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.443 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.443 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.443 DEBUG:boto:Params: {} 2024-04-16T18:18:14.444 DEBUG:boto:Token: None 2024-04-16T18:18:14.444 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.450 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000575c3b2c72f24091-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.450 DEBUG:boto:b'lybtug-51obj000000000000000000141000falseobj000000000000000000142024-04-16T18:17:57.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.451 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000015 2024-04-16T18:18:14.451 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.451 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.451 DEBUG:boto:path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.451 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.451 DEBUG:boto:Method: GET 2024-04-16T18:18:14.451 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.451 DEBUG:boto:Data: 2024-04-16T18:18:14.451 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.452 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.452 DEBUG:boto:Params: {} 2024-04-16T18:18:14.452 DEBUG:boto:Token: None 2024-04-16T18:18:14.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w= 2024-04-16T18:18:14.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w=', 'Content-Length': '0'} 2024-04-16T18:18:14.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afc01f6c1da01aae-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.502 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.502 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.502 DEBUG:boto:path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.503 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.503 DEBUG:boto:Method: GET 2024-04-16T18:18:14.503 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.503 DEBUG:boto:Data: 2024-04-16T18:18:14.503 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.503 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.503 DEBUG:boto:Params: {} 2024-04-16T18:18:14.503 DEBUG:boto:Token: None 2024-04-16T18:18:14.503 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.503 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w= 2024-04-16T18:18:14.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w=', 'Content-Length': '0'} 2024-04-16T18:18:14.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ced68a9292f06a7f-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.507 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.507 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.507 DEBUG:boto:path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.507 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.507 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.508 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.508 DEBUG:boto:Data: 2024-04-16T18:18:14.508 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.508 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.508 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.508 DEBUG:boto:Params: {} 2024-04-16T18:18:14.508 DEBUG:boto:Token: None 2024-04-16T18:18:14.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VMk2fPsre6/teaEhK3yxbyLphWc= 2024-04-16T18:18:14.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VMk2fPsre6/teaEhK3yxbyLphWc=', 'Content-Length': '0'} 2024-04-16T18:18:14.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000875ffdbfc09aae41-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.512 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.512 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.512 DEBUG:boto:path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.512 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.512 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.512 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.512 DEBUG:boto:Data: 2024-04-16T18:18:14.512 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.512 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.513 DEBUG:boto:Params: {} 2024-04-16T18:18:14.513 DEBUG:boto:Token: None 2024-04-16T18:18:14.513 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VMk2fPsre6/teaEhK3yxbyLphWc= 2024-04-16T18:18:14.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VMk2fPsre6/teaEhK3yxbyLphWc=', 'Content-Length': '0'} 2024-04-16T18:18:14.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c72eb2ebf911d904-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.516 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000015 2024-04-16T18:18:14.516 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.517 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.517 DEBUG:boto:path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.517 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.517 DEBUG:boto:Method: GET 2024-04-16T18:18:14.517 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.517 DEBUG:boto:Data: 2024-04-16T18:18:14.517 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.517 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.517 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.517 DEBUG:boto:Params: {} 2024-04-16T18:18:14.517 DEBUG:boto:Token: None 2024-04-16T18:18:14.518 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w= 2024-04-16T18:18:14.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w=', 'Content-Length': '0'} 2024-04-16T18:18:14.521 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6b896faa6a192b4-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.562 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.562 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.562 DEBUG:boto:path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.563 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.563 DEBUG:boto:Method: GET 2024-04-16T18:18:14.563 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.563 DEBUG:boto:Data: 2024-04-16T18:18:14.563 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.563 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.563 DEBUG:boto:Params: {} 2024-04-16T18:18:14.563 DEBUG:boto:Token: None 2024-04-16T18:18:14.563 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015?versionId=null 2024-04-16T18:18:14.565 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w= 2024-04-16T18:18:14.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnmdwS2A41xamK2sXBVAnFa8Z0w=', 'Content-Length': '0'} 2024-04-16T18:18:14.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000faa61f6f761ef7f7-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.569 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.569 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.569 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.569 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.569 DEBUG:boto:Method: GET 2024-04-16T18:18:14.569 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.569 DEBUG:boto:Data: 2024-04-16T18:18:14.570 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.570 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.570 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.570 DEBUG:boto:Params: {} 2024-04-16T18:18:14.570 DEBUG:boto:Token: None 2024-04-16T18:18:14.570 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.570 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000971ee646fec74c13-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.575 DEBUG:boto:b'lybtug-51obj000000000000000000151000falseobj000000000000000000152024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.575 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.576 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.576 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.576 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.576 DEBUG:boto:Method: GET 2024-04-16T18:18:14.576 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.576 DEBUG:boto:Data: 2024-04-16T18:18:14.576 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.576 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.576 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.576 DEBUG:boto:Params: {} 2024-04-16T18:18:14.576 DEBUG:boto:Token: None 2024-04-16T18:18:14.577 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.583 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000349743261b169439-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.630 DEBUG:boto:b'lybtug-51obj000000000000000000151000falseobj000000000000000000152024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.631 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.631 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.631 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.631 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.631 DEBUG:boto:Data: 2024-04-16T18:18:14.631 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.631 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.632 DEBUG:boto:Params: {} 2024-04-16T18:18:14.632 DEBUG:boto:Token: None 2024-04-16T18:18:14.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cUkRGY2BHCvRafhrQHGo730nEks= 2024-04-16T18:18:14.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cUkRGY2BHCvRafhrQHGo730nEks=', 'Content-Length': '0'} 2024-04-16T18:18:14.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0ece077bb59c1e4-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.636 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.636 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.636 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.636 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.636 DEBUG:boto:Data: 2024-04-16T18:18:14.636 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.636 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.636 DEBUG:boto:Params: {} 2024-04-16T18:18:14.637 DEBUG:boto:Token: None 2024-04-16T18:18:14.637 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cUkRGY2BHCvRafhrQHGo730nEks= 2024-04-16T18:18:14.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cUkRGY2BHCvRafhrQHGo730nEks=', 'Content-Length': '0'} 2024-04-16T18:18:14.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003305134c5858713e-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000015 2024-04-16T18:18:14.640 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.640 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.640 DEBUG:boto:Method: GET 2024-04-16T18:18:14.641 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.641 DEBUG:boto:Data: 2024-04-16T18:18:14.641 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.641 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.641 DEBUG:boto:Params: {} 2024-04-16T18:18:14.641 DEBUG:boto:Token: None 2024-04-16T18:18:14.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.642 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:91aQq7LcG5rWICiMyEuLGwnTqws= 2024-04-16T18:18:14.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:91aQq7LcG5rWICiMyEuLGwnTqws=', 'Content-Length': '0'} 2024-04-16T18:18:14.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000898e858d318c1a65-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.686 DEBUG:boto:path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.686 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.686 DEBUG:boto:Method: GET 2024-04-16T18:18:14.687 DEBUG:boto:Path: /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.687 DEBUG:boto:Data: 2024-04-16T18:18:14.687 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.687 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.687 DEBUG:boto:Params: {} 2024-04-16T18:18:14.687 DEBUG:boto:Token: None 2024-04-16T18:18:14.687 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000015 2024-04-16T18:18:14.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:91aQq7LcG5rWICiMyEuLGwnTqws= 2024-04-16T18:18:14.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:91aQq7LcG5rWICiMyEuLGwnTqws=', 'Content-Length': '0'} 2024-04-16T18:18:14.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032a1bac265523af4-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.691 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.691 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.691 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.691 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.691 DEBUG:boto:Method: GET 2024-04-16T18:18:14.691 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.692 DEBUG:boto:Data: 2024-04-16T18:18:14.692 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.692 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.692 DEBUG:boto:Params: {} 2024-04-16T18:18:14.692 DEBUG:boto:Token: None 2024-04-16T18:18:14.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.698 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9e64344f0fe7c68-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.738 DEBUG:boto:b'lybtug-51obj000000000000000000151000falseobj000000000000000000152024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.739 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.739 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.739 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.739 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.739 DEBUG:boto:Method: GET 2024-04-16T18:18:14.739 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000015 2024-04-16T18:18:14.739 DEBUG:boto:Data: 2024-04-16T18:18:14.739 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.739 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.740 DEBUG:boto:Params: {} 2024-04-16T18:18:14.740 DEBUG:boto:Token: None 2024-04-16T18:18:14.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.746 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069745f17084df60f-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.746 DEBUG:boto:b'lybtug-51obj000000000000000000151000falseobj000000000000000000152024-04-16T18:18:01.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000016 2024-04-16T18:18:14.747 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.747 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.747 DEBUG:boto:path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.747 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.747 DEBUG:boto:Method: GET 2024-04-16T18:18:14.747 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.748 DEBUG:boto:Data: 2024-04-16T18:18:14.748 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.748 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.748 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.748 DEBUG:boto:Params: {} 2024-04-16T18:18:14.748 DEBUG:boto:Token: None 2024-04-16T18:18:14.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM= 2024-04-16T18:18:14.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM=', 'Content-Length': '0'} 2024-04-16T18:18:14.752 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c009a9aa74a05e9b-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.794 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.795 DEBUG:boto:path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.795 DEBUG:boto:Method: GET 2024-04-16T18:18:14.795 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.795 DEBUG:boto:Data: 2024-04-16T18:18:14.795 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.795 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.795 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.795 DEBUG:boto:Params: {} 2024-04-16T18:18:14.796 DEBUG:boto:Token: None 2024-04-16T18:18:14.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM= 2024-04-16T18:18:14.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM=', 'Content-Length': '0'} 2024-04-16T18:18:14.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3b5f9ac71612d12-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.799 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.799 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.799 DEBUG:boto:path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.800 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.800 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.800 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.800 DEBUG:boto:Data: 2024-04-16T18:18:14.800 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.800 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.800 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.800 DEBUG:boto:Params: {} 2024-04-16T18:18:14.800 DEBUG:boto:Token: None 2024-04-16T18:18:14.801 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aS7kZeeP+FEj2a3+OlblVq8uDmc= 2024-04-16T18:18:14.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aS7kZeeP+FEj2a3+OlblVq8uDmc=', 'Content-Length': '0'} 2024-04-16T18:18:14.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6e23b1c81a86ab3-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.804 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.804 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.804 DEBUG:boto:path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.804 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.804 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.804 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.805 DEBUG:boto:Data: 2024-04-16T18:18:14.805 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.805 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.805 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.805 DEBUG:boto:Params: {} 2024-04-16T18:18:14.805 DEBUG:boto:Token: None 2024-04-16T18:18:14.805 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aS7kZeeP+FEj2a3+OlblVq8uDmc= 2024-04-16T18:18:14.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aS7kZeeP+FEj2a3+OlblVq8uDmc=', 'Content-Length': '0'} 2024-04-16T18:18:14.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f874d9001cd58afc-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000016 2024-04-16T18:18:14.808 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.808 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.809 DEBUG:boto:path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.809 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.809 DEBUG:boto:Method: GET 2024-04-16T18:18:14.809 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.809 DEBUG:boto:Data: 2024-04-16T18:18:14.809 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.809 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.809 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.809 DEBUG:boto:Params: {} 2024-04-16T18:18:14.809 DEBUG:boto:Token: None 2024-04-16T18:18:14.809 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.810 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM= 2024-04-16T18:18:14.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM=', 'Content-Length': '0'} 2024-04-16T18:18:14.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a8b3f4ccda6cb43-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.854 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.854 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.855 DEBUG:boto:path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.855 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.855 DEBUG:boto:Method: GET 2024-04-16T18:18:14.855 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.855 DEBUG:boto:Data: 2024-04-16T18:18:14.855 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.855 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.855 DEBUG:boto:Params: {} 2024-04-16T18:18:14.855 DEBUG:boto:Token: None 2024-04-16T18:18:14.855 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016?versionId=null 2024-04-16T18:18:14.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM= 2024-04-16T18:18:14.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o6+xHKTh5bnbuF/aSeEHNte8uRM=', 'Content-Length': '0'} 2024-04-16T18:18:14.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b3905e58c808269-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.859 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.859 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.859 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.859 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.859 DEBUG:boto:Method: GET 2024-04-16T18:18:14.859 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.859 DEBUG:boto:Data: 2024-04-16T18:18:14.860 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.860 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.860 DEBUG:boto:Params: {} 2024-04-16T18:18:14.860 DEBUG:boto:Token: None 2024-04-16T18:18:14.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098e423a8f0343238-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.864 DEBUG:boto:b'lybtug-51obj000000000000000000161000falseobj000000000000000000162024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.865 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.865 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.865 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.865 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.865 DEBUG:boto:Method: GET 2024-04-16T18:18:14.865 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.865 DEBUG:boto:Data: 2024-04-16T18:18:14.866 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.866 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.866 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.866 DEBUG:boto:Params: {} 2024-04-16T18:18:14.866 DEBUG:boto:Token: None 2024-04-16T18:18:14.866 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.866 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008fa81f508ea19daa-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.914 DEBUG:boto:b'lybtug-51obj000000000000000000161000falseobj000000000000000000162024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:14.915 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.915 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.915 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.915 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.915 DEBUG:boto:Data: 2024-04-16T18:18:14.915 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.915 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.915 DEBUG:boto:Params: {} 2024-04-16T18:18:14.916 DEBUG:boto:Token: None 2024-04-16T18:18:14.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/t0qGgmvYQl6Du69w9O5CUMflDY= 2024-04-16T18:18:14.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/t0qGgmvYQl6Du69w9O5CUMflDY=', 'Content-Length': '0'} 2024-04-16T18:18:14.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025264a215500f9f0-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.919 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.919 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.919 DEBUG:boto:Method: HEAD 2024-04-16T18:18:14.919 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.919 DEBUG:boto:Data: 2024-04-16T18:18:14.920 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.920 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.920 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.920 DEBUG:boto:Params: {} 2024-04-16T18:18:14.920 DEBUG:boto:Token: None 2024-04-16T18:18:14.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/t0qGgmvYQl6Du69w9O5CUMflDY= 2024-04-16T18:18:14.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/t0qGgmvYQl6Du69w9O5CUMflDY=', 'Content-Length': '0'} 2024-04-16T18:18:14.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000248658924e28f8f6-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.923 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000016 2024-04-16T18:18:14.923 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.923 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.924 DEBUG:boto:Method: GET 2024-04-16T18:18:14.924 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.924 DEBUG:boto:Data: 2024-04-16T18:18:14.924 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.924 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.924 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.924 DEBUG:boto:Params: {} 2024-04-16T18:18:14.924 DEBUG:boto:Token: None 2024-04-16T18:18:14.924 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OsJ7OCNhFL1cTsuWuziYMQjiHPQ= 2024-04-16T18:18:14.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OsJ7OCNhFL1cTsuWuziYMQjiHPQ=', 'Content-Length': '0'} 2024-04-16T18:18:14.927 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a140dd06cde5e3a-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.970 DEBUG:boto:path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.970 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.970 DEBUG:boto:Method: GET 2024-04-16T18:18:14.971 DEBUG:boto:Path: /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.971 DEBUG:boto:Data: 2024-04-16T18:18:14.971 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:14.971 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.971 DEBUG:boto:Params: {} 2024-04-16T18:18:14.971 DEBUG:boto:Token: None 2024-04-16T18:18:14.971 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/obj00000000000000000016 2024-04-16T18:18:14.971 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OsJ7OCNhFL1cTsuWuziYMQjiHPQ= 2024-04-16T18:18:14.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OsJ7OCNhFL1cTsuWuziYMQjiHPQ=', 'Content-Length': '0'} 2024-04-16T18:18:14.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d348a7eb40f1342-00661ec0e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:14.974 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:14.975 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:14.975 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.975 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.975 DEBUG:boto:Method: GET 2024-04-16T18:18:14.975 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:14.975 DEBUG:boto:Data: 2024-04-16T18:18:14.975 DEBUG:boto:Headers: {} 2024-04-16T18:18:14.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:14.975 DEBUG:boto:Port: 8000 2024-04-16T18:18:14.975 DEBUG:boto:Params: {} 2024-04-16T18:18:14.976 DEBUG:boto:Token: None 2024-04-16T18:18:14.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:14 GMT /lybtug-51/ 2024-04-16T18:18:14.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0= 2024-04-16T18:18:14.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mgYfDFToY8C3+VYgk8tBqEkx+V0=', 'Content-Length': '0'} 2024-04-16T18:18:14.980 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031db33f194f24ebe-00661ec0e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.022 DEBUG:boto:b'lybtug-51obj000000000000000000161000falseobj000000000000000000162024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.023 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.023 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.023 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:15.023 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:15.023 DEBUG:boto:Method: GET 2024-04-16T18:18:15.023 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000016 2024-04-16T18:18:15.023 DEBUG:boto:Data: 2024-04-16T18:18:15.023 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.024 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.024 DEBUG:boto:Params: {} 2024-04-16T18:18:15.024 DEBUG:boto:Token: None 2024-04-16T18:18:15.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f88515e9f0be6ed-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.030 DEBUG:boto:b'lybtug-51obj000000000000000000161000falseobj000000000000000000162024-04-16T18:18:00.286Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.030 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000017 2024-04-16T18:18:15.031 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.031 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.031 DEBUG:boto:path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.031 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.031 DEBUG:boto:Method: GET 2024-04-16T18:18:15.031 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.031 DEBUG:boto:Data: 2024-04-16T18:18:15.031 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.031 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.031 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.031 DEBUG:boto:Params: {} 2024-04-16T18:18:15.032 DEBUG:boto:Token: None 2024-04-16T18:18:15.032 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY= 2024-04-16T18:18:15.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY=', 'Content-Length': '0'} 2024-04-16T18:18:15.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061d4c1012644f9e1-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.079 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.079 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.079 DEBUG:boto:path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.079 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.079 DEBUG:boto:Method: GET 2024-04-16T18:18:15.079 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.079 DEBUG:boto:Data: 2024-04-16T18:18:15.079 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.079 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.079 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.079 DEBUG:boto:Params: {} 2024-04-16T18:18:15.080 DEBUG:boto:Token: None 2024-04-16T18:18:15.080 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY= 2024-04-16T18:18:15.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY=', 'Content-Length': '0'} 2024-04-16T18:18:15.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040cc94326af9b583-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.084 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.084 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.084 DEBUG:boto:Data: 2024-04-16T18:18:15.084 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.084 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.084 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.084 DEBUG:boto:Params: {} 2024-04-16T18:18:15.084 DEBUG:boto:Token: None 2024-04-16T18:18:15.084 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HFVmf3fouLDdXGFiyZuQjAQGTtw= 2024-04-16T18:18:15.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HFVmf3fouLDdXGFiyZuQjAQGTtw=', 'Content-Length': '0'} 2024-04-16T18:18:15.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c57a07c88757ecd-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.088 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.088 DEBUG:boto:path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.088 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.088 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.088 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.088 DEBUG:boto:Data: 2024-04-16T18:18:15.088 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.088 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.088 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.088 DEBUG:boto:Params: {} 2024-04-16T18:18:15.088 DEBUG:boto:Token: None 2024-04-16T18:18:15.089 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.089 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HFVmf3fouLDdXGFiyZuQjAQGTtw= 2024-04-16T18:18:15.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HFVmf3fouLDdXGFiyZuQjAQGTtw=', 'Content-Length': '0'} 2024-04-16T18:18:15.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000591a8b2256e20d0-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000017 2024-04-16T18:18:15.092 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.092 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.092 DEBUG:boto:path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.092 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.092 DEBUG:boto:Method: GET 2024-04-16T18:18:15.092 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.092 DEBUG:boto:Data: 2024-04-16T18:18:15.092 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.092 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.092 DEBUG:boto:Params: {} 2024-04-16T18:18:15.093 DEBUG:boto:Token: None 2024-04-16T18:18:15.093 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.093 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY= 2024-04-16T18:18:15.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY=', 'Content-Length': '0'} 2024-04-16T18:18:15.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b0fd71d8da8f67e-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.138 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.138 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.139 DEBUG:boto:path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.139 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.139 DEBUG:boto:Method: GET 2024-04-16T18:18:15.139 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.139 DEBUG:boto:Data: 2024-04-16T18:18:15.139 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.139 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.139 DEBUG:boto:Params: {} 2024-04-16T18:18:15.139 DEBUG:boto:Token: None 2024-04-16T18:18:15.139 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017?versionId=null 2024-04-16T18:18:15.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY= 2024-04-16T18:18:15.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o0TqO6woxyTu9A1q7Akh0GO78wY=', 'Content-Length': '0'} 2024-04-16T18:18:15.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083ac103f9c0f8375-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.143 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.143 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.143 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.143 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.143 DEBUG:boto:Method: GET 2024-04-16T18:18:15.143 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.143 DEBUG:boto:Data: 2024-04-16T18:18:15.144 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.144 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.144 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.144 DEBUG:boto:Params: {} 2024-04-16T18:18:15.144 DEBUG:boto:Token: None 2024-04-16T18:18:15.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038d4f4e4e6e0b6c9-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.149 DEBUG:boto:b'lybtug-51obj000000000000000000171000falseobj000000000000000000172024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.149 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.149 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.149 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.149 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.150 DEBUG:boto:Method: GET 2024-04-16T18:18:15.150 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.150 DEBUG:boto:Data: 2024-04-16T18:18:15.150 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.150 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.150 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.150 DEBUG:boto:Params: {} 2024-04-16T18:18:15.150 DEBUG:boto:Token: None 2024-04-16T18:18:15.150 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.150 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.156 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006f69c5435487512-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.202 DEBUG:boto:b'lybtug-51obj000000000000000000171000falseobj000000000000000000172024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.203 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.203 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.203 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.203 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.203 DEBUG:boto:Data: 2024-04-16T18:18:15.203 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.203 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.203 DEBUG:boto:Params: {} 2024-04-16T18:18:15.204 DEBUG:boto:Token: None 2024-04-16T18:18:15.204 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yfHMQLxkZg9lFh7TvkvDaptg884= 2024-04-16T18:18:15.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yfHMQLxkZg9lFh7TvkvDaptg884=', 'Content-Length': '0'} 2024-04-16T18:18:15.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c5dccceb2ec862d-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.207 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.207 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.207 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.207 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.207 DEBUG:boto:Data: 2024-04-16T18:18:15.208 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.208 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.208 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.208 DEBUG:boto:Params: {} 2024-04-16T18:18:15.208 DEBUG:boto:Token: None 2024-04-16T18:18:15.208 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yfHMQLxkZg9lFh7TvkvDaptg884= 2024-04-16T18:18:15.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yfHMQLxkZg9lFh7TvkvDaptg884=', 'Content-Length': '0'} 2024-04-16T18:18:15.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a23281f8bb31aa8-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.211 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000017 2024-04-16T18:18:15.212 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.212 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.212 DEBUG:boto:Method: GET 2024-04-16T18:18:15.212 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.212 DEBUG:boto:Data: 2024-04-16T18:18:15.212 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.212 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.212 DEBUG:boto:Params: {} 2024-04-16T18:18:15.212 DEBUG:boto:Token: None 2024-04-16T18:18:15.212 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kC9iOQe+NppJwsta/JreG78rn24= 2024-04-16T18:18:15.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kC9iOQe+NppJwsta/JreG78rn24=', 'Content-Length': '0'} 2024-04-16T18:18:15.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1d21e8a4c602510-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.258 DEBUG:boto:path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.258 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.259 DEBUG:boto:Method: GET 2024-04-16T18:18:15.259 DEBUG:boto:Path: /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.259 DEBUG:boto:Data: 2024-04-16T18:18:15.259 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.259 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.259 DEBUG:boto:Params: {} 2024-04-16T18:18:15.259 DEBUG:boto:Token: None 2024-04-16T18:18:15.259 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000017 2024-04-16T18:18:15.259 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kC9iOQe+NppJwsta/JreG78rn24= 2024-04-16T18:18:15.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kC9iOQe+NppJwsta/JreG78rn24=', 'Content-Length': '0'} 2024-04-16T18:18:15.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048b8045d2f6e2b11-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.262 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.263 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.263 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.263 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.263 DEBUG:boto:Method: GET 2024-04-16T18:18:15.263 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.263 DEBUG:boto:Data: 2024-04-16T18:18:15.263 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.263 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.263 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.263 DEBUG:boto:Params: {} 2024-04-16T18:18:15.263 DEBUG:boto:Token: None 2024-04-16T18:18:15.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2fa7b84c9a03cc1-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.310 DEBUG:boto:b'lybtug-51obj000000000000000000171000falseobj000000000000000000172024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.311 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.311 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.311 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.311 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.311 DEBUG:boto:Method: GET 2024-04-16T18:18:15.311 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000017 2024-04-16T18:18:15.311 DEBUG:boto:Data: 2024-04-16T18:18:15.312 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.312 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.312 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.312 DEBUG:boto:Params: {} 2024-04-16T18:18:15.312 DEBUG:boto:Token: None 2024-04-16T18:18:15.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff054dcfee5dc5d8-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.319 DEBUG:boto:b'lybtug-51obj000000000000000000171000falseobj000000000000000000172024-04-16T18:18:00.178Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.319 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000018 2024-04-16T18:18:15.319 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.319 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.319 DEBUG:boto:path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.319 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.319 DEBUG:boto:Method: GET 2024-04-16T18:18:15.320 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.320 DEBUG:boto:Data: 2024-04-16T18:18:15.320 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.320 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.320 DEBUG:boto:Params: {} 2024-04-16T18:18:15.320 DEBUG:boto:Token: None 2024-04-16T18:18:15.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8= 2024-04-16T18:18:15.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8=', 'Content-Length': '0'} 2024-04-16T18:18:15.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8fb945dd5a81a2b-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.366 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.366 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.367 DEBUG:boto:path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.367 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.367 DEBUG:boto:Method: GET 2024-04-16T18:18:15.367 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.367 DEBUG:boto:Data: 2024-04-16T18:18:15.367 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.367 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.367 DEBUG:boto:Params: {} 2024-04-16T18:18:15.367 DEBUG:boto:Token: None 2024-04-16T18:18:15.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8= 2024-04-16T18:18:15.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8=', 'Content-Length': '0'} 2024-04-16T18:18:15.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001df35ee2b1e7f6fd-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.371 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.372 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.372 DEBUG:boto:path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.372 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.372 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.372 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.372 DEBUG:boto:Data: 2024-04-16T18:18:15.372 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.372 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.372 DEBUG:boto:Params: {} 2024-04-16T18:18:15.373 DEBUG:boto:Token: None 2024-04-16T18:18:15.373 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.373 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:80rIGY1ulE4KI2fZ+owszw0yUiM= 2024-04-16T18:18:15.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:80rIGY1ulE4KI2fZ+owszw0yUiM=', 'Content-Length': '0'} 2024-04-16T18:18:15.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e43e82bbf7063e2-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.376 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.376 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.376 DEBUG:boto:path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.376 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.376 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.376 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.377 DEBUG:boto:Data: 2024-04-16T18:18:15.377 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.377 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.377 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.377 DEBUG:boto:Params: {} 2024-04-16T18:18:15.377 DEBUG:boto:Token: None 2024-04-16T18:18:15.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:80rIGY1ulE4KI2fZ+owszw0yUiM= 2024-04-16T18:18:15.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:80rIGY1ulE4KI2fZ+owszw0yUiM=', 'Content-Length': '0'} 2024-04-16T18:18:15.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e15a7d2e723a7295-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.381 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000018 2024-04-16T18:18:15.381 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.381 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.381 DEBUG:boto:path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.381 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.381 DEBUG:boto:Method: GET 2024-04-16T18:18:15.381 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.381 DEBUG:boto:Data: 2024-04-16T18:18:15.381 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.381 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.382 DEBUG:boto:Params: {} 2024-04-16T18:18:15.382 DEBUG:boto:Token: None 2024-04-16T18:18:15.382 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.382 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8= 2024-04-16T18:18:15.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8=', 'Content-Length': '0'} 2024-04-16T18:18:15.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b26ff2ce5d9e9f69-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.426 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.426 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.427 DEBUG:boto:path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.427 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.427 DEBUG:boto:Method: GET 2024-04-16T18:18:15.427 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.427 DEBUG:boto:Data: 2024-04-16T18:18:15.427 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.427 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.427 DEBUG:boto:Params: {} 2024-04-16T18:18:15.427 DEBUG:boto:Token: None 2024-04-16T18:18:15.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018?versionId=null 2024-04-16T18:18:15.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8= 2024-04-16T18:18:15.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tiO4e180MhXMD+o8XyY9gf6Bhl8=', 'Content-Length': '0'} 2024-04-16T18:18:15.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000969447c7679a3c2a-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.431 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.431 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.431 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.431 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.432 DEBUG:boto:Method: GET 2024-04-16T18:18:15.432 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.432 DEBUG:boto:Data: 2024-04-16T18:18:15.432 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.432 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.432 DEBUG:boto:Params: {} 2024-04-16T18:18:15.432 DEBUG:boto:Token: None 2024-04-16T18:18:15.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000834deacf53b01f0f-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.437 DEBUG:boto:b'lybtug-51obj000000000000000000181000falseobj000000000000000000182024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.437 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.437 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.437 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.437 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.437 DEBUG:boto:Method: GET 2024-04-16T18:18:15.437 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.438 DEBUG:boto:Data: 2024-04-16T18:18:15.438 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.438 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.438 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.438 DEBUG:boto:Params: {} 2024-04-16T18:18:15.438 DEBUG:boto:Token: None 2024-04-16T18:18:15.438 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.438 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e468e554b3b04459-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.486 DEBUG:boto:b'lybtug-51obj000000000000000000181000falseobj000000000000000000182024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.487 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.487 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.487 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.487 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.487 DEBUG:boto:Data: 2024-04-16T18:18:15.487 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.487 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.487 DEBUG:boto:Params: {} 2024-04-16T18:18:15.488 DEBUG:boto:Token: None 2024-04-16T18:18:15.488 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V+mP8E3TyVki34fAVkcgU/sv6FE= 2024-04-16T18:18:15.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V+mP8E3TyVki34fAVkcgU/sv6FE=', 'Content-Length': '0'} 2024-04-16T18:18:15.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6717f8c91173090-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.491 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.491 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.491 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.492 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.492 DEBUG:boto:Data: 2024-04-16T18:18:15.492 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.492 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.492 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.492 DEBUG:boto:Params: {} 2024-04-16T18:18:15.492 DEBUG:boto:Token: None 2024-04-16T18:18:15.492 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V+mP8E3TyVki34fAVkcgU/sv6FE= 2024-04-16T18:18:15.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V+mP8E3TyVki34fAVkcgU/sv6FE=', 'Content-Length': '0'} 2024-04-16T18:18:15.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000253d46f01f03fade-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000018 2024-04-16T18:18:15.495 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.496 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.496 DEBUG:boto:Method: GET 2024-04-16T18:18:15.496 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.496 DEBUG:boto:Data: 2024-04-16T18:18:15.496 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.496 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.496 DEBUG:boto:Params: {} 2024-04-16T18:18:15.496 DEBUG:boto:Token: None 2024-04-16T18:18:15.496 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m6Z6RT1FvA2qxuJNo+j3TDWsyZE= 2024-04-16T18:18:15.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m6Z6RT1FvA2qxuJNo+j3TDWsyZE=', 'Content-Length': '0'} 2024-04-16T18:18:15.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023ca666deaec4c28-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.543 DEBUG:boto:path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.543 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.543 DEBUG:boto:Method: GET 2024-04-16T18:18:15.543 DEBUG:boto:Path: /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.543 DEBUG:boto:Data: 2024-04-16T18:18:15.543 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.543 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.543 DEBUG:boto:Params: {} 2024-04-16T18:18:15.543 DEBUG:boto:Token: None 2024-04-16T18:18:15.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000018 2024-04-16T18:18:15.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m6Z6RT1FvA2qxuJNo+j3TDWsyZE= 2024-04-16T18:18:15.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m6Z6RT1FvA2qxuJNo+j3TDWsyZE=', 'Content-Length': '0'} 2024-04-16T18:18:15.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000857c47da2eb3a77a-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.547 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.547 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.548 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.548 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.548 DEBUG:boto:Method: GET 2024-04-16T18:18:15.548 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.548 DEBUG:boto:Data: 2024-04-16T18:18:15.548 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.548 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.548 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.548 DEBUG:boto:Params: {} 2024-04-16T18:18:15.548 DEBUG:boto:Token: None 2024-04-16T18:18:15.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.553 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002585ec7060b45228-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.594 DEBUG:boto:b'lybtug-51obj000000000000000000181000falseobj000000000000000000182024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.595 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.595 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.595 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.595 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.595 DEBUG:boto:Method: GET 2024-04-16T18:18:15.595 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000018 2024-04-16T18:18:15.595 DEBUG:boto:Data: 2024-04-16T18:18:15.595 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.596 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.596 DEBUG:boto:Params: {} 2024-04-16T18:18:15.596 DEBUG:boto:Token: None 2024-04-16T18:18:15.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b26ccc874039205-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.602 DEBUG:boto:b'lybtug-51obj000000000000000000181000falseobj000000000000000000182024-04-16T18:18:01.323Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.602 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000019 2024-04-16T18:18:15.602 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.603 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.603 DEBUG:boto:path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.603 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.603 DEBUG:boto:Method: GET 2024-04-16T18:18:15.603 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.603 DEBUG:boto:Data: 2024-04-16T18:18:15.603 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.603 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.603 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.603 DEBUG:boto:Params: {} 2024-04-16T18:18:15.603 DEBUG:boto:Token: None 2024-04-16T18:18:15.603 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U= 2024-04-16T18:18:15.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U=', 'Content-Length': '0'} 2024-04-16T18:18:15.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b85fe2805ca204f4-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.650 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.650 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.651 DEBUG:boto:path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.651 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.651 DEBUG:boto:Method: GET 2024-04-16T18:18:15.651 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.651 DEBUG:boto:Data: 2024-04-16T18:18:15.651 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.651 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.651 DEBUG:boto:Params: {} 2024-04-16T18:18:15.651 DEBUG:boto:Token: None 2024-04-16T18:18:15.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U= 2024-04-16T18:18:15.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U=', 'Content-Length': '0'} 2024-04-16T18:18:15.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030c2cb08c63500ea-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.655 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.655 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.655 DEBUG:boto:path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.655 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.655 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.655 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.655 DEBUG:boto:Data: 2024-04-16T18:18:15.655 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.656 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.656 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.656 DEBUG:boto:Params: {} 2024-04-16T18:18:15.656 DEBUG:boto:Token: None 2024-04-16T18:18:15.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzVY9XCwpzxasg1o5V/GpmxGC6U= 2024-04-16T18:18:15.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzVY9XCwpzxasg1o5V/GpmxGC6U=', 'Content-Length': '0'} 2024-04-16T18:18:15.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cb8606526a23a23-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.659 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.659 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.659 DEBUG:boto:path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.659 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.659 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.660 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.660 DEBUG:boto:Data: 2024-04-16T18:18:15.660 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.660 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.660 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.660 DEBUG:boto:Params: {} 2024-04-16T18:18:15.660 DEBUG:boto:Token: None 2024-04-16T18:18:15.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzVY9XCwpzxasg1o5V/GpmxGC6U= 2024-04-16T18:18:15.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzVY9XCwpzxasg1o5V/GpmxGC6U=', 'Content-Length': '0'} 2024-04-16T18:18:15.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000103e000eebf403a5-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000019 2024-04-16T18:18:15.663 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.663 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.663 DEBUG:boto:path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.663 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.663 DEBUG:boto:Method: GET 2024-04-16T18:18:15.664 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.664 DEBUG:boto:Data: 2024-04-16T18:18:15.664 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.664 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.664 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.664 DEBUG:boto:Params: {} 2024-04-16T18:18:15.664 DEBUG:boto:Token: None 2024-04-16T18:18:15.664 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U= 2024-04-16T18:18:15.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U=', 'Content-Length': '0'} 2024-04-16T18:18:15.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbe79ebe57333317-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.710 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.710 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.711 DEBUG:boto:path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.711 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.711 DEBUG:boto:Method: GET 2024-04-16T18:18:15.711 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.711 DEBUG:boto:Data: 2024-04-16T18:18:15.711 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.711 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.711 DEBUG:boto:Params: {} 2024-04-16T18:18:15.711 DEBUG:boto:Token: None 2024-04-16T18:18:15.711 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019?versionId=null 2024-04-16T18:18:15.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U= 2024-04-16T18:18:15.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bXpW3fjl6ZIuVOZBrxa4k0jon7U=', 'Content-Length': '0'} 2024-04-16T18:18:15.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb89143cdafdde2f-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.715 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.715 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.715 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.715 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.715 DEBUG:boto:Method: GET 2024-04-16T18:18:15.715 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.716 DEBUG:boto:Data: 2024-04-16T18:18:15.716 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.716 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.716 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.716 DEBUG:boto:Params: {} 2024-04-16T18:18:15.716 DEBUG:boto:Token: None 2024-04-16T18:18:15.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.720 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083e8512e82b495c9-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.721 DEBUG:boto:b'lybtug-51obj000000000000000000191000falseobj000000000000000000192024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.721 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.721 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.721 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.721 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.721 DEBUG:boto:Method: GET 2024-04-16T18:18:15.722 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.722 DEBUG:boto:Data: 2024-04-16T18:18:15.722 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.722 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.722 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.722 DEBUG:boto:Params: {} 2024-04-16T18:18:15.722 DEBUG:boto:Token: None 2024-04-16T18:18:15.722 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.722 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.728 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053ea4a11a5a1ba3a-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.770 DEBUG:boto:b'lybtug-51obj000000000000000000191000falseobj000000000000000000192024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.771 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.771 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.771 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.771 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.771 DEBUG:boto:Data: 2024-04-16T18:18:15.771 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.771 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.771 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.771 DEBUG:boto:Params: {} 2024-04-16T18:18:15.771 DEBUG:boto:Token: None 2024-04-16T18:18:15.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9hfTpfjHJsHejN2ww2EPAzb066w= 2024-04-16T18:18:15.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9hfTpfjHJsHejN2ww2EPAzb066w=', 'Content-Length': '0'} 2024-04-16T18:18:15.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020d0e7dee30a86c5-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.775 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.775 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.775 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.775 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.775 DEBUG:boto:Data: 2024-04-16T18:18:15.775 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.776 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.776 DEBUG:boto:Params: {} 2024-04-16T18:18:15.776 DEBUG:boto:Token: None 2024-04-16T18:18:15.776 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9hfTpfjHJsHejN2ww2EPAzb066w= 2024-04-16T18:18:15.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9hfTpfjHJsHejN2ww2EPAzb066w=', 'Content-Length': '0'} 2024-04-16T18:18:15.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bc1d5c2366cbd08-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.779 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000019 2024-04-16T18:18:15.779 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.779 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.780 DEBUG:boto:Method: GET 2024-04-16T18:18:15.780 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.780 DEBUG:boto:Data: 2024-04-16T18:18:15.780 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.780 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.780 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.780 DEBUG:boto:Params: {} 2024-04-16T18:18:15.780 DEBUG:boto:Token: None 2024-04-16T18:18:15.780 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7vAWFy3gF9XWOwX9/ybLxZw7Kk4= 2024-04-16T18:18:15.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7vAWFy3gF9XWOwX9/ybLxZw7Kk4=', 'Content-Length': '0'} 2024-04-16T18:18:15.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000104af9c1937a88cb-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.826 DEBUG:boto:path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.826 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.826 DEBUG:boto:Method: GET 2024-04-16T18:18:15.827 DEBUG:boto:Path: /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.827 DEBUG:boto:Data: 2024-04-16T18:18:15.827 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.827 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.827 DEBUG:boto:Params: {} 2024-04-16T18:18:15.827 DEBUG:boto:Token: None 2024-04-16T18:18:15.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000019 2024-04-16T18:18:15.827 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7vAWFy3gF9XWOwX9/ybLxZw7Kk4= 2024-04-16T18:18:15.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7vAWFy3gF9XWOwX9/ybLxZw7Kk4=', 'Content-Length': '0'} 2024-04-16T18:18:15.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d49f365aeed9b1e3-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.830 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.830 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.830 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.831 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.831 DEBUG:boto:Method: GET 2024-04-16T18:18:15.831 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.831 DEBUG:boto:Data: 2024-04-16T18:18:15.831 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.831 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.831 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.831 DEBUG:boto:Params: {} 2024-04-16T18:18:15.831 DEBUG:boto:Token: None 2024-04-16T18:18:15.831 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006084fcde73d85d6a-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.878 DEBUG:boto:b'lybtug-51obj000000000000000000191000falseobj000000000000000000192024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.879 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.879 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.879 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.879 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.879 DEBUG:boto:Method: GET 2024-04-16T18:18:15.879 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000019 2024-04-16T18:18:15.879 DEBUG:boto:Data: 2024-04-16T18:18:15.879 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.879 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.879 DEBUG:boto:Params: {} 2024-04-16T18:18:15.879 DEBUG:boto:Token: None 2024-04-16T18:18:15.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/ 2024-04-16T18:18:15.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM= 2024-04-16T18:18:15.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bT6RAEBUQoENnkDKCOzUeVDEuUM=', 'Content-Length': '0'} 2024-04-16T18:18:15.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f33db496e3450fd-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.885 DEBUG:boto:b'lybtug-51obj000000000000000000191000falseobj000000000000000000192024-04-16T18:18:00.101Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:15.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000020 2024-04-16T18:18:15.886 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.886 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.886 DEBUG:boto:path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.886 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.886 DEBUG:boto:Method: GET 2024-04-16T18:18:15.886 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.886 DEBUG:boto:Data: 2024-04-16T18:18:15.886 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.887 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.887 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.887 DEBUG:boto:Params: {} 2024-04-16T18:18:15.887 DEBUG:boto:Token: None 2024-04-16T18:18:15.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.887 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM= 2024-04-16T18:18:15.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM=', 'Content-Length': '0'} 2024-04-16T18:18:15.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039bb03fd631a9e4a-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.934 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.934 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.935 DEBUG:boto:path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.935 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.935 DEBUG:boto:Method: GET 2024-04-16T18:18:15.935 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.935 DEBUG:boto:Data: 2024-04-16T18:18:15.935 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.935 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.935 DEBUG:boto:Params: {} 2024-04-16T18:18:15.935 DEBUG:boto:Token: None 2024-04-16T18:18:15.935 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM= 2024-04-16T18:18:15.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM=', 'Content-Length': '0'} 2024-04-16T18:18:15.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d654e5c5664693a-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.939 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.939 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.939 DEBUG:boto:path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.939 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.939 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.939 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.939 DEBUG:boto:Data: 2024-04-16T18:18:15.939 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.939 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.940 DEBUG:boto:Params: {} 2024-04-16T18:18:15.940 DEBUG:boto:Token: None 2024-04-16T18:18:15.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7F6dB7WshQFcPQbKg+Ito7+0Wcc= 2024-04-16T18:18:15.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7F6dB7WshQFcPQbKg+Ito7+0Wcc=', 'Content-Length': '0'} 2024-04-16T18:18:15.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bf61a18e480b155-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.943 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.943 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.943 DEBUG:boto:path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.943 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.944 DEBUG:boto:Method: HEAD 2024-04-16T18:18:15.944 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.944 DEBUG:boto:Data: 2024-04-16T18:18:15.944 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.944 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.944 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.944 DEBUG:boto:Params: {} 2024-04-16T18:18:15.944 DEBUG:boto:Token: None 2024-04-16T18:18:15.944 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7F6dB7WshQFcPQbKg+Ito7+0Wcc= 2024-04-16T18:18:15.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7F6dB7WshQFcPQbKg+Ito7+0Wcc=', 'Content-Length': '0'} 2024-04-16T18:18:15.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007181a3e7265a5271-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.948 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000020 2024-04-16T18:18:15.948 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.948 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.948 DEBUG:boto:path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.948 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.948 DEBUG:boto:Method: GET 2024-04-16T18:18:15.948 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.949 DEBUG:boto:Data: 2024-04-16T18:18:15.949 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.949 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:15.949 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.949 DEBUG:boto:Params: {} 2024-04-16T18:18:15.949 DEBUG:boto:Token: None 2024-04-16T18:18:15.949 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM= 2024-04-16T18:18:15.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM=', 'Content-Length': '0'} 2024-04-16T18:18:15.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1e1f3ee6b260008-00661ec0e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.994 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.994 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:15.994 DEBUG:boto:path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.995 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.995 DEBUG:boto:Method: GET 2024-04-16T18:18:15.995 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.995 DEBUG:boto:Data: 2024-04-16T18:18:15.995 DEBUG:boto:Headers: {} 2024-04-16T18:18:15.995 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:15.995 DEBUG:boto:Port: 8000 2024-04-16T18:18:15.995 DEBUG:boto:Params: {} 2024-04-16T18:18:15.995 DEBUG:boto:Token: None 2024-04-16T18:18:15.995 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:15 GMT /lybtug-51/obj00000000000000000020?versionId=null 2024-04-16T18:18:15.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM= 2024-04-16T18:18:15.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hcBH6KJKNdTvNHQBf6kbUknYvfM=', 'Content-Length': '0'} 2024-04-16T18:18:15.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000feda2995ce5b9ee8-00661ec0e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:15.999 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:15.999 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:15.999 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:15.999 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:15.999 DEBUG:boto:Method: GET 2024-04-16T18:18:15.999 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:15.999 DEBUG:boto:Data: 2024-04-16T18:18:16.000 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.000 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.000 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.000 DEBUG:boto:Params: {} 2024-04-16T18:18:16.000 DEBUG:boto:Token: None 2024-04-16T18:18:16.000 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000faf153af8798d2c4-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.005 DEBUG:boto:b'lybtug-51obj000000000000000000201000falseobj000000000000000000202024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.005 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.005 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.005 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.005 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.005 DEBUG:boto:Method: GET 2024-04-16T18:18:16.006 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.006 DEBUG:boto:Data: 2024-04-16T18:18:16.006 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.006 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.006 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.006 DEBUG:boto:Params: {} 2024-04-16T18:18:16.006 DEBUG:boto:Token: None 2024-04-16T18:18:16.006 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.006 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a0bc72aee53e5fa-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.054 DEBUG:boto:b'lybtug-51obj000000000000000000201000falseobj000000000000000000202024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.055 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.055 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.055 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.055 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.055 DEBUG:boto:Data: 2024-04-16T18:18:16.055 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.055 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.055 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.055 DEBUG:boto:Params: {} 2024-04-16T18:18:16.055 DEBUG:boto:Token: None 2024-04-16T18:18:16.055 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wsXBSr79x4eANBOID6OUd42/wsc= 2024-04-16T18:18:16.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wsXBSr79x4eANBOID6OUd42/wsc=', 'Content-Length': '0'} 2024-04-16T18:18:16.058 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f056a3854ebc5dc0-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.059 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.059 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.059 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.059 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.059 DEBUG:boto:Data: 2024-04-16T18:18:16.059 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.059 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.059 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.059 DEBUG:boto:Params: {} 2024-04-16T18:18:16.059 DEBUG:boto:Token: None 2024-04-16T18:18:16.060 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wsXBSr79x4eANBOID6OUd42/wsc= 2024-04-16T18:18:16.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wsXBSr79x4eANBOID6OUd42/wsc=', 'Content-Length': '0'} 2024-04-16T18:18:16.062 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000548a2996233924ab-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.063 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000020 2024-04-16T18:18:16.063 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.063 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.063 DEBUG:boto:Method: GET 2024-04-16T18:18:16.063 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.063 DEBUG:boto:Data: 2024-04-16T18:18:16.063 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.063 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.063 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.063 DEBUG:boto:Params: {} 2024-04-16T18:18:16.064 DEBUG:boto:Token: None 2024-04-16T18:18:16.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zDzbpSeOOKSOilO7lHryL6kICzg= 2024-04-16T18:18:16.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zDzbpSeOOKSOilO7lHryL6kICzg=', 'Content-Length': '0'} 2024-04-16T18:18:16.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7a2c29f3ea75eb1-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.110 DEBUG:boto:path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.110 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.110 DEBUG:boto:Method: GET 2024-04-16T18:18:16.111 DEBUG:boto:Path: /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.111 DEBUG:boto:Data: 2024-04-16T18:18:16.111 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.111 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.111 DEBUG:boto:Params: {} 2024-04-16T18:18:16.111 DEBUG:boto:Token: None 2024-04-16T18:18:16.111 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000020 2024-04-16T18:18:16.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zDzbpSeOOKSOilO7lHryL6kICzg= 2024-04-16T18:18:16.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zDzbpSeOOKSOilO7lHryL6kICzg=', 'Content-Length': '0'} 2024-04-16T18:18:16.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2a0014f62ef886d-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.115 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.115 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.115 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.115 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.115 DEBUG:boto:Method: GET 2024-04-16T18:18:16.115 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.115 DEBUG:boto:Data: 2024-04-16T18:18:16.116 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.116 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.116 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.116 DEBUG:boto:Params: {} 2024-04-16T18:18:16.116 DEBUG:boto:Token: None 2024-04-16T18:18:16.116 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.121 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003db3dce060057cfb-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.162 DEBUG:boto:b'lybtug-51obj000000000000000000201000falseobj000000000000000000202024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.163 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.163 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.163 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.163 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.163 DEBUG:boto:Method: GET 2024-04-16T18:18:16.163 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000020 2024-04-16T18:18:16.163 DEBUG:boto:Data: 2024-04-16T18:18:16.163 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.163 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.163 DEBUG:boto:Params: {} 2024-04-16T18:18:16.164 DEBUG:boto:Token: None 2024-04-16T18:18:16.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8af15033a604c43-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.170 DEBUG:boto:b'lybtug-51obj000000000000000000201000falseobj000000000000000000202024-04-16T18:18:00.819Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.170 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000021 2024-04-16T18:18:16.170 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.170 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.170 DEBUG:boto:path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.171 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.171 DEBUG:boto:Method: GET 2024-04-16T18:18:16.171 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.171 DEBUG:boto:Data: 2024-04-16T18:18:16.171 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.171 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.171 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.171 DEBUG:boto:Params: {} 2024-04-16T18:18:16.171 DEBUG:boto:Token: None 2024-04-16T18:18:16.171 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8= 2024-04-16T18:18:16.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8=', 'Content-Length': '0'} 2024-04-16T18:18:16.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b8c5ea5eb04048b-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.218 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.218 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.218 DEBUG:boto:path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.219 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.219 DEBUG:boto:Method: GET 2024-04-16T18:18:16.219 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.219 DEBUG:boto:Data: 2024-04-16T18:18:16.219 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.219 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.219 DEBUG:boto:Params: {} 2024-04-16T18:18:16.219 DEBUG:boto:Token: None 2024-04-16T18:18:16.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8= 2024-04-16T18:18:16.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8=', 'Content-Length': '0'} 2024-04-16T18:18:16.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000453170a04cedbbd1-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.223 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.223 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.223 DEBUG:boto:path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.223 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.223 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.224 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.224 DEBUG:boto:Data: 2024-04-16T18:18:16.224 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.224 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.224 DEBUG:boto:Params: {} 2024-04-16T18:18:16.224 DEBUG:boto:Token: None 2024-04-16T18:18:16.224 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cEwQT/mU5+7PEaRePWIMkg1iAk4= 2024-04-16T18:18:16.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cEwQT/mU5+7PEaRePWIMkg1iAk4=', 'Content-Length': '0'} 2024-04-16T18:18:16.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ba5d10e77ea905a-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.227 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.227 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.228 DEBUG:boto:path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.228 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.228 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.228 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.228 DEBUG:boto:Data: 2024-04-16T18:18:16.228 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.228 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.228 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.228 DEBUG:boto:Params: {} 2024-04-16T18:18:16.228 DEBUG:boto:Token: None 2024-04-16T18:18:16.228 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.229 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cEwQT/mU5+7PEaRePWIMkg1iAk4= 2024-04-16T18:18:16.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cEwQT/mU5+7PEaRePWIMkg1iAk4=', 'Content-Length': '0'} 2024-04-16T18:18:16.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000378f7a65fb1c4fc5-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.232 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000021 2024-04-16T18:18:16.232 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.232 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.232 DEBUG:boto:path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.232 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.232 DEBUG:boto:Method: GET 2024-04-16T18:18:16.232 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.232 DEBUG:boto:Data: 2024-04-16T18:18:16.232 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.232 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.233 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.233 DEBUG:boto:Params: {} 2024-04-16T18:18:16.233 DEBUG:boto:Token: None 2024-04-16T18:18:16.233 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.233 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8= 2024-04-16T18:18:16.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8=', 'Content-Length': '0'} 2024-04-16T18:18:16.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000774d6f8f1c7d846-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.278 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.278 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.278 DEBUG:boto:path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.279 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.279 DEBUG:boto:Method: GET 2024-04-16T18:18:16.279 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.279 DEBUG:boto:Data: 2024-04-16T18:18:16.279 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.279 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.279 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.279 DEBUG:boto:Params: {} 2024-04-16T18:18:16.279 DEBUG:boto:Token: None 2024-04-16T18:18:16.279 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021?versionId=null 2024-04-16T18:18:16.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8= 2024-04-16T18:18:16.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3JxQscCN8QZItDzUJPZKkVVFQc8=', 'Content-Length': '0'} 2024-04-16T18:18:16.283 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000444e180e34e36ab4-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.283 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.283 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.283 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.283 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.283 DEBUG:boto:Method: GET 2024-04-16T18:18:16.283 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.284 DEBUG:boto:Data: 2024-04-16T18:18:16.284 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.284 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.284 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.284 DEBUG:boto:Params: {} 2024-04-16T18:18:16.284 DEBUG:boto:Token: None 2024-04-16T18:18:16.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070c055fb839a4918-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.288 DEBUG:boto:b'lybtug-51obj000000000000000000211000falseobj000000000000000000212024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.289 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.289 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.289 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.289 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.289 DEBUG:boto:Method: GET 2024-04-16T18:18:16.289 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.289 DEBUG:boto:Data: 2024-04-16T18:18:16.289 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.289 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.289 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.289 DEBUG:boto:Params: {} 2024-04-16T18:18:16.290 DEBUG:boto:Token: None 2024-04-16T18:18:16.290 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.290 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6decf09c05d7431-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.338 DEBUG:boto:b'lybtug-51obj000000000000000000211000falseobj000000000000000000212024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.339 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.339 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.339 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.339 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.339 DEBUG:boto:Data: 2024-04-16T18:18:16.339 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.339 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.339 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.339 DEBUG:boto:Params: {} 2024-04-16T18:18:16.340 DEBUG:boto:Token: None 2024-04-16T18:18:16.340 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lqp3LjyU0iW7So7md7ERAnOUIAQ= 2024-04-16T18:18:16.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lqp3LjyU0iW7So7md7ERAnOUIAQ=', 'Content-Length': '0'} 2024-04-16T18:18:16.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015a6c91f9b219976-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.343 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.343 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.343 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.343 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.343 DEBUG:boto:Data: 2024-04-16T18:18:16.343 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.344 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.344 DEBUG:boto:Params: {} 2024-04-16T18:18:16.344 DEBUG:boto:Token: None 2024-04-16T18:18:16.344 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lqp3LjyU0iW7So7md7ERAnOUIAQ= 2024-04-16T18:18:16.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lqp3LjyU0iW7So7md7ERAnOUIAQ=', 'Content-Length': '0'} 2024-04-16T18:18:16.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000898518485be8770f-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.347 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000021 2024-04-16T18:18:16.347 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.347 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.347 DEBUG:boto:Method: GET 2024-04-16T18:18:16.347 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.347 DEBUG:boto:Data: 2024-04-16T18:18:16.348 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.348 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.348 DEBUG:boto:Params: {} 2024-04-16T18:18:16.348 DEBUG:boto:Token: None 2024-04-16T18:18:16.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rkopZ2udC9P262hHHJx1OWI5LFU= 2024-04-16T18:18:16.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rkopZ2udC9P262hHHJx1OWI5LFU=', 'Content-Length': '0'} 2024-04-16T18:18:16.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000937b2fcdab145cb8-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.394 DEBUG:boto:path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.394 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.394 DEBUG:boto:Method: GET 2024-04-16T18:18:16.395 DEBUG:boto:Path: /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.395 DEBUG:boto:Data: 2024-04-16T18:18:16.395 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.395 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.395 DEBUG:boto:Params: {} 2024-04-16T18:18:16.395 DEBUG:boto:Token: None 2024-04-16T18:18:16.395 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000021 2024-04-16T18:18:16.395 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rkopZ2udC9P262hHHJx1OWI5LFU= 2024-04-16T18:18:16.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rkopZ2udC9P262hHHJx1OWI5LFU=', 'Content-Length': '0'} 2024-04-16T18:18:16.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069ce312498c196d2-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.399 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.399 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.399 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.399 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.399 DEBUG:boto:Method: GET 2024-04-16T18:18:16.399 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.399 DEBUG:boto:Data: 2024-04-16T18:18:16.400 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.400 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.400 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.400 DEBUG:boto:Params: {} 2024-04-16T18:18:16.400 DEBUG:boto:Token: None 2024-04-16T18:18:16.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.406 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee8658d7324f79f2-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.446 DEBUG:boto:b'lybtug-51obj000000000000000000211000falseobj000000000000000000212024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.447 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.447 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.447 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.447 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.447 DEBUG:boto:Method: GET 2024-04-16T18:18:16.447 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000021 2024-04-16T18:18:16.447 DEBUG:boto:Data: 2024-04-16T18:18:16.447 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.447 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.447 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.447 DEBUG:boto:Params: {} 2024-04-16T18:18:16.447 DEBUG:boto:Token: None 2024-04-16T18:18:16.448 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ea4d523b4b7697d-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.454 DEBUG:boto:b'lybtug-51obj000000000000000000211000falseobj000000000000000000212024-04-16T18:18:00.441Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.455 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000022 2024-04-16T18:18:16.455 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.455 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.455 DEBUG:boto:path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.455 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.455 DEBUG:boto:Method: GET 2024-04-16T18:18:16.455 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.455 DEBUG:boto:Data: 2024-04-16T18:18:16.455 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.456 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.456 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.456 DEBUG:boto:Params: {} 2024-04-16T18:18:16.456 DEBUG:boto:Token: None 2024-04-16T18:18:16.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I= 2024-04-16T18:18:16.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I=', 'Content-Length': '0'} 2024-04-16T18:18:16.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cf262718672414d-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.502 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.502 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.503 DEBUG:boto:path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.503 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.503 DEBUG:boto:Method: GET 2024-04-16T18:18:16.503 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.503 DEBUG:boto:Data: 2024-04-16T18:18:16.503 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.503 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.503 DEBUG:boto:Params: {} 2024-04-16T18:18:16.503 DEBUG:boto:Token: None 2024-04-16T18:18:16.503 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.503 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I= 2024-04-16T18:18:16.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I=', 'Content-Length': '0'} 2024-04-16T18:18:16.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe1c77a185760ff8-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.507 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.507 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.507 DEBUG:boto:path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.507 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.507 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.507 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.507 DEBUG:boto:Data: 2024-04-16T18:18:16.507 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.508 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.508 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.508 DEBUG:boto:Params: {} 2024-04-16T18:18:16.508 DEBUG:boto:Token: None 2024-04-16T18:18:16.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JgBaimCPl9xD8KnSoz03gPBjKZQ= 2024-04-16T18:18:16.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JgBaimCPl9xD8KnSoz03gPBjKZQ=', 'Content-Length': '0'} 2024-04-16T18:18:16.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b804cb6f257fa6ad-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.511 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.511 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.512 DEBUG:boto:path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.512 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.512 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.512 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.512 DEBUG:boto:Data: 2024-04-16T18:18:16.512 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.512 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.512 DEBUG:boto:Params: {} 2024-04-16T18:18:16.512 DEBUG:boto:Token: None 2024-04-16T18:18:16.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JgBaimCPl9xD8KnSoz03gPBjKZQ= 2024-04-16T18:18:16.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JgBaimCPl9xD8KnSoz03gPBjKZQ=', 'Content-Length': '0'} 2024-04-16T18:18:16.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b77ee11af802028-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.516 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000022 2024-04-16T18:18:16.516 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.516 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.516 DEBUG:boto:path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.516 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.516 DEBUG:boto:Method: GET 2024-04-16T18:18:16.516 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.516 DEBUG:boto:Data: 2024-04-16T18:18:16.516 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.516 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.516 DEBUG:boto:Params: {} 2024-04-16T18:18:16.517 DEBUG:boto:Token: None 2024-04-16T18:18:16.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I= 2024-04-16T18:18:16.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I=', 'Content-Length': '0'} 2024-04-16T18:18:16.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba3ec5f275731bd3-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.562 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.562 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.563 DEBUG:boto:path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.563 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.563 DEBUG:boto:Method: GET 2024-04-16T18:18:16.563 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.563 DEBUG:boto:Data: 2024-04-16T18:18:16.563 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.563 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.563 DEBUG:boto:Params: {} 2024-04-16T18:18:16.563 DEBUG:boto:Token: None 2024-04-16T18:18:16.563 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022?versionId=null 2024-04-16T18:18:16.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I= 2024-04-16T18:18:16.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rD7KNwx0qIKvzl/C+c1wbp91k4I=', 'Content-Length': '0'} 2024-04-16T18:18:16.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a041010796668c9-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.567 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.567 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.567 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.567 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.567 DEBUG:boto:Method: GET 2024-04-16T18:18:16.568 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.568 DEBUG:boto:Data: 2024-04-16T18:18:16.568 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.568 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.568 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.568 DEBUG:boto:Params: {} 2024-04-16T18:18:16.568 DEBUG:boto:Token: None 2024-04-16T18:18:16.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c2e41fa31245b98-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.572 DEBUG:boto:b'lybtug-51obj000000000000000000221000falseobj000000000000000000222024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.573 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.573 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.573 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.573 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.573 DEBUG:boto:Method: GET 2024-04-16T18:18:16.573 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.573 DEBUG:boto:Data: 2024-04-16T18:18:16.573 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.573 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.573 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.574 DEBUG:boto:Params: {} 2024-04-16T18:18:16.574 DEBUG:boto:Token: None 2024-04-16T18:18:16.574 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.574 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000034a302e6a19bc51-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.622 DEBUG:boto:b'lybtug-51obj000000000000000000221000falseobj000000000000000000222024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.623 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.623 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.623 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.623 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.623 DEBUG:boto:Data: 2024-04-16T18:18:16.623 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.623 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.623 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.623 DEBUG:boto:Params: {} 2024-04-16T18:18:16.623 DEBUG:boto:Token: None 2024-04-16T18:18:16.624 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:19HJNyKk0WsJORD0wb+sJOGTf2k= 2024-04-16T18:18:16.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:19HJNyKk0WsJORD0wb+sJOGTf2k=', 'Content-Length': '0'} 2024-04-16T18:18:16.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b20d895a655c806-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.627 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.627 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.627 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.627 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.627 DEBUG:boto:Data: 2024-04-16T18:18:16.627 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.627 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.628 DEBUG:boto:Params: {} 2024-04-16T18:18:16.628 DEBUG:boto:Token: None 2024-04-16T18:18:16.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:19HJNyKk0WsJORD0wb+sJOGTf2k= 2024-04-16T18:18:16.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:19HJNyKk0WsJORD0wb+sJOGTf2k=', 'Content-Length': '0'} 2024-04-16T18:18:16.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a7e078486474636-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000022 2024-04-16T18:18:16.631 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.631 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.631 DEBUG:boto:Method: GET 2024-04-16T18:18:16.631 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.631 DEBUG:boto:Data: 2024-04-16T18:18:16.632 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.632 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.632 DEBUG:boto:Params: {} 2024-04-16T18:18:16.632 DEBUG:boto:Token: None 2024-04-16T18:18:16.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w3MP2Zf70YpfKxZG/Vpu9X0qq6k= 2024-04-16T18:18:16.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w3MP2Zf70YpfKxZG/Vpu9X0qq6k=', 'Content-Length': '0'} 2024-04-16T18:18:16.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be2299e72ac4c3a3-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.678 DEBUG:boto:path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.679 DEBUG:boto:Method: GET 2024-04-16T18:18:16.679 DEBUG:boto:Path: /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.679 DEBUG:boto:Data: 2024-04-16T18:18:16.679 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.679 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.679 DEBUG:boto:Params: {} 2024-04-16T18:18:16.679 DEBUG:boto:Token: None 2024-04-16T18:18:16.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000022 2024-04-16T18:18:16.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w3MP2Zf70YpfKxZG/Vpu9X0qq6k= 2024-04-16T18:18:16.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w3MP2Zf70YpfKxZG/Vpu9X0qq6k=', 'Content-Length': '0'} 2024-04-16T18:18:16.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a23afb1013887638-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.683 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.683 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.684 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.684 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.684 DEBUG:boto:Method: GET 2024-04-16T18:18:16.684 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.684 DEBUG:boto:Data: 2024-04-16T18:18:16.684 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.684 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.684 DEBUG:boto:Params: {} 2024-04-16T18:18:16.684 DEBUG:boto:Token: None 2024-04-16T18:18:16.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae23f82cbe67422c-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.730 DEBUG:boto:b'lybtug-51obj000000000000000000221000falseobj000000000000000000222024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.731 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.731 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.731 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.731 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.731 DEBUG:boto:Method: GET 2024-04-16T18:18:16.731 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000022 2024-04-16T18:18:16.731 DEBUG:boto:Data: 2024-04-16T18:18:16.731 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.731 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.732 DEBUG:boto:Params: {} 2024-04-16T18:18:16.732 DEBUG:boto:Token: None 2024-04-16T18:18:16.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000270951638d581c99-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.739 DEBUG:boto:b'lybtug-51obj000000000000000000221000falseobj000000000000000000222024-04-16T18:17:59.202Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.739 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000023 2024-04-16T18:18:16.739 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.739 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.739 DEBUG:boto:path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.739 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.739 DEBUG:boto:Method: GET 2024-04-16T18:18:16.739 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.739 DEBUG:boto:Data: 2024-04-16T18:18:16.740 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.740 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.740 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.740 DEBUG:boto:Params: {} 2024-04-16T18:18:16.740 DEBUG:boto:Token: None 2024-04-16T18:18:16.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog= 2024-04-16T18:18:16.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog=', 'Content-Length': '0'} 2024-04-16T18:18:16.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8e3682070081971-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.786 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.786 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.786 DEBUG:boto:path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.787 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.787 DEBUG:boto:Method: GET 2024-04-16T18:18:16.787 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.787 DEBUG:boto:Data: 2024-04-16T18:18:16.787 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.787 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.787 DEBUG:boto:Params: {} 2024-04-16T18:18:16.787 DEBUG:boto:Token: None 2024-04-16T18:18:16.787 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.787 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog= 2024-04-16T18:18:16.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog=', 'Content-Length': '0'} 2024-04-16T18:18:16.790 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000476defade76ea229-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.791 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.791 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.791 DEBUG:boto:path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.791 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.791 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.791 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.791 DEBUG:boto:Data: 2024-04-16T18:18:16.791 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.791 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.792 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.792 DEBUG:boto:Params: {} 2024-04-16T18:18:16.792 DEBUG:boto:Token: None 2024-04-16T18:18:16.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hWatTJZKUyjjD/H09dwkd8hur6I= 2024-04-16T18:18:16.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hWatTJZKUyjjD/H09dwkd8hur6I=', 'Content-Length': '0'} 2024-04-16T18:18:16.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a19e3a97afbb2942-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.795 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.795 DEBUG:boto:path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.795 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.795 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.796 DEBUG:boto:Data: 2024-04-16T18:18:16.796 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.796 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.796 DEBUG:boto:Params: {} 2024-04-16T18:18:16.796 DEBUG:boto:Token: None 2024-04-16T18:18:16.796 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hWatTJZKUyjjD/H09dwkd8hur6I= 2024-04-16T18:18:16.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hWatTJZKUyjjD/H09dwkd8hur6I=', 'Content-Length': '0'} 2024-04-16T18:18:16.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007be1a6c1c7914164-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000023 2024-04-16T18:18:16.800 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.800 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.800 DEBUG:boto:path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.800 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.800 DEBUG:boto:Method: GET 2024-04-16T18:18:16.800 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.800 DEBUG:boto:Data: 2024-04-16T18:18:16.800 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.801 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.801 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.801 DEBUG:boto:Params: {} 2024-04-16T18:18:16.801 DEBUG:boto:Token: None 2024-04-16T18:18:16.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog= 2024-04-16T18:18:16.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog=', 'Content-Length': '0'} 2024-04-16T18:18:16.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008904240eb9a82fc6-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.846 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.847 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.847 DEBUG:boto:path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.847 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.847 DEBUG:boto:Method: GET 2024-04-16T18:18:16.847 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.847 DEBUG:boto:Data: 2024-04-16T18:18:16.847 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.847 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.847 DEBUG:boto:Params: {} 2024-04-16T18:18:16.847 DEBUG:boto:Token: None 2024-04-16T18:18:16.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023?versionId=null 2024-04-16T18:18:16.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog= 2024-04-16T18:18:16.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4hlZvllugMVJQ07J1oH3unGmqog=', 'Content-Length': '0'} 2024-04-16T18:18:16.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c9f342dfdba675e-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.851 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.851 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.851 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.851 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.851 DEBUG:boto:Method: GET 2024-04-16T18:18:16.851 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.852 DEBUG:boto:Data: 2024-04-16T18:18:16.852 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.852 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.852 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.852 DEBUG:boto:Params: {} 2024-04-16T18:18:16.852 DEBUG:boto:Token: None 2024-04-16T18:18:16.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.857 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e0467fc8e7cf6fc-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.857 DEBUG:boto:b'lybtug-51obj000000000000000000231000falseobj000000000000000000232024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.857 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.857 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.857 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.857 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.858 DEBUG:boto:Method: GET 2024-04-16T18:18:16.858 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.858 DEBUG:boto:Data: 2024-04-16T18:18:16.858 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.858 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.858 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.858 DEBUG:boto:Params: {} 2024-04-16T18:18:16.858 DEBUG:boto:Token: None 2024-04-16T18:18:16.858 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.858 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000340af8eca88db28a-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.906 DEBUG:boto:b'lybtug-51obj000000000000000000231000falseobj000000000000000000232024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:16.907 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.907 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.907 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.907 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.907 DEBUG:boto:Data: 2024-04-16T18:18:16.907 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.907 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.907 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.907 DEBUG:boto:Params: {} 2024-04-16T18:18:16.907 DEBUG:boto:Token: None 2024-04-16T18:18:16.908 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ryFGTfO2VyVYbow3+fb/Kx8dOCI= 2024-04-16T18:18:16.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ryFGTfO2VyVYbow3+fb/Kx8dOCI=', 'Content-Length': '0'} 2024-04-16T18:18:16.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1a7038e911c5a3a-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.911 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.911 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.911 DEBUG:boto:Method: HEAD 2024-04-16T18:18:16.911 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.911 DEBUG:boto:Data: 2024-04-16T18:18:16.911 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.912 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.912 DEBUG:boto:Params: {} 2024-04-16T18:18:16.912 DEBUG:boto:Token: None 2024-04-16T18:18:16.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ryFGTfO2VyVYbow3+fb/Kx8dOCI= 2024-04-16T18:18:16.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ryFGTfO2VyVYbow3+fb/Kx8dOCI=', 'Content-Length': '0'} 2024-04-16T18:18:16.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dbdbcd6107c74ca-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000023 2024-04-16T18:18:16.916 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.916 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.916 DEBUG:boto:Method: GET 2024-04-16T18:18:16.916 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.916 DEBUG:boto:Data: 2024-04-16T18:18:16.916 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.916 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.916 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.916 DEBUG:boto:Params: {} 2024-04-16T18:18:16.916 DEBUG:boto:Token: None 2024-04-16T18:18:16.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.917 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FiobJaW7jckTJ8hw9sphIRu5D1U= 2024-04-16T18:18:16.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FiobJaW7jckTJ8hw9sphIRu5D1U=', 'Content-Length': '0'} 2024-04-16T18:18:16.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b56bfb7832e8219d-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.962 DEBUG:boto:path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.962 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.963 DEBUG:boto:Method: GET 2024-04-16T18:18:16.963 DEBUG:boto:Path: /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.963 DEBUG:boto:Data: 2024-04-16T18:18:16.963 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:16.963 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.963 DEBUG:boto:Params: {} 2024-04-16T18:18:16.963 DEBUG:boto:Token: None 2024-04-16T18:18:16.963 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/obj00000000000000000023 2024-04-16T18:18:16.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FiobJaW7jckTJ8hw9sphIRu5D1U= 2024-04-16T18:18:16.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FiobJaW7jckTJ8hw9sphIRu5D1U=', 'Content-Length': '0'} 2024-04-16T18:18:16.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030fc8d7ee5699941-00661ec0e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:16.967 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:16.967 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:16.967 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.967 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.967 DEBUG:boto:Method: GET 2024-04-16T18:18:16.967 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:16.967 DEBUG:boto:Data: 2024-04-16T18:18:16.967 DEBUG:boto:Headers: {} 2024-04-16T18:18:16.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:16.968 DEBUG:boto:Port: 8000 2024-04-16T18:18:16.968 DEBUG:boto:Params: {} 2024-04-16T18:18:16.968 DEBUG:boto:Token: None 2024-04-16T18:18:16.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:16 GMT /lybtug-51/ 2024-04-16T18:18:16.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0= 2024-04-16T18:18:16.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7trBjl+HLnwE5gBHpcoAhdZlZf0=', 'Content-Length': '0'} 2024-04-16T18:18:16.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077cd59fc3669fb9e-00661ec0e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.014 DEBUG:boto:b'lybtug-51obj000000000000000000231000falseobj000000000000000000232024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.015 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.015 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.015 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:17.015 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:17.015 DEBUG:boto:Method: GET 2024-04-16T18:18:17.015 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000023 2024-04-16T18:18:17.015 DEBUG:boto:Data: 2024-04-16T18:18:17.015 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.015 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.015 DEBUG:boto:Params: {} 2024-04-16T18:18:17.016 DEBUG:boto:Token: None 2024-04-16T18:18:17.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.022 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000235991260d2b0e8a-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.022 DEBUG:boto:b'lybtug-51obj000000000000000000231000falseobj000000000000000000232024-04-16T18:17:58.848Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.022 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000024 2024-04-16T18:18:17.022 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.023 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.023 DEBUG:boto:path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.023 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.023 DEBUG:boto:Method: GET 2024-04-16T18:18:17.023 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.023 DEBUG:boto:Data: 2024-04-16T18:18:17.023 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.023 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.023 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.023 DEBUG:boto:Params: {} 2024-04-16T18:18:17.023 DEBUG:boto:Token: None 2024-04-16T18:18:17.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y= 2024-04-16T18:18:17.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y=', 'Content-Length': '0'} 2024-04-16T18:18:17.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdb77d035c065103-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.070 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.070 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.071 DEBUG:boto:path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.071 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.071 DEBUG:boto:Method: GET 2024-04-16T18:18:17.071 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.071 DEBUG:boto:Data: 2024-04-16T18:18:17.071 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.071 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.071 DEBUG:boto:Params: {} 2024-04-16T18:18:17.071 DEBUG:boto:Token: None 2024-04-16T18:18:17.072 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y= 2024-04-16T18:18:17.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y=', 'Content-Length': '0'} 2024-04-16T18:18:17.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fef719e825d006f-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.075 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.075 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.075 DEBUG:boto:path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.076 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.076 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.076 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.076 DEBUG:boto:Data: 2024-04-16T18:18:17.076 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.076 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.076 DEBUG:boto:Params: {} 2024-04-16T18:18:17.076 DEBUG:boto:Token: None 2024-04-16T18:18:17.076 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.077 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0wVnQqJvX4s3vGbXa3iHAMQ4v/4= 2024-04-16T18:18:17.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0wVnQqJvX4s3vGbXa3iHAMQ4v/4=', 'Content-Length': '0'} 2024-04-16T18:18:17.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca3546ee36b5aab2-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.080 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.080 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.080 DEBUG:boto:path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.080 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.081 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.081 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.081 DEBUG:boto:Data: 2024-04-16T18:18:17.081 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.081 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.081 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.081 DEBUG:boto:Params: {} 2024-04-16T18:18:17.081 DEBUG:boto:Token: None 2024-04-16T18:18:17.081 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0wVnQqJvX4s3vGbXa3iHAMQ4v/4= 2024-04-16T18:18:17.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0wVnQqJvX4s3vGbXa3iHAMQ4v/4=', 'Content-Length': '0'} 2024-04-16T18:18:17.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005650c02fc5b17d02-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000024 2024-04-16T18:18:17.084 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.085 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.085 DEBUG:boto:path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.085 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.085 DEBUG:boto:Method: GET 2024-04-16T18:18:17.085 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.085 DEBUG:boto:Data: 2024-04-16T18:18:17.085 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.085 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.085 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.085 DEBUG:boto:Params: {} 2024-04-16T18:18:17.086 DEBUG:boto:Token: None 2024-04-16T18:18:17.086 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.086 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y= 2024-04-16T18:18:17.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y=', 'Content-Length': '0'} 2024-04-16T18:18:17.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035f3ab4f4919ee1e-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.130 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.130 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.130 DEBUG:boto:path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.131 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.131 DEBUG:boto:Method: GET 2024-04-16T18:18:17.131 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.131 DEBUG:boto:Data: 2024-04-16T18:18:17.131 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.131 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.131 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.131 DEBUG:boto:Params: {} 2024-04-16T18:18:17.131 DEBUG:boto:Token: None 2024-04-16T18:18:17.131 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024?versionId=null 2024-04-16T18:18:17.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y= 2024-04-16T18:18:17.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GeubaYBqAEL090xENlXKHoLaS7Y=', 'Content-Length': '0'} 2024-04-16T18:18:17.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1a667cf413c9804-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.135 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.135 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.135 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.135 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.136 DEBUG:boto:Method: GET 2024-04-16T18:18:17.136 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.136 DEBUG:boto:Data: 2024-04-16T18:18:17.136 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.136 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.136 DEBUG:boto:Params: {} 2024-04-16T18:18:17.136 DEBUG:boto:Token: None 2024-04-16T18:18:17.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.141 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ccb097e688ca3e93-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.141 DEBUG:boto:b'lybtug-51obj000000000000000000241000falseobj000000000000000000242024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.141 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.141 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.142 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.142 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.142 DEBUG:boto:Method: GET 2024-04-16T18:18:17.142 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.142 DEBUG:boto:Data: 2024-04-16T18:18:17.142 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.142 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.142 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.142 DEBUG:boto:Params: {} 2024-04-16T18:18:17.142 DEBUG:boto:Token: None 2024-04-16T18:18:17.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.143 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066e3e132da1da1e0-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.190 DEBUG:boto:b'lybtug-51obj000000000000000000241000falseobj000000000000000000242024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.191 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.191 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.191 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.191 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.191 DEBUG:boto:Data: 2024-04-16T18:18:17.191 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.191 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.191 DEBUG:boto:Params: {} 2024-04-16T18:18:17.192 DEBUG:boto:Token: None 2024-04-16T18:18:17.192 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1HjoFEuspjCOtq1KyiU2BGPJYFA= 2024-04-16T18:18:17.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1HjoFEuspjCOtq1KyiU2BGPJYFA=', 'Content-Length': '0'} 2024-04-16T18:18:17.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4fc2f7c1d3598cc-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.195 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.195 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.195 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.195 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.195 DEBUG:boto:Data: 2024-04-16T18:18:17.196 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.196 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.196 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.196 DEBUG:boto:Params: {} 2024-04-16T18:18:17.196 DEBUG:boto:Token: None 2024-04-16T18:18:17.196 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1HjoFEuspjCOtq1KyiU2BGPJYFA= 2024-04-16T18:18:17.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1HjoFEuspjCOtq1KyiU2BGPJYFA=', 'Content-Length': '0'} 2024-04-16T18:18:17.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048446f7568d2a930-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.199 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000024 2024-04-16T18:18:17.199 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.200 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.200 DEBUG:boto:Method: GET 2024-04-16T18:18:17.200 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.200 DEBUG:boto:Data: 2024-04-16T18:18:17.200 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.200 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.200 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.200 DEBUG:boto:Params: {} 2024-04-16T18:18:17.200 DEBUG:boto:Token: None 2024-04-16T18:18:17.200 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RyuRTdFE98jGAdmP35I/3Pu+7iA= 2024-04-16T18:18:17.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RyuRTdFE98jGAdmP35I/3Pu+7iA=', 'Content-Length': '0'} 2024-04-16T18:18:17.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000256562a130c78f31-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.246 DEBUG:boto:path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.246 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.247 DEBUG:boto:Method: GET 2024-04-16T18:18:17.247 DEBUG:boto:Path: /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.247 DEBUG:boto:Data: 2024-04-16T18:18:17.247 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.247 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.247 DEBUG:boto:Params: {} 2024-04-16T18:18:17.247 DEBUG:boto:Token: None 2024-04-16T18:18:17.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000024 2024-04-16T18:18:17.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RyuRTdFE98jGAdmP35I/3Pu+7iA= 2024-04-16T18:18:17.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RyuRTdFE98jGAdmP35I/3Pu+7iA=', 'Content-Length': '0'} 2024-04-16T18:18:17.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b16f2c842ded2d58-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.251 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.251 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.251 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.251 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.251 DEBUG:boto:Method: GET 2024-04-16T18:18:17.251 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.251 DEBUG:boto:Data: 2024-04-16T18:18:17.251 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.251 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.251 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.252 DEBUG:boto:Params: {} 2024-04-16T18:18:17.252 DEBUG:boto:Token: None 2024-04-16T18:18:17.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ea005a43d25301d-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.298 DEBUG:boto:b'lybtug-51obj000000000000000000241000falseobj000000000000000000242024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.299 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.299 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.299 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.299 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.299 DEBUG:boto:Method: GET 2024-04-16T18:18:17.299 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000024 2024-04-16T18:18:17.299 DEBUG:boto:Data: 2024-04-16T18:18:17.299 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.299 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.300 DEBUG:boto:Params: {} 2024-04-16T18:18:17.300 DEBUG:boto:Token: None 2024-04-16T18:18:17.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.309 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cdb8334b3981982e-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.309 DEBUG:boto:b'lybtug-51obj000000000000000000241000falseobj000000000000000000242024-04-16T18:18:01.167Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.310 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000025 2024-04-16T18:18:17.310 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.310 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.310 DEBUG:boto:path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.310 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.310 DEBUG:boto:Method: GET 2024-04-16T18:18:17.310 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.310 DEBUG:boto:Data: 2024-04-16T18:18:17.311 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.311 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.311 DEBUG:boto:Params: {} 2024-04-16T18:18:17.311 DEBUG:boto:Token: None 2024-04-16T18:18:17.311 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.311 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs= 2024-04-16T18:18:17.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs=', 'Content-Length': '0'} 2024-04-16T18:18:17.314 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af4575df3567f06e-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.358 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.358 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.358 DEBUG:boto:path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.358 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.359 DEBUG:boto:Method: GET 2024-04-16T18:18:17.359 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.359 DEBUG:boto:Data: 2024-04-16T18:18:17.359 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.359 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.359 DEBUG:boto:Params: {} 2024-04-16T18:18:17.359 DEBUG:boto:Token: None 2024-04-16T18:18:17.359 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.359 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs= 2024-04-16T18:18:17.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs=', 'Content-Length': '0'} 2024-04-16T18:18:17.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e326acc273afc3e-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.363 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.363 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.363 DEBUG:boto:path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.363 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.363 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.363 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.363 DEBUG:boto:Data: 2024-04-16T18:18:17.363 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.363 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.363 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.364 DEBUG:boto:Params: {} 2024-04-16T18:18:17.364 DEBUG:boto:Token: None 2024-04-16T18:18:17.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4dT+Yqs3F9BH682e5fksrex8Fw4= 2024-04-16T18:18:17.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4dT+Yqs3F9BH682e5fksrex8Fw4=', 'Content-Length': '0'} 2024-04-16T18:18:17.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a31a88bfb9a83a58-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.367 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.367 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.367 DEBUG:boto:path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.367 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.367 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.367 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.367 DEBUG:boto:Data: 2024-04-16T18:18:17.367 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.367 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.368 DEBUG:boto:Params: {} 2024-04-16T18:18:17.368 DEBUG:boto:Token: None 2024-04-16T18:18:17.368 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4dT+Yqs3F9BH682e5fksrex8Fw4= 2024-04-16T18:18:17.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4dT+Yqs3F9BH682e5fksrex8Fw4=', 'Content-Length': '0'} 2024-04-16T18:18:17.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bfb4f16701f276b-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.371 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000025 2024-04-16T18:18:17.371 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.371 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.371 DEBUG:boto:path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.371 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.372 DEBUG:boto:Method: GET 2024-04-16T18:18:17.372 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.372 DEBUG:boto:Data: 2024-04-16T18:18:17.372 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.372 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.372 DEBUG:boto:Params: {} 2024-04-16T18:18:17.372 DEBUG:boto:Token: None 2024-04-16T18:18:17.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs= 2024-04-16T18:18:17.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs=', 'Content-Length': '0'} 2024-04-16T18:18:17.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6826c42df1c0ab4-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.418 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.418 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.418 DEBUG:boto:path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.419 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.419 DEBUG:boto:Method: GET 2024-04-16T18:18:17.419 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.419 DEBUG:boto:Data: 2024-04-16T18:18:17.419 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.419 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.419 DEBUG:boto:Params: {} 2024-04-16T18:18:17.419 DEBUG:boto:Token: None 2024-04-16T18:18:17.419 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025?versionId=null 2024-04-16T18:18:17.419 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs= 2024-04-16T18:18:17.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlkXoO+bnCj6gT+iohVgVI8NPFs=', 'Content-Length': '0'} 2024-04-16T18:18:17.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000219149ac0cb2f572-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.423 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.423 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.423 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.423 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.423 DEBUG:boto:Method: GET 2024-04-16T18:18:17.423 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.423 DEBUG:boto:Data: 2024-04-16T18:18:17.425 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.425 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.425 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.425 DEBUG:boto:Params: {} 2024-04-16T18:18:17.426 DEBUG:boto:Token: None 2024-04-16T18:18:17.426 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.426 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9fca5410b9bda58-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.430 DEBUG:boto:b'lybtug-51obj000000000000000000251000falseobj000000000000000000252024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.431 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.431 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.431 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.431 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.431 DEBUG:boto:Method: GET 2024-04-16T18:18:17.431 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.431 DEBUG:boto:Data: 2024-04-16T18:18:17.431 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.431 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.431 DEBUG:boto:Params: {} 2024-04-16T18:18:17.431 DEBUG:boto:Token: None 2024-04-16T18:18:17.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.438 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075d0b35eda1f378d-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.478 DEBUG:boto:b'lybtug-51obj000000000000000000251000falseobj000000000000000000252024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.478 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.479 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.479 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.479 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.479 DEBUG:boto:Data: 2024-04-16T18:18:17.479 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.479 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.479 DEBUG:boto:Params: {} 2024-04-16T18:18:17.479 DEBUG:boto:Token: None 2024-04-16T18:18:17.479 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OPxS1K6ln4gDMRWrhdmIfguVDr0= 2024-04-16T18:18:17.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OPxS1K6ln4gDMRWrhdmIfguVDr0=', 'Content-Length': '0'} 2024-04-16T18:18:17.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac9c33855386f355-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.482 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.483 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.483 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.483 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.483 DEBUG:boto:Data: 2024-04-16T18:18:17.483 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.483 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.483 DEBUG:boto:Params: {} 2024-04-16T18:18:17.483 DEBUG:boto:Token: None 2024-04-16T18:18:17.484 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OPxS1K6ln4gDMRWrhdmIfguVDr0= 2024-04-16T18:18:17.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OPxS1K6ln4gDMRWrhdmIfguVDr0=', 'Content-Length': '0'} 2024-04-16T18:18:17.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef7088a4cc77b66f-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.487 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000025 2024-04-16T18:18:17.487 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.487 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.487 DEBUG:boto:Method: GET 2024-04-16T18:18:17.487 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.487 DEBUG:boto:Data: 2024-04-16T18:18:17.487 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.487 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.487 DEBUG:boto:Params: {} 2024-04-16T18:18:17.488 DEBUG:boto:Token: None 2024-04-16T18:18:17.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CmxoQa4Ayh5QRI2hioy0Zl/SAMk= 2024-04-16T18:18:17.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CmxoQa4Ayh5QRI2hioy0Zl/SAMk=', 'Content-Length': '0'} 2024-04-16T18:18:17.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000620956ed1e30144f-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.534 DEBUG:boto:path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.535 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.535 DEBUG:boto:Method: GET 2024-04-16T18:18:17.535 DEBUG:boto:Path: /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.535 DEBUG:boto:Data: 2024-04-16T18:18:17.535 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.535 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.535 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.535 DEBUG:boto:Params: {} 2024-04-16T18:18:17.535 DEBUG:boto:Token: None 2024-04-16T18:18:17.535 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000025 2024-04-16T18:18:17.535 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CmxoQa4Ayh5QRI2hioy0Zl/SAMk= 2024-04-16T18:18:17.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CmxoQa4Ayh5QRI2hioy0Zl/SAMk=', 'Content-Length': '0'} 2024-04-16T18:18:17.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007fa004bbc6b0227-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.539 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.539 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.539 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.539 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.539 DEBUG:boto:Method: GET 2024-04-16T18:18:17.539 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.539 DEBUG:boto:Data: 2024-04-16T18:18:17.539 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.539 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.540 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.540 DEBUG:boto:Params: {} 2024-04-16T18:18:17.540 DEBUG:boto:Token: None 2024-04-16T18:18:17.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.544 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000828143d05f135b87-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.586 DEBUG:boto:b'lybtug-51obj000000000000000000251000falseobj000000000000000000252024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.587 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.587 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.587 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.587 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.587 DEBUG:boto:Method: GET 2024-04-16T18:18:17.587 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000025 2024-04-16T18:18:17.587 DEBUG:boto:Data: 2024-04-16T18:18:17.587 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.587 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.587 DEBUG:boto:Params: {} 2024-04-16T18:18:17.588 DEBUG:boto:Token: None 2024-04-16T18:18:17.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.594 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081b741e645b40f14-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.594 DEBUG:boto:b'lybtug-51obj000000000000000000251000falseobj000000000000000000252024-04-16T18:17:57.520Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.594 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000026 2024-04-16T18:18:17.594 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.595 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.595 DEBUG:boto:path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.595 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.595 DEBUG:boto:Method: GET 2024-04-16T18:18:17.595 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.595 DEBUG:boto:Data: 2024-04-16T18:18:17.595 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.595 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.595 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.595 DEBUG:boto:Params: {} 2024-04-16T18:18:17.596 DEBUG:boto:Token: None 2024-04-16T18:18:17.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q= 2024-04-16T18:18:17.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q=', 'Content-Length': '0'} 2024-04-16T18:18:17.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b8bc37f5b721571-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.642 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.642 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.643 DEBUG:boto:path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.643 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.643 DEBUG:boto:Method: GET 2024-04-16T18:18:17.643 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.643 DEBUG:boto:Data: 2024-04-16T18:18:17.643 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.643 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.643 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.643 DEBUG:boto:Params: {} 2024-04-16T18:18:17.643 DEBUG:boto:Token: None 2024-04-16T18:18:17.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q= 2024-04-16T18:18:17.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q=', 'Content-Length': '0'} 2024-04-16T18:18:17.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e42c74f0b4cc32d-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.647 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.647 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.647 DEBUG:boto:path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.647 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.648 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.648 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.648 DEBUG:boto:Data: 2024-04-16T18:18:17.648 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.648 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.648 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.648 DEBUG:boto:Params: {} 2024-04-16T18:18:17.648 DEBUG:boto:Token: None 2024-04-16T18:18:17.648 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IoHoWptKkP+rLTkjPU5fB3BgxbI= 2024-04-16T18:18:17.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IoHoWptKkP+rLTkjPU5fB3BgxbI=', 'Content-Length': '0'} 2024-04-16T18:18:17.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b732cbf3f30f99a-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.651 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.652 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.652 DEBUG:boto:path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.652 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.652 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.652 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.652 DEBUG:boto:Data: 2024-04-16T18:18:17.652 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.652 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.652 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.652 DEBUG:boto:Params: {} 2024-04-16T18:18:17.652 DEBUG:boto:Token: None 2024-04-16T18:18:17.653 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.653 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IoHoWptKkP+rLTkjPU5fB3BgxbI= 2024-04-16T18:18:17.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IoHoWptKkP+rLTkjPU5fB3BgxbI=', 'Content-Length': '0'} 2024-04-16T18:18:17.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edf2b64f7b7727b0-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.656 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000026 2024-04-16T18:18:17.656 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.656 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.656 DEBUG:boto:path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.656 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.656 DEBUG:boto:Method: GET 2024-04-16T18:18:17.656 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.656 DEBUG:boto:Data: 2024-04-16T18:18:17.656 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.656 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.656 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.657 DEBUG:boto:Params: {} 2024-04-16T18:18:17.657 DEBUG:boto:Token: None 2024-04-16T18:18:17.657 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q= 2024-04-16T18:18:17.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q=', 'Content-Length': '0'} 2024-04-16T18:18:17.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026953282fffa4dbd-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.702 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.702 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.702 DEBUG:boto:path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.703 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.703 DEBUG:boto:Method: GET 2024-04-16T18:18:17.703 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.703 DEBUG:boto:Data: 2024-04-16T18:18:17.703 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.703 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.703 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.703 DEBUG:boto:Params: {} 2024-04-16T18:18:17.703 DEBUG:boto:Token: None 2024-04-16T18:18:17.703 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026?versionId=null 2024-04-16T18:18:17.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q= 2024-04-16T18:18:17.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dfVxzA/QVXEjO2JtxrsGUJfN/0Q=', 'Content-Length': '0'} 2024-04-16T18:18:17.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e780fc4aec670e47-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.707 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.707 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.707 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.707 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.707 DEBUG:boto:Method: GET 2024-04-16T18:18:17.707 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.707 DEBUG:boto:Data: 2024-04-16T18:18:17.708 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.708 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.708 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.708 DEBUG:boto:Params: {} 2024-04-16T18:18:17.708 DEBUG:boto:Token: None 2024-04-16T18:18:17.708 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f47aa29cecad254a-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.712 DEBUG:boto:b'lybtug-51obj000000000000000000261000falseobj000000000000000000262024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.713 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.713 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.713 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.713 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.713 DEBUG:boto:Method: GET 2024-04-16T18:18:17.713 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.713 DEBUG:boto:Data: 2024-04-16T18:18:17.713 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.714 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.714 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.714 DEBUG:boto:Params: {} 2024-04-16T18:18:17.714 DEBUG:boto:Token: None 2024-04-16T18:18:17.714 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.714 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8de77fcc7b7f0f3-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.762 DEBUG:boto:b'lybtug-51obj000000000000000000261000falseobj000000000000000000262024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.824 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.824 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.825 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.825 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.825 DEBUG:boto:Data: 2024-04-16T18:18:17.825 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.825 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.825 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.825 DEBUG:boto:Params: {} 2024-04-16T18:18:17.825 DEBUG:boto:Token: None 2024-04-16T18:18:17.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.826 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4nTfv9W2TnFRcZpy9hlSu29Aod0= 2024-04-16T18:18:17.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4nTfv9W2TnFRcZpy9hlSu29Aod0=', 'Content-Length': '0'} 2024-04-16T18:18:17.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7ff95dc999484df-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.829 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.829 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.829 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.829 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.829 DEBUG:boto:Data: 2024-04-16T18:18:17.829 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.829 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.829 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.830 DEBUG:boto:Params: {} 2024-04-16T18:18:17.830 DEBUG:boto:Token: None 2024-04-16T18:18:17.830 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.830 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4nTfv9W2TnFRcZpy9hlSu29Aod0= 2024-04-16T18:18:17.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4nTfv9W2TnFRcZpy9hlSu29Aod0=', 'Content-Length': '0'} 2024-04-16T18:18:17.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033d450672084ab3b-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.834 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000026 2024-04-16T18:18:17.834 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.834 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.834 DEBUG:boto:Method: GET 2024-04-16T18:18:17.834 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.834 DEBUG:boto:Data: 2024-04-16T18:18:17.834 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.834 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.835 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.835 DEBUG:boto:Params: {} 2024-04-16T18:18:17.835 DEBUG:boto:Token: None 2024-04-16T18:18:17.835 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TJlqxZR3Y4Pl5VbVk9Oe/3yTio4= 2024-04-16T18:18:17.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TJlqxZR3Y4Pl5VbVk9Oe/3yTio4=', 'Content-Length': '0'} 2024-04-16T18:18:17.838 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074f884e67bda1e5a-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.878 DEBUG:boto:path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.878 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.879 DEBUG:boto:Method: GET 2024-04-16T18:18:17.879 DEBUG:boto:Path: /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.879 DEBUG:boto:Data: 2024-04-16T18:18:17.879 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.879 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.879 DEBUG:boto:Params: {} 2024-04-16T18:18:17.879 DEBUG:boto:Token: None 2024-04-16T18:18:17.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000026 2024-04-16T18:18:17.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TJlqxZR3Y4Pl5VbVk9Oe/3yTio4= 2024-04-16T18:18:17.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TJlqxZR3Y4Pl5VbVk9Oe/3yTio4=', 'Content-Length': '0'} 2024-04-16T18:18:17.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e33a1c8c6b816bc-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.883 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.883 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.883 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.883 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.883 DEBUG:boto:Method: GET 2024-04-16T18:18:17.883 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.883 DEBUG:boto:Data: 2024-04-16T18:18:17.884 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.884 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.884 DEBUG:boto:Params: {} 2024-04-16T18:18:17.884 DEBUG:boto:Token: None 2024-04-16T18:18:17.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026c891145837d6db-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.889 DEBUG:boto:b'lybtug-51obj000000000000000000261000falseobj000000000000000000262024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.889 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:17.889 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:17.889 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.890 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.890 DEBUG:boto:Method: GET 2024-04-16T18:18:17.890 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000026 2024-04-16T18:18:17.890 DEBUG:boto:Data: 2024-04-16T18:18:17.890 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.890 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.890 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.891 DEBUG:boto:Params: {} 2024-04-16T18:18:17.891 DEBUG:boto:Token: None 2024-04-16T18:18:17.891 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/ 2024-04-16T18:18:17.891 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8= 2024-04-16T18:18:17.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6zNm/C+Mlha6tJFDKwXt/jiBVL8=', 'Content-Length': '0'} 2024-04-16T18:18:17.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028918a7533ec91ba-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.897 DEBUG:boto:b'lybtug-51obj000000000000000000261000falseobj000000000000000000262024-04-16T18:18:00.338Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:17.898 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000027 2024-04-16T18:18:17.898 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.898 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.898 DEBUG:boto:path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.898 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.898 DEBUG:boto:Method: GET 2024-04-16T18:18:17.898 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.899 DEBUG:boto:Data: 2024-04-16T18:18:17.899 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.899 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.899 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.899 DEBUG:boto:Params: {} 2024-04-16T18:18:17.899 DEBUG:boto:Token: None 2024-04-16T18:18:17.899 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.899 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jM9nRlMjUbIr5Ms67IQyxy/50/4= 2024-04-16T18:18:17.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jM9nRlMjUbIr5Ms67IQyxy/50/4=', 'Content-Length': '0'} 2024-04-16T18:18:17.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022e0243a4754cdd5-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.946 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.946 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.947 DEBUG:boto:path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.947 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.947 DEBUG:boto:Method: GET 2024-04-16T18:18:17.947 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.947 DEBUG:boto:Data: 2024-04-16T18:18:17.947 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.947 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.947 DEBUG:boto:Params: {} 2024-04-16T18:18:17.947 DEBUG:boto:Token: None 2024-04-16T18:18:17.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jM9nRlMjUbIr5Ms67IQyxy/50/4= 2024-04-16T18:18:17.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jM9nRlMjUbIr5Ms67IQyxy/50/4=', 'Content-Length': '0'} 2024-04-16T18:18:17.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce64cb2dbd1e67d8-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.951 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.951 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.951 DEBUG:boto:Data: 2024-04-16T18:18:17.952 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.952 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.952 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.952 DEBUG:boto:Params: {} 2024-04-16T18:18:17.952 DEBUG:boto:Token: None 2024-04-16T18:18:17.952 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MoFDlXa/gPRczogGmuPzj7TPJ3s= 2024-04-16T18:18:17.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MoFDlXa/gPRczogGmuPzj7TPJ3s=', 'Content-Length': '0'} 2024-04-16T18:18:17.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014b3e04a07c50a46-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.955 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.955 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.955 DEBUG:boto:path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.955 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.956 DEBUG:boto:Method: HEAD 2024-04-16T18:18:17.956 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.956 DEBUG:boto:Data: 2024-04-16T18:18:17.956 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.956 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:17.956 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.956 DEBUG:boto:Params: {} 2024-04-16T18:18:17.956 DEBUG:boto:Token: None 2024-04-16T18:18:17.957 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.957 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MoFDlXa/gPRczogGmuPzj7TPJ3s= 2024-04-16T18:18:17.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MoFDlXa/gPRczogGmuPzj7TPJ3s=', 'Content-Length': '0'} 2024-04-16T18:18:17.960 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043edb7d047c7de65-00661ec0e9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:17.960 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000027 2024-04-16T18:18:17.961 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.961 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:17.961 DEBUG:boto:path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.961 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.961 DEBUG:boto:Method: GET 2024-04-16T18:18:17.961 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.961 DEBUG:boto:Data: 2024-04-16T18:18:17.961 DEBUG:boto:Headers: {} 2024-04-16T18:18:17.961 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:17.962 DEBUG:boto:Port: 8000 2024-04-16T18:18:17.962 DEBUG:boto:Params: {} 2024-04-16T18:18:17.962 DEBUG:boto:Token: None 2024-04-16T18:18:17.962 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:17 GMT /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:17.962 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jM9nRlMjUbIr5Ms67IQyxy/50/4= 2024-04-16T18:18:17.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jM9nRlMjUbIr5Ms67IQyxy/50/4=', 'Content-Length': '0'} 2024-04-16T18:18:17.965 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0f37ccfbf444d6a-00661ec0e9-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.006 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.006 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.007 DEBUG:boto:path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:18.007 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:18.007 DEBUG:boto:Method: GET 2024-04-16T18:18:18.007 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:18.007 DEBUG:boto:Data: 2024-04-16T18:18:18.007 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.007 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.007 DEBUG:boto:Params: {} 2024-04-16T18:18:18.007 DEBUG:boto:Token: None 2024-04-16T18:18:18.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000027?versionId=null 2024-04-16T18:18:18.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DNJgxD+Rf+0nzH+ot44HsvTs0ws= 2024-04-16T18:18:18.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DNJgxD+Rf+0nzH+ot44HsvTs0ws=', 'Content-Length': '0'} 2024-04-16T18:18:18.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000238f7eafb54f1500-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.011 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.011 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.011 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.011 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.011 DEBUG:boto:Method: GET 2024-04-16T18:18:18.011 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.011 DEBUG:boto:Data: 2024-04-16T18:18:18.011 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.012 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.012 DEBUG:boto:Params: {} 2024-04-16T18:18:18.012 DEBUG:boto:Token: None 2024-04-16T18:18:18.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6dfffb15c9b3b1d-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.058 DEBUG:boto:b'lybtug-51obj000000000000000000271000falseobj000000000000000000272024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.059 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.059 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.060 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.060 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.060 DEBUG:boto:Method: GET 2024-04-16T18:18:18.060 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.060 DEBUG:boto:Data: 2024-04-16T18:18:18.060 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.060 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.060 DEBUG:boto:Params: {} 2024-04-16T18:18:18.060 DEBUG:boto:Token: None 2024-04-16T18:18:18.061 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.068 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000346f680caa733231-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.068 DEBUG:boto:b'lybtug-51obj000000000000000000271000falseobj000000000000000000272024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.068 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.069 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.069 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.069 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.069 DEBUG:boto:Data: 2024-04-16T18:18:18.069 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.069 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.069 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.069 DEBUG:boto:Params: {} 2024-04-16T18:18:18.069 DEBUG:boto:Token: None 2024-04-16T18:18:18.069 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.070 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/ofDWONDwpzRyyFKueY3S/6VZ8= 2024-04-16T18:18:18.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/ofDWONDwpzRyyFKueY3S/6VZ8=', 'Content-Length': '0'} 2024-04-16T18:18:18.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084581f9359a3acc8-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.073 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.073 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.073 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.073 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.073 DEBUG:boto:Data: 2024-04-16T18:18:18.073 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.073 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.073 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.074 DEBUG:boto:Params: {} 2024-04-16T18:18:18.074 DEBUG:boto:Token: None 2024-04-16T18:18:18.074 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.074 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z/ofDWONDwpzRyyFKueY3S/6VZ8= 2024-04-16T18:18:18.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z/ofDWONDwpzRyyFKueY3S/6VZ8=', 'Content-Length': '0'} 2024-04-16T18:18:18.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c68f24e725463bee-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.077 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000027 2024-04-16T18:18:18.077 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.077 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.077 DEBUG:boto:Method: GET 2024-04-16T18:18:18.077 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.077 DEBUG:boto:Data: 2024-04-16T18:18:18.077 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.078 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.078 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.078 DEBUG:boto:Params: {} 2024-04-16T18:18:18.078 DEBUG:boto:Token: None 2024-04-16T18:18:18.078 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.078 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3EVFCw+2+XqOsjHiQCEbZJxYgPg= 2024-04-16T18:18:18.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3EVFCw+2+XqOsjHiQCEbZJxYgPg=', 'Content-Length': '0'} 2024-04-16T18:18:18.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041c4f943ce7ee211-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.122 DEBUG:boto:path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.122 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.123 DEBUG:boto:Method: GET 2024-04-16T18:18:18.123 DEBUG:boto:Path: /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.123 DEBUG:boto:Data: 2024-04-16T18:18:18.123 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.123 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.123 DEBUG:boto:Params: {} 2024-04-16T18:18:18.123 DEBUG:boto:Token: None 2024-04-16T18:18:18.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000027 2024-04-16T18:18:18.123 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3EVFCw+2+XqOsjHiQCEbZJxYgPg= 2024-04-16T18:18:18.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3EVFCw+2+XqOsjHiQCEbZJxYgPg=', 'Content-Length': '0'} 2024-04-16T18:18:18.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ef5c34db5fcc68c-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.127 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.127 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.127 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.127 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.127 DEBUG:boto:Method: GET 2024-04-16T18:18:18.127 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.127 DEBUG:boto:Data: 2024-04-16T18:18:18.127 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.128 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.128 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.128 DEBUG:boto:Params: {} 2024-04-16T18:18:18.128 DEBUG:boto:Token: None 2024-04-16T18:18:18.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002dc8152c1eac54f9-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.174 DEBUG:boto:b'lybtug-51obj000000000000000000271000falseobj000000000000000000272024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.175 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.175 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.175 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.175 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.175 DEBUG:boto:Method: GET 2024-04-16T18:18:18.175 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000027 2024-04-16T18:18:18.175 DEBUG:boto:Data: 2024-04-16T18:18:18.175 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.176 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.176 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.176 DEBUG:boto:Params: {} 2024-04-16T18:18:18.176 DEBUG:boto:Token: None 2024-04-16T18:18:18.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.182 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090f2db59ac749497-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.182 DEBUG:boto:b'lybtug-51obj000000000000000000271000falseobj000000000000000000272024-04-16T18:18:00.646Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.183 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000028 2024-04-16T18:18:18.183 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.183 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.183 DEBUG:boto:path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.183 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.183 DEBUG:boto:Method: GET 2024-04-16T18:18:18.183 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.183 DEBUG:boto:Data: 2024-04-16T18:18:18.183 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.184 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.184 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.184 DEBUG:boto:Params: {} 2024-04-16T18:18:18.184 DEBUG:boto:Token: None 2024-04-16T18:18:18.184 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc= 2024-04-16T18:18:18.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc=', 'Content-Length': '0'} 2024-04-16T18:18:18.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8244f792f2c9da2-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.230 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.231 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.231 DEBUG:boto:path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.231 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.231 DEBUG:boto:Method: GET 2024-04-16T18:18:18.231 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.231 DEBUG:boto:Data: 2024-04-16T18:18:18.231 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.231 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.231 DEBUG:boto:Params: {} 2024-04-16T18:18:18.231 DEBUG:boto:Token: None 2024-04-16T18:18:18.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc= 2024-04-16T18:18:18.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc=', 'Content-Length': '0'} 2024-04-16T18:18:18.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070ef9c4b423f8c69-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.235 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.235 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.235 DEBUG:boto:path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.235 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.236 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.236 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.236 DEBUG:boto:Data: 2024-04-16T18:18:18.236 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.236 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.236 DEBUG:boto:Params: {} 2024-04-16T18:18:18.236 DEBUG:boto:Token: None 2024-04-16T18:18:18.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dDmRqmkyf2uNfiA6HggfTTlom34= 2024-04-16T18:18:18.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dDmRqmkyf2uNfiA6HggfTTlom34=', 'Content-Length': '0'} 2024-04-16T18:18:18.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004119c05ffee4051f-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.241 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.241 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.242 DEBUG:boto:path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.242 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.242 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.242 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.242 DEBUG:boto:Data: 2024-04-16T18:18:18.242 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.242 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.242 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.242 DEBUG:boto:Params: {} 2024-04-16T18:18:18.242 DEBUG:boto:Token: None 2024-04-16T18:18:18.243 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.243 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dDmRqmkyf2uNfiA6HggfTTlom34= 2024-04-16T18:18:18.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dDmRqmkyf2uNfiA6HggfTTlom34=', 'Content-Length': '0'} 2024-04-16T18:18:18.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba50d3dab7c91a9b-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000028 2024-04-16T18:18:18.246 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.246 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.246 DEBUG:boto:path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.246 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.246 DEBUG:boto:Method: GET 2024-04-16T18:18:18.246 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.247 DEBUG:boto:Data: 2024-04-16T18:18:18.247 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.247 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.247 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.247 DEBUG:boto:Params: {} 2024-04-16T18:18:18.247 DEBUG:boto:Token: None 2024-04-16T18:18:18.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.247 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc= 2024-04-16T18:18:18.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc=', 'Content-Length': '0'} 2024-04-16T18:18:18.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2c8a282392a33bd-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.294 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.295 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.295 DEBUG:boto:path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.295 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.295 DEBUG:boto:Method: GET 2024-04-16T18:18:18.295 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.295 DEBUG:boto:Data: 2024-04-16T18:18:18.295 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.295 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.295 DEBUG:boto:Params: {} 2024-04-16T18:18:18.295 DEBUG:boto:Token: None 2024-04-16T18:18:18.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028?versionId=null 2024-04-16T18:18:18.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc= 2024-04-16T18:18:18.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AXmTK+vuTneV1KfmqT3n0aj+Irc=', 'Content-Length': '0'} 2024-04-16T18:18:18.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002286cb1974e526ef-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.299 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.299 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.299 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.299 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.300 DEBUG:boto:Method: GET 2024-04-16T18:18:18.300 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.300 DEBUG:boto:Data: 2024-04-16T18:18:18.300 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.300 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.300 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.300 DEBUG:boto:Params: {} 2024-04-16T18:18:18.300 DEBUG:boto:Token: None 2024-04-16T18:18:18.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.305 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002941f363ac9cf2e0-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.305 DEBUG:boto:b'lybtug-51obj000000000000000000281000falseobj000000000000000000282024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.305 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.306 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.306 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.306 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.306 DEBUG:boto:Method: GET 2024-04-16T18:18:18.306 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.306 DEBUG:boto:Data: 2024-04-16T18:18:18.306 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.306 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.306 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.306 DEBUG:boto:Params: {} 2024-04-16T18:18:18.307 DEBUG:boto:Token: None 2024-04-16T18:18:18.307 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.307 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.313 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022f940e6a8435d4c-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.313 DEBUG:boto:b'lybtug-51obj000000000000000000281000falseobj000000000000000000282024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.314 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.314 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.314 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.314 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.314 DEBUG:boto:Data: 2024-04-16T18:18:18.314 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.314 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.314 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.314 DEBUG:boto:Params: {} 2024-04-16T18:18:18.315 DEBUG:boto:Token: None 2024-04-16T18:18:18.315 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.315 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:onEWEds+pg7//N45K9XrxAqwPZU= 2024-04-16T18:18:18.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:onEWEds+pg7//N45K9XrxAqwPZU=', 'Content-Length': '0'} 2024-04-16T18:18:18.318 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c2741ab19f3ca8b-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.318 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.318 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.318 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.318 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.318 DEBUG:boto:Data: 2024-04-16T18:18:18.319 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.319 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.319 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.319 DEBUG:boto:Params: {} 2024-04-16T18:18:18.319 DEBUG:boto:Token: None 2024-04-16T18:18:18.319 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.319 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:onEWEds+pg7//N45K9XrxAqwPZU= 2024-04-16T18:18:18.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:onEWEds+pg7//N45K9XrxAqwPZU=', 'Content-Length': '0'} 2024-04-16T18:18:18.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026a49f1468fe245c-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.322 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000028 2024-04-16T18:18:18.322 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.323 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.323 DEBUG:boto:Method: GET 2024-04-16T18:18:18.323 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.323 DEBUG:boto:Data: 2024-04-16T18:18:18.323 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.323 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.323 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.323 DEBUG:boto:Params: {} 2024-04-16T18:18:18.323 DEBUG:boto:Token: None 2024-04-16T18:18:18.323 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:392OPzgeZdQ7xMjO1mynHqMUf9A= 2024-04-16T18:18:18.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:392OPzgeZdQ7xMjO1mynHqMUf9A=', 'Content-Length': '0'} 2024-04-16T18:18:18.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d6e8cd4b4072b6a-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.370 DEBUG:boto:path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.370 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.371 DEBUG:boto:Method: GET 2024-04-16T18:18:18.371 DEBUG:boto:Path: /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.371 DEBUG:boto:Data: 2024-04-16T18:18:18.371 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.371 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.371 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.371 DEBUG:boto:Params: {} 2024-04-16T18:18:18.371 DEBUG:boto:Token: None 2024-04-16T18:18:18.371 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000028 2024-04-16T18:18:18.371 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:392OPzgeZdQ7xMjO1mynHqMUf9A= 2024-04-16T18:18:18.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:392OPzgeZdQ7xMjO1mynHqMUf9A=', 'Content-Length': '0'} 2024-04-16T18:18:18.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019e24fb9ca3795f7-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.375 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.375 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.375 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.375 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.375 DEBUG:boto:Method: GET 2024-04-16T18:18:18.375 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.376 DEBUG:boto:Data: 2024-04-16T18:18:18.376 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.376 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.376 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.376 DEBUG:boto:Params: {} 2024-04-16T18:18:18.376 DEBUG:boto:Token: None 2024-04-16T18:18:18.376 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.381 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1a819793fa40193-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.422 DEBUG:boto:b'lybtug-51obj000000000000000000281000falseobj000000000000000000282024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.423 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.423 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.423 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.423 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.423 DEBUG:boto:Method: GET 2024-04-16T18:18:18.423 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000028 2024-04-16T18:18:18.423 DEBUG:boto:Data: 2024-04-16T18:18:18.423 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.424 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.424 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.424 DEBUG:boto:Params: {} 2024-04-16T18:18:18.424 DEBUG:boto:Token: None 2024-04-16T18:18:18.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a914ba687b7751f-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.431 DEBUG:boto:b'lybtug-51obj000000000000000000281000falseobj000000000000000000282024-04-16T18:17:53.925Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.431 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000029 2024-04-16T18:18:18.431 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.431 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.431 DEBUG:boto:path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.431 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.432 DEBUG:boto:Method: GET 2024-04-16T18:18:18.432 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.432 DEBUG:boto:Data: 2024-04-16T18:18:18.432 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.432 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.432 DEBUG:boto:Params: {} 2024-04-16T18:18:18.432 DEBUG:boto:Token: None 2024-04-16T18:18:18.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo= 2024-04-16T18:18:18.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo=', 'Content-Length': '0'} 2024-04-16T18:18:18.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d59f5dfc4d374fa-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.478 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.478 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.479 DEBUG:boto:path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.479 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.479 DEBUG:boto:Method: GET 2024-04-16T18:18:18.479 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.479 DEBUG:boto:Data: 2024-04-16T18:18:18.479 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.479 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.479 DEBUG:boto:Params: {} 2024-04-16T18:18:18.479 DEBUG:boto:Token: None 2024-04-16T18:18:18.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo= 2024-04-16T18:18:18.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo=', 'Content-Length': '0'} 2024-04-16T18:18:18.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f90329ac994122eb-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.483 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.483 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.483 DEBUG:boto:path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.483 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.484 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.484 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.484 DEBUG:boto:Data: 2024-04-16T18:18:18.484 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.484 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.484 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.484 DEBUG:boto:Params: {} 2024-04-16T18:18:18.484 DEBUG:boto:Token: None 2024-04-16T18:18:18.484 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8+CGRMFePC4xJnszZwnGUGipql8= 2024-04-16T18:18:18.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8+CGRMFePC4xJnszZwnGUGipql8=', 'Content-Length': '0'} 2024-04-16T18:18:18.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cd8c071a1220422-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.488 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.488 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.488 DEBUG:boto:path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.488 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.488 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.488 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.488 DEBUG:boto:Data: 2024-04-16T18:18:18.489 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.489 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.489 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.489 DEBUG:boto:Params: {} 2024-04-16T18:18:18.489 DEBUG:boto:Token: None 2024-04-16T18:18:18.489 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8+CGRMFePC4xJnszZwnGUGipql8= 2024-04-16T18:18:18.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8+CGRMFePC4xJnszZwnGUGipql8=', 'Content-Length': '0'} 2024-04-16T18:18:18.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076035f9256e35924-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.492 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000029 2024-04-16T18:18:18.493 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.493 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.493 DEBUG:boto:path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.493 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.493 DEBUG:boto:Method: GET 2024-04-16T18:18:18.493 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.493 DEBUG:boto:Data: 2024-04-16T18:18:18.493 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.493 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.493 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.493 DEBUG:boto:Params: {} 2024-04-16T18:18:18.494 DEBUG:boto:Token: None 2024-04-16T18:18:18.494 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.494 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo= 2024-04-16T18:18:18.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo=', 'Content-Length': '0'} 2024-04-16T18:18:18.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005466bfed79eb6052-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.538 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.538 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.539 DEBUG:boto:path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.539 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.539 DEBUG:boto:Method: GET 2024-04-16T18:18:18.539 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.539 DEBUG:boto:Data: 2024-04-16T18:18:18.539 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.539 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.539 DEBUG:boto:Params: {} 2024-04-16T18:18:18.539 DEBUG:boto:Token: None 2024-04-16T18:18:18.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029?versionId=null 2024-04-16T18:18:18.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo= 2024-04-16T18:18:18.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/2lPOYPuAHdROMEu08nHe7Awzo=', 'Content-Length': '0'} 2024-04-16T18:18:18.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b29cacabcdadd31-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.543 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.543 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.543 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.543 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.543 DEBUG:boto:Method: GET 2024-04-16T18:18:18.543 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.544 DEBUG:boto:Data: 2024-04-16T18:18:18.544 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.544 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.544 DEBUG:boto:Params: {} 2024-04-16T18:18:18.544 DEBUG:boto:Token: None 2024-04-16T18:18:18.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006eb3995c67666408-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.549 DEBUG:boto:b'lybtug-51obj000000000000000000291000falseobj000000000000000000292024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.549 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.550 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.550 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.550 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.550 DEBUG:boto:Method: GET 2024-04-16T18:18:18.550 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.550 DEBUG:boto:Data: 2024-04-16T18:18:18.551 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.551 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.551 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.551 DEBUG:boto:Params: {} 2024-04-16T18:18:18.551 DEBUG:boto:Token: None 2024-04-16T18:18:18.551 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ab526692b05d683-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.558 DEBUG:boto:b'lybtug-51obj000000000000000000291000falseobj000000000000000000292024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.558 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.559 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.559 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.559 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.559 DEBUG:boto:Data: 2024-04-16T18:18:18.559 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.559 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.559 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.559 DEBUG:boto:Params: {} 2024-04-16T18:18:18.559 DEBUG:boto:Token: None 2024-04-16T18:18:18.560 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vLrDRH941lLo5/ZCK4zGhaG3dVI= 2024-04-16T18:18:18.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vLrDRH941lLo5/ZCK4zGhaG3dVI=', 'Content-Length': '0'} 2024-04-16T18:18:18.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f27268fb20ce9460-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.563 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.563 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.563 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.563 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.563 DEBUG:boto:Data: 2024-04-16T18:18:18.563 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.563 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.564 DEBUG:boto:Params: {} 2024-04-16T18:18:18.564 DEBUG:boto:Token: None 2024-04-16T18:18:18.564 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vLrDRH941lLo5/ZCK4zGhaG3dVI= 2024-04-16T18:18:18.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vLrDRH941lLo5/ZCK4zGhaG3dVI=', 'Content-Length': '0'} 2024-04-16T18:18:18.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053ed77114693bc3e-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.567 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000029 2024-04-16T18:18:18.567 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.567 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.567 DEBUG:boto:Method: GET 2024-04-16T18:18:18.568 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.568 DEBUG:boto:Data: 2024-04-16T18:18:18.568 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.568 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.568 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.568 DEBUG:boto:Params: {} 2024-04-16T18:18:18.568 DEBUG:boto:Token: None 2024-04-16T18:18:18.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M2YzwhHmKybLKY5xrNkz72W1PiY= 2024-04-16T18:18:18.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M2YzwhHmKybLKY5xrNkz72W1PiY=', 'Content-Length': '0'} 2024-04-16T18:18:18.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1a2c661c49493ca-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.614 DEBUG:boto:path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.615 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.615 DEBUG:boto:Method: GET 2024-04-16T18:18:18.615 DEBUG:boto:Path: /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.615 DEBUG:boto:Data: 2024-04-16T18:18:18.615 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.615 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.615 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.615 DEBUG:boto:Params: {} 2024-04-16T18:18:18.615 DEBUG:boto:Token: None 2024-04-16T18:18:18.615 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000029 2024-04-16T18:18:18.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M2YzwhHmKybLKY5xrNkz72W1PiY= 2024-04-16T18:18:18.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M2YzwhHmKybLKY5xrNkz72W1PiY=', 'Content-Length': '0'} 2024-04-16T18:18:18.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000305b0e05927f92fc-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.619 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.619 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.619 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.619 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.619 DEBUG:boto:Method: GET 2024-04-16T18:18:18.619 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.620 DEBUG:boto:Data: 2024-04-16T18:18:18.620 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.620 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.620 DEBUG:boto:Params: {} 2024-04-16T18:18:18.620 DEBUG:boto:Token: None 2024-04-16T18:18:18.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.625 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc4d02f7d6ca9206-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.625 DEBUG:boto:b'lybtug-51obj000000000000000000291000falseobj000000000000000000292024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.625 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.625 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.625 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.625 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.626 DEBUG:boto:Method: GET 2024-04-16T18:18:18.626 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000029 2024-04-16T18:18:18.626 DEBUG:boto:Data: 2024-04-16T18:18:18.626 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.626 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.626 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.626 DEBUG:boto:Params: {} 2024-04-16T18:18:18.626 DEBUG:boto:Token: None 2024-04-16T18:18:18.626 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.626 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001891dd753e7d7f51-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.633 DEBUG:boto:b'lybtug-51obj000000000000000000291000falseobj000000000000000000292024-04-16T18:17:59.075Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.633 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000030 2024-04-16T18:18:18.634 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.634 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.634 DEBUG:boto:path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.634 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.634 DEBUG:boto:Method: GET 2024-04-16T18:18:18.634 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.634 DEBUG:boto:Data: 2024-04-16T18:18:18.634 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.634 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.634 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.635 DEBUG:boto:Params: {} 2024-04-16T18:18:18.635 DEBUG:boto:Token: None 2024-04-16T18:18:18.635 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.635 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU= 2024-04-16T18:18:18.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU=', 'Content-Length': '0'} 2024-04-16T18:18:18.638 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029d3bc98f0b77d6d-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.679 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.680 DEBUG:boto:path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.680 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.680 DEBUG:boto:Method: GET 2024-04-16T18:18:18.680 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.680 DEBUG:boto:Data: 2024-04-16T18:18:18.680 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.680 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.680 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.680 DEBUG:boto:Params: {} 2024-04-16T18:18:18.680 DEBUG:boto:Token: None 2024-04-16T18:18:18.681 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.681 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU= 2024-04-16T18:18:18.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU=', 'Content-Length': '0'} 2024-04-16T18:18:18.684 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d6d7e32c93c5ca4-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.684 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.684 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.684 DEBUG:boto:path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.684 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.685 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.685 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.685 DEBUG:boto:Data: 2024-04-16T18:18:18.685 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.685 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.685 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.685 DEBUG:boto:Params: {} 2024-04-16T18:18:18.685 DEBUG:boto:Token: None 2024-04-16T18:18:18.685 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/DD4sQkwG4HUQCqXBVnZEJzKgFg= 2024-04-16T18:18:18.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/DD4sQkwG4HUQCqXBVnZEJzKgFg=', 'Content-Length': '0'} 2024-04-16T18:18:18.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e490fb71c2977fe4-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.688 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.689 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.689 DEBUG:boto:path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.689 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.689 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.689 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.689 DEBUG:boto:Data: 2024-04-16T18:18:18.689 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.689 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.689 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.689 DEBUG:boto:Params: {} 2024-04-16T18:18:18.689 DEBUG:boto:Token: None 2024-04-16T18:18:18.690 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.690 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/DD4sQkwG4HUQCqXBVnZEJzKgFg= 2024-04-16T18:18:18.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/DD4sQkwG4HUQCqXBVnZEJzKgFg=', 'Content-Length': '0'} 2024-04-16T18:18:18.693 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9934cb584d7d881-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.693 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000030 2024-04-16T18:18:18.693 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.693 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.693 DEBUG:boto:path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.694 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.694 DEBUG:boto:Method: GET 2024-04-16T18:18:18.694 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.694 DEBUG:boto:Data: 2024-04-16T18:18:18.694 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.694 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.694 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.694 DEBUG:boto:Params: {} 2024-04-16T18:18:18.694 DEBUG:boto:Token: None 2024-04-16T18:18:18.695 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.695 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU= 2024-04-16T18:18:18.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU=', 'Content-Length': '0'} 2024-04-16T18:18:18.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc05c9adcfe32459-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.738 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.738 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.739 DEBUG:boto:path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.739 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.739 DEBUG:boto:Method: GET 2024-04-16T18:18:18.739 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.739 DEBUG:boto:Data: 2024-04-16T18:18:18.739 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.739 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.739 DEBUG:boto:Params: {} 2024-04-16T18:18:18.739 DEBUG:boto:Token: None 2024-04-16T18:18:18.739 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030?versionId=null 2024-04-16T18:18:18.739 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU= 2024-04-16T18:18:18.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QKCRrZN+TlwYN1mWCUgzXCRlfwU=', 'Content-Length': '0'} 2024-04-16T18:18:18.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7d1883c3b4aceab-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.743 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.743 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.743 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.744 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.744 DEBUG:boto:Method: GET 2024-04-16T18:18:18.744 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.744 DEBUG:boto:Data: 2024-04-16T18:18:18.744 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.744 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.744 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.744 DEBUG:boto:Params: {} 2024-04-16T18:18:18.744 DEBUG:boto:Token: None 2024-04-16T18:18:18.744 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002353866af3ee4376-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.749 DEBUG:boto:b'lybtug-51obj000000000000000000301000falseobj000000000000000000302024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.749 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.749 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.749 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.749 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.750 DEBUG:boto:Method: GET 2024-04-16T18:18:18.750 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.750 DEBUG:boto:Data: 2024-04-16T18:18:18.750 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.750 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.750 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.750 DEBUG:boto:Params: {} 2024-04-16T18:18:18.750 DEBUG:boto:Token: None 2024-04-16T18:18:18.750 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.751 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.756 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ef6703b8eac8d59-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.798 DEBUG:boto:b'lybtug-51obj000000000000000000301000falseobj000000000000000000302024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.799 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.799 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.799 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.799 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.799 DEBUG:boto:Data: 2024-04-16T18:18:18.799 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.799 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.799 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.799 DEBUG:boto:Params: {} 2024-04-16T18:18:18.799 DEBUG:boto:Token: None 2024-04-16T18:18:18.799 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AS5rbR5SRPmuMMq3MFjP36AqWho= 2024-04-16T18:18:18.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AS5rbR5SRPmuMMq3MFjP36AqWho=', 'Content-Length': '0'} 2024-04-16T18:18:18.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000528918f383672c24-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.803 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.803 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.803 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.803 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.803 DEBUG:boto:Data: 2024-04-16T18:18:18.803 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.803 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.803 DEBUG:boto:Params: {} 2024-04-16T18:18:18.804 DEBUG:boto:Token: None 2024-04-16T18:18:18.804 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AS5rbR5SRPmuMMq3MFjP36AqWho= 2024-04-16T18:18:18.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AS5rbR5SRPmuMMq3MFjP36AqWho=', 'Content-Length': '0'} 2024-04-16T18:18:18.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b21ee04149b2639-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.807 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000030 2024-04-16T18:18:18.807 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.807 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.807 DEBUG:boto:Method: GET 2024-04-16T18:18:18.807 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.807 DEBUG:boto:Data: 2024-04-16T18:18:18.807 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.807 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.808 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.808 DEBUG:boto:Params: {} 2024-04-16T18:18:18.808 DEBUG:boto:Token: None 2024-04-16T18:18:18.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mIzvnjwyWU/qmeETc42f6UQZ6wQ= 2024-04-16T18:18:18.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mIzvnjwyWU/qmeETc42f6UQZ6wQ=', 'Content-Length': '0'} 2024-04-16T18:18:18.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef457f4aba3db299-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.854 DEBUG:boto:path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.854 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.855 DEBUG:boto:Method: GET 2024-04-16T18:18:18.855 DEBUG:boto:Path: /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.855 DEBUG:boto:Data: 2024-04-16T18:18:18.855 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.855 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.855 DEBUG:boto:Params: {} 2024-04-16T18:18:18.855 DEBUG:boto:Token: None 2024-04-16T18:18:18.855 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000030 2024-04-16T18:18:18.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mIzvnjwyWU/qmeETc42f6UQZ6wQ= 2024-04-16T18:18:18.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mIzvnjwyWU/qmeETc42f6UQZ6wQ=', 'Content-Length': '0'} 2024-04-16T18:18:18.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064fa3ef5be8f8348-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.859 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.859 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.859 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.859 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.860 DEBUG:boto:Method: GET 2024-04-16T18:18:18.860 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.860 DEBUG:boto:Data: 2024-04-16T18:18:18.860 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.860 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.860 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.860 DEBUG:boto:Params: {} 2024-04-16T18:18:18.860 DEBUG:boto:Token: None 2024-04-16T18:18:18.861 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cb8a26f887ec44f-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.906 DEBUG:boto:b'lybtug-51obj000000000000000000301000falseobj000000000000000000302024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.907 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:18.907 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:18.907 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.907 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.907 DEBUG:boto:Method: GET 2024-04-16T18:18:18.907 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000030 2024-04-16T18:18:18.907 DEBUG:boto:Data: 2024-04-16T18:18:18.907 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.907 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.908 DEBUG:boto:Params: {} 2024-04-16T18:18:18.908 DEBUG:boto:Token: None 2024-04-16T18:18:18.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/ 2024-04-16T18:18:18.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II= 2024-04-16T18:18:18.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CvEYmpttq3217/QI2al/lcLI+II=', 'Content-Length': '0'} 2024-04-16T18:18:18.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a558ab0c2b4c1f4-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.914 DEBUG:boto:b'lybtug-51obj000000000000000000301000falseobj000000000000000000302024-04-16T18:18:00.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:18.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000031 2024-04-16T18:18:18.915 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.915 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.915 DEBUG:boto:path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.915 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.915 DEBUG:boto:Method: GET 2024-04-16T18:18:18.915 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.915 DEBUG:boto:Data: 2024-04-16T18:18:18.915 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.915 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.916 DEBUG:boto:Params: {} 2024-04-16T18:18:18.916 DEBUG:boto:Token: None 2024-04-16T18:18:18.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TZngnZHSfiCgG6l/mq2tA+xJ8gU= 2024-04-16T18:18:18.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TZngnZHSfiCgG6l/mq2tA+xJ8gU=', 'Content-Length': '0'} 2024-04-16T18:18:18.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000afe012d2806aaee-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.962 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.962 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.962 DEBUG:boto:path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.963 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.963 DEBUG:boto:Method: GET 2024-04-16T18:18:18.963 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.963 DEBUG:boto:Data: 2024-04-16T18:18:18.963 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.963 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.963 DEBUG:boto:Params: {} 2024-04-16T18:18:18.963 DEBUG:boto:Token: None 2024-04-16T18:18:18.963 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TZngnZHSfiCgG6l/mq2tA+xJ8gU= 2024-04-16T18:18:18.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TZngnZHSfiCgG6l/mq2tA+xJ8gU=', 'Content-Length': '0'} 2024-04-16T18:18:18.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008738763e7ad67fcf-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.967 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.967 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.967 DEBUG:boto:Data: 2024-04-16T18:18:18.967 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.967 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.968 DEBUG:boto:Params: {} 2024-04-16T18:18:18.968 DEBUG:boto:Token: None 2024-04-16T18:18:18.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hhVQ/2vm6B/igT3kzj+uKQKQVbA= 2024-04-16T18:18:18.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hhVQ/2vm6B/igT3kzj+uKQKQVbA=', 'Content-Length': '0'} 2024-04-16T18:18:18.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eaf63bd4ebcf0db7-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.971 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.971 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.972 DEBUG:boto:path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.972 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.972 DEBUG:boto:Method: HEAD 2024-04-16T18:18:18.972 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.972 DEBUG:boto:Data: 2024-04-16T18:18:18.972 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.972 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:18.972 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.972 DEBUG:boto:Params: {} 2024-04-16T18:18:18.972 DEBUG:boto:Token: None 2024-04-16T18:18:18.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hhVQ/2vm6B/igT3kzj+uKQKQVbA= 2024-04-16T18:18:18.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hhVQ/2vm6B/igT3kzj+uKQKQVbA=', 'Content-Length': '0'} 2024-04-16T18:18:18.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e560b8024c21f3fe-00661ec0ea-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:18.976 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000031 2024-04-16T18:18:18.976 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.976 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:18.976 DEBUG:boto:path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.976 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.976 DEBUG:boto:Method: GET 2024-04-16T18:18:18.976 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.977 DEBUG:boto:Data: 2024-04-16T18:18:18.977 DEBUG:boto:Headers: {} 2024-04-16T18:18:18.977 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:18.977 DEBUG:boto:Port: 8000 2024-04-16T18:18:18.977 DEBUG:boto:Params: {} 2024-04-16T18:18:18.977 DEBUG:boto:Token: None 2024-04-16T18:18:18.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:18 GMT /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:18.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TZngnZHSfiCgG6l/mq2tA+xJ8gU= 2024-04-16T18:18:18.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TZngnZHSfiCgG6l/mq2tA+xJ8gU=', 'Content-Length': '0'} 2024-04-16T18:18:18.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c4576d6b7e6cafc-00661ec0ea-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.022 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.022 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.022 DEBUG:boto:path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:19.023 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:19.023 DEBUG:boto:Method: GET 2024-04-16T18:18:19.023 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:19.023 DEBUG:boto:Data: 2024-04-16T18:18:19.023 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.023 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.023 DEBUG:boto:Params: {} 2024-04-16T18:18:19.023 DEBUG:boto:Token: None 2024-04-16T18:18:19.023 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000031?versionId=null 2024-04-16T18:18:19.023 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UH+SZkZuU6ButYqA2pWyw+T3C58= 2024-04-16T18:18:19.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UH+SZkZuU6ButYqA2pWyw+T3C58=', 'Content-Length': '0'} 2024-04-16T18:18:19.026 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000567f1d450cba2443-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.027 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.027 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.027 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.027 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.027 DEBUG:boto:Method: GET 2024-04-16T18:18:19.027 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.027 DEBUG:boto:Data: 2024-04-16T18:18:19.027 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.027 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.027 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.027 DEBUG:boto:Params: {} 2024-04-16T18:18:19.028 DEBUG:boto:Token: None 2024-04-16T18:18:19.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbf10d3a2d759061-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.032 DEBUG:boto:b'lybtug-51obj000000000000000000311000falseobj000000000000000000312024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.033 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.033 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.033 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.033 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.033 DEBUG:boto:Method: GET 2024-04-16T18:18:19.033 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.033 DEBUG:boto:Data: 2024-04-16T18:18:19.033 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.033 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.034 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.034 DEBUG:boto:Params: {} 2024-04-16T18:18:19.034 DEBUG:boto:Token: None 2024-04-16T18:18:19.034 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.034 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a25a5a492a6d495-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.082 DEBUG:boto:b'lybtug-51obj000000000000000000311000falseobj000000000000000000312024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.083 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.083 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.083 DEBUG:boto:Data: 2024-04-16T18:18:19.083 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.083 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.083 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.083 DEBUG:boto:Params: {} 2024-04-16T18:18:19.083 DEBUG:boto:Token: None 2024-04-16T18:18:19.084 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x1KPPf18vSHViMeJDDfeqShYMrU= 2024-04-16T18:18:19.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x1KPPf18vSHViMeJDDfeqShYMrU=', 'Content-Length': '0'} 2024-04-16T18:18:19.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a8a708a439c26e1-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.087 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.087 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.087 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.087 DEBUG:boto:Data: 2024-04-16T18:18:19.087 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.087 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.087 DEBUG:boto:Params: {} 2024-04-16T18:18:19.088 DEBUG:boto:Token: None 2024-04-16T18:18:19.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x1KPPf18vSHViMeJDDfeqShYMrU= 2024-04-16T18:18:19.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x1KPPf18vSHViMeJDDfeqShYMrU=', 'Content-Length': '0'} 2024-04-16T18:18:19.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a2d28340ca0871e-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000031 2024-04-16T18:18:19.091 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.091 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.091 DEBUG:boto:Method: GET 2024-04-16T18:18:19.091 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.091 DEBUG:boto:Data: 2024-04-16T18:18:19.092 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.092 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.092 DEBUG:boto:Params: {} 2024-04-16T18:18:19.092 DEBUG:boto:Token: None 2024-04-16T18:18:19.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SKKlF/6GqxcynJaah/0KrhsYaa8= 2024-04-16T18:18:19.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SKKlF/6GqxcynJaah/0KrhsYaa8=', 'Content-Length': '0'} 2024-04-16T18:18:19.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000598366459d52c4a6-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.138 DEBUG:boto:path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.138 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.139 DEBUG:boto:Method: GET 2024-04-16T18:18:19.139 DEBUG:boto:Path: /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.139 DEBUG:boto:Data: 2024-04-16T18:18:19.139 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.139 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.139 DEBUG:boto:Params: {} 2024-04-16T18:18:19.139 DEBUG:boto:Token: None 2024-04-16T18:18:19.139 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000031 2024-04-16T18:18:19.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SKKlF/6GqxcynJaah/0KrhsYaa8= 2024-04-16T18:18:19.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SKKlF/6GqxcynJaah/0KrhsYaa8=', 'Content-Length': '0'} 2024-04-16T18:18:19.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007ad4edbe773a91e-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.143 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.143 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.143 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.143 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.143 DEBUG:boto:Method: GET 2024-04-16T18:18:19.143 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.143 DEBUG:boto:Data: 2024-04-16T18:18:19.143 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.144 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.144 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.144 DEBUG:boto:Params: {} 2024-04-16T18:18:19.144 DEBUG:boto:Token: None 2024-04-16T18:18:19.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094328751b1dc7742-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.194 DEBUG:boto:b'lybtug-51obj000000000000000000311000falseobj000000000000000000312024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.195 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.195 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.195 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.195 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.195 DEBUG:boto:Method: GET 2024-04-16T18:18:19.195 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000031 2024-04-16T18:18:19.195 DEBUG:boto:Data: 2024-04-16T18:18:19.195 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.195 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.195 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.196 DEBUG:boto:Params: {} 2024-04-16T18:18:19.196 DEBUG:boto:Token: None 2024-04-16T18:18:19.196 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f77842cf92f74e74-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.202 DEBUG:boto:b'lybtug-51obj000000000000000000311000falseobj000000000000000000312024-04-16T18:17:57.064Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.202 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000032 2024-04-16T18:18:19.203 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.203 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.203 DEBUG:boto:path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.203 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.203 DEBUG:boto:Method: GET 2024-04-16T18:18:19.203 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.203 DEBUG:boto:Data: 2024-04-16T18:18:19.203 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.203 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.203 DEBUG:boto:Params: {} 2024-04-16T18:18:19.203 DEBUG:boto:Token: None 2024-04-16T18:18:19.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4= 2024-04-16T18:18:19.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4=', 'Content-Length': '0'} 2024-04-16T18:18:19.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f9bc9f1db609e9e-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.250 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.250 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.250 DEBUG:boto:path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.251 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.251 DEBUG:boto:Method: GET 2024-04-16T18:18:19.251 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.251 DEBUG:boto:Data: 2024-04-16T18:18:19.251 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.251 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.251 DEBUG:boto:Params: {} 2024-04-16T18:18:19.251 DEBUG:boto:Token: None 2024-04-16T18:18:19.251 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.251 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4= 2024-04-16T18:18:19.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4=', 'Content-Length': '0'} 2024-04-16T18:18:19.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a049c26613cf1516-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.255 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.255 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.255 DEBUG:boto:path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.255 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.255 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.255 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.255 DEBUG:boto:Data: 2024-04-16T18:18:19.255 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.255 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.255 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.255 DEBUG:boto:Params: {} 2024-04-16T18:18:19.256 DEBUG:boto:Token: None 2024-04-16T18:18:19.256 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MG8eSaGtOgy6dqK4EHLsICkkEh8= 2024-04-16T18:18:19.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MG8eSaGtOgy6dqK4EHLsICkkEh8=', 'Content-Length': '0'} 2024-04-16T18:18:19.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7f1502b87ca2887-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.261 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.262 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.262 DEBUG:boto:path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.262 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.262 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.262 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.262 DEBUG:boto:Data: 2024-04-16T18:18:19.262 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.262 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.262 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.262 DEBUG:boto:Params: {} 2024-04-16T18:18:19.263 DEBUG:boto:Token: None 2024-04-16T18:18:19.263 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MG8eSaGtOgy6dqK4EHLsICkkEh8= 2024-04-16T18:18:19.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MG8eSaGtOgy6dqK4EHLsICkkEh8=', 'Content-Length': '0'} 2024-04-16T18:18:19.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a183f837385101d-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.266 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000032 2024-04-16T18:18:19.266 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.266 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.266 DEBUG:boto:path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.266 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.266 DEBUG:boto:Method: GET 2024-04-16T18:18:19.266 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.266 DEBUG:boto:Data: 2024-04-16T18:18:19.266 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.267 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.267 DEBUG:boto:Params: {} 2024-04-16T18:18:19.267 DEBUG:boto:Token: None 2024-04-16T18:18:19.267 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.267 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4= 2024-04-16T18:18:19.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4=', 'Content-Length': '0'} 2024-04-16T18:18:19.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f70fde80898d79d-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.314 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.314 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.315 DEBUG:boto:path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.315 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.315 DEBUG:boto:Method: GET 2024-04-16T18:18:19.315 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.315 DEBUG:boto:Data: 2024-04-16T18:18:19.315 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.315 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.315 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.315 DEBUG:boto:Params: {} 2024-04-16T18:18:19.315 DEBUG:boto:Token: None 2024-04-16T18:18:19.315 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032?versionId=null 2024-04-16T18:18:19.315 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4= 2024-04-16T18:18:19.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SqGUrD45K0EVnUUZccZKRlFzxW4=', 'Content-Length': '0'} 2024-04-16T18:18:19.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e05f4dafa050fb83-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.319 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.320 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.320 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.320 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.320 DEBUG:boto:Method: GET 2024-04-16T18:18:19.320 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.320 DEBUG:boto:Data: 2024-04-16T18:18:19.320 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.320 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.320 DEBUG:boto:Params: {} 2024-04-16T18:18:19.320 DEBUG:boto:Token: None 2024-04-16T18:18:19.321 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000483a1e401e48cf2e-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.326 DEBUG:boto:b'lybtug-51obj000000000000000000321000falseobj000000000000000000322024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.326 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.326 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.326 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.326 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.326 DEBUG:boto:Method: GET 2024-04-16T18:18:19.327 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.327 DEBUG:boto:Data: 2024-04-16T18:18:19.327 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.327 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.327 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.327 DEBUG:boto:Params: {} 2024-04-16T18:18:19.327 DEBUG:boto:Token: None 2024-04-16T18:18:19.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.327 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.333 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073362def072c5281-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.374 DEBUG:boto:b'lybtug-51obj000000000000000000321000falseobj000000000000000000322024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.374 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.375 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.375 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.375 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.375 DEBUG:boto:Data: 2024-04-16T18:18:19.375 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.375 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.375 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.375 DEBUG:boto:Params: {} 2024-04-16T18:18:19.375 DEBUG:boto:Token: None 2024-04-16T18:18:19.375 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dXAOY2QueFyqX59h1neIkT2AnjE= 2024-04-16T18:18:19.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dXAOY2QueFyqX59h1neIkT2AnjE=', 'Content-Length': '0'} 2024-04-16T18:18:19.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005680a4aed57686e9-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.379 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.379 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.379 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.379 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.379 DEBUG:boto:Data: 2024-04-16T18:18:19.379 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.379 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.379 DEBUG:boto:Params: {} 2024-04-16T18:18:19.380 DEBUG:boto:Token: None 2024-04-16T18:18:19.380 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dXAOY2QueFyqX59h1neIkT2AnjE= 2024-04-16T18:18:19.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dXAOY2QueFyqX59h1neIkT2AnjE=', 'Content-Length': '0'} 2024-04-16T18:18:19.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cefec92236abfc6-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.383 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000032 2024-04-16T18:18:19.383 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.383 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.383 DEBUG:boto:Method: GET 2024-04-16T18:18:19.383 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.383 DEBUG:boto:Data: 2024-04-16T18:18:19.383 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.383 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.383 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.383 DEBUG:boto:Params: {} 2024-04-16T18:18:19.384 DEBUG:boto:Token: None 2024-04-16T18:18:19.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/1H5l/IKK+rxPJq0aV53qgUT0MI= 2024-04-16T18:18:19.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/1H5l/IKK+rxPJq0aV53qgUT0MI=', 'Content-Length': '0'} 2024-04-16T18:18:19.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010357344896fb193-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.430 DEBUG:boto:path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.430 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.431 DEBUG:boto:Method: GET 2024-04-16T18:18:19.431 DEBUG:boto:Path: /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.431 DEBUG:boto:Data: 2024-04-16T18:18:19.431 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.431 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.431 DEBUG:boto:Params: {} 2024-04-16T18:18:19.431 DEBUG:boto:Token: None 2024-04-16T18:18:19.431 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000032 2024-04-16T18:18:19.431 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/1H5l/IKK+rxPJq0aV53qgUT0MI= 2024-04-16T18:18:19.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/1H5l/IKK+rxPJq0aV53qgUT0MI=', 'Content-Length': '0'} 2024-04-16T18:18:19.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e19b8687cccff337-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.435 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.435 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.435 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.435 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.435 DEBUG:boto:Method: GET 2024-04-16T18:18:19.435 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.435 DEBUG:boto:Data: 2024-04-16T18:18:19.435 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.435 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.436 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.436 DEBUG:boto:Params: {} 2024-04-16T18:18:19.436 DEBUG:boto:Token: None 2024-04-16T18:18:19.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000088c056c9f406147-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.482 DEBUG:boto:b'lybtug-51obj000000000000000000321000falseobj000000000000000000322024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.483 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.483 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.483 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.483 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.483 DEBUG:boto:Method: GET 2024-04-16T18:18:19.483 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000032 2024-04-16T18:18:19.483 DEBUG:boto:Data: 2024-04-16T18:18:19.483 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.483 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.483 DEBUG:boto:Params: {} 2024-04-16T18:18:19.484 DEBUG:boto:Token: None 2024-04-16T18:18:19.484 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d76c0a328e76793-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.490 DEBUG:boto:b'lybtug-51obj000000000000000000321000falseobj000000000000000000322024-04-16T18:17:53.876Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.490 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000033 2024-04-16T18:18:19.490 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.490 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.490 DEBUG:boto:path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.491 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.491 DEBUG:boto:Method: GET 2024-04-16T18:18:19.491 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.491 DEBUG:boto:Data: 2024-04-16T18:18:19.491 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.491 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.491 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.491 DEBUG:boto:Params: {} 2024-04-16T18:18:19.491 DEBUG:boto:Token: None 2024-04-16T18:18:19.491 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458= 2024-04-16T18:18:19.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458=', 'Content-Length': '0'} 2024-04-16T18:18:19.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a85f9adf97e5dd00-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.538 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.538 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.539 DEBUG:boto:path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.539 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.539 DEBUG:boto:Method: GET 2024-04-16T18:18:19.539 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.539 DEBUG:boto:Data: 2024-04-16T18:18:19.539 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.539 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.539 DEBUG:boto:Params: {} 2024-04-16T18:18:19.539 DEBUG:boto:Token: None 2024-04-16T18:18:19.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458= 2024-04-16T18:18:19.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458=', 'Content-Length': '0'} 2024-04-16T18:18:19.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000456704b8c733149d-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.543 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.544 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.544 DEBUG:boto:path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.544 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.544 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.544 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.544 DEBUG:boto:Data: 2024-04-16T18:18:19.544 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.544 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.544 DEBUG:boto:Params: {} 2024-04-16T18:18:19.544 DEBUG:boto:Token: None 2024-04-16T18:18:19.545 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MZB74oyAaWdD9dd7eKnVqp29LzQ= 2024-04-16T18:18:19.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MZB74oyAaWdD9dd7eKnVqp29LzQ=', 'Content-Length': '0'} 2024-04-16T18:18:19.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed48dfdee1a398fd-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.548 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.548 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.548 DEBUG:boto:path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.548 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.549 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.549 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.549 DEBUG:boto:Data: 2024-04-16T18:18:19.549 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.549 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.549 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.549 DEBUG:boto:Params: {} 2024-04-16T18:18:19.549 DEBUG:boto:Token: None 2024-04-16T18:18:19.549 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MZB74oyAaWdD9dd7eKnVqp29LzQ= 2024-04-16T18:18:19.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MZB74oyAaWdD9dd7eKnVqp29LzQ=', 'Content-Length': '0'} 2024-04-16T18:18:19.553 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048784b420149d802-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.553 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000033 2024-04-16T18:18:19.553 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.553 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.553 DEBUG:boto:path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.553 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.553 DEBUG:boto:Method: GET 2024-04-16T18:18:19.554 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.554 DEBUG:boto:Data: 2024-04-16T18:18:19.554 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.554 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.554 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.554 DEBUG:boto:Params: {} 2024-04-16T18:18:19.554 DEBUG:boto:Token: None 2024-04-16T18:18:19.554 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.554 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458= 2024-04-16T18:18:19.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458=', 'Content-Length': '0'} 2024-04-16T18:18:19.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054ce2ab90376620f-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.598 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.598 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.598 DEBUG:boto:path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.599 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.599 DEBUG:boto:Method: GET 2024-04-16T18:18:19.599 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.599 DEBUG:boto:Data: 2024-04-16T18:18:19.599 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.599 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.599 DEBUG:boto:Params: {} 2024-04-16T18:18:19.599 DEBUG:boto:Token: None 2024-04-16T18:18:19.599 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033?versionId=null 2024-04-16T18:18:19.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458= 2024-04-16T18:18:19.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w35Y8e6NzZ1hki3ZpWQ5V2x3458=', 'Content-Length': '0'} 2024-04-16T18:18:19.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e07599c70fd6a60-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.603 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.603 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.603 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.603 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.604 DEBUG:boto:Method: GET 2024-04-16T18:18:19.604 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.604 DEBUG:boto:Data: 2024-04-16T18:18:19.604 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.604 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.604 DEBUG:boto:Params: {} 2024-04-16T18:18:19.604 DEBUG:boto:Token: None 2024-04-16T18:18:19.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e6db2b04e285628-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.609 DEBUG:boto:b'lybtug-51obj000000000000000000331000falseobj000000000000000000332024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.609 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.609 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.609 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.610 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.610 DEBUG:boto:Method: GET 2024-04-16T18:18:19.610 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.610 DEBUG:boto:Data: 2024-04-16T18:18:19.610 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.610 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.610 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.610 DEBUG:boto:Params: {} 2024-04-16T18:18:19.610 DEBUG:boto:Token: None 2024-04-16T18:18:19.610 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.611 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.617 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fea895a4e152880b-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.658 DEBUG:boto:b'lybtug-51obj000000000000000000331000falseobj000000000000000000332024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.659 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.659 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.659 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.659 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.659 DEBUG:boto:Data: 2024-04-16T18:18:19.659 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.659 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.659 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.659 DEBUG:boto:Params: {} 2024-04-16T18:18:19.659 DEBUG:boto:Token: None 2024-04-16T18:18:19.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BIsO0cPQO84B2EnpBeDjFYQhPvI= 2024-04-16T18:18:19.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BIsO0cPQO84B2EnpBeDjFYQhPvI=', 'Content-Length': '0'} 2024-04-16T18:18:19.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f3474311c9faef0-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.663 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.663 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.663 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.663 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.663 DEBUG:boto:Data: 2024-04-16T18:18:19.663 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.663 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.663 DEBUG:boto:Params: {} 2024-04-16T18:18:19.664 DEBUG:boto:Token: None 2024-04-16T18:18:19.664 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BIsO0cPQO84B2EnpBeDjFYQhPvI= 2024-04-16T18:18:19.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BIsO0cPQO84B2EnpBeDjFYQhPvI=', 'Content-Length': '0'} 2024-04-16T18:18:19.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb38f34ba2f9b9bc-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.667 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000033 2024-04-16T18:18:19.667 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.668 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.668 DEBUG:boto:Method: GET 2024-04-16T18:18:19.668 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.668 DEBUG:boto:Data: 2024-04-16T18:18:19.668 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.668 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.668 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.668 DEBUG:boto:Params: {} 2024-04-16T18:18:19.668 DEBUG:boto:Token: None 2024-04-16T18:18:19.668 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cioa8w5FqsMa5PqpGvgUGH2WNdM= 2024-04-16T18:18:19.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cioa8w5FqsMa5PqpGvgUGH2WNdM=', 'Content-Length': '0'} 2024-04-16T18:18:19.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e33a1808693ce0c-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.714 DEBUG:boto:path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.714 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.715 DEBUG:boto:Method: GET 2024-04-16T18:18:19.715 DEBUG:boto:Path: /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.715 DEBUG:boto:Data: 2024-04-16T18:18:19.715 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.715 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.715 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.715 DEBUG:boto:Params: {} 2024-04-16T18:18:19.715 DEBUG:boto:Token: None 2024-04-16T18:18:19.715 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000033 2024-04-16T18:18:19.715 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cioa8w5FqsMa5PqpGvgUGH2WNdM= 2024-04-16T18:18:19.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cioa8w5FqsMa5PqpGvgUGH2WNdM=', 'Content-Length': '0'} 2024-04-16T18:18:19.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004da962c12080a50b-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.719 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.719 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.719 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.719 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.719 DEBUG:boto:Method: GET 2024-04-16T18:18:19.719 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.719 DEBUG:boto:Data: 2024-04-16T18:18:19.719 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.720 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.720 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.720 DEBUG:boto:Params: {} 2024-04-16T18:18:19.720 DEBUG:boto:Token: None 2024-04-16T18:18:19.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.725 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c29571ab3860b482-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.766 DEBUG:boto:b'lybtug-51obj000000000000000000331000falseobj000000000000000000332024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.767 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.767 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.767 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.767 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.767 DEBUG:boto:Method: GET 2024-04-16T18:18:19.767 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000033 2024-04-16T18:18:19.767 DEBUG:boto:Data: 2024-04-16T18:18:19.767 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.767 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.767 DEBUG:boto:Params: {} 2024-04-16T18:18:19.768 DEBUG:boto:Token: None 2024-04-16T18:18:19.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.774 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e690470c7a13adbf-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.775 DEBUG:boto:b'lybtug-51obj000000000000000000331000falseobj000000000000000000332024-04-16T18:18:00.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000036 2024-04-16T18:18:19.775 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.775 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.776 DEBUG:boto:path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.776 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.776 DEBUG:boto:Method: GET 2024-04-16T18:18:19.776 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.776 DEBUG:boto:Data: 2024-04-16T18:18:19.776 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.776 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.776 DEBUG:boto:Params: {} 2024-04-16T18:18:19.776 DEBUG:boto:Token: None 2024-04-16T18:18:19.777 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo= 2024-04-16T18:18:19.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo=', 'Content-Length': '0'} 2024-04-16T18:18:19.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032f1bc6ced4cf60f-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.822 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.823 DEBUG:boto:Method: GET 2024-04-16T18:18:19.823 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.823 DEBUG:boto:Data: 2024-04-16T18:18:19.823 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.823 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.823 DEBUG:boto:Params: {} 2024-04-16T18:18:19.823 DEBUG:boto:Token: None 2024-04-16T18:18:19.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo= 2024-04-16T18:18:19.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo=', 'Content-Length': '0'} 2024-04-16T18:18:19.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073517ccf086dbf5d-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.827 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.827 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.827 DEBUG:boto:path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.827 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.827 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.827 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.828 DEBUG:boto:Data: 2024-04-16T18:18:19.828 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.828 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.828 DEBUG:boto:Params: {} 2024-04-16T18:18:19.828 DEBUG:boto:Token: None 2024-04-16T18:18:19.828 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ggpih6ozjpMzxgkfkmPmnGuYANw= 2024-04-16T18:18:19.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ggpih6ozjpMzxgkfkmPmnGuYANw=', 'Content-Length': '0'} 2024-04-16T18:18:19.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bfc46fe6f4181e4-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.832 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.832 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.832 DEBUG:boto:path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.832 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.832 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.832 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.832 DEBUG:boto:Data: 2024-04-16T18:18:19.832 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.833 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.833 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.833 DEBUG:boto:Params: {} 2024-04-16T18:18:19.833 DEBUG:boto:Token: None 2024-04-16T18:18:19.833 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ggpih6ozjpMzxgkfkmPmnGuYANw= 2024-04-16T18:18:19.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ggpih6ozjpMzxgkfkmPmnGuYANw=', 'Content-Length': '0'} 2024-04-16T18:18:19.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003334625c1b7a3a69-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.836 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000036 2024-04-16T18:18:19.837 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.837 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.837 DEBUG:boto:path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.837 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.837 DEBUG:boto:Method: GET 2024-04-16T18:18:19.837 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.837 DEBUG:boto:Data: 2024-04-16T18:18:19.837 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.837 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.837 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.837 DEBUG:boto:Params: {} 2024-04-16T18:18:19.838 DEBUG:boto:Token: None 2024-04-16T18:18:19.838 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.838 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo= 2024-04-16T18:18:19.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo=', 'Content-Length': '0'} 2024-04-16T18:18:19.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066d983cf8954fa38-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.882 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.882 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.883 DEBUG:boto:path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.883 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.883 DEBUG:boto:Method: GET 2024-04-16T18:18:19.883 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.883 DEBUG:boto:Data: 2024-04-16T18:18:19.883 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.883 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.883 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.883 DEBUG:boto:Params: {} 2024-04-16T18:18:19.883 DEBUG:boto:Token: None 2024-04-16T18:18:19.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036?versionId=null 2024-04-16T18:18:19.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo= 2024-04-16T18:18:19.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLTbW4OWnjYHqjNqw0eBREuNTOo=', 'Content-Length': '0'} 2024-04-16T18:18:19.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c937aa65a64c90b9-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.887 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.887 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.888 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:19.888 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:19.888 DEBUG:boto:Method: GET 2024-04-16T18:18:19.888 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:19.888 DEBUG:boto:Data: 2024-04-16T18:18:19.888 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.888 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.888 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.888 DEBUG:boto:Params: {} 2024-04-16T18:18:19.888 DEBUG:boto:Token: None 2024-04-16T18:18:19.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.889 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021c7037e1a51225d-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.893 DEBUG:boto:b'lybtug-51obj000000000000000000361000falseobj000000000000000000362024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.893 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:19.893 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:19.893 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:19.893 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:19.894 DEBUG:boto:Method: GET 2024-04-16T18:18:19.894 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:19.894 DEBUG:boto:Data: 2024-04-16T18:18:19.894 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.894 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.894 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.894 DEBUG:boto:Params: {} 2024-04-16T18:18:19.894 DEBUG:boto:Token: None 2024-04-16T18:18:19.894 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/ 2024-04-16T18:18:19.895 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI= 2024-04-16T18:18:19.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CaM81mdx7XAHv8cvZLvBXJlALSI=', 'Content-Length': '0'} 2024-04-16T18:18:19.901 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c620e5fc9b360c34-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.942 DEBUG:boto:b'lybtug-51obj000000000000000000361000falseobj000000000000000000362024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:19.943 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.943 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.943 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.943 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.943 DEBUG:boto:Data: 2024-04-16T18:18:19.943 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.943 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.943 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.943 DEBUG:boto:Params: {} 2024-04-16T18:18:19.943 DEBUG:boto:Token: None 2024-04-16T18:18:19.944 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wf3RUHaKYUQMz7M3fqJV1QWTUMY= 2024-04-16T18:18:19.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wf3RUHaKYUQMz7M3fqJV1QWTUMY=', 'Content-Length': '0'} 2024-04-16T18:18:19.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e8388d121092202-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.947 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.947 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.947 DEBUG:boto:Method: HEAD 2024-04-16T18:18:19.947 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.947 DEBUG:boto:Data: 2024-04-16T18:18:19.947 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.947 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.948 DEBUG:boto:Params: {} 2024-04-16T18:18:19.948 DEBUG:boto:Token: None 2024-04-16T18:18:19.948 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wf3RUHaKYUQMz7M3fqJV1QWTUMY= 2024-04-16T18:18:19.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wf3RUHaKYUQMz7M3fqJV1QWTUMY=', 'Content-Length': '0'} 2024-04-16T18:18:19.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075f157171755b2de-00661ec0eb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.951 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000036 2024-04-16T18:18:19.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.952 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.952 DEBUG:boto:Method: GET 2024-04-16T18:18:19.952 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.952 DEBUG:boto:Data: 2024-04-16T18:18:19.952 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.952 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:19.952 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.952 DEBUG:boto:Params: {} 2024-04-16T18:18:19.952 DEBUG:boto:Token: None 2024-04-16T18:18:19.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlbhZcNximxaqD/Be1jJE4TOVzM= 2024-04-16T18:18:19.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlbhZcNximxaqD/Be1jJE4TOVzM=', 'Content-Length': '0'} 2024-04-16T18:18:19.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000045bd1323e975492-00661ec0eb-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:19.998 DEBUG:boto:path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.999 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.999 DEBUG:boto:Method: GET 2024-04-16T18:18:19.999 DEBUG:boto:Path: /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.999 DEBUG:boto:Data: 2024-04-16T18:18:19.999 DEBUG:boto:Headers: {} 2024-04-16T18:18:19.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:19.999 DEBUG:boto:Port: 8000 2024-04-16T18:18:19.999 DEBUG:boto:Params: {} 2024-04-16T18:18:19.999 DEBUG:boto:Token: None 2024-04-16T18:18:19.999 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:19 GMT /lybtug-51/obj00000000000000000036 2024-04-16T18:18:19.999 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlbhZcNximxaqD/Be1jJE4TOVzM= 2024-04-16T18:18:20.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlbhZcNximxaqD/Be1jJE4TOVzM=', 'Content-Length': '0'} 2024-04-16T18:18:20.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000333591d2371b962c-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.003 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.003 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.003 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:20.003 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:20.003 DEBUG:boto:Method: GET 2024-04-16T18:18:20.003 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:20.003 DEBUG:boto:Data: 2024-04-16T18:18:20.003 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.003 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.004 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.004 DEBUG:boto:Params: {} 2024-04-16T18:18:20.004 DEBUG:boto:Token: None 2024-04-16T18:18:20.004 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.008 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045302299219f98ca-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.050 DEBUG:boto:b'lybtug-51obj000000000000000000361000falseobj000000000000000000362024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.051 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.051 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.051 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:20.051 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:20.051 DEBUG:boto:Method: GET 2024-04-16T18:18:20.051 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000036 2024-04-16T18:18:20.051 DEBUG:boto:Data: 2024-04-16T18:18:20.051 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.051 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.051 DEBUG:boto:Params: {} 2024-04-16T18:18:20.052 DEBUG:boto:Token: None 2024-04-16T18:18:20.052 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064ef46081e020616-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.058 DEBUG:boto:b'lybtug-51obj000000000000000000361000falseobj000000000000000000362024-04-16T18:18:00.205Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.059 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000037 2024-04-16T18:18:20.059 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.059 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.059 DEBUG:boto:path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.059 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.059 DEBUG:boto:Method: GET 2024-04-16T18:18:20.059 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.059 DEBUG:boto:Data: 2024-04-16T18:18:20.059 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.060 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.060 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.060 DEBUG:boto:Params: {} 2024-04-16T18:18:20.060 DEBUG:boto:Token: None 2024-04-16T18:18:20.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk= 2024-04-16T18:18:20.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk=', 'Content-Length': '0'} 2024-04-16T18:18:20.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c36a76f6f7aa2155-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.106 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.107 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.107 DEBUG:boto:path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.107 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.107 DEBUG:boto:Method: GET 2024-04-16T18:18:20.107 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.107 DEBUG:boto:Data: 2024-04-16T18:18:20.107 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.107 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.107 DEBUG:boto:Params: {} 2024-04-16T18:18:20.107 DEBUG:boto:Token: None 2024-04-16T18:18:20.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk= 2024-04-16T18:18:20.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk=', 'Content-Length': '0'} 2024-04-16T18:18:20.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e02c2446e35647f4-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.111 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.111 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.111 DEBUG:boto:path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.111 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.111 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.111 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.111 DEBUG:boto:Data: 2024-04-16T18:18:20.112 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.112 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.112 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.112 DEBUG:boto:Params: {} 2024-04-16T18:18:20.112 DEBUG:boto:Token: None 2024-04-16T18:18:20.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qicO8hCi7QIyLYJAbPR9eih+uMk= 2024-04-16T18:18:20.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qicO8hCi7QIyLYJAbPR9eih+uMk=', 'Content-Length': '0'} 2024-04-16T18:18:20.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071382e71fa037f54-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.116 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.116 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.116 DEBUG:boto:path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.116 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.116 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.116 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.116 DEBUG:boto:Data: 2024-04-16T18:18:20.117 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.117 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.117 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.117 DEBUG:boto:Params: {} 2024-04-16T18:18:20.117 DEBUG:boto:Token: None 2024-04-16T18:18:20.117 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.117 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qicO8hCi7QIyLYJAbPR9eih+uMk= 2024-04-16T18:18:20.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qicO8hCi7QIyLYJAbPR9eih+uMk=', 'Content-Length': '0'} 2024-04-16T18:18:20.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ec4ae66aa199065-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000037 2024-04-16T18:18:20.120 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.120 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.120 DEBUG:boto:path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.121 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.121 DEBUG:boto:Method: GET 2024-04-16T18:18:20.121 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.121 DEBUG:boto:Data: 2024-04-16T18:18:20.121 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.121 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.121 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.121 DEBUG:boto:Params: {} 2024-04-16T18:18:20.121 DEBUG:boto:Token: None 2024-04-16T18:18:20.121 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.122 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk= 2024-04-16T18:18:20.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk=', 'Content-Length': '0'} 2024-04-16T18:18:20.125 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000737984c410126b58-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.166 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.166 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.166 DEBUG:boto:path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.167 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.167 DEBUG:boto:Method: GET 2024-04-16T18:18:20.167 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.167 DEBUG:boto:Data: 2024-04-16T18:18:20.167 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.167 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.167 DEBUG:boto:Params: {} 2024-04-16T18:18:20.167 DEBUG:boto:Token: None 2024-04-16T18:18:20.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037?versionId=null 2024-04-16T18:18:20.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk= 2024-04-16T18:18:20.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LuVVFIDDeZ42b0D02YOSzGvPJnk=', 'Content-Length': '0'} 2024-04-16T18:18:20.170 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e9fc695fcce7a02-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.171 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.171 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.171 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.171 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.171 DEBUG:boto:Method: GET 2024-04-16T18:18:20.171 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.171 DEBUG:boto:Data: 2024-04-16T18:18:20.171 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.171 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.171 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.172 DEBUG:boto:Params: {} 2024-04-16T18:18:20.172 DEBUG:boto:Token: None 2024-04-16T18:18:20.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000861af3f48ea2f746-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.176 DEBUG:boto:b'lybtug-51obj000000000000000000371000falseobj000000000000000000372024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.177 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.177 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.177 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.177 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.177 DEBUG:boto:Method: GET 2024-04-16T18:18:20.177 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.177 DEBUG:boto:Data: 2024-04-16T18:18:20.177 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.177 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.177 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.177 DEBUG:boto:Params: {} 2024-04-16T18:18:20.178 DEBUG:boto:Token: None 2024-04-16T18:18:20.178 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.178 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.184 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9d34b4ad69df07a-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.230 DEBUG:boto:b'lybtug-51obj000000000000000000371000falseobj000000000000000000372024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.231 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.231 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.231 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.231 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.231 DEBUG:boto:Data: 2024-04-16T18:18:20.231 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.231 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.231 DEBUG:boto:Params: {} 2024-04-16T18:18:20.231 DEBUG:boto:Token: None 2024-04-16T18:18:20.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0IDHoO3D5FPFLRCnp82xaO7RWqo= 2024-04-16T18:18:20.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0IDHoO3D5FPFLRCnp82xaO7RWqo=', 'Content-Length': '0'} 2024-04-16T18:18:20.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e8e4ce7667e7b79-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.235 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.235 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.235 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.235 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.235 DEBUG:boto:Data: 2024-04-16T18:18:20.235 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.235 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.235 DEBUG:boto:Params: {} 2024-04-16T18:18:20.236 DEBUG:boto:Token: None 2024-04-16T18:18:20.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0IDHoO3D5FPFLRCnp82xaO7RWqo= 2024-04-16T18:18:20.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0IDHoO3D5FPFLRCnp82xaO7RWqo=', 'Content-Length': '0'} 2024-04-16T18:18:20.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001cb3dafe550de828-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000037 2024-04-16T18:18:20.239 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.239 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.239 DEBUG:boto:Method: GET 2024-04-16T18:18:20.239 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.240 DEBUG:boto:Data: 2024-04-16T18:18:20.240 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.240 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.240 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.240 DEBUG:boto:Params: {} 2024-04-16T18:18:20.240 DEBUG:boto:Token: None 2024-04-16T18:18:20.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TJSAlxmJyXr/SAQnE7wXgT5WnI4= 2024-04-16T18:18:20.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TJSAlxmJyXr/SAQnE7wXgT5WnI4=', 'Content-Length': '0'} 2024-04-16T18:18:20.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b860233a7ddad53-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.286 DEBUG:boto:path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.286 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.287 DEBUG:boto:Method: GET 2024-04-16T18:18:20.287 DEBUG:boto:Path: /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.287 DEBUG:boto:Data: 2024-04-16T18:18:20.287 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.287 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.287 DEBUG:boto:Params: {} 2024-04-16T18:18:20.287 DEBUG:boto:Token: None 2024-04-16T18:18:20.287 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000037 2024-04-16T18:18:20.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TJSAlxmJyXr/SAQnE7wXgT5WnI4= 2024-04-16T18:18:20.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TJSAlxmJyXr/SAQnE7wXgT5WnI4=', 'Content-Length': '0'} 2024-04-16T18:18:20.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000161631853dc1f965-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.291 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.291 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.291 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.291 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.291 DEBUG:boto:Method: GET 2024-04-16T18:18:20.291 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.291 DEBUG:boto:Data: 2024-04-16T18:18:20.291 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.292 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.292 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.292 DEBUG:boto:Params: {} 2024-04-16T18:18:20.292 DEBUG:boto:Token: None 2024-04-16T18:18:20.292 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000db765fcd03441da-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.338 DEBUG:boto:b'lybtug-51obj000000000000000000371000falseobj000000000000000000372024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.339 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.339 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.339 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.339 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.339 DEBUG:boto:Method: GET 2024-04-16T18:18:20.339 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000037 2024-04-16T18:18:20.339 DEBUG:boto:Data: 2024-04-16T18:18:20.339 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.339 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.339 DEBUG:boto:Params: {} 2024-04-16T18:18:20.340 DEBUG:boto:Token: None 2024-04-16T18:18:20.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8b69a7895d7bf1c-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.347 DEBUG:boto:b'lybtug-51obj000000000000000000371000falseobj000000000000000000372024-04-16T18:17:57.195Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000039 2024-04-16T18:18:20.348 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.348 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.348 DEBUG:boto:path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.348 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.348 DEBUG:boto:Method: GET 2024-04-16T18:18:20.348 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.348 DEBUG:boto:Data: 2024-04-16T18:18:20.348 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.349 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.349 DEBUG:boto:Params: {} 2024-04-16T18:18:20.349 DEBUG:boto:Token: None 2024-04-16T18:18:20.349 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54= 2024-04-16T18:18:20.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54=', 'Content-Length': '0'} 2024-04-16T18:18:20.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000137abe02f0b6cb7f-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.394 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.395 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.395 DEBUG:boto:path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.395 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.395 DEBUG:boto:Method: GET 2024-04-16T18:18:20.395 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.395 DEBUG:boto:Data: 2024-04-16T18:18:20.395 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.395 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.395 DEBUG:boto:Params: {} 2024-04-16T18:18:20.395 DEBUG:boto:Token: None 2024-04-16T18:18:20.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54= 2024-04-16T18:18:20.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54=', 'Content-Length': '0'} 2024-04-16T18:18:20.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f609dc482abd8f8-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.399 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.399 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.399 DEBUG:boto:path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.399 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.400 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.400 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.400 DEBUG:boto:Data: 2024-04-16T18:18:20.400 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.400 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.400 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.400 DEBUG:boto:Params: {} 2024-04-16T18:18:20.400 DEBUG:boto:Token: None 2024-04-16T18:18:20.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p6WjLt83OiIPqdB1aE/OYD+Kv5w= 2024-04-16T18:18:20.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p6WjLt83OiIPqdB1aE/OYD+Kv5w=', 'Content-Length': '0'} 2024-04-16T18:18:20.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000feab3621ba34e330-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.404 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.404 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.404 DEBUG:boto:path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.404 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.404 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.404 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.404 DEBUG:boto:Data: 2024-04-16T18:18:20.404 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.404 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.405 DEBUG:boto:Params: {} 2024-04-16T18:18:20.405 DEBUG:boto:Token: None 2024-04-16T18:18:20.405 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p6WjLt83OiIPqdB1aE/OYD+Kv5w= 2024-04-16T18:18:20.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p6WjLt83OiIPqdB1aE/OYD+Kv5w=', 'Content-Length': '0'} 2024-04-16T18:18:20.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db5916287e1a92b0-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.408 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000039 2024-04-16T18:18:20.408 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.408 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.408 DEBUG:boto:path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.409 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.409 DEBUG:boto:Method: GET 2024-04-16T18:18:20.409 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.409 DEBUG:boto:Data: 2024-04-16T18:18:20.409 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.409 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.409 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.409 DEBUG:boto:Params: {} 2024-04-16T18:18:20.409 DEBUG:boto:Token: None 2024-04-16T18:18:20.409 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.409 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54= 2024-04-16T18:18:20.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54=', 'Content-Length': '0'} 2024-04-16T18:18:20.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fec99b3ed0bbf4b1-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.454 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.454 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.455 DEBUG:boto:path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.455 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.455 DEBUG:boto:Method: GET 2024-04-16T18:18:20.455 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.455 DEBUG:boto:Data: 2024-04-16T18:18:20.455 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.455 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.455 DEBUG:boto:Params: {} 2024-04-16T18:18:20.455 DEBUG:boto:Token: None 2024-04-16T18:18:20.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039?versionId=null 2024-04-16T18:18:20.455 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54= 2024-04-16T18:18:20.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k57DbsHksLsjQa86o3H39nW7x54=', 'Content-Length': '0'} 2024-04-16T18:18:20.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3540dbfd959053a-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.459 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.459 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.459 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.459 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.459 DEBUG:boto:Method: GET 2024-04-16T18:18:20.459 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.460 DEBUG:boto:Data: 2024-04-16T18:18:20.460 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.460 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.460 DEBUG:boto:Params: {} 2024-04-16T18:18:20.460 DEBUG:boto:Token: None 2024-04-16T18:18:20.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001f5f8f2ef7e5141-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.464 DEBUG:boto:b'lybtug-51obj000000000000000000391000falseobj000000000000000000392024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.465 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.465 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.465 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.465 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.465 DEBUG:boto:Method: GET 2024-04-16T18:18:20.465 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.465 DEBUG:boto:Data: 2024-04-16T18:18:20.465 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.465 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.465 DEBUG:boto:Params: {} 2024-04-16T18:18:20.465 DEBUG:boto:Token: None 2024-04-16T18:18:20.466 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.472 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c6e50834b180654-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.514 DEBUG:boto:b'lybtug-51obj000000000000000000391000falseobj000000000000000000392024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.515 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.515 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.515 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.515 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.515 DEBUG:boto:Data: 2024-04-16T18:18:20.515 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.515 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.515 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.515 DEBUG:boto:Params: {} 2024-04-16T18:18:20.515 DEBUG:boto:Token: None 2024-04-16T18:18:20.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rxiRWYxDm1YYMQGt3ze8ukmAQ0c= 2024-04-16T18:18:20.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rxiRWYxDm1YYMQGt3ze8ukmAQ0c=', 'Content-Length': '0'} 2024-04-16T18:18:20.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a6e22c2bfa09499-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.519 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.519 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.519 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.519 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.519 DEBUG:boto:Data: 2024-04-16T18:18:20.520 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.520 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.520 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.520 DEBUG:boto:Params: {} 2024-04-16T18:18:20.520 DEBUG:boto:Token: None 2024-04-16T18:18:20.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rxiRWYxDm1YYMQGt3ze8ukmAQ0c= 2024-04-16T18:18:20.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rxiRWYxDm1YYMQGt3ze8ukmAQ0c=', 'Content-Length': '0'} 2024-04-16T18:18:20.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bdec52b4d4b01c8-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.524 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000039 2024-04-16T18:18:20.524 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.524 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.524 DEBUG:boto:Method: GET 2024-04-16T18:18:20.524 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.524 DEBUG:boto:Data: 2024-04-16T18:18:20.524 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.524 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.524 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.524 DEBUG:boto:Params: {} 2024-04-16T18:18:20.525 DEBUG:boto:Token: None 2024-04-16T18:18:20.525 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OPcmdIadwbPRnmoiJJuAu2Xdm90= 2024-04-16T18:18:20.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OPcmdIadwbPRnmoiJJuAu2Xdm90=', 'Content-Length': '0'} 2024-04-16T18:18:20.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8335af266651b51-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.570 DEBUG:boto:path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.571 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.571 DEBUG:boto:Method: GET 2024-04-16T18:18:20.571 DEBUG:boto:Path: /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.571 DEBUG:boto:Data: 2024-04-16T18:18:20.571 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.571 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.571 DEBUG:boto:Params: {} 2024-04-16T18:18:20.571 DEBUG:boto:Token: None 2024-04-16T18:18:20.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000039 2024-04-16T18:18:20.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OPcmdIadwbPRnmoiJJuAu2Xdm90= 2024-04-16T18:18:20.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OPcmdIadwbPRnmoiJJuAu2Xdm90=', 'Content-Length': '0'} 2024-04-16T18:18:20.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001403758c8e27364a-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.575 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.575 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.575 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.575 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.575 DEBUG:boto:Method: GET 2024-04-16T18:18:20.575 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.575 DEBUG:boto:Data: 2024-04-16T18:18:20.575 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.575 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.576 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.576 DEBUG:boto:Params: {} 2024-04-16T18:18:20.576 DEBUG:boto:Token: None 2024-04-16T18:18:20.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d028064326c97e5e-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.622 DEBUG:boto:b'lybtug-51obj000000000000000000391000falseobj000000000000000000392024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.623 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.623 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.623 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.623 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.623 DEBUG:boto:Method: GET 2024-04-16T18:18:20.623 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000039 2024-04-16T18:18:20.623 DEBUG:boto:Data: 2024-04-16T18:18:20.623 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.623 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.623 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.623 DEBUG:boto:Params: {} 2024-04-16T18:18:20.624 DEBUG:boto:Token: None 2024-04-16T18:18:20.624 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.630 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2730634d1706316-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.630 DEBUG:boto:b'lybtug-51obj000000000000000000391000falseobj000000000000000000392024-04-16T18:18:00.742Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.630 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000040 2024-04-16T18:18:20.630 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.630 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.630 DEBUG:boto:path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.631 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.631 DEBUG:boto:Method: GET 2024-04-16T18:18:20.631 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.631 DEBUG:boto:Data: 2024-04-16T18:18:20.631 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.631 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.631 DEBUG:boto:Params: {} 2024-04-16T18:18:20.631 DEBUG:boto:Token: None 2024-04-16T18:18:20.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog= 2024-04-16T18:18:20.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog=', 'Content-Length': '0'} 2024-04-16T18:18:20.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048ac49beab8b1db4-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.678 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.678 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.678 DEBUG:boto:path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.679 DEBUG:boto:Method: GET 2024-04-16T18:18:20.679 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.679 DEBUG:boto:Data: 2024-04-16T18:18:20.679 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.679 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.679 DEBUG:boto:Params: {} 2024-04-16T18:18:20.679 DEBUG:boto:Token: None 2024-04-16T18:18:20.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog= 2024-04-16T18:18:20.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog=', 'Content-Length': '0'} 2024-04-16T18:18:20.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b55b267a112e2e1-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.683 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.683 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.683 DEBUG:boto:path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.683 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.683 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.684 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.684 DEBUG:boto:Data: 2024-04-16T18:18:20.684 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.684 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.684 DEBUG:boto:Params: {} 2024-04-16T18:18:20.684 DEBUG:boto:Token: None 2024-04-16T18:18:20.684 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jYbRZlGH5B9WZWD9Mrh41AIEPuI= 2024-04-16T18:18:20.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jYbRZlGH5B9WZWD9Mrh41AIEPuI=', 'Content-Length': '0'} 2024-04-16T18:18:20.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005afc05c8c3d0eb06-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.690 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.690 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.691 DEBUG:boto:path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.691 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.691 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.691 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.691 DEBUG:boto:Data: 2024-04-16T18:18:20.691 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.691 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.691 DEBUG:boto:Params: {} 2024-04-16T18:18:20.691 DEBUG:boto:Token: None 2024-04-16T18:18:20.692 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jYbRZlGH5B9WZWD9Mrh41AIEPuI= 2024-04-16T18:18:20.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jYbRZlGH5B9WZWD9Mrh41AIEPuI=', 'Content-Length': '0'} 2024-04-16T18:18:20.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000566542e24a809c30-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.695 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000040 2024-04-16T18:18:20.695 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.695 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.695 DEBUG:boto:path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.695 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.695 DEBUG:boto:Method: GET 2024-04-16T18:18:20.696 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.696 DEBUG:boto:Data: 2024-04-16T18:18:20.696 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.696 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.696 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.696 DEBUG:boto:Params: {} 2024-04-16T18:18:20.696 DEBUG:boto:Token: None 2024-04-16T18:18:20.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog= 2024-04-16T18:18:20.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog=', 'Content-Length': '0'} 2024-04-16T18:18:20.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff0264a9f037eed9-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.742 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.742 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.742 DEBUG:boto:path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.743 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.743 DEBUG:boto:Method: GET 2024-04-16T18:18:20.743 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.743 DEBUG:boto:Data: 2024-04-16T18:18:20.743 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.743 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.743 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.743 DEBUG:boto:Params: {} 2024-04-16T18:18:20.743 DEBUG:boto:Token: None 2024-04-16T18:18:20.743 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040?versionId=null 2024-04-16T18:18:20.743 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog= 2024-04-16T18:18:20.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c/20tlc+cRE2UCawaeAxfEn0Sog=', 'Content-Length': '0'} 2024-04-16T18:18:20.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8f68ebda2544e6e-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.747 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.747 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.747 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.747 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.747 DEBUG:boto:Method: GET 2024-04-16T18:18:20.747 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.747 DEBUG:boto:Data: 2024-04-16T18:18:20.748 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.748 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.748 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.748 DEBUG:boto:Params: {} 2024-04-16T18:18:20.748 DEBUG:boto:Token: None 2024-04-16T18:18:20.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc974276eabff0a0-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.752 DEBUG:boto:b'lybtug-51obj000000000000000000401000falseobj000000000000000000402024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.753 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.753 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.753 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.754 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.754 DEBUG:boto:Method: GET 2024-04-16T18:18:20.754 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.754 DEBUG:boto:Data: 2024-04-16T18:18:20.754 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.754 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.754 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.754 DEBUG:boto:Params: {} 2024-04-16T18:18:20.754 DEBUG:boto:Token: None 2024-04-16T18:18:20.754 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.755 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.760 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f42e53a7ca36f32a-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.802 DEBUG:boto:b'lybtug-51obj000000000000000000401000falseobj000000000000000000402024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.803 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.803 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.803 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.803 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.803 DEBUG:boto:Data: 2024-04-16T18:18:20.803 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.803 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.803 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.803 DEBUG:boto:Params: {} 2024-04-16T18:18:20.803 DEBUG:boto:Token: None 2024-04-16T18:18:20.803 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:slfbxBJO/TxOSrE1vcyOmcTnQ+8= 2024-04-16T18:18:20.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:slfbxBJO/TxOSrE1vcyOmcTnQ+8=', 'Content-Length': '0'} 2024-04-16T18:18:20.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af94a2e750564eb5-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.807 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.807 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.807 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.807 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.807 DEBUG:boto:Data: 2024-04-16T18:18:20.807 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.807 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.807 DEBUG:boto:Params: {} 2024-04-16T18:18:20.807 DEBUG:boto:Token: None 2024-04-16T18:18:20.808 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:slfbxBJO/TxOSrE1vcyOmcTnQ+8= 2024-04-16T18:18:20.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:slfbxBJO/TxOSrE1vcyOmcTnQ+8=', 'Content-Length': '0'} 2024-04-16T18:18:20.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000812ffbe0e5c70da-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000040 2024-04-16T18:18:20.811 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.811 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.811 DEBUG:boto:Method: GET 2024-04-16T18:18:20.811 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.812 DEBUG:boto:Data: 2024-04-16T18:18:20.812 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.812 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.812 DEBUG:boto:Params: {} 2024-04-16T18:18:20.812 DEBUG:boto:Token: None 2024-04-16T18:18:20.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GhzQk4G2y+xpJQlRki0RwtbMSs= 2024-04-16T18:18:20.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GhzQk4G2y+xpJQlRki0RwtbMSs=', 'Content-Length': '0'} 2024-04-16T18:18:20.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6e14f8e28daba10-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.858 DEBUG:boto:path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.858 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.859 DEBUG:boto:Method: GET 2024-04-16T18:18:20.859 DEBUG:boto:Path: /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.859 DEBUG:boto:Data: 2024-04-16T18:18:20.859 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.859 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.859 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.859 DEBUG:boto:Params: {} 2024-04-16T18:18:20.859 DEBUG:boto:Token: None 2024-04-16T18:18:20.859 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000040 2024-04-16T18:18:20.859 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3GhzQk4G2y+xpJQlRki0RwtbMSs= 2024-04-16T18:18:20.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3GhzQk4G2y+xpJQlRki0RwtbMSs=', 'Content-Length': '0'} 2024-04-16T18:18:20.862 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e79b55186f75e249-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.863 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.863 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.863 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.863 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.863 DEBUG:boto:Method: GET 2024-04-16T18:18:20.863 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.863 DEBUG:boto:Data: 2024-04-16T18:18:20.863 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.863 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.863 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.864 DEBUG:boto:Params: {} 2024-04-16T18:18:20.864 DEBUG:boto:Token: None 2024-04-16T18:18:20.864 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000166efb5646b2432a-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.910 DEBUG:boto:b'lybtug-51obj000000000000000000401000falseobj000000000000000000402024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.911 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:20.911 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:20.911 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.911 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.912 DEBUG:boto:Method: GET 2024-04-16T18:18:20.912 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000040 2024-04-16T18:18:20.912 DEBUG:boto:Data: 2024-04-16T18:18:20.912 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.912 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.912 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.912 DEBUG:boto:Params: {} 2024-04-16T18:18:20.912 DEBUG:boto:Token: None 2024-04-16T18:18:20.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/ 2024-04-16T18:18:20.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs= 2024-04-16T18:18:20.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kGzFzqUo8qkQ+Oy9kcPfp8XWAKs=', 'Content-Length': '0'} 2024-04-16T18:18:20.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d37d917663c1680-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.919 DEBUG:boto:b'lybtug-51obj000000000000000000401000falseobj000000000000000000402024-04-16T18:18:00.259Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:20.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000042 2024-04-16T18:18:20.920 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.920 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.920 DEBUG:boto:path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.920 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.920 DEBUG:boto:Method: GET 2024-04-16T18:18:20.920 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.920 DEBUG:boto:Data: 2024-04-16T18:18:20.921 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.921 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.921 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.921 DEBUG:boto:Params: {} 2024-04-16T18:18:20.921 DEBUG:boto:Token: None 2024-04-16T18:18:20.921 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIlDECCKqpTX4AYKnINheItwAP0= 2024-04-16T18:18:20.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIlDECCKqpTX4AYKnINheItwAP0=', 'Content-Length': '0'} 2024-04-16T18:18:20.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b7b7538deadd461-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.966 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.966 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.967 DEBUG:boto:Method: GET 2024-04-16T18:18:20.967 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.967 DEBUG:boto:Data: 2024-04-16T18:18:20.967 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.967 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.967 DEBUG:boto:Params: {} 2024-04-16T18:18:20.967 DEBUG:boto:Token: None 2024-04-16T18:18:20.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIlDECCKqpTX4AYKnINheItwAP0= 2024-04-16T18:18:20.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIlDECCKqpTX4AYKnINheItwAP0=', 'Content-Length': '0'} 2024-04-16T18:18:20.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6911ab866a0dae0-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.971 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.971 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.971 DEBUG:boto:path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.971 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.971 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.971 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.971 DEBUG:boto:Data: 2024-04-16T18:18:20.971 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.971 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.972 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.972 DEBUG:boto:Params: {} 2024-04-16T18:18:20.972 DEBUG:boto:Token: None 2024-04-16T18:18:20.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oCnSx6fPDiESDrKjMUZgHLtT6hM= 2024-04-16T18:18:20.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oCnSx6fPDiESDrKjMUZgHLtT6hM=', 'Content-Length': '0'} 2024-04-16T18:18:20.975 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002cde237024593fb1-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.975 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.975 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.975 DEBUG:boto:path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.975 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.975 DEBUG:boto:Method: HEAD 2024-04-16T18:18:20.975 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.976 DEBUG:boto:Data: 2024-04-16T18:18:20.976 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.976 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:20.976 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.976 DEBUG:boto:Params: {} 2024-04-16T18:18:20.976 DEBUG:boto:Token: None 2024-04-16T18:18:20.976 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oCnSx6fPDiESDrKjMUZgHLtT6hM= 2024-04-16T18:18:20.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oCnSx6fPDiESDrKjMUZgHLtT6hM=', 'Content-Length': '0'} 2024-04-16T18:18:20.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1331f5d41d76103-00661ec0ec-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:20.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000042 2024-04-16T18:18:20.979 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.979 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:20.979 DEBUG:boto:path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.979 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.980 DEBUG:boto:Method: GET 2024-04-16T18:18:20.980 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.980 DEBUG:boto:Data: 2024-04-16T18:18:20.980 DEBUG:boto:Headers: {} 2024-04-16T18:18:20.980 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:20.980 DEBUG:boto:Port: 8000 2024-04-16T18:18:20.980 DEBUG:boto:Params: {} 2024-04-16T18:18:20.980 DEBUG:boto:Token: None 2024-04-16T18:18:20.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:20 GMT /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:20.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AIlDECCKqpTX4AYKnINheItwAP0= 2024-04-16T18:18:20.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AIlDECCKqpTX4AYKnINheItwAP0=', 'Content-Length': '0'} 2024-04-16T18:18:20.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a21663bccf84fd6f-00661ec0ec-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.026 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.026 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.026 DEBUG:boto:path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:21.027 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:21.027 DEBUG:boto:Method: GET 2024-04-16T18:18:21.027 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:21.027 DEBUG:boto:Data: 2024-04-16T18:18:21.027 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.027 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.027 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.027 DEBUG:boto:Params: {} 2024-04-16T18:18:21.027 DEBUG:boto:Token: None 2024-04-16T18:18:21.027 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000042?versionId=null 2024-04-16T18:18:21.027 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Th+kCK1mIpJLLMgsbK/Rxmm9UK0= 2024-04-16T18:18:21.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Th+kCK1mIpJLLMgsbK/Rxmm9UK0=', 'Content-Length': '0'} 2024-04-16T18:18:21.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065fc1284f8591a4c-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.031 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.031 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.031 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.031 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.031 DEBUG:boto:Method: GET 2024-04-16T18:18:21.031 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.032 DEBUG:boto:Data: 2024-04-16T18:18:21.032 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.032 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.032 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.032 DEBUG:boto:Params: {} 2024-04-16T18:18:21.032 DEBUG:boto:Token: None 2024-04-16T18:18:21.032 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006432b0ebe7315597-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.037 DEBUG:boto:b'lybtug-51obj000000000000000000421000falseobj000000000000000000422024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.037 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.037 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.037 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.037 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.037 DEBUG:boto:Method: GET 2024-04-16T18:18:21.037 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.037 DEBUG:boto:Data: 2024-04-16T18:18:21.037 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.038 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.038 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.038 DEBUG:boto:Params: {} 2024-04-16T18:18:21.038 DEBUG:boto:Token: None 2024-04-16T18:18:21.038 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.038 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b828ef062e312f2-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.086 DEBUG:boto:b'lybtug-51obj000000000000000000421000falseobj000000000000000000422024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.087 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.087 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.087 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.087 DEBUG:boto:Data: 2024-04-16T18:18:21.087 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.087 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.087 DEBUG:boto:Params: {} 2024-04-16T18:18:21.087 DEBUG:boto:Token: None 2024-04-16T18:18:21.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9cUXYRz48wsCh/X26W+CEeySCg4= 2024-04-16T18:18:21.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9cUXYRz48wsCh/X26W+CEeySCg4=', 'Content-Length': '0'} 2024-04-16T18:18:21.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c916053a650dd129-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.091 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.091 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.091 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.091 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.091 DEBUG:boto:Data: 2024-04-16T18:18:21.091 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.091 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.092 DEBUG:boto:Params: {} 2024-04-16T18:18:21.092 DEBUG:boto:Token: None 2024-04-16T18:18:21.092 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9cUXYRz48wsCh/X26W+CEeySCg4= 2024-04-16T18:18:21.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9cUXYRz48wsCh/X26W+CEeySCg4=', 'Content-Length': '0'} 2024-04-16T18:18:21.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b3119f15c796808-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000042 2024-04-16T18:18:21.098 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.098 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.098 DEBUG:boto:Method: GET 2024-04-16T18:18:21.098 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.098 DEBUG:boto:Data: 2024-04-16T18:18:21.098 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.098 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.098 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.098 DEBUG:boto:Params: {} 2024-04-16T18:18:21.099 DEBUG:boto:Token: None 2024-04-16T18:18:21.099 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.099 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvceXnHZcCemeBj/NMAp/nFzoM= 2024-04-16T18:18:21.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvceXnHZcCemeBj/NMAp/nFzoM=', 'Content-Length': '0'} 2024-04-16T18:18:21.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000131d5ab90be939bb-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.142 DEBUG:boto:path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.142 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.142 DEBUG:boto:Method: GET 2024-04-16T18:18:21.143 DEBUG:boto:Path: /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.143 DEBUG:boto:Data: 2024-04-16T18:18:21.143 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.143 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.143 DEBUG:boto:Params: {} 2024-04-16T18:18:21.143 DEBUG:boto:Token: None 2024-04-16T18:18:21.143 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000042 2024-04-16T18:18:21.143 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6uvceXnHZcCemeBj/NMAp/nFzoM= 2024-04-16T18:18:21.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6uvceXnHZcCemeBj/NMAp/nFzoM=', 'Content-Length': '0'} 2024-04-16T18:18:21.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af70cf21709cc554-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.146 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.147 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.147 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.147 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.147 DEBUG:boto:Method: GET 2024-04-16T18:18:21.147 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.147 DEBUG:boto:Data: 2024-04-16T18:18:21.147 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.147 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.147 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.147 DEBUG:boto:Params: {} 2024-04-16T18:18:21.147 DEBUG:boto:Token: None 2024-04-16T18:18:21.147 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d42cbbae3915dc8-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.198 DEBUG:boto:b'lybtug-51obj000000000000000000421000falseobj000000000000000000422024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.199 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.199 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.199 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.199 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.199 DEBUG:boto:Method: GET 2024-04-16T18:18:21.199 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000042 2024-04-16T18:18:21.199 DEBUG:boto:Data: 2024-04-16T18:18:21.199 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.199 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.199 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.200 DEBUG:boto:Params: {} 2024-04-16T18:18:21.200 DEBUG:boto:Token: None 2024-04-16T18:18:21.200 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.206 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d26f9805ef106c13-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.206 DEBUG:boto:b'lybtug-51obj000000000000000000421000falseobj000000000000000000422024-04-16T18:17:57.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.207 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000045 2024-04-16T18:18:21.207 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.207 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.207 DEBUG:boto:path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.207 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.207 DEBUG:boto:Method: GET 2024-04-16T18:18:21.207 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.207 DEBUG:boto:Data: 2024-04-16T18:18:21.207 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.207 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.208 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.208 DEBUG:boto:Params: {} 2024-04-16T18:18:21.208 DEBUG:boto:Token: None 2024-04-16T18:18:21.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw= 2024-04-16T18:18:21.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw=', 'Content-Length': '0'} 2024-04-16T18:18:21.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091fa300538e7135e-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.258 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.258 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.259 DEBUG:boto:path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.259 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.259 DEBUG:boto:Method: GET 2024-04-16T18:18:21.259 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.259 DEBUG:boto:Data: 2024-04-16T18:18:21.259 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.259 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.259 DEBUG:boto:Params: {} 2024-04-16T18:18:21.259 DEBUG:boto:Token: None 2024-04-16T18:18:21.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw= 2024-04-16T18:18:21.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw=', 'Content-Length': '0'} 2024-04-16T18:18:21.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094080bc6e85c9fe8-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.263 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.263 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.263 DEBUG:boto:path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.263 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.264 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.264 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.264 DEBUG:boto:Data: 2024-04-16T18:18:21.264 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.264 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.264 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.264 DEBUG:boto:Params: {} 2024-04-16T18:18:21.264 DEBUG:boto:Token: None 2024-04-16T18:18:21.264 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qe1DD7xCOKnRh/Dob8CzxtiaoDs= 2024-04-16T18:18:21.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qe1DD7xCOKnRh/Dob8CzxtiaoDs=', 'Content-Length': '0'} 2024-04-16T18:18:21.268 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049f09bd17f4a917f-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.268 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.268 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.268 DEBUG:boto:path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.268 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.268 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.268 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.268 DEBUG:boto:Data: 2024-04-16T18:18:21.268 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.268 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.268 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.269 DEBUG:boto:Params: {} 2024-04-16T18:18:21.269 DEBUG:boto:Token: None 2024-04-16T18:18:21.269 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.269 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qe1DD7xCOKnRh/Dob8CzxtiaoDs= 2024-04-16T18:18:21.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qe1DD7xCOKnRh/Dob8CzxtiaoDs=', 'Content-Length': '0'} 2024-04-16T18:18:21.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca30dd1173433b71-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.272 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000045 2024-04-16T18:18:21.272 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.272 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.272 DEBUG:boto:path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.272 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.273 DEBUG:boto:Method: GET 2024-04-16T18:18:21.273 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.273 DEBUG:boto:Data: 2024-04-16T18:18:21.273 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.273 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.273 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.273 DEBUG:boto:Params: {} 2024-04-16T18:18:21.273 DEBUG:boto:Token: None 2024-04-16T18:18:21.273 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw= 2024-04-16T18:18:21.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw=', 'Content-Length': '0'} 2024-04-16T18:18:21.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e04c2cfdd118309-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.318 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.318 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.319 DEBUG:boto:path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.319 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.319 DEBUG:boto:Method: GET 2024-04-16T18:18:21.319 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.319 DEBUG:boto:Data: 2024-04-16T18:18:21.319 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.319 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.319 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.319 DEBUG:boto:Params: {} 2024-04-16T18:18:21.319 DEBUG:boto:Token: None 2024-04-16T18:18:21.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045?versionId=null 2024-04-16T18:18:21.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw= 2024-04-16T18:18:21.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a8Q9V80BgSdWyCgaat9uDJC7KQw=', 'Content-Length': '0'} 2024-04-16T18:18:21.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005c82a26196de890-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.323 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.323 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.323 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.323 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.323 DEBUG:boto:Method: GET 2024-04-16T18:18:21.324 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.324 DEBUG:boto:Data: 2024-04-16T18:18:21.324 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.324 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.324 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.324 DEBUG:boto:Params: {} 2024-04-16T18:18:21.324 DEBUG:boto:Token: None 2024-04-16T18:18:21.324 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d972b7be8f57308-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.329 DEBUG:boto:b'lybtug-51obj000000000000000000451000falseobj000000000000000000452024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.329 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.329 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.329 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.329 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.329 DEBUG:boto:Method: GET 2024-04-16T18:18:21.330 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.330 DEBUG:boto:Data: 2024-04-16T18:18:21.330 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.330 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.330 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.330 DEBUG:boto:Params: {} 2024-04-16T18:18:21.330 DEBUG:boto:Token: None 2024-04-16T18:18:21.330 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.330 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.336 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066a57a8559892af9-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.378 DEBUG:boto:b'lybtug-51obj000000000000000000451000falseobj000000000000000000452024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.379 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.379 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.379 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.379 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.379 DEBUG:boto:Data: 2024-04-16T18:18:21.379 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.379 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.379 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.379 DEBUG:boto:Params: {} 2024-04-16T18:18:21.379 DEBUG:boto:Token: None 2024-04-16T18:18:21.380 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MhA7EMlxOFe50eFsRKnBgMbPe7w= 2024-04-16T18:18:21.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MhA7EMlxOFe50eFsRKnBgMbPe7w=', 'Content-Length': '0'} 2024-04-16T18:18:21.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf458333fc3eee20-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.383 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.383 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.383 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.383 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.384 DEBUG:boto:Data: 2024-04-16T18:18:21.384 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.384 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.384 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.384 DEBUG:boto:Params: {} 2024-04-16T18:18:21.384 DEBUG:boto:Token: None 2024-04-16T18:18:21.384 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MhA7EMlxOFe50eFsRKnBgMbPe7w= 2024-04-16T18:18:21.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MhA7EMlxOFe50eFsRKnBgMbPe7w=', 'Content-Length': '0'} 2024-04-16T18:18:21.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023d7ce55caf80d46-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.387 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000045 2024-04-16T18:18:21.387 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.388 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.388 DEBUG:boto:Method: GET 2024-04-16T18:18:21.388 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.388 DEBUG:boto:Data: 2024-04-16T18:18:21.388 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.388 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.388 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.388 DEBUG:boto:Params: {} 2024-04-16T18:18:21.388 DEBUG:boto:Token: None 2024-04-16T18:18:21.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K/E7kLUWfb8HVOic2+fvypvwPrM= 2024-04-16T18:18:21.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K/E7kLUWfb8HVOic2+fvypvwPrM=', 'Content-Length': '0'} 2024-04-16T18:18:21.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9df6ac68e3c4351-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.434 DEBUG:boto:path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.434 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.435 DEBUG:boto:Method: GET 2024-04-16T18:18:21.435 DEBUG:boto:Path: /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.435 DEBUG:boto:Data: 2024-04-16T18:18:21.435 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.435 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.435 DEBUG:boto:Params: {} 2024-04-16T18:18:21.435 DEBUG:boto:Token: None 2024-04-16T18:18:21.435 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000045 2024-04-16T18:18:21.435 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K/E7kLUWfb8HVOic2+fvypvwPrM= 2024-04-16T18:18:21.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K/E7kLUWfb8HVOic2+fvypvwPrM=', 'Content-Length': '0'} 2024-04-16T18:18:21.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb45d73ebf58f707-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.439 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.439 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.439 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.439 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.439 DEBUG:boto:Method: GET 2024-04-16T18:18:21.439 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.439 DEBUG:boto:Data: 2024-04-16T18:18:21.439 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.439 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.439 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.440 DEBUG:boto:Params: {} 2024-04-16T18:18:21.440 DEBUG:boto:Token: None 2024-04-16T18:18:21.440 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e372080e1919397b-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.486 DEBUG:boto:b'lybtug-51obj000000000000000000451000falseobj000000000000000000452024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.487 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.487 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.487 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.487 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.487 DEBUG:boto:Method: GET 2024-04-16T18:18:21.487 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000045 2024-04-16T18:18:21.487 DEBUG:boto:Data: 2024-04-16T18:18:21.487 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.487 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.487 DEBUG:boto:Params: {} 2024-04-16T18:18:21.488 DEBUG:boto:Token: None 2024-04-16T18:18:21.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.494 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d366e5c1c3541969-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.494 DEBUG:boto:b'lybtug-51obj000000000000000000451000falseobj000000000000000000452024-04-16T18:18:00.047Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.494 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000046 2024-04-16T18:18:21.494 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.495 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.495 DEBUG:boto:path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.495 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.495 DEBUG:boto:Method: GET 2024-04-16T18:18:21.495 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.495 DEBUG:boto:Data: 2024-04-16T18:18:21.495 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.495 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.495 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.495 DEBUG:boto:Params: {} 2024-04-16T18:18:21.495 DEBUG:boto:Token: None 2024-04-16T18:18:21.495 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc= 2024-04-16T18:18:21.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc=', 'Content-Length': '0'} 2024-04-16T18:18:21.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a27ae1ebd923846c-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.542 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.542 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.542 DEBUG:boto:path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.543 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.543 DEBUG:boto:Method: GET 2024-04-16T18:18:21.543 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.543 DEBUG:boto:Data: 2024-04-16T18:18:21.543 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.543 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.543 DEBUG:boto:Params: {} 2024-04-16T18:18:21.543 DEBUG:boto:Token: None 2024-04-16T18:18:21.543 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc= 2024-04-16T18:18:21.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc=', 'Content-Length': '0'} 2024-04-16T18:18:21.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020d4a76bcf250c8b-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.547 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.547 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.547 DEBUG:boto:path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.547 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.547 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.547 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.547 DEBUG:boto:Data: 2024-04-16T18:18:21.547 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.547 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.547 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.548 DEBUG:boto:Params: {} 2024-04-16T18:18:21.548 DEBUG:boto:Token: None 2024-04-16T18:18:21.548 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U5cLvsRZK/lKZCODA2Cq0ckcTIM= 2024-04-16T18:18:21.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U5cLvsRZK/lKZCODA2Cq0ckcTIM=', 'Content-Length': '0'} 2024-04-16T18:18:21.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089b2d2614f5acbd2-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.551 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.551 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.551 DEBUG:boto:path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.551 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.552 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.552 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.552 DEBUG:boto:Data: 2024-04-16T18:18:21.552 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.552 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.552 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.552 DEBUG:boto:Params: {} 2024-04-16T18:18:21.552 DEBUG:boto:Token: None 2024-04-16T18:18:21.552 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U5cLvsRZK/lKZCODA2Cq0ckcTIM= 2024-04-16T18:18:21.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U5cLvsRZK/lKZCODA2Cq0ckcTIM=', 'Content-Length': '0'} 2024-04-16T18:18:21.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d93eab6fdde405b-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.555 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000046 2024-04-16T18:18:21.556 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.556 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.556 DEBUG:boto:path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.556 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.556 DEBUG:boto:Method: GET 2024-04-16T18:18:21.556 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.556 DEBUG:boto:Data: 2024-04-16T18:18:21.556 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.556 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.556 DEBUG:boto:Params: {} 2024-04-16T18:18:21.556 DEBUG:boto:Token: None 2024-04-16T18:18:21.557 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.557 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc= 2024-04-16T18:18:21.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc=', 'Content-Length': '0'} 2024-04-16T18:18:21.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bbb03af76f29fcb-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.602 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.602 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.603 DEBUG:boto:path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.603 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.603 DEBUG:boto:Method: GET 2024-04-16T18:18:21.603 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.603 DEBUG:boto:Data: 2024-04-16T18:18:21.603 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.603 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.603 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.603 DEBUG:boto:Params: {} 2024-04-16T18:18:21.603 DEBUG:boto:Token: None 2024-04-16T18:18:21.603 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046?versionId=null 2024-04-16T18:18:21.603 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc= 2024-04-16T18:18:21.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AreXfx+7C/T/rNjVrCIzTsVcdjc=', 'Content-Length': '0'} 2024-04-16T18:18:21.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009412a77e591c897e-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.607 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.607 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.607 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.607 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.607 DEBUG:boto:Method: GET 2024-04-16T18:18:21.607 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.607 DEBUG:boto:Data: 2024-04-16T18:18:21.607 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.608 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.608 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.608 DEBUG:boto:Params: {} 2024-04-16T18:18:21.608 DEBUG:boto:Token: None 2024-04-16T18:18:21.608 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.612 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d06d2eb01e8f101-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.612 DEBUG:boto:b'lybtug-51obj000000000000000000461000falseobj000000000000000000462024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.613 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.613 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.613 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.613 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.613 DEBUG:boto:Method: GET 2024-04-16T18:18:21.613 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.613 DEBUG:boto:Data: 2024-04-16T18:18:21.613 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.613 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.613 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.613 DEBUG:boto:Params: {} 2024-04-16T18:18:21.613 DEBUG:boto:Token: None 2024-04-16T18:18:21.614 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.614 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.620 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000accd2e78f21a186a-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.662 DEBUG:boto:b'lybtug-51obj000000000000000000461000falseobj000000000000000000462024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.663 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.663 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.663 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.663 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.663 DEBUG:boto:Data: 2024-04-16T18:18:21.663 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.663 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.663 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.663 DEBUG:boto:Params: {} 2024-04-16T18:18:21.663 DEBUG:boto:Token: None 2024-04-16T18:18:21.664 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zj07FGOMV8LbK8s8qokO99SDBBw= 2024-04-16T18:18:21.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zj07FGOMV8LbK8s8qokO99SDBBw=', 'Content-Length': '0'} 2024-04-16T18:18:21.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f82f990419187a7-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.667 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.667 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.667 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.667 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.667 DEBUG:boto:Data: 2024-04-16T18:18:21.667 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.667 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.667 DEBUG:boto:Params: {} 2024-04-16T18:18:21.667 DEBUG:boto:Token: None 2024-04-16T18:18:21.668 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zj07FGOMV8LbK8s8qokO99SDBBw= 2024-04-16T18:18:21.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zj07FGOMV8LbK8s8qokO99SDBBw=', 'Content-Length': '0'} 2024-04-16T18:18:21.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001feb613eaccdfd4-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000046 2024-04-16T18:18:21.671 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.671 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.671 DEBUG:boto:Method: GET 2024-04-16T18:18:21.671 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.671 DEBUG:boto:Data: 2024-04-16T18:18:21.671 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.671 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.672 DEBUG:boto:Params: {} 2024-04-16T18:18:21.672 DEBUG:boto:Token: None 2024-04-16T18:18:21.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PFw4cBkjCG/vrIjT8zYxi+8MOwQ= 2024-04-16T18:18:21.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PFw4cBkjCG/vrIjT8zYxi+8MOwQ=', 'Content-Length': '0'} 2024-04-16T18:18:21.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c58a1286c17a41d-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.718 DEBUG:boto:path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.718 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.719 DEBUG:boto:Method: GET 2024-04-16T18:18:21.719 DEBUG:boto:Path: /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.719 DEBUG:boto:Data: 2024-04-16T18:18:21.719 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.719 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.719 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.719 DEBUG:boto:Params: {} 2024-04-16T18:18:21.719 DEBUG:boto:Token: None 2024-04-16T18:18:21.719 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000046 2024-04-16T18:18:21.719 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PFw4cBkjCG/vrIjT8zYxi+8MOwQ= 2024-04-16T18:18:21.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PFw4cBkjCG/vrIjT8zYxi+8MOwQ=', 'Content-Length': '0'} 2024-04-16T18:18:21.722 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0bcf202e6789268-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.723 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.723 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.723 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.723 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.723 DEBUG:boto:Method: GET 2024-04-16T18:18:21.723 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.723 DEBUG:boto:Data: 2024-04-16T18:18:21.724 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.724 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.724 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.724 DEBUG:boto:Params: {} 2024-04-16T18:18:21.724 DEBUG:boto:Token: None 2024-04-16T18:18:21.724 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.728 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae00028bcd512023-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.770 DEBUG:boto:b'lybtug-51obj000000000000000000461000falseobj000000000000000000462024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.771 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.771 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.771 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.771 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.771 DEBUG:boto:Method: GET 2024-04-16T18:18:21.771 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000046 2024-04-16T18:18:21.771 DEBUG:boto:Data: 2024-04-16T18:18:21.771 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.772 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.772 DEBUG:boto:Params: {} 2024-04-16T18:18:21.772 DEBUG:boto:Token: None 2024-04-16T18:18:21.772 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046d1cd9654da7e98-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.778 DEBUG:boto:b'lybtug-51obj000000000000000000461000falseobj000000000000000000462024-04-16T18:17:56.102Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.779 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000048 2024-04-16T18:18:21.779 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.779 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.779 DEBUG:boto:path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.779 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.779 DEBUG:boto:Method: GET 2024-04-16T18:18:21.779 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.779 DEBUG:boto:Data: 2024-04-16T18:18:21.779 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.780 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.780 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.780 DEBUG:boto:Params: {} 2024-04-16T18:18:21.780 DEBUG:boto:Token: None 2024-04-16T18:18:21.780 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA= 2024-04-16T18:18:21.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA=', 'Content-Length': '0'} 2024-04-16T18:18:21.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074d6646d7b5ed03d-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.826 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.826 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.827 DEBUG:boto:path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.827 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.827 DEBUG:boto:Method: GET 2024-04-16T18:18:21.827 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.827 DEBUG:boto:Data: 2024-04-16T18:18:21.827 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.827 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.827 DEBUG:boto:Params: {} 2024-04-16T18:18:21.827 DEBUG:boto:Token: None 2024-04-16T18:18:21.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.827 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA= 2024-04-16T18:18:21.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA=', 'Content-Length': '0'} 2024-04-16T18:18:21.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004d11c6265f8a0ea-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.831 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.831 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.831 DEBUG:boto:path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.831 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.831 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.831 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.831 DEBUG:boto:Data: 2024-04-16T18:18:21.831 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.831 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.831 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.831 DEBUG:boto:Params: {} 2024-04-16T18:18:21.832 DEBUG:boto:Token: None 2024-04-16T18:18:21.832 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xh9L+hAmeoqpQe3GSvzRdFjR7B0= 2024-04-16T18:18:21.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xh9L+hAmeoqpQe3GSvzRdFjR7B0=', 'Content-Length': '0'} 2024-04-16T18:18:21.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf6ef9ab9fda0145-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.835 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.835 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.835 DEBUG:boto:path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.835 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.835 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.835 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.836 DEBUG:boto:Data: 2024-04-16T18:18:21.836 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.836 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.836 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.836 DEBUG:boto:Params: {} 2024-04-16T18:18:21.836 DEBUG:boto:Token: None 2024-04-16T18:18:21.836 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xh9L+hAmeoqpQe3GSvzRdFjR7B0= 2024-04-16T18:18:21.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xh9L+hAmeoqpQe3GSvzRdFjR7B0=', 'Content-Length': '0'} 2024-04-16T18:18:21.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099c30127bbf836d3-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000048 2024-04-16T18:18:21.839 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.839 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.839 DEBUG:boto:path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.840 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.840 DEBUG:boto:Method: GET 2024-04-16T18:18:21.840 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.840 DEBUG:boto:Data: 2024-04-16T18:18:21.840 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.840 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.840 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.840 DEBUG:boto:Params: {} 2024-04-16T18:18:21.840 DEBUG:boto:Token: None 2024-04-16T18:18:21.840 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA= 2024-04-16T18:18:21.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA=', 'Content-Length': '0'} 2024-04-16T18:18:21.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d4e8a5c8981a389-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.886 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.887 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.887 DEBUG:boto:path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.887 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.887 DEBUG:boto:Method: GET 2024-04-16T18:18:21.887 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.887 DEBUG:boto:Data: 2024-04-16T18:18:21.887 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.887 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.887 DEBUG:boto:Params: {} 2024-04-16T18:18:21.887 DEBUG:boto:Token: None 2024-04-16T18:18:21.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048?versionId=null 2024-04-16T18:18:21.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA= 2024-04-16T18:18:21.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C9JiLY6hi5o8/AiaeKI88l92qxA=', 'Content-Length': '0'} 2024-04-16T18:18:21.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000892111a28f9798ac-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.891 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.891 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.891 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:21.891 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:21.891 DEBUG:boto:Method: GET 2024-04-16T18:18:21.891 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:21.891 DEBUG:boto:Data: 2024-04-16T18:18:21.892 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.892 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.892 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.892 DEBUG:boto:Params: {} 2024-04-16T18:18:21.892 DEBUG:boto:Token: None 2024-04-16T18:18:21.892 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e650a4edd7b8f387-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.896 DEBUG:boto:b'lybtug-51obj000000000000000000481000falseobj000000000000000000482024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.897 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:21.897 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:21.897 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:21.897 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:21.897 DEBUG:boto:Method: GET 2024-04-16T18:18:21.897 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:21.897 DEBUG:boto:Data: 2024-04-16T18:18:21.897 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.897 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.897 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.897 DEBUG:boto:Params: {} 2024-04-16T18:18:21.898 DEBUG:boto:Token: None 2024-04-16T18:18:21.898 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/ 2024-04-16T18:18:21.898 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA= 2024-04-16T18:18:21.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NhmGP/Bxkv+R8Mo9wOhT4LOjjrA=', 'Content-Length': '0'} 2024-04-16T18:18:21.904 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bdab955da12efdb-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.946 DEBUG:boto:b'lybtug-51obj000000000000000000481000falseobj000000000000000000482024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:21.947 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.947 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.947 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.947 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.947 DEBUG:boto:Data: 2024-04-16T18:18:21.947 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.947 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.947 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.947 DEBUG:boto:Params: {} 2024-04-16T18:18:21.947 DEBUG:boto:Token: None 2024-04-16T18:18:21.948 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zYGd59tYXIsblFkm8uJVIoTZUoM= 2024-04-16T18:18:21.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zYGd59tYXIsblFkm8uJVIoTZUoM=', 'Content-Length': '0'} 2024-04-16T18:18:21.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d64fc38cc4231a13-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.951 DEBUG:boto:Method: HEAD 2024-04-16T18:18:21.951 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.952 DEBUG:boto:Data: 2024-04-16T18:18:21.952 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.952 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:21.952 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.952 DEBUG:boto:Params: {} 2024-04-16T18:18:21.952 DEBUG:boto:Token: None 2024-04-16T18:18:21.952 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zYGd59tYXIsblFkm8uJVIoTZUoM= 2024-04-16T18:18:21.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zYGd59tYXIsblFkm8uJVIoTZUoM=', 'Content-Length': '0'} 2024-04-16T18:18:21.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad5efb69b81e6804-00661ec0ed-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:21.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000048 2024-04-16T18:18:21.956 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.956 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.956 DEBUG:boto:Method: GET 2024-04-16T18:18:21.956 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.956 DEBUG:boto:Data: 2024-04-16T18:18:21.956 DEBUG:boto:Headers: {} 2024-04-16T18:18:21.956 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:21.956 DEBUG:boto:Port: 8000 2024-04-16T18:18:21.956 DEBUG:boto:Params: {} 2024-04-16T18:18:21.956 DEBUG:boto:Token: None 2024-04-16T18:18:21.957 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:21 GMT /lybtug-51/obj00000000000000000048 2024-04-16T18:18:21.957 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QlQBq1sQBRJZLt5jpjlSDvGd6y0= 2024-04-16T18:18:21.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QlQBq1sQBRJZLt5jpjlSDvGd6y0=', 'Content-Length': '0'} 2024-04-16T18:18:21.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000412e81a823cbbc04-00661ec0ed-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.002 DEBUG:boto:path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:22.003 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000048 2024-04-16T18:18:22.003 DEBUG:boto:Method: GET 2024-04-16T18:18:22.003 DEBUG:boto:Path: /lybtug-51/obj00000000000000000048 2024-04-16T18:18:22.003 DEBUG:boto:Data: 2024-04-16T18:18:22.003 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.003 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.003 DEBUG:boto:Params: {} 2024-04-16T18:18:22.003 DEBUG:boto:Token: None 2024-04-16T18:18:22.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000048 2024-04-16T18:18:22.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jY9yZ8vleLc/Tc2zpa41c2fnpPA= 2024-04-16T18:18:22.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jY9yZ8vleLc/Tc2zpa41c2fnpPA=', 'Content-Length': '0'} 2024-04-16T18:18:22.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000701ad84ca47366d8-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.006 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.007 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.007 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:22.007 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:22.007 DEBUG:boto:Method: GET 2024-04-16T18:18:22.007 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:22.007 DEBUG:boto:Data: 2024-04-16T18:18:22.007 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.007 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.007 DEBUG:boto:Params: {} 2024-04-16T18:18:22.008 DEBUG:boto:Token: None 2024-04-16T18:18:22.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021abc51521815ce7-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.054 DEBUG:boto:b'lybtug-51obj000000000000000000481000falseobj000000000000000000482024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.055 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.055 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.055 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:22.055 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:22.055 DEBUG:boto:Method: GET 2024-04-16T18:18:22.055 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000048 2024-04-16T18:18:22.055 DEBUG:boto:Data: 2024-04-16T18:18:22.055 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.055 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.056 DEBUG:boto:Params: {} 2024-04-16T18:18:22.056 DEBUG:boto:Token: None 2024-04-16T18:18:22.056 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.062 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009a087ebedf559be-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.063 DEBUG:boto:b'lybtug-51obj000000000000000000481000falseobj000000000000000000482024-04-16T18:18:00.574Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.063 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000049 2024-04-16T18:18:22.063 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.063 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.063 DEBUG:boto:path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.063 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.063 DEBUG:boto:Method: GET 2024-04-16T18:18:22.064 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.064 DEBUG:boto:Data: 2024-04-16T18:18:22.064 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.064 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.064 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.064 DEBUG:boto:Params: {} 2024-04-16T18:18:22.064 DEBUG:boto:Token: None 2024-04-16T18:18:22.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk= 2024-04-16T18:18:22.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk=', 'Content-Length': '0'} 2024-04-16T18:18:22.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc6963d1e48e1f27-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.110 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.110 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.110 DEBUG:boto:path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.111 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.111 DEBUG:boto:Method: GET 2024-04-16T18:18:22.111 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.111 DEBUG:boto:Data: 2024-04-16T18:18:22.111 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.111 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.111 DEBUG:boto:Params: {} 2024-04-16T18:18:22.111 DEBUG:boto:Token: None 2024-04-16T18:18:22.111 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk= 2024-04-16T18:18:22.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk=', 'Content-Length': '0'} 2024-04-16T18:18:22.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d071b00a6df5a8d5-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.115 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.115 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.115 DEBUG:boto:path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.115 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.115 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.115 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.115 DEBUG:boto:Data: 2024-04-16T18:18:22.115 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.115 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.116 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.116 DEBUG:boto:Params: {} 2024-04-16T18:18:22.116 DEBUG:boto:Token: None 2024-04-16T18:18:22.116 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4DVWmIQikj59VuENUKNuCypUy3Y= 2024-04-16T18:18:22.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4DVWmIQikj59VuENUKNuCypUy3Y=', 'Content-Length': '0'} 2024-04-16T18:18:22.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000409ba4ce52545669-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.119 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.119 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.119 DEBUG:boto:path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.119 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.120 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.120 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.120 DEBUG:boto:Data: 2024-04-16T18:18:22.120 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.120 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.120 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.120 DEBUG:boto:Params: {} 2024-04-16T18:18:22.120 DEBUG:boto:Token: None 2024-04-16T18:18:22.120 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4DVWmIQikj59VuENUKNuCypUy3Y= 2024-04-16T18:18:22.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4DVWmIQikj59VuENUKNuCypUy3Y=', 'Content-Length': '0'} 2024-04-16T18:18:22.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000484dc106b0417023-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.123 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000049 2024-04-16T18:18:22.123 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.123 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.124 DEBUG:boto:path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.124 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.124 DEBUG:boto:Method: GET 2024-04-16T18:18:22.124 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.124 DEBUG:boto:Data: 2024-04-16T18:18:22.124 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.124 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.124 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.124 DEBUG:boto:Params: {} 2024-04-16T18:18:22.124 DEBUG:boto:Token: None 2024-04-16T18:18:22.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk= 2024-04-16T18:18:22.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk=', 'Content-Length': '0'} 2024-04-16T18:18:22.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f398bdcb9354908-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.170 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.170 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.170 DEBUG:boto:path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.171 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.171 DEBUG:boto:Method: GET 2024-04-16T18:18:22.171 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.171 DEBUG:boto:Data: 2024-04-16T18:18:22.171 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.171 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.171 DEBUG:boto:Params: {} 2024-04-16T18:18:22.171 DEBUG:boto:Token: None 2024-04-16T18:18:22.171 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049?versionId=null 2024-04-16T18:18:22.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk= 2024-04-16T18:18:22.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yZU3syPktVqX0AgRNixhCOz2Qrk=', 'Content-Length': '0'} 2024-04-16T18:18:22.174 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b8b137d212e3f46-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.175 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.175 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.175 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.175 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.175 DEBUG:boto:Method: GET 2024-04-16T18:18:22.175 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.175 DEBUG:boto:Data: 2024-04-16T18:18:22.175 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.175 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.176 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.176 DEBUG:boto:Params: {} 2024-04-16T18:18:22.176 DEBUG:boto:Token: None 2024-04-16T18:18:22.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000305aa04e5a3b3716-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.180 DEBUG:boto:b'lybtug-51obj000000000000000000491000falseobj000000000000000000492024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.180 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.180 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.180 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.181 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.181 DEBUG:boto:Method: GET 2024-04-16T18:18:22.181 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.181 DEBUG:boto:Data: 2024-04-16T18:18:22.181 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.181 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.181 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.181 DEBUG:boto:Params: {} 2024-04-16T18:18:22.181 DEBUG:boto:Token: None 2024-04-16T18:18:22.181 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.182 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e31cd29266194f4b-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.230 DEBUG:boto:b'lybtug-51obj000000000000000000491000falseobj000000000000000000492024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.231 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.231 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.231 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.231 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.231 DEBUG:boto:Data: 2024-04-16T18:18:22.231 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.231 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.231 DEBUG:boto:Params: {} 2024-04-16T18:18:22.232 DEBUG:boto:Token: None 2024-04-16T18:18:22.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SE0IdPZDnNtD+QhY31v3JlrvsgA= 2024-04-16T18:18:22.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SE0IdPZDnNtD+QhY31v3JlrvsgA=', 'Content-Length': '0'} 2024-04-16T18:18:22.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000066410188d2978c9-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.235 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.235 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.235 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.235 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.235 DEBUG:boto:Data: 2024-04-16T18:18:22.235 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.236 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.236 DEBUG:boto:Params: {} 2024-04-16T18:18:22.236 DEBUG:boto:Token: None 2024-04-16T18:18:22.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SE0IdPZDnNtD+QhY31v3JlrvsgA= 2024-04-16T18:18:22.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SE0IdPZDnNtD+QhY31v3JlrvsgA=', 'Content-Length': '0'} 2024-04-16T18:18:22.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000680df89d3a43e75b-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000049 2024-04-16T18:18:22.239 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.239 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.239 DEBUG:boto:Method: GET 2024-04-16T18:18:22.239 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.239 DEBUG:boto:Data: 2024-04-16T18:18:22.239 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.239 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.240 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.240 DEBUG:boto:Params: {} 2024-04-16T18:18:22.240 DEBUG:boto:Token: None 2024-04-16T18:18:22.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FteET22Z+8RyVucgoUgJFjpFyCI= 2024-04-16T18:18:22.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FteET22Z+8RyVucgoUgJFjpFyCI=', 'Content-Length': '0'} 2024-04-16T18:18:22.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b445dc1cdbd73d7c-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.286 DEBUG:boto:path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.286 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.287 DEBUG:boto:Method: GET 2024-04-16T18:18:22.287 DEBUG:boto:Path: /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.287 DEBUG:boto:Data: 2024-04-16T18:18:22.287 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.287 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.287 DEBUG:boto:Params: {} 2024-04-16T18:18:22.287 DEBUG:boto:Token: None 2024-04-16T18:18:22.287 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000049 2024-04-16T18:18:22.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FteET22Z+8RyVucgoUgJFjpFyCI= 2024-04-16T18:18:22.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FteET22Z+8RyVucgoUgJFjpFyCI=', 'Content-Length': '0'} 2024-04-16T18:18:22.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:57 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1db7b0b396c3e0b-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.291 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.291 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.291 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.291 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.291 DEBUG:boto:Method: GET 2024-04-16T18:18:22.291 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.291 DEBUG:boto:Data: 2024-04-16T18:18:22.291 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.291 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.292 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.292 DEBUG:boto:Params: {} 2024-04-16T18:18:22.292 DEBUG:boto:Token: None 2024-04-16T18:18:22.292 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010ba432626342dca-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.338 DEBUG:boto:b'lybtug-51obj000000000000000000491000falseobj000000000000000000492024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.339 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.339 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.339 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.339 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.339 DEBUG:boto:Method: GET 2024-04-16T18:18:22.339 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000049 2024-04-16T18:18:22.339 DEBUG:boto:Data: 2024-04-16T18:18:22.339 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.339 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.340 DEBUG:boto:Params: {} 2024-04-16T18:18:22.340 DEBUG:boto:Token: None 2024-04-16T18:18:22.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.346 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ae62bb0eee68e2e-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.346 DEBUG:boto:b'lybtug-51obj000000000000000000491000falseobj000000000000000000492024-04-16T18:17:57.946Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000050 2024-04-16T18:18:22.347 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.347 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.347 DEBUG:boto:path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.347 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.347 DEBUG:boto:Method: GET 2024-04-16T18:18:22.347 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.347 DEBUG:boto:Data: 2024-04-16T18:18:22.347 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.347 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.347 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.347 DEBUG:boto:Params: {} 2024-04-16T18:18:22.348 DEBUG:boto:Token: None 2024-04-16T18:18:22.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA= 2024-04-16T18:18:22.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA=', 'Content-Length': '0'} 2024-04-16T18:18:22.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc7f5938a77bca9c-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.394 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.394 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.395 DEBUG:boto:path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.395 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.395 DEBUG:boto:Method: GET 2024-04-16T18:18:22.395 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.395 DEBUG:boto:Data: 2024-04-16T18:18:22.395 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.395 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.395 DEBUG:boto:Params: {} 2024-04-16T18:18:22.395 DEBUG:boto:Token: None 2024-04-16T18:18:22.395 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA= 2024-04-16T18:18:22.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA=', 'Content-Length': '0'} 2024-04-16T18:18:22.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000354f3866f2c3bcc0-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.399 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.399 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.399 DEBUG:boto:path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.399 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.399 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.400 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.400 DEBUG:boto:Data: 2024-04-16T18:18:22.400 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.400 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.400 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.400 DEBUG:boto:Params: {} 2024-04-16T18:18:22.400 DEBUG:boto:Token: None 2024-04-16T18:18:22.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qLDzIjOHqoGVrj7GuV1hYhTflFE= 2024-04-16T18:18:22.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qLDzIjOHqoGVrj7GuV1hYhTflFE=', 'Content-Length': '0'} 2024-04-16T18:18:22.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057c2ef829eea63d6-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.404 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.404 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.404 DEBUG:boto:path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.404 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.404 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.404 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.404 DEBUG:boto:Data: 2024-04-16T18:18:22.404 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.404 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.404 DEBUG:boto:Params: {} 2024-04-16T18:18:22.405 DEBUG:boto:Token: None 2024-04-16T18:18:22.405 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qLDzIjOHqoGVrj7GuV1hYhTflFE= 2024-04-16T18:18:22.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qLDzIjOHqoGVrj7GuV1hYhTflFE=', 'Content-Length': '0'} 2024-04-16T18:18:22.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da40760d930e78c2-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.408 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000050 2024-04-16T18:18:22.408 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.408 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.408 DEBUG:boto:path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.408 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.409 DEBUG:boto:Method: GET 2024-04-16T18:18:22.409 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.409 DEBUG:boto:Data: 2024-04-16T18:18:22.409 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.409 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.409 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.409 DEBUG:boto:Params: {} 2024-04-16T18:18:22.409 DEBUG:boto:Token: None 2024-04-16T18:18:22.409 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.409 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA= 2024-04-16T18:18:22.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA=', 'Content-Length': '0'} 2024-04-16T18:18:22.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097ea91b3725f5fc1-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.454 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.454 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.454 DEBUG:boto:path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.455 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.455 DEBUG:boto:Method: GET 2024-04-16T18:18:22.455 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.455 DEBUG:boto:Data: 2024-04-16T18:18:22.455 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.455 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.455 DEBUG:boto:Params: {} 2024-04-16T18:18:22.455 DEBUG:boto:Token: None 2024-04-16T18:18:22.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050?versionId=null 2024-04-16T18:18:22.455 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA= 2024-04-16T18:18:22.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h/g1rhf6XG+SLwS/YVEnYM10SjA=', 'Content-Length': '0'} 2024-04-16T18:18:22.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004880b8c753c48f7c-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.459 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.459 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.459 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.459 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.459 DEBUG:boto:Method: GET 2024-04-16T18:18:22.459 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.459 DEBUG:boto:Data: 2024-04-16T18:18:22.459 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.460 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.460 DEBUG:boto:Params: {} 2024-04-16T18:18:22.460 DEBUG:boto:Token: None 2024-04-16T18:18:22.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c61e0958a39338a-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.465 DEBUG:boto:b'lybtug-51obj000000000000000000501000falseobj000000000000000000502024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.465 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.465 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.465 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.465 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.465 DEBUG:boto:Method: GET 2024-04-16T18:18:22.465 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.465 DEBUG:boto:Data: 2024-04-16T18:18:22.466 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.466 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.466 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.466 DEBUG:boto:Params: {} 2024-04-16T18:18:22.466 DEBUG:boto:Token: None 2024-04-16T18:18:22.466 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.472 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b565a437f26ce771-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.514 DEBUG:boto:b'lybtug-51obj000000000000000000501000falseobj000000000000000000502024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.515 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.515 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.515 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.515 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.515 DEBUG:boto:Data: 2024-04-16T18:18:22.515 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.515 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.515 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.515 DEBUG:boto:Params: {} 2024-04-16T18:18:22.515 DEBUG:boto:Token: None 2024-04-16T18:18:22.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wfOz2vMmTFuwWcRLdaiJ2elgLlE= 2024-04-16T18:18:22.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wfOz2vMmTFuwWcRLdaiJ2elgLlE=', 'Content-Length': '0'} 2024-04-16T18:18:22.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2a7a3826d24d642-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.519 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.519 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.519 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.519 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.519 DEBUG:boto:Data: 2024-04-16T18:18:22.519 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.519 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.519 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.519 DEBUG:boto:Params: {} 2024-04-16T18:18:22.520 DEBUG:boto:Token: None 2024-04-16T18:18:22.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wfOz2vMmTFuwWcRLdaiJ2elgLlE= 2024-04-16T18:18:22.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wfOz2vMmTFuwWcRLdaiJ2elgLlE=', 'Content-Length': '0'} 2024-04-16T18:18:22.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000586f593d8adba595-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.523 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000050 2024-04-16T18:18:22.523 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.523 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.524 DEBUG:boto:Method: GET 2024-04-16T18:18:22.524 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.524 DEBUG:boto:Data: 2024-04-16T18:18:22.524 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.524 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.524 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.524 DEBUG:boto:Params: {} 2024-04-16T18:18:22.524 DEBUG:boto:Token: None 2024-04-16T18:18:22.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6CDKHgL1zNLBecBjZqcCbM2sYPQ= 2024-04-16T18:18:22.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6CDKHgL1zNLBecBjZqcCbM2sYPQ=', 'Content-Length': '0'} 2024-04-16T18:18:22.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6f013621b5378c0-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.570 DEBUG:boto:path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.570 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.570 DEBUG:boto:Method: GET 2024-04-16T18:18:22.571 DEBUG:boto:Path: /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.571 DEBUG:boto:Data: 2024-04-16T18:18:22.571 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.571 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.571 DEBUG:boto:Params: {} 2024-04-16T18:18:22.571 DEBUG:boto:Token: None 2024-04-16T18:18:22.571 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000050 2024-04-16T18:18:22.571 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6CDKHgL1zNLBecBjZqcCbM2sYPQ= 2024-04-16T18:18:22.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6CDKHgL1zNLBecBjZqcCbM2sYPQ=', 'Content-Length': '0'} 2024-04-16T18:18:22.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025a130665c5cd960-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.575 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.575 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.575 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.575 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.575 DEBUG:boto:Method: GET 2024-04-16T18:18:22.575 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.576 DEBUG:boto:Data: 2024-04-16T18:18:22.576 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.576 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.576 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.576 DEBUG:boto:Params: {} 2024-04-16T18:18:22.576 DEBUG:boto:Token: None 2024-04-16T18:18:22.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7c29c67d44d89ce-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.622 DEBUG:boto:b'lybtug-51obj000000000000000000501000falseobj000000000000000000502024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.623 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.623 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.623 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.623 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.623 DEBUG:boto:Method: GET 2024-04-16T18:18:22.623 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000050 2024-04-16T18:18:22.623 DEBUG:boto:Data: 2024-04-16T18:18:22.623 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.623 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.623 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.623 DEBUG:boto:Params: {} 2024-04-16T18:18:22.624 DEBUG:boto:Token: None 2024-04-16T18:18:22.624 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.630 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075d51180978d91c0-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.630 DEBUG:boto:b'lybtug-51obj000000000000000000501000falseobj000000000000000000502024-04-16T18:17:59.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.630 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000053 2024-04-16T18:18:22.631 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.631 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.631 DEBUG:boto:path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.631 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.631 DEBUG:boto:Method: GET 2024-04-16T18:18:22.631 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.631 DEBUG:boto:Data: 2024-04-16T18:18:22.631 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.631 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.631 DEBUG:boto:Params: {} 2024-04-16T18:18:22.631 DEBUG:boto:Token: None 2024-04-16T18:18:22.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo= 2024-04-16T18:18:22.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo=', 'Content-Length': '0'} 2024-04-16T18:18:22.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000848fbd3fdf66865e-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.679 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.679 DEBUG:boto:path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.679 DEBUG:boto:Method: GET 2024-04-16T18:18:22.679 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.679 DEBUG:boto:Data: 2024-04-16T18:18:22.679 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.679 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.679 DEBUG:boto:Params: {} 2024-04-16T18:18:22.680 DEBUG:boto:Token: None 2024-04-16T18:18:22.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo= 2024-04-16T18:18:22.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo=', 'Content-Length': '0'} 2024-04-16T18:18:22.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007457ffd8a95d7da9-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.683 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.683 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.683 DEBUG:boto:path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.683 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.683 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.684 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.684 DEBUG:boto:Data: 2024-04-16T18:18:22.684 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.684 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.684 DEBUG:boto:Params: {} 2024-04-16T18:18:22.684 DEBUG:boto:Token: None 2024-04-16T18:18:22.684 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hP/E7PdPfZd/SypF+pvhalJS5pU= 2024-04-16T18:18:22.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hP/E7PdPfZd/SypF+pvhalJS5pU=', 'Content-Length': '0'} 2024-04-16T18:18:22.687 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c3ce801f454c740-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.688 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.688 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.688 DEBUG:boto:path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.688 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.688 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.688 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.688 DEBUG:boto:Data: 2024-04-16T18:18:22.688 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.688 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.688 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.688 DEBUG:boto:Params: {} 2024-04-16T18:18:22.688 DEBUG:boto:Token: None 2024-04-16T18:18:22.689 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.689 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hP/E7PdPfZd/SypF+pvhalJS5pU= 2024-04-16T18:18:22.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hP/E7PdPfZd/SypF+pvhalJS5pU=', 'Content-Length': '0'} 2024-04-16T18:18:22.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b991a832ad2f8347-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.691 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000053 2024-04-16T18:18:22.692 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.692 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.692 DEBUG:boto:path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.692 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.692 DEBUG:boto:Method: GET 2024-04-16T18:18:22.692 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.692 DEBUG:boto:Data: 2024-04-16T18:18:22.692 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.692 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.692 DEBUG:boto:Params: {} 2024-04-16T18:18:22.692 DEBUG:boto:Token: None 2024-04-16T18:18:22.693 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.693 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo= 2024-04-16T18:18:22.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo=', 'Content-Length': '0'} 2024-04-16T18:18:22.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfb55cb658e195cc-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.738 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.738 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.739 DEBUG:boto:path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.739 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.739 DEBUG:boto:Method: GET 2024-04-16T18:18:22.739 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.739 DEBUG:boto:Data: 2024-04-16T18:18:22.739 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.739 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.739 DEBUG:boto:Params: {} 2024-04-16T18:18:22.739 DEBUG:boto:Token: None 2024-04-16T18:18:22.739 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053?versionId=null 2024-04-16T18:18:22.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo= 2024-04-16T18:18:22.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4DsJlToqTaKtrr5rK/TVbK0FPGo=', 'Content-Length': '0'} 2024-04-16T18:18:22.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046fec7386598cf40-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.743 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.743 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.743 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.743 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.743 DEBUG:boto:Method: GET 2024-04-16T18:18:22.743 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.744 DEBUG:boto:Data: 2024-04-16T18:18:22.744 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.744 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.744 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.744 DEBUG:boto:Params: {} 2024-04-16T18:18:22.744 DEBUG:boto:Token: None 2024-04-16T18:18:22.744 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5ac1d4496bd4ad6-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.749 DEBUG:boto:b'lybtug-51obj000000000000000000531000falseobj000000000000000000532024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.749 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.749 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.750 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.750 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.750 DEBUG:boto:Method: GET 2024-04-16T18:18:22.750 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.750 DEBUG:boto:Data: 2024-04-16T18:18:22.750 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.750 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.750 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.750 DEBUG:boto:Params: {} 2024-04-16T18:18:22.750 DEBUG:boto:Token: None 2024-04-16T18:18:22.751 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.751 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab0aa4d663702623-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.798 DEBUG:boto:b'lybtug-51obj000000000000000000531000falseobj000000000000000000532024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.799 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.799 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.799 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.799 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.799 DEBUG:boto:Data: 2024-04-16T18:18:22.799 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.799 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.799 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.799 DEBUG:boto:Params: {} 2024-04-16T18:18:22.799 DEBUG:boto:Token: None 2024-04-16T18:18:22.800 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gF2nGjr62TXNhfVebOfvvo2vJ78= 2024-04-16T18:18:22.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gF2nGjr62TXNhfVebOfvvo2vJ78=', 'Content-Length': '0'} 2024-04-16T18:18:22.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fc9f38a0800a7cb-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.803 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.803 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.803 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.803 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.803 DEBUG:boto:Data: 2024-04-16T18:18:22.803 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.804 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.804 DEBUG:boto:Params: {} 2024-04-16T18:18:22.804 DEBUG:boto:Token: None 2024-04-16T18:18:22.804 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gF2nGjr62TXNhfVebOfvvo2vJ78= 2024-04-16T18:18:22.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gF2nGjr62TXNhfVebOfvvo2vJ78=', 'Content-Length': '0'} 2024-04-16T18:18:22.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062c9f8b168a623bc-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.807 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000053 2024-04-16T18:18:22.807 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.807 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.807 DEBUG:boto:Method: GET 2024-04-16T18:18:22.807 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.807 DEBUG:boto:Data: 2024-04-16T18:18:22.807 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.808 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.808 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.808 DEBUG:boto:Params: {} 2024-04-16T18:18:22.808 DEBUG:boto:Token: None 2024-04-16T18:18:22.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AfJbTuWt2yYxfUx7cyXUvAiDZEQ= 2024-04-16T18:18:22.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AfJbTuWt2yYxfUx7cyXUvAiDZEQ=', 'Content-Length': '0'} 2024-04-16T18:18:22.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ecda71703f46913-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.854 DEBUG:boto:path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.854 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.855 DEBUG:boto:Method: GET 2024-04-16T18:18:22.855 DEBUG:boto:Path: /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.855 DEBUG:boto:Data: 2024-04-16T18:18:22.855 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.855 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.855 DEBUG:boto:Params: {} 2024-04-16T18:18:22.855 DEBUG:boto:Token: None 2024-04-16T18:18:22.855 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000053 2024-04-16T18:18:22.855 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AfJbTuWt2yYxfUx7cyXUvAiDZEQ= 2024-04-16T18:18:22.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AfJbTuWt2yYxfUx7cyXUvAiDZEQ=', 'Content-Length': '0'} 2024-04-16T18:18:22.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000466557b8352fd1fd-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.859 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.859 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.859 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.859 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.859 DEBUG:boto:Method: GET 2024-04-16T18:18:22.859 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.859 DEBUG:boto:Data: 2024-04-16T18:18:22.859 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.859 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.859 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.859 DEBUG:boto:Params: {} 2024-04-16T18:18:22.860 DEBUG:boto:Token: None 2024-04-16T18:18:22.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7b90ed32e16fb38-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.906 DEBUG:boto:b'lybtug-51obj000000000000000000531000falseobj000000000000000000532024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.907 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:22.907 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:22.907 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.907 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.907 DEBUG:boto:Method: GET 2024-04-16T18:18:22.907 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000053 2024-04-16T18:18:22.907 DEBUG:boto:Data: 2024-04-16T18:18:22.907 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.907 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.907 DEBUG:boto:Params: {} 2024-04-16T18:18:22.908 DEBUG:boto:Token: None 2024-04-16T18:18:22.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/ 2024-04-16T18:18:22.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo= 2024-04-16T18:18:22.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yk8FfFAePxWwEAeY+NuzH5sKAIo=', 'Content-Length': '0'} 2024-04-16T18:18:22.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a40ef1e6121d53f4-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.914 DEBUG:boto:b'lybtug-51obj000000000000000000531000falseobj000000000000000000532024-04-16T18:18:00.722Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:22.914 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000055 2024-04-16T18:18:22.915 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.915 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.915 DEBUG:boto:path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.915 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.915 DEBUG:boto:Method: GET 2024-04-16T18:18:22.915 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.915 DEBUG:boto:Data: 2024-04-16T18:18:22.915 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.915 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.915 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.915 DEBUG:boto:Params: {} 2024-04-16T18:18:22.916 DEBUG:boto:Token: None 2024-04-16T18:18:22.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fAF+NAUjVuiYSfIUjKc8jR+VpL8= 2024-04-16T18:18:22.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fAF+NAUjVuiYSfIUjKc8jR+VpL8=', 'Content-Length': '0'} 2024-04-16T18:18:22.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba48bd1191c967d3-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.962 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.962 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.962 DEBUG:boto:path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.962 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.963 DEBUG:boto:Method: GET 2024-04-16T18:18:22.963 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.963 DEBUG:boto:Data: 2024-04-16T18:18:22.963 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.963 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.963 DEBUG:boto:Params: {} 2024-04-16T18:18:22.963 DEBUG:boto:Token: None 2024-04-16T18:18:22.963 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fAF+NAUjVuiYSfIUjKc8jR+VpL8= 2024-04-16T18:18:22.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fAF+NAUjVuiYSfIUjKc8jR+VpL8=', 'Content-Length': '0'} 2024-04-16T18:18:22.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005142aaa11a970f7c-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.967 DEBUG:boto:path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.967 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.967 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.967 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.968 DEBUG:boto:Data: 2024-04-16T18:18:22.968 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.968 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.968 DEBUG:boto:Params: {} 2024-04-16T18:18:22.968 DEBUG:boto:Token: None 2024-04-16T18:18:22.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a9njy9uX0QSoRZll5Ws30uAr68Q= 2024-04-16T18:18:22.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a9njy9uX0QSoRZll5Ws30uAr68Q=', 'Content-Length': '0'} 2024-04-16T18:18:22.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b74183b2ff5773e7-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.971 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.971 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.972 DEBUG:boto:path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.972 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.972 DEBUG:boto:Method: HEAD 2024-04-16T18:18:22.972 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.972 DEBUG:boto:Data: 2024-04-16T18:18:22.972 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.972 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:22.972 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.972 DEBUG:boto:Params: {} 2024-04-16T18:18:22.972 DEBUG:boto:Token: None 2024-04-16T18:18:22.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.973 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:a9njy9uX0QSoRZll5Ws30uAr68Q= 2024-04-16T18:18:22.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a9njy9uX0QSoRZll5Ws30uAr68Q=', 'Content-Length': '0'} 2024-04-16T18:18:22.975 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000886adbe51d6c16aa-00661ec0ee-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:22.976 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000055 2024-04-16T18:18:22.976 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.976 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:22.976 DEBUG:boto:path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.976 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.976 DEBUG:boto:Method: GET 2024-04-16T18:18:22.976 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.977 DEBUG:boto:Data: 2024-04-16T18:18:22.977 DEBUG:boto:Headers: {} 2024-04-16T18:18:22.977 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:22.977 DEBUG:boto:Port: 8000 2024-04-16T18:18:22.977 DEBUG:boto:Params: {} 2024-04-16T18:18:22.977 DEBUG:boto:Token: None 2024-04-16T18:18:22.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:22 GMT /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:22.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fAF+NAUjVuiYSfIUjKc8jR+VpL8= 2024-04-16T18:18:22.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fAF+NAUjVuiYSfIUjKc8jR+VpL8=', 'Content-Length': '0'} 2024-04-16T18:18:22.981 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002786adc957dcc2f0-00661ec0ee-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.022 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.023 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.023 DEBUG:boto:path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:23.023 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:23.023 DEBUG:boto:Method: GET 2024-04-16T18:18:23.023 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:23.023 DEBUG:boto:Data: 2024-04-16T18:18:23.023 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.023 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.023 DEBUG:boto:Params: {} 2024-04-16T18:18:23.023 DEBUG:boto:Token: None 2024-04-16T18:18:23.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000055?versionId=null 2024-04-16T18:18:23.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KArUm1pxmdJ9eGHyEcbvlMVjbYo= 2024-04-16T18:18:23.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KArUm1pxmdJ9eGHyEcbvlMVjbYo=', 'Content-Length': '0'} 2024-04-16T18:18:23.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4e7b2e8fb8a4cd0-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.027 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.028 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.028 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.028 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.028 DEBUG:boto:Method: GET 2024-04-16T18:18:23.028 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.028 DEBUG:boto:Data: 2024-04-16T18:18:23.028 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.028 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.028 DEBUG:boto:Params: {} 2024-04-16T18:18:23.028 DEBUG:boto:Token: None 2024-04-16T18:18:23.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba14e07378e6c7f2-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.033 DEBUG:boto:b'lybtug-51obj000000000000000000551000falseobj000000000000000000552024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.034 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.034 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.034 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.034 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.034 DEBUG:boto:Method: GET 2024-04-16T18:18:23.034 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.034 DEBUG:boto:Data: 2024-04-16T18:18:23.034 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.034 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.034 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.034 DEBUG:boto:Params: {} 2024-04-16T18:18:23.035 DEBUG:boto:Token: None 2024-04-16T18:18:23.035 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.035 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.041 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ec2c8ad71002209-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.082 DEBUG:boto:b'lybtug-51obj000000000000000000551000falseobj000000000000000000552024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.083 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.083 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.083 DEBUG:boto:Data: 2024-04-16T18:18:23.083 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.083 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.083 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.083 DEBUG:boto:Params: {} 2024-04-16T18:18:23.083 DEBUG:boto:Token: None 2024-04-16T18:18:23.084 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JeYnOidFEb7DiboO4abmS+TP0EY= 2024-04-16T18:18:23.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JeYnOidFEb7DiboO4abmS+TP0EY=', 'Content-Length': '0'} 2024-04-16T18:18:23.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c8378a3e4bd19e4-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.087 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.087 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.087 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.087 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.087 DEBUG:boto:Data: 2024-04-16T18:18:23.087 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.088 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.088 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.088 DEBUG:boto:Params: {} 2024-04-16T18:18:23.088 DEBUG:boto:Token: None 2024-04-16T18:18:23.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JeYnOidFEb7DiboO4abmS+TP0EY= 2024-04-16T18:18:23.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JeYnOidFEb7DiboO4abmS+TP0EY=', 'Content-Length': '0'} 2024-04-16T18:18:23.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cba15690ebf29a46-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000055 2024-04-16T18:18:23.091 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.092 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.092 DEBUG:boto:Method: GET 2024-04-16T18:18:23.092 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.092 DEBUG:boto:Data: 2024-04-16T18:18:23.092 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.092 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.092 DEBUG:boto:Params: {} 2024-04-16T18:18:23.092 DEBUG:boto:Token: None 2024-04-16T18:18:23.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.093 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zO+ZiqhbBw0YiziHUKMK04Tc4U4= 2024-04-16T18:18:23.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zO+ZiqhbBw0YiziHUKMK04Tc4U4=', 'Content-Length': '0'} 2024-04-16T18:18:23.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000348d591a0574c8b7-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.138 DEBUG:boto:path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.138 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.139 DEBUG:boto:Method: GET 2024-04-16T18:18:23.139 DEBUG:boto:Path: /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.139 DEBUG:boto:Data: 2024-04-16T18:18:23.139 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.139 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.139 DEBUG:boto:Params: {} 2024-04-16T18:18:23.139 DEBUG:boto:Token: None 2024-04-16T18:18:23.139 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000055 2024-04-16T18:18:23.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zO+ZiqhbBw0YiziHUKMK04Tc4U4= 2024-04-16T18:18:23.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zO+ZiqhbBw0YiziHUKMK04Tc4U4=', 'Content-Length': '0'} 2024-04-16T18:18:23.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:55 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006098500e5cfff95e-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.143 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.143 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.143 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.143 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.143 DEBUG:boto:Method: GET 2024-04-16T18:18:23.143 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.143 DEBUG:boto:Data: 2024-04-16T18:18:23.143 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.143 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.144 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.144 DEBUG:boto:Params: {} 2024-04-16T18:18:23.144 DEBUG:boto:Token: None 2024-04-16T18:18:23.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c9dced54676d248-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.190 DEBUG:boto:b'lybtug-51obj000000000000000000551000falseobj000000000000000000552024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.191 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.191 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.191 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.191 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.191 DEBUG:boto:Method: GET 2024-04-16T18:18:23.191 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000055 2024-04-16T18:18:23.191 DEBUG:boto:Data: 2024-04-16T18:18:23.191 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.191 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.192 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.192 DEBUG:boto:Params: {} 2024-04-16T18:18:23.192 DEBUG:boto:Token: None 2024-04-16T18:18:23.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.198 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008142780c60a72798-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.198 DEBUG:boto:b'lybtug-51obj000000000000000000551000falseobj000000000000000000552024-04-16T18:17:55.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.199 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000056 2024-04-16T18:18:23.199 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.199 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.199 DEBUG:boto:path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.199 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.199 DEBUG:boto:Method: GET 2024-04-16T18:18:23.199 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.199 DEBUG:boto:Data: 2024-04-16T18:18:23.199 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.200 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.200 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.200 DEBUG:boto:Params: {} 2024-04-16T18:18:23.200 DEBUG:boto:Token: None 2024-04-16T18:18:23.200 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8= 2024-04-16T18:18:23.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8=', 'Content-Length': '0'} 2024-04-16T18:18:23.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fbe24a4b9b8d1f8-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.246 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.246 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.247 DEBUG:boto:path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.247 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.247 DEBUG:boto:Method: GET 2024-04-16T18:18:23.247 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.247 DEBUG:boto:Data: 2024-04-16T18:18:23.247 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.247 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.247 DEBUG:boto:Params: {} 2024-04-16T18:18:23.247 DEBUG:boto:Token: None 2024-04-16T18:18:23.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8= 2024-04-16T18:18:23.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8=', 'Content-Length': '0'} 2024-04-16T18:18:23.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c1ceae38c900766-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.251 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.252 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.252 DEBUG:boto:path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.252 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.252 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.252 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.252 DEBUG:boto:Data: 2024-04-16T18:18:23.252 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.252 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.252 DEBUG:boto:Params: {} 2024-04-16T18:18:23.252 DEBUG:boto:Token: None 2024-04-16T18:18:23.253 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.253 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+FxWywrOYg44V5C+NblGEZhznW4= 2024-04-16T18:18:23.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+FxWywrOYg44V5C+NblGEZhznW4=', 'Content-Length': '0'} 2024-04-16T18:18:23.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000836639ba4b193016-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.256 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.256 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.256 DEBUG:boto:path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.256 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.256 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.256 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.257 DEBUG:boto:Data: 2024-04-16T18:18:23.257 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.257 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.257 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.257 DEBUG:boto:Params: {} 2024-04-16T18:18:23.257 DEBUG:boto:Token: None 2024-04-16T18:18:23.257 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+FxWywrOYg44V5C+NblGEZhznW4= 2024-04-16T18:18:23.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+FxWywrOYg44V5C+NblGEZhznW4=', 'Content-Length': '0'} 2024-04-16T18:18:23.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011e7d86ed0c5b11a-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.261 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000056 2024-04-16T18:18:23.261 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.261 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.261 DEBUG:boto:path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.261 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.261 DEBUG:boto:Method: GET 2024-04-16T18:18:23.261 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.261 DEBUG:boto:Data: 2024-04-16T18:18:23.261 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.261 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.261 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.261 DEBUG:boto:Params: {} 2024-04-16T18:18:23.262 DEBUG:boto:Token: None 2024-04-16T18:18:23.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8= 2024-04-16T18:18:23.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8=', 'Content-Length': '0'} 2024-04-16T18:18:23.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025a41be25bb55f3b-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.306 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.307 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.307 DEBUG:boto:path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.307 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.307 DEBUG:boto:Method: GET 2024-04-16T18:18:23.307 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.307 DEBUG:boto:Data: 2024-04-16T18:18:23.307 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.307 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.307 DEBUG:boto:Params: {} 2024-04-16T18:18:23.307 DEBUG:boto:Token: None 2024-04-16T18:18:23.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056?versionId=null 2024-04-16T18:18:23.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8= 2024-04-16T18:18:23.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/J/qRTmyuyQ7Uk1ZqULiYeHl4/8=', 'Content-Length': '0'} 2024-04-16T18:18:23.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e93d1dc3b1f42b9-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.311 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.311 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.311 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.312 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.312 DEBUG:boto:Method: GET 2024-04-16T18:18:23.312 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.312 DEBUG:boto:Data: 2024-04-16T18:18:23.312 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.312 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.312 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.312 DEBUG:boto:Params: {} 2024-04-16T18:18:23.312 DEBUG:boto:Token: None 2024-04-16T18:18:23.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.317 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006bf591205e414be0-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.317 DEBUG:boto:b'lybtug-51obj000000000000000000561000falseobj000000000000000000562024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.317 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.318 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.318 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.318 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.318 DEBUG:boto:Method: GET 2024-04-16T18:18:23.318 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.318 DEBUG:boto:Data: 2024-04-16T18:18:23.318 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.318 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.318 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.318 DEBUG:boto:Params: {} 2024-04-16T18:18:23.318 DEBUG:boto:Token: None 2024-04-16T18:18:23.318 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.319 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5e731fa036abbc0-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.366 DEBUG:boto:b'lybtug-51obj000000000000000000561000falseobj000000000000000000562024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.367 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.367 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.367 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.367 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.367 DEBUG:boto:Data: 2024-04-16T18:18:23.367 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.367 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.367 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.367 DEBUG:boto:Params: {} 2024-04-16T18:18:23.367 DEBUG:boto:Token: None 2024-04-16T18:18:23.367 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c9pjoSxed7YYWTamD4Eoh8XGQrw= 2024-04-16T18:18:23.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c9pjoSxed7YYWTamD4Eoh8XGQrw=', 'Content-Length': '0'} 2024-04-16T18:18:23.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031ad8f62ac8b9039-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.371 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.371 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.371 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.371 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.371 DEBUG:boto:Data: 2024-04-16T18:18:23.372 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.372 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.372 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.372 DEBUG:boto:Params: {} 2024-04-16T18:18:23.372 DEBUG:boto:Token: None 2024-04-16T18:18:23.372 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c9pjoSxed7YYWTamD4Eoh8XGQrw= 2024-04-16T18:18:23.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c9pjoSxed7YYWTamD4Eoh8XGQrw=', 'Content-Length': '0'} 2024-04-16T18:18:23.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000242d975f49a406dd-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.375 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000056 2024-04-16T18:18:23.375 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.376 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.376 DEBUG:boto:Method: GET 2024-04-16T18:18:23.376 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.376 DEBUG:boto:Data: 2024-04-16T18:18:23.376 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.376 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.376 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.376 DEBUG:boto:Params: {} 2024-04-16T18:18:23.376 DEBUG:boto:Token: None 2024-04-16T18:18:23.376 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AaSZOK5gbJuc+nJ8G2h9XST6z1c= 2024-04-16T18:18:23.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AaSZOK5gbJuc+nJ8G2h9XST6z1c=', 'Content-Length': '0'} 2024-04-16T18:18:23.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf8aec78152d280f-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.422 DEBUG:boto:path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.422 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.423 DEBUG:boto:Method: GET 2024-04-16T18:18:23.423 DEBUG:boto:Path: /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.423 DEBUG:boto:Data: 2024-04-16T18:18:23.423 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.423 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.423 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.423 DEBUG:boto:Params: {} 2024-04-16T18:18:23.423 DEBUG:boto:Token: None 2024-04-16T18:18:23.423 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000056 2024-04-16T18:18:23.423 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AaSZOK5gbJuc+nJ8G2h9XST6z1c= 2024-04-16T18:18:23.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AaSZOK5gbJuc+nJ8G2h9XST6z1c=', 'Content-Length': '0'} 2024-04-16T18:18:23.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc70b639c5d77b4f-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.426 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.427 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.427 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.427 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.427 DEBUG:boto:Method: GET 2024-04-16T18:18:23.427 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.427 DEBUG:boto:Data: 2024-04-16T18:18:23.427 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.427 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.427 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.427 DEBUG:boto:Params: {} 2024-04-16T18:18:23.427 DEBUG:boto:Token: None 2024-04-16T18:18:23.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.432 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000dae9acdcdb0793b-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.474 DEBUG:boto:b'lybtug-51obj000000000000000000561000falseobj000000000000000000562024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.475 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.475 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.475 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.475 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.475 DEBUG:boto:Method: GET 2024-04-16T18:18:23.475 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000056 2024-04-16T18:18:23.475 DEBUG:boto:Data: 2024-04-16T18:18:23.475 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.475 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.476 DEBUG:boto:Params: {} 2024-04-16T18:18:23.476 DEBUG:boto:Token: None 2024-04-16T18:18:23.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.482 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000633606b6530ec09d-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.482 DEBUG:boto:b'lybtug-51obj000000000000000000561000falseobj000000000000000000562024-04-16T18:17:59.463Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.482 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000057 2024-04-16T18:18:23.482 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.482 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.483 DEBUG:boto:path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.483 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.483 DEBUG:boto:Method: GET 2024-04-16T18:18:23.483 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.483 DEBUG:boto:Data: 2024-04-16T18:18:23.483 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.483 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.483 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.483 DEBUG:boto:Params: {} 2024-04-16T18:18:23.483 DEBUG:boto:Token: None 2024-04-16T18:18:23.483 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo= 2024-04-16T18:18:23.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo=', 'Content-Length': '0'} 2024-04-16T18:18:23.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046aaa7cbc132f267-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.530 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.530 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.531 DEBUG:boto:path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.531 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.531 DEBUG:boto:Method: GET 2024-04-16T18:18:23.531 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.531 DEBUG:boto:Data: 2024-04-16T18:18:23.531 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.531 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.531 DEBUG:boto:Params: {} 2024-04-16T18:18:23.531 DEBUG:boto:Token: None 2024-04-16T18:18:23.531 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo= 2024-04-16T18:18:23.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo=', 'Content-Length': '0'} 2024-04-16T18:18:23.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012607c3faf78bd5a-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.535 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.535 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.535 DEBUG:boto:path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.535 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.535 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.535 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.536 DEBUG:boto:Data: 2024-04-16T18:18:23.536 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.536 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.536 DEBUG:boto:Params: {} 2024-04-16T18:18:23.536 DEBUG:boto:Token: None 2024-04-16T18:18:23.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1/pPgnx6BL8hEoK/w46jqd8opA= 2024-04-16T18:18:23.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1/pPgnx6BL8hEoK/w46jqd8opA=', 'Content-Length': '0'} 2024-04-16T18:18:23.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b13d26fc86a33d7a-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.540 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.540 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.540 DEBUG:boto:path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.540 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.540 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.540 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.540 DEBUG:boto:Data: 2024-04-16T18:18:23.540 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.540 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.540 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.541 DEBUG:boto:Params: {} 2024-04-16T18:18:23.541 DEBUG:boto:Token: None 2024-04-16T18:18:23.541 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1/pPgnx6BL8hEoK/w46jqd8opA= 2024-04-16T18:18:23.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1/pPgnx6BL8hEoK/w46jqd8opA=', 'Content-Length': '0'} 2024-04-16T18:18:23.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c7d15357e0449e6-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000057 2024-04-16T18:18:23.544 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.544 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.544 DEBUG:boto:path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.544 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.544 DEBUG:boto:Method: GET 2024-04-16T18:18:23.544 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.544 DEBUG:boto:Data: 2024-04-16T18:18:23.544 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.545 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.545 DEBUG:boto:Params: {} 2024-04-16T18:18:23.545 DEBUG:boto:Token: None 2024-04-16T18:18:23.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo= 2024-04-16T18:18:23.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo=', 'Content-Length': '0'} 2024-04-16T18:18:23.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d80848d02fab2bda-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.590 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.591 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.591 DEBUG:boto:path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.591 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.591 DEBUG:boto:Method: GET 2024-04-16T18:18:23.591 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.591 DEBUG:boto:Data: 2024-04-16T18:18:23.591 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.591 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.591 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.591 DEBUG:boto:Params: {} 2024-04-16T18:18:23.592 DEBUG:boto:Token: None 2024-04-16T18:18:23.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057?versionId=null 2024-04-16T18:18:23.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo= 2024-04-16T18:18:23.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8HhLrR2solFs3CvXVGirPc89BZo=', 'Content-Length': '0'} 2024-04-16T18:18:23.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b4519cc3e4fd52-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.595 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.595 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.595 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.595 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.596 DEBUG:boto:Method: GET 2024-04-16T18:18:23.596 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.596 DEBUG:boto:Data: 2024-04-16T18:18:23.596 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.596 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.596 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.596 DEBUG:boto:Params: {} 2024-04-16T18:18:23.596 DEBUG:boto:Token: None 2024-04-16T18:18:23.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.601 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2921e2ac3b03252-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.601 DEBUG:boto:b'lybtug-51obj000000000000000000571000falseobj000000000000000000572024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.601 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.601 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.601 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.601 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.602 DEBUG:boto:Method: GET 2024-04-16T18:18:23.602 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.602 DEBUG:boto:Data: 2024-04-16T18:18:23.602 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.602 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.602 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.602 DEBUG:boto:Params: {} 2024-04-16T18:18:23.602 DEBUG:boto:Token: None 2024-04-16T18:18:23.602 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.602 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c9609764a372132-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.650 DEBUG:boto:b'lybtug-51obj000000000000000000571000falseobj000000000000000000572024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.651 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.651 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.651 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.651 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.651 DEBUG:boto:Data: 2024-04-16T18:18:23.651 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.651 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.651 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.651 DEBUG:boto:Params: {} 2024-04-16T18:18:23.652 DEBUG:boto:Token: None 2024-04-16T18:18:23.652 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ctC+se9ekMCHq+7HgKzCug5xZw= 2024-04-16T18:18:23.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ctC+se9ekMCHq+7HgKzCug5xZw=', 'Content-Length': '0'} 2024-04-16T18:18:23.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe59454734457433-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.655 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.655 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.655 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.656 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.656 DEBUG:boto:Data: 2024-04-16T18:18:23.656 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.656 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.656 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.656 DEBUG:boto:Params: {} 2024-04-16T18:18:23.656 DEBUG:boto:Token: None 2024-04-16T18:18:23.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ctC+se9ekMCHq+7HgKzCug5xZw= 2024-04-16T18:18:23.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ctC+se9ekMCHq+7HgKzCug5xZw=', 'Content-Length': '0'} 2024-04-16T18:18:23.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080a9a6d4816b4671-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.660 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000057 2024-04-16T18:18:23.660 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.660 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.660 DEBUG:boto:Method: GET 2024-04-16T18:18:23.660 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.660 DEBUG:boto:Data: 2024-04-16T18:18:23.660 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.660 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.660 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.660 DEBUG:boto:Params: {} 2024-04-16T18:18:23.661 DEBUG:boto:Token: None 2024-04-16T18:18:23.661 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.661 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GJgnAf6sv50IyG71BOf1db56xpo= 2024-04-16T18:18:23.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GJgnAf6sv50IyG71BOf1db56xpo=', 'Content-Length': '0'} 2024-04-16T18:18:23.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5c1bb74ecf07a91-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.706 DEBUG:boto:path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.706 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.707 DEBUG:boto:Method: GET 2024-04-16T18:18:23.707 DEBUG:boto:Path: /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.707 DEBUG:boto:Data: 2024-04-16T18:18:23.707 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.707 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.707 DEBUG:boto:Params: {} 2024-04-16T18:18:23.707 DEBUG:boto:Token: None 2024-04-16T18:18:23.707 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000057 2024-04-16T18:18:23.707 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GJgnAf6sv50IyG71BOf1db56xpo= 2024-04-16T18:18:23.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GJgnAf6sv50IyG71BOf1db56xpo=', 'Content-Length': '0'} 2024-04-16T18:18:23.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac39861db8449adf-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.711 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.711 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.711 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.711 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.711 DEBUG:boto:Method: GET 2024-04-16T18:18:23.712 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.712 DEBUG:boto:Data: 2024-04-16T18:18:23.712 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.712 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.712 DEBUG:boto:Params: {} 2024-04-16T18:18:23.712 DEBUG:boto:Token: None 2024-04-16T18:18:23.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080964e4c1585d568-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.758 DEBUG:boto:b'lybtug-51obj000000000000000000571000falseobj000000000000000000572024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.759 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.759 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.759 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.759 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.759 DEBUG:boto:Method: GET 2024-04-16T18:18:23.759 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000057 2024-04-16T18:18:23.759 DEBUG:boto:Data: 2024-04-16T18:18:23.759 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.759 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.759 DEBUG:boto:Params: {} 2024-04-16T18:18:23.760 DEBUG:boto:Token: None 2024-04-16T18:18:23.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a24e4a607ab39511-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.766 DEBUG:boto:b'lybtug-51obj000000000000000000571000falseobj000000000000000000572024-04-16T18:18:00.671Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.767 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000058 2024-04-16T18:18:23.767 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.767 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.767 DEBUG:boto:path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.767 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.767 DEBUG:boto:Method: GET 2024-04-16T18:18:23.767 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.767 DEBUG:boto:Data: 2024-04-16T18:18:23.767 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.767 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.767 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.768 DEBUG:boto:Params: {} 2024-04-16T18:18:23.768 DEBUG:boto:Token: None 2024-04-16T18:18:23.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ= 2024-04-16T18:18:23.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ=', 'Content-Length': '0'} 2024-04-16T18:18:23.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c35a6dba3e4d3ea2-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.814 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.815 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.815 DEBUG:boto:path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.815 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.815 DEBUG:boto:Method: GET 2024-04-16T18:18:23.815 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.815 DEBUG:boto:Data: 2024-04-16T18:18:23.815 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.815 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.815 DEBUG:boto:Params: {} 2024-04-16T18:18:23.815 DEBUG:boto:Token: None 2024-04-16T18:18:23.816 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ= 2024-04-16T18:18:23.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ=', 'Content-Length': '0'} 2024-04-16T18:18:23.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057e08697cbc810ac-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.819 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.819 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.819 DEBUG:boto:path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.819 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.820 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.820 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.820 DEBUG:boto:Data: 2024-04-16T18:18:23.820 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.820 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.820 DEBUG:boto:Params: {} 2024-04-16T18:18:23.820 DEBUG:boto:Token: None 2024-04-16T18:18:23.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ZpUm1oCEQqlZepxn6WQCdnjWEc= 2024-04-16T18:18:23.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ZpUm1oCEQqlZepxn6WQCdnjWEc=', 'Content-Length': '0'} 2024-04-16T18:18:23.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088faa6b05bc296eb-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.823 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.823 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.824 DEBUG:boto:path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.824 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.824 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.824 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.824 DEBUG:boto:Data: 2024-04-16T18:18:23.824 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.824 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.824 DEBUG:boto:Params: {} 2024-04-16T18:18:23.824 DEBUG:boto:Token: None 2024-04-16T18:18:23.824 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ZpUm1oCEQqlZepxn6WQCdnjWEc= 2024-04-16T18:18:23.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ZpUm1oCEQqlZepxn6WQCdnjWEc=', 'Content-Length': '0'} 2024-04-16T18:18:23.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009977237f539cea50-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.828 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000058 2024-04-16T18:18:23.828 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.828 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.828 DEBUG:boto:path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.828 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.828 DEBUG:boto:Method: GET 2024-04-16T18:18:23.828 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.828 DEBUG:boto:Data: 2024-04-16T18:18:23.828 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.829 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.829 DEBUG:boto:Params: {} 2024-04-16T18:18:23.829 DEBUG:boto:Token: None 2024-04-16T18:18:23.829 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ= 2024-04-16T18:18:23.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ=', 'Content-Length': '0'} 2024-04-16T18:18:23.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca56fe36013e7da5-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.874 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.874 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.875 DEBUG:boto:path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.875 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.875 DEBUG:boto:Method: GET 2024-04-16T18:18:23.875 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.875 DEBUG:boto:Data: 2024-04-16T18:18:23.875 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.875 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.875 DEBUG:boto:Params: {} 2024-04-16T18:18:23.875 DEBUG:boto:Token: None 2024-04-16T18:18:23.875 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058?versionId=null 2024-04-16T18:18:23.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ= 2024-04-16T18:18:23.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kc2UXZupjotGGLCMbMQbZ+9/YIQ=', 'Content-Length': '0'} 2024-04-16T18:18:23.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b5ea438b764af24-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.879 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.879 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.879 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.880 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.880 DEBUG:boto:Method: GET 2024-04-16T18:18:23.880 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.880 DEBUG:boto:Data: 2024-04-16T18:18:23.880 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.880 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.880 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.880 DEBUG:boto:Params: {} 2024-04-16T18:18:23.880 DEBUG:boto:Token: None 2024-04-16T18:18:23.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddba6569f9ca0290-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.885 DEBUG:boto:b'lybtug-51obj000000000000000000581000falseobj000000000000000000582024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.886 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.886 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.886 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.886 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.886 DEBUG:boto:Method: GET 2024-04-16T18:18:23.886 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.886 DEBUG:boto:Data: 2024-04-16T18:18:23.886 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.886 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.886 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.886 DEBUG:boto:Params: {} 2024-04-16T18:18:23.887 DEBUG:boto:Token: None 2024-04-16T18:18:23.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.887 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:23.893 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb95690e8df68d79-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.934 DEBUG:boto:b'lybtug-51obj000000000000000000581000falseobj000000000000000000582024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:23.935 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.935 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.935 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.935 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.935 DEBUG:boto:Data: 2024-04-16T18:18:23.935 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.935 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.935 DEBUG:boto:Params: {} 2024-04-16T18:18:23.935 DEBUG:boto:Token: None 2024-04-16T18:18:23.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5wQ0Bp5htnIDyV9khnksnnAc5tc= 2024-04-16T18:18:23.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5wQ0Bp5htnIDyV9khnksnnAc5tc=', 'Content-Length': '0'} 2024-04-16T18:18:23.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000783ad50e5c09c542-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.939 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.939 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.939 DEBUG:boto:Method: HEAD 2024-04-16T18:18:23.939 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.939 DEBUG:boto:Data: 2024-04-16T18:18:23.940 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.940 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.940 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.940 DEBUG:boto:Params: {} 2024-04-16T18:18:23.940 DEBUG:boto:Token: None 2024-04-16T18:18:23.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5wQ0Bp5htnIDyV9khnksnnAc5tc= 2024-04-16T18:18:23.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5wQ0Bp5htnIDyV9khnksnnAc5tc=', 'Content-Length': '0'} 2024-04-16T18:18:23.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034721d398df19836-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.943 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000058 2024-04-16T18:18:23.943 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.943 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.943 DEBUG:boto:Method: GET 2024-04-16T18:18:23.944 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.944 DEBUG:boto:Data: 2024-04-16T18:18:23.944 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.944 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.944 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.944 DEBUG:boto:Params: {} 2024-04-16T18:18:23.944 DEBUG:boto:Token: None 2024-04-16T18:18:23.944 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4chxAU/2Xr/UNyQ+YgHbZzow1+g= 2024-04-16T18:18:23.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4chxAU/2Xr/UNyQ+YgHbZzow1+g=', 'Content-Length': '0'} 2024-04-16T18:18:23.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079e4bb97c81ec736-00661ec0ef-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.990 DEBUG:boto:path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.991 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.991 DEBUG:boto:Method: GET 2024-04-16T18:18:23.991 DEBUG:boto:Path: /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.991 DEBUG:boto:Data: 2024-04-16T18:18:23.991 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:23.991 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.991 DEBUG:boto:Params: {} 2024-04-16T18:18:23.991 DEBUG:boto:Token: None 2024-04-16T18:18:23.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/obj00000000000000000058 2024-04-16T18:18:23.991 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4chxAU/2Xr/UNyQ+YgHbZzow1+g= 2024-04-16T18:18:23.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4chxAU/2Xr/UNyQ+YgHbZzow1+g=', 'Content-Length': '0'} 2024-04-16T18:18:23.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:56 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad3e986d9c1ad70b-00661ec0ef-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:23.995 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:23.995 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:23.995 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.995 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.995 DEBUG:boto:Method: GET 2024-04-16T18:18:23.995 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:23.995 DEBUG:boto:Data: 2024-04-16T18:18:23.995 DEBUG:boto:Headers: {} 2024-04-16T18:18:23.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:23.996 DEBUG:boto:Port: 8000 2024-04-16T18:18:23.996 DEBUG:boto:Params: {} 2024-04-16T18:18:23.996 DEBUG:boto:Token: None 2024-04-16T18:18:23.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:23 GMT /lybtug-51/ 2024-04-16T18:18:23.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I= 2024-04-16T18:18:23.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dRIAl0sEmxRHmuL0ony3pOkAs8I=', 'Content-Length': '0'} 2024-04-16T18:18:24.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000023181e4f7c7ff18-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.042 DEBUG:boto:b'lybtug-51obj000000000000000000581000falseobj000000000000000000582024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.043 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.043 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.043 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:24.043 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:24.043 DEBUG:boto:Method: GET 2024-04-16T18:18:24.043 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000058 2024-04-16T18:18:24.043 DEBUG:boto:Data: 2024-04-16T18:18:24.043 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.043 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.043 DEBUG:boto:Params: {} 2024-04-16T18:18:24.044 DEBUG:boto:Token: None 2024-04-16T18:18:24.044 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053765a05b048324b-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.050 DEBUG:boto:b'lybtug-51obj000000000000000000581000falseobj000000000000000000582024-04-16T18:17:56.349Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.051 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000060 2024-04-16T18:18:24.051 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.051 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.051 DEBUG:boto:path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.051 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.051 DEBUG:boto:Method: GET 2024-04-16T18:18:24.051 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.051 DEBUG:boto:Data: 2024-04-16T18:18:24.052 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.052 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.052 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.052 DEBUG:boto:Params: {} 2024-04-16T18:18:24.052 DEBUG:boto:Token: None 2024-04-16T18:18:24.052 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk= 2024-04-16T18:18:24.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk=', 'Content-Length': '0'} 2024-04-16T18:18:24.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001681d54171a8822b-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.098 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.098 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.099 DEBUG:boto:path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.099 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.099 DEBUG:boto:Method: GET 2024-04-16T18:18:24.099 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.099 DEBUG:boto:Data: 2024-04-16T18:18:24.099 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.099 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.099 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.099 DEBUG:boto:Params: {} 2024-04-16T18:18:24.099 DEBUG:boto:Token: None 2024-04-16T18:18:24.099 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk= 2024-04-16T18:18:24.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk=', 'Content-Length': '0'} 2024-04-16T18:18:24.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4db33b072d30938-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.103 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.103 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.103 DEBUG:boto:path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.103 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.103 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.103 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.103 DEBUG:boto:Data: 2024-04-16T18:18:24.103 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.104 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.104 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.104 DEBUG:boto:Params: {} 2024-04-16T18:18:24.104 DEBUG:boto:Token: None 2024-04-16T18:18:24.104 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4w5qWr2aw/HnUizKowoqlTG2pB4= 2024-04-16T18:18:24.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4w5qWr2aw/HnUizKowoqlTG2pB4=', 'Content-Length': '0'} 2024-04-16T18:18:24.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005aae06e143d7142-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.107 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.107 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.107 DEBUG:boto:path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.108 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.108 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.108 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.108 DEBUG:boto:Data: 2024-04-16T18:18:24.108 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.108 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.108 DEBUG:boto:Params: {} 2024-04-16T18:18:24.108 DEBUG:boto:Token: None 2024-04-16T18:18:24.108 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4w5qWr2aw/HnUizKowoqlTG2pB4= 2024-04-16T18:18:24.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4w5qWr2aw/HnUizKowoqlTG2pB4=', 'Content-Length': '0'} 2024-04-16T18:18:24.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ed8fa8ab2ef5789-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.112 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000060 2024-04-16T18:18:24.112 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.112 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.112 DEBUG:boto:path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.112 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.112 DEBUG:boto:Method: GET 2024-04-16T18:18:24.112 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.112 DEBUG:boto:Data: 2024-04-16T18:18:24.112 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.113 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.113 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.113 DEBUG:boto:Params: {} 2024-04-16T18:18:24.113 DEBUG:boto:Token: None 2024-04-16T18:18:24.113 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk= 2024-04-16T18:18:24.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk=', 'Content-Length': '0'} 2024-04-16T18:18:24.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb1cd9e9d9dd8f5a-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.158 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.158 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.158 DEBUG:boto:path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.159 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.159 DEBUG:boto:Method: GET 2024-04-16T18:18:24.159 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.159 DEBUG:boto:Data: 2024-04-16T18:18:24.159 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.159 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.159 DEBUG:boto:Params: {} 2024-04-16T18:18:24.159 DEBUG:boto:Token: None 2024-04-16T18:18:24.159 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060?versionId=null 2024-04-16T18:18:24.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk= 2024-04-16T18:18:24.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NcJbBaaC23+bkMacFudw355yPPk=', 'Content-Length': '0'} 2024-04-16T18:18:24.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000507a5a2edf8577bb-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.163 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.163 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.163 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.163 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.163 DEBUG:boto:Method: GET 2024-04-16T18:18:24.163 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.163 DEBUG:boto:Data: 2024-04-16T18:18:24.163 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.163 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.163 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.164 DEBUG:boto:Params: {} 2024-04-16T18:18:24.164 DEBUG:boto:Token: None 2024-04-16T18:18:24.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005736a805adc903ca-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.168 DEBUG:boto:b'lybtug-51obj000000000000000000601000falseobj000000000000000000602024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.169 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.169 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.169 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.169 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.169 DEBUG:boto:Method: GET 2024-04-16T18:18:24.169 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.169 DEBUG:boto:Data: 2024-04-16T18:18:24.169 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.169 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.169 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.169 DEBUG:boto:Params: {} 2024-04-16T18:18:24.170 DEBUG:boto:Token: None 2024-04-16T18:18:24.170 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.170 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bc25f8f5d1868e1-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.218 DEBUG:boto:b'lybtug-51obj000000000000000000601000falseobj000000000000000000602024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.219 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.219 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.219 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.219 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.219 DEBUG:boto:Data: 2024-04-16T18:18:24.219 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.219 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.219 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.219 DEBUG:boto:Params: {} 2024-04-16T18:18:24.219 DEBUG:boto:Token: None 2024-04-16T18:18:24.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RA1zsLh2H2V5l/i0kCWd5PGR3Qs= 2024-04-16T18:18:24.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RA1zsLh2H2V5l/i0kCWd5PGR3Qs=', 'Content-Length': '0'} 2024-04-16T18:18:24.222 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019afc03f5c2e6cda-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.223 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.223 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.223 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.223 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.223 DEBUG:boto:Data: 2024-04-16T18:18:24.223 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.223 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.223 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.223 DEBUG:boto:Params: {} 2024-04-16T18:18:24.223 DEBUG:boto:Token: None 2024-04-16T18:18:24.223 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RA1zsLh2H2V5l/i0kCWd5PGR3Qs= 2024-04-16T18:18:24.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RA1zsLh2H2V5l/i0kCWd5PGR3Qs=', 'Content-Length': '0'} 2024-04-16T18:18:24.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003370c38792a793ba-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.227 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000060 2024-04-16T18:18:24.227 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.227 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.227 DEBUG:boto:Method: GET 2024-04-16T18:18:24.227 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.227 DEBUG:boto:Data: 2024-04-16T18:18:24.227 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.227 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.227 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.228 DEBUG:boto:Params: {} 2024-04-16T18:18:24.228 DEBUG:boto:Token: None 2024-04-16T18:18:24.228 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6wYkB+JjYR2Pga6tIFWPoWW3/Jc= 2024-04-16T18:18:24.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6wYkB+JjYR2Pga6tIFWPoWW3/Jc=', 'Content-Length': '0'} 2024-04-16T18:18:24.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:03 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acbf689861dd03f7-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.274 DEBUG:boto:path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.275 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.275 DEBUG:boto:Method: GET 2024-04-16T18:18:24.275 DEBUG:boto:Path: /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.275 DEBUG:boto:Data: 2024-04-16T18:18:24.275 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.275 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.275 DEBUG:boto:Params: {} 2024-04-16T18:18:24.275 DEBUG:boto:Token: None 2024-04-16T18:18:24.275 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000060 2024-04-16T18:18:24.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6wYkB+JjYR2Pga6tIFWPoWW3/Jc= 2024-04-16T18:18:24.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6wYkB+JjYR2Pga6tIFWPoWW3/Jc=', 'Content-Length': '0'} 2024-04-16T18:18:24.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b74e27157a9a15ed-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.279 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.279 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.279 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.279 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.279 DEBUG:boto:Method: GET 2024-04-16T18:18:24.280 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.280 DEBUG:boto:Data: 2024-04-16T18:18:24.280 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.280 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.280 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.280 DEBUG:boto:Params: {} 2024-04-16T18:18:24.280 DEBUG:boto:Token: None 2024-04-16T18:18:24.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f5df97e27d84914-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.326 DEBUG:boto:b'lybtug-51obj000000000000000000601000falseobj000000000000000000602024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.327 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.327 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.327 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.327 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.327 DEBUG:boto:Method: GET 2024-04-16T18:18:24.327 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000060 2024-04-16T18:18:24.327 DEBUG:boto:Data: 2024-04-16T18:18:24.327 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.327 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.328 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.328 DEBUG:boto:Params: {} 2024-04-16T18:18:24.328 DEBUG:boto:Token: None 2024-04-16T18:18:24.328 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.333 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7ad38b23c1884b1-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.334 DEBUG:boto:b'lybtug-51obj000000000000000000601000falseobj000000000000000000602024-04-16T18:18:01.022Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.334 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000062 2024-04-16T18:18:24.334 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.334 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.334 DEBUG:boto:path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.334 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.334 DEBUG:boto:Method: GET 2024-04-16T18:18:24.335 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.335 DEBUG:boto:Data: 2024-04-16T18:18:24.335 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.335 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.335 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.335 DEBUG:boto:Params: {} 2024-04-16T18:18:24.335 DEBUG:boto:Token: None 2024-04-16T18:18:24.335 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0= 2024-04-16T18:18:24.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0=', 'Content-Length': '0'} 2024-04-16T18:18:24.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d8f9945a6f1909e-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.382 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.382 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.383 DEBUG:boto:path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.383 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.383 DEBUG:boto:Method: GET 2024-04-16T18:18:24.383 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.383 DEBUG:boto:Data: 2024-04-16T18:18:24.383 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.383 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.383 DEBUG:boto:Params: {} 2024-04-16T18:18:24.383 DEBUG:boto:Token: None 2024-04-16T18:18:24.383 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0= 2024-04-16T18:18:24.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0=', 'Content-Length': '0'} 2024-04-16T18:18:24.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fcc251cc8024c4d-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.387 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.387 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.387 DEBUG:boto:path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.387 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.387 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.387 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.387 DEBUG:boto:Data: 2024-04-16T18:18:24.387 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.387 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.387 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.387 DEBUG:boto:Params: {} 2024-04-16T18:18:24.388 DEBUG:boto:Token: None 2024-04-16T18:18:24.388 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x/kK/tfIOg4bMLJqB7vnhQBHULY= 2024-04-16T18:18:24.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x/kK/tfIOg4bMLJqB7vnhQBHULY=', 'Content-Length': '0'} 2024-04-16T18:18:24.390 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008714d759b63b6278-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.391 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.391 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.391 DEBUG:boto:path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.391 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.391 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.391 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.391 DEBUG:boto:Data: 2024-04-16T18:18:24.391 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.392 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.392 DEBUG:boto:Params: {} 2024-04-16T18:18:24.392 DEBUG:boto:Token: None 2024-04-16T18:18:24.392 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x/kK/tfIOg4bMLJqB7vnhQBHULY= 2024-04-16T18:18:24.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x/kK/tfIOg4bMLJqB7vnhQBHULY=', 'Content-Length': '0'} 2024-04-16T18:18:24.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082179ba62f7b40f7-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.395 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000062 2024-04-16T18:18:24.395 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.395 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.395 DEBUG:boto:path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.395 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.395 DEBUG:boto:Method: GET 2024-04-16T18:18:24.396 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.396 DEBUG:boto:Data: 2024-04-16T18:18:24.396 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.396 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.396 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.396 DEBUG:boto:Params: {} 2024-04-16T18:18:24.396 DEBUG:boto:Token: None 2024-04-16T18:18:24.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0= 2024-04-16T18:18:24.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0=', 'Content-Length': '0'} 2024-04-16T18:18:24.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090b1891d2c7dd957-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.442 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.442 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.443 DEBUG:boto:path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.443 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.443 DEBUG:boto:Method: GET 2024-04-16T18:18:24.443 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.443 DEBUG:boto:Data: 2024-04-16T18:18:24.443 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.443 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.443 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.443 DEBUG:boto:Params: {} 2024-04-16T18:18:24.443 DEBUG:boto:Token: None 2024-04-16T18:18:24.443 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062?versionId=null 2024-04-16T18:18:24.443 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0= 2024-04-16T18:18:24.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1L7fdhFRUMGnJ6UXSfa7V5ER6i0=', 'Content-Length': '0'} 2024-04-16T18:18:24.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a214f8d015650f91-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.446 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.447 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.447 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.447 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.447 DEBUG:boto:Method: GET 2024-04-16T18:18:24.447 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.447 DEBUG:boto:Data: 2024-04-16T18:18:24.447 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.447 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.447 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.447 DEBUG:boto:Params: {} 2024-04-16T18:18:24.447 DEBUG:boto:Token: None 2024-04-16T18:18:24.448 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009216a77b5242fd97-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.453 DEBUG:boto:b'lybtug-51obj000000000000000000621000falseobj000000000000000000622024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.453 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.453 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.453 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.453 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.453 DEBUG:boto:Method: GET 2024-04-16T18:18:24.453 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.453 DEBUG:boto:Data: 2024-04-16T18:18:24.453 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.454 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.454 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.454 DEBUG:boto:Params: {} 2024-04-16T18:18:24.454 DEBUG:boto:Token: None 2024-04-16T18:18:24.454 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.454 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.460 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000274f154254bd96b1-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.502 DEBUG:boto:b'lybtug-51obj000000000000000000621000falseobj000000000000000000622024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.503 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.503 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.503 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.503 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.503 DEBUG:boto:Data: 2024-04-16T18:18:24.503 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.503 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.503 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.503 DEBUG:boto:Params: {} 2024-04-16T18:18:24.504 DEBUG:boto:Token: None 2024-04-16T18:18:24.504 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ay8jOMTuE7yxlDV0ptM5gU4laPs= 2024-04-16T18:18:24.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ay8jOMTuE7yxlDV0ptM5gU4laPs=', 'Content-Length': '0'} 2024-04-16T18:18:24.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e5032270cd97922-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.507 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.507 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.507 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.507 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.507 DEBUG:boto:Data: 2024-04-16T18:18:24.507 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.508 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.508 DEBUG:boto:Params: {} 2024-04-16T18:18:24.508 DEBUG:boto:Token: None 2024-04-16T18:18:24.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ay8jOMTuE7yxlDV0ptM5gU4laPs= 2024-04-16T18:18:24.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ay8jOMTuE7yxlDV0ptM5gU4laPs=', 'Content-Length': '0'} 2024-04-16T18:18:24.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abbc2fc770dba383-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.511 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000062 2024-04-16T18:18:24.511 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.512 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.512 DEBUG:boto:Method: GET 2024-04-16T18:18:24.512 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.512 DEBUG:boto:Data: 2024-04-16T18:18:24.512 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.512 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.512 DEBUG:boto:Params: {} 2024-04-16T18:18:24.512 DEBUG:boto:Token: None 2024-04-16T18:18:24.512 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:djzdcCDPJc1q1v3e4YRbth4dl7Q= 2024-04-16T18:18:24.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:djzdcCDPJc1q1v3e4YRbth4dl7Q=', 'Content-Length': '0'} 2024-04-16T18:18:24.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043a02e7d8418f4bd-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.558 DEBUG:boto:path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.558 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.558 DEBUG:boto:Method: GET 2024-04-16T18:18:24.559 DEBUG:boto:Path: /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.559 DEBUG:boto:Data: 2024-04-16T18:18:24.559 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.559 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.559 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.559 DEBUG:boto:Params: {} 2024-04-16T18:18:24.559 DEBUG:boto:Token: None 2024-04-16T18:18:24.559 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000062 2024-04-16T18:18:24.559 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:djzdcCDPJc1q1v3e4YRbth4dl7Q= 2024-04-16T18:18:24.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:djzdcCDPJc1q1v3e4YRbth4dl7Q=', 'Content-Length': '0'} 2024-04-16T18:18:24.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fefd29654184230-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.563 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.563 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.563 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.563 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.563 DEBUG:boto:Method: GET 2024-04-16T18:18:24.563 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.563 DEBUG:boto:Data: 2024-04-16T18:18:24.563 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.564 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.564 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.564 DEBUG:boto:Params: {} 2024-04-16T18:18:24.564 DEBUG:boto:Token: None 2024-04-16T18:18:24.564 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbf22bd34070b39c-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.610 DEBUG:boto:b'lybtug-51obj000000000000000000621000falseobj000000000000000000622024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.611 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.611 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.611 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.611 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.611 DEBUG:boto:Method: GET 2024-04-16T18:18:24.611 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000062 2024-04-16T18:18:24.611 DEBUG:boto:Data: 2024-04-16T18:18:24.611 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.611 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.611 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.611 DEBUG:boto:Params: {} 2024-04-16T18:18:24.612 DEBUG:boto:Token: None 2024-04-16T18:18:24.612 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3ce7704a0987a6c-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.618 DEBUG:boto:b'lybtug-51obj000000000000000000621000falseobj000000000000000000622024-04-16T18:18:01.094Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000065 2024-04-16T18:18:24.619 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.619 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.619 DEBUG:boto:path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.619 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.619 DEBUG:boto:Method: GET 2024-04-16T18:18:24.619 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.619 DEBUG:boto:Data: 2024-04-16T18:18:24.619 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.619 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.619 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.620 DEBUG:boto:Params: {} 2024-04-16T18:18:24.620 DEBUG:boto:Token: None 2024-04-16T18:18:24.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU= 2024-04-16T18:18:24.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU=', 'Content-Length': '0'} 2024-04-16T18:18:24.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ada3ad6ef316457-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.666 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.666 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.667 DEBUG:boto:path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.667 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.667 DEBUG:boto:Method: GET 2024-04-16T18:18:24.667 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.667 DEBUG:boto:Data: 2024-04-16T18:18:24.667 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.667 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.667 DEBUG:boto:Params: {} 2024-04-16T18:18:24.667 DEBUG:boto:Token: None 2024-04-16T18:18:24.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU= 2024-04-16T18:18:24.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU=', 'Content-Length': '0'} 2024-04-16T18:18:24.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040e09a4a2fe1c088-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.671 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.671 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.671 DEBUG:boto:path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.671 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.671 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.671 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.671 DEBUG:boto:Data: 2024-04-16T18:18:24.671 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.672 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.672 DEBUG:boto:Params: {} 2024-04-16T18:18:24.672 DEBUG:boto:Token: None 2024-04-16T18:18:24.672 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L7p2APIokoxDrsrPKRyrPnZUL7Y= 2024-04-16T18:18:24.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L7p2APIokoxDrsrPKRyrPnZUL7Y=', 'Content-Length': '0'} 2024-04-16T18:18:24.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e56e1120a498525d-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.675 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.675 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.675 DEBUG:boto:path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.675 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.675 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.676 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.676 DEBUG:boto:Data: 2024-04-16T18:18:24.676 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.676 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.676 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.676 DEBUG:boto:Params: {} 2024-04-16T18:18:24.676 DEBUG:boto:Token: None 2024-04-16T18:18:24.676 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L7p2APIokoxDrsrPKRyrPnZUL7Y= 2024-04-16T18:18:24.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L7p2APIokoxDrsrPKRyrPnZUL7Y=', 'Content-Length': '0'} 2024-04-16T18:18:24.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f6381379de47b0a-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.679 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000065 2024-04-16T18:18:24.679 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.679 DEBUG:boto:path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.679 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.679 DEBUG:boto:Method: GET 2024-04-16T18:18:24.680 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.680 DEBUG:boto:Data: 2024-04-16T18:18:24.680 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.680 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.680 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.680 DEBUG:boto:Params: {} 2024-04-16T18:18:24.680 DEBUG:boto:Token: None 2024-04-16T18:18:24.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU= 2024-04-16T18:18:24.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU=', 'Content-Length': '0'} 2024-04-16T18:18:24.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009730c279279eca3f-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.726 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.727 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.727 DEBUG:boto:path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.727 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.727 DEBUG:boto:Method: GET 2024-04-16T18:18:24.727 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.727 DEBUG:boto:Data: 2024-04-16T18:18:24.727 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.727 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.727 DEBUG:boto:Params: {} 2024-04-16T18:18:24.727 DEBUG:boto:Token: None 2024-04-16T18:18:24.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065?versionId=null 2024-04-16T18:18:24.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU= 2024-04-16T18:18:24.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CFUBmBwQKYLCN25/LDR8FWGhThU=', 'Content-Length': '0'} 2024-04-16T18:18:24.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9e3247d57573e94-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.731 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.731 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.731 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.731 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.731 DEBUG:boto:Method: GET 2024-04-16T18:18:24.731 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.732 DEBUG:boto:Data: 2024-04-16T18:18:24.732 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.732 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.732 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.732 DEBUG:boto:Params: {} 2024-04-16T18:18:24.732 DEBUG:boto:Token: None 2024-04-16T18:18:24.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007937e1faa2cdd06-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.736 DEBUG:boto:b'lybtug-51obj000000000000000000651000falseobj000000000000000000652024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.737 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.737 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.737 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.737 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.737 DEBUG:boto:Method: GET 2024-04-16T18:18:24.737 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.737 DEBUG:boto:Data: 2024-04-16T18:18:24.737 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.737 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.737 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.738 DEBUG:boto:Params: {} 2024-04-16T18:18:24.738 DEBUG:boto:Token: None 2024-04-16T18:18:24.738 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.738 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000374d6780985768aa-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.786 DEBUG:boto:b'lybtug-51obj000000000000000000651000falseobj000000000000000000652024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.787 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.787 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.787 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.787 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.787 DEBUG:boto:Data: 2024-04-16T18:18:24.787 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.787 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.787 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.788 DEBUG:boto:Params: {} 2024-04-16T18:18:24.788 DEBUG:boto:Token: None 2024-04-16T18:18:24.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vs7junOYN/o75YJ15BZOfj6izFk= 2024-04-16T18:18:24.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vs7junOYN/o75YJ15BZOfj6izFk=', 'Content-Length': '0'} 2024-04-16T18:18:24.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b720d95ec08948-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.791 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.791 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.792 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.792 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.792 DEBUG:boto:Data: 2024-04-16T18:18:24.792 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.792 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.792 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.792 DEBUG:boto:Params: {} 2024-04-16T18:18:24.792 DEBUG:boto:Token: None 2024-04-16T18:18:24.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vs7junOYN/o75YJ15BZOfj6izFk= 2024-04-16T18:18:24.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vs7junOYN/o75YJ15BZOfj6izFk=', 'Content-Length': '0'} 2024-04-16T18:18:24.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e763eb2f043ec811-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000065 2024-04-16T18:18:24.795 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.795 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.795 DEBUG:boto:Method: GET 2024-04-16T18:18:24.796 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.796 DEBUG:boto:Data: 2024-04-16T18:18:24.796 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.796 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.796 DEBUG:boto:Params: {} 2024-04-16T18:18:24.796 DEBUG:boto:Token: None 2024-04-16T18:18:24.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zT9Y1gQTxotNmmB8bqxmfMsY/Fw= 2024-04-16T18:18:24.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zT9Y1gQTxotNmmB8bqxmfMsY/Fw=', 'Content-Length': '0'} 2024-04-16T18:18:24.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cced18b090301f7-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.842 DEBUG:boto:path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.842 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.843 DEBUG:boto:Method: GET 2024-04-16T18:18:24.843 DEBUG:boto:Path: /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.843 DEBUG:boto:Data: 2024-04-16T18:18:24.843 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.843 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.843 DEBUG:boto:Params: {} 2024-04-16T18:18:24.843 DEBUG:boto:Token: None 2024-04-16T18:18:24.843 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000065 2024-04-16T18:18:24.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zT9Y1gQTxotNmmB8bqxmfMsY/Fw= 2024-04-16T18:18:24.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zT9Y1gQTxotNmmB8bqxmfMsY/Fw=', 'Content-Length': '0'} 2024-04-16T18:18:24.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:18:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2a65bcfc514be90-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.847 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.847 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.847 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.847 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.847 DEBUG:boto:Method: GET 2024-04-16T18:18:24.847 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.847 DEBUG:boto:Data: 2024-04-16T18:18:24.847 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.848 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.848 DEBUG:boto:Params: {} 2024-04-16T18:18:24.848 DEBUG:boto:Token: None 2024-04-16T18:18:24.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.853 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ae258c0b8d8d86d-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.894 DEBUG:boto:b'lybtug-51obj000000000000000000651000falseobj000000000000000000652024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.895 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:24.895 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:24.895 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.895 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.895 DEBUG:boto:Method: GET 2024-04-16T18:18:24.895 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000065 2024-04-16T18:18:24.895 DEBUG:boto:Data: 2024-04-16T18:18:24.895 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.895 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.896 DEBUG:boto:Params: {} 2024-04-16T18:18:24.896 DEBUG:boto:Token: None 2024-04-16T18:18:24.896 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/ 2024-04-16T18:18:24.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs= 2024-04-16T18:18:24.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:luyyyCMIi7Y2CbeVZLlhKunrHrs=', 'Content-Length': '0'} 2024-04-16T18:18:24.902 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029a2fbc1eff1ba15-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.903 DEBUG:boto:b'lybtug-51obj000000000000000000651000falseobj000000000000000000652024-04-16T18:18:00.595Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:24.903 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000076 2024-04-16T18:18:24.903 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.903 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.903 DEBUG:boto:path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.903 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.903 DEBUG:boto:Method: GET 2024-04-16T18:18:24.903 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.904 DEBUG:boto:Data: 2024-04-16T18:18:24.904 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.904 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.904 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.904 DEBUG:boto:Params: {} 2024-04-16T18:18:24.904 DEBUG:boto:Token: None 2024-04-16T18:18:24.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DpP/AgcwITvXAi2yLLIxqlff7Lk= 2024-04-16T18:18:24.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DpP/AgcwITvXAi2yLLIxqlff7Lk=', 'Content-Length': '0'} 2024-04-16T18:18:24.908 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fef3545636827c27-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.950 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.951 DEBUG:boto:path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.951 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.951 DEBUG:boto:Method: GET 2024-04-16T18:18:24.951 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.951 DEBUG:boto:Data: 2024-04-16T18:18:24.951 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.951 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.951 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.951 DEBUG:boto:Params: {} 2024-04-16T18:18:24.951 DEBUG:boto:Token: None 2024-04-16T18:18:24.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DpP/AgcwITvXAi2yLLIxqlff7Lk= 2024-04-16T18:18:24.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DpP/AgcwITvXAi2yLLIxqlff7Lk=', 'Content-Length': '0'} 2024-04-16T18:18:24.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000caa0fb867341e8b6-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.955 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.955 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.955 DEBUG:boto:path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.955 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.955 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.956 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.956 DEBUG:boto:Data: 2024-04-16T18:18:24.956 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.956 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.956 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.956 DEBUG:boto:Params: {} 2024-04-16T18:18:24.956 DEBUG:boto:Token: None 2024-04-16T18:18:24.956 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kr6MKxr1UysidYy1rGosdZy8Zw= 2024-04-16T18:18:24.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kr6MKxr1UysidYy1rGosdZy8Zw=', 'Content-Length': '0'} 2024-04-16T18:18:24.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c0017d8e43bb091-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.960 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.960 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.960 DEBUG:boto:path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.960 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.960 DEBUG:boto:Method: HEAD 2024-04-16T18:18:24.960 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.960 DEBUG:boto:Data: 2024-04-16T18:18:24.960 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.960 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:24.960 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.960 DEBUG:boto:Params: {} 2024-04-16T18:18:24.961 DEBUG:boto:Token: None 2024-04-16T18:18:24.961 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6kr6MKxr1UysidYy1rGosdZy8Zw= 2024-04-16T18:18:24.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6kr6MKxr1UysidYy1rGosdZy8Zw=', 'Content-Length': '0'} 2024-04-16T18:18:24.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c273a0b960bfe88-00661ec0f0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:24.964 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000076 2024-04-16T18:18:24.964 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.964 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:24.964 DEBUG:boto:path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.964 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.965 DEBUG:boto:Method: GET 2024-04-16T18:18:24.965 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.965 DEBUG:boto:Data: 2024-04-16T18:18:24.965 DEBUG:boto:Headers: {} 2024-04-16T18:18:24.965 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:24.965 DEBUG:boto:Port: 8000 2024-04-16T18:18:24.965 DEBUG:boto:Params: {} 2024-04-16T18:18:24.965 DEBUG:boto:Token: None 2024-04-16T18:18:24.965 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:24 GMT /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:24.965 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DpP/AgcwITvXAi2yLLIxqlff7Lk= 2024-04-16T18:18:24.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DpP/AgcwITvXAi2yLLIxqlff7Lk=', 'Content-Length': '0'} 2024-04-16T18:18:24.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029b6c7b3170700d3-00661ec0f0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.010 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.010 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.011 DEBUG:boto:path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:25.011 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:25.011 DEBUG:boto:Method: GET 2024-04-16T18:18:25.011 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:25.011 DEBUG:boto:Data: 2024-04-16T18:18:25.011 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.011 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.011 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.011 DEBUG:boto:Params: {} 2024-04-16T18:18:25.011 DEBUG:boto:Token: None 2024-04-16T18:18:25.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000076?versionId=null 2024-04-16T18:18:25.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:h1gOCetsrlMQnIsBhGtI4vW3H2M= 2024-04-16T18:18:25.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h1gOCetsrlMQnIsBhGtI4vW3H2M=', 'Content-Length': '0'} 2024-04-16T18:18:25.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004612046bf979d9b0-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.015 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.015 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.015 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.015 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.015 DEBUG:boto:Method: GET 2024-04-16T18:18:25.016 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.016 DEBUG:boto:Data: 2024-04-16T18:18:25.016 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.016 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.016 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.016 DEBUG:boto:Params: {} 2024-04-16T18:18:25.016 DEBUG:boto:Token: None 2024-04-16T18:18:25.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011a3b5535f1c7861-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.021 DEBUG:boto:b'lybtug-51obj000000000000000000761000falseobj000000000000000000762024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.021 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.021 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.021 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.022 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.022 DEBUG:boto:Method: GET 2024-04-16T18:18:25.022 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.022 DEBUG:boto:Data: 2024-04-16T18:18:25.022 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.022 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.022 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.022 DEBUG:boto:Params: {} 2024-04-16T18:18:25.022 DEBUG:boto:Token: None 2024-04-16T18:18:25.022 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.023 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.028 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba2bcd6e3acbb998-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.070 DEBUG:boto:b'lybtug-51obj000000000000000000761000falseobj000000000000000000762024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.071 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.071 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.071 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.071 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.071 DEBUG:boto:Data: 2024-04-16T18:18:25.071 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.071 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.071 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.072 DEBUG:boto:Params: {} 2024-04-16T18:18:25.072 DEBUG:boto:Token: None 2024-04-16T18:18:25.072 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7/0EqPFk4+vTTrOxrRgrzU6/F7o= 2024-04-16T18:18:25.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7/0EqPFk4+vTTrOxrRgrzU6/F7o=', 'Content-Length': '0'} 2024-04-16T18:18:25.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcd27fd80c47ef6e-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.076 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.076 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.076 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.076 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.076 DEBUG:boto:Data: 2024-04-16T18:18:25.076 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.076 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.076 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.076 DEBUG:boto:Params: {} 2024-04-16T18:18:25.076 DEBUG:boto:Token: None 2024-04-16T18:18:25.076 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.077 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7/0EqPFk4+vTTrOxrRgrzU6/F7o= 2024-04-16T18:18:25.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7/0EqPFk4+vTTrOxrRgrzU6/F7o=', 'Content-Length': '0'} 2024-04-16T18:18:25.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f017b2610724fb72-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.080 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000076 2024-04-16T18:18:25.080 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.080 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.080 DEBUG:boto:Method: GET 2024-04-16T18:18:25.080 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.081 DEBUG:boto:Data: 2024-04-16T18:18:25.081 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.081 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.081 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.081 DEBUG:boto:Params: {} 2024-04-16T18:18:25.081 DEBUG:boto:Token: None 2024-04-16T18:18:25.081 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.081 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hrd87LX3MH2xQQk3sRJuL8XJm7c= 2024-04-16T18:18:25.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hrd87LX3MH2xQQk3sRJuL8XJm7c=', 'Content-Length': '0'} 2024-04-16T18:18:25.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:01 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cf80d402e7be373-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.126 DEBUG:boto:path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.127 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.127 DEBUG:boto:Method: GET 2024-04-16T18:18:25.127 DEBUG:boto:Path: /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.127 DEBUG:boto:Data: 2024-04-16T18:18:25.127 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.127 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.127 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.127 DEBUG:boto:Params: {} 2024-04-16T18:18:25.127 DEBUG:boto:Token: None 2024-04-16T18:18:25.127 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000076 2024-04-16T18:18:25.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hrd87LX3MH2xQQk3sRJuL8XJm7c= 2024-04-16T18:18:25.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hrd87LX3MH2xQQk3sRJuL8XJm7c=', 'Content-Length': '0'} 2024-04-16T18:18:25.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096390e411ecbc76b-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.131 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.132 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.132 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.132 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.132 DEBUG:boto:Method: GET 2024-04-16T18:18:25.132 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.132 DEBUG:boto:Data: 2024-04-16T18:18:25.132 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.132 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.132 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.132 DEBUG:boto:Params: {} 2024-04-16T18:18:25.132 DEBUG:boto:Token: None 2024-04-16T18:18:25.133 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000798ed68abb14ea1c-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.178 DEBUG:boto:b'lybtug-51obj000000000000000000761000falseobj000000000000000000762024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.179 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.179 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.179 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.179 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.179 DEBUG:boto:Method: GET 2024-04-16T18:18:25.179 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000076 2024-04-16T18:18:25.179 DEBUG:boto:Data: 2024-04-16T18:18:25.179 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.180 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.180 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.180 DEBUG:boto:Params: {} 2024-04-16T18:18:25.180 DEBUG:boto:Token: None 2024-04-16T18:18:25.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.186 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7c4fa436b03abb3-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.187 DEBUG:boto:b'lybtug-51obj000000000000000000761000falseobj000000000000000000762024-04-16T18:17:53.672Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.187 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000133 2024-04-16T18:18:25.187 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.187 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.187 DEBUG:boto:path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.187 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.187 DEBUG:boto:Method: GET 2024-04-16T18:18:25.188 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.188 DEBUG:boto:Data: 2024-04-16T18:18:25.188 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.188 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.188 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.188 DEBUG:boto:Params: {} 2024-04-16T18:18:25.188 DEBUG:boto:Token: None 2024-04-16T18:18:25.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg= 2024-04-16T18:18:25.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg=', 'Content-Length': '0'} 2024-04-16T18:18:25.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003dc73e39d0706d6e-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.238 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.239 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.239 DEBUG:boto:path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.239 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.239 DEBUG:boto:Method: GET 2024-04-16T18:18:25.239 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.239 DEBUG:boto:Data: 2024-04-16T18:18:25.239 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.239 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.239 DEBUG:boto:Params: {} 2024-04-16T18:18:25.239 DEBUG:boto:Token: None 2024-04-16T18:18:25.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg= 2024-04-16T18:18:25.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg=', 'Content-Length': '0'} 2024-04-16T18:18:25.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6d98e21aea14493-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.243 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.243 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.243 DEBUG:boto:path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.243 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.243 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.244 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.244 DEBUG:boto:Data: 2024-04-16T18:18:25.244 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.244 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.244 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.244 DEBUG:boto:Params: {} 2024-04-16T18:18:25.244 DEBUG:boto:Token: None 2024-04-16T18:18:25.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nWuwJ+ErFLYXJPLWyWmywH/6vJ4= 2024-04-16T18:18:25.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nWuwJ+ErFLYXJPLWyWmywH/6vJ4=', 'Content-Length': '0'} 2024-04-16T18:18:25.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007be25f23a620b3b9-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.248 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.248 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.248 DEBUG:boto:path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.248 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.248 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.248 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.248 DEBUG:boto:Data: 2024-04-16T18:18:25.248 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.248 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.248 DEBUG:boto:Params: {} 2024-04-16T18:18:25.249 DEBUG:boto:Token: None 2024-04-16T18:18:25.249 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nWuwJ+ErFLYXJPLWyWmywH/6vJ4= 2024-04-16T18:18:25.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nWuwJ+ErFLYXJPLWyWmywH/6vJ4=', 'Content-Length': '0'} 2024-04-16T18:18:25.252 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004160761ac252fcd0-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.252 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000133 2024-04-16T18:18:25.252 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.252 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.252 DEBUG:boto:path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.252 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.253 DEBUG:boto:Method: GET 2024-04-16T18:18:25.253 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.253 DEBUG:boto:Data: 2024-04-16T18:18:25.253 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.253 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.253 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.253 DEBUG:boto:Params: {} 2024-04-16T18:18:25.253 DEBUG:boto:Token: None 2024-04-16T18:18:25.253 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.253 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg= 2024-04-16T18:18:25.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg=', 'Content-Length': '0'} 2024-04-16T18:18:25.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000946013ce8deac8ed-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.298 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.299 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.299 DEBUG:boto:path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.299 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.299 DEBUG:boto:Method: GET 2024-04-16T18:18:25.299 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.299 DEBUG:boto:Data: 2024-04-16T18:18:25.299 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.299 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.299 DEBUG:boto:Params: {} 2024-04-16T18:18:25.300 DEBUG:boto:Token: None 2024-04-16T18:18:25.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133?versionId=null 2024-04-16T18:18:25.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg= 2024-04-16T18:18:25.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:reyufYawaO+vvDe8ZAXuz8OEOjg=', 'Content-Length': '0'} 2024-04-16T18:18:25.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003dc6e515a2b93d06-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.304 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.304 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.304 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.304 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.304 DEBUG:boto:Method: GET 2024-04-16T18:18:25.304 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.305 DEBUG:boto:Data: 2024-04-16T18:18:25.305 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.305 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.305 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.305 DEBUG:boto:Params: {} 2024-04-16T18:18:25.305 DEBUG:boto:Token: None 2024-04-16T18:18:25.305 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.305 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.309 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028e5406422387626-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.310 DEBUG:boto:b'lybtug-51obj000000000000000001331000falseobj000000000000000001332024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.310 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.310 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.310 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.310 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.311 DEBUG:boto:Method: GET 2024-04-16T18:18:25.311 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.311 DEBUG:boto:Data: 2024-04-16T18:18:25.311 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.311 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.311 DEBUG:boto:Params: {} 2024-04-16T18:18:25.311 DEBUG:boto:Token: None 2024-04-16T18:18:25.311 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.311 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a9b27a876260354-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.358 DEBUG:boto:b'lybtug-51obj000000000000000001331000falseobj000000000000000001332024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.359 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.359 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.359 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.359 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.359 DEBUG:boto:Data: 2024-04-16T18:18:25.359 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.359 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.359 DEBUG:boto:Params: {} 2024-04-16T18:18:25.359 DEBUG:boto:Token: None 2024-04-16T18:18:25.360 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HRkW2UcbDwQHr71gX1HvAzhOmgw= 2024-04-16T18:18:25.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HRkW2UcbDwQHr71gX1HvAzhOmgw=', 'Content-Length': '0'} 2024-04-16T18:18:25.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8038aa260f9f400-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.363 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.363 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.363 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.363 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.363 DEBUG:boto:Data: 2024-04-16T18:18:25.363 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.363 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.364 DEBUG:boto:Params: {} 2024-04-16T18:18:25.364 DEBUG:boto:Token: None 2024-04-16T18:18:25.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HRkW2UcbDwQHr71gX1HvAzhOmgw= 2024-04-16T18:18:25.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HRkW2UcbDwQHr71gX1HvAzhOmgw=', 'Content-Length': '0'} 2024-04-16T18:18:25.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e379d8a91b1b9c4-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000133 2024-04-16T18:18:25.367 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.368 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.368 DEBUG:boto:Method: GET 2024-04-16T18:18:25.368 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.368 DEBUG:boto:Data: 2024-04-16T18:18:25.368 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.368 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.368 DEBUG:boto:Params: {} 2024-04-16T18:18:25.368 DEBUG:boto:Token: None 2024-04-16T18:18:25.368 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4kJ41r7kgRhj+OkjHBRKrJ6eqf0= 2024-04-16T18:18:25.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4kJ41r7kgRhj+OkjHBRKrJ6eqf0=', 'Content-Length': '0'} 2024-04-16T18:18:25.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000569618bed60e439f-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.414 DEBUG:boto:path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.414 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.415 DEBUG:boto:Method: GET 2024-04-16T18:18:25.415 DEBUG:boto:Path: /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.415 DEBUG:boto:Data: 2024-04-16T18:18:25.415 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.415 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.415 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.415 DEBUG:boto:Params: {} 2024-04-16T18:18:25.415 DEBUG:boto:Token: None 2024-04-16T18:18:25.415 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000133 2024-04-16T18:18:25.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4kJ41r7kgRhj+OkjHBRKrJ6eqf0= 2024-04-16T18:18:25.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4kJ41r7kgRhj+OkjHBRKrJ6eqf0=', 'Content-Length': '0'} 2024-04-16T18:18:25.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ea611a7570c3f31-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.419 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.419 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.419 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.419 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.419 DEBUG:boto:Method: GET 2024-04-16T18:18:25.419 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.419 DEBUG:boto:Data: 2024-04-16T18:18:25.420 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.420 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.420 DEBUG:boto:Params: {} 2024-04-16T18:18:25.420 DEBUG:boto:Token: None 2024-04-16T18:18:25.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.425 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad3899b609074f7a-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.466 DEBUG:boto:b'lybtug-51obj000000000000000001331000falseobj000000000000000001332024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.467 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.467 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.467 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.467 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.467 DEBUG:boto:Method: GET 2024-04-16T18:18:25.467 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000133 2024-04-16T18:18:25.467 DEBUG:boto:Data: 2024-04-16T18:18:25.467 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.467 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.467 DEBUG:boto:Params: {} 2024-04-16T18:18:25.468 DEBUG:boto:Token: None 2024-04-16T18:18:25.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d36c46008521e6ff-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.474 DEBUG:boto:b'lybtug-51obj000000000000000001331000falseobj000000000000000001332024-04-16T18:17:58.827Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.474 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000162 2024-04-16T18:18:25.475 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.475 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.475 DEBUG:boto:path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.475 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.475 DEBUG:boto:Method: GET 2024-04-16T18:18:25.475 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.475 DEBUG:boto:Data: 2024-04-16T18:18:25.475 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.475 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.475 DEBUG:boto:Params: {} 2024-04-16T18:18:25.476 DEBUG:boto:Token: None 2024-04-16T18:18:25.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY= 2024-04-16T18:18:25.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY=', 'Content-Length': '0'} 2024-04-16T18:18:25.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9af101741d978f2-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.522 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.522 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.523 DEBUG:boto:path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.523 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.523 DEBUG:boto:Method: GET 2024-04-16T18:18:25.523 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.523 DEBUG:boto:Data: 2024-04-16T18:18:25.523 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.523 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.523 DEBUG:boto:Params: {} 2024-04-16T18:18:25.523 DEBUG:boto:Token: None 2024-04-16T18:18:25.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY= 2024-04-16T18:18:25.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY=', 'Content-Length': '0'} 2024-04-16T18:18:25.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025fbfdffedd21642-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.527 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.527 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.527 DEBUG:boto:path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.527 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.527 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.528 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.528 DEBUG:boto:Data: 2024-04-16T18:18:25.528 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.528 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.528 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.528 DEBUG:boto:Params: {} 2024-04-16T18:18:25.528 DEBUG:boto:Token: None 2024-04-16T18:18:25.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K3Y5DdIVjbDSqqqh0VjSp47y7hY= 2024-04-16T18:18:25.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K3Y5DdIVjbDSqqqh0VjSp47y7hY=', 'Content-Length': '0'} 2024-04-16T18:18:25.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000590e64a59bab2a48-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.531 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.532 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.532 DEBUG:boto:path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.532 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.532 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.532 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.532 DEBUG:boto:Data: 2024-04-16T18:18:25.532 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.532 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.532 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.532 DEBUG:boto:Params: {} 2024-04-16T18:18:25.533 DEBUG:boto:Token: None 2024-04-16T18:18:25.533 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.533 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K3Y5DdIVjbDSqqqh0VjSp47y7hY= 2024-04-16T18:18:25.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K3Y5DdIVjbDSqqqh0VjSp47y7hY=', 'Content-Length': '0'} 2024-04-16T18:18:25.536 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065bbd3e905bd2cf2-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.536 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000162 2024-04-16T18:18:25.536 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.536 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.536 DEBUG:boto:path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.536 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.537 DEBUG:boto:Method: GET 2024-04-16T18:18:25.537 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.537 DEBUG:boto:Data: 2024-04-16T18:18:25.537 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.537 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.537 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.537 DEBUG:boto:Params: {} 2024-04-16T18:18:25.537 DEBUG:boto:Token: None 2024-04-16T18:18:25.537 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.538 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY= 2024-04-16T18:18:25.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY=', 'Content-Length': '0'} 2024-04-16T18:18:25.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006dc1346917ac9a2c-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.582 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.582 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.583 DEBUG:boto:path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.583 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.583 DEBUG:boto:Method: GET 2024-04-16T18:18:25.583 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.583 DEBUG:boto:Data: 2024-04-16T18:18:25.583 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.583 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.583 DEBUG:boto:Params: {} 2024-04-16T18:18:25.583 DEBUG:boto:Token: None 2024-04-16T18:18:25.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162?versionId=null 2024-04-16T18:18:25.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY= 2024-04-16T18:18:25.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jQK+WTU8WLHiKG5Sboc9pM0BEhY=', 'Content-Length': '0'} 2024-04-16T18:18:25.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e6794544ad9855b-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.587 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.587 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.587 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.587 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.587 DEBUG:boto:Method: GET 2024-04-16T18:18:25.587 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.587 DEBUG:boto:Data: 2024-04-16T18:18:25.587 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.588 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.588 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.588 DEBUG:boto:Params: {} 2024-04-16T18:18:25.588 DEBUG:boto:Token: None 2024-04-16T18:18:25.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8a1aec07b2cc5e6-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.592 DEBUG:boto:b'lybtug-51obj000000000000000001621000falseobj000000000000000001622024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.593 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.593 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.593 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.593 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.593 DEBUG:boto:Method: GET 2024-04-16T18:18:25.593 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.593 DEBUG:boto:Data: 2024-04-16T18:18:25.593 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.593 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.593 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.593 DEBUG:boto:Params: {} 2024-04-16T18:18:25.594 DEBUG:boto:Token: None 2024-04-16T18:18:25.594 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.594 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f4d5c270284e99e-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.642 DEBUG:boto:b'lybtug-51obj000000000000000001621000falseobj000000000000000001622024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.643 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.643 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.643 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.643 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.643 DEBUG:boto:Data: 2024-04-16T18:18:25.643 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.644 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.644 DEBUG:boto:Params: {} 2024-04-16T18:18:25.644 DEBUG:boto:Token: None 2024-04-16T18:18:25.644 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bsgaOcsRufIRQzC/M4/5hPlfj6I= 2024-04-16T18:18:25.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bsgaOcsRufIRQzC/M4/5hPlfj6I=', 'Content-Length': '0'} 2024-04-16T18:18:25.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd64d53458424e22-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.647 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.647 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.647 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.648 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.648 DEBUG:boto:Data: 2024-04-16T18:18:25.648 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.648 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.648 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.648 DEBUG:boto:Params: {} 2024-04-16T18:18:25.648 DEBUG:boto:Token: None 2024-04-16T18:18:25.648 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bsgaOcsRufIRQzC/M4/5hPlfj6I= 2024-04-16T18:18:25.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bsgaOcsRufIRQzC/M4/5hPlfj6I=', 'Content-Length': '0'} 2024-04-16T18:18:25.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d05dd5ca3fa5c5e-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.651 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000162 2024-04-16T18:18:25.652 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.652 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.652 DEBUG:boto:Method: GET 2024-04-16T18:18:25.652 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.652 DEBUG:boto:Data: 2024-04-16T18:18:25.652 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.652 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.652 DEBUG:boto:Params: {} 2024-04-16T18:18:25.653 DEBUG:boto:Token: None 2024-04-16T18:18:25.653 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.653 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JzySc25YQuLQwN3FVLQl4u6qXCc= 2024-04-16T18:18:25.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JzySc25YQuLQwN3FVLQl4u6qXCc=', 'Content-Length': '0'} 2024-04-16T18:18:25.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003dbef8629251457-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.698 DEBUG:boto:path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.699 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.699 DEBUG:boto:Method: GET 2024-04-16T18:18:25.699 DEBUG:boto:Path: /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.699 DEBUG:boto:Data: 2024-04-16T18:18:25.699 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.699 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.699 DEBUG:boto:Params: {} 2024-04-16T18:18:25.699 DEBUG:boto:Token: None 2024-04-16T18:18:25.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000162 2024-04-16T18:18:25.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JzySc25YQuLQwN3FVLQl4u6qXCc= 2024-04-16T18:18:25.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JzySc25YQuLQwN3FVLQl4u6qXCc=', 'Content-Length': '0'} 2024-04-16T18:18:25.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe86e41099b67721-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.703 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.703 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.703 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.703 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.703 DEBUG:boto:Method: GET 2024-04-16T18:18:25.704 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.704 DEBUG:boto:Data: 2024-04-16T18:18:25.704 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.704 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.704 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.704 DEBUG:boto:Params: {} 2024-04-16T18:18:25.704 DEBUG:boto:Token: None 2024-04-16T18:18:25.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.709 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d800443893025eb-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.750 DEBUG:boto:b'lybtug-51obj000000000000000001621000falseobj000000000000000001622024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.751 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.751 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.751 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.751 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.751 DEBUG:boto:Method: GET 2024-04-16T18:18:25.751 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000162 2024-04-16T18:18:25.751 DEBUG:boto:Data: 2024-04-16T18:18:25.751 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.752 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.752 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.752 DEBUG:boto:Params: {} 2024-04-16T18:18:25.752 DEBUG:boto:Token: None 2024-04-16T18:18:25.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.758 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093f2c5bd7f5f8493-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.759 DEBUG:boto:b'lybtug-51obj000000000000000001621000falseobj000000000000000001622024-04-16T18:17:58.598Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000271 2024-04-16T18:18:25.759 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.759 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.759 DEBUG:boto:path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.759 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.759 DEBUG:boto:Method: GET 2024-04-16T18:18:25.760 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.760 DEBUG:boto:Data: 2024-04-16T18:18:25.760 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.760 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.760 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.760 DEBUG:boto:Params: {} 2024-04-16T18:18:25.760 DEBUG:boto:Token: None 2024-04-16T18:18:25.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0= 2024-04-16T18:18:25.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0=', 'Content-Length': '0'} 2024-04-16T18:18:25.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d832fb5a155f2444-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.806 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.806 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.806 DEBUG:boto:path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.807 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.807 DEBUG:boto:Method: GET 2024-04-16T18:18:25.807 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.807 DEBUG:boto:Data: 2024-04-16T18:18:25.808 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.808 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.808 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.808 DEBUG:boto:Params: {} 2024-04-16T18:18:25.808 DEBUG:boto:Token: None 2024-04-16T18:18:25.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0= 2024-04-16T18:18:25.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0=', 'Content-Length': '0'} 2024-04-16T18:18:25.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b48585f602286177-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.812 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.812 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.812 DEBUG:boto:path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.812 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.812 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.813 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.813 DEBUG:boto:Data: 2024-04-16T18:18:25.813 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.813 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.813 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.813 DEBUG:boto:Params: {} 2024-04-16T18:18:25.813 DEBUG:boto:Token: None 2024-04-16T18:18:25.813 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.813 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uCXyJAsSWiV1zyY15JFS2vPBeOQ= 2024-04-16T18:18:25.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uCXyJAsSWiV1zyY15JFS2vPBeOQ=', 'Content-Length': '0'} 2024-04-16T18:18:25.816 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3a902af6b3d5ab4-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.817 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.817 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.817 DEBUG:boto:path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.817 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.817 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.817 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.817 DEBUG:boto:Data: 2024-04-16T18:18:25.817 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.817 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.817 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.818 DEBUG:boto:Params: {} 2024-04-16T18:18:25.818 DEBUG:boto:Token: None 2024-04-16T18:18:25.818 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.818 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uCXyJAsSWiV1zyY15JFS2vPBeOQ= 2024-04-16T18:18:25.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uCXyJAsSWiV1zyY15JFS2vPBeOQ=', 'Content-Length': '0'} 2024-04-16T18:18:25.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c887f9a78c123f19-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.822 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000271 2024-04-16T18:18:25.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.822 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.822 DEBUG:boto:path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.822 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.822 DEBUG:boto:Method: GET 2024-04-16T18:18:25.822 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.822 DEBUG:boto:Data: 2024-04-16T18:18:25.823 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.823 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.823 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.823 DEBUG:boto:Params: {} 2024-04-16T18:18:25.823 DEBUG:boto:Token: None 2024-04-16T18:18:25.823 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.823 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0= 2024-04-16T18:18:25.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0=', 'Content-Length': '0'} 2024-04-16T18:18:25.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f1f5480274da739-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.866 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.866 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.867 DEBUG:boto:path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.867 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.867 DEBUG:boto:Method: GET 2024-04-16T18:18:25.867 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.867 DEBUG:boto:Data: 2024-04-16T18:18:25.867 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.867 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.867 DEBUG:boto:Params: {} 2024-04-16T18:18:25.867 DEBUG:boto:Token: None 2024-04-16T18:18:25.868 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271?versionId=null 2024-04-16T18:18:25.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0= 2024-04-16T18:18:25.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pN3t9Z5presvACsnIHHRSNSmZT0=', 'Content-Length': '0'} 2024-04-16T18:18:25.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000035038fe825afa1d-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.871 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.871 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.871 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.871 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.871 DEBUG:boto:Method: GET 2024-04-16T18:18:25.872 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.872 DEBUG:boto:Data: 2024-04-16T18:18:25.872 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.872 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.872 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.872 DEBUG:boto:Params: {} 2024-04-16T18:18:25.872 DEBUG:boto:Token: None 2024-04-16T18:18:25.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042efc40a0aa11425-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.877 DEBUG:boto:b'lybtug-51obj000000000000000002711000falseobj000000000000000002712024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.877 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.877 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.877 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.877 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.877 DEBUG:boto:Method: GET 2024-04-16T18:18:25.878 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.878 DEBUG:boto:Data: 2024-04-16T18:18:25.878 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.878 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.878 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.878 DEBUG:boto:Params: {} 2024-04-16T18:18:25.878 DEBUG:boto:Token: None 2024-04-16T18:18:25.878 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.878 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009929aa32982572be-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.926 DEBUG:boto:b'lybtug-51obj000000000000000002711000falseobj000000000000000002712024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:25.927 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.927 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.927 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.927 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.927 DEBUG:boto:Data: 2024-04-16T18:18:25.927 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.927 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.928 DEBUG:boto:Params: {} 2024-04-16T18:18:25.928 DEBUG:boto:Token: None 2024-04-16T18:18:25.928 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sNqbRB5cmKfYaX05b5LhtP0gM7I= 2024-04-16T18:18:25.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sNqbRB5cmKfYaX05b5LhtP0gM7I=', 'Content-Length': '0'} 2024-04-16T18:18:25.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b241b6ffa7a66863-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.931 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.931 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.931 DEBUG:boto:Method: HEAD 2024-04-16T18:18:25.931 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.932 DEBUG:boto:Data: 2024-04-16T18:18:25.932 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.932 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.932 DEBUG:boto:Params: {} 2024-04-16T18:18:25.932 DEBUG:boto:Token: None 2024-04-16T18:18:25.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sNqbRB5cmKfYaX05b5LhtP0gM7I= 2024-04-16T18:18:25.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sNqbRB5cmKfYaX05b5LhtP0gM7I=', 'Content-Length': '0'} 2024-04-16T18:18:25.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000846e4bcf38af2a37-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.936 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000271 2024-04-16T18:18:25.936 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.936 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.936 DEBUG:boto:Method: GET 2024-04-16T18:18:25.936 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.936 DEBUG:boto:Data: 2024-04-16T18:18:25.936 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.936 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.937 DEBUG:boto:Params: {} 2024-04-16T18:18:25.937 DEBUG:boto:Token: None 2024-04-16T18:18:25.937 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.937 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PAhNRZ2ynZgerXFUHPJhG/TFMpY= 2024-04-16T18:18:25.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PAhNRZ2ynZgerXFUHPJhG/TFMpY=', 'Content-Length': '0'} 2024-04-16T18:18:25.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000997ce11cc54d525d-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.982 DEBUG:boto:path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.983 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.983 DEBUG:boto:Method: GET 2024-04-16T18:18:25.983 DEBUG:boto:Path: /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.983 DEBUG:boto:Data: 2024-04-16T18:18:25.983 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:25.983 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.983 DEBUG:boto:Params: {} 2024-04-16T18:18:25.983 DEBUG:boto:Token: None 2024-04-16T18:18:25.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/obj00000000000000000271 2024-04-16T18:18:25.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PAhNRZ2ynZgerXFUHPJhG/TFMpY= 2024-04-16T18:18:25.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PAhNRZ2ynZgerXFUHPJhG/TFMpY=', 'Content-Length': '0'} 2024-04-16T18:18:25.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022252c3d6462a6f3-00661ec0f1-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:25.987 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:25.987 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:25.987 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.988 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.988 DEBUG:boto:Method: GET 2024-04-16T18:18:25.988 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:25.988 DEBUG:boto:Data: 2024-04-16T18:18:25.988 DEBUG:boto:Headers: {} 2024-04-16T18:18:25.988 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:25.988 DEBUG:boto:Port: 8000 2024-04-16T18:18:25.988 DEBUG:boto:Params: {} 2024-04-16T18:18:25.988 DEBUG:boto:Token: None 2024-04-16T18:18:25.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:25 GMT /lybtug-51/ 2024-04-16T18:18:25.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc= 2024-04-16T18:18:25.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BUaP3h14XRfY71kYIkU2nUn7Vgc=', 'Content-Length': '0'} 2024-04-16T18:18:25.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5e69466f29a5222-00661ec0f1-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.034 DEBUG:boto:b'lybtug-51obj000000000000000002711000falseobj000000000000000002712024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.035 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.035 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.035 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:26.035 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:26.035 DEBUG:boto:Method: GET 2024-04-16T18:18:26.036 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000271 2024-04-16T18:18:26.036 DEBUG:boto:Data: 2024-04-16T18:18:26.036 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.036 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.036 DEBUG:boto:Params: {} 2024-04-16T18:18:26.036 DEBUG:boto:Token: None 2024-04-16T18:18:26.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.042 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000331810c9958cae73-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.043 DEBUG:boto:b'lybtug-51obj000000000000000002711000falseobj000000000000000002712024-04-16T18:17:58.547Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000283 2024-04-16T18:18:26.043 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.043 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.044 DEBUG:boto:path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.044 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.044 DEBUG:boto:Method: GET 2024-04-16T18:18:26.044 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.044 DEBUG:boto:Data: 2024-04-16T18:18:26.044 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.044 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.044 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.044 DEBUG:boto:Params: {} 2024-04-16T18:18:26.045 DEBUG:boto:Token: None 2024-04-16T18:18:26.045 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.045 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08= 2024-04-16T18:18:26.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08=', 'Content-Length': '0'} 2024-04-16T18:18:26.048 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbaabc043d93c68d-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.090 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.091 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.091 DEBUG:boto:path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.091 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.091 DEBUG:boto:Method: GET 2024-04-16T18:18:26.091 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.091 DEBUG:boto:Data: 2024-04-16T18:18:26.091 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.091 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.092 DEBUG:boto:Params: {} 2024-04-16T18:18:26.092 DEBUG:boto:Token: None 2024-04-16T18:18:26.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08= 2024-04-16T18:18:26.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08=', 'Content-Length': '0'} 2024-04-16T18:18:26.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075ded21997ed8904-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.096 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.096 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.096 DEBUG:boto:path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.096 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.096 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.096 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.096 DEBUG:boto:Data: 2024-04-16T18:18:26.097 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.097 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.097 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.097 DEBUG:boto:Params: {} 2024-04-16T18:18:26.097 DEBUG:boto:Token: None 2024-04-16T18:18:26.097 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KzQ4gIP0tGs7QiUmqvG3s3reumM= 2024-04-16T18:18:26.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KzQ4gIP0tGs7QiUmqvG3s3reumM=', 'Content-Length': '0'} 2024-04-16T18:18:26.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d22f5a94fd0dd627-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.101 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.101 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.101 DEBUG:boto:path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.101 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.102 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.102 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.102 DEBUG:boto:Data: 2024-04-16T18:18:26.102 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.102 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.102 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.102 DEBUG:boto:Params: {} 2024-04-16T18:18:26.102 DEBUG:boto:Token: None 2024-04-16T18:18:26.103 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.103 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KzQ4gIP0tGs7QiUmqvG3s3reumM= 2024-04-16T18:18:26.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KzQ4gIP0tGs7QiUmqvG3s3reumM=', 'Content-Length': '0'} 2024-04-16T18:18:26.106 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000470dbee892ab782f-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.106 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000283 2024-04-16T18:18:26.107 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.107 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.107 DEBUG:boto:path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.107 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.107 DEBUG:boto:Method: GET 2024-04-16T18:18:26.107 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.107 DEBUG:boto:Data: 2024-04-16T18:18:26.108 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.108 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.108 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.108 DEBUG:boto:Params: {} 2024-04-16T18:18:26.108 DEBUG:boto:Token: None 2024-04-16T18:18:26.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08= 2024-04-16T18:18:26.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08=', 'Content-Length': '0'} 2024-04-16T18:18:26.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011a8cb3e7acc0f54-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.154 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.155 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.155 DEBUG:boto:path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.155 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.155 DEBUG:boto:Method: GET 2024-04-16T18:18:26.155 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.155 DEBUG:boto:Data: 2024-04-16T18:18:26.155 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.155 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.156 DEBUG:boto:Params: {} 2024-04-16T18:18:26.156 DEBUG:boto:Token: None 2024-04-16T18:18:26.156 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283?versionId=null 2024-04-16T18:18:26.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08= 2024-04-16T18:18:26.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vb+T3eeI93+s3ODG3NJGvWYk/08=', 'Content-Length': '0'} 2024-04-16T18:18:26.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b538ce42f3faaf71-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.159 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.160 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.160 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.160 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.160 DEBUG:boto:Method: GET 2024-04-16T18:18:26.160 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.160 DEBUG:boto:Data: 2024-04-16T18:18:26.160 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.160 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.160 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.161 DEBUG:boto:Params: {} 2024-04-16T18:18:26.161 DEBUG:boto:Token: None 2024-04-16T18:18:26.161 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.166 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006569ef3d860d508b-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.166 DEBUG:boto:b'lybtug-51obj000000000000000002831000falseobj000000000000000002832024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.166 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.167 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.167 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.167 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.167 DEBUG:boto:Method: GET 2024-04-16T18:18:26.167 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.167 DEBUG:boto:Data: 2024-04-16T18:18:26.167 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.167 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.168 DEBUG:boto:Params: {} 2024-04-16T18:18:26.168 DEBUG:boto:Token: None 2024-04-16T18:18:26.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084a6b82e28731ae7-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.218 DEBUG:boto:b'lybtug-51obj000000000000000002831000falseobj000000000000000002832024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.219 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.219 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.219 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.219 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.219 DEBUG:boto:Data: 2024-04-16T18:18:26.219 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.219 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.220 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.220 DEBUG:boto:Params: {} 2024-04-16T18:18:26.220 DEBUG:boto:Token: None 2024-04-16T18:18:26.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jRkVCzhurCOswIJFvspnovIrcn0= 2024-04-16T18:18:26.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jRkVCzhurCOswIJFvspnovIrcn0=', 'Content-Length': '0'} 2024-04-16T18:18:26.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000365b78e985512b0c-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.223 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.223 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.224 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.224 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.224 DEBUG:boto:Data: 2024-04-16T18:18:26.224 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.224 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.224 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.224 DEBUG:boto:Params: {} 2024-04-16T18:18:26.224 DEBUG:boto:Token: None 2024-04-16T18:18:26.225 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jRkVCzhurCOswIJFvspnovIrcn0= 2024-04-16T18:18:26.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jRkVCzhurCOswIJFvspnovIrcn0=', 'Content-Length': '0'} 2024-04-16T18:18:26.228 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdfb7ade9a947186-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.228 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000283 2024-04-16T18:18:26.228 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.228 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.229 DEBUG:boto:Method: GET 2024-04-16T18:18:26.229 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.229 DEBUG:boto:Data: 2024-04-16T18:18:26.229 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.229 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.229 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.229 DEBUG:boto:Params: {} 2024-04-16T18:18:26.229 DEBUG:boto:Token: None 2024-04-16T18:18:26.229 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.230 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ekrgB2Ah16LcZJ9XaC0S3KYHcms= 2024-04-16T18:18:26.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ekrgB2Ah16LcZJ9XaC0S3KYHcms=', 'Content-Length': '0'} 2024-04-16T18:18:26.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0a3c7f35ead2fdb-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.274 DEBUG:boto:path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.275 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.275 DEBUG:boto:Method: GET 2024-04-16T18:18:26.275 DEBUG:boto:Path: /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.275 DEBUG:boto:Data: 2024-04-16T18:18:26.275 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.275 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.275 DEBUG:boto:Params: {} 2024-04-16T18:18:26.275 DEBUG:boto:Token: None 2024-04-16T18:18:26.276 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000283 2024-04-16T18:18:26.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ekrgB2Ah16LcZJ9XaC0S3KYHcms= 2024-04-16T18:18:26.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ekrgB2Ah16LcZJ9XaC0S3KYHcms=', 'Content-Length': '0'} 2024-04-16T18:18:26.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab5848e1b4d332a0-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.279 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.280 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.280 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.280 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.280 DEBUG:boto:Method: GET 2024-04-16T18:18:26.280 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.280 DEBUG:boto:Data: 2024-04-16T18:18:26.280 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.281 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.281 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.281 DEBUG:boto:Params: {} 2024-04-16T18:18:26.281 DEBUG:boto:Token: None 2024-04-16T18:18:26.281 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.281 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.286 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096db5f0b5d0ec21b-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.326 DEBUG:boto:b'lybtug-51obj000000000000000002831000falseobj000000000000000002832024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.327 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.327 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.327 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.327 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.327 DEBUG:boto:Method: GET 2024-04-16T18:18:26.327 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000283 2024-04-16T18:18:26.328 DEBUG:boto:Data: 2024-04-16T18:18:26.328 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.328 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.328 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.328 DEBUG:boto:Params: {} 2024-04-16T18:18:26.328 DEBUG:boto:Token: None 2024-04-16T18:18:26.328 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.329 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001cb0ba48ba69932e-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.335 DEBUG:boto:b'lybtug-51obj000000000000000002831000falseobj000000000000000002832024-04-16T18:17:59.515Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.335 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000317 2024-04-16T18:18:26.335 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.336 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.336 DEBUG:boto:path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.336 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.336 DEBUG:boto:Method: GET 2024-04-16T18:18:26.336 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.336 DEBUG:boto:Data: 2024-04-16T18:18:26.336 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.336 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.336 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.337 DEBUG:boto:Params: {} 2024-04-16T18:18:26.337 DEBUG:boto:Token: None 2024-04-16T18:18:26.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI= 2024-04-16T18:18:26.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI=', 'Content-Length': '0'} 2024-04-16T18:18:26.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d20b717d92656eb7-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.382 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.382 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.383 DEBUG:boto:path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.383 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.383 DEBUG:boto:Method: GET 2024-04-16T18:18:26.383 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.383 DEBUG:boto:Data: 2024-04-16T18:18:26.383 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.383 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.384 DEBUG:boto:Params: {} 2024-04-16T18:18:26.384 DEBUG:boto:Token: None 2024-04-16T18:18:26.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI= 2024-04-16T18:18:26.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI=', 'Content-Length': '0'} 2024-04-16T18:18:26.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f361219c8e4a9884-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.388 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.388 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.388 DEBUG:boto:path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.388 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.388 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.388 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.389 DEBUG:boto:Data: 2024-04-16T18:18:26.389 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.389 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.389 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.389 DEBUG:boto:Params: {} 2024-04-16T18:18:26.389 DEBUG:boto:Token: None 2024-04-16T18:18:26.389 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.389 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BsTVxiRiU2W2CP7Fmlxr0xOmuFY= 2024-04-16T18:18:26.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BsTVxiRiU2W2CP7Fmlxr0xOmuFY=', 'Content-Length': '0'} 2024-04-16T18:18:26.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009d2994a1b51791d-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.393 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.393 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.393 DEBUG:boto:path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.393 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.393 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.393 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.393 DEBUG:boto:Data: 2024-04-16T18:18:26.394 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.394 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.394 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.394 DEBUG:boto:Params: {} 2024-04-16T18:18:26.394 DEBUG:boto:Token: None 2024-04-16T18:18:26.394 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.394 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BsTVxiRiU2W2CP7Fmlxr0xOmuFY= 2024-04-16T18:18:26.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BsTVxiRiU2W2CP7Fmlxr0xOmuFY=', 'Content-Length': '0'} 2024-04-16T18:18:26.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070570c240461a9e1-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.398 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000317 2024-04-16T18:18:26.398 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.398 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.399 DEBUG:boto:path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.399 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.399 DEBUG:boto:Method: GET 2024-04-16T18:18:26.399 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.399 DEBUG:boto:Data: 2024-04-16T18:18:26.399 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.399 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.399 DEBUG:boto:Params: {} 2024-04-16T18:18:26.400 DEBUG:boto:Token: None 2024-04-16T18:18:26.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI= 2024-04-16T18:18:26.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI=', 'Content-Length': '0'} 2024-04-16T18:18:26.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c5cd0d1b1b23ea2-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.446 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.447 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.447 DEBUG:boto:path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.447 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.447 DEBUG:boto:Method: GET 2024-04-16T18:18:26.447 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.447 DEBUG:boto:Data: 2024-04-16T18:18:26.447 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.447 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.448 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.448 DEBUG:boto:Params: {} 2024-04-16T18:18:26.448 DEBUG:boto:Token: None 2024-04-16T18:18:26.448 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317?versionId=null 2024-04-16T18:18:26.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI= 2024-04-16T18:18:26.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DdJ+MDJc80m8P4loXxQJW/2IGCI=', 'Content-Length': '0'} 2024-04-16T18:18:26.452 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006dc6d84bfc2b22d0-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.452 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.452 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.452 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.453 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.453 DEBUG:boto:Method: GET 2024-04-16T18:18:26.453 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.453 DEBUG:boto:Data: 2024-04-16T18:18:26.453 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.453 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.453 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.453 DEBUG:boto:Params: {} 2024-04-16T18:18:26.454 DEBUG:boto:Token: None 2024-04-16T18:18:26.454 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.454 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.458 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086f6e215d910b0b1-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.459 DEBUG:boto:b'lybtug-51obj000000000000000003171000falseobj000000000000000003172024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.459 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.459 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.459 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.460 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.460 DEBUG:boto:Method: GET 2024-04-16T18:18:26.460 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.460 DEBUG:boto:Data: 2024-04-16T18:18:26.460 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.460 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.460 DEBUG:boto:Params: {} 2024-04-16T18:18:26.461 DEBUG:boto:Token: None 2024-04-16T18:18:26.461 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020798a7394fbc47e-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.510 DEBUG:boto:b'lybtug-51obj000000000000000003171000falseobj000000000000000003172024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.511 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.511 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.511 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.511 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.512 DEBUG:boto:Data: 2024-04-16T18:18:26.512 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.512 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.512 DEBUG:boto:Params: {} 2024-04-16T18:18:26.512 DEBUG:boto:Token: None 2024-04-16T18:18:26.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r1jJS2pgGWTgwUNBae1jgbV8yLM= 2024-04-16T18:18:26.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r1jJS2pgGWTgwUNBae1jgbV8yLM=', 'Content-Length': '0'} 2024-04-16T18:18:26.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9b615fb3be58d6a-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.516 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.516 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.517 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.517 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.517 DEBUG:boto:Data: 2024-04-16T18:18:26.517 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.517 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.517 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.517 DEBUG:boto:Params: {} 2024-04-16T18:18:26.517 DEBUG:boto:Token: None 2024-04-16T18:18:26.518 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r1jJS2pgGWTgwUNBae1jgbV8yLM= 2024-04-16T18:18:26.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r1jJS2pgGWTgwUNBae1jgbV8yLM=', 'Content-Length': '0'} 2024-04-16T18:18:26.521 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006503cdf0257d073b-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000317 2024-04-16T18:18:26.521 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.522 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.522 DEBUG:boto:Method: GET 2024-04-16T18:18:26.522 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.522 DEBUG:boto:Data: 2024-04-16T18:18:26.522 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.522 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.522 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.523 DEBUG:boto:Params: {} 2024-04-16T18:18:26.523 DEBUG:boto:Token: None 2024-04-16T18:18:26.523 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.523 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zy8VyuJzchKYmMCVLCw/KuPI6YY= 2024-04-16T18:18:26.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zy8VyuJzchKYmMCVLCw/KuPI6YY=', 'Content-Length': '0'} 2024-04-16T18:18:26.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1b98484ae72cee0-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.570 DEBUG:boto:path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.571 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.571 DEBUG:boto:Method: GET 2024-04-16T18:18:26.571 DEBUG:boto:Path: /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.571 DEBUG:boto:Data: 2024-04-16T18:18:26.571 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.571 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.572 DEBUG:boto:Params: {} 2024-04-16T18:18:26.572 DEBUG:boto:Token: None 2024-04-16T18:18:26.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000317 2024-04-16T18:18:26.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zy8VyuJzchKYmMCVLCw/KuPI6YY= 2024-04-16T18:18:26.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zy8VyuJzchKYmMCVLCw/KuPI6YY=', 'Content-Length': '0'} 2024-04-16T18:18:26.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8b75bd739029d74-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.576 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.576 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.576 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.576 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.577 DEBUG:boto:Method: GET 2024-04-16T18:18:26.577 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.577 DEBUG:boto:Data: 2024-04-16T18:18:26.577 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.577 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.577 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.577 DEBUG:boto:Params: {} 2024-04-16T18:18:26.577 DEBUG:boto:Token: None 2024-04-16T18:18:26.578 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.583 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cd599ffa38de9d2-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.626 DEBUG:boto:b'lybtug-51obj000000000000000003171000falseobj000000000000000003172024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.627 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.627 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.627 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.627 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.628 DEBUG:boto:Method: GET 2024-04-16T18:18:26.628 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000317 2024-04-16T18:18:26.628 DEBUG:boto:Data: 2024-04-16T18:18:26.628 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.628 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.628 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.628 DEBUG:boto:Params: {} 2024-04-16T18:18:26.629 DEBUG:boto:Token: None 2024-04-16T18:18:26.629 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.629 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.636 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd2c4acf191941d6-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.636 DEBUG:boto:b'lybtug-51obj000000000000000003171000falseobj000000000000000003172024-04-16T18:17:58.751Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.636 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000423 2024-04-16T18:18:26.637 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.637 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.637 DEBUG:boto:path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.637 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.637 DEBUG:boto:Method: GET 2024-04-16T18:18:26.637 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.637 DEBUG:boto:Data: 2024-04-16T18:18:26.638 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.638 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.638 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.638 DEBUG:boto:Params: {} 2024-04-16T18:18:26.638 DEBUG:boto:Token: None 2024-04-16T18:18:26.638 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.639 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s= 2024-04-16T18:18:26.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s=', 'Content-Length': '0'} 2024-04-16T18:18:26.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000533a98a68b0eaf98-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.682 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.683 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.683 DEBUG:boto:path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.683 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.683 DEBUG:boto:Method: GET 2024-04-16T18:18:26.683 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.683 DEBUG:boto:Data: 2024-04-16T18:18:26.683 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.684 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.684 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.684 DEBUG:boto:Params: {} 2024-04-16T18:18:26.684 DEBUG:boto:Token: None 2024-04-16T18:18:26.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s= 2024-04-16T18:18:26.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s=', 'Content-Length': '0'} 2024-04-16T18:18:26.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e50340b0dd2b7a2-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.688 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.688 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.688 DEBUG:boto:path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.689 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.689 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.689 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.689 DEBUG:boto:Data: 2024-04-16T18:18:26.689 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.689 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.689 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.689 DEBUG:boto:Params: {} 2024-04-16T18:18:26.690 DEBUG:boto:Token: None 2024-04-16T18:18:26.690 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.690 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HGL85jyceBWDDtgXIC8uM7mvy4U= 2024-04-16T18:18:26.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HGL85jyceBWDDtgXIC8uM7mvy4U=', 'Content-Length': '0'} 2024-04-16T18:18:26.694 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008826ebb8c5e1e407-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.694 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.694 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.694 DEBUG:boto:path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.694 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.694 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.695 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.695 DEBUG:boto:Data: 2024-04-16T18:18:26.695 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.695 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.695 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.695 DEBUG:boto:Params: {} 2024-04-16T18:18:26.695 DEBUG:boto:Token: None 2024-04-16T18:18:26.696 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HGL85jyceBWDDtgXIC8uM7mvy4U= 2024-04-16T18:18:26.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HGL85jyceBWDDtgXIC8uM7mvy4U=', 'Content-Length': '0'} 2024-04-16T18:18:26.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e01be31d0d5535a-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.699 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000423 2024-04-16T18:18:26.700 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.700 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.700 DEBUG:boto:path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.700 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.700 DEBUG:boto:Method: GET 2024-04-16T18:18:26.700 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.700 DEBUG:boto:Data: 2024-04-16T18:18:26.701 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.701 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.701 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.701 DEBUG:boto:Params: {} 2024-04-16T18:18:26.701 DEBUG:boto:Token: None 2024-04-16T18:18:26.701 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.701 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s= 2024-04-16T18:18:26.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s=', 'Content-Length': '0'} 2024-04-16T18:18:26.705 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ff0d8acfa81989b-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.746 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.747 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.747 DEBUG:boto:path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.747 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.747 DEBUG:boto:Method: GET 2024-04-16T18:18:26.747 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.747 DEBUG:boto:Data: 2024-04-16T18:18:26.748 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.748 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.748 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.748 DEBUG:boto:Params: {} 2024-04-16T18:18:26.748 DEBUG:boto:Token: None 2024-04-16T18:18:26.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423?versionId=null 2024-04-16T18:18:26.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s= 2024-04-16T18:18:26.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lvbb4s3xCiEBMEzBVd+mQQEKv4s=', 'Content-Length': '0'} 2024-04-16T18:18:26.752 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000082d747c1fbf69e1-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.752 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.753 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.753 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.753 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.753 DEBUG:boto:Method: GET 2024-04-16T18:18:26.753 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.753 DEBUG:boto:Data: 2024-04-16T18:18:26.753 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.754 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.754 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.754 DEBUG:boto:Params: {} 2024-04-16T18:18:26.754 DEBUG:boto:Token: None 2024-04-16T18:18:26.754 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.755 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.759 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009791002b0303396-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.803 DEBUG:boto:b'lybtug-51obj000000000000000004231000falseobj000000000000000004232024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.803 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.803 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.803 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.804 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.804 DEBUG:boto:Method: GET 2024-04-16T18:18:26.804 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.804 DEBUG:boto:Data: 2024-04-16T18:18:26.804 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.804 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.804 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.804 DEBUG:boto:Params: {} 2024-04-16T18:18:26.805 DEBUG:boto:Token: None 2024-04-16T18:18:26.805 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.811 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002fed3908bec28a69-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.812 DEBUG:boto:b'lybtug-51obj000000000000000004231000falseobj000000000000000004232024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.812 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.813 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.813 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.813 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.813 DEBUG:boto:Data: 2024-04-16T18:18:26.813 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.813 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.814 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.814 DEBUG:boto:Params: {} 2024-04-16T18:18:26.814 DEBUG:boto:Token: None 2024-04-16T18:18:26.814 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JOv688XVq2x25V7sqi8lfoOz6iY= 2024-04-16T18:18:26.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JOv688XVq2x25V7sqi8lfoOz6iY=', 'Content-Length': '0'} 2024-04-16T18:18:26.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0c6dfd53e0a6b44-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.818 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.818 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.818 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.819 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.819 DEBUG:boto:Data: 2024-04-16T18:18:26.819 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.819 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.819 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.819 DEBUG:boto:Params: {} 2024-04-16T18:18:26.819 DEBUG:boto:Token: None 2024-04-16T18:18:26.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JOv688XVq2x25V7sqi8lfoOz6iY= 2024-04-16T18:18:26.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JOv688XVq2x25V7sqi8lfoOz6iY=', 'Content-Length': '0'} 2024-04-16T18:18:26.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f716b93a8d5e49fe-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.823 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000423 2024-04-16T18:18:26.824 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.824 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.824 DEBUG:boto:Method: GET 2024-04-16T18:18:26.824 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.824 DEBUG:boto:Data: 2024-04-16T18:18:26.824 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.825 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.825 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.825 DEBUG:boto:Params: {} 2024-04-16T18:18:26.825 DEBUG:boto:Token: None 2024-04-16T18:18:26.825 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DeuhVL3gk+odiIQGSOqjW/V+ZyM= 2024-04-16T18:18:26.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DeuhVL3gk+odiIQGSOqjW/V+ZyM=', 'Content-Length': '0'} 2024-04-16T18:18:26.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a2c9d5bc56da838-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.870 DEBUG:boto:path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.871 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.871 DEBUG:boto:Method: GET 2024-04-16T18:18:26.871 DEBUG:boto:Path: /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.871 DEBUG:boto:Data: 2024-04-16T18:18:26.871 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.872 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.872 DEBUG:boto:Params: {} 2024-04-16T18:18:26.872 DEBUG:boto:Token: None 2024-04-16T18:18:26.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000423 2024-04-16T18:18:26.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DeuhVL3gk+odiIQGSOqjW/V+ZyM= 2024-04-16T18:18:26.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DeuhVL3gk+odiIQGSOqjW/V+ZyM=', 'Content-Length': '0'} 2024-04-16T18:18:26.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b45be45328693f2e-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.877 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.877 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.877 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.877 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.877 DEBUG:boto:Method: GET 2024-04-16T18:18:26.878 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.878 DEBUG:boto:Data: 2024-04-16T18:18:26.878 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.878 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.878 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.878 DEBUG:boto:Params: {} 2024-04-16T18:18:26.878 DEBUG:boto:Token: None 2024-04-16T18:18:26.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.883 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a12b7c21d72021e-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.884 DEBUG:boto:b'lybtug-51obj000000000000000004231000falseobj000000000000000004232024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.884 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:26.884 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:26.885 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.885 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.885 DEBUG:boto:Method: GET 2024-04-16T18:18:26.885 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000423 2024-04-16T18:18:26.885 DEBUG:boto:Data: 2024-04-16T18:18:26.885 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.885 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.886 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.886 DEBUG:boto:Params: {} 2024-04-16T18:18:26.886 DEBUG:boto:Token: None 2024-04-16T18:18:26.886 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/ 2024-04-16T18:18:26.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac= 2024-04-16T18:18:26.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+z9m6tywBjMg8BjzeDtR37Mgjac=', 'Content-Length': '0'} 2024-04-16T18:18:26.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9cf57c1afdb2465-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.893 DEBUG:boto:b'lybtug-51obj000000000000000004231000falseobj000000000000000004232024-04-16T18:17:58.978Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:26.893 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000467 2024-04-16T18:18:26.893 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.894 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.894 DEBUG:boto:path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.894 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.894 DEBUG:boto:Method: GET 2024-04-16T18:18:26.894 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.894 DEBUG:boto:Data: 2024-04-16T18:18:26.895 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.895 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.895 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.895 DEBUG:boto:Params: {} 2024-04-16T18:18:26.895 DEBUG:boto:Token: None 2024-04-16T18:18:26.895 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jIFxS6cwBrQmhZiWem+bqNGHUiA= 2024-04-16T18:18:26.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jIFxS6cwBrQmhZiWem+bqNGHUiA=', 'Content-Length': '0'} 2024-04-16T18:18:26.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cd9483a06c6fb31-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.942 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.943 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.943 DEBUG:boto:path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.943 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.943 DEBUG:boto:Method: GET 2024-04-16T18:18:26.943 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.943 DEBUG:boto:Data: 2024-04-16T18:18:26.944 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.944 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.944 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.944 DEBUG:boto:Params: {} 2024-04-16T18:18:26.944 DEBUG:boto:Token: None 2024-04-16T18:18:26.944 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jIFxS6cwBrQmhZiWem+bqNGHUiA= 2024-04-16T18:18:26.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jIFxS6cwBrQmhZiWem+bqNGHUiA=', 'Content-Length': '0'} 2024-04-16T18:18:26.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000399c578827fce559-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.949 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.949 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.949 DEBUG:boto:path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.949 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.949 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.949 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.949 DEBUG:boto:Data: 2024-04-16T18:18:26.950 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.950 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.950 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.950 DEBUG:boto:Params: {} 2024-04-16T18:18:26.950 DEBUG:boto:Token: None 2024-04-16T18:18:26.950 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.951 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnYjec2rIE50OMdwEyemmw5puBI= 2024-04-16T18:18:26.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnYjec2rIE50OMdwEyemmw5puBI=', 'Content-Length': '0'} 2024-04-16T18:18:26.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058148d227a2ad46a-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.954 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.954 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.955 DEBUG:boto:path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.955 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.955 DEBUG:boto:Method: HEAD 2024-04-16T18:18:26.955 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.955 DEBUG:boto:Data: 2024-04-16T18:18:26.955 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:26.956 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.956 DEBUG:boto:Params: {} 2024-04-16T18:18:26.956 DEBUG:boto:Token: None 2024-04-16T18:18:26.956 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NnYjec2rIE50OMdwEyemmw5puBI= 2024-04-16T18:18:26.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NnYjec2rIE50OMdwEyemmw5puBI=', 'Content-Length': '0'} 2024-04-16T18:18:26.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036ac73050cb589d2-00661ec0f2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:26.960 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000467 2024-04-16T18:18:26.960 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.960 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:26.960 DEBUG:boto:path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.961 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.961 DEBUG:boto:Method: GET 2024-04-16T18:18:26.961 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.961 DEBUG:boto:Data: 2024-04-16T18:18:26.961 DEBUG:boto:Headers: {} 2024-04-16T18:18:26.961 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:26.961 DEBUG:boto:Port: 8000 2024-04-16T18:18:26.962 DEBUG:boto:Params: {} 2024-04-16T18:18:26.962 DEBUG:boto:Token: None 2024-04-16T18:18:26.962 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:26 GMT /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:26.962 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jIFxS6cwBrQmhZiWem+bqNGHUiA= 2024-04-16T18:18:26.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jIFxS6cwBrQmhZiWem+bqNGHUiA=', 'Content-Length': '0'} 2024-04-16T18:18:26.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b763be8d0d5981c-00661ec0f2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.007 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.007 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.007 DEBUG:boto:path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:27.007 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:27.007 DEBUG:boto:Method: GET 2024-04-16T18:18:27.007 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:27.008 DEBUG:boto:Data: 2024-04-16T18:18:27.008 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.008 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:27.008 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.008 DEBUG:boto:Params: {} 2024-04-16T18:18:27.008 DEBUG:boto:Token: None 2024-04-16T18:18:27.009 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/obj00000000000000000467?versionId=null 2024-04-16T18:18:27.009 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AtlHD+ug5Oa7ZeW3TYrTd0JVbmY= 2024-04-16T18:18:27.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AtlHD+ug5Oa7ZeW3TYrTd0JVbmY=', 'Content-Length': '0'} 2024-04-16T18:18:27.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065a668278968026d-00661ec0f3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.013 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:27.013 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:27.013 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.013 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.014 DEBUG:boto:Method: GET 2024-04-16T18:18:27.014 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.014 DEBUG:boto:Data: 2024-04-16T18:18:27.014 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.014 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:27.014 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.015 DEBUG:boto:Params: {} 2024-04-16T18:18:27.015 DEBUG:boto:Token: None 2024-04-16T18:18:27.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/ 2024-04-16T18:18:27.015 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q= 2024-04-16T18:18:27.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q=', 'Content-Length': '0'} 2024-04-16T18:18:27.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046e4dfc19caa01bf-00661ec0f3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.020 DEBUG:boto:b'lybtug-51obj000000000000000004671000falseobj000000000000000004672024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:27.021 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:27.021 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:27.021 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.022 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.022 DEBUG:boto:Method: GET 2024-04-16T18:18:27.022 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.022 DEBUG:boto:Data: 2024-04-16T18:18:27.022 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.022 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:27.022 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.023 DEBUG:boto:Params: {} 2024-04-16T18:18:27.023 DEBUG:boto:Token: None 2024-04-16T18:18:27.023 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/ 2024-04-16T18:18:27.023 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q= 2024-04-16T18:18:27.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q=', 'Content-Length': '0'} 2024-04-16T18:18:27.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ad1bc79f4133603-00661ec0f3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.071 DEBUG:boto:b'lybtug-51obj000000000000000004671000falseobj000000000000000004672024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:27.071 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.071 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.071 DEBUG:boto:Method: HEAD 2024-04-16T18:18:27.072 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.072 DEBUG:boto:Data: 2024-04-16T18:18:27.072 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.072 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:27.072 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.072 DEBUG:boto:Params: {} 2024-04-16T18:18:27.073 DEBUG:boto:Token: None 2024-04-16T18:18:27.073 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+SEXIaPGwwm5jKbCCKjyc5EphvY= 2024-04-16T18:18:27.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+SEXIaPGwwm5jKbCCKjyc5EphvY=', 'Content-Length': '0'} 2024-04-16T18:18:27.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bb4ce3867cbda15-00661ec0f3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.077 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.077 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.077 DEBUG:boto:Method: HEAD 2024-04-16T18:18:27.077 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.077 DEBUG:boto:Data: 2024-04-16T18:18:27.077 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.078 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:27.078 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.078 DEBUG:boto:Params: {} 2024-04-16T18:18:27.078 DEBUG:boto:Token: None 2024-04-16T18:18:27.078 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.079 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+SEXIaPGwwm5jKbCCKjyc5EphvY= 2024-04-16T18:18:27.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+SEXIaPGwwm5jKbCCKjyc5EphvY=', 'Content-Length': '0'} 2024-04-16T18:18:27.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071876a821ca70c45-00661ec0f3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.083 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj00000000000000000467 2024-04-16T18:18:27.083 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.083 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.083 DEBUG:boto:Method: GET 2024-04-16T18:18:27.083 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.083 DEBUG:boto:Data: 2024-04-16T18:18:27.084 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.084 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:27.084 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.084 DEBUG:boto:Params: {} 2024-04-16T18:18:27.084 DEBUG:boto:Token: None 2024-04-16T18:18:27.084 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.085 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZzsnrYEBBPVM2SuQ/t/xAYPXp1o= 2024-04-16T18:18:27.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZzsnrYEBBPVM2SuQ/t/xAYPXp1o=', 'Content-Length': '0'} 2024-04-16T18:18:27.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-51:054102a3-8d94-4596-8b3b-6341eab312e1.4804.20'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:18:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8b110defed3c818-00661ec0f3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.131 DEBUG:boto:path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.131 DEBUG:boto:auth_path=/lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.131 DEBUG:boto:Method: GET 2024-04-16T18:18:27.131 DEBUG:boto:Path: /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.131 DEBUG:boto:Data: 2024-04-16T18:18:27.131 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.132 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:27.132 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.132 DEBUG:boto:Params: {} 2024-04-16T18:18:27.132 DEBUG:boto:Token: None 2024-04-16T18:18:27.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/obj00000000000000000467 2024-04-16T18:18:27.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZzsnrYEBBPVM2SuQ/t/xAYPXp1o= 2024-04-16T18:18:27.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZzsnrYEBBPVM2SuQ/t/xAYPXp1o=', 'Content-Length': '0'} 2024-04-16T18:18:27.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:17:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4190babe03929a4-00661ec0f3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.136 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:27.137 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:27.137 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.137 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.137 DEBUG:boto:Method: GET 2024-04-16T18:18:27.137 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.137 DEBUG:boto:Data: 2024-04-16T18:18:27.138 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.138 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:18:27.138 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.138 DEBUG:boto:Params: {} 2024-04-16T18:18:27.138 DEBUG:boto:Token: None 2024-04-16T18:18:27.138 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/ 2024-04-16T18:18:27.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q= 2024-04-16T18:18:27.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q=', 'Content-Length': '0'} 2024-04-16T18:18:27.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec7b0139cd328bd2-00661ec0f3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.144 DEBUG:boto:b'lybtug-51obj000000000000000004671000falseobj000000000000000004672024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:27.145 DEBUG:boto:path=/lybtug-51/ 2024-04-16T18:18:27.145 DEBUG:boto:auth_path=/lybtug-51/ 2024-04-16T18:18:27.145 DEBUG:boto:path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.145 DEBUG:boto:auth_path=/lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.145 DEBUG:boto:Method: GET 2024-04-16T18:18:27.145 DEBUG:boto:Path: /lybtug-51/?prefix=obj00000000000000000467 2024-04-16T18:18:27.145 DEBUG:boto:Data: 2024-04-16T18:18:27.146 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.146 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:27.146 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.146 DEBUG:boto:Params: {} 2024-04-16T18:18:27.146 DEBUG:boto:Token: None 2024-04-16T18:18:27.146 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:18:27 GMT /lybtug-51/ 2024-04-16T18:18:27.147 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q= 2024-04-16T18:18:27.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t74Xiou/Lk8TQ80c90n0QOQGm/Q=', 'Content-Length': '0'} 2024-04-16T18:18:27.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000893bc3c5c076ffe5-00661ec0f3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.154 DEBUG:boto:b'lybtug-51obj000000000000000004671000falseobj000000000000000004672024-04-16T18:17:58.773Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:18:27.155 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-51 zones={test-zone2, test-zone1} 2024-04-16T18:18:27.155 INFO:tasks.rgw_multisite_tests:Create several generations of objects, then run bucket sync ... ok 2024-04-16T18:18:27.159 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:18:27.159 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:18:27.159 DEBUG:boto:Method: PUT 2024-04-16T18:18:27.159 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:18:27.159 DEBUG:boto:Data: 2024-04-16T18:18:27.160 DEBUG:boto:Headers: {} 2024-04-16T18:18:27.160 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:18:27.160 DEBUG:boto:Port: 8000 2024-04-16T18:18:27.160 DEBUG:boto:Params: {} 2024-04-16T18:18:27.160 DEBUG:boto:Token: None 2024-04-16T18:18:27.161 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:18:27 GMT /lybtug-52/ 2024-04-16T18:18:27.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xeDWpjTlMkIk7xQR2+Qh2QCT6i0= 2024-04-16T18:18:27.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:18:27 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xeDWpjTlMkIk7xQR2+Qh2QCT6i0=', 'Content-Length': '0'} 2024-04-16T18:18:27.196 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000007c112dd0a76d47c-00661ec0f3-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:18:27 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:18:27.196 DEBUG:rgw_multi.tests:created bucket=lybtug-52 2024-04-16T18:18:27.197 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:27.197 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:27.197 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:27.242 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:27.242 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:27.312 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:18:27.347 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713291507.196311_646.1","last_update":"2024-04-16T18:18:27.196311Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:18:27.347 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:27.347 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:18:27.348 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:27.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:27.348 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:27.391 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:27.391 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:27.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:27.463+0000 7f92e8500980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:27.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:27.467+0000 7f92e8500980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:27.495 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:27Z"} 2024-04-16T18:18:27.496 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:27Z"} 2024-04-16T18:18:27.496 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:27.496 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:27.496 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:27.496 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:18:32.497 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:32.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:32.498 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:32.543 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:32.543 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:32.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:32.615+0000 7f5191aa3980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:32.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:32.619+0000 7f5191aa3980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:32.648 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:32Z"} 2024-04-16T18:18:32.649 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:32Z"} 2024-04-16T18:18:32.649 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:32.649 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:32.649 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:32.649 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:18:37.650 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:37.651 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:37.651 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:37.767+0000 7f7b8ece7980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:37.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:37.767+0000 7f7b8ece7980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:37.799 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:37Z"} 2024-04-16T18:18:37.799 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:37Z"} 2024-04-16T18:18:37.799 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:37.799 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:37.799 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:37.799 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:18:42.801 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:42.801 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:42.801 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:42.844 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:42.845 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:42.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:42.919+0000 7f35b3894980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:42.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:42.923+0000 7f35b3894980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:42.952 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:42Z"} 2024-04-16T18:18:42.952 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:42Z"} 2024-04-16T18:18:42.952 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:42.952 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:42.953 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:42.953 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:18:47.953 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:47.954 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:47.954 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:47.996 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:48.066+0000 7f412eea2980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:48.070+0000 7f412eea2980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:48.100 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:48Z"} 2024-04-16T18:18:48.100 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:48Z"} 2024-04-16T18:18:48.100 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:48.100 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:48.100 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:48.101 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:18:53.101 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:53.102 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:53.102 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:53.146 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:53.146 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:53.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:53.214+0000 7fb45bcd5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:53.218+0000 7fb45bcd5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:53.249 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:53Z"} 2024-04-16T18:18:53.249 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:53Z"} 2024-04-16T18:18:53.250 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:53.250 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:53.250 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:53.250 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:18:58.251 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:58.251 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:18:58.251 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:18:58.294 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:18:58.294 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:18:58.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:58.366+0000 7f64e9707980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:58.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:18:58.366+0000 7f64e9707980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:18:58.400 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:58Z"} 2024-04-16T18:18:58.400 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:18:58Z"} 2024-04-16T18:18:58.400 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:18:58.401 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:58.401 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:18:58.401 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:03.402 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:03.403 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:03.403 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:03.446 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:03.446 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:03.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:03.518+0000 7f48a0252980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:03.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:03.518+0000 7f48a0252980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:03.550 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:03Z"} 2024-04-16T18:19:03.550 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:03Z"} 2024-04-16T18:19:03.551 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:03.551 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:03.551 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:03.551 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:08.552 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:08.552 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:08.552 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:08.595 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:08.596 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:08.666+0000 7f260f949980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:08.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:08.670+0000 7f260f949980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:08.701 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:08Z"} 2024-04-16T18:19:08.701 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:08Z"} 2024-04-16T18:19:08.702 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:08.702 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:08.702 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:08.702 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:13.703 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:13.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:13.704 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:13.746 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:13.746 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:13.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:13.818+0000 7f1c1eea0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:13.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:13.822+0000 7f1c1eea0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:13.852 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:13Z"} 2024-04-16T18:19:13.852 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:13Z"} 2024-04-16T18:19:13.852 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:13.852 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:13.853 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:13.853 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:18.853 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:18.854 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:18.854 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:18.901 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:18.901 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:18.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:18.969+0000 7f8600148980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:18.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:18.973+0000 7f8600148980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:19.005 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:18Z"} 2024-04-16T18:19:19.006 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:18Z"} 2024-04-16T18:19:19.006 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:19.006 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:19.006 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:19.006 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:24.007 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:24.007 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:24.008 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:24.051 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:24.051 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:24.121+0000 7f32448dc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:24.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:24.121+0000 7f32448dc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:24.155 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:24Z"} 2024-04-16T18:19:24.155 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:24Z"} 2024-04-16T18:19:24.156 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:24.156 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:24.156 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:24.156 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:29.157 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:29.157 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:29.157 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:29.200 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:29.201 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:29.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:29.277+0000 7fd78c233980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:29.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:29.277+0000 7fd78c233980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:29.311 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:29Z"} 2024-04-16T18:19:29.311 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:29Z"} 2024-04-16T18:19:29.311 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:29.311 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:29.311 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:29.312 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:34.313 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:34.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:34.313 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:34.356 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:34.356 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:34.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:34.425+0000 7fe181cd5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:34.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:34.429+0000 7fe181cd5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:34.461 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:34Z"} 2024-04-16T18:19:34.462 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:34Z"} 2024-04-16T18:19:34.462 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:34.462 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:34.462 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:34.462 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:39.463 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:39.464 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:39.464 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:39.506 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:39.507 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:39.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:39.581+0000 7fa58633e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:39.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:39.585+0000 7fa58633e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:39.613 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:39Z"} 2024-04-16T18:19:39.613 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:39Z"} 2024-04-16T18:19:39.613 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:39.614 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:39.614 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:39.614 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:44.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:44.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:44.616 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:44.658 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:44.658 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:44.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:44.737+0000 7f72adbbb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:44.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:44.737+0000 7f72adbbb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:44.769 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:44Z"} 2024-04-16T18:19:44.769 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:44Z"} 2024-04-16T18:19:44.769 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:44.769 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:44.769 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:44.769 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:49.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:49.771 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:49.771 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:49.814 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:49.815 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:49.888+0000 7f58dffd8980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:49.888+0000 7f58dffd8980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:49.920 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:49Z"} 2024-04-16T18:19:49.920 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:49Z"} 2024-04-16T18:19:49.920 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:49.920 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:49.920 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:49.920 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:19:54.921 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:54.922 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:19:54.922 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:19:54.964 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:19:54.964 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:19:55.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:55.036+0000 7ffa39c05980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:55.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:19:55.040+0000 7ffa39c05980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:19:55.069 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:55Z"} 2024-04-16T18:19:55.069 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291453.320908_569.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:17:33.320908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:19:55Z"} 2024-04-16T18:19:55.069 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291453.320908_569.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:17:33.320908Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:19:55.069 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:55.069 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291453.320908_569.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:19:55.070 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713291507.196311_646.1 target=1_1713291453.320908_569.1 2024-04-16T18:20:00.070 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:20:00.070 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:20:00.071 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:20:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:20:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:20:00.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:00.184+0000 7f0c4c4e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:20:00.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:00.188+0000 7f0c4c4e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:20:00.217 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291507.196311_646.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:18:27.196311Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:20:00Z"} 2024-04-16T18:20:00.217 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291507.196311_646.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:18:27.196311Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:20:00Z"} 2024-04-16T18:20:00.217 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291507.196311_646.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:18:27.196311Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:20:00.217 DEBUG:rgw_multi.tests:log_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:20:00.217 DEBUG:rgw_multi.tests:sync_status={0: '1_1713291507.196311_646.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:20:00.218 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:20:00.218 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:00.218 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:00.218 DEBUG:boto:Method: HEAD 2024-04-16T18:20:00.218 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:00.218 DEBUG:boto:Data: 2024-04-16T18:20:00.218 DEBUG:boto:Headers: {} 2024-04-16T18:20:00.218 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.219 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.219 DEBUG:boto:Params: {} 2024-04-16T18:20:00.219 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:20:00.219 DEBUG:boto:Token: None 2024-04-16T18:20:00.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/ 2024-04-16T18:20:00.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo= 2024-04-16T18:20:00.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo=', 'Content-Length': '0'} 2024-04-16T18:20:00.227 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000db6893c4efdbf77d-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.227 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:00.228 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:00.228 DEBUG:boto:Method: PUT 2024-04-16T18:20:00.228 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:20:00.228 DEBUG:boto:Data: 2024-04-16T18:20:00.228 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:00.228 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.228 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.228 DEBUG:boto:Params: {} 2024-04-16T18:20:00.228 DEBUG:boto:Token: None 2024-04-16T18:20:00.229 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/obj11 2024-04-16T18:20:00.229 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/ipHwXsIIKdRS3Bm3/E/miz/LJo= 2024-04-16T18:20:00.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/ipHwXsIIKdRS3Bm3/E/miz/LJo='} 2024-04-16T18:20:00.243 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d0d750678c659b1f-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.243 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:00.243 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:00.243 DEBUG:boto:Method: HEAD 2024-04-16T18:20:00.243 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:00.243 DEBUG:boto:Data: 2024-04-16T18:20:00.244 DEBUG:boto:Headers: {} 2024-04-16T18:20:00.244 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.244 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.244 DEBUG:boto:Params: {} 2024-04-16T18:20:00.244 DEBUG:boto:Token: None 2024-04-16T18:20:00.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/ 2024-04-16T18:20:00.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo= 2024-04-16T18:20:00.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo=', 'Content-Length': '0'} 2024-04-16T18:20:00.248 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000063d9cfcad24c5eb2-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.248 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:00.248 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:00.248 DEBUG:boto:Method: PUT 2024-04-16T18:20:00.248 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:20:00.248 DEBUG:boto:Data: 2024-04-16T18:20:00.248 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:00.249 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.249 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.249 DEBUG:boto:Params: {} 2024-04-16T18:20:00.249 DEBUG:boto:Token: None 2024-04-16T18:20:00.249 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/obj22 2024-04-16T18:20:00.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EsmK+atym14S/wUo49IUIVikzcU= 2024-04-16T18:20:00.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EsmK+atym14S/wUo49IUIVikzcU='} 2024-04-16T18:20:00.294 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002b7acf01e96e8c79-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.294 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:00.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:00.295 DEBUG:boto:Method: HEAD 2024-04-16T18:20:00.295 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:00.295 DEBUG:boto:Data: 2024-04-16T18:20:00.295 DEBUG:boto:Headers: {} 2024-04-16T18:20:00.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.295 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.295 DEBUG:boto:Params: {} 2024-04-16T18:20:00.295 DEBUG:boto:Token: None 2024-04-16T18:20:00.295 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/ 2024-04-16T18:20:00.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo= 2024-04-16T18:20:00.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo=', 'Content-Length': '0'} 2024-04-16T18:20:00.299 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000950f7ab18244b03f-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.299 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:00.300 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:00.300 DEBUG:boto:Method: PUT 2024-04-16T18:20:00.300 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:20:00.300 DEBUG:boto:Data: 2024-04-16T18:20:00.300 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:00.300 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.300 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.300 DEBUG:boto:Params: {} 2024-04-16T18:20:00.300 DEBUG:boto:Token: None 2024-04-16T18:20:00.300 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/obj33 2024-04-16T18:20:00.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6o87NAkeHyz4i4TwfI2kepFPot0= 2024-04-16T18:20:00.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6o87NAkeHyz4i4TwfI2kepFPot0='} 2024-04-16T18:20:00.312 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000017336334009ea27a-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.313 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:00.313 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:00.313 DEBUG:boto:Method: HEAD 2024-04-16T18:20:00.313 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:00.313 DEBUG:boto:Data: 2024-04-16T18:20:00.313 DEBUG:boto:Headers: {} 2024-04-16T18:20:00.313 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.313 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.313 DEBUG:boto:Params: {} 2024-04-16T18:20:00.314 DEBUG:boto:Token: None 2024-04-16T18:20:00.314 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/ 2024-04-16T18:20:00.314 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo= 2024-04-16T18:20:00.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo=', 'Content-Length': '0'} 2024-04-16T18:20:00.318 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003884a8c238b50cfe-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.318 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:00.318 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:00.318 DEBUG:boto:Method: PUT 2024-04-16T18:20:00.318 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:20:00.319 DEBUG:boto:Data: 2024-04-16T18:20:00.319 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:00.319 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.319 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.319 DEBUG:boto:Params: {} 2024-04-16T18:20:00.319 DEBUG:boto:Token: None 2024-04-16T18:20:00.319 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/obj44 2024-04-16T18:20:00.319 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0T5eWUn8NzO7R1gku2lYFr6caXk= 2024-04-16T18:20:00.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0T5eWUn8NzO7R1gku2lYFr6caXk='} 2024-04-16T18:20:00.362 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000312b4069d877925f-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.363 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:00.363 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:00.363 DEBUG:boto:Method: HEAD 2024-04-16T18:20:00.363 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:00.363 DEBUG:boto:Data: 2024-04-16T18:20:00.363 DEBUG:boto:Headers: {} 2024-04-16T18:20:00.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.363 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.363 DEBUG:boto:Params: {} 2024-04-16T18:20:00.364 DEBUG:boto:Token: None 2024-04-16T18:20:00.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/ 2024-04-16T18:20:00.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo= 2024-04-16T18:20:00.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mhHT6r6n4ltjwDicTqUonIwfyCo=', 'Content-Length': '0'} 2024-04-16T18:20:00.368 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a2feee9f91cf4cfc-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.368 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:00.368 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:00.368 DEBUG:boto:Method: PUT 2024-04-16T18:20:00.368 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:20:00.368 DEBUG:boto:Data: 2024-04-16T18:20:00.368 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:00.369 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:00.369 DEBUG:boto:Port: 8000 2024-04-16T18:20:00.369 DEBUG:boto:Params: {} 2024-04-16T18:20:00.369 DEBUG:boto:Token: None 2024-04-16T18:20:00.369 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:00 GMT /lybtug-52/obj55 2024-04-16T18:20:00.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+T2D8zD7ejbHqmLPyor4gyXITaw= 2024-04-16T18:20:00.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:00 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+T2D8zD7ejbHqmLPyor4gyXITaw='} 2024-04-16T18:20:00.379 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000952ebac28915e849-00661ec150-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:00.379 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:20:00.379 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:20:00.379 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:20:00.379 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:20:00.430 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:20:00.430 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:20:00.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:00.512+0000 7f6ec984f980 1 waiting to reach incremental sync.. 2024-04-16T18:20:05.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:05.516+0000 7f6ec984f980 1 bucket sync caught up with source: 2024-04-16T18:20:05.514 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.77.6, 00000000002.105.6, 00000000002.96.6] 2024-04-16T18:20:05.514 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , , , 00000000001.77.6, 00000000002.105.6, 00000000002.96.6] 2024-04-16T18:20:05.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:05.516+0000 7f6ec984f980 0 bucket checkpoint complete 2024-04-16T18:20:05.561 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:20:05.561 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:20:05.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:20:05.562 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:20:05.612 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:20:05.612 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:20:05.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:20:05.698+0000 7fc0c8b1b980 1 waiting to reach incremental sync.. 2024-04-16T18:20:10.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:20:10.702+0000 7fc0c8b1b980 1 bucket sync caught up with source: 2024-04-16T18:20:10.698 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, , , , , , , , 00000000001.458.6, 00000000002.313.6, 00000000002.404.6] 2024-04-16T18:20:10.698 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, , , , , , , , 00000000001.458.6, 00000000002.313.6, 00000000002.404.6] 2024-04-16T18:20:10.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:20:10.702+0000 7fc0c8b1b980 0 bucket checkpoint complete 2024-04-16T18:20:10.741 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:20:10.741 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.741 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.741 DEBUG:boto:Method: HEAD 2024-04-16T18:20:10.742 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:10.742 DEBUG:boto:Data: 2024-04-16T18:20:10.742 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.742 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.742 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.742 DEBUG:boto:Params: {} 2024-04-16T18:20:10.742 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:20:10.743 DEBUG:boto:Token: None 2024-04-16T18:20:10.743 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/ 2024-04-16T18:20:10.743 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:72d25tlcvsDNZnESQV2yhscUbvs= 2024-04-16T18:20:10.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:72d25tlcvsDNZnESQV2yhscUbvs=', 'Content-Length': '0'} 2024-04-16T18:20:10.749 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000982be5abf339cf59-00661ec15a-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.749 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.749 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.749 DEBUG:boto:Method: HEAD 2024-04-16T18:20:10.749 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:10.749 DEBUG:boto:Data: 2024-04-16T18:20:10.750 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.750 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.750 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.750 DEBUG:boto:Params: {} 2024-04-16T18:20:10.750 DEBUG:boto:Token: None 2024-04-16T18:20:10.750 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/ 2024-04-16T18:20:10.750 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:72d25tlcvsDNZnESQV2yhscUbvs= 2024-04-16T18:20:10.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:72d25tlcvsDNZnESQV2yhscUbvs=', 'Content-Length': '0'} 2024-04-16T18:20:10.754 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004f46493fe1ce4469-00661ec15a-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.754 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:20:10.755 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.755 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.755 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.755 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.755 DEBUG:boto:Method: GET 2024-04-16T18:20:10.755 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.755 DEBUG:boto:Data: 2024-04-16T18:20:10.755 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.755 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.755 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.755 DEBUG:boto:Params: {} 2024-04-16T18:20:10.756 DEBUG:boto:Token: None 2024-04-16T18:20:10.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/?versions 2024-04-16T18:20:10.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g= 2024-04-16T18:20:10.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g=', 'Content-Length': '0'} 2024-04-16T18:20:10.759 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b935441d4b439431-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.759 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:10.760 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:20:10.760 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:20:10.760 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:20:10.760 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:20:10.761 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:20:10.761 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:20:10.761 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.761 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.761 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.761 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.761 DEBUG:boto:Method: GET 2024-04-16T18:20:10.761 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.761 DEBUG:boto:Data: 2024-04-16T18:20:10.761 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.761 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.762 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.762 DEBUG:boto:Params: {} 2024-04-16T18:20:10.762 DEBUG:boto:Token: None 2024-04-16T18:20:10.762 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/?versions 2024-04-16T18:20:10.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g= 2024-04-16T18:20:10.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g=', 'Content-Length': '0'} 2024-04-16T18:20:10.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000963166e6b1f52b8-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.766 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:10.766 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:20:10.767 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:20:10.767 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:20:10.767 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:20:10.767 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:20:10.767 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.767 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.767 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.767 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.767 DEBUG:boto:Method: GET 2024-04-16T18:20:10.768 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.768 DEBUG:boto:Data: 2024-04-16T18:20:10.768 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.768 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.768 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.768 DEBUG:boto:Params: {} 2024-04-16T18:20:10.768 DEBUG:boto:Token: None 2024-04-16T18:20:10.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/?versions 2024-04-16T18:20:10.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g= 2024-04-16T18:20:10.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g=', 'Content-Length': '0'} 2024-04-16T18:20:10.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004202d77909167e48-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.772 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:10.773 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.773 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.773 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.773 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.773 DEBUG:boto:Method: GET 2024-04-16T18:20:10.773 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:20:10.773 DEBUG:boto:Data: 2024-04-16T18:20:10.773 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.773 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.773 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.774 DEBUG:boto:Params: {} 2024-04-16T18:20:10.774 DEBUG:boto:Token: None 2024-04-16T18:20:10.774 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/?versions 2024-04-16T18:20:10.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g= 2024-04-16T18:20:10.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+jT0jDRxLyp+ShT6u9pq4bepv8g=', 'Content-Length': '0'} 2024-04-16T18:20:10.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003b83acd59582bb8-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.778 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:10.778 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:20:10.779 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.779 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.779 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.779 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.779 DEBUG:boto:Method: GET 2024-04-16T18:20:10.779 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.779 DEBUG:boto:Data: 2024-04-16T18:20:10.779 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.779 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.779 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.779 DEBUG:boto:Params: {} 2024-04-16T18:20:10.780 DEBUG:boto:Token: None 2024-04-16T18:20:10.780 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs= 2024-04-16T18:20:10.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs=', 'Content-Length': '0'} 2024-04-16T18:20:10.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bc5ab71cddf5380-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.826 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.827 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.827 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.827 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.827 DEBUG:boto:Method: GET 2024-04-16T18:20:10.827 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.827 DEBUG:boto:Data: 2024-04-16T18:20:10.827 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.827 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.827 DEBUG:boto:Params: {} 2024-04-16T18:20:10.827 DEBUG:boto:Token: None 2024-04-16T18:20:10.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs= 2024-04-16T18:20:10.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs=', 'Content-Length': '0'} 2024-04-16T18:20:10.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c57761e52b9dd63b-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.831 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.831 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.831 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.831 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.831 DEBUG:boto:Method: HEAD 2024-04-16T18:20:10.831 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.832 DEBUG:boto:Data: 2024-04-16T18:20:10.832 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.832 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.832 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.832 DEBUG:boto:Params: {} 2024-04-16T18:20:10.832 DEBUG:boto:Token: None 2024-04-16T18:20:10.832 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IwEesMKiAEc0hfr58Ze266Ve9I8= 2024-04-16T18:20:10.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IwEesMKiAEc0hfr58Ze266Ve9I8=', 'Content-Length': '0'} 2024-04-16T18:20:10.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f86dc3c97aee85b4-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.836 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.836 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.836 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.836 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.836 DEBUG:boto:Method: HEAD 2024-04-16T18:20:10.836 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.836 DEBUG:boto:Data: 2024-04-16T18:20:10.836 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.836 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.836 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.836 DEBUG:boto:Params: {} 2024-04-16T18:20:10.837 DEBUG:boto:Token: None 2024-04-16T18:20:10.837 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IwEesMKiAEc0hfr58Ze266Ve9I8= 2024-04-16T18:20:10.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IwEesMKiAEc0hfr58Ze266Ve9I8=', 'Content-Length': '0'} 2024-04-16T18:20:10.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d52b987b6da4b158-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:20:10.840 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.840 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.840 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.840 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.841 DEBUG:boto:Method: GET 2024-04-16T18:20:10.841 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.841 DEBUG:boto:Data: 2024-04-16T18:20:10.841 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.841 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.841 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.841 DEBUG:boto:Params: {} 2024-04-16T18:20:10.841 DEBUG:boto:Token: None 2024-04-16T18:20:10.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs= 2024-04-16T18:20:10.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs=', 'Content-Length': '0'} 2024-04-16T18:20:10.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cafc1173a9ce8695-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.886 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.886 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.886 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.887 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.887 DEBUG:boto:Method: GET 2024-04-16T18:20:10.887 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.887 DEBUG:boto:Data: 2024-04-16T18:20:10.887 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.887 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.887 DEBUG:boto:Params: {} 2024-04-16T18:20:10.887 DEBUG:boto:Token: None 2024-04-16T18:20:10.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:20:10.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs= 2024-04-16T18:20:10.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mx+/5lhaxY8hxjfoX2Jn/h2u1bs=', 'Content-Length': '0'} 2024-04-16T18:20:10.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f910c7d0151d03bc-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.891 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.891 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.891 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:10.892 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:10.892 DEBUG:boto:Method: GET 2024-04-16T18:20:10.892 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:20:10.892 DEBUG:boto:Data: 2024-04-16T18:20:10.892 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.892 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.892 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.892 DEBUG:boto:Params: {} 2024-04-16T18:20:10.892 DEBUG:boto:Token: None 2024-04-16T18:20:10.893 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/ 2024-04-16T18:20:10.893 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5D8rOvX9+LiZloCRoX6dlQR4HEU= 2024-04-16T18:20:10.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5D8rOvX9+LiZloCRoX6dlQR4HEU=', 'Content-Length': '0'} 2024-04-16T18:20:10.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004cf99a917a40781c-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.938 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:10.939 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:10.939 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:10.939 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:10.939 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:10.939 DEBUG:boto:Method: GET 2024-04-16T18:20:10.939 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:20:10.939 DEBUG:boto:Data: 2024-04-16T18:20:10.939 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.939 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.939 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.939 DEBUG:boto:Params: {} 2024-04-16T18:20:10.940 DEBUG:boto:Token: None 2024-04-16T18:20:10.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/ 2024-04-16T18:20:10.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5D8rOvX9+LiZloCRoX6dlQR4HEU= 2024-04-16T18:20:10.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5D8rOvX9+LiZloCRoX6dlQR4HEU=', 'Content-Length': '0'} 2024-04-16T18:20:10.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d4b237dcae20bb5-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.943 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:10.944 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.944 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.944 DEBUG:boto:Method: HEAD 2024-04-16T18:20:10.944 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:20:10.944 DEBUG:boto:Data: 2024-04-16T18:20:10.944 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.944 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.944 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.944 DEBUG:boto:Params: {} 2024-04-16T18:20:10.945 DEBUG:boto:Token: None 2024-04-16T18:20:10.945 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11 2024-04-16T18:20:10.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R+dL/e1IywjOMyOuSZFomJPC814= 2024-04-16T18:20:10.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R+dL/e1IywjOMyOuSZFomJPC814=', 'Content-Length': '0'} 2024-04-16T18:20:10.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d03d602199f25445-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.948 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.948 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.948 DEBUG:boto:Method: HEAD 2024-04-16T18:20:10.948 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:20:10.948 DEBUG:boto:Data: 2024-04-16T18:20:10.949 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.949 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.949 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.949 DEBUG:boto:Params: {} 2024-04-16T18:20:10.949 DEBUG:boto:Token: None 2024-04-16T18:20:10.949 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11 2024-04-16T18:20:10.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R+dL/e1IywjOMyOuSZFomJPC814= 2024-04-16T18:20:10.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R+dL/e1IywjOMyOuSZFomJPC814=', 'Content-Length': '0'} 2024-04-16T18:20:10.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000404b04c8f926162-00661ec15a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.953 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:20:10.953 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.953 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.953 DEBUG:boto:Method: GET 2024-04-16T18:20:10.953 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:20:10.953 DEBUG:boto:Data: 2024-04-16T18:20:10.953 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.953 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:10.953 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.953 DEBUG:boto:Params: {} 2024-04-16T18:20:10.954 DEBUG:boto:Token: None 2024-04-16T18:20:10.954 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11 2024-04-16T18:20:10.954 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7QrnsYVTEn3xMDV8s9GCEZYGLhw= 2024-04-16T18:20:10.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7QrnsYVTEn3xMDV8s9GCEZYGLhw=', 'Content-Length': '0'} 2024-04-16T18:20:10.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9ea3250957488a8-00661ec15a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:10 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:10.998 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:20:10.999 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:20:10.999 DEBUG:boto:Method: GET 2024-04-16T18:20:10.999 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:20:10.999 DEBUG:boto:Data: 2024-04-16T18:20:10.999 DEBUG:boto:Headers: {} 2024-04-16T18:20:10.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:10.999 DEBUG:boto:Port: 8000 2024-04-16T18:20:10.999 DEBUG:boto:Params: {} 2024-04-16T18:20:10.999 DEBUG:boto:Token: None 2024-04-16T18:20:11.000 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:10 GMT /lybtug-52/obj11 2024-04-16T18:20:11.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7QrnsYVTEn3xMDV8s9GCEZYGLhw= 2024-04-16T18:20:11.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:10 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7QrnsYVTEn3xMDV8s9GCEZYGLhw=', 'Content-Length': '0'} 2024-04-16T18:20:11.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4ba4a8e920445da-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.003 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.003 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.003 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:11.003 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:11.004 DEBUG:boto:Method: GET 2024-04-16T18:20:11.004 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:20:11.004 DEBUG:boto:Data: 2024-04-16T18:20:11.004 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.004 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.004 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.004 DEBUG:boto:Params: {} 2024-04-16T18:20:11.004 DEBUG:boto:Token: None 2024-04-16T18:20:11.004 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.009 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052eb759ff5423be6-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.050 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.051 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.051 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.051 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:11.051 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:20:11.051 DEBUG:boto:Method: GET 2024-04-16T18:20:11.051 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:20:11.051 DEBUG:boto:Data: 2024-04-16T18:20:11.051 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.052 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.052 DEBUG:boto:Params: {} 2024-04-16T18:20:11.052 DEBUG:boto:Token: None 2024-04-16T18:20:11.052 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c24303adf2443066-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.056 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.057 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:20:11.057 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.057 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.057 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.057 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.057 DEBUG:boto:Method: GET 2024-04-16T18:20:11.057 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.057 DEBUG:boto:Data: 2024-04-16T18:20:11.057 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.057 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.058 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.058 DEBUG:boto:Params: {} 2024-04-16T18:20:11.058 DEBUG:boto:Token: None 2024-04-16T18:20:11.058 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.058 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo= 2024-04-16T18:20:11.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo=', 'Content-Length': '0'} 2024-04-16T18:20:11.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092029a6a89c78398-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.102 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.103 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.103 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.103 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.103 DEBUG:boto:Method: GET 2024-04-16T18:20:11.103 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.103 DEBUG:boto:Data: 2024-04-16T18:20:11.103 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.103 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.103 DEBUG:boto:Params: {} 2024-04-16T18:20:11.103 DEBUG:boto:Token: None 2024-04-16T18:20:11.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo= 2024-04-16T18:20:11.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo=', 'Content-Length': '0'} 2024-04-16T18:20:11.106 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000908aee84e0396140-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.107 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.107 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.107 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.107 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.107 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.107 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.107 DEBUG:boto:Data: 2024-04-16T18:20:11.108 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.108 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.108 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.108 DEBUG:boto:Params: {} 2024-04-16T18:20:11.108 DEBUG:boto:Token: None 2024-04-16T18:20:11.108 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PDw9Nfl//NkQWRdFYTyeSmD0LKk= 2024-04-16T18:20:11.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PDw9Nfl//NkQWRdFYTyeSmD0LKk=', 'Content-Length': '0'} 2024-04-16T18:20:11.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075fe1ae1a336eaf4-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.111 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.112 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.112 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.112 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.112 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.112 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.112 DEBUG:boto:Data: 2024-04-16T18:20:11.112 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.112 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.112 DEBUG:boto:Params: {} 2024-04-16T18:20:11.112 DEBUG:boto:Token: None 2024-04-16T18:20:11.113 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PDw9Nfl//NkQWRdFYTyeSmD0LKk= 2024-04-16T18:20:11.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PDw9Nfl//NkQWRdFYTyeSmD0LKk=', 'Content-Length': '0'} 2024-04-16T18:20:11.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000496dd02ceb4685ec-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.116 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:20:11.116 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.116 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.116 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.116 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.116 DEBUG:boto:Method: GET 2024-04-16T18:20:11.117 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.117 DEBUG:boto:Data: 2024-04-16T18:20:11.117 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.117 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.117 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.117 DEBUG:boto:Params: {} 2024-04-16T18:20:11.117 DEBUG:boto:Token: None 2024-04-16T18:20:11.117 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.117 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo= 2024-04-16T18:20:11.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo=', 'Content-Length': '0'} 2024-04-16T18:20:11.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006273d8c53d6b252c-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.162 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.162 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.163 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.163 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.163 DEBUG:boto:Method: GET 2024-04-16T18:20:11.163 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.163 DEBUG:boto:Data: 2024-04-16T18:20:11.163 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.163 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.163 DEBUG:boto:Params: {} 2024-04-16T18:20:11.163 DEBUG:boto:Token: None 2024-04-16T18:20:11.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:20:11.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo= 2024-04-16T18:20:11.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pH7uUtS8az/yfJ3kTc1DRNY0lYo=', 'Content-Length': '0'} 2024-04-16T18:20:11.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d94eb3d816caf3d3-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.167 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.167 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.167 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.167 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.167 DEBUG:boto:Method: GET 2024-04-16T18:20:11.167 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:20:11.167 DEBUG:boto:Data: 2024-04-16T18:20:11.167 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.168 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.168 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.168 DEBUG:boto:Params: {} 2024-04-16T18:20:11.168 DEBUG:boto:Token: None 2024-04-16T18:20:11.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.172 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc64039262c07266-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.172 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.172 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.173 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.173 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.173 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.173 DEBUG:boto:Method: GET 2024-04-16T18:20:11.173 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:20:11.173 DEBUG:boto:Data: 2024-04-16T18:20:11.173 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.173 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.173 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.173 DEBUG:boto:Params: {} 2024-04-16T18:20:11.174 DEBUG:boto:Token: None 2024-04-16T18:20:11.174 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.174 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.178 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046a373158f2493c4-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.178 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.178 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.178 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.179 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.179 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:20:11.179 DEBUG:boto:Data: 2024-04-16T18:20:11.179 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.179 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.179 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.179 DEBUG:boto:Params: {} 2024-04-16T18:20:11.179 DEBUG:boto:Token: None 2024-04-16T18:20:11.179 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22 2024-04-16T18:20:11.179 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NKcbzv+CXn9qZnPFjggdcY/bPys= 2024-04-16T18:20:11.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NKcbzv+CXn9qZnPFjggdcY/bPys=', 'Content-Length': '0'} 2024-04-16T18:20:11.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b67aece7a6e9e23-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.182 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.182 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.183 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.183 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:20:11.183 DEBUG:boto:Data: 2024-04-16T18:20:11.183 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.183 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.183 DEBUG:boto:Params: {} 2024-04-16T18:20:11.183 DEBUG:boto:Token: None 2024-04-16T18:20:11.183 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22 2024-04-16T18:20:11.183 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NKcbzv+CXn9qZnPFjggdcY/bPys= 2024-04-16T18:20:11.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NKcbzv+CXn9qZnPFjggdcY/bPys=', 'Content-Length': '0'} 2024-04-16T18:20:11.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbc78a56a37bf413-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.187 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:20:11.187 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.187 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.187 DEBUG:boto:Method: GET 2024-04-16T18:20:11.187 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:20:11.187 DEBUG:boto:Data: 2024-04-16T18:20:11.187 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.187 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.187 DEBUG:boto:Params: {} 2024-04-16T18:20:11.187 DEBUG:boto:Token: None 2024-04-16T18:20:11.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22 2024-04-16T18:20:11.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXWL0g/cPYsXXqVgH7nKaC/cIwo= 2024-04-16T18:20:11.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXWL0g/cPYsXXqVgH7nKaC/cIwo=', 'Content-Length': '0'} 2024-04-16T18:20:11.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e54cb9448aff58a3-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.234 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:20:11.234 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:20:11.234 DEBUG:boto:Method: GET 2024-04-16T18:20:11.235 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:20:11.235 DEBUG:boto:Data: 2024-04-16T18:20:11.235 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.235 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.235 DEBUG:boto:Params: {} 2024-04-16T18:20:11.235 DEBUG:boto:Token: None 2024-04-16T18:20:11.235 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj22 2024-04-16T18:20:11.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CXWL0g/cPYsXXqVgH7nKaC/cIwo= 2024-04-16T18:20:11.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXWL0g/cPYsXXqVgH7nKaC/cIwo=', 'Content-Length': '0'} 2024-04-16T18:20:11.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003973c36a5876ba83-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.239 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.239 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.239 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.239 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.239 DEBUG:boto:Method: GET 2024-04-16T18:20:11.239 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:20:11.239 DEBUG:boto:Data: 2024-04-16T18:20:11.239 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.239 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.239 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.239 DEBUG:boto:Params: {} 2024-04-16T18:20:11.240 DEBUG:boto:Token: None 2024-04-16T18:20:11.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ae9818d04cc6013-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.286 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.287 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.287 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:20:11.287 DEBUG:boto:Method: GET 2024-04-16T18:20:11.287 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:20:11.287 DEBUG:boto:Data: 2024-04-16T18:20:11.287 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.287 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.288 DEBUG:boto:Params: {} 2024-04-16T18:20:11.288 DEBUG:boto:Token: None 2024-04-16T18:20:11.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb3c0bf1597bdc25-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.292 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:20:11.292 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.292 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.293 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.293 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.293 DEBUG:boto:Method: GET 2024-04-16T18:20:11.293 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.293 DEBUG:boto:Data: 2024-04-16T18:20:11.293 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.293 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.293 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.293 DEBUG:boto:Params: {} 2024-04-16T18:20:11.293 DEBUG:boto:Token: None 2024-04-16T18:20:11.293 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc= 2024-04-16T18:20:11.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc=', 'Content-Length': '0'} 2024-04-16T18:20:11.296 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb0b1465dca7783e-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.338 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.339 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.339 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.339 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.339 DEBUG:boto:Method: GET 2024-04-16T18:20:11.339 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.339 DEBUG:boto:Data: 2024-04-16T18:20:11.339 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.339 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.339 DEBUG:boto:Params: {} 2024-04-16T18:20:11.340 DEBUG:boto:Token: None 2024-04-16T18:20:11.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc= 2024-04-16T18:20:11.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc=', 'Content-Length': '0'} 2024-04-16T18:20:11.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1180b257eab32c6-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.343 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.343 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.343 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.343 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.343 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.344 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.344 DEBUG:boto:Data: 2024-04-16T18:20:11.344 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.344 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.344 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.344 DEBUG:boto:Params: {} 2024-04-16T18:20:11.344 DEBUG:boto:Token: None 2024-04-16T18:20:11.344 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZHA3Qm7mdc7pf0keQIlFHpiy0no= 2024-04-16T18:20:11.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZHA3Qm7mdc7pf0keQIlFHpiy0no=', 'Content-Length': '0'} 2024-04-16T18:20:11.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ffe6b59d07a6812-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.347 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.348 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.348 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.348 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.348 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.348 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.348 DEBUG:boto:Data: 2024-04-16T18:20:11.348 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.348 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.348 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.348 DEBUG:boto:Params: {} 2024-04-16T18:20:11.348 DEBUG:boto:Token: None 2024-04-16T18:20:11.349 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZHA3Qm7mdc7pf0keQIlFHpiy0no= 2024-04-16T18:20:11.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZHA3Qm7mdc7pf0keQIlFHpiy0no=', 'Content-Length': '0'} 2024-04-16T18:20:11.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbc121f29a36fd9d-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.352 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:20:11.352 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.352 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.353 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.353 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.353 DEBUG:boto:Method: GET 2024-04-16T18:20:11.353 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.353 DEBUG:boto:Data: 2024-04-16T18:20:11.353 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.353 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.353 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.353 DEBUG:boto:Params: {} 2024-04-16T18:20:11.353 DEBUG:boto:Token: None 2024-04-16T18:20:11.354 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.354 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc= 2024-04-16T18:20:11.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc=', 'Content-Length': '0'} 2024-04-16T18:20:11.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067df30683c4ae8d9-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.398 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.398 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.398 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.399 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.399 DEBUG:boto:Method: GET 2024-04-16T18:20:11.399 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.399 DEBUG:boto:Data: 2024-04-16T18:20:11.399 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.399 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.399 DEBUG:boto:Params: {} 2024-04-16T18:20:11.399 DEBUG:boto:Token: None 2024-04-16T18:20:11.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:20:11.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc= 2024-04-16T18:20:11.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZLZnWuqFNvceLU7FTiQLIYzadc=', 'Content-Length': '0'} 2024-04-16T18:20:11.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4d57d00e4b019e8-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.403 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.403 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.403 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.403 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.403 DEBUG:boto:Method: GET 2024-04-16T18:20:11.403 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:20:11.403 DEBUG:boto:Data: 2024-04-16T18:20:11.404 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.404 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.404 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.404 DEBUG:boto:Params: {} 2024-04-16T18:20:11.404 DEBUG:boto:Token: None 2024-04-16T18:20:11.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.408 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e914c5da0143803b-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.408 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.408 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.409 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.409 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.409 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.409 DEBUG:boto:Method: GET 2024-04-16T18:20:11.409 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:20:11.409 DEBUG:boto:Data: 2024-04-16T18:20:11.409 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.409 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.409 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.409 DEBUG:boto:Params: {} 2024-04-16T18:20:11.410 DEBUG:boto:Token: None 2024-04-16T18:20:11.410 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a871962cd0b0e854-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.454 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.455 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.455 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.455 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.455 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:20:11.455 DEBUG:boto:Data: 2024-04-16T18:20:11.455 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.455 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.455 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.456 DEBUG:boto:Params: {} 2024-04-16T18:20:11.456 DEBUG:boto:Token: None 2024-04-16T18:20:11.456 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33 2024-04-16T18:20:11.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NCtd7eMN6kiOjVum7Jzyt2WaHlQ= 2024-04-16T18:20:11.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NCtd7eMN6kiOjVum7Jzyt2WaHlQ=', 'Content-Length': '0'} 2024-04-16T18:20:11.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004424bb117e59540d-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.459 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.459 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.460 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.460 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:20:11.460 DEBUG:boto:Data: 2024-04-16T18:20:11.460 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.460 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.460 DEBUG:boto:Params: {} 2024-04-16T18:20:11.460 DEBUG:boto:Token: None 2024-04-16T18:20:11.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33 2024-04-16T18:20:11.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NCtd7eMN6kiOjVum7Jzyt2WaHlQ= 2024-04-16T18:20:11.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NCtd7eMN6kiOjVum7Jzyt2WaHlQ=', 'Content-Length': '0'} 2024-04-16T18:20:11.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a48dd582a5443533-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.464 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:20:11.464 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.464 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.464 DEBUG:boto:Method: GET 2024-04-16T18:20:11.464 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:20:11.464 DEBUG:boto:Data: 2024-04-16T18:20:11.464 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.464 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.464 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.464 DEBUG:boto:Params: {} 2024-04-16T18:20:11.465 DEBUG:boto:Token: None 2024-04-16T18:20:11.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33 2024-04-16T18:20:11.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B9dnLZ/qkrJMAp4uAedLBj+F7CQ= 2024-04-16T18:20:11.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B9dnLZ/qkrJMAp4uAedLBj+F7CQ=', 'Content-Length': '0'} 2024-04-16T18:20:11.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088dd80b6e8e731d4-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.510 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:20:11.510 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:20:11.511 DEBUG:boto:Method: GET 2024-04-16T18:20:11.511 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:20:11.511 DEBUG:boto:Data: 2024-04-16T18:20:11.511 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.511 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.511 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.511 DEBUG:boto:Params: {} 2024-04-16T18:20:11.511 DEBUG:boto:Token: None 2024-04-16T18:20:11.511 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj33 2024-04-16T18:20:11.511 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B9dnLZ/qkrJMAp4uAedLBj+F7CQ= 2024-04-16T18:20:11.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B9dnLZ/qkrJMAp4uAedLBj+F7CQ=', 'Content-Length': '0'} 2024-04-16T18:20:11.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1406d303481e2e0-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.515 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.515 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.515 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.515 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.515 DEBUG:boto:Method: GET 2024-04-16T18:20:11.516 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:20:11.516 DEBUG:boto:Data: 2024-04-16T18:20:11.516 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.516 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.516 DEBUG:boto:Params: {} 2024-04-16T18:20:11.516 DEBUG:boto:Token: None 2024-04-16T18:20:11.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.520 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d5cca1d783fe4cb-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.562 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.563 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.563 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.563 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.563 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:20:11.563 DEBUG:boto:Method: GET 2024-04-16T18:20:11.563 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:20:11.563 DEBUG:boto:Data: 2024-04-16T18:20:11.563 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.563 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.563 DEBUG:boto:Params: {} 2024-04-16T18:20:11.564 DEBUG:boto:Token: None 2024-04-16T18:20:11.564 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000130f0d86b6465339-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.568 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.568 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:20:11.568 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.569 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.569 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.569 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.569 DEBUG:boto:Method: GET 2024-04-16T18:20:11.569 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.569 DEBUG:boto:Data: 2024-04-16T18:20:11.569 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.569 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.569 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.569 DEBUG:boto:Params: {} 2024-04-16T18:20:11.569 DEBUG:boto:Token: None 2024-04-16T18:20:11.570 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.570 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE= 2024-04-16T18:20:11.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE=', 'Content-Length': '0'} 2024-04-16T18:20:11.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070383890b534069e-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.614 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.614 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.614 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.615 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.615 DEBUG:boto:Method: GET 2024-04-16T18:20:11.615 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.615 DEBUG:boto:Data: 2024-04-16T18:20:11.615 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.615 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.615 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.615 DEBUG:boto:Params: {} 2024-04-16T18:20:11.615 DEBUG:boto:Token: None 2024-04-16T18:20:11.615 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE= 2024-04-16T18:20:11.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE=', 'Content-Length': '0'} 2024-04-16T18:20:11.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a57e71c20ac5df05-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.619 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.619 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.619 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.619 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.619 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.619 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.619 DEBUG:boto:Data: 2024-04-16T18:20:11.620 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.620 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.620 DEBUG:boto:Params: {} 2024-04-16T18:20:11.620 DEBUG:boto:Token: None 2024-04-16T18:20:11.620 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VudyWhHFWdrTSv1wl1VLLUNzMJo= 2024-04-16T18:20:11.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VudyWhHFWdrTSv1wl1VLLUNzMJo=', 'Content-Length': '0'} 2024-04-16T18:20:11.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8bca3f94e728b3e-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.623 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.624 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.624 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.624 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.624 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.624 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.624 DEBUG:boto:Data: 2024-04-16T18:20:11.624 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.624 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.624 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.624 DEBUG:boto:Params: {} 2024-04-16T18:20:11.625 DEBUG:boto:Token: None 2024-04-16T18:20:11.625 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.625 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VudyWhHFWdrTSv1wl1VLLUNzMJo= 2024-04-16T18:20:11.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VudyWhHFWdrTSv1wl1VLLUNzMJo=', 'Content-Length': '0'} 2024-04-16T18:20:11.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048831e068e45525a-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.628 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:20:11.628 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.628 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.628 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.628 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.628 DEBUG:boto:Method: GET 2024-04-16T18:20:11.629 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.629 DEBUG:boto:Data: 2024-04-16T18:20:11.629 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.629 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.629 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.629 DEBUG:boto:Params: {} 2024-04-16T18:20:11.629 DEBUG:boto:Token: None 2024-04-16T18:20:11.629 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.629 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE= 2024-04-16T18:20:11.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE=', 'Content-Length': '0'} 2024-04-16T18:20:11.632 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e2f89780d16c6f2-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.674 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.674 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.674 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.675 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.675 DEBUG:boto:Method: GET 2024-04-16T18:20:11.675 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.675 DEBUG:boto:Data: 2024-04-16T18:20:11.675 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.675 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.675 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.675 DEBUG:boto:Params: {} 2024-04-16T18:20:11.675 DEBUG:boto:Token: None 2024-04-16T18:20:11.675 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:20:11.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE= 2024-04-16T18:20:11.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:unquD6puNJb5wPWfYL4YHiO75BE=', 'Content-Length': '0'} 2024-04-16T18:20:11.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1d31858ec7d5851-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.679 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.679 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.679 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.679 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.679 DEBUG:boto:Method: GET 2024-04-16T18:20:11.679 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:20:11.679 DEBUG:boto:Data: 2024-04-16T18:20:11.679 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.680 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.680 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.680 DEBUG:boto:Params: {} 2024-04-16T18:20:11.680 DEBUG:boto:Token: None 2024-04-16T18:20:11.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.683 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4b362cc076cc01a-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.684 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.684 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.684 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.684 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.684 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.684 DEBUG:boto:Method: GET 2024-04-16T18:20:11.684 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:20:11.684 DEBUG:boto:Data: 2024-04-16T18:20:11.684 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.685 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.685 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.685 DEBUG:boto:Params: {} 2024-04-16T18:20:11.685 DEBUG:boto:Token: None 2024-04-16T18:20:11.685 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b38860f6af15e9c-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.730 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.730 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.731 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.731 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.731 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:20:11.731 DEBUG:boto:Data: 2024-04-16T18:20:11.731 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.731 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.731 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.731 DEBUG:boto:Params: {} 2024-04-16T18:20:11.731 DEBUG:boto:Token: None 2024-04-16T18:20:11.732 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44 2024-04-16T18:20:11.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TNq/B6UI1YmOxmHuPjem/6bksCw= 2024-04-16T18:20:11.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TNq/B6UI1YmOxmHuPjem/6bksCw=', 'Content-Length': '0'} 2024-04-16T18:20:11.735 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2a308f210b73060-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.735 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.735 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.735 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.735 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:20:11.735 DEBUG:boto:Data: 2024-04-16T18:20:11.735 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.735 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.736 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.736 DEBUG:boto:Params: {} 2024-04-16T18:20:11.736 DEBUG:boto:Token: None 2024-04-16T18:20:11.736 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44 2024-04-16T18:20:11.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TNq/B6UI1YmOxmHuPjem/6bksCw= 2024-04-16T18:20:11.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TNq/B6UI1YmOxmHuPjem/6bksCw=', 'Content-Length': '0'} 2024-04-16T18:20:11.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066ecf4708195ebd6-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.739 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:20:11.739 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.739 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.739 DEBUG:boto:Method: GET 2024-04-16T18:20:11.739 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:20:11.740 DEBUG:boto:Data: 2024-04-16T18:20:11.740 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.740 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.740 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.740 DEBUG:boto:Params: {} 2024-04-16T18:20:11.740 DEBUG:boto:Token: None 2024-04-16T18:20:11.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44 2024-04-16T18:20:11.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m6yCpLkofeEYun4mnFAkJarBdyA= 2024-04-16T18:20:11.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m6yCpLkofeEYun4mnFAkJarBdyA=', 'Content-Length': '0'} 2024-04-16T18:20:11.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d146ad83a94c018-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.786 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:20:11.786 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:20:11.787 DEBUG:boto:Method: GET 2024-04-16T18:20:11.787 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:20:11.787 DEBUG:boto:Data: 2024-04-16T18:20:11.787 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.787 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.787 DEBUG:boto:Params: {} 2024-04-16T18:20:11.787 DEBUG:boto:Token: None 2024-04-16T18:20:11.787 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj44 2024-04-16T18:20:11.787 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:m6yCpLkofeEYun4mnFAkJarBdyA= 2024-04-16T18:20:11.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m6yCpLkofeEYun4mnFAkJarBdyA=', 'Content-Length': '0'} 2024-04-16T18:20:11.790 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007884f2a3de99d345-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.791 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.791 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.791 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.791 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.791 DEBUG:boto:Method: GET 2024-04-16T18:20:11.791 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:20:11.791 DEBUG:boto:Data: 2024-04-16T18:20:11.791 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.792 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.792 DEBUG:boto:Params: {} 2024-04-16T18:20:11.792 DEBUG:boto:Token: None 2024-04-16T18:20:11.792 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000147300bbc29ec0f8-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.838 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.839 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.839 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.839 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.839 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:20:11.839 DEBUG:boto:Method: GET 2024-04-16T18:20:11.839 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:20:11.839 DEBUG:boto:Data: 2024-04-16T18:20:11.839 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.839 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.839 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.839 DEBUG:boto:Params: {} 2024-04-16T18:20:11.840 DEBUG:boto:Token: None 2024-04-16T18:20:11.840 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e210050f8b3c1eff-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.844 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.844 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:20:11.844 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:11.845 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:11.845 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.845 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.845 DEBUG:boto:Method: GET 2024-04-16T18:20:11.845 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.845 DEBUG:boto:Data: 2024-04-16T18:20:11.845 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.845 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.845 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.845 DEBUG:boto:Params: {} 2024-04-16T18:20:11.845 DEBUG:boto:Token: None 2024-04-16T18:20:11.845 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.846 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI= 2024-04-16T18:20:11.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI=', 'Content-Length': '0'} 2024-04-16T18:20:11.848 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e5e4c91e8722f2f-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.890 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:11.891 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:11.891 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.891 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.891 DEBUG:boto:Method: GET 2024-04-16T18:20:11.891 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.891 DEBUG:boto:Data: 2024-04-16T18:20:11.891 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.891 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.891 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.892 DEBUG:boto:Params: {} 2024-04-16T18:20:11.892 DEBUG:boto:Token: None 2024-04-16T18:20:11.892 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI= 2024-04-16T18:20:11.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI=', 'Content-Length': '0'} 2024-04-16T18:20:11.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000570e7b2593356ae-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.896 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:11.896 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:11.896 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.896 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.896 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.896 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.896 DEBUG:boto:Data: 2024-04-16T18:20:11.896 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.896 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.896 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.896 DEBUG:boto:Params: {} 2024-04-16T18:20:11.896 DEBUG:boto:Token: None 2024-04-16T18:20:11.897 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.897 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6U8//nDvpshTJjyxjRM3ufbWC/I= 2024-04-16T18:20:11.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6U8//nDvpshTJjyxjRM3ufbWC/I=', 'Content-Length': '0'} 2024-04-16T18:20:11.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000696f4c8cee59fc26-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.900 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:11.900 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:11.900 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.901 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.901 DEBUG:boto:Method: HEAD 2024-04-16T18:20:11.901 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.901 DEBUG:boto:Data: 2024-04-16T18:20:11.901 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.901 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.901 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.901 DEBUG:boto:Params: {} 2024-04-16T18:20:11.901 DEBUG:boto:Token: None 2024-04-16T18:20:11.901 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.902 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6U8//nDvpshTJjyxjRM3ufbWC/I= 2024-04-16T18:20:11.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6U8//nDvpshTJjyxjRM3ufbWC/I=', 'Content-Length': '0'} 2024-04-16T18:20:11.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f2f816852f65088-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.905 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:20:11.905 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:11.905 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:11.905 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.905 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.905 DEBUG:boto:Method: GET 2024-04-16T18:20:11.905 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.905 DEBUG:boto:Data: 2024-04-16T18:20:11.905 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.906 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.906 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.906 DEBUG:boto:Params: {} 2024-04-16T18:20:11.906 DEBUG:boto:Token: None 2024-04-16T18:20:11.906 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.906 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI= 2024-04-16T18:20:11.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI=', 'Content-Length': '0'} 2024-04-16T18:20:11.909 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0406079e3fd133e-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.954 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:11.954 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:11.955 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.955 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.955 DEBUG:boto:Method: GET 2024-04-16T18:20:11.955 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.955 DEBUG:boto:Data: 2024-04-16T18:20:11.955 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.955 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.955 DEBUG:boto:Params: {} 2024-04-16T18:20:11.955 DEBUG:boto:Token: None 2024-04-16T18:20:11.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:20:11.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI= 2024-04-16T18:20:11.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+ycOZkSBdl7ylPG7ePQ/Voy5GQI=', 'Content-Length': '0'} 2024-04-16T18:20:11.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095d097e08c062d9b-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.960 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.960 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.960 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:11.960 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:11.960 DEBUG:boto:Method: GET 2024-04-16T18:20:11.960 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:20:11.960 DEBUG:boto:Data: 2024-04-16T18:20:11.961 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.961 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:11.961 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.961 DEBUG:boto:Params: {} 2024-04-16T18:20:11.961 DEBUG:boto:Token: None 2024-04-16T18:20:11.961 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.965 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012cd94ed22a2a871-00661ec15b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:11.965 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:11.965 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:11.966 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:11.966 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:11.966 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:11.966 DEBUG:boto:Method: GET 2024-04-16T18:20:11.966 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:20:11.966 DEBUG:boto:Data: 2024-04-16T18:20:11.966 DEBUG:boto:Headers: {} 2024-04-16T18:20:11.966 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:11.966 DEBUG:boto:Port: 8000 2024-04-16T18:20:11.966 DEBUG:boto:Params: {} 2024-04-16T18:20:11.966 DEBUG:boto:Token: None 2024-04-16T18:20:11.967 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:11 GMT /lybtug-52/ 2024-04-16T18:20:11.967 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k= 2024-04-16T18:20:11.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z2D3BCn20Dd6y+M7uu6dN0+Bg+k=', 'Content-Length': '0'} 2024-04-16T18:20:11.970 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c00b6254569ebdc1-00661ec15b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.014 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:12.015 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:12.015 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:12.015 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.015 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:20:12.015 DEBUG:boto:Data: 2024-04-16T18:20:12.015 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.015 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:12.015 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.015 DEBUG:boto:Params: {} 2024-04-16T18:20:12.016 DEBUG:boto:Token: None 2024-04-16T18:20:12.016 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj55 2024-04-16T18:20:12.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5INpljxXTjIt8qB3ZLAIpn4pLS8= 2024-04-16T18:20:12.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5INpljxXTjIt8qB3ZLAIpn4pLS8=', 'Content-Length': '0'} 2024-04-16T18:20:12.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000429965953f53e6fa-00661ec15c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.019 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:12.019 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:12.019 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.019 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:20:12.020 DEBUG:boto:Data: 2024-04-16T18:20:12.020 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.020 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.020 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.020 DEBUG:boto:Params: {} 2024-04-16T18:20:12.020 DEBUG:boto:Token: None 2024-04-16T18:20:12.020 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj55 2024-04-16T18:20:12.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5INpljxXTjIt8qB3ZLAIpn4pLS8= 2024-04-16T18:20:12.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5INpljxXTjIt8qB3ZLAIpn4pLS8=', 'Content-Length': '0'} 2024-04-16T18:20:12.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061185a7c075947ae-00661ec15c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:20:12.023 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:12.024 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:12.024 DEBUG:boto:Method: GET 2024-04-16T18:20:12.024 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:20:12.024 DEBUG:boto:Data: 2024-04-16T18:20:12.024 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.024 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:12.024 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.024 DEBUG:boto:Params: {} 2024-04-16T18:20:12.024 DEBUG:boto:Token: None 2024-04-16T18:20:12.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj55 2024-04-16T18:20:12.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y//jIWI60NT6GHvnhRxUqDqdlvg= 2024-04-16T18:20:12.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y//jIWI60NT6GHvnhRxUqDqdlvg=', 'Content-Length': '0'} 2024-04-16T18:20:12.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc4d5b7e4431a087-00661ec15c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.070 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:20:12.070 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:20:12.071 DEBUG:boto:Method: GET 2024-04-16T18:20:12.071 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:20:12.071 DEBUG:boto:Data: 2024-04-16T18:20:12.071 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.071 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.071 DEBUG:boto:Params: {} 2024-04-16T18:20:12.071 DEBUG:boto:Token: None 2024-04-16T18:20:12.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj55 2024-04-16T18:20:12.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y//jIWI60NT6GHvnhRxUqDqdlvg= 2024-04-16T18:20:12.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y//jIWI60NT6GHvnhRxUqDqdlvg=', 'Content-Length': '0'} 2024-04-16T18:20:12.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1dde430f5ebcccc-00661ec15c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.075 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.075 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.075 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:12.075 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:12.075 DEBUG:boto:Method: GET 2024-04-16T18:20:12.075 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:20:12.075 DEBUG:boto:Data: 2024-04-16T18:20:12.075 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:20:12.076 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.076 DEBUG:boto:Params: {} 2024-04-16T18:20:12.076 DEBUG:boto:Token: None 2024-04-16T18:20:12.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CPxfmyGbaEmdBYpyaeMIfr3dPCA= 2024-04-16T18:20:12.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CPxfmyGbaEmdBYpyaeMIfr3dPCA=', 'Content-Length': '0'} 2024-04-16T18:20:12.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc5464b79ff2138e-00661ec15c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.122 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:12.123 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.123 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.123 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:12.123 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:20:12.123 DEBUG:boto:Method: GET 2024-04-16T18:20:12.123 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:20:12.123 DEBUG:boto:Data: 2024-04-16T18:20:12.123 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.124 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.124 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.124 DEBUG:boto:Params: {} 2024-04-16T18:20:12.124 DEBUG:boto:Token: None 2024-04-16T18:20:12.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CPxfmyGbaEmdBYpyaeMIfr3dPCA= 2024-04-16T18:20:12.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CPxfmyGbaEmdBYpyaeMIfr3dPCA=', 'Content-Length': '0'} 2024-04-16T18:20:12.127 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f679a48f7e62ceef-00661ec15c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.128 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:20:12.128 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:20:12.128 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '17', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:20:12.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '17', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:20:12.129 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --num-shards 17 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:20:12.175 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:20:12.175 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:20:12.271 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:20:12.271 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-52 2024-04-16T18:20:12.281 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 5 2024-04-16T18:20:12.375 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.376 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.376 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.376 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:12.376 DEBUG:boto:Data: 2024-04-16T18:20:12.376 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.376 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.376 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.376 DEBUG:boto:Params: {} 2024-04-16T18:20:12.376 DEBUG:boto:Token: None 2024-04-16T18:20:12.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ= 2024-04-16T18:20:12.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ=', 'Content-Length': '0'} 2024-04-16T18:20:12.381 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002cafa9bdcea3fc0e-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.381 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:20:12.381 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:20:12.381 DEBUG:boto:Method: PUT 2024-04-16T18:20:12.381 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:20:12.381 DEBUG:boto:Data: 2024-04-16T18:20:12.381 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:12.381 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.382 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.382 DEBUG:boto:Params: {} 2024-04-16T18:20:12.382 DEBUG:boto:Token: None 2024-04-16T18:20:12.382 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj17 2024-04-16T18:20:12.382 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K+VflhGElxrXs8ULs300ey8LbHY= 2024-04-16T18:20:12.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K+VflhGElxrXs8ULs300ey8LbHY='} 2024-04-16T18:20:12.393 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001f41b503289f7f39-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.394 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.394 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.394 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.394 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:12.394 DEBUG:boto:Data: 2024-04-16T18:20:12.394 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.394 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.394 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.394 DEBUG:boto:Params: {} 2024-04-16T18:20:12.394 DEBUG:boto:Token: None 2024-04-16T18:20:12.394 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.395 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ= 2024-04-16T18:20:12.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ=', 'Content-Length': '0'} 2024-04-16T18:20:12.398 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000035437474ea1842c0-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.398 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:20:12.398 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:20:12.398 DEBUG:boto:Method: PUT 2024-04-16T18:20:12.398 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:20:12.398 DEBUG:boto:Data: 2024-04-16T18:20:12.398 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:12.398 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.398 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.399 DEBUG:boto:Params: {} 2024-04-16T18:20:12.399 DEBUG:boto:Token: None 2024-04-16T18:20:12.399 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj34 2024-04-16T18:20:12.399 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7jDeJYjf5zn/GAZ9Fr0Vfs0RwgU= 2024-04-16T18:20:12.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7jDeJYjf5zn/GAZ9Fr0Vfs0RwgU='} 2024-04-16T18:20:12.442 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc34e851ba309011-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.443 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.443 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.443 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.443 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:12.443 DEBUG:boto:Data: 2024-04-16T18:20:12.443 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.443 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.443 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.444 DEBUG:boto:Params: {} 2024-04-16T18:20:12.444 DEBUG:boto:Token: None 2024-04-16T18:20:12.444 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ= 2024-04-16T18:20:12.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ=', 'Content-Length': '0'} 2024-04-16T18:20:12.448 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000684b8042c876cdf1-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.448 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:20:12.448 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:20:12.449 DEBUG:boto:Method: PUT 2024-04-16T18:20:12.449 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:20:12.449 DEBUG:boto:Data: 2024-04-16T18:20:12.449 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:12.449 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.449 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.449 DEBUG:boto:Params: {} 2024-04-16T18:20:12.449 DEBUG:boto:Token: None 2024-04-16T18:20:12.449 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj51 2024-04-16T18:20:12.450 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3o18JlxIXpFOZUJiHLwDxg59R10= 2024-04-16T18:20:12.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3o18JlxIXpFOZUJiHLwDxg59R10='} 2024-04-16T18:20:12.461 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000381c1e4c1ba262d1-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.461 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.461 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.462 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.462 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:12.462 DEBUG:boto:Data: 2024-04-16T18:20:12.462 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.462 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.462 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.462 DEBUG:boto:Params: {} 2024-04-16T18:20:12.462 DEBUG:boto:Token: None 2024-04-16T18:20:12.462 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.463 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ= 2024-04-16T18:20:12.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ=', 'Content-Length': '0'} 2024-04-16T18:20:12.466 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003e25d0bf50c118e6-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.466 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:20:12.466 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:20:12.466 DEBUG:boto:Method: PUT 2024-04-16T18:20:12.467 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:20:12.467 DEBUG:boto:Data: 2024-04-16T18:20:12.467 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:12.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.467 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.467 DEBUG:boto:Params: {} 2024-04-16T18:20:12.467 DEBUG:boto:Token: None 2024-04-16T18:20:12.467 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj68 2024-04-16T18:20:12.467 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AEkhdySOENhKAQ1yc0kw+51o7Jk= 2024-04-16T18:20:12.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AEkhdySOENhKAQ1yc0kw+51o7Jk='} 2024-04-16T18:20:12.510 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005ca2e2c5a12244da-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.511 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:20:12.511 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:20:12.511 DEBUG:boto:Method: HEAD 2024-04-16T18:20:12.511 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:20:12.511 DEBUG:boto:Data: 2024-04-16T18:20:12.511 DEBUG:boto:Headers: {} 2024-04-16T18:20:12.511 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.511 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.511 DEBUG:boto:Params: {} 2024-04-16T18:20:12.511 DEBUG:boto:Token: None 2024-04-16T18:20:12.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/ 2024-04-16T18:20:12.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ= 2024-04-16T18:20:12.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dqxh45+04G27Qm96rPJBpo5c3WQ=', 'Content-Length': '0'} 2024-04-16T18:20:12.516 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '27'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000099d335060b31cf0f-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.517 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:20:12.517 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:20:12.517 DEBUG:boto:Method: PUT 2024-04-16T18:20:12.517 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:20:12.517 DEBUG:boto:Data: 2024-04-16T18:20:12.517 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:20:12.517 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:20:12.517 DEBUG:boto:Port: 8000 2024-04-16T18:20:12.517 DEBUG:boto:Params: {} 2024-04-16T18:20:12.517 DEBUG:boto:Token: None 2024-04-16T18:20:12.518 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:20:12 GMT /lybtug-52/obj85 2024-04-16T18:20:12.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vwVAR3cooYh9s+mfEk7UXo3/GR4= 2024-04-16T18:20:12.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:20:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vwVAR3cooYh9s+mfEk7UXo3/GR4='} 2024-04-16T18:20:12.530 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e21820113365b1f6-00661ec15c-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:20:12.530 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:20:12.530 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:20:12.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:20:12.531 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:20:12.580 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:20:12.581 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:20:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:12.664+0000 7f670f83d980 1 waiting to reach latest gen 1, on 0.. 2024-04-16T18:20:17.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:17.668+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:17.667 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, , , , ] 2024-04-16T18:20:17.667 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:22.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:22.672+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:22.670 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, , , , ] 2024-04-16T18:20:22.670 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:27.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:27.675+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:27.673 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, , , , ] 2024-04-16T18:20:27.673 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:32.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:32.675+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:32.675 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, , , , ] 2024-04-16T18:20:32.675 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:37.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:37.679+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:37.679 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, , , , ] 2024-04-16T18:20:37.679 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:42.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:42.683+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:42.681 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:42.681 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:47.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:47.687+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:47.685 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:47.685 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:52.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:52.687+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:52.687 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:52.687 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:57.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:20:57.691+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:20:57.690 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:20:57.690 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:02.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:02.694+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:02.693 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:02.693 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:07.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:07.698+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:07.696 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:07.696 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:12.698+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:12.700 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:12.700 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:17.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:17.702+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:17.702 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:17.702 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:22.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:22.706+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:22.704 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:22.705 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:27.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:27.706+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:27.707 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:27.707 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:32.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:32.710+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:32.709 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:32.709 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:37.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:37.714+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:37.712 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:37.713 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:42.713+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:42.715 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:42.715 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:47.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:47.717+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:47.718 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:47.718 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:52.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:52.721+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:52.720 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:52.720 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:21:57.721+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:21:57.723 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:21:57.723 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:02.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:02.725+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:02.725 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:02.726 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:07.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:07.729+0000 7f670f83d980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:07.728 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , , , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:07.728 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:12.733+0000 7f670f83d980 1 bucket sync caught up with source: 2024-04-16T18:22:12.731 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:12.731 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000001.105.6, , , , , , 00000000001.53.6, , 00000000001.104.6, , , , 00000000001.145.6, 00000000001.84.6, , , ] 2024-04-16T18:22:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:12.733+0000 7f670f83d980 0 bucket checkpoint complete 2024-04-16T18:22:12.776 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:22:12.776 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:22:12.776 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:22:12.777 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:22:12.828 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:22:12.828 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:22:12.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:22:12.909+0000 7fdedba3f980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:12.906 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.628.6, 00000000001.553.6, , , 00000000001.513.6, , , 00000000001.458.6, 00000000003.319.6, 00000000002.404.6] 2024-04-16T18:22:12.906 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.628.6, 00000000001.553.6, , , 00000000001.513.6, , , 00000000001.458.6, 00000000003.319.6, 00000000003.410.6] 2024-04-16T18:22:17.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:22:17.913+0000 7fdedba3f980 1 bucket sync caught up with source: 2024-04-16T18:22:17.909 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000001.628.6, 00000000001.553.6, , , 00000000001.513.6, , , 00000000001.458.6, 00000000003.319.6, 00000000003.410.6] 2024-04-16T18:22:17.910 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000001.628.6, 00000000001.553.6, , , 00000000001.513.6, , , 00000000001.458.6, 00000000003.319.6, 00000000003.410.6] 2024-04-16T18:22:17.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:22:17.913+0000 7fdedba3f980 0 bucket checkpoint complete 2024-04-16T18:22:17.956 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:22:17.957 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:17.957 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:17.957 DEBUG:boto:Method: HEAD 2024-04-16T18:22:17.957 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:17.957 DEBUG:boto:Data: 2024-04-16T18:22:17.957 DEBUG:boto:Headers: {} 2024-04-16T18:22:17.957 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:17.958 DEBUG:boto:Port: 8000 2024-04-16T18:22:17.958 DEBUG:boto:Params: {} 2024-04-16T18:22:17.958 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:22:17.959 DEBUG:boto:Token: None 2024-04-16T18:22:17.959 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:17 GMT /lybtug-52/ 2024-04-16T18:22:17.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dHV5h19pAnBCZ4bj9RW0AhxfPeM= 2024-04-16T18:22:17.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dHV5h19pAnBCZ4bj9RW0AhxfPeM=', 'Content-Length': '0'} 2024-04-16T18:22:17.967 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006b96c4d74050a1bf-00661ec1d9-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:17.968 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:17.968 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:17.968 DEBUG:boto:Method: HEAD 2024-04-16T18:22:17.968 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:17.968 DEBUG:boto:Data: 2024-04-16T18:22:17.968 DEBUG:boto:Headers: {} 2024-04-16T18:22:17.968 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:17.968 DEBUG:boto:Port: 8000 2024-04-16T18:22:17.969 DEBUG:boto:Params: {} 2024-04-16T18:22:17.969 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:22:17.969 DEBUG:boto:Token: None 2024-04-16T18:22:17.969 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:17 GMT /lybtug-52/ 2024-04-16T18:22:17.969 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dHV5h19pAnBCZ4bj9RW0AhxfPeM= 2024-04-16T18:22:17.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dHV5h19pAnBCZ4bj9RW0AhxfPeM=', 'Content-Length': '0'} 2024-04-16T18:22:17.975 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a2a6e6c86c59c1d6-00661ec1d9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:17.976 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:22:17.976 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:17.976 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:17.976 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.976 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.976 DEBUG:boto:Method: GET 2024-04-16T18:22:17.977 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.977 DEBUG:boto:Data: 2024-04-16T18:22:17.977 DEBUG:boto:Headers: {} 2024-04-16T18:22:17.977 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:17.977 DEBUG:boto:Port: 8000 2024-04-16T18:22:17.977 DEBUG:boto:Params: {} 2024-04-16T18:22:17.977 DEBUG:boto:Token: None 2024-04-16T18:22:17.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:17 GMT /lybtug-52/?versions 2024-04-16T18:22:17.978 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/1dcOqDNtlAhIrOsGUXQfvVOKaI= 2024-04-16T18:22:17.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/1dcOqDNtlAhIrOsGUXQfvVOKaI=', 'Content-Length': '0'} 2024-04-16T18:22:17.982 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc68291b0328255f-00661ec1d9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:17.982 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:17.983 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:22:17.983 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:22:17.984 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:22:17.985 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:17.985 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:17.985 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.985 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.985 DEBUG:boto:Method: GET 2024-04-16T18:22:17.985 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.985 DEBUG:boto:Data: 2024-04-16T18:22:17.985 DEBUG:boto:Headers: {} 2024-04-16T18:22:17.985 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:17.986 DEBUG:boto:Port: 8000 2024-04-16T18:22:17.986 DEBUG:boto:Params: {} 2024-04-16T18:22:17.986 DEBUG:boto:Token: None 2024-04-16T18:22:17.986 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:17 GMT /lybtug-52/?versions 2024-04-16T18:22:17.986 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/1dcOqDNtlAhIrOsGUXQfvVOKaI= 2024-04-16T18:22:17.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/1dcOqDNtlAhIrOsGUXQfvVOKaI=', 'Content-Length': '0'} 2024-04-16T18:22:17.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082528953e823bdda-00661ec1d9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:17.991 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:17.991 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:22:17.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:22:17.993 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:17.993 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:17.993 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.993 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.993 DEBUG:boto:Method: GET 2024-04-16T18:22:17.993 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.993 DEBUG:boto:Data: 2024-04-16T18:22:17.993 DEBUG:boto:Headers: {} 2024-04-16T18:22:17.993 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:17.993 DEBUG:boto:Port: 8000 2024-04-16T18:22:17.994 DEBUG:boto:Params: {} 2024-04-16T18:22:17.994 DEBUG:boto:Token: None 2024-04-16T18:22:17.994 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:17 GMT /lybtug-52/?versions 2024-04-16T18:22:17.994 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/1dcOqDNtlAhIrOsGUXQfvVOKaI= 2024-04-16T18:22:17.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/1dcOqDNtlAhIrOsGUXQfvVOKaI=', 'Content-Length': '0'} 2024-04-16T18:22:17.998 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a4942b0964efbb9-00661ec1d9-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:17.998 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:17.999 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:17.999 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:17.999 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:17.999 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:22:18.000 DEBUG:boto:Method: GET 2024-04-16T18:22:18.000 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:22:18.000 DEBUG:boto:Data: 2024-04-16T18:22:18.000 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.000 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.000 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.000 DEBUG:boto:Params: {} 2024-04-16T18:22:18.000 DEBUG:boto:Token: None 2024-04-16T18:22:18.001 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/?versions 2024-04-16T18:22:18.001 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hwc9uzWedvvHujWhcVc+3NA85Uo= 2024-04-16T18:22:18.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hwc9uzWedvvHujWhcVc+3NA85Uo=', 'Content-Length': '0'} 2024-04-16T18:22:18.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6c288f040511d9d-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.006 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:22:18.007 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.007 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.007 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.007 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.007 DEBUG:boto:Method: GET 2024-04-16T18:22:18.007 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.007 DEBUG:boto:Data: 2024-04-16T18:22:18.007 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.008 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.008 DEBUG:boto:Params: {} 2024-04-16T18:22:18.008 DEBUG:boto:Token: None 2024-04-16T18:22:18.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0= 2024-04-16T18:22:18.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0=', 'Content-Length': '0'} 2024-04-16T18:22:18.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007115ee33693cf1a0-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.012 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.012 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.012 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.012 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.012 DEBUG:boto:Method: GET 2024-04-16T18:22:18.013 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.013 DEBUG:boto:Data: 2024-04-16T18:22:18.013 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.013 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.013 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.013 DEBUG:boto:Params: {} 2024-04-16T18:22:18.013 DEBUG:boto:Token: None 2024-04-16T18:22:18.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0= 2024-04-16T18:22:18.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0=', 'Content-Length': '0'} 2024-04-16T18:22:18.017 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9faad3e6920d1b4-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.058 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.059 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.059 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.059 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.059 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.059 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.059 DEBUG:boto:Data: 2024-04-16T18:22:18.059 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.059 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.060 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.060 DEBUG:boto:Params: {} 2024-04-16T18:22:18.060 DEBUG:boto:Token: None 2024-04-16T18:22:18.060 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iI0azvpQ4nlaUKO7WzHYhD0El84= 2024-04-16T18:22:18.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iI0azvpQ4nlaUKO7WzHYhD0El84=', 'Content-Length': '0'} 2024-04-16T18:22:18.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc2562788435f51a-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.064 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.064 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.064 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.064 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.064 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.064 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.064 DEBUG:boto:Data: 2024-04-16T18:22:18.064 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.065 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.065 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.065 DEBUG:boto:Params: {} 2024-04-16T18:22:18.065 DEBUG:boto:Token: None 2024-04-16T18:22:18.065 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.065 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iI0azvpQ4nlaUKO7WzHYhD0El84= 2024-04-16T18:22:18.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iI0azvpQ4nlaUKO7WzHYhD0El84=', 'Content-Length': '0'} 2024-04-16T18:22:18.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca2965a39efd6b9d-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.069 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:22:18.069 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.069 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.069 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.069 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.069 DEBUG:boto:Method: GET 2024-04-16T18:22:18.069 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.069 DEBUG:boto:Data: 2024-04-16T18:22:18.070 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.070 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.070 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.070 DEBUG:boto:Params: {} 2024-04-16T18:22:18.070 DEBUG:boto:Token: None 2024-04-16T18:22:18.070 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.070 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0= 2024-04-16T18:22:18.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0=', 'Content-Length': '0'} 2024-04-16T18:22:18.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000591e931ae81eb8d0-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.114 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.115 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.115 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.115 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.115 DEBUG:boto:Method: GET 2024-04-16T18:22:18.115 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.115 DEBUG:boto:Data: 2024-04-16T18:22:18.115 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.115 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.115 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.115 DEBUG:boto:Params: {} 2024-04-16T18:22:18.116 DEBUG:boto:Token: None 2024-04-16T18:22:18.116 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:22:18.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0= 2024-04-16T18:22:18.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cjs/btp2EWbyVuel4itdFn1YrC0=', 'Content-Length': '0'} 2024-04-16T18:22:18.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b9eed8cef9d3a6-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.120 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.120 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.120 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.120 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.120 DEBUG:boto:Method: GET 2024-04-16T18:22:18.120 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:22:18.120 DEBUG:boto:Data: 2024-04-16T18:22:18.121 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.121 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.121 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.121 DEBUG:boto:Params: {} 2024-04-16T18:22:18.121 DEBUG:boto:Token: None 2024-04-16T18:22:18.121 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.125 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006702a17b1f665075-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.125 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.125 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.125 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.126 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.126 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.126 DEBUG:boto:Method: GET 2024-04-16T18:22:18.126 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:22:18.126 DEBUG:boto:Data: 2024-04-16T18:22:18.126 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.126 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.126 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.126 DEBUG:boto:Params: {} 2024-04-16T18:22:18.127 DEBUG:boto:Token: None 2024-04-16T18:22:18.127 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.127 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099905943c16a02b7-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.132 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.132 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.132 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.132 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.132 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:22:18.133 DEBUG:boto:Data: 2024-04-16T18:22:18.133 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.133 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.133 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.133 DEBUG:boto:Params: {} 2024-04-16T18:22:18.133 DEBUG:boto:Token: None 2024-04-16T18:22:18.133 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11 2024-04-16T18:22:18.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EYqUOYHn8B2P87RvmaoCZ9+F5xs= 2024-04-16T18:22:18.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EYqUOYHn8B2P87RvmaoCZ9+F5xs=', 'Content-Length': '0'} 2024-04-16T18:22:18.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd409d354523a7df-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.137 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.137 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.137 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.137 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:22:18.137 DEBUG:boto:Data: 2024-04-16T18:22:18.137 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.137 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.138 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.138 DEBUG:boto:Params: {} 2024-04-16T18:22:18.138 DEBUG:boto:Token: None 2024-04-16T18:22:18.138 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11 2024-04-16T18:22:18.138 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EYqUOYHn8B2P87RvmaoCZ9+F5xs= 2024-04-16T18:22:18.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EYqUOYHn8B2P87RvmaoCZ9+F5xs=', 'Content-Length': '0'} 2024-04-16T18:22:18.141 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009585b1d6a2a8906a-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.141 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:22:18.142 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.142 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.142 DEBUG:boto:Method: GET 2024-04-16T18:22:18.142 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:22:18.142 DEBUG:boto:Data: 2024-04-16T18:22:18.142 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.142 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.142 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.142 DEBUG:boto:Params: {} 2024-04-16T18:22:18.143 DEBUG:boto:Token: None 2024-04-16T18:22:18.143 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11 2024-04-16T18:22:18.143 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5Ob7Yfu74NhLLtKi4Bl1y43SIek= 2024-04-16T18:22:18.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5Ob7Yfu74NhLLtKi4Bl1y43SIek=', 'Content-Length': '0'} 2024-04-16T18:22:18.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036ebc71587084858-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.186 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:22:18.187 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:22:18.187 DEBUG:boto:Method: GET 2024-04-16T18:22:18.187 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:22:18.187 DEBUG:boto:Data: 2024-04-16T18:22:18.187 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.187 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.187 DEBUG:boto:Params: {} 2024-04-16T18:22:18.187 DEBUG:boto:Token: None 2024-04-16T18:22:18.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj11 2024-04-16T18:22:18.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5Ob7Yfu74NhLLtKi4Bl1y43SIek= 2024-04-16T18:22:18.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5Ob7Yfu74NhLLtKi4Bl1y43SIek=', 'Content-Length': '0'} 2024-04-16T18:22:18.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ad4fe94ffd99c4e-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.191 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.191 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.191 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.191 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.192 DEBUG:boto:Method: GET 2024-04-16T18:22:18.192 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:22:18.192 DEBUG:boto:Data: 2024-04-16T18:22:18.192 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.192 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.192 DEBUG:boto:Params: {} 2024-04-16T18:22:18.192 DEBUG:boto:Token: None 2024-04-16T18:22:18.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.193 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089afad42b273757e-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.196 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.197 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.197 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.197 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.197 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:22:18.197 DEBUG:boto:Method: GET 2024-04-16T18:22:18.197 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:22:18.198 DEBUG:boto:Data: 2024-04-16T18:22:18.198 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.198 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.198 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.198 DEBUG:boto:Params: {} 2024-04-16T18:22:18.198 DEBUG:boto:Token: None 2024-04-16T18:22:18.198 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.198 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068fedfb2c626b509-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.203 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:22:18.204 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.204 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.204 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.204 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.204 DEBUG:boto:Method: GET 2024-04-16T18:22:18.204 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.204 DEBUG:boto:Data: 2024-04-16T18:22:18.204 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.204 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.205 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.205 DEBUG:boto:Params: {} 2024-04-16T18:22:18.205 DEBUG:boto:Token: None 2024-04-16T18:22:18.205 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.205 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk= 2024-04-16T18:22:18.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk=', 'Content-Length': '0'} 2024-04-16T18:22:18.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003acbc6934585a0be-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.250 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.251 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.251 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.251 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.251 DEBUG:boto:Method: GET 2024-04-16T18:22:18.251 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.251 DEBUG:boto:Data: 2024-04-16T18:22:18.251 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.251 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.251 DEBUG:boto:Params: {} 2024-04-16T18:22:18.252 DEBUG:boto:Token: None 2024-04-16T18:22:18.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk= 2024-04-16T18:22:18.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk=', 'Content-Length': '0'} 2024-04-16T18:22:18.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fdeb5de3571864da-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.255 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.255 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.255 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.256 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.256 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.256 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.256 DEBUG:boto:Data: 2024-04-16T18:22:18.256 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.256 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.256 DEBUG:boto:Params: {} 2024-04-16T18:22:18.256 DEBUG:boto:Token: None 2024-04-16T18:22:18.257 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c08/So0EAlyp6WsgK3qMPXTNnto= 2024-04-16T18:22:18.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c08/So0EAlyp6WsgK3qMPXTNnto=', 'Content-Length': '0'} 2024-04-16T18:22:18.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f64c0678196f30b-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.260 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.260 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.260 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.261 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.261 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.261 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.261 DEBUG:boto:Data: 2024-04-16T18:22:18.261 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.261 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.261 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.261 DEBUG:boto:Params: {} 2024-04-16T18:22:18.261 DEBUG:boto:Token: None 2024-04-16T18:22:18.261 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c08/So0EAlyp6WsgK3qMPXTNnto= 2024-04-16T18:22:18.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c08/So0EAlyp6WsgK3qMPXTNnto=', 'Content-Length': '0'} 2024-04-16T18:22:18.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006168036861d0e037-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.265 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:22:18.265 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.265 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.265 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.265 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.265 DEBUG:boto:Method: GET 2024-04-16T18:22:18.265 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.266 DEBUG:boto:Data: 2024-04-16T18:22:18.266 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.266 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.266 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.266 DEBUG:boto:Params: {} 2024-04-16T18:22:18.266 DEBUG:boto:Token: None 2024-04-16T18:22:18.266 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.266 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk= 2024-04-16T18:22:18.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk=', 'Content-Length': '0'} 2024-04-16T18:22:18.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4c4ad73259f66f9-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.310 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.311 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.311 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.311 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.311 DEBUG:boto:Method: GET 2024-04-16T18:22:18.311 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.311 DEBUG:boto:Data: 2024-04-16T18:22:18.311 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.311 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.311 DEBUG:boto:Params: {} 2024-04-16T18:22:18.312 DEBUG:boto:Token: None 2024-04-16T18:22:18.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:22:18.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk= 2024-04-16T18:22:18.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:p17cyWyFgfbt2n8V5is/8nq5KRk=', 'Content-Length': '0'} 2024-04-16T18:22:18.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e451f850f1c6f6bf-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.316 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.316 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.316 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.316 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.316 DEBUG:boto:Method: GET 2024-04-16T18:22:18.316 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:22:18.316 DEBUG:boto:Data: 2024-04-16T18:22:18.316 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.316 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.317 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.317 DEBUG:boto:Params: {} 2024-04-16T18:22:18.317 DEBUG:boto:Token: None 2024-04-16T18:22:18.317 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.317 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.321 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4bb691504f4d53b-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.321 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.322 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.322 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.322 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.322 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.322 DEBUG:boto:Method: GET 2024-04-16T18:22:18.322 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:22:18.322 DEBUG:boto:Data: 2024-04-16T18:22:18.322 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.323 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.323 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.323 DEBUG:boto:Params: {} 2024-04-16T18:22:18.323 DEBUG:boto:Token: None 2024-04-16T18:22:18.323 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.323 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004415943ab91814d7-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.370 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.371 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.371 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.371 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.371 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:22:18.371 DEBUG:boto:Data: 2024-04-16T18:22:18.371 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.372 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.372 DEBUG:boto:Params: {} 2024-04-16T18:22:18.372 DEBUG:boto:Token: None 2024-04-16T18:22:18.372 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17 2024-04-16T18:22:18.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2ttyud5+oNJkUlbHTdsCqtce9Wg= 2024-04-16T18:22:18.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2ttyud5+oNJkUlbHTdsCqtce9Wg=', 'Content-Length': '0'} 2024-04-16T18:22:18.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbe9bf20ee61f07a-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.375 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.376 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.376 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.376 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:22:18.376 DEBUG:boto:Data: 2024-04-16T18:22:18.376 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.376 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.376 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.376 DEBUG:boto:Params: {} 2024-04-16T18:22:18.376 DEBUG:boto:Token: None 2024-04-16T18:22:18.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17 2024-04-16T18:22:18.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2ttyud5+oNJkUlbHTdsCqtce9Wg= 2024-04-16T18:22:18.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2ttyud5+oNJkUlbHTdsCqtce9Wg=', 'Content-Length': '0'} 2024-04-16T18:22:18.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae82c21b5b22eea4-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:22:18.380 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.380 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.380 DEBUG:boto:Method: GET 2024-04-16T18:22:18.380 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:22:18.381 DEBUG:boto:Data: 2024-04-16T18:22:18.381 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.381 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.381 DEBUG:boto:Params: {} 2024-04-16T18:22:18.381 DEBUG:boto:Token: None 2024-04-16T18:22:18.381 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17 2024-04-16T18:22:18.382 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QT35ykNeeINxlpERHdZaMGC+i8s= 2024-04-16T18:22:18.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QT35ykNeeINxlpERHdZaMGC+i8s=', 'Content-Length': '0'} 2024-04-16T18:22:18.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d1005d1399c52ee-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.426 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:22:18.427 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:22:18.427 DEBUG:boto:Method: GET 2024-04-16T18:22:18.427 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:22:18.427 DEBUG:boto:Data: 2024-04-16T18:22:18.427 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.427 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.427 DEBUG:boto:Params: {} 2024-04-16T18:22:18.428 DEBUG:boto:Token: None 2024-04-16T18:22:18.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj17 2024-04-16T18:22:18.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QT35ykNeeINxlpERHdZaMGC+i8s= 2024-04-16T18:22:18.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QT35ykNeeINxlpERHdZaMGC+i8s=', 'Content-Length': '0'} 2024-04-16T18:22:18.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000eb6865d2784be9f-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.431 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.431 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.431 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.432 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.432 DEBUG:boto:Method: GET 2024-04-16T18:22:18.432 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:22:18.432 DEBUG:boto:Data: 2024-04-16T18:22:18.432 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.432 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.432 DEBUG:boto:Params: {} 2024-04-16T18:22:18.432 DEBUG:boto:Token: None 2024-04-16T18:22:18.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.433 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000320ddc240fa8c50b-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.478 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.479 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.479 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.479 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.479 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:22:18.479 DEBUG:boto:Method: GET 2024-04-16T18:22:18.479 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:22:18.479 DEBUG:boto:Data: 2024-04-16T18:22:18.479 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.480 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.480 DEBUG:boto:Params: {} 2024-04-16T18:22:18.480 DEBUG:boto:Token: None 2024-04-16T18:22:18.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000042d1ca24ab172c2-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.485 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.485 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:22:18.485 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.485 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.485 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.485 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.486 DEBUG:boto:Method: GET 2024-04-16T18:22:18.486 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.486 DEBUG:boto:Data: 2024-04-16T18:22:18.486 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.486 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.486 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.486 DEBUG:boto:Params: {} 2024-04-16T18:22:18.486 DEBUG:boto:Token: None 2024-04-16T18:22:18.486 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.487 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg= 2024-04-16T18:22:18.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg=', 'Content-Length': '0'} 2024-04-16T18:22:18.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a4089861d09fe25-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.530 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.531 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.531 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.531 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.531 DEBUG:boto:Method: GET 2024-04-16T18:22:18.531 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.531 DEBUG:boto:Data: 2024-04-16T18:22:18.531 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.531 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.531 DEBUG:boto:Params: {} 2024-04-16T18:22:18.531 DEBUG:boto:Token: None 2024-04-16T18:22:18.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg= 2024-04-16T18:22:18.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg=', 'Content-Length': '0'} 2024-04-16T18:22:18.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dd04c169525d1d2-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.535 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.535 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.535 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.535 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.536 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.536 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.536 DEBUG:boto:Data: 2024-04-16T18:22:18.536 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.536 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.536 DEBUG:boto:Params: {} 2024-04-16T18:22:18.536 DEBUG:boto:Token: None 2024-04-16T18:22:18.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s1ssmok2HpsCHOsdPYn1U6zpppk= 2024-04-16T18:22:18.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s1ssmok2HpsCHOsdPYn1U6zpppk=', 'Content-Length': '0'} 2024-04-16T18:22:18.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9f0a0ba41d165dc-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.540 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.540 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.540 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.540 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.540 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.540 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.540 DEBUG:boto:Data: 2024-04-16T18:22:18.541 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.541 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.541 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.541 DEBUG:boto:Params: {} 2024-04-16T18:22:18.541 DEBUG:boto:Token: None 2024-04-16T18:22:18.541 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:s1ssmok2HpsCHOsdPYn1U6zpppk= 2024-04-16T18:22:18.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s1ssmok2HpsCHOsdPYn1U6zpppk=', 'Content-Length': '0'} 2024-04-16T18:22:18.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c467b7ffd92f41f-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:22:18.545 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.545 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.545 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.545 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.545 DEBUG:boto:Method: GET 2024-04-16T18:22:18.545 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.545 DEBUG:boto:Data: 2024-04-16T18:22:18.545 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.545 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.546 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.546 DEBUG:boto:Params: {} 2024-04-16T18:22:18.546 DEBUG:boto:Token: None 2024-04-16T18:22:18.546 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.546 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg= 2024-04-16T18:22:18.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg=', 'Content-Length': '0'} 2024-04-16T18:22:18.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000917aef50f602012e-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.590 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.590 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.591 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.591 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.591 DEBUG:boto:Method: GET 2024-04-16T18:22:18.591 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.591 DEBUG:boto:Data: 2024-04-16T18:22:18.591 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.591 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.591 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.591 DEBUG:boto:Params: {} 2024-04-16T18:22:18.591 DEBUG:boto:Token: None 2024-04-16T18:22:18.591 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:22:18.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg= 2024-04-16T18:22:18.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cW7i9EcmiLlgmhcDJx+RDEtkaYg=', 'Content-Length': '0'} 2024-04-16T18:22:18.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d0deb6e25810fe6-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.595 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.595 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.595 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.595 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.595 DEBUG:boto:Method: GET 2024-04-16T18:22:18.595 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:22:18.595 DEBUG:boto:Data: 2024-04-16T18:22:18.595 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.595 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.595 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.596 DEBUG:boto:Params: {} 2024-04-16T18:22:18.596 DEBUG:boto:Token: None 2024-04-16T18:22:18.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000554e64e4c64bb74b-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.600 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.601 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.601 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.601 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.601 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.601 DEBUG:boto:Method: GET 2024-04-16T18:22:18.601 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:22:18.601 DEBUG:boto:Data: 2024-04-16T18:22:18.601 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.601 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.602 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.602 DEBUG:boto:Params: {} 2024-04-16T18:22:18.602 DEBUG:boto:Token: None 2024-04-16T18:22:18.602 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.602 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.606 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1d230acf5ad5c98-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.650 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.651 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.651 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.651 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.651 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:22:18.651 DEBUG:boto:Data: 2024-04-16T18:22:18.651 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.651 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.651 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.651 DEBUG:boto:Params: {} 2024-04-16T18:22:18.652 DEBUG:boto:Token: None 2024-04-16T18:22:18.652 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22 2024-04-16T18:22:18.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0ai0tDz4qUjz5+Se1UXZqxFkvTI= 2024-04-16T18:22:18.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0ai0tDz4qUjz5+Se1UXZqxFkvTI=', 'Content-Length': '0'} 2024-04-16T18:22:18.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bed2c47d039be3c-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.655 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.655 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.655 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.655 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:22:18.655 DEBUG:boto:Data: 2024-04-16T18:22:18.655 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.655 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.655 DEBUG:boto:Params: {} 2024-04-16T18:22:18.656 DEBUG:boto:Token: None 2024-04-16T18:22:18.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22 2024-04-16T18:22:18.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0ai0tDz4qUjz5+Se1UXZqxFkvTI= 2024-04-16T18:22:18.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0ai0tDz4qUjz5+Se1UXZqxFkvTI=', 'Content-Length': '0'} 2024-04-16T18:22:18.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c9c9daf89f81ae1-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.659 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:22:18.659 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.659 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.660 DEBUG:boto:Method: GET 2024-04-16T18:22:18.660 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:22:18.660 DEBUG:boto:Data: 2024-04-16T18:22:18.660 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.660 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.660 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.660 DEBUG:boto:Params: {} 2024-04-16T18:22:18.660 DEBUG:boto:Token: None 2024-04-16T18:22:18.660 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22 2024-04-16T18:22:18.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1WnkrEnbZIaBjCOgMtZecjW3XS8= 2024-04-16T18:22:18.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1WnkrEnbZIaBjCOgMtZecjW3XS8=', 'Content-Length': '0'} 2024-04-16T18:22:18.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e57e53cefb6ae45f-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.710 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:22:18.711 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:22:18.711 DEBUG:boto:Method: GET 2024-04-16T18:22:18.711 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:22:18.711 DEBUG:boto:Data: 2024-04-16T18:22:18.711 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.711 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.711 DEBUG:boto:Params: {} 2024-04-16T18:22:18.711 DEBUG:boto:Token: None 2024-04-16T18:22:18.711 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj22 2024-04-16T18:22:18.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1WnkrEnbZIaBjCOgMtZecjW3XS8= 2024-04-16T18:22:18.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1WnkrEnbZIaBjCOgMtZecjW3XS8=', 'Content-Length': '0'} 2024-04-16T18:22:18.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c570e3c35731fa60-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.715 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.715 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.715 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.715 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.715 DEBUG:boto:Method: GET 2024-04-16T18:22:18.715 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:22:18.715 DEBUG:boto:Data: 2024-04-16T18:22:18.715 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.715 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.716 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.716 DEBUG:boto:Params: {} 2024-04-16T18:22:18.716 DEBUG:boto:Token: None 2024-04-16T18:22:18.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.720 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eaa86e78853b5b2a-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.762 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.763 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.763 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.763 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.763 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:22:18.763 DEBUG:boto:Method: GET 2024-04-16T18:22:18.763 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:22:18.763 DEBUG:boto:Data: 2024-04-16T18:22:18.763 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.763 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.764 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.764 DEBUG:boto:Params: {} 2024-04-16T18:22:18.764 DEBUG:boto:Token: None 2024-04-16T18:22:18.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.769 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044f1b5779dbbe85b-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.769 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.770 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:22:18.770 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.770 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.770 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.770 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.770 DEBUG:boto:Method: GET 2024-04-16T18:22:18.770 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.770 DEBUG:boto:Data: 2024-04-16T18:22:18.770 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.771 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.771 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.771 DEBUG:boto:Params: {} 2024-04-16T18:22:18.771 DEBUG:boto:Token: None 2024-04-16T18:22:18.771 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.771 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ= 2024-04-16T18:22:18.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ=', 'Content-Length': '0'} 2024-04-16T18:22:18.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018859130efe865f7-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.814 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.814 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.815 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.815 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.815 DEBUG:boto:Method: GET 2024-04-16T18:22:18.815 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.815 DEBUG:boto:Data: 2024-04-16T18:22:18.815 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.815 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.815 DEBUG:boto:Params: {} 2024-04-16T18:22:18.815 DEBUG:boto:Token: None 2024-04-16T18:22:18.815 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ= 2024-04-16T18:22:18.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ=', 'Content-Length': '0'} 2024-04-16T18:22:18.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b68d4f51147228da-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.819 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.819 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.819 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.819 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.820 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.820 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.820 DEBUG:boto:Data: 2024-04-16T18:22:18.820 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.820 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.820 DEBUG:boto:Params: {} 2024-04-16T18:22:18.820 DEBUG:boto:Token: None 2024-04-16T18:22:18.820 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fOQd8Jfq93ZNguIu+Pceb1rFmmI= 2024-04-16T18:22:18.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fOQd8Jfq93ZNguIu+Pceb1rFmmI=', 'Content-Length': '0'} 2024-04-16T18:22:18.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060c63be9e9f77980-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.823 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.824 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.824 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.824 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.824 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.824 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.824 DEBUG:boto:Data: 2024-04-16T18:22:18.824 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.824 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.824 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.824 DEBUG:boto:Params: {} 2024-04-16T18:22:18.824 DEBUG:boto:Token: None 2024-04-16T18:22:18.824 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fOQd8Jfq93ZNguIu+Pceb1rFmmI= 2024-04-16T18:22:18.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fOQd8Jfq93ZNguIu+Pceb1rFmmI=', 'Content-Length': '0'} 2024-04-16T18:22:18.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086887cf33b66ccc7-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.828 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:22:18.828 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.828 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.828 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.828 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.828 DEBUG:boto:Method: GET 2024-04-16T18:22:18.828 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.829 DEBUG:boto:Data: 2024-04-16T18:22:18.829 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.829 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.829 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.829 DEBUG:boto:Params: {} 2024-04-16T18:22:18.829 DEBUG:boto:Token: None 2024-04-16T18:22:18.829 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ= 2024-04-16T18:22:18.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ=', 'Content-Length': '0'} 2024-04-16T18:22:18.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a57f14e8de71bba6-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.874 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.875 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.875 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.875 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.875 DEBUG:boto:Method: GET 2024-04-16T18:22:18.875 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.875 DEBUG:boto:Data: 2024-04-16T18:22:18.875 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.875 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.875 DEBUG:boto:Params: {} 2024-04-16T18:22:18.876 DEBUG:boto:Token: None 2024-04-16T18:22:18.876 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:22:18.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ= 2024-04-16T18:22:18.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mwLZKNtrHWuN7zsVVTsW48GDjIQ=', 'Content-Length': '0'} 2024-04-16T18:22:18.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cac7f032edcf794-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.879 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.880 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.880 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:18.880 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:18.880 DEBUG:boto:Method: GET 2024-04-16T18:22:18.880 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:22:18.880 DEBUG:boto:Data: 2024-04-16T18:22:18.880 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.880 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.880 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.880 DEBUG:boto:Params: {} 2024-04-16T18:22:18.881 DEBUG:boto:Token: None 2024-04-16T18:22:18.881 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.881 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c161ef6083d0ea3e-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.885 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.885 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.885 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.885 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:18.885 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:18.885 DEBUG:boto:Method: GET 2024-04-16T18:22:18.885 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:22:18.886 DEBUG:boto:Data: 2024-04-16T18:22:18.886 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.886 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.886 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.886 DEBUG:boto:Params: {} 2024-04-16T18:22:18.886 DEBUG:boto:Token: None 2024-04-16T18:22:18.886 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.886 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:18.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059674b0ab343b47b-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.934 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:18.935 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.935 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.935 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.935 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:22:18.935 DEBUG:boto:Data: 2024-04-16T18:22:18.935 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.935 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.935 DEBUG:boto:Params: {} 2024-04-16T18:22:18.936 DEBUG:boto:Token: None 2024-04-16T18:22:18.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33 2024-04-16T18:22:18.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xpwIuRnn/SFfF1JJZ4zOweo/fUA= 2024-04-16T18:22:18.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xpwIuRnn/SFfF1JJZ4zOweo/fUA=', 'Content-Length': '0'} 2024-04-16T18:22:18.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070638fe3ef6aff6e-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.939 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.939 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.939 DEBUG:boto:Method: HEAD 2024-04-16T18:22:18.939 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:22:18.939 DEBUG:boto:Data: 2024-04-16T18:22:18.940 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.940 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.940 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.940 DEBUG:boto:Params: {} 2024-04-16T18:22:18.940 DEBUG:boto:Token: None 2024-04-16T18:22:18.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33 2024-04-16T18:22:18.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xpwIuRnn/SFfF1JJZ4zOweo/fUA= 2024-04-16T18:22:18.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xpwIuRnn/SFfF1JJZ4zOweo/fUA=', 'Content-Length': '0'} 2024-04-16T18:22:18.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0f9eaf8eb7dc7a3-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.943 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:22:18.944 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.944 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.944 DEBUG:boto:Method: GET 2024-04-16T18:22:18.944 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:22:18.944 DEBUG:boto:Data: 2024-04-16T18:22:18.944 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.944 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.944 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.944 DEBUG:boto:Params: {} 2024-04-16T18:22:18.945 DEBUG:boto:Token: None 2024-04-16T18:22:18.945 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33 2024-04-16T18:22:18.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPE+RpL5CT0gGl5DDZR+JOS/Wdc= 2024-04-16T18:22:18.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPE+RpL5CT0gGl5DDZR+JOS/Wdc=', 'Content-Length': '0'} 2024-04-16T18:22:18.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fba6acc9889ce302-00661ec1da-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.990 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:22:18.990 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:22:18.991 DEBUG:boto:Method: GET 2024-04-16T18:22:18.991 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:22:18.991 DEBUG:boto:Data: 2024-04-16T18:22:18.991 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:18.991 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.991 DEBUG:boto:Params: {} 2024-04-16T18:22:18.991 DEBUG:boto:Token: None 2024-04-16T18:22:18.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/obj33 2024-04-16T18:22:18.991 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPE+RpL5CT0gGl5DDZR+JOS/Wdc= 2024-04-16T18:22:18.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPE+RpL5CT0gGl5DDZR+JOS/Wdc=', 'Content-Length': '0'} 2024-04-16T18:22:18.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0a6be9da3c11889-00661ec1da-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:18.995 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:18.995 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:18.995 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:18.995 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:18.995 DEBUG:boto:Method: GET 2024-04-16T18:22:18.995 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:22:18.995 DEBUG:boto:Data: 2024-04-16T18:22:18.995 DEBUG:boto:Headers: {} 2024-04-16T18:22:18.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:18.996 DEBUG:boto:Port: 8000 2024-04-16T18:22:18.996 DEBUG:boto:Params: {} 2024-04-16T18:22:18.996 DEBUG:boto:Token: None 2024-04-16T18:22:18.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:18 GMT /lybtug-52/ 2024-04-16T18:22:18.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU= 2024-04-16T18:22:18.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:66sBxJhWZBp2UWUrDzxbSeHMSsU=', 'Content-Length': '0'} 2024-04-16T18:22:19.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd32be5c28172b7f-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.042 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.043 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.043 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.043 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:19.043 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:22:19.043 DEBUG:boto:Method: GET 2024-04-16T18:22:19.043 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:22:19.043 DEBUG:boto:Data: 2024-04-16T18:22:19.043 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.044 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.044 DEBUG:boto:Params: {} 2024-04-16T18:22:19.044 DEBUG:boto:Token: None 2024-04-16T18:22:19.044 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.048 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfc634445c0cfb33-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.048 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.048 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:22:19.049 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.049 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.049 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.049 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.049 DEBUG:boto:Method: GET 2024-04-16T18:22:19.049 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.049 DEBUG:boto:Data: 2024-04-16T18:22:19.049 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.049 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.049 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.050 DEBUG:boto:Params: {} 2024-04-16T18:22:19.050 DEBUG:boto:Token: None 2024-04-16T18:22:19.050 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.050 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI= 2024-04-16T18:22:19.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI=', 'Content-Length': '0'} 2024-04-16T18:22:19.053 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ca66972d012b00e-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.094 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.094 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.095 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.095 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.095 DEBUG:boto:Method: GET 2024-04-16T18:22:19.095 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.095 DEBUG:boto:Data: 2024-04-16T18:22:19.095 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.095 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.095 DEBUG:boto:Params: {} 2024-04-16T18:22:19.095 DEBUG:boto:Token: None 2024-04-16T18:22:19.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI= 2024-04-16T18:22:19.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI=', 'Content-Length': '0'} 2024-04-16T18:22:19.099 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ca165b8ddb67816-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.099 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.099 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.099 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.099 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.099 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.100 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.100 DEBUG:boto:Data: 2024-04-16T18:22:19.100 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.100 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.100 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.100 DEBUG:boto:Params: {} 2024-04-16T18:22:19.100 DEBUG:boto:Token: None 2024-04-16T18:22:19.100 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N1MmXoYxQux3f73Dqm1Zdewv7UM= 2024-04-16T18:22:19.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N1MmXoYxQux3f73Dqm1Zdewv7UM=', 'Content-Length': '0'} 2024-04-16T18:22:19.103 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d35bdf579bdbbf14-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.103 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.103 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.104 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.104 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.104 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.104 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.104 DEBUG:boto:Data: 2024-04-16T18:22:19.104 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.104 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.104 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.104 DEBUG:boto:Params: {} 2024-04-16T18:22:19.104 DEBUG:boto:Token: None 2024-04-16T18:22:19.104 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.105 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N1MmXoYxQux3f73Dqm1Zdewv7UM= 2024-04-16T18:22:19.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N1MmXoYxQux3f73Dqm1Zdewv7UM=', 'Content-Length': '0'} 2024-04-16T18:22:19.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b4c6348bae4d4d1-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.108 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:22:19.109 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.109 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.109 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.110 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.110 DEBUG:boto:Method: GET 2024-04-16T18:22:19.110 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.110 DEBUG:boto:Data: 2024-04-16T18:22:19.110 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.110 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.110 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.110 DEBUG:boto:Params: {} 2024-04-16T18:22:19.110 DEBUG:boto:Token: None 2024-04-16T18:22:19.111 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI= 2024-04-16T18:22:19.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI=', 'Content-Length': '0'} 2024-04-16T18:22:19.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006952880f2d7cb59c-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.154 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.154 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.154 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.155 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.155 DEBUG:boto:Method: GET 2024-04-16T18:22:19.155 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.155 DEBUG:boto:Data: 2024-04-16T18:22:19.155 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.155 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.155 DEBUG:boto:Params: {} 2024-04-16T18:22:19.155 DEBUG:boto:Token: None 2024-04-16T18:22:19.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:22:19.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI= 2024-04-16T18:22:19.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eEqOrWrurv9YemjONzHIBRBkJVI=', 'Content-Length': '0'} 2024-04-16T18:22:19.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2b2b37c79834b90-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.159 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.159 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.159 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.159 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.159 DEBUG:boto:Method: GET 2024-04-16T18:22:19.159 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:22:19.159 DEBUG:boto:Data: 2024-04-16T18:22:19.159 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.160 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.160 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.160 DEBUG:boto:Params: {} 2024-04-16T18:22:19.160 DEBUG:boto:Token: None 2024-04-16T18:22:19.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c07e72d2d9523e48-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.164 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.164 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.164 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.165 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.165 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.165 DEBUG:boto:Method: GET 2024-04-16T18:22:19.165 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:22:19.165 DEBUG:boto:Data: 2024-04-16T18:22:19.165 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.165 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.165 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.165 DEBUG:boto:Params: {} 2024-04-16T18:22:19.165 DEBUG:boto:Token: None 2024-04-16T18:22:19.165 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.166 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.170 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2a610435e56a692-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.210 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.211 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.211 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.211 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.211 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:22:19.211 DEBUG:boto:Data: 2024-04-16T18:22:19.211 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.211 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.211 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.211 DEBUG:boto:Params: {} 2024-04-16T18:22:19.211 DEBUG:boto:Token: None 2024-04-16T18:22:19.212 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34 2024-04-16T18:22:19.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D43eY3yPxOL63tOFIdvO39/vwN0= 2024-04-16T18:22:19.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D43eY3yPxOL63tOFIdvO39/vwN0=', 'Content-Length': '0'} 2024-04-16T18:22:19.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054de36b43e3b8b53-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.215 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.215 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.215 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.215 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:22:19.215 DEBUG:boto:Data: 2024-04-16T18:22:19.215 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.215 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.215 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.216 DEBUG:boto:Params: {} 2024-04-16T18:22:19.216 DEBUG:boto:Token: None 2024-04-16T18:22:19.216 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34 2024-04-16T18:22:19.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D43eY3yPxOL63tOFIdvO39/vwN0= 2024-04-16T18:22:19.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D43eY3yPxOL63tOFIdvO39/vwN0=', 'Content-Length': '0'} 2024-04-16T18:22:19.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a68f0253538a605-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.219 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:22:19.219 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.219 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.219 DEBUG:boto:Method: GET 2024-04-16T18:22:19.219 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:22:19.219 DEBUG:boto:Data: 2024-04-16T18:22:19.220 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.220 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.220 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.220 DEBUG:boto:Params: {} 2024-04-16T18:22:19.220 DEBUG:boto:Token: None 2024-04-16T18:22:19.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34 2024-04-16T18:22:19.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lY5H1eH9MPkHM09sPENp/JMSBBI= 2024-04-16T18:22:19.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lY5H1eH9MPkHM09sPENp/JMSBBI=', 'Content-Length': '0'} 2024-04-16T18:22:19.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f6a25a665f584b0-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.266 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:22:19.266 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:22:19.267 DEBUG:boto:Method: GET 2024-04-16T18:22:19.267 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:22:19.267 DEBUG:boto:Data: 2024-04-16T18:22:19.267 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.267 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.267 DEBUG:boto:Params: {} 2024-04-16T18:22:19.267 DEBUG:boto:Token: None 2024-04-16T18:22:19.267 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj34 2024-04-16T18:22:19.267 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lY5H1eH9MPkHM09sPENp/JMSBBI= 2024-04-16T18:22:19.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lY5H1eH9MPkHM09sPENp/JMSBBI=', 'Content-Length': '0'} 2024-04-16T18:22:19.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000089520e04aee6b33-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.271 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.271 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.271 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.271 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.271 DEBUG:boto:Method: GET 2024-04-16T18:22:19.271 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:22:19.271 DEBUG:boto:Data: 2024-04-16T18:22:19.271 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.271 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.271 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.271 DEBUG:boto:Params: {} 2024-04-16T18:22:19.272 DEBUG:boto:Token: None 2024-04-16T18:22:19.272 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.272 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.276 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000976bba2be56abb94-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.318 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.319 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.319 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.319 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.319 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:22:19.319 DEBUG:boto:Method: GET 2024-04-16T18:22:19.319 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:22:19.319 DEBUG:boto:Data: 2024-04-16T18:22:19.319 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.319 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.319 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.319 DEBUG:boto:Params: {} 2024-04-16T18:22:19.320 DEBUG:boto:Token: None 2024-04-16T18:22:19.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.324 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000076f77c8b08482bb-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.325 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:22:19.325 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.325 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.325 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.325 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.325 DEBUG:boto:Method: GET 2024-04-16T18:22:19.325 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.326 DEBUG:boto:Data: 2024-04-16T18:22:19.326 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.326 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.326 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.326 DEBUG:boto:Params: {} 2024-04-16T18:22:19.326 DEBUG:boto:Token: None 2024-04-16T18:22:19.326 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.326 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k= 2024-04-16T18:22:19.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k=', 'Content-Length': '0'} 2024-04-16T18:22:19.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd940552cb8676c3-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.370 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.370 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.371 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.371 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.371 DEBUG:boto:Method: GET 2024-04-16T18:22:19.371 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.371 DEBUG:boto:Data: 2024-04-16T18:22:19.371 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.371 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.371 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.371 DEBUG:boto:Params: {} 2024-04-16T18:22:19.371 DEBUG:boto:Token: None 2024-04-16T18:22:19.371 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k= 2024-04-16T18:22:19.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k=', 'Content-Length': '0'} 2024-04-16T18:22:19.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bea161f5afce1e3-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.375 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.375 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.375 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.375 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.375 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.375 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.375 DEBUG:boto:Data: 2024-04-16T18:22:19.375 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.375 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.376 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.376 DEBUG:boto:Params: {} 2024-04-16T18:22:19.376 DEBUG:boto:Token: None 2024-04-16T18:22:19.376 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fzKNmi4vKp+kNOT4APpjrd1UE8c= 2024-04-16T18:22:19.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fzKNmi4vKp+kNOT4APpjrd1UE8c=', 'Content-Length': '0'} 2024-04-16T18:22:19.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002c508af20665413-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.379 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.379 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.379 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.379 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.379 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.379 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.380 DEBUG:boto:Data: 2024-04-16T18:22:19.380 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.380 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.380 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.380 DEBUG:boto:Params: {} 2024-04-16T18:22:19.380 DEBUG:boto:Token: None 2024-04-16T18:22:19.380 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fzKNmi4vKp+kNOT4APpjrd1UE8c= 2024-04-16T18:22:19.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fzKNmi4vKp+kNOT4APpjrd1UE8c=', 'Content-Length': '0'} 2024-04-16T18:22:19.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7aede9c1479fd49-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.384 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:22:19.384 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.384 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.384 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.384 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.384 DEBUG:boto:Method: GET 2024-04-16T18:22:19.384 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.384 DEBUG:boto:Data: 2024-04-16T18:22:19.384 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.384 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.384 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.384 DEBUG:boto:Params: {} 2024-04-16T18:22:19.385 DEBUG:boto:Token: None 2024-04-16T18:22:19.385 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.385 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k= 2024-04-16T18:22:19.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k=', 'Content-Length': '0'} 2024-04-16T18:22:19.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061d0e2b9aa6822b2-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.430 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.430 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.430 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.431 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.431 DEBUG:boto:Method: GET 2024-04-16T18:22:19.431 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.431 DEBUG:boto:Data: 2024-04-16T18:22:19.431 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.431 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.431 DEBUG:boto:Params: {} 2024-04-16T18:22:19.431 DEBUG:boto:Token: None 2024-04-16T18:22:19.431 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:22:19.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k= 2024-04-16T18:22:19.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/XSDNAijBOlSiRd00p7HbxdXX5k=', 'Content-Length': '0'} 2024-04-16T18:22:19.435 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000812e3659fd2e1410-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.435 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.435 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.435 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.435 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.435 DEBUG:boto:Method: GET 2024-04-16T18:22:19.435 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:22:19.435 DEBUG:boto:Data: 2024-04-16T18:22:19.435 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.436 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.436 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.436 DEBUG:boto:Params: {} 2024-04-16T18:22:19.436 DEBUG:boto:Token: None 2024-04-16T18:22:19.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000efa546ad58f4fe9a-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.439 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.440 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.440 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.440 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.440 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.440 DEBUG:boto:Method: GET 2024-04-16T18:22:19.440 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:22:19.440 DEBUG:boto:Data: 2024-04-16T18:22:19.440 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.440 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.441 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.441 DEBUG:boto:Params: {} 2024-04-16T18:22:19.441 DEBUG:boto:Token: None 2024-04-16T18:22:19.441 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.441 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e304c19d0e9aae8-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.486 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.487 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.487 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.487 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.487 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:22:19.487 DEBUG:boto:Data: 2024-04-16T18:22:19.487 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.487 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.487 DEBUG:boto:Params: {} 2024-04-16T18:22:19.487 DEBUG:boto:Token: None 2024-04-16T18:22:19.487 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44 2024-04-16T18:22:19.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2SYcMKSls1uxMNK6b6Yzc5l9fY8= 2024-04-16T18:22:19.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2SYcMKSls1uxMNK6b6Yzc5l9fY8=', 'Content-Length': '0'} 2024-04-16T18:22:19.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d099e0090a22c40f-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.491 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.491 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.491 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.491 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:22:19.491 DEBUG:boto:Data: 2024-04-16T18:22:19.491 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.491 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.491 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.492 DEBUG:boto:Params: {} 2024-04-16T18:22:19.492 DEBUG:boto:Token: None 2024-04-16T18:22:19.492 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44 2024-04-16T18:22:19.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2SYcMKSls1uxMNK6b6Yzc5l9fY8= 2024-04-16T18:22:19.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2SYcMKSls1uxMNK6b6Yzc5l9fY8=', 'Content-Length': '0'} 2024-04-16T18:22:19.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b669f14bf11c6c44-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:22:19.496 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.496 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.496 DEBUG:boto:Method: GET 2024-04-16T18:22:19.496 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:22:19.496 DEBUG:boto:Data: 2024-04-16T18:22:19.496 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.496 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.496 DEBUG:boto:Params: {} 2024-04-16T18:22:19.496 DEBUG:boto:Token: None 2024-04-16T18:22:19.497 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44 2024-04-16T18:22:19.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MrNYGmPnPOoptSGXzkD0b+H1/vo= 2024-04-16T18:22:19.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MrNYGmPnPOoptSGXzkD0b+H1/vo=', 'Content-Length': '0'} 2024-04-16T18:22:19.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e475c2cc4ce19e7-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.542 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:22:19.542 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:22:19.543 DEBUG:boto:Method: GET 2024-04-16T18:22:19.543 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:22:19.543 DEBUG:boto:Data: 2024-04-16T18:22:19.543 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.543 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.543 DEBUG:boto:Params: {} 2024-04-16T18:22:19.543 DEBUG:boto:Token: None 2024-04-16T18:22:19.543 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj44 2024-04-16T18:22:19.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MrNYGmPnPOoptSGXzkD0b+H1/vo= 2024-04-16T18:22:19.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MrNYGmPnPOoptSGXzkD0b+H1/vo=', 'Content-Length': '0'} 2024-04-16T18:22:19.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9387edd53427da5-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.547 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.547 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.547 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.548 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.548 DEBUG:boto:Method: GET 2024-04-16T18:22:19.548 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:22:19.548 DEBUG:boto:Data: 2024-04-16T18:22:19.548 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.548 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.548 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.548 DEBUG:boto:Params: {} 2024-04-16T18:22:19.548 DEBUG:boto:Token: None 2024-04-16T18:22:19.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.552 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000768ee7f966057c79-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.594 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.595 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.595 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.595 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.595 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:22:19.595 DEBUG:boto:Method: GET 2024-04-16T18:22:19.595 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:22:19.595 DEBUG:boto:Data: 2024-04-16T18:22:19.595 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.595 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.595 DEBUG:boto:Params: {} 2024-04-16T18:22:19.596 DEBUG:boto:Token: None 2024-04-16T18:22:19.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a2b1165da5af872-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.600 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.601 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:22:19.601 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.601 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.601 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.601 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.601 DEBUG:boto:Method: GET 2024-04-16T18:22:19.601 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.601 DEBUG:boto:Data: 2024-04-16T18:22:19.601 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.602 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.602 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.602 DEBUG:boto:Params: {} 2024-04-16T18:22:19.602 DEBUG:boto:Token: None 2024-04-16T18:22:19.602 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.602 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g= 2024-04-16T18:22:19.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g=', 'Content-Length': '0'} 2024-04-16T18:22:19.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091ab26ac304592ad-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.646 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.646 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.647 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.647 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.647 DEBUG:boto:Method: GET 2024-04-16T18:22:19.647 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.647 DEBUG:boto:Data: 2024-04-16T18:22:19.647 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.647 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.647 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.647 DEBUG:boto:Params: {} 2024-04-16T18:22:19.647 DEBUG:boto:Token: None 2024-04-16T18:22:19.648 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g= 2024-04-16T18:22:19.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g=', 'Content-Length': '0'} 2024-04-16T18:22:19.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f14e528f54e715a2-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.651 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.652 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.652 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.652 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.652 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.653 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.653 DEBUG:boto:Data: 2024-04-16T18:22:19.653 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.653 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.653 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.653 DEBUG:boto:Params: {} 2024-04-16T18:22:19.653 DEBUG:boto:Token: None 2024-04-16T18:22:19.654 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.654 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/YzCyZ0TnyED5J4y19EkxP/lDcQ= 2024-04-16T18:22:19.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/YzCyZ0TnyED5J4y19EkxP/lDcQ=', 'Content-Length': '0'} 2024-04-16T18:22:19.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d59393ea4183627-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.657 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.657 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.657 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.657 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.657 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.657 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.657 DEBUG:boto:Data: 2024-04-16T18:22:19.657 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.657 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.658 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.658 DEBUG:boto:Params: {} 2024-04-16T18:22:19.658 DEBUG:boto:Token: None 2024-04-16T18:22:19.658 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.658 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/YzCyZ0TnyED5J4y19EkxP/lDcQ= 2024-04-16T18:22:19.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/YzCyZ0TnyED5J4y19EkxP/lDcQ=', 'Content-Length': '0'} 2024-04-16T18:22:19.661 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c4839282222ab0e-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:22:19.662 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.662 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.662 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.662 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.662 DEBUG:boto:Method: GET 2024-04-16T18:22:19.662 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.662 DEBUG:boto:Data: 2024-04-16T18:22:19.662 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.662 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.662 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.663 DEBUG:boto:Params: {} 2024-04-16T18:22:19.663 DEBUG:boto:Token: None 2024-04-16T18:22:19.663 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g= 2024-04-16T18:22:19.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g=', 'Content-Length': '0'} 2024-04-16T18:22:19.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc538fa952aea592-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.706 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.706 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.707 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.707 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.707 DEBUG:boto:Method: GET 2024-04-16T18:22:19.707 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.707 DEBUG:boto:Data: 2024-04-16T18:22:19.707 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.707 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.707 DEBUG:boto:Params: {} 2024-04-16T18:22:19.707 DEBUG:boto:Token: None 2024-04-16T18:22:19.707 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:22:19.707 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g= 2024-04-16T18:22:19.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8AcA3OJqrnLueGmPLqI+71B9o7g=', 'Content-Length': '0'} 2024-04-16T18:22:19.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f6c6955cf3e3936-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.711 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.711 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.711 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.711 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.711 DEBUG:boto:Method: GET 2024-04-16T18:22:19.712 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:22:19.712 DEBUG:boto:Data: 2024-04-16T18:22:19.712 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.712 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.712 DEBUG:boto:Params: {} 2024-04-16T18:22:19.712 DEBUG:boto:Token: None 2024-04-16T18:22:19.712 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070a9a0f2de669b0c-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.758 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.759 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.759 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.759 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.759 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.759 DEBUG:boto:Method: GET 2024-04-16T18:22:19.759 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:22:19.759 DEBUG:boto:Data: 2024-04-16T18:22:19.759 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.759 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.760 DEBUG:boto:Params: {} 2024-04-16T18:22:19.760 DEBUG:boto:Token: None 2024-04-16T18:22:19.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.765 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000044a7d4adba59d15-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.765 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.765 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.765 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.765 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.765 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:22:19.765 DEBUG:boto:Data: 2024-04-16T18:22:19.766 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.766 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.766 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.766 DEBUG:boto:Params: {} 2024-04-16T18:22:19.766 DEBUG:boto:Token: None 2024-04-16T18:22:19.766 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51 2024-04-16T18:22:19.766 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KYbidyl8++AF73jqHPQGgm8g9c0= 2024-04-16T18:22:19.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KYbidyl8++AF73jqHPQGgm8g9c0=', 'Content-Length': '0'} 2024-04-16T18:22:19.769 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d17b279e12e2b508-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.769 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.769 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.769 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.769 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:22:19.769 DEBUG:boto:Data: 2024-04-16T18:22:19.769 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.769 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.770 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.770 DEBUG:boto:Params: {} 2024-04-16T18:22:19.770 DEBUG:boto:Token: None 2024-04-16T18:22:19.770 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51 2024-04-16T18:22:19.770 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KYbidyl8++AF73jqHPQGgm8g9c0= 2024-04-16T18:22:19.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KYbidyl8++AF73jqHPQGgm8g9c0=', 'Content-Length': '0'} 2024-04-16T18:22:19.773 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006199063e846f75eb-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.773 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:22:19.773 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.774 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.774 DEBUG:boto:Method: GET 2024-04-16T18:22:19.774 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:22:19.774 DEBUG:boto:Data: 2024-04-16T18:22:19.774 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.774 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.774 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.774 DEBUG:boto:Params: {} 2024-04-16T18:22:19.774 DEBUG:boto:Token: None 2024-04-16T18:22:19.774 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51 2024-04-16T18:22:19.774 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZAg0FQajovdg+JqnZB2L+hYQfo= 2024-04-16T18:22:19.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZAg0FQajovdg+JqnZB2L+hYQfo=', 'Content-Length': '0'} 2024-04-16T18:22:19.777 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005464464a34f5ce32-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.818 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:22:19.818 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:22:19.818 DEBUG:boto:Method: GET 2024-04-16T18:22:19.819 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:22:19.819 DEBUG:boto:Data: 2024-04-16T18:22:19.819 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.819 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.819 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.819 DEBUG:boto:Params: {} 2024-04-16T18:22:19.819 DEBUG:boto:Token: None 2024-04-16T18:22:19.819 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj51 2024-04-16T18:22:19.819 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1ZAg0FQajovdg+JqnZB2L+hYQfo= 2024-04-16T18:22:19.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1ZAg0FQajovdg+JqnZB2L+hYQfo=', 'Content-Length': '0'} 2024-04-16T18:22:19.822 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc97f24c04ec60ba-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.823 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.823 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.823 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.823 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.823 DEBUG:boto:Method: GET 2024-04-16T18:22:19.823 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:22:19.823 DEBUG:boto:Data: 2024-04-16T18:22:19.823 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.824 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.824 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.824 DEBUG:boto:Params: {} 2024-04-16T18:22:19.824 DEBUG:boto:Token: None 2024-04-16T18:22:19.824 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044995db14f47f51f-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.828 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.828 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.828 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.828 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.828 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:22:19.828 DEBUG:boto:Method: GET 2024-04-16T18:22:19.829 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:22:19.829 DEBUG:boto:Data: 2024-04-16T18:22:19.829 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.829 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.829 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.829 DEBUG:boto:Params: {} 2024-04-16T18:22:19.829 DEBUG:boto:Token: None 2024-04-16T18:22:19.829 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.833 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9944e7869091377-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.834 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.834 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:22:19.834 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:19.834 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:19.834 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.834 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.834 DEBUG:boto:Method: GET 2024-04-16T18:22:19.835 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.835 DEBUG:boto:Data: 2024-04-16T18:22:19.835 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.835 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.835 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.835 DEBUG:boto:Params: {} 2024-04-16T18:22:19.835 DEBUG:boto:Token: None 2024-04-16T18:22:19.835 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk= 2024-04-16T18:22:19.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk=', 'Content-Length': '0'} 2024-04-16T18:22:19.838 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081c621526654e85b-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.882 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:19.882 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:19.882 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.883 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.883 DEBUG:boto:Method: GET 2024-04-16T18:22:19.883 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.883 DEBUG:boto:Data: 2024-04-16T18:22:19.883 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.883 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.883 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.883 DEBUG:boto:Params: {} 2024-04-16T18:22:19.883 DEBUG:boto:Token: None 2024-04-16T18:22:19.883 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk= 2024-04-16T18:22:19.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk=', 'Content-Length': '0'} 2024-04-16T18:22:19.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020ed2bda61ffe015-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.887 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:19.887 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:19.887 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.887 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.887 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.887 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.887 DEBUG:boto:Data: 2024-04-16T18:22:19.887 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.888 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.888 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.888 DEBUG:boto:Params: {} 2024-04-16T18:22:19.888 DEBUG:boto:Token: None 2024-04-16T18:22:19.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lFzhDjAVsbfZkcupuaXsyEY92IQ= 2024-04-16T18:22:19.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lFzhDjAVsbfZkcupuaXsyEY92IQ=', 'Content-Length': '0'} 2024-04-16T18:22:19.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099f079d80ddf29e1-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.891 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:19.891 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:19.891 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.891 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.891 DEBUG:boto:Method: HEAD 2024-04-16T18:22:19.892 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.892 DEBUG:boto:Data: 2024-04-16T18:22:19.892 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.892 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.892 DEBUG:boto:Params: {} 2024-04-16T18:22:19.892 DEBUG:boto:Token: None 2024-04-16T18:22:19.892 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lFzhDjAVsbfZkcupuaXsyEY92IQ= 2024-04-16T18:22:19.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lFzhDjAVsbfZkcupuaXsyEY92IQ=', 'Content-Length': '0'} 2024-04-16T18:22:19.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7c697021eb16c74-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.895 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:22:19.895 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:19.895 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:19.896 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.896 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.896 DEBUG:boto:Method: GET 2024-04-16T18:22:19.896 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.896 DEBUG:boto:Data: 2024-04-16T18:22:19.896 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.896 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.896 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.896 DEBUG:boto:Params: {} 2024-04-16T18:22:19.896 DEBUG:boto:Token: None 2024-04-16T18:22:19.896 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.897 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk= 2024-04-16T18:22:19.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk=', 'Content-Length': '0'} 2024-04-16T18:22:19.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dc420e4d7890e27-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.942 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:19.942 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:19.942 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.943 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.943 DEBUG:boto:Method: GET 2024-04-16T18:22:19.943 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.943 DEBUG:boto:Data: 2024-04-16T18:22:19.943 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.943 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.943 DEBUG:boto:Params: {} 2024-04-16T18:22:19.943 DEBUG:boto:Token: None 2024-04-16T18:22:19.943 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:22:19.943 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk= 2024-04-16T18:22:19.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LWeSbxrNOsT6RW9Uod84gkrnXAk=', 'Content-Length': '0'} 2024-04-16T18:22:19.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000423dc732c2cbb6a3-00661ec1db-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.947 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.947 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.947 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:19.947 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:19.948 DEBUG:boto:Method: GET 2024-04-16T18:22:19.948 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:22:19.948 DEBUG:boto:Data: 2024-04-16T18:22:19.948 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:19.948 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.948 DEBUG:boto:Params: {} 2024-04-16T18:22:19.948 DEBUG:boto:Token: None 2024-04-16T18:22:19.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:19.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb699b5fb70bd72d-00661ec1db-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:19.994 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:19.995 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:19.995 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:19.995 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:19.995 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:19.995 DEBUG:boto:Method: GET 2024-04-16T18:22:19.995 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:22:19.995 DEBUG:boto:Data: 2024-04-16T18:22:19.995 DEBUG:boto:Headers: {} 2024-04-16T18:22:19.995 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:19.995 DEBUG:boto:Port: 8000 2024-04-16T18:22:19.995 DEBUG:boto:Params: {} 2024-04-16T18:22:19.996 DEBUG:boto:Token: None 2024-04-16T18:22:19.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:19 GMT /lybtug-52/ 2024-04-16T18:22:19.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss= 2024-04-16T18:22:19.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixn3U6NO04ACRLNx4PgPt88/Uss=', 'Content-Length': '0'} 2024-04-16T18:22:20.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3ba245c346da460-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.000 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.001 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:20.001 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:20.001 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.001 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:22:20.001 DEBUG:boto:Data: 2024-04-16T18:22:20.001 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.001 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.001 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.001 DEBUG:boto:Params: {} 2024-04-16T18:22:20.001 DEBUG:boto:Token: None 2024-04-16T18:22:20.001 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj55 2024-04-16T18:22:20.002 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gdnF3uRT1gOWIAZz8AW8gfHm21g= 2024-04-16T18:22:20.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gdnF3uRT1gOWIAZz8AW8gfHm21g=', 'Content-Length': '0'} 2024-04-16T18:22:20.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a31fc8686ac1e7c-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.004 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:20.004 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:20.004 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.004 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:22:20.005 DEBUG:boto:Data: 2024-04-16T18:22:20.005 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.005 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.005 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.005 DEBUG:boto:Params: {} 2024-04-16T18:22:20.005 DEBUG:boto:Token: None 2024-04-16T18:22:20.005 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj55 2024-04-16T18:22:20.005 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gdnF3uRT1gOWIAZz8AW8gfHm21g= 2024-04-16T18:22:20.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gdnF3uRT1gOWIAZz8AW8gfHm21g=', 'Content-Length': '0'} 2024-04-16T18:22:20.008 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b09ee3e05c8c82c0-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.008 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:22:20.008 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:20.008 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:20.009 DEBUG:boto:Method: GET 2024-04-16T18:22:20.009 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:22:20.009 DEBUG:boto:Data: 2024-04-16T18:22:20.009 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.009 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.009 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.009 DEBUG:boto:Params: {} 2024-04-16T18:22:20.009 DEBUG:boto:Token: None 2024-04-16T18:22:20.009 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj55 2024-04-16T18:22:20.009 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KGWSv4d30AyiEwF74kiI2LVZrAw= 2024-04-16T18:22:20.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KGWSv4d30AyiEwF74kiI2LVZrAw=', 'Content-Length': '0'} 2024-04-16T18:22:20.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002758a96b3e536595-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.054 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:22:20.054 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:22:20.054 DEBUG:boto:Method: GET 2024-04-16T18:22:20.055 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:22:20.055 DEBUG:boto:Data: 2024-04-16T18:22:20.055 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.055 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.055 DEBUG:boto:Params: {} 2024-04-16T18:22:20.055 DEBUG:boto:Token: None 2024-04-16T18:22:20.055 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj55 2024-04-16T18:22:20.055 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KGWSv4d30AyiEwF74kiI2LVZrAw= 2024-04-16T18:22:20.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KGWSv4d30AyiEwF74kiI2LVZrAw=', 'Content-Length': '0'} 2024-04-16T18:22:20.058 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040fed76eb1e91ab4-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.058 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.059 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.059 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:20.059 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:20.059 DEBUG:boto:Method: GET 2024-04-16T18:22:20.059 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:22:20.059 DEBUG:boto:Data: 2024-04-16T18:22:20.060 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.060 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.060 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.060 DEBUG:boto:Params: {} 2024-04-16T18:22:20.060 DEBUG:boto:Token: None 2024-04-16T18:22:20.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.064 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007243a26123a6507c-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.064 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.065 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.065 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.065 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:20.065 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:22:20.065 DEBUG:boto:Method: GET 2024-04-16T18:22:20.065 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:22:20.065 DEBUG:boto:Data: 2024-04-16T18:22:20.065 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.065 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.065 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.066 DEBUG:boto:Params: {} 2024-04-16T18:22:20.066 DEBUG:boto:Token: None 2024-04-16T18:22:20.066 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.066 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.071 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000185bc37eff90da58-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.071 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:22:20.072 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.072 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.072 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.072 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.072 DEBUG:boto:Method: GET 2024-04-16T18:22:20.072 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.072 DEBUG:boto:Data: 2024-04-16T18:22:20.072 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.072 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.072 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.073 DEBUG:boto:Params: {} 2024-04-16T18:22:20.073 DEBUG:boto:Token: None 2024-04-16T18:22:20.073 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY= 2024-04-16T18:22:20.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY=', 'Content-Length': '0'} 2024-04-16T18:22:20.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcd4f0bd0cc15e29-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.118 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.118 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.118 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.119 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.119 DEBUG:boto:Method: GET 2024-04-16T18:22:20.119 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.119 DEBUG:boto:Data: 2024-04-16T18:22:20.119 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.119 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.119 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.119 DEBUG:boto:Params: {} 2024-04-16T18:22:20.119 DEBUG:boto:Token: None 2024-04-16T18:22:20.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY= 2024-04-16T18:22:20.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY=', 'Content-Length': '0'} 2024-04-16T18:22:20.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a65ea809875c2716-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.123 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.123 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.123 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.123 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.123 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.123 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.123 DEBUG:boto:Data: 2024-04-16T18:22:20.123 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.123 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.123 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.123 DEBUG:boto:Params: {} 2024-04-16T18:22:20.124 DEBUG:boto:Token: None 2024-04-16T18:22:20.124 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e/RDMmWZNn8Jjn1rcsazwHSn2K8= 2024-04-16T18:22:20.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e/RDMmWZNn8Jjn1rcsazwHSn2K8=', 'Content-Length': '0'} 2024-04-16T18:22:20.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ef962a811942639-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.127 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.127 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.127 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.127 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.127 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.127 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.128 DEBUG:boto:Data: 2024-04-16T18:22:20.128 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.128 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.128 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.128 DEBUG:boto:Params: {} 2024-04-16T18:22:20.128 DEBUG:boto:Token: None 2024-04-16T18:22:20.128 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e/RDMmWZNn8Jjn1rcsazwHSn2K8= 2024-04-16T18:22:20.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e/RDMmWZNn8Jjn1rcsazwHSn2K8=', 'Content-Length': '0'} 2024-04-16T18:22:20.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f757528c2e7f91f5-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.131 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:22:20.132 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.132 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.132 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.132 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.132 DEBUG:boto:Method: GET 2024-04-16T18:22:20.132 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.132 DEBUG:boto:Data: 2024-04-16T18:22:20.132 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.132 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.132 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.132 DEBUG:boto:Params: {} 2024-04-16T18:22:20.133 DEBUG:boto:Token: None 2024-04-16T18:22:20.133 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY= 2024-04-16T18:22:20.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY=', 'Content-Length': '0'} 2024-04-16T18:22:20.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c11809a19de3c3bc-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.178 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.178 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.178 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.179 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.179 DEBUG:boto:Method: GET 2024-04-16T18:22:20.179 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.179 DEBUG:boto:Data: 2024-04-16T18:22:20.179 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.179 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.180 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.180 DEBUG:boto:Params: {} 2024-04-16T18:22:20.180 DEBUG:boto:Token: None 2024-04-16T18:22:20.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:22:20.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY= 2024-04-16T18:22:20.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0c0vypWJVe4tef6znHuT0CqhxCY=', 'Content-Length': '0'} 2024-04-16T18:22:20.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ac15810abf69232-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.184 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.184 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.184 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.184 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.184 DEBUG:boto:Method: GET 2024-04-16T18:22:20.184 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:22:20.184 DEBUG:boto:Data: 2024-04-16T18:22:20.184 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.184 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.184 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.184 DEBUG:boto:Params: {} 2024-04-16T18:22:20.185 DEBUG:boto:Token: None 2024-04-16T18:22:20.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.189 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025580b405935aa19-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.230 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.231 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.231 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.231 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.231 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.231 DEBUG:boto:Method: GET 2024-04-16T18:22:20.231 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:22:20.231 DEBUG:boto:Data: 2024-04-16T18:22:20.231 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.231 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.231 DEBUG:boto:Params: {} 2024-04-16T18:22:20.232 DEBUG:boto:Token: None 2024-04-16T18:22:20.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.236 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065db11648b3b2545-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.236 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.236 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.236 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.236 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.237 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:22:20.237 DEBUG:boto:Data: 2024-04-16T18:22:20.237 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.237 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.237 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.237 DEBUG:boto:Params: {} 2024-04-16T18:22:20.237 DEBUG:boto:Token: None 2024-04-16T18:22:20.237 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68 2024-04-16T18:22:20.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DHCCYdkM7iJuzWaXcxec3bgZaeI= 2024-04-16T18:22:20.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DHCCYdkM7iJuzWaXcxec3bgZaeI=', 'Content-Length': '0'} 2024-04-16T18:22:20.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e10d90a27ef2af74-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.241 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.241 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.241 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.241 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:22:20.241 DEBUG:boto:Data: 2024-04-16T18:22:20.241 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.241 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.241 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.241 DEBUG:boto:Params: {} 2024-04-16T18:22:20.241 DEBUG:boto:Token: None 2024-04-16T18:22:20.241 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68 2024-04-16T18:22:20.242 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DHCCYdkM7iJuzWaXcxec3bgZaeI= 2024-04-16T18:22:20.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DHCCYdkM7iJuzWaXcxec3bgZaeI=', 'Content-Length': '0'} 2024-04-16T18:22:20.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de8e132108df20f5-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:22:20.245 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.245 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.245 DEBUG:boto:Method: GET 2024-04-16T18:22:20.245 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:22:20.245 DEBUG:boto:Data: 2024-04-16T18:22:20.245 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.245 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.245 DEBUG:boto:Params: {} 2024-04-16T18:22:20.246 DEBUG:boto:Token: None 2024-04-16T18:22:20.246 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68 2024-04-16T18:22:20.246 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BuWyshkJAPO4XTd2R/+wsEegh0A= 2024-04-16T18:22:20.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BuWyshkJAPO4XTd2R/+wsEegh0A=', 'Content-Length': '0'} 2024-04-16T18:22:20.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091743a708582d6c2-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.290 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:22:20.290 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:22:20.291 DEBUG:boto:Method: GET 2024-04-16T18:22:20.291 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:22:20.291 DEBUG:boto:Data: 2024-04-16T18:22:20.291 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.291 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.291 DEBUG:boto:Params: {} 2024-04-16T18:22:20.291 DEBUG:boto:Token: None 2024-04-16T18:22:20.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj68 2024-04-16T18:22:20.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BuWyshkJAPO4XTd2R/+wsEegh0A= 2024-04-16T18:22:20.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BuWyshkJAPO4XTd2R/+wsEegh0A=', 'Content-Length': '0'} 2024-04-16T18:22:20.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b4c825f0ad5b0b4-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.295 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.295 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.295 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.295 DEBUG:boto:Method: GET 2024-04-16T18:22:20.295 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:22:20.295 DEBUG:boto:Data: 2024-04-16T18:22:20.295 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.296 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.296 DEBUG:boto:Params: {} 2024-04-16T18:22:20.296 DEBUG:boto:Token: None 2024-04-16T18:22:20.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.299 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009df0882c3d388fe9-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.342 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.343 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.343 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.343 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.343 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:22:20.343 DEBUG:boto:Method: GET 2024-04-16T18:22:20.343 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:22:20.343 DEBUG:boto:Data: 2024-04-16T18:22:20.343 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.343 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.343 DEBUG:boto:Params: {} 2024-04-16T18:22:20.344 DEBUG:boto:Token: None 2024-04-16T18:22:20.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054f91bc10937268b-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.348 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:22:20.348 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.349 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.349 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.349 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.349 DEBUG:boto:Method: GET 2024-04-16T18:22:20.349 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.349 DEBUG:boto:Data: 2024-04-16T18:22:20.349 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.349 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.349 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.349 DEBUG:boto:Params: {} 2024-04-16T18:22:20.349 DEBUG:boto:Token: None 2024-04-16T18:22:20.349 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.350 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw= 2024-04-16T18:22:20.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw=', 'Content-Length': '0'} 2024-04-16T18:22:20.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8c9eeb37f88493e-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.394 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.394 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.395 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.395 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.395 DEBUG:boto:Method: GET 2024-04-16T18:22:20.395 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.395 DEBUG:boto:Data: 2024-04-16T18:22:20.395 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.395 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.395 DEBUG:boto:Params: {} 2024-04-16T18:22:20.395 DEBUG:boto:Token: None 2024-04-16T18:22:20.395 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw= 2024-04-16T18:22:20.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw=', 'Content-Length': '0'} 2024-04-16T18:22:20.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000936ccd6cca948b45-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.399 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.399 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.399 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.399 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.399 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.399 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.399 DEBUG:boto:Data: 2024-04-16T18:22:20.399 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.400 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.400 DEBUG:boto:Params: {} 2024-04-16T18:22:20.400 DEBUG:boto:Token: None 2024-04-16T18:22:20.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9ja/SSYYRzDC2P6WZ9Z6Huh3J2M= 2024-04-16T18:22:20.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9ja/SSYYRzDC2P6WZ9Z6Huh3J2M=', 'Content-Length': '0'} 2024-04-16T18:22:20.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bca180d72ba05378-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.403 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.403 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.403 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.403 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.403 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.404 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.404 DEBUG:boto:Data: 2024-04-16T18:22:20.404 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.404 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.404 DEBUG:boto:Params: {} 2024-04-16T18:22:20.404 DEBUG:boto:Token: None 2024-04-16T18:22:20.404 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9ja/SSYYRzDC2P6WZ9Z6Huh3J2M= 2024-04-16T18:22:20.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9ja/SSYYRzDC2P6WZ9Z6Huh3J2M=', 'Content-Length': '0'} 2024-04-16T18:22:20.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000810661b70fdb33c9-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.407 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:22:20.407 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.407 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.407 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.407 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.408 DEBUG:boto:Method: GET 2024-04-16T18:22:20.408 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.408 DEBUG:boto:Data: 2024-04-16T18:22:20.408 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.408 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.408 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.408 DEBUG:boto:Params: {} 2024-04-16T18:22:20.408 DEBUG:boto:Token: None 2024-04-16T18:22:20.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw= 2024-04-16T18:22:20.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw=', 'Content-Length': '0'} 2024-04-16T18:22:20.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bc54a675c1cd354-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.454 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.454 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.455 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.455 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.455 DEBUG:boto:Method: GET 2024-04-16T18:22:20.455 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.455 DEBUG:boto:Data: 2024-04-16T18:22:20.455 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.455 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.455 DEBUG:boto:Params: {} 2024-04-16T18:22:20.455 DEBUG:boto:Token: None 2024-04-16T18:22:20.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:22:20.455 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw= 2024-04-16T18:22:20.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tPc9W/idkAIJhzam9VZWvUy8cxw=', 'Content-Length': '0'} 2024-04-16T18:22:20.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013d3d7a4a295d909-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.459 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.459 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.459 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.459 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.459 DEBUG:boto:Method: GET 2024-04-16T18:22:20.459 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:22:20.459 DEBUG:boto:Data: 2024-04-16T18:22:20.459 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.459 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.459 DEBUG:boto:Params: {} 2024-04-16T18:22:20.460 DEBUG:boto:Token: None 2024-04-16T18:22:20.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.463 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086d1745727167bfa-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.464 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.464 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.464 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.464 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.464 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.464 DEBUG:boto:Method: GET 2024-04-16T18:22:20.465 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:22:20.465 DEBUG:boto:Data: 2024-04-16T18:22:20.465 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.465 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.465 DEBUG:boto:Params: {} 2024-04-16T18:22:20.465 DEBUG:boto:Token: None 2024-04-16T18:22:20.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c21652a386e697a-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.470 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.470 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.470 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.471 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.471 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:22:20.471 DEBUG:boto:Data: 2024-04-16T18:22:20.471 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.471 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.471 DEBUG:boto:Params: {} 2024-04-16T18:22:20.471 DEBUG:boto:Token: None 2024-04-16T18:22:20.471 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85 2024-04-16T18:22:20.471 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x3tkmvzRpuAt01wgHQ8XBK8hXZc= 2024-04-16T18:22:20.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x3tkmvzRpuAt01wgHQ8XBK8hXZc=', 'Content-Length': '0'} 2024-04-16T18:22:20.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb62f049cb9220ca-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.474 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.474 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.475 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.475 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:22:20.475 DEBUG:boto:Data: 2024-04-16T18:22:20.475 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.475 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.475 DEBUG:boto:Params: {} 2024-04-16T18:22:20.475 DEBUG:boto:Token: None 2024-04-16T18:22:20.475 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85 2024-04-16T18:22:20.475 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x3tkmvzRpuAt01wgHQ8XBK8hXZc= 2024-04-16T18:22:20.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x3tkmvzRpuAt01wgHQ8XBK8hXZc=', 'Content-Length': '0'} 2024-04-16T18:22:20.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000143ccb07f5781eb8-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.478 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:22:20.479 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.479 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.479 DEBUG:boto:Method: GET 2024-04-16T18:22:20.479 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:22:20.479 DEBUG:boto:Data: 2024-04-16T18:22:20.479 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.479 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.479 DEBUG:boto:Params: {} 2024-04-16T18:22:20.480 DEBUG:boto:Token: None 2024-04-16T18:22:20.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85 2024-04-16T18:22:20.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FnjRFWfdcKnOhCm1uDDLlp+UXPg= 2024-04-16T18:22:20.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FnjRFWfdcKnOhCm1uDDLlp+UXPg=', 'Content-Length': '0'} 2024-04-16T18:22:20.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fd2fa870930d87f-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.526 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:22:20.526 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:22:20.527 DEBUG:boto:Method: GET 2024-04-16T18:22:20.527 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:22:20.527 DEBUG:boto:Data: 2024-04-16T18:22:20.527 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.527 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.527 DEBUG:boto:Params: {} 2024-04-16T18:22:20.527 DEBUG:boto:Token: None 2024-04-16T18:22:20.527 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj85 2024-04-16T18:22:20.527 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FnjRFWfdcKnOhCm1uDDLlp+UXPg= 2024-04-16T18:22:20.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FnjRFWfdcKnOhCm1uDDLlp+UXPg=', 'Content-Length': '0'} 2024-04-16T18:22:20.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b82085b5ddd8a67-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.530 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.531 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.531 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.531 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.531 DEBUG:boto:Method: GET 2024-04-16T18:22:20.531 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:22:20.531 DEBUG:boto:Data: 2024-04-16T18:22:20.531 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.531 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:22:20.531 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.531 DEBUG:boto:Params: {} 2024-04-16T18:22:20.531 DEBUG:boto:Token: None 2024-04-16T18:22:20.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.535 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da4bb9ce59fbd198-00661ec1dc-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.578 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.579 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.579 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.579 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.579 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:22:20.579 DEBUG:boto:Method: GET 2024-04-16T18:22:20.579 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:22:20.579 DEBUG:boto:Data: 2024-04-16T18:22:20.579 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.579 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.579 DEBUG:boto:Params: {} 2024-04-16T18:22:20.580 DEBUG:boto:Token: None 2024-04-16T18:22:20.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8= 2024-04-16T18:22:20.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L+G4/CaXEeivjuyDIHqiUupQVd8=', 'Content-Length': '0'} 2024-04-16T18:22:20.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4b2d584fef8321b-00661ec1dc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.585 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:22:20.585 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:22:20.585 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '19', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:22:20.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '19', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:22:20.585 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --num-shards 19 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:22:20.631 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:22:20.631 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:22:20.727 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:22:20.727 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-52 2024-04-16T18:22:20.740 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 10 2024-04-16T18:22:20.856 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.856 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.856 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.856 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:20.856 DEBUG:boto:Data: 2024-04-16T18:22:20.856 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.856 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.857 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.857 DEBUG:boto:Params: {} 2024-04-16T18:22:20.857 DEBUG:boto:Token: None 2024-04-16T18:22:20.857 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8= 2024-04-16T18:22:20.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8=', 'Content-Length': '0'} 2024-04-16T18:22:20.861 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cd6a4206c083cc91-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.861 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:22:20.861 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:22:20.861 DEBUG:boto:Method: PUT 2024-04-16T18:22:20.862 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:22:20.862 DEBUG:boto:Data: 2024-04-16T18:22:20.862 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:22:20.862 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.862 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.862 DEBUG:boto:Params: {} 2024-04-16T18:22:20.862 DEBUG:boto:Token: None 2024-04-16T18:22:20.862 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj19 2024-04-16T18:22:20.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:znZjVXVhloAxVuDwaVMoJ4AxYHw= 2024-04-16T18:22:20.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:znZjVXVhloAxVuDwaVMoJ4AxYHw='} 2024-04-16T18:22:20.874 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a162f06c47a375f2-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.874 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.874 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.874 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.874 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:20.874 DEBUG:boto:Data: 2024-04-16T18:22:20.874 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.874 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.875 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.875 DEBUG:boto:Params: {} 2024-04-16T18:22:20.875 DEBUG:boto:Token: None 2024-04-16T18:22:20.875 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8= 2024-04-16T18:22:20.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8=', 'Content-Length': '0'} 2024-04-16T18:22:20.878 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '33'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000055b2befebd691136-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.878 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:22:20.878 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:22:20.878 DEBUG:boto:Method: PUT 2024-04-16T18:22:20.879 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:22:20.879 DEBUG:boto:Data: 2024-04-16T18:22:20.879 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:22:20.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.879 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.879 DEBUG:boto:Params: {} 2024-04-16T18:22:20.879 DEBUG:boto:Token: None 2024-04-16T18:22:20.879 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj38 2024-04-16T18:22:20.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:duC8mhT1GqwmjQ3BGLaShiijnwo= 2024-04-16T18:22:20.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duC8mhT1GqwmjQ3BGLaShiijnwo='} 2024-04-16T18:22:20.887 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000020208f9b72e6780d-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.887 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.887 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.887 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.887 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:20.887 DEBUG:boto:Data: 2024-04-16T18:22:20.887 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.888 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.888 DEBUG:boto:Params: {} 2024-04-16T18:22:20.888 DEBUG:boto:Token: None 2024-04-16T18:22:20.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8= 2024-04-16T18:22:20.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8=', 'Content-Length': '0'} 2024-04-16T18:22:20.891 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a1684a5406312b08-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.892 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:22:20.892 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:22:20.892 DEBUG:boto:Method: PUT 2024-04-16T18:22:20.892 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:22:20.892 DEBUG:boto:Data: 2024-04-16T18:22:20.892 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:22:20.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.892 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.892 DEBUG:boto:Params: {} 2024-04-16T18:22:20.893 DEBUG:boto:Token: None 2024-04-16T18:22:20.893 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj57 2024-04-16T18:22:20.893 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mX81UB85F87noomGGAvM2G6NFsE= 2024-04-16T18:22:20.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mX81UB85F87noomGGAvM2G6NFsE='} 2024-04-16T18:22:20.938 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003c4d13dbdf7212be-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.939 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.939 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.939 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.939 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:20.939 DEBUG:boto:Data: 2024-04-16T18:22:20.939 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.939 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.939 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.939 DEBUG:boto:Params: {} 2024-04-16T18:22:20.939 DEBUG:boto:Token: None 2024-04-16T18:22:20.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8= 2024-04-16T18:22:20.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8=', 'Content-Length': '0'} 2024-04-16T18:22:20.944 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '39'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005e9b192436b459b0-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.944 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:22:20.944 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:22:20.944 DEBUG:boto:Method: PUT 2024-04-16T18:22:20.944 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:22:20.945 DEBUG:boto:Data: 2024-04-16T18:22:20.945 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:22:20.945 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.945 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.945 DEBUG:boto:Params: {} 2024-04-16T18:22:20.945 DEBUG:boto:Token: None 2024-04-16T18:22:20.945 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj76 2024-04-16T18:22:20.945 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x1ENWEnVRbanEQVKG86yDMJKd3g= 2024-04-16T18:22:20.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x1ENWEnVRbanEQVKG86yDMJKd3g='} 2024-04-16T18:22:20.958 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000db256e8480633113-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.958 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:22:20.958 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:22:20.958 DEBUG:boto:Method: HEAD 2024-04-16T18:22:20.958 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:22:20.958 DEBUG:boto:Data: 2024-04-16T18:22:20.958 DEBUG:boto:Headers: {} 2024-04-16T18:22:20.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.959 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.959 DEBUG:boto:Params: {} 2024-04-16T18:22:20.959 DEBUG:boto:Token: None 2024-04-16T18:22:20.959 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/ 2024-04-16T18:22:20.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8= 2024-04-16T18:22:20.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C71TEPncq2KRGE+FX8VWLTAxIc8=', 'Content-Length': '0'} 2024-04-16T18:22:20.963 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005da026e10431c6ab-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:20.963 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:22:20.963 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:22:20.963 DEBUG:boto:Method: PUT 2024-04-16T18:22:20.963 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:22:20.963 DEBUG:boto:Data: 2024-04-16T18:22:20.963 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:22:20.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:22:20.964 DEBUG:boto:Port: 8000 2024-04-16T18:22:20.964 DEBUG:boto:Params: {} 2024-04-16T18:22:20.964 DEBUG:boto:Token: None 2024-04-16T18:22:20.964 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:22:20 GMT /lybtug-52/obj95 2024-04-16T18:22:20.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bz+1w2Kd01z6C2jZdOL2yHGaWrw= 2024-04-16T18:22:20.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:22:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bz+1w2Kd01z6C2jZdOL2yHGaWrw='} 2024-04-16T18:22:21.007 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000059610f4f80e57dc1-00661ec1dc-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:22:21.007 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:22:21.007 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:22:21.007 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:22:21.007 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:22:21.058 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:22:21.058 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:22:21.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:21.152+0000 7fc1e9d32980 1 waiting to reach latest gen 2, on 1.. 2024-04-16T18:22:26.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:26.156+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:26.154 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:26.154 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:22:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:31.160+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:31.157 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:31.157 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:22:36.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:36.160+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:36.159 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:36.159 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:22:41.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:41.164+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:41.162 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:41.162 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:22:46.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:46.168+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:46.164 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:46.164 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:22:51.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:51.168+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:51.167 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:51.167 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:22:56.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:22:56.172+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:22:56.170 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:22:56.170 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:01.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:01.176+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:01.173 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:01.173 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:06.179+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:06.176 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:06.176 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:11.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:11.179+0000 7fc1e9d32980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:11.179 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:11.179 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:16.183+0000 7fc1e9d32980 1 bucket sync caught up with source: 2024-04-16T18:23:16.182 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:16.182 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.112.6, 00000000004.114.6, , , , , , , , , , , ] 2024-04-16T18:23:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:16.183+0000 7fc1e9d32980 0 bucket checkpoint complete 2024-04-16T18:23:16.230 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:23:16.231 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:23:16.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:23:16.231 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:23:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:23:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:23:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:23:16.368+0000 7fcc88822980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:16.367 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.632.6, 00000000001.553.6, 00000000001.470.6, , 00000000001.513.6, , , 00000000002.466.6, 00000000004.323.6, 00000000003.410.6] 2024-04-16T18:23:16.367 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.632.6, 00000000001.553.6, 00000000001.470.6, 00000000001.1217.6, 00000000001.513.6, , , 00000000002.466.6, 00000000004.323.6, 00000000003.410.6] 2024-04-16T18:23:21.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:23:21.372+0000 7fcc88822980 1 bucket sync caught up with source: 2024-04-16T18:23:21.369 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.632.6, 00000000001.553.6, 00000000001.470.6, 00000000001.1217.6, 00000000001.513.6, , , 00000000002.466.6, 00000000004.323.6, 00000000003.410.6] 2024-04-16T18:23:21.369 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.632.6, 00000000001.553.6, 00000000001.470.6, 00000000001.1217.6, 00000000001.513.6, , , 00000000002.466.6, 00000000004.323.6, 00000000003.410.6] 2024-04-16T18:23:21.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:23:21.372+0000 7fcc88822980 0 bucket checkpoint complete 2024-04-16T18:23:21.417 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:23:21.417 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.417 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.417 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.417 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:21.417 DEBUG:boto:Data: 2024-04-16T18:23:21.417 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.417 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.418 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.418 DEBUG:boto:Params: {} 2024-04-16T18:23:21.418 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:23:21.418 DEBUG:boto:Token: None 2024-04-16T18:23:21.418 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.419 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CDzyyZqHqMQBRAakD7oVFw8dymQ= 2024-04-16T18:23:21.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CDzyyZqHqMQBRAakD7oVFw8dymQ=', 'Content-Length': '0'} 2024-04-16T18:23:21.425 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bcb5c60610e36023-00661ec219-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.426 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.426 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.426 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.426 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:21.426 DEBUG:boto:Data: 2024-04-16T18:23:21.427 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.427 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.427 DEBUG:boto:Params: {} 2024-04-16T18:23:21.427 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:23:21.427 DEBUG:boto:Token: None 2024-04-16T18:23:21.428 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CDzyyZqHqMQBRAakD7oVFw8dymQ= 2024-04-16T18:23:21.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CDzyyZqHqMQBRAakD7oVFw8dymQ=', 'Content-Length': '0'} 2024-04-16T18:23:21.435 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e62c3dcb2c6c7be9-00661ec219-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.436 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:23:21.436 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.436 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.436 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.436 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.436 DEBUG:boto:Method: GET 2024-04-16T18:23:21.437 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.437 DEBUG:boto:Data: 2024-04-16T18:23:21.437 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.437 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.437 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.438 DEBUG:boto:Params: {} 2024-04-16T18:23:21.438 DEBUG:boto:Token: None 2024-04-16T18:23:21.438 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/?versions 2024-04-16T18:23:21.438 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE= 2024-04-16T18:23:21.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE=', 'Content-Length': '0'} 2024-04-16T18:23:21.442 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dad9743d895a833f-00661ec219-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.442 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:23:21.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:23:21.445 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:23:21.445 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.445 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.445 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.446 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.446 DEBUG:boto:Method: GET 2024-04-16T18:23:21.446 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.446 DEBUG:boto:Data: 2024-04-16T18:23:21.446 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.446 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.446 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.447 DEBUG:boto:Params: {} 2024-04-16T18:23:21.447 DEBUG:boto:Token: None 2024-04-16T18:23:21.447 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/?versions 2024-04-16T18:23:21.447 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE= 2024-04-16T18:23:21.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE=', 'Content-Length': '0'} 2024-04-16T18:23:21.451 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009582772afab3d854-00661ec219-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.451 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:23:21.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:23:21.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:23:21.453 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.453 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.453 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.453 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.453 DEBUG:boto:Method: GET 2024-04-16T18:23:21.454 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.454 DEBUG:boto:Data: 2024-04-16T18:23:21.454 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.454 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.454 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.454 DEBUG:boto:Params: {} 2024-04-16T18:23:21.454 DEBUG:boto:Token: None 2024-04-16T18:23:21.454 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/?versions 2024-04-16T18:23:21.454 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE= 2024-04-16T18:23:21.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE=', 'Content-Length': '0'} 2024-04-16T18:23:21.458 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c476dda61dcb5dfe-00661ec219-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.459 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.460 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.460 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.460 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.460 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.460 DEBUG:boto:Method: GET 2024-04-16T18:23:21.460 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:23:21.460 DEBUG:boto:Data: 2024-04-16T18:23:21.460 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.460 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.461 DEBUG:boto:Params: {} 2024-04-16T18:23:21.461 DEBUG:boto:Token: None 2024-04-16T18:23:21.461 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/?versions 2024-04-16T18:23:21.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE= 2024-04-16T18:23:21.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PK98osXqXH7WMIKV0WfHxoalFiE=', 'Content-Length': '0'} 2024-04-16T18:23:21.465 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4d2b64dd8360756-00661ec219-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.465 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:23:21.467 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.467 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.467 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.467 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.467 DEBUG:boto:Method: GET 2024-04-16T18:23:21.467 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.467 DEBUG:boto:Data: 2024-04-16T18:23:21.467 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.467 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.467 DEBUG:boto:Params: {} 2024-04-16T18:23:21.468 DEBUG:boto:Token: None 2024-04-16T18:23:21.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs= 2024-04-16T18:23:21.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs=', 'Content-Length': '0'} 2024-04-16T18:23:21.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1194dda842a0318-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.471 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.472 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.472 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.472 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.472 DEBUG:boto:Method: GET 2024-04-16T18:23:21.472 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.472 DEBUG:boto:Data: 2024-04-16T18:23:21.472 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.472 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.472 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.472 DEBUG:boto:Params: {} 2024-04-16T18:23:21.473 DEBUG:boto:Token: None 2024-04-16T18:23:21.473 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.473 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs= 2024-04-16T18:23:21.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs=', 'Content-Length': '0'} 2024-04-16T18:23:21.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae2f2ff87f32c730-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.518 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.518 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.519 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.519 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.519 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.519 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.519 DEBUG:boto:Data: 2024-04-16T18:23:21.519 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.519 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.519 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.519 DEBUG:boto:Params: {} 2024-04-16T18:23:21.519 DEBUG:boto:Token: None 2024-04-16T18:23:21.519 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wXCKsKfWwohpQLvKov/f3M6J6m0= 2024-04-16T18:23:21.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wXCKsKfWwohpQLvKov/f3M6J6m0=', 'Content-Length': '0'} 2024-04-16T18:23:21.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039536a3d417aec60-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.523 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.523 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.523 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.523 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.523 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.524 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.524 DEBUG:boto:Data: 2024-04-16T18:23:21.524 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.524 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.524 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.524 DEBUG:boto:Params: {} 2024-04-16T18:23:21.524 DEBUG:boto:Token: None 2024-04-16T18:23:21.524 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wXCKsKfWwohpQLvKov/f3M6J6m0= 2024-04-16T18:23:21.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wXCKsKfWwohpQLvKov/f3M6J6m0=', 'Content-Length': '0'} 2024-04-16T18:23:21.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adaef8f23dfa8b4b-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.528 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:23:21.528 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.528 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.528 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.528 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.529 DEBUG:boto:Method: GET 2024-04-16T18:23:21.529 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.529 DEBUG:boto:Data: 2024-04-16T18:23:21.529 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.529 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.529 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.529 DEBUG:boto:Params: {} 2024-04-16T18:23:21.529 DEBUG:boto:Token: None 2024-04-16T18:23:21.529 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.529 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs= 2024-04-16T18:23:21.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs=', 'Content-Length': '0'} 2024-04-16T18:23:21.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066ada0b3076dbe4e-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.574 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.574 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.575 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.575 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.575 DEBUG:boto:Method: GET 2024-04-16T18:23:21.575 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.575 DEBUG:boto:Data: 2024-04-16T18:23:21.575 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.575 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.575 DEBUG:boto:Params: {} 2024-04-16T18:23:21.575 DEBUG:boto:Token: None 2024-04-16T18:23:21.575 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:23:21.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs= 2024-04-16T18:23:21.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XL3hYhIt33X6olz0frtIOs0Ytrs=', 'Content-Length': '0'} 2024-04-16T18:23:21.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023469694d1762be7-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.579 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.579 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.579 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.579 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.579 DEBUG:boto:Method: GET 2024-04-16T18:23:21.579 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:23:21.579 DEBUG:boto:Data: 2024-04-16T18:23:21.580 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.580 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.580 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.580 DEBUG:boto:Params: {} 2024-04-16T18:23:21.580 DEBUG:boto:Token: None 2024-04-16T18:23:21.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059f1c94ab9e6f389-00661ec219-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.584 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.585 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.585 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.585 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.585 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.585 DEBUG:boto:Method: GET 2024-04-16T18:23:21.585 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:23:21.585 DEBUG:boto:Data: 2024-04-16T18:23:21.585 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.585 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.585 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.585 DEBUG:boto:Params: {} 2024-04-16T18:23:21.586 DEBUG:boto:Token: None 2024-04-16T18:23:21.586 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.586 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2af40bad4cb2b87-00661ec219-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.590 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.590 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.591 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.591 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.591 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:23:21.591 DEBUG:boto:Data: 2024-04-16T18:23:21.591 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.591 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.591 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.591 DEBUG:boto:Params: {} 2024-04-16T18:23:21.591 DEBUG:boto:Token: None 2024-04-16T18:23:21.591 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11 2024-04-16T18:23:21.591 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:piNoTq9vNyEdVfScFo09HLW6QQ0= 2024-04-16T18:23:21.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:piNoTq9vNyEdVfScFo09HLW6QQ0=', 'Content-Length': '0'} 2024-04-16T18:23:21.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b6331436d0b8ff-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.595 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.595 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.595 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.595 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:23:21.595 DEBUG:boto:Data: 2024-04-16T18:23:21.595 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.595 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.595 DEBUG:boto:Params: {} 2024-04-16T18:23:21.596 DEBUG:boto:Token: None 2024-04-16T18:23:21.596 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11 2024-04-16T18:23:21.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:piNoTq9vNyEdVfScFo09HLW6QQ0= 2024-04-16T18:23:21.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:piNoTq9vNyEdVfScFo09HLW6QQ0=', 'Content-Length': '0'} 2024-04-16T18:23:21.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034a398d0ee7dd8b4-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.599 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:23:21.599 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.599 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.599 DEBUG:boto:Method: GET 2024-04-16T18:23:21.599 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:23:21.600 DEBUG:boto:Data: 2024-04-16T18:23:21.600 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.600 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.600 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.600 DEBUG:boto:Params: {} 2024-04-16T18:23:21.600 DEBUG:boto:Token: None 2024-04-16T18:23:21.600 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11 2024-04-16T18:23:21.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o4DjNOUjJFkwXnJOEtBjGTOkePo= 2024-04-16T18:23:21.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o4DjNOUjJFkwXnJOEtBjGTOkePo=', 'Content-Length': '0'} 2024-04-16T18:23:21.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000536b8bcea5a2715a-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.646 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:23:21.646 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:23:21.647 DEBUG:boto:Method: GET 2024-04-16T18:23:21.647 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:23:21.647 DEBUG:boto:Data: 2024-04-16T18:23:21.647 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.647 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.647 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.647 DEBUG:boto:Params: {} 2024-04-16T18:23:21.647 DEBUG:boto:Token: None 2024-04-16T18:23:21.647 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj11 2024-04-16T18:23:21.647 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o4DjNOUjJFkwXnJOEtBjGTOkePo= 2024-04-16T18:23:21.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o4DjNOUjJFkwXnJOEtBjGTOkePo=', 'Content-Length': '0'} 2024-04-16T18:23:21.650 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071d8712caa7c3961-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.651 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.651 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.651 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.651 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.651 DEBUG:boto:Method: GET 2024-04-16T18:23:21.651 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:23:21.651 DEBUG:boto:Data: 2024-04-16T18:23:21.651 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.651 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.652 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.652 DEBUG:boto:Params: {} 2024-04-16T18:23:21.652 DEBUG:boto:Token: None 2024-04-16T18:23:21.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.655 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbd3d63851de3765-00661ec219-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.656 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.656 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.656 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.656 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.656 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:23:21.656 DEBUG:boto:Method: GET 2024-04-16T18:23:21.657 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:23:21.657 DEBUG:boto:Data: 2024-04-16T18:23:21.657 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.657 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.657 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.657 DEBUG:boto:Params: {} 2024-04-16T18:23:21.657 DEBUG:boto:Token: None 2024-04-16T18:23:21.657 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.662 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002aab11c35f8adc13-00661ec219-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.702 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.703 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:23:21.703 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.703 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.703 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.703 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.703 DEBUG:boto:Method: GET 2024-04-16T18:23:21.703 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.703 DEBUG:boto:Data: 2024-04-16T18:23:21.703 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.703 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.703 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.704 DEBUG:boto:Params: {} 2024-04-16T18:23:21.704 DEBUG:boto:Token: None 2024-04-16T18:23:21.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo= 2024-04-16T18:23:21.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo=', 'Content-Length': '0'} 2024-04-16T18:23:21.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c987a8552c5d86c6-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.750 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.750 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.751 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.751 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.751 DEBUG:boto:Method: GET 2024-04-16T18:23:21.751 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.751 DEBUG:boto:Data: 2024-04-16T18:23:21.751 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.751 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.751 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.751 DEBUG:boto:Params: {} 2024-04-16T18:23:21.751 DEBUG:boto:Token: None 2024-04-16T18:23:21.751 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo= 2024-04-16T18:23:21.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo=', 'Content-Length': '0'} 2024-04-16T18:23:21.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ece234883a4ffc3-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.755 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.755 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.755 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.755 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.755 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.755 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.756 DEBUG:boto:Data: 2024-04-16T18:23:21.756 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.756 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.756 DEBUG:boto:Params: {} 2024-04-16T18:23:21.756 DEBUG:boto:Token: None 2024-04-16T18:23:21.756 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z9cTdSBZNr0doqHOWSM+8DpJh7I= 2024-04-16T18:23:21.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z9cTdSBZNr0doqHOWSM+8DpJh7I=', 'Content-Length': '0'} 2024-04-16T18:23:21.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9bd0c90ce164e90-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.759 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.759 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.759 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.760 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.760 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.760 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.760 DEBUG:boto:Data: 2024-04-16T18:23:21.760 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.760 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.760 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.760 DEBUG:boto:Params: {} 2024-04-16T18:23:21.760 DEBUG:boto:Token: None 2024-04-16T18:23:21.760 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z9cTdSBZNr0doqHOWSM+8DpJh7I= 2024-04-16T18:23:21.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z9cTdSBZNr0doqHOWSM+8DpJh7I=', 'Content-Length': '0'} 2024-04-16T18:23:21.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb4fbcd052aec7a0-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:23:21.764 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.764 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.764 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.764 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.764 DEBUG:boto:Method: GET 2024-04-16T18:23:21.765 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.765 DEBUG:boto:Data: 2024-04-16T18:23:21.765 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.765 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.765 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.765 DEBUG:boto:Params: {} 2024-04-16T18:23:21.765 DEBUG:boto:Token: None 2024-04-16T18:23:21.765 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.765 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo= 2024-04-16T18:23:21.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo=', 'Content-Length': '0'} 2024-04-16T18:23:21.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bc4e7be29828b3c-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.810 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.810 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.811 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.811 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.811 DEBUG:boto:Method: GET 2024-04-16T18:23:21.811 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.811 DEBUG:boto:Data: 2024-04-16T18:23:21.811 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.811 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.811 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.811 DEBUG:boto:Params: {} 2024-04-16T18:23:21.811 DEBUG:boto:Token: None 2024-04-16T18:23:21.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:23:21.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo= 2024-04-16T18:23:21.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:efAnbVa1xZ8ow5Cs43vuDK3Zdjo=', 'Content-Length': '0'} 2024-04-16T18:23:21.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6e44e3172055bb8-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.816 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.816 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.816 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.816 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.816 DEBUG:boto:Method: GET 2024-04-16T18:23:21.816 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:23:21.816 DEBUG:boto:Data: 2024-04-16T18:23:21.816 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.816 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.817 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.817 DEBUG:boto:Params: {} 2024-04-16T18:23:21.817 DEBUG:boto:Token: None 2024-04-16T18:23:21.817 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.817 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.821 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032f393f8df21e889-00661ec219-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.821 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.821 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.821 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.821 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.821 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.822 DEBUG:boto:Method: GET 2024-04-16T18:23:21.822 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:23:21.822 DEBUG:boto:Data: 2024-04-16T18:23:21.822 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.822 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.822 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.822 DEBUG:boto:Params: {} 2024-04-16T18:23:21.822 DEBUG:boto:Token: None 2024-04-16T18:23:21.822 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.823 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003295d4394aec4e7-00661ec219-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.828 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.828 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.828 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.828 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.828 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:23:21.828 DEBUG:boto:Data: 2024-04-16T18:23:21.828 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.828 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.829 DEBUG:boto:Params: {} 2024-04-16T18:23:21.829 DEBUG:boto:Token: None 2024-04-16T18:23:21.829 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17 2024-04-16T18:23:21.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OoXcqE6WZQZUPlqrb359HbgduPM= 2024-04-16T18:23:21.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OoXcqE6WZQZUPlqrb359HbgduPM=', 'Content-Length': '0'} 2024-04-16T18:23:21.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a81b7fe9aa6a2f9e-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.832 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.832 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.832 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.832 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:23:21.832 DEBUG:boto:Data: 2024-04-16T18:23:21.832 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.832 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.832 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.832 DEBUG:boto:Params: {} 2024-04-16T18:23:21.833 DEBUG:boto:Token: None 2024-04-16T18:23:21.833 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17 2024-04-16T18:23:21.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OoXcqE6WZQZUPlqrb359HbgduPM= 2024-04-16T18:23:21.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OoXcqE6WZQZUPlqrb359HbgduPM=', 'Content-Length': '0'} 2024-04-16T18:23:21.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a73c79b749c68ca0-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.836 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:23:21.836 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.836 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.836 DEBUG:boto:Method: GET 2024-04-16T18:23:21.836 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:23:21.836 DEBUG:boto:Data: 2024-04-16T18:23:21.837 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.837 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.837 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.837 DEBUG:boto:Params: {} 2024-04-16T18:23:21.837 DEBUG:boto:Token: None 2024-04-16T18:23:21.837 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17 2024-04-16T18:23:21.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vM/TjwYXgj0RTNhkH7bExFyIer8= 2024-04-16T18:23:21.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vM/TjwYXgj0RTNhkH7bExFyIer8=', 'Content-Length': '0'} 2024-04-16T18:23:21.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049eac72281c589e3-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.882 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:23:21.882 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:23:21.883 DEBUG:boto:Method: GET 2024-04-16T18:23:21.883 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:23:21.883 DEBUG:boto:Data: 2024-04-16T18:23:21.883 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.883 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.883 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.883 DEBUG:boto:Params: {} 2024-04-16T18:23:21.883 DEBUG:boto:Token: None 2024-04-16T18:23:21.883 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj17 2024-04-16T18:23:21.883 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vM/TjwYXgj0RTNhkH7bExFyIer8= 2024-04-16T18:23:21.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vM/TjwYXgj0RTNhkH7bExFyIer8=', 'Content-Length': '0'} 2024-04-16T18:23:21.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004003003f09ff9777-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.887 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.887 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.887 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.887 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.887 DEBUG:boto:Method: GET 2024-04-16T18:23:21.887 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:23:21.887 DEBUG:boto:Data: 2024-04-16T18:23:21.887 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.887 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.887 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.887 DEBUG:boto:Params: {} 2024-04-16T18:23:21.888 DEBUG:boto:Token: None 2024-04-16T18:23:21.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.891 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0a1898f3818b2b3-00661ec219-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.891 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.892 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:21.892 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:21.892 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.892 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:23:21.892 DEBUG:boto:Method: GET 2024-04-16T18:23:21.892 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:23:21.892 DEBUG:boto:Data: 2024-04-16T18:23:21.892 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.892 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.892 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.892 DEBUG:boto:Params: {} 2024-04-16T18:23:21.892 DEBUG:boto:Token: None 2024-04-16T18:23:21.893 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/ 2024-04-16T18:23:21.893 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU= 2024-04-16T18:23:21.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJiZ6lkfbU51XOf5m0JzxGTvfaU=', 'Content-Length': '0'} 2024-04-16T18:23:21.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7d7749044f12191-00661ec219-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.897 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:21.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:23:21.898 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:21.898 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:21.898 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.898 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.898 DEBUG:boto:Method: GET 2024-04-16T18:23:21.898 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.898 DEBUG:boto:Data: 2024-04-16T18:23:21.898 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.898 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.898 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.898 DEBUG:boto:Params: {} 2024-04-16T18:23:21.899 DEBUG:boto:Token: None 2024-04-16T18:23:21.899 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.899 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9skY48BD7Xv9QxQv0FMhNOkSJs= 2024-04-16T18:23:21.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9skY48BD7Xv9QxQv0FMhNOkSJs=', 'Content-Length': '0'} 2024-04-16T18:23:21.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3e1b03c7c6fb348-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.942 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:21.942 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:21.943 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.943 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.943 DEBUG:boto:Method: GET 2024-04-16T18:23:21.943 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.943 DEBUG:boto:Data: 2024-04-16T18:23:21.943 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.943 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.943 DEBUG:boto:Params: {} 2024-04-16T18:23:21.943 DEBUG:boto:Token: None 2024-04-16T18:23:21.944 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9skY48BD7Xv9QxQv0FMhNOkSJs= 2024-04-16T18:23:21.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9skY48BD7Xv9QxQv0FMhNOkSJs=', 'Content-Length': '0'} 2024-04-16T18:23:21.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0a4d8df6aa4d814-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.948 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:21.948 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:21.948 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.948 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.948 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.948 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.948 DEBUG:boto:Data: 2024-04-16T18:23:21.948 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.948 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.949 DEBUG:boto:Params: {} 2024-04-16T18:23:21.949 DEBUG:boto:Token: None 2024-04-16T18:23:21.949 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.949 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ieOXUxXuHQkup6ubhVSyjGbj31A= 2024-04-16T18:23:21.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ieOXUxXuHQkup6ubhVSyjGbj31A=', 'Content-Length': '0'} 2024-04-16T18:23:21.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7e6b11b55536056-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.952 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:21.952 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:21.952 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.952 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.952 DEBUG:boto:Method: HEAD 2024-04-16T18:23:21.952 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.953 DEBUG:boto:Data: 2024-04-16T18:23:21.953 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.953 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:21.953 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.953 DEBUG:boto:Params: {} 2024-04-16T18:23:21.953 DEBUG:boto:Token: None 2024-04-16T18:23:21.953 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ieOXUxXuHQkup6ubhVSyjGbj31A= 2024-04-16T18:23:21.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ieOXUxXuHQkup6ubhVSyjGbj31A=', 'Content-Length': '0'} 2024-04-16T18:23:21.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad1fca56cfbdc0eb-00661ec219-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:21.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:23:21.956 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:21.957 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:21.957 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.957 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.957 DEBUG:boto:Method: GET 2024-04-16T18:23:21.957 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.957 DEBUG:boto:Data: 2024-04-16T18:23:21.957 DEBUG:boto:Headers: {} 2024-04-16T18:23:21.957 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:21.957 DEBUG:boto:Port: 8000 2024-04-16T18:23:21.957 DEBUG:boto:Params: {} 2024-04-16T18:23:21.957 DEBUG:boto:Token: None 2024-04-16T18:23:21.957 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:21 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:23:21.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9skY48BD7Xv9QxQv0FMhNOkSJs= 2024-04-16T18:23:21.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9skY48BD7Xv9QxQv0FMhNOkSJs=', 'Content-Length': '0'} 2024-04-16T18:23:21.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004e72d6ab710e6f6-00661ec219-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.006 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:22.006 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:22.007 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:22.007 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:23:22.007 DEBUG:boto:Method: GET 2024-04-16T18:23:22.007 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:23:22.007 DEBUG:boto:Data: 2024-04-16T18:23:22.007 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.007 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.008 DEBUG:boto:Params: {} 2024-04-16T18:23:22.008 DEBUG:boto:Token: None 2024-04-16T18:23:22.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:23:22.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:izvkhqhS/DeM9IRa7dxIETJIrsc= 2024-04-16T18:23:22.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:izvkhqhS/DeM9IRa7dxIETJIrsc=', 'Content-Length': '0'} 2024-04-16T18:23:22.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000632d9ed283774e52-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.012 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.012 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.012 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.012 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.013 DEBUG:boto:Method: GET 2024-04-16T18:23:22.013 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:23:22.013 DEBUG:boto:Data: 2024-04-16T18:23:22.013 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.013 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.013 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.013 DEBUG:boto:Params: {} 2024-04-16T18:23:22.013 DEBUG:boto:Token: None 2024-04-16T18:23:22.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.014 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.018 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ff3070658da26dd-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.018 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.018 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.019 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.019 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.019 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.019 DEBUG:boto:Method: GET 2024-04-16T18:23:22.019 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:23:22.019 DEBUG:boto:Data: 2024-04-16T18:23:22.019 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.019 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.019 DEBUG:boto:Params: {} 2024-04-16T18:23:22.020 DEBUG:boto:Token: None 2024-04-16T18:23:22.020 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006fdad62d50dceca-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.066 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.067 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:22.067 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:22.067 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.067 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:23:22.067 DEBUG:boto:Data: 2024-04-16T18:23:22.068 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.068 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.068 DEBUG:boto:Params: {} 2024-04-16T18:23:22.068 DEBUG:boto:Token: None 2024-04-16T18:23:22.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj19 2024-04-16T18:23:22.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TJ0xCYFrx5WKpqTM9RXsfEgi+lM= 2024-04-16T18:23:22.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TJ0xCYFrx5WKpqTM9RXsfEgi+lM=', 'Content-Length': '0'} 2024-04-16T18:23:22.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbe08ce13702eb96-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.072 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:22.072 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:22.072 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.072 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:23:22.072 DEBUG:boto:Data: 2024-04-16T18:23:22.073 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.073 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.073 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.073 DEBUG:boto:Params: {} 2024-04-16T18:23:22.073 DEBUG:boto:Token: None 2024-04-16T18:23:22.073 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj19 2024-04-16T18:23:22.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TJ0xCYFrx5WKpqTM9RXsfEgi+lM= 2024-04-16T18:23:22.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TJ0xCYFrx5WKpqTM9RXsfEgi+lM=', 'Content-Length': '0'} 2024-04-16T18:23:22.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096084296187866cb-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:23:22.077 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:22.077 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:22.077 DEBUG:boto:Method: GET 2024-04-16T18:23:22.077 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:23:22.077 DEBUG:boto:Data: 2024-04-16T18:23:22.077 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.077 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.077 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.077 DEBUG:boto:Params: {} 2024-04-16T18:23:22.078 DEBUG:boto:Token: None 2024-04-16T18:23:22.078 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj19 2024-04-16T18:23:22.078 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v8fxG2mdYNuDnPLGTeF7XvM5ivw= 2024-04-16T18:23:22.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v8fxG2mdYNuDnPLGTeF7XvM5ivw=', 'Content-Length': '0'} 2024-04-16T18:23:22.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002cb480fd4a3077e-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.126 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:23:22.126 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:23:22.127 DEBUG:boto:Method: GET 2024-04-16T18:23:22.127 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:23:22.127 DEBUG:boto:Data: 2024-04-16T18:23:22.127 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.127 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.127 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.127 DEBUG:boto:Params: {} 2024-04-16T18:23:22.127 DEBUG:boto:Token: None 2024-04-16T18:23:22.127 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj19 2024-04-16T18:23:22.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v8fxG2mdYNuDnPLGTeF7XvM5ivw= 2024-04-16T18:23:22.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v8fxG2mdYNuDnPLGTeF7XvM5ivw=', 'Content-Length': '0'} 2024-04-16T18:23:22.130 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7b087ffaea2bdbb-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.131 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.131 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.131 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.131 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.131 DEBUG:boto:Method: GET 2024-04-16T18:23:22.131 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:23:22.131 DEBUG:boto:Data: 2024-04-16T18:23:22.131 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.132 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.132 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.132 DEBUG:boto:Params: {} 2024-04-16T18:23:22.132 DEBUG:boto:Token: None 2024-04-16T18:23:22.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.136 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056dfd4d4a69de447-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.178 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.179 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.179 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.179 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.179 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:23:22.179 DEBUG:boto:Method: GET 2024-04-16T18:23:22.179 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:23:22.179 DEBUG:boto:Data: 2024-04-16T18:23:22.179 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.179 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.180 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.180 DEBUG:boto:Params: {} 2024-04-16T18:23:22.180 DEBUG:boto:Token: None 2024-04-16T18:23:22.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.185 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009dfe5caed4f6c530-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.185 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.186 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:23:22.186 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.186 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.186 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.186 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.186 DEBUG:boto:Method: GET 2024-04-16T18:23:22.186 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.186 DEBUG:boto:Data: 2024-04-16T18:23:22.186 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.186 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.187 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.187 DEBUG:boto:Params: {} 2024-04-16T18:23:22.187 DEBUG:boto:Token: None 2024-04-16T18:23:22.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc= 2024-04-16T18:23:22.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc=', 'Content-Length': '0'} 2024-04-16T18:23:22.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aefadf8e74ed4b5d-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.230 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.231 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.231 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.231 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.231 DEBUG:boto:Method: GET 2024-04-16T18:23:22.231 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.231 DEBUG:boto:Data: 2024-04-16T18:23:22.231 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.231 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.231 DEBUG:boto:Params: {} 2024-04-16T18:23:22.231 DEBUG:boto:Token: None 2024-04-16T18:23:22.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc= 2024-04-16T18:23:22.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc=', 'Content-Length': '0'} 2024-04-16T18:23:22.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c8e6e72c3b23c30-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.235 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.235 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.235 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.235 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.235 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.235 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.235 DEBUG:boto:Data: 2024-04-16T18:23:22.236 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.236 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.236 DEBUG:boto:Params: {} 2024-04-16T18:23:22.236 DEBUG:boto:Token: None 2024-04-16T18:23:22.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:INxb8HpP73OV/zSLEMaak6p/eZU= 2024-04-16T18:23:22.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:INxb8HpP73OV/zSLEMaak6p/eZU=', 'Content-Length': '0'} 2024-04-16T18:23:22.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000758c2e6b36482e64-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.240 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.240 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.240 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.240 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.240 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.240 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.240 DEBUG:boto:Data: 2024-04-16T18:23:22.240 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.240 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.240 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.240 DEBUG:boto:Params: {} 2024-04-16T18:23:22.241 DEBUG:boto:Token: None 2024-04-16T18:23:22.241 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:INxb8HpP73OV/zSLEMaak6p/eZU= 2024-04-16T18:23:22.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:INxb8HpP73OV/zSLEMaak6p/eZU=', 'Content-Length': '0'} 2024-04-16T18:23:22.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019f39d5aa9f16993-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.244 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:23:22.244 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.244 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.244 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.244 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.245 DEBUG:boto:Method: GET 2024-04-16T18:23:22.245 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.245 DEBUG:boto:Data: 2024-04-16T18:23:22.245 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.245 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.245 DEBUG:boto:Params: {} 2024-04-16T18:23:22.245 DEBUG:boto:Token: None 2024-04-16T18:23:22.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc= 2024-04-16T18:23:22.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc=', 'Content-Length': '0'} 2024-04-16T18:23:22.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4210bf2daae47bb-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.290 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.290 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.291 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.291 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.291 DEBUG:boto:Method: GET 2024-04-16T18:23:22.291 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.291 DEBUG:boto:Data: 2024-04-16T18:23:22.291 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.291 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.291 DEBUG:boto:Params: {} 2024-04-16T18:23:22.291 DEBUG:boto:Token: None 2024-04-16T18:23:22.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:23:22.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc= 2024-04-16T18:23:22.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pkxRQBF2QjlMc/MkGHfUDEziFWc=', 'Content-Length': '0'} 2024-04-16T18:23:22.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce2342bca15b23eb-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.295 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.295 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.295 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.295 DEBUG:boto:Method: GET 2024-04-16T18:23:22.295 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:23:22.295 DEBUG:boto:Data: 2024-04-16T18:23:22.295 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.295 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.296 DEBUG:boto:Params: {} 2024-04-16T18:23:22.296 DEBUG:boto:Token: None 2024-04-16T18:23:22.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000294158e0dd5f81a7-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.300 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.301 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.301 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.301 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.301 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.301 DEBUG:boto:Method: GET 2024-04-16T18:23:22.301 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:23:22.301 DEBUG:boto:Data: 2024-04-16T18:23:22.301 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.301 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.301 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.301 DEBUG:boto:Params: {} 2024-04-16T18:23:22.302 DEBUG:boto:Token: None 2024-04-16T18:23:22.302 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.306 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ea48bea4f2b3b6e-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.346 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.347 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.347 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.347 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.347 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:23:22.347 DEBUG:boto:Data: 2024-04-16T18:23:22.347 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.347 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.347 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.347 DEBUG:boto:Params: {} 2024-04-16T18:23:22.348 DEBUG:boto:Token: None 2024-04-16T18:23:22.348 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22 2024-04-16T18:23:22.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3MaW96mzzoHIBE++s80HsBNXTbk= 2024-04-16T18:23:22.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3MaW96mzzoHIBE++s80HsBNXTbk=', 'Content-Length': '0'} 2024-04-16T18:23:22.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e766956a622bb84-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.351 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.351 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.351 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.351 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:23:22.351 DEBUG:boto:Data: 2024-04-16T18:23:22.351 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.351 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.352 DEBUG:boto:Params: {} 2024-04-16T18:23:22.352 DEBUG:boto:Token: None 2024-04-16T18:23:22.352 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22 2024-04-16T18:23:22.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3MaW96mzzoHIBE++s80HsBNXTbk= 2024-04-16T18:23:22.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3MaW96mzzoHIBE++s80HsBNXTbk=', 'Content-Length': '0'} 2024-04-16T18:23:22.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3361c555e26642b-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.355 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:23:22.355 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.355 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.355 DEBUG:boto:Method: GET 2024-04-16T18:23:22.356 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:23:22.356 DEBUG:boto:Data: 2024-04-16T18:23:22.356 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.356 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.356 DEBUG:boto:Params: {} 2024-04-16T18:23:22.356 DEBUG:boto:Token: None 2024-04-16T18:23:22.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22 2024-04-16T18:23:22.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+RYJ1frEksHQDU7vtL2owvoXPw4= 2024-04-16T18:23:22.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+RYJ1frEksHQDU7vtL2owvoXPw4=', 'Content-Length': '0'} 2024-04-16T18:23:22.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017340ae3569de9b6-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.402 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:23:22.402 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:23:22.403 DEBUG:boto:Method: GET 2024-04-16T18:23:22.403 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:23:22.403 DEBUG:boto:Data: 2024-04-16T18:23:22.403 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.403 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.403 DEBUG:boto:Params: {} 2024-04-16T18:23:22.403 DEBUG:boto:Token: None 2024-04-16T18:23:22.403 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj22 2024-04-16T18:23:22.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+RYJ1frEksHQDU7vtL2owvoXPw4= 2024-04-16T18:23:22.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+RYJ1frEksHQDU7vtL2owvoXPw4=', 'Content-Length': '0'} 2024-04-16T18:23:22.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003031b7eddf7ea987-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.406 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.407 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.407 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.407 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.407 DEBUG:boto:Method: GET 2024-04-16T18:23:22.407 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:23:22.407 DEBUG:boto:Data: 2024-04-16T18:23:22.407 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.407 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.407 DEBUG:boto:Params: {} 2024-04-16T18:23:22.407 DEBUG:boto:Token: None 2024-04-16T18:23:22.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d72d1b80c820637e-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.454 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.455 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.455 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.455 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.455 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:23:22.455 DEBUG:boto:Method: GET 2024-04-16T18:23:22.455 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:23:22.455 DEBUG:boto:Data: 2024-04-16T18:23:22.455 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.455 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.456 DEBUG:boto:Params: {} 2024-04-16T18:23:22.456 DEBUG:boto:Token: None 2024-04-16T18:23:22.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.461 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a63b01843eee44e-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.461 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.461 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:23:22.461 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.461 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.461 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.462 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.462 DEBUG:boto:Method: GET 2024-04-16T18:23:22.462 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.462 DEBUG:boto:Data: 2024-04-16T18:23:22.462 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.462 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.462 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.462 DEBUG:boto:Params: {} 2024-04-16T18:23:22.462 DEBUG:boto:Token: None 2024-04-16T18:23:22.462 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.463 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc= 2024-04-16T18:23:22.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc=', 'Content-Length': '0'} 2024-04-16T18:23:22.465 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0e85f724c06955f-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.506 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.507 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.507 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.507 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.507 DEBUG:boto:Method: GET 2024-04-16T18:23:22.507 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.507 DEBUG:boto:Data: 2024-04-16T18:23:22.507 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.507 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.507 DEBUG:boto:Params: {} 2024-04-16T18:23:22.507 DEBUG:boto:Token: None 2024-04-16T18:23:22.508 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc= 2024-04-16T18:23:22.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc=', 'Content-Length': '0'} 2024-04-16T18:23:22.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ff4a1dc0b8b1214-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.511 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.511 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.511 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.511 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.511 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.511 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.511 DEBUG:boto:Data: 2024-04-16T18:23:22.512 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.512 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.512 DEBUG:boto:Params: {} 2024-04-16T18:23:22.512 DEBUG:boto:Token: None 2024-04-16T18:23:22.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fpSi/lLnuPUBUPbUpaPiB1+CgCk= 2024-04-16T18:23:22.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fpSi/lLnuPUBUPbUpaPiB1+CgCk=', 'Content-Length': '0'} 2024-04-16T18:23:22.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c60922f1eb8eefd-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.515 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.515 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.515 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.515 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.515 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.515 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.516 DEBUG:boto:Data: 2024-04-16T18:23:22.516 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.516 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.516 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.516 DEBUG:boto:Params: {} 2024-04-16T18:23:22.516 DEBUG:boto:Token: None 2024-04-16T18:23:22.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fpSi/lLnuPUBUPbUpaPiB1+CgCk= 2024-04-16T18:23:22.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fpSi/lLnuPUBUPbUpaPiB1+CgCk=', 'Content-Length': '0'} 2024-04-16T18:23:22.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005505119bab7b0740-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.519 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:23:22.519 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.519 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.519 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.520 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.520 DEBUG:boto:Method: GET 2024-04-16T18:23:22.520 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.520 DEBUG:boto:Data: 2024-04-16T18:23:22.520 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.520 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.520 DEBUG:boto:Params: {} 2024-04-16T18:23:22.520 DEBUG:boto:Token: None 2024-04-16T18:23:22.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc= 2024-04-16T18:23:22.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc=', 'Content-Length': '0'} 2024-04-16T18:23:22.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004357c58d61778d6d-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.566 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.566 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.566 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.567 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.567 DEBUG:boto:Method: GET 2024-04-16T18:23:22.567 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.567 DEBUG:boto:Data: 2024-04-16T18:23:22.567 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.567 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.567 DEBUG:boto:Params: {} 2024-04-16T18:23:22.567 DEBUG:boto:Token: None 2024-04-16T18:23:22.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:23:22.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc= 2024-04-16T18:23:22.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pfXDJh6AOJJ5me/3UiuqCt6Rrc=', 'Content-Length': '0'} 2024-04-16T18:23:22.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067c16fb27a1453c4-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.571 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.571 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.571 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.572 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.572 DEBUG:boto:Method: GET 2024-04-16T18:23:22.572 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:23:22.572 DEBUG:boto:Data: 2024-04-16T18:23:22.572 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.572 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.572 DEBUG:boto:Params: {} 2024-04-16T18:23:22.572 DEBUG:boto:Token: None 2024-04-16T18:23:22.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fbb0a9a227e304c7-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.576 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.576 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.577 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.577 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.577 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.577 DEBUG:boto:Method: GET 2024-04-16T18:23:22.577 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:23:22.577 DEBUG:boto:Data: 2024-04-16T18:23:22.577 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.577 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.577 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.577 DEBUG:boto:Params: {} 2024-04-16T18:23:22.577 DEBUG:boto:Token: None 2024-04-16T18:23:22.578 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.582 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095b6a8793f0544bb-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.626 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.627 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.627 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.627 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.627 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:23:22.627 DEBUG:boto:Data: 2024-04-16T18:23:22.627 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.627 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.627 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.628 DEBUG:boto:Params: {} 2024-04-16T18:23:22.628 DEBUG:boto:Token: None 2024-04-16T18:23:22.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33 2024-04-16T18:23:22.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NtjlmU72FcQWbGJdB7YYhl5mVlM= 2024-04-16T18:23:22.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NtjlmU72FcQWbGJdB7YYhl5mVlM=', 'Content-Length': '0'} 2024-04-16T18:23:22.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d0d3e2c8ca71a96-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.631 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.631 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.631 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.631 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:23:22.631 DEBUG:boto:Data: 2024-04-16T18:23:22.632 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.632 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.632 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.632 DEBUG:boto:Params: {} 2024-04-16T18:23:22.632 DEBUG:boto:Token: None 2024-04-16T18:23:22.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33 2024-04-16T18:23:22.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NtjlmU72FcQWbGJdB7YYhl5mVlM= 2024-04-16T18:23:22.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NtjlmU72FcQWbGJdB7YYhl5mVlM=', 'Content-Length': '0'} 2024-04-16T18:23:22.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff6b655bc9ed38b9-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.635 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:23:22.635 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.636 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.636 DEBUG:boto:Method: GET 2024-04-16T18:23:22.636 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:23:22.636 DEBUG:boto:Data: 2024-04-16T18:23:22.636 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.636 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.636 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.636 DEBUG:boto:Params: {} 2024-04-16T18:23:22.636 DEBUG:boto:Token: None 2024-04-16T18:23:22.637 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33 2024-04-16T18:23:22.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l3fA51re6SHdSX9v2s0HP4AnLqs= 2024-04-16T18:23:22.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l3fA51re6SHdSX9v2s0HP4AnLqs=', 'Content-Length': '0'} 2024-04-16T18:23:22.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c4e3211f23a705b-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.682 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:23:22.682 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:23:22.683 DEBUG:boto:Method: GET 2024-04-16T18:23:22.683 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:23:22.683 DEBUG:boto:Data: 2024-04-16T18:23:22.683 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.683 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.683 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.683 DEBUG:boto:Params: {} 2024-04-16T18:23:22.683 DEBUG:boto:Token: None 2024-04-16T18:23:22.683 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj33 2024-04-16T18:23:22.683 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l3fA51re6SHdSX9v2s0HP4AnLqs= 2024-04-16T18:23:22.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l3fA51re6SHdSX9v2s0HP4AnLqs=', 'Content-Length': '0'} 2024-04-16T18:23:22.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1b193b39bed4606-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.687 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.687 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.687 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.687 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.687 DEBUG:boto:Method: GET 2024-04-16T18:23:22.687 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:23:22.687 DEBUG:boto:Data: 2024-04-16T18:23:22.687 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.687 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.688 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.688 DEBUG:boto:Params: {} 2024-04-16T18:23:22.688 DEBUG:boto:Token: None 2024-04-16T18:23:22.688 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.692 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dcdf153280586b24-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.734 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.735 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.735 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.735 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.735 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:23:22.735 DEBUG:boto:Method: GET 2024-04-16T18:23:22.735 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:23:22.735 DEBUG:boto:Data: 2024-04-16T18:23:22.735 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.735 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.735 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.735 DEBUG:boto:Params: {} 2024-04-16T18:23:22.735 DEBUG:boto:Token: None 2024-04-16T18:23:22.736 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029a58d87c147edae-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.740 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:23:22.741 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.741 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.741 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.741 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.741 DEBUG:boto:Method: GET 2024-04-16T18:23:22.741 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.741 DEBUG:boto:Data: 2024-04-16T18:23:22.741 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.741 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.741 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.741 DEBUG:boto:Params: {} 2024-04-16T18:23:22.742 DEBUG:boto:Token: None 2024-04-16T18:23:22.742 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.742 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I= 2024-04-16T18:23:22.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I=', 'Content-Length': '0'} 2024-04-16T18:23:22.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000028376c5ffc7eb76-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.786 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.786 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.787 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.787 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.787 DEBUG:boto:Method: GET 2024-04-16T18:23:22.787 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.787 DEBUG:boto:Data: 2024-04-16T18:23:22.787 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.787 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.787 DEBUG:boto:Params: {} 2024-04-16T18:23:22.787 DEBUG:boto:Token: None 2024-04-16T18:23:22.787 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I= 2024-04-16T18:23:22.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I=', 'Content-Length': '0'} 2024-04-16T18:23:22.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006eda1588a410cce7-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.791 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.791 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.791 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.791 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.791 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.791 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.792 DEBUG:boto:Data: 2024-04-16T18:23:22.792 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.792 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.792 DEBUG:boto:Params: {} 2024-04-16T18:23:22.792 DEBUG:boto:Token: None 2024-04-16T18:23:22.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rtmhJ1jvhIkDnREMVSlFmM3La8o= 2024-04-16T18:23:22.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rtmhJ1jvhIkDnREMVSlFmM3La8o=', 'Content-Length': '0'} 2024-04-16T18:23:22.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000343998b811cd060d-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.795 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.796 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.796 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.796 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.796 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.796 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.796 DEBUG:boto:Data: 2024-04-16T18:23:22.796 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.796 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.796 DEBUG:boto:Params: {} 2024-04-16T18:23:22.796 DEBUG:boto:Token: None 2024-04-16T18:23:22.797 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rtmhJ1jvhIkDnREMVSlFmM3La8o= 2024-04-16T18:23:22.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rtmhJ1jvhIkDnREMVSlFmM3La8o=', 'Content-Length': '0'} 2024-04-16T18:23:22.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007fb9f04111b4cfe-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:23:22.800 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.800 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.800 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.800 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.800 DEBUG:boto:Method: GET 2024-04-16T18:23:22.800 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.800 DEBUG:boto:Data: 2024-04-16T18:23:22.801 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.801 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.801 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.801 DEBUG:boto:Params: {} 2024-04-16T18:23:22.801 DEBUG:boto:Token: None 2024-04-16T18:23:22.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I= 2024-04-16T18:23:22.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I=', 'Content-Length': '0'} 2024-04-16T18:23:22.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad497e4032dbfd6f-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.846 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.846 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.846 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.847 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.847 DEBUG:boto:Method: GET 2024-04-16T18:23:22.847 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.847 DEBUG:boto:Data: 2024-04-16T18:23:22.847 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.847 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.847 DEBUG:boto:Params: {} 2024-04-16T18:23:22.847 DEBUG:boto:Token: None 2024-04-16T18:23:22.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:23:22.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I= 2024-04-16T18:23:22.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E0nvSGn1lBSK1sFgS+EcM51Tu/I=', 'Content-Length': '0'} 2024-04-16T18:23:22.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eb895c976021bf7-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.851 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.851 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.851 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:22.851 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:22.851 DEBUG:boto:Method: GET 2024-04-16T18:23:22.851 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:23:22.851 DEBUG:boto:Data: 2024-04-16T18:23:22.851 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.852 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.852 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.852 DEBUG:boto:Params: {} 2024-04-16T18:23:22.852 DEBUG:boto:Token: None 2024-04-16T18:23:22.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b86963393c4c763-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.856 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.857 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.857 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.857 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:22.857 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:22.857 DEBUG:boto:Method: GET 2024-04-16T18:23:22.857 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:23:22.857 DEBUG:boto:Data: 2024-04-16T18:23:22.857 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.857 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.857 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.857 DEBUG:boto:Params: {} 2024-04-16T18:23:22.858 DEBUG:boto:Token: None 2024-04-16T18:23:22.858 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.858 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.862 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000559f70c956cf152b-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.902 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:22.903 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.903 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.903 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.903 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:23:22.903 DEBUG:boto:Data: 2024-04-16T18:23:22.903 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.903 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.903 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.903 DEBUG:boto:Params: {} 2024-04-16T18:23:22.904 DEBUG:boto:Token: None 2024-04-16T18:23:22.904 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34 2024-04-16T18:23:22.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qRaMcsR2HUf40A3jlJrGxbgdD/A= 2024-04-16T18:23:22.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qRaMcsR2HUf40A3jlJrGxbgdD/A=', 'Content-Length': '0'} 2024-04-16T18:23:22.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6a63760ba651b85-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.907 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.907 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.907 DEBUG:boto:Method: HEAD 2024-04-16T18:23:22.907 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:23:22.907 DEBUG:boto:Data: 2024-04-16T18:23:22.907 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.907 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.908 DEBUG:boto:Params: {} 2024-04-16T18:23:22.908 DEBUG:boto:Token: None 2024-04-16T18:23:22.908 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34 2024-04-16T18:23:22.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qRaMcsR2HUf40A3jlJrGxbgdD/A= 2024-04-16T18:23:22.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qRaMcsR2HUf40A3jlJrGxbgdD/A=', 'Content-Length': '0'} 2024-04-16T18:23:22.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000793ebcdc26c931d9-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.911 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:23:22.911 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.911 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.911 DEBUG:boto:Method: GET 2024-04-16T18:23:22.911 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:23:22.911 DEBUG:boto:Data: 2024-04-16T18:23:22.911 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.912 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.912 DEBUG:boto:Params: {} 2024-04-16T18:23:22.912 DEBUG:boto:Token: None 2024-04-16T18:23:22.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34 2024-04-16T18:23:22.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Sw8b9v1N+t3YjZ7g/Vn844axRKs= 2024-04-16T18:23:22.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sw8b9v1N+t3YjZ7g/Vn844axRKs=', 'Content-Length': '0'} 2024-04-16T18:23:22.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000583956e0d809a121-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.958 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:23:22.958 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:23:22.959 DEBUG:boto:Method: GET 2024-04-16T18:23:22.959 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:23:22.959 DEBUG:boto:Data: 2024-04-16T18:23:22.959 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:22.959 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.959 DEBUG:boto:Params: {} 2024-04-16T18:23:22.959 DEBUG:boto:Token: None 2024-04-16T18:23:22.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/obj34 2024-04-16T18:23:22.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Sw8b9v1N+t3YjZ7g/Vn844axRKs= 2024-04-16T18:23:22.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sw8b9v1N+t3YjZ7g/Vn844axRKs=', 'Content-Length': '0'} 2024-04-16T18:23:22.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065dfb2dbeb31f855-00661ec21a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:22.963 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:22.963 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:22.963 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:22.963 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:22.963 DEBUG:boto:Method: GET 2024-04-16T18:23:22.963 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:23:22.963 DEBUG:boto:Data: 2024-04-16T18:23:22.963 DEBUG:boto:Headers: {} 2024-04-16T18:23:22.964 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:22.964 DEBUG:boto:Port: 8000 2024-04-16T18:23:22.964 DEBUG:boto:Params: {} 2024-04-16T18:23:22.964 DEBUG:boto:Token: None 2024-04-16T18:23:22.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:22 GMT /lybtug-52/ 2024-04-16T18:23:22.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU= 2024-04-16T18:23:22.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G1VdiMd5eCzBAuXVg2xSwLsNGOU=', 'Content-Length': '0'} 2024-04-16T18:23:22.968 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c1d1e6226f79edf-00661ec21a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.010 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.011 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.011 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.011 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:23.011 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:23:23.011 DEBUG:boto:Method: GET 2024-04-16T18:23:23.011 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:23:23.011 DEBUG:boto:Data: 2024-04-16T18:23:23.011 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.011 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.011 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.012 DEBUG:boto:Params: {} 2024-04-16T18:23:23.012 DEBUG:boto:Token: None 2024-04-16T18:23:23.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8708877c65f5301-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.017 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.017 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:23:23.017 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.017 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.017 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.017 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.018 DEBUG:boto:Method: GET 2024-04-16T18:23:23.018 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.018 DEBUG:boto:Data: 2024-04-16T18:23:23.018 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.018 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.018 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.018 DEBUG:boto:Params: {} 2024-04-16T18:23:23.018 DEBUG:boto:Token: None 2024-04-16T18:23:23.018 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.018 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs= 2024-04-16T18:23:23.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs=', 'Content-Length': '0'} 2024-04-16T18:23:23.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000092d7df86e9cde65-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.062 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.062 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.063 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.063 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.063 DEBUG:boto:Method: GET 2024-04-16T18:23:23.063 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.063 DEBUG:boto:Data: 2024-04-16T18:23:23.063 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.063 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.063 DEBUG:boto:Params: {} 2024-04-16T18:23:23.063 DEBUG:boto:Token: None 2024-04-16T18:23:23.063 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs= 2024-04-16T18:23:23.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs=', 'Content-Length': '0'} 2024-04-16T18:23:23.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020893f21a2fb08da-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.067 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.067 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.067 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.068 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.068 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.068 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.068 DEBUG:boto:Data: 2024-04-16T18:23:23.068 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.068 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.068 DEBUG:boto:Params: {} 2024-04-16T18:23:23.068 DEBUG:boto:Token: None 2024-04-16T18:23:23.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7K2ctKYg21rHMkVMlhhxByP7Ky4= 2024-04-16T18:23:23.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7K2ctKYg21rHMkVMlhhxByP7Ky4=', 'Content-Length': '0'} 2024-04-16T18:23:23.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0750098012349d6-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.071 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.071 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.071 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.072 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.072 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.072 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.072 DEBUG:boto:Data: 2024-04-16T18:23:23.072 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.072 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.072 DEBUG:boto:Params: {} 2024-04-16T18:23:23.072 DEBUG:boto:Token: None 2024-04-16T18:23:23.072 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7K2ctKYg21rHMkVMlhhxByP7Ky4= 2024-04-16T18:23:23.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7K2ctKYg21rHMkVMlhhxByP7Ky4=', 'Content-Length': '0'} 2024-04-16T18:23:23.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045fbf20d27b1ed23-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:23:23.076 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.076 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.076 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.076 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.076 DEBUG:boto:Method: GET 2024-04-16T18:23:23.076 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.076 DEBUG:boto:Data: 2024-04-16T18:23:23.076 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.077 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.077 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.077 DEBUG:boto:Params: {} 2024-04-16T18:23:23.077 DEBUG:boto:Token: None 2024-04-16T18:23:23.077 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.077 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs= 2024-04-16T18:23:23.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs=', 'Content-Length': '0'} 2024-04-16T18:23:23.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d97992d2134efa8-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.122 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.123 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.123 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.123 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.123 DEBUG:boto:Method: GET 2024-04-16T18:23:23.123 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.123 DEBUG:boto:Data: 2024-04-16T18:23:23.123 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.123 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.123 DEBUG:boto:Params: {} 2024-04-16T18:23:23.123 DEBUG:boto:Token: None 2024-04-16T18:23:23.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:23:23.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs= 2024-04-16T18:23:23.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4tgXFuF1p7TEi9JhFbIOQRVZIWs=', 'Content-Length': '0'} 2024-04-16T18:23:23.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003577dd684af333d9-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.127 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.127 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.127 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.127 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.127 DEBUG:boto:Method: GET 2024-04-16T18:23:23.127 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:23:23.127 DEBUG:boto:Data: 2024-04-16T18:23:23.127 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.128 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.128 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.128 DEBUG:boto:Params: {} 2024-04-16T18:23:23.128 DEBUG:boto:Token: None 2024-04-16T18:23:23.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc30abd38e0257fb-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.132 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.132 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.132 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.132 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.132 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.132 DEBUG:boto:Method: GET 2024-04-16T18:23:23.133 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:23:23.133 DEBUG:boto:Data: 2024-04-16T18:23:23.133 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.133 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.133 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.133 DEBUG:boto:Params: {} 2024-04-16T18:23:23.133 DEBUG:boto:Token: None 2024-04-16T18:23:23.133 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.138 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000659b0e4d372539b8-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.178 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.179 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.179 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.179 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.179 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:23:23.179 DEBUG:boto:Data: 2024-04-16T18:23:23.179 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.179 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.179 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.179 DEBUG:boto:Params: {} 2024-04-16T18:23:23.179 DEBUG:boto:Token: None 2024-04-16T18:23:23.179 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38 2024-04-16T18:23:23.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CCaTbAz1PMFnexuyPd/+jw+qBgI= 2024-04-16T18:23:23.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CCaTbAz1PMFnexuyPd/+jw+qBgI=', 'Content-Length': '0'} 2024-04-16T18:23:23.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078247a1de50a6a26-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.183 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.183 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.183 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.183 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:23:23.183 DEBUG:boto:Data: 2024-04-16T18:23:23.183 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.183 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.183 DEBUG:boto:Params: {} 2024-04-16T18:23:23.183 DEBUG:boto:Token: None 2024-04-16T18:23:23.183 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38 2024-04-16T18:23:23.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CCaTbAz1PMFnexuyPd/+jw+qBgI= 2024-04-16T18:23:23.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CCaTbAz1PMFnexuyPd/+jw+qBgI=', 'Content-Length': '0'} 2024-04-16T18:23:23.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4a007935a2615b9-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.187 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:23:23.187 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.187 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.187 DEBUG:boto:Method: GET 2024-04-16T18:23:23.187 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:23:23.187 DEBUG:boto:Data: 2024-04-16T18:23:23.187 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.187 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.187 DEBUG:boto:Params: {} 2024-04-16T18:23:23.188 DEBUG:boto:Token: None 2024-04-16T18:23:23.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38 2024-04-16T18:23:23.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j31kNP6Y/PUJ8C3di6Kjh2A9q1k= 2024-04-16T18:23:23.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j31kNP6Y/PUJ8C3di6Kjh2A9q1k=', 'Content-Length': '0'} 2024-04-16T18:23:23.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014888fdfd1198d76-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.234 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:23:23.235 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:23:23.235 DEBUG:boto:Method: GET 2024-04-16T18:23:23.235 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:23:23.235 DEBUG:boto:Data: 2024-04-16T18:23:23.235 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.235 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.235 DEBUG:boto:Params: {} 2024-04-16T18:23:23.235 DEBUG:boto:Token: None 2024-04-16T18:23:23.235 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj38 2024-04-16T18:23:23.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j31kNP6Y/PUJ8C3di6Kjh2A9q1k= 2024-04-16T18:23:23.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j31kNP6Y/PUJ8C3di6Kjh2A9q1k=', 'Content-Length': '0'} 2024-04-16T18:23:23.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002143c4e74b59c530-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.239 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.239 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.239 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.239 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.239 DEBUG:boto:Method: GET 2024-04-16T18:23:23.239 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:23:23.239 DEBUG:boto:Data: 2024-04-16T18:23:23.239 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.239 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.239 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.240 DEBUG:boto:Params: {} 2024-04-16T18:23:23.240 DEBUG:boto:Token: None 2024-04-16T18:23:23.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004052754849db0762-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.286 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.287 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.287 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:23:23.287 DEBUG:boto:Method: GET 2024-04-16T18:23:23.287 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:23:23.287 DEBUG:boto:Data: 2024-04-16T18:23:23.287 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.287 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.287 DEBUG:boto:Params: {} 2024-04-16T18:23:23.288 DEBUG:boto:Token: None 2024-04-16T18:23:23.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001c7bbeeb8e7a1ea-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.292 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:23:23.293 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.293 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.293 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.293 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.293 DEBUG:boto:Method: GET 2024-04-16T18:23:23.293 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.293 DEBUG:boto:Data: 2024-04-16T18:23:23.293 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.293 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.293 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.293 DEBUG:boto:Params: {} 2024-04-16T18:23:23.294 DEBUG:boto:Token: None 2024-04-16T18:23:23.294 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs= 2024-04-16T18:23:23.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs=', 'Content-Length': '0'} 2024-04-16T18:23:23.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001098bd837521652f-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.338 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.339 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.339 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.339 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.339 DEBUG:boto:Method: GET 2024-04-16T18:23:23.339 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.339 DEBUG:boto:Data: 2024-04-16T18:23:23.339 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.339 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.339 DEBUG:boto:Params: {} 2024-04-16T18:23:23.339 DEBUG:boto:Token: None 2024-04-16T18:23:23.339 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs= 2024-04-16T18:23:23.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs=', 'Content-Length': '0'} 2024-04-16T18:23:23.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f16261671e4ba681-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.343 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.343 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.343 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.343 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.343 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.343 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.343 DEBUG:boto:Data: 2024-04-16T18:23:23.344 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.344 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.344 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.344 DEBUG:boto:Params: {} 2024-04-16T18:23:23.344 DEBUG:boto:Token: None 2024-04-16T18:23:23.344 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EYsq0GcBxJoopwx9wEWRVIUicoM= 2024-04-16T18:23:23.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EYsq0GcBxJoopwx9wEWRVIUicoM=', 'Content-Length': '0'} 2024-04-16T18:23:23.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9b71a9e41904d89-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.347 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.347 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.348 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.348 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.348 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.348 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.348 DEBUG:boto:Data: 2024-04-16T18:23:23.348 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.348 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.348 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.348 DEBUG:boto:Params: {} 2024-04-16T18:23:23.348 DEBUG:boto:Token: None 2024-04-16T18:23:23.348 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EYsq0GcBxJoopwx9wEWRVIUicoM= 2024-04-16T18:23:23.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EYsq0GcBxJoopwx9wEWRVIUicoM=', 'Content-Length': '0'} 2024-04-16T18:23:23.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000750b3959939920ab-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.352 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:23:23.352 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.352 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.352 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.352 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.352 DEBUG:boto:Method: GET 2024-04-16T18:23:23.353 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.353 DEBUG:boto:Data: 2024-04-16T18:23:23.353 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.353 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.353 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.353 DEBUG:boto:Params: {} 2024-04-16T18:23:23.353 DEBUG:boto:Token: None 2024-04-16T18:23:23.353 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs= 2024-04-16T18:23:23.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs=', 'Content-Length': '0'} 2024-04-16T18:23:23.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2afca48db573cb3-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.398 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.398 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.398 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.399 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.399 DEBUG:boto:Method: GET 2024-04-16T18:23:23.399 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.399 DEBUG:boto:Data: 2024-04-16T18:23:23.399 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.399 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.399 DEBUG:boto:Params: {} 2024-04-16T18:23:23.399 DEBUG:boto:Token: None 2024-04-16T18:23:23.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:23:23.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs= 2024-04-16T18:23:23.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fY4+eDg4UfVyIfveFEsiEH3bzVs=', 'Content-Length': '0'} 2024-04-16T18:23:23.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c768016d5cce8d88-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.404 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.404 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.404 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.404 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.404 DEBUG:boto:Method: GET 2024-04-16T18:23:23.404 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:23:23.404 DEBUG:boto:Data: 2024-04-16T18:23:23.404 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.404 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.404 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.404 DEBUG:boto:Params: {} 2024-04-16T18:23:23.405 DEBUG:boto:Token: None 2024-04-16T18:23:23.405 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.409 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d5a5a8a98756811-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.409 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.409 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.409 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.409 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.409 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.410 DEBUG:boto:Method: GET 2024-04-16T18:23:23.410 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:23:23.410 DEBUG:boto:Data: 2024-04-16T18:23:23.410 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.410 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.410 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.410 DEBUG:boto:Params: {} 2024-04-16T18:23:23.410 DEBUG:boto:Token: None 2024-04-16T18:23:23.410 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.411 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa9d58bfd5e8a3b2-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.458 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.459 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.459 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.459 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.459 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:23:23.459 DEBUG:boto:Data: 2024-04-16T18:23:23.459 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.460 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.460 DEBUG:boto:Params: {} 2024-04-16T18:23:23.460 DEBUG:boto:Token: None 2024-04-16T18:23:23.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44 2024-04-16T18:23:23.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Ek8FGHXIF5k9RmdhHry40gXSPU= 2024-04-16T18:23:23.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Ek8FGHXIF5k9RmdhHry40gXSPU=', 'Content-Length': '0'} 2024-04-16T18:23:23.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb0f5c91eed24544-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.463 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.463 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.464 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.464 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:23:23.464 DEBUG:boto:Data: 2024-04-16T18:23:23.464 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.464 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.464 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.464 DEBUG:boto:Params: {} 2024-04-16T18:23:23.464 DEBUG:boto:Token: None 2024-04-16T18:23:23.464 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44 2024-04-16T18:23:23.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Ek8FGHXIF5k9RmdhHry40gXSPU= 2024-04-16T18:23:23.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Ek8FGHXIF5k9RmdhHry40gXSPU=', 'Content-Length': '0'} 2024-04-16T18:23:23.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000014e1cabd9404f0f-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.468 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:23:23.468 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.468 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.468 DEBUG:boto:Method: GET 2024-04-16T18:23:23.468 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:23:23.468 DEBUG:boto:Data: 2024-04-16T18:23:23.468 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.468 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.469 DEBUG:boto:Params: {} 2024-04-16T18:23:23.469 DEBUG:boto:Token: None 2024-04-16T18:23:23.469 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44 2024-04-16T18:23:23.469 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k4PJed1O+PUO2+Gjz0+vNobnKRc= 2024-04-16T18:23:23.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k4PJed1O+PUO2+Gjz0+vNobnKRc=', 'Content-Length': '0'} 2024-04-16T18:23:23.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068c0fac2881bb76a-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.514 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:23:23.514 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:23:23.514 DEBUG:boto:Method: GET 2024-04-16T18:23:23.515 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:23:23.515 DEBUG:boto:Data: 2024-04-16T18:23:23.515 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.515 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.515 DEBUG:boto:Params: {} 2024-04-16T18:23:23.515 DEBUG:boto:Token: None 2024-04-16T18:23:23.515 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj44 2024-04-16T18:23:23.515 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k4PJed1O+PUO2+Gjz0+vNobnKRc= 2024-04-16T18:23:23.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k4PJed1O+PUO2+Gjz0+vNobnKRc=', 'Content-Length': '0'} 2024-04-16T18:23:23.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d910a4a6431cb12-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.519 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.519 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.519 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.519 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.519 DEBUG:boto:Method: GET 2024-04-16T18:23:23.519 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:23:23.519 DEBUG:boto:Data: 2024-04-16T18:23:23.519 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.519 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.519 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.520 DEBUG:boto:Params: {} 2024-04-16T18:23:23.520 DEBUG:boto:Token: None 2024-04-16T18:23:23.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.523 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff8ddd5cf74c6c26-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.566 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.567 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.567 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.567 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.567 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:23:23.567 DEBUG:boto:Method: GET 2024-04-16T18:23:23.567 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:23:23.567 DEBUG:boto:Data: 2024-04-16T18:23:23.567 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.567 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.567 DEBUG:boto:Params: {} 2024-04-16T18:23:23.567 DEBUG:boto:Token: None 2024-04-16T18:23:23.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000942c737513f0a259-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.572 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.572 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:23:23.573 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.573 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.573 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.573 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.573 DEBUG:boto:Method: GET 2024-04-16T18:23:23.573 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.573 DEBUG:boto:Data: 2024-04-16T18:23:23.573 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.573 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.573 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.573 DEBUG:boto:Params: {} 2024-04-16T18:23:23.573 DEBUG:boto:Token: None 2024-04-16T18:23:23.574 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.574 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w= 2024-04-16T18:23:23.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w=', 'Content-Length': '0'} 2024-04-16T18:23:23.577 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069b35742537fb3bd-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.618 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.618 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.618 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.619 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.619 DEBUG:boto:Method: GET 2024-04-16T18:23:23.619 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.619 DEBUG:boto:Data: 2024-04-16T18:23:23.619 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.619 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.619 DEBUG:boto:Params: {} 2024-04-16T18:23:23.619 DEBUG:boto:Token: None 2024-04-16T18:23:23.619 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.619 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w= 2024-04-16T18:23:23.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w=', 'Content-Length': '0'} 2024-04-16T18:23:23.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009efda968c30621fa-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.623 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.623 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.623 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.623 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.623 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.623 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.623 DEBUG:boto:Data: 2024-04-16T18:23:23.623 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.624 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.624 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.624 DEBUG:boto:Params: {} 2024-04-16T18:23:23.624 DEBUG:boto:Token: None 2024-04-16T18:23:23.624 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DTTaEf3d8JDZd7PEC9Xd/HJYSt0= 2024-04-16T18:23:23.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DTTaEf3d8JDZd7PEC9Xd/HJYSt0=', 'Content-Length': '0'} 2024-04-16T18:23:23.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3bd6cb34364c07a-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.627 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.627 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.627 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.627 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.628 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.628 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.628 DEBUG:boto:Data: 2024-04-16T18:23:23.628 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.628 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.628 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.628 DEBUG:boto:Params: {} 2024-04-16T18:23:23.628 DEBUG:boto:Token: None 2024-04-16T18:23:23.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DTTaEf3d8JDZd7PEC9Xd/HJYSt0= 2024-04-16T18:23:23.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DTTaEf3d8JDZd7PEC9Xd/HJYSt0=', 'Content-Length': '0'} 2024-04-16T18:23:23.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005aac3370541da5c-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.632 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:23:23.632 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.632 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.632 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.632 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.632 DEBUG:boto:Method: GET 2024-04-16T18:23:23.632 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.632 DEBUG:boto:Data: 2024-04-16T18:23:23.632 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.633 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.633 DEBUG:boto:Params: {} 2024-04-16T18:23:23.633 DEBUG:boto:Token: None 2024-04-16T18:23:23.633 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.633 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w= 2024-04-16T18:23:23.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w=', 'Content-Length': '0'} 2024-04-16T18:23:23.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077fb4d3ac72b475e-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.678 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.678 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.678 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.679 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.679 DEBUG:boto:Method: GET 2024-04-16T18:23:23.679 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.679 DEBUG:boto:Data: 2024-04-16T18:23:23.679 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.679 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.679 DEBUG:boto:Params: {} 2024-04-16T18:23:23.679 DEBUG:boto:Token: None 2024-04-16T18:23:23.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:23:23.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w= 2024-04-16T18:23:23.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2dEuauIGJwSgEl6LNk+/4iNx35w=', 'Content-Length': '0'} 2024-04-16T18:23:23.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d9646d093aee26b-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.683 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.683 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.683 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.683 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.683 DEBUG:boto:Method: GET 2024-04-16T18:23:23.683 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:23:23.683 DEBUG:boto:Data: 2024-04-16T18:23:23.684 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.684 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.684 DEBUG:boto:Params: {} 2024-04-16T18:23:23.684 DEBUG:boto:Token: None 2024-04-16T18:23:23.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.688 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051c1fc47662cdda0-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.688 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.688 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.688 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.688 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.688 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.688 DEBUG:boto:Method: GET 2024-04-16T18:23:23.689 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:23:23.689 DEBUG:boto:Data: 2024-04-16T18:23:23.689 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.689 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.689 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.689 DEBUG:boto:Params: {} 2024-04-16T18:23:23.689 DEBUG:boto:Token: None 2024-04-16T18:23:23.689 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.689 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.693 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a1a26f62d8025a6-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.734 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.735 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.735 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.735 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.735 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:23:23.735 DEBUG:boto:Data: 2024-04-16T18:23:23.735 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.735 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.735 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.735 DEBUG:boto:Params: {} 2024-04-16T18:23:23.735 DEBUG:boto:Token: None 2024-04-16T18:23:23.735 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51 2024-04-16T18:23:23.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oYJJS2+ejNR6iIRZchrrxCfrAvs= 2024-04-16T18:23:23.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oYJJS2+ejNR6iIRZchrrxCfrAvs=', 'Content-Length': '0'} 2024-04-16T18:23:23.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001df93c1133c8b1f4-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.739 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.739 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.739 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.739 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:23:23.739 DEBUG:boto:Data: 2024-04-16T18:23:23.740 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.740 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.740 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.740 DEBUG:boto:Params: {} 2024-04-16T18:23:23.740 DEBUG:boto:Token: None 2024-04-16T18:23:23.740 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51 2024-04-16T18:23:23.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oYJJS2+ejNR6iIRZchrrxCfrAvs= 2024-04-16T18:23:23.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oYJJS2+ejNR6iIRZchrrxCfrAvs=', 'Content-Length': '0'} 2024-04-16T18:23:23.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bccac7f4570801d-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:23:23.744 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.744 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.744 DEBUG:boto:Method: GET 2024-04-16T18:23:23.744 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:23:23.744 DEBUG:boto:Data: 2024-04-16T18:23:23.744 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.744 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.744 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.744 DEBUG:boto:Params: {} 2024-04-16T18:23:23.744 DEBUG:boto:Token: None 2024-04-16T18:23:23.745 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51 2024-04-16T18:23:23.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z44MRkFjvVex8QIFLyZIQgNy0Ok= 2024-04-16T18:23:23.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z44MRkFjvVex8QIFLyZIQgNy0Ok=', 'Content-Length': '0'} 2024-04-16T18:23:23.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8e89d177ba440be-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.790 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:23:23.790 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:23:23.791 DEBUG:boto:Method: GET 2024-04-16T18:23:23.791 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:23:23.791 DEBUG:boto:Data: 2024-04-16T18:23:23.791 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.791 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.791 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.791 DEBUG:boto:Params: {} 2024-04-16T18:23:23.791 DEBUG:boto:Token: None 2024-04-16T18:23:23.791 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj51 2024-04-16T18:23:23.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z44MRkFjvVex8QIFLyZIQgNy0Ok= 2024-04-16T18:23:23.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z44MRkFjvVex8QIFLyZIQgNy0Ok=', 'Content-Length': '0'} 2024-04-16T18:23:23.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083942df9dc5e4920-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.795 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.795 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.795 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.795 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.795 DEBUG:boto:Method: GET 2024-04-16T18:23:23.795 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:23:23.795 DEBUG:boto:Data: 2024-04-16T18:23:23.796 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.796 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.796 DEBUG:boto:Params: {} 2024-04-16T18:23:23.796 DEBUG:boto:Token: None 2024-04-16T18:23:23.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa5b31230884f00d-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.842 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.843 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.843 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.843 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.843 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:23:23.843 DEBUG:boto:Method: GET 2024-04-16T18:23:23.843 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:23:23.843 DEBUG:boto:Data: 2024-04-16T18:23:23.843 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.843 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.843 DEBUG:boto:Params: {} 2024-04-16T18:23:23.844 DEBUG:boto:Token: None 2024-04-16T18:23:23.844 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.848 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ef52273637e77e2-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.849 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.849 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:23:23.849 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:23.849 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:23.849 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.849 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.849 DEBUG:boto:Method: GET 2024-04-16T18:23:23.850 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.850 DEBUG:boto:Data: 2024-04-16T18:23:23.850 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.850 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.850 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.850 DEBUG:boto:Params: {} 2024-04-16T18:23:23.850 DEBUG:boto:Token: None 2024-04-16T18:23:23.850 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.850 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE= 2024-04-16T18:23:23.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE=', 'Content-Length': '0'} 2024-04-16T18:23:23.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e11672e78d6fc35-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.894 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:23.894 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:23.895 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.895 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.895 DEBUG:boto:Method: GET 2024-04-16T18:23:23.895 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.895 DEBUG:boto:Data: 2024-04-16T18:23:23.895 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.895 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.895 DEBUG:boto:Params: {} 2024-04-16T18:23:23.895 DEBUG:boto:Token: None 2024-04-16T18:23:23.896 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE= 2024-04-16T18:23:23.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE=', 'Content-Length': '0'} 2024-04-16T18:23:23.898 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f7bfb4a2b6e3564-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.899 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:23.899 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:23.899 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.899 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.899 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.899 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.899 DEBUG:boto:Data: 2024-04-16T18:23:23.900 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.900 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.900 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.900 DEBUG:boto:Params: {} 2024-04-16T18:23:23.900 DEBUG:boto:Token: None 2024-04-16T18:23:23.900 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UAdvAllyrPGpDzuRyR4ovJDJnJI= 2024-04-16T18:23:23.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UAdvAllyrPGpDzuRyR4ovJDJnJI=', 'Content-Length': '0'} 2024-04-16T18:23:23.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004465651ba80ee17-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.903 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:23.903 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:23.904 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.904 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.904 DEBUG:boto:Method: HEAD 2024-04-16T18:23:23.904 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.904 DEBUG:boto:Data: 2024-04-16T18:23:23.904 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.904 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.904 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.904 DEBUG:boto:Params: {} 2024-04-16T18:23:23.904 DEBUG:boto:Token: None 2024-04-16T18:23:23.905 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.905 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UAdvAllyrPGpDzuRyR4ovJDJnJI= 2024-04-16T18:23:23.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UAdvAllyrPGpDzuRyR4ovJDJnJI=', 'Content-Length': '0'} 2024-04-16T18:23:23.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fac9183fe58d95e0-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.908 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:23:23.908 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:23.908 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:23.908 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.908 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.908 DEBUG:boto:Method: GET 2024-04-16T18:23:23.908 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.908 DEBUG:boto:Data: 2024-04-16T18:23:23.909 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.909 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.909 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.909 DEBUG:boto:Params: {} 2024-04-16T18:23:23.909 DEBUG:boto:Token: None 2024-04-16T18:23:23.909 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE= 2024-04-16T18:23:23.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE=', 'Content-Length': '0'} 2024-04-16T18:23:23.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016988e3698df5814-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.954 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:23.954 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:23.955 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.955 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.955 DEBUG:boto:Method: GET 2024-04-16T18:23:23.955 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.955 DEBUG:boto:Data: 2024-04-16T18:23:23.955 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.955 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.955 DEBUG:boto:Params: {} 2024-04-16T18:23:23.955 DEBUG:boto:Token: None 2024-04-16T18:23:23.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:23:23.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE= 2024-04-16T18:23:23.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cT4oP7BApAs5SobHCvyav3rcjUE=', 'Content-Length': '0'} 2024-04-16T18:23:23.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb9683829ed18f02-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.959 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.959 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.959 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:23.959 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:23.959 DEBUG:boto:Method: GET 2024-04-16T18:23:23.960 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:23:23.960 DEBUG:boto:Data: 2024-04-16T18:23:23.960 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.960 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:23.960 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.960 DEBUG:boto:Params: {} 2024-04-16T18:23:23.960 DEBUG:boto:Token: None 2024-04-16T18:23:23.960 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096e0712e76cde93e-00661ec21b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:23.964 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:23.965 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:23.965 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:23.965 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:23.965 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:23.965 DEBUG:boto:Method: GET 2024-04-16T18:23:23.965 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:23:23.965 DEBUG:boto:Data: 2024-04-16T18:23:23.965 DEBUG:boto:Headers: {} 2024-04-16T18:23:23.966 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:23.966 DEBUG:boto:Port: 8000 2024-04-16T18:23:23.966 DEBUG:boto:Params: {} 2024-04-16T18:23:23.966 DEBUG:boto:Token: None 2024-04-16T18:23:23.966 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:23 GMT /lybtug-52/ 2024-04-16T18:23:23.966 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M= 2024-04-16T18:23:23.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RCxnj9eHSuOaKtVWcWBrgJbGD7M=', 'Content-Length': '0'} 2024-04-16T18:23:23.971 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b30297cf127193a4-00661ec21b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.014 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.015 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:24.015 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:24.015 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.015 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:23:24.015 DEBUG:boto:Data: 2024-04-16T18:23:24.015 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.015 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.015 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.015 DEBUG:boto:Params: {} 2024-04-16T18:23:24.015 DEBUG:boto:Token: None 2024-04-16T18:23:24.015 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj55 2024-04-16T18:23:24.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+4WOerEBnYw1PxgAe+mAS68+aAk= 2024-04-16T18:23:24.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+4WOerEBnYw1PxgAe+mAS68+aAk=', 'Content-Length': '0'} 2024-04-16T18:23:24.018 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8aae97796e311c6-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.019 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:24.019 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:24.019 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.019 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:23:24.019 DEBUG:boto:Data: 2024-04-16T18:23:24.019 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.019 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.019 DEBUG:boto:Params: {} 2024-04-16T18:23:24.019 DEBUG:boto:Token: None 2024-04-16T18:23:24.019 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj55 2024-04-16T18:23:24.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+4WOerEBnYw1PxgAe+mAS68+aAk= 2024-04-16T18:23:24.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+4WOerEBnYw1PxgAe+mAS68+aAk=', 'Content-Length': '0'} 2024-04-16T18:23:24.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd10f72d75671e7c-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:23:24.023 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:24.023 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:24.023 DEBUG:boto:Method: GET 2024-04-16T18:23:24.023 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:23:24.023 DEBUG:boto:Data: 2024-04-16T18:23:24.023 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.023 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.023 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.024 DEBUG:boto:Params: {} 2024-04-16T18:23:24.024 DEBUG:boto:Token: None 2024-04-16T18:23:24.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj55 2024-04-16T18:23:24.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KKkEiP+7bLeuwinWyW4Jait5UpI= 2024-04-16T18:23:24.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KKkEiP+7bLeuwinWyW4Jait5UpI=', 'Content-Length': '0'} 2024-04-16T18:23:24.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8c2b9134d01e8f7-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.070 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:23:24.070 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:23:24.071 DEBUG:boto:Method: GET 2024-04-16T18:23:24.071 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:23:24.071 DEBUG:boto:Data: 2024-04-16T18:23:24.071 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.071 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.071 DEBUG:boto:Params: {} 2024-04-16T18:23:24.071 DEBUG:boto:Token: None 2024-04-16T18:23:24.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj55 2024-04-16T18:23:24.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KKkEiP+7bLeuwinWyW4Jait5UpI= 2024-04-16T18:23:24.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KKkEiP+7bLeuwinWyW4Jait5UpI=', 'Content-Length': '0'} 2024-04-16T18:23:24.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074a14f36c22bcac6-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.075 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.075 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.075 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:24.075 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:24.075 DEBUG:boto:Method: GET 2024-04-16T18:23:24.075 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:23:24.075 DEBUG:boto:Data: 2024-04-16T18:23:24.075 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.075 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.075 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.076 DEBUG:boto:Params: {} 2024-04-16T18:23:24.076 DEBUG:boto:Token: None 2024-04-16T18:23:24.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cf2bf2a55fec399-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.122 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.123 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.123 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.123 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:24.123 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:23:24.123 DEBUG:boto:Method: GET 2024-04-16T18:23:24.123 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:23:24.123 DEBUG:boto:Data: 2024-04-16T18:23:24.123 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.123 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.124 DEBUG:boto:Params: {} 2024-04-16T18:23:24.124 DEBUG:boto:Token: None 2024-04-16T18:23:24.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2c66098b0c9c0bd-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.129 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.129 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:23:24.129 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.129 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.129 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.129 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.130 DEBUG:boto:Method: GET 2024-04-16T18:23:24.130 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.130 DEBUG:boto:Data: 2024-04-16T18:23:24.130 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.130 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.130 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.130 DEBUG:boto:Params: {} 2024-04-16T18:23:24.130 DEBUG:boto:Token: None 2024-04-16T18:23:24.130 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.130 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q= 2024-04-16T18:23:24.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cfb3edcf1cca8da-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.174 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.174 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.175 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.175 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.175 DEBUG:boto:Method: GET 2024-04-16T18:23:24.175 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.175 DEBUG:boto:Data: 2024-04-16T18:23:24.175 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.175 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.175 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.175 DEBUG:boto:Params: {} 2024-04-16T18:23:24.175 DEBUG:boto:Token: None 2024-04-16T18:23:24.175 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q= 2024-04-16T18:23:24.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c438cec64a3e70fe-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.179 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.179 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.179 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.179 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.179 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.179 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.179 DEBUG:boto:Data: 2024-04-16T18:23:24.179 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.179 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.179 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.179 DEBUG:boto:Params: {} 2024-04-16T18:23:24.180 DEBUG:boto:Token: None 2024-04-16T18:23:24.180 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3EJ/SHr8vWIi+ujjozIGimY9GFM= 2024-04-16T18:23:24.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3EJ/SHr8vWIi+ujjozIGimY9GFM=', 'Content-Length': '0'} 2024-04-16T18:23:24.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff2e8809931448c7-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.183 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.183 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.183 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.183 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.183 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.183 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.183 DEBUG:boto:Data: 2024-04-16T18:23:24.183 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.184 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.184 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.184 DEBUG:boto:Params: {} 2024-04-16T18:23:24.184 DEBUG:boto:Token: None 2024-04-16T18:23:24.184 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3EJ/SHr8vWIi+ujjozIGimY9GFM= 2024-04-16T18:23:24.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3EJ/SHr8vWIi+ujjozIGimY9GFM=', 'Content-Length': '0'} 2024-04-16T18:23:24.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e330595ce9b7e568-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.187 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:23:24.187 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.187 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.187 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.187 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.187 DEBUG:boto:Method: GET 2024-04-16T18:23:24.187 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.187 DEBUG:boto:Data: 2024-04-16T18:23:24.188 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.188 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.188 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.188 DEBUG:boto:Params: {} 2024-04-16T18:23:24.188 DEBUG:boto:Token: None 2024-04-16T18:23:24.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q= 2024-04-16T18:23:24.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da3e00c8505643ce-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.234 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.234 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.235 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.235 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.235 DEBUG:boto:Method: GET 2024-04-16T18:23:24.235 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.235 DEBUG:boto:Data: 2024-04-16T18:23:24.235 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.235 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.235 DEBUG:boto:Params: {} 2024-04-16T18:23:24.235 DEBUG:boto:Token: None 2024-04-16T18:23:24.235 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:23:24.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q= 2024-04-16T18:23:24.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HE23L9Xm0r2WCc5Ye1pZq/o+9/Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d94c3a46d1f841b-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.239 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.239 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.239 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.239 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.240 DEBUG:boto:Method: GET 2024-04-16T18:23:24.240 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:23:24.240 DEBUG:boto:Data: 2024-04-16T18:23:24.240 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.240 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.240 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.240 DEBUG:boto:Params: {} 2024-04-16T18:23:24.240 DEBUG:boto:Token: None 2024-04-16T18:23:24.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041d2384b1993f0ca-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.244 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.244 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.244 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.244 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.245 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.245 DEBUG:boto:Method: GET 2024-04-16T18:23:24.245 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:23:24.245 DEBUG:boto:Data: 2024-04-16T18:23:24.245 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.245 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.245 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.245 DEBUG:boto:Params: {} 2024-04-16T18:23:24.245 DEBUG:boto:Token: None 2024-04-16T18:23:24.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6ec2e595c004d41-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.294 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.295 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.295 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.295 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.295 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:23:24.295 DEBUG:boto:Data: 2024-04-16T18:23:24.295 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.295 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.295 DEBUG:boto:Params: {} 2024-04-16T18:23:24.296 DEBUG:boto:Token: None 2024-04-16T18:23:24.296 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57 2024-04-16T18:23:24.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QpOuRvwXxw+658XlPAwG7pBWiRw= 2024-04-16T18:23:24.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QpOuRvwXxw+658XlPAwG7pBWiRw=', 'Content-Length': '0'} 2024-04-16T18:23:24.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b51875cddd67806-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.299 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.300 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.300 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.300 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:23:24.300 DEBUG:boto:Data: 2024-04-16T18:23:24.300 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.300 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.300 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.300 DEBUG:boto:Params: {} 2024-04-16T18:23:24.300 DEBUG:boto:Token: None 2024-04-16T18:23:24.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57 2024-04-16T18:23:24.301 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QpOuRvwXxw+658XlPAwG7pBWiRw= 2024-04-16T18:23:24.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QpOuRvwXxw+658XlPAwG7pBWiRw=', 'Content-Length': '0'} 2024-04-16T18:23:24.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea1111dc6516c177-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.304 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:23:24.304 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.304 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.304 DEBUG:boto:Method: GET 2024-04-16T18:23:24.304 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:23:24.304 DEBUG:boto:Data: 2024-04-16T18:23:24.304 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.304 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.305 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.305 DEBUG:boto:Params: {} 2024-04-16T18:23:24.305 DEBUG:boto:Token: None 2024-04-16T18:23:24.305 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57 2024-04-16T18:23:24.305 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jBYT4GzRu+3kNRUnD3CcdBtWe+g= 2024-04-16T18:23:24.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jBYT4GzRu+3kNRUnD3CcdBtWe+g=', 'Content-Length': '0'} 2024-04-16T18:23:24.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000590e4f031092c1cf-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.350 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:23:24.350 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:23:24.351 DEBUG:boto:Method: GET 2024-04-16T18:23:24.351 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:23:24.351 DEBUG:boto:Data: 2024-04-16T18:23:24.351 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.351 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.351 DEBUG:boto:Params: {} 2024-04-16T18:23:24.351 DEBUG:boto:Token: None 2024-04-16T18:23:24.351 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj57 2024-04-16T18:23:24.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jBYT4GzRu+3kNRUnD3CcdBtWe+g= 2024-04-16T18:23:24.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jBYT4GzRu+3kNRUnD3CcdBtWe+g=', 'Content-Length': '0'} 2024-04-16T18:23:24.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000828e8f3f3084f6ab-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.355 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.355 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.355 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.355 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.355 DEBUG:boto:Method: GET 2024-04-16T18:23:24.355 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:23:24.356 DEBUG:boto:Data: 2024-04-16T18:23:24.356 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.356 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.356 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.356 DEBUG:boto:Params: {} 2024-04-16T18:23:24.356 DEBUG:boto:Token: None 2024-04-16T18:23:24.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edb50d1f5c10af91-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.402 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.403 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.403 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.403 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.403 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:23:24.403 DEBUG:boto:Method: GET 2024-04-16T18:23:24.403 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:23:24.403 DEBUG:boto:Data: 2024-04-16T18:23:24.403 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.403 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.404 DEBUG:boto:Params: {} 2024-04-16T18:23:24.404 DEBUG:boto:Token: None 2024-04-16T18:23:24.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.408 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063fa6a178aae6016-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.409 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.409 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:23:24.409 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.409 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.409 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.409 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.409 DEBUG:boto:Method: GET 2024-04-16T18:23:24.410 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.410 DEBUG:boto:Data: 2024-04-16T18:23:24.410 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.410 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.410 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.410 DEBUG:boto:Params: {} 2024-04-16T18:23:24.410 DEBUG:boto:Token: None 2024-04-16T18:23:24.410 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY= 2024-04-16T18:23:24.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY=', 'Content-Length': '0'} 2024-04-16T18:23:24.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057ec65b7e263eff8-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.454 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.455 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.455 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.455 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.455 DEBUG:boto:Method: GET 2024-04-16T18:23:24.455 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.455 DEBUG:boto:Data: 2024-04-16T18:23:24.455 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.455 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.455 DEBUG:boto:Params: {} 2024-04-16T18:23:24.455 DEBUG:boto:Token: None 2024-04-16T18:23:24.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY= 2024-04-16T18:23:24.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY=', 'Content-Length': '0'} 2024-04-16T18:23:24.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0ecf970e828b6ed-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.459 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.459 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.459 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.459 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.459 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.459 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.459 DEBUG:boto:Data: 2024-04-16T18:23:24.460 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.460 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.460 DEBUG:boto:Params: {} 2024-04-16T18:23:24.460 DEBUG:boto:Token: None 2024-04-16T18:23:24.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7uv+w3OuXcBU8fZwivtfo52aSJM= 2024-04-16T18:23:24.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7uv+w3OuXcBU8fZwivtfo52aSJM=', 'Content-Length': '0'} 2024-04-16T18:23:24.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000484f910d9e3b68d-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.463 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.463 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.463 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.463 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.463 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.464 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.464 DEBUG:boto:Data: 2024-04-16T18:23:24.464 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.464 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.464 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.464 DEBUG:boto:Params: {} 2024-04-16T18:23:24.464 DEBUG:boto:Token: None 2024-04-16T18:23:24.464 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7uv+w3OuXcBU8fZwivtfo52aSJM= 2024-04-16T18:23:24.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7uv+w3OuXcBU8fZwivtfo52aSJM=', 'Content-Length': '0'} 2024-04-16T18:23:24.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000157768384ed96180-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:23:24.467 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.467 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.468 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.468 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.468 DEBUG:boto:Method: GET 2024-04-16T18:23:24.468 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.468 DEBUG:boto:Data: 2024-04-16T18:23:24.468 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.468 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.468 DEBUG:boto:Params: {} 2024-04-16T18:23:24.468 DEBUG:boto:Token: None 2024-04-16T18:23:24.469 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.469 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY= 2024-04-16T18:23:24.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY=', 'Content-Length': '0'} 2024-04-16T18:23:24.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027427307da12c23b-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.514 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.514 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.515 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.515 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.515 DEBUG:boto:Method: GET 2024-04-16T18:23:24.515 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.515 DEBUG:boto:Data: 2024-04-16T18:23:24.515 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.515 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.515 DEBUG:boto:Params: {} 2024-04-16T18:23:24.515 DEBUG:boto:Token: None 2024-04-16T18:23:24.515 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:23:24.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY= 2024-04-16T18:23:24.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k3QMsY5i48c6yHlhiSCXFqNqBuY=', 'Content-Length': '0'} 2024-04-16T18:23:24.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000976c3694a5386cef-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.519 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.519 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.519 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.519 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.519 DEBUG:boto:Method: GET 2024-04-16T18:23:24.520 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:23:24.520 DEBUG:boto:Data: 2024-04-16T18:23:24.520 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.520 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.520 DEBUG:boto:Params: {} 2024-04-16T18:23:24.520 DEBUG:boto:Token: None 2024-04-16T18:23:24.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe82cf5501ec44c4-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.524 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.525 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.525 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.525 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.525 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.525 DEBUG:boto:Method: GET 2024-04-16T18:23:24.525 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:23:24.525 DEBUG:boto:Data: 2024-04-16T18:23:24.525 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.525 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.525 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.526 DEBUG:boto:Params: {} 2024-04-16T18:23:24.526 DEBUG:boto:Token: None 2024-04-16T18:23:24.526 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.526 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe999d26b3d63990-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.574 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.574 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.575 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.575 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.575 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:23:24.575 DEBUG:boto:Data: 2024-04-16T18:23:24.575 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.575 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.575 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.575 DEBUG:boto:Params: {} 2024-04-16T18:23:24.575 DEBUG:boto:Token: None 2024-04-16T18:23:24.575 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68 2024-04-16T18:23:24.575 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fk7AX/cbxT0HQXeFLToOaTlnSHk= 2024-04-16T18:23:24.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fk7AX/cbxT0HQXeFLToOaTlnSHk=', 'Content-Length': '0'} 2024-04-16T18:23:24.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4b9bfabcaca38e1-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.578 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.579 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.579 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.579 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:23:24.579 DEBUG:boto:Data: 2024-04-16T18:23:24.579 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.579 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.579 DEBUG:boto:Params: {} 2024-04-16T18:23:24.579 DEBUG:boto:Token: None 2024-04-16T18:23:24.579 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68 2024-04-16T18:23:24.579 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fk7AX/cbxT0HQXeFLToOaTlnSHk= 2024-04-16T18:23:24.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fk7AX/cbxT0HQXeFLToOaTlnSHk=', 'Content-Length': '0'} 2024-04-16T18:23:24.582 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4b34bcd1ba12a82-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.583 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:23:24.583 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.583 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.583 DEBUG:boto:Method: GET 2024-04-16T18:23:24.583 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:23:24.583 DEBUG:boto:Data: 2024-04-16T18:23:24.583 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.583 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.583 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.583 DEBUG:boto:Params: {} 2024-04-16T18:23:24.583 DEBUG:boto:Token: None 2024-04-16T18:23:24.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68 2024-04-16T18:23:24.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WKqSwlvSg5wFdvv7tb++SPJ3UxE= 2024-04-16T18:23:24.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WKqSwlvSg5wFdvv7tb++SPJ3UxE=', 'Content-Length': '0'} 2024-04-16T18:23:24.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028c699d8ea2330d9-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.630 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:23:24.630 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:23:24.631 DEBUG:boto:Method: GET 2024-04-16T18:23:24.631 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:23:24.631 DEBUG:boto:Data: 2024-04-16T18:23:24.631 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.631 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.631 DEBUG:boto:Params: {} 2024-04-16T18:23:24.631 DEBUG:boto:Token: None 2024-04-16T18:23:24.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj68 2024-04-16T18:23:24.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WKqSwlvSg5wFdvv7tb++SPJ3UxE= 2024-04-16T18:23:24.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WKqSwlvSg5wFdvv7tb++SPJ3UxE=', 'Content-Length': '0'} 2024-04-16T18:23:24.634 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f40bdbc89cfa5e0-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.635 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.635 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.635 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.635 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.635 DEBUG:boto:Method: GET 2024-04-16T18:23:24.635 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:23:24.635 DEBUG:boto:Data: 2024-04-16T18:23:24.635 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.636 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.636 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.636 DEBUG:boto:Params: {} 2024-04-16T18:23:24.636 DEBUG:boto:Token: None 2024-04-16T18:23:24.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000217e325c4240483e-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.640 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.640 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.640 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.640 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.641 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:23:24.641 DEBUG:boto:Method: GET 2024-04-16T18:23:24.641 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:23:24.641 DEBUG:boto:Data: 2024-04-16T18:23:24.641 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.641 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.641 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.641 DEBUG:boto:Params: {} 2024-04-16T18:23:24.641 DEBUG:boto:Token: None 2024-04-16T18:23:24.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.646 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4ec6629249f3935-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.646 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.647 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:23:24.647 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.647 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.647 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.647 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.647 DEBUG:boto:Method: GET 2024-04-16T18:23:24.647 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.647 DEBUG:boto:Data: 2024-04-16T18:23:24.647 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.647 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.647 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.648 DEBUG:boto:Params: {} 2024-04-16T18:23:24.648 DEBUG:boto:Token: None 2024-04-16T18:23:24.648 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs= 2024-04-16T18:23:24.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs=', 'Content-Length': '0'} 2024-04-16T18:23:24.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e2e5938310f7d81-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.694 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.695 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.695 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.695 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.695 DEBUG:boto:Method: GET 2024-04-16T18:23:24.695 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.695 DEBUG:boto:Data: 2024-04-16T18:23:24.695 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.695 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.695 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.695 DEBUG:boto:Params: {} 2024-04-16T18:23:24.695 DEBUG:boto:Token: None 2024-04-16T18:23:24.695 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs= 2024-04-16T18:23:24.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs=', 'Content-Length': '0'} 2024-04-16T18:23:24.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057f05512e702b14a-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.699 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.699 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.699 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.699 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.699 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.699 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.699 DEBUG:boto:Data: 2024-04-16T18:23:24.699 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.699 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.699 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.700 DEBUG:boto:Params: {} 2024-04-16T18:23:24.700 DEBUG:boto:Token: None 2024-04-16T18:23:24.700 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CzwHNNqtX238MzmvQ5aap85LIx8= 2024-04-16T18:23:24.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CzwHNNqtX238MzmvQ5aap85LIx8=', 'Content-Length': '0'} 2024-04-16T18:23:24.702 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d46c65eadfa6154-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.703 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.703 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.703 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.703 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.703 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.703 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.703 DEBUG:boto:Data: 2024-04-16T18:23:24.703 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.703 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.703 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.704 DEBUG:boto:Params: {} 2024-04-16T18:23:24.704 DEBUG:boto:Token: None 2024-04-16T18:23:24.704 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CzwHNNqtX238MzmvQ5aap85LIx8= 2024-04-16T18:23:24.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CzwHNNqtX238MzmvQ5aap85LIx8=', 'Content-Length': '0'} 2024-04-16T18:23:24.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aec85cb88d5ff74d-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.707 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:23:24.707 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.707 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.707 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.708 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.708 DEBUG:boto:Method: GET 2024-04-16T18:23:24.708 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.708 DEBUG:boto:Data: 2024-04-16T18:23:24.708 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.708 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.708 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.708 DEBUG:boto:Params: {} 2024-04-16T18:23:24.708 DEBUG:boto:Token: None 2024-04-16T18:23:24.708 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs= 2024-04-16T18:23:24.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs=', 'Content-Length': '0'} 2024-04-16T18:23:24.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b5589f12c50d1ff-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.754 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.755 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.755 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.755 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.755 DEBUG:boto:Method: GET 2024-04-16T18:23:24.755 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.755 DEBUG:boto:Data: 2024-04-16T18:23:24.755 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.755 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.755 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.755 DEBUG:boto:Params: {} 2024-04-16T18:23:24.755 DEBUG:boto:Token: None 2024-04-16T18:23:24.755 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:23:24.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs= 2024-04-16T18:23:24.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AFCFooYSl8wEsAFufRxXwVQmucs=', 'Content-Length': '0'} 2024-04-16T18:23:24.758 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2050f034c28896d-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.759 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.759 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.759 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.759 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.759 DEBUG:boto:Method: GET 2024-04-16T18:23:24.759 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:23:24.759 DEBUG:boto:Data: 2024-04-16T18:23:24.759 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.759 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.759 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.760 DEBUG:boto:Params: {} 2024-04-16T18:23:24.760 DEBUG:boto:Token: None 2024-04-16T18:23:24.760 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001944e3a18e65ccc0-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.806 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.807 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.807 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.807 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.807 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.807 DEBUG:boto:Method: GET 2024-04-16T18:23:24.807 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:23:24.807 DEBUG:boto:Data: 2024-04-16T18:23:24.807 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.807 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.807 DEBUG:boto:Params: {} 2024-04-16T18:23:24.808 DEBUG:boto:Token: None 2024-04-16T18:23:24.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005425b8e7208704ae-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.813 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.813 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.813 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.813 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.813 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:23:24.813 DEBUG:boto:Data: 2024-04-16T18:23:24.813 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.813 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.813 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.814 DEBUG:boto:Params: {} 2024-04-16T18:23:24.814 DEBUG:boto:Token: None 2024-04-16T18:23:24.814 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76 2024-04-16T18:23:24.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LQ1H1DbD/+QlI3u48c4RrtICOSM= 2024-04-16T18:23:24.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LQ1H1DbD/+QlI3u48c4RrtICOSM=', 'Content-Length': '0'} 2024-04-16T18:23:24.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007836d7274ff52b78-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.817 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.817 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.817 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.817 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:23:24.817 DEBUG:boto:Data: 2024-04-16T18:23:24.817 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.818 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.818 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.818 DEBUG:boto:Params: {} 2024-04-16T18:23:24.818 DEBUG:boto:Token: None 2024-04-16T18:23:24.818 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76 2024-04-16T18:23:24.818 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LQ1H1DbD/+QlI3u48c4RrtICOSM= 2024-04-16T18:23:24.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LQ1H1DbD/+QlI3u48c4RrtICOSM=', 'Content-Length': '0'} 2024-04-16T18:23:24.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052f859e591cbc88a-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.821 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:23:24.821 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.821 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.821 DEBUG:boto:Method: GET 2024-04-16T18:23:24.821 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:23:24.821 DEBUG:boto:Data: 2024-04-16T18:23:24.821 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.821 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.821 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.822 DEBUG:boto:Params: {} 2024-04-16T18:23:24.822 DEBUG:boto:Token: None 2024-04-16T18:23:24.822 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76 2024-04-16T18:23:24.822 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O6+gTUThE6/8nDDbeOVOg0HUhlM= 2024-04-16T18:23:24.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O6+gTUThE6/8nDDbeOVOg0HUhlM=', 'Content-Length': '0'} 2024-04-16T18:23:24.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000210a0590adbf3274-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.866 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:23:24.866 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:23:24.867 DEBUG:boto:Method: GET 2024-04-16T18:23:24.867 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:23:24.867 DEBUG:boto:Data: 2024-04-16T18:23:24.867 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.867 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.867 DEBUG:boto:Params: {} 2024-04-16T18:23:24.867 DEBUG:boto:Token: None 2024-04-16T18:23:24.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj76 2024-04-16T18:23:24.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O6+gTUThE6/8nDDbeOVOg0HUhlM= 2024-04-16T18:23:24.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O6+gTUThE6/8nDDbeOVOg0HUhlM=', 'Content-Length': '0'} 2024-04-16T18:23:24.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061bf19ad8b62e128-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.871 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.871 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.871 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.871 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.871 DEBUG:boto:Method: GET 2024-04-16T18:23:24.871 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:23:24.871 DEBUG:boto:Data: 2024-04-16T18:23:24.871 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.872 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.872 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.872 DEBUG:boto:Params: {} 2024-04-16T18:23:24.872 DEBUG:boto:Token: None 2024-04-16T18:23:24.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad94a1f0bbd7a041-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.876 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.876 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.876 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.876 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.876 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:23:24.877 DEBUG:boto:Method: GET 2024-04-16T18:23:24.877 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:23:24.877 DEBUG:boto:Data: 2024-04-16T18:23:24.877 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.877 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.877 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.877 DEBUG:boto:Params: {} 2024-04-16T18:23:24.877 DEBUG:boto:Token: None 2024-04-16T18:23:24.877 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.877 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:24.882 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7d072638ccd691e-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.882 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:24.883 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:23:24.883 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:24.883 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:24.883 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.883 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.883 DEBUG:boto:Method: GET 2024-04-16T18:23:24.883 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.883 DEBUG:boto:Data: 2024-04-16T18:23:24.883 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.884 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.884 DEBUG:boto:Params: {} 2024-04-16T18:23:24.884 DEBUG:boto:Token: None 2024-04-16T18:23:24.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q= 2024-04-16T18:23:24.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd7001b453434f4d-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.930 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:24.931 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:24.931 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.931 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.931 DEBUG:boto:Method: GET 2024-04-16T18:23:24.931 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.931 DEBUG:boto:Data: 2024-04-16T18:23:24.931 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.931 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.932 DEBUG:boto:Params: {} 2024-04-16T18:23:24.932 DEBUG:boto:Token: None 2024-04-16T18:23:24.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q= 2024-04-16T18:23:24.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066b94dfc047a3074-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.935 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:24.935 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:24.935 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.935 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.936 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.936 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.936 DEBUG:boto:Data: 2024-04-16T18:23:24.936 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.936 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.936 DEBUG:boto:Params: {} 2024-04-16T18:23:24.936 DEBUG:boto:Token: None 2024-04-16T18:23:24.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OmrCX0CpVYNwJ2laAf7UqZ1IeCA= 2024-04-16T18:23:24.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OmrCX0CpVYNwJ2laAf7UqZ1IeCA=', 'Content-Length': '0'} 2024-04-16T18:23:24.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000969282d07f4ef4f-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.940 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:24.940 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:24.940 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.940 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.940 DEBUG:boto:Method: HEAD 2024-04-16T18:23:24.940 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.940 DEBUG:boto:Data: 2024-04-16T18:23:24.940 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.940 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.940 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.941 DEBUG:boto:Params: {} 2024-04-16T18:23:24.941 DEBUG:boto:Token: None 2024-04-16T18:23:24.941 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.941 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OmrCX0CpVYNwJ2laAf7UqZ1IeCA= 2024-04-16T18:23:24.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OmrCX0CpVYNwJ2laAf7UqZ1IeCA=', 'Content-Length': '0'} 2024-04-16T18:23:24.944 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041e5eb7e33f8d69c-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.944 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:23:24.944 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:24.945 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:24.945 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.945 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.945 DEBUG:boto:Method: GET 2024-04-16T18:23:24.945 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.945 DEBUG:boto:Data: 2024-04-16T18:23:24.945 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.945 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.945 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.945 DEBUG:boto:Params: {} 2024-04-16T18:23:24.946 DEBUG:boto:Token: None 2024-04-16T18:23:24.946 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.946 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q= 2024-04-16T18:23:24.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.949 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023510db22f70b2f9-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.990 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:24.990 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:24.990 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.991 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.991 DEBUG:boto:Method: GET 2024-04-16T18:23:24.991 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.991 DEBUG:boto:Data: 2024-04-16T18:23:24.991 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:24.991 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.991 DEBUG:boto:Params: {} 2024-04-16T18:23:24.991 DEBUG:boto:Token: None 2024-04-16T18:23:24.991 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:23:24.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q= 2024-04-16T18:23:24.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeczFtBaje9gU+EMv4Iit8PTX1Q=', 'Content-Length': '0'} 2024-04-16T18:23:24.995 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdf2fd1ebfc6d7fd-00661ec21c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:24.995 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:24.995 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:24.996 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:24.996 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:24.996 DEBUG:boto:Method: GET 2024-04-16T18:23:24.996 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:23:24.996 DEBUG:boto:Data: 2024-04-16T18:23:24.996 DEBUG:boto:Headers: {} 2024-04-16T18:23:24.996 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:24.996 DEBUG:boto:Port: 8000 2024-04-16T18:23:24.996 DEBUG:boto:Params: {} 2024-04-16T18:23:24.996 DEBUG:boto:Token: None 2024-04-16T18:23:24.997 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:24 GMT /lybtug-52/ 2024-04-16T18:23:24.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys= 2024-04-16T18:23:24.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R7uU4tjvbrn6GtSuCJu4TOxQIys=', 'Content-Length': '0'} 2024-04-16T18:23:25.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea2cf62db2fc33fd-00661ec21c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.042 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.043 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.043 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.043 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:25.043 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:25.043 DEBUG:boto:Method: GET 2024-04-16T18:23:25.043 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:23:25.043 DEBUG:boto:Data: 2024-04-16T18:23:25.043 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.043 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.043 DEBUG:boto:Params: {} 2024-04-16T18:23:25.043 DEBUG:boto:Token: None 2024-04-16T18:23:25.044 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.048 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd7cd64827c0e09b-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.049 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.049 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:25.049 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:25.049 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.049 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:23:25.050 DEBUG:boto:Data: 2024-04-16T18:23:25.050 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.050 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.050 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.050 DEBUG:boto:Params: {} 2024-04-16T18:23:25.050 DEBUG:boto:Token: None 2024-04-16T18:23:25.050 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj85 2024-04-16T18:23:25.050 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P6j0qimZCi1me3xDlk0NXq88uD8= 2024-04-16T18:23:25.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P6j0qimZCi1me3xDlk0NXq88uD8=', 'Content-Length': '0'} 2024-04-16T18:23:25.053 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005632f1327c27ef1c-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.053 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:25.053 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:25.053 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.054 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:23:25.054 DEBUG:boto:Data: 2024-04-16T18:23:25.054 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.054 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.054 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.054 DEBUG:boto:Params: {} 2024-04-16T18:23:25.054 DEBUG:boto:Token: None 2024-04-16T18:23:25.054 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj85 2024-04-16T18:23:25.054 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P6j0qimZCi1me3xDlk0NXq88uD8= 2024-04-16T18:23:25.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P6j0qimZCi1me3xDlk0NXq88uD8=', 'Content-Length': '0'} 2024-04-16T18:23:25.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000999e22ddd3901749-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.057 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:23:25.057 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:25.057 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:25.057 DEBUG:boto:Method: GET 2024-04-16T18:23:25.058 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:23:25.058 DEBUG:boto:Data: 2024-04-16T18:23:25.058 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.058 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.058 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.058 DEBUG:boto:Params: {} 2024-04-16T18:23:25.058 DEBUG:boto:Token: None 2024-04-16T18:23:25.058 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj85 2024-04-16T18:23:25.058 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yc0rYyIMUSZDO7ycVbI7D0m1+Y0= 2024-04-16T18:23:25.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yc0rYyIMUSZDO7ycVbI7D0m1+Y0=', 'Content-Length': '0'} 2024-04-16T18:23:25.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053e4dc005cbd364b-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.102 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:23:25.102 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:23:25.103 DEBUG:boto:Method: GET 2024-04-16T18:23:25.103 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:23:25.103 DEBUG:boto:Data: 2024-04-16T18:23:25.103 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.103 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.103 DEBUG:boto:Params: {} 2024-04-16T18:23:25.103 DEBUG:boto:Token: None 2024-04-16T18:23:25.103 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj85 2024-04-16T18:23:25.103 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yc0rYyIMUSZDO7ycVbI7D0m1+Y0= 2024-04-16T18:23:25.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yc0rYyIMUSZDO7ycVbI7D0m1+Y0=', 'Content-Length': '0'} 2024-04-16T18:23:25.106 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c742dcd6d072dcd-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.107 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.107 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.107 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:25.107 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:25.107 DEBUG:boto:Method: GET 2024-04-16T18:23:25.107 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:23:25.107 DEBUG:boto:Data: 2024-04-16T18:23:25.107 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.107 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.107 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.107 DEBUG:boto:Params: {} 2024-04-16T18:23:25.108 DEBUG:boto:Token: None 2024-04-16T18:23:25.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6eff2d78aa437cf-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.111 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.112 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.112 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.112 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:25.112 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:23:25.112 DEBUG:boto:Method: GET 2024-04-16T18:23:25.112 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:23:25.112 DEBUG:boto:Data: 2024-04-16T18:23:25.112 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.113 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.113 DEBUG:boto:Params: {} 2024-04-16T18:23:25.113 DEBUG:boto:Token: None 2024-04-16T18:23:25.113 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.117 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046e152364a34b4e8-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.118 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.118 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:23:25.118 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.118 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.118 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.118 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.118 DEBUG:boto:Method: GET 2024-04-16T18:23:25.119 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.119 DEBUG:boto:Data: 2024-04-16T18:23:25.119 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.119 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.119 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.119 DEBUG:boto:Params: {} 2024-04-16T18:23:25.119 DEBUG:boto:Token: None 2024-04-16T18:23:25.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY= 2024-04-16T18:23:25.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY=', 'Content-Length': '0'} 2024-04-16T18:23:25.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000296b5f88eaed6662-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.170 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.171 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.171 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.171 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.171 DEBUG:boto:Method: GET 2024-04-16T18:23:25.171 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.171 DEBUG:boto:Data: 2024-04-16T18:23:25.171 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.171 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.171 DEBUG:boto:Params: {} 2024-04-16T18:23:25.171 DEBUG:boto:Token: None 2024-04-16T18:23:25.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY= 2024-04-16T18:23:25.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY=', 'Content-Length': '0'} 2024-04-16T18:23:25.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000653eb08cd8789e34-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.175 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.175 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.176 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.176 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.176 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.176 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.176 DEBUG:boto:Data: 2024-04-16T18:23:25.176 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.176 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.176 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.176 DEBUG:boto:Params: {} 2024-04-16T18:23:25.176 DEBUG:boto:Token: None 2024-04-16T18:23:25.176 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lw9LxV1r7dMlT3DuQpp80OyDYWQ= 2024-04-16T18:23:25.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lw9LxV1r7dMlT3DuQpp80OyDYWQ=', 'Content-Length': '0'} 2024-04-16T18:23:25.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c265878225b40977-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.180 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.180 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.180 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.180 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.180 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.180 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.180 DEBUG:boto:Data: 2024-04-16T18:23:25.180 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.181 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.181 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.181 DEBUG:boto:Params: {} 2024-04-16T18:23:25.181 DEBUG:boto:Token: None 2024-04-16T18:23:25.181 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.181 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lw9LxV1r7dMlT3DuQpp80OyDYWQ= 2024-04-16T18:23:25.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lw9LxV1r7dMlT3DuQpp80OyDYWQ=', 'Content-Length': '0'} 2024-04-16T18:23:25.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000736c866866281258-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.185 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:23:25.185 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.185 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.185 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.185 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.185 DEBUG:boto:Method: GET 2024-04-16T18:23:25.185 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.185 DEBUG:boto:Data: 2024-04-16T18:23:25.185 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.186 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.186 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.186 DEBUG:boto:Params: {} 2024-04-16T18:23:25.186 DEBUG:boto:Token: None 2024-04-16T18:23:25.186 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.186 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY= 2024-04-16T18:23:25.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY=', 'Content-Length': '0'} 2024-04-16T18:23:25.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0635d55f57f110a-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.230 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.230 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.231 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.231 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.231 DEBUG:boto:Method: GET 2024-04-16T18:23:25.231 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.231 DEBUG:boto:Data: 2024-04-16T18:23:25.231 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.231 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.231 DEBUG:boto:Params: {} 2024-04-16T18:23:25.231 DEBUG:boto:Token: None 2024-04-16T18:23:25.231 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:23:25.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY= 2024-04-16T18:23:25.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZRfe2nDG/kUR8uFnUpxuMNpHQZY=', 'Content-Length': '0'} 2024-04-16T18:23:25.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033e73c7054c5c28e-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.235 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.235 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.235 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.235 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.235 DEBUG:boto:Method: GET 2024-04-16T18:23:25.235 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:23:25.235 DEBUG:boto:Data: 2024-04-16T18:23:25.236 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.236 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.236 DEBUG:boto:Params: {} 2024-04-16T18:23:25.236 DEBUG:boto:Token: None 2024-04-16T18:23:25.236 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8ae2ab862377d0b-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.286 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.287 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.287 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.287 DEBUG:boto:Method: GET 2024-04-16T18:23:25.287 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:23:25.287 DEBUG:boto:Data: 2024-04-16T18:23:25.287 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.287 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.287 DEBUG:boto:Params: {} 2024-04-16T18:23:25.287 DEBUG:boto:Token: None 2024-04-16T18:23:25.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2018223ba0ace2f-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.293 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.293 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.293 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.293 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.293 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:23:25.293 DEBUG:boto:Data: 2024-04-16T18:23:25.293 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.293 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.293 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.294 DEBUG:boto:Params: {} 2024-04-16T18:23:25.294 DEBUG:boto:Token: None 2024-04-16T18:23:25.294 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95 2024-04-16T18:23:25.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VdWn1/ihUDmdF9Ecc1e3CAtYVGg= 2024-04-16T18:23:25.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VdWn1/ihUDmdF9Ecc1e3CAtYVGg=', 'Content-Length': '0'} 2024-04-16T18:23:25.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000037aff90f2555822-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.297 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.297 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.297 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.297 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:23:25.297 DEBUG:boto:Data: 2024-04-16T18:23:25.298 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.298 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.298 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.298 DEBUG:boto:Params: {} 2024-04-16T18:23:25.298 DEBUG:boto:Token: None 2024-04-16T18:23:25.298 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95 2024-04-16T18:23:25.298 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VdWn1/ihUDmdF9Ecc1e3CAtYVGg= 2024-04-16T18:23:25.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VdWn1/ihUDmdF9Ecc1e3CAtYVGg=', 'Content-Length': '0'} 2024-04-16T18:23:25.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5007a49b0ba504e-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.301 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:23:25.301 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.301 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.301 DEBUG:boto:Method: GET 2024-04-16T18:23:25.301 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:23:25.301 DEBUG:boto:Data: 2024-04-16T18:23:25.302 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.302 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.302 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.302 DEBUG:boto:Params: {} 2024-04-16T18:23:25.302 DEBUG:boto:Token: None 2024-04-16T18:23:25.302 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95 2024-04-16T18:23:25.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qgWrLVx7pKGx1GwGt7qwWXR4dWY= 2024-04-16T18:23:25.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qgWrLVx7pKGx1GwGt7qwWXR4dWY=', 'Content-Length': '0'} 2024-04-16T18:23:25.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1508ad55856d9d1-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.346 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:23:25.346 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:23:25.347 DEBUG:boto:Method: GET 2024-04-16T18:23:25.347 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:23:25.347 DEBUG:boto:Data: 2024-04-16T18:23:25.347 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.347 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.347 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.347 DEBUG:boto:Params: {} 2024-04-16T18:23:25.347 DEBUG:boto:Token: None 2024-04-16T18:23:25.347 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj95 2024-04-16T18:23:25.347 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qgWrLVx7pKGx1GwGt7qwWXR4dWY= 2024-04-16T18:23:25.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qgWrLVx7pKGx1GwGt7qwWXR4dWY=', 'Content-Length': '0'} 2024-04-16T18:23:25.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a4c472742773b8c-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.351 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.351 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.351 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.351 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.351 DEBUG:boto:Method: GET 2024-04-16T18:23:25.351 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:23:25.351 DEBUG:boto:Data: 2024-04-16T18:23:25.351 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.352 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:23:25.352 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.352 DEBUG:boto:Params: {} 2024-04-16T18:23:25.352 DEBUG:boto:Token: None 2024-04-16T18:23:25.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.355 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f36cd15f0cbe8cd-00661ec21d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.356 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.356 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.356 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.356 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.356 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:23:25.356 DEBUG:boto:Method: GET 2024-04-16T18:23:25.356 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:23:25.357 DEBUG:boto:Data: 2024-04-16T18:23:25.357 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.357 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.357 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.357 DEBUG:boto:Params: {} 2024-04-16T18:23:25.357 DEBUG:boto:Token: None 2024-04-16T18:23:25.357 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI= 2024-04-16T18:23:25.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u3/okZm9PjCo8PHAkUkJeb0eSUI=', 'Content-Length': '0'} 2024-04-16T18:23:25.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aaedebf82888da34-00661ec21d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.362 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:23:25.362 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:23:25.362 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '23', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:23:25.363 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '23', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:23:25.363 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --num-shards 23 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:23:25.409 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:23:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:23:25.511 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:23:25.511 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-52 2024-04-16T18:23:25.525 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 15 2024-04-16T18:23:25.648 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.648 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.648 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.648 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:25.648 DEBUG:boto:Data: 2024-04-16T18:23:25.648 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.649 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.649 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.649 DEBUG:boto:Params: {} 2024-04-16T18:23:25.649 DEBUG:boto:Token: None 2024-04-16T18:23:25.649 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg= 2024-04-16T18:23:25.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg=', 'Content-Length': '0'} 2024-04-16T18:23:25.654 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000806c54500e779e65-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.654 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:23:25.654 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:23:25.654 DEBUG:boto:Method: PUT 2024-04-16T18:23:25.654 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:23:25.654 DEBUG:boto:Data: 2024-04-16T18:23:25.655 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:23:25.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.655 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.655 DEBUG:boto:Params: {} 2024-04-16T18:23:25.655 DEBUG:boto:Token: None 2024-04-16T18:23:25.655 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj23 2024-04-16T18:23:25.655 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H7Iib2r0wQHnPlhBrVORVCxnagM= 2024-04-16T18:23:25.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H7Iib2r0wQHnPlhBrVORVCxnagM='} 2024-04-16T18:23:25.698 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e6a1bc3cee175f56-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.699 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.699 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.699 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.699 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:25.699 DEBUG:boto:Data: 2024-04-16T18:23:25.699 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.699 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.699 DEBUG:boto:Params: {} 2024-04-16T18:23:25.699 DEBUG:boto:Token: None 2024-04-16T18:23:25.700 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg= 2024-04-16T18:23:25.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg=', 'Content-Length': '0'} 2024-04-16T18:23:25.704 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000487d14cbf5cdb7eb-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.704 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:23:25.704 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:23:25.704 DEBUG:boto:Method: PUT 2024-04-16T18:23:25.704 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:23:25.705 DEBUG:boto:Data: 2024-04-16T18:23:25.705 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:23:25.705 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.705 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.705 DEBUG:boto:Params: {} 2024-04-16T18:23:25.705 DEBUG:boto:Token: None 2024-04-16T18:23:25.705 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj46 2024-04-16T18:23:25.705 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PCpifJvXpP7QyeDGc5qXBNpAYo4= 2024-04-16T18:23:25.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PCpifJvXpP7QyeDGc5qXBNpAYo4='} 2024-04-16T18:23:25.716 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002126a108d9928b41-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.717 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.717 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.717 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.717 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:25.717 DEBUG:boto:Data: 2024-04-16T18:23:25.717 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.717 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.717 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.717 DEBUG:boto:Params: {} 2024-04-16T18:23:25.717 DEBUG:boto:Token: None 2024-04-16T18:23:25.718 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg= 2024-04-16T18:23:25.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg=', 'Content-Length': '0'} 2024-04-16T18:23:25.722 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '51'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ebf225db9d62081a-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.722 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:23:25.722 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:23:25.722 DEBUG:boto:Method: PUT 2024-04-16T18:23:25.722 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:23:25.722 DEBUG:boto:Data: 2024-04-16T18:23:25.722 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:23:25.723 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.723 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.723 DEBUG:boto:Params: {} 2024-04-16T18:23:25.723 DEBUG:boto:Token: None 2024-04-16T18:23:25.723 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj69 2024-04-16T18:23:25.723 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OLAY6ruIvrSaJndAfNNLboKPs34= 2024-04-16T18:23:25.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OLAY6ruIvrSaJndAfNNLboKPs34='} 2024-04-16T18:23:25.766 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004ec84b1644534dc3-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.767 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.767 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.767 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.767 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:25.767 DEBUG:boto:Data: 2024-04-16T18:23:25.767 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.767 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.767 DEBUG:boto:Params: {} 2024-04-16T18:23:25.767 DEBUG:boto:Token: None 2024-04-16T18:23:25.768 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg= 2024-04-16T18:23:25.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg=', 'Content-Length': '0'} 2024-04-16T18:23:25.772 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a89b3c432cb86e80-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.772 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:23:25.772 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:23:25.773 DEBUG:boto:Method: PUT 2024-04-16T18:23:25.773 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:23:25.773 DEBUG:boto:Data: 2024-04-16T18:23:25.773 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:23:25.773 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.773 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.773 DEBUG:boto:Params: {} 2024-04-16T18:23:25.773 DEBUG:boto:Token: None 2024-04-16T18:23:25.773 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj92 2024-04-16T18:23:25.773 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:A6qqvOOlkcZ6Agd6Nqk8sD3oBmc= 2024-04-16T18:23:25.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A6qqvOOlkcZ6Agd6Nqk8sD3oBmc='} 2024-04-16T18:23:25.785 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d2d6898c3a6a10de-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.786 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:23:25.786 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:23:25.786 DEBUG:boto:Method: HEAD 2024-04-16T18:23:25.786 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:23:25.786 DEBUG:boto:Data: 2024-04-16T18:23:25.786 DEBUG:boto:Headers: {} 2024-04-16T18:23:25.786 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.786 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.786 DEBUG:boto:Params: {} 2024-04-16T18:23:25.786 DEBUG:boto:Token: None 2024-04-16T18:23:25.787 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/ 2024-04-16T18:23:25.787 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg= 2024-04-16T18:23:25.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iGwCnpeJpB7qCgAxUJH4ZLROKIg=', 'Content-Length': '0'} 2024-04-16T18:23:25.791 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '57'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b1f2c5e02b719677-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.792 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:23:25.792 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:23:25.792 DEBUG:boto:Method: PUT 2024-04-16T18:23:25.792 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:23:25.792 DEBUG:boto:Data: 2024-04-16T18:23:25.792 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:23:25.792 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:23:25.792 DEBUG:boto:Port: 8000 2024-04-16T18:23:25.792 DEBUG:boto:Params: {} 2024-04-16T18:23:25.792 DEBUG:boto:Token: None 2024-04-16T18:23:25.793 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:23:25 GMT /lybtug-52/obj115 2024-04-16T18:23:25.793 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Pv0vGQKLQbhRTEfm29DavnaV7+E= 2024-04-16T18:23:25.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:23:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Pv0vGQKLQbhRTEfm29DavnaV7+E='} 2024-04-16T18:23:25.838 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000564cb3edb2402bea-00661ec21d-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:23:25.839 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:23:25.839 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:23:25.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:23:25.839 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:23:25.889 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:23:25.889 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:23:25.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:25.971+0000 7f648a5b1980 1 waiting to reach latest gen 3, on 2.. 2024-04-16T18:23:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:30.975+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:30.981 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:30.982 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:23:35.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:35.979+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:35.977 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:35.978 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:23:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:40.983+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:40.980 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:40.980 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:23:45.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:45.983+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:45.983 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:45.983 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:23:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:50.986+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:50.985 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:50.986 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:23:55.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:23:55.990+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:23:55.988 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , , , , , ] 2024-04-16T18:23:55.989 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:00.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:00.990+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:00.991 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , , , , , ] 2024-04-16T18:24:00.991 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:05.994+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:05.994 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , , , , , ] 2024-04-16T18:24:05.994 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:10.998+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:10.997 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , , , , , ] 2024-04-16T18:24:10.997 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:15.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:15.998+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:16.000 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , 00000000001.57.6, , , , ] 2024-04-16T18:24:16.000 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:21.002+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:21.003 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , 00000000001.57.6, , , , ] 2024-04-16T18:24:21.003 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:26.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:26.005+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:26.005 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , 00000000001.57.6, , , , ] 2024-04-16T18:24:26.005 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:31.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:31.009+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:31.008 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , , , , , , , , , , , 00000000001.57.6, , , , ] 2024-04-16T18:24:31.008 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:36.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:36.009+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:36.011 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:36.011 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:41.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:41.013+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:41.014 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:41.014 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:46.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:46.021+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:46.017 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:46.017 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:51.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:51.025+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:51.020 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:51.021 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:56.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:24:56.024+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:24:56.023 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:24:56.023 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:01.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:01.028+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:01.026 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:01.026 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:06.032+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:06.030 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:06.030 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:11.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:11.032+0000 7f648a5b1980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:11.032 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , , , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:11.033 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:16.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:16.036+0000 7f648a5b1980 1 bucket sync caught up with source: 2024-04-16T18:25:16.035 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:16.036 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , 00000000001.109.6, , , 00000000002.87.6, , 00000000002.76.6, , , , , , 00000000001.81.6, , 00000000001.57.6, , , , ] 2024-04-16T18:25:16.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:16.036+0000 7f648a5b1980 0 bucket checkpoint complete 2024-04-16T18:25:16.079 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:25:16.079 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:25:16.080 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:25:16.080 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:25:16.132 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:25:16.132 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:25:16.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:25:16.218+0000 7fcb98081980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:16.213 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000002.1221.6, 00000000002.519.6, , , 00000000002.466.6, 00000000005.327.6, 00000000003.410.6] 2024-04-16T18:25:16.213 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000002.1221.6, 00000000002.519.6, , , 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:25:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:25:21.218+0000 7fcb98081980 1 bucket sync caught up with source: 2024-04-16T18:25:21.216 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000002.1221.6, 00000000002.519.6, , , 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:25:21.216 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000002.1221.6, 00000000002.519.6, , , 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:25:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:25:21.218+0000 7fcb98081980 0 bucket checkpoint complete 2024-04-16T18:25:21.263 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:25:21.264 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.264 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.264 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.264 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:21.264 DEBUG:boto:Data: 2024-04-16T18:25:21.265 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.265 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.265 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.265 DEBUG:boto:Params: {} 2024-04-16T18:25:21.266 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:25:21.266 DEBUG:boto:Token: None 2024-04-16T18:25:21.266 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.267 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K8TUZUzV4zjwmYE7kYH0BNDDf4Q= 2024-04-16T18:25:21.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K8TUZUzV4zjwmYE7kYH0BNDDf4Q=', 'Content-Length': '0'} 2024-04-16T18:25:21.275 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000246dafb0a89cc32e-00661ec291-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.275 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.275 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.275 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.275 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:21.275 DEBUG:boto:Data: 2024-04-16T18:25:21.276 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.276 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.276 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.276 DEBUG:boto:Params: {} 2024-04-16T18:25:21.276 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:25:21.277 DEBUG:boto:Token: None 2024-04-16T18:25:21.277 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K8TUZUzV4zjwmYE7kYH0BNDDf4Q= 2024-04-16T18:25:21.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K8TUZUzV4zjwmYE7kYH0BNDDf4Q=', 'Content-Length': '0'} 2024-04-16T18:25:21.283 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008f01da84c95d97ad-00661ec291-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.283 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:25:21.284 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.284 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.284 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.284 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.284 DEBUG:boto:Method: GET 2024-04-16T18:25:21.284 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.284 DEBUG:boto:Data: 2024-04-16T18:25:21.285 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.285 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.285 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.285 DEBUG:boto:Params: {} 2024-04-16T18:25:21.285 DEBUG:boto:Token: None 2024-04-16T18:25:21.285 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/?versions 2024-04-16T18:25:21.286 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8= 2024-04-16T18:25:21.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8=', 'Content-Length': '0'} 2024-04-16T18:25:21.290 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045d550b4d7f6ffe6-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.290 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.292 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:25:21.292 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:25:21.292 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:25:21.292 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:25:21.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:25:21.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:25:21.295 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:25:21.295 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.295 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.295 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.295 DEBUG:boto:Method: GET 2024-04-16T18:25:21.295 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.295 DEBUG:boto:Data: 2024-04-16T18:25:21.296 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.296 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.296 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.296 DEBUG:boto:Params: {} 2024-04-16T18:25:21.296 DEBUG:boto:Token: None 2024-04-16T18:25:21.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/?versions 2024-04-16T18:25:21.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8= 2024-04-16T18:25:21.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8=', 'Content-Length': '0'} 2024-04-16T18:25:21.302 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002b10106dab0e3c9-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.303 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:25:21.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:25:21.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:25:21.306 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:25:21.306 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:25:21.306 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:25:21.306 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:25:21.306 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:25:21.307 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.307 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.307 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.307 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.307 DEBUG:boto:Method: GET 2024-04-16T18:25:21.307 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.307 DEBUG:boto:Data: 2024-04-16T18:25:21.307 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.308 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.308 DEBUG:boto:Params: {} 2024-04-16T18:25:21.308 DEBUG:boto:Token: None 2024-04-16T18:25:21.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/?versions 2024-04-16T18:25:21.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8= 2024-04-16T18:25:21.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8=', 'Content-Length': '0'} 2024-04-16T18:25:21.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000762d28af44e152b1-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.314 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.315 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.316 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.316 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.316 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.316 DEBUG:boto:Method: GET 2024-04-16T18:25:21.316 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:25:21.316 DEBUG:boto:Data: 2024-04-16T18:25:21.316 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.317 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.317 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.317 DEBUG:boto:Params: {} 2024-04-16T18:25:21.317 DEBUG:boto:Token: None 2024-04-16T18:25:21.317 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/?versions 2024-04-16T18:25:21.317 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8= 2024-04-16T18:25:21.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1eXi8dQzjeNMqzlYenbt/Mn0Jf8=', 'Content-Length': '0'} 2024-04-16T18:25:21.324 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f432697ee2924854-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.325 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:25:21.326 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.326 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.326 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.326 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.327 DEBUG:boto:Method: GET 2024-04-16T18:25:21.327 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.327 DEBUG:boto:Data: 2024-04-16T18:25:21.327 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.327 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.327 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.327 DEBUG:boto:Params: {} 2024-04-16T18:25:21.328 DEBUG:boto:Token: None 2024-04-16T18:25:21.328 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88= 2024-04-16T18:25:21.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88=', 'Content-Length': '0'} 2024-04-16T18:25:21.332 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eee6e2c0b4c9d454-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.332 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.332 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.333 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.333 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.333 DEBUG:boto:Method: GET 2024-04-16T18:25:21.333 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.333 DEBUG:boto:Data: 2024-04-16T18:25:21.333 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.333 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.333 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.334 DEBUG:boto:Params: {} 2024-04-16T18:25:21.334 DEBUG:boto:Token: None 2024-04-16T18:25:21.334 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.334 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88= 2024-04-16T18:25:21.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88=', 'Content-Length': '0'} 2024-04-16T18:25:21.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017a481d283fc983f-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.378 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.379 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.379 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.379 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.379 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.379 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.379 DEBUG:boto:Data: 2024-04-16T18:25:21.379 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.380 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.380 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.380 DEBUG:boto:Params: {} 2024-04-16T18:25:21.380 DEBUG:boto:Token: None 2024-04-16T18:25:21.380 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oZsbHfqr5GXkV1UMmvK49wKyDnE= 2024-04-16T18:25:21.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oZsbHfqr5GXkV1UMmvK49wKyDnE=', 'Content-Length': '0'} 2024-04-16T18:25:21.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096da6ca160bc7176-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.384 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.385 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.385 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.385 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.385 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.385 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.385 DEBUG:boto:Data: 2024-04-16T18:25:21.385 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.385 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.386 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.386 DEBUG:boto:Params: {} 2024-04-16T18:25:21.386 DEBUG:boto:Token: None 2024-04-16T18:25:21.386 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.386 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oZsbHfqr5GXkV1UMmvK49wKyDnE= 2024-04-16T18:25:21.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oZsbHfqr5GXkV1UMmvK49wKyDnE=', 'Content-Length': '0'} 2024-04-16T18:25:21.390 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000423b4552f99ccd86-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.390 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:25:21.390 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.390 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.391 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.391 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.391 DEBUG:boto:Method: GET 2024-04-16T18:25:21.391 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.391 DEBUG:boto:Data: 2024-04-16T18:25:21.391 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.391 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.391 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.392 DEBUG:boto:Params: {} 2024-04-16T18:25:21.392 DEBUG:boto:Token: None 2024-04-16T18:25:21.392 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88= 2024-04-16T18:25:21.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88=', 'Content-Length': '0'} 2024-04-16T18:25:21.396 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080132ada547a3fed-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.438 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.439 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.439 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.439 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.439 DEBUG:boto:Method: GET 2024-04-16T18:25:21.439 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.439 DEBUG:boto:Data: 2024-04-16T18:25:21.439 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.440 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.440 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.440 DEBUG:boto:Params: {} 2024-04-16T18:25:21.440 DEBUG:boto:Token: None 2024-04-16T18:25:21.440 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:25:21.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88= 2024-04-16T18:25:21.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H3jBZjbBRVD8a8m35KsVaelAw88=', 'Content-Length': '0'} 2024-04-16T18:25:21.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ca14516ccfa2342-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.444 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.444 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.445 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.445 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.445 DEBUG:boto:Method: GET 2024-04-16T18:25:21.445 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:25:21.445 DEBUG:boto:Data: 2024-04-16T18:25:21.445 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.446 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.446 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.446 DEBUG:boto:Params: {} 2024-04-16T18:25:21.446 DEBUG:boto:Token: None 2024-04-16T18:25:21.446 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.446 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.450 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a82a13ce2472c40-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.451 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.451 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.451 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.451 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.452 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.452 DEBUG:boto:Method: GET 2024-04-16T18:25:21.452 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:25:21.452 DEBUG:boto:Data: 2024-04-16T18:25:21.452 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.452 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.452 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.452 DEBUG:boto:Params: {} 2024-04-16T18:25:21.453 DEBUG:boto:Token: None 2024-04-16T18:25:21.453 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.453 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.458 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000126675343f981333-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.459 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.459 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.459 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.459 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.460 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:25:21.460 DEBUG:boto:Data: 2024-04-16T18:25:21.460 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.460 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.460 DEBUG:boto:Params: {} 2024-04-16T18:25:21.460 DEBUG:boto:Token: None 2024-04-16T18:25:21.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11 2024-04-16T18:25:21.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RAUtQYi+c1qGjTpjRWywg9atVpI= 2024-04-16T18:25:21.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RAUtQYi+c1qGjTpjRWywg9atVpI=', 'Content-Length': '0'} 2024-04-16T18:25:21.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083cb23a10bf60244-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.464 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.465 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.465 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.465 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:25:21.465 DEBUG:boto:Data: 2024-04-16T18:25:21.465 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.465 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.465 DEBUG:boto:Params: {} 2024-04-16T18:25:21.466 DEBUG:boto:Token: None 2024-04-16T18:25:21.466 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11 2024-04-16T18:25:21.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RAUtQYi+c1qGjTpjRWywg9atVpI= 2024-04-16T18:25:21.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RAUtQYi+c1qGjTpjRWywg9atVpI=', 'Content-Length': '0'} 2024-04-16T18:25:21.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024012d0e0265d846-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.470 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:25:21.470 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.470 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.470 DEBUG:boto:Method: GET 2024-04-16T18:25:21.471 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:25:21.471 DEBUG:boto:Data: 2024-04-16T18:25:21.471 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.471 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.471 DEBUG:boto:Params: {} 2024-04-16T18:25:21.471 DEBUG:boto:Token: None 2024-04-16T18:25:21.471 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11 2024-04-16T18:25:21.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T6AWL3AeEHux6ZaFAvQ0hnrBCO0= 2024-04-16T18:25:21.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T6AWL3AeEHux6ZaFAvQ0hnrBCO0=', 'Content-Length': '0'} 2024-04-16T18:25:21.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043cf3b6da551824d-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.518 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:25:21.519 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:25:21.519 DEBUG:boto:Method: GET 2024-04-16T18:25:21.519 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:25:21.519 DEBUG:boto:Data: 2024-04-16T18:25:21.519 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.519 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.519 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.520 DEBUG:boto:Params: {} 2024-04-16T18:25:21.520 DEBUG:boto:Token: None 2024-04-16T18:25:21.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj11 2024-04-16T18:25:21.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:T6AWL3AeEHux6ZaFAvQ0hnrBCO0= 2024-04-16T18:25:21.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T6AWL3AeEHux6ZaFAvQ0hnrBCO0=', 'Content-Length': '0'} 2024-04-16T18:25:21.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020f60dc5702c6f0c-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.524 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.524 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.524 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.524 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.525 DEBUG:boto:Method: GET 2024-04-16T18:25:21.525 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:25:21.525 DEBUG:boto:Data: 2024-04-16T18:25:21.525 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.525 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.525 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.525 DEBUG:boto:Params: {} 2024-04-16T18:25:21.525 DEBUG:boto:Token: None 2024-04-16T18:25:21.526 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.526 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ec527b1684e0114-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.530 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.531 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.531 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.531 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.531 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:25:21.531 DEBUG:boto:Method: GET 2024-04-16T18:25:21.531 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:25:21.532 DEBUG:boto:Data: 2024-04-16T18:25:21.532 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.532 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.532 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.532 DEBUG:boto:Params: {} 2024-04-16T18:25:21.532 DEBUG:boto:Token: None 2024-04-16T18:25:21.532 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.533 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.537 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057d476f01c48920f-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.538 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:25:21.538 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.538 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.539 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.539 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.539 DEBUG:boto:Method: GET 2024-04-16T18:25:21.539 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.539 DEBUG:boto:Data: 2024-04-16T18:25:21.539 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.539 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.539 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.539 DEBUG:boto:Params: {} 2024-04-16T18:25:21.540 DEBUG:boto:Token: None 2024-04-16T18:25:21.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o= 2024-04-16T18:25:21.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o=', 'Content-Length': '0'} 2024-04-16T18:25:21.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d314462063e46b2-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.544 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.544 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.544 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.544 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.544 DEBUG:boto:Method: GET 2024-04-16T18:25:21.545 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.545 DEBUG:boto:Data: 2024-04-16T18:25:21.545 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.545 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.545 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.545 DEBUG:boto:Params: {} 2024-04-16T18:25:21.545 DEBUG:boto:Token: None 2024-04-16T18:25:21.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.546 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o= 2024-04-16T18:25:21.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o=', 'Content-Length': '0'} 2024-04-16T18:25:21.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b561befbf2fef015-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.549 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.549 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.549 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.549 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.550 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.550 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.550 DEBUG:boto:Data: 2024-04-16T18:25:21.550 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.550 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.550 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.550 DEBUG:boto:Params: {} 2024-04-16T18:25:21.551 DEBUG:boto:Token: None 2024-04-16T18:25:21.551 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.551 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dmk2DmceZZxUls8/NdOZX7UH8Cg= 2024-04-16T18:25:21.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dmk2DmceZZxUls8/NdOZX7UH8Cg=', 'Content-Length': '0'} 2024-04-16T18:25:21.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cde1ac207e53c690-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.555 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.555 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.555 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.555 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.555 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.555 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.555 DEBUG:boto:Data: 2024-04-16T18:25:21.556 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.556 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.556 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.556 DEBUG:boto:Params: {} 2024-04-16T18:25:21.556 DEBUG:boto:Token: None 2024-04-16T18:25:21.556 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dmk2DmceZZxUls8/NdOZX7UH8Cg= 2024-04-16T18:25:21.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dmk2DmceZZxUls8/NdOZX7UH8Cg=', 'Content-Length': '0'} 2024-04-16T18:25:21.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9e6ae7187f5f14e-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.560 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:25:21.560 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.560 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.561 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.561 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.561 DEBUG:boto:Method: GET 2024-04-16T18:25:21.561 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.561 DEBUG:boto:Data: 2024-04-16T18:25:21.561 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.561 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.561 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.561 DEBUG:boto:Params: {} 2024-04-16T18:25:21.562 DEBUG:boto:Token: None 2024-04-16T18:25:21.562 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.562 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o= 2024-04-16T18:25:21.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o=', 'Content-Length': '0'} 2024-04-16T18:25:21.565 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000367d6f2ff0286f10-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.606 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.607 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.607 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.607 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.607 DEBUG:boto:Method: GET 2024-04-16T18:25:21.607 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.607 DEBUG:boto:Data: 2024-04-16T18:25:21.607 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.607 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.607 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.608 DEBUG:boto:Params: {} 2024-04-16T18:25:21.608 DEBUG:boto:Token: None 2024-04-16T18:25:21.608 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:25:21.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o= 2024-04-16T18:25:21.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yJuDADDibJtIj2WtLZs42yCxg9o=', 'Content-Length': '0'} 2024-04-16T18:25:21.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc5cda543693cc73-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.612 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.612 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.613 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.613 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.613 DEBUG:boto:Method: GET 2024-04-16T18:25:21.613 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:25:21.613 DEBUG:boto:Data: 2024-04-16T18:25:21.613 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.613 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.613 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.614 DEBUG:boto:Params: {} 2024-04-16T18:25:21.614 DEBUG:boto:Token: None 2024-04-16T18:25:21.614 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.614 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eac1d8bf8df668ca-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.619 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.619 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.619 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.619 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.619 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.620 DEBUG:boto:Method: GET 2024-04-16T18:25:21.620 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:25:21.620 DEBUG:boto:Data: 2024-04-16T18:25:21.620 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.620 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.620 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.620 DEBUG:boto:Params: {} 2024-04-16T18:25:21.620 DEBUG:boto:Token: None 2024-04-16T18:25:21.621 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.621 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.626 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8153ca5742ef1f0-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.666 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.667 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.667 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.667 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.667 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:25:21.668 DEBUG:boto:Data: 2024-04-16T18:25:21.668 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.668 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.668 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.668 DEBUG:boto:Params: {} 2024-04-16T18:25:21.669 DEBUG:boto:Token: None 2024-04-16T18:25:21.669 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115 2024-04-16T18:25:21.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+Eok2Mkox72hkaJONhwZB6lCKuY= 2024-04-16T18:25:21.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+Eok2Mkox72hkaJONhwZB6lCKuY=', 'Content-Length': '0'} 2024-04-16T18:25:21.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001250d247a6f7b39a-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.673 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.673 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.673 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.673 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:25:21.673 DEBUG:boto:Data: 2024-04-16T18:25:21.673 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.674 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.674 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.674 DEBUG:boto:Params: {} 2024-04-16T18:25:21.674 DEBUG:boto:Token: None 2024-04-16T18:25:21.674 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115 2024-04-16T18:25:21.674 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+Eok2Mkox72hkaJONhwZB6lCKuY= 2024-04-16T18:25:21.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+Eok2Mkox72hkaJONhwZB6lCKuY=', 'Content-Length': '0'} 2024-04-16T18:25:21.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e4bc3a3e704d050-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.678 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:25:21.678 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.678 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.679 DEBUG:boto:Method: GET 2024-04-16T18:25:21.679 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:25:21.679 DEBUG:boto:Data: 2024-04-16T18:25:21.679 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.679 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.679 DEBUG:boto:Params: {} 2024-04-16T18:25:21.680 DEBUG:boto:Token: None 2024-04-16T18:25:21.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115 2024-04-16T18:25:21.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:stkLy49ErkC9YgiUnMp4ytgY6Ow= 2024-04-16T18:25:21.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:stkLy49ErkC9YgiUnMp4ytgY6Ow=', 'Content-Length': '0'} 2024-04-16T18:25:21.684 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1c21a9295b758f5-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.727 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:25:21.727 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:25:21.727 DEBUG:boto:Method: GET 2024-04-16T18:25:21.727 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:25:21.727 DEBUG:boto:Data: 2024-04-16T18:25:21.727 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.728 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.728 DEBUG:boto:Params: {} 2024-04-16T18:25:21.728 DEBUG:boto:Token: None 2024-04-16T18:25:21.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj115 2024-04-16T18:25:21.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:stkLy49ErkC9YgiUnMp4ytgY6Ow= 2024-04-16T18:25:21.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:stkLy49ErkC9YgiUnMp4ytgY6Ow=', 'Content-Length': '0'} 2024-04-16T18:25:21.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fbc691bb64028b5-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.732 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.732 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.732 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.733 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.733 DEBUG:boto:Method: GET 2024-04-16T18:25:21.733 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:25:21.733 DEBUG:boto:Data: 2024-04-16T18:25:21.733 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.733 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.733 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.733 DEBUG:boto:Params: {} 2024-04-16T18:25:21.734 DEBUG:boto:Token: None 2024-04-16T18:25:21.734 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.734 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013c8e9528778ff4a-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.779 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.779 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.779 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.779 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.779 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:25:21.780 DEBUG:boto:Method: GET 2024-04-16T18:25:21.780 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:25:21.780 DEBUG:boto:Data: 2024-04-16T18:25:21.780 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.780 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.780 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.780 DEBUG:boto:Params: {} 2024-04-16T18:25:21.781 DEBUG:boto:Token: None 2024-04-16T18:25:21.781 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5deaa74e3999a80-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.787 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.787 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:25:21.787 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.787 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.788 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.788 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.788 DEBUG:boto:Method: GET 2024-04-16T18:25:21.788 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.788 DEBUG:boto:Data: 2024-04-16T18:25:21.788 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.789 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.789 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.789 DEBUG:boto:Params: {} 2024-04-16T18:25:21.789 DEBUG:boto:Token: None 2024-04-16T18:25:21.789 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.789 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY= 2024-04-16T18:25:21.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY=', 'Content-Length': '0'} 2024-04-16T18:25:21.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062356a8d38f8a0bc-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.834 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.835 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.835 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.835 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.835 DEBUG:boto:Method: GET 2024-04-16T18:25:21.835 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.835 DEBUG:boto:Data: 2024-04-16T18:25:21.835 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.836 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.836 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.836 DEBUG:boto:Params: {} 2024-04-16T18:25:21.836 DEBUG:boto:Token: None 2024-04-16T18:25:21.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY= 2024-04-16T18:25:21.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY=', 'Content-Length': '0'} 2024-04-16T18:25:21.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6e0f23ad8fcacc5-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.840 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.840 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.840 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.841 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.841 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.841 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.841 DEBUG:boto:Data: 2024-04-16T18:25:21.841 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.841 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.841 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.841 DEBUG:boto:Params: {} 2024-04-16T18:25:21.842 DEBUG:boto:Token: None 2024-04-16T18:25:21.842 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dWHVM2IZosAMzLOSMDjmMNAykeI= 2024-04-16T18:25:21.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dWHVM2IZosAMzLOSMDjmMNAykeI=', 'Content-Length': '0'} 2024-04-16T18:25:21.845 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af418259aea63b53-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.845 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.845 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.845 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.846 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.846 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.846 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.846 DEBUG:boto:Data: 2024-04-16T18:25:21.846 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.846 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.846 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.846 DEBUG:boto:Params: {} 2024-04-16T18:25:21.847 DEBUG:boto:Token: None 2024-04-16T18:25:21.847 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dWHVM2IZosAMzLOSMDjmMNAykeI= 2024-04-16T18:25:21.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dWHVM2IZosAMzLOSMDjmMNAykeI=', 'Content-Length': '0'} 2024-04-16T18:25:21.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000429685ae45ebf93-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.850 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:25:21.851 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.851 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.851 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.851 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.851 DEBUG:boto:Method: GET 2024-04-16T18:25:21.851 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.851 DEBUG:boto:Data: 2024-04-16T18:25:21.851 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.852 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.852 DEBUG:boto:Params: {} 2024-04-16T18:25:21.852 DEBUG:boto:Token: None 2024-04-16T18:25:21.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY= 2024-04-16T18:25:21.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY=', 'Content-Length': '0'} 2024-04-16T18:25:21.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050bbcd88f077de02-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.898 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.898 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.899 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.899 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.899 DEBUG:boto:Method: GET 2024-04-16T18:25:21.899 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.899 DEBUG:boto:Data: 2024-04-16T18:25:21.899 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.899 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.899 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.899 DEBUG:boto:Params: {} 2024-04-16T18:25:21.899 DEBUG:boto:Token: None 2024-04-16T18:25:21.900 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:25:21.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY= 2024-04-16T18:25:21.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z1Q65PNOQqGtH6nsId/+engTmyY=', 'Content-Length': '0'} 2024-04-16T18:25:21.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e73e370dae35505-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.903 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.903 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.903 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:21.903 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:21.903 DEBUG:boto:Method: GET 2024-04-16T18:25:21.904 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:25:21.904 DEBUG:boto:Data: 2024-04-16T18:25:21.904 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.904 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.904 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.904 DEBUG:boto:Params: {} 2024-04-16T18:25:21.904 DEBUG:boto:Token: None 2024-04-16T18:25:21.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.908 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e1e21439fbcd0cb-00661ec291-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.950 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.951 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:21.951 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:21.951 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:21.951 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:21.951 DEBUG:boto:Method: GET 2024-04-16T18:25:21.951 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:25:21.951 DEBUG:boto:Data: 2024-04-16T18:25:21.952 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.952 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.952 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.952 DEBUG:boto:Params: {} 2024-04-16T18:25:21.952 DEBUG:boto:Token: None 2024-04-16T18:25:21.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/ 2024-04-16T18:25:21.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg= 2024-04-16T18:25:21.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cY3m8iDzzKDJYijrqiKbIt4xsBg=', 'Content-Length': '0'} 2024-04-16T18:25:21.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062143e2335a78bf6-00661ec291-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.957 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:21.958 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.958 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.958 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.958 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:25:21.958 DEBUG:boto:Data: 2024-04-16T18:25:21.958 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.958 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.959 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.959 DEBUG:boto:Params: {} 2024-04-16T18:25:21.959 DEBUG:boto:Token: None 2024-04-16T18:25:21.959 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17 2024-04-16T18:25:21.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rVGyz6AVCTfnMiYEGHogWr9COnw= 2024-04-16T18:25:21.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rVGyz6AVCTfnMiYEGHogWr9COnw=', 'Content-Length': '0'} 2024-04-16T18:25:21.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc901e7ac1448f29-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.962 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.962 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.962 DEBUG:boto:Method: HEAD 2024-04-16T18:25:21.962 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:25:21.962 DEBUG:boto:Data: 2024-04-16T18:25:21.963 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:21.963 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.963 DEBUG:boto:Params: {} 2024-04-16T18:25:21.963 DEBUG:boto:Token: None 2024-04-16T18:25:21.963 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17 2024-04-16T18:25:21.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rVGyz6AVCTfnMiYEGHogWr9COnw= 2024-04-16T18:25:21.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rVGyz6AVCTfnMiYEGHogWr9COnw=', 'Content-Length': '0'} 2024-04-16T18:25:21.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000085201f25eb3b022-00661ec291-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:21.966 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:25:21.966 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:21.966 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:21.967 DEBUG:boto:Method: GET 2024-04-16T18:25:21.967 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:25:21.967 DEBUG:boto:Data: 2024-04-16T18:25:21.967 DEBUG:boto:Headers: {} 2024-04-16T18:25:21.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:21.967 DEBUG:boto:Port: 8000 2024-04-16T18:25:21.967 DEBUG:boto:Params: {} 2024-04-16T18:25:21.967 DEBUG:boto:Token: None 2024-04-16T18:25:21.967 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:21 GMT /lybtug-52/obj17 2024-04-16T18:25:21.967 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sgyg6I6fHNaJfOKKK2Cy42i3k/I= 2024-04-16T18:25:21.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sgyg6I6fHNaJfOKKK2Cy42i3k/I=', 'Content-Length': '0'} 2024-04-16T18:25:21.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d57d7406e8a99e4-00661ec291-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.014 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:25:22.014 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:25:22.015 DEBUG:boto:Method: GET 2024-04-16T18:25:22.015 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:25:22.015 DEBUG:boto:Data: 2024-04-16T18:25:22.015 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.015 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.015 DEBUG:boto:Params: {} 2024-04-16T18:25:22.015 DEBUG:boto:Token: None 2024-04-16T18:25:22.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj17 2024-04-16T18:25:22.015 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZQDmcf1ClsY9xNhv+FEJaV7mN2Q= 2024-04-16T18:25:22.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZQDmcf1ClsY9xNhv+FEJaV7mN2Q=', 'Content-Length': '0'} 2024-04-16T18:25:22.018 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000caaa9e02e430963d-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.021 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.021 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.021 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:22.022 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:22.022 DEBUG:boto:Method: GET 2024-04-16T18:25:22.022 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:25:22.022 DEBUG:boto:Data: 2024-04-16T18:25:22.022 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.022 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.022 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.022 DEBUG:boto:Params: {} 2024-04-16T18:25:22.022 DEBUG:boto:Token: None 2024-04-16T18:25:22.023 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.023 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.027 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4211e00e3efa32f-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.027 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.028 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.028 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.028 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:22.028 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:25:22.029 DEBUG:boto:Method: GET 2024-04-16T18:25:22.029 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:25:22.029 DEBUG:boto:Data: 2024-04-16T18:25:22.029 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.029 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.029 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.029 DEBUG:boto:Params: {} 2024-04-16T18:25:22.029 DEBUG:boto:Token: None 2024-04-16T18:25:22.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.034 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5f9ddcfd6be03ea-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.034 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.035 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:25:22.035 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.035 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.035 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.035 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.035 DEBUG:boto:Method: GET 2024-04-16T18:25:22.035 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.035 DEBUG:boto:Data: 2024-04-16T18:25:22.036 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.036 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.036 DEBUG:boto:Params: {} 2024-04-16T18:25:22.036 DEBUG:boto:Token: None 2024-04-16T18:25:22.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts= 2024-04-16T18:25:22.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts=', 'Content-Length': '0'} 2024-04-16T18:25:22.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a6d7e45ab46bc2e-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.083 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.083 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.083 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.083 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.083 DEBUG:boto:Method: GET 2024-04-16T18:25:22.083 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.083 DEBUG:boto:Data: 2024-04-16T18:25:22.083 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.083 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.083 DEBUG:boto:Params: {} 2024-04-16T18:25:22.084 DEBUG:boto:Token: None 2024-04-16T18:25:22.084 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts= 2024-04-16T18:25:22.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts=', 'Content-Length': '0'} 2024-04-16T18:25:22.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025a72301e93a8cd9-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.087 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.087 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.087 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.087 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.087 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.087 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.087 DEBUG:boto:Data: 2024-04-16T18:25:22.087 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.088 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.088 DEBUG:boto:Params: {} 2024-04-16T18:25:22.088 DEBUG:boto:Token: None 2024-04-16T18:25:22.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcPr/y8luM6+ulrGMLHnQc6yf7E= 2024-04-16T18:25:22.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcPr/y8luM6+ulrGMLHnQc6yf7E=', 'Content-Length': '0'} 2024-04-16T18:25:22.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000471081af7d239ab8-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.093 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.093 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.093 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.093 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.093 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.093 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.093 DEBUG:boto:Data: 2024-04-16T18:25:22.093 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.093 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.093 DEBUG:boto:Params: {} 2024-04-16T18:25:22.094 DEBUG:boto:Token: None 2024-04-16T18:25:22.094 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gcPr/y8luM6+ulrGMLHnQc6yf7E= 2024-04-16T18:25:22.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gcPr/y8luM6+ulrGMLHnQc6yf7E=', 'Content-Length': '0'} 2024-04-16T18:25:22.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000176e3fdab08bdffd-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:25:22.097 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.097 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.097 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.097 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.098 DEBUG:boto:Method: GET 2024-04-16T18:25:22.098 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.098 DEBUG:boto:Data: 2024-04-16T18:25:22.098 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.098 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.098 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.098 DEBUG:boto:Params: {} 2024-04-16T18:25:22.098 DEBUG:boto:Token: None 2024-04-16T18:25:22.098 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.098 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts= 2024-04-16T18:25:22.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts=', 'Content-Length': '0'} 2024-04-16T18:25:22.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000660afa699d520fe8-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.143 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.143 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.143 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.143 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.143 DEBUG:boto:Method: GET 2024-04-16T18:25:22.143 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.143 DEBUG:boto:Data: 2024-04-16T18:25:22.143 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.143 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.143 DEBUG:boto:Params: {} 2024-04-16T18:25:22.144 DEBUG:boto:Token: None 2024-04-16T18:25:22.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:25:22.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts= 2024-04-16T18:25:22.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uWqAX8OXvXOf9W7z/YMlAZJ13Ts=', 'Content-Length': '0'} 2024-04-16T18:25:22.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0d1a20d0288d507-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.147 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.147 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.147 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.147 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.147 DEBUG:boto:Method: GET 2024-04-16T18:25:22.148 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:25:22.148 DEBUG:boto:Data: 2024-04-16T18:25:22.148 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.148 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.148 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.148 DEBUG:boto:Params: {} 2024-04-16T18:25:22.148 DEBUG:boto:Token: None 2024-04-16T18:25:22.148 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000968e3fc4cfc2a3eb-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.194 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.195 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.195 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.195 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.195 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.195 DEBUG:boto:Method: GET 2024-04-16T18:25:22.195 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:25:22.195 DEBUG:boto:Data: 2024-04-16T18:25:22.195 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.195 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.196 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.196 DEBUG:boto:Params: {} 2024-04-16T18:25:22.196 DEBUG:boto:Token: None 2024-04-16T18:25:22.196 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.200 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006371bc237227f95c-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.201 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.201 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.201 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.201 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.201 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:25:22.201 DEBUG:boto:Data: 2024-04-16T18:25:22.201 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.202 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.202 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.202 DEBUG:boto:Params: {} 2024-04-16T18:25:22.202 DEBUG:boto:Token: None 2024-04-16T18:25:22.202 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19 2024-04-16T18:25:22.202 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UJQxxGJKqS9iljaSs1lfVOY6sHQ= 2024-04-16T18:25:22.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UJQxxGJKqS9iljaSs1lfVOY6sHQ=', 'Content-Length': '0'} 2024-04-16T18:25:22.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5fdb3962e3f58fe-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.206 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.206 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.206 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.206 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:25:22.206 DEBUG:boto:Data: 2024-04-16T18:25:22.206 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.206 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.206 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.206 DEBUG:boto:Params: {} 2024-04-16T18:25:22.206 DEBUG:boto:Token: None 2024-04-16T18:25:22.206 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19 2024-04-16T18:25:22.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UJQxxGJKqS9iljaSs1lfVOY6sHQ= 2024-04-16T18:25:22.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UJQxxGJKqS9iljaSs1lfVOY6sHQ=', 'Content-Length': '0'} 2024-04-16T18:25:22.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088b2fb219a4113f1-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.210 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:25:22.210 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.210 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.210 DEBUG:boto:Method: GET 2024-04-16T18:25:22.210 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:25:22.210 DEBUG:boto:Data: 2024-04-16T18:25:22.210 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.210 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.210 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.211 DEBUG:boto:Params: {} 2024-04-16T18:25:22.211 DEBUG:boto:Token: None 2024-04-16T18:25:22.211 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19 2024-04-16T18:25:22.211 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MyXErhnT15SgLhuP1uZQ/Xswf6w= 2024-04-16T18:25:22.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MyXErhnT15SgLhuP1uZQ/Xswf6w=', 'Content-Length': '0'} 2024-04-16T18:25:22.214 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006281727136fc94f2-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.258 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:25:22.258 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:25:22.258 DEBUG:boto:Method: GET 2024-04-16T18:25:22.259 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:25:22.259 DEBUG:boto:Data: 2024-04-16T18:25:22.259 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.259 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.259 DEBUG:boto:Params: {} 2024-04-16T18:25:22.259 DEBUG:boto:Token: None 2024-04-16T18:25:22.259 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj19 2024-04-16T18:25:22.259 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MyXErhnT15SgLhuP1uZQ/Xswf6w= 2024-04-16T18:25:22.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MyXErhnT15SgLhuP1uZQ/Xswf6w=', 'Content-Length': '0'} 2024-04-16T18:25:22.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b4d522cd5671540-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.262 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.263 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.263 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.263 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.263 DEBUG:boto:Method: GET 2024-04-16T18:25:22.263 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:25:22.263 DEBUG:boto:Data: 2024-04-16T18:25:22.263 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.263 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.263 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.263 DEBUG:boto:Params: {} 2024-04-16T18:25:22.263 DEBUG:boto:Token: None 2024-04-16T18:25:22.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095aacee02b3f1375-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.268 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.268 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.268 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.268 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.268 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:25:22.268 DEBUG:boto:Method: GET 2024-04-16T18:25:22.269 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:25:22.269 DEBUG:boto:Data: 2024-04-16T18:25:22.269 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.269 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.269 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.269 DEBUG:boto:Params: {} 2024-04-16T18:25:22.269 DEBUG:boto:Token: None 2024-04-16T18:25:22.269 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.269 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a545a8bdeb7eb7a3-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.275 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.275 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:25:22.275 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.275 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.275 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.275 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.275 DEBUG:boto:Method: GET 2024-04-16T18:25:22.276 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.276 DEBUG:boto:Data: 2024-04-16T18:25:22.276 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.276 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.276 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.276 DEBUG:boto:Params: {} 2024-04-16T18:25:22.276 DEBUG:boto:Token: None 2024-04-16T18:25:22.276 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws= 2024-04-16T18:25:22.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws=', 'Content-Length': '0'} 2024-04-16T18:25:22.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a44e7a4da747b8a0-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.322 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.322 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.322 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.322 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.323 DEBUG:boto:Method: GET 2024-04-16T18:25:22.323 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.323 DEBUG:boto:Data: 2024-04-16T18:25:22.323 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.323 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.323 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.323 DEBUG:boto:Params: {} 2024-04-16T18:25:22.323 DEBUG:boto:Token: None 2024-04-16T18:25:22.323 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.323 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws= 2024-04-16T18:25:22.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws=', 'Content-Length': '0'} 2024-04-16T18:25:22.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c63682753091328-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.327 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.327 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.327 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.327 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.327 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.327 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.327 DEBUG:boto:Data: 2024-04-16T18:25:22.327 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.327 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.328 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.328 DEBUG:boto:Params: {} 2024-04-16T18:25:22.328 DEBUG:boto:Token: None 2024-04-16T18:25:22.328 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VJTNW5xpyp20K/i9C7jZCPN6b4c= 2024-04-16T18:25:22.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VJTNW5xpyp20K/i9C7jZCPN6b4c=', 'Content-Length': '0'} 2024-04-16T18:25:22.331 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077cb7b2d40dba6c6-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.331 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.331 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.331 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.331 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.331 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.331 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.331 DEBUG:boto:Data: 2024-04-16T18:25:22.331 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.331 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.332 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.332 DEBUG:boto:Params: {} 2024-04-16T18:25:22.332 DEBUG:boto:Token: None 2024-04-16T18:25:22.332 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VJTNW5xpyp20K/i9C7jZCPN6b4c= 2024-04-16T18:25:22.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VJTNW5xpyp20K/i9C7jZCPN6b4c=', 'Content-Length': '0'} 2024-04-16T18:25:22.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000caaf480ec88da014-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.335 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:25:22.335 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.335 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.335 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.335 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.335 DEBUG:boto:Method: GET 2024-04-16T18:25:22.335 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.336 DEBUG:boto:Data: 2024-04-16T18:25:22.336 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.336 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.336 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.336 DEBUG:boto:Params: {} 2024-04-16T18:25:22.336 DEBUG:boto:Token: None 2024-04-16T18:25:22.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws= 2024-04-16T18:25:22.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws=', 'Content-Length': '0'} 2024-04-16T18:25:22.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7b20c7b95de8dfe-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.382 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.382 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.383 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.383 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.383 DEBUG:boto:Method: GET 2024-04-16T18:25:22.383 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.383 DEBUG:boto:Data: 2024-04-16T18:25:22.383 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.383 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.383 DEBUG:boto:Params: {} 2024-04-16T18:25:22.383 DEBUG:boto:Token: None 2024-04-16T18:25:22.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:25:22.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws= 2024-04-16T18:25:22.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/cKYQPLibDOx9PZaosz0Sbh6rws=', 'Content-Length': '0'} 2024-04-16T18:25:22.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4eb0fc195065fca-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.387 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.387 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.387 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.387 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.387 DEBUG:boto:Method: GET 2024-04-16T18:25:22.387 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:25:22.387 DEBUG:boto:Data: 2024-04-16T18:25:22.388 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.388 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.388 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.388 DEBUG:boto:Params: {} 2024-04-16T18:25:22.388 DEBUG:boto:Token: None 2024-04-16T18:25:22.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e4d3e2ea9282272-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.434 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.435 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.435 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.435 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.435 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.435 DEBUG:boto:Method: GET 2024-04-16T18:25:22.435 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:25:22.435 DEBUG:boto:Data: 2024-04-16T18:25:22.435 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.435 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.435 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.435 DEBUG:boto:Params: {} 2024-04-16T18:25:22.435 DEBUG:boto:Token: None 2024-04-16T18:25:22.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.441 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc4c7db39272405e-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.441 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.442 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.442 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.442 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.442 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:25:22.442 DEBUG:boto:Data: 2024-04-16T18:25:22.442 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.442 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.442 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.442 DEBUG:boto:Params: {} 2024-04-16T18:25:22.442 DEBUG:boto:Token: None 2024-04-16T18:25:22.443 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22 2024-04-16T18:25:22.443 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SJJJj3BklbHwohYa3upnB8CN7LU= 2024-04-16T18:25:22.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SJJJj3BklbHwohYa3upnB8CN7LU=', 'Content-Length': '0'} 2024-04-16T18:25:22.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e0ab9fa0c470d81-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.446 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.446 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.446 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.446 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:25:22.446 DEBUG:boto:Data: 2024-04-16T18:25:22.446 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.446 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.446 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.447 DEBUG:boto:Params: {} 2024-04-16T18:25:22.447 DEBUG:boto:Token: None 2024-04-16T18:25:22.447 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22 2024-04-16T18:25:22.447 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SJJJj3BklbHwohYa3upnB8CN7LU= 2024-04-16T18:25:22.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SJJJj3BklbHwohYa3upnB8CN7LU=', 'Content-Length': '0'} 2024-04-16T18:25:22.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d3041c5219d805e-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.450 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:25:22.450 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.450 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.450 DEBUG:boto:Method: GET 2024-04-16T18:25:22.450 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:25:22.450 DEBUG:boto:Data: 2024-04-16T18:25:22.450 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.450 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.451 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.451 DEBUG:boto:Params: {} 2024-04-16T18:25:22.451 DEBUG:boto:Token: None 2024-04-16T18:25:22.451 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22 2024-04-16T18:25:22.451 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lABkWjdoRbvbHWZA/969vxZJemo= 2024-04-16T18:25:22.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lABkWjdoRbvbHWZA/969vxZJemo=', 'Content-Length': '0'} 2024-04-16T18:25:22.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000934807daeed9b8eb-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.494 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:25:22.494 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:25:22.495 DEBUG:boto:Method: GET 2024-04-16T18:25:22.495 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:25:22.495 DEBUG:boto:Data: 2024-04-16T18:25:22.495 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.495 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.495 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.495 DEBUG:boto:Params: {} 2024-04-16T18:25:22.495 DEBUG:boto:Token: None 2024-04-16T18:25:22.495 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj22 2024-04-16T18:25:22.495 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lABkWjdoRbvbHWZA/969vxZJemo= 2024-04-16T18:25:22.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lABkWjdoRbvbHWZA/969vxZJemo=', 'Content-Length': '0'} 2024-04-16T18:25:22.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2ecf7d5fb54ea94-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.499 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.499 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.499 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.499 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.499 DEBUG:boto:Method: GET 2024-04-16T18:25:22.499 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:25:22.499 DEBUG:boto:Data: 2024-04-16T18:25:22.499 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.499 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.499 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.499 DEBUG:boto:Params: {} 2024-04-16T18:25:22.500 DEBUG:boto:Token: None 2024-04-16T18:25:22.500 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c521bd95bfc7a868-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.504 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.504 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.504 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.504 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.504 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:25:22.505 DEBUG:boto:Method: GET 2024-04-16T18:25:22.505 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:25:22.505 DEBUG:boto:Data: 2024-04-16T18:25:22.505 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.505 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.505 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.505 DEBUG:boto:Params: {} 2024-04-16T18:25:22.505 DEBUG:boto:Token: None 2024-04-16T18:25:22.505 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.511 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd8b532e15daad26-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.511 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.512 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:25:22.512 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.512 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.512 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.512 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.512 DEBUG:boto:Method: GET 2024-04-16T18:25:22.512 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.512 DEBUG:boto:Data: 2024-04-16T18:25:22.512 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.513 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.513 DEBUG:boto:Params: {} 2024-04-16T18:25:22.513 DEBUG:boto:Token: None 2024-04-16T18:25:22.513 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA= 2024-04-16T18:25:22.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA=', 'Content-Length': '0'} 2024-04-16T18:25:22.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff4e68aa6c3d58cc-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.558 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.558 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.559 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.559 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.559 DEBUG:boto:Method: GET 2024-04-16T18:25:22.559 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.559 DEBUG:boto:Data: 2024-04-16T18:25:22.559 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.559 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.559 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.559 DEBUG:boto:Params: {} 2024-04-16T18:25:22.559 DEBUG:boto:Token: None 2024-04-16T18:25:22.559 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA= 2024-04-16T18:25:22.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA=', 'Content-Length': '0'} 2024-04-16T18:25:22.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000431f940d0fe4be34-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.563 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.563 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.563 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.563 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.563 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.563 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.563 DEBUG:boto:Data: 2024-04-16T18:25:22.564 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.564 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.564 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.564 DEBUG:boto:Params: {} 2024-04-16T18:25:22.564 DEBUG:boto:Token: None 2024-04-16T18:25:22.564 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r8I72p2cLgNrDAxTt6PMH1ysHtc= 2024-04-16T18:25:22.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r8I72p2cLgNrDAxTt6PMH1ysHtc=', 'Content-Length': '0'} 2024-04-16T18:25:22.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000667ec3fc7ef08ddd-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.567 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.567 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.568 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.568 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.568 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.568 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.568 DEBUG:boto:Data: 2024-04-16T18:25:22.568 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.568 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.568 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.568 DEBUG:boto:Params: {} 2024-04-16T18:25:22.568 DEBUG:boto:Token: None 2024-04-16T18:25:22.568 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r8I72p2cLgNrDAxTt6PMH1ysHtc= 2024-04-16T18:25:22.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r8I72p2cLgNrDAxTt6PMH1ysHtc=', 'Content-Length': '0'} 2024-04-16T18:25:22.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006afc4dd208c2aaaf-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.572 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:25:22.572 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.572 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.572 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.572 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.572 DEBUG:boto:Method: GET 2024-04-16T18:25:22.572 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.573 DEBUG:boto:Data: 2024-04-16T18:25:22.573 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.573 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.573 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.573 DEBUG:boto:Params: {} 2024-04-16T18:25:22.573 DEBUG:boto:Token: None 2024-04-16T18:25:22.573 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA= 2024-04-16T18:25:22.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA=', 'Content-Length': '0'} 2024-04-16T18:25:22.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087d8430beed4dc5a-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.618 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.619 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.619 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.619 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.619 DEBUG:boto:Method: GET 2024-04-16T18:25:22.619 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.619 DEBUG:boto:Data: 2024-04-16T18:25:22.619 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.619 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.619 DEBUG:boto:Params: {} 2024-04-16T18:25:22.619 DEBUG:boto:Token: None 2024-04-16T18:25:22.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:25:22.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA= 2024-04-16T18:25:22.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X3ez16vTbTcK7V58dF4qdhJQDsA=', 'Content-Length': '0'} 2024-04-16T18:25:22.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a6a5f52718711ae-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.623 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.623 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.623 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.623 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.623 DEBUG:boto:Method: GET 2024-04-16T18:25:22.623 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:25:22.623 DEBUG:boto:Data: 2024-04-16T18:25:22.623 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.623 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.623 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.624 DEBUG:boto:Params: {} 2024-04-16T18:25:22.624 DEBUG:boto:Token: None 2024-04-16T18:25:22.624 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.628 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b718cebe9efa0790-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.670 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.671 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.671 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.671 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.671 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.671 DEBUG:boto:Method: GET 2024-04-16T18:25:22.671 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:25:22.671 DEBUG:boto:Data: 2024-04-16T18:25:22.671 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.671 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.671 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.671 DEBUG:boto:Params: {} 2024-04-16T18:25:22.671 DEBUG:boto:Token: None 2024-04-16T18:25:22.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ded34a2a080aa4d6-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.677 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.677 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.677 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.677 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.677 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:25:22.677 DEBUG:boto:Data: 2024-04-16T18:25:22.677 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.677 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.678 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.678 DEBUG:boto:Params: {} 2024-04-16T18:25:22.678 DEBUG:boto:Token: None 2024-04-16T18:25:22.678 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23 2024-04-16T18:25:22.678 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCyUtKp3bkGpYAgd717bZh1gOPI= 2024-04-16T18:25:22.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCyUtKp3bkGpYAgd717bZh1gOPI=', 'Content-Length': '0'} 2024-04-16T18:25:22.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b76e0764539959e4-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.681 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.681 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.681 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.681 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:25:22.681 DEBUG:boto:Data: 2024-04-16T18:25:22.682 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.682 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.682 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.682 DEBUG:boto:Params: {} 2024-04-16T18:25:22.682 DEBUG:boto:Token: None 2024-04-16T18:25:22.682 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23 2024-04-16T18:25:22.682 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yCyUtKp3bkGpYAgd717bZh1gOPI= 2024-04-16T18:25:22.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yCyUtKp3bkGpYAgd717bZh1gOPI=', 'Content-Length': '0'} 2024-04-16T18:25:22.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa0eb8133075aa1d-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.685 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:25:22.686 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.686 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.686 DEBUG:boto:Method: GET 2024-04-16T18:25:22.686 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:25:22.686 DEBUG:boto:Data: 2024-04-16T18:25:22.686 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.686 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.686 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.686 DEBUG:boto:Params: {} 2024-04-16T18:25:22.686 DEBUG:boto:Token: None 2024-04-16T18:25:22.686 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23 2024-04-16T18:25:22.687 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PSgvchM38IIxLhY9C0a2GCxKwkY= 2024-04-16T18:25:22.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PSgvchM38IIxLhY9C0a2GCxKwkY=', 'Content-Length': '0'} 2024-04-16T18:25:22.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007add05e99dec39e9-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.730 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:25:22.731 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:25:22.731 DEBUG:boto:Method: GET 2024-04-16T18:25:22.731 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:25:22.731 DEBUG:boto:Data: 2024-04-16T18:25:22.731 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.731 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.731 DEBUG:boto:Params: {} 2024-04-16T18:25:22.731 DEBUG:boto:Token: None 2024-04-16T18:25:22.731 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj23 2024-04-16T18:25:22.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PSgvchM38IIxLhY9C0a2GCxKwkY= 2024-04-16T18:25:22.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PSgvchM38IIxLhY9C0a2GCxKwkY=', 'Content-Length': '0'} 2024-04-16T18:25:22.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004eafa1181e35303e-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.735 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.735 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.735 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.735 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.735 DEBUG:boto:Method: GET 2024-04-16T18:25:22.735 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:25:22.735 DEBUG:boto:Data: 2024-04-16T18:25:22.735 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.735 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.735 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.736 DEBUG:boto:Params: {} 2024-04-16T18:25:22.736 DEBUG:boto:Token: None 2024-04-16T18:25:22.736 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.739 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f1345b8184054d4-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.740 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.740 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.740 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.740 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.740 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:25:22.740 DEBUG:boto:Method: GET 2024-04-16T18:25:22.741 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:25:22.741 DEBUG:boto:Data: 2024-04-16T18:25:22.741 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.741 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.741 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.741 DEBUG:boto:Params: {} 2024-04-16T18:25:22.741 DEBUG:boto:Token: None 2024-04-16T18:25:22.741 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.741 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.746 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000111cbcb9a6e1cd52-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.746 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:25:22.747 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.747 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.747 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.747 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.747 DEBUG:boto:Method: GET 2024-04-16T18:25:22.747 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.747 DEBUG:boto:Data: 2024-04-16T18:25:22.747 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.747 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.747 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.747 DEBUG:boto:Params: {} 2024-04-16T18:25:22.748 DEBUG:boto:Token: None 2024-04-16T18:25:22.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs= 2024-04-16T18:25:22.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs=', 'Content-Length': '0'} 2024-04-16T18:25:22.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b458c6db8f6e9420-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.794 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.794 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.795 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.795 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.795 DEBUG:boto:Method: GET 2024-04-16T18:25:22.795 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.795 DEBUG:boto:Data: 2024-04-16T18:25:22.795 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.795 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.795 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.795 DEBUG:boto:Params: {} 2024-04-16T18:25:22.795 DEBUG:boto:Token: None 2024-04-16T18:25:22.796 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs= 2024-04-16T18:25:22.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs=', 'Content-Length': '0'} 2024-04-16T18:25:22.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000428cf6f9df225b7a-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.799 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.799 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.799 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.799 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.799 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.799 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.799 DEBUG:boto:Data: 2024-04-16T18:25:22.799 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.800 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.800 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.800 DEBUG:boto:Params: {} 2024-04-16T18:25:22.800 DEBUG:boto:Token: None 2024-04-16T18:25:22.800 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MwkPxvlzZteR6LRe/yPuq1o4n5s= 2024-04-16T18:25:22.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MwkPxvlzZteR6LRe/yPuq1o4n5s=', 'Content-Length': '0'} 2024-04-16T18:25:22.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d727e4a59b8c3585-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.803 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.804 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.804 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.804 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.804 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.804 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.804 DEBUG:boto:Data: 2024-04-16T18:25:22.804 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.804 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.804 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.804 DEBUG:boto:Params: {} 2024-04-16T18:25:22.804 DEBUG:boto:Token: None 2024-04-16T18:25:22.805 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MwkPxvlzZteR6LRe/yPuq1o4n5s= 2024-04-16T18:25:22.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MwkPxvlzZteR6LRe/yPuq1o4n5s=', 'Content-Length': '0'} 2024-04-16T18:25:22.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c340fd42b286aa2-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:25:22.808 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.808 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.808 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.808 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.808 DEBUG:boto:Method: GET 2024-04-16T18:25:22.808 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.808 DEBUG:boto:Data: 2024-04-16T18:25:22.808 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.809 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.809 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.809 DEBUG:boto:Params: {} 2024-04-16T18:25:22.809 DEBUG:boto:Token: None 2024-04-16T18:25:22.809 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs= 2024-04-16T18:25:22.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs=', 'Content-Length': '0'} 2024-04-16T18:25:22.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eb73a45c3c91bcd-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.854 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.854 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.855 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.855 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.855 DEBUG:boto:Method: GET 2024-04-16T18:25:22.855 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.855 DEBUG:boto:Data: 2024-04-16T18:25:22.855 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.855 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.855 DEBUG:boto:Params: {} 2024-04-16T18:25:22.855 DEBUG:boto:Token: None 2024-04-16T18:25:22.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:25:22.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs= 2024-04-16T18:25:22.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1uiXRcMmPSgvMMfEtIXZLzy2RTs=', 'Content-Length': '0'} 2024-04-16T18:25:22.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1013b5bebbe5168-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.859 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.859 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.859 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.859 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.859 DEBUG:boto:Method: GET 2024-04-16T18:25:22.859 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:25:22.859 DEBUG:boto:Data: 2024-04-16T18:25:22.859 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.859 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.859 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.860 DEBUG:boto:Params: {} 2024-04-16T18:25:22.860 DEBUG:boto:Token: None 2024-04-16T18:25:22.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.863 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093ddf544d2ad9829-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.906 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.907 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.907 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.907 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.907 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.907 DEBUG:boto:Method: GET 2024-04-16T18:25:22.907 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:25:22.907 DEBUG:boto:Data: 2024-04-16T18:25:22.907 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.907 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.908 DEBUG:boto:Params: {} 2024-04-16T18:25:22.908 DEBUG:boto:Token: None 2024-04-16T18:25:22.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058a15f3732604e0d-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.913 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.914 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.914 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.914 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.914 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:25:22.914 DEBUG:boto:Data: 2024-04-16T18:25:22.914 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.914 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.914 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.914 DEBUG:boto:Params: {} 2024-04-16T18:25:22.914 DEBUG:boto:Token: None 2024-04-16T18:25:22.915 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33 2024-04-16T18:25:22.915 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0OGuTarPTs5hIj2yckduMwYGCc4= 2024-04-16T18:25:22.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0OGuTarPTs5hIj2yckduMwYGCc4=', 'Content-Length': '0'} 2024-04-16T18:25:22.917 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056ec47f2891f435a-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.918 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.918 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.918 DEBUG:boto:Method: HEAD 2024-04-16T18:25:22.918 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:25:22.918 DEBUG:boto:Data: 2024-04-16T18:25:22.918 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.918 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.919 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.919 DEBUG:boto:Params: {} 2024-04-16T18:25:22.919 DEBUG:boto:Token: None 2024-04-16T18:25:22.919 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33 2024-04-16T18:25:22.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0OGuTarPTs5hIj2yckduMwYGCc4= 2024-04-16T18:25:22.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0OGuTarPTs5hIj2yckduMwYGCc4=', 'Content-Length': '0'} 2024-04-16T18:25:22.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099bd0b12ad5255bb-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.922 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:25:22.922 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.923 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.923 DEBUG:boto:Method: GET 2024-04-16T18:25:22.923 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:25:22.923 DEBUG:boto:Data: 2024-04-16T18:25:22.923 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.923 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.923 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.923 DEBUG:boto:Params: {} 2024-04-16T18:25:22.923 DEBUG:boto:Token: None 2024-04-16T18:25:22.923 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33 2024-04-16T18:25:22.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5/cZndhd1lrixR0Bqf8EOWcWrtE= 2024-04-16T18:25:22.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5/cZndhd1lrixR0Bqf8EOWcWrtE=', 'Content-Length': '0'} 2024-04-16T18:25:22.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8aba0289494c600-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.970 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:25:22.970 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:25:22.971 DEBUG:boto:Method: GET 2024-04-16T18:25:22.971 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:25:22.971 DEBUG:boto:Data: 2024-04-16T18:25:22.971 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.971 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.971 DEBUG:boto:Params: {} 2024-04-16T18:25:22.971 DEBUG:boto:Token: None 2024-04-16T18:25:22.971 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj33 2024-04-16T18:25:22.971 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5/cZndhd1lrixR0Bqf8EOWcWrtE= 2024-04-16T18:25:22.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5/cZndhd1lrixR0Bqf8EOWcWrtE=', 'Content-Length': '0'} 2024-04-16T18:25:22.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020c1e105ae3180fc-00661ec292-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.975 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.975 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.975 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.975 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.975 DEBUG:boto:Method: GET 2024-04-16T18:25:22.975 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:25:22.975 DEBUG:boto:Data: 2024-04-16T18:25:22.975 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.975 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.975 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.975 DEBUG:boto:Params: {} 2024-04-16T18:25:22.976 DEBUG:boto:Token: None 2024-04-16T18:25:22.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.979 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092dc1d0c96802d4b-00661ec292-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.980 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.980 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:22.980 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:22.980 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.980 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:25:22.980 DEBUG:boto:Method: GET 2024-04-16T18:25:22.980 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:25:22.980 DEBUG:boto:Data: 2024-04-16T18:25:22.981 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.981 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:22.981 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.981 DEBUG:boto:Params: {} 2024-04-16T18:25:22.981 DEBUG:boto:Token: None 2024-04-16T18:25:22.981 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/ 2024-04-16T18:25:22.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc= 2024-04-16T18:25:22.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VFsynTuo3n8eAUXc4XCM2sky7Xc=', 'Content-Length': '0'} 2024-04-16T18:25:22.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b770bbe7b6c9072-00661ec292-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:22.986 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:22.986 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:25:22.986 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:22.986 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:22.987 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:22.987 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:22.987 DEBUG:boto:Method: GET 2024-04-16T18:25:22.987 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:25:22.987 DEBUG:boto:Data: 2024-04-16T18:25:22.987 DEBUG:boto:Headers: {} 2024-04-16T18:25:22.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:22.987 DEBUG:boto:Port: 8000 2024-04-16T18:25:22.987 DEBUG:boto:Params: {} 2024-04-16T18:25:22.987 DEBUG:boto:Token: None 2024-04-16T18:25:22.987 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:22 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:25:22.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Tpy0WJOcYNz4SxAr7lytzfsfUiw= 2024-04-16T18:25:22.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:22 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Tpy0WJOcYNz4SxAr7lytzfsfUiw=', 'Content-Length': '0'} 2024-04-16T18:25:22.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a3363eb3efe7c3a-00661ec292-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:22 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.034 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.034 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.034 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.035 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.035 DEBUG:boto:Method: GET 2024-04-16T18:25:23.035 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.035 DEBUG:boto:Data: 2024-04-16T18:25:23.035 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.035 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.035 DEBUG:boto:Params: {} 2024-04-16T18:25:23.035 DEBUG:boto:Token: None 2024-04-16T18:25:23.035 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.035 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XlaJKaLoQ2epmn3sqA/OnelRqZo= 2024-04-16T18:25:23.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XlaJKaLoQ2epmn3sqA/OnelRqZo=', 'Content-Length': '0'} 2024-04-16T18:25:23.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006383a1a09fc38836-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.039 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.039 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.039 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.039 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.039 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.041 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.041 DEBUG:boto:Data: 2024-04-16T18:25:23.041 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.041 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.041 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.041 DEBUG:boto:Params: {} 2024-04-16T18:25:23.041 DEBUG:boto:Token: None 2024-04-16T18:25:23.041 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wi3HzCGGMBQPtJd+Up0OVHbYH3E= 2024-04-16T18:25:23.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wi3HzCGGMBQPtJd+Up0OVHbYH3E=', 'Content-Length': '0'} 2024-04-16T18:25:23.045 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033db9a075d4d1528-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.045 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.045 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.045 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.045 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.045 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.045 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.045 DEBUG:boto:Data: 2024-04-16T18:25:23.045 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.046 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.046 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.046 DEBUG:boto:Params: {} 2024-04-16T18:25:23.046 DEBUG:boto:Token: None 2024-04-16T18:25:23.046 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.046 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wi3HzCGGMBQPtJd+Up0OVHbYH3E= 2024-04-16T18:25:23.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wi3HzCGGMBQPtJd+Up0OVHbYH3E=', 'Content-Length': '0'} 2024-04-16T18:25:23.049 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9bb79dc9f901c96-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.049 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:25:23.049 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.049 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.049 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.049 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.049 DEBUG:boto:Method: GET 2024-04-16T18:25:23.050 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.050 DEBUG:boto:Data: 2024-04-16T18:25:23.050 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.050 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.050 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.050 DEBUG:boto:Params: {} 2024-04-16T18:25:23.050 DEBUG:boto:Token: None 2024-04-16T18:25:23.050 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.050 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XlaJKaLoQ2epmn3sqA/OnelRqZo= 2024-04-16T18:25:23.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XlaJKaLoQ2epmn3sqA/OnelRqZo=', 'Content-Length': '0'} 2024-04-16T18:25:23.053 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf3842be106d97e6-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.094 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.095 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.095 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.095 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.095 DEBUG:boto:Method: GET 2024-04-16T18:25:23.095 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.095 DEBUG:boto:Data: 2024-04-16T18:25:23.095 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.095 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.095 DEBUG:boto:Params: {} 2024-04-16T18:25:23.095 DEBUG:boto:Token: None 2024-04-16T18:25:23.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:25:23.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XlaJKaLoQ2epmn3sqA/OnelRqZo= 2024-04-16T18:25:23.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XlaJKaLoQ2epmn3sqA/OnelRqZo=', 'Content-Length': '0'} 2024-04-16T18:25:23.098 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9d2cbc7f538fb32-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.099 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.099 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.099 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.099 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.099 DEBUG:boto:Method: GET 2024-04-16T18:25:23.099 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:25:23.099 DEBUG:boto:Data: 2024-04-16T18:25:23.099 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.099 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.100 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.100 DEBUG:boto:Params: {} 2024-04-16T18:25:23.100 DEBUG:boto:Token: None 2024-04-16T18:25:23.100 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.104 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c224a9c7e17d81cb-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.104 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.104 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.104 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.104 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.104 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.105 DEBUG:boto:Method: GET 2024-04-16T18:25:23.105 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:25:23.105 DEBUG:boto:Data: 2024-04-16T18:25:23.105 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.105 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.105 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.105 DEBUG:boto:Params: {} 2024-04-16T18:25:23.105 DEBUG:boto:Token: None 2024-04-16T18:25:23.105 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.105 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4d9426dd3cef906-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.150 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.151 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.151 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.151 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.151 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:25:23.151 DEBUG:boto:Data: 2024-04-16T18:25:23.151 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.151 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.151 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.151 DEBUG:boto:Params: {} 2024-04-16T18:25:23.151 DEBUG:boto:Token: None 2024-04-16T18:25:23.152 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34 2024-04-16T18:25:23.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ubbYqNgfBRwy0kztADjsV83YLoA= 2024-04-16T18:25:23.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ubbYqNgfBRwy0kztADjsV83YLoA=', 'Content-Length': '0'} 2024-04-16T18:25:23.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000422a5bb64d2def12-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.155 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.155 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.155 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.155 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:25:23.155 DEBUG:boto:Data: 2024-04-16T18:25:23.155 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.155 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.155 DEBUG:boto:Params: {} 2024-04-16T18:25:23.155 DEBUG:boto:Token: None 2024-04-16T18:25:23.156 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34 2024-04-16T18:25:23.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ubbYqNgfBRwy0kztADjsV83YLoA= 2024-04-16T18:25:23.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ubbYqNgfBRwy0kztADjsV83YLoA=', 'Content-Length': '0'} 2024-04-16T18:25:23.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d7cf304f67c3b87-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.159 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:25:23.159 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.159 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.159 DEBUG:boto:Method: GET 2024-04-16T18:25:23.159 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:25:23.159 DEBUG:boto:Data: 2024-04-16T18:25:23.159 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.159 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.159 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.159 DEBUG:boto:Params: {} 2024-04-16T18:25:23.160 DEBUG:boto:Token: None 2024-04-16T18:25:23.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34 2024-04-16T18:25:23.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g6tV6Q4qCcvcO1ehxhG2g6vns1Y= 2024-04-16T18:25:23.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g6tV6Q4qCcvcO1ehxhG2g6vns1Y=', 'Content-Length': '0'} 2024-04-16T18:25:23.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec8dc571ed082dbb-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.206 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:25:23.206 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:25:23.207 DEBUG:boto:Method: GET 2024-04-16T18:25:23.207 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:25:23.207 DEBUG:boto:Data: 2024-04-16T18:25:23.207 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.207 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.207 DEBUG:boto:Params: {} 2024-04-16T18:25:23.207 DEBUG:boto:Token: None 2024-04-16T18:25:23.207 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj34 2024-04-16T18:25:23.207 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g6tV6Q4qCcvcO1ehxhG2g6vns1Y= 2024-04-16T18:25:23.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g6tV6Q4qCcvcO1ehxhG2g6vns1Y=', 'Content-Length': '0'} 2024-04-16T18:25:23.210 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044781bc640efa60f-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.211 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.211 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.211 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.211 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.211 DEBUG:boto:Method: GET 2024-04-16T18:25:23.211 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:25:23.211 DEBUG:boto:Data: 2024-04-16T18:25:23.211 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.211 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.212 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.212 DEBUG:boto:Params: {} 2024-04-16T18:25:23.212 DEBUG:boto:Token: None 2024-04-16T18:25:23.212 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054d5e6c97aef305c-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.216 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.216 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.216 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.216 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.217 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:25:23.217 DEBUG:boto:Method: GET 2024-04-16T18:25:23.217 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:25:23.217 DEBUG:boto:Data: 2024-04-16T18:25:23.217 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.217 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.217 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.217 DEBUG:boto:Params: {} 2024-04-16T18:25:23.217 DEBUG:boto:Token: None 2024-04-16T18:25:23.217 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.218 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.223 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c126112e4fd1686-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.223 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.223 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:25:23.223 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.223 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.224 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.224 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.224 DEBUG:boto:Method: GET 2024-04-16T18:25:23.224 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.224 DEBUG:boto:Data: 2024-04-16T18:25:23.224 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.224 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.224 DEBUG:boto:Params: {} 2024-04-16T18:25:23.224 DEBUG:boto:Token: None 2024-04-16T18:25:23.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc= 2024-04-16T18:25:23.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc=', 'Content-Length': '0'} 2024-04-16T18:25:23.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086db185018c84cfe-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.274 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.274 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.274 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.275 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.275 DEBUG:boto:Method: GET 2024-04-16T18:25:23.275 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.275 DEBUG:boto:Data: 2024-04-16T18:25:23.275 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.275 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.275 DEBUG:boto:Params: {} 2024-04-16T18:25:23.275 DEBUG:boto:Token: None 2024-04-16T18:25:23.275 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.275 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc= 2024-04-16T18:25:23.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc=', 'Content-Length': '0'} 2024-04-16T18:25:23.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000078228fbc698d48c-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.279 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.279 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.279 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.279 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.279 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.279 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.279 DEBUG:boto:Data: 2024-04-16T18:25:23.280 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.280 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.280 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.280 DEBUG:boto:Params: {} 2024-04-16T18:25:23.280 DEBUG:boto:Token: None 2024-04-16T18:25:23.280 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nAJxXNlhDyACwCt/QmHYO3+8RHk= 2024-04-16T18:25:23.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nAJxXNlhDyACwCt/QmHYO3+8RHk=', 'Content-Length': '0'} 2024-04-16T18:25:23.283 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001637b4c66d9c2d51-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.283 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.283 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.283 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.283 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.283 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.284 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.284 DEBUG:boto:Data: 2024-04-16T18:25:23.284 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.284 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.284 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.284 DEBUG:boto:Params: {} 2024-04-16T18:25:23.284 DEBUG:boto:Token: None 2024-04-16T18:25:23.284 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nAJxXNlhDyACwCt/QmHYO3+8RHk= 2024-04-16T18:25:23.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nAJxXNlhDyACwCt/QmHYO3+8RHk=', 'Content-Length': '0'} 2024-04-16T18:25:23.287 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6e02832116103d1-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.287 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:25:23.287 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.287 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.287 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.288 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.288 DEBUG:boto:Method: GET 2024-04-16T18:25:23.288 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.288 DEBUG:boto:Data: 2024-04-16T18:25:23.288 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.288 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.288 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.288 DEBUG:boto:Params: {} 2024-04-16T18:25:23.288 DEBUG:boto:Token: None 2024-04-16T18:25:23.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc= 2024-04-16T18:25:23.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc=', 'Content-Length': '0'} 2024-04-16T18:25:23.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000124a805b6938415c-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.334 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.334 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.335 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.335 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.335 DEBUG:boto:Method: GET 2024-04-16T18:25:23.335 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.335 DEBUG:boto:Data: 2024-04-16T18:25:23.335 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.335 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.335 DEBUG:boto:Params: {} 2024-04-16T18:25:23.335 DEBUG:boto:Token: None 2024-04-16T18:25:23.335 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:25:23.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc= 2024-04-16T18:25:23.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qZ8XxaQepKrZIO/v84yHAlc/1nc=', 'Content-Length': '0'} 2024-04-16T18:25:23.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d420fe58b3b47ace-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.339 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.339 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.339 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.339 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.339 DEBUG:boto:Method: GET 2024-04-16T18:25:23.339 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:25:23.340 DEBUG:boto:Data: 2024-04-16T18:25:23.340 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.340 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.340 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.340 DEBUG:boto:Params: {} 2024-04-16T18:25:23.340 DEBUG:boto:Token: None 2024-04-16T18:25:23.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c188a1c6dc64945-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.344 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.344 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.344 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.344 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.344 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.344 DEBUG:boto:Method: GET 2024-04-16T18:25:23.345 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:25:23.345 DEBUG:boto:Data: 2024-04-16T18:25:23.345 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.345 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.345 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.345 DEBUG:boto:Params: {} 2024-04-16T18:25:23.345 DEBUG:boto:Token: None 2024-04-16T18:25:23.345 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.345 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.350 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6cd091e42f20703-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.350 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.350 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.350 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.350 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.351 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:25:23.351 DEBUG:boto:Data: 2024-04-16T18:25:23.351 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.351 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.351 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.351 DEBUG:boto:Params: {} 2024-04-16T18:25:23.351 DEBUG:boto:Token: None 2024-04-16T18:25:23.351 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38 2024-04-16T18:25:23.351 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THfm02hp3tIsngq5zFvY4E6ScxE= 2024-04-16T18:25:23.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THfm02hp3tIsngq5zFvY4E6ScxE=', 'Content-Length': '0'} 2024-04-16T18:25:23.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a939808b576ea83-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.354 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.354 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.354 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.354 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:25:23.354 DEBUG:boto:Data: 2024-04-16T18:25:23.355 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.355 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.355 DEBUG:boto:Params: {} 2024-04-16T18:25:23.355 DEBUG:boto:Token: None 2024-04-16T18:25:23.355 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38 2024-04-16T18:25:23.355 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:THfm02hp3tIsngq5zFvY4E6ScxE= 2024-04-16T18:25:23.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:THfm02hp3tIsngq5zFvY4E6ScxE=', 'Content-Length': '0'} 2024-04-16T18:25:23.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a23679f2c44fc41-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.358 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:25:23.358 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.358 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.358 DEBUG:boto:Method: GET 2024-04-16T18:25:23.359 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:25:23.359 DEBUG:boto:Data: 2024-04-16T18:25:23.359 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.359 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.359 DEBUG:boto:Params: {} 2024-04-16T18:25:23.359 DEBUG:boto:Token: None 2024-04-16T18:25:23.359 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38 2024-04-16T18:25:23.359 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pZKmYK2NauNkkGckOH8RiZ683H0= 2024-04-16T18:25:23.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pZKmYK2NauNkkGckOH8RiZ683H0=', 'Content-Length': '0'} 2024-04-16T18:25:23.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ec4b5db4deca7a1-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.402 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:25:23.402 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:25:23.403 DEBUG:boto:Method: GET 2024-04-16T18:25:23.403 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:25:23.403 DEBUG:boto:Data: 2024-04-16T18:25:23.403 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.403 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.403 DEBUG:boto:Params: {} 2024-04-16T18:25:23.403 DEBUG:boto:Token: None 2024-04-16T18:25:23.403 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj38 2024-04-16T18:25:23.403 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pZKmYK2NauNkkGckOH8RiZ683H0= 2024-04-16T18:25:23.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pZKmYK2NauNkkGckOH8RiZ683H0=', 'Content-Length': '0'} 2024-04-16T18:25:23.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea9f5aabdd79fc84-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.407 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.407 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.407 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.407 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.407 DEBUG:boto:Method: GET 2024-04-16T18:25:23.407 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:25:23.407 DEBUG:boto:Data: 2024-04-16T18:25:23.407 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.407 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.408 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.408 DEBUG:boto:Params: {} 2024-04-16T18:25:23.408 DEBUG:boto:Token: None 2024-04-16T18:25:23.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.411 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066ffd7a6b1503b85-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.454 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.455 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.455 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.455 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.455 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:25:23.455 DEBUG:boto:Method: GET 2024-04-16T18:25:23.455 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:25:23.455 DEBUG:boto:Data: 2024-04-16T18:25:23.455 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.455 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.455 DEBUG:boto:Params: {} 2024-04-16T18:25:23.456 DEBUG:boto:Token: None 2024-04-16T18:25:23.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.461 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e86219875f6f782b-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.461 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.462 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:25:23.462 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.462 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.462 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.462 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.462 DEBUG:boto:Method: GET 2024-04-16T18:25:23.462 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.462 DEBUG:boto:Data: 2024-04-16T18:25:23.462 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.462 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.462 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.463 DEBUG:boto:Params: {} 2024-04-16T18:25:23.463 DEBUG:boto:Token: None 2024-04-16T18:25:23.463 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.463 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio= 2024-04-16T18:25:23.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio=', 'Content-Length': '0'} 2024-04-16T18:25:23.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009998242bc07f61cf-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.506 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.506 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.507 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.507 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.507 DEBUG:boto:Method: GET 2024-04-16T18:25:23.507 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.507 DEBUG:boto:Data: 2024-04-16T18:25:23.507 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.507 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.507 DEBUG:boto:Params: {} 2024-04-16T18:25:23.507 DEBUG:boto:Token: None 2024-04-16T18:25:23.507 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio= 2024-04-16T18:25:23.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio=', 'Content-Length': '0'} 2024-04-16T18:25:23.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afcc91e57fcd5e5a-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.511 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.511 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.511 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.511 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.511 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.511 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.511 DEBUG:boto:Data: 2024-04-16T18:25:23.512 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.512 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.512 DEBUG:boto:Params: {} 2024-04-16T18:25:23.512 DEBUG:boto:Token: None 2024-04-16T18:25:23.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mekv6en63FP4j5pTcjZaWfCDaOc= 2024-04-16T18:25:23.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mekv6en63FP4j5pTcjZaWfCDaOc=', 'Content-Length': '0'} 2024-04-16T18:25:23.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006398f1660dbea7e3-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.515 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.515 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.516 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.516 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.516 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.516 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.516 DEBUG:boto:Data: 2024-04-16T18:25:23.516 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.516 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.516 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.516 DEBUG:boto:Params: {} 2024-04-16T18:25:23.516 DEBUG:boto:Token: None 2024-04-16T18:25:23.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mekv6en63FP4j5pTcjZaWfCDaOc= 2024-04-16T18:25:23.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mekv6en63FP4j5pTcjZaWfCDaOc=', 'Content-Length': '0'} 2024-04-16T18:25:23.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ff5901665b89307-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.519 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:25:23.520 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.520 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.520 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.520 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.520 DEBUG:boto:Method: GET 2024-04-16T18:25:23.520 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.520 DEBUG:boto:Data: 2024-04-16T18:25:23.520 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.520 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.520 DEBUG:boto:Params: {} 2024-04-16T18:25:23.521 DEBUG:boto:Token: None 2024-04-16T18:25:23.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio= 2024-04-16T18:25:23.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio=', 'Content-Length': '0'} 2024-04-16T18:25:23.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007354a612a7753763-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.566 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.567 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.567 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.567 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.567 DEBUG:boto:Method: GET 2024-04-16T18:25:23.567 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.567 DEBUG:boto:Data: 2024-04-16T18:25:23.567 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.567 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.567 DEBUG:boto:Params: {} 2024-04-16T18:25:23.567 DEBUG:boto:Token: None 2024-04-16T18:25:23.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:25:23.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio= 2024-04-16T18:25:23.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5S+ZAWpyIjCbPoaGf86muT7DPio=', 'Content-Length': '0'} 2024-04-16T18:25:23.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048228da08877627b-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.571 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.571 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.571 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.571 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.572 DEBUG:boto:Method: GET 2024-04-16T18:25:23.572 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:25:23.572 DEBUG:boto:Data: 2024-04-16T18:25:23.572 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.572 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.572 DEBUG:boto:Params: {} 2024-04-16T18:25:23.572 DEBUG:boto:Token: None 2024-04-16T18:25:23.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf5337b36391c5d3-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.576 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.576 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.577 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.577 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.577 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.577 DEBUG:boto:Method: GET 2024-04-16T18:25:23.577 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:25:23.577 DEBUG:boto:Data: 2024-04-16T18:25:23.577 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.577 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.577 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.577 DEBUG:boto:Params: {} 2024-04-16T18:25:23.577 DEBUG:boto:Token: None 2024-04-16T18:25:23.577 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.582 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9e4f0f4437e4494-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.582 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.582 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.583 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.583 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.583 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:25:23.583 DEBUG:boto:Data: 2024-04-16T18:25:23.583 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.583 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.583 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.583 DEBUG:boto:Params: {} 2024-04-16T18:25:23.583 DEBUG:boto:Token: None 2024-04-16T18:25:23.583 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44 2024-04-16T18:25:23.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DSTXJnEdNnbSQXODcKrMfIWP8uU= 2024-04-16T18:25:23.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DSTXJnEdNnbSQXODcKrMfIWP8uU=', 'Content-Length': '0'} 2024-04-16T18:25:23.587 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b943de394865051-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.587 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.587 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.587 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.587 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:25:23.587 DEBUG:boto:Data: 2024-04-16T18:25:23.587 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.587 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.588 DEBUG:boto:Params: {} 2024-04-16T18:25:23.588 DEBUG:boto:Token: None 2024-04-16T18:25:23.588 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44 2024-04-16T18:25:23.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DSTXJnEdNnbSQXODcKrMfIWP8uU= 2024-04-16T18:25:23.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DSTXJnEdNnbSQXODcKrMfIWP8uU=', 'Content-Length': '0'} 2024-04-16T18:25:23.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2588102379c044a-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.591 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:25:23.591 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.591 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.591 DEBUG:boto:Method: GET 2024-04-16T18:25:23.591 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:25:23.591 DEBUG:boto:Data: 2024-04-16T18:25:23.591 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.591 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.592 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.592 DEBUG:boto:Params: {} 2024-04-16T18:25:23.592 DEBUG:boto:Token: None 2024-04-16T18:25:23.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44 2024-04-16T18:25:23.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E1NTTxMGXNkFujQm9u2cxSh2nKA= 2024-04-16T18:25:23.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E1NTTxMGXNkFujQm9u2cxSh2nKA=', 'Content-Length': '0'} 2024-04-16T18:25:23.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b36184ecc1433de-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.638 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:25:23.638 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:25:23.639 DEBUG:boto:Method: GET 2024-04-16T18:25:23.639 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:25:23.639 DEBUG:boto:Data: 2024-04-16T18:25:23.639 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.639 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.639 DEBUG:boto:Params: {} 2024-04-16T18:25:23.639 DEBUG:boto:Token: None 2024-04-16T18:25:23.639 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj44 2024-04-16T18:25:23.639 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E1NTTxMGXNkFujQm9u2cxSh2nKA= 2024-04-16T18:25:23.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E1NTTxMGXNkFujQm9u2cxSh2nKA=', 'Content-Length': '0'} 2024-04-16T18:25:23.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f65aa8cf9a9ac94f-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.643 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.643 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.643 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.643 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.643 DEBUG:boto:Method: GET 2024-04-16T18:25:23.643 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:25:23.643 DEBUG:boto:Data: 2024-04-16T18:25:23.643 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.643 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.643 DEBUG:boto:Params: {} 2024-04-16T18:25:23.644 DEBUG:boto:Token: None 2024-04-16T18:25:23.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.648 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000559584ae401f0453-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.648 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.648 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.648 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.649 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.649 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:25:23.649 DEBUG:boto:Method: GET 2024-04-16T18:25:23.649 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:25:23.649 DEBUG:boto:Data: 2024-04-16T18:25:23.649 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.649 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.649 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.649 DEBUG:boto:Params: {} 2024-04-16T18:25:23.649 DEBUG:boto:Token: None 2024-04-16T18:25:23.649 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.654 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb576aa6e2dc1d38-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.698 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.699 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:25:23.699 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.699 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.699 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.699 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.699 DEBUG:boto:Method: GET 2024-04-16T18:25:23.699 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.699 DEBUG:boto:Data: 2024-04-16T18:25:23.699 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.699 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.699 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.700 DEBUG:boto:Params: {} 2024-04-16T18:25:23.700 DEBUG:boto:Token: None 2024-04-16T18:25:23.700 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg= 2024-04-16T18:25:23.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg=', 'Content-Length': '0'} 2024-04-16T18:25:23.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b31d8edb66a862dd-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.746 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.747 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.747 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.747 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.747 DEBUG:boto:Method: GET 2024-04-16T18:25:23.747 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.747 DEBUG:boto:Data: 2024-04-16T18:25:23.747 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.747 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.747 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.747 DEBUG:boto:Params: {} 2024-04-16T18:25:23.747 DEBUG:boto:Token: None 2024-04-16T18:25:23.748 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg= 2024-04-16T18:25:23.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg=', 'Content-Length': '0'} 2024-04-16T18:25:23.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f15838fd68adfb78-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.751 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.751 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.751 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.751 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.752 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.752 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.752 DEBUG:boto:Data: 2024-04-16T18:25:23.752 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.752 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.752 DEBUG:boto:Params: {} 2024-04-16T18:25:23.752 DEBUG:boto:Token: None 2024-04-16T18:25:23.752 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WtMSH6ir0yF9P7Ped4F8cS6RPfg= 2024-04-16T18:25:23.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WtMSH6ir0yF9P7Ped4F8cS6RPfg=', 'Content-Length': '0'} 2024-04-16T18:25:23.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd1d92e0ccb86736-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.756 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.756 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.756 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.756 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.756 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.756 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.756 DEBUG:boto:Data: 2024-04-16T18:25:23.756 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.756 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.757 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.757 DEBUG:boto:Params: {} 2024-04-16T18:25:23.757 DEBUG:boto:Token: None 2024-04-16T18:25:23.757 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WtMSH6ir0yF9P7Ped4F8cS6RPfg= 2024-04-16T18:25:23.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WtMSH6ir0yF9P7Ped4F8cS6RPfg=', 'Content-Length': '0'} 2024-04-16T18:25:23.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f8097b888cff8da-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:25:23.760 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.760 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.761 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.761 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.761 DEBUG:boto:Method: GET 2024-04-16T18:25:23.761 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.761 DEBUG:boto:Data: 2024-04-16T18:25:23.761 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.761 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.761 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.761 DEBUG:boto:Params: {} 2024-04-16T18:25:23.761 DEBUG:boto:Token: None 2024-04-16T18:25:23.761 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg= 2024-04-16T18:25:23.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg=', 'Content-Length': '0'} 2024-04-16T18:25:23.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000695cd51275dfc32a-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.806 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.807 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.807 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.807 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.807 DEBUG:boto:Method: GET 2024-04-16T18:25:23.807 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.807 DEBUG:boto:Data: 2024-04-16T18:25:23.807 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.807 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.807 DEBUG:boto:Params: {} 2024-04-16T18:25:23.807 DEBUG:boto:Token: None 2024-04-16T18:25:23.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:25:23.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg= 2024-04-16T18:25:23.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YgI83zIj3UnhO0llxmIOKky6dZg=', 'Content-Length': '0'} 2024-04-16T18:25:23.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e25c4d5caa486c8f-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.811 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.811 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.811 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.811 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.811 DEBUG:boto:Method: GET 2024-04-16T18:25:23.811 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:25:23.811 DEBUG:boto:Data: 2024-04-16T18:25:23.812 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.812 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.812 DEBUG:boto:Params: {} 2024-04-16T18:25:23.812 DEBUG:boto:Token: None 2024-04-16T18:25:23.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8d991f68498c3ac-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.815 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.816 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.816 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.816 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.816 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.816 DEBUG:boto:Method: GET 2024-04-16T18:25:23.816 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:25:23.816 DEBUG:boto:Data: 2024-04-16T18:25:23.816 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.817 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.817 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.817 DEBUG:boto:Params: {} 2024-04-16T18:25:23.817 DEBUG:boto:Token: None 2024-04-16T18:25:23.817 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.817 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a2ad24e1a0069b0-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.822 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.822 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.823 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.823 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.823 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:25:23.823 DEBUG:boto:Data: 2024-04-16T18:25:23.823 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.823 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.823 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.823 DEBUG:boto:Params: {} 2024-04-16T18:25:23.823 DEBUG:boto:Token: None 2024-04-16T18:25:23.823 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46 2024-04-16T18:25:23.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Onf/WAoIcnmcGZxsv1KGM4Y1Qhg= 2024-04-16T18:25:23.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Onf/WAoIcnmcGZxsv1KGM4Y1Qhg=', 'Content-Length': '0'} 2024-04-16T18:25:23.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bc8a87c76392a1b-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.827 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.827 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.827 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.827 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:25:23.827 DEBUG:boto:Data: 2024-04-16T18:25:23.827 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.827 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.827 DEBUG:boto:Params: {} 2024-04-16T18:25:23.827 DEBUG:boto:Token: None 2024-04-16T18:25:23.828 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46 2024-04-16T18:25:23.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Onf/WAoIcnmcGZxsv1KGM4Y1Qhg= 2024-04-16T18:25:23.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Onf/WAoIcnmcGZxsv1KGM4Y1Qhg=', 'Content-Length': '0'} 2024-04-16T18:25:23.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070196f9e6b7a1a8b-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.831 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:25:23.832 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.832 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.832 DEBUG:boto:Method: GET 2024-04-16T18:25:23.832 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:25:23.832 DEBUG:boto:Data: 2024-04-16T18:25:23.832 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.832 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.832 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.832 DEBUG:boto:Params: {} 2024-04-16T18:25:23.832 DEBUG:boto:Token: None 2024-04-16T18:25:23.833 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46 2024-04-16T18:25:23.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTpCbAq+yUTQpdhtwGxx/BKLmGg= 2024-04-16T18:25:23.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTpCbAq+yUTQpdhtwGxx/BKLmGg=', 'Content-Length': '0'} 2024-04-16T18:25:23.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011081d2d1346c920-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.878 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:25:23.878 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:25:23.879 DEBUG:boto:Method: GET 2024-04-16T18:25:23.879 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:25:23.879 DEBUG:boto:Data: 2024-04-16T18:25:23.879 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.879 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.879 DEBUG:boto:Params: {} 2024-04-16T18:25:23.879 DEBUG:boto:Token: None 2024-04-16T18:25:23.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj46 2024-04-16T18:25:23.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lTpCbAq+yUTQpdhtwGxx/BKLmGg= 2024-04-16T18:25:23.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lTpCbAq+yUTQpdhtwGxx/BKLmGg=', 'Content-Length': '0'} 2024-04-16T18:25:23.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085dcae461efa8715-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.883 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.883 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.883 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.883 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.884 DEBUG:boto:Method: GET 2024-04-16T18:25:23.884 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:25:23.884 DEBUG:boto:Data: 2024-04-16T18:25:23.884 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.884 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.884 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.884 DEBUG:boto:Params: {} 2024-04-16T18:25:23.884 DEBUG:boto:Token: None 2024-04-16T18:25:23.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecf191e71b8126f3-00661ec293-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.888 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.889 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:23.889 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:23.889 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.889 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:25:23.889 DEBUG:boto:Method: GET 2024-04-16T18:25:23.889 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:25:23.889 DEBUG:boto:Data: 2024-04-16T18:25:23.889 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.889 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.890 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.890 DEBUG:boto:Params: {} 2024-04-16T18:25:23.890 DEBUG:boto:Token: None 2024-04-16T18:25:23.890 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/ 2024-04-16T18:25:23.890 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE= 2024-04-16T18:25:23.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bYijH6sg2xK4ayW79GhsQlqe7zE=', 'Content-Length': '0'} 2024-04-16T18:25:23.895 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee938fdff94afd9a-00661ec293-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.895 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:23.895 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:25:23.895 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:23.895 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:23.895 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.896 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.896 DEBUG:boto:Method: GET 2024-04-16T18:25:23.896 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.896 DEBUG:boto:Data: 2024-04-16T18:25:23.896 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.896 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.896 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.896 DEBUG:boto:Params: {} 2024-04-16T18:25:23.896 DEBUG:boto:Token: None 2024-04-16T18:25:23.896 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXplAU4rjtNSUTN7XK9+GpiTli8= 2024-04-16T18:25:23.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXplAU4rjtNSUTN7XK9+GpiTli8=', 'Content-Length': '0'} 2024-04-16T18:25:23.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018f3614641218d1f-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.942 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:23.942 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:23.943 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.943 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.943 DEBUG:boto:Method: GET 2024-04-16T18:25:23.943 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.943 DEBUG:boto:Data: 2024-04-16T18:25:23.943 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.943 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.943 DEBUG:boto:Params: {} 2024-04-16T18:25:23.943 DEBUG:boto:Token: None 2024-04-16T18:25:23.944 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXplAU4rjtNSUTN7XK9+GpiTli8= 2024-04-16T18:25:23.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXplAU4rjtNSUTN7XK9+GpiTli8=', 'Content-Length': '0'} 2024-04-16T18:25:23.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029c09cd521ef02b1-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.947 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:23.947 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:23.947 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.947 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.947 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.948 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.948 DEBUG:boto:Data: 2024-04-16T18:25:23.948 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.948 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.948 DEBUG:boto:Params: {} 2024-04-16T18:25:23.948 DEBUG:boto:Token: None 2024-04-16T18:25:23.948 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bp0NLRd9NgG2orEITdguvaG5QDM= 2024-04-16T18:25:23.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bp0NLRd9NgG2orEITdguvaG5QDM=', 'Content-Length': '0'} 2024-04-16T18:25:23.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006b63d6a321b472c-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.951 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:23.952 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:23.952 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.952 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.952 DEBUG:boto:Method: HEAD 2024-04-16T18:25:23.952 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.952 DEBUG:boto:Data: 2024-04-16T18:25:23.952 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.952 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:23.952 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.952 DEBUG:boto:Params: {} 2024-04-16T18:25:23.952 DEBUG:boto:Token: None 2024-04-16T18:25:23.953 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bp0NLRd9NgG2orEITdguvaG5QDM= 2024-04-16T18:25:23.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bp0NLRd9NgG2orEITdguvaG5QDM=', 'Content-Length': '0'} 2024-04-16T18:25:23.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0ca04a6faca9665-00661ec293-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:23.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:25:23.956 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:23.956 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:23.957 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.957 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.957 DEBUG:boto:Method: GET 2024-04-16T18:25:23.957 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.957 DEBUG:boto:Data: 2024-04-16T18:25:23.957 DEBUG:boto:Headers: {} 2024-04-16T18:25:23.957 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:23.957 DEBUG:boto:Port: 8000 2024-04-16T18:25:23.957 DEBUG:boto:Params: {} 2024-04-16T18:25:23.957 DEBUG:boto:Token: None 2024-04-16T18:25:23.957 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:23 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:25:23.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SXplAU4rjtNSUTN7XK9+GpiTli8= 2024-04-16T18:25:23.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:23 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SXplAU4rjtNSUTN7XK9+GpiTli8=', 'Content-Length': '0'} 2024-04-16T18:25:23.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057d179be13dacb98-00661ec293-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:23 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.002 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:24.002 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:24.002 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:24.003 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:25:24.003 DEBUG:boto:Method: GET 2024-04-16T18:25:24.003 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:25:24.003 DEBUG:boto:Data: 2024-04-16T18:25:24.003 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.003 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.003 DEBUG:boto:Params: {} 2024-04-16T18:25:24.003 DEBUG:boto:Token: None 2024-04-16T18:25:24.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:25:24.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O7iBvtbhtQOjALVz4gD4QpOm8Uo= 2024-04-16T18:25:24.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O7iBvtbhtQOjALVz4gD4QpOm8Uo=', 'Content-Length': '0'} 2024-04-16T18:25:24.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b90dfd8da6cdc00e-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.007 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.007 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.008 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.008 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.008 DEBUG:boto:Method: GET 2024-04-16T18:25:24.008 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:25:24.008 DEBUG:boto:Data: 2024-04-16T18:25:24.008 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.008 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.008 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.008 DEBUG:boto:Params: {} 2024-04-16T18:25:24.008 DEBUG:boto:Token: None 2024-04-16T18:25:24.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.009 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d682acf6e05405a-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.012 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.013 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.013 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.013 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.013 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.013 DEBUG:boto:Method: GET 2024-04-16T18:25:24.013 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:25:24.013 DEBUG:boto:Data: 2024-04-16T18:25:24.013 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.014 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.014 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.014 DEBUG:boto:Params: {} 2024-04-16T18:25:24.014 DEBUG:boto:Token: None 2024-04-16T18:25:24.014 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.014 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.019 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000730e58c399937145-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.062 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.063 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:24.063 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:24.063 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.063 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:25:24.063 DEBUG:boto:Data: 2024-04-16T18:25:24.063 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.063 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.063 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.063 DEBUG:boto:Params: {} 2024-04-16T18:25:24.064 DEBUG:boto:Token: None 2024-04-16T18:25:24.064 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj51 2024-04-16T18:25:24.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kBTr3amaORbMX1TrF7ejci7incg= 2024-04-16T18:25:24.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kBTr3amaORbMX1TrF7ejci7incg=', 'Content-Length': '0'} 2024-04-16T18:25:24.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065643cd71a1d4b64-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.067 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:24.067 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:24.067 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.067 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:25:24.067 DEBUG:boto:Data: 2024-04-16T18:25:24.067 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.068 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.068 DEBUG:boto:Params: {} 2024-04-16T18:25:24.068 DEBUG:boto:Token: None 2024-04-16T18:25:24.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj51 2024-04-16T18:25:24.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kBTr3amaORbMX1TrF7ejci7incg= 2024-04-16T18:25:24.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kBTr3amaORbMX1TrF7ejci7incg=', 'Content-Length': '0'} 2024-04-16T18:25:24.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007365d551779a1cab-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:25:24.071 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:24.071 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:24.071 DEBUG:boto:Method: GET 2024-04-16T18:25:24.072 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:25:24.072 DEBUG:boto:Data: 2024-04-16T18:25:24.072 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.072 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.072 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.072 DEBUG:boto:Params: {} 2024-04-16T18:25:24.072 DEBUG:boto:Token: None 2024-04-16T18:25:24.072 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj51 2024-04-16T18:25:24.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Izuqc19K1NbSEKA7+W4LYKEcXVE= 2024-04-16T18:25:24.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Izuqc19K1NbSEKA7+W4LYKEcXVE=', 'Content-Length': '0'} 2024-04-16T18:25:24.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aadd2b32910c108c-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.118 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:25:24.118 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:25:24.118 DEBUG:boto:Method: GET 2024-04-16T18:25:24.119 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:25:24.119 DEBUG:boto:Data: 2024-04-16T18:25:24.119 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.119 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.119 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.119 DEBUG:boto:Params: {} 2024-04-16T18:25:24.119 DEBUG:boto:Token: None 2024-04-16T18:25:24.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj51 2024-04-16T18:25:24.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Izuqc19K1NbSEKA7+W4LYKEcXVE= 2024-04-16T18:25:24.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Izuqc19K1NbSEKA7+W4LYKEcXVE=', 'Content-Length': '0'} 2024-04-16T18:25:24.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c83087643352acbc-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.123 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.123 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.123 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.123 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.123 DEBUG:boto:Method: GET 2024-04-16T18:25:24.123 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:25:24.123 DEBUG:boto:Data: 2024-04-16T18:25:24.123 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.123 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.123 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.124 DEBUG:boto:Params: {} 2024-04-16T18:25:24.124 DEBUG:boto:Token: None 2024-04-16T18:25:24.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.127 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c4dbf45975b91ec-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.170 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.171 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.171 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.171 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.171 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:25:24.171 DEBUG:boto:Method: GET 2024-04-16T18:25:24.171 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:25:24.171 DEBUG:boto:Data: 2024-04-16T18:25:24.171 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.171 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.171 DEBUG:boto:Params: {} 2024-04-16T18:25:24.172 DEBUG:boto:Token: None 2024-04-16T18:25:24.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc589e8a0ac1e465-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.177 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.177 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:25:24.177 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.177 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.177 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.177 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.177 DEBUG:boto:Method: GET 2024-04-16T18:25:24.177 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.178 DEBUG:boto:Data: 2024-04-16T18:25:24.178 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.178 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.178 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.178 DEBUG:boto:Params: {} 2024-04-16T18:25:24.178 DEBUG:boto:Token: None 2024-04-16T18:25:24.178 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.178 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ= 2024-04-16T18:25:24.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ=', 'Content-Length': '0'} 2024-04-16T18:25:24.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9d7b17fd8313ef0-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.222 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.222 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.223 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.223 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.223 DEBUG:boto:Method: GET 2024-04-16T18:25:24.223 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.223 DEBUG:boto:Data: 2024-04-16T18:25:24.223 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.223 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.223 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.223 DEBUG:boto:Params: {} 2024-04-16T18:25:24.223 DEBUG:boto:Token: None 2024-04-16T18:25:24.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ= 2024-04-16T18:25:24.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ=', 'Content-Length': '0'} 2024-04-16T18:25:24.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdae13f3688a2ea3-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.227 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.227 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.227 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.227 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.227 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.227 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.227 DEBUG:boto:Data: 2024-04-16T18:25:24.227 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.228 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.228 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.228 DEBUG:boto:Params: {} 2024-04-16T18:25:24.228 DEBUG:boto:Token: None 2024-04-16T18:25:24.228 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sPU7AffAy21qiunbJspcAq5If2o= 2024-04-16T18:25:24.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sPU7AffAy21qiunbJspcAq5If2o=', 'Content-Length': '0'} 2024-04-16T18:25:24.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001285c8e6507a5ac7-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.231 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.231 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.231 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.231 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.231 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.232 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.232 DEBUG:boto:Data: 2024-04-16T18:25:24.232 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.232 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.232 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.232 DEBUG:boto:Params: {} 2024-04-16T18:25:24.232 DEBUG:boto:Token: None 2024-04-16T18:25:24.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sPU7AffAy21qiunbJspcAq5If2o= 2024-04-16T18:25:24.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sPU7AffAy21qiunbJspcAq5If2o=', 'Content-Length': '0'} 2024-04-16T18:25:24.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023f9f4d14e885f9c-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:25:24.235 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.235 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.236 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.236 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.236 DEBUG:boto:Method: GET 2024-04-16T18:25:24.236 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.236 DEBUG:boto:Data: 2024-04-16T18:25:24.236 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.236 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.236 DEBUG:boto:Params: {} 2024-04-16T18:25:24.236 DEBUG:boto:Token: None 2024-04-16T18:25:24.237 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ= 2024-04-16T18:25:24.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ=', 'Content-Length': '0'} 2024-04-16T18:25:24.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000071b3bf27d8cb489-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.282 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.282 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.282 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.283 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.283 DEBUG:boto:Method: GET 2024-04-16T18:25:24.283 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.283 DEBUG:boto:Data: 2024-04-16T18:25:24.283 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.283 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.283 DEBUG:boto:Params: {} 2024-04-16T18:25:24.283 DEBUG:boto:Token: None 2024-04-16T18:25:24.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:25:24.283 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ= 2024-04-16T18:25:24.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:381rwVK5cbsum/12pLlnwzyetbQ=', 'Content-Length': '0'} 2024-04-16T18:25:24.287 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5e2c37ebd999988-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.287 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.287 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.287 DEBUG:boto:Method: GET 2024-04-16T18:25:24.288 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:25:24.288 DEBUG:boto:Data: 2024-04-16T18:25:24.288 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.288 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.288 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.288 DEBUG:boto:Params: {} 2024-04-16T18:25:24.288 DEBUG:boto:Token: None 2024-04-16T18:25:24.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021cafb60234f6fb5-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.292 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.293 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.293 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.293 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.293 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.293 DEBUG:boto:Method: GET 2024-04-16T18:25:24.293 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:25:24.293 DEBUG:boto:Data: 2024-04-16T18:25:24.293 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.293 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.293 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.293 DEBUG:boto:Params: {} 2024-04-16T18:25:24.294 DEBUG:boto:Token: None 2024-04-16T18:25:24.294 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.299 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000666335265ce1803f-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.342 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.343 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.343 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.343 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.343 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:25:24.343 DEBUG:boto:Data: 2024-04-16T18:25:24.343 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.343 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.343 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.343 DEBUG:boto:Params: {} 2024-04-16T18:25:24.343 DEBUG:boto:Token: None 2024-04-16T18:25:24.343 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55 2024-04-16T18:25:24.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OxfUYzot83d/y/gU33mH8yCpRDc= 2024-04-16T18:25:24.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OxfUYzot83d/y/gU33mH8yCpRDc=', 'Content-Length': '0'} 2024-04-16T18:25:24.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000898f4a199658d083-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.347 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.347 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.347 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.347 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:25:24.347 DEBUG:boto:Data: 2024-04-16T18:25:24.347 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.347 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.347 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.347 DEBUG:boto:Params: {} 2024-04-16T18:25:24.347 DEBUG:boto:Token: None 2024-04-16T18:25:24.348 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55 2024-04-16T18:25:24.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OxfUYzot83d/y/gU33mH8yCpRDc= 2024-04-16T18:25:24.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OxfUYzot83d/y/gU33mH8yCpRDc=', 'Content-Length': '0'} 2024-04-16T18:25:24.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b7535d96a703b2f-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.351 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:25:24.351 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.351 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.351 DEBUG:boto:Method: GET 2024-04-16T18:25:24.351 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:25:24.351 DEBUG:boto:Data: 2024-04-16T18:25:24.351 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.351 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.351 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.351 DEBUG:boto:Params: {} 2024-04-16T18:25:24.351 DEBUG:boto:Token: None 2024-04-16T18:25:24.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55 2024-04-16T18:25:24.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hx6y0xx7G6SIJDd8XN6hk8VnGXQ= 2024-04-16T18:25:24.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hx6y0xx7G6SIJDd8XN6hk8VnGXQ=', 'Content-Length': '0'} 2024-04-16T18:25:24.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071eb3bf84db13c6d-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.398 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:25:24.398 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:25:24.399 DEBUG:boto:Method: GET 2024-04-16T18:25:24.399 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:25:24.399 DEBUG:boto:Data: 2024-04-16T18:25:24.399 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.399 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.399 DEBUG:boto:Params: {} 2024-04-16T18:25:24.399 DEBUG:boto:Token: None 2024-04-16T18:25:24.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj55 2024-04-16T18:25:24.399 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hx6y0xx7G6SIJDd8XN6hk8VnGXQ= 2024-04-16T18:25:24.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hx6y0xx7G6SIJDd8XN6hk8VnGXQ=', 'Content-Length': '0'} 2024-04-16T18:25:24.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000599898167bf0dc80-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.403 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.403 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.403 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.403 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.403 DEBUG:boto:Method: GET 2024-04-16T18:25:24.403 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:25:24.403 DEBUG:boto:Data: 2024-04-16T18:25:24.403 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.403 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.403 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.403 DEBUG:boto:Params: {} 2024-04-16T18:25:24.404 DEBUG:boto:Token: None 2024-04-16T18:25:24.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003827dbe7210dc451-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.450 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.451 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.451 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.451 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.451 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:25:24.451 DEBUG:boto:Method: GET 2024-04-16T18:25:24.451 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:25:24.451 DEBUG:boto:Data: 2024-04-16T18:25:24.451 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.452 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.452 DEBUG:boto:Params: {} 2024-04-16T18:25:24.452 DEBUG:boto:Token: None 2024-04-16T18:25:24.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.457 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c269871fb2a98582-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.458 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:25:24.458 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.458 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.458 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.459 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.459 DEBUG:boto:Method: GET 2024-04-16T18:25:24.459 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.459 DEBUG:boto:Data: 2024-04-16T18:25:24.459 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.459 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.459 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.459 DEBUG:boto:Params: {} 2024-04-16T18:25:24.459 DEBUG:boto:Token: None 2024-04-16T18:25:24.459 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc= 2024-04-16T18:25:24.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc=', 'Content-Length': '0'} 2024-04-16T18:25:24.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079ca46ae36a57f1c-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.506 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.506 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.507 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.507 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.507 DEBUG:boto:Method: GET 2024-04-16T18:25:24.507 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.507 DEBUG:boto:Data: 2024-04-16T18:25:24.507 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.507 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.508 DEBUG:boto:Params: {} 2024-04-16T18:25:24.508 DEBUG:boto:Token: None 2024-04-16T18:25:24.508 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc= 2024-04-16T18:25:24.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc=', 'Content-Length': '0'} 2024-04-16T18:25:24.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000160c13850d7b0bb5-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.511 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.512 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.512 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.512 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.512 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.512 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.512 DEBUG:boto:Data: 2024-04-16T18:25:24.512 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.512 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.512 DEBUG:boto:Params: {} 2024-04-16T18:25:24.512 DEBUG:boto:Token: None 2024-04-16T18:25:24.513 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:488joVCyp3QXVdXZJcgmebBvao0= 2024-04-16T18:25:24.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:488joVCyp3QXVdXZJcgmebBvao0=', 'Content-Length': '0'} 2024-04-16T18:25:24.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e534d8c64c4b776-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.516 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.516 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.517 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.517 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.517 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.517 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.517 DEBUG:boto:Data: 2024-04-16T18:25:24.517 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.517 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.517 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.517 DEBUG:boto:Params: {} 2024-04-16T18:25:24.517 DEBUG:boto:Token: None 2024-04-16T18:25:24.517 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:488joVCyp3QXVdXZJcgmebBvao0= 2024-04-16T18:25:24.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:488joVCyp3QXVdXZJcgmebBvao0=', 'Content-Length': '0'} 2024-04-16T18:25:24.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e3d2b00aab168f6-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:25:24.521 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.521 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.521 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.521 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.521 DEBUG:boto:Method: GET 2024-04-16T18:25:24.521 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.521 DEBUG:boto:Data: 2024-04-16T18:25:24.521 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.521 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.522 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.522 DEBUG:boto:Params: {} 2024-04-16T18:25:24.522 DEBUG:boto:Token: None 2024-04-16T18:25:24.522 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc= 2024-04-16T18:25:24.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc=', 'Content-Length': '0'} 2024-04-16T18:25:24.525 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000607ef25bdab88c2b-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.566 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.566 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.566 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.567 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.567 DEBUG:boto:Method: GET 2024-04-16T18:25:24.567 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.567 DEBUG:boto:Data: 2024-04-16T18:25:24.567 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.567 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.567 DEBUG:boto:Params: {} 2024-04-16T18:25:24.567 DEBUG:boto:Token: None 2024-04-16T18:25:24.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:25:24.567 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc= 2024-04-16T18:25:24.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G0Hc755hQKF5EeXgT6yHtNCINnc=', 'Content-Length': '0'} 2024-04-16T18:25:24.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc53266b9cb57fa5-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.571 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.571 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.571 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.571 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.571 DEBUG:boto:Method: GET 2024-04-16T18:25:24.571 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:25:24.571 DEBUG:boto:Data: 2024-04-16T18:25:24.572 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.572 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.572 DEBUG:boto:Params: {} 2024-04-16T18:25:24.572 DEBUG:boto:Token: None 2024-04-16T18:25:24.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3e5e4b3247e7a75-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.576 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.576 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.576 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.576 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.576 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.577 DEBUG:boto:Method: GET 2024-04-16T18:25:24.577 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:25:24.577 DEBUG:boto:Data: 2024-04-16T18:25:24.577 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.577 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.577 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.577 DEBUG:boto:Params: {} 2024-04-16T18:25:24.577 DEBUG:boto:Token: None 2024-04-16T18:25:24.577 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.582 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ac494fcb512a05e-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.622 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.623 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.623 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.623 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.623 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:25:24.623 DEBUG:boto:Data: 2024-04-16T18:25:24.623 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.623 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.623 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.623 DEBUG:boto:Params: {} 2024-04-16T18:25:24.624 DEBUG:boto:Token: None 2024-04-16T18:25:24.624 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57 2024-04-16T18:25:24.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zOhUAMRJUqCUMLEDOLYL3Gj0NW0= 2024-04-16T18:25:24.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zOhUAMRJUqCUMLEDOLYL3Gj0NW0=', 'Content-Length': '0'} 2024-04-16T18:25:24.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002004de09bf07af62-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.627 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.627 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.627 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.627 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:25:24.627 DEBUG:boto:Data: 2024-04-16T18:25:24.627 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.628 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.628 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.628 DEBUG:boto:Params: {} 2024-04-16T18:25:24.628 DEBUG:boto:Token: None 2024-04-16T18:25:24.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57 2024-04-16T18:25:24.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zOhUAMRJUqCUMLEDOLYL3Gj0NW0= 2024-04-16T18:25:24.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zOhUAMRJUqCUMLEDOLYL3Gj0NW0=', 'Content-Length': '0'} 2024-04-16T18:25:24.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d63728eaa5e53f80-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:25:24.631 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.631 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.632 DEBUG:boto:Method: GET 2024-04-16T18:25:24.632 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:25:24.632 DEBUG:boto:Data: 2024-04-16T18:25:24.632 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.632 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.632 DEBUG:boto:Params: {} 2024-04-16T18:25:24.632 DEBUG:boto:Token: None 2024-04-16T18:25:24.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57 2024-04-16T18:25:24.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ESIPPYVO7Rnqj8dUNMqoI8yldHs= 2024-04-16T18:25:24.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ESIPPYVO7Rnqj8dUNMqoI8yldHs=', 'Content-Length': '0'} 2024-04-16T18:25:24.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000400d657ddecfd712-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.678 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:25:24.678 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:25:24.679 DEBUG:boto:Method: GET 2024-04-16T18:25:24.679 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:25:24.679 DEBUG:boto:Data: 2024-04-16T18:25:24.679 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.679 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.679 DEBUG:boto:Params: {} 2024-04-16T18:25:24.679 DEBUG:boto:Token: None 2024-04-16T18:25:24.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj57 2024-04-16T18:25:24.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ESIPPYVO7Rnqj8dUNMqoI8yldHs= 2024-04-16T18:25:24.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ESIPPYVO7Rnqj8dUNMqoI8yldHs=', 'Content-Length': '0'} 2024-04-16T18:25:24.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c0213b368af4d3c-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.683 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.683 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.683 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.683 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.684 DEBUG:boto:Method: GET 2024-04-16T18:25:24.684 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:25:24.684 DEBUG:boto:Data: 2024-04-16T18:25:24.684 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.684 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.684 DEBUG:boto:Params: {} 2024-04-16T18:25:24.684 DEBUG:boto:Token: None 2024-04-16T18:25:24.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.688 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb599e4903e9e976-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.730 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.731 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.731 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.731 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.731 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:25:24.731 DEBUG:boto:Method: GET 2024-04-16T18:25:24.731 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:25:24.731 DEBUG:boto:Data: 2024-04-16T18:25:24.731 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.731 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.731 DEBUG:boto:Params: {} 2024-04-16T18:25:24.732 DEBUG:boto:Token: None 2024-04-16T18:25:24.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.737 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050dc2b3089bfa70e-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.737 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.737 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:25:24.737 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.737 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.738 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.738 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.738 DEBUG:boto:Method: GET 2024-04-16T18:25:24.738 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.738 DEBUG:boto:Data: 2024-04-16T18:25:24.738 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.738 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.738 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.738 DEBUG:boto:Params: {} 2024-04-16T18:25:24.738 DEBUG:boto:Token: None 2024-04-16T18:25:24.739 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.739 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M= 2024-04-16T18:25:24.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M=', 'Content-Length': '0'} 2024-04-16T18:25:24.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8a6d45fbeea54f0-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.782 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.782 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.783 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.783 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.783 DEBUG:boto:Method: GET 2024-04-16T18:25:24.783 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.783 DEBUG:boto:Data: 2024-04-16T18:25:24.783 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.783 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.783 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.783 DEBUG:boto:Params: {} 2024-04-16T18:25:24.783 DEBUG:boto:Token: None 2024-04-16T18:25:24.783 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.783 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M= 2024-04-16T18:25:24.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M=', 'Content-Length': '0'} 2024-04-16T18:25:24.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b33c598431433131-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.787 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.787 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.787 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.787 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.787 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.787 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.787 DEBUG:boto:Data: 2024-04-16T18:25:24.787 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.788 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.788 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.788 DEBUG:boto:Params: {} 2024-04-16T18:25:24.788 DEBUG:boto:Token: None 2024-04-16T18:25:24.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E+8R/e9tXJAJA2hvnqksemlP7mU= 2024-04-16T18:25:24.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E+8R/e9tXJAJA2hvnqksemlP7mU=', 'Content-Length': '0'} 2024-04-16T18:25:24.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b545be391695e89-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.791 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.791 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.791 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.791 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.792 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.792 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.792 DEBUG:boto:Data: 2024-04-16T18:25:24.792 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.792 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.792 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.792 DEBUG:boto:Params: {} 2024-04-16T18:25:24.792 DEBUG:boto:Token: None 2024-04-16T18:25:24.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E+8R/e9tXJAJA2hvnqksemlP7mU= 2024-04-16T18:25:24.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E+8R/e9tXJAJA2hvnqksemlP7mU=', 'Content-Length': '0'} 2024-04-16T18:25:24.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db5a29ad8b674623-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:25:24.796 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.796 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.796 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.796 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.796 DEBUG:boto:Method: GET 2024-04-16T18:25:24.796 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.796 DEBUG:boto:Data: 2024-04-16T18:25:24.796 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.796 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.796 DEBUG:boto:Params: {} 2024-04-16T18:25:24.797 DEBUG:boto:Token: None 2024-04-16T18:25:24.797 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M= 2024-04-16T18:25:24.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M=', 'Content-Length': '0'} 2024-04-16T18:25:24.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056b0fc0e20256590-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.842 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.842 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.843 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.843 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.843 DEBUG:boto:Method: GET 2024-04-16T18:25:24.843 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.843 DEBUG:boto:Data: 2024-04-16T18:25:24.843 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.843 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.843 DEBUG:boto:Params: {} 2024-04-16T18:25:24.843 DEBUG:boto:Token: None 2024-04-16T18:25:24.844 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:25:24.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M= 2024-04-16T18:25:24.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sBgWQJlf1HamZmtSGzrBV/d2h9M=', 'Content-Length': '0'} 2024-04-16T18:25:24.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2602f79881c4de6-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.847 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.847 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.848 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:24.848 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:24.848 DEBUG:boto:Method: GET 2024-04-16T18:25:24.848 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:25:24.848 DEBUG:boto:Data: 2024-04-16T18:25:24.848 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.848 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.848 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.848 DEBUG:boto:Params: {} 2024-04-16T18:25:24.849 DEBUG:boto:Token: None 2024-04-16T18:25:24.849 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.849 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.853 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c08a5b5af4c10da4-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.853 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.853 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.853 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.853 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:24.854 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:24.854 DEBUG:boto:Method: GET 2024-04-16T18:25:24.854 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:25:24.854 DEBUG:boto:Data: 2024-04-16T18:25:24.854 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.854 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.854 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.854 DEBUG:boto:Params: {} 2024-04-16T18:25:24.854 DEBUG:boto:Token: None 2024-04-16T18:25:24.854 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfefbc9428cf527e-00661ec294-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.902 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:24.902 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.903 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.903 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.903 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:25:24.903 DEBUG:boto:Data: 2024-04-16T18:25:24.903 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.903 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.903 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.903 DEBUG:boto:Params: {} 2024-04-16T18:25:24.903 DEBUG:boto:Token: None 2024-04-16T18:25:24.903 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68 2024-04-16T18:25:24.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V/lCz9KvuKWpDHbjg6TGqGHiEGo= 2024-04-16T18:25:24.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V/lCz9KvuKWpDHbjg6TGqGHiEGo=', 'Content-Length': '0'} 2024-04-16T18:25:24.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1edafdc6fb51ce0-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.906 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.907 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.907 DEBUG:boto:Method: HEAD 2024-04-16T18:25:24.907 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:25:24.907 DEBUG:boto:Data: 2024-04-16T18:25:24.907 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.907 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.907 DEBUG:boto:Params: {} 2024-04-16T18:25:24.907 DEBUG:boto:Token: None 2024-04-16T18:25:24.907 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68 2024-04-16T18:25:24.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V/lCz9KvuKWpDHbjg6TGqGHiEGo= 2024-04-16T18:25:24.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V/lCz9KvuKWpDHbjg6TGqGHiEGo=', 'Content-Length': '0'} 2024-04-16T18:25:24.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003b2f3e9248b01cb-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.911 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:25:24.911 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.911 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.911 DEBUG:boto:Method: GET 2024-04-16T18:25:24.911 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:25:24.911 DEBUG:boto:Data: 2024-04-16T18:25:24.911 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.912 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.912 DEBUG:boto:Params: {} 2024-04-16T18:25:24.912 DEBUG:boto:Token: None 2024-04-16T18:25:24.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68 2024-04-16T18:25:24.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j1iPbV20gYGsYymnfAYEYMaKR8A= 2024-04-16T18:25:24.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j1iPbV20gYGsYymnfAYEYMaKR8A=', 'Content-Length': '0'} 2024-04-16T18:25:24.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000211bda47d2c0546c-00661ec294-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.958 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:25:24.958 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:25:24.959 DEBUG:boto:Method: GET 2024-04-16T18:25:24.959 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:25:24.959 DEBUG:boto:Data: 2024-04-16T18:25:24.959 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:24.959 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.959 DEBUG:boto:Params: {} 2024-04-16T18:25:24.959 DEBUG:boto:Token: None 2024-04-16T18:25:24.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/obj68 2024-04-16T18:25:24.959 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j1iPbV20gYGsYymnfAYEYMaKR8A= 2024-04-16T18:25:24.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j1iPbV20gYGsYymnfAYEYMaKR8A=', 'Content-Length': '0'} 2024-04-16T18:25:24.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7c4d37df721f9d2-00661ec294-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:24.963 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:24.963 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:24.963 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:24.963 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:24.963 DEBUG:boto:Method: GET 2024-04-16T18:25:24.963 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:25:24.963 DEBUG:boto:Data: 2024-04-16T18:25:24.963 DEBUG:boto:Headers: {} 2024-04-16T18:25:24.963 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:24.963 DEBUG:boto:Port: 8000 2024-04-16T18:25:24.963 DEBUG:boto:Params: {} 2024-04-16T18:25:24.964 DEBUG:boto:Token: None 2024-04-16T18:25:24.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:24 GMT /lybtug-52/ 2024-04-16T18:25:24.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg= 2024-04-16T18:25:24.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3AQt5zS8fOubSx14918eJMsyKmg=', 'Content-Length': '0'} 2024-04-16T18:25:24.967 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e02c416f724a6f29-00661ec294-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.010 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.011 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.011 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.011 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:25.011 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:25:25.011 DEBUG:boto:Method: GET 2024-04-16T18:25:25.011 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:25:25.011 DEBUG:boto:Data: 2024-04-16T18:25:25.011 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.011 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.011 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.011 DEBUG:boto:Params: {} 2024-04-16T18:25:25.012 DEBUG:boto:Token: None 2024-04-16T18:25:25.012 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.012 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.017 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002703eba848b9c1e0-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.017 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.017 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:25:25.017 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.018 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.018 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.018 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.018 DEBUG:boto:Method: GET 2024-04-16T18:25:25.018 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.018 DEBUG:boto:Data: 2024-04-16T18:25:25.018 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.018 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.018 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.018 DEBUG:boto:Params: {} 2024-04-16T18:25:25.018 DEBUG:boto:Token: None 2024-04-16T18:25:25.019 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.019 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc= 2024-04-16T18:25:25.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc=', 'Content-Length': '0'} 2024-04-16T18:25:25.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c328369f7d96c146-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.062 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.062 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.063 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.063 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.063 DEBUG:boto:Method: GET 2024-04-16T18:25:25.063 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.063 DEBUG:boto:Data: 2024-04-16T18:25:25.063 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.063 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.063 DEBUG:boto:Params: {} 2024-04-16T18:25:25.063 DEBUG:boto:Token: None 2024-04-16T18:25:25.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc= 2024-04-16T18:25:25.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc=', 'Content-Length': '0'} 2024-04-16T18:25:25.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cefb7c14854ead4-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.067 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.067 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.067 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.067 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.067 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.067 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.067 DEBUG:boto:Data: 2024-04-16T18:25:25.068 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.068 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.068 DEBUG:boto:Params: {} 2024-04-16T18:25:25.068 DEBUG:boto:Token: None 2024-04-16T18:25:25.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cRgoF4/AlF7y4GZvn2whX/8THbs= 2024-04-16T18:25:25.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cRgoF4/AlF7y4GZvn2whX/8THbs=', 'Content-Length': '0'} 2024-04-16T18:25:25.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000971de5537b006b2f-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.071 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.072 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.072 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.072 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.072 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.072 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.072 DEBUG:boto:Data: 2024-04-16T18:25:25.072 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.072 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.072 DEBUG:boto:Params: {} 2024-04-16T18:25:25.072 DEBUG:boto:Token: None 2024-04-16T18:25:25.073 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cRgoF4/AlF7y4GZvn2whX/8THbs= 2024-04-16T18:25:25.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cRgoF4/AlF7y4GZvn2whX/8THbs=', 'Content-Length': '0'} 2024-04-16T18:25:25.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0e2b228e2b5e19a-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:25:25.076 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.076 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.076 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.076 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.077 DEBUG:boto:Method: GET 2024-04-16T18:25:25.077 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.077 DEBUG:boto:Data: 2024-04-16T18:25:25.077 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.077 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.077 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.077 DEBUG:boto:Params: {} 2024-04-16T18:25:25.077 DEBUG:boto:Token: None 2024-04-16T18:25:25.077 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.077 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc= 2024-04-16T18:25:25.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc=', 'Content-Length': '0'} 2024-04-16T18:25:25.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000118400451c4bb68f-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.122 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.122 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.123 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.123 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.123 DEBUG:boto:Method: GET 2024-04-16T18:25:25.123 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.123 DEBUG:boto:Data: 2024-04-16T18:25:25.123 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.123 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.123 DEBUG:boto:Params: {} 2024-04-16T18:25:25.123 DEBUG:boto:Token: None 2024-04-16T18:25:25.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:25:25.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc= 2024-04-16T18:25:25.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+KB34lb9mls9KoF8q6bjTzgWUrc=', 'Content-Length': '0'} 2024-04-16T18:25:25.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9bd3909c4e60ab9-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.127 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.127 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.127 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.127 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.127 DEBUG:boto:Method: GET 2024-04-16T18:25:25.128 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:25:25.128 DEBUG:boto:Data: 2024-04-16T18:25:25.128 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.128 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.128 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.128 DEBUG:boto:Params: {} 2024-04-16T18:25:25.128 DEBUG:boto:Token: None 2024-04-16T18:25:25.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9418b3cb73593e9-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.132 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.132 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.132 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.133 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.133 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.133 DEBUG:boto:Method: GET 2024-04-16T18:25:25.133 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:25:25.133 DEBUG:boto:Data: 2024-04-16T18:25:25.133 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.133 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.133 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.133 DEBUG:boto:Params: {} 2024-04-16T18:25:25.133 DEBUG:boto:Token: None 2024-04-16T18:25:25.133 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.139 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005344fc35df01e59e-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.182 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.183 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.183 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.183 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.183 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:25:25.183 DEBUG:boto:Data: 2024-04-16T18:25:25.183 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.183 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.183 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.183 DEBUG:boto:Params: {} 2024-04-16T18:25:25.184 DEBUG:boto:Token: None 2024-04-16T18:25:25.184 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69 2024-04-16T18:25:25.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o7mDye8Z8L3ZguJEBa3Bx1xnXEc= 2024-04-16T18:25:25.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o7mDye8Z8L3ZguJEBa3Bx1xnXEc=', 'Content-Length': '0'} 2024-04-16T18:25:25.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064957c4711cff0f0-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.187 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.187 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.187 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.187 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:25:25.187 DEBUG:boto:Data: 2024-04-16T18:25:25.187 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.187 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.188 DEBUG:boto:Params: {} 2024-04-16T18:25:25.188 DEBUG:boto:Token: None 2024-04-16T18:25:25.188 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69 2024-04-16T18:25:25.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o7mDye8Z8L3ZguJEBa3Bx1xnXEc= 2024-04-16T18:25:25.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o7mDye8Z8L3ZguJEBa3Bx1xnXEc=', 'Content-Length': '0'} 2024-04-16T18:25:25.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052305382fcb587c4-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.191 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:25:25.191 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.191 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.191 DEBUG:boto:Method: GET 2024-04-16T18:25:25.191 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:25:25.191 DEBUG:boto:Data: 2024-04-16T18:25:25.192 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.192 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.192 DEBUG:boto:Params: {} 2024-04-16T18:25:25.192 DEBUG:boto:Token: None 2024-04-16T18:25:25.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69 2024-04-16T18:25:25.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e16LCsA2nX2f5nVr4uWjXKw+YLY= 2024-04-16T18:25:25.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e16LCsA2nX2f5nVr4uWjXKw+YLY=', 'Content-Length': '0'} 2024-04-16T18:25:25.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d2463410b4f5354-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.238 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:25:25.238 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:25:25.239 DEBUG:boto:Method: GET 2024-04-16T18:25:25.239 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:25:25.239 DEBUG:boto:Data: 2024-04-16T18:25:25.239 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.239 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.239 DEBUG:boto:Params: {} 2024-04-16T18:25:25.239 DEBUG:boto:Token: None 2024-04-16T18:25:25.239 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj69 2024-04-16T18:25:25.239 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e16LCsA2nX2f5nVr4uWjXKw+YLY= 2024-04-16T18:25:25.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e16LCsA2nX2f5nVr4uWjXKw+YLY=', 'Content-Length': '0'} 2024-04-16T18:25:25.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b690b522d29e1b5-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.243 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.243 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.243 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.243 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.243 DEBUG:boto:Method: GET 2024-04-16T18:25:25.243 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:25:25.243 DEBUG:boto:Data: 2024-04-16T18:25:25.243 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.243 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.243 DEBUG:boto:Params: {} 2024-04-16T18:25:25.244 DEBUG:boto:Token: None 2024-04-16T18:25:25.244 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.247 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073cfe528dc5df7e3-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.290 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.291 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.291 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.291 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.291 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:25:25.291 DEBUG:boto:Method: GET 2024-04-16T18:25:25.291 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:25:25.291 DEBUG:boto:Data: 2024-04-16T18:25:25.291 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.291 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.292 DEBUG:boto:Params: {} 2024-04-16T18:25:25.292 DEBUG:boto:Token: None 2024-04-16T18:25:25.292 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.297 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9f930b2c5f77f49-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.297 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.298 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:25:25.298 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.299 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.299 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.299 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.299 DEBUG:boto:Method: GET 2024-04-16T18:25:25.299 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.299 DEBUG:boto:Data: 2024-04-16T18:25:25.299 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.299 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.299 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.299 DEBUG:boto:Params: {} 2024-04-16T18:25:25.300 DEBUG:boto:Token: None 2024-04-16T18:25:25.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo= 2024-04-16T18:25:25.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo=', 'Content-Length': '0'} 2024-04-16T18:25:25.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010c26e29d693a8fb-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.346 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.346 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.347 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.347 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.347 DEBUG:boto:Method: GET 2024-04-16T18:25:25.347 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.347 DEBUG:boto:Data: 2024-04-16T18:25:25.347 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.347 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.347 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.347 DEBUG:boto:Params: {} 2024-04-16T18:25:25.347 DEBUG:boto:Token: None 2024-04-16T18:25:25.347 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo= 2024-04-16T18:25:25.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo=', 'Content-Length': '0'} 2024-04-16T18:25:25.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef9ac2c7746c2c13-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.351 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.351 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.351 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.351 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.352 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.352 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.352 DEBUG:boto:Data: 2024-04-16T18:25:25.352 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.353 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.353 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.353 DEBUG:boto:Params: {} 2024-04-16T18:25:25.353 DEBUG:boto:Token: None 2024-04-16T18:25:25.353 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JdRpQdCxrwTAHLDSTymlq9rBikI= 2024-04-16T18:25:25.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JdRpQdCxrwTAHLDSTymlq9rBikI=', 'Content-Length': '0'} 2024-04-16T18:25:25.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078923ed3b06e6933-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.357 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.357 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.357 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.357 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.357 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.357 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.357 DEBUG:boto:Data: 2024-04-16T18:25:25.357 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.357 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.357 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.358 DEBUG:boto:Params: {} 2024-04-16T18:25:25.358 DEBUG:boto:Token: None 2024-04-16T18:25:25.358 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.358 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JdRpQdCxrwTAHLDSTymlq9rBikI= 2024-04-16T18:25:25.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JdRpQdCxrwTAHLDSTymlq9rBikI=', 'Content-Length': '0'} 2024-04-16T18:25:25.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a4d17898d5486df-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.361 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:25:25.361 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.361 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.361 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.361 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.361 DEBUG:boto:Method: GET 2024-04-16T18:25:25.361 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.362 DEBUG:boto:Data: 2024-04-16T18:25:25.362 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.362 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.362 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.362 DEBUG:boto:Params: {} 2024-04-16T18:25:25.363 DEBUG:boto:Token: None 2024-04-16T18:25:25.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.363 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo= 2024-04-16T18:25:25.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo=', 'Content-Length': '0'} 2024-04-16T18:25:25.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000375460c62bea48ee-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.406 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.406 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.406 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.407 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.407 DEBUG:boto:Method: GET 2024-04-16T18:25:25.407 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.407 DEBUG:boto:Data: 2024-04-16T18:25:25.407 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.407 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.407 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.407 DEBUG:boto:Params: {} 2024-04-16T18:25:25.407 DEBUG:boto:Token: None 2024-04-16T18:25:25.407 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:25:25.407 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo= 2024-04-16T18:25:25.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ocGz/LUOC43CWADNpgeTagGdnYo=', 'Content-Length': '0'} 2024-04-16T18:25:25.410 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062891713227111ce-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.411 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.411 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.411 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.411 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.411 DEBUG:boto:Method: GET 2024-04-16T18:25:25.411 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:25:25.411 DEBUG:boto:Data: 2024-04-16T18:25:25.411 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.411 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.411 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.412 DEBUG:boto:Params: {} 2024-04-16T18:25:25.412 DEBUG:boto:Token: None 2024-04-16T18:25:25.412 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.416 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a9c228a16d145ab-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.416 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.416 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.416 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.416 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.417 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.417 DEBUG:boto:Method: GET 2024-04-16T18:25:25.417 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:25:25.417 DEBUG:boto:Data: 2024-04-16T18:25:25.417 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.417 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.417 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.417 DEBUG:boto:Params: {} 2024-04-16T18:25:25.417 DEBUG:boto:Token: None 2024-04-16T18:25:25.417 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.418 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b3673055a9ac64e-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.466 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.467 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.467 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.467 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.467 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:25:25.467 DEBUG:boto:Data: 2024-04-16T18:25:25.467 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.467 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.467 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.467 DEBUG:boto:Params: {} 2024-04-16T18:25:25.467 DEBUG:boto:Token: None 2024-04-16T18:25:25.468 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76 2024-04-16T18:25:25.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jfjK4IZv+MbabBgN2QGqjNyc5rY= 2024-04-16T18:25:25.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jfjK4IZv+MbabBgN2QGqjNyc5rY=', 'Content-Length': '0'} 2024-04-16T18:25:25.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000457abe93e501deff-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.471 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.471 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.471 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.471 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:25:25.471 DEBUG:boto:Data: 2024-04-16T18:25:25.471 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.471 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.471 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.471 DEBUG:boto:Params: {} 2024-04-16T18:25:25.472 DEBUG:boto:Token: None 2024-04-16T18:25:25.472 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76 2024-04-16T18:25:25.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jfjK4IZv+MbabBgN2QGqjNyc5rY= 2024-04-16T18:25:25.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jfjK4IZv+MbabBgN2QGqjNyc5rY=', 'Content-Length': '0'} 2024-04-16T18:25:25.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e842b6acc6c6d3bc-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:25:25.475 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.475 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.475 DEBUG:boto:Method: GET 2024-04-16T18:25:25.476 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:25:25.476 DEBUG:boto:Data: 2024-04-16T18:25:25.476 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.476 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.476 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.476 DEBUG:boto:Params: {} 2024-04-16T18:25:25.476 DEBUG:boto:Token: None 2024-04-16T18:25:25.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76 2024-04-16T18:25:25.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rgxKnIpdiyYNJtwviZwc6JWW0SQ= 2024-04-16T18:25:25.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rgxKnIpdiyYNJtwviZwc6JWW0SQ=', 'Content-Length': '0'} 2024-04-16T18:25:25.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e23dbf6afed0d021-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.522 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:25:25.522 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:25:25.522 DEBUG:boto:Method: GET 2024-04-16T18:25:25.523 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:25:25.523 DEBUG:boto:Data: 2024-04-16T18:25:25.523 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.523 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.523 DEBUG:boto:Params: {} 2024-04-16T18:25:25.523 DEBUG:boto:Token: None 2024-04-16T18:25:25.523 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj76 2024-04-16T18:25:25.523 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rgxKnIpdiyYNJtwviZwc6JWW0SQ= 2024-04-16T18:25:25.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rgxKnIpdiyYNJtwviZwc6JWW0SQ=', 'Content-Length': '0'} 2024-04-16T18:25:25.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004319903dfa34b758-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.527 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.527 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.527 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.527 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.527 DEBUG:boto:Method: GET 2024-04-16T18:25:25.527 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:25:25.527 DEBUG:boto:Data: 2024-04-16T18:25:25.527 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.527 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.527 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.527 DEBUG:boto:Params: {} 2024-04-16T18:25:25.528 DEBUG:boto:Token: None 2024-04-16T18:25:25.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038d6dc4660d47d7a-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.574 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.575 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.575 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.575 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.575 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:25:25.575 DEBUG:boto:Method: GET 2024-04-16T18:25:25.575 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:25:25.575 DEBUG:boto:Data: 2024-04-16T18:25:25.575 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.575 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.576 DEBUG:boto:Params: {} 2024-04-16T18:25:25.576 DEBUG:boto:Token: None 2024-04-16T18:25:25.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.581 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017029dddfd8a956c-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.581 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.581 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:25:25.582 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.582 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.582 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.582 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.582 DEBUG:boto:Method: GET 2024-04-16T18:25:25.582 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.582 DEBUG:boto:Data: 2024-04-16T18:25:25.582 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.582 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.582 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.582 DEBUG:boto:Params: {} 2024-04-16T18:25:25.583 DEBUG:boto:Token: None 2024-04-16T18:25:25.583 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.583 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o= 2024-04-16T18:25:25.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o=', 'Content-Length': '0'} 2024-04-16T18:25:25.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e94befdf4bb9070-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.626 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.626 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.627 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.627 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.627 DEBUG:boto:Method: GET 2024-04-16T18:25:25.627 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.627 DEBUG:boto:Data: 2024-04-16T18:25:25.627 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.627 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.627 DEBUG:boto:Params: {} 2024-04-16T18:25:25.627 DEBUG:boto:Token: None 2024-04-16T18:25:25.627 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o= 2024-04-16T18:25:25.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o=', 'Content-Length': '0'} 2024-04-16T18:25:25.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f086257d2bf995fb-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.631 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.631 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.631 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.631 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.631 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.632 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.632 DEBUG:boto:Data: 2024-04-16T18:25:25.632 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.632 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.632 DEBUG:boto:Params: {} 2024-04-16T18:25:25.632 DEBUG:boto:Token: None 2024-04-16T18:25:25.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2e3RFcDVoHLbSm4A33S4qpstJ2g= 2024-04-16T18:25:25.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2e3RFcDVoHLbSm4A33S4qpstJ2g=', 'Content-Length': '0'} 2024-04-16T18:25:25.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058ecfed6faeb10f5-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.635 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.635 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.636 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.636 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.636 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.636 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.636 DEBUG:boto:Data: 2024-04-16T18:25:25.636 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.636 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.636 DEBUG:boto:Params: {} 2024-04-16T18:25:25.636 DEBUG:boto:Token: None 2024-04-16T18:25:25.636 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2e3RFcDVoHLbSm4A33S4qpstJ2g= 2024-04-16T18:25:25.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2e3RFcDVoHLbSm4A33S4qpstJ2g=', 'Content-Length': '0'} 2024-04-16T18:25:25.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df2e1cf29c282ca8-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:25:25.640 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.640 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.640 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.640 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.640 DEBUG:boto:Method: GET 2024-04-16T18:25:25.640 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.640 DEBUG:boto:Data: 2024-04-16T18:25:25.641 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.641 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.641 DEBUG:boto:Params: {} 2024-04-16T18:25:25.641 DEBUG:boto:Token: None 2024-04-16T18:25:25.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o= 2024-04-16T18:25:25.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o=', 'Content-Length': '0'} 2024-04-16T18:25:25.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000934ac456d617a8c6-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.686 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.686 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.686 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.687 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.687 DEBUG:boto:Method: GET 2024-04-16T18:25:25.687 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.687 DEBUG:boto:Data: 2024-04-16T18:25:25.687 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.687 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.687 DEBUG:boto:Params: {} 2024-04-16T18:25:25.687 DEBUG:boto:Token: None 2024-04-16T18:25:25.687 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:25:25.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o= 2024-04-16T18:25:25.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fi+oLoN6kPNABiSKoEKKdnLpK2o=', 'Content-Length': '0'} 2024-04-16T18:25:25.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f162c705527d226-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.691 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.691 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.691 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.691 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.691 DEBUG:boto:Method: GET 2024-04-16T18:25:25.691 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:25:25.691 DEBUG:boto:Data: 2024-04-16T18:25:25.692 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.692 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.692 DEBUG:boto:Params: {} 2024-04-16T18:25:25.692 DEBUG:boto:Token: None 2024-04-16T18:25:25.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef8bba1e3ea4ee84-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.696 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.697 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.697 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.697 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.697 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.697 DEBUG:boto:Method: GET 2024-04-16T18:25:25.697 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:25:25.697 DEBUG:boto:Data: 2024-04-16T18:25:25.697 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.697 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.697 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.698 DEBUG:boto:Params: {} 2024-04-16T18:25:25.698 DEBUG:boto:Token: None 2024-04-16T18:25:25.698 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.698 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016fb8073576e951c-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.746 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.747 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.747 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.747 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.747 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:25:25.747 DEBUG:boto:Data: 2024-04-16T18:25:25.747 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.747 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.747 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.747 DEBUG:boto:Params: {} 2024-04-16T18:25:25.747 DEBUG:boto:Token: None 2024-04-16T18:25:25.747 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85 2024-04-16T18:25:25.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:horv8yI8nxRcmr8L83qmLFhFqVc= 2024-04-16T18:25:25.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:horv8yI8nxRcmr8L83qmLFhFqVc=', 'Content-Length': '0'} 2024-04-16T18:25:25.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d069f81ef89cfc3a-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.751 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.751 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.751 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.751 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:25:25.751 DEBUG:boto:Data: 2024-04-16T18:25:25.751 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.751 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.752 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.752 DEBUG:boto:Params: {} 2024-04-16T18:25:25.752 DEBUG:boto:Token: None 2024-04-16T18:25:25.752 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85 2024-04-16T18:25:25.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:horv8yI8nxRcmr8L83qmLFhFqVc= 2024-04-16T18:25:25.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:horv8yI8nxRcmr8L83qmLFhFqVc=', 'Content-Length': '0'} 2024-04-16T18:25:25.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae300f15f19f5e38-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.755 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:25:25.755 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.755 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.755 DEBUG:boto:Method: GET 2024-04-16T18:25:25.755 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:25:25.755 DEBUG:boto:Data: 2024-04-16T18:25:25.755 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.756 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.756 DEBUG:boto:Params: {} 2024-04-16T18:25:25.756 DEBUG:boto:Token: None 2024-04-16T18:25:25.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85 2024-04-16T18:25:25.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nGpOyo+9qZvMmPUzhPW7Peg4wHk= 2024-04-16T18:25:25.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nGpOyo+9qZvMmPUzhPW7Peg4wHk=', 'Content-Length': '0'} 2024-04-16T18:25:25.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c628d03c2a5e4117-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.802 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:25:25.802 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:25:25.802 DEBUG:boto:Method: GET 2024-04-16T18:25:25.803 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:25:25.803 DEBUG:boto:Data: 2024-04-16T18:25:25.803 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.803 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.803 DEBUG:boto:Params: {} 2024-04-16T18:25:25.803 DEBUG:boto:Token: None 2024-04-16T18:25:25.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj85 2024-04-16T18:25:25.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nGpOyo+9qZvMmPUzhPW7Peg4wHk= 2024-04-16T18:25:25.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nGpOyo+9qZvMmPUzhPW7Peg4wHk=', 'Content-Length': '0'} 2024-04-16T18:25:25.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022723379e55ab43f-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.806 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.807 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.807 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.807 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.807 DEBUG:boto:Method: GET 2024-04-16T18:25:25.807 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:25:25.807 DEBUG:boto:Data: 2024-04-16T18:25:25.807 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.807 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.807 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.807 DEBUG:boto:Params: {} 2024-04-16T18:25:25.807 DEBUG:boto:Token: None 2024-04-16T18:25:25.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.811 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c8f106076be8986-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.854 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.854 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.855 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.855 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.855 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:25:25.855 DEBUG:boto:Method: GET 2024-04-16T18:25:25.855 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:25:25.855 DEBUG:boto:Data: 2024-04-16T18:25:25.855 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.855 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.855 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.855 DEBUG:boto:Params: {} 2024-04-16T18:25:25.856 DEBUG:boto:Token: None 2024-04-16T18:25:25.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.861 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa0a345c5fc133d8-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.861 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.861 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:25:25.861 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:25.861 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:25.861 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.862 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.862 DEBUG:boto:Method: GET 2024-04-16T18:25:25.862 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.862 DEBUG:boto:Data: 2024-04-16T18:25:25.862 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.862 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.862 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.862 DEBUG:boto:Params: {} 2024-04-16T18:25:25.862 DEBUG:boto:Token: None 2024-04-16T18:25:25.862 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.863 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk= 2024-04-16T18:25:25.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk=', 'Content-Length': '0'} 2024-04-16T18:25:25.866 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f08c0ad5321d251-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.906 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:25.906 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:25.907 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.907 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.907 DEBUG:boto:Method: GET 2024-04-16T18:25:25.907 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.907 DEBUG:boto:Data: 2024-04-16T18:25:25.907 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.907 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.907 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.907 DEBUG:boto:Params: {} 2024-04-16T18:25:25.907 DEBUG:boto:Token: None 2024-04-16T18:25:25.907 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk= 2024-04-16T18:25:25.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk=', 'Content-Length': '0'} 2024-04-16T18:25:25.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077a7aa5862a44a0f-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.911 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:25.911 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:25.911 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.911 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.911 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.911 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.911 DEBUG:boto:Data: 2024-04-16T18:25:25.911 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.911 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.911 DEBUG:boto:Params: {} 2024-04-16T18:25:25.912 DEBUG:boto:Token: None 2024-04-16T18:25:25.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yf7TVrtp5bR2Av6pxNuH61rQdng= 2024-04-16T18:25:25.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yf7TVrtp5bR2Av6pxNuH61rQdng=', 'Content-Length': '0'} 2024-04-16T18:25:25.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094e3f0872dfb41ef-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.915 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:25.915 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:25.915 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.915 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.915 DEBUG:boto:Method: HEAD 2024-04-16T18:25:25.915 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.915 DEBUG:boto:Data: 2024-04-16T18:25:25.916 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.916 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.916 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.916 DEBUG:boto:Params: {} 2024-04-16T18:25:25.916 DEBUG:boto:Token: None 2024-04-16T18:25:25.916 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yf7TVrtp5bR2Av6pxNuH61rQdng= 2024-04-16T18:25:25.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yf7TVrtp5bR2Av6pxNuH61rQdng=', 'Content-Length': '0'} 2024-04-16T18:25:25.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c14f887113d29a70-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:25:25.919 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:25.919 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:25.919 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.919 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.920 DEBUG:boto:Method: GET 2024-04-16T18:25:25.920 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.920 DEBUG:boto:Data: 2024-04-16T18:25:25.920 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.920 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.920 DEBUG:boto:Params: {} 2024-04-16T18:25:25.920 DEBUG:boto:Token: None 2024-04-16T18:25:25.920 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk= 2024-04-16T18:25:25.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk=', 'Content-Length': '0'} 2024-04-16T18:25:25.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d294e07f0e102a6-00661ec295-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.966 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:25.966 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:25.967 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.967 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.967 DEBUG:boto:Method: GET 2024-04-16T18:25:25.967 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.967 DEBUG:boto:Data: 2024-04-16T18:25:25.967 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.967 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.967 DEBUG:boto:Params: {} 2024-04-16T18:25:25.967 DEBUG:boto:Token: None 2024-04-16T18:25:25.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:25:25.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk= 2024-04-16T18:25:25.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBinci0dyY6OcyhSg58WHgsDFgk=', 'Content-Length': '0'} 2024-04-16T18:25:25.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e65dab2185a95a9d-00661ec295-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.971 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.971 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.971 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:25.971 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:25.971 DEBUG:boto:Method: GET 2024-04-16T18:25:25.972 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:25:25.972 DEBUG:boto:Data: 2024-04-16T18:25:25.972 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:25.972 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.972 DEBUG:boto:Params: {} 2024-04-16T18:25:25.972 DEBUG:boto:Token: None 2024-04-16T18:25:25.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004744c316c9b637b3-00661ec295-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:25.976 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:25.977 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:25.977 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:25.977 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:25.977 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:25.977 DEBUG:boto:Method: GET 2024-04-16T18:25:25.978 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:25:25.978 DEBUG:boto:Data: 2024-04-16T18:25:25.978 DEBUG:boto:Headers: {} 2024-04-16T18:25:25.978 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:25.978 DEBUG:boto:Port: 8000 2024-04-16T18:25:25.978 DEBUG:boto:Params: {} 2024-04-16T18:25:25.978 DEBUG:boto:Token: None 2024-04-16T18:25:25.979 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:25 GMT /lybtug-52/ 2024-04-16T18:25:25.979 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM= 2024-04-16T18:25:25.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O2N+fg1tItK0TJrXyKpnY+TU7vM=', 'Content-Length': '0'} 2024-04-16T18:25:25.984 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000391cd6e4c44e0b13-00661ec295-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.030 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.031 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:26.031 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:26.031 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.031 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:25:26.031 DEBUG:boto:Data: 2024-04-16T18:25:26.031 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.031 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.031 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.032 DEBUG:boto:Params: {} 2024-04-16T18:25:26.032 DEBUG:boto:Token: None 2024-04-16T18:25:26.032 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj92 2024-04-16T18:25:26.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Obxh4bZXBDVwuRb/QLkFOAAnKJA= 2024-04-16T18:25:26.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Obxh4bZXBDVwuRb/QLkFOAAnKJA=', 'Content-Length': '0'} 2024-04-16T18:25:26.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cde9c4ac253a416b-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.035 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:26.036 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:26.036 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.036 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:25:26.036 DEBUG:boto:Data: 2024-04-16T18:25:26.036 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.036 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.036 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.036 DEBUG:boto:Params: {} 2024-04-16T18:25:26.036 DEBUG:boto:Token: None 2024-04-16T18:25:26.037 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj92 2024-04-16T18:25:26.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Obxh4bZXBDVwuRb/QLkFOAAnKJA= 2024-04-16T18:25:26.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Obxh4bZXBDVwuRb/QLkFOAAnKJA=', 'Content-Length': '0'} 2024-04-16T18:25:26.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cd52a2aaa32f320-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:25:26.040 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:26.040 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:26.040 DEBUG:boto:Method: GET 2024-04-16T18:25:26.040 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:25:26.040 DEBUG:boto:Data: 2024-04-16T18:25:26.041 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.041 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.041 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.041 DEBUG:boto:Params: {} 2024-04-16T18:25:26.041 DEBUG:boto:Token: None 2024-04-16T18:25:26.041 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj92 2024-04-16T18:25:26.041 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4jKDTfqIUpXuVZLgVPKHP0OMJjM= 2024-04-16T18:25:26.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4jKDTfqIUpXuVZLgVPKHP0OMJjM=', 'Content-Length': '0'} 2024-04-16T18:25:26.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023bb061c883101af-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.086 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:25:26.086 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:25:26.087 DEBUG:boto:Method: GET 2024-04-16T18:25:26.087 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:25:26.087 DEBUG:boto:Data: 2024-04-16T18:25:26.087 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.087 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.087 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.087 DEBUG:boto:Params: {} 2024-04-16T18:25:26.087 DEBUG:boto:Token: None 2024-04-16T18:25:26.087 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj92 2024-04-16T18:25:26.087 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4jKDTfqIUpXuVZLgVPKHP0OMJjM= 2024-04-16T18:25:26.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4jKDTfqIUpXuVZLgVPKHP0OMJjM=', 'Content-Length': '0'} 2024-04-16T18:25:26.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022e9a21ca57e18d8-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.091 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.091 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.091 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:26.091 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:26.091 DEBUG:boto:Method: GET 2024-04-16T18:25:26.091 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:25:26.091 DEBUG:boto:Data: 2024-04-16T18:25:26.091 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.091 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.091 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.091 DEBUG:boto:Params: {} 2024-04-16T18:25:26.092 DEBUG:boto:Token: None 2024-04-16T18:25:26.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI= 2024-04-16T18:25:26.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI=', 'Content-Length': '0'} 2024-04-16T18:25:26.096 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc19a6937fab75dc-00661ec296-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.138 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.139 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.139 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.139 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:26.139 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:25:26.139 DEBUG:boto:Method: GET 2024-04-16T18:25:26.139 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:25:26.139 DEBUG:boto:Data: 2024-04-16T18:25:26.139 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.139 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.139 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.140 DEBUG:boto:Params: {} 2024-04-16T18:25:26.140 DEBUG:boto:Token: None 2024-04-16T18:25:26.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI= 2024-04-16T18:25:26.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI=', 'Content-Length': '0'} 2024-04-16T18:25:26.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096b55279d9a0ab65-00661ec296-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.144 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.145 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:25:26.145 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.145 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.145 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.145 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.145 DEBUG:boto:Method: GET 2024-04-16T18:25:26.145 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.145 DEBUG:boto:Data: 2024-04-16T18:25:26.146 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.146 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.146 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.146 DEBUG:boto:Params: {} 2024-04-16T18:25:26.146 DEBUG:boto:Token: None 2024-04-16T18:25:26.146 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.146 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg= 2024-04-16T18:25:26.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg=', 'Content-Length': '0'} 2024-04-16T18:25:26.149 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000492ac5e909a4c1ff-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.190 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.190 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.191 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.191 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.191 DEBUG:boto:Method: GET 2024-04-16T18:25:26.191 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.191 DEBUG:boto:Data: 2024-04-16T18:25:26.191 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.191 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.191 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.191 DEBUG:boto:Params: {} 2024-04-16T18:25:26.191 DEBUG:boto:Token: None 2024-04-16T18:25:26.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg= 2024-04-16T18:25:26.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg=', 'Content-Length': '0'} 2024-04-16T18:25:26.194 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000491fcb97e860f38a-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.195 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.195 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.195 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.195 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.195 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.195 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.195 DEBUG:boto:Data: 2024-04-16T18:25:26.195 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.195 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.195 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.196 DEBUG:boto:Params: {} 2024-04-16T18:25:26.196 DEBUG:boto:Token: None 2024-04-16T18:25:26.196 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2FjbV1+lv6g/SdhT1Qg6reRxKEU= 2024-04-16T18:25:26.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2FjbV1+lv6g/SdhT1Qg6reRxKEU=', 'Content-Length': '0'} 2024-04-16T18:25:26.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da49643d2785e2ac-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.199 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.199 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.199 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.199 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.199 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.200 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.200 DEBUG:boto:Data: 2024-04-16T18:25:26.200 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.200 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.200 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.200 DEBUG:boto:Params: {} 2024-04-16T18:25:26.200 DEBUG:boto:Token: None 2024-04-16T18:25:26.200 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2FjbV1+lv6g/SdhT1Qg6reRxKEU= 2024-04-16T18:25:26.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2FjbV1+lv6g/SdhT1Qg6reRxKEU=', 'Content-Length': '0'} 2024-04-16T18:25:26.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005446ebc7a7289b35-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:25:26.203 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.203 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.204 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.204 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.204 DEBUG:boto:Method: GET 2024-04-16T18:25:26.204 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.204 DEBUG:boto:Data: 2024-04-16T18:25:26.204 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.204 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.204 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.204 DEBUG:boto:Params: {} 2024-04-16T18:25:26.204 DEBUG:boto:Token: None 2024-04-16T18:25:26.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.205 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg= 2024-04-16T18:25:26.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg=', 'Content-Length': '0'} 2024-04-16T18:25:26.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e0e15a845289138-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.250 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.250 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.251 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.251 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.251 DEBUG:boto:Method: GET 2024-04-16T18:25:26.251 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.251 DEBUG:boto:Data: 2024-04-16T18:25:26.251 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.251 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.251 DEBUG:boto:Params: {} 2024-04-16T18:25:26.251 DEBUG:boto:Token: None 2024-04-16T18:25:26.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:25:26.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg= 2024-04-16T18:25:26.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fdyca8h/vn5rnTp3PxMuTjgYiXg=', 'Content-Length': '0'} 2024-04-16T18:25:26.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052baec3a10eb5238-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.255 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.255 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.255 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.255 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.256 DEBUG:boto:Method: GET 2024-04-16T18:25:26.256 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:25:26.256 DEBUG:boto:Data: 2024-04-16T18:25:26.256 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.256 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.256 DEBUG:boto:Params: {} 2024-04-16T18:25:26.256 DEBUG:boto:Token: None 2024-04-16T18:25:26.256 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI= 2024-04-16T18:25:26.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI=', 'Content-Length': '0'} 2024-04-16T18:25:26.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0483136460f8d34-00661ec296-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.260 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.260 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.261 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.261 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.261 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.261 DEBUG:boto:Method: GET 2024-04-16T18:25:26.261 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:25:26.261 DEBUG:boto:Data: 2024-04-16T18:25:26.261 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.261 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.261 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.261 DEBUG:boto:Params: {} 2024-04-16T18:25:26.261 DEBUG:boto:Token: None 2024-04-16T18:25:26.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI= 2024-04-16T18:25:26.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI=', 'Content-Length': '0'} 2024-04-16T18:25:26.267 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4cf7e94d1c34aaa-00661ec296-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.310 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.311 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.311 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.311 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.311 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:25:26.311 DEBUG:boto:Data: 2024-04-16T18:25:26.311 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.311 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.311 DEBUG:boto:Params: {} 2024-04-16T18:25:26.311 DEBUG:boto:Token: None 2024-04-16T18:25:26.312 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95 2024-04-16T18:25:26.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oPOfCaFjJ0xCEGBLp0PeTqx2jE0= 2024-04-16T18:25:26.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oPOfCaFjJ0xCEGBLp0PeTqx2jE0=', 'Content-Length': '0'} 2024-04-16T18:25:26.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d100f03b65374a92-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.315 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.315 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.315 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.315 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:25:26.315 DEBUG:boto:Data: 2024-04-16T18:25:26.315 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.315 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.315 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.316 DEBUG:boto:Params: {} 2024-04-16T18:25:26.316 DEBUG:boto:Token: None 2024-04-16T18:25:26.316 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95 2024-04-16T18:25:26.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oPOfCaFjJ0xCEGBLp0PeTqx2jE0= 2024-04-16T18:25:26.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oPOfCaFjJ0xCEGBLp0PeTqx2jE0=', 'Content-Length': '0'} 2024-04-16T18:25:26.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006dbf88013bf78c1a-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.319 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:25:26.319 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.319 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.319 DEBUG:boto:Method: GET 2024-04-16T18:25:26.319 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:25:26.320 DEBUG:boto:Data: 2024-04-16T18:25:26.320 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.320 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.320 DEBUG:boto:Params: {} 2024-04-16T18:25:26.320 DEBUG:boto:Token: None 2024-04-16T18:25:26.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95 2024-04-16T18:25:26.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PLwLD7hGTTkZA+VwUpr6tnqur6c= 2024-04-16T18:25:26.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PLwLD7hGTTkZA+VwUpr6tnqur6c=', 'Content-Length': '0'} 2024-04-16T18:25:26.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000219e5d14c1690b34-00661ec296-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.366 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:25:26.366 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:25:26.366 DEBUG:boto:Method: GET 2024-04-16T18:25:26.367 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:25:26.367 DEBUG:boto:Data: 2024-04-16T18:25:26.367 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.367 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.367 DEBUG:boto:Params: {} 2024-04-16T18:25:26.367 DEBUG:boto:Token: None 2024-04-16T18:25:26.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj95 2024-04-16T18:25:26.367 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PLwLD7hGTTkZA+VwUpr6tnqur6c= 2024-04-16T18:25:26.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PLwLD7hGTTkZA+VwUpr6tnqur6c=', 'Content-Length': '0'} 2024-04-16T18:25:26.370 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000031efb37c1824964-00661ec296-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.370 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.371 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.371 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.371 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.371 DEBUG:boto:Method: GET 2024-04-16T18:25:26.371 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:25:26.371 DEBUG:boto:Data: 2024-04-16T18:25:26.371 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.371 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:25:26.371 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.371 DEBUG:boto:Params: {} 2024-04-16T18:25:26.372 DEBUG:boto:Token: None 2024-04-16T18:25:26.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI= 2024-04-16T18:25:26.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI=', 'Content-Length': '0'} 2024-04-16T18:25:26.375 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000233f0ad272c33e26-00661ec296-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.418 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.419 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.419 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.419 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.419 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:25:26.419 DEBUG:boto:Method: GET 2024-04-16T18:25:26.419 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:25:26.419 DEBUG:boto:Data: 2024-04-16T18:25:26.419 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.419 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.420 DEBUG:boto:Params: {} 2024-04-16T18:25:26.420 DEBUG:boto:Token: None 2024-04-16T18:25:26.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI= 2024-04-16T18:25:26.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sg87RrOM0uZRTfH+Xu/MzS48GdI=', 'Content-Length': '0'} 2024-04-16T18:25:26.425 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e61be702cb5b35e-00661ec296-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.425 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:25:26.425 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:25:26.426 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '29', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:25:26.426 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '29', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:25:26.426 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --num-shards 29 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:25:26.472 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:25:26.473 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:25:26.573 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:25:26.573 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-52 2024-04-16T18:25:26.592 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 20 2024-04-16T18:25:26.748 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.748 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.748 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.748 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:26.748 DEBUG:boto:Data: 2024-04-16T18:25:26.748 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.749 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.749 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.749 DEBUG:boto:Params: {} 2024-04-16T18:25:26.749 DEBUG:boto:Token: None 2024-04-16T18:25:26.749 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.749 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0= 2024-04-16T18:25:26.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0=', 'Content-Length': '0'} 2024-04-16T18:25:26.753 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005832ee248e96e51d-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.754 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:25:26.754 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:25:26.754 DEBUG:boto:Method: PUT 2024-04-16T18:25:26.754 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:25:26.754 DEBUG:boto:Data: 2024-04-16T18:25:26.754 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:25:26.754 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.754 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.754 DEBUG:boto:Params: {} 2024-04-16T18:25:26.754 DEBUG:boto:Token: None 2024-04-16T18:25:26.755 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj29 2024-04-16T18:25:26.755 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BYQo3YhDaU0eVHKYq/aeLQWR8rY= 2024-04-16T18:25:26.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BYQo3YhDaU0eVHKYq/aeLQWR8rY='} 2024-04-16T18:25:26.766 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f63493ad12c805af-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.766 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.766 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.766 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.767 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:26.767 DEBUG:boto:Data: 2024-04-16T18:25:26.767 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.767 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.767 DEBUG:boto:Params: {} 2024-04-16T18:25:26.767 DEBUG:boto:Token: None 2024-04-16T18:25:26.767 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.767 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0= 2024-04-16T18:25:26.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0=', 'Content-Length': '0'} 2024-04-16T18:25:26.771 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '63'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aee793ee8fa68f09-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.772 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:25:26.772 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:25:26.772 DEBUG:boto:Method: PUT 2024-04-16T18:25:26.772 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:25:26.772 DEBUG:boto:Data: 2024-04-16T18:25:26.772 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:25:26.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.772 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.772 DEBUG:boto:Params: {} 2024-04-16T18:25:26.773 DEBUG:boto:Token: None 2024-04-16T18:25:26.773 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj58 2024-04-16T18:25:26.773 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3pB/surDrgq8W3erCGzAhfgMcIs= 2024-04-16T18:25:26.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3pB/surDrgq8W3erCGzAhfgMcIs='} 2024-04-16T18:25:26.814 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d854b5cee188e939-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.814 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.815 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.815 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.815 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:26.815 DEBUG:boto:Data: 2024-04-16T18:25:26.815 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.815 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.815 DEBUG:boto:Params: {} 2024-04-16T18:25:26.815 DEBUG:boto:Token: None 2024-04-16T18:25:26.815 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0= 2024-04-16T18:25:26.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0=', 'Content-Length': '0'} 2024-04-16T18:25:26.821 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000a2b02a272c8368d-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.822 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:25:26.822 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:25:26.822 DEBUG:boto:Method: PUT 2024-04-16T18:25:26.822 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:25:26.822 DEBUG:boto:Data: 2024-04-16T18:25:26.822 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:25:26.822 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.822 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.823 DEBUG:boto:Params: {} 2024-04-16T18:25:26.823 DEBUG:boto:Token: None 2024-04-16T18:25:26.823 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj87 2024-04-16T18:25:26.823 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HJUSiZddXYhtQmJh3tq2Zy/TkiA= 2024-04-16T18:25:26.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HJUSiZddXYhtQmJh3tq2Zy/TkiA='} 2024-04-16T18:25:26.835 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f99c851f3f616bc7-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.835 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.836 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.836 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.836 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:26.836 DEBUG:boto:Data: 2024-04-16T18:25:26.836 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.836 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.836 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.836 DEBUG:boto:Params: {} 2024-04-16T18:25:26.836 DEBUG:boto:Token: None 2024-04-16T18:25:26.836 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0= 2024-04-16T18:25:26.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0=', 'Content-Length': '0'} 2024-04-16T18:25:26.842 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009a0302350f022bd6-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.842 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:25:26.842 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:25:26.842 DEBUG:boto:Method: PUT 2024-04-16T18:25:26.842 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:25:26.842 DEBUG:boto:Data: 2024-04-16T18:25:26.843 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:25:26.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.843 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.843 DEBUG:boto:Params: {} 2024-04-16T18:25:26.843 DEBUG:boto:Token: None 2024-04-16T18:25:26.843 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj116 2024-04-16T18:25:26.843 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q9L2xCOlDmXAOx/6slZtYv3C5X4= 2024-04-16T18:25:26.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q9L2xCOlDmXAOx/6slZtYv3C5X4='} 2024-04-16T18:25:26.886 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002897e66b76d24a82-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.887 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:25:26.887 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:25:26.887 DEBUG:boto:Method: HEAD 2024-04-16T18:25:26.887 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:25:26.887 DEBUG:boto:Data: 2024-04-16T18:25:26.887 DEBUG:boto:Headers: {} 2024-04-16T18:25:26.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.887 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.887 DEBUG:boto:Params: {} 2024-04-16T18:25:26.887 DEBUG:boto:Token: None 2024-04-16T18:25:26.888 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/ 2024-04-16T18:25:26.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0= 2024-04-16T18:25:26.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DN1Tomz+wWRm8k3I3Hk2Sf4bmc0=', 'Content-Length': '0'} 2024-04-16T18:25:26.893 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c4cd52ec74cc5838-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.893 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:25:26.893 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:25:26.894 DEBUG:boto:Method: PUT 2024-04-16T18:25:26.894 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:25:26.894 DEBUG:boto:Data: 2024-04-16T18:25:26.894 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:25:26.894 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:25:26.894 DEBUG:boto:Port: 8000 2024-04-16T18:25:26.894 DEBUG:boto:Params: {} 2024-04-16T18:25:26.894 DEBUG:boto:Token: None 2024-04-16T18:25:26.894 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:25:26 GMT /lybtug-52/obj145 2024-04-16T18:25:26.894 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xgnNXuXX61exlqctoiKhVTLKuvM= 2024-04-16T18:25:26.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:25:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xgnNXuXX61exlqctoiKhVTLKuvM='} 2024-04-16T18:25:26.906 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004103077cd7b628c4-00661ec296-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:25:26.907 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:25:26.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:25:26.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:25:26.907 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:25:26.957 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:25:26.958 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:25:27.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:27.040+0000 7f287fa35980 1 waiting to reach latest gen 4, on 3.. 2024-04-16T18:25:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:32.043+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:32.045 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:25:32.045 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:25:37.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:37.047+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:37.048 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:25:37.048 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:25:42.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:42.051+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:42.051 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:25:42.051 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:25:47.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:47.055+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:47.055 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:25:47.055 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:25:52.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:52.059+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:52.058 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:25:52.058 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:25:57.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:25:57.063+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:25:57.062 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:25:57.062 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:02.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:02.062+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:02.065 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:02.065 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:07.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:07.066+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:07.068 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:07.068 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:12.070+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:12.072 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:12.072 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:17.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:17.074+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:17.075 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:17.075 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:22.078+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:22.078 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:22.078 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:27.082+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:27.081 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:27.082 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:32.082+0000 7f287fa35980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:32.085 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000001.62.6, , , , , , , , , , , , 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:32.085 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:37.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:37.085+0000 7f287fa35980 1 bucket sync caught up with source: 2024-04-16T18:26:37.089 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:37.089 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [00000000002.89.6, 00000000001.61.6, , , , , 00000000001.62.6, , , , , , , , , , , 00000000002.111.6, 00000000001.116.6, , , , , , , , , , ] 2024-04-16T18:26:37.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:37.085+0000 7f287fa35980 0 bucket checkpoint complete 2024-04-16T18:26:37.133 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:26:37.133 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:26:37.133 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:26:37.134 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:26:37.184 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:26:37.184 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:26:37.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:26:37.269+0000 7f55477cd980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:37.266 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000002.1221.6, 00000000004.529.6, 00000000001.961.6, , 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:26:37.266 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000003.1229.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:26:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:26:42.273+0000 7f55477cd980 1 bucket sync caught up with source: 2024-04-16T18:26:42.269 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000003.1229.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:26:42.269 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000002.632.6, 00000000002.559.6, 00000000001.470.6, 00000000003.1229.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000002.466.6, 00000000005.327.6, 00000000004.416.6] 2024-04-16T18:26:42.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:26:42.273+0000 7f55477cd980 0 bucket checkpoint complete 2024-04-16T18:26:42.313 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:26:42.314 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.314 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.314 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.314 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:42.314 DEBUG:boto:Data: 2024-04-16T18:26:42.314 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.314 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.314 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.315 DEBUG:boto:Params: {} 2024-04-16T18:26:42.315 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:26:42.315 DEBUG:boto:Token: None 2024-04-16T18:26:42.316 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+N/k//wU4wIfr6Ffun71ymQ5MHE= 2024-04-16T18:26:42.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+N/k//wU4wIfr6Ffun71ymQ5MHE=', 'Content-Length': '0'} 2024-04-16T18:26:42.323 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d9e8463959172920-00661ec2e2-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.323 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.323 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.323 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.323 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:42.324 DEBUG:boto:Data: 2024-04-16T18:26:42.324 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.324 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.324 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.324 DEBUG:boto:Params: {} 2024-04-16T18:26:42.324 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:26:42.324 DEBUG:boto:Token: None 2024-04-16T18:26:42.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.325 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+N/k//wU4wIfr6Ffun71ymQ5MHE= 2024-04-16T18:26:42.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+N/k//wU4wIfr6Ffun71ymQ5MHE=', 'Content-Length': '0'} 2024-04-16T18:26:42.330 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ad9387a9ab13b2ba-00661ec2e2-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.331 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:26:42.331 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.331 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.331 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.331 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.331 DEBUG:boto:Method: GET 2024-04-16T18:26:42.331 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.331 DEBUG:boto:Data: 2024-04-16T18:26:42.332 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.332 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.332 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.332 DEBUG:boto:Params: {} 2024-04-16T18:26:42.332 DEBUG:boto:Token: None 2024-04-16T18:26:42.332 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/?versions 2024-04-16T18:26:42.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro= 2024-04-16T18:26:42.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro=', 'Content-Length': '0'} 2024-04-16T18:26:42.336 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000473bc39dac6173b0-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.337 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:26:42.339 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:26:42.340 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:26:42.341 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:26:42.341 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.341 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.341 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.341 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.342 DEBUG:boto:Method: GET 2024-04-16T18:26:42.342 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.342 DEBUG:boto:Data: 2024-04-16T18:26:42.342 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.342 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.342 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.342 DEBUG:boto:Params: {} 2024-04-16T18:26:42.343 DEBUG:boto:Token: None 2024-04-16T18:26:42.343 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/?versions 2024-04-16T18:26:42.343 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro= 2024-04-16T18:26:42.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro=', 'Content-Length': '0'} 2024-04-16T18:26:42.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002068384132ea2a53-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.348 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:26:42.349 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:26:42.350 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:26:42.351 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:26:42.351 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.351 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.351 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.351 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.352 DEBUG:boto:Method: GET 2024-04-16T18:26:42.352 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.352 DEBUG:boto:Data: 2024-04-16T18:26:42.352 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.352 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.352 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.352 DEBUG:boto:Params: {} 2024-04-16T18:26:42.352 DEBUG:boto:Token: None 2024-04-16T18:26:42.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/?versions 2024-04-16T18:26:42.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro= 2024-04-16T18:26:42.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro=', 'Content-Length': '0'} 2024-04-16T18:26:42.356 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000950f0217446b05bb-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.357 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.358 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.359 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.359 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.359 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.359 DEBUG:boto:Method: GET 2024-04-16T18:26:42.359 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:26:42.359 DEBUG:boto:Data: 2024-04-16T18:26:42.359 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.359 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.359 DEBUG:boto:Params: {} 2024-04-16T18:26:42.360 DEBUG:boto:Token: None 2024-04-16T18:26:42.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/?versions 2024-04-16T18:26:42.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro= 2024-04-16T18:26:42.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cFYvVckzwTCmNR4itrWaGxGBAro=', 'Content-Length': '0'} 2024-04-16T18:26:42.366 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013d7802702df5646-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.366 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:26:42.367 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.368 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.368 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.368 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.368 DEBUG:boto:Method: GET 2024-04-16T18:26:42.368 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.368 DEBUG:boto:Data: 2024-04-16T18:26:42.368 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.368 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.368 DEBUG:boto:Params: {} 2024-04-16T18:26:42.369 DEBUG:boto:Token: None 2024-04-16T18:26:42.369 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4= 2024-04-16T18:26:42.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4=', 'Content-Length': '0'} 2024-04-16T18:26:42.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d57ae2f997bac18-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.414 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.414 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.415 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.415 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.415 DEBUG:boto:Method: GET 2024-04-16T18:26:42.415 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.415 DEBUG:boto:Data: 2024-04-16T18:26:42.415 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.415 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.415 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.415 DEBUG:boto:Params: {} 2024-04-16T18:26:42.415 DEBUG:boto:Token: None 2024-04-16T18:26:42.416 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4= 2024-04-16T18:26:42.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4=', 'Content-Length': '0'} 2024-04-16T18:26:42.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cdcb078461a122e-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.419 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.419 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.419 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.419 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.419 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.420 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.420 DEBUG:boto:Data: 2024-04-16T18:26:42.420 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.420 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.420 DEBUG:boto:Params: {} 2024-04-16T18:26:42.420 DEBUG:boto:Token: None 2024-04-16T18:26:42.420 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+Sv52IQhyv5qHU0oDS4btKfhPTs= 2024-04-16T18:26:42.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+Sv52IQhyv5qHU0oDS4btKfhPTs=', 'Content-Length': '0'} 2024-04-16T18:26:42.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f98f27b464b9bd9-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.424 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.424 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.424 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.424 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.424 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.424 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.424 DEBUG:boto:Data: 2024-04-16T18:26:42.424 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.425 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.425 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.425 DEBUG:boto:Params: {} 2024-04-16T18:26:42.425 DEBUG:boto:Token: None 2024-04-16T18:26:42.425 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.425 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+Sv52IQhyv5qHU0oDS4btKfhPTs= 2024-04-16T18:26:42.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+Sv52IQhyv5qHU0oDS4btKfhPTs=', 'Content-Length': '0'} 2024-04-16T18:26:42.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4fe187af7fd5f1b-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:26:42.429 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.429 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.429 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.429 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.429 DEBUG:boto:Method: GET 2024-04-16T18:26:42.429 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.429 DEBUG:boto:Data: 2024-04-16T18:26:42.429 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.430 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.430 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.430 DEBUG:boto:Params: {} 2024-04-16T18:26:42.430 DEBUG:boto:Token: None 2024-04-16T18:26:42.430 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.430 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4= 2024-04-16T18:26:42.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4=', 'Content-Length': '0'} 2024-04-16T18:26:42.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b71d4eb4edd8361-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.474 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.475 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.475 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.475 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.475 DEBUG:boto:Method: GET 2024-04-16T18:26:42.475 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.475 DEBUG:boto:Data: 2024-04-16T18:26:42.475 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.475 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.475 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.475 DEBUG:boto:Params: {} 2024-04-16T18:26:42.475 DEBUG:boto:Token: None 2024-04-16T18:26:42.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:26:42.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4= 2024-04-16T18:26:42.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:59lt9eOxQgi3dK0aNjvhlTCgcy4=', 'Content-Length': '0'} 2024-04-16T18:26:42.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b29c8035b6e4b15-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.479 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.479 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.479 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.479 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.480 DEBUG:boto:Method: GET 2024-04-16T18:26:42.480 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:26:42.480 DEBUG:boto:Data: 2024-04-16T18:26:42.480 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.480 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.480 DEBUG:boto:Params: {} 2024-04-16T18:26:42.480 DEBUG:boto:Token: None 2024-04-16T18:26:42.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8417284a088a82b-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.526 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.527 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.527 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.527 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.527 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.527 DEBUG:boto:Method: GET 2024-04-16T18:26:42.527 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:26:42.527 DEBUG:boto:Data: 2024-04-16T18:26:42.528 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.528 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.528 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.528 DEBUG:boto:Params: {} 2024-04-16T18:26:42.528 DEBUG:boto:Token: None 2024-04-16T18:26:42.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d42a46fbf68ee2f5-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.535 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.535 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.535 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.535 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.535 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:26:42.535 DEBUG:boto:Data: 2024-04-16T18:26:42.535 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.536 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.536 DEBUG:boto:Params: {} 2024-04-16T18:26:42.536 DEBUG:boto:Token: None 2024-04-16T18:26:42.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11 2024-04-16T18:26:42.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7r0Z70/oDYqZ4P0rwzgQ6lORO7c= 2024-04-16T18:26:42.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7r0Z70/oDYqZ4P0rwzgQ6lORO7c=', 'Content-Length': '0'} 2024-04-16T18:26:42.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fdaf24e5f48ea93-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.540 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.540 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.540 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.540 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:26:42.540 DEBUG:boto:Data: 2024-04-16T18:26:42.540 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.540 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.540 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.540 DEBUG:boto:Params: {} 2024-04-16T18:26:42.541 DEBUG:boto:Token: None 2024-04-16T18:26:42.541 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11 2024-04-16T18:26:42.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7r0Z70/oDYqZ4P0rwzgQ6lORO7c= 2024-04-16T18:26:42.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7r0Z70/oDYqZ4P0rwzgQ6lORO7c=', 'Content-Length': '0'} 2024-04-16T18:26:42.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000657df11c48cab670-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:26:42.544 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.544 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.544 DEBUG:boto:Method: GET 2024-04-16T18:26:42.545 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:26:42.545 DEBUG:boto:Data: 2024-04-16T18:26:42.545 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.545 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.545 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.545 DEBUG:boto:Params: {} 2024-04-16T18:26:42.545 DEBUG:boto:Token: None 2024-04-16T18:26:42.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11 2024-04-16T18:26:42.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NBm3IpjQ+lwy/z3jp9oeDTz04EA= 2024-04-16T18:26:42.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NBm3IpjQ+lwy/z3jp9oeDTz04EA=', 'Content-Length': '0'} 2024-04-16T18:26:42.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ce028f9adc435f2-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.549 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:26:42.549 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:26:42.549 DEBUG:boto:Method: GET 2024-04-16T18:26:42.549 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:26:42.549 DEBUG:boto:Data: 2024-04-16T18:26:42.549 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.549 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.549 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.549 DEBUG:boto:Params: {} 2024-04-16T18:26:42.550 DEBUG:boto:Token: None 2024-04-16T18:26:42.550 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj11 2024-04-16T18:26:42.550 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NBm3IpjQ+lwy/z3jp9oeDTz04EA= 2024-04-16T18:26:42.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NBm3IpjQ+lwy/z3jp9oeDTz04EA=', 'Content-Length': '0'} 2024-04-16T18:26:42.553 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4b9eb54612d0a7e-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.553 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.553 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.553 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.553 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.554 DEBUG:boto:Method: GET 2024-04-16T18:26:42.554 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:26:42.554 DEBUG:boto:Data: 2024-04-16T18:26:42.554 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.554 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.554 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.554 DEBUG:boto:Params: {} 2024-04-16T18:26:42.554 DEBUG:boto:Token: None 2024-04-16T18:26:42.554 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.554 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e62b657cbda2b71-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.602 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.603 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.603 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.603 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.603 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:26:42.603 DEBUG:boto:Method: GET 2024-04-16T18:26:42.603 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:26:42.603 DEBUG:boto:Data: 2024-04-16T18:26:42.603 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.603 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.604 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.604 DEBUG:boto:Params: {} 2024-04-16T18:26:42.604 DEBUG:boto:Token: None 2024-04-16T18:26:42.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.610 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001930c23a3b7c6fe6-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.611 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.611 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:26:42.611 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.611 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.611 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.611 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.612 DEBUG:boto:Method: GET 2024-04-16T18:26:42.612 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.612 DEBUG:boto:Data: 2024-04-16T18:26:42.612 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.612 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.612 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.612 DEBUG:boto:Params: {} 2024-04-16T18:26:42.612 DEBUG:boto:Token: None 2024-04-16T18:26:42.612 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM= 2024-04-16T18:26:42.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM=', 'Content-Length': '0'} 2024-04-16T18:26:42.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000076275668489ac5a-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.616 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.616 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.616 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.616 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.616 DEBUG:boto:Method: GET 2024-04-16T18:26:42.617 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.617 DEBUG:boto:Data: 2024-04-16T18:26:42.617 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.617 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.617 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.617 DEBUG:boto:Params: {} 2024-04-16T18:26:42.617 DEBUG:boto:Token: None 2024-04-16T18:26:42.617 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.618 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM= 2024-04-16T18:26:42.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM=', 'Content-Length': '0'} 2024-04-16T18:26:42.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcbc61783458493d-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.662 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.662 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.662 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.663 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.663 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.663 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.663 DEBUG:boto:Data: 2024-04-16T18:26:42.663 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.663 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.663 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.663 DEBUG:boto:Params: {} 2024-04-16T18:26:42.663 DEBUG:boto:Token: None 2024-04-16T18:26:42.663 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZvuU2YwLciVSJztc7q4hEmjFPvo= 2024-04-16T18:26:42.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZvuU2YwLciVSJztc7q4hEmjFPvo=', 'Content-Length': '0'} 2024-04-16T18:26:42.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a45185cab70a01c5-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.667 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.667 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.667 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.668 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.668 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.668 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.668 DEBUG:boto:Data: 2024-04-16T18:26:42.668 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.668 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.668 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.668 DEBUG:boto:Params: {} 2024-04-16T18:26:42.668 DEBUG:boto:Token: None 2024-04-16T18:26:42.668 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZvuU2YwLciVSJztc7q4hEmjFPvo= 2024-04-16T18:26:42.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZvuU2YwLciVSJztc7q4hEmjFPvo=', 'Content-Length': '0'} 2024-04-16T18:26:42.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ac7b97a31a60c80-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.672 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:26:42.672 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.672 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.672 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.672 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.672 DEBUG:boto:Method: GET 2024-04-16T18:26:42.672 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.672 DEBUG:boto:Data: 2024-04-16T18:26:42.672 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.672 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.672 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.673 DEBUG:boto:Params: {} 2024-04-16T18:26:42.673 DEBUG:boto:Token: None 2024-04-16T18:26:42.673 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.673 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM= 2024-04-16T18:26:42.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM=', 'Content-Length': '0'} 2024-04-16T18:26:42.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029947d7d0123eb14-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.677 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.677 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.677 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.677 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.677 DEBUG:boto:Method: GET 2024-04-16T18:26:42.677 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.677 DEBUG:boto:Data: 2024-04-16T18:26:42.678 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.678 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.678 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.678 DEBUG:boto:Params: {} 2024-04-16T18:26:42.678 DEBUG:boto:Token: None 2024-04-16T18:26:42.678 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:26:42.678 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM= 2024-04-16T18:26:42.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0FsuhzW100DJIRpySI3c2FJFbtM=', 'Content-Length': '0'} 2024-04-16T18:26:42.682 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bad642b626157917-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.682 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.682 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.682 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.682 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.683 DEBUG:boto:Method: GET 2024-04-16T18:26:42.683 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:26:42.683 DEBUG:boto:Data: 2024-04-16T18:26:42.683 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.683 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.683 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.683 DEBUG:boto:Params: {} 2024-04-16T18:26:42.683 DEBUG:boto:Token: None 2024-04-16T18:26:42.683 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.683 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.687 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f33360dd976f241f-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.730 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.731 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.731 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.731 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.731 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.731 DEBUG:boto:Method: GET 2024-04-16T18:26:42.731 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:26:42.731 DEBUG:boto:Data: 2024-04-16T18:26:42.731 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.731 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.731 DEBUG:boto:Params: {} 2024-04-16T18:26:42.732 DEBUG:boto:Token: None 2024-04-16T18:26:42.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acafa5194c44428b-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.738 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.739 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.739 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.739 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.739 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:26:42.739 DEBUG:boto:Data: 2024-04-16T18:26:42.739 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.740 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.740 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.740 DEBUG:boto:Params: {} 2024-04-16T18:26:42.740 DEBUG:boto:Token: None 2024-04-16T18:26:42.740 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115 2024-04-16T18:26:42.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:61fnBSJcyBHYHb7Y/V/faZ4AZg4= 2024-04-16T18:26:42.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:61fnBSJcyBHYHb7Y/V/faZ4AZg4=', 'Content-Length': '0'} 2024-04-16T18:26:42.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b299aa45428f8cfa-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.743 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.744 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.744 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.744 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:26:42.744 DEBUG:boto:Data: 2024-04-16T18:26:42.744 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.744 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.744 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.744 DEBUG:boto:Params: {} 2024-04-16T18:26:42.744 DEBUG:boto:Token: None 2024-04-16T18:26:42.744 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115 2024-04-16T18:26:42.745 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:61fnBSJcyBHYHb7Y/V/faZ4AZg4= 2024-04-16T18:26:42.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:61fnBSJcyBHYHb7Y/V/faZ4AZg4=', 'Content-Length': '0'} 2024-04-16T18:26:42.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001998ceed180f7369-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.748 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:26:42.748 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.748 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.748 DEBUG:boto:Method: GET 2024-04-16T18:26:42.748 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:26:42.748 DEBUG:boto:Data: 2024-04-16T18:26:42.748 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.748 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.749 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.749 DEBUG:boto:Params: {} 2024-04-16T18:26:42.749 DEBUG:boto:Token: None 2024-04-16T18:26:42.749 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115 2024-04-16T18:26:42.749 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H948KP9ZXpP0FaR1YpH/nvNVo5Y= 2024-04-16T18:26:42.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H948KP9ZXpP0FaR1YpH/nvNVo5Y=', 'Content-Length': '0'} 2024-04-16T18:26:42.752 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2c15d5d3eaeec23-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.752 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:26:42.752 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:26:42.752 DEBUG:boto:Method: GET 2024-04-16T18:26:42.753 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:26:42.753 DEBUG:boto:Data: 2024-04-16T18:26:42.753 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.753 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.753 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.753 DEBUG:boto:Params: {} 2024-04-16T18:26:42.753 DEBUG:boto:Token: None 2024-04-16T18:26:42.753 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj115 2024-04-16T18:26:42.753 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H948KP9ZXpP0FaR1YpH/nvNVo5Y= 2024-04-16T18:26:42.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H948KP9ZXpP0FaR1YpH/nvNVo5Y=', 'Content-Length': '0'} 2024-04-16T18:26:42.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dff7c470872811e3-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.798 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.798 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.799 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.799 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.799 DEBUG:boto:Method: GET 2024-04-16T18:26:42.799 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:26:42.799 DEBUG:boto:Data: 2024-04-16T18:26:42.799 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.799 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.799 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.799 DEBUG:boto:Params: {} 2024-04-16T18:26:42.799 DEBUG:boto:Token: None 2024-04-16T18:26:42.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.803 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5d3015ee8b4b9be-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.804 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.804 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.804 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.804 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.804 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:26:42.804 DEBUG:boto:Method: GET 2024-04-16T18:26:42.804 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:26:42.804 DEBUG:boto:Data: 2024-04-16T18:26:42.805 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.805 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.805 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.805 DEBUG:boto:Params: {} 2024-04-16T18:26:42.805 DEBUG:boto:Token: None 2024-04-16T18:26:42.805 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ecd33c308cf7943-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.812 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.812 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:26:42.812 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.812 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.812 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.813 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.813 DEBUG:boto:Method: GET 2024-04-16T18:26:42.813 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.813 DEBUG:boto:Data: 2024-04-16T18:26:42.813 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.813 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.813 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.813 DEBUG:boto:Params: {} 2024-04-16T18:26:42.813 DEBUG:boto:Token: None 2024-04-16T18:26:42.813 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw= 2024-04-16T18:26:42.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw=', 'Content-Length': '0'} 2024-04-16T18:26:42.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037a417d50da19ef0-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.862 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.862 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.863 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.863 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.863 DEBUG:boto:Method: GET 2024-04-16T18:26:42.863 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.863 DEBUG:boto:Data: 2024-04-16T18:26:42.863 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.863 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.863 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.863 DEBUG:boto:Params: {} 2024-04-16T18:26:42.863 DEBUG:boto:Token: None 2024-04-16T18:26:42.864 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw= 2024-04-16T18:26:42.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw=', 'Content-Length': '0'} 2024-04-16T18:26:42.867 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb31cebaf9333670-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.867 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.867 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.867 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.867 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.867 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.867 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.867 DEBUG:boto:Data: 2024-04-16T18:26:42.868 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.868 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.868 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.868 DEBUG:boto:Params: {} 2024-04-16T18:26:42.868 DEBUG:boto:Token: None 2024-04-16T18:26:42.868 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5L6L+/u7SQYbfLdpghendntTZVM= 2024-04-16T18:26:42.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5L6L+/u7SQYbfLdpghendntTZVM=', 'Content-Length': '0'} 2024-04-16T18:26:42.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000638901bc617f62ae-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.871 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.871 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.871 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.871 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.871 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.872 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.872 DEBUG:boto:Data: 2024-04-16T18:26:42.872 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.872 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.872 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.872 DEBUG:boto:Params: {} 2024-04-16T18:26:42.872 DEBUG:boto:Token: None 2024-04-16T18:26:42.872 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5L6L+/u7SQYbfLdpghendntTZVM= 2024-04-16T18:26:42.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5L6L+/u7SQYbfLdpghendntTZVM=', 'Content-Length': '0'} 2024-04-16T18:26:42.875 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000115eea0b9733dd4a-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:26:42.876 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.876 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.876 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.876 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.876 DEBUG:boto:Method: GET 2024-04-16T18:26:42.876 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.876 DEBUG:boto:Data: 2024-04-16T18:26:42.876 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.876 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.876 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.876 DEBUG:boto:Params: {} 2024-04-16T18:26:42.876 DEBUG:boto:Token: None 2024-04-16T18:26:42.877 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.877 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw= 2024-04-16T18:26:42.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw=', 'Content-Length': '0'} 2024-04-16T18:26:42.880 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5c95a401566a4bf-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.880 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.880 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.880 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.880 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.880 DEBUG:boto:Method: GET 2024-04-16T18:26:42.881 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.881 DEBUG:boto:Data: 2024-04-16T18:26:42.881 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.881 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.881 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.881 DEBUG:boto:Params: {} 2024-04-16T18:26:42.881 DEBUG:boto:Token: None 2024-04-16T18:26:42.881 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:26:42.881 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw= 2024-04-16T18:26:42.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w78sONHnsaLFR/AAHzqxu1/F8Qw=', 'Content-Length': '0'} 2024-04-16T18:26:42.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ed988569ba80a92-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.926 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.926 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.927 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:42.927 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:42.927 DEBUG:boto:Method: GET 2024-04-16T18:26:42.927 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:26:42.927 DEBUG:boto:Data: 2024-04-16T18:26:42.927 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.927 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.927 DEBUG:boto:Params: {} 2024-04-16T18:26:42.927 DEBUG:boto:Token: None 2024-04-16T18:26:42.927 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.931 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001fe0b0dcb174993b-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.931 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.932 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:42.932 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:42.932 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:42.932 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:42.932 DEBUG:boto:Method: GET 2024-04-16T18:26:42.932 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:26:42.932 DEBUG:boto:Data: 2024-04-16T18:26:42.933 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.933 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.933 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.933 DEBUG:boto:Params: {} 2024-04-16T18:26:42.933 DEBUG:boto:Token: None 2024-04-16T18:26:42.933 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/ 2024-04-16T18:26:42.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po= 2024-04-16T18:26:42.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D9bnx22lZw5lbEztygf2zmIZ6Po=', 'Content-Length': '0'} 2024-04-16T18:26:42.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bddb31ab8bca2fa7-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.939 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:42.939 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.939 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.939 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.940 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:26:42.940 DEBUG:boto:Data: 2024-04-16T18:26:42.940 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.940 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.940 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.940 DEBUG:boto:Params: {} 2024-04-16T18:26:42.940 DEBUG:boto:Token: None 2024-04-16T18:26:42.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116 2024-04-16T18:26:42.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+cbz7FsO9+sbZsJ+HXbS2s9BTZ0= 2024-04-16T18:26:42.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+cbz7FsO9+sbZsJ+HXbS2s9BTZ0=', 'Content-Length': '0'} 2024-04-16T18:26:42.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dc78999d71846c5-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.943 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.943 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.943 DEBUG:boto:Method: HEAD 2024-04-16T18:26:42.944 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:26:42.944 DEBUG:boto:Data: 2024-04-16T18:26:42.944 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.944 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.944 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.944 DEBUG:boto:Params: {} 2024-04-16T18:26:42.944 DEBUG:boto:Token: None 2024-04-16T18:26:42.944 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116 2024-04-16T18:26:42.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+cbz7FsO9+sbZsJ+HXbS2s9BTZ0= 2024-04-16T18:26:42.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+cbz7FsO9+sbZsJ+HXbS2s9BTZ0=', 'Content-Length': '0'} 2024-04-16T18:26:42.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c89b996f9c8ac0a-00661ec2e2-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:26:42.947 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.947 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.948 DEBUG:boto:Method: GET 2024-04-16T18:26:42.948 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:26:42.948 DEBUG:boto:Data: 2024-04-16T18:26:42.948 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:42.948 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.948 DEBUG:boto:Params: {} 2024-04-16T18:26:42.948 DEBUG:boto:Token: None 2024-04-16T18:26:42.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116 2024-04-16T18:26:42.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SF+0YjhQa1km1KQKruYT6U3l0bA= 2024-04-16T18:26:42.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SF+0YjhQa1km1KQKruYT6U3l0bA=', 'Content-Length': '0'} 2024-04-16T18:26:42.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087d6d6b933fd135a-00661ec2e2-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:42.998 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:26:42.998 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:26:42.999 DEBUG:boto:Method: GET 2024-04-16T18:26:42.999 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:26:42.999 DEBUG:boto:Data: 2024-04-16T18:26:42.999 DEBUG:boto:Headers: {} 2024-04-16T18:26:42.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:42.999 DEBUG:boto:Port: 8000 2024-04-16T18:26:42.999 DEBUG:boto:Params: {} 2024-04-16T18:26:42.999 DEBUG:boto:Token: None 2024-04-16T18:26:42.999 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:42 GMT /lybtug-52/obj116 2024-04-16T18:26:43.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SF+0YjhQa1km1KQKruYT6U3l0bA= 2024-04-16T18:26:43.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SF+0YjhQa1km1KQKruYT6U3l0bA=', 'Content-Length': '0'} 2024-04-16T18:26:43.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000526c12d320e0c521-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.003 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.004 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.004 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:43.004 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:43.004 DEBUG:boto:Method: GET 2024-04-16T18:26:43.004 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:26:43.004 DEBUG:boto:Data: 2024-04-16T18:26:43.004 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.004 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.004 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.004 DEBUG:boto:Params: {} 2024-04-16T18:26:43.005 DEBUG:boto:Token: None 2024-04-16T18:26:43.005 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.005 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.009 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000377a510820d34083-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.009 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.009 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.009 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.009 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:43.010 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:26:43.010 DEBUG:boto:Method: GET 2024-04-16T18:26:43.010 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:26:43.010 DEBUG:boto:Data: 2024-04-16T18:26:43.010 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.010 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.010 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.010 DEBUG:boto:Params: {} 2024-04-16T18:26:43.010 DEBUG:boto:Token: None 2024-04-16T18:26:43.011 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.011 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.017 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001554cd6f8fa3c842-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.058 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.059 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:26:43.059 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.059 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.059 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.059 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.059 DEBUG:boto:Method: GET 2024-04-16T18:26:43.059 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.059 DEBUG:boto:Data: 2024-04-16T18:26:43.059 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.059 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.060 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.060 DEBUG:boto:Params: {} 2024-04-16T18:26:43.060 DEBUG:boto:Token: None 2024-04-16T18:26:43.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw= 2024-04-16T18:26:43.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw=', 'Content-Length': '0'} 2024-04-16T18:26:43.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc8ea999020a266b-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.063 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.064 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.064 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.064 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.064 DEBUG:boto:Method: GET 2024-04-16T18:26:43.064 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.064 DEBUG:boto:Data: 2024-04-16T18:26:43.064 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.064 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.064 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.064 DEBUG:boto:Params: {} 2024-04-16T18:26:43.064 DEBUG:boto:Token: None 2024-04-16T18:26:43.065 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.065 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw= 2024-04-16T18:26:43.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw=', 'Content-Length': '0'} 2024-04-16T18:26:43.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017e6feaf9d19c765-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.068 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.068 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.068 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.068 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.068 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.069 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.069 DEBUG:boto:Data: 2024-04-16T18:26:43.069 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.069 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.069 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.069 DEBUG:boto:Params: {} 2024-04-16T18:26:43.069 DEBUG:boto:Token: None 2024-04-16T18:26:43.069 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.069 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Teq3tNm3eXLxTgtGzGt1bs0iyHs= 2024-04-16T18:26:43.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Teq3tNm3eXLxTgtGzGt1bs0iyHs=', 'Content-Length': '0'} 2024-04-16T18:26:43.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6b3b3c7c4f51f00-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.072 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.072 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.073 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.073 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.073 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.073 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.073 DEBUG:boto:Data: 2024-04-16T18:26:43.073 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.073 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.073 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.073 DEBUG:boto:Params: {} 2024-04-16T18:26:43.073 DEBUG:boto:Token: None 2024-04-16T18:26:43.074 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.074 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Teq3tNm3eXLxTgtGzGt1bs0iyHs= 2024-04-16T18:26:43.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Teq3tNm3eXLxTgtGzGt1bs0iyHs=', 'Content-Length': '0'} 2024-04-16T18:26:43.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2298cfe432bc101-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.077 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:26:43.077 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.077 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.077 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.077 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.078 DEBUG:boto:Method: GET 2024-04-16T18:26:43.078 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.078 DEBUG:boto:Data: 2024-04-16T18:26:43.078 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.078 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.078 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.078 DEBUG:boto:Params: {} 2024-04-16T18:26:43.078 DEBUG:boto:Token: None 2024-04-16T18:26:43.078 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.078 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw= 2024-04-16T18:26:43.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw=', 'Content-Length': '0'} 2024-04-16T18:26:43.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a879297b3342a88d-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.122 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.122 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.122 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.123 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.123 DEBUG:boto:Method: GET 2024-04-16T18:26:43.123 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.123 DEBUG:boto:Data: 2024-04-16T18:26:43.123 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.123 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.123 DEBUG:boto:Params: {} 2024-04-16T18:26:43.123 DEBUG:boto:Token: None 2024-04-16T18:26:43.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:26:43.123 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw= 2024-04-16T18:26:43.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:umdFAhNAQZZ1oNszLIoKDwmzerw=', 'Content-Length': '0'} 2024-04-16T18:26:43.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052e43b9c407bac5c-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.127 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.127 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.127 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.127 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.127 DEBUG:boto:Method: GET 2024-04-16T18:26:43.127 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:26:43.127 DEBUG:boto:Data: 2024-04-16T18:26:43.127 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.127 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.127 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.127 DEBUG:boto:Params: {} 2024-04-16T18:26:43.127 DEBUG:boto:Token: None 2024-04-16T18:26:43.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc842b7da079e193-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.131 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.132 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.132 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.132 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.132 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.132 DEBUG:boto:Method: GET 2024-04-16T18:26:43.132 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:26:43.132 DEBUG:boto:Data: 2024-04-16T18:26:43.132 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.132 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.132 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.133 DEBUG:boto:Params: {} 2024-04-16T18:26:43.133 DEBUG:boto:Token: None 2024-04-16T18:26:43.133 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.139 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077d2c40b7d4050f9-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.182 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.183 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.183 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.183 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.183 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:26:43.183 DEBUG:boto:Data: 2024-04-16T18:26:43.183 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.183 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.183 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.183 DEBUG:boto:Params: {} 2024-04-16T18:26:43.183 DEBUG:boto:Token: None 2024-04-16T18:26:43.184 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145 2024-04-16T18:26:43.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YSC/4R3k6V9I11VYkGXmMz9dlS8= 2024-04-16T18:26:43.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YSC/4R3k6V9I11VYkGXmMz9dlS8=', 'Content-Length': '0'} 2024-04-16T18:26:43.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9e2109449d74239-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.187 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.187 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.187 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.187 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:26:43.187 DEBUG:boto:Data: 2024-04-16T18:26:43.188 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.188 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.188 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.188 DEBUG:boto:Params: {} 2024-04-16T18:26:43.188 DEBUG:boto:Token: None 2024-04-16T18:26:43.188 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145 2024-04-16T18:26:43.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YSC/4R3k6V9I11VYkGXmMz9dlS8= 2024-04-16T18:26:43.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YSC/4R3k6V9I11VYkGXmMz9dlS8=', 'Content-Length': '0'} 2024-04-16T18:26:43.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000210e85e07cb53999-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.192 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:26:43.192 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.192 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.192 DEBUG:boto:Method: GET 2024-04-16T18:26:43.192 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:26:43.192 DEBUG:boto:Data: 2024-04-16T18:26:43.192 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.192 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.193 DEBUG:boto:Params: {} 2024-04-16T18:26:43.193 DEBUG:boto:Token: None 2024-04-16T18:26:43.193 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145 2024-04-16T18:26:43.193 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7fmohrhTDKYVItuZCrgxyyIsKE= 2024-04-16T18:26:43.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7fmohrhTDKYVItuZCrgxyyIsKE=', 'Content-Length': '0'} 2024-04-16T18:26:43.196 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ba6746a8bb17a7a-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.196 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:26:43.196 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:26:43.196 DEBUG:boto:Method: GET 2024-04-16T18:26:43.197 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:26:43.197 DEBUG:boto:Data: 2024-04-16T18:26:43.197 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.197 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.197 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.197 DEBUG:boto:Params: {} 2024-04-16T18:26:43.197 DEBUG:boto:Token: None 2024-04-16T18:26:43.197 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj145 2024-04-16T18:26:43.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N7fmohrhTDKYVItuZCrgxyyIsKE= 2024-04-16T18:26:43.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N7fmohrhTDKYVItuZCrgxyyIsKE=', 'Content-Length': '0'} 2024-04-16T18:26:43.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000233874c912154486-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.200 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.201 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.201 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.201 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.201 DEBUG:boto:Method: GET 2024-04-16T18:26:43.201 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:26:43.201 DEBUG:boto:Data: 2024-04-16T18:26:43.201 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.201 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.201 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.201 DEBUG:boto:Params: {} 2024-04-16T18:26:43.202 DEBUG:boto:Token: None 2024-04-16T18:26:43.202 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.202 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.206 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6f99d449fa552c4-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.246 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.247 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.247 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.247 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.247 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:26:43.247 DEBUG:boto:Method: GET 2024-04-16T18:26:43.248 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:26:43.248 DEBUG:boto:Data: 2024-04-16T18:26:43.248 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.248 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.249 DEBUG:boto:Params: {} 2024-04-16T18:26:43.249 DEBUG:boto:Token: None 2024-04-16T18:26:43.249 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.255 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc2702b479ac8792-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.256 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:26:43.256 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.256 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.256 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.257 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.257 DEBUG:boto:Method: GET 2024-04-16T18:26:43.257 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.257 DEBUG:boto:Data: 2024-04-16T18:26:43.257 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.257 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.257 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.257 DEBUG:boto:Params: {} 2024-04-16T18:26:43.257 DEBUG:boto:Token: None 2024-04-16T18:26:43.258 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.258 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s= 2024-04-16T18:26:43.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s=', 'Content-Length': '0'} 2024-04-16T18:26:43.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091f59316db43aae8-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.261 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.261 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.262 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.262 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.262 DEBUG:boto:Method: GET 2024-04-16T18:26:43.262 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.262 DEBUG:boto:Data: 2024-04-16T18:26:43.262 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.262 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.262 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.262 DEBUG:boto:Params: {} 2024-04-16T18:26:43.263 DEBUG:boto:Token: None 2024-04-16T18:26:43.263 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s= 2024-04-16T18:26:43.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s=', 'Content-Length': '0'} 2024-04-16T18:26:43.266 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049ce10b9c821146b-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.306 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.307 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.307 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.307 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.307 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.307 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.307 DEBUG:boto:Data: 2024-04-16T18:26:43.307 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.307 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.307 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.308 DEBUG:boto:Params: {} 2024-04-16T18:26:43.308 DEBUG:boto:Token: None 2024-04-16T18:26:43.308 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:f4+YUZ/TJNlL8fzIhgekLMi5ASc= 2024-04-16T18:26:43.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:f4+YUZ/TJNlL8fzIhgekLMi5ASc=', 'Content-Length': '0'} 2024-04-16T18:26:43.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea181f5b8308a534-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.311 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.311 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.312 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.312 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.312 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.312 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.312 DEBUG:boto:Data: 2024-04-16T18:26:43.312 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.312 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.312 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.312 DEBUG:boto:Params: {} 2024-04-16T18:26:43.312 DEBUG:boto:Token: None 2024-04-16T18:26:43.312 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.313 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:f4+YUZ/TJNlL8fzIhgekLMi5ASc= 2024-04-16T18:26:43.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:f4+YUZ/TJNlL8fzIhgekLMi5ASc=', 'Content-Length': '0'} 2024-04-16T18:26:43.316 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003fc853bdf2c3b6f-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.316 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:26:43.316 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.316 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.317 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.317 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.317 DEBUG:boto:Method: GET 2024-04-16T18:26:43.317 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.317 DEBUG:boto:Data: 2024-04-16T18:26:43.317 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.317 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.317 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.317 DEBUG:boto:Params: {} 2024-04-16T18:26:43.317 DEBUG:boto:Token: None 2024-04-16T18:26:43.317 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.317 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s= 2024-04-16T18:26:43.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s=', 'Content-Length': '0'} 2024-04-16T18:26:43.321 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab850174243384a7-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.321 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.321 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.321 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.321 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.321 DEBUG:boto:Method: GET 2024-04-16T18:26:43.321 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.321 DEBUG:boto:Data: 2024-04-16T18:26:43.321 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.321 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.322 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.322 DEBUG:boto:Params: {} 2024-04-16T18:26:43.322 DEBUG:boto:Token: None 2024-04-16T18:26:43.322 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:26:43.322 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s= 2024-04-16T18:26:43.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DtgUz8jhZhbhWoMO3xqNEimGY/s=', 'Content-Length': '0'} 2024-04-16T18:26:43.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5bfe5f89c295a89-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.325 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.326 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.326 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.326 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.326 DEBUG:boto:Method: GET 2024-04-16T18:26:43.326 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:26:43.326 DEBUG:boto:Data: 2024-04-16T18:26:43.326 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.326 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.326 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.326 DEBUG:boto:Params: {} 2024-04-16T18:26:43.326 DEBUG:boto:Token: None 2024-04-16T18:26:43.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.327 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.330 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025f5ffecdb12aa18-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.374 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.375 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.375 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.375 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.375 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.375 DEBUG:boto:Method: GET 2024-04-16T18:26:43.375 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:26:43.375 DEBUG:boto:Data: 2024-04-16T18:26:43.375 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.375 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.375 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.376 DEBUG:boto:Params: {} 2024-04-16T18:26:43.376 DEBUG:boto:Token: None 2024-04-16T18:26:43.376 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.382 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e536e009568c075-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.382 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.383 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.383 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.383 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.383 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:26:43.383 DEBUG:boto:Data: 2024-04-16T18:26:43.383 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.383 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.384 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.384 DEBUG:boto:Params: {} 2024-04-16T18:26:43.384 DEBUG:boto:Token: None 2024-04-16T18:26:43.384 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17 2024-04-16T18:26:43.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9yVNUPZsrdieLoB0ZCeNK0aRFts= 2024-04-16T18:26:43.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9yVNUPZsrdieLoB0ZCeNK0aRFts=', 'Content-Length': '0'} 2024-04-16T18:26:43.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbba827054aca77b-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.387 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.387 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.387 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.387 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:26:43.387 DEBUG:boto:Data: 2024-04-16T18:26:43.387 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.388 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.388 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.388 DEBUG:boto:Params: {} 2024-04-16T18:26:43.388 DEBUG:boto:Token: None 2024-04-16T18:26:43.388 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17 2024-04-16T18:26:43.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9yVNUPZsrdieLoB0ZCeNK0aRFts= 2024-04-16T18:26:43.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9yVNUPZsrdieLoB0ZCeNK0aRFts=', 'Content-Length': '0'} 2024-04-16T18:26:43.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000099def1218fd9cb7-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.392 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:26:43.392 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.392 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.392 DEBUG:boto:Method: GET 2024-04-16T18:26:43.392 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:26:43.393 DEBUG:boto:Data: 2024-04-16T18:26:43.393 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.393 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.393 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.393 DEBUG:boto:Params: {} 2024-04-16T18:26:43.393 DEBUG:boto:Token: None 2024-04-16T18:26:43.393 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17 2024-04-16T18:26:43.393 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KkG+YeVHQSNUuesItLUiC7J9oks= 2024-04-16T18:26:43.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KkG+YeVHQSNUuesItLUiC7J9oks=', 'Content-Length': '0'} 2024-04-16T18:26:43.396 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ffd7dba769f370f-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.396 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:26:43.396 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:26:43.397 DEBUG:boto:Method: GET 2024-04-16T18:26:43.397 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:26:43.397 DEBUG:boto:Data: 2024-04-16T18:26:43.397 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.397 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.397 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.397 DEBUG:boto:Params: {} 2024-04-16T18:26:43.397 DEBUG:boto:Token: None 2024-04-16T18:26:43.397 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj17 2024-04-16T18:26:43.397 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KkG+YeVHQSNUuesItLUiC7J9oks= 2024-04-16T18:26:43.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KkG+YeVHQSNUuesItLUiC7J9oks=', 'Content-Length': '0'} 2024-04-16T18:26:43.400 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000859dfa0c76a101b6-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.442 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.442 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.443 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.443 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.443 DEBUG:boto:Method: GET 2024-04-16T18:26:43.443 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:26:43.443 DEBUG:boto:Data: 2024-04-16T18:26:43.443 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.443 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.443 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.443 DEBUG:boto:Params: {} 2024-04-16T18:26:43.443 DEBUG:boto:Token: None 2024-04-16T18:26:43.444 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad88f058bc958245-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.448 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.448 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.448 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.448 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.448 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:26:43.449 DEBUG:boto:Method: GET 2024-04-16T18:26:43.449 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:26:43.449 DEBUG:boto:Data: 2024-04-16T18:26:43.449 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.449 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.449 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.449 DEBUG:boto:Params: {} 2024-04-16T18:26:43.449 DEBUG:boto:Token: None 2024-04-16T18:26:43.449 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.449 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.456 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b3e8ad5a1216a85-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.456 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.456 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:26:43.456 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.456 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.457 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.457 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.457 DEBUG:boto:Method: GET 2024-04-16T18:26:43.457 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.457 DEBUG:boto:Data: 2024-04-16T18:26:43.457 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.457 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.457 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.457 DEBUG:boto:Params: {} 2024-04-16T18:26:43.457 DEBUG:boto:Token: None 2024-04-16T18:26:43.458 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.458 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI= 2024-04-16T18:26:43.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI=', 'Content-Length': '0'} 2024-04-16T18:26:43.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000435a2bff1c958064-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.502 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.502 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.503 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.503 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.503 DEBUG:boto:Method: GET 2024-04-16T18:26:43.503 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.503 DEBUG:boto:Data: 2024-04-16T18:26:43.503 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.503 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.503 DEBUG:boto:Params: {} 2024-04-16T18:26:43.503 DEBUG:boto:Token: None 2024-04-16T18:26:43.504 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI= 2024-04-16T18:26:43.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI=', 'Content-Length': '0'} 2024-04-16T18:26:43.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a84ff1bd8fec799-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.507 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.507 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.507 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.507 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.507 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.507 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.507 DEBUG:boto:Data: 2024-04-16T18:26:43.508 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.508 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.508 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.508 DEBUG:boto:Params: {} 2024-04-16T18:26:43.508 DEBUG:boto:Token: None 2024-04-16T18:26:43.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YnFVXnn8L6/cA3WbGWIem8A6nG0= 2024-04-16T18:26:43.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YnFVXnn8L6/cA3WbGWIem8A6nG0=', 'Content-Length': '0'} 2024-04-16T18:26:43.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000508f6330a938a8e-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.511 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.512 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.512 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.512 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.512 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.512 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.512 DEBUG:boto:Data: 2024-04-16T18:26:43.512 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.512 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.512 DEBUG:boto:Params: {} 2024-04-16T18:26:43.512 DEBUG:boto:Token: None 2024-04-16T18:26:43.513 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YnFVXnn8L6/cA3WbGWIem8A6nG0= 2024-04-16T18:26:43.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YnFVXnn8L6/cA3WbGWIem8A6nG0=', 'Content-Length': '0'} 2024-04-16T18:26:43.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c70cf8d12d2dd3b-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.516 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:26:43.516 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.516 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.516 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.516 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.516 DEBUG:boto:Method: GET 2024-04-16T18:26:43.517 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.517 DEBUG:boto:Data: 2024-04-16T18:26:43.517 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.517 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.517 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.517 DEBUG:boto:Params: {} 2024-04-16T18:26:43.517 DEBUG:boto:Token: None 2024-04-16T18:26:43.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI= 2024-04-16T18:26:43.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI=', 'Content-Length': '0'} 2024-04-16T18:26:43.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050bbc7e4ed93509a-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.562 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.563 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.563 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.563 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.563 DEBUG:boto:Method: GET 2024-04-16T18:26:43.563 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.563 DEBUG:boto:Data: 2024-04-16T18:26:43.563 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.563 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.563 DEBUG:boto:Params: {} 2024-04-16T18:26:43.563 DEBUG:boto:Token: None 2024-04-16T18:26:43.563 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:26:43.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI= 2024-04-16T18:26:43.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W6MiEJ+KpViKlnffx6C2wqrPyWI=', 'Content-Length': '0'} 2024-04-16T18:26:43.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000225dec4a7f111395-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.567 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.567 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.567 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.567 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.567 DEBUG:boto:Method: GET 2024-04-16T18:26:43.568 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:26:43.568 DEBUG:boto:Data: 2024-04-16T18:26:43.568 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.568 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.568 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.568 DEBUG:boto:Params: {} 2024-04-16T18:26:43.568 DEBUG:boto:Token: None 2024-04-16T18:26:43.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008eb5e97cd712d30f-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.614 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.615 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.615 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.615 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.615 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.615 DEBUG:boto:Method: GET 2024-04-16T18:26:43.615 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:26:43.615 DEBUG:boto:Data: 2024-04-16T18:26:43.615 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.615 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.616 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.616 DEBUG:boto:Params: {} 2024-04-16T18:26:43.616 DEBUG:boto:Token: None 2024-04-16T18:26:43.616 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.622 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001eeecf752532807-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.622 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.623 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.623 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.623 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.623 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:26:43.623 DEBUG:boto:Data: 2024-04-16T18:26:43.623 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.623 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.623 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.623 DEBUG:boto:Params: {} 2024-04-16T18:26:43.624 DEBUG:boto:Token: None 2024-04-16T18:26:43.624 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19 2024-04-16T18:26:43.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MaG2YivTjwq/rTxM9PmWntht5vo= 2024-04-16T18:26:43.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MaG2YivTjwq/rTxM9PmWntht5vo=', 'Content-Length': '0'} 2024-04-16T18:26:43.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cef6c0da18127d9-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.627 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.627 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.628 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.628 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:26:43.628 DEBUG:boto:Data: 2024-04-16T18:26:43.628 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.628 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.628 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.628 DEBUG:boto:Params: {} 2024-04-16T18:26:43.628 DEBUG:boto:Token: None 2024-04-16T18:26:43.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19 2024-04-16T18:26:43.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MaG2YivTjwq/rTxM9PmWntht5vo= 2024-04-16T18:26:43.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MaG2YivTjwq/rTxM9PmWntht5vo=', 'Content-Length': '0'} 2024-04-16T18:26:43.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000181a48a4eac5c911-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.632 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:26:43.632 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.632 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.632 DEBUG:boto:Method: GET 2024-04-16T18:26:43.632 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:26:43.632 DEBUG:boto:Data: 2024-04-16T18:26:43.632 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.632 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.632 DEBUG:boto:Params: {} 2024-04-16T18:26:43.633 DEBUG:boto:Token: None 2024-04-16T18:26:43.633 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19 2024-04-16T18:26:43.633 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gioKvyyMbu0mRAbScKCtF/eG9mw= 2024-04-16T18:26:43.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gioKvyyMbu0mRAbScKCtF/eG9mw=', 'Content-Length': '0'} 2024-04-16T18:26:43.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fce21040c7ae904-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.678 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:26:43.678 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:26:43.679 DEBUG:boto:Method: GET 2024-04-16T18:26:43.679 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:26:43.679 DEBUG:boto:Data: 2024-04-16T18:26:43.679 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.679 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.679 DEBUG:boto:Params: {} 2024-04-16T18:26:43.679 DEBUG:boto:Token: None 2024-04-16T18:26:43.680 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj19 2024-04-16T18:26:43.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gioKvyyMbu0mRAbScKCtF/eG9mw= 2024-04-16T18:26:43.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gioKvyyMbu0mRAbScKCtF/eG9mw=', 'Content-Length': '0'} 2024-04-16T18:26:43.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071f54298f1fb485c-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.683 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.684 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.684 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.684 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.684 DEBUG:boto:Method: GET 2024-04-16T18:26:43.684 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:26:43.684 DEBUG:boto:Data: 2024-04-16T18:26:43.684 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.684 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.684 DEBUG:boto:Params: {} 2024-04-16T18:26:43.685 DEBUG:boto:Token: None 2024-04-16T18:26:43.685 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.685 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001de5a849a980e3b5-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.730 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.731 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.731 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.731 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.731 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:26:43.731 DEBUG:boto:Method: GET 2024-04-16T18:26:43.731 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:26:43.731 DEBUG:boto:Data: 2024-04-16T18:26:43.731 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.732 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.732 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.732 DEBUG:boto:Params: {} 2024-04-16T18:26:43.732 DEBUG:boto:Token: None 2024-04-16T18:26:43.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f500d1b8bd9217ec-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.739 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.739 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:26:43.739 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.739 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.739 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.739 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.739 DEBUG:boto:Method: GET 2024-04-16T18:26:43.739 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.739 DEBUG:boto:Data: 2024-04-16T18:26:43.739 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.740 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.740 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.740 DEBUG:boto:Params: {} 2024-04-16T18:26:43.740 DEBUG:boto:Token: None 2024-04-16T18:26:43.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU= 2024-04-16T18:26:43.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU=', 'Content-Length': '0'} 2024-04-16T18:26:43.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080bbb551440d2c00-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.786 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.787 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.787 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.787 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.787 DEBUG:boto:Method: GET 2024-04-16T18:26:43.787 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.787 DEBUG:boto:Data: 2024-04-16T18:26:43.787 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.787 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.787 DEBUG:boto:Params: {} 2024-04-16T18:26:43.787 DEBUG:boto:Token: None 2024-04-16T18:26:43.788 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU= 2024-04-16T18:26:43.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU=', 'Content-Length': '0'} 2024-04-16T18:26:43.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bde305600f6bb90-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.791 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.791 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.791 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.791 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.791 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.791 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.791 DEBUG:boto:Data: 2024-04-16T18:26:43.791 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.792 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.792 DEBUG:boto:Params: {} 2024-04-16T18:26:43.792 DEBUG:boto:Token: None 2024-04-16T18:26:43.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:brdQmYiFLHq1hrzxlcT9ApWpRNI= 2024-04-16T18:26:43.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:brdQmYiFLHq1hrzxlcT9ApWpRNI=', 'Content-Length': '0'} 2024-04-16T18:26:43.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef493484a10cc361-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.795 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.795 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.796 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.796 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.796 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.796 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.796 DEBUG:boto:Data: 2024-04-16T18:26:43.796 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.796 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.796 DEBUG:boto:Params: {} 2024-04-16T18:26:43.796 DEBUG:boto:Token: None 2024-04-16T18:26:43.797 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:brdQmYiFLHq1hrzxlcT9ApWpRNI= 2024-04-16T18:26:43.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:brdQmYiFLHq1hrzxlcT9ApWpRNI=', 'Content-Length': '0'} 2024-04-16T18:26:43.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000877124a60c77e074-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.799 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:26:43.800 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.800 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.800 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.800 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.800 DEBUG:boto:Method: GET 2024-04-16T18:26:43.800 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.800 DEBUG:boto:Data: 2024-04-16T18:26:43.800 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.800 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.800 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.800 DEBUG:boto:Params: {} 2024-04-16T18:26:43.801 DEBUG:boto:Token: None 2024-04-16T18:26:43.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU= 2024-04-16T18:26:43.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU=', 'Content-Length': '0'} 2024-04-16T18:26:43.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e85b4a93e1910b41-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.846 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.846 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.847 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.847 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.847 DEBUG:boto:Method: GET 2024-04-16T18:26:43.847 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.847 DEBUG:boto:Data: 2024-04-16T18:26:43.847 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.847 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.847 DEBUG:boto:Params: {} 2024-04-16T18:26:43.848 DEBUG:boto:Token: None 2024-04-16T18:26:43.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:26:43.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU= 2024-04-16T18:26:43.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J0Qv8G0bcEy7yZ8/hdJPb6+e1HU=', 'Content-Length': '0'} 2024-04-16T18:26:43.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030ad56dbc58e64d2-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.851 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.851 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.851 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.851 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.852 DEBUG:boto:Method: GET 2024-04-16T18:26:43.852 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:26:43.852 DEBUG:boto:Data: 2024-04-16T18:26:43.852 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.852 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.852 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.852 DEBUG:boto:Params: {} 2024-04-16T18:26:43.852 DEBUG:boto:Token: None 2024-04-16T18:26:43.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.853 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008263c40abe2a5bd2-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.856 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.857 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.857 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.857 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.857 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.857 DEBUG:boto:Method: GET 2024-04-16T18:26:43.857 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:26:43.857 DEBUG:boto:Data: 2024-04-16T18:26:43.857 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.858 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.858 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.858 DEBUG:boto:Params: {} 2024-04-16T18:26:43.858 DEBUG:boto:Token: None 2024-04-16T18:26:43.858 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.858 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000082dd323504a3b27-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.864 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.864 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.865 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.865 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.865 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:26:43.865 DEBUG:boto:Data: 2024-04-16T18:26:43.865 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.865 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.865 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.865 DEBUG:boto:Params: {} 2024-04-16T18:26:43.865 DEBUG:boto:Token: None 2024-04-16T18:26:43.866 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22 2024-04-16T18:26:43.866 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+zURJmh+3KIGt6KRqNbEDb98/8I= 2024-04-16T18:26:43.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+zURJmh+3KIGt6KRqNbEDb98/8I=', 'Content-Length': '0'} 2024-04-16T18:26:43.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5c1524956ffdb4f-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.869 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.869 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.869 DEBUG:boto:Method: HEAD 2024-04-16T18:26:43.869 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:26:43.869 DEBUG:boto:Data: 2024-04-16T18:26:43.870 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.870 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.870 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.870 DEBUG:boto:Params: {} 2024-04-16T18:26:43.870 DEBUG:boto:Token: None 2024-04-16T18:26:43.870 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22 2024-04-16T18:26:43.870 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+zURJmh+3KIGt6KRqNbEDb98/8I= 2024-04-16T18:26:43.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+zURJmh+3KIGt6KRqNbEDb98/8I=', 'Content-Length': '0'} 2024-04-16T18:26:43.873 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df6bc1fab96cf937-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.873 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:26:43.873 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.874 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.874 DEBUG:boto:Method: GET 2024-04-16T18:26:43.874 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:26:43.874 DEBUG:boto:Data: 2024-04-16T18:26:43.874 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.874 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.874 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.874 DEBUG:boto:Params: {} 2024-04-16T18:26:43.874 DEBUG:boto:Token: None 2024-04-16T18:26:43.875 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22 2024-04-16T18:26:43.875 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wYvYUqQdoiRWmKSwCOPR9JGwO6c= 2024-04-16T18:26:43.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wYvYUqQdoiRWmKSwCOPR9JGwO6c=', 'Content-Length': '0'} 2024-04-16T18:26:43.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000789b434a505256fd-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.918 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:26:43.918 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:26:43.919 DEBUG:boto:Method: GET 2024-04-16T18:26:43.919 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:26:43.919 DEBUG:boto:Data: 2024-04-16T18:26:43.919 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.919 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.919 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.919 DEBUG:boto:Params: {} 2024-04-16T18:26:43.919 DEBUG:boto:Token: None 2024-04-16T18:26:43.919 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj22 2024-04-16T18:26:43.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wYvYUqQdoiRWmKSwCOPR9JGwO6c= 2024-04-16T18:26:43.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wYvYUqQdoiRWmKSwCOPR9JGwO6c=', 'Content-Length': '0'} 2024-04-16T18:26:43.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dfcf97f7eccad5a-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.923 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.923 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.923 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.923 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.923 DEBUG:boto:Method: GET 2024-04-16T18:26:43.923 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:26:43.923 DEBUG:boto:Data: 2024-04-16T18:26:43.923 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.923 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.923 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.924 DEBUG:boto:Params: {} 2024-04-16T18:26:43.924 DEBUG:boto:Token: None 2024-04-16T18:26:43.924 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.927 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000291c4804db07cc43-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.970 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.971 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:43.971 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:43.971 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.971 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:26:43.971 DEBUG:boto:Method: GET 2024-04-16T18:26:43.971 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:26:43.971 DEBUG:boto:Data: 2024-04-16T18:26:43.971 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.971 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:43.971 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.971 DEBUG:boto:Params: {} 2024-04-16T18:26:43.972 DEBUG:boto:Token: None 2024-04-16T18:26:43.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/ 2024-04-16T18:26:43.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4= 2024-04-16T18:26:43.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywflmvdZdFpfkaYjiPdkMPau/t4=', 'Content-Length': '0'} 2024-04-16T18:26:43.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3f21a60a77e6b48-00661ec2e3-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:43.978 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:43.978 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:26:43.978 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:43.979 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:43.979 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:43.979 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:43.979 DEBUG:boto:Method: GET 2024-04-16T18:26:43.979 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:26:43.979 DEBUG:boto:Data: 2024-04-16T18:26:43.979 DEBUG:boto:Headers: {} 2024-04-16T18:26:43.979 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:43.979 DEBUG:boto:Port: 8000 2024-04-16T18:26:43.979 DEBUG:boto:Params: {} 2024-04-16T18:26:43.979 DEBUG:boto:Token: None 2024-04-16T18:26:43.979 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:43 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:26:43.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rd+pkoWbt1ovxVTaw67gQ60Ka78= 2024-04-16T18:26:43.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rd+pkoWbt1ovxVTaw67gQ60Ka78=', 'Content-Length': '0'} 2024-04-16T18:26:43.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019cc92eca4521b9f-00661ec2e3-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.030 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.030 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.031 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.031 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.031 DEBUG:boto:Method: GET 2024-04-16T18:26:44.031 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.031 DEBUG:boto:Data: 2024-04-16T18:26:44.031 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.031 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.031 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.031 DEBUG:boto:Params: {} 2024-04-16T18:26:44.031 DEBUG:boto:Token: None 2024-04-16T18:26:44.032 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DU0Aai3wuGQovkFiQjg8mbID3fs= 2024-04-16T18:26:44.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DU0Aai3wuGQovkFiQjg8mbID3fs=', 'Content-Length': '0'} 2024-04-16T18:26:44.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d1a07049b1da3e3-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.035 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.035 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.035 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.035 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.035 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.035 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.035 DEBUG:boto:Data: 2024-04-16T18:26:44.036 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.036 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.036 DEBUG:boto:Params: {} 2024-04-16T18:26:44.036 DEBUG:boto:Token: None 2024-04-16T18:26:44.036 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:apotUYg+OeK2wS9rR50KnxUXhTA= 2024-04-16T18:26:44.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:apotUYg+OeK2wS9rR50KnxUXhTA=', 'Content-Length': '0'} 2024-04-16T18:26:44.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8537ae44007e1ef-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.039 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.039 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.039 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.039 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.040 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.040 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.040 DEBUG:boto:Data: 2024-04-16T18:26:44.040 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.040 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.040 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.040 DEBUG:boto:Params: {} 2024-04-16T18:26:44.040 DEBUG:boto:Token: None 2024-04-16T18:26:44.040 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:apotUYg+OeK2wS9rR50KnxUXhTA= 2024-04-16T18:26:44.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:apotUYg+OeK2wS9rR50KnxUXhTA=', 'Content-Length': '0'} 2024-04-16T18:26:44.043 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079f421132d804f79-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:26:44.043 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.044 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.044 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.044 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.044 DEBUG:boto:Method: GET 2024-04-16T18:26:44.044 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.044 DEBUG:boto:Data: 2024-04-16T18:26:44.044 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.044 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.044 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.044 DEBUG:boto:Params: {} 2024-04-16T18:26:44.045 DEBUG:boto:Token: None 2024-04-16T18:26:44.045 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.045 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DU0Aai3wuGQovkFiQjg8mbID3fs= 2024-04-16T18:26:44.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DU0Aai3wuGQovkFiQjg8mbID3fs=', 'Content-Length': '0'} 2024-04-16T18:26:44.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b029b87cc57a0d05-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.090 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.090 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.091 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.091 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.091 DEBUG:boto:Method: GET 2024-04-16T18:26:44.091 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.091 DEBUG:boto:Data: 2024-04-16T18:26:44.091 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.091 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.091 DEBUG:boto:Params: {} 2024-04-16T18:26:44.092 DEBUG:boto:Token: None 2024-04-16T18:26:44.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:26:44.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DU0Aai3wuGQovkFiQjg8mbID3fs= 2024-04-16T18:26:44.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DU0Aai3wuGQovkFiQjg8mbID3fs=', 'Content-Length': '0'} 2024-04-16T18:26:44.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000979dd7b3190359cd-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.095 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.095 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.096 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.096 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.096 DEBUG:boto:Method: GET 2024-04-16T18:26:44.096 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:26:44.096 DEBUG:boto:Data: 2024-04-16T18:26:44.096 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.096 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.096 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.096 DEBUG:boto:Params: {} 2024-04-16T18:26:44.097 DEBUG:boto:Token: None 2024-04-16T18:26:44.097 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ce9f9f1368a4a9f-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.101 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.101 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.101 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.101 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.101 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.101 DEBUG:boto:Method: GET 2024-04-16T18:26:44.102 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:26:44.102 DEBUG:boto:Data: 2024-04-16T18:26:44.102 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.102 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.102 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.102 DEBUG:boto:Params: {} 2024-04-16T18:26:44.102 DEBUG:boto:Token: None 2024-04-16T18:26:44.102 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.102 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.109 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000e246b85398c7399-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.109 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.109 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.109 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.110 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.110 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:26:44.110 DEBUG:boto:Data: 2024-04-16T18:26:44.110 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.110 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.110 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.110 DEBUG:boto:Params: {} 2024-04-16T18:26:44.110 DEBUG:boto:Token: None 2024-04-16T18:26:44.110 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23 2024-04-16T18:26:44.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:09UBOZHmYpqNXUSpqyI1T1jCrkw= 2024-04-16T18:26:44.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:09UBOZHmYpqNXUSpqyI1T1jCrkw=', 'Content-Length': '0'} 2024-04-16T18:26:44.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cede4a5b0c5173fc-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.113 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.114 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.114 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.114 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:26:44.114 DEBUG:boto:Data: 2024-04-16T18:26:44.114 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.114 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.114 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.114 DEBUG:boto:Params: {} 2024-04-16T18:26:44.114 DEBUG:boto:Token: None 2024-04-16T18:26:44.115 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23 2024-04-16T18:26:44.115 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:09UBOZHmYpqNXUSpqyI1T1jCrkw= 2024-04-16T18:26:44.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:09UBOZHmYpqNXUSpqyI1T1jCrkw=', 'Content-Length': '0'} 2024-04-16T18:26:44.118 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000786c28e1a8313b18-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.118 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:26:44.118 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.118 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.118 DEBUG:boto:Method: GET 2024-04-16T18:26:44.118 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:26:44.118 DEBUG:boto:Data: 2024-04-16T18:26:44.119 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.119 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.119 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.119 DEBUG:boto:Params: {} 2024-04-16T18:26:44.119 DEBUG:boto:Token: None 2024-04-16T18:26:44.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23 2024-04-16T18:26:44.119 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dy7+vxaHjmUxV81yPQ/xfITdxRg= 2024-04-16T18:26:44.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dy7+vxaHjmUxV81yPQ/xfITdxRg=', 'Content-Length': '0'} 2024-04-16T18:26:44.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c97bcb5c96dc7d65-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.166 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:26:44.167 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:26:44.167 DEBUG:boto:Method: GET 2024-04-16T18:26:44.167 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:26:44.167 DEBUG:boto:Data: 2024-04-16T18:26:44.167 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.167 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.167 DEBUG:boto:Params: {} 2024-04-16T18:26:44.167 DEBUG:boto:Token: None 2024-04-16T18:26:44.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj23 2024-04-16T18:26:44.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dy7+vxaHjmUxV81yPQ/xfITdxRg= 2024-04-16T18:26:44.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dy7+vxaHjmUxV81yPQ/xfITdxRg=', 'Content-Length': '0'} 2024-04-16T18:26:44.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000333a685b32497325-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.171 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.171 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.171 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.171 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.171 DEBUG:boto:Method: GET 2024-04-16T18:26:44.171 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:26:44.171 DEBUG:boto:Data: 2024-04-16T18:26:44.172 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.172 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.172 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.172 DEBUG:boto:Params: {} 2024-04-16T18:26:44.172 DEBUG:boto:Token: None 2024-04-16T18:26:44.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f2b97d2f14b8fac-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.218 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.219 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.219 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.219 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.219 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:26:44.219 DEBUG:boto:Method: GET 2024-04-16T18:26:44.219 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:26:44.219 DEBUG:boto:Data: 2024-04-16T18:26:44.219 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.220 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.220 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.220 DEBUG:boto:Params: {} 2024-04-16T18:26:44.220 DEBUG:boto:Token: None 2024-04-16T18:26:44.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019d95c96b33664d0-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.227 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.227 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:26:44.227 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.227 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.227 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.227 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.227 DEBUG:boto:Method: GET 2024-04-16T18:26:44.227 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.227 DEBUG:boto:Data: 2024-04-16T18:26:44.228 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.228 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.228 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.228 DEBUG:boto:Params: {} 2024-04-16T18:26:44.228 DEBUG:boto:Token: None 2024-04-16T18:26:44.228 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g= 2024-04-16T18:26:44.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g=', 'Content-Length': '0'} 2024-04-16T18:26:44.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087fb852683d987a7-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.274 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.274 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.275 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.275 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.275 DEBUG:boto:Method: GET 2024-04-16T18:26:44.275 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.275 DEBUG:boto:Data: 2024-04-16T18:26:44.275 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.275 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.275 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.275 DEBUG:boto:Params: {} 2024-04-16T18:26:44.275 DEBUG:boto:Token: None 2024-04-16T18:26:44.276 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g= 2024-04-16T18:26:44.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g=', 'Content-Length': '0'} 2024-04-16T18:26:44.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e9a064729bcc835-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.281 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.281 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.281 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.281 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.281 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.281 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.281 DEBUG:boto:Data: 2024-04-16T18:26:44.281 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.281 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.281 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.282 DEBUG:boto:Params: {} 2024-04-16T18:26:44.282 DEBUG:boto:Token: None 2024-04-16T18:26:44.282 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.282 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DL864J2cx/n0WZHpPE2h6/Ph00w= 2024-04-16T18:26:44.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DL864J2cx/n0WZHpPE2h6/Ph00w=', 'Content-Length': '0'} 2024-04-16T18:26:44.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a08e473cb7973edd-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.286 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.286 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.286 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.286 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.286 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.286 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.286 DEBUG:boto:Data: 2024-04-16T18:26:44.286 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.286 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.287 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.287 DEBUG:boto:Params: {} 2024-04-16T18:26:44.287 DEBUG:boto:Token: None 2024-04-16T18:26:44.287 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DL864J2cx/n0WZHpPE2h6/Ph00w= 2024-04-16T18:26:44.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DL864J2cx/n0WZHpPE2h6/Ph00w=', 'Content-Length': '0'} 2024-04-16T18:26:44.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b5e528ead83680f-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:26:44.290 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.290 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.291 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.291 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.291 DEBUG:boto:Method: GET 2024-04-16T18:26:44.291 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.291 DEBUG:boto:Data: 2024-04-16T18:26:44.291 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.291 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.291 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.291 DEBUG:boto:Params: {} 2024-04-16T18:26:44.291 DEBUG:boto:Token: None 2024-04-16T18:26:44.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g= 2024-04-16T18:26:44.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g=', 'Content-Length': '0'} 2024-04-16T18:26:44.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007acda8b6bcca301b-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.338 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.338 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.339 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.339 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.339 DEBUG:boto:Method: GET 2024-04-16T18:26:44.339 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.339 DEBUG:boto:Data: 2024-04-16T18:26:44.339 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.339 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.339 DEBUG:boto:Params: {} 2024-04-16T18:26:44.339 DEBUG:boto:Token: None 2024-04-16T18:26:44.339 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:26:44.339 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g= 2024-04-16T18:26:44.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNzJG9s44q4dN/f2sR1trEP9k5g=', 'Content-Length': '0'} 2024-04-16T18:26:44.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cea40090a7b57fb7-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.343 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.343 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.343 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.343 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.343 DEBUG:boto:Method: GET 2024-04-16T18:26:44.343 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:26:44.344 DEBUG:boto:Data: 2024-04-16T18:26:44.344 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.344 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.344 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.344 DEBUG:boto:Params: {} 2024-04-16T18:26:44.344 DEBUG:boto:Token: None 2024-04-16T18:26:44.344 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e20e5fe8b366674-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.390 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.391 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.391 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.391 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.391 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.391 DEBUG:boto:Method: GET 2024-04-16T18:26:44.391 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:26:44.391 DEBUG:boto:Data: 2024-04-16T18:26:44.391 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.392 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.392 DEBUG:boto:Params: {} 2024-04-16T18:26:44.392 DEBUG:boto:Token: None 2024-04-16T18:26:44.392 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.398 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e31bc11ad9efa6b-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.399 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.399 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.399 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.399 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.399 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:26:44.399 DEBUG:boto:Data: 2024-04-16T18:26:44.399 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.399 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.399 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.400 DEBUG:boto:Params: {} 2024-04-16T18:26:44.400 DEBUG:boto:Token: None 2024-04-16T18:26:44.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29 2024-04-16T18:26:44.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8spIs8q+XalftJuzzusQoGRXj/4= 2024-04-16T18:26:44.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8spIs8q+XalftJuzzusQoGRXj/4=', 'Content-Length': '0'} 2024-04-16T18:26:44.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009839b1ac069ac767-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.403 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.403 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.403 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.403 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:26:44.403 DEBUG:boto:Data: 2024-04-16T18:26:44.404 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.404 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.404 DEBUG:boto:Params: {} 2024-04-16T18:26:44.404 DEBUG:boto:Token: None 2024-04-16T18:26:44.404 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29 2024-04-16T18:26:44.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8spIs8q+XalftJuzzusQoGRXj/4= 2024-04-16T18:26:44.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8spIs8q+XalftJuzzusQoGRXj/4=', 'Content-Length': '0'} 2024-04-16T18:26:44.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aaef1fcdc668166c-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.407 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:26:44.408 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.408 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.408 DEBUG:boto:Method: GET 2024-04-16T18:26:44.408 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:26:44.408 DEBUG:boto:Data: 2024-04-16T18:26:44.408 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.408 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.408 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.408 DEBUG:boto:Params: {} 2024-04-16T18:26:44.408 DEBUG:boto:Token: None 2024-04-16T18:26:44.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29 2024-04-16T18:26:44.409 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ta27d8zzhYW+uuKgQ8eOk1e+RfQ= 2024-04-16T18:26:44.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ta27d8zzhYW+uuKgQ8eOk1e+RfQ=', 'Content-Length': '0'} 2024-04-16T18:26:44.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da1728d26498f48a-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.454 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:26:44.454 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:26:44.455 DEBUG:boto:Method: GET 2024-04-16T18:26:44.455 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:26:44.455 DEBUG:boto:Data: 2024-04-16T18:26:44.455 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.455 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.455 DEBUG:boto:Params: {} 2024-04-16T18:26:44.455 DEBUG:boto:Token: None 2024-04-16T18:26:44.455 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj29 2024-04-16T18:26:44.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ta27d8zzhYW+uuKgQ8eOk1e+RfQ= 2024-04-16T18:26:44.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ta27d8zzhYW+uuKgQ8eOk1e+RfQ=', 'Content-Length': '0'} 2024-04-16T18:26:44.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018aa7e886008c7e8-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.460 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.460 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.460 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.460 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.460 DEBUG:boto:Method: GET 2024-04-16T18:26:44.460 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:26:44.460 DEBUG:boto:Data: 2024-04-16T18:26:44.460 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.460 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.461 DEBUG:boto:Params: {} 2024-04-16T18:26:44.461 DEBUG:boto:Token: None 2024-04-16T18:26:44.461 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.465 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000416c2e2ac4e6617c-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.506 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.507 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.507 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.507 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.507 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:26:44.507 DEBUG:boto:Method: GET 2024-04-16T18:26:44.507 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:26:44.507 DEBUG:boto:Data: 2024-04-16T18:26:44.507 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.508 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.508 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.508 DEBUG:boto:Params: {} 2024-04-16T18:26:44.508 DEBUG:boto:Token: None 2024-04-16T18:26:44.508 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca29372a872811a3-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.515 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:26:44.516 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.516 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.516 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.516 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.516 DEBUG:boto:Method: GET 2024-04-16T18:26:44.516 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.516 DEBUG:boto:Data: 2024-04-16T18:26:44.516 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.516 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.517 DEBUG:boto:Params: {} 2024-04-16T18:26:44.517 DEBUG:boto:Token: None 2024-04-16T18:26:44.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI= 2024-04-16T18:26:44.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI=', 'Content-Length': '0'} 2024-04-16T18:26:44.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000403bb9a62144e477-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.562 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.562 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.563 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.563 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.563 DEBUG:boto:Method: GET 2024-04-16T18:26:44.563 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.563 DEBUG:boto:Data: 2024-04-16T18:26:44.563 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.563 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.563 DEBUG:boto:Params: {} 2024-04-16T18:26:44.563 DEBUG:boto:Token: None 2024-04-16T18:26:44.563 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.564 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI= 2024-04-16T18:26:44.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI=', 'Content-Length': '0'} 2024-04-16T18:26:44.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002788d4fdf28c3745-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.567 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.567 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.567 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.567 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.568 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.568 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.568 DEBUG:boto:Data: 2024-04-16T18:26:44.568 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.568 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.568 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.568 DEBUG:boto:Params: {} 2024-04-16T18:26:44.568 DEBUG:boto:Token: None 2024-04-16T18:26:44.568 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l4EZkLPvJQkwQNqWDF+6uLNt788= 2024-04-16T18:26:44.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l4EZkLPvJQkwQNqWDF+6uLNt788=', 'Content-Length': '0'} 2024-04-16T18:26:44.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef33888803c6d6ce-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.572 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.572 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.572 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.572 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.572 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.572 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.572 DEBUG:boto:Data: 2024-04-16T18:26:44.572 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.572 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.572 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.572 DEBUG:boto:Params: {} 2024-04-16T18:26:44.573 DEBUG:boto:Token: None 2024-04-16T18:26:44.573 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l4EZkLPvJQkwQNqWDF+6uLNt788= 2024-04-16T18:26:44.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l4EZkLPvJQkwQNqWDF+6uLNt788=', 'Content-Length': '0'} 2024-04-16T18:26:44.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3939b7d713ff6c2-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.576 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:26:44.576 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.576 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.576 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.576 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.576 DEBUG:boto:Method: GET 2024-04-16T18:26:44.576 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.576 DEBUG:boto:Data: 2024-04-16T18:26:44.576 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.577 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.577 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.577 DEBUG:boto:Params: {} 2024-04-16T18:26:44.577 DEBUG:boto:Token: None 2024-04-16T18:26:44.577 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.577 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI= 2024-04-16T18:26:44.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI=', 'Content-Length': '0'} 2024-04-16T18:26:44.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efae1b28e03d9cd6-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.622 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.623 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.623 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.623 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.623 DEBUG:boto:Method: GET 2024-04-16T18:26:44.623 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.623 DEBUG:boto:Data: 2024-04-16T18:26:44.623 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.623 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.623 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.623 DEBUG:boto:Params: {} 2024-04-16T18:26:44.623 DEBUG:boto:Token: None 2024-04-16T18:26:44.623 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:26:44.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI= 2024-04-16T18:26:44.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gMnIJilZhYM7moc7RUx1O4r9kXI=', 'Content-Length': '0'} 2024-04-16T18:26:44.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002223e412bc160b29-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.627 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.627 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.627 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.627 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.627 DEBUG:boto:Method: GET 2024-04-16T18:26:44.627 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:26:44.627 DEBUG:boto:Data: 2024-04-16T18:26:44.628 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.628 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.628 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.628 DEBUG:boto:Params: {} 2024-04-16T18:26:44.628 DEBUG:boto:Token: None 2024-04-16T18:26:44.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d333c7260be36c1c-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.632 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.632 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.632 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.633 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.633 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.633 DEBUG:boto:Method: GET 2024-04-16T18:26:44.633 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:26:44.633 DEBUG:boto:Data: 2024-04-16T18:26:44.633 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.633 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.633 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.633 DEBUG:boto:Params: {} 2024-04-16T18:26:44.633 DEBUG:boto:Token: None 2024-04-16T18:26:44.633 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.634 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009eb9caccf47df49c-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.670 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.670 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.670 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.670 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.671 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:26:44.671 DEBUG:boto:Data: 2024-04-16T18:26:44.671 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.671 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.671 DEBUG:boto:Params: {} 2024-04-16T18:26:44.671 DEBUG:boto:Token: None 2024-04-16T18:26:44.671 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33 2024-04-16T18:26:44.671 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LvRSQpJ66eroemm/yquhj83GqtQ= 2024-04-16T18:26:44.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LvRSQpJ66eroemm/yquhj83GqtQ=', 'Content-Length': '0'} 2024-04-16T18:26:44.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005c58072bbd2002a-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.674 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.674 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.674 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.675 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:26:44.675 DEBUG:boto:Data: 2024-04-16T18:26:44.675 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.675 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.675 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.675 DEBUG:boto:Params: {} 2024-04-16T18:26:44.675 DEBUG:boto:Token: None 2024-04-16T18:26:44.675 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33 2024-04-16T18:26:44.675 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LvRSQpJ66eroemm/yquhj83GqtQ= 2024-04-16T18:26:44.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LvRSQpJ66eroemm/yquhj83GqtQ=', 'Content-Length': '0'} 2024-04-16T18:26:44.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052c2dec224530590-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.678 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:26:44.679 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.679 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.679 DEBUG:boto:Method: GET 2024-04-16T18:26:44.679 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:26:44.679 DEBUG:boto:Data: 2024-04-16T18:26:44.679 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.679 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.679 DEBUG:boto:Params: {} 2024-04-16T18:26:44.679 DEBUG:boto:Token: None 2024-04-16T18:26:44.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33 2024-04-16T18:26:44.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NXpZoj/XCSOqYRq/bXVmsHwnnjw= 2024-04-16T18:26:44.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NXpZoj/XCSOqYRq/bXVmsHwnnjw=', 'Content-Length': '0'} 2024-04-16T18:26:44.682 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1543dc9ddfafd79-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.722 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:26:44.723 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:26:44.723 DEBUG:boto:Method: GET 2024-04-16T18:26:44.723 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:26:44.723 DEBUG:boto:Data: 2024-04-16T18:26:44.723 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.723 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.723 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.723 DEBUG:boto:Params: {} 2024-04-16T18:26:44.723 DEBUG:boto:Token: None 2024-04-16T18:26:44.723 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj33 2024-04-16T18:26:44.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NXpZoj/XCSOqYRq/bXVmsHwnnjw= 2024-04-16T18:26:44.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NXpZoj/XCSOqYRq/bXVmsHwnnjw=', 'Content-Length': '0'} 2024-04-16T18:26:44.726 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e46fd1eec0c376e4-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.727 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.727 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.727 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.727 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.727 DEBUG:boto:Method: GET 2024-04-16T18:26:44.727 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:26:44.727 DEBUG:boto:Data: 2024-04-16T18:26:44.727 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.727 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.728 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.728 DEBUG:boto:Params: {} 2024-04-16T18:26:44.728 DEBUG:boto:Token: None 2024-04-16T18:26:44.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.731 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006bb73f5b6865d9ae-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.731 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.732 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.732 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.732 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.732 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:26:44.732 DEBUG:boto:Method: GET 2024-04-16T18:26:44.732 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:26:44.732 DEBUG:boto:Data: 2024-04-16T18:26:44.732 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.732 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.733 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.733 DEBUG:boto:Params: {} 2024-04-16T18:26:44.733 DEBUG:boto:Token: None 2024-04-16T18:26:44.733 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.739 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000539560b819ad7984-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.782 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.783 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:26:44.783 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.783 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.783 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.783 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.783 DEBUG:boto:Method: GET 2024-04-16T18:26:44.783 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.783 DEBUG:boto:Data: 2024-04-16T18:26:44.784 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.784 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.784 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.784 DEBUG:boto:Params: {} 2024-04-16T18:26:44.784 DEBUG:boto:Token: None 2024-04-16T18:26:44.784 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4= 2024-04-16T18:26:44.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4=', 'Content-Length': '0'} 2024-04-16T18:26:44.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000542ffa20e2de8d7d-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.830 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.831 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.831 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.831 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.831 DEBUG:boto:Method: GET 2024-04-16T18:26:44.831 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.831 DEBUG:boto:Data: 2024-04-16T18:26:44.831 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.831 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.831 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.831 DEBUG:boto:Params: {} 2024-04-16T18:26:44.831 DEBUG:boto:Token: None 2024-04-16T18:26:44.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4= 2024-04-16T18:26:44.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4=', 'Content-Length': '0'} 2024-04-16T18:26:44.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8c6012b1bd811bc-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.835 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.835 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.835 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.835 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.835 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.835 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.836 DEBUG:boto:Data: 2024-04-16T18:26:44.836 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.836 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.836 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.836 DEBUG:boto:Params: {} 2024-04-16T18:26:44.836 DEBUG:boto:Token: None 2024-04-16T18:26:44.836 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+lUQzYVZ0ODvrtKALuhjRf5BWA0= 2024-04-16T18:26:44.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+lUQzYVZ0ODvrtKALuhjRf5BWA0=', 'Content-Length': '0'} 2024-04-16T18:26:44.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084da8b6fbc88b08b-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.839 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.839 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.840 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.840 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.840 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.840 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.840 DEBUG:boto:Data: 2024-04-16T18:26:44.840 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.840 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.840 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.840 DEBUG:boto:Params: {} 2024-04-16T18:26:44.840 DEBUG:boto:Token: None 2024-04-16T18:26:44.840 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+lUQzYVZ0ODvrtKALuhjRf5BWA0= 2024-04-16T18:26:44.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+lUQzYVZ0ODvrtKALuhjRf5BWA0=', 'Content-Length': '0'} 2024-04-16T18:26:44.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0b71dbc69292281-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.843 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:26:44.844 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.844 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.844 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.844 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.844 DEBUG:boto:Method: GET 2024-04-16T18:26:44.844 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.844 DEBUG:boto:Data: 2024-04-16T18:26:44.844 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.844 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.844 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.844 DEBUG:boto:Params: {} 2024-04-16T18:26:44.845 DEBUG:boto:Token: None 2024-04-16T18:26:44.845 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.845 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4= 2024-04-16T18:26:44.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4=', 'Content-Length': '0'} 2024-04-16T18:26:44.848 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019de36ed3057c556-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.894 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.894 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.895 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.895 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.895 DEBUG:boto:Method: GET 2024-04-16T18:26:44.895 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.895 DEBUG:boto:Data: 2024-04-16T18:26:44.895 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.895 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.895 DEBUG:boto:Params: {} 2024-04-16T18:26:44.895 DEBUG:boto:Token: None 2024-04-16T18:26:44.895 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:26:44.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4= 2024-04-16T18:26:44.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6C8BLLGWnn6jjw2Z0KBJ8j+6qQ4=', 'Content-Length': '0'} 2024-04-16T18:26:44.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b152689e6d5e1c64-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.899 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.899 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.899 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.899 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.899 DEBUG:boto:Method: GET 2024-04-16T18:26:44.899 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:26:44.899 DEBUG:boto:Data: 2024-04-16T18:26:44.899 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.899 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.900 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.900 DEBUG:boto:Params: {} 2024-04-16T18:26:44.900 DEBUG:boto:Token: None 2024-04-16T18:26:44.900 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.903 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8474dfb41880e01-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.904 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.904 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.904 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.904 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.904 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.904 DEBUG:boto:Method: GET 2024-04-16T18:26:44.904 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:26:44.905 DEBUG:boto:Data: 2024-04-16T18:26:44.905 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.905 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.905 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.905 DEBUG:boto:Params: {} 2024-04-16T18:26:44.905 DEBUG:boto:Token: None 2024-04-16T18:26:44.905 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.905 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.912 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005456061b65ff338d-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.912 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.912 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.912 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.912 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.912 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:26:44.912 DEBUG:boto:Data: 2024-04-16T18:26:44.912 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.913 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.913 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.913 DEBUG:boto:Params: {} 2024-04-16T18:26:44.913 DEBUG:boto:Token: None 2024-04-16T18:26:44.913 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34 2024-04-16T18:26:44.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XUL2qanQLoi5KZa6Wi8ht2URmwQ= 2024-04-16T18:26:44.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XUL2qanQLoi5KZa6Wi8ht2URmwQ=', 'Content-Length': '0'} 2024-04-16T18:26:44.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054ec4922f5bb6ad2-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.916 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.916 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.916 DEBUG:boto:Method: HEAD 2024-04-16T18:26:44.917 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:26:44.917 DEBUG:boto:Data: 2024-04-16T18:26:44.917 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.917 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.917 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.917 DEBUG:boto:Params: {} 2024-04-16T18:26:44.917 DEBUG:boto:Token: None 2024-04-16T18:26:44.917 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34 2024-04-16T18:26:44.917 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XUL2qanQLoi5KZa6Wi8ht2URmwQ= 2024-04-16T18:26:44.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XUL2qanQLoi5KZa6Wi8ht2URmwQ=', 'Content-Length': '0'} 2024-04-16T18:26:44.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc5eccacf6278602-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:26:44.921 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.921 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.921 DEBUG:boto:Method: GET 2024-04-16T18:26:44.921 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:26:44.921 DEBUG:boto:Data: 2024-04-16T18:26:44.921 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.921 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.921 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.921 DEBUG:boto:Params: {} 2024-04-16T18:26:44.922 DEBUG:boto:Token: None 2024-04-16T18:26:44.922 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34 2024-04-16T18:26:44.922 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o9OnJTvh6AQCIGh09lNr9LzSjgU= 2024-04-16T18:26:44.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o9OnJTvh6AQCIGh09lNr9LzSjgU=', 'Content-Length': '0'} 2024-04-16T18:26:44.925 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020cfeb5be18a7fe6-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.966 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:26:44.967 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:26:44.967 DEBUG:boto:Method: GET 2024-04-16T18:26:44.967 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:26:44.967 DEBUG:boto:Data: 2024-04-16T18:26:44.967 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.967 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.967 DEBUG:boto:Params: {} 2024-04-16T18:26:44.968 DEBUG:boto:Token: None 2024-04-16T18:26:44.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj34 2024-04-16T18:26:44.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o9OnJTvh6AQCIGh09lNr9LzSjgU= 2024-04-16T18:26:44.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o9OnJTvh6AQCIGh09lNr9LzSjgU=', 'Content-Length': '0'} 2024-04-16T18:26:44.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7c9941ce71304cd-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.971 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.971 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.971 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.971 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.971 DEBUG:boto:Method: GET 2024-04-16T18:26:44.971 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:26:44.972 DEBUG:boto:Data: 2024-04-16T18:26:44.972 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.972 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.972 DEBUG:boto:Params: {} 2024-04-16T18:26:44.972 DEBUG:boto:Token: None 2024-04-16T18:26:44.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d71ee60efeab9a7b-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.976 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.976 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:44.977 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:44.977 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.977 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:26:44.977 DEBUG:boto:Method: GET 2024-04-16T18:26:44.977 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:26:44.977 DEBUG:boto:Data: 2024-04-16T18:26:44.977 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.977 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:44.977 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.977 DEBUG:boto:Params: {} 2024-04-16T18:26:44.977 DEBUG:boto:Token: None 2024-04-16T18:26:44.978 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/ 2024-04-16T18:26:44.978 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs= 2024-04-16T18:26:44.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:atRAzLJ+/jybVINYk4QbGIuCQGs=', 'Content-Length': '0'} 2024-04-16T18:26:44.984 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003115e1b6e2d4998b-00661ec2e4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:44.984 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:44.985 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:26:44.985 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:44.985 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:44.985 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:44.985 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:44.985 DEBUG:boto:Method: GET 2024-04-16T18:26:44.985 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:26:44.985 DEBUG:boto:Data: 2024-04-16T18:26:44.985 DEBUG:boto:Headers: {} 2024-04-16T18:26:44.985 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:44.986 DEBUG:boto:Port: 8000 2024-04-16T18:26:44.986 DEBUG:boto:Params: {} 2024-04-16T18:26:44.986 DEBUG:boto:Token: None 2024-04-16T18:26:44.986 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:44 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:26:44.986 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BIW+QGAu1ZfT9R1nxC+AMpqV4D0= 2024-04-16T18:26:44.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BIW+QGAu1ZfT9R1nxC+AMpqV4D0=', 'Content-Length': '0'} 2024-04-16T18:26:44.989 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053df4f3f041b81d6-00661ec2e4-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.030 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.030 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.031 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.031 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.031 DEBUG:boto:Method: GET 2024-04-16T18:26:45.031 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.031 DEBUG:boto:Data: 2024-04-16T18:26:45.031 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.031 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.031 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.031 DEBUG:boto:Params: {} 2024-04-16T18:26:45.031 DEBUG:boto:Token: None 2024-04-16T18:26:45.032 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gKS7dInhqMqZ7FxXrKHWk3K0oC0= 2024-04-16T18:26:45.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gKS7dInhqMqZ7FxXrKHWk3K0oC0=', 'Content-Length': '0'} 2024-04-16T18:26:45.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052903e05ad70195f-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.035 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.035 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.035 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.035 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.036 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.036 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.036 DEBUG:boto:Data: 2024-04-16T18:26:45.036 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.036 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.036 DEBUG:boto:Params: {} 2024-04-16T18:26:45.036 DEBUG:boto:Token: None 2024-04-16T18:26:45.036 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v95SrJQw7nb0VxbFuXqsPfL+CCI= 2024-04-16T18:26:45.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v95SrJQw7nb0VxbFuXqsPfL+CCI=', 'Content-Length': '0'} 2024-04-16T18:26:45.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048d6655c43d8b1e6-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.040 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.040 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.040 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.040 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.040 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.040 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.040 DEBUG:boto:Data: 2024-04-16T18:26:45.040 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.040 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.040 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.041 DEBUG:boto:Params: {} 2024-04-16T18:26:45.041 DEBUG:boto:Token: None 2024-04-16T18:26:45.041 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.041 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v95SrJQw7nb0VxbFuXqsPfL+CCI= 2024-04-16T18:26:45.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v95SrJQw7nb0VxbFuXqsPfL+CCI=', 'Content-Length': '0'} 2024-04-16T18:26:45.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f1cdff6a4f63f17-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.044 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:26:45.044 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.044 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.044 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.044 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.044 DEBUG:boto:Method: GET 2024-04-16T18:26:45.044 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.045 DEBUG:boto:Data: 2024-04-16T18:26:45.045 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.045 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.045 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.045 DEBUG:boto:Params: {} 2024-04-16T18:26:45.045 DEBUG:boto:Token: None 2024-04-16T18:26:45.045 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.045 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gKS7dInhqMqZ7FxXrKHWk3K0oC0= 2024-04-16T18:26:45.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gKS7dInhqMqZ7FxXrKHWk3K0oC0=', 'Content-Length': '0'} 2024-04-16T18:26:45.048 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d050ae5180897e39-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.090 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.091 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.091 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.091 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.091 DEBUG:boto:Method: GET 2024-04-16T18:26:45.091 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.091 DEBUG:boto:Data: 2024-04-16T18:26:45.091 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.091 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.091 DEBUG:boto:Params: {} 2024-04-16T18:26:45.092 DEBUG:boto:Token: None 2024-04-16T18:26:45.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:26:45.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gKS7dInhqMqZ7FxXrKHWk3K0oC0= 2024-04-16T18:26:45.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gKS7dInhqMqZ7FxXrKHWk3K0oC0=', 'Content-Length': '0'} 2024-04-16T18:26:45.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a23ef49d1d820fb-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.096 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.096 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.096 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.096 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.096 DEBUG:boto:Method: GET 2024-04-16T18:26:45.096 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:26:45.096 DEBUG:boto:Data: 2024-04-16T18:26:45.096 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.096 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.096 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.096 DEBUG:boto:Params: {} 2024-04-16T18:26:45.097 DEBUG:boto:Token: None 2024-04-16T18:26:45.097 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.101 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021ea15e616dfac91-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.101 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.101 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.102 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.102 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.102 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.102 DEBUG:boto:Method: GET 2024-04-16T18:26:45.102 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:26:45.102 DEBUG:boto:Data: 2024-04-16T18:26:45.102 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.102 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.102 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.103 DEBUG:boto:Params: {} 2024-04-16T18:26:45.103 DEBUG:boto:Token: None 2024-04-16T18:26:45.103 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.103 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.109 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec5bd40c501175b1-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.150 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.151 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.151 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.151 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.151 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:26:45.151 DEBUG:boto:Data: 2024-04-16T18:26:45.151 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.151 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.152 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.152 DEBUG:boto:Params: {} 2024-04-16T18:26:45.152 DEBUG:boto:Token: None 2024-04-16T18:26:45.152 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38 2024-04-16T18:26:45.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M33Z2TADx0hRGGReWx+jdXhKp1o= 2024-04-16T18:26:45.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M33Z2TADx0hRGGReWx+jdXhKp1o=', 'Content-Length': '0'} 2024-04-16T18:26:45.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067d5866fc7859d7a-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.156 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.156 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.156 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.156 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:26:45.156 DEBUG:boto:Data: 2024-04-16T18:26:45.156 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.156 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.156 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.156 DEBUG:boto:Params: {} 2024-04-16T18:26:45.156 DEBUG:boto:Token: None 2024-04-16T18:26:45.157 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38 2024-04-16T18:26:45.157 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M33Z2TADx0hRGGReWx+jdXhKp1o= 2024-04-16T18:26:45.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M33Z2TADx0hRGGReWx+jdXhKp1o=', 'Content-Length': '0'} 2024-04-16T18:26:45.160 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d231b4e531f2e149-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.160 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:26:45.160 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.160 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.161 DEBUG:boto:Method: GET 2024-04-16T18:26:45.161 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:26:45.161 DEBUG:boto:Data: 2024-04-16T18:26:45.161 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.161 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.161 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.161 DEBUG:boto:Params: {} 2024-04-16T18:26:45.161 DEBUG:boto:Token: None 2024-04-16T18:26:45.161 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38 2024-04-16T18:26:45.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vRlaHKEuEI37IukQiV+Kh13K5Gw= 2024-04-16T18:26:45.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vRlaHKEuEI37IukQiV+Kh13K5Gw=', 'Content-Length': '0'} 2024-04-16T18:26:45.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000116ffc5bd8825a21-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.206 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:26:45.207 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:26:45.207 DEBUG:boto:Method: GET 2024-04-16T18:26:45.207 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:26:45.207 DEBUG:boto:Data: 2024-04-16T18:26:45.207 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.207 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.207 DEBUG:boto:Params: {} 2024-04-16T18:26:45.207 DEBUG:boto:Token: None 2024-04-16T18:26:45.208 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj38 2024-04-16T18:26:45.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vRlaHKEuEI37IukQiV+Kh13K5Gw= 2024-04-16T18:26:45.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vRlaHKEuEI37IukQiV+Kh13K5Gw=', 'Content-Length': '0'} 2024-04-16T18:26:45.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a4644328d37f7e2-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.211 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.212 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.212 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.212 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.212 DEBUG:boto:Method: GET 2024-04-16T18:26:45.212 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:26:45.212 DEBUG:boto:Data: 2024-04-16T18:26:45.212 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.212 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.212 DEBUG:boto:Params: {} 2024-04-16T18:26:45.213 DEBUG:boto:Token: None 2024-04-16T18:26:45.213 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6f0e23385657c68-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.258 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.259 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.259 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.259 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.259 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:26:45.259 DEBUG:boto:Method: GET 2024-04-16T18:26:45.259 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:26:45.260 DEBUG:boto:Data: 2024-04-16T18:26:45.260 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.260 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.260 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.260 DEBUG:boto:Params: {} 2024-04-16T18:26:45.260 DEBUG:boto:Token: None 2024-04-16T18:26:45.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c018093dd3b4528-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.269 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.269 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:26:45.269 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.269 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.269 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.270 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.270 DEBUG:boto:Method: GET 2024-04-16T18:26:45.270 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.270 DEBUG:boto:Data: 2024-04-16T18:26:45.270 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.270 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.270 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.270 DEBUG:boto:Params: {} 2024-04-16T18:26:45.270 DEBUG:boto:Token: None 2024-04-16T18:26:45.271 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.271 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs= 2024-04-16T18:26:45.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs=', 'Content-Length': '0'} 2024-04-16T18:26:45.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003103f571b1bebce1-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.314 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.315 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.315 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.315 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.315 DEBUG:boto:Method: GET 2024-04-16T18:26:45.315 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.315 DEBUG:boto:Data: 2024-04-16T18:26:45.315 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.315 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.315 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.315 DEBUG:boto:Params: {} 2024-04-16T18:26:45.315 DEBUG:boto:Token: None 2024-04-16T18:26:45.316 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs= 2024-04-16T18:26:45.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs=', 'Content-Length': '0'} 2024-04-16T18:26:45.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026e2a2ff36ec45e4-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.320 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.320 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.320 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.320 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.320 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.320 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.320 DEBUG:boto:Data: 2024-04-16T18:26:45.320 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.320 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.321 DEBUG:boto:Params: {} 2024-04-16T18:26:45.321 DEBUG:boto:Token: None 2024-04-16T18:26:45.321 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eOQMTVmjabthXeGofX7as46krZY= 2024-04-16T18:26:45.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eOQMTVmjabthXeGofX7as46krZY=', 'Content-Length': '0'} 2024-04-16T18:26:45.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067e21f46bd5fd0ad-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.324 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.325 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.325 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.325 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.325 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.325 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.325 DEBUG:boto:Data: 2024-04-16T18:26:45.325 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.325 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.325 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.325 DEBUG:boto:Params: {} 2024-04-16T18:26:45.326 DEBUG:boto:Token: None 2024-04-16T18:26:45.326 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.326 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eOQMTVmjabthXeGofX7as46krZY= 2024-04-16T18:26:45.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eOQMTVmjabthXeGofX7as46krZY=', 'Content-Length': '0'} 2024-04-16T18:26:45.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6f86a3202291152-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.330 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:26:45.330 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.330 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.330 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.330 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.330 DEBUG:boto:Method: GET 2024-04-16T18:26:45.330 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.331 DEBUG:boto:Data: 2024-04-16T18:26:45.331 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.331 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.331 DEBUG:boto:Params: {} 2024-04-16T18:26:45.331 DEBUG:boto:Token: None 2024-04-16T18:26:45.331 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.331 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs= 2024-04-16T18:26:45.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs=', 'Content-Length': '0'} 2024-04-16T18:26:45.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ab2dc305e61269d-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.378 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.379 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.379 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.379 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.379 DEBUG:boto:Method: GET 2024-04-16T18:26:45.379 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.379 DEBUG:boto:Data: 2024-04-16T18:26:45.379 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.379 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.379 DEBUG:boto:Params: {} 2024-04-16T18:26:45.379 DEBUG:boto:Token: None 2024-04-16T18:26:45.380 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:26:45.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs= 2024-04-16T18:26:45.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w4NZiXuPQziqH/Vet9LEmjNepbs=', 'Content-Length': '0'} 2024-04-16T18:26:45.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d763af2ac57710da-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.384 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.384 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.384 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.384 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.384 DEBUG:boto:Method: GET 2024-04-16T18:26:45.384 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:26:45.384 DEBUG:boto:Data: 2024-04-16T18:26:45.384 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.384 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.384 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.385 DEBUG:boto:Params: {} 2024-04-16T18:26:45.385 DEBUG:boto:Token: None 2024-04-16T18:26:45.385 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.385 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.389 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dde2c779afa48b5a-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.389 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.390 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.390 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.390 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.390 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.390 DEBUG:boto:Method: GET 2024-04-16T18:26:45.390 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:26:45.390 DEBUG:boto:Data: 2024-04-16T18:26:45.390 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.391 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.391 DEBUG:boto:Params: {} 2024-04-16T18:26:45.391 DEBUG:boto:Token: None 2024-04-16T18:26:45.391 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.391 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.397 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b761c1e80886c31-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.438 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.439 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.439 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.439 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.439 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:26:45.439 DEBUG:boto:Data: 2024-04-16T18:26:45.439 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.439 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.439 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.439 DEBUG:boto:Params: {} 2024-04-16T18:26:45.440 DEBUG:boto:Token: None 2024-04-16T18:26:45.440 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44 2024-04-16T18:26:45.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xs6X59I3cgAS8lL94U2kUDPuldQ= 2024-04-16T18:26:45.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xs6X59I3cgAS8lL94U2kUDPuldQ=', 'Content-Length': '0'} 2024-04-16T18:26:45.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004357781139bab395-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.443 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.443 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.444 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.444 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:26:45.444 DEBUG:boto:Data: 2024-04-16T18:26:45.444 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.444 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.444 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.444 DEBUG:boto:Params: {} 2024-04-16T18:26:45.444 DEBUG:boto:Token: None 2024-04-16T18:26:45.444 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44 2024-04-16T18:26:45.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xs6X59I3cgAS8lL94U2kUDPuldQ= 2024-04-16T18:26:45.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xs6X59I3cgAS8lL94U2kUDPuldQ=', 'Content-Length': '0'} 2024-04-16T18:26:45.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a20924b1046de18f-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.448 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:26:45.448 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.448 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.448 DEBUG:boto:Method: GET 2024-04-16T18:26:45.448 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:26:45.448 DEBUG:boto:Data: 2024-04-16T18:26:45.449 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.449 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.449 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.449 DEBUG:boto:Params: {} 2024-04-16T18:26:45.449 DEBUG:boto:Token: None 2024-04-16T18:26:45.449 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44 2024-04-16T18:26:45.449 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YNoY8UZgivSOYgNywfmnYPyW9Qo= 2024-04-16T18:26:45.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YNoY8UZgivSOYgNywfmnYPyW9Qo=', 'Content-Length': '0'} 2024-04-16T18:26:45.453 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4d30b12c3afb9c1-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.494 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:26:45.495 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:26:45.495 DEBUG:boto:Method: GET 2024-04-16T18:26:45.495 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:26:45.495 DEBUG:boto:Data: 2024-04-16T18:26:45.495 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.495 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.495 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.495 DEBUG:boto:Params: {} 2024-04-16T18:26:45.495 DEBUG:boto:Token: None 2024-04-16T18:26:45.495 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj44 2024-04-16T18:26:45.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YNoY8UZgivSOYgNywfmnYPyW9Qo= 2024-04-16T18:26:45.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YNoY8UZgivSOYgNywfmnYPyW9Qo=', 'Content-Length': '0'} 2024-04-16T18:26:45.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eba8bec8f86bc4f6-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.499 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.500 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.500 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.500 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.500 DEBUG:boto:Method: GET 2024-04-16T18:26:45.500 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:26:45.500 DEBUG:boto:Data: 2024-04-16T18:26:45.500 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.500 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.500 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.500 DEBUG:boto:Params: {} 2024-04-16T18:26:45.501 DEBUG:boto:Token: None 2024-04-16T18:26:45.501 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cc4d99a4c42be62-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.546 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.547 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.547 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.547 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.547 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:26:45.547 DEBUG:boto:Method: GET 2024-04-16T18:26:45.547 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:26:45.547 DEBUG:boto:Data: 2024-04-16T18:26:45.547 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.548 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.548 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.548 DEBUG:boto:Params: {} 2024-04-16T18:26:45.548 DEBUG:boto:Token: None 2024-04-16T18:26:45.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1c85151a5879dba-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.555 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.555 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:26:45.555 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.555 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.555 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.556 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.556 DEBUG:boto:Method: GET 2024-04-16T18:26:45.556 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.556 DEBUG:boto:Data: 2024-04-16T18:26:45.556 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.556 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.556 DEBUG:boto:Params: {} 2024-04-16T18:26:45.556 DEBUG:boto:Token: None 2024-04-16T18:26:45.557 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.557 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc= 2024-04-16T18:26:45.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc=', 'Content-Length': '0'} 2024-04-16T18:26:45.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9ad7f64945a9fe8-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.602 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.602 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.603 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.603 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.603 DEBUG:boto:Method: GET 2024-04-16T18:26:45.603 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.603 DEBUG:boto:Data: 2024-04-16T18:26:45.603 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.603 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.603 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.603 DEBUG:boto:Params: {} 2024-04-16T18:26:45.603 DEBUG:boto:Token: None 2024-04-16T18:26:45.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc= 2024-04-16T18:26:45.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc=', 'Content-Length': '0'} 2024-04-16T18:26:45.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009bf8288d127e09a-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.607 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.608 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.608 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.608 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.608 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.608 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.608 DEBUG:boto:Data: 2024-04-16T18:26:45.608 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.608 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.608 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.608 DEBUG:boto:Params: {} 2024-04-16T18:26:45.609 DEBUG:boto:Token: None 2024-04-16T18:26:45.609 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.609 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gYafk9+AnFrHS67+uR6QTtDFmEo= 2024-04-16T18:26:45.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gYafk9+AnFrHS67+uR6QTtDFmEo=', 'Content-Length': '0'} 2024-04-16T18:26:45.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000988560f1572c024c-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.612 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.612 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.612 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.612 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.612 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.613 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.613 DEBUG:boto:Data: 2024-04-16T18:26:45.613 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.613 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.613 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.613 DEBUG:boto:Params: {} 2024-04-16T18:26:45.613 DEBUG:boto:Token: None 2024-04-16T18:26:45.613 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.613 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gYafk9+AnFrHS67+uR6QTtDFmEo= 2024-04-16T18:26:45.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gYafk9+AnFrHS67+uR6QTtDFmEo=', 'Content-Length': '0'} 2024-04-16T18:26:45.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000857d9b985cf01667-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.617 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:26:45.617 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.617 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.617 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.617 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.617 DEBUG:boto:Method: GET 2024-04-16T18:26:45.618 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.618 DEBUG:boto:Data: 2024-04-16T18:26:45.618 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.618 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.618 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.618 DEBUG:boto:Params: {} 2024-04-16T18:26:45.618 DEBUG:boto:Token: None 2024-04-16T18:26:45.618 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.618 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc= 2024-04-16T18:26:45.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc=', 'Content-Length': '0'} 2024-04-16T18:26:45.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac38879335ba2755-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.662 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.663 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.663 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.663 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.663 DEBUG:boto:Method: GET 2024-04-16T18:26:45.663 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.663 DEBUG:boto:Data: 2024-04-16T18:26:45.663 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.663 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.663 DEBUG:boto:Params: {} 2024-04-16T18:26:45.663 DEBUG:boto:Token: None 2024-04-16T18:26:45.664 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:26:45.665 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc= 2024-04-16T18:26:45.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lD/BIuqsDxfAofVxEVQj60HgWdc=', 'Content-Length': '0'} 2024-04-16T18:26:45.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e0c05b1b664d075-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.668 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.668 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.668 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.668 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.669 DEBUG:boto:Method: GET 2024-04-16T18:26:45.669 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:26:45.669 DEBUG:boto:Data: 2024-04-16T18:26:45.669 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.669 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.669 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.669 DEBUG:boto:Params: {} 2024-04-16T18:26:45.669 DEBUG:boto:Token: None 2024-04-16T18:26:45.669 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.673 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055fbf8c94b26658a-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.673 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.674 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.674 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.674 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.674 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.674 DEBUG:boto:Method: GET 2024-04-16T18:26:45.674 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:26:45.674 DEBUG:boto:Data: 2024-04-16T18:26:45.674 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.674 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.674 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.674 DEBUG:boto:Params: {} 2024-04-16T18:26:45.675 DEBUG:boto:Token: None 2024-04-16T18:26:45.675 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.675 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.682 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1bb5abbff99d5f6-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.722 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.723 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.723 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.723 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.723 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:26:45.723 DEBUG:boto:Data: 2024-04-16T18:26:45.723 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.723 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.723 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.723 DEBUG:boto:Params: {} 2024-04-16T18:26:45.723 DEBUG:boto:Token: None 2024-04-16T18:26:45.724 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46 2024-04-16T18:26:45.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:74o++jYlodhs3o9H9JE4EsVywL0= 2024-04-16T18:26:45.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:74o++jYlodhs3o9H9JE4EsVywL0=', 'Content-Length': '0'} 2024-04-16T18:26:45.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fbb0f18c4cd809b-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.727 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.727 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.727 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.727 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:26:45.727 DEBUG:boto:Data: 2024-04-16T18:26:45.727 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.727 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.727 DEBUG:boto:Params: {} 2024-04-16T18:26:45.728 DEBUG:boto:Token: None 2024-04-16T18:26:45.728 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46 2024-04-16T18:26:45.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:74o++jYlodhs3o9H9JE4EsVywL0= 2024-04-16T18:26:45.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:74o++jYlodhs3o9H9JE4EsVywL0=', 'Content-Length': '0'} 2024-04-16T18:26:45.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d70ad188b25e204-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.732 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:26:45.732 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.732 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.732 DEBUG:boto:Method: GET 2024-04-16T18:26:45.732 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:26:45.732 DEBUG:boto:Data: 2024-04-16T18:26:45.732 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.732 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.732 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.732 DEBUG:boto:Params: {} 2024-04-16T18:26:45.732 DEBUG:boto:Token: None 2024-04-16T18:26:45.733 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46 2024-04-16T18:26:45.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jGXqQwxMb+egCRPROK7exhNTEsg= 2024-04-16T18:26:45.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jGXqQwxMb+egCRPROK7exhNTEsg=', 'Content-Length': '0'} 2024-04-16T18:26:45.736 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006798e23e2df8d26d-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.778 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:26:45.779 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:26:45.779 DEBUG:boto:Method: GET 2024-04-16T18:26:45.779 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:26:45.779 DEBUG:boto:Data: 2024-04-16T18:26:45.779 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.779 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.779 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.779 DEBUG:boto:Params: {} 2024-04-16T18:26:45.779 DEBUG:boto:Token: None 2024-04-16T18:26:45.779 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj46 2024-04-16T18:26:45.779 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jGXqQwxMb+egCRPROK7exhNTEsg= 2024-04-16T18:26:45.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jGXqQwxMb+egCRPROK7exhNTEsg=', 'Content-Length': '0'} 2024-04-16T18:26:45.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036a9829d8ad71564-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.783 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.783 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.783 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.783 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.783 DEBUG:boto:Method: GET 2024-04-16T18:26:45.783 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:26:45.784 DEBUG:boto:Data: 2024-04-16T18:26:45.784 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.784 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.784 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.784 DEBUG:boto:Params: {} 2024-04-16T18:26:45.784 DEBUG:boto:Token: None 2024-04-16T18:26:45.784 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.787 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c3b70d653ed2295-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.830 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.831 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.831 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.831 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.831 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:26:45.831 DEBUG:boto:Method: GET 2024-04-16T18:26:45.831 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:26:45.831 DEBUG:boto:Data: 2024-04-16T18:26:45.831 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.831 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.831 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.831 DEBUG:boto:Params: {} 2024-04-16T18:26:45.831 DEBUG:boto:Token: None 2024-04-16T18:26:45.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033e52c6cc5fb5709-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.838 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.838 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:26:45.838 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:45.838 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:45.839 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.839 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.839 DEBUG:boto:Method: GET 2024-04-16T18:26:45.839 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.839 DEBUG:boto:Data: 2024-04-16T18:26:45.839 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.839 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.839 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.839 DEBUG:boto:Params: {} 2024-04-16T18:26:45.839 DEBUG:boto:Token: None 2024-04-16T18:26:45.839 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.839 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q= 2024-04-16T18:26:45.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q=', 'Content-Length': '0'} 2024-04-16T18:26:45.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b025d9fa4985e04-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.886 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:45.886 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:45.887 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.887 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.887 DEBUG:boto:Method: GET 2024-04-16T18:26:45.887 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.887 DEBUG:boto:Data: 2024-04-16T18:26:45.887 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.887 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.887 DEBUG:boto:Params: {} 2024-04-16T18:26:45.887 DEBUG:boto:Token: None 2024-04-16T18:26:45.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q= 2024-04-16T18:26:45.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q=', 'Content-Length': '0'} 2024-04-16T18:26:45.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af13136e5e77ea36-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.891 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:45.891 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:45.891 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.891 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.891 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.891 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.891 DEBUG:boto:Data: 2024-04-16T18:26:45.891 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.891 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.891 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.891 DEBUG:boto:Params: {} 2024-04-16T18:26:45.892 DEBUG:boto:Token: None 2024-04-16T18:26:45.892 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xJ7W0x4Alnr3BcnuyXaXzvar23s= 2024-04-16T18:26:45.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xJ7W0x4Alnr3BcnuyXaXzvar23s=', 'Content-Length': '0'} 2024-04-16T18:26:45.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c8f88411437faea-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.895 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:45.895 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:45.895 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.895 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.896 DEBUG:boto:Method: HEAD 2024-04-16T18:26:45.896 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.896 DEBUG:boto:Data: 2024-04-16T18:26:45.896 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.896 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.896 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.896 DEBUG:boto:Params: {} 2024-04-16T18:26:45.896 DEBUG:boto:Token: None 2024-04-16T18:26:45.896 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xJ7W0x4Alnr3BcnuyXaXzvar23s= 2024-04-16T18:26:45.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xJ7W0x4Alnr3BcnuyXaXzvar23s=', 'Content-Length': '0'} 2024-04-16T18:26:45.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009dfddb211285e9c9-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.900 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:26:45.900 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:45.900 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:45.900 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.900 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.900 DEBUG:boto:Method: GET 2024-04-16T18:26:45.900 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.900 DEBUG:boto:Data: 2024-04-16T18:26:45.900 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.901 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.901 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.901 DEBUG:boto:Params: {} 2024-04-16T18:26:45.901 DEBUG:boto:Token: None 2024-04-16T18:26:45.901 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q= 2024-04-16T18:26:45.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q=', 'Content-Length': '0'} 2024-04-16T18:26:45.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020fa24900904e6bf-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.946 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:45.947 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:45.947 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.947 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.947 DEBUG:boto:Method: GET 2024-04-16T18:26:45.947 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.947 DEBUG:boto:Data: 2024-04-16T18:26:45.947 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.947 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.947 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.947 DEBUG:boto:Params: {} 2024-04-16T18:26:45.947 DEBUG:boto:Token: None 2024-04-16T18:26:45.947 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:26:45.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q= 2024-04-16T18:26:45.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HZDTUkLGlQAMg+pLIUgHWCfJb0Q=', 'Content-Length': '0'} 2024-04-16T18:26:45.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002733e340e24a5da6-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.951 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.951 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.951 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:45.951 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:45.952 DEBUG:boto:Method: GET 2024-04-16T18:26:45.952 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:26:45.952 DEBUG:boto:Data: 2024-04-16T18:26:45.952 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.952 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:45.952 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.952 DEBUG:boto:Params: {} 2024-04-16T18:26:45.952 DEBUG:boto:Token: None 2024-04-16T18:26:45.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.956 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000592d76fe8b0a0322-00661ec2e5-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:45.956 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:45.957 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:45.957 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:45.957 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:45.957 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:45.957 DEBUG:boto:Method: GET 2024-04-16T18:26:45.957 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:26:45.957 DEBUG:boto:Data: 2024-04-16T18:26:45.957 DEBUG:boto:Headers: {} 2024-04-16T18:26:45.957 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:45.957 DEBUG:boto:Port: 8000 2024-04-16T18:26:45.958 DEBUG:boto:Params: {} 2024-04-16T18:26:45.958 DEBUG:boto:Token: None 2024-04-16T18:26:45.958 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:45 GMT /lybtug-52/ 2024-04-16T18:26:45.958 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs= 2024-04-16T18:26:45.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8FR4rb3u1pSWmRq6Bp/5A4tXtUs=', 'Content-Length': '0'} 2024-04-16T18:26:45.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a179a0e5ea8b293-00661ec2e5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.006 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.007 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:46.007 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:46.007 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.007 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:26:46.007 DEBUG:boto:Data: 2024-04-16T18:26:46.007 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.007 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.007 DEBUG:boto:Params: {} 2024-04-16T18:26:46.008 DEBUG:boto:Token: None 2024-04-16T18:26:46.008 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj51 2024-04-16T18:26:46.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C0SCmgJsnMvRb4IpB/+WaPjcyOk= 2024-04-16T18:26:46.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C0SCmgJsnMvRb4IpB/+WaPjcyOk=', 'Content-Length': '0'} 2024-04-16T18:26:46.011 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018aecae41459acdf-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.011 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:46.012 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:46.012 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.012 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:26:46.012 DEBUG:boto:Data: 2024-04-16T18:26:46.012 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.012 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.012 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.012 DEBUG:boto:Params: {} 2024-04-16T18:26:46.012 DEBUG:boto:Token: None 2024-04-16T18:26:46.012 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj51 2024-04-16T18:26:46.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:C0SCmgJsnMvRb4IpB/+WaPjcyOk= 2024-04-16T18:26:46.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C0SCmgJsnMvRb4IpB/+WaPjcyOk=', 'Content-Length': '0'} 2024-04-16T18:26:46.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a579a4919462bd7b-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.016 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:26:46.016 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:46.016 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:46.016 DEBUG:boto:Method: GET 2024-04-16T18:26:46.016 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:26:46.016 DEBUG:boto:Data: 2024-04-16T18:26:46.016 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.016 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.017 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.017 DEBUG:boto:Params: {} 2024-04-16T18:26:46.017 DEBUG:boto:Token: None 2024-04-16T18:26:46.017 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj51 2024-04-16T18:26:46.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PvuFRX1R1WZXLlLkPf71Nx0Xu6A= 2024-04-16T18:26:46.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PvuFRX1R1WZXLlLkPf71Nx0Xu6A=', 'Content-Length': '0'} 2024-04-16T18:26:46.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006202dbf84b652a76-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.062 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:26:46.063 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:26:46.063 DEBUG:boto:Method: GET 2024-04-16T18:26:46.063 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:26:46.063 DEBUG:boto:Data: 2024-04-16T18:26:46.063 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.063 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.063 DEBUG:boto:Params: {} 2024-04-16T18:26:46.063 DEBUG:boto:Token: None 2024-04-16T18:26:46.063 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj51 2024-04-16T18:26:46.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PvuFRX1R1WZXLlLkPf71Nx0Xu6A= 2024-04-16T18:26:46.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PvuFRX1R1WZXLlLkPf71Nx0Xu6A=', 'Content-Length': '0'} 2024-04-16T18:26:46.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c01783398ef5b0a5-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.067 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.067 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.067 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:46.067 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:46.067 DEBUG:boto:Method: GET 2024-04-16T18:26:46.067 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:26:46.067 DEBUG:boto:Data: 2024-04-16T18:26:46.067 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.068 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.068 DEBUG:boto:Params: {} 2024-04-16T18:26:46.068 DEBUG:boto:Token: None 2024-04-16T18:26:46.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.072 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067876284b48dfa3c-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.114 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.115 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.115 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.115 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:46.115 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:26:46.115 DEBUG:boto:Method: GET 2024-04-16T18:26:46.115 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:26:46.115 DEBUG:boto:Data: 2024-04-16T18:26:46.115 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.116 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.116 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.116 DEBUG:boto:Params: {} 2024-04-16T18:26:46.116 DEBUG:boto:Token: None 2024-04-16T18:26:46.116 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000caab59004417a44-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.123 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.123 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:26:46.123 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.124 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.124 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.124 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.124 DEBUG:boto:Method: GET 2024-04-16T18:26:46.124 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.124 DEBUG:boto:Data: 2024-04-16T18:26:46.124 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.124 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.124 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.124 DEBUG:boto:Params: {} 2024-04-16T18:26:46.125 DEBUG:boto:Token: None 2024-04-16T18:26:46.125 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.125 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w= 2024-04-16T18:26:46.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w=', 'Content-Length': '0'} 2024-04-16T18:26:46.128 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c74e4789ab3f672b-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.170 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.171 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.171 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.171 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.171 DEBUG:boto:Method: GET 2024-04-16T18:26:46.171 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.171 DEBUG:boto:Data: 2024-04-16T18:26:46.171 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.171 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.171 DEBUG:boto:Params: {} 2024-04-16T18:26:46.171 DEBUG:boto:Token: None 2024-04-16T18:26:46.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w= 2024-04-16T18:26:46.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w=', 'Content-Length': '0'} 2024-04-16T18:26:46.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000154ed1d5fe29815d-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.176 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.176 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.176 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.176 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.176 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.176 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.176 DEBUG:boto:Data: 2024-04-16T18:26:46.176 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.176 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.176 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.177 DEBUG:boto:Params: {} 2024-04-16T18:26:46.177 DEBUG:boto:Token: None 2024-04-16T18:26:46.177 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:twgU5K2aL3U37MuRvnz6SlJPE9M= 2024-04-16T18:26:46.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:twgU5K2aL3U37MuRvnz6SlJPE9M=', 'Content-Length': '0'} 2024-04-16T18:26:46.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001185c91c8697954-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.180 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.180 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.180 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.180 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.180 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.181 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.181 DEBUG:boto:Data: 2024-04-16T18:26:46.181 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.181 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.181 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.181 DEBUG:boto:Params: {} 2024-04-16T18:26:46.181 DEBUG:boto:Token: None 2024-04-16T18:26:46.181 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.181 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:twgU5K2aL3U37MuRvnz6SlJPE9M= 2024-04-16T18:26:46.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:twgU5K2aL3U37MuRvnz6SlJPE9M=', 'Content-Length': '0'} 2024-04-16T18:26:46.185 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c3e1d1c52dc5b22-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.185 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:26:46.185 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.185 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.185 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.185 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.185 DEBUG:boto:Method: GET 2024-04-16T18:26:46.185 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.186 DEBUG:boto:Data: 2024-04-16T18:26:46.186 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.186 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.186 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.186 DEBUG:boto:Params: {} 2024-04-16T18:26:46.186 DEBUG:boto:Token: None 2024-04-16T18:26:46.186 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.186 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w= 2024-04-16T18:26:46.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w=', 'Content-Length': '0'} 2024-04-16T18:26:46.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9e1b7181bf3f6f9-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.230 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.231 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.231 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.231 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.231 DEBUG:boto:Method: GET 2024-04-16T18:26:46.231 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.231 DEBUG:boto:Data: 2024-04-16T18:26:46.231 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.231 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.231 DEBUG:boto:Params: {} 2024-04-16T18:26:46.232 DEBUG:boto:Token: None 2024-04-16T18:26:46.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:26:46.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w= 2024-04-16T18:26:46.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vzPQrsDdOMCJxtamSrs7r8OBf5w=', 'Content-Length': '0'} 2024-04-16T18:26:46.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bee06b2003bb5cf-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.235 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.235 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.235 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.235 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.236 DEBUG:boto:Method: GET 2024-04-16T18:26:46.236 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:26:46.236 DEBUG:boto:Data: 2024-04-16T18:26:46.236 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.236 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.236 DEBUG:boto:Params: {} 2024-04-16T18:26:46.236 DEBUG:boto:Token: None 2024-04-16T18:26:46.236 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d944db6d0a36b7a-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.240 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.241 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.241 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.241 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.241 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.241 DEBUG:boto:Method: GET 2024-04-16T18:26:46.241 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:26:46.241 DEBUG:boto:Data: 2024-04-16T18:26:46.241 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.241 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.242 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.242 DEBUG:boto:Params: {} 2024-04-16T18:26:46.242 DEBUG:boto:Token: None 2024-04-16T18:26:46.242 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.242 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010c244b8415299ac-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.290 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.291 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.291 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.291 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.291 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:26:46.291 DEBUG:boto:Data: 2024-04-16T18:26:46.291 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.291 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.291 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.291 DEBUG:boto:Params: {} 2024-04-16T18:26:46.292 DEBUG:boto:Token: None 2024-04-16T18:26:46.292 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55 2024-04-16T18:26:46.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PaQuNv6GW9il2kaxscPBqodwGdE= 2024-04-16T18:26:46.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PaQuNv6GW9il2kaxscPBqodwGdE=', 'Content-Length': '0'} 2024-04-16T18:26:46.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000decc3520e63ab38f-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.295 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.295 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.295 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.295 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:26:46.295 DEBUG:boto:Data: 2024-04-16T18:26:46.295 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.296 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.296 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.296 DEBUG:boto:Params: {} 2024-04-16T18:26:46.296 DEBUG:boto:Token: None 2024-04-16T18:26:46.296 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55 2024-04-16T18:26:46.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PaQuNv6GW9il2kaxscPBqodwGdE= 2024-04-16T18:26:46.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PaQuNv6GW9il2kaxscPBqodwGdE=', 'Content-Length': '0'} 2024-04-16T18:26:46.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037699ff41162d4aa-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.300 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:26:46.300 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.300 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.300 DEBUG:boto:Method: GET 2024-04-16T18:26:46.300 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:26:46.300 DEBUG:boto:Data: 2024-04-16T18:26:46.300 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.300 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.300 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.300 DEBUG:boto:Params: {} 2024-04-16T18:26:46.301 DEBUG:boto:Token: None 2024-04-16T18:26:46.301 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55 2024-04-16T18:26:46.301 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PC9ODCyi/a49U3tfQu/6gleSL/w= 2024-04-16T18:26:46.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PC9ODCyi/a49U3tfQu/6gleSL/w=', 'Content-Length': '0'} 2024-04-16T18:26:46.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039c7a8125eb8cb69-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.346 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:26:46.346 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:26:46.347 DEBUG:boto:Method: GET 2024-04-16T18:26:46.347 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:26:46.347 DEBUG:boto:Data: 2024-04-16T18:26:46.347 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.347 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.347 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.347 DEBUG:boto:Params: {} 2024-04-16T18:26:46.347 DEBUG:boto:Token: None 2024-04-16T18:26:46.347 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj55 2024-04-16T18:26:46.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PC9ODCyi/a49U3tfQu/6gleSL/w= 2024-04-16T18:26:46.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PC9ODCyi/a49U3tfQu/6gleSL/w=', 'Content-Length': '0'} 2024-04-16T18:26:46.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068b4cd99c97aae83-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.351 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.351 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.351 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.352 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.352 DEBUG:boto:Method: GET 2024-04-16T18:26:46.352 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:26:46.352 DEBUG:boto:Data: 2024-04-16T18:26:46.352 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.352 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.352 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.352 DEBUG:boto:Params: {} 2024-04-16T18:26:46.352 DEBUG:boto:Token: None 2024-04-16T18:26:46.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.356 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000280182455619a346-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.398 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.399 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.399 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.399 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.399 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:26:46.399 DEBUG:boto:Method: GET 2024-04-16T18:26:46.399 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:26:46.399 DEBUG:boto:Data: 2024-04-16T18:26:46.399 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.400 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.400 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.400 DEBUG:boto:Params: {} 2024-04-16T18:26:46.400 DEBUG:boto:Token: None 2024-04-16T18:26:46.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7d70a71ae8424b7-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.408 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.408 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:26:46.408 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.408 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.408 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.408 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.408 DEBUG:boto:Method: GET 2024-04-16T18:26:46.409 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.409 DEBUG:boto:Data: 2024-04-16T18:26:46.409 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.409 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.409 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.409 DEBUG:boto:Params: {} 2024-04-16T18:26:46.409 DEBUG:boto:Token: None 2024-04-16T18:26:46.409 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.409 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw= 2024-04-16T18:26:46.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw=', 'Content-Length': '0'} 2024-04-16T18:26:46.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051d7d6f29b639424-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.454 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.454 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.455 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.455 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.455 DEBUG:boto:Method: GET 2024-04-16T18:26:46.455 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.455 DEBUG:boto:Data: 2024-04-16T18:26:46.455 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.455 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.455 DEBUG:boto:Params: {} 2024-04-16T18:26:46.455 DEBUG:boto:Token: None 2024-04-16T18:26:46.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw= 2024-04-16T18:26:46.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw=', 'Content-Length': '0'} 2024-04-16T18:26:46.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000189e00caf8147694-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.459 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.459 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.459 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.459 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.459 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.460 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.460 DEBUG:boto:Data: 2024-04-16T18:26:46.460 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.460 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.460 DEBUG:boto:Params: {} 2024-04-16T18:26:46.460 DEBUG:boto:Token: None 2024-04-16T18:26:46.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UTF4GFgm+9DaKGt3duwwhF0EPEw= 2024-04-16T18:26:46.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UTF4GFgm+9DaKGt3duwwhF0EPEw=', 'Content-Length': '0'} 2024-04-16T18:26:46.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004abe27f06968f048-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.464 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.464 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.464 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.464 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.464 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.464 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.464 DEBUG:boto:Data: 2024-04-16T18:26:46.464 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.464 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.464 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.464 DEBUG:boto:Params: {} 2024-04-16T18:26:46.465 DEBUG:boto:Token: None 2024-04-16T18:26:46.465 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UTF4GFgm+9DaKGt3duwwhF0EPEw= 2024-04-16T18:26:46.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UTF4GFgm+9DaKGt3duwwhF0EPEw=', 'Content-Length': '0'} 2024-04-16T18:26:46.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006075fc51c23a4f3e-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.468 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:26:46.468 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.469 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.469 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.469 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.469 DEBUG:boto:Method: GET 2024-04-16T18:26:46.469 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.469 DEBUG:boto:Data: 2024-04-16T18:26:46.469 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.469 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.469 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.469 DEBUG:boto:Params: {} 2024-04-16T18:26:46.469 DEBUG:boto:Token: None 2024-04-16T18:26:46.470 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.470 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw= 2024-04-16T18:26:46.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw=', 'Content-Length': '0'} 2024-04-16T18:26:46.473 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea52176b3dba486a-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.514 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.514 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.514 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.515 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.515 DEBUG:boto:Method: GET 2024-04-16T18:26:46.515 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.515 DEBUG:boto:Data: 2024-04-16T18:26:46.515 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.515 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.515 DEBUG:boto:Params: {} 2024-04-16T18:26:46.515 DEBUG:boto:Token: None 2024-04-16T18:26:46.515 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:26:46.515 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw= 2024-04-16T18:26:46.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IPDus2xEkZ9ytxV8ajeR0web3nw=', 'Content-Length': '0'} 2024-04-16T18:26:46.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c7f8c7c81c3b6f6-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.519 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.519 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.519 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.519 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.519 DEBUG:boto:Method: GET 2024-04-16T18:26:46.519 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:26:46.519 DEBUG:boto:Data: 2024-04-16T18:26:46.519 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.519 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.519 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.520 DEBUG:boto:Params: {} 2024-04-16T18:26:46.520 DEBUG:boto:Token: None 2024-04-16T18:26:46.520 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000454f5f8626d47c6c-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.524 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.524 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.524 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.524 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.524 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.524 DEBUG:boto:Method: GET 2024-04-16T18:26:46.525 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:26:46.525 DEBUG:boto:Data: 2024-04-16T18:26:46.525 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.525 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.525 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.525 DEBUG:boto:Params: {} 2024-04-16T18:26:46.525 DEBUG:boto:Token: None 2024-04-16T18:26:46.525 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.532 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de6e0fbb9de134fd-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.574 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.575 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.575 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.575 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.575 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:26:46.575 DEBUG:boto:Data: 2024-04-16T18:26:46.575 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.575 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.575 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.575 DEBUG:boto:Params: {} 2024-04-16T18:26:46.575 DEBUG:boto:Token: None 2024-04-16T18:26:46.576 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57 2024-04-16T18:26:46.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lzSzyPqpG3ywb26Fq7p+JaGSdhg= 2024-04-16T18:26:46.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lzSzyPqpG3ywb26Fq7p+JaGSdhg=', 'Content-Length': '0'} 2024-04-16T18:26:46.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a248f1b9bcca7fc5-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.579 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.579 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.579 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.579 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:26:46.579 DEBUG:boto:Data: 2024-04-16T18:26:46.579 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.579 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.579 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.579 DEBUG:boto:Params: {} 2024-04-16T18:26:46.580 DEBUG:boto:Token: None 2024-04-16T18:26:46.580 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57 2024-04-16T18:26:46.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lzSzyPqpG3ywb26Fq7p+JaGSdhg= 2024-04-16T18:26:46.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lzSzyPqpG3ywb26Fq7p+JaGSdhg=', 'Content-Length': '0'} 2024-04-16T18:26:46.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f082a2dadf1ced24-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.583 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:26:46.583 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.584 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.584 DEBUG:boto:Method: GET 2024-04-16T18:26:46.584 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:26:46.584 DEBUG:boto:Data: 2024-04-16T18:26:46.584 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.584 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.584 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.584 DEBUG:boto:Params: {} 2024-04-16T18:26:46.584 DEBUG:boto:Token: None 2024-04-16T18:26:46.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57 2024-04-16T18:26:46.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NdfhnDMHOn14LzLWbvYKa2ug/Pk= 2024-04-16T18:26:46.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NdfhnDMHOn14LzLWbvYKa2ug/Pk=', 'Content-Length': '0'} 2024-04-16T18:26:46.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bd00a16ff4d0dd3-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.630 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:26:46.630 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:26:46.631 DEBUG:boto:Method: GET 2024-04-16T18:26:46.631 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:26:46.631 DEBUG:boto:Data: 2024-04-16T18:26:46.631 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.631 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.631 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.631 DEBUG:boto:Params: {} 2024-04-16T18:26:46.631 DEBUG:boto:Token: None 2024-04-16T18:26:46.631 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj57 2024-04-16T18:26:46.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NdfhnDMHOn14LzLWbvYKa2ug/Pk= 2024-04-16T18:26:46.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NdfhnDMHOn14LzLWbvYKa2ug/Pk=', 'Content-Length': '0'} 2024-04-16T18:26:46.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c3e001663f8ef37-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.635 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.635 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.635 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.635 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.635 DEBUG:boto:Method: GET 2024-04-16T18:26:46.636 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:26:46.636 DEBUG:boto:Data: 2024-04-16T18:26:46.636 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.636 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.636 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.636 DEBUG:boto:Params: {} 2024-04-16T18:26:46.636 DEBUG:boto:Token: None 2024-04-16T18:26:46.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d24918f830ed11c-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.682 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.683 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.683 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.683 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.683 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:26:46.683 DEBUG:boto:Method: GET 2024-04-16T18:26:46.683 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:26:46.683 DEBUG:boto:Data: 2024-04-16T18:26:46.683 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.683 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.683 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.684 DEBUG:boto:Params: {} 2024-04-16T18:26:46.684 DEBUG:boto:Token: None 2024-04-16T18:26:46.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.691 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b2c7f5640078708-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.691 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.691 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:26:46.691 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.691 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.691 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.691 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.692 DEBUG:boto:Method: GET 2024-04-16T18:26:46.692 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.692 DEBUG:boto:Data: 2024-04-16T18:26:46.692 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.692 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.692 DEBUG:boto:Params: {} 2024-04-16T18:26:46.692 DEBUG:boto:Token: None 2024-04-16T18:26:46.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8= 2024-04-16T18:26:46.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8=', 'Content-Length': '0'} 2024-04-16T18:26:46.696 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aaa96b9a870be667-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.738 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.738 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.739 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.739 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.739 DEBUG:boto:Method: GET 2024-04-16T18:26:46.739 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.739 DEBUG:boto:Data: 2024-04-16T18:26:46.739 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.739 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.739 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.739 DEBUG:boto:Params: {} 2024-04-16T18:26:46.739 DEBUG:boto:Token: None 2024-04-16T18:26:46.739 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8= 2024-04-16T18:26:46.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8=', 'Content-Length': '0'} 2024-04-16T18:26:46.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f29a9a89bdc914e0-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.743 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.743 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.743 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.743 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.743 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.743 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.743 DEBUG:boto:Data: 2024-04-16T18:26:46.743 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.743 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.744 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.744 DEBUG:boto:Params: {} 2024-04-16T18:26:46.744 DEBUG:boto:Token: None 2024-04-16T18:26:46.744 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ljGQ3+X9BQwMMLR9VXepNnGpFM0= 2024-04-16T18:26:46.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ljGQ3+X9BQwMMLR9VXepNnGpFM0=', 'Content-Length': '0'} 2024-04-16T18:26:46.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e860b008a60afc73-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.747 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.747 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.747 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.747 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.747 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.747 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.748 DEBUG:boto:Data: 2024-04-16T18:26:46.748 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.748 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.748 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.748 DEBUG:boto:Params: {} 2024-04-16T18:26:46.748 DEBUG:boto:Token: None 2024-04-16T18:26:46.748 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ljGQ3+X9BQwMMLR9VXepNnGpFM0= 2024-04-16T18:26:46.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ljGQ3+X9BQwMMLR9VXepNnGpFM0=', 'Content-Length': '0'} 2024-04-16T18:26:46.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f26fd80c4fe6a3ed-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.751 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:26:46.751 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.751 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.751 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.751 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.752 DEBUG:boto:Method: GET 2024-04-16T18:26:46.752 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.752 DEBUG:boto:Data: 2024-04-16T18:26:46.752 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.752 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.752 DEBUG:boto:Params: {} 2024-04-16T18:26:46.752 DEBUG:boto:Token: None 2024-04-16T18:26:46.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8= 2024-04-16T18:26:46.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8=', 'Content-Length': '0'} 2024-04-16T18:26:46.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c55dc31107ef70d9-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.798 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.798 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.799 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.799 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.799 DEBUG:boto:Method: GET 2024-04-16T18:26:46.799 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.799 DEBUG:boto:Data: 2024-04-16T18:26:46.799 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.799 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.799 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.799 DEBUG:boto:Params: {} 2024-04-16T18:26:46.799 DEBUG:boto:Token: None 2024-04-16T18:26:46.799 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:26:46.799 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8= 2024-04-16T18:26:46.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pBnmMx8lZfWfhjs42ssIkAfv8I8=', 'Content-Length': '0'} 2024-04-16T18:26:46.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6fbf5985385d70f-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.803 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.803 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.803 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.803 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.803 DEBUG:boto:Method: GET 2024-04-16T18:26:46.803 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:26:46.803 DEBUG:boto:Data: 2024-04-16T18:26:46.803 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.803 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.803 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.803 DEBUG:boto:Params: {} 2024-04-16T18:26:46.804 DEBUG:boto:Token: None 2024-04-16T18:26:46.804 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.804 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6c2d4b6072d6743-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.808 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.808 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.808 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.808 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.808 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.808 DEBUG:boto:Method: GET 2024-04-16T18:26:46.808 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:26:46.809 DEBUG:boto:Data: 2024-04-16T18:26:46.809 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.809 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.809 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.809 DEBUG:boto:Params: {} 2024-04-16T18:26:46.809 DEBUG:boto:Token: None 2024-04-16T18:26:46.809 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.817 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008002b607e3f0c383-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.858 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.859 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.859 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.859 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.859 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:26:46.859 DEBUG:boto:Data: 2024-04-16T18:26:46.859 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.859 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.859 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.859 DEBUG:boto:Params: {} 2024-04-16T18:26:46.859 DEBUG:boto:Token: None 2024-04-16T18:26:46.859 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58 2024-04-16T18:26:46.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1DTkzRYZ1EfnMsU3roLTS8gzjNs= 2024-04-16T18:26:46.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1DTkzRYZ1EfnMsU3roLTS8gzjNs=', 'Content-Length': '0'} 2024-04-16T18:26:46.862 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e862690188388a4-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.863 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.863 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.863 DEBUG:boto:Method: HEAD 2024-04-16T18:26:46.863 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:26:46.863 DEBUG:boto:Data: 2024-04-16T18:26:46.863 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.863 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.863 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.863 DEBUG:boto:Params: {} 2024-04-16T18:26:46.863 DEBUG:boto:Token: None 2024-04-16T18:26:46.864 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58 2024-04-16T18:26:46.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1DTkzRYZ1EfnMsU3roLTS8gzjNs= 2024-04-16T18:26:46.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1DTkzRYZ1EfnMsU3roLTS8gzjNs=', 'Content-Length': '0'} 2024-04-16T18:26:46.866 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000321926e15a2e2a8-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.867 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:26:46.867 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.867 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.867 DEBUG:boto:Method: GET 2024-04-16T18:26:46.867 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:26:46.867 DEBUG:boto:Data: 2024-04-16T18:26:46.867 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.867 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.867 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.867 DEBUG:boto:Params: {} 2024-04-16T18:26:46.868 DEBUG:boto:Token: None 2024-04-16T18:26:46.868 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58 2024-04-16T18:26:46.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZVLjS6eby2ZDJbacm4uf2nvMCrg= 2024-04-16T18:26:46.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZVLjS6eby2ZDJbacm4uf2nvMCrg=', 'Content-Length': '0'} 2024-04-16T18:26:46.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e446b2ee2a033f15-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.914 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:26:46.915 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:26:46.915 DEBUG:boto:Method: GET 2024-04-16T18:26:46.915 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:26:46.915 DEBUG:boto:Data: 2024-04-16T18:26:46.915 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.915 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.915 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.915 DEBUG:boto:Params: {} 2024-04-16T18:26:46.915 DEBUG:boto:Token: None 2024-04-16T18:26:46.915 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj58 2024-04-16T18:26:46.915 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZVLjS6eby2ZDJbacm4uf2nvMCrg= 2024-04-16T18:26:46.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZVLjS6eby2ZDJbacm4uf2nvMCrg=', 'Content-Length': '0'} 2024-04-16T18:26:46.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049534717cf74e519-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.919 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.919 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.919 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.919 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.919 DEBUG:boto:Method: GET 2024-04-16T18:26:46.919 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:26:46.919 DEBUG:boto:Data: 2024-04-16T18:26:46.919 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.919 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.919 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.919 DEBUG:boto:Params: {} 2024-04-16T18:26:46.920 DEBUG:boto:Token: None 2024-04-16T18:26:46.920 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.923 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000363d30787d7e6743-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.966 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.967 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:46.967 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:46.967 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.967 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:26:46.967 DEBUG:boto:Method: GET 2024-04-16T18:26:46.967 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:26:46.967 DEBUG:boto:Data: 2024-04-16T18:26:46.967 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:46.967 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.967 DEBUG:boto:Params: {} 2024-04-16T18:26:46.968 DEBUG:boto:Token: None 2024-04-16T18:26:46.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/ 2024-04-16T18:26:46.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s= 2024-04-16T18:26:46.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EzBKFXcsRIWtKv6Oek0w8UpLC6s=', 'Content-Length': '0'} 2024-04-16T18:26:46.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058f05620b18d2a5d-00661ec2e6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:46.975 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:46.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:26:46.975 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:46.975 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:46.975 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:46.975 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:46.975 DEBUG:boto:Method: GET 2024-04-16T18:26:46.975 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:26:46.976 DEBUG:boto:Data: 2024-04-16T18:26:46.976 DEBUG:boto:Headers: {} 2024-04-16T18:26:46.976 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:46.976 DEBUG:boto:Port: 8000 2024-04-16T18:26:46.976 DEBUG:boto:Params: {} 2024-04-16T18:26:46.976 DEBUG:boto:Token: None 2024-04-16T18:26:46.976 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:46 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:26:46.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:e/3XkKeF9f1P9gxGxGCUcIRn/Hk= 2024-04-16T18:26:46.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:e/3XkKeF9f1P9gxGxGCUcIRn/Hk=', 'Content-Length': '0'} 2024-04-16T18:26:46.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006654c8a57730618c-00661ec2e6-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.022 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.022 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.023 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.023 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.023 DEBUG:boto:Method: GET 2024-04-16T18:26:47.023 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.023 DEBUG:boto:Data: 2024-04-16T18:26:47.023 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.023 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.023 DEBUG:boto:Params: {} 2024-04-16T18:26:47.023 DEBUG:boto:Token: None 2024-04-16T18:26:47.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1dZtAFgSibi7JmgpVBfuRDUUvY= 2024-04-16T18:26:47.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1dZtAFgSibi7JmgpVBfuRDUUvY=', 'Content-Length': '0'} 2024-04-16T18:26:47.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dedaf5acf7060016-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.027 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.027 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.027 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.027 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.028 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.028 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.028 DEBUG:boto:Data: 2024-04-16T18:26:47.028 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.028 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.028 DEBUG:boto:Params: {} 2024-04-16T18:26:47.028 DEBUG:boto:Token: None 2024-04-16T18:26:47.028 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JWNM3ObthK5lG8T1Gy4CO16fQIc= 2024-04-16T18:26:47.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JWNM3ObthK5lG8T1Gy4CO16fQIc=', 'Content-Length': '0'} 2024-04-16T18:26:47.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082e1aaed4fb5014c-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.032 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.032 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.032 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.032 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.032 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.032 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.032 DEBUG:boto:Data: 2024-04-16T18:26:47.032 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.032 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.032 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.033 DEBUG:boto:Params: {} 2024-04-16T18:26:47.033 DEBUG:boto:Token: None 2024-04-16T18:26:47.033 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.033 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JWNM3ObthK5lG8T1Gy4CO16fQIc= 2024-04-16T18:26:47.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JWNM3ObthK5lG8T1Gy4CO16fQIc=', 'Content-Length': '0'} 2024-04-16T18:26:47.036 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df9721b277f18aee-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.036 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:26:47.036 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.036 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.036 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.036 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.036 DEBUG:boto:Method: GET 2024-04-16T18:26:47.036 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.037 DEBUG:boto:Data: 2024-04-16T18:26:47.037 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.037 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.037 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.037 DEBUG:boto:Params: {} 2024-04-16T18:26:47.037 DEBUG:boto:Token: None 2024-04-16T18:26:47.037 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1dZtAFgSibi7JmgpVBfuRDUUvY= 2024-04-16T18:26:47.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1dZtAFgSibi7JmgpVBfuRDUUvY=', 'Content-Length': '0'} 2024-04-16T18:26:47.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a87aca440901fda9-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.082 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.082 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.082 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.083 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.083 DEBUG:boto:Method: GET 2024-04-16T18:26:47.083 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.083 DEBUG:boto:Data: 2024-04-16T18:26:47.083 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.083 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.083 DEBUG:boto:Params: {} 2024-04-16T18:26:47.083 DEBUG:boto:Token: None 2024-04-16T18:26:47.083 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:26:47.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y1dZtAFgSibi7JmgpVBfuRDUUvY= 2024-04-16T18:26:47.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y1dZtAFgSibi7JmgpVBfuRDUUvY=', 'Content-Length': '0'} 2024-04-16T18:26:47.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a86475ea1580462-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.087 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.087 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.087 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.087 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.087 DEBUG:boto:Method: GET 2024-04-16T18:26:47.087 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:26:47.087 DEBUG:boto:Data: 2024-04-16T18:26:47.087 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.088 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.088 DEBUG:boto:Params: {} 2024-04-16T18:26:47.088 DEBUG:boto:Token: None 2024-04-16T18:26:47.088 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.092 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6e89d0b079d62b2-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.092 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.092 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.092 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.092 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.092 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.093 DEBUG:boto:Method: GET 2024-04-16T18:26:47.093 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:26:47.093 DEBUG:boto:Data: 2024-04-16T18:26:47.093 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.093 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.093 DEBUG:boto:Params: {} 2024-04-16T18:26:47.093 DEBUG:boto:Token: None 2024-04-16T18:26:47.093 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.093 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.101 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000985d7fef6380f18f-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.142 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.143 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.143 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.143 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.143 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:26:47.143 DEBUG:boto:Data: 2024-04-16T18:26:47.143 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.143 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.143 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.143 DEBUG:boto:Params: {} 2024-04-16T18:26:47.144 DEBUG:boto:Token: None 2024-04-16T18:26:47.144 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68 2024-04-16T18:26:47.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RY0rHXIVRrg3ORTYdXr4UGBOpOw= 2024-04-16T18:26:47.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RY0rHXIVRrg3ORTYdXr4UGBOpOw=', 'Content-Length': '0'} 2024-04-16T18:26:47.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005742825e41f16aeb-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.148 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.148 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.148 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.148 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:26:47.148 DEBUG:boto:Data: 2024-04-16T18:26:47.148 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.148 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.148 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.148 DEBUG:boto:Params: {} 2024-04-16T18:26:47.148 DEBUG:boto:Token: None 2024-04-16T18:26:47.149 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68 2024-04-16T18:26:47.149 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RY0rHXIVRrg3ORTYdXr4UGBOpOw= 2024-04-16T18:26:47.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RY0rHXIVRrg3ORTYdXr4UGBOpOw=', 'Content-Length': '0'} 2024-04-16T18:26:47.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d316ca04adfa1bcf-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.152 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:26:47.152 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.152 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.152 DEBUG:boto:Method: GET 2024-04-16T18:26:47.152 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:26:47.152 DEBUG:boto:Data: 2024-04-16T18:26:47.152 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.153 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.153 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.153 DEBUG:boto:Params: {} 2024-04-16T18:26:47.153 DEBUG:boto:Token: None 2024-04-16T18:26:47.153 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68 2024-04-16T18:26:47.153 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UxvilK0OgH6PcBEOrzSSkihZEyk= 2024-04-16T18:26:47.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UxvilK0OgH6PcBEOrzSSkihZEyk=', 'Content-Length': '0'} 2024-04-16T18:26:47.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d270b47300ee6a96-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.198 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:26:47.199 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:26:47.199 DEBUG:boto:Method: GET 2024-04-16T18:26:47.199 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:26:47.199 DEBUG:boto:Data: 2024-04-16T18:26:47.199 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.199 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.199 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.199 DEBUG:boto:Params: {} 2024-04-16T18:26:47.199 DEBUG:boto:Token: None 2024-04-16T18:26:47.199 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj68 2024-04-16T18:26:47.200 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UxvilK0OgH6PcBEOrzSSkihZEyk= 2024-04-16T18:26:47.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UxvilK0OgH6PcBEOrzSSkihZEyk=', 'Content-Length': '0'} 2024-04-16T18:26:47.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ccbd4fca0eb7482-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.203 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.203 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.203 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.203 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.203 DEBUG:boto:Method: GET 2024-04-16T18:26:47.203 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:26:47.204 DEBUG:boto:Data: 2024-04-16T18:26:47.204 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.204 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.204 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.204 DEBUG:boto:Params: {} 2024-04-16T18:26:47.204 DEBUG:boto:Token: None 2024-04-16T18:26:47.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.208 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b8c29156f8ba7af-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.250 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.251 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.251 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.251 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.251 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:26:47.251 DEBUG:boto:Method: GET 2024-04-16T18:26:47.251 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:26:47.251 DEBUG:boto:Data: 2024-04-16T18:26:47.251 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.251 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.252 DEBUG:boto:Params: {} 2024-04-16T18:26:47.252 DEBUG:boto:Token: None 2024-04-16T18:26:47.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.258 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f7813596a9cb5e3-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.258 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.259 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:26:47.259 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.259 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.259 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.259 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.259 DEBUG:boto:Method: GET 2024-04-16T18:26:47.259 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.259 DEBUG:boto:Data: 2024-04-16T18:26:47.260 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.260 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.260 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.260 DEBUG:boto:Params: {} 2024-04-16T18:26:47.260 DEBUG:boto:Token: None 2024-04-16T18:26:47.260 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY= 2024-04-16T18:26:47.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY=', 'Content-Length': '0'} 2024-04-16T18:26:47.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021c798d6e6e89772-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.306 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.307 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.307 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.307 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.307 DEBUG:boto:Method: GET 2024-04-16T18:26:47.307 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.307 DEBUG:boto:Data: 2024-04-16T18:26:47.307 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.307 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.307 DEBUG:boto:Params: {} 2024-04-16T18:26:47.308 DEBUG:boto:Token: None 2024-04-16T18:26:47.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY= 2024-04-16T18:26:47.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY=', 'Content-Length': '0'} 2024-04-16T18:26:47.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a94bf4f84e527287-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.311 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.311 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.311 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.311 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.312 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.312 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.312 DEBUG:boto:Data: 2024-04-16T18:26:47.312 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.312 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.312 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.312 DEBUG:boto:Params: {} 2024-04-16T18:26:47.312 DEBUG:boto:Token: None 2024-04-16T18:26:47.312 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZMvdBReqEbVfSrMpyKOEmnSbifg= 2024-04-16T18:26:47.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZMvdBReqEbVfSrMpyKOEmnSbifg=', 'Content-Length': '0'} 2024-04-16T18:26:47.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb551c4b34179dab-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.316 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.316 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.316 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.316 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.316 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.316 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.316 DEBUG:boto:Data: 2024-04-16T18:26:47.316 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.316 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.316 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.316 DEBUG:boto:Params: {} 2024-04-16T18:26:47.317 DEBUG:boto:Token: None 2024-04-16T18:26:47.317 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.317 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZMvdBReqEbVfSrMpyKOEmnSbifg= 2024-04-16T18:26:47.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZMvdBReqEbVfSrMpyKOEmnSbifg=', 'Content-Length': '0'} 2024-04-16T18:26:47.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f60ffccd4f08c27-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.320 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:26:47.320 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.320 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.320 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.320 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.320 DEBUG:boto:Method: GET 2024-04-16T18:26:47.320 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.320 DEBUG:boto:Data: 2024-04-16T18:26:47.321 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.321 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.321 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.321 DEBUG:boto:Params: {} 2024-04-16T18:26:47.321 DEBUG:boto:Token: None 2024-04-16T18:26:47.321 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY= 2024-04-16T18:26:47.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY=', 'Content-Length': '0'} 2024-04-16T18:26:47.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003618c7d2516b3522-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.366 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.366 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.367 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.367 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.367 DEBUG:boto:Method: GET 2024-04-16T18:26:47.367 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.367 DEBUG:boto:Data: 2024-04-16T18:26:47.367 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.367 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.367 DEBUG:boto:Params: {} 2024-04-16T18:26:47.367 DEBUG:boto:Token: None 2024-04-16T18:26:47.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:26:47.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY= 2024-04-16T18:26:47.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9j4sh25pPSZ1JQYdO6F4+AnBJNY=', 'Content-Length': '0'} 2024-04-16T18:26:47.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f97466bb2c3b76b6-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.371 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.371 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.371 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.371 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.371 DEBUG:boto:Method: GET 2024-04-16T18:26:47.371 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:26:47.372 DEBUG:boto:Data: 2024-04-16T18:26:47.372 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.372 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.372 DEBUG:boto:Params: {} 2024-04-16T18:26:47.372 DEBUG:boto:Token: None 2024-04-16T18:26:47.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000008c7536b894e6a3a-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.376 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.376 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.376 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.376 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.376 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.376 DEBUG:boto:Method: GET 2024-04-16T18:26:47.376 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:26:47.377 DEBUG:boto:Data: 2024-04-16T18:26:47.377 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.377 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.377 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.377 DEBUG:boto:Params: {} 2024-04-16T18:26:47.377 DEBUG:boto:Token: None 2024-04-16T18:26:47.377 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.384 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b061e01411f7c3e4-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.426 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.427 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.427 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.427 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.427 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:26:47.427 DEBUG:boto:Data: 2024-04-16T18:26:47.427 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.427 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.427 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.427 DEBUG:boto:Params: {} 2024-04-16T18:26:47.427 DEBUG:boto:Token: None 2024-04-16T18:26:47.428 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69 2024-04-16T18:26:47.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JTSZ/saX9z3bASpewJ3lhV01CoI= 2024-04-16T18:26:47.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JTSZ/saX9z3bASpewJ3lhV01CoI=', 'Content-Length': '0'} 2024-04-16T18:26:47.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d745a428dd25dc27-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.431 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.431 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.431 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.431 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:26:47.431 DEBUG:boto:Data: 2024-04-16T18:26:47.432 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.432 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.432 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.432 DEBUG:boto:Params: {} 2024-04-16T18:26:47.432 DEBUG:boto:Token: None 2024-04-16T18:26:47.432 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69 2024-04-16T18:26:47.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JTSZ/saX9z3bASpewJ3lhV01CoI= 2024-04-16T18:26:47.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JTSZ/saX9z3bASpewJ3lhV01CoI=', 'Content-Length': '0'} 2024-04-16T18:26:47.435 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c692375238ad643-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.436 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:26:47.436 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.436 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.436 DEBUG:boto:Method: GET 2024-04-16T18:26:47.436 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:26:47.436 DEBUG:boto:Data: 2024-04-16T18:26:47.436 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.436 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.436 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.436 DEBUG:boto:Params: {} 2024-04-16T18:26:47.436 DEBUG:boto:Token: None 2024-04-16T18:26:47.437 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69 2024-04-16T18:26:47.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4X95lo+J/AstihFH2ZH1wTvgVdo= 2024-04-16T18:26:47.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4X95lo+J/AstihFH2ZH1wTvgVdo=', 'Content-Length': '0'} 2024-04-16T18:26:47.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b825f38ea6b7868-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.482 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:26:47.482 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:26:47.483 DEBUG:boto:Method: GET 2024-04-16T18:26:47.483 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:26:47.483 DEBUG:boto:Data: 2024-04-16T18:26:47.483 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.483 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.483 DEBUG:boto:Params: {} 2024-04-16T18:26:47.483 DEBUG:boto:Token: None 2024-04-16T18:26:47.483 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj69 2024-04-16T18:26:47.483 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4X95lo+J/AstihFH2ZH1wTvgVdo= 2024-04-16T18:26:47.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4X95lo+J/AstihFH2ZH1wTvgVdo=', 'Content-Length': '0'} 2024-04-16T18:26:47.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000000666ad486476ae-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.487 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.487 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.487 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.488 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.488 DEBUG:boto:Method: GET 2024-04-16T18:26:47.488 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:26:47.488 DEBUG:boto:Data: 2024-04-16T18:26:47.488 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.488 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.488 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.488 DEBUG:boto:Params: {} 2024-04-16T18:26:47.488 DEBUG:boto:Token: None 2024-04-16T18:26:47.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.492 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3004f5746b42f20-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.534 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.535 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.535 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.535 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.535 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:26:47.535 DEBUG:boto:Method: GET 2024-04-16T18:26:47.535 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:26:47.535 DEBUG:boto:Data: 2024-04-16T18:26:47.535 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.535 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.535 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.536 DEBUG:boto:Params: {} 2024-04-16T18:26:47.536 DEBUG:boto:Token: None 2024-04-16T18:26:47.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000008743335e7cbb9ee-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.542 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.542 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:26:47.542 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.543 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.543 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.543 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.543 DEBUG:boto:Method: GET 2024-04-16T18:26:47.543 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.543 DEBUG:boto:Data: 2024-04-16T18:26:47.543 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.543 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.543 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.543 DEBUG:boto:Params: {} 2024-04-16T18:26:47.545 DEBUG:boto:Token: None 2024-04-16T18:26:47.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA= 2024-04-16T18:26:47.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA=', 'Content-Length': '0'} 2024-04-16T18:26:47.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d35d6a6ae678ed39-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.590 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.590 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.590 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.591 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.591 DEBUG:boto:Method: GET 2024-04-16T18:26:47.591 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.591 DEBUG:boto:Data: 2024-04-16T18:26:47.591 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.591 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.591 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.591 DEBUG:boto:Params: {} 2024-04-16T18:26:47.591 DEBUG:boto:Token: None 2024-04-16T18:26:47.591 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA= 2024-04-16T18:26:47.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA=', 'Content-Length': '0'} 2024-04-16T18:26:47.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a3fb8fe72177ab4-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.595 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.595 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.595 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.595 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.595 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.595 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.595 DEBUG:boto:Data: 2024-04-16T18:26:47.595 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.595 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.595 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.596 DEBUG:boto:Params: {} 2024-04-16T18:26:47.596 DEBUG:boto:Token: None 2024-04-16T18:26:47.596 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d/LBVU08IaRPH1s5KOIEbSnVxjI= 2024-04-16T18:26:47.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d/LBVU08IaRPH1s5KOIEbSnVxjI=', 'Content-Length': '0'} 2024-04-16T18:26:47.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d874c210512ad8a-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.599 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.599 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.599 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.599 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.600 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.600 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.600 DEBUG:boto:Data: 2024-04-16T18:26:47.600 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.600 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.600 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.600 DEBUG:boto:Params: {} 2024-04-16T18:26:47.600 DEBUG:boto:Token: None 2024-04-16T18:26:47.600 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.601 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:d/LBVU08IaRPH1s5KOIEbSnVxjI= 2024-04-16T18:26:47.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d/LBVU08IaRPH1s5KOIEbSnVxjI=', 'Content-Length': '0'} 2024-04-16T18:26:47.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a0e07f37621ac53-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:26:47.604 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.604 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.604 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.604 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.604 DEBUG:boto:Method: GET 2024-04-16T18:26:47.604 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.604 DEBUG:boto:Data: 2024-04-16T18:26:47.605 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.605 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.605 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.605 DEBUG:boto:Params: {} 2024-04-16T18:26:47.605 DEBUG:boto:Token: None 2024-04-16T18:26:47.605 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA= 2024-04-16T18:26:47.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA=', 'Content-Length': '0'} 2024-04-16T18:26:47.609 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2f9c5760c424179-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.650 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.650 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.651 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.651 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.651 DEBUG:boto:Method: GET 2024-04-16T18:26:47.651 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.651 DEBUG:boto:Data: 2024-04-16T18:26:47.651 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.651 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.651 DEBUG:boto:Params: {} 2024-04-16T18:26:47.651 DEBUG:boto:Token: None 2024-04-16T18:26:47.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:26:47.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA= 2024-04-16T18:26:47.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:425qI5c06pWyk0Ye9II23AKqnRA=', 'Content-Length': '0'} 2024-04-16T18:26:47.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000729823847a28cf96-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.656 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.656 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.656 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.656 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.656 DEBUG:boto:Method: GET 2024-04-16T18:26:47.656 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:26:47.656 DEBUG:boto:Data: 2024-04-16T18:26:47.656 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.656 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.657 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.657 DEBUG:boto:Params: {} 2024-04-16T18:26:47.657 DEBUG:boto:Token: None 2024-04-16T18:26:47.657 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.657 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.661 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000023f3a5b8123f4e7-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.661 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.661 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.661 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.661 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.662 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.662 DEBUG:boto:Method: GET 2024-04-16T18:26:47.662 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:26:47.662 DEBUG:boto:Data: 2024-04-16T18:26:47.662 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.662 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.662 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.662 DEBUG:boto:Params: {} 2024-04-16T18:26:47.662 DEBUG:boto:Token: None 2024-04-16T18:26:47.662 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.662 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7f175407acf5422-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.710 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.711 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.711 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.711 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.711 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:26:47.711 DEBUG:boto:Data: 2024-04-16T18:26:47.711 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.711 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.711 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.711 DEBUG:boto:Params: {} 2024-04-16T18:26:47.711 DEBUG:boto:Token: None 2024-04-16T18:26:47.712 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76 2024-04-16T18:26:47.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mo5oVUwl1EMF8fuqLt6qk6q1Unw= 2024-04-16T18:26:47.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mo5oVUwl1EMF8fuqLt6qk6q1Unw=', 'Content-Length': '0'} 2024-04-16T18:26:47.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051abe6b312bfa31c-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.715 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.715 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.715 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.715 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:26:47.715 DEBUG:boto:Data: 2024-04-16T18:26:47.715 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.715 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.715 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.715 DEBUG:boto:Params: {} 2024-04-16T18:26:47.715 DEBUG:boto:Token: None 2024-04-16T18:26:47.716 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76 2024-04-16T18:26:47.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mo5oVUwl1EMF8fuqLt6qk6q1Unw= 2024-04-16T18:26:47.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mo5oVUwl1EMF8fuqLt6qk6q1Unw=', 'Content-Length': '0'} 2024-04-16T18:26:47.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f800dd042176a62-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:26:47.719 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.719 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.719 DEBUG:boto:Method: GET 2024-04-16T18:26:47.719 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:26:47.719 DEBUG:boto:Data: 2024-04-16T18:26:47.719 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.719 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.719 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.719 DEBUG:boto:Params: {} 2024-04-16T18:26:47.720 DEBUG:boto:Token: None 2024-04-16T18:26:47.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76 2024-04-16T18:26:47.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ipHY4sggjhhzFYKqtOQOvOXSDdU= 2024-04-16T18:26:47.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ipHY4sggjhhzFYKqtOQOvOXSDdU=', 'Content-Length': '0'} 2024-04-16T18:26:47.723 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adb899e42e1ee749-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.766 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:26:47.766 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:26:47.767 DEBUG:boto:Method: GET 2024-04-16T18:26:47.767 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:26:47.767 DEBUG:boto:Data: 2024-04-16T18:26:47.767 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.767 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.767 DEBUG:boto:Params: {} 2024-04-16T18:26:47.767 DEBUG:boto:Token: None 2024-04-16T18:26:47.767 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj76 2024-04-16T18:26:47.767 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ipHY4sggjhhzFYKqtOQOvOXSDdU= 2024-04-16T18:26:47.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ipHY4sggjhhzFYKqtOQOvOXSDdU=', 'Content-Length': '0'} 2024-04-16T18:26:47.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093917ebc1beb3940-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.771 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.771 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.771 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.771 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.771 DEBUG:boto:Method: GET 2024-04-16T18:26:47.771 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:26:47.771 DEBUG:boto:Data: 2024-04-16T18:26:47.771 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.771 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.771 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.771 DEBUG:boto:Params: {} 2024-04-16T18:26:47.771 DEBUG:boto:Token: None 2024-04-16T18:26:47.772 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.775 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e50ea62ab55d0f5-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.818 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.819 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.819 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.819 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.819 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:26:47.819 DEBUG:boto:Method: GET 2024-04-16T18:26:47.819 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:26:47.819 DEBUG:boto:Data: 2024-04-16T18:26:47.819 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.819 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.819 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.819 DEBUG:boto:Params: {} 2024-04-16T18:26:47.820 DEBUG:boto:Token: None 2024-04-16T18:26:47.820 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.826 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010ad2bbcef70ae63-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.827 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.827 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:26:47.827 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.827 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.827 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.827 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.828 DEBUG:boto:Method: GET 2024-04-16T18:26:47.828 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.828 DEBUG:boto:Data: 2024-04-16T18:26:47.828 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.828 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.828 DEBUG:boto:Params: {} 2024-04-16T18:26:47.828 DEBUG:boto:Token: None 2024-04-16T18:26:47.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4= 2024-04-16T18:26:47.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4=', 'Content-Length': '0'} 2024-04-16T18:26:47.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e30cd9c1352934c-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.874 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.874 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.875 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.875 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.875 DEBUG:boto:Method: GET 2024-04-16T18:26:47.875 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.875 DEBUG:boto:Data: 2024-04-16T18:26:47.875 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.875 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.875 DEBUG:boto:Params: {} 2024-04-16T18:26:47.875 DEBUG:boto:Token: None 2024-04-16T18:26:47.875 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4= 2024-04-16T18:26:47.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4=', 'Content-Length': '0'} 2024-04-16T18:26:47.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018100426a6c1829b-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.879 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.879 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.879 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.879 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.879 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.879 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.879 DEBUG:boto:Data: 2024-04-16T18:26:47.879 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.879 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.880 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.880 DEBUG:boto:Params: {} 2024-04-16T18:26:47.880 DEBUG:boto:Token: None 2024-04-16T18:26:47.880 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fcq3Wh3FK+WhpeeOVHJo9BIJ17U= 2024-04-16T18:26:47.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fcq3Wh3FK+WhpeeOVHJo9BIJ17U=', 'Content-Length': '0'} 2024-04-16T18:26:47.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6e49a7c59262cdf-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.883 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.883 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.883 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.883 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.883 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.883 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.883 DEBUG:boto:Data: 2024-04-16T18:26:47.884 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.884 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.884 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.884 DEBUG:boto:Params: {} 2024-04-16T18:26:47.884 DEBUG:boto:Token: None 2024-04-16T18:26:47.884 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Fcq3Wh3FK+WhpeeOVHJo9BIJ17U= 2024-04-16T18:26:47.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Fcq3Wh3FK+WhpeeOVHJo9BIJ17U=', 'Content-Length': '0'} 2024-04-16T18:26:47.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a1e6cd19e7c4dbd-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.887 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:26:47.887 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.887 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.887 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.887 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.888 DEBUG:boto:Method: GET 2024-04-16T18:26:47.888 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.888 DEBUG:boto:Data: 2024-04-16T18:26:47.888 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.888 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.888 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.888 DEBUG:boto:Params: {} 2024-04-16T18:26:47.888 DEBUG:boto:Token: None 2024-04-16T18:26:47.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4= 2024-04-16T18:26:47.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4=', 'Content-Length': '0'} 2024-04-16T18:26:47.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9cb90f0e006046e-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.934 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.934 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.935 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.935 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.935 DEBUG:boto:Method: GET 2024-04-16T18:26:47.935 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.935 DEBUG:boto:Data: 2024-04-16T18:26:47.935 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.935 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.935 DEBUG:boto:Params: {} 2024-04-16T18:26:47.935 DEBUG:boto:Token: None 2024-04-16T18:26:47.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:26:47.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4= 2024-04-16T18:26:47.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zVpLzU5wciRONMIq0/xI9U45Sy4=', 'Content-Length': '0'} 2024-04-16T18:26:47.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3448b6d652d21d1-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.939 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.939 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.939 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:47.939 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:47.939 DEBUG:boto:Method: GET 2024-04-16T18:26:47.940 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:26:47.940 DEBUG:boto:Data: 2024-04-16T18:26:47.940 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.940 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.940 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.940 DEBUG:boto:Params: {} 2024-04-16T18:26:47.940 DEBUG:boto:Token: None 2024-04-16T18:26:47.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009672bb1832ec11a-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.944 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.945 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:47.945 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:47.945 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:47.945 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:47.945 DEBUG:boto:Method: GET 2024-04-16T18:26:47.945 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:26:47.945 DEBUG:boto:Data: 2024-04-16T18:26:47.945 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.945 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.945 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.945 DEBUG:boto:Params: {} 2024-04-16T18:26:47.946 DEBUG:boto:Token: None 2024-04-16T18:26:47.946 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/ 2024-04-16T18:26:47.946 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY= 2024-04-16T18:26:47.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v3JsJaibCy5in08LcMqlAfsLAPY=', 'Content-Length': '0'} 2024-04-16T18:26:47.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c2433e8e90903ee-00661ec2e7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.994 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:47.995 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.995 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.995 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.995 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:26:47.995 DEBUG:boto:Data: 2024-04-16T18:26:47.995 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:47.995 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.995 DEBUG:boto:Params: {} 2024-04-16T18:26:47.995 DEBUG:boto:Token: None 2024-04-16T18:26:47.996 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:47 GMT /lybtug-52/obj85 2024-04-16T18:26:47.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SmRiq52JCIDc1mu61om7apfRffE= 2024-04-16T18:26:47.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SmRiq52JCIDc1mu61om7apfRffE=', 'Content-Length': '0'} 2024-04-16T18:26:47.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025eb2180481cd930-00661ec2e7-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:47.999 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:47.999 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:47.999 DEBUG:boto:Method: HEAD 2024-04-16T18:26:47.999 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:26:47.999 DEBUG:boto:Data: 2024-04-16T18:26:47.999 DEBUG:boto:Headers: {} 2024-04-16T18:26:47.999 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:47.999 DEBUG:boto:Port: 8000 2024-04-16T18:26:47.999 DEBUG:boto:Params: {} 2024-04-16T18:26:48.000 DEBUG:boto:Token: None 2024-04-16T18:26:48.000 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj85 2024-04-16T18:26:48.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EvfF/HFTniT0iDc8N8sUi+wq9B0= 2024-04-16T18:26:48.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EvfF/HFTniT0iDc8N8sUi+wq9B0=', 'Content-Length': '0'} 2024-04-16T18:26:48.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f4e6349a977f327-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.003 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:26:48.003 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:48.003 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:48.003 DEBUG:boto:Method: GET 2024-04-16T18:26:48.003 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:26:48.003 DEBUG:boto:Data: 2024-04-16T18:26:48.003 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.003 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.004 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.004 DEBUG:boto:Params: {} 2024-04-16T18:26:48.004 DEBUG:boto:Token: None 2024-04-16T18:26:48.004 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj85 2024-04-16T18:26:48.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zuegzAHmUf7+p2ICT9QIpwRg5a4= 2024-04-16T18:26:48.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zuegzAHmUf7+p2ICT9QIpwRg5a4=', 'Content-Length': '0'} 2024-04-16T18:26:48.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080cb83e25f5314fa-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.050 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:26:48.051 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:26:48.051 DEBUG:boto:Method: GET 2024-04-16T18:26:48.051 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:26:48.051 DEBUG:boto:Data: 2024-04-16T18:26:48.051 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.051 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.051 DEBUG:boto:Params: {} 2024-04-16T18:26:48.051 DEBUG:boto:Token: None 2024-04-16T18:26:48.051 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj85 2024-04-16T18:26:48.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zuegzAHmUf7+p2ICT9QIpwRg5a4= 2024-04-16T18:26:48.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zuegzAHmUf7+p2ICT9QIpwRg5a4=', 'Content-Length': '0'} 2024-04-16T18:26:48.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007098521e7e0f140d-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.055 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.055 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.055 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:48.055 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:48.055 DEBUG:boto:Method: GET 2024-04-16T18:26:48.055 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:26:48.055 DEBUG:boto:Data: 2024-04-16T18:26:48.055 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.055 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.055 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.055 DEBUG:boto:Params: {} 2024-04-16T18:26:48.056 DEBUG:boto:Token: None 2024-04-16T18:26:48.056 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.059 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ddf5f0ff039181a-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.102 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.103 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.103 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.103 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:48.103 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:26:48.103 DEBUG:boto:Method: GET 2024-04-16T18:26:48.103 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:26:48.103 DEBUG:boto:Data: 2024-04-16T18:26:48.103 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.103 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.103 DEBUG:boto:Params: {} 2024-04-16T18:26:48.104 DEBUG:boto:Token: None 2024-04-16T18:26:48.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005603c1ce49441004-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.110 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.110 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:26:48.111 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.111 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.111 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.111 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.111 DEBUG:boto:Method: GET 2024-04-16T18:26:48.111 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.111 DEBUG:boto:Data: 2024-04-16T18:26:48.111 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.111 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.111 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.111 DEBUG:boto:Params: {} 2024-04-16T18:26:48.112 DEBUG:boto:Token: None 2024-04-16T18:26:48.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU= 2024-04-16T18:26:48.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU=', 'Content-Length': '0'} 2024-04-16T18:26:48.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cd43906ee897d6e-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.158 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.158 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.159 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.159 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.159 DEBUG:boto:Method: GET 2024-04-16T18:26:48.159 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.159 DEBUG:boto:Data: 2024-04-16T18:26:48.159 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.159 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.159 DEBUG:boto:Params: {} 2024-04-16T18:26:48.161 DEBUG:boto:Token: None 2024-04-16T18:26:48.161 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.162 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU= 2024-04-16T18:26:48.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU=', 'Content-Length': '0'} 2024-04-16T18:26:48.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c207c0740dbc550b-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.165 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.165 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.165 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.166 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.166 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.166 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.166 DEBUG:boto:Data: 2024-04-16T18:26:48.166 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.166 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.166 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.166 DEBUG:boto:Params: {} 2024-04-16T18:26:48.166 DEBUG:boto:Token: None 2024-04-16T18:26:48.167 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.167 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hk/FOZWUgj1pLQnbZqklWy07FFs= 2024-04-16T18:26:48.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hk/FOZWUgj1pLQnbZqklWy07FFs=', 'Content-Length': '0'} 2024-04-16T18:26:48.170 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e68904e5412057cc-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.170 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.170 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.170 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.170 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.171 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.171 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.171 DEBUG:boto:Data: 2024-04-16T18:26:48.171 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.171 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.171 DEBUG:boto:Params: {} 2024-04-16T18:26:48.171 DEBUG:boto:Token: None 2024-04-16T18:26:48.171 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hk/FOZWUgj1pLQnbZqklWy07FFs= 2024-04-16T18:26:48.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hk/FOZWUgj1pLQnbZqklWy07FFs=', 'Content-Length': '0'} 2024-04-16T18:26:48.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022ff742b564a3c9f-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.175 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:26:48.175 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.175 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.175 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.176 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.176 DEBUG:boto:Method: GET 2024-04-16T18:26:48.176 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.176 DEBUG:boto:Data: 2024-04-16T18:26:48.176 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.176 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.176 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.176 DEBUG:boto:Params: {} 2024-04-16T18:26:48.176 DEBUG:boto:Token: None 2024-04-16T18:26:48.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU= 2024-04-16T18:26:48.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU=', 'Content-Length': '0'} 2024-04-16T18:26:48.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ec55f31a4c5e4bb-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.222 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.222 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.223 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.223 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.223 DEBUG:boto:Method: GET 2024-04-16T18:26:48.223 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.223 DEBUG:boto:Data: 2024-04-16T18:26:48.223 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.223 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.223 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.223 DEBUG:boto:Params: {} 2024-04-16T18:26:48.223 DEBUG:boto:Token: None 2024-04-16T18:26:48.223 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:26:48.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU= 2024-04-16T18:26:48.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rCYhNN0lzBpYTCDrmhGmdJkf8WU=', 'Content-Length': '0'} 2024-04-16T18:26:48.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f23c84d4fd7b2f2d-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.227 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.227 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.227 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.227 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.227 DEBUG:boto:Method: GET 2024-04-16T18:26:48.227 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:26:48.227 DEBUG:boto:Data: 2024-04-16T18:26:48.228 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.228 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.228 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.228 DEBUG:boto:Params: {} 2024-04-16T18:26:48.228 DEBUG:boto:Token: None 2024-04-16T18:26:48.228 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.232 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005146d3573ca037c2-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.278 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.279 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.279 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.279 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.279 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.279 DEBUG:boto:Method: GET 2024-04-16T18:26:48.279 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:26:48.279 DEBUG:boto:Data: 2024-04-16T18:26:48.279 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.279 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.279 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.280 DEBUG:boto:Params: {} 2024-04-16T18:26:48.280 DEBUG:boto:Token: None 2024-04-16T18:26:48.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.286 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000905047e3cc00d37a-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.286 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.286 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.286 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.286 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.287 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:26:48.287 DEBUG:boto:Data: 2024-04-16T18:26:48.287 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.287 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.287 DEBUG:boto:Params: {} 2024-04-16T18:26:48.287 DEBUG:boto:Token: None 2024-04-16T18:26:48.287 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87 2024-04-16T18:26:48.287 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dzlkjgWCRJ+ON/HtGwrcGrnVMVI= 2024-04-16T18:26:48.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dzlkjgWCRJ+ON/HtGwrcGrnVMVI=', 'Content-Length': '0'} 2024-04-16T18:26:48.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcbd8c3a0d904dcb-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.291 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.291 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.291 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.291 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:26:48.291 DEBUG:boto:Data: 2024-04-16T18:26:48.291 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.291 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.291 DEBUG:boto:Params: {} 2024-04-16T18:26:48.291 DEBUG:boto:Token: None 2024-04-16T18:26:48.291 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87 2024-04-16T18:26:48.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dzlkjgWCRJ+ON/HtGwrcGrnVMVI= 2024-04-16T18:26:48.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dzlkjgWCRJ+ON/HtGwrcGrnVMVI=', 'Content-Length': '0'} 2024-04-16T18:26:48.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041f5b9bdcf14f387-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:26:48.295 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.295 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.295 DEBUG:boto:Method: GET 2024-04-16T18:26:48.295 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:26:48.295 DEBUG:boto:Data: 2024-04-16T18:26:48.295 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.295 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.296 DEBUG:boto:Params: {} 2024-04-16T18:26:48.296 DEBUG:boto:Token: None 2024-04-16T18:26:48.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87 2024-04-16T18:26:48.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+exQe93GohX5MY/SAQWRYL88XWI= 2024-04-16T18:26:48.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+exQe93GohX5MY/SAQWRYL88XWI=', 'Content-Length': '0'} 2024-04-16T18:26:48.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc7913302fd4d028-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.342 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:26:48.342 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:26:48.342 DEBUG:boto:Method: GET 2024-04-16T18:26:48.343 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:26:48.343 DEBUG:boto:Data: 2024-04-16T18:26:48.343 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.343 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.343 DEBUG:boto:Params: {} 2024-04-16T18:26:48.343 DEBUG:boto:Token: None 2024-04-16T18:26:48.343 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj87 2024-04-16T18:26:48.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+exQe93GohX5MY/SAQWRYL88XWI= 2024-04-16T18:26:48.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+exQe93GohX5MY/SAQWRYL88XWI=', 'Content-Length': '0'} 2024-04-16T18:26:48.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ef502dc334abf2d-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.347 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.347 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.347 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.347 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.347 DEBUG:boto:Method: GET 2024-04-16T18:26:48.347 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:26:48.347 DEBUG:boto:Data: 2024-04-16T18:26:48.347 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.347 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.348 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.348 DEBUG:boto:Params: {} 2024-04-16T18:26:48.348 DEBUG:boto:Token: None 2024-04-16T18:26:48.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000127c59de79ef2a6d-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.394 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.395 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.395 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.395 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.395 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:26:48.395 DEBUG:boto:Method: GET 2024-04-16T18:26:48.395 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:26:48.395 DEBUG:boto:Data: 2024-04-16T18:26:48.395 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.396 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.396 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.396 DEBUG:boto:Params: {} 2024-04-16T18:26:48.396 DEBUG:boto:Token: None 2024-04-16T18:26:48.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.402 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9a49f61472876d4-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.403 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.403 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:26:48.403 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.403 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.404 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.404 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.404 DEBUG:boto:Method: GET 2024-04-16T18:26:48.404 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.404 DEBUG:boto:Data: 2024-04-16T18:26:48.404 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.404 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.404 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.404 DEBUG:boto:Params: {} 2024-04-16T18:26:48.404 DEBUG:boto:Token: None 2024-04-16T18:26:48.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY= 2024-04-16T18:26:48.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY=', 'Content-Length': '0'} 2024-04-16T18:26:48.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000784daf8f074caffa-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.450 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.451 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.451 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.451 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.451 DEBUG:boto:Method: GET 2024-04-16T18:26:48.451 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.451 DEBUG:boto:Data: 2024-04-16T18:26:48.451 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.451 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.451 DEBUG:boto:Params: {} 2024-04-16T18:26:48.451 DEBUG:boto:Token: None 2024-04-16T18:26:48.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY= 2024-04-16T18:26:48.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY=', 'Content-Length': '0'} 2024-04-16T18:26:48.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064e9e849dd7c717f-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.455 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.455 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.455 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.455 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.455 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.455 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.455 DEBUG:boto:Data: 2024-04-16T18:26:48.455 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.455 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.456 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.456 DEBUG:boto:Params: {} 2024-04-16T18:26:48.456 DEBUG:boto:Token: None 2024-04-16T18:26:48.456 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zCUsOudYcuBK2OQ0zJJdUwbucn8= 2024-04-16T18:26:48.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zCUsOudYcuBK2OQ0zJJdUwbucn8=', 'Content-Length': '0'} 2024-04-16T18:26:48.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5fd6bfce98709b5-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.459 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.459 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.460 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.460 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.460 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.460 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.460 DEBUG:boto:Data: 2024-04-16T18:26:48.460 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.460 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.460 DEBUG:boto:Params: {} 2024-04-16T18:26:48.460 DEBUG:boto:Token: None 2024-04-16T18:26:48.460 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zCUsOudYcuBK2OQ0zJJdUwbucn8= 2024-04-16T18:26:48.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zCUsOudYcuBK2OQ0zJJdUwbucn8=', 'Content-Length': '0'} 2024-04-16T18:26:48.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd6edabb7bfbfc32-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.464 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:26:48.464 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.464 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.464 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.464 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.464 DEBUG:boto:Method: GET 2024-04-16T18:26:48.464 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.464 DEBUG:boto:Data: 2024-04-16T18:26:48.464 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.464 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.464 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.465 DEBUG:boto:Params: {} 2024-04-16T18:26:48.465 DEBUG:boto:Token: None 2024-04-16T18:26:48.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.465 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY= 2024-04-16T18:26:48.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY=', 'Content-Length': '0'} 2024-04-16T18:26:48.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001069edfff3691e7c-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.510 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.510 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.511 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.511 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.511 DEBUG:boto:Method: GET 2024-04-16T18:26:48.511 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.511 DEBUG:boto:Data: 2024-04-16T18:26:48.511 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.511 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.511 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.511 DEBUG:boto:Params: {} 2024-04-16T18:26:48.512 DEBUG:boto:Token: None 2024-04-16T18:26:48.512 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:26:48.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY= 2024-04-16T18:26:48.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W+1uTVzFBdivPYELpFsLc3C7lXY=', 'Content-Length': '0'} 2024-04-16T18:26:48.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037b46dbcee502c11-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.516 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.516 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.516 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.516 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.516 DEBUG:boto:Method: GET 2024-04-16T18:26:48.516 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:26:48.516 DEBUG:boto:Data: 2024-04-16T18:26:48.516 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.516 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.516 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.517 DEBUG:boto:Params: {} 2024-04-16T18:26:48.517 DEBUG:boto:Token: None 2024-04-16T18:26:48.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.520 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009568ae844192a499-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.521 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.521 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.521 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.521 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.521 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.521 DEBUG:boto:Method: GET 2024-04-16T18:26:48.521 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:26:48.522 DEBUG:boto:Data: 2024-04-16T18:26:48.522 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.522 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.522 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.522 DEBUG:boto:Params: {} 2024-04-16T18:26:48.522 DEBUG:boto:Token: None 2024-04-16T18:26:48.522 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6da60059a455ad6-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.529 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.529 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.529 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.529 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.529 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:26:48.529 DEBUG:boto:Data: 2024-04-16T18:26:48.529 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.530 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.530 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.530 DEBUG:boto:Params: {} 2024-04-16T18:26:48.530 DEBUG:boto:Token: None 2024-04-16T18:26:48.530 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92 2024-04-16T18:26:48.530 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YHbGuiXGGwOfOevKgeogD0WWacs= 2024-04-16T18:26:48.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YHbGuiXGGwOfOevKgeogD0WWacs=', 'Content-Length': '0'} 2024-04-16T18:26:48.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edcfc825117e8fc3-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.533 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.533 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.533 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.534 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:26:48.534 DEBUG:boto:Data: 2024-04-16T18:26:48.534 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.534 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.534 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.534 DEBUG:boto:Params: {} 2024-04-16T18:26:48.534 DEBUG:boto:Token: None 2024-04-16T18:26:48.534 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92 2024-04-16T18:26:48.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YHbGuiXGGwOfOevKgeogD0WWacs= 2024-04-16T18:26:48.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YHbGuiXGGwOfOevKgeogD0WWacs=', 'Content-Length': '0'} 2024-04-16T18:26:48.537 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b86cf0cb0ec411d-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.537 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:26:48.537 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.537 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.538 DEBUG:boto:Method: GET 2024-04-16T18:26:48.538 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:26:48.538 DEBUG:boto:Data: 2024-04-16T18:26:48.538 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.538 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.538 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.538 DEBUG:boto:Params: {} 2024-04-16T18:26:48.538 DEBUG:boto:Token: None 2024-04-16T18:26:48.538 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92 2024-04-16T18:26:48.538 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hAZIOCyr04fsO8SJm/qLJgBs/Ao= 2024-04-16T18:26:48.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hAZIOCyr04fsO8SJm/qLJgBs/Ao=', 'Content-Length': '0'} 2024-04-16T18:26:48.541 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e566e945f934b60-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.582 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:26:48.583 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:26:48.583 DEBUG:boto:Method: GET 2024-04-16T18:26:48.583 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:26:48.583 DEBUG:boto:Data: 2024-04-16T18:26:48.583 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.583 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.583 DEBUG:boto:Params: {} 2024-04-16T18:26:48.583 DEBUG:boto:Token: None 2024-04-16T18:26:48.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj92 2024-04-16T18:26:48.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hAZIOCyr04fsO8SJm/qLJgBs/Ao= 2024-04-16T18:26:48.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hAZIOCyr04fsO8SJm/qLJgBs/Ao=', 'Content-Length': '0'} 2024-04-16T18:26:48.587 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bb541cf8313fb9d-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.587 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.587 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.587 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.587 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.587 DEBUG:boto:Method: GET 2024-04-16T18:26:48.588 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:26:48.588 DEBUG:boto:Data: 2024-04-16T18:26:48.588 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.588 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.588 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.588 DEBUG:boto:Params: {} 2024-04-16T18:26:48.588 DEBUG:boto:Token: None 2024-04-16T18:26:48.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e163c432344005c7-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.634 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.635 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.635 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.635 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.635 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:26:48.635 DEBUG:boto:Method: GET 2024-04-16T18:26:48.635 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:26:48.635 DEBUG:boto:Data: 2024-04-16T18:26:48.635 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.635 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.636 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.636 DEBUG:boto:Params: {} 2024-04-16T18:26:48.636 DEBUG:boto:Token: None 2024-04-16T18:26:48.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.643 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb48a7d950e8a542-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.643 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.643 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:26:48.643 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.644 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.644 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.644 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.644 DEBUG:boto:Method: GET 2024-04-16T18:26:48.644 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.644 DEBUG:boto:Data: 2024-04-16T18:26:48.644 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.644 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.644 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.644 DEBUG:boto:Params: {} 2024-04-16T18:26:48.645 DEBUG:boto:Token: None 2024-04-16T18:26:48.645 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic= 2024-04-16T18:26:48.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic=', 'Content-Length': '0'} 2024-04-16T18:26:48.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000167457378642f539-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.690 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.691 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.691 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.691 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.691 DEBUG:boto:Method: GET 2024-04-16T18:26:48.691 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.691 DEBUG:boto:Data: 2024-04-16T18:26:48.691 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.691 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.691 DEBUG:boto:Params: {} 2024-04-16T18:26:48.691 DEBUG:boto:Token: None 2024-04-16T18:26:48.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic= 2024-04-16T18:26:48.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic=', 'Content-Length': '0'} 2024-04-16T18:26:48.694 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000582fe143fe1e4ab5-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.695 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.695 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.695 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.695 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.695 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.695 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.695 DEBUG:boto:Data: 2024-04-16T18:26:48.695 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.695 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.695 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.696 DEBUG:boto:Params: {} 2024-04-16T18:26:48.696 DEBUG:boto:Token: None 2024-04-16T18:26:48.696 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jeqmRub2sZRTJ/YAaN0R4oRh6KM= 2024-04-16T18:26:48.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jeqmRub2sZRTJ/YAaN0R4oRh6KM=', 'Content-Length': '0'} 2024-04-16T18:26:48.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007645f20efb1766e5-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.699 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.699 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.699 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.699 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.699 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.699 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.699 DEBUG:boto:Data: 2024-04-16T18:26:48.699 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.700 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.700 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.700 DEBUG:boto:Params: {} 2024-04-16T18:26:48.700 DEBUG:boto:Token: None 2024-04-16T18:26:48.700 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jeqmRub2sZRTJ/YAaN0R4oRh6KM= 2024-04-16T18:26:48.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jeqmRub2sZRTJ/YAaN0R4oRh6KM=', 'Content-Length': '0'} 2024-04-16T18:26:48.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4af7de0c827f0b0-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.703 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:26:48.703 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.703 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.704 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.704 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.704 DEBUG:boto:Method: GET 2024-04-16T18:26:48.704 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.704 DEBUG:boto:Data: 2024-04-16T18:26:48.704 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.704 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.704 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.704 DEBUG:boto:Params: {} 2024-04-16T18:26:48.704 DEBUG:boto:Token: None 2024-04-16T18:26:48.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.705 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic= 2024-04-16T18:26:48.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic=', 'Content-Length': '0'} 2024-04-16T18:26:48.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fe845edd4eecc3d-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.750 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.750 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.751 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.751 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.751 DEBUG:boto:Method: GET 2024-04-16T18:26:48.751 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.751 DEBUG:boto:Data: 2024-04-16T18:26:48.751 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.751 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.751 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.751 DEBUG:boto:Params: {} 2024-04-16T18:26:48.751 DEBUG:boto:Token: None 2024-04-16T18:26:48.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:26:48.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic= 2024-04-16T18:26:48.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O5GXUrnt8XNbMTC+eWFNB8HD7Ic=', 'Content-Length': '0'} 2024-04-16T18:26:48.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c15e0b8f051cdb2b-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.755 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.755 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.755 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.755 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.755 DEBUG:boto:Method: GET 2024-04-16T18:26:48.756 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:26:48.756 DEBUG:boto:Data: 2024-04-16T18:26:48.756 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.756 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.756 DEBUG:boto:Params: {} 2024-04-16T18:26:48.756 DEBUG:boto:Token: None 2024-04-16T18:26:48.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.760 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044998778e4b51f4a-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.760 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.761 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.761 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.761 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.761 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.761 DEBUG:boto:Method: GET 2024-04-16T18:26:48.761 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:26:48.761 DEBUG:boto:Data: 2024-04-16T18:26:48.761 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.761 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.761 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.762 DEBUG:boto:Params: {} 2024-04-16T18:26:48.762 DEBUG:boto:Token: None 2024-04-16T18:26:48.762 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.769 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001876c03556af4ab5-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.769 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.769 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.770 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.770 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.770 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:26:48.770 DEBUG:boto:Data: 2024-04-16T18:26:48.770 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.770 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.770 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.770 DEBUG:boto:Params: {} 2024-04-16T18:26:48.770 DEBUG:boto:Token: None 2024-04-16T18:26:48.771 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95 2024-04-16T18:26:48.771 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GvlWosAN9rc5DBwIbqhvmnvnjTE= 2024-04-16T18:26:48.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GvlWosAN9rc5DBwIbqhvmnvnjTE=', 'Content-Length': '0'} 2024-04-16T18:26:48.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000816fc5831c63712d-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.774 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.774 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.774 DEBUG:boto:Method: HEAD 2024-04-16T18:26:48.774 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:26:48.774 DEBUG:boto:Data: 2024-04-16T18:26:48.775 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.775 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.775 DEBUG:boto:Params: {} 2024-04-16T18:26:48.775 DEBUG:boto:Token: None 2024-04-16T18:26:48.775 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95 2024-04-16T18:26:48.775 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GvlWosAN9rc5DBwIbqhvmnvnjTE= 2024-04-16T18:26:48.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GvlWosAN9rc5DBwIbqhvmnvnjTE=', 'Content-Length': '0'} 2024-04-16T18:26:48.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000951475148fd5b9e2-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.778 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:26:48.778 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.779 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.779 DEBUG:boto:Method: GET 2024-04-16T18:26:48.779 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:26:48.779 DEBUG:boto:Data: 2024-04-16T18:26:48.779 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.779 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.779 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.779 DEBUG:boto:Params: {} 2024-04-16T18:26:48.779 DEBUG:boto:Token: None 2024-04-16T18:26:48.779 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95 2024-04-16T18:26:48.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cjd0YX7NnstNR1E0pLAhuzpMWuU= 2024-04-16T18:26:48.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cjd0YX7NnstNR1E0pLAhuzpMWuU=', 'Content-Length': '0'} 2024-04-16T18:26:48.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b16786ef43bfe9-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.826 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:26:48.826 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:26:48.827 DEBUG:boto:Method: GET 2024-04-16T18:26:48.827 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:26:48.827 DEBUG:boto:Data: 2024-04-16T18:26:48.827 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.827 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.827 DEBUG:boto:Params: {} 2024-04-16T18:26:48.827 DEBUG:boto:Token: None 2024-04-16T18:26:48.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/obj95 2024-04-16T18:26:48.827 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cjd0YX7NnstNR1E0pLAhuzpMWuU= 2024-04-16T18:26:48.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cjd0YX7NnstNR1E0pLAhuzpMWuU=', 'Content-Length': '0'} 2024-04-16T18:26:48.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0294fea6fd8891c-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.831 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.831 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.831 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.831 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.831 DEBUG:boto:Method: GET 2024-04-16T18:26:48.831 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:26:48.831 DEBUG:boto:Data: 2024-04-16T18:26:48.831 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.831 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:26:48.831 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.831 DEBUG:boto:Params: {} 2024-04-16T18:26:48.832 DEBUG:boto:Token: None 2024-04-16T18:26:48.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.835 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae7fc767d3fc9c3e-00661ec2e8-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.878 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.879 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:48.879 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:48.879 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.879 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:26:48.879 DEBUG:boto:Method: GET 2024-04-16T18:26:48.879 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:26:48.880 DEBUG:boto:Data: 2024-04-16T18:26:48.880 DEBUG:boto:Headers: {} 2024-04-16T18:26:48.880 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:48.880 DEBUG:boto:Port: 8000 2024-04-16T18:26:48.880 DEBUG:boto:Params: {} 2024-04-16T18:26:48.880 DEBUG:boto:Token: None 2024-04-16T18:26:48.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:26:48 GMT /lybtug-52/ 2024-04-16T18:26:48.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y= 2024-04-16T18:26:48.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3jqMokGreRP+jjM0TdB/hqvFv/Y=', 'Content-Length': '0'} 2024-04-16T18:26:48.886 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fcae7cd0547995a-00661ec2e8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:48.886 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:26:48.887 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:26:48.887 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '31', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:26:48.887 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '31', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:26:48.888 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --num-shards 31 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:26:48.933 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:26:48.933 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:26:49.041 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:26:49.041 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-52 2024-04-16T18:26:49.063 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 25 2024-04-16T18:26:49.228 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:49.228 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:49.229 DEBUG:boto:Method: HEAD 2024-04-16T18:26:49.229 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:49.229 DEBUG:boto:Data: 2024-04-16T18:26:49.229 DEBUG:boto:Headers: {} 2024-04-16T18:26:49.229 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.229 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.229 DEBUG:boto:Params: {} 2024-04-16T18:26:49.229 DEBUG:boto:Token: None 2024-04-16T18:26:49.229 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/ 2024-04-16T18:26:49.229 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw= 2024-04-16T18:26:49.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw=', 'Content-Length': '0'} 2024-04-16T18:26:49.235 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f6173b8d06342c51-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.235 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:26:49.235 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:26:49.235 DEBUG:boto:Method: PUT 2024-04-16T18:26:49.235 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:26:49.236 DEBUG:boto:Data: 2024-04-16T18:26:49.236 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:26:49.236 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.236 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.236 DEBUG:boto:Params: {} 2024-04-16T18:26:49.236 DEBUG:boto:Token: None 2024-04-16T18:26:49.236 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/obj31 2024-04-16T18:26:49.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ubyKPXrlsClEAhLdCoB7ijIl+2Y= 2024-04-16T18:26:49.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ubyKPXrlsClEAhLdCoB7ijIl+2Y='} 2024-04-16T18:26:49.247 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c173f1073fdd97f1-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.248 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:49.248 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:49.248 DEBUG:boto:Method: HEAD 2024-04-16T18:26:49.248 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:49.248 DEBUG:boto:Data: 2024-04-16T18:26:49.248 DEBUG:boto:Headers: {} 2024-04-16T18:26:49.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.248 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.248 DEBUG:boto:Params: {} 2024-04-16T18:26:49.248 DEBUG:boto:Token: None 2024-04-16T18:26:49.249 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/ 2024-04-16T18:26:49.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw= 2024-04-16T18:26:49.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw=', 'Content-Length': '0'} 2024-04-16T18:26:49.254 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a29d56a284d2e676-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.254 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:26:49.255 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:26:49.255 DEBUG:boto:Method: PUT 2024-04-16T18:26:49.255 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:26:49.255 DEBUG:boto:Data: 2024-04-16T18:26:49.255 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:26:49.255 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.255 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.255 DEBUG:boto:Params: {} 2024-04-16T18:26:49.256 DEBUG:boto:Token: None 2024-04-16T18:26:49.256 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/obj62 2024-04-16T18:26:49.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6apxqbMaDFXrOUrAJ+pP+fmIY2s= 2024-04-16T18:26:49.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6apxqbMaDFXrOUrAJ+pP+fmIY2s='} 2024-04-16T18:26:49.298 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000d84bf2f5634e1bf-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.299 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:49.299 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:49.299 DEBUG:boto:Method: HEAD 2024-04-16T18:26:49.299 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:49.299 DEBUG:boto:Data: 2024-04-16T18:26:49.299 DEBUG:boto:Headers: {} 2024-04-16T18:26:49.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.299 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.299 DEBUG:boto:Params: {} 2024-04-16T18:26:49.300 DEBUG:boto:Token: None 2024-04-16T18:26:49.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/ 2024-04-16T18:26:49.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw= 2024-04-16T18:26:49.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw=', 'Content-Length': '0'} 2024-04-16T18:26:49.306 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '81'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002c0f718f2393b0d3-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.306 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:26:49.306 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:26:49.306 DEBUG:boto:Method: PUT 2024-04-16T18:26:49.306 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:26:49.306 DEBUG:boto:Data: 2024-04-16T18:26:49.306 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:26:49.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.307 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.307 DEBUG:boto:Params: {} 2024-04-16T18:26:49.307 DEBUG:boto:Token: None 2024-04-16T18:26:49.307 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/obj93 2024-04-16T18:26:49.307 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TbYOrkioUKlBondTPttX+B79L4M= 2024-04-16T18:26:49.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TbYOrkioUKlBondTPttX+B79L4M='} 2024-04-16T18:26:49.320 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bd9414bc136d43a3-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.320 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:49.320 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:49.320 DEBUG:boto:Method: HEAD 2024-04-16T18:26:49.320 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:49.320 DEBUG:boto:Data: 2024-04-16T18:26:49.320 DEBUG:boto:Headers: {} 2024-04-16T18:26:49.321 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.321 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.321 DEBUG:boto:Params: {} 2024-04-16T18:26:49.321 DEBUG:boto:Token: None 2024-04-16T18:26:49.321 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/ 2024-04-16T18:26:49.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw= 2024-04-16T18:26:49.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw=', 'Content-Length': '0'} 2024-04-16T18:26:49.329 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000024950cf4d8b784fe-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.329 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:26:49.329 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:26:49.329 DEBUG:boto:Method: PUT 2024-04-16T18:26:49.329 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:26:49.329 DEBUG:boto:Data: 2024-04-16T18:26:49.329 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:26:49.329 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.329 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.329 DEBUG:boto:Params: {} 2024-04-16T18:26:49.330 DEBUG:boto:Token: None 2024-04-16T18:26:49.330 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/obj124 2024-04-16T18:26:49.330 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iysxDmDk5t//rccBbUJQgHBj6LQ= 2024-04-16T18:26:49.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iysxDmDk5t//rccBbUJQgHBj6LQ='} 2024-04-16T18:26:49.374 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d84ae84fda8b8579-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.375 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:26:49.375 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:26:49.375 DEBUG:boto:Method: HEAD 2024-04-16T18:26:49.375 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:26:49.375 DEBUG:boto:Data: 2024-04-16T18:26:49.375 DEBUG:boto:Headers: {} 2024-04-16T18:26:49.375 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.375 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.375 DEBUG:boto:Params: {} 2024-04-16T18:26:49.376 DEBUG:boto:Token: None 2024-04-16T18:26:49.376 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/ 2024-04-16T18:26:49.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw= 2024-04-16T18:26:49.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/+lZomXgaXHxBWCYeNtUX596Iaw=', 'Content-Length': '0'} 2024-04-16T18:26:49.381 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '87'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009e9b3560c5307f45-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.382 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:26:49.382 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:26:49.382 DEBUG:boto:Method: PUT 2024-04-16T18:26:49.382 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:26:49.382 DEBUG:boto:Data: 2024-04-16T18:26:49.382 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:26:49.382 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:26:49.382 DEBUG:boto:Port: 8000 2024-04-16T18:26:49.382 DEBUG:boto:Params: {} 2024-04-16T18:26:49.382 DEBUG:boto:Token: None 2024-04-16T18:26:49.383 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:26:49 GMT /lybtug-52/obj155 2024-04-16T18:26:49.383 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:F7uzqQK55DJo1OlOwEz6oA1KGFQ= 2024-04-16T18:26:49.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:26:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:F7uzqQK55DJo1OlOwEz6oA1KGFQ='} 2024-04-16T18:26:49.395 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002272deaa3d2601c5-00661ec2e9-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:26:49.395 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:26:49.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:26:49.395 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:26:49.395 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:26:49.445 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:26:49.445 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:26:49.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:49.529+0000 7f87739d9980 1 waiting to reach latest gen 5, on 4.. 2024-04-16T18:26:54.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:54.533+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:54.536 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:54.537 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:26:59.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:26:59.541+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:26:59.540 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:26:59.540 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:04.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:04.545+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:04.544 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:27:04.544 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:09.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:09.549+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:09.548 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:27:09.548 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:14.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:14.552+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:14.552 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:27:14.552 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:19.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:19.556+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:19.556 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:27:19.556 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:24.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:24.560+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:24.559 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:27:24.560 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:29.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:29.564+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:29.563 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2024-04-16T18:27:29.563 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:34.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:34.568+0000 7f87739d9980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:34.567 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , 00000000002.104.6, , , , 00000000001.74.6, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:34.567 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:39.572+0000 7f87739d9980 1 bucket sync caught up with source: 2024-04-16T18:27:39.571 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:39.571 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , 00000000002.104.6, , 00000000001.72.6, , 00000000001.74.6, , , , , , , , , , 00000000001.94.6, , , , , , , , , , , , , , 00000000001.51.6] 2024-04-16T18:27:39.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:39.572+0000 7f87739d9980 0 bucket checkpoint complete 2024-04-16T18:27:39.621 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:27:39.621 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:27:39.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:27:39.621 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:27:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:27:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:27:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:27:39.756+0000 7f2b6b850980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:39.753 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000004.1234.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000002.466.6, 00000000006.338.6, 00000000004.416.6] 2024-04-16T18:27:39.753 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000004.1234.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000003.479.6, 00000000006.338.6, 00000000004.416.6] 2024-04-16T18:27:44.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:27:44.760+0000 7f2b6b850980 1 bucket sync caught up with source: 2024-04-16T18:27:44.756 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000004.1234.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000003.479.6, 00000000006.338.6, 00000000004.416.6] 2024-04-16T18:27:44.756 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000004.1234.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000003.479.6, 00000000006.338.6, 00000000004.416.6] 2024-04-16T18:27:44.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:27:44.760+0000 7f2b6b850980 0 bucket checkpoint complete 2024-04-16T18:27:44.804 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:27:44.804 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.804 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.804 DEBUG:boto:Method: HEAD 2024-04-16T18:27:44.804 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:44.804 DEBUG:boto:Data: 2024-04-16T18:27:44.805 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.805 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.805 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.805 DEBUG:boto:Params: {} 2024-04-16T18:27:44.805 DEBUG:boto:Token: None 2024-04-16T18:27:44.805 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/ 2024-04-16T18:27:44.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NVhlv8VFBtbpEIMNQv3AxLKlTYk= 2024-04-16T18:27:44.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NVhlv8VFBtbpEIMNQv3AxLKlTYk=', 'Content-Length': '0'} 2024-04-16T18:27:44.809 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000077b1bb2f0c2aeb9a-00661ec320-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.809 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.810 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.810 DEBUG:boto:Method: HEAD 2024-04-16T18:27:44.810 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:44.810 DEBUG:boto:Data: 2024-04-16T18:27:44.810 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.810 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.810 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.810 DEBUG:boto:Params: {} 2024-04-16T18:27:44.810 DEBUG:boto:Token: None 2024-04-16T18:27:44.811 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/ 2024-04-16T18:27:44.811 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NVhlv8VFBtbpEIMNQv3AxLKlTYk= 2024-04-16T18:27:44.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NVhlv8VFBtbpEIMNQv3AxLKlTYk=', 'Content-Length': '0'} 2024-04-16T18:27:44.815 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000323d68b0f2538cfe-00661ec320-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.815 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:27:44.816 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.816 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.816 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.816 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.816 DEBUG:boto:Method: GET 2024-04-16T18:27:44.816 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.816 DEBUG:boto:Data: 2024-04-16T18:27:44.816 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.816 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.816 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.816 DEBUG:boto:Params: {} 2024-04-16T18:27:44.817 DEBUG:boto:Token: None 2024-04-16T18:27:44.817 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/?versions 2024-04-16T18:27:44.817 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg= 2024-04-16T18:27:44.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg=', 'Content-Length': '0'} 2024-04-16T18:27:44.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1d45641ef87269d-00661ec320-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.822 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:44.824 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:27:44.824 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:27:44.824 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:27:44.824 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2024-04-16T18:27:44.824 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:27:44.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:27:44.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2024-04-16T18:27:44.827 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:27:44.827 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:27:44.827 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.827 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.827 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.827 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.827 DEBUG:boto:Method: GET 2024-04-16T18:27:44.827 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.827 DEBUG:boto:Data: 2024-04-16T18:27:44.827 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.828 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.828 DEBUG:boto:Params: {} 2024-04-16T18:27:44.828 DEBUG:boto:Token: None 2024-04-16T18:27:44.828 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/?versions 2024-04-16T18:27:44.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg= 2024-04-16T18:27:44.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg=', 'Content-Length': '0'} 2024-04-16T18:27:44.834 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0df7801db052dba-00661ec320-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.835 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2024-04-16T18:27:44.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:27:44.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2024-04-16T18:27:44.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:27:44.839 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.839 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.839 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.839 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.839 DEBUG:boto:Method: GET 2024-04-16T18:27:44.839 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.839 DEBUG:boto:Data: 2024-04-16T18:27:44.839 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.839 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.839 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.839 DEBUG:boto:Params: {} 2024-04-16T18:27:44.840 DEBUG:boto:Token: None 2024-04-16T18:27:44.840 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/?versions 2024-04-16T18:27:44.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg= 2024-04-16T18:27:44.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg=', 'Content-Length': '0'} 2024-04-16T18:27:44.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbedf9175944795f-00661ec320-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.845 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:44.846 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.846 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.846 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.846 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.846 DEBUG:boto:Method: GET 2024-04-16T18:27:44.847 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:27:44.847 DEBUG:boto:Data: 2024-04-16T18:27:44.847 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.847 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.847 DEBUG:boto:Params: {} 2024-04-16T18:27:44.847 DEBUG:boto:Token: None 2024-04-16T18:27:44.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/?versions 2024-04-16T18:27:44.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg= 2024-04-16T18:27:44.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pskhsQcqmrbr1fiJDxwiy7TWpwg=', 'Content-Length': '0'} 2024-04-16T18:27:44.854 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec515c23f1aed2f1-00661ec320-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.855 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:44.856 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:27:44.856 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.856 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.856 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.856 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.856 DEBUG:boto:Method: GET 2024-04-16T18:27:44.856 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.857 DEBUG:boto:Data: 2024-04-16T18:27:44.857 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.857 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.857 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.857 DEBUG:boto:Params: {} 2024-04-16T18:27:44.857 DEBUG:boto:Token: None 2024-04-16T18:27:44.857 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M= 2024-04-16T18:27:44.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M=', 'Content-Length': '0'} 2024-04-16T18:27:44.860 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baa80a1e71c354f7-00661ec320-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.861 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.861 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.861 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.861 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.861 DEBUG:boto:Method: GET 2024-04-16T18:27:44.861 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.861 DEBUG:boto:Data: 2024-04-16T18:27:44.861 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.861 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.862 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.862 DEBUG:boto:Params: {} 2024-04-16T18:27:44.862 DEBUG:boto:Token: None 2024-04-16T18:27:44.862 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M= 2024-04-16T18:27:44.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M=', 'Content-Length': '0'} 2024-04-16T18:27:44.865 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba5bb8a94e04e9d7-00661ec320-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.906 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.906 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.907 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.907 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.907 DEBUG:boto:Method: HEAD 2024-04-16T18:27:44.907 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.907 DEBUG:boto:Data: 2024-04-16T18:27:44.907 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.907 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.907 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.907 DEBUG:boto:Params: {} 2024-04-16T18:27:44.909 DEBUG:boto:Token: None 2024-04-16T18:27:44.909 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2fDJo195XoN/viiD+60PQ6WYwAY= 2024-04-16T18:27:44.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2fDJo195XoN/viiD+60PQ6WYwAY=', 'Content-Length': '0'} 2024-04-16T18:27:44.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfbf26e23e29f6d8-00661ec320-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.913 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.913 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.913 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.913 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.913 DEBUG:boto:Method: HEAD 2024-04-16T18:27:44.913 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.913 DEBUG:boto:Data: 2024-04-16T18:27:44.913 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.913 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.913 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.913 DEBUG:boto:Params: {} 2024-04-16T18:27:44.914 DEBUG:boto:Token: None 2024-04-16T18:27:44.914 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.914 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2fDJo195XoN/viiD+60PQ6WYwAY= 2024-04-16T18:27:44.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2fDJo195XoN/viiD+60PQ6WYwAY=', 'Content-Length': '0'} 2024-04-16T18:27:44.917 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065c8da9a33edaf29-00661ec320-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.917 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:27:44.918 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.918 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.918 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.918 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.918 DEBUG:boto:Method: GET 2024-04-16T18:27:44.918 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.918 DEBUG:boto:Data: 2024-04-16T18:27:44.918 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.918 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.918 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.918 DEBUG:boto:Params: {} 2024-04-16T18:27:44.919 DEBUG:boto:Token: None 2024-04-16T18:27:44.919 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M= 2024-04-16T18:27:44.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M=', 'Content-Length': '0'} 2024-04-16T18:27:44.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c1dafa9b178d192-00661ec320-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.962 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.963 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.963 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.963 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.963 DEBUG:boto:Method: GET 2024-04-16T18:27:44.963 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.963 DEBUG:boto:Data: 2024-04-16T18:27:44.963 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.963 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.963 DEBUG:boto:Params: {} 2024-04-16T18:27:44.964 DEBUG:boto:Token: None 2024-04-16T18:27:44.964 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:27:44.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M= 2024-04-16T18:27:44.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JhqjSCsQjAF+JKkyxa9sRzLW24M=', 'Content-Length': '0'} 2024-04-16T18:27:44.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003eaf3b5bca080030-00661ec320-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.967 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.967 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.967 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:44.967 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:44.967 DEBUG:boto:Method: GET 2024-04-16T18:27:44.967 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:27:44.967 DEBUG:boto:Data: 2024-04-16T18:27:44.968 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.968 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.968 DEBUG:boto:Params: {} 2024-04-16T18:27:44.968 DEBUG:boto:Token: None 2024-04-16T18:27:44.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/ 2024-04-16T18:27:44.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vMXA/i+RIM3vtw6K9am1yThLZdo= 2024-04-16T18:27:44.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vMXA/i+RIM3vtw6K9am1yThLZdo=', 'Content-Length': '0'} 2024-04-16T18:27:44.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f923589b1a9dc65-00661ec320-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.972 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:44.973 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:44.973 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:44.973 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:44.973 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:44.973 DEBUG:boto:Method: GET 2024-04-16T18:27:44.973 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:27:44.973 DEBUG:boto:Data: 2024-04-16T18:27:44.973 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.973 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.973 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.973 DEBUG:boto:Params: {} 2024-04-16T18:27:44.973 DEBUG:boto:Token: None 2024-04-16T18:27:44.974 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/ 2024-04-16T18:27:44.974 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vMXA/i+RIM3vtw6K9am1yThLZdo= 2024-04-16T18:27:44.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vMXA/i+RIM3vtw6K9am1yThLZdo=', 'Content-Length': '0'} 2024-04-16T18:27:44.979 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000116b0996ced99fd-00661ec320-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.980 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:44.980 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.980 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.980 DEBUG:boto:Method: HEAD 2024-04-16T18:27:44.980 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:27:44.980 DEBUG:boto:Data: 2024-04-16T18:27:44.980 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.980 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.980 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.981 DEBUG:boto:Params: {} 2024-04-16T18:27:44.981 DEBUG:boto:Token: None 2024-04-16T18:27:44.981 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11 2024-04-16T18:27:44.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fy5QF2cztpNrWgmmKXrkFoVLV8g= 2024-04-16T18:27:44.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fy5QF2cztpNrWgmmKXrkFoVLV8g=', 'Content-Length': '0'} 2024-04-16T18:27:44.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000423399820a793014-00661ec320-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.984 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.984 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.984 DEBUG:boto:Method: HEAD 2024-04-16T18:27:44.984 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:27:44.984 DEBUG:boto:Data: 2024-04-16T18:27:44.984 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.984 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:44.984 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.985 DEBUG:boto:Params: {} 2024-04-16T18:27:44.985 DEBUG:boto:Token: None 2024-04-16T18:27:44.985 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11 2024-04-16T18:27:44.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fy5QF2cztpNrWgmmKXrkFoVLV8g= 2024-04-16T18:27:44.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fy5QF2cztpNrWgmmKXrkFoVLV8g=', 'Content-Length': '0'} 2024-04-16T18:27:44.988 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2fa660fde9e7566-00661ec320-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:44.988 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:27:44.988 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:44.989 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:44.989 DEBUG:boto:Method: GET 2024-04-16T18:27:44.989 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:27:44.989 DEBUG:boto:Data: 2024-04-16T18:27:44.989 DEBUG:boto:Headers: {} 2024-04-16T18:27:44.989 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:44.989 DEBUG:boto:Port: 8000 2024-04-16T18:27:44.989 DEBUG:boto:Params: {} 2024-04-16T18:27:44.989 DEBUG:boto:Token: None 2024-04-16T18:27:44.989 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:44 GMT /lybtug-52/obj11 2024-04-16T18:27:44.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aOxyDodOL4kBlSPGbalCAM2j0bs= 2024-04-16T18:27:44.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aOxyDodOL4kBlSPGbalCAM2j0bs=', 'Content-Length': '0'} 2024-04-16T18:27:44.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c49985036395290-00661ec320-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.034 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:27:45.035 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:27:45.035 DEBUG:boto:Method: GET 2024-04-16T18:27:45.035 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:27:45.035 DEBUG:boto:Data: 2024-04-16T18:27:45.035 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.035 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.035 DEBUG:boto:Params: {} 2024-04-16T18:27:45.035 DEBUG:boto:Token: None 2024-04-16T18:27:45.035 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj11 2024-04-16T18:27:45.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QUy0c9F2kU4J69HXSs14iYlcIxU= 2024-04-16T18:27:45.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QUy0c9F2kU4J69HXSs14iYlcIxU=', 'Content-Length': '0'} 2024-04-16T18:27:45.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ddf7dd3bfffa4a3-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.039 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.039 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.039 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:45.039 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:45.039 DEBUG:boto:Method: GET 2024-04-16T18:27:45.039 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:27:45.039 DEBUG:boto:Data: 2024-04-16T18:27:45.039 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.039 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.039 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.040 DEBUG:boto:Params: {} 2024-04-16T18:27:45.040 DEBUG:boto:Token: None 2024-04-16T18:27:45.040 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a7ddfa3f1d4e469-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.044 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.044 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.045 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.045 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:45.045 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:27:45.045 DEBUG:boto:Method: GET 2024-04-16T18:27:45.045 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:27:45.045 DEBUG:boto:Data: 2024-04-16T18:27:45.045 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.045 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.045 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.045 DEBUG:boto:Params: {} 2024-04-16T18:27:45.045 DEBUG:boto:Token: None 2024-04-16T18:27:45.045 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.046 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005697c448b154080-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.052 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.052 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:27:45.052 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.052 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.052 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.052 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.052 DEBUG:boto:Method: GET 2024-04-16T18:27:45.052 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.053 DEBUG:boto:Data: 2024-04-16T18:27:45.053 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.053 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.053 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.053 DEBUG:boto:Params: {} 2024-04-16T18:27:45.053 DEBUG:boto:Token: None 2024-04-16T18:27:45.053 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.053 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk= 2024-04-16T18:27:45.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk=', 'Content-Length': '0'} 2024-04-16T18:27:45.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f55171dc96d28d01-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.056 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.057 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.057 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.057 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.057 DEBUG:boto:Method: GET 2024-04-16T18:27:45.057 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.057 DEBUG:boto:Data: 2024-04-16T18:27:45.057 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.057 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.057 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.057 DEBUG:boto:Params: {} 2024-04-16T18:27:45.057 DEBUG:boto:Token: None 2024-04-16T18:27:45.057 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.058 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk= 2024-04-16T18:27:45.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk=', 'Content-Length': '0'} 2024-04-16T18:27:45.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f40cf2692bce5ec-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.061 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.061 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.061 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.061 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.061 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.061 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.061 DEBUG:boto:Data: 2024-04-16T18:27:45.062 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.062 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.062 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.062 DEBUG:boto:Params: {} 2024-04-16T18:27:45.062 DEBUG:boto:Token: None 2024-04-16T18:27:45.062 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.062 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pm1xlpGfdo4CMS5q+0xay/cdb8c= 2024-04-16T18:27:45.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pm1xlpGfdo4CMS5q+0xay/cdb8c=', 'Content-Length': '0'} 2024-04-16T18:27:45.065 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f0ff2a863ca28c8-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.065 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.066 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.066 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.066 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.066 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.066 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.066 DEBUG:boto:Data: 2024-04-16T18:27:45.066 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.066 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.066 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.066 DEBUG:boto:Params: {} 2024-04-16T18:27:45.066 DEBUG:boto:Token: None 2024-04-16T18:27:45.067 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.067 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pm1xlpGfdo4CMS5q+0xay/cdb8c= 2024-04-16T18:27:45.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pm1xlpGfdo4CMS5q+0xay/cdb8c=', 'Content-Length': '0'} 2024-04-16T18:27:45.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f051b258c5e4e15e-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.070 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:27:45.070 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.070 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.070 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.070 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.070 DEBUG:boto:Method: GET 2024-04-16T18:27:45.070 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.070 DEBUG:boto:Data: 2024-04-16T18:27:45.070 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.070 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.071 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.071 DEBUG:boto:Params: {} 2024-04-16T18:27:45.071 DEBUG:boto:Token: None 2024-04-16T18:27:45.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk= 2024-04-16T18:27:45.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk=', 'Content-Length': '0'} 2024-04-16T18:27:45.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9c7e3ce9d5de4a7-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.114 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.115 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.115 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.115 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.115 DEBUG:boto:Method: GET 2024-04-16T18:27:45.115 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.115 DEBUG:boto:Data: 2024-04-16T18:27:45.115 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.115 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.115 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.115 DEBUG:boto:Params: {} 2024-04-16T18:27:45.115 DEBUG:boto:Token: None 2024-04-16T18:27:45.115 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:27:45.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk= 2024-04-16T18:27:45.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dc7RUd9B9Dt5lBpHr5ZBWCnm9Pk=', 'Content-Length': '0'} 2024-04-16T18:27:45.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9b9328e546a0253-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.119 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.119 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.120 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.120 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.120 DEBUG:boto:Method: GET 2024-04-16T18:27:45.120 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:27:45.120 DEBUG:boto:Data: 2024-04-16T18:27:45.120 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.120 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.120 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.120 DEBUG:boto:Params: {} 2024-04-16T18:27:45.120 DEBUG:boto:Token: None 2024-04-16T18:27:45.121 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfa6b16672fa3106-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.124 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.125 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.125 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.125 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.125 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.125 DEBUG:boto:Method: GET 2024-04-16T18:27:45.125 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:27:45.125 DEBUG:boto:Data: 2024-04-16T18:27:45.125 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.125 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.125 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.126 DEBUG:boto:Params: {} 2024-04-16T18:27:45.126 DEBUG:boto:Token: None 2024-04-16T18:27:45.126 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.126 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000097514efa58ee714-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.132 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.132 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.132 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.132 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.132 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:27:45.132 DEBUG:boto:Data: 2024-04-16T18:27:45.133 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.133 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.133 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.133 DEBUG:boto:Params: {} 2024-04-16T18:27:45.133 DEBUG:boto:Token: None 2024-04-16T18:27:45.133 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115 2024-04-16T18:27:45.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uJk7/URqFY9mmnJTDCXb8FZO/90= 2024-04-16T18:27:45.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uJk7/URqFY9mmnJTDCXb8FZO/90=', 'Content-Length': '0'} 2024-04-16T18:27:45.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022c0c75c31be5b89-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.136 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.136 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.136 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.136 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:27:45.137 DEBUG:boto:Data: 2024-04-16T18:27:45.137 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.137 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.137 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.137 DEBUG:boto:Params: {} 2024-04-16T18:27:45.137 DEBUG:boto:Token: None 2024-04-16T18:27:45.137 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115 2024-04-16T18:27:45.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uJk7/URqFY9mmnJTDCXb8FZO/90= 2024-04-16T18:27:45.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uJk7/URqFY9mmnJTDCXb8FZO/90=', 'Content-Length': '0'} 2024-04-16T18:27:45.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056e0f34939864f75-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.140 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:27:45.140 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.140 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.140 DEBUG:boto:Method: GET 2024-04-16T18:27:45.141 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:27:45.141 DEBUG:boto:Data: 2024-04-16T18:27:45.141 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.141 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.141 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.141 DEBUG:boto:Params: {} 2024-04-16T18:27:45.141 DEBUG:boto:Token: None 2024-04-16T18:27:45.141 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115 2024-04-16T18:27:45.141 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b+FsH4cTs02MSY3k0Cv6/Rbxy3A= 2024-04-16T18:27:45.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b+FsH4cTs02MSY3k0Cv6/Rbxy3A=', 'Content-Length': '0'} 2024-04-16T18:27:45.145 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000099a1ad9a2495ade-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.186 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:27:45.187 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:27:45.187 DEBUG:boto:Method: GET 2024-04-16T18:27:45.187 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:27:45.187 DEBUG:boto:Data: 2024-04-16T18:27:45.187 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.187 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.187 DEBUG:boto:Params: {} 2024-04-16T18:27:45.187 DEBUG:boto:Token: None 2024-04-16T18:27:45.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj115 2024-04-16T18:27:45.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b+FsH4cTs02MSY3k0Cv6/Rbxy3A= 2024-04-16T18:27:45.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b+FsH4cTs02MSY3k0Cv6/Rbxy3A=', 'Content-Length': '0'} 2024-04-16T18:27:45.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059b64e7738db37a9-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.191 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.191 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.191 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.191 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.191 DEBUG:boto:Method: GET 2024-04-16T18:27:45.192 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:27:45.192 DEBUG:boto:Data: 2024-04-16T18:27:45.192 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.192 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.192 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.192 DEBUG:boto:Params: {} 2024-04-16T18:27:45.192 DEBUG:boto:Token: None 2024-04-16T18:27:45.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.195 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000289c385fae01d70e-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.196 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.196 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.196 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.196 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.196 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:27:45.196 DEBUG:boto:Method: GET 2024-04-16T18:27:45.196 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:27:45.197 DEBUG:boto:Data: 2024-04-16T18:27:45.197 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.197 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.197 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.197 DEBUG:boto:Params: {} 2024-04-16T18:27:45.197 DEBUG:boto:Token: None 2024-04-16T18:27:45.197 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005e091a39d710b70-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.246 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.247 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:27:45.247 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.247 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.247 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.247 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.247 DEBUG:boto:Method: GET 2024-04-16T18:27:45.247 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.247 DEBUG:boto:Data: 2024-04-16T18:27:45.247 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.248 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.248 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.248 DEBUG:boto:Params: {} 2024-04-16T18:27:45.248 DEBUG:boto:Token: None 2024-04-16T18:27:45.248 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw= 2024-04-16T18:27:45.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw=', 'Content-Length': '0'} 2024-04-16T18:27:45.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c17da8113b73577-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.294 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.294 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.295 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.295 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.295 DEBUG:boto:Method: GET 2024-04-16T18:27:45.295 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.295 DEBUG:boto:Data: 2024-04-16T18:27:45.295 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.295 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.295 DEBUG:boto:Params: {} 2024-04-16T18:27:45.295 DEBUG:boto:Token: None 2024-04-16T18:27:45.295 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.295 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw= 2024-04-16T18:27:45.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw=', 'Content-Length': '0'} 2024-04-16T18:27:45.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003079dc04ded0dd77-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.299 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.299 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.299 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.299 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.299 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.299 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.299 DEBUG:boto:Data: 2024-04-16T18:27:45.299 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.299 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.300 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.300 DEBUG:boto:Params: {} 2024-04-16T18:27:45.300 DEBUG:boto:Token: None 2024-04-16T18:27:45.300 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kabu6zfzYZu7azisks0uPE7EUKg= 2024-04-16T18:27:45.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kabu6zfzYZu7azisks0uPE7EUKg=', 'Content-Length': '0'} 2024-04-16T18:27:45.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002aeb4312cb548a2a-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.303 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.303 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.303 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.303 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.303 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.304 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.304 DEBUG:boto:Data: 2024-04-16T18:27:45.304 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.304 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.304 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.304 DEBUG:boto:Params: {} 2024-04-16T18:27:45.304 DEBUG:boto:Token: None 2024-04-16T18:27:45.304 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kabu6zfzYZu7azisks0uPE7EUKg= 2024-04-16T18:27:45.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kabu6zfzYZu7azisks0uPE7EUKg=', 'Content-Length': '0'} 2024-04-16T18:27:45.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085a9e50d13543167-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.308 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:27:45.308 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.308 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.308 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.308 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.308 DEBUG:boto:Method: GET 2024-04-16T18:27:45.308 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.308 DEBUG:boto:Data: 2024-04-16T18:27:45.308 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.308 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.308 DEBUG:boto:Params: {} 2024-04-16T18:27:45.309 DEBUG:boto:Token: None 2024-04-16T18:27:45.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw= 2024-04-16T18:27:45.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw=', 'Content-Length': '0'} 2024-04-16T18:27:45.312 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b622c0306e89c61-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.354 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.355 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.355 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.355 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.355 DEBUG:boto:Method: GET 2024-04-16T18:27:45.355 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.355 DEBUG:boto:Data: 2024-04-16T18:27:45.355 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.355 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.355 DEBUG:boto:Params: {} 2024-04-16T18:27:45.355 DEBUG:boto:Token: None 2024-04-16T18:27:45.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:27:45.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw= 2024-04-16T18:27:45.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8EZWcrznY4A3ZFBgDwW+6vYPSjw=', 'Content-Length': '0'} 2024-04-16T18:27:45.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000408432aacc252d8e-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.359 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.359 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.359 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.359 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.359 DEBUG:boto:Method: GET 2024-04-16T18:27:45.360 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:27:45.360 DEBUG:boto:Data: 2024-04-16T18:27:45.360 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.360 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.360 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.360 DEBUG:boto:Params: {} 2024-04-16T18:27:45.360 DEBUG:boto:Token: None 2024-04-16T18:27:45.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003caf075e1b5e3e5c-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.364 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.364 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.364 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.364 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.365 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.365 DEBUG:boto:Method: GET 2024-04-16T18:27:45.365 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:27:45.365 DEBUG:boto:Data: 2024-04-16T18:27:45.365 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.365 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.365 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.365 DEBUG:boto:Params: {} 2024-04-16T18:27:45.365 DEBUG:boto:Token: None 2024-04-16T18:27:45.365 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.365 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015923b9090c0810b-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.372 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.372 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.372 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.372 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.372 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:27:45.372 DEBUG:boto:Data: 2024-04-16T18:27:45.372 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.372 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.373 DEBUG:boto:Params: {} 2024-04-16T18:27:45.373 DEBUG:boto:Token: None 2024-04-16T18:27:45.373 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116 2024-04-16T18:27:45.373 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mQwJxtxxOh9S+m0IWh0vGR597Z8= 2024-04-16T18:27:45.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mQwJxtxxOh9S+m0IWh0vGR597Z8=', 'Content-Length': '0'} 2024-04-16T18:27:45.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a3d5569d87247fb-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.376 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.376 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.376 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.376 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:27:45.377 DEBUG:boto:Data: 2024-04-16T18:27:45.377 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.377 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.377 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.377 DEBUG:boto:Params: {} 2024-04-16T18:27:45.377 DEBUG:boto:Token: None 2024-04-16T18:27:45.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116 2024-04-16T18:27:45.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mQwJxtxxOh9S+m0IWh0vGR597Z8= 2024-04-16T18:27:45.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mQwJxtxxOh9S+m0IWh0vGR597Z8=', 'Content-Length': '0'} 2024-04-16T18:27:45.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000856ed829caaa0ce7-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:27:45.380 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.380 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.380 DEBUG:boto:Method: GET 2024-04-16T18:27:45.380 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:27:45.381 DEBUG:boto:Data: 2024-04-16T18:27:45.381 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.381 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.381 DEBUG:boto:Params: {} 2024-04-16T18:27:45.381 DEBUG:boto:Token: None 2024-04-16T18:27:45.381 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116 2024-04-16T18:27:45.381 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NjOvpUtzSJTWMLY09Krg7yHO570= 2024-04-16T18:27:45.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NjOvpUtzSJTWMLY09Krg7yHO570=', 'Content-Length': '0'} 2024-04-16T18:27:45.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8cb0af5f1506b8f-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.426 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:27:45.427 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:27:45.427 DEBUG:boto:Method: GET 2024-04-16T18:27:45.427 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:27:45.427 DEBUG:boto:Data: 2024-04-16T18:27:45.427 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.427 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.427 DEBUG:boto:Params: {} 2024-04-16T18:27:45.427 DEBUG:boto:Token: None 2024-04-16T18:27:45.427 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj116 2024-04-16T18:27:45.427 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NjOvpUtzSJTWMLY09Krg7yHO570= 2024-04-16T18:27:45.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NjOvpUtzSJTWMLY09Krg7yHO570=', 'Content-Length': '0'} 2024-04-16T18:27:45.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cd85381494429a4-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.431 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.431 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.431 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.431 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.431 DEBUG:boto:Method: GET 2024-04-16T18:27:45.431 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:27:45.432 DEBUG:boto:Data: 2024-04-16T18:27:45.432 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.432 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.432 DEBUG:boto:Params: {} 2024-04-16T18:27:45.432 DEBUG:boto:Token: None 2024-04-16T18:27:45.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef28dbdd0a556c55-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.436 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.436 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.436 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.436 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.437 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:27:45.437 DEBUG:boto:Method: GET 2024-04-16T18:27:45.437 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:27:45.437 DEBUG:boto:Data: 2024-04-16T18:27:45.437 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.437 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.437 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.437 DEBUG:boto:Params: {} 2024-04-16T18:27:45.437 DEBUG:boto:Token: None 2024-04-16T18:27:45.437 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d36914a9e1f6607b-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.443 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.444 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:27:45.444 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.444 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.444 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.444 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.444 DEBUG:boto:Method: GET 2024-04-16T18:27:45.444 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.444 DEBUG:boto:Data: 2024-04-16T18:27:45.444 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.444 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.444 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.444 DEBUG:boto:Params: {} 2024-04-16T18:27:45.445 DEBUG:boto:Token: None 2024-04-16T18:27:45.445 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.445 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA= 2024-04-16T18:27:45.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA=', 'Content-Length': '0'} 2024-04-16T18:27:45.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9b0364c98bc4764-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.490 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.490 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.491 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.491 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.491 DEBUG:boto:Method: GET 2024-04-16T18:27:45.491 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.491 DEBUG:boto:Data: 2024-04-16T18:27:45.491 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.491 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.491 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.491 DEBUG:boto:Params: {} 2024-04-16T18:27:45.491 DEBUG:boto:Token: None 2024-04-16T18:27:45.491 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA= 2024-04-16T18:27:45.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA=', 'Content-Length': '0'} 2024-04-16T18:27:45.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000211f9fb741df752b-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.495 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.495 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.495 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.495 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.495 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.495 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.495 DEBUG:boto:Data: 2024-04-16T18:27:45.495 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.496 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.496 DEBUG:boto:Params: {} 2024-04-16T18:27:45.496 DEBUG:boto:Token: None 2024-04-16T18:27:45.496 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.496 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:29Oz79h4sbZTR2uqHfeK6uOZmxo= 2024-04-16T18:27:45.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:29Oz79h4sbZTR2uqHfeK6uOZmxo=', 'Content-Length': '0'} 2024-04-16T18:27:45.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041f2632618ce6539-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.499 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.499 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.499 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.499 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.499 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.499 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.500 DEBUG:boto:Data: 2024-04-16T18:27:45.500 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.500 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.500 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.500 DEBUG:boto:Params: {} 2024-04-16T18:27:45.500 DEBUG:boto:Token: None 2024-04-16T18:27:45.500 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:29Oz79h4sbZTR2uqHfeK6uOZmxo= 2024-04-16T18:27:45.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:29Oz79h4sbZTR2uqHfeK6uOZmxo=', 'Content-Length': '0'} 2024-04-16T18:27:45.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003059ea13fcc69aac-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:27:45.503 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.504 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.504 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.504 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.504 DEBUG:boto:Method: GET 2024-04-16T18:27:45.504 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.504 DEBUG:boto:Data: 2024-04-16T18:27:45.504 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.504 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.504 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.504 DEBUG:boto:Params: {} 2024-04-16T18:27:45.504 DEBUG:boto:Token: None 2024-04-16T18:27:45.504 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.505 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA= 2024-04-16T18:27:45.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA=', 'Content-Length': '0'} 2024-04-16T18:27:45.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001517ac24b5ef08fb-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.550 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.550 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.551 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.551 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.551 DEBUG:boto:Method: GET 2024-04-16T18:27:45.551 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.551 DEBUG:boto:Data: 2024-04-16T18:27:45.551 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.551 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.551 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.551 DEBUG:boto:Params: {} 2024-04-16T18:27:45.551 DEBUG:boto:Token: None 2024-04-16T18:27:45.551 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:27:45.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA= 2024-04-16T18:27:45.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xv+hX0WqjmsfH9Yc2yeuBWEk1pA=', 'Content-Length': '0'} 2024-04-16T18:27:45.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4e969747c72840e-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.555 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.555 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.555 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.555 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.555 DEBUG:boto:Method: GET 2024-04-16T18:27:45.555 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:27:45.555 DEBUG:boto:Data: 2024-04-16T18:27:45.555 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.555 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.555 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.555 DEBUG:boto:Params: {} 2024-04-16T18:27:45.555 DEBUG:boto:Token: None 2024-04-16T18:27:45.556 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007219644979aa5706-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.560 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.560 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.560 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.560 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.560 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.560 DEBUG:boto:Method: GET 2024-04-16T18:27:45.560 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:27:45.561 DEBUG:boto:Data: 2024-04-16T18:27:45.561 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.561 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.561 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.561 DEBUG:boto:Params: {} 2024-04-16T18:27:45.561 DEBUG:boto:Token: None 2024-04-16T18:27:45.561 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.561 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.567 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1d7134ed70f7362-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.610 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.611 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.611 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.611 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.611 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:27:45.611 DEBUG:boto:Data: 2024-04-16T18:27:45.611 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.611 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.611 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.611 DEBUG:boto:Params: {} 2024-04-16T18:27:45.612 DEBUG:boto:Token: None 2024-04-16T18:27:45.612 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124 2024-04-16T18:27:45.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gV/OAGLkN2EPnq4EbFBvGl1bSpM= 2024-04-16T18:27:45.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gV/OAGLkN2EPnq4EbFBvGl1bSpM=', 'Content-Length': '0'} 2024-04-16T18:27:45.615 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b5c974d6a3114ad-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.615 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.615 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.615 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.615 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:27:45.615 DEBUG:boto:Data: 2024-04-16T18:27:45.616 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.616 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.616 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.616 DEBUG:boto:Params: {} 2024-04-16T18:27:45.616 DEBUG:boto:Token: None 2024-04-16T18:27:45.616 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124 2024-04-16T18:27:45.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gV/OAGLkN2EPnq4EbFBvGl1bSpM= 2024-04-16T18:27:45.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gV/OAGLkN2EPnq4EbFBvGl1bSpM=', 'Content-Length': '0'} 2024-04-16T18:27:45.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000115c62834b1e17f8-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:27:45.619 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.619 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.619 DEBUG:boto:Method: GET 2024-04-16T18:27:45.619 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:27:45.619 DEBUG:boto:Data: 2024-04-16T18:27:45.620 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.620 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.620 DEBUG:boto:Params: {} 2024-04-16T18:27:45.620 DEBUG:boto:Token: None 2024-04-16T18:27:45.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124 2024-04-16T18:27:45.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UnRGZFBkMuDuJt7U917/J7wB5LY= 2024-04-16T18:27:45.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UnRGZFBkMuDuJt7U917/J7wB5LY=', 'Content-Length': '0'} 2024-04-16T18:27:45.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009580180bc357be0-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.666 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:27:45.666 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:27:45.666 DEBUG:boto:Method: GET 2024-04-16T18:27:45.667 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:27:45.667 DEBUG:boto:Data: 2024-04-16T18:27:45.667 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.667 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.667 DEBUG:boto:Params: {} 2024-04-16T18:27:45.667 DEBUG:boto:Token: None 2024-04-16T18:27:45.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj124 2024-04-16T18:27:45.667 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UnRGZFBkMuDuJt7U917/J7wB5LY= 2024-04-16T18:27:45.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UnRGZFBkMuDuJt7U917/J7wB5LY=', 'Content-Length': '0'} 2024-04-16T18:27:45.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d7eecaff9cfe15f-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.671 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.671 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.671 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.671 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.671 DEBUG:boto:Method: GET 2024-04-16T18:27:45.671 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:27:45.671 DEBUG:boto:Data: 2024-04-16T18:27:45.671 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.671 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.672 DEBUG:boto:Params: {} 2024-04-16T18:27:45.672 DEBUG:boto:Token: None 2024-04-16T18:27:45.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000559933e1a3d6c214-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.718 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.719 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.719 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.719 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.719 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:27:45.719 DEBUG:boto:Method: GET 2024-04-16T18:27:45.719 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:27:45.719 DEBUG:boto:Data: 2024-04-16T18:27:45.719 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.719 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.719 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.719 DEBUG:boto:Params: {} 2024-04-16T18:27:45.720 DEBUG:boto:Token: None 2024-04-16T18:27:45.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.729 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7b9830e89b41ded-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.729 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.730 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:27:45.730 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.730 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.730 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.730 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.730 DEBUG:boto:Method: GET 2024-04-16T18:27:45.730 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.730 DEBUG:boto:Data: 2024-04-16T18:27:45.730 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.730 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.730 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.731 DEBUG:boto:Params: {} 2024-04-16T18:27:45.731 DEBUG:boto:Token: None 2024-04-16T18:27:45.731 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.731 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g= 2024-04-16T18:27:45.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g=', 'Content-Length': '0'} 2024-04-16T18:27:45.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056b5ea1ea764e980-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.774 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.774 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.774 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.775 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.775 DEBUG:boto:Method: GET 2024-04-16T18:27:45.775 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.775 DEBUG:boto:Data: 2024-04-16T18:27:45.775 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.775 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.775 DEBUG:boto:Params: {} 2024-04-16T18:27:45.775 DEBUG:boto:Token: None 2024-04-16T18:27:45.775 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.775 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g= 2024-04-16T18:27:45.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g=', 'Content-Length': '0'} 2024-04-16T18:27:45.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000722bcc5d6a3ea8e-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.779 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.779 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.779 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.779 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.779 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.779 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.779 DEBUG:boto:Data: 2024-04-16T18:27:45.779 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.779 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.779 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.779 DEBUG:boto:Params: {} 2024-04-16T18:27:45.780 DEBUG:boto:Token: None 2024-04-16T18:27:45.780 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7OQO87S4x7g5F1/itFCGDR7iZro= 2024-04-16T18:27:45.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7OQO87S4x7g5F1/itFCGDR7iZro=', 'Content-Length': '0'} 2024-04-16T18:27:45.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ccec48e7259eaa75-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.783 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.783 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.783 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.783 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.783 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.783 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.784 DEBUG:boto:Data: 2024-04-16T18:27:45.784 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.784 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.784 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.784 DEBUG:boto:Params: {} 2024-04-16T18:27:45.784 DEBUG:boto:Token: None 2024-04-16T18:27:45.784 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7OQO87S4x7g5F1/itFCGDR7iZro= 2024-04-16T18:27:45.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7OQO87S4x7g5F1/itFCGDR7iZro=', 'Content-Length': '0'} 2024-04-16T18:27:45.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be7ded0580b324ac-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.787 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:27:45.787 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.787 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.788 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.788 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.788 DEBUG:boto:Method: GET 2024-04-16T18:27:45.788 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.788 DEBUG:boto:Data: 2024-04-16T18:27:45.788 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.788 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.788 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.788 DEBUG:boto:Params: {} 2024-04-16T18:27:45.788 DEBUG:boto:Token: None 2024-04-16T18:27:45.788 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.789 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g= 2024-04-16T18:27:45.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g=', 'Content-Length': '0'} 2024-04-16T18:27:45.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031d3fb95e10531c4-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.834 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.834 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.834 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.835 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.835 DEBUG:boto:Method: GET 2024-04-16T18:27:45.835 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.835 DEBUG:boto:Data: 2024-04-16T18:27:45.835 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.835 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.835 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.835 DEBUG:boto:Params: {} 2024-04-16T18:27:45.835 DEBUG:boto:Token: None 2024-04-16T18:27:45.835 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:27:45.835 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g= 2024-04-16T18:27:45.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kalYGWzRQdXgWGwff5ew4MD/92g=', 'Content-Length': '0'} 2024-04-16T18:27:45.838 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000103580ca105eeaaa-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.839 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.839 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.839 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:45.839 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:45.839 DEBUG:boto:Method: GET 2024-04-16T18:27:45.839 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:27:45.839 DEBUG:boto:Data: 2024-04-16T18:27:45.839 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.839 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.839 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.839 DEBUG:boto:Params: {} 2024-04-16T18:27:45.839 DEBUG:boto:Token: None 2024-04-16T18:27:45.840 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.843 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e752e46048efc30-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.844 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.844 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.844 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.844 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:45.844 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:45.844 DEBUG:boto:Method: GET 2024-04-16T18:27:45.844 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:27:45.845 DEBUG:boto:Data: 2024-04-16T18:27:45.845 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.845 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.845 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.845 DEBUG:boto:Params: {} 2024-04-16T18:27:45.845 DEBUG:boto:Token: None 2024-04-16T18:27:45.845 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.845 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.851 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d426dfdb37e8e26-00661ec321-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.894 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:45.895 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.895 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.895 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.895 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:27:45.895 DEBUG:boto:Data: 2024-04-16T18:27:45.895 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.895 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.895 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.895 DEBUG:boto:Params: {} 2024-04-16T18:27:45.895 DEBUG:boto:Token: None 2024-04-16T18:27:45.895 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145 2024-04-16T18:27:45.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WK15yhWduUvYM5Lf8FifYD1WMdE= 2024-04-16T18:27:45.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WK15yhWduUvYM5Lf8FifYD1WMdE=', 'Content-Length': '0'} 2024-04-16T18:27:45.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca8963dc59d4137a-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.899 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.899 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.899 DEBUG:boto:Method: HEAD 2024-04-16T18:27:45.899 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:27:45.899 DEBUG:boto:Data: 2024-04-16T18:27:45.899 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.900 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.900 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.900 DEBUG:boto:Params: {} 2024-04-16T18:27:45.900 DEBUG:boto:Token: None 2024-04-16T18:27:45.900 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145 2024-04-16T18:27:45.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WK15yhWduUvYM5Lf8FifYD1WMdE= 2024-04-16T18:27:45.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WK15yhWduUvYM5Lf8FifYD1WMdE=', 'Content-Length': '0'} 2024-04-16T18:27:45.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000114a58948ad1ebcf-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.903 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:27:45.903 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.903 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.904 DEBUG:boto:Method: GET 2024-04-16T18:27:45.904 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:27:45.904 DEBUG:boto:Data: 2024-04-16T18:27:45.904 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.904 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.904 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.904 DEBUG:boto:Params: {} 2024-04-16T18:27:45.904 DEBUG:boto:Token: None 2024-04-16T18:27:45.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145 2024-04-16T18:27:45.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N0VCNRPyJJ4qo0HUmNAPOpANM2g= 2024-04-16T18:27:45.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N0VCNRPyJJ4qo0HUmNAPOpANM2g=', 'Content-Length': '0'} 2024-04-16T18:27:45.908 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ceedb1d972d1698-00661ec321-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.950 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:27:45.950 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:27:45.951 DEBUG:boto:Method: GET 2024-04-16T18:27:45.951 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:27:45.951 DEBUG:boto:Data: 2024-04-16T18:27:45.951 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.951 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:45.951 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.951 DEBUG:boto:Params: {} 2024-04-16T18:27:45.951 DEBUG:boto:Token: None 2024-04-16T18:27:45.951 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/obj145 2024-04-16T18:27:45.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N0VCNRPyJJ4qo0HUmNAPOpANM2g= 2024-04-16T18:27:45.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N0VCNRPyJJ4qo0HUmNAPOpANM2g=', 'Content-Length': '0'} 2024-04-16T18:27:45.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d63c18d02e154a02-00661ec321-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:45.955 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:45.955 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:45.955 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:45.955 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:45.955 DEBUG:boto:Method: GET 2024-04-16T18:27:45.955 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:27:45.955 DEBUG:boto:Data: 2024-04-16T18:27:45.955 DEBUG:boto:Headers: {} 2024-04-16T18:27:45.956 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:45.956 DEBUG:boto:Port: 8000 2024-04-16T18:27:45.956 DEBUG:boto:Params: {} 2024-04-16T18:27:45.956 DEBUG:boto:Token: None 2024-04-16T18:27:45.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:45 GMT /lybtug-52/ 2024-04-16T18:27:45.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ= 2024-04-16T18:27:45.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dczPua2D4O5Vn/k1lFwNFEtDgBQ=', 'Content-Length': '0'} 2024-04-16T18:27:45.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4dc7d875bfc0a7f-00661ec321-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.002 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.003 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.003 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.003 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:46.003 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:27:46.003 DEBUG:boto:Method: GET 2024-04-16T18:27:46.003 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:27:46.003 DEBUG:boto:Data: 2024-04-16T18:27:46.003 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.003 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.003 DEBUG:boto:Params: {} 2024-04-16T18:27:46.004 DEBUG:boto:Token: None 2024-04-16T18:27:46.004 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000189cd5125e5e69be-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.011 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.011 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:27:46.012 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.012 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.012 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.012 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.012 DEBUG:boto:Method: GET 2024-04-16T18:27:46.012 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.012 DEBUG:boto:Data: 2024-04-16T18:27:46.012 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.012 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.012 DEBUG:boto:Params: {} 2024-04-16T18:27:46.013 DEBUG:boto:Token: None 2024-04-16T18:27:46.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0= 2024-04-16T18:27:46.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0=', 'Content-Length': '0'} 2024-04-16T18:27:46.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f17ef9b0487d485e-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.058 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.058 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.059 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.059 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.059 DEBUG:boto:Method: GET 2024-04-16T18:27:46.059 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.059 DEBUG:boto:Data: 2024-04-16T18:27:46.059 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.059 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.059 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.059 DEBUG:boto:Params: {} 2024-04-16T18:27:46.059 DEBUG:boto:Token: None 2024-04-16T18:27:46.059 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0= 2024-04-16T18:27:46.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0=', 'Content-Length': '0'} 2024-04-16T18:27:46.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c658d20cc931dba-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.063 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.063 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.063 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.063 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.063 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.063 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.064 DEBUG:boto:Data: 2024-04-16T18:27:46.064 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.064 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.064 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.064 DEBUG:boto:Params: {} 2024-04-16T18:27:46.064 DEBUG:boto:Token: None 2024-04-16T18:27:46.064 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PK6GMdQET9tAXFn9K8etY1MN0yI= 2024-04-16T18:27:46.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PK6GMdQET9tAXFn9K8etY1MN0yI=', 'Content-Length': '0'} 2024-04-16T18:27:46.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf159d9e6beb1038-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.068 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.068 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.068 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.068 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.068 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.068 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.068 DEBUG:boto:Data: 2024-04-16T18:27:46.068 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.068 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.068 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.069 DEBUG:boto:Params: {} 2024-04-16T18:27:46.069 DEBUG:boto:Token: None 2024-04-16T18:27:46.069 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.069 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PK6GMdQET9tAXFn9K8etY1MN0yI= 2024-04-16T18:27:46.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PK6GMdQET9tAXFn9K8etY1MN0yI=', 'Content-Length': '0'} 2024-04-16T18:27:46.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006caccf88d828d051-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:27:46.072 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.072 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.072 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.072 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.073 DEBUG:boto:Method: GET 2024-04-16T18:27:46.073 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.073 DEBUG:boto:Data: 2024-04-16T18:27:46.073 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.073 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.073 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.073 DEBUG:boto:Params: {} 2024-04-16T18:27:46.073 DEBUG:boto:Token: None 2024-04-16T18:27:46.073 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0= 2024-04-16T18:27:46.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0=', 'Content-Length': '0'} 2024-04-16T18:27:46.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000373ae3e899a696a7-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.118 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.118 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.119 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.119 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.119 DEBUG:boto:Method: GET 2024-04-16T18:27:46.119 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.119 DEBUG:boto:Data: 2024-04-16T18:27:46.119 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.119 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.119 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.119 DEBUG:boto:Params: {} 2024-04-16T18:27:46.119 DEBUG:boto:Token: None 2024-04-16T18:27:46.119 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:27:46.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0= 2024-04-16T18:27:46.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9afjMmqe/sT/gZDHyR9rHQXto0=', 'Content-Length': '0'} 2024-04-16T18:27:46.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f320a7e17d5cb734-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.123 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.123 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.123 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.123 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.123 DEBUG:boto:Method: GET 2024-04-16T18:27:46.123 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:27:46.123 DEBUG:boto:Data: 2024-04-16T18:27:46.124 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.124 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.124 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.124 DEBUG:boto:Params: {} 2024-04-16T18:27:46.124 DEBUG:boto:Token: None 2024-04-16T18:27:46.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.128 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bf405f693978d22-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.129 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.129 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.129 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.129 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.129 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.129 DEBUG:boto:Method: GET 2024-04-16T18:27:46.129 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:27:46.129 DEBUG:boto:Data: 2024-04-16T18:27:46.130 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.130 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.130 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.130 DEBUG:boto:Params: {} 2024-04-16T18:27:46.130 DEBUG:boto:Token: None 2024-04-16T18:27:46.130 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.130 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a64961379e0af0a-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.178 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.179 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.179 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.179 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.179 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:27:46.179 DEBUG:boto:Data: 2024-04-16T18:27:46.179 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.179 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.179 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.179 DEBUG:boto:Params: {} 2024-04-16T18:27:46.180 DEBUG:boto:Token: None 2024-04-16T18:27:46.180 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155 2024-04-16T18:27:46.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cvS2oogkilJ65dKAX55r3nUftGQ= 2024-04-16T18:27:46.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cvS2oogkilJ65dKAX55r3nUftGQ=', 'Content-Length': '0'} 2024-04-16T18:27:46.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000135882b16276f14d-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.183 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.183 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.183 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.183 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:27:46.184 DEBUG:boto:Data: 2024-04-16T18:27:46.184 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.184 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.184 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.184 DEBUG:boto:Params: {} 2024-04-16T18:27:46.184 DEBUG:boto:Token: None 2024-04-16T18:27:46.184 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155 2024-04-16T18:27:46.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cvS2oogkilJ65dKAX55r3nUftGQ= 2024-04-16T18:27:46.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cvS2oogkilJ65dKAX55r3nUftGQ=', 'Content-Length': '0'} 2024-04-16T18:27:46.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbf5968281051735-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:27:46.188 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.188 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.188 DEBUG:boto:Method: GET 2024-04-16T18:27:46.188 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:27:46.188 DEBUG:boto:Data: 2024-04-16T18:27:46.188 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.188 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.188 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.188 DEBUG:boto:Params: {} 2024-04-16T18:27:46.189 DEBUG:boto:Token: None 2024-04-16T18:27:46.189 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155 2024-04-16T18:27:46.189 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MyDgrYAjDUqch+LNoGEbFQibVAs= 2024-04-16T18:27:46.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MyDgrYAjDUqch+LNoGEbFQibVAs=', 'Content-Length': '0'} 2024-04-16T18:27:46.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000704a6eae8c2c9c7-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.234 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:27:46.234 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:27:46.235 DEBUG:boto:Method: GET 2024-04-16T18:27:46.235 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:27:46.235 DEBUG:boto:Data: 2024-04-16T18:27:46.235 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.235 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.235 DEBUG:boto:Params: {} 2024-04-16T18:27:46.235 DEBUG:boto:Token: None 2024-04-16T18:27:46.235 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj155 2024-04-16T18:27:46.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MyDgrYAjDUqch+LNoGEbFQibVAs= 2024-04-16T18:27:46.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MyDgrYAjDUqch+LNoGEbFQibVAs=', 'Content-Length': '0'} 2024-04-16T18:27:46.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000588558bdd2f64251-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.239 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.239 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.239 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.239 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.239 DEBUG:boto:Method: GET 2024-04-16T18:27:46.239 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:27:46.239 DEBUG:boto:Data: 2024-04-16T18:27:46.240 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.240 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.240 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.240 DEBUG:boto:Params: {} 2024-04-16T18:27:46.240 DEBUG:boto:Token: None 2024-04-16T18:27:46.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b6c0ce437d50554-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.286 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.287 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.287 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:27:46.287 DEBUG:boto:Method: GET 2024-04-16T18:27:46.287 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:27:46.287 DEBUG:boto:Data: 2024-04-16T18:27:46.287 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.287 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.287 DEBUG:boto:Params: {} 2024-04-16T18:27:46.288 DEBUG:boto:Token: None 2024-04-16T18:27:46.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.294 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da2f6b2a558f2a13-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.295 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:27:46.295 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.295 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.295 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.295 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.295 DEBUG:boto:Method: GET 2024-04-16T18:27:46.296 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.296 DEBUG:boto:Data: 2024-04-16T18:27:46.296 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.296 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.296 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.296 DEBUG:boto:Params: {} 2024-04-16T18:27:46.296 DEBUG:boto:Token: None 2024-04-16T18:27:46.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw= 2024-04-16T18:27:46.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw=', 'Content-Length': '0'} 2024-04-16T18:27:46.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f89321082472b711-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.342 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.342 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.343 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.343 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.343 DEBUG:boto:Method: GET 2024-04-16T18:27:46.343 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.343 DEBUG:boto:Data: 2024-04-16T18:27:46.343 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.343 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.343 DEBUG:boto:Params: {} 2024-04-16T18:27:46.343 DEBUG:boto:Token: None 2024-04-16T18:27:46.343 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw= 2024-04-16T18:27:46.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw=', 'Content-Length': '0'} 2024-04-16T18:27:46.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf9615b34e8085ce-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.348 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.348 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.348 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.348 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.348 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.348 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.348 DEBUG:boto:Data: 2024-04-16T18:27:46.348 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.348 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.348 DEBUG:boto:Params: {} 2024-04-16T18:27:46.349 DEBUG:boto:Token: None 2024-04-16T18:27:46.349 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ojXc/v8P6dVkLb65RpDH4tQupaY= 2024-04-16T18:27:46.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ojXc/v8P6dVkLb65RpDH4tQupaY=', 'Content-Length': '0'} 2024-04-16T18:27:46.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d82beed2f7f7b8b-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.352 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.352 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.352 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.352 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.352 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.352 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.352 DEBUG:boto:Data: 2024-04-16T18:27:46.352 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.352 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.353 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.353 DEBUG:boto:Params: {} 2024-04-16T18:27:46.353 DEBUG:boto:Token: None 2024-04-16T18:27:46.353 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ojXc/v8P6dVkLb65RpDH4tQupaY= 2024-04-16T18:27:46.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ojXc/v8P6dVkLb65RpDH4tQupaY=', 'Content-Length': '0'} 2024-04-16T18:27:46.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000674c109f487fdd28-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:27:46.356 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.356 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.357 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.357 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.357 DEBUG:boto:Method: GET 2024-04-16T18:27:46.357 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.357 DEBUG:boto:Data: 2024-04-16T18:27:46.357 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.357 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.357 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.357 DEBUG:boto:Params: {} 2024-04-16T18:27:46.357 DEBUG:boto:Token: None 2024-04-16T18:27:46.357 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.358 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw= 2024-04-16T18:27:46.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw=', 'Content-Length': '0'} 2024-04-16T18:27:46.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2624308d8b7bfc4-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.402 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.402 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.402 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.403 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.403 DEBUG:boto:Method: GET 2024-04-16T18:27:46.403 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.403 DEBUG:boto:Data: 2024-04-16T18:27:46.403 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.403 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.403 DEBUG:boto:Params: {} 2024-04-16T18:27:46.403 DEBUG:boto:Token: None 2024-04-16T18:27:46.403 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:27:46.403 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw= 2024-04-16T18:27:46.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4G7M8puDz9pozYWZw8m3KVlBurw=', 'Content-Length': '0'} 2024-04-16T18:27:46.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f61eb17cd9bf0d6-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.407 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.407 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.407 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.407 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.407 DEBUG:boto:Method: GET 2024-04-16T18:27:46.407 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:27:46.408 DEBUG:boto:Data: 2024-04-16T18:27:46.408 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.408 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.408 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.408 DEBUG:boto:Params: {} 2024-04-16T18:27:46.408 DEBUG:boto:Token: None 2024-04-16T18:27:46.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007aa0f9d0525fadcc-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.412 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.412 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.412 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.412 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.413 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.413 DEBUG:boto:Method: GET 2024-04-16T18:27:46.413 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:27:46.413 DEBUG:boto:Data: 2024-04-16T18:27:46.413 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.413 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.413 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.413 DEBUG:boto:Params: {} 2024-04-16T18:27:46.413 DEBUG:boto:Token: None 2024-04-16T18:27:46.413 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.413 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.421 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b05e225da6ce899f-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.462 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.462 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.463 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.463 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.463 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:27:46.463 DEBUG:boto:Data: 2024-04-16T18:27:46.463 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.463 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.463 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.463 DEBUG:boto:Params: {} 2024-04-16T18:27:46.463 DEBUG:boto:Token: None 2024-04-16T18:27:46.463 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17 2024-04-16T18:27:46.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pzdVKZlociLAw/mYEOsfYHao0eI= 2024-04-16T18:27:46.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pzdVKZlociLAw/mYEOsfYHao0eI=', 'Content-Length': '0'} 2024-04-16T18:27:46.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdc4e46bbbd06d20-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.467 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.467 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.467 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.467 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:27:46.467 DEBUG:boto:Data: 2024-04-16T18:27:46.467 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.467 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.467 DEBUG:boto:Params: {} 2024-04-16T18:27:46.467 DEBUG:boto:Token: None 2024-04-16T18:27:46.468 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17 2024-04-16T18:27:46.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pzdVKZlociLAw/mYEOsfYHao0eI= 2024-04-16T18:27:46.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pzdVKZlociLAw/mYEOsfYHao0eI=', 'Content-Length': '0'} 2024-04-16T18:27:46.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005fc2f34d70dc65b9-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.471 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:27:46.471 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.471 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.471 DEBUG:boto:Method: GET 2024-04-16T18:27:46.471 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:27:46.471 DEBUG:boto:Data: 2024-04-16T18:27:46.471 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.471 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.471 DEBUG:boto:Params: {} 2024-04-16T18:27:46.472 DEBUG:boto:Token: None 2024-04-16T18:27:46.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17 2024-04-16T18:27:46.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5YI/RgpeFWTt1iUrNCfFRPTX34I= 2024-04-16T18:27:46.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5YI/RgpeFWTt1iUrNCfFRPTX34I=', 'Content-Length': '0'} 2024-04-16T18:27:46.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfb60c44b2e9fed8-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.518 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:27:46.518 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:27:46.519 DEBUG:boto:Method: GET 2024-04-16T18:27:46.519 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:27:46.519 DEBUG:boto:Data: 2024-04-16T18:27:46.519 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.519 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.519 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.519 DEBUG:boto:Params: {} 2024-04-16T18:27:46.519 DEBUG:boto:Token: None 2024-04-16T18:27:46.519 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj17 2024-04-16T18:27:46.519 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5YI/RgpeFWTt1iUrNCfFRPTX34I= 2024-04-16T18:27:46.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5YI/RgpeFWTt1iUrNCfFRPTX34I=', 'Content-Length': '0'} 2024-04-16T18:27:46.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca5e41333a9f51ce-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.523 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.523 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.523 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.523 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.523 DEBUG:boto:Method: GET 2024-04-16T18:27:46.523 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:27:46.523 DEBUG:boto:Data: 2024-04-16T18:27:46.524 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.524 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.524 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.524 DEBUG:boto:Params: {} 2024-04-16T18:27:46.524 DEBUG:boto:Token: None 2024-04-16T18:27:46.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.527 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b843832ccfa6bc34-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.570 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.571 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.571 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.571 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.571 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:27:46.571 DEBUG:boto:Method: GET 2024-04-16T18:27:46.571 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:27:46.571 DEBUG:boto:Data: 2024-04-16T18:27:46.571 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.571 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.571 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.572 DEBUG:boto:Params: {} 2024-04-16T18:27:46.572 DEBUG:boto:Token: None 2024-04-16T18:27:46.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f343b0cd750e0f90-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.578 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.578 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:27:46.578 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.579 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.579 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.579 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.579 DEBUG:boto:Method: GET 2024-04-16T18:27:46.579 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.579 DEBUG:boto:Data: 2024-04-16T18:27:46.579 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.579 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.579 DEBUG:boto:Params: {} 2024-04-16T18:27:46.579 DEBUG:boto:Token: None 2024-04-16T18:27:46.579 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ= 2024-04-16T18:27:46.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ=', 'Content-Length': '0'} 2024-04-16T18:27:46.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c26e7cecbdeafd5-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.626 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.627 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.627 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.627 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.627 DEBUG:boto:Method: GET 2024-04-16T18:27:46.627 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.627 DEBUG:boto:Data: 2024-04-16T18:27:46.627 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.627 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.627 DEBUG:boto:Params: {} 2024-04-16T18:27:46.627 DEBUG:boto:Token: None 2024-04-16T18:27:46.627 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ= 2024-04-16T18:27:46.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ=', 'Content-Length': '0'} 2024-04-16T18:27:46.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017a88a6e158ea7b6-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.631 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.631 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.631 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.631 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.632 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.632 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.632 DEBUG:boto:Data: 2024-04-16T18:27:46.632 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.632 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.632 DEBUG:boto:Params: {} 2024-04-16T18:27:46.632 DEBUG:boto:Token: None 2024-04-16T18:27:46.632 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z40758swJKUw89NLfpK8nUCx0Z0= 2024-04-16T18:27:46.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z40758swJKUw89NLfpK8nUCx0Z0=', 'Content-Length': '0'} 2024-04-16T18:27:46.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1d5d68cb49c9d4d-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.636 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.636 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.636 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.636 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.636 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.636 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.636 DEBUG:boto:Data: 2024-04-16T18:27:46.636 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.636 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.637 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.637 DEBUG:boto:Params: {} 2024-04-16T18:27:46.637 DEBUG:boto:Token: None 2024-04-16T18:27:46.637 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.637 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z40758swJKUw89NLfpK8nUCx0Z0= 2024-04-16T18:27:46.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z40758swJKUw89NLfpK8nUCx0Z0=', 'Content-Length': '0'} 2024-04-16T18:27:46.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db8bb01771805efa-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:27:46.640 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.640 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.640 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.641 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.641 DEBUG:boto:Method: GET 2024-04-16T18:27:46.641 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.641 DEBUG:boto:Data: 2024-04-16T18:27:46.641 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.641 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.641 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.641 DEBUG:boto:Params: {} 2024-04-16T18:27:46.641 DEBUG:boto:Token: None 2024-04-16T18:27:46.641 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.641 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ= 2024-04-16T18:27:46.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ=', 'Content-Length': '0'} 2024-04-16T18:27:46.645 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b12ff37ac17dc05-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.686 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.687 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.687 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.687 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.687 DEBUG:boto:Method: GET 2024-04-16T18:27:46.687 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.687 DEBUG:boto:Data: 2024-04-16T18:27:46.687 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.687 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.687 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.687 DEBUG:boto:Params: {} 2024-04-16T18:27:46.687 DEBUG:boto:Token: None 2024-04-16T18:27:46.688 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:27:46.688 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ= 2024-04-16T18:27:46.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UG4oodp2XDl1596+rV11RM41rqQ=', 'Content-Length': '0'} 2024-04-16T18:27:46.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f518c15c84328ba-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.691 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.691 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.691 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.691 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.691 DEBUG:boto:Method: GET 2024-04-16T18:27:46.691 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:27:46.692 DEBUG:boto:Data: 2024-04-16T18:27:46.692 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.692 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.692 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.692 DEBUG:boto:Params: {} 2024-04-16T18:27:46.692 DEBUG:boto:Token: None 2024-04-16T18:27:46.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063cee152b71c07f7-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.696 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.697 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.697 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.697 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.697 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.697 DEBUG:boto:Method: GET 2024-04-16T18:27:46.697 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:27:46.697 DEBUG:boto:Data: 2024-04-16T18:27:46.697 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.697 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.697 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.697 DEBUG:boto:Params: {} 2024-04-16T18:27:46.698 DEBUG:boto:Token: None 2024-04-16T18:27:46.698 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.698 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000552fbf4f0eb417cf-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.746 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.749 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.749 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.749 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.749 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:27:46.749 DEBUG:boto:Data: 2024-04-16T18:27:46.750 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.750 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.750 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.750 DEBUG:boto:Params: {} 2024-04-16T18:27:46.750 DEBUG:boto:Token: None 2024-04-16T18:27:46.750 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19 2024-04-16T18:27:46.750 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MLcSIO+TCG2ed3YT3AckjDWOZ3E= 2024-04-16T18:27:46.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MLcSIO+TCG2ed3YT3AckjDWOZ3E=', 'Content-Length': '0'} 2024-04-16T18:27:46.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002423f31c57cc4275-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.753 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.753 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.754 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.754 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:27:46.754 DEBUG:boto:Data: 2024-04-16T18:27:46.754 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.754 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.754 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.754 DEBUG:boto:Params: {} 2024-04-16T18:27:46.754 DEBUG:boto:Token: None 2024-04-16T18:27:46.754 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19 2024-04-16T18:27:46.754 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MLcSIO+TCG2ed3YT3AckjDWOZ3E= 2024-04-16T18:27:46.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MLcSIO+TCG2ed3YT3AckjDWOZ3E=', 'Content-Length': '0'} 2024-04-16T18:27:46.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e06e1f47b9da5982-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.757 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:27:46.758 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.758 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.758 DEBUG:boto:Method: GET 2024-04-16T18:27:46.758 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:27:46.758 DEBUG:boto:Data: 2024-04-16T18:27:46.758 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.758 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.758 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.758 DEBUG:boto:Params: {} 2024-04-16T18:27:46.758 DEBUG:boto:Token: None 2024-04-16T18:27:46.759 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19 2024-04-16T18:27:46.759 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:URHEKalwEi15oQ4oGFvX8QrNotM= 2024-04-16T18:27:46.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:URHEKalwEi15oQ4oGFvX8QrNotM=', 'Content-Length': '0'} 2024-04-16T18:27:46.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014597e142d63044d-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.802 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:27:46.802 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:27:46.803 DEBUG:boto:Method: GET 2024-04-16T18:27:46.803 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:27:46.803 DEBUG:boto:Data: 2024-04-16T18:27:46.803 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.803 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.803 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.803 DEBUG:boto:Params: {} 2024-04-16T18:27:46.803 DEBUG:boto:Token: None 2024-04-16T18:27:46.803 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj19 2024-04-16T18:27:46.803 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:URHEKalwEi15oQ4oGFvX8QrNotM= 2024-04-16T18:27:46.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:URHEKalwEi15oQ4oGFvX8QrNotM=', 'Content-Length': '0'} 2024-04-16T18:27:46.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb4cb5e36543d22a-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.807 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.807 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.807 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.807 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.807 DEBUG:boto:Method: GET 2024-04-16T18:27:46.807 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:27:46.807 DEBUG:boto:Data: 2024-04-16T18:27:46.807 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.807 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.807 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.807 DEBUG:boto:Params: {} 2024-04-16T18:27:46.807 DEBUG:boto:Token: None 2024-04-16T18:27:46.808 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.811 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfd11cc5b42f194d-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.811 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.812 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.812 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.812 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.812 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:27:46.812 DEBUG:boto:Method: GET 2024-04-16T18:27:46.812 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:27:46.812 DEBUG:boto:Data: 2024-04-16T18:27:46.812 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.812 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.813 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.813 DEBUG:boto:Params: {} 2024-04-16T18:27:46.813 DEBUG:boto:Token: None 2024-04-16T18:27:46.813 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.813 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.819 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec8a02e6ff64d089-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.819 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:27:46.820 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.820 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.820 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.820 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.820 DEBUG:boto:Method: GET 2024-04-16T18:27:46.820 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.820 DEBUG:boto:Data: 2024-04-16T18:27:46.820 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.820 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.821 DEBUG:boto:Params: {} 2024-04-16T18:27:46.821 DEBUG:boto:Token: None 2024-04-16T18:27:46.821 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs= 2024-04-16T18:27:46.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs=', 'Content-Length': '0'} 2024-04-16T18:27:46.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004be26fa7ce985159-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.866 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.866 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.866 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.867 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.867 DEBUG:boto:Method: GET 2024-04-16T18:27:46.867 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.867 DEBUG:boto:Data: 2024-04-16T18:27:46.867 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.867 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.867 DEBUG:boto:Params: {} 2024-04-16T18:27:46.867 DEBUG:boto:Token: None 2024-04-16T18:27:46.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs= 2024-04-16T18:27:46.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs=', 'Content-Length': '0'} 2024-04-16T18:27:46.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009169db3333c6d670-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.871 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.871 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.871 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.871 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.871 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.871 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.871 DEBUG:boto:Data: 2024-04-16T18:27:46.871 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.871 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.871 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.871 DEBUG:boto:Params: {} 2024-04-16T18:27:46.872 DEBUG:boto:Token: None 2024-04-16T18:27:46.872 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NrhgyQRju68W8nBkViilxS1J+UE= 2024-04-16T18:27:46.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NrhgyQRju68W8nBkViilxS1J+UE=', 'Content-Length': '0'} 2024-04-16T18:27:46.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000522cbf4ca6d4cc7f-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.875 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.875 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.875 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.875 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.875 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.875 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.875 DEBUG:boto:Data: 2024-04-16T18:27:46.875 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.875 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.875 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.875 DEBUG:boto:Params: {} 2024-04-16T18:27:46.875 DEBUG:boto:Token: None 2024-04-16T18:27:46.876 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:NrhgyQRju68W8nBkViilxS1J+UE= 2024-04-16T18:27:46.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:NrhgyQRju68W8nBkViilxS1J+UE=', 'Content-Length': '0'} 2024-04-16T18:27:46.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005dc9e87aee558b5a-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.879 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:27:46.879 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.879 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.879 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.879 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.879 DEBUG:boto:Method: GET 2024-04-16T18:27:46.879 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.879 DEBUG:boto:Data: 2024-04-16T18:27:46.879 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.879 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.879 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.880 DEBUG:boto:Params: {} 2024-04-16T18:27:46.880 DEBUG:boto:Token: None 2024-04-16T18:27:46.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs= 2024-04-16T18:27:46.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs=', 'Content-Length': '0'} 2024-04-16T18:27:46.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062dc519a51731229-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.926 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.927 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.927 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.927 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.927 DEBUG:boto:Method: GET 2024-04-16T18:27:46.927 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.927 DEBUG:boto:Data: 2024-04-16T18:27:46.927 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.927 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.927 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.927 DEBUG:boto:Params: {} 2024-04-16T18:27:46.927 DEBUG:boto:Token: None 2024-04-16T18:27:46.927 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:27:46.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs= 2024-04-16T18:27:46.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9t0HUQohGkojszMNAiJ5OSe6sDs=', 'Content-Length': '0'} 2024-04-16T18:27:46.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc237cbf0b1ba0f9-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.931 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.931 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.931 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:46.931 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:46.931 DEBUG:boto:Method: GET 2024-04-16T18:27:46.931 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:27:46.931 DEBUG:boto:Data: 2024-04-16T18:27:46.931 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.931 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.931 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.931 DEBUG:boto:Params: {} 2024-04-16T18:27:46.932 DEBUG:boto:Token: None 2024-04-16T18:27:46.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac9f94d58f4d0478-00661ec322-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.978 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.979 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:46.979 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:46.979 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:46.979 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:46.979 DEBUG:boto:Method: GET 2024-04-16T18:27:46.979 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:27:46.979 DEBUG:boto:Data: 2024-04-16T18:27:46.979 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.979 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.979 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.979 DEBUG:boto:Params: {} 2024-04-16T18:27:46.980 DEBUG:boto:Token: None 2024-04-16T18:27:46.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/ 2024-04-16T18:27:46.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk= 2024-04-16T18:27:46.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K32w10jQ+OKlt5onxwl+XEsRukk=', 'Content-Length': '0'} 2024-04-16T18:27:46.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003bbd85b04032b718-00661ec322-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.986 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:46.986 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.986 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.986 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.987 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:27:46.987 DEBUG:boto:Data: 2024-04-16T18:27:46.987 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.987 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.987 DEBUG:boto:Params: {} 2024-04-16T18:27:46.987 DEBUG:boto:Token: None 2024-04-16T18:27:46.987 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22 2024-04-16T18:27:46.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L6EZK4uFm8d5Bx+fARNBAI4KfK0= 2024-04-16T18:27:46.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L6EZK4uFm8d5Bx+fARNBAI4KfK0=', 'Content-Length': '0'} 2024-04-16T18:27:46.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee8685d249612d26-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.990 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.990 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.991 DEBUG:boto:Method: HEAD 2024-04-16T18:27:46.991 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:27:46.991 DEBUG:boto:Data: 2024-04-16T18:27:46.991 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.991 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:46.991 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.991 DEBUG:boto:Params: {} 2024-04-16T18:27:46.991 DEBUG:boto:Token: None 2024-04-16T18:27:46.991 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22 2024-04-16T18:27:46.991 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L6EZK4uFm8d5Bx+fARNBAI4KfK0= 2024-04-16T18:27:46.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L6EZK4uFm8d5Bx+fARNBAI4KfK0=', 'Content-Length': '0'} 2024-04-16T18:27:46.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057c8e4c32d238ba7-00661ec322-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:46.994 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:27:46.995 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:46.995 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:46.995 DEBUG:boto:Method: GET 2024-04-16T18:27:46.995 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:27:46.995 DEBUG:boto:Data: 2024-04-16T18:27:46.995 DEBUG:boto:Headers: {} 2024-04-16T18:27:46.995 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:46.995 DEBUG:boto:Port: 8000 2024-04-16T18:27:46.995 DEBUG:boto:Params: {} 2024-04-16T18:27:46.995 DEBUG:boto:Token: None 2024-04-16T18:27:46.996 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:46 GMT /lybtug-52/obj22 2024-04-16T18:27:46.996 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y/sXPaq9BnTN0vXI20eH4uFbc/c= 2024-04-16T18:27:46.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y/sXPaq9BnTN0vXI20eH4uFbc/c=', 'Content-Length': '0'} 2024-04-16T18:27:46.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005146cb4ab87f3cff-00661ec322-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.042 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:27:47.042 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:27:47.043 DEBUG:boto:Method: GET 2024-04-16T18:27:47.043 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:27:47.043 DEBUG:boto:Data: 2024-04-16T18:27:47.043 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.043 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.043 DEBUG:boto:Params: {} 2024-04-16T18:27:47.043 DEBUG:boto:Token: None 2024-04-16T18:27:47.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj22 2024-04-16T18:27:47.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qfWQv5+XVQ0gTTK4Kd67hDUI3nc= 2024-04-16T18:27:47.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qfWQv5+XVQ0gTTK4Kd67hDUI3nc=', 'Content-Length': '0'} 2024-04-16T18:27:47.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000415c764d3544e2e5-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.047 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.047 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.047 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:47.047 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:47.047 DEBUG:boto:Method: GET 2024-04-16T18:27:47.047 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:27:47.047 DEBUG:boto:Data: 2024-04-16T18:27:47.048 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.048 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.048 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.048 DEBUG:boto:Params: {} 2024-04-16T18:27:47.048 DEBUG:boto:Token: None 2024-04-16T18:27:47.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d29dba73180426c4-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.052 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.052 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.052 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.053 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:47.053 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:27:47.053 DEBUG:boto:Method: GET 2024-04-16T18:27:47.053 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:27:47.053 DEBUG:boto:Data: 2024-04-16T18:27:47.053 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.053 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.053 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.053 DEBUG:boto:Params: {} 2024-04-16T18:27:47.053 DEBUG:boto:Token: None 2024-04-16T18:27:47.054 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.054 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b05500ae77ee4000-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.061 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.061 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:27:47.061 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.061 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.062 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.062 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.062 DEBUG:boto:Method: GET 2024-04-16T18:27:47.062 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.062 DEBUG:boto:Data: 2024-04-16T18:27:47.062 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.062 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.062 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.062 DEBUG:boto:Params: {} 2024-04-16T18:27:47.062 DEBUG:boto:Token: None 2024-04-16T18:27:47.062 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.063 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI= 2024-04-16T18:27:47.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI=', 'Content-Length': '0'} 2024-04-16T18:27:47.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000eb53d45febed8f8-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.106 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.106 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.107 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.107 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.107 DEBUG:boto:Method: GET 2024-04-16T18:27:47.107 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.107 DEBUG:boto:Data: 2024-04-16T18:27:47.107 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.107 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.107 DEBUG:boto:Params: {} 2024-04-16T18:27:47.107 DEBUG:boto:Token: None 2024-04-16T18:27:47.107 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI= 2024-04-16T18:27:47.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI=', 'Content-Length': '0'} 2024-04-16T18:27:47.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ac99ce48991fd54-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.111 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.111 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.111 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.111 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.111 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.111 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.112 DEBUG:boto:Data: 2024-04-16T18:27:47.112 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.112 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.112 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.112 DEBUG:boto:Params: {} 2024-04-16T18:27:47.112 DEBUG:boto:Token: None 2024-04-16T18:27:47.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ivi41UWIp7XuynKwQPqA697tL3s= 2024-04-16T18:27:47.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ivi41UWIp7XuynKwQPqA697tL3s=', 'Content-Length': '0'} 2024-04-16T18:27:47.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e373e5868df7e7d2-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.115 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.115 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.115 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.116 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.116 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.116 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.116 DEBUG:boto:Data: 2024-04-16T18:27:47.116 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.116 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.116 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.116 DEBUG:boto:Params: {} 2024-04-16T18:27:47.116 DEBUG:boto:Token: None 2024-04-16T18:27:47.116 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ivi41UWIp7XuynKwQPqA697tL3s= 2024-04-16T18:27:47.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ivi41UWIp7XuynKwQPqA697tL3s=', 'Content-Length': '0'} 2024-04-16T18:27:47.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c3204271e97839e-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.119 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:27:47.119 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.119 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.120 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.120 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.120 DEBUG:boto:Method: GET 2024-04-16T18:27:47.120 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.120 DEBUG:boto:Data: 2024-04-16T18:27:47.120 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.120 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.120 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.120 DEBUG:boto:Params: {} 2024-04-16T18:27:47.120 DEBUG:boto:Token: None 2024-04-16T18:27:47.120 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI= 2024-04-16T18:27:47.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI=', 'Content-Length': '0'} 2024-04-16T18:27:47.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000966e39386326d9ad-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.166 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.166 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.167 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.167 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.167 DEBUG:boto:Method: GET 2024-04-16T18:27:47.167 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.167 DEBUG:boto:Data: 2024-04-16T18:27:47.167 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.167 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.167 DEBUG:boto:Params: {} 2024-04-16T18:27:47.167 DEBUG:boto:Token: None 2024-04-16T18:27:47.167 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:27:47.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI= 2024-04-16T18:27:47.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:43w2pY+4b3iKYGxFQeouW8DqIrI=', 'Content-Length': '0'} 2024-04-16T18:27:47.170 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000096f3d2ca675bfb0-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.171 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.171 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.171 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.171 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.171 DEBUG:boto:Method: GET 2024-04-16T18:27:47.171 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:27:47.171 DEBUG:boto:Data: 2024-04-16T18:27:47.171 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.171 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.172 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.172 DEBUG:boto:Params: {} 2024-04-16T18:27:47.172 DEBUG:boto:Token: None 2024-04-16T18:27:47.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c9add0383df19b0-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.218 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.219 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.219 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.219 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.219 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.219 DEBUG:boto:Method: GET 2024-04-16T18:27:47.219 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:27:47.219 DEBUG:boto:Data: 2024-04-16T18:27:47.219 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.219 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.219 DEBUG:boto:Params: {} 2024-04-16T18:27:47.220 DEBUG:boto:Token: None 2024-04-16T18:27:47.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000953289b27d1489ad-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.226 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.227 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.227 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.227 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.227 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:27:47.227 DEBUG:boto:Data: 2024-04-16T18:27:47.227 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.227 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.227 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.227 DEBUG:boto:Params: {} 2024-04-16T18:27:47.228 DEBUG:boto:Token: None 2024-04-16T18:27:47.228 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23 2024-04-16T18:27:47.228 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cEoMHDDnkYyqJRSFdsvaqQ55BEQ= 2024-04-16T18:27:47.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cEoMHDDnkYyqJRSFdsvaqQ55BEQ=', 'Content-Length': '0'} 2024-04-16T18:27:47.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1826abfec558cb5-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.231 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.231 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.231 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.232 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:27:47.232 DEBUG:boto:Data: 2024-04-16T18:27:47.232 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.232 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.232 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.232 DEBUG:boto:Params: {} 2024-04-16T18:27:47.232 DEBUG:boto:Token: None 2024-04-16T18:27:47.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23 2024-04-16T18:27:47.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cEoMHDDnkYyqJRSFdsvaqQ55BEQ= 2024-04-16T18:27:47.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cEoMHDDnkYyqJRSFdsvaqQ55BEQ=', 'Content-Length': '0'} 2024-04-16T18:27:47.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ab1ec14e33dac29-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:27:47.236 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.236 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.236 DEBUG:boto:Method: GET 2024-04-16T18:27:47.236 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:27:47.236 DEBUG:boto:Data: 2024-04-16T18:27:47.236 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.236 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.236 DEBUG:boto:Params: {} 2024-04-16T18:27:47.236 DEBUG:boto:Token: None 2024-04-16T18:27:47.237 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23 2024-04-16T18:27:47.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jsiiTM79blIS1BeLSpcIabhNCC0= 2024-04-16T18:27:47.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jsiiTM79blIS1BeLSpcIabhNCC0=', 'Content-Length': '0'} 2024-04-16T18:27:47.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4288985d21b1f50-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.282 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:27:47.283 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:27:47.283 DEBUG:boto:Method: GET 2024-04-16T18:27:47.283 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:27:47.283 DEBUG:boto:Data: 2024-04-16T18:27:47.283 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.283 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.283 DEBUG:boto:Params: {} 2024-04-16T18:27:47.283 DEBUG:boto:Token: None 2024-04-16T18:27:47.283 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj23 2024-04-16T18:27:47.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jsiiTM79blIS1BeLSpcIabhNCC0= 2024-04-16T18:27:47.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jsiiTM79blIS1BeLSpcIabhNCC0=', 'Content-Length': '0'} 2024-04-16T18:27:47.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f4fde234bc2b21e-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.287 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.287 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.287 DEBUG:boto:Method: GET 2024-04-16T18:27:47.287 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:27:47.287 DEBUG:boto:Data: 2024-04-16T18:27:47.287 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.287 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.287 DEBUG:boto:Params: {} 2024-04-16T18:27:47.288 DEBUG:boto:Token: None 2024-04-16T18:27:47.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac7f963626c1cbbc-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.334 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.335 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.335 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.335 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.335 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:27:47.335 DEBUG:boto:Method: GET 2024-04-16T18:27:47.335 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:27:47.335 DEBUG:boto:Data: 2024-04-16T18:27:47.335 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.335 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.335 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.335 DEBUG:boto:Params: {} 2024-04-16T18:27:47.335 DEBUG:boto:Token: None 2024-04-16T18:27:47.336 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.336 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.345 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b359149fed509b67-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.345 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:27:47.346 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.346 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.346 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.346 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.346 DEBUG:boto:Method: GET 2024-04-16T18:27:47.346 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.346 DEBUG:boto:Data: 2024-04-16T18:27:47.346 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.346 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.346 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.347 DEBUG:boto:Params: {} 2024-04-16T18:27:47.347 DEBUG:boto:Token: None 2024-04-16T18:27:47.347 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.347 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI= 2024-04-16T18:27:47.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI=', 'Content-Length': '0'} 2024-04-16T18:27:47.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001567ddbd567e68e2-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.390 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.390 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.391 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.391 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.391 DEBUG:boto:Method: GET 2024-04-16T18:27:47.391 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.391 DEBUG:boto:Data: 2024-04-16T18:27:47.391 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.391 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.391 DEBUG:boto:Params: {} 2024-04-16T18:27:47.391 DEBUG:boto:Token: None 2024-04-16T18:27:47.392 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI= 2024-04-16T18:27:47.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI=', 'Content-Length': '0'} 2024-04-16T18:27:47.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b791a16515b21fad-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.395 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.395 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.395 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.395 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.395 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.395 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.396 DEBUG:boto:Data: 2024-04-16T18:27:47.396 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.396 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.396 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.396 DEBUG:boto:Params: {} 2024-04-16T18:27:47.396 DEBUG:boto:Token: None 2024-04-16T18:27:47.396 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eOaXCDeA0iR5Ge1Xlew9IRoxF08= 2024-04-16T18:27:47.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eOaXCDeA0iR5Ge1Xlew9IRoxF08=', 'Content-Length': '0'} 2024-04-16T18:27:47.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d21523208b1930df-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.399 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.400 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.400 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.400 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.400 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.400 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.400 DEBUG:boto:Data: 2024-04-16T18:27:47.400 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.400 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.400 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.400 DEBUG:boto:Params: {} 2024-04-16T18:27:47.400 DEBUG:boto:Token: None 2024-04-16T18:27:47.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.401 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eOaXCDeA0iR5Ge1Xlew9IRoxF08= 2024-04-16T18:27:47.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eOaXCDeA0iR5Ge1Xlew9IRoxF08=', 'Content-Length': '0'} 2024-04-16T18:27:47.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2645003e5f3f154-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:27:47.404 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.404 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.404 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.404 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.404 DEBUG:boto:Method: GET 2024-04-16T18:27:47.404 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.405 DEBUG:boto:Data: 2024-04-16T18:27:47.405 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.405 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.405 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.405 DEBUG:boto:Params: {} 2024-04-16T18:27:47.405 DEBUG:boto:Token: None 2024-04-16T18:27:47.405 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI= 2024-04-16T18:27:47.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI=', 'Content-Length': '0'} 2024-04-16T18:27:47.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b848aa22743537a-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.450 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.450 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.451 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.451 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.451 DEBUG:boto:Method: GET 2024-04-16T18:27:47.451 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.451 DEBUG:boto:Data: 2024-04-16T18:27:47.451 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.451 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.451 DEBUG:boto:Params: {} 2024-04-16T18:27:47.451 DEBUG:boto:Token: None 2024-04-16T18:27:47.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:27:47.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI= 2024-04-16T18:27:47.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:buXJ4ogIZPdY+EDYZG0Hdo4KEaI=', 'Content-Length': '0'} 2024-04-16T18:27:47.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9aabc97a9df69a3-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.455 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.455 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.455 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.455 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.455 DEBUG:boto:Method: GET 2024-04-16T18:27:47.456 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:27:47.456 DEBUG:boto:Data: 2024-04-16T18:27:47.456 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.456 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.456 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.456 DEBUG:boto:Params: {} 2024-04-16T18:27:47.456 DEBUG:boto:Token: None 2024-04-16T18:27:47.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.460 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8f3b5320c2eb789-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.460 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.461 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.461 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.461 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.461 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.461 DEBUG:boto:Method: GET 2024-04-16T18:27:47.461 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:27:47.461 DEBUG:boto:Data: 2024-04-16T18:27:47.461 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.461 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.461 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.462 DEBUG:boto:Params: {} 2024-04-16T18:27:47.462 DEBUG:boto:Token: None 2024-04-16T18:27:47.462 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.462 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab29f0e2706645bb-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.510 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.511 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.511 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.511 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.511 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:27:47.511 DEBUG:boto:Data: 2024-04-16T18:27:47.511 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.511 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.511 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.511 DEBUG:boto:Params: {} 2024-04-16T18:27:47.512 DEBUG:boto:Token: None 2024-04-16T18:27:47.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29 2024-04-16T18:27:47.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WPJ6Gm5tFeO76R839jBrFomt8xo= 2024-04-16T18:27:47.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WPJ6Gm5tFeO76R839jBrFomt8xo=', 'Content-Length': '0'} 2024-04-16T18:27:47.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a13dc74038d40b14-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.515 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.515 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.515 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.516 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:27:47.516 DEBUG:boto:Data: 2024-04-16T18:27:47.516 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.516 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.516 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.516 DEBUG:boto:Params: {} 2024-04-16T18:27:47.516 DEBUG:boto:Token: None 2024-04-16T18:27:47.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29 2024-04-16T18:27:47.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WPJ6Gm5tFeO76R839jBrFomt8xo= 2024-04-16T18:27:47.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WPJ6Gm5tFeO76R839jBrFomt8xo=', 'Content-Length': '0'} 2024-04-16T18:27:47.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f940d47b397c7001-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:27:47.520 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.520 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.520 DEBUG:boto:Method: GET 2024-04-16T18:27:47.520 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:27:47.520 DEBUG:boto:Data: 2024-04-16T18:27:47.520 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.520 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.520 DEBUG:boto:Params: {} 2024-04-16T18:27:47.520 DEBUG:boto:Token: None 2024-04-16T18:27:47.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29 2024-04-16T18:27:47.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CDcNHibUSaGyrhaNhi6AQeuBidI= 2024-04-16T18:27:47.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CDcNHibUSaGyrhaNhi6AQeuBidI=', 'Content-Length': '0'} 2024-04-16T18:27:47.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000349e4e34b7c87397-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.566 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:27:47.566 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:27:47.567 DEBUG:boto:Method: GET 2024-04-16T18:27:47.567 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:27:47.567 DEBUG:boto:Data: 2024-04-16T18:27:47.567 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.567 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.567 DEBUG:boto:Params: {} 2024-04-16T18:27:47.567 DEBUG:boto:Token: None 2024-04-16T18:27:47.567 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj29 2024-04-16T18:27:47.567 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CDcNHibUSaGyrhaNhi6AQeuBidI= 2024-04-16T18:27:47.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CDcNHibUSaGyrhaNhi6AQeuBidI=', 'Content-Length': '0'} 2024-04-16T18:27:47.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e083b95ccdf1a2aa-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.571 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.571 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.571 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.571 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.571 DEBUG:boto:Method: GET 2024-04-16T18:27:47.571 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:27:47.571 DEBUG:boto:Data: 2024-04-16T18:27:47.571 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.571 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.571 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.571 DEBUG:boto:Params: {} 2024-04-16T18:27:47.572 DEBUG:boto:Token: None 2024-04-16T18:27:47.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1155f4f9ff5fe4f-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.618 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.619 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.619 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.619 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.619 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:27:47.619 DEBUG:boto:Method: GET 2024-04-16T18:27:47.619 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:27:47.619 DEBUG:boto:Data: 2024-04-16T18:27:47.619 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.619 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.619 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.619 DEBUG:boto:Params: {} 2024-04-16T18:27:47.620 DEBUG:boto:Token: None 2024-04-16T18:27:47.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.625 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de20d4b8025b1985-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.626 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.626 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:27:47.626 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.626 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.626 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.626 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.626 DEBUG:boto:Method: GET 2024-04-16T18:27:47.627 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.627 DEBUG:boto:Data: 2024-04-16T18:27:47.627 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.627 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.627 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.627 DEBUG:boto:Params: {} 2024-04-16T18:27:47.627 DEBUG:boto:Token: None 2024-04-16T18:27:47.627 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.627 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs= 2024-04-16T18:27:47.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs=', 'Content-Length': '0'} 2024-04-16T18:27:47.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2cbb7466dafd90f-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.674 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.674 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.675 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.675 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.675 DEBUG:boto:Method: GET 2024-04-16T18:27:47.675 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.675 DEBUG:boto:Data: 2024-04-16T18:27:47.675 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.675 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.675 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.675 DEBUG:boto:Params: {} 2024-04-16T18:27:47.675 DEBUG:boto:Token: None 2024-04-16T18:27:47.675 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs= 2024-04-16T18:27:47.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs=', 'Content-Length': '0'} 2024-04-16T18:27:47.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca183f0753462cce-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.679 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.679 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.679 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.679 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.679 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.679 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.679 DEBUG:boto:Data: 2024-04-16T18:27:47.679 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.679 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.680 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.680 DEBUG:boto:Params: {} 2024-04-16T18:27:47.680 DEBUG:boto:Token: None 2024-04-16T18:27:47.680 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.680 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UB4QiYjVzbSYR4cGtHYnHbeW2ko= 2024-04-16T18:27:47.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UB4QiYjVzbSYR4cGtHYnHbeW2ko=', 'Content-Length': '0'} 2024-04-16T18:27:47.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000589ac02acd12eee9-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.683 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.683 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.683 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.683 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.684 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.684 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.684 DEBUG:boto:Data: 2024-04-16T18:27:47.684 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.684 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.684 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.684 DEBUG:boto:Params: {} 2024-04-16T18:27:47.684 DEBUG:boto:Token: None 2024-04-16T18:27:47.684 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UB4QiYjVzbSYR4cGtHYnHbeW2ko= 2024-04-16T18:27:47.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UB4QiYjVzbSYR4cGtHYnHbeW2ko=', 'Content-Length': '0'} 2024-04-16T18:27:47.687 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038aad25c4b06e1a9-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.688 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:27:47.688 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.688 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.688 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.688 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.688 DEBUG:boto:Method: GET 2024-04-16T18:27:47.688 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.688 DEBUG:boto:Data: 2024-04-16T18:27:47.688 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.688 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.689 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.689 DEBUG:boto:Params: {} 2024-04-16T18:27:47.689 DEBUG:boto:Token: None 2024-04-16T18:27:47.689 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.689 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs= 2024-04-16T18:27:47.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs=', 'Content-Length': '0'} 2024-04-16T18:27:47.692 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fefef7ac2e7ef6f-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.734 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.734 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.735 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.735 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.735 DEBUG:boto:Method: GET 2024-04-16T18:27:47.735 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.735 DEBUG:boto:Data: 2024-04-16T18:27:47.735 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.735 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.735 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.735 DEBUG:boto:Params: {} 2024-04-16T18:27:47.735 DEBUG:boto:Token: None 2024-04-16T18:27:47.735 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:27:47.736 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs= 2024-04-16T18:27:47.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tcfXrO/htWPuoHFPUsE3JDW8lUs=', 'Content-Length': '0'} 2024-04-16T18:27:47.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004172667df5852280-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.739 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.739 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.739 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.739 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.739 DEBUG:boto:Method: GET 2024-04-16T18:27:47.740 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:27:47.740 DEBUG:boto:Data: 2024-04-16T18:27:47.740 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.740 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.740 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.740 DEBUG:boto:Params: {} 2024-04-16T18:27:47.740 DEBUG:boto:Token: None 2024-04-16T18:27:47.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a2430b87365d17a-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.745 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.745 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.745 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.745 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.745 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.745 DEBUG:boto:Method: GET 2024-04-16T18:27:47.746 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:27:47.746 DEBUG:boto:Data: 2024-04-16T18:27:47.746 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.746 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.746 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.746 DEBUG:boto:Params: {} 2024-04-16T18:27:47.746 DEBUG:boto:Token: None 2024-04-16T18:27:47.746 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.746 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070b2fd072b3974dd-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.794 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.795 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.795 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.795 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.795 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:27:47.795 DEBUG:boto:Data: 2024-04-16T18:27:47.795 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.795 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.795 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.795 DEBUG:boto:Params: {} 2024-04-16T18:27:47.796 DEBUG:boto:Token: None 2024-04-16T18:27:47.796 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31 2024-04-16T18:27:47.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PZFRUmFwNidxKxYZbYU3t3Q1t70= 2024-04-16T18:27:47.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PZFRUmFwNidxKxYZbYU3t3Q1t70=', 'Content-Length': '0'} 2024-04-16T18:27:47.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003efc20ae399b81d8-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.799 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.799 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.799 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.799 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:27:47.799 DEBUG:boto:Data: 2024-04-16T18:27:47.799 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.800 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.800 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.800 DEBUG:boto:Params: {} 2024-04-16T18:27:47.800 DEBUG:boto:Token: None 2024-04-16T18:27:47.800 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31 2024-04-16T18:27:47.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PZFRUmFwNidxKxYZbYU3t3Q1t70= 2024-04-16T18:27:47.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PZFRUmFwNidxKxYZbYU3t3Q1t70=', 'Content-Length': '0'} 2024-04-16T18:27:47.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000935f7250f118e11f-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.803 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:27:47.804 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.804 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.804 DEBUG:boto:Method: GET 2024-04-16T18:27:47.804 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:27:47.804 DEBUG:boto:Data: 2024-04-16T18:27:47.804 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.804 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.804 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.804 DEBUG:boto:Params: {} 2024-04-16T18:27:47.804 DEBUG:boto:Token: None 2024-04-16T18:27:47.805 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31 2024-04-16T18:27:47.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/yf+l0EKVW5euCFKKqdHxbUhkKQ= 2024-04-16T18:27:47.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/yf+l0EKVW5euCFKKqdHxbUhkKQ=', 'Content-Length': '0'} 2024-04-16T18:27:47.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3a1dd1e40f4525b-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.850 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:27:47.850 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:27:47.851 DEBUG:boto:Method: GET 2024-04-16T18:27:47.851 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:27:47.851 DEBUG:boto:Data: 2024-04-16T18:27:47.851 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.851 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.851 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.851 DEBUG:boto:Params: {} 2024-04-16T18:27:47.851 DEBUG:boto:Token: None 2024-04-16T18:27:47.851 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj31 2024-04-16T18:27:47.851 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/yf+l0EKVW5euCFKKqdHxbUhkKQ= 2024-04-16T18:27:47.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/yf+l0EKVW5euCFKKqdHxbUhkKQ=', 'Content-Length': '0'} 2024-04-16T18:27:47.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcb883aef8bb6c5a-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.855 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.855 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.855 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.855 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.855 DEBUG:boto:Method: GET 2024-04-16T18:27:47.855 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:27:47.855 DEBUG:boto:Data: 2024-04-16T18:27:47.856 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.856 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.856 DEBUG:boto:Params: {} 2024-04-16T18:27:47.856 DEBUG:boto:Token: None 2024-04-16T18:27:47.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055022b2b2eb91ade-00661ec323-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.902 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.903 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:47.903 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:47.903 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.903 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:27:47.903 DEBUG:boto:Method: GET 2024-04-16T18:27:47.903 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:27:47.903 DEBUG:boto:Data: 2024-04-16T18:27:47.903 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.903 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.903 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.904 DEBUG:boto:Params: {} 2024-04-16T18:27:47.904 DEBUG:boto:Token: None 2024-04-16T18:27:47.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/ 2024-04-16T18:27:47.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc= 2024-04-16T18:27:47.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PeOt9htnuDO1nL+DMlmyRu5boc=', 'Content-Length': '0'} 2024-04-16T18:27:47.910 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b25e972c76b63c83-00661ec323-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.910 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:47.910 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:27:47.910 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:47.911 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:47.911 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.911 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.911 DEBUG:boto:Method: GET 2024-04-16T18:27:47.911 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.911 DEBUG:boto:Data: 2024-04-16T18:27:47.911 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.911 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.911 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.911 DEBUG:boto:Params: {} 2024-04-16T18:27:47.911 DEBUG:boto:Token: None 2024-04-16T18:27:47.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QHuFTJ1J2q6hYvA1jonxH5TH/Qg= 2024-04-16T18:27:47.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QHuFTJ1J2q6hYvA1jonxH5TH/Qg=', 'Content-Length': '0'} 2024-04-16T18:27:47.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001af32525364b4aa2-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.958 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:47.958 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:47.958 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.959 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.959 DEBUG:boto:Method: GET 2024-04-16T18:27:47.959 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.959 DEBUG:boto:Data: 2024-04-16T18:27:47.959 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.959 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.959 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.959 DEBUG:boto:Params: {} 2024-04-16T18:27:47.959 DEBUG:boto:Token: None 2024-04-16T18:27:47.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QHuFTJ1J2q6hYvA1jonxH5TH/Qg= 2024-04-16T18:27:47.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QHuFTJ1J2q6hYvA1jonxH5TH/Qg=', 'Content-Length': '0'} 2024-04-16T18:27:47.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a2e46f09faee5b4-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.963 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:47.963 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:47.963 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.963 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.963 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.963 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.963 DEBUG:boto:Data: 2024-04-16T18:27:47.963 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.963 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.963 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.964 DEBUG:boto:Params: {} 2024-04-16T18:27:47.964 DEBUG:boto:Token: None 2024-04-16T18:27:47.964 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5MhOFBCWsh13fZjBqNvvbo/fQqA= 2024-04-16T18:27:47.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5MhOFBCWsh13fZjBqNvvbo/fQqA=', 'Content-Length': '0'} 2024-04-16T18:27:47.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f774815625e07677-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.967 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:47.967 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:47.967 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.967 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.968 DEBUG:boto:Method: HEAD 2024-04-16T18:27:47.968 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.968 DEBUG:boto:Data: 2024-04-16T18:27:47.968 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.968 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:47.968 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.968 DEBUG:boto:Params: {} 2024-04-16T18:27:47.968 DEBUG:boto:Token: None 2024-04-16T18:27:47.968 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5MhOFBCWsh13fZjBqNvvbo/fQqA= 2024-04-16T18:27:47.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5MhOFBCWsh13fZjBqNvvbo/fQqA=', 'Content-Length': '0'} 2024-04-16T18:27:47.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1d05f52e7067f1b-00661ec323-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:47.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:27:47.971 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:47.971 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:47.971 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.972 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.972 DEBUG:boto:Method: GET 2024-04-16T18:27:47.972 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.972 DEBUG:boto:Data: 2024-04-16T18:27:47.972 DEBUG:boto:Headers: {} 2024-04-16T18:27:47.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:47.972 DEBUG:boto:Port: 8000 2024-04-16T18:27:47.972 DEBUG:boto:Params: {} 2024-04-16T18:27:47.972 DEBUG:boto:Token: None 2024-04-16T18:27:47.972 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:27:47.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QHuFTJ1J2q6hYvA1jonxH5TH/Qg= 2024-04-16T18:27:47.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QHuFTJ1J2q6hYvA1jonxH5TH/Qg=', 'Content-Length': '0'} 2024-04-16T18:27:47.975 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077259e27b0cff205-00661ec323-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.018 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:48.018 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:48.019 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:48.019 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:27:48.019 DEBUG:boto:Method: GET 2024-04-16T18:27:48.019 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:27:48.019 DEBUG:boto:Data: 2024-04-16T18:27:48.019 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.019 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.019 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.019 DEBUG:boto:Params: {} 2024-04-16T18:27:48.019 DEBUG:boto:Token: None 2024-04-16T18:27:48.019 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:27:48.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pRmj6UUwra1t6G28CODym3DcwQ= 2024-04-16T18:27:48.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pRmj6UUwra1t6G28CODym3DcwQ=', 'Content-Length': '0'} 2024-04-16T18:27:48.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e66a88e51edc7fad-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.023 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.023 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.023 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.023 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.023 DEBUG:boto:Method: GET 2024-04-16T18:27:48.023 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:27:48.023 DEBUG:boto:Data: 2024-04-16T18:27:48.023 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.023 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.023 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.023 DEBUG:boto:Params: {} 2024-04-16T18:27:48.024 DEBUG:boto:Token: None 2024-04-16T18:27:48.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.027 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c589926e7fb63a8d-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.027 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.028 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.028 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.028 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.028 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.028 DEBUG:boto:Method: GET 2024-04-16T18:27:48.028 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:27:48.028 DEBUG:boto:Data: 2024-04-16T18:27:48.028 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.028 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.028 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.029 DEBUG:boto:Params: {} 2024-04-16T18:27:48.029 DEBUG:boto:Token: None 2024-04-16T18:27:48.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.035 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bebfd6c34e7e048-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.078 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.079 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:48.079 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:48.079 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.079 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:27:48.079 DEBUG:boto:Data: 2024-04-16T18:27:48.079 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.079 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.079 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.079 DEBUG:boto:Params: {} 2024-04-16T18:27:48.079 DEBUG:boto:Token: None 2024-04-16T18:27:48.080 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj33 2024-04-16T18:27:48.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WrlNV+Csi4SaEtDpQ3aRE+0rux8= 2024-04-16T18:27:48.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WrlNV+Csi4SaEtDpQ3aRE+0rux8=', 'Content-Length': '0'} 2024-04-16T18:27:48.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a380b711a0e59475-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.083 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:48.083 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:48.083 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.083 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:27:48.083 DEBUG:boto:Data: 2024-04-16T18:27:48.083 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.083 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.083 DEBUG:boto:Params: {} 2024-04-16T18:27:48.084 DEBUG:boto:Token: None 2024-04-16T18:27:48.084 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj33 2024-04-16T18:27:48.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WrlNV+Csi4SaEtDpQ3aRE+0rux8= 2024-04-16T18:27:48.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WrlNV+Csi4SaEtDpQ3aRE+0rux8=', 'Content-Length': '0'} 2024-04-16T18:27:48.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000741e7209a5299298-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.087 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:27:48.087 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:48.087 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:48.087 DEBUG:boto:Method: GET 2024-04-16T18:27:48.087 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:27:48.088 DEBUG:boto:Data: 2024-04-16T18:27:48.088 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.088 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.088 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.088 DEBUG:boto:Params: {} 2024-04-16T18:27:48.088 DEBUG:boto:Token: None 2024-04-16T18:27:48.088 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj33 2024-04-16T18:27:48.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:st3o0nXnUY2hfU9y0wkAGt+qqUI= 2024-04-16T18:27:48.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:st3o0nXnUY2hfU9y0wkAGt+qqUI=', 'Content-Length': '0'} 2024-04-16T18:27:48.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bcbf7c928c27d61-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.134 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:27:48.135 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:27:48.135 DEBUG:boto:Method: GET 2024-04-16T18:27:48.135 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:27:48.135 DEBUG:boto:Data: 2024-04-16T18:27:48.135 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.135 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.135 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.135 DEBUG:boto:Params: {} 2024-04-16T18:27:48.135 DEBUG:boto:Token: None 2024-04-16T18:27:48.135 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj33 2024-04-16T18:27:48.135 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:st3o0nXnUY2hfU9y0wkAGt+qqUI= 2024-04-16T18:27:48.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:st3o0nXnUY2hfU9y0wkAGt+qqUI=', 'Content-Length': '0'} 2024-04-16T18:27:48.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbb9e644a49550d1-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.139 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.139 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.139 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.139 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.139 DEBUG:boto:Method: GET 2024-04-16T18:27:48.139 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:27:48.139 DEBUG:boto:Data: 2024-04-16T18:27:48.140 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.140 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.140 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.140 DEBUG:boto:Params: {} 2024-04-16T18:27:48.140 DEBUG:boto:Token: None 2024-04-16T18:27:48.140 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.140 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d2880d607a51c31-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.186 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.187 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.187 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.187 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.187 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:27:48.187 DEBUG:boto:Method: GET 2024-04-16T18:27:48.187 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:27:48.187 DEBUG:boto:Data: 2024-04-16T18:27:48.187 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.187 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.187 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.188 DEBUG:boto:Params: {} 2024-04-16T18:27:48.188 DEBUG:boto:Token: None 2024-04-16T18:27:48.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000032d1cf100dd4b0-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.194 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.194 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:27:48.194 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.194 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.194 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.194 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.195 DEBUG:boto:Method: GET 2024-04-16T18:27:48.195 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.195 DEBUG:boto:Data: 2024-04-16T18:27:48.195 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.195 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.195 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.195 DEBUG:boto:Params: {} 2024-04-16T18:27:48.195 DEBUG:boto:Token: None 2024-04-16T18:27:48.195 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.195 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ= 2024-04-16T18:27:48.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ=', 'Content-Length': '0'} 2024-04-16T18:27:48.198 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009635b161654c904d-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.242 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.242 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.242 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.243 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.243 DEBUG:boto:Method: GET 2024-04-16T18:27:48.243 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.243 DEBUG:boto:Data: 2024-04-16T18:27:48.243 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.243 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.243 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.243 DEBUG:boto:Params: {} 2024-04-16T18:27:48.243 DEBUG:boto:Token: None 2024-04-16T18:27:48.243 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ= 2024-04-16T18:27:48.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ=', 'Content-Length': '0'} 2024-04-16T18:27:48.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b785c2ee7f80698-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.247 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.247 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.247 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.247 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.247 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.247 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.248 DEBUG:boto:Data: 2024-04-16T18:27:48.248 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.248 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.248 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.248 DEBUG:boto:Params: {} 2024-04-16T18:27:48.248 DEBUG:boto:Token: None 2024-04-16T18:27:48.248 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lgq0z7VxOf4eo0/XX7oLISU2IOs= 2024-04-16T18:27:48.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgq0z7VxOf4eo0/XX7oLISU2IOs=', 'Content-Length': '0'} 2024-04-16T18:27:48.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069dc4dbad5769c3c-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.251 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.251 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.251 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.251 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.252 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.252 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.252 DEBUG:boto:Data: 2024-04-16T18:27:48.252 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.252 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.252 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.252 DEBUG:boto:Params: {} 2024-04-16T18:27:48.252 DEBUG:boto:Token: None 2024-04-16T18:27:48.252 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lgq0z7VxOf4eo0/XX7oLISU2IOs= 2024-04-16T18:27:48.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgq0z7VxOf4eo0/XX7oLISU2IOs=', 'Content-Length': '0'} 2024-04-16T18:27:48.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa9deac7c0ce0904-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:27:48.256 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.256 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.256 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.256 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.256 DEBUG:boto:Method: GET 2024-04-16T18:27:48.256 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.256 DEBUG:boto:Data: 2024-04-16T18:27:48.256 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.256 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.256 DEBUG:boto:Params: {} 2024-04-16T18:27:48.257 DEBUG:boto:Token: None 2024-04-16T18:27:48.257 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ= 2024-04-16T18:27:48.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ=', 'Content-Length': '0'} 2024-04-16T18:27:48.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011f23f91268de6f8-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.302 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.302 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.303 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.303 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.303 DEBUG:boto:Method: GET 2024-04-16T18:27:48.303 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.303 DEBUG:boto:Data: 2024-04-16T18:27:48.303 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.303 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.303 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.303 DEBUG:boto:Params: {} 2024-04-16T18:27:48.303 DEBUG:boto:Token: None 2024-04-16T18:27:48.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:27:48.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ= 2024-04-16T18:27:48.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ctpzR/PSvkKR9kgIBja/j8A/XgQ=', 'Content-Length': '0'} 2024-04-16T18:27:48.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be4cc1f24971f7fc-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.307 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.307 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.307 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.307 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.307 DEBUG:boto:Method: GET 2024-04-16T18:27:48.308 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:27:48.308 DEBUG:boto:Data: 2024-04-16T18:27:48.308 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.308 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.308 DEBUG:boto:Params: {} 2024-04-16T18:27:48.308 DEBUG:boto:Token: None 2024-04-16T18:27:48.308 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9a4b2a471089ca9-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.312 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.313 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.313 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.313 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.313 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.313 DEBUG:boto:Method: GET 2024-04-16T18:27:48.313 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:27:48.313 DEBUG:boto:Data: 2024-04-16T18:27:48.313 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.313 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.313 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.313 DEBUG:boto:Params: {} 2024-04-16T18:27:48.314 DEBUG:boto:Token: None 2024-04-16T18:27:48.314 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.314 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.319 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c48a420f8fd0542-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.362 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.363 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.363 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.363 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.363 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:27:48.363 DEBUG:boto:Data: 2024-04-16T18:27:48.363 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.363 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.363 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.363 DEBUG:boto:Params: {} 2024-04-16T18:27:48.363 DEBUG:boto:Token: None 2024-04-16T18:27:48.363 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34 2024-04-16T18:27:48.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HilzEXNRh39eDKojBonvJbPhIoM= 2024-04-16T18:27:48.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HilzEXNRh39eDKojBonvJbPhIoM=', 'Content-Length': '0'} 2024-04-16T18:27:48.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ba4e9ca61509f3a-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.367 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.367 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.367 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.367 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:27:48.367 DEBUG:boto:Data: 2024-04-16T18:27:48.367 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.367 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.367 DEBUG:boto:Params: {} 2024-04-16T18:27:48.367 DEBUG:boto:Token: None 2024-04-16T18:27:48.367 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34 2024-04-16T18:27:48.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HilzEXNRh39eDKojBonvJbPhIoM= 2024-04-16T18:27:48.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HilzEXNRh39eDKojBonvJbPhIoM=', 'Content-Length': '0'} 2024-04-16T18:27:48.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000612508e108415fed-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.371 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:27:48.371 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.371 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.371 DEBUG:boto:Method: GET 2024-04-16T18:27:48.371 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:27:48.371 DEBUG:boto:Data: 2024-04-16T18:27:48.372 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.372 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.372 DEBUG:boto:Params: {} 2024-04-16T18:27:48.372 DEBUG:boto:Token: None 2024-04-16T18:27:48.372 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34 2024-04-16T18:27:48.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gEvowU6EaJjzWWnMukZfYVfnmwU= 2024-04-16T18:27:48.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gEvowU6EaJjzWWnMukZfYVfnmwU=', 'Content-Length': '0'} 2024-04-16T18:27:48.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009399f8a6f4c1b837-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.418 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:27:48.419 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:27:48.419 DEBUG:boto:Method: GET 2024-04-16T18:27:48.419 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:27:48.419 DEBUG:boto:Data: 2024-04-16T18:27:48.419 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.419 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.419 DEBUG:boto:Params: {} 2024-04-16T18:27:48.419 DEBUG:boto:Token: None 2024-04-16T18:27:48.419 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj34 2024-04-16T18:27:48.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gEvowU6EaJjzWWnMukZfYVfnmwU= 2024-04-16T18:27:48.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gEvowU6EaJjzWWnMukZfYVfnmwU=', 'Content-Length': '0'} 2024-04-16T18:27:48.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e5d9d560e92cd3e-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.423 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.423 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.423 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.423 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.423 DEBUG:boto:Method: GET 2024-04-16T18:27:48.423 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:27:48.423 DEBUG:boto:Data: 2024-04-16T18:27:48.423 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.423 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.424 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.424 DEBUG:boto:Params: {} 2024-04-16T18:27:48.424 DEBUG:boto:Token: None 2024-04-16T18:27:48.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.427 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cd952700c0390cd-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.470 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.471 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.471 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.471 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.471 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:27:48.471 DEBUG:boto:Method: GET 2024-04-16T18:27:48.471 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:27:48.471 DEBUG:boto:Data: 2024-04-16T18:27:48.471 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.471 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.471 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.472 DEBUG:boto:Params: {} 2024-04-16T18:27:48.472 DEBUG:boto:Token: None 2024-04-16T18:27:48.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.478 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eaedec7ca8cdd499-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.479 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:27:48.479 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.479 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.479 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.479 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.479 DEBUG:boto:Method: GET 2024-04-16T18:27:48.479 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.480 DEBUG:boto:Data: 2024-04-16T18:27:48.480 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.480 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.480 DEBUG:boto:Params: {} 2024-04-16T18:27:48.480 DEBUG:boto:Token: None 2024-04-16T18:27:48.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8= 2024-04-16T18:27:48.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8=', 'Content-Length': '0'} 2024-04-16T18:27:48.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a4b2d7ef2036feb-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.526 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.526 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.527 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.527 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.527 DEBUG:boto:Method: GET 2024-04-16T18:27:48.527 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.527 DEBUG:boto:Data: 2024-04-16T18:27:48.527 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.527 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.527 DEBUG:boto:Params: {} 2024-04-16T18:27:48.527 DEBUG:boto:Token: None 2024-04-16T18:27:48.527 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8= 2024-04-16T18:27:48.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8=', 'Content-Length': '0'} 2024-04-16T18:27:48.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048d478e7c74abab7-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.531 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.531 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.531 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.531 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.531 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.531 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.531 DEBUG:boto:Data: 2024-04-16T18:27:48.532 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.532 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.532 DEBUG:boto:Params: {} 2024-04-16T18:27:48.532 DEBUG:boto:Token: None 2024-04-16T18:27:48.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+yizRfdzyMlgczgaDc7yNkdDuso= 2024-04-16T18:27:48.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+yizRfdzyMlgczgaDc7yNkdDuso=', 'Content-Length': '0'} 2024-04-16T18:27:48.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007df790409862bf3e-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.535 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.535 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.535 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.535 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.535 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.535 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.536 DEBUG:boto:Data: 2024-04-16T18:27:48.536 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.536 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.536 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.536 DEBUG:boto:Params: {} 2024-04-16T18:27:48.536 DEBUG:boto:Token: None 2024-04-16T18:27:48.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+yizRfdzyMlgczgaDc7yNkdDuso= 2024-04-16T18:27:48.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+yizRfdzyMlgczgaDc7yNkdDuso=', 'Content-Length': '0'} 2024-04-16T18:27:48.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aaa0a58d0729bdf7-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.539 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:27:48.539 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.539 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.540 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.540 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.540 DEBUG:boto:Method: GET 2024-04-16T18:27:48.540 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.540 DEBUG:boto:Data: 2024-04-16T18:27:48.540 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.540 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.540 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.540 DEBUG:boto:Params: {} 2024-04-16T18:27:48.540 DEBUG:boto:Token: None 2024-04-16T18:27:48.541 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8= 2024-04-16T18:27:48.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8=', 'Content-Length': '0'} 2024-04-16T18:27:48.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd3982fa60a97f51-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.586 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.586 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.586 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.587 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.587 DEBUG:boto:Method: GET 2024-04-16T18:27:48.587 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.587 DEBUG:boto:Data: 2024-04-16T18:27:48.587 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.587 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.587 DEBUG:boto:Params: {} 2024-04-16T18:27:48.587 DEBUG:boto:Token: None 2024-04-16T18:27:48.587 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:27:48.587 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8= 2024-04-16T18:27:48.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZfR7g6OW143jMmpF4S2b2OMiYw8=', 'Content-Length': '0'} 2024-04-16T18:27:48.590 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c7bda411c7f077d-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.590 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.591 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.591 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.591 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.591 DEBUG:boto:Method: GET 2024-04-16T18:27:48.591 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:27:48.591 DEBUG:boto:Data: 2024-04-16T18:27:48.591 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.591 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.591 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.591 DEBUG:boto:Params: {} 2024-04-16T18:27:48.592 DEBUG:boto:Token: None 2024-04-16T18:27:48.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.595 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a42acc285ada3124-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.595 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.596 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.596 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.596 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.596 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.596 DEBUG:boto:Method: GET 2024-04-16T18:27:48.596 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:27:48.596 DEBUG:boto:Data: 2024-04-16T18:27:48.596 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.596 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.596 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.597 DEBUG:boto:Params: {} 2024-04-16T18:27:48.597 DEBUG:boto:Token: None 2024-04-16T18:27:48.597 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.597 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.603 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0e4b557aaeaf46d-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.646 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.647 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.647 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.647 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.647 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:27:48.647 DEBUG:boto:Data: 2024-04-16T18:27:48.647 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.647 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.647 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.647 DEBUG:boto:Params: {} 2024-04-16T18:27:48.647 DEBUG:boto:Token: None 2024-04-16T18:27:48.648 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38 2024-04-16T18:27:48.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fJjunxUwnuUUMjWikQAqPAM/Mlc= 2024-04-16T18:27:48.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fJjunxUwnuUUMjWikQAqPAM/Mlc=', 'Content-Length': '0'} 2024-04-16T18:27:48.650 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079d69bee09d5a203-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.651 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.651 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.651 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.651 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:27:48.651 DEBUG:boto:Data: 2024-04-16T18:27:48.651 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.651 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.651 DEBUG:boto:Params: {} 2024-04-16T18:27:48.651 DEBUG:boto:Token: None 2024-04-16T18:27:48.652 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38 2024-04-16T18:27:48.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fJjunxUwnuUUMjWikQAqPAM/Mlc= 2024-04-16T18:27:48.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fJjunxUwnuUUMjWikQAqPAM/Mlc=', 'Content-Length': '0'} 2024-04-16T18:27:48.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dceb2e6c34bf9f0-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.654 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:27:48.655 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.655 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.655 DEBUG:boto:Method: GET 2024-04-16T18:27:48.655 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:27:48.655 DEBUG:boto:Data: 2024-04-16T18:27:48.655 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.655 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.655 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.655 DEBUG:boto:Params: {} 2024-04-16T18:27:48.655 DEBUG:boto:Token: None 2024-04-16T18:27:48.655 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38 2024-04-16T18:27:48.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qTrluuNfxp2VKGnPJnB6uxv+CPo= 2024-04-16T18:27:48.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qTrluuNfxp2VKGnPJnB6uxv+CPo=', 'Content-Length': '0'} 2024-04-16T18:27:48.658 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ef764462bd381b4-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.702 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:27:48.702 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:27:48.703 DEBUG:boto:Method: GET 2024-04-16T18:27:48.703 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:27:48.703 DEBUG:boto:Data: 2024-04-16T18:27:48.703 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.703 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.703 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.703 DEBUG:boto:Params: {} 2024-04-16T18:27:48.703 DEBUG:boto:Token: None 2024-04-16T18:27:48.703 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj38 2024-04-16T18:27:48.703 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qTrluuNfxp2VKGnPJnB6uxv+CPo= 2024-04-16T18:27:48.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qTrluuNfxp2VKGnPJnB6uxv+CPo=', 'Content-Length': '0'} 2024-04-16T18:27:48.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048f55649a213df2d-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.707 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.707 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.707 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.707 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.707 DEBUG:boto:Method: GET 2024-04-16T18:27:48.708 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:27:48.708 DEBUG:boto:Data: 2024-04-16T18:27:48.708 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.708 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.708 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.708 DEBUG:boto:Params: {} 2024-04-16T18:27:48.708 DEBUG:boto:Token: None 2024-04-16T18:27:48.708 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.711 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bba94e6562116785-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.754 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.755 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.755 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.755 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.755 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:27:48.755 DEBUG:boto:Method: GET 2024-04-16T18:27:48.755 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:27:48.755 DEBUG:boto:Data: 2024-04-16T18:27:48.755 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.755 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.755 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.755 DEBUG:boto:Params: {} 2024-04-16T18:27:48.756 DEBUG:boto:Token: None 2024-04-16T18:27:48.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab67f424bdfdfd0e-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.763 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.763 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:27:48.763 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.763 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.763 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.763 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.763 DEBUG:boto:Method: GET 2024-04-16T18:27:48.763 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.764 DEBUG:boto:Data: 2024-04-16T18:27:48.764 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.764 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.764 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.764 DEBUG:boto:Params: {} 2024-04-16T18:27:48.764 DEBUG:boto:Token: None 2024-04-16T18:27:48.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8= 2024-04-16T18:27:48.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8=', 'Content-Length': '0'} 2024-04-16T18:27:48.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0632dea7d2dab86-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.811 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.811 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.811 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.811 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.811 DEBUG:boto:Method: GET 2024-04-16T18:27:48.811 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.811 DEBUG:boto:Data: 2024-04-16T18:27:48.811 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.811 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.811 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.812 DEBUG:boto:Params: {} 2024-04-16T18:27:48.812 DEBUG:boto:Token: None 2024-04-16T18:27:48.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8= 2024-04-16T18:27:48.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8=', 'Content-Length': '0'} 2024-04-16T18:27:48.816 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008260136fd9b9fe00-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.816 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.816 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.816 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.816 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.816 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.816 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.816 DEBUG:boto:Data: 2024-04-16T18:27:48.817 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.817 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.817 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.817 DEBUG:boto:Params: {} 2024-04-16T18:27:48.817 DEBUG:boto:Token: None 2024-04-16T18:27:48.817 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.817 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Quv4j/xZJ1ZopwY/U4PBwq+H1Wk= 2024-04-16T18:27:48.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Quv4j/xZJ1ZopwY/U4PBwq+H1Wk=', 'Content-Length': '0'} 2024-04-16T18:27:48.820 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b3abd46ae02222-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.820 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.820 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.820 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.821 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.821 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.821 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.821 DEBUG:boto:Data: 2024-04-16T18:27:48.821 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.821 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.821 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.821 DEBUG:boto:Params: {} 2024-04-16T18:27:48.821 DEBUG:boto:Token: None 2024-04-16T18:27:48.821 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Quv4j/xZJ1ZopwY/U4PBwq+H1Wk= 2024-04-16T18:27:48.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Quv4j/xZJ1ZopwY/U4PBwq+H1Wk=', 'Content-Length': '0'} 2024-04-16T18:27:48.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001eb10eccf9c18f63-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:27:48.825 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.825 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.825 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.825 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.825 DEBUG:boto:Method: GET 2024-04-16T18:27:48.826 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.826 DEBUG:boto:Data: 2024-04-16T18:27:48.826 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.826 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.826 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.826 DEBUG:boto:Params: {} 2024-04-16T18:27:48.826 DEBUG:boto:Token: None 2024-04-16T18:27:48.826 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.826 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8= 2024-04-16T18:27:48.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8=', 'Content-Length': '0'} 2024-04-16T18:27:48.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d18673f25e418d2-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.870 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.870 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.871 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.871 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.871 DEBUG:boto:Method: GET 2024-04-16T18:27:48.871 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.871 DEBUG:boto:Data: 2024-04-16T18:27:48.871 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.871 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.871 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.871 DEBUG:boto:Params: {} 2024-04-16T18:27:48.871 DEBUG:boto:Token: None 2024-04-16T18:27:48.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:27:48.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8= 2024-04-16T18:27:48.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FOZaNuOa/I99C6Kwt5IH4rIa9q8=', 'Content-Length': '0'} 2024-04-16T18:27:48.875 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2b5a9179e0cabca-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.875 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.875 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.876 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:48.876 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:48.876 DEBUG:boto:Method: GET 2024-04-16T18:27:48.876 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:27:48.876 DEBUG:boto:Data: 2024-04-16T18:27:48.876 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.876 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.876 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.876 DEBUG:boto:Params: {} 2024-04-16T18:27:48.876 DEBUG:boto:Token: None 2024-04-16T18:27:48.877 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.877 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.880 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c6e2bfab85b9d5f-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.881 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.881 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.881 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.882 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:48.882 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:48.882 DEBUG:boto:Method: GET 2024-04-16T18:27:48.882 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:27:48.882 DEBUG:boto:Data: 2024-04-16T18:27:48.882 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.883 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.883 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.883 DEBUG:boto:Params: {} 2024-04-16T18:27:48.883 DEBUG:boto:Token: None 2024-04-16T18:27:48.883 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.883 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000210b6803e38680b3-00661ec324-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.930 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:48.931 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.931 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.931 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.931 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:27:48.931 DEBUG:boto:Data: 2024-04-16T18:27:48.931 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.931 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.931 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.931 DEBUG:boto:Params: {} 2024-04-16T18:27:48.931 DEBUG:boto:Token: None 2024-04-16T18:27:48.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44 2024-04-16T18:27:48.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gWlBvVviakP1pVQYGT81shia5KY= 2024-04-16T18:27:48.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gWlBvVviakP1pVQYGT81shia5KY=', 'Content-Length': '0'} 2024-04-16T18:27:48.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018b93ce56fb565d9-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.935 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.935 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.935 DEBUG:boto:Method: HEAD 2024-04-16T18:27:48.935 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:27:48.935 DEBUG:boto:Data: 2024-04-16T18:27:48.936 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.936 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.936 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.936 DEBUG:boto:Params: {} 2024-04-16T18:27:48.936 DEBUG:boto:Token: None 2024-04-16T18:27:48.936 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44 2024-04-16T18:27:48.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gWlBvVviakP1pVQYGT81shia5KY= 2024-04-16T18:27:48.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gWlBvVviakP1pVQYGT81shia5KY=', 'Content-Length': '0'} 2024-04-16T18:27:48.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a3556b65ecad0c9-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.939 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:27:48.939 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.940 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.940 DEBUG:boto:Method: GET 2024-04-16T18:27:48.940 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:27:48.940 DEBUG:boto:Data: 2024-04-16T18:27:48.940 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.940 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.940 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.940 DEBUG:boto:Params: {} 2024-04-16T18:27:48.940 DEBUG:boto:Token: None 2024-04-16T18:27:48.940 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44 2024-04-16T18:27:48.941 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:S/o6WNQty2P0V8MqMHbztoC/4gc= 2024-04-16T18:27:48.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:S/o6WNQty2P0V8MqMHbztoC/4gc=', 'Content-Length': '0'} 2024-04-16T18:27:48.944 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f533d272e5af7be4-00661ec324-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.986 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:27:48.986 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:27:48.987 DEBUG:boto:Method: GET 2024-04-16T18:27:48.987 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:27:48.987 DEBUG:boto:Data: 2024-04-16T18:27:48.987 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.987 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:48.987 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.987 DEBUG:boto:Params: {} 2024-04-16T18:27:48.987 DEBUG:boto:Token: None 2024-04-16T18:27:48.987 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/obj44 2024-04-16T18:27:48.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:S/o6WNQty2P0V8MqMHbztoC/4gc= 2024-04-16T18:27:48.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:S/o6WNQty2P0V8MqMHbztoC/4gc=', 'Content-Length': '0'} 2024-04-16T18:27:48.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca139803f0775cec-00661ec324-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:48.991 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:48.991 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:48.991 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:48.991 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:48.991 DEBUG:boto:Method: GET 2024-04-16T18:27:48.991 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:27:48.991 DEBUG:boto:Data: 2024-04-16T18:27:48.991 DEBUG:boto:Headers: {} 2024-04-16T18:27:48.991 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:48.991 DEBUG:boto:Port: 8000 2024-04-16T18:27:48.991 DEBUG:boto:Params: {} 2024-04-16T18:27:48.992 DEBUG:boto:Token: None 2024-04-16T18:27:48.992 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:48 GMT /lybtug-52/ 2024-04-16T18:27:48.992 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4= 2024-04-16T18:27:48.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XSUt46GhlOA9Q+lQ3Ufc7LuRrt4=', 'Content-Length': '0'} 2024-04-16T18:27:48.995 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000221bed95a6a6d3ff-00661ec324-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.038 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.039 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.039 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.039 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:49.039 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:27:49.039 DEBUG:boto:Method: GET 2024-04-16T18:27:49.039 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:27:49.039 DEBUG:boto:Data: 2024-04-16T18:27:49.039 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.039 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.039 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.039 DEBUG:boto:Params: {} 2024-04-16T18:27:49.039 DEBUG:boto:Token: None 2024-04-16T18:27:49.040 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009319c3a7b493e8fe-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.046 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.046 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:27:49.046 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.047 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.047 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.047 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.047 DEBUG:boto:Method: GET 2024-04-16T18:27:49.047 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.047 DEBUG:boto:Data: 2024-04-16T18:27:49.047 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.047 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.047 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.047 DEBUG:boto:Params: {} 2024-04-16T18:27:49.047 DEBUG:boto:Token: None 2024-04-16T18:27:49.047 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao= 2024-04-16T18:27:49.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao=', 'Content-Length': '0'} 2024-04-16T18:27:49.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bc6ba4b98c8d73d-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.094 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.094 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.095 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.095 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.095 DEBUG:boto:Method: GET 2024-04-16T18:27:49.095 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.095 DEBUG:boto:Data: 2024-04-16T18:27:49.095 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.095 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.095 DEBUG:boto:Params: {} 2024-04-16T18:27:49.095 DEBUG:boto:Token: None 2024-04-16T18:27:49.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao= 2024-04-16T18:27:49.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao=', 'Content-Length': '0'} 2024-04-16T18:27:49.099 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ccd99d169270707-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.100 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.100 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.100 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.100 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.100 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.100 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.100 DEBUG:boto:Data: 2024-04-16T18:27:49.100 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.100 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.100 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.100 DEBUG:boto:Params: {} 2024-04-16T18:27:49.101 DEBUG:boto:Token: None 2024-04-16T18:27:49.101 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.101 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RA9uWYpwHVcEm1H5kRhtHQQ7llg= 2024-04-16T18:27:49.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RA9uWYpwHVcEm1H5kRhtHQQ7llg=', 'Content-Length': '0'} 2024-04-16T18:27:49.104 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dfb940e477e5339-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.104 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.105 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.105 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.105 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.105 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.105 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.105 DEBUG:boto:Data: 2024-04-16T18:27:49.105 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.105 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.105 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.105 DEBUG:boto:Params: {} 2024-04-16T18:27:49.105 DEBUG:boto:Token: None 2024-04-16T18:27:49.106 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.106 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RA9uWYpwHVcEm1H5kRhtHQQ7llg= 2024-04-16T18:27:49.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RA9uWYpwHVcEm1H5kRhtHQQ7llg=', 'Content-Length': '0'} 2024-04-16T18:27:49.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de346d868f373270-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.109 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:27:49.109 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.109 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.109 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.109 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.110 DEBUG:boto:Method: GET 2024-04-16T18:27:49.110 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.110 DEBUG:boto:Data: 2024-04-16T18:27:49.110 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.110 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.110 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.110 DEBUG:boto:Params: {} 2024-04-16T18:27:49.110 DEBUG:boto:Token: None 2024-04-16T18:27:49.110 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.110 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao= 2024-04-16T18:27:49.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao=', 'Content-Length': '0'} 2024-04-16T18:27:49.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042100900cd3f2600-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.154 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.154 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.155 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.155 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.155 DEBUG:boto:Method: GET 2024-04-16T18:27:49.155 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.155 DEBUG:boto:Data: 2024-04-16T18:27:49.155 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.155 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.155 DEBUG:boto:Params: {} 2024-04-16T18:27:49.155 DEBUG:boto:Token: None 2024-04-16T18:27:49.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:27:49.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao= 2024-04-16T18:27:49.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VTED731SQv0oRO+VYY0woWqO6ao=', 'Content-Length': '0'} 2024-04-16T18:27:49.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea9a0604219d8bee-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.159 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.159 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.159 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.159 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.159 DEBUG:boto:Method: GET 2024-04-16T18:27:49.159 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:27:49.160 DEBUG:boto:Data: 2024-04-16T18:27:49.160 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.160 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.160 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.160 DEBUG:boto:Params: {} 2024-04-16T18:27:49.160 DEBUG:boto:Token: None 2024-04-16T18:27:49.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057ac105b641ced79-00661ec325-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.164 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.164 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.164 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.164 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.164 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.165 DEBUG:boto:Method: GET 2024-04-16T18:27:49.165 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:27:49.165 DEBUG:boto:Data: 2024-04-16T18:27:49.165 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.165 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.165 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.165 DEBUG:boto:Params: {} 2024-04-16T18:27:49.165 DEBUG:boto:Token: None 2024-04-16T18:27:49.165 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.165 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.171 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095997d06495adc5f-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.214 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.215 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.215 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.215 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.215 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:27:49.215 DEBUG:boto:Data: 2024-04-16T18:27:49.215 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.215 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.215 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.215 DEBUG:boto:Params: {} 2024-04-16T18:27:49.215 DEBUG:boto:Token: None 2024-04-16T18:27:49.216 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46 2024-04-16T18:27:49.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fW8EJbTg5ILnbpKShgO0M4fku+U= 2024-04-16T18:27:49.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fW8EJbTg5ILnbpKShgO0M4fku+U=', 'Content-Length': '0'} 2024-04-16T18:27:49.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edf8bd6592d57612-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.219 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.219 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.219 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.219 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:27:49.219 DEBUG:boto:Data: 2024-04-16T18:27:49.219 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.219 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.219 DEBUG:boto:Params: {} 2024-04-16T18:27:49.220 DEBUG:boto:Token: None 2024-04-16T18:27:49.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46 2024-04-16T18:27:49.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fW8EJbTg5ILnbpKShgO0M4fku+U= 2024-04-16T18:27:49.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fW8EJbTg5ILnbpKShgO0M4fku+U=', 'Content-Length': '0'} 2024-04-16T18:27:49.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adfd3841defec8bc-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.223 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:27:49.223 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.223 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.223 DEBUG:boto:Method: GET 2024-04-16T18:27:49.224 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:27:49.224 DEBUG:boto:Data: 2024-04-16T18:27:49.224 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.224 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.224 DEBUG:boto:Params: {} 2024-04-16T18:27:49.224 DEBUG:boto:Token: None 2024-04-16T18:27:49.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46 2024-04-16T18:27:49.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SFbhMYxOzoFIeg9WK3MBNC7bNbg= 2024-04-16T18:27:49.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SFbhMYxOzoFIeg9WK3MBNC7bNbg=', 'Content-Length': '0'} 2024-04-16T18:27:49.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078bb2c9fadf28a55-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.270 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:27:49.270 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:27:49.271 DEBUG:boto:Method: GET 2024-04-16T18:27:49.271 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:27:49.271 DEBUG:boto:Data: 2024-04-16T18:27:49.271 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.271 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.271 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.271 DEBUG:boto:Params: {} 2024-04-16T18:27:49.271 DEBUG:boto:Token: None 2024-04-16T18:27:49.271 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj46 2024-04-16T18:27:49.271 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:SFbhMYxOzoFIeg9WK3MBNC7bNbg= 2024-04-16T18:27:49.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SFbhMYxOzoFIeg9WK3MBNC7bNbg=', 'Content-Length': '0'} 2024-04-16T18:27:49.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024c4d074d790882f-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.274 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.275 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.275 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.275 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.275 DEBUG:boto:Method: GET 2024-04-16T18:27:49.275 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:27:49.275 DEBUG:boto:Data: 2024-04-16T18:27:49.275 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.275 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.275 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.275 DEBUG:boto:Params: {} 2024-04-16T18:27:49.275 DEBUG:boto:Token: None 2024-04-16T18:27:49.276 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.276 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004fd7ae4172853bed-00661ec325-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.322 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.323 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.323 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.323 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.323 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:27:49.323 DEBUG:boto:Method: GET 2024-04-16T18:27:49.323 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:27:49.323 DEBUG:boto:Data: 2024-04-16T18:27:49.323 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.323 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.323 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.324 DEBUG:boto:Params: {} 2024-04-16T18:27:49.324 DEBUG:boto:Token: None 2024-04-16T18:27:49.324 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.330 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0d287b7d85125c0-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.330 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.330 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:27:49.330 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.330 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.330 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.331 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.331 DEBUG:boto:Method: GET 2024-04-16T18:27:49.331 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.331 DEBUG:boto:Data: 2024-04-16T18:27:49.331 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.331 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.331 DEBUG:boto:Params: {} 2024-04-16T18:27:49.331 DEBUG:boto:Token: None 2024-04-16T18:27:49.331 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.331 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM= 2024-04-16T18:27:49.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM=', 'Content-Length': '0'} 2024-04-16T18:27:49.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000919f5aa6a378dc5-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.378 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.378 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.379 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.379 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.379 DEBUG:boto:Method: GET 2024-04-16T18:27:49.379 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.379 DEBUG:boto:Data: 2024-04-16T18:27:49.379 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.379 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.379 DEBUG:boto:Params: {} 2024-04-16T18:27:49.379 DEBUG:boto:Token: None 2024-04-16T18:27:49.379 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.379 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM= 2024-04-16T18:27:49.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM=', 'Content-Length': '0'} 2024-04-16T18:27:49.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000accb85e6185c24ea-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.383 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.383 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.383 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.383 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.383 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.383 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.384 DEBUG:boto:Data: 2024-04-16T18:27:49.384 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.384 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.384 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.384 DEBUG:boto:Params: {} 2024-04-16T18:27:49.384 DEBUG:boto:Token: None 2024-04-16T18:27:49.384 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rEw6ntzFYBay+Ahya9FhYST+0dY= 2024-04-16T18:27:49.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rEw6ntzFYBay+Ahya9FhYST+0dY=', 'Content-Length': '0'} 2024-04-16T18:27:49.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000568058522e4d1eac-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.387 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.387 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.388 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.388 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.388 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.388 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.388 DEBUG:boto:Data: 2024-04-16T18:27:49.388 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.388 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.388 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.388 DEBUG:boto:Params: {} 2024-04-16T18:27:49.388 DEBUG:boto:Token: None 2024-04-16T18:27:49.388 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rEw6ntzFYBay+Ahya9FhYST+0dY= 2024-04-16T18:27:49.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rEw6ntzFYBay+Ahya9FhYST+0dY=', 'Content-Length': '0'} 2024-04-16T18:27:49.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd442d80d9019dbd-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.393 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:27:49.393 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.393 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.393 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.393 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.393 DEBUG:boto:Method: GET 2024-04-16T18:27:49.393 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.393 DEBUG:boto:Data: 2024-04-16T18:27:49.393 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.393 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.394 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.394 DEBUG:boto:Params: {} 2024-04-16T18:27:49.394 DEBUG:boto:Token: None 2024-04-16T18:27:49.394 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.394 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM= 2024-04-16T18:27:49.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM=', 'Content-Length': '0'} 2024-04-16T18:27:49.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048e655547040394c-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.438 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.439 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.439 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.439 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.439 DEBUG:boto:Method: GET 2024-04-16T18:27:49.439 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.439 DEBUG:boto:Data: 2024-04-16T18:27:49.439 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.439 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.439 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.440 DEBUG:boto:Params: {} 2024-04-16T18:27:49.440 DEBUG:boto:Token: None 2024-04-16T18:27:49.440 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:27:49.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM= 2024-04-16T18:27:49.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O/4XdHagWOBNHO1bcJfsAlGz5qM=', 'Content-Length': '0'} 2024-04-16T18:27:49.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085830d9427d6fd45-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.443 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.444 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.444 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.444 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.444 DEBUG:boto:Method: GET 2024-04-16T18:27:49.444 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:27:49.444 DEBUG:boto:Data: 2024-04-16T18:27:49.444 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.444 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.444 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.444 DEBUG:boto:Params: {} 2024-04-16T18:27:49.445 DEBUG:boto:Token: None 2024-04-16T18:27:49.445 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.445 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfff2faf152d695c-00661ec325-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.448 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.449 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.449 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.449 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.449 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.449 DEBUG:boto:Method: GET 2024-04-16T18:27:49.449 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:27:49.450 DEBUG:boto:Data: 2024-04-16T18:27:49.450 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.450 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.450 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.450 DEBUG:boto:Params: {} 2024-04-16T18:27:49.450 DEBUG:boto:Token: None 2024-04-16T18:27:49.450 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.450 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.456 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000454a45990916b6e5-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.502 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.503 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.503 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.503 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.503 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:27:49.503 DEBUG:boto:Data: 2024-04-16T18:27:49.503 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.503 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.504 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.504 DEBUG:boto:Params: {} 2024-04-16T18:27:49.504 DEBUG:boto:Token: None 2024-04-16T18:27:49.504 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51 2024-04-16T18:27:49.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:37+gAhxzVKg9yWkbo5MvrqQtUy8= 2024-04-16T18:27:49.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:37+gAhxzVKg9yWkbo5MvrqQtUy8=', 'Content-Length': '0'} 2024-04-16T18:27:49.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e486f3991dad71c5-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.508 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.508 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.508 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.508 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:27:49.508 DEBUG:boto:Data: 2024-04-16T18:27:49.508 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.509 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.509 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.509 DEBUG:boto:Params: {} 2024-04-16T18:27:49.509 DEBUG:boto:Token: None 2024-04-16T18:27:49.509 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51 2024-04-16T18:27:49.509 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:37+gAhxzVKg9yWkbo5MvrqQtUy8= 2024-04-16T18:27:49.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:37+gAhxzVKg9yWkbo5MvrqQtUy8=', 'Content-Length': '0'} 2024-04-16T18:27:49.512 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2c4deeeeaff82a5-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.513 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:27:49.513 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.513 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.513 DEBUG:boto:Method: GET 2024-04-16T18:27:49.513 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:27:49.513 DEBUG:boto:Data: 2024-04-16T18:27:49.513 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.513 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.514 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.514 DEBUG:boto:Params: {} 2024-04-16T18:27:49.514 DEBUG:boto:Token: None 2024-04-16T18:27:49.514 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51 2024-04-16T18:27:49.514 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TxKyvD8qJ9n+vRrfe5cXKBnSPBg= 2024-04-16T18:27:49.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TxKyvD8qJ9n+vRrfe5cXKBnSPBg=', 'Content-Length': '0'} 2024-04-16T18:27:49.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065e27b0d5870e981-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.558 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:27:49.558 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:27:49.559 DEBUG:boto:Method: GET 2024-04-16T18:27:49.559 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:27:49.559 DEBUG:boto:Data: 2024-04-16T18:27:49.559 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.559 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.559 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.559 DEBUG:boto:Params: {} 2024-04-16T18:27:49.559 DEBUG:boto:Token: None 2024-04-16T18:27:49.559 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj51 2024-04-16T18:27:49.559 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TxKyvD8qJ9n+vRrfe5cXKBnSPBg= 2024-04-16T18:27:49.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TxKyvD8qJ9n+vRrfe5cXKBnSPBg=', 'Content-Length': '0'} 2024-04-16T18:27:49.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056f5b6aedf83bb61-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.563 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.563 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.564 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.564 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.564 DEBUG:boto:Method: GET 2024-04-16T18:27:49.564 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:27:49.564 DEBUG:boto:Data: 2024-04-16T18:27:49.564 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.564 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.564 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.564 DEBUG:boto:Params: {} 2024-04-16T18:27:49.564 DEBUG:boto:Token: None 2024-04-16T18:27:49.564 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.565 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a7711e9b49658d9-00661ec325-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.610 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.611 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.611 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.611 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.611 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:27:49.611 DEBUG:boto:Method: GET 2024-04-16T18:27:49.611 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:27:49.611 DEBUG:boto:Data: 2024-04-16T18:27:49.611 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.611 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.611 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.612 DEBUG:boto:Params: {} 2024-04-16T18:27:49.612 DEBUG:boto:Token: None 2024-04-16T18:27:49.612 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ca6c7c70072f056-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.618 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:27:49.619 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.619 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.619 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.619 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.619 DEBUG:boto:Method: GET 2024-04-16T18:27:49.619 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.619 DEBUG:boto:Data: 2024-04-16T18:27:49.619 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.620 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.620 DEBUG:boto:Params: {} 2024-04-16T18:27:49.620 DEBUG:boto:Token: None 2024-04-16T18:27:49.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE= 2024-04-16T18:27:49.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE=', 'Content-Length': '0'} 2024-04-16T18:27:49.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d73f1a6a82c58f8f-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.666 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.666 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.667 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.667 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.667 DEBUG:boto:Method: GET 2024-04-16T18:27:49.667 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.667 DEBUG:boto:Data: 2024-04-16T18:27:49.667 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.667 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.667 DEBUG:boto:Params: {} 2024-04-16T18:27:49.667 DEBUG:boto:Token: None 2024-04-16T18:27:49.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE= 2024-04-16T18:27:49.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE=', 'Content-Length': '0'} 2024-04-16T18:27:49.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ab0b72464cd5d8c-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.671 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.671 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.671 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.671 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.671 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.671 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.671 DEBUG:boto:Data: 2024-04-16T18:27:49.671 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.672 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.672 DEBUG:boto:Params: {} 2024-04-16T18:27:49.672 DEBUG:boto:Token: None 2024-04-16T18:27:49.672 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2Be6+P41an/808wTMVtuuuCjNnc= 2024-04-16T18:27:49.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Be6+P41an/808wTMVtuuuCjNnc=', 'Content-Length': '0'} 2024-04-16T18:27:49.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000008dfedb5275fe85-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.675 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.675 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.675 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.675 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.675 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.675 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.675 DEBUG:boto:Data: 2024-04-16T18:27:49.676 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.676 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.676 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.676 DEBUG:boto:Params: {} 2024-04-16T18:27:49.676 DEBUG:boto:Token: None 2024-04-16T18:27:49.676 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.676 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2Be6+P41an/808wTMVtuuuCjNnc= 2024-04-16T18:27:49.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Be6+P41an/808wTMVtuuuCjNnc=', 'Content-Length': '0'} 2024-04-16T18:27:49.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fe8807272f07d72-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:27:49.680 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.680 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.680 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.680 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.680 DEBUG:boto:Method: GET 2024-04-16T18:27:49.680 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.680 DEBUG:boto:Data: 2024-04-16T18:27:49.680 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.680 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.681 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.681 DEBUG:boto:Params: {} 2024-04-16T18:27:49.681 DEBUG:boto:Token: None 2024-04-16T18:27:49.681 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.681 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE= 2024-04-16T18:27:49.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE=', 'Content-Length': '0'} 2024-04-16T18:27:49.684 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046d2dac5dee79cd4-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.726 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.727 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.727 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.727 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.727 DEBUG:boto:Method: GET 2024-04-16T18:27:49.727 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.727 DEBUG:boto:Data: 2024-04-16T18:27:49.727 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.727 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.727 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.727 DEBUG:boto:Params: {} 2024-04-16T18:27:49.727 DEBUG:boto:Token: None 2024-04-16T18:27:49.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:27:49.728 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE= 2024-04-16T18:27:49.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ReV+X3G1sr/23QC3rHdLZ+Rz7CE=', 'Content-Length': '0'} 2024-04-16T18:27:49.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc3841ecb20223c3-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.731 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.731 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.732 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.732 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.732 DEBUG:boto:Method: GET 2024-04-16T18:27:49.732 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:27:49.732 DEBUG:boto:Data: 2024-04-16T18:27:49.732 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.732 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.732 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.732 DEBUG:boto:Params: {} 2024-04-16T18:27:49.732 DEBUG:boto:Token: None 2024-04-16T18:27:49.733 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c42196f561f85b7c-00661ec325-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.737 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.737 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.737 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.737 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.737 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.737 DEBUG:boto:Method: GET 2024-04-16T18:27:49.738 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:27:49.738 DEBUG:boto:Data: 2024-04-16T18:27:49.738 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.738 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.738 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.738 DEBUG:boto:Params: {} 2024-04-16T18:27:49.738 DEBUG:boto:Token: None 2024-04-16T18:27:49.738 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.738 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.745 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002207a6ec40a1aa6b-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.786 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.787 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.787 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.787 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.787 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:27:49.787 DEBUG:boto:Data: 2024-04-16T18:27:49.787 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.787 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.787 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.787 DEBUG:boto:Params: {} 2024-04-16T18:27:49.788 DEBUG:boto:Token: None 2024-04-16T18:27:49.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55 2024-04-16T18:27:49.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Cx4go6s/uAn0nlUIeNPFQuSMRUE= 2024-04-16T18:27:49.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Cx4go6s/uAn0nlUIeNPFQuSMRUE=', 'Content-Length': '0'} 2024-04-16T18:27:49.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d6c2c485bef4ce1-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.791 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.791 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.791 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.791 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:27:49.792 DEBUG:boto:Data: 2024-04-16T18:27:49.792 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.792 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.792 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.792 DEBUG:boto:Params: {} 2024-04-16T18:27:49.792 DEBUG:boto:Token: None 2024-04-16T18:27:49.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55 2024-04-16T18:27:49.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Cx4go6s/uAn0nlUIeNPFQuSMRUE= 2024-04-16T18:27:49.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Cx4go6s/uAn0nlUIeNPFQuSMRUE=', 'Content-Length': '0'} 2024-04-16T18:27:49.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df5fdbb01f244822-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:27:49.796 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.796 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.796 DEBUG:boto:Method: GET 2024-04-16T18:27:49.796 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:27:49.796 DEBUG:boto:Data: 2024-04-16T18:27:49.796 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.796 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.796 DEBUG:boto:Params: {} 2024-04-16T18:27:49.796 DEBUG:boto:Token: None 2024-04-16T18:27:49.797 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55 2024-04-16T18:27:49.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6tMLVHzqnSjQy0MzOc3f5T47cGE= 2024-04-16T18:27:49.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6tMLVHzqnSjQy0MzOc3f5T47cGE=', 'Content-Length': '0'} 2024-04-16T18:27:49.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054b2d465fe62f4f5-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.842 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:27:49.843 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:27:49.843 DEBUG:boto:Method: GET 2024-04-16T18:27:49.843 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:27:49.843 DEBUG:boto:Data: 2024-04-16T18:27:49.843 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.843 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.843 DEBUG:boto:Params: {} 2024-04-16T18:27:49.843 DEBUG:boto:Token: None 2024-04-16T18:27:49.843 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj55 2024-04-16T18:27:49.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6tMLVHzqnSjQy0MzOc3f5T47cGE= 2024-04-16T18:27:49.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6tMLVHzqnSjQy0MzOc3f5T47cGE=', 'Content-Length': '0'} 2024-04-16T18:27:49.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000817bb360ba8e8bc0-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.847 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.847 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.847 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.847 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.847 DEBUG:boto:Method: GET 2024-04-16T18:27:49.848 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:27:49.848 DEBUG:boto:Data: 2024-04-16T18:27:49.848 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.848 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.848 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.848 DEBUG:boto:Params: {} 2024-04-16T18:27:49.848 DEBUG:boto:Token: None 2024-04-16T18:27:49.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.854 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036572e81520070f8-00661ec325-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.898 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.899 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:49.899 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:49.899 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.899 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:27:49.899 DEBUG:boto:Method: GET 2024-04-16T18:27:49.899 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:27:49.899 DEBUG:boto:Data: 2024-04-16T18:27:49.899 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.899 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.899 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.899 DEBUG:boto:Params: {} 2024-04-16T18:27:49.900 DEBUG:boto:Token: None 2024-04-16T18:27:49.900 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/ 2024-04-16T18:27:49.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs= 2024-04-16T18:27:49.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5+yKCeMvHsECynFr8gn7fDw3wKs=', 'Content-Length': '0'} 2024-04-16T18:27:49.906 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d3ec51a3302ffd0-00661ec325-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.906 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:49.907 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:27:49.907 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:49.907 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:49.907 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.907 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.907 DEBUG:boto:Method: GET 2024-04-16T18:27:49.907 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.907 DEBUG:boto:Data: 2024-04-16T18:27:49.907 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.907 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.907 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.908 DEBUG:boto:Params: {} 2024-04-16T18:27:49.908 DEBUG:boto:Token: None 2024-04-16T18:27:49.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CqqtKof+dinqGrs0Ri7/laqipYE= 2024-04-16T18:27:49.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CqqtKof+dinqGrs0Ri7/laqipYE=', 'Content-Length': '0'} 2024-04-16T18:27:49.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf1cbf92145a7b22-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.954 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:49.954 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:49.955 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.955 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.955 DEBUG:boto:Method: GET 2024-04-16T18:27:49.955 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.955 DEBUG:boto:Data: 2024-04-16T18:27:49.955 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.955 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.955 DEBUG:boto:Params: {} 2024-04-16T18:27:49.955 DEBUG:boto:Token: None 2024-04-16T18:27:49.955 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CqqtKof+dinqGrs0Ri7/laqipYE= 2024-04-16T18:27:49.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CqqtKof+dinqGrs0Ri7/laqipYE=', 'Content-Length': '0'} 2024-04-16T18:27:49.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f8be12a12a8ec54-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.959 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:49.959 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:49.959 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.959 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.959 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.959 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.960 DEBUG:boto:Data: 2024-04-16T18:27:49.960 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.960 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.960 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.960 DEBUG:boto:Params: {} 2024-04-16T18:27:49.960 DEBUG:boto:Token: None 2024-04-16T18:27:49.960 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n+d2nGDdp4FNte19jFN1XwAvmQQ= 2024-04-16T18:27:49.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n+d2nGDdp4FNte19jFN1XwAvmQQ=', 'Content-Length': '0'} 2024-04-16T18:27:49.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026720c877961a066-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.964 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:49.964 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:49.964 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.964 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.964 DEBUG:boto:Method: HEAD 2024-04-16T18:27:49.964 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.964 DEBUG:boto:Data: 2024-04-16T18:27:49.964 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:49.965 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.965 DEBUG:boto:Params: {} 2024-04-16T18:27:49.965 DEBUG:boto:Token: None 2024-04-16T18:27:49.965 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.965 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n+d2nGDdp4FNte19jFN1XwAvmQQ= 2024-04-16T18:27:49.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n+d2nGDdp4FNte19jFN1XwAvmQQ=', 'Content-Length': '0'} 2024-04-16T18:27:49.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dec9e45bb63a8b9b-00661ec325-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:49.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:27:49.968 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:49.968 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:49.968 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.968 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.968 DEBUG:boto:Method: GET 2024-04-16T18:27:49.969 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.969 DEBUG:boto:Data: 2024-04-16T18:27:49.969 DEBUG:boto:Headers: {} 2024-04-16T18:27:49.969 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:49.969 DEBUG:boto:Port: 8000 2024-04-16T18:27:49.969 DEBUG:boto:Params: {} 2024-04-16T18:27:49.969 DEBUG:boto:Token: None 2024-04-16T18:27:49.969 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:27:49.969 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CqqtKof+dinqGrs0Ri7/laqipYE= 2024-04-16T18:27:49.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CqqtKof+dinqGrs0Ri7/laqipYE=', 'Content-Length': '0'} 2024-04-16T18:27:49.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000883dba1ef81ad36-00661ec325-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.014 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:50.014 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:50.015 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:50.015 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:27:50.015 DEBUG:boto:Method: GET 2024-04-16T18:27:50.015 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:27:50.015 DEBUG:boto:Data: 2024-04-16T18:27:50.015 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.015 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.015 DEBUG:boto:Params: {} 2024-04-16T18:27:50.015 DEBUG:boto:Token: None 2024-04-16T18:27:50.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:27:50.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n8/ASOi2uwB6ULeucARUlz3B/oI= 2024-04-16T18:27:50.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n8/ASOi2uwB6ULeucARUlz3B/oI=', 'Content-Length': '0'} 2024-04-16T18:27:50.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d48a47143260d68-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.020 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.020 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.020 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.020 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.020 DEBUG:boto:Method: GET 2024-04-16T18:27:50.021 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:27:50.021 DEBUG:boto:Data: 2024-04-16T18:27:50.021 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.021 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.021 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.021 DEBUG:boto:Params: {} 2024-04-16T18:27:50.021 DEBUG:boto:Token: None 2024-04-16T18:27:50.021 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.021 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044f91b62e9069f50-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.025 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.026 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.026 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.026 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.026 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.026 DEBUG:boto:Method: GET 2024-04-16T18:27:50.026 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:27:50.026 DEBUG:boto:Data: 2024-04-16T18:27:50.026 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.026 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.027 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.027 DEBUG:boto:Params: {} 2024-04-16T18:27:50.027 DEBUG:boto:Token: None 2024-04-16T18:27:50.027 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.027 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.034 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1fb9bf4b38bf5ed-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.074 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.075 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:50.075 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:50.075 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.075 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:27:50.075 DEBUG:boto:Data: 2024-04-16T18:27:50.075 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.075 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.075 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.075 DEBUG:boto:Params: {} 2024-04-16T18:27:50.075 DEBUG:boto:Token: None 2024-04-16T18:27:50.076 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj57 2024-04-16T18:27:50.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:f8w4cKRxpyrFZZgRdawQLmbgApY= 2024-04-16T18:27:50.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:f8w4cKRxpyrFZZgRdawQLmbgApY=', 'Content-Length': '0'} 2024-04-16T18:27:50.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdadd591c88c41f4-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.079 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:50.079 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:50.079 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.079 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:27:50.080 DEBUG:boto:Data: 2024-04-16T18:27:50.080 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.080 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.080 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.080 DEBUG:boto:Params: {} 2024-04-16T18:27:50.080 DEBUG:boto:Token: None 2024-04-16T18:27:50.080 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj57 2024-04-16T18:27:50.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:f8w4cKRxpyrFZZgRdawQLmbgApY= 2024-04-16T18:27:50.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:f8w4cKRxpyrFZZgRdawQLmbgApY=', 'Content-Length': '0'} 2024-04-16T18:27:50.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e63aab8292ddec8-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:27:50.084 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:50.084 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:50.084 DEBUG:boto:Method: GET 2024-04-16T18:27:50.084 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:27:50.084 DEBUG:boto:Data: 2024-04-16T18:27:50.084 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.085 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.085 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.085 DEBUG:boto:Params: {} 2024-04-16T18:27:50.085 DEBUG:boto:Token: None 2024-04-16T18:27:50.085 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj57 2024-04-16T18:27:50.085 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k8xUXRP7ySuqeqdHCthSfqg+oHk= 2024-04-16T18:27:50.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k8xUXRP7ySuqeqdHCthSfqg+oHk=', 'Content-Length': '0'} 2024-04-16T18:27:50.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c595510805a3df78-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.130 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:27:50.130 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:27:50.131 DEBUG:boto:Method: GET 2024-04-16T18:27:50.131 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:27:50.131 DEBUG:boto:Data: 2024-04-16T18:27:50.131 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.131 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.131 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.131 DEBUG:boto:Params: {} 2024-04-16T18:27:50.131 DEBUG:boto:Token: None 2024-04-16T18:27:50.131 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj57 2024-04-16T18:27:50.131 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k8xUXRP7ySuqeqdHCthSfqg+oHk= 2024-04-16T18:27:50.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k8xUXRP7ySuqeqdHCthSfqg+oHk=', 'Content-Length': '0'} 2024-04-16T18:27:50.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000466f341ec3d1fb28-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.135 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.135 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.135 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.135 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.136 DEBUG:boto:Method: GET 2024-04-16T18:27:50.136 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:27:50.136 DEBUG:boto:Data: 2024-04-16T18:27:50.136 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.136 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.136 DEBUG:boto:Params: {} 2024-04-16T18:27:50.136 DEBUG:boto:Token: None 2024-04-16T18:27:50.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1c09d4132291f6d-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.182 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.183 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.183 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.183 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.183 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:27:50.183 DEBUG:boto:Method: GET 2024-04-16T18:27:50.183 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:27:50.183 DEBUG:boto:Data: 2024-04-16T18:27:50.183 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.183 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.183 DEBUG:boto:Params: {} 2024-04-16T18:27:50.184 DEBUG:boto:Token: None 2024-04-16T18:27:50.184 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.189 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f6a9f5f24625f2b-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.190 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.190 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:27:50.190 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.191 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.191 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.191 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.191 DEBUG:boto:Method: GET 2024-04-16T18:27:50.191 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.191 DEBUG:boto:Data: 2024-04-16T18:27:50.191 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.191 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.191 DEBUG:boto:Params: {} 2024-04-16T18:27:50.191 DEBUG:boto:Token: None 2024-04-16T18:27:50.192 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ= 2024-04-16T18:27:50.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ=', 'Content-Length': '0'} 2024-04-16T18:27:50.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0888046811c7ddb-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.238 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.238 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.238 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.239 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.239 DEBUG:boto:Method: GET 2024-04-16T18:27:50.239 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.239 DEBUG:boto:Data: 2024-04-16T18:27:50.239 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.239 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.239 DEBUG:boto:Params: {} 2024-04-16T18:27:50.239 DEBUG:boto:Token: None 2024-04-16T18:27:50.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ= 2024-04-16T18:27:50.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ=', 'Content-Length': '0'} 2024-04-16T18:27:50.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1de37e320d91ad8-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.243 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.243 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.243 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.243 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.243 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.243 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.244 DEBUG:boto:Data: 2024-04-16T18:27:50.244 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.244 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.244 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.244 DEBUG:boto:Params: {} 2024-04-16T18:27:50.244 DEBUG:boto:Token: None 2024-04-16T18:27:50.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EON1IFUNwCki1GwiU3GtenP2FNw= 2024-04-16T18:27:50.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EON1IFUNwCki1GwiU3GtenP2FNw=', 'Content-Length': '0'} 2024-04-16T18:27:50.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003976289492ec76a9-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.247 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.247 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.247 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.247 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.247 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.248 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.248 DEBUG:boto:Data: 2024-04-16T18:27:50.248 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.248 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.248 DEBUG:boto:Params: {} 2024-04-16T18:27:50.248 DEBUG:boto:Token: None 2024-04-16T18:27:50.248 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EON1IFUNwCki1GwiU3GtenP2FNw= 2024-04-16T18:27:50.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EON1IFUNwCki1GwiU3GtenP2FNw=', 'Content-Length': '0'} 2024-04-16T18:27:50.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008140469140b2e908-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:27:50.251 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.252 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.252 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.252 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.252 DEBUG:boto:Method: GET 2024-04-16T18:27:50.252 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.252 DEBUG:boto:Data: 2024-04-16T18:27:50.252 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.252 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.252 DEBUG:boto:Params: {} 2024-04-16T18:27:50.253 DEBUG:boto:Token: None 2024-04-16T18:27:50.253 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.253 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ= 2024-04-16T18:27:50.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ=', 'Content-Length': '0'} 2024-04-16T18:27:50.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000031af64a31d35439-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.298 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.298 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.299 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.299 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.299 DEBUG:boto:Method: GET 2024-04-16T18:27:50.299 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.299 DEBUG:boto:Data: 2024-04-16T18:27:50.299 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.299 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.299 DEBUG:boto:Params: {} 2024-04-16T18:27:50.299 DEBUG:boto:Token: None 2024-04-16T18:27:50.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:27:50.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ= 2024-04-16T18:27:50.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bI/E8zdWgQdWpMBluRcnJQ9vshQ=', 'Content-Length': '0'} 2024-04-16T18:27:50.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071cc05bd4beb5e3f-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.303 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.303 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.303 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.303 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.303 DEBUG:boto:Method: GET 2024-04-16T18:27:50.303 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:27:50.303 DEBUG:boto:Data: 2024-04-16T18:27:50.303 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.304 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.304 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.304 DEBUG:boto:Params: {} 2024-04-16T18:27:50.304 DEBUG:boto:Token: None 2024-04-16T18:27:50.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4f9ac42d3557f50-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.308 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.308 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.308 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.308 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.309 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.309 DEBUG:boto:Method: GET 2024-04-16T18:27:50.309 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:27:50.309 DEBUG:boto:Data: 2024-04-16T18:27:50.309 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.309 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.309 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.309 DEBUG:boto:Params: {} 2024-04-16T18:27:50.309 DEBUG:boto:Token: None 2024-04-16T18:27:50.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.316 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000214cefb1fee604e8-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.358 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.359 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.359 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.359 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.359 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:27:50.359 DEBUG:boto:Data: 2024-04-16T18:27:50.359 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.359 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.359 DEBUG:boto:Params: {} 2024-04-16T18:27:50.360 DEBUG:boto:Token: None 2024-04-16T18:27:50.360 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58 2024-04-16T18:27:50.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0azak8hTnWKS4U3ek37KYzNmY9k= 2024-04-16T18:27:50.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0azak8hTnWKS4U3ek37KYzNmY9k=', 'Content-Length': '0'} 2024-04-16T18:27:50.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084487b795424ef34-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.363 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.363 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.363 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.363 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:27:50.363 DEBUG:boto:Data: 2024-04-16T18:27:50.363 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.363 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.364 DEBUG:boto:Params: {} 2024-04-16T18:27:50.364 DEBUG:boto:Token: None 2024-04-16T18:27:50.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58 2024-04-16T18:27:50.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0azak8hTnWKS4U3ek37KYzNmY9k= 2024-04-16T18:27:50.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0azak8hTnWKS4U3ek37KYzNmY9k=', 'Content-Length': '0'} 2024-04-16T18:27:50.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098ad42ae4b36fa17-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:27:50.367 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.367 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.367 DEBUG:boto:Method: GET 2024-04-16T18:27:50.367 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:27:50.367 DEBUG:boto:Data: 2024-04-16T18:27:50.367 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.367 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.367 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.367 DEBUG:boto:Params: {} 2024-04-16T18:27:50.368 DEBUG:boto:Token: None 2024-04-16T18:27:50.368 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58 2024-04-16T18:27:50.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mBSbsBPBLsVY+w42qZB/kkhgoTI= 2024-04-16T18:27:50.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mBSbsBPBLsVY+w42qZB/kkhgoTI=', 'Content-Length': '0'} 2024-04-16T18:27:50.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001dfdf2b29e9a2128-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.414 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:27:50.414 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:27:50.415 DEBUG:boto:Method: GET 2024-04-16T18:27:50.415 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:27:50.415 DEBUG:boto:Data: 2024-04-16T18:27:50.415 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.415 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.415 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.415 DEBUG:boto:Params: {} 2024-04-16T18:27:50.415 DEBUG:boto:Token: None 2024-04-16T18:27:50.415 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj58 2024-04-16T18:27:50.415 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mBSbsBPBLsVY+w42qZB/kkhgoTI= 2024-04-16T18:27:50.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mBSbsBPBLsVY+w42qZB/kkhgoTI=', 'Content-Length': '0'} 2024-04-16T18:27:50.418 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000199a1f1fc47abb29-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.419 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.419 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.419 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.419 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.419 DEBUG:boto:Method: GET 2024-04-16T18:27:50.419 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:27:50.419 DEBUG:boto:Data: 2024-04-16T18:27:50.419 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.419 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.419 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.420 DEBUG:boto:Params: {} 2024-04-16T18:27:50.420 DEBUG:boto:Token: None 2024-04-16T18:27:50.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e76b29103b00ce22-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.466 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.467 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.467 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.467 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.467 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:27:50.467 DEBUG:boto:Method: GET 2024-04-16T18:27:50.467 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:27:50.467 DEBUG:boto:Data: 2024-04-16T18:27:50.467 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.467 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.467 DEBUG:boto:Params: {} 2024-04-16T18:27:50.468 DEBUG:boto:Token: None 2024-04-16T18:27:50.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4fed3e2d8198186-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.474 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.474 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:27:50.475 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.475 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.475 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.475 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.475 DEBUG:boto:Method: GET 2024-04-16T18:27:50.475 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.475 DEBUG:boto:Data: 2024-04-16T18:27:50.475 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.475 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.475 DEBUG:boto:Params: {} 2024-04-16T18:27:50.476 DEBUG:boto:Token: None 2024-04-16T18:27:50.476 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc= 2024-04-16T18:27:50.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc=', 'Content-Length': '0'} 2024-04-16T18:27:50.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065831cd66fa9c9df-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.522 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.522 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.523 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.523 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.523 DEBUG:boto:Method: GET 2024-04-16T18:27:50.523 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.523 DEBUG:boto:Data: 2024-04-16T18:27:50.523 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.523 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.523 DEBUG:boto:Params: {} 2024-04-16T18:27:50.523 DEBUG:boto:Token: None 2024-04-16T18:27:50.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc= 2024-04-16T18:27:50.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc=', 'Content-Length': '0'} 2024-04-16T18:27:50.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cecc27fcbf7d2918-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.527 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.527 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.527 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.527 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.527 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.527 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.528 DEBUG:boto:Data: 2024-04-16T18:27:50.528 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.528 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.528 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.528 DEBUG:boto:Params: {} 2024-04-16T18:27:50.528 DEBUG:boto:Token: None 2024-04-16T18:27:50.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DC20t+2jLj5EdNkSSdugR9iUQJ8= 2024-04-16T18:27:50.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DC20t+2jLj5EdNkSSdugR9iUQJ8=', 'Content-Length': '0'} 2024-04-16T18:27:50.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd3708821b46b09f-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.531 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.531 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.531 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.531 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.531 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.532 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.532 DEBUG:boto:Data: 2024-04-16T18:27:50.532 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.532 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.532 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.532 DEBUG:boto:Params: {} 2024-04-16T18:27:50.532 DEBUG:boto:Token: None 2024-04-16T18:27:50.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:DC20t+2jLj5EdNkSSdugR9iUQJ8= 2024-04-16T18:27:50.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DC20t+2jLj5EdNkSSdugR9iUQJ8=', 'Content-Length': '0'} 2024-04-16T18:27:50.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078967532d978bdad-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.535 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:27:50.536 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.536 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.536 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.536 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.536 DEBUG:boto:Method: GET 2024-04-16T18:27:50.536 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.536 DEBUG:boto:Data: 2024-04-16T18:27:50.536 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.536 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.536 DEBUG:boto:Params: {} 2024-04-16T18:27:50.537 DEBUG:boto:Token: None 2024-04-16T18:27:50.537 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.537 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc= 2024-04-16T18:27:50.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc=', 'Content-Length': '0'} 2024-04-16T18:27:50.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f2fb422ecf96359-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.582 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.582 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.583 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.583 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.583 DEBUG:boto:Method: GET 2024-04-16T18:27:50.583 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.583 DEBUG:boto:Data: 2024-04-16T18:27:50.583 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.583 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.583 DEBUG:boto:Params: {} 2024-04-16T18:27:50.583 DEBUG:boto:Token: None 2024-04-16T18:27:50.583 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:27:50.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc= 2024-04-16T18:27:50.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v2zH9VSCbVKkuYgUUsvWFvmeYcc=', 'Content-Length': '0'} 2024-04-16T18:27:50.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0a8c8284f39058b-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.587 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.587 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.587 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.587 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.587 DEBUG:boto:Method: GET 2024-04-16T18:27:50.587 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:27:50.587 DEBUG:boto:Data: 2024-04-16T18:27:50.587 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.587 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.588 DEBUG:boto:Params: {} 2024-04-16T18:27:50.588 DEBUG:boto:Token: None 2024-04-16T18:27:50.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f73da04400e693f-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.592 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.592 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.593 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.593 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.593 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.593 DEBUG:boto:Method: GET 2024-04-16T18:27:50.593 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:27:50.593 DEBUG:boto:Data: 2024-04-16T18:27:50.593 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.593 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.593 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.593 DEBUG:boto:Params: {} 2024-04-16T18:27:50.593 DEBUG:boto:Token: None 2024-04-16T18:27:50.594 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.594 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000854090418e7d8ba8-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.642 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.643 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.643 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.643 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.643 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:27:50.643 DEBUG:boto:Data: 2024-04-16T18:27:50.643 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.643 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.643 DEBUG:boto:Params: {} 2024-04-16T18:27:50.643 DEBUG:boto:Token: None 2024-04-16T18:27:50.644 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62 2024-04-16T18:27:50.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+4a4pdpiH/K62wJPkOFsYxaHSyA= 2024-04-16T18:27:50.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+4a4pdpiH/K62wJPkOFsYxaHSyA=', 'Content-Length': '0'} 2024-04-16T18:27:50.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e608115a9f3a11bf-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.647 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.647 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.647 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.647 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:27:50.647 DEBUG:boto:Data: 2024-04-16T18:27:50.647 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.648 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.648 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.648 DEBUG:boto:Params: {} 2024-04-16T18:27:50.648 DEBUG:boto:Token: None 2024-04-16T18:27:50.648 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62 2024-04-16T18:27:50.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+4a4pdpiH/K62wJPkOFsYxaHSyA= 2024-04-16T18:27:50.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+4a4pdpiH/K62wJPkOFsYxaHSyA=', 'Content-Length': '0'} 2024-04-16T18:27:50.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000324baa6ac214bad1-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.651 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:27:50.651 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.652 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.652 DEBUG:boto:Method: GET 2024-04-16T18:27:50.652 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:27:50.652 DEBUG:boto:Data: 2024-04-16T18:27:50.652 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.652 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.652 DEBUG:boto:Params: {} 2024-04-16T18:27:50.652 DEBUG:boto:Token: None 2024-04-16T18:27:50.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62 2024-04-16T18:27:50.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VlOUG9WvzKpdByqMf1Y+brPVdPg= 2024-04-16T18:27:50.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VlOUG9WvzKpdByqMf1Y+brPVdPg=', 'Content-Length': '0'} 2024-04-16T18:27:50.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049913a9294f537e7-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.698 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:27:50.698 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:27:50.699 DEBUG:boto:Method: GET 2024-04-16T18:27:50.699 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:27:50.699 DEBUG:boto:Data: 2024-04-16T18:27:50.699 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.699 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.699 DEBUG:boto:Params: {} 2024-04-16T18:27:50.699 DEBUG:boto:Token: None 2024-04-16T18:27:50.699 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj62 2024-04-16T18:27:50.699 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VlOUG9WvzKpdByqMf1Y+brPVdPg= 2024-04-16T18:27:50.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VlOUG9WvzKpdByqMf1Y+brPVdPg=', 'Content-Length': '0'} 2024-04-16T18:27:50.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078d3f687e46e9f7e-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.703 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.703 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.703 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.703 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.703 DEBUG:boto:Method: GET 2024-04-16T18:27:50.703 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:27:50.703 DEBUG:boto:Data: 2024-04-16T18:27:50.704 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.704 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.704 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.704 DEBUG:boto:Params: {} 2024-04-16T18:27:50.704 DEBUG:boto:Token: None 2024-04-16T18:27:50.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a22296db11345fcd-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.750 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.751 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.751 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.751 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.751 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:27:50.751 DEBUG:boto:Method: GET 2024-04-16T18:27:50.751 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:27:50.751 DEBUG:boto:Data: 2024-04-16T18:27:50.751 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.751 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.751 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.752 DEBUG:boto:Params: {} 2024-04-16T18:27:50.752 DEBUG:boto:Token: None 2024-04-16T18:27:50.752 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.759 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b271ca25cf7d929-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.759 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:27:50.760 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.760 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.760 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.760 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.760 DEBUG:boto:Method: GET 2024-04-16T18:27:50.760 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.760 DEBUG:boto:Data: 2024-04-16T18:27:50.760 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.760 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.760 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.761 DEBUG:boto:Params: {} 2024-04-16T18:27:50.761 DEBUG:boto:Token: None 2024-04-16T18:27:50.761 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.761 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k= 2024-04-16T18:27:50.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k=', 'Content-Length': '0'} 2024-04-16T18:27:50.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005445f622292de8dd-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.806 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.806 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.807 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.807 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.807 DEBUG:boto:Method: GET 2024-04-16T18:27:50.807 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.807 DEBUG:boto:Data: 2024-04-16T18:27:50.807 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.807 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.807 DEBUG:boto:Params: {} 2024-04-16T18:27:50.807 DEBUG:boto:Token: None 2024-04-16T18:27:50.807 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k= 2024-04-16T18:27:50.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k=', 'Content-Length': '0'} 2024-04-16T18:27:50.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e714d7cc7677c5c7-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.811 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.811 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.811 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.811 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.811 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.811 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.812 DEBUG:boto:Data: 2024-04-16T18:27:50.812 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.812 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.812 DEBUG:boto:Params: {} 2024-04-16T18:27:50.812 DEBUG:boto:Token: None 2024-04-16T18:27:50.812 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9/J4ZS7HRnNt7d+tg22GgUdp0/s= 2024-04-16T18:27:50.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9/J4ZS7HRnNt7d+tg22GgUdp0/s=', 'Content-Length': '0'} 2024-04-16T18:27:50.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000deb1e525e7b223e6-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.815 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.815 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.815 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.816 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.816 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.816 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.816 DEBUG:boto:Data: 2024-04-16T18:27:50.816 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.816 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.816 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.816 DEBUG:boto:Params: {} 2024-04-16T18:27:50.816 DEBUG:boto:Token: None 2024-04-16T18:27:50.816 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9/J4ZS7HRnNt7d+tg22GgUdp0/s= 2024-04-16T18:27:50.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9/J4ZS7HRnNt7d+tg22GgUdp0/s=', 'Content-Length': '0'} 2024-04-16T18:27:50.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041b5dc440e49c1a7-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:27:50.820 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.820 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.820 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.820 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.820 DEBUG:boto:Method: GET 2024-04-16T18:27:50.820 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.820 DEBUG:boto:Data: 2024-04-16T18:27:50.820 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.821 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.821 DEBUG:boto:Params: {} 2024-04-16T18:27:50.821 DEBUG:boto:Token: None 2024-04-16T18:27:50.821 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k= 2024-04-16T18:27:50.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k=', 'Content-Length': '0'} 2024-04-16T18:27:50.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f16c73cdc89fad52-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.866 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.866 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.866 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.867 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.867 DEBUG:boto:Method: GET 2024-04-16T18:27:50.867 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.867 DEBUG:boto:Data: 2024-04-16T18:27:50.867 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.867 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.867 DEBUG:boto:Params: {} 2024-04-16T18:27:50.867 DEBUG:boto:Token: None 2024-04-16T18:27:50.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:27:50.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k= 2024-04-16T18:27:50.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uzb0bm3IER968xqtLNfPVPWMl3k=', 'Content-Length': '0'} 2024-04-16T18:27:50.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014a2480881c582c5-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.871 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.871 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.871 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:50.871 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:50.871 DEBUG:boto:Method: GET 2024-04-16T18:27:50.871 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:27:50.871 DEBUG:boto:Data: 2024-04-16T18:27:50.872 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.872 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.872 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.872 DEBUG:boto:Params: {} 2024-04-16T18:27:50.872 DEBUG:boto:Token: None 2024-04-16T18:27:50.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df42174fdb0c229e-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.876 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.876 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.877 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.877 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:50.877 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:50.877 DEBUG:boto:Method: GET 2024-04-16T18:27:50.877 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:27:50.877 DEBUG:boto:Data: 2024-04-16T18:27:50.877 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.877 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.877 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.877 DEBUG:boto:Params: {} 2024-04-16T18:27:50.877 DEBUG:boto:Token: None 2024-04-16T18:27:50.877 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.878 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cffa406f2d5c0e3-00661ec326-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.926 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:50.927 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.927 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.927 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.927 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:27:50.927 DEBUG:boto:Data: 2024-04-16T18:27:50.927 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.927 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.927 DEBUG:boto:Params: {} 2024-04-16T18:27:50.927 DEBUG:boto:Token: None 2024-04-16T18:27:50.927 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68 2024-04-16T18:27:50.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LQdcloE5MjespZPclK/RddalDVw= 2024-04-16T18:27:50.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LQdcloE5MjespZPclK/RddalDVw=', 'Content-Length': '0'} 2024-04-16T18:27:50.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac8dd65df5b4cf5b-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.931 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.931 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.931 DEBUG:boto:Method: HEAD 2024-04-16T18:27:50.931 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:27:50.931 DEBUG:boto:Data: 2024-04-16T18:27:50.931 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.931 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.931 DEBUG:boto:Params: {} 2024-04-16T18:27:50.932 DEBUG:boto:Token: None 2024-04-16T18:27:50.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68 2024-04-16T18:27:50.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LQdcloE5MjespZPclK/RddalDVw= 2024-04-16T18:27:50.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LQdcloE5MjespZPclK/RddalDVw=', 'Content-Length': '0'} 2024-04-16T18:27:50.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073b5aea56ae4eb45-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:27:50.935 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.935 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.935 DEBUG:boto:Method: GET 2024-04-16T18:27:50.935 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:27:50.935 DEBUG:boto:Data: 2024-04-16T18:27:50.935 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.935 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.935 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.936 DEBUG:boto:Params: {} 2024-04-16T18:27:50.936 DEBUG:boto:Token: None 2024-04-16T18:27:50.936 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68 2024-04-16T18:27:50.936 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3uep5VFRodF6fYx5ZZOgxLr/l/w= 2024-04-16T18:27:50.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3uep5VFRodF6fYx5ZZOgxLr/l/w=', 'Content-Length': '0'} 2024-04-16T18:27:50.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000844ee71e1f157bed-00661ec326-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.982 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:27:50.982 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:27:50.983 DEBUG:boto:Method: GET 2024-04-16T18:27:50.983 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:27:50.983 DEBUG:boto:Data: 2024-04-16T18:27:50.983 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:50.983 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.983 DEBUG:boto:Params: {} 2024-04-16T18:27:50.983 DEBUG:boto:Token: None 2024-04-16T18:27:50.983 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/obj68 2024-04-16T18:27:50.983 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3uep5VFRodF6fYx5ZZOgxLr/l/w= 2024-04-16T18:27:50.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3uep5VFRodF6fYx5ZZOgxLr/l/w=', 'Content-Length': '0'} 2024-04-16T18:27:50.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eca9cd0b4d05ab11-00661ec326-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:50.986 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:50.987 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:50.987 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:50.987 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:50.987 DEBUG:boto:Method: GET 2024-04-16T18:27:50.987 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:27:50.987 DEBUG:boto:Data: 2024-04-16T18:27:50.987 DEBUG:boto:Headers: {} 2024-04-16T18:27:50.987 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:50.987 DEBUG:boto:Port: 8000 2024-04-16T18:27:50.987 DEBUG:boto:Params: {} 2024-04-16T18:27:50.988 DEBUG:boto:Token: None 2024-04-16T18:27:50.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:50 GMT /lybtug-52/ 2024-04-16T18:27:50.988 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU= 2024-04-16T18:27:50.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r9tLD0PXNEX6oea2PdDz/SMabIU=', 'Content-Length': '0'} 2024-04-16T18:27:50.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbfbb82f26065a3e-00661ec326-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.034 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.035 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.035 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.035 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:51.035 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:27:51.035 DEBUG:boto:Method: GET 2024-04-16T18:27:51.035 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:27:51.035 DEBUG:boto:Data: 2024-04-16T18:27:51.035 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.035 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.035 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.035 DEBUG:boto:Params: {} 2024-04-16T18:27:51.036 DEBUG:boto:Token: None 2024-04-16T18:27:51.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.042 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032230969ebf2ad5d-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.043 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:27:51.043 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.043 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.043 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.043 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.043 DEBUG:boto:Method: GET 2024-04-16T18:27:51.044 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.044 DEBUG:boto:Data: 2024-04-16T18:27:51.044 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.044 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.044 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.044 DEBUG:boto:Params: {} 2024-04-16T18:27:51.044 DEBUG:boto:Token: None 2024-04-16T18:27:51.044 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8= 2024-04-16T18:27:51.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8=', 'Content-Length': '0'} 2024-04-16T18:27:51.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070fc9d856da2953a-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.090 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.090 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.090 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.091 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.091 DEBUG:boto:Method: GET 2024-04-16T18:27:51.091 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.091 DEBUG:boto:Data: 2024-04-16T18:27:51.091 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.091 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.091 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.091 DEBUG:boto:Params: {} 2024-04-16T18:27:51.091 DEBUG:boto:Token: None 2024-04-16T18:27:51.091 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8= 2024-04-16T18:27:51.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8=', 'Content-Length': '0'} 2024-04-16T18:27:51.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014cc437e09955367-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.095 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.095 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.095 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.095 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.095 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.095 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.096 DEBUG:boto:Data: 2024-04-16T18:27:51.096 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.096 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.096 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.096 DEBUG:boto:Params: {} 2024-04-16T18:27:51.096 DEBUG:boto:Token: None 2024-04-16T18:27:51.096 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0emY+zWt1zq8EbFliOfhfINBNFc= 2024-04-16T18:27:51.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0emY+zWt1zq8EbFliOfhfINBNFc=', 'Content-Length': '0'} 2024-04-16T18:27:51.099 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dbf6f08af28242d-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.099 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.099 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.099 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.100 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.100 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.100 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.100 DEBUG:boto:Data: 2024-04-16T18:27:51.100 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.100 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.100 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.100 DEBUG:boto:Params: {} 2024-04-16T18:27:51.100 DEBUG:boto:Token: None 2024-04-16T18:27:51.100 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0emY+zWt1zq8EbFliOfhfINBNFc= 2024-04-16T18:27:51.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0emY+zWt1zq8EbFliOfhfINBNFc=', 'Content-Length': '0'} 2024-04-16T18:27:51.103 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000693f42d1cb7c3941-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.103 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:27:51.104 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.104 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.104 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.104 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.104 DEBUG:boto:Method: GET 2024-04-16T18:27:51.104 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.104 DEBUG:boto:Data: 2024-04-16T18:27:51.104 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.104 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.104 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.104 DEBUG:boto:Params: {} 2024-04-16T18:27:51.104 DEBUG:boto:Token: None 2024-04-16T18:27:51.105 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.105 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8= 2024-04-16T18:27:51.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8=', 'Content-Length': '0'} 2024-04-16T18:27:51.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003947dcffd4dda00d-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.150 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.150 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.151 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.151 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.151 DEBUG:boto:Method: GET 2024-04-16T18:27:51.151 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.151 DEBUG:boto:Data: 2024-04-16T18:27:51.151 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.151 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.151 DEBUG:boto:Params: {} 2024-04-16T18:27:51.151 DEBUG:boto:Token: None 2024-04-16T18:27:51.151 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:27:51.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8= 2024-04-16T18:27:51.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ws2fTOQwkbln7X2pZtm+g+A7dr8=', 'Content-Length': '0'} 2024-04-16T18:27:51.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000767d90dc816e7705-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.155 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.155 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.155 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.155 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.155 DEBUG:boto:Method: GET 2024-04-16T18:27:51.155 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:27:51.155 DEBUG:boto:Data: 2024-04-16T18:27:51.155 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.156 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.156 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.156 DEBUG:boto:Params: {} 2024-04-16T18:27:51.156 DEBUG:boto:Token: None 2024-04-16T18:27:51.156 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9b2858b47f46c9d-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.160 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.160 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.160 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.161 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.161 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.161 DEBUG:boto:Method: GET 2024-04-16T18:27:51.161 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:27:51.161 DEBUG:boto:Data: 2024-04-16T18:27:51.161 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.161 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.161 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.161 DEBUG:boto:Params: {} 2024-04-16T18:27:51.161 DEBUG:boto:Token: None 2024-04-16T18:27:51.161 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.162 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cb133b7b8b0dcbd-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.210 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.234 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.234 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.234 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.234 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:27:51.234 DEBUG:boto:Data: 2024-04-16T18:27:51.234 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.234 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.235 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.235 DEBUG:boto:Params: {} 2024-04-16T18:27:51.235 DEBUG:boto:Token: None 2024-04-16T18:27:51.235 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69 2024-04-16T18:27:51.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmoYttsPHIxRvEooisVXRjrX75Y= 2024-04-16T18:27:51.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmoYttsPHIxRvEooisVXRjrX75Y=', 'Content-Length': '0'} 2024-04-16T18:27:51.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ceb778bc23d47ede-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.238 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.239 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.239 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.239 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:27:51.239 DEBUG:boto:Data: 2024-04-16T18:27:51.239 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.239 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.239 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.239 DEBUG:boto:Params: {} 2024-04-16T18:27:51.239 DEBUG:boto:Token: None 2024-04-16T18:27:51.239 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69 2024-04-16T18:27:51.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmoYttsPHIxRvEooisVXRjrX75Y= 2024-04-16T18:27:51.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmoYttsPHIxRvEooisVXRjrX75Y=', 'Content-Length': '0'} 2024-04-16T18:27:51.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f00f77b3cebd7e68-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:27:51.243 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.243 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.243 DEBUG:boto:Method: GET 2024-04-16T18:27:51.243 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:27:51.243 DEBUG:boto:Data: 2024-04-16T18:27:51.243 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.243 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.244 DEBUG:boto:Params: {} 2024-04-16T18:27:51.244 DEBUG:boto:Token: None 2024-04-16T18:27:51.244 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69 2024-04-16T18:27:51.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WQYE/7qGc3QLtLSe0gC2JL29FBk= 2024-04-16T18:27:51.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WQYE/7qGc3QLtLSe0gC2JL29FBk=', 'Content-Length': '0'} 2024-04-16T18:27:51.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6924d0523b69ae7-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.290 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:27:51.290 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:27:51.291 DEBUG:boto:Method: GET 2024-04-16T18:27:51.291 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:27:51.291 DEBUG:boto:Data: 2024-04-16T18:27:51.291 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.291 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.291 DEBUG:boto:Params: {} 2024-04-16T18:27:51.291 DEBUG:boto:Token: None 2024-04-16T18:27:51.291 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj69 2024-04-16T18:27:51.291 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WQYE/7qGc3QLtLSe0gC2JL29FBk= 2024-04-16T18:27:51.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WQYE/7qGc3QLtLSe0gC2JL29FBk=', 'Content-Length': '0'} 2024-04-16T18:27:51.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9fd3da5b9c23de4-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.295 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.295 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.295 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.295 DEBUG:boto:Method: GET 2024-04-16T18:27:51.295 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:27:51.296 DEBUG:boto:Data: 2024-04-16T18:27:51.296 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.296 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.296 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.296 DEBUG:boto:Params: {} 2024-04-16T18:27:51.296 DEBUG:boto:Token: None 2024-04-16T18:27:51.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dffc53a9ddc3a361-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.300 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.300 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.301 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.301 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.301 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:27:51.301 DEBUG:boto:Method: GET 2024-04-16T18:27:51.301 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:27:51.301 DEBUG:boto:Data: 2024-04-16T18:27:51.301 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.301 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.301 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.301 DEBUG:boto:Params: {} 2024-04-16T18:27:51.301 DEBUG:boto:Token: None 2024-04-16T18:27:51.302 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.309 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051c412bff84705f7-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.350 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.351 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:27:51.351 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.351 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.351 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.351 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.351 DEBUG:boto:Method: GET 2024-04-16T18:27:51.351 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.351 DEBUG:boto:Data: 2024-04-16T18:27:51.351 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.351 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.351 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.351 DEBUG:boto:Params: {} 2024-04-16T18:27:51.352 DEBUG:boto:Token: None 2024-04-16T18:27:51.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4= 2024-04-16T18:27:51.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4=', 'Content-Length': '0'} 2024-04-16T18:27:51.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000747f575fd687217d-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.398 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.398 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.398 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.399 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.399 DEBUG:boto:Method: GET 2024-04-16T18:27:51.399 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.399 DEBUG:boto:Data: 2024-04-16T18:27:51.399 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.399 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.399 DEBUG:boto:Params: {} 2024-04-16T18:27:51.399 DEBUG:boto:Token: None 2024-04-16T18:27:51.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.399 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4= 2024-04-16T18:27:51.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4=', 'Content-Length': '0'} 2024-04-16T18:27:51.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000746a805e267af4d3-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.404 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.404 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.404 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.404 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.404 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.404 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.404 DEBUG:boto:Data: 2024-04-16T18:27:51.404 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.404 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.404 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.404 DEBUG:boto:Params: {} 2024-04-16T18:27:51.405 DEBUG:boto:Token: None 2024-04-16T18:27:51.405 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nbLKZddzuCrHadHt/QSNkutoBuM= 2024-04-16T18:27:51.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nbLKZddzuCrHadHt/QSNkutoBuM=', 'Content-Length': '0'} 2024-04-16T18:27:51.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000369d79065e740ab2-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.408 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.408 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.409 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.409 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.409 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.409 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.409 DEBUG:boto:Data: 2024-04-16T18:27:51.409 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.409 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.409 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.409 DEBUG:boto:Params: {} 2024-04-16T18:27:51.409 DEBUG:boto:Token: None 2024-04-16T18:27:51.409 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nbLKZddzuCrHadHt/QSNkutoBuM= 2024-04-16T18:27:51.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nbLKZddzuCrHadHt/QSNkutoBuM=', 'Content-Length': '0'} 2024-04-16T18:27:51.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d96f8a98ee2dcabf-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.413 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:27:51.413 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.413 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.413 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.413 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.413 DEBUG:boto:Method: GET 2024-04-16T18:27:51.413 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.414 DEBUG:boto:Data: 2024-04-16T18:27:51.414 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.414 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.414 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.414 DEBUG:boto:Params: {} 2024-04-16T18:27:51.414 DEBUG:boto:Token: None 2024-04-16T18:27:51.414 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.414 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4= 2024-04-16T18:27:51.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4=', 'Content-Length': '0'} 2024-04-16T18:27:51.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df9964a0a0d8ef62-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.458 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.458 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.459 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.459 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.459 DEBUG:boto:Method: GET 2024-04-16T18:27:51.459 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.459 DEBUG:boto:Data: 2024-04-16T18:27:51.459 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.459 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.459 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.459 DEBUG:boto:Params: {} 2024-04-16T18:27:51.459 DEBUG:boto:Token: None 2024-04-16T18:27:51.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:27:51.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4= 2024-04-16T18:27:51.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JkgShOCZjagyhfdgKMMjzhyxJZ4=', 'Content-Length': '0'} 2024-04-16T18:27:51.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c8212534cfa6587-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.463 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.463 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.463 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.463 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.463 DEBUG:boto:Method: GET 2024-04-16T18:27:51.463 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:27:51.464 DEBUG:boto:Data: 2024-04-16T18:27:51.464 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.464 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.464 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.464 DEBUG:boto:Params: {} 2024-04-16T18:27:51.464 DEBUG:boto:Token: None 2024-04-16T18:27:51.464 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000795160d6b615c7d5-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.468 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.468 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.468 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.469 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.469 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.469 DEBUG:boto:Method: GET 2024-04-16T18:27:51.469 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:27:51.469 DEBUG:boto:Data: 2024-04-16T18:27:51.469 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.469 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.469 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.469 DEBUG:boto:Params: {} 2024-04-16T18:27:51.469 DEBUG:boto:Token: None 2024-04-16T18:27:51.469 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.470 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fc0ee7127c9c600-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.476 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.477 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.477 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.477 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.477 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:27:51.477 DEBUG:boto:Data: 2024-04-16T18:27:51.477 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.477 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.477 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.477 DEBUG:boto:Params: {} 2024-04-16T18:27:51.477 DEBUG:boto:Token: None 2024-04-16T18:27:51.478 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76 2024-04-16T18:27:51.478 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3Zz0+6vb37FWY4hQTJGihPiaIQc= 2024-04-16T18:27:51.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3Zz0+6vb37FWY4hQTJGihPiaIQc=', 'Content-Length': '0'} 2024-04-16T18:27:51.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c8476a5e70399a9-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.481 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.481 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.481 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.481 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:27:51.481 DEBUG:boto:Data: 2024-04-16T18:27:51.481 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.481 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.481 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.481 DEBUG:boto:Params: {} 2024-04-16T18:27:51.481 DEBUG:boto:Token: None 2024-04-16T18:27:51.481 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76 2024-04-16T18:27:51.482 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3Zz0+6vb37FWY4hQTJGihPiaIQc= 2024-04-16T18:27:51.482 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3Zz0+6vb37FWY4hQTJGihPiaIQc=', 'Content-Length': '0'} 2024-04-16T18:27:51.484 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d308a1b06b58575-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.484 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:27:51.485 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.485 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.485 DEBUG:boto:Method: GET 2024-04-16T18:27:51.485 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:27:51.485 DEBUG:boto:Data: 2024-04-16T18:27:51.485 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.485 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.485 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.485 DEBUG:boto:Params: {} 2024-04-16T18:27:51.485 DEBUG:boto:Token: None 2024-04-16T18:27:51.485 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76 2024-04-16T18:27:51.486 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wg22y+GNhQj3ewI/xaL5TmSO9Ys= 2024-04-16T18:27:51.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wg22y+GNhQj3ewI/xaL5TmSO9Ys=', 'Content-Length': '0'} 2024-04-16T18:27:51.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c179746393ade634-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.530 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:27:51.530 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:27:51.531 DEBUG:boto:Method: GET 2024-04-16T18:27:51.531 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:27:51.531 DEBUG:boto:Data: 2024-04-16T18:27:51.531 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.531 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.531 DEBUG:boto:Params: {} 2024-04-16T18:27:51.531 DEBUG:boto:Token: None 2024-04-16T18:27:51.531 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj76 2024-04-16T18:27:51.531 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wg22y+GNhQj3ewI/xaL5TmSO9Ys= 2024-04-16T18:27:51.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wg22y+GNhQj3ewI/xaL5TmSO9Ys=', 'Content-Length': '0'} 2024-04-16T18:27:51.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000264f136309c7ee35-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.535 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.535 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.535 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.535 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.535 DEBUG:boto:Method: GET 2024-04-16T18:27:51.535 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:27:51.535 DEBUG:boto:Data: 2024-04-16T18:27:51.536 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.536 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.536 DEBUG:boto:Params: {} 2024-04-16T18:27:51.536 DEBUG:boto:Token: None 2024-04-16T18:27:51.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.540 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adc4d99af643f73c-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.582 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.583 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.583 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.583 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.583 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:27:51.583 DEBUG:boto:Method: GET 2024-04-16T18:27:51.583 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:27:51.583 DEBUG:boto:Data: 2024-04-16T18:27:51.583 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.583 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.583 DEBUG:boto:Params: {} 2024-04-16T18:27:51.584 DEBUG:boto:Token: None 2024-04-16T18:27:51.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbb30f8422b6bba5-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.591 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.591 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:27:51.591 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.591 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.591 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.591 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.591 DEBUG:boto:Method: GET 2024-04-16T18:27:51.591 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.592 DEBUG:boto:Data: 2024-04-16T18:27:51.592 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.592 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.592 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.592 DEBUG:boto:Params: {} 2024-04-16T18:27:51.592 DEBUG:boto:Token: None 2024-04-16T18:27:51.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc= 2024-04-16T18:27:51.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc=', 'Content-Length': '0'} 2024-04-16T18:27:51.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c2a2f68ba74cde4-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.638 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.638 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.639 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.639 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.639 DEBUG:boto:Method: GET 2024-04-16T18:27:51.639 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.639 DEBUG:boto:Data: 2024-04-16T18:27:51.639 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.639 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.639 DEBUG:boto:Params: {} 2024-04-16T18:27:51.639 DEBUG:boto:Token: None 2024-04-16T18:27:51.639 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc= 2024-04-16T18:27:51.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc=', 'Content-Length': '0'} 2024-04-16T18:27:51.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec711e983b5e6499-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.643 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.643 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.643 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.643 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.643 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.643 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.643 DEBUG:boto:Data: 2024-04-16T18:27:51.643 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.644 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.644 DEBUG:boto:Params: {} 2024-04-16T18:27:51.644 DEBUG:boto:Token: None 2024-04-16T18:27:51.644 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gkveOMtxPRyKbo86ydXpmQFV1R8= 2024-04-16T18:27:51.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gkveOMtxPRyKbo86ydXpmQFV1R8=', 'Content-Length': '0'} 2024-04-16T18:27:51.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b1b78cfe4c3b994-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.647 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.647 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.647 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.647 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.647 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.648 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.648 DEBUG:boto:Data: 2024-04-16T18:27:51.648 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.648 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.648 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.648 DEBUG:boto:Params: {} 2024-04-16T18:27:51.648 DEBUG:boto:Token: None 2024-04-16T18:27:51.648 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gkveOMtxPRyKbo86ydXpmQFV1R8= 2024-04-16T18:27:51.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gkveOMtxPRyKbo86ydXpmQFV1R8=', 'Content-Length': '0'} 2024-04-16T18:27:51.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007b3b65630410068-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.652 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:27:51.652 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.652 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.652 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.652 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.652 DEBUG:boto:Method: GET 2024-04-16T18:27:51.652 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.652 DEBUG:boto:Data: 2024-04-16T18:27:51.652 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.652 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.652 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.652 DEBUG:boto:Params: {} 2024-04-16T18:27:51.653 DEBUG:boto:Token: None 2024-04-16T18:27:51.653 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.653 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc= 2024-04-16T18:27:51.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc=', 'Content-Length': '0'} 2024-04-16T18:27:51.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007864a7764ecbcf7d-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.698 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.698 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.698 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.699 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.699 DEBUG:boto:Method: GET 2024-04-16T18:27:51.699 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.699 DEBUG:boto:Data: 2024-04-16T18:27:51.699 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.699 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.699 DEBUG:boto:Params: {} 2024-04-16T18:27:51.699 DEBUG:boto:Token: None 2024-04-16T18:27:51.699 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:27:51.699 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc= 2024-04-16T18:27:51.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WyOR4rpTQrGsUD+ENyCLnhMm7wc=', 'Content-Length': '0'} 2024-04-16T18:27:51.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095a9c1a5a12c0f65-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.703 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.703 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.703 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.703 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.703 DEBUG:boto:Method: GET 2024-04-16T18:27:51.703 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:27:51.703 DEBUG:boto:Data: 2024-04-16T18:27:51.704 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.704 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.704 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.704 DEBUG:boto:Params: {} 2024-04-16T18:27:51.704 DEBUG:boto:Token: None 2024-04-16T18:27:51.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d403078fd5c24995-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.708 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.708 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.708 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.709 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.709 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.709 DEBUG:boto:Method: GET 2024-04-16T18:27:51.709 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:27:51.709 DEBUG:boto:Data: 2024-04-16T18:27:51.709 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.709 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.709 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.709 DEBUG:boto:Params: {} 2024-04-16T18:27:51.709 DEBUG:boto:Token: None 2024-04-16T18:27:51.709 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.710 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006dc3d24bef3966cd-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.716 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.716 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.716 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.716 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.717 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:27:51.717 DEBUG:boto:Data: 2024-04-16T18:27:51.717 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.717 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.717 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.717 DEBUG:boto:Params: {} 2024-04-16T18:27:51.717 DEBUG:boto:Token: None 2024-04-16T18:27:51.717 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85 2024-04-16T18:27:51.717 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B4tkM4K2uJsIFONcIwaVJ4ZQNs4= 2024-04-16T18:27:51.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B4tkM4K2uJsIFONcIwaVJ4ZQNs4=', 'Content-Length': '0'} 2024-04-16T18:27:51.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000adf3f9562122e84-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.720 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.720 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.720 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.721 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:27:51.721 DEBUG:boto:Data: 2024-04-16T18:27:51.721 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.721 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.721 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.721 DEBUG:boto:Params: {} 2024-04-16T18:27:51.721 DEBUG:boto:Token: None 2024-04-16T18:27:51.721 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85 2024-04-16T18:27:51.721 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B4tkM4K2uJsIFONcIwaVJ4ZQNs4= 2024-04-16T18:27:51.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B4tkM4K2uJsIFONcIwaVJ4ZQNs4=', 'Content-Length': '0'} 2024-04-16T18:27:51.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002421e07a47a9f99f-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.724 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:27:51.725 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.725 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.725 DEBUG:boto:Method: GET 2024-04-16T18:27:51.725 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:27:51.725 DEBUG:boto:Data: 2024-04-16T18:27:51.725 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.725 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.725 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.725 DEBUG:boto:Params: {} 2024-04-16T18:27:51.725 DEBUG:boto:Token: None 2024-04-16T18:27:51.725 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85 2024-04-16T18:27:51.725 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UGroK80PcflYyvOD8nKFxuUeG0Q= 2024-04-16T18:27:51.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UGroK80PcflYyvOD8nKFxuUeG0Q=', 'Content-Length': '0'} 2024-04-16T18:27:51.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd5bedeb8755bc1e-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.774 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:27:51.774 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:27:51.775 DEBUG:boto:Method: GET 2024-04-16T18:27:51.775 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:27:51.775 DEBUG:boto:Data: 2024-04-16T18:27:51.775 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.775 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.775 DEBUG:boto:Params: {} 2024-04-16T18:27:51.775 DEBUG:boto:Token: None 2024-04-16T18:27:51.775 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj85 2024-04-16T18:27:51.775 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UGroK80PcflYyvOD8nKFxuUeG0Q= 2024-04-16T18:27:51.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UGroK80PcflYyvOD8nKFxuUeG0Q=', 'Content-Length': '0'} 2024-04-16T18:27:51.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b48ea237ae405295-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.779 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.779 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.779 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.779 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.779 DEBUG:boto:Method: GET 2024-04-16T18:27:51.779 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:27:51.779 DEBUG:boto:Data: 2024-04-16T18:27:51.780 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.780 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.780 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.780 DEBUG:boto:Params: {} 2024-04-16T18:27:51.780 DEBUG:boto:Token: None 2024-04-16T18:27:51.780 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.784 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4b8b619c296b814-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.784 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.784 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.784 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.785 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.785 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:27:51.785 DEBUG:boto:Method: GET 2024-04-16T18:27:51.785 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:27:51.785 DEBUG:boto:Data: 2024-04-16T18:27:51.785 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.785 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.785 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.785 DEBUG:boto:Params: {} 2024-04-16T18:27:51.785 DEBUG:boto:Token: None 2024-04-16T18:27:51.785 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.786 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061f8b0221405fc90-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.792 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.792 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:27:51.792 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.792 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.792 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.793 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.793 DEBUG:boto:Method: GET 2024-04-16T18:27:51.793 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.793 DEBUG:boto:Data: 2024-04-16T18:27:51.793 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.793 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.793 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.793 DEBUG:boto:Params: {} 2024-04-16T18:27:51.793 DEBUG:boto:Token: None 2024-04-16T18:27:51.793 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.793 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ= 2024-04-16T18:27:51.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ=', 'Content-Length': '0'} 2024-04-16T18:27:51.797 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c0e9852f42bd59a-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.838 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.838 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.839 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.839 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.839 DEBUG:boto:Method: GET 2024-04-16T18:27:51.839 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.839 DEBUG:boto:Data: 2024-04-16T18:27:51.839 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.839 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.839 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.839 DEBUG:boto:Params: {} 2024-04-16T18:27:51.839 DEBUG:boto:Token: None 2024-04-16T18:27:51.840 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.840 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ= 2024-04-16T18:27:51.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ=', 'Content-Length': '0'} 2024-04-16T18:27:51.842 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f44828c209badac8-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.843 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.843 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.843 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.843 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.843 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.843 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.843 DEBUG:boto:Data: 2024-04-16T18:27:51.843 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.844 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.844 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.844 DEBUG:boto:Params: {} 2024-04-16T18:27:51.844 DEBUG:boto:Token: None 2024-04-16T18:27:51.844 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HYcL8VTNReJvGVgZs+TXxPOR9EI= 2024-04-16T18:27:51.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HYcL8VTNReJvGVgZs+TXxPOR9EI=', 'Content-Length': '0'} 2024-04-16T18:27:51.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000488e91856daeb21c-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.847 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.847 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.848 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.848 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.848 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.848 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.848 DEBUG:boto:Data: 2024-04-16T18:27:51.848 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.848 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.848 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.848 DEBUG:boto:Params: {} 2024-04-16T18:27:51.848 DEBUG:boto:Token: None 2024-04-16T18:27:51.848 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HYcL8VTNReJvGVgZs+TXxPOR9EI= 2024-04-16T18:27:51.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HYcL8VTNReJvGVgZs+TXxPOR9EI=', 'Content-Length': '0'} 2024-04-16T18:27:51.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000342e4d9247151cab-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.852 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:27:51.852 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.852 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.852 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.852 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.852 DEBUG:boto:Method: GET 2024-04-16T18:27:51.852 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.852 DEBUG:boto:Data: 2024-04-16T18:27:51.852 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.852 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.852 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.853 DEBUG:boto:Params: {} 2024-04-16T18:27:51.853 DEBUG:boto:Token: None 2024-04-16T18:27:51.853 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.853 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ= 2024-04-16T18:27:51.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ=', 'Content-Length': '0'} 2024-04-16T18:27:51.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017b8f9880329134b-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.898 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.898 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.899 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.899 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.899 DEBUG:boto:Method: GET 2024-04-16T18:27:51.899 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.899 DEBUG:boto:Data: 2024-04-16T18:27:51.899 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.899 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.899 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.899 DEBUG:boto:Params: {} 2024-04-16T18:27:51.899 DEBUG:boto:Token: None 2024-04-16T18:27:51.899 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:27:51.899 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ= 2024-04-16T18:27:51.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:E4fO0Fgx0d4Tq1CJTfrgq70zhaQ=', 'Content-Length': '0'} 2024-04-16T18:27:51.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d3b2fc89c90bf45-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.903 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.903 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.903 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:51.903 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:51.903 DEBUG:boto:Method: GET 2024-04-16T18:27:51.903 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:27:51.903 DEBUG:boto:Data: 2024-04-16T18:27:51.903 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.903 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.904 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.904 DEBUG:boto:Params: {} 2024-04-16T18:27:51.904 DEBUG:boto:Token: None 2024-04-16T18:27:51.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.908 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b8f8116668b3884-00661ec327-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.908 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.908 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:51.908 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:51.908 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:51.909 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:51.909 DEBUG:boto:Method: GET 2024-04-16T18:27:51.909 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:27:51.909 DEBUG:boto:Data: 2024-04-16T18:27:51.909 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.909 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.909 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.909 DEBUG:boto:Params: {} 2024-04-16T18:27:51.909 DEBUG:boto:Token: None 2024-04-16T18:27:51.909 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/ 2024-04-16T18:27:51.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0= 2024-04-16T18:27:51.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9XFBJtMnq4cAgQXk/TNOec/uQr0=', 'Content-Length': '0'} 2024-04-16T18:27:51.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f3f78ca432a54cd-00661ec327-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.958 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:51.959 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.959 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.959 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.959 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:27:51.959 DEBUG:boto:Data: 2024-04-16T18:27:51.959 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.959 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.959 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.959 DEBUG:boto:Params: {} 2024-04-16T18:27:51.960 DEBUG:boto:Token: None 2024-04-16T18:27:51.960 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87 2024-04-16T18:27:51.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:caDP61ZAnXTrNqNltkfF8ydT+nE= 2024-04-16T18:27:51.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:caDP61ZAnXTrNqNltkfF8ydT+nE=', 'Content-Length': '0'} 2024-04-16T18:27:51.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ddd429abb0cad37-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.963 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.963 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.963 DEBUG:boto:Method: HEAD 2024-04-16T18:27:51.963 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:27:51.964 DEBUG:boto:Data: 2024-04-16T18:27:51.964 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:51.964 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.964 DEBUG:boto:Params: {} 2024-04-16T18:27:51.964 DEBUG:boto:Token: None 2024-04-16T18:27:51.964 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87 2024-04-16T18:27:51.964 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:caDP61ZAnXTrNqNltkfF8ydT+nE= 2024-04-16T18:27:51.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:caDP61ZAnXTrNqNltkfF8ydT+nE=', 'Content-Length': '0'} 2024-04-16T18:27:51.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb00b8d38ccd085a-00661ec327-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:51.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:27:51.968 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:51.968 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:51.968 DEBUG:boto:Method: GET 2024-04-16T18:27:51.968 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:27:51.968 DEBUG:boto:Data: 2024-04-16T18:27:51.968 DEBUG:boto:Headers: {} 2024-04-16T18:27:51.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:51.968 DEBUG:boto:Port: 8000 2024-04-16T18:27:51.968 DEBUG:boto:Params: {} 2024-04-16T18:27:51.968 DEBUG:boto:Token: None 2024-04-16T18:27:51.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:51 GMT /lybtug-52/obj87 2024-04-16T18:27:51.969 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nanNfDPSQvFt5AKtae6LquKl8Ds= 2024-04-16T18:27:51.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nanNfDPSQvFt5AKtae6LquKl8Ds=', 'Content-Length': '0'} 2024-04-16T18:27:51.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cccca1f7ea0c1c3f-00661ec327-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.014 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:27:52.014 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:27:52.014 DEBUG:boto:Method: GET 2024-04-16T18:27:52.015 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:27:52.015 DEBUG:boto:Data: 2024-04-16T18:27:52.015 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.015 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.015 DEBUG:boto:Params: {} 2024-04-16T18:27:52.015 DEBUG:boto:Token: None 2024-04-16T18:27:52.015 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj87 2024-04-16T18:27:52.015 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:twq7kVp2wQxIylev0ke3uXq18HI= 2024-04-16T18:27:52.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:twq7kVp2wQxIylev0ke3uXq18HI=', 'Content-Length': '0'} 2024-04-16T18:27:52.018 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002751ced5ba7db5d7-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.019 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.019 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.019 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:52.019 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:52.019 DEBUG:boto:Method: GET 2024-04-16T18:27:52.019 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:27:52.019 DEBUG:boto:Data: 2024-04-16T18:27:52.019 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.019 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.020 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.020 DEBUG:boto:Params: {} 2024-04-16T18:27:52.020 DEBUG:boto:Token: None 2024-04-16T18:27:52.020 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.020 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b04d53ba083523cb-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.066 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.067 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.067 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.067 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:52.067 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:27:52.067 DEBUG:boto:Method: GET 2024-04-16T18:27:52.067 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:27:52.067 DEBUG:boto:Data: 2024-04-16T18:27:52.067 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.067 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.067 DEBUG:boto:Params: {} 2024-04-16T18:27:52.068 DEBUG:boto:Token: None 2024-04-16T18:27:52.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.074 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe1811c6c411cf4b-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.074 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.075 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:27:52.075 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.075 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.075 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.075 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.075 DEBUG:boto:Method: GET 2024-04-16T18:27:52.075 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.075 DEBUG:boto:Data: 2024-04-16T18:27:52.075 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.075 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.075 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.075 DEBUG:boto:Params: {} 2024-04-16T18:27:52.076 DEBUG:boto:Token: None 2024-04-16T18:27:52.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ= 2024-04-16T18:27:52.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ=', 'Content-Length': '0'} 2024-04-16T18:27:52.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d48158d71ddd1d9c-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.122 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.122 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.122 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.123 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.123 DEBUG:boto:Method: GET 2024-04-16T18:27:52.123 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.123 DEBUG:boto:Data: 2024-04-16T18:27:52.123 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.123 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.123 DEBUG:boto:Params: {} 2024-04-16T18:27:52.123 DEBUG:boto:Token: None 2024-04-16T18:27:52.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ= 2024-04-16T18:27:52.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ=', 'Content-Length': '0'} 2024-04-16T18:27:52.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096732fc36ebf3ee0-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.127 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.127 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.127 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.127 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.127 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.127 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.127 DEBUG:boto:Data: 2024-04-16T18:27:52.127 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.127 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.128 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.128 DEBUG:boto:Params: {} 2024-04-16T18:27:52.128 DEBUG:boto:Token: None 2024-04-16T18:27:52.128 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lZ8iKZUmfRzTK36nkjXqo32BaGI= 2024-04-16T18:27:52.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lZ8iKZUmfRzTK36nkjXqo32BaGI=', 'Content-Length': '0'} 2024-04-16T18:27:52.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000054ed7b1612a3341-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.131 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.131 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.131 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.131 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.131 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.131 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.132 DEBUG:boto:Data: 2024-04-16T18:27:52.132 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.132 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.132 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.132 DEBUG:boto:Params: {} 2024-04-16T18:27:52.132 DEBUG:boto:Token: None 2024-04-16T18:27:52.132 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lZ8iKZUmfRzTK36nkjXqo32BaGI= 2024-04-16T18:27:52.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lZ8iKZUmfRzTK36nkjXqo32BaGI=', 'Content-Length': '0'} 2024-04-16T18:27:52.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a235d0c7ed6723bd-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.135 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:27:52.136 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.136 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.136 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.136 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.136 DEBUG:boto:Method: GET 2024-04-16T18:27:52.136 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.136 DEBUG:boto:Data: 2024-04-16T18:27:52.136 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.136 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.136 DEBUG:boto:Params: {} 2024-04-16T18:27:52.137 DEBUG:boto:Token: None 2024-04-16T18:27:52.137 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.137 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ= 2024-04-16T18:27:52.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ=', 'Content-Length': '0'} 2024-04-16T18:27:52.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032213c9dffa11663-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.182 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.182 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.183 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.183 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.183 DEBUG:boto:Method: GET 2024-04-16T18:27:52.183 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.183 DEBUG:boto:Data: 2024-04-16T18:27:52.183 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.183 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.183 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.183 DEBUG:boto:Params: {} 2024-04-16T18:27:52.183 DEBUG:boto:Token: None 2024-04-16T18:27:52.183 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:27:52.184 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ= 2024-04-16T18:27:52.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4xby30MIsZUCdAUFcKyO32/tksQ=', 'Content-Length': '0'} 2024-04-16T18:27:52.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009be65704ef1f21e6-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.187 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.187 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.187 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.187 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.187 DEBUG:boto:Method: GET 2024-04-16T18:27:52.187 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:27:52.187 DEBUG:boto:Data: 2024-04-16T18:27:52.187 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.187 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.188 DEBUG:boto:Params: {} 2024-04-16T18:27:52.188 DEBUG:boto:Token: None 2024-04-16T18:27:52.188 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.194 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e3a7dd0cb4b0421-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.194 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.194 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.194 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.194 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.195 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.195 DEBUG:boto:Method: GET 2024-04-16T18:27:52.195 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:27:52.195 DEBUG:boto:Data: 2024-04-16T18:27:52.195 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.195 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.195 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.195 DEBUG:boto:Params: {} 2024-04-16T18:27:52.195 DEBUG:boto:Token: None 2024-04-16T18:27:52.195 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.195 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044b3f689732eed50-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.242 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.243 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.243 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.243 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.243 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:27:52.243 DEBUG:boto:Data: 2024-04-16T18:27:52.243 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.243 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.243 DEBUG:boto:Params: {} 2024-04-16T18:27:52.244 DEBUG:boto:Token: None 2024-04-16T18:27:52.244 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92 2024-04-16T18:27:52.244 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YGF4JrkiQFEtfZXUTHC4d9vadD8= 2024-04-16T18:27:52.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YGF4JrkiQFEtfZXUTHC4d9vadD8=', 'Content-Length': '0'} 2024-04-16T18:27:52.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a721e1bea665b0d-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.247 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.247 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.247 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.247 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:27:52.247 DEBUG:boto:Data: 2024-04-16T18:27:52.247 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.248 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.248 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.248 DEBUG:boto:Params: {} 2024-04-16T18:27:52.248 DEBUG:boto:Token: None 2024-04-16T18:27:52.248 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92 2024-04-16T18:27:52.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YGF4JrkiQFEtfZXUTHC4d9vadD8= 2024-04-16T18:27:52.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YGF4JrkiQFEtfZXUTHC4d9vadD8=', 'Content-Length': '0'} 2024-04-16T18:27:52.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df9d3ac39888be84-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:27:52.251 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.252 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.252 DEBUG:boto:Method: GET 2024-04-16T18:27:52.252 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:27:52.252 DEBUG:boto:Data: 2024-04-16T18:27:52.252 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.252 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.252 DEBUG:boto:Params: {} 2024-04-16T18:27:52.252 DEBUG:boto:Token: None 2024-04-16T18:27:52.253 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92 2024-04-16T18:27:52.253 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l0BevLlDOETpxO4M7G+SYVLFuKw= 2024-04-16T18:27:52.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l0BevLlDOETpxO4M7G+SYVLFuKw=', 'Content-Length': '0'} 2024-04-16T18:27:52.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006384b6162d8b92ae-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.298 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:27:52.298 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:27:52.299 DEBUG:boto:Method: GET 2024-04-16T18:27:52.299 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:27:52.299 DEBUG:boto:Data: 2024-04-16T18:27:52.299 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.299 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.299 DEBUG:boto:Params: {} 2024-04-16T18:27:52.299 DEBUG:boto:Token: None 2024-04-16T18:27:52.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj92 2024-04-16T18:27:52.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l0BevLlDOETpxO4M7G+SYVLFuKw= 2024-04-16T18:27:52.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l0BevLlDOETpxO4M7G+SYVLFuKw=', 'Content-Length': '0'} 2024-04-16T18:27:52.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c074655b3186333-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.302 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.303 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.303 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.303 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.303 DEBUG:boto:Method: GET 2024-04-16T18:27:52.303 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:27:52.303 DEBUG:boto:Data: 2024-04-16T18:27:52.303 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.303 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.303 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.303 DEBUG:boto:Params: {} 2024-04-16T18:27:52.303 DEBUG:boto:Token: None 2024-04-16T18:27:52.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032b9b5f831cd3903-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.350 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.351 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.351 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.351 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.351 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:27:52.351 DEBUG:boto:Method: GET 2024-04-16T18:27:52.351 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:27:52.351 DEBUG:boto:Data: 2024-04-16T18:27:52.351 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.351 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.352 DEBUG:boto:Params: {} 2024-04-16T18:27:52.352 DEBUG:boto:Token: None 2024-04-16T18:27:52.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003563fb64188b90f7-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.358 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.358 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:27:52.358 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.358 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.359 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.359 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.359 DEBUG:boto:Method: GET 2024-04-16T18:27:52.359 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.359 DEBUG:boto:Data: 2024-04-16T18:27:52.359 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.359 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.359 DEBUG:boto:Params: {} 2024-04-16T18:27:52.359 DEBUG:boto:Token: None 2024-04-16T18:27:52.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg= 2024-04-16T18:27:52.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg=', 'Content-Length': '0'} 2024-04-16T18:27:52.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056e2872c88a83814-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.406 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.406 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.406 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.407 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.407 DEBUG:boto:Method: GET 2024-04-16T18:27:52.407 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.407 DEBUG:boto:Data: 2024-04-16T18:27:52.407 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.407 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.407 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.407 DEBUG:boto:Params: {} 2024-04-16T18:27:52.407 DEBUG:boto:Token: None 2024-04-16T18:27:52.407 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg= 2024-04-16T18:27:52.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg=', 'Content-Length': '0'} 2024-04-16T18:27:52.410 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b5c7f495de01785-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.411 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.411 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.411 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.411 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.411 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.411 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.411 DEBUG:boto:Data: 2024-04-16T18:27:52.412 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.412 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.412 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.412 DEBUG:boto:Params: {} 2024-04-16T18:27:52.412 DEBUG:boto:Token: None 2024-04-16T18:27:52.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W2VRsDzmYKtMj21XSEG3+qtb2hU= 2024-04-16T18:27:52.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W2VRsDzmYKtMj21XSEG3+qtb2hU=', 'Content-Length': '0'} 2024-04-16T18:27:52.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030d3a545eac14d9d-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.415 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.415 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.415 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.415 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.416 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.416 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.416 DEBUG:boto:Data: 2024-04-16T18:27:52.416 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.416 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.416 DEBUG:boto:Params: {} 2024-04-16T18:27:52.416 DEBUG:boto:Token: None 2024-04-16T18:27:52.416 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:W2VRsDzmYKtMj21XSEG3+qtb2hU= 2024-04-16T18:27:52.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:W2VRsDzmYKtMj21XSEG3+qtb2hU=', 'Content-Length': '0'} 2024-04-16T18:27:52.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b91c34baf125ba1e-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:27:52.420 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.420 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.420 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.420 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.420 DEBUG:boto:Method: GET 2024-04-16T18:27:52.420 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.420 DEBUG:boto:Data: 2024-04-16T18:27:52.420 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.421 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.421 DEBUG:boto:Params: {} 2024-04-16T18:27:52.421 DEBUG:boto:Token: None 2024-04-16T18:27:52.421 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.421 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg= 2024-04-16T18:27:52.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg=', 'Content-Length': '0'} 2024-04-16T18:27:52.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000948b8f41daa892fd-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.466 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.466 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.467 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.467 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.467 DEBUG:boto:Method: GET 2024-04-16T18:27:52.467 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.467 DEBUG:boto:Data: 2024-04-16T18:27:52.467 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.467 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.467 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.467 DEBUG:boto:Params: {} 2024-04-16T18:27:52.468 DEBUG:boto:Token: None 2024-04-16T18:27:52.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:27:52.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg= 2024-04-16T18:27:52.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gRf2R0ZEjdKrib8Olk2m8g/tXXg=', 'Content-Length': '0'} 2024-04-16T18:27:52.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5afd1eafd41bd13-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.471 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.472 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.472 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.472 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.472 DEBUG:boto:Method: GET 2024-04-16T18:27:52.472 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:27:52.472 DEBUG:boto:Data: 2024-04-16T18:27:52.472 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.472 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.472 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.473 DEBUG:boto:Params: {} 2024-04-16T18:27:52.473 DEBUG:boto:Token: None 2024-04-16T18:27:52.473 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.473 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c1568acf2c21da5-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.477 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.477 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.477 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.477 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.477 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.477 DEBUG:boto:Method: GET 2024-04-16T18:27:52.478 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:27:52.478 DEBUG:boto:Data: 2024-04-16T18:27:52.478 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.478 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.478 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.478 DEBUG:boto:Params: {} 2024-04-16T18:27:52.478 DEBUG:boto:Token: None 2024-04-16T18:27:52.478 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.479 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f5b42ff23aa2c1c-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.526 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.527 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.527 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.527 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.527 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:27:52.527 DEBUG:boto:Data: 2024-04-16T18:27:52.527 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.527 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.527 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.527 DEBUG:boto:Params: {} 2024-04-16T18:27:52.527 DEBUG:boto:Token: None 2024-04-16T18:27:52.528 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93 2024-04-16T18:27:52.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+PaEWiGVonFYBrwVYTWFQPhxni4= 2024-04-16T18:27:52.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+PaEWiGVonFYBrwVYTWFQPhxni4=', 'Content-Length': '0'} 2024-04-16T18:27:52.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c2da999a2fd437b-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.531 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.531 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.531 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.531 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:27:52.531 DEBUG:boto:Data: 2024-04-16T18:27:52.531 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.531 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.531 DEBUG:boto:Params: {} 2024-04-16T18:27:52.532 DEBUG:boto:Token: None 2024-04-16T18:27:52.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93 2024-04-16T18:27:52.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+PaEWiGVonFYBrwVYTWFQPhxni4= 2024-04-16T18:27:52.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+PaEWiGVonFYBrwVYTWFQPhxni4=', 'Content-Length': '0'} 2024-04-16T18:27:52.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050ad54e8048e9c69-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.535 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:27:52.535 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.535 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.535 DEBUG:boto:Method: GET 2024-04-16T18:27:52.536 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:27:52.536 DEBUG:boto:Data: 2024-04-16T18:27:52.536 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.536 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.536 DEBUG:boto:Params: {} 2024-04-16T18:27:52.536 DEBUG:boto:Token: None 2024-04-16T18:27:52.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93 2024-04-16T18:27:52.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OjnIFiNBDJ8X9rE1wEkgfcmPlmg= 2024-04-16T18:27:52.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OjnIFiNBDJ8X9rE1wEkgfcmPlmg=', 'Content-Length': '0'} 2024-04-16T18:27:52.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000635ddb9ac0833853-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.582 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:27:52.583 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:27:52.583 DEBUG:boto:Method: GET 2024-04-16T18:27:52.583 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:27:52.583 DEBUG:boto:Data: 2024-04-16T18:27:52.583 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.583 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.583 DEBUG:boto:Params: {} 2024-04-16T18:27:52.583 DEBUG:boto:Token: None 2024-04-16T18:27:52.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj93 2024-04-16T18:27:52.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OjnIFiNBDJ8X9rE1wEkgfcmPlmg= 2024-04-16T18:27:52.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OjnIFiNBDJ8X9rE1wEkgfcmPlmg=', 'Content-Length': '0'} 2024-04-16T18:27:52.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c0173de76b810d8-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.587 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.587 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.587 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.587 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.587 DEBUG:boto:Method: GET 2024-04-16T18:27:52.587 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:27:52.587 DEBUG:boto:Data: 2024-04-16T18:27:52.587 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.587 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.588 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.588 DEBUG:boto:Params: {} 2024-04-16T18:27:52.588 DEBUG:boto:Token: None 2024-04-16T18:27:52.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004701c1f06cc17073-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.634 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.635 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.635 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.635 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.635 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:27:52.635 DEBUG:boto:Method: GET 2024-04-16T18:27:52.635 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:27:52.635 DEBUG:boto:Data: 2024-04-16T18:27:52.635 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.635 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.635 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.635 DEBUG:boto:Params: {} 2024-04-16T18:27:52.636 DEBUG:boto:Token: None 2024-04-16T18:27:52.636 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.642 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef5293c7ef81d76e-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.642 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.643 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:27:52.643 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.643 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.643 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.643 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.643 DEBUG:boto:Method: GET 2024-04-16T18:27:52.643 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.643 DEBUG:boto:Data: 2024-04-16T18:27:52.643 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.644 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.644 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.644 DEBUG:boto:Params: {} 2024-04-16T18:27:52.644 DEBUG:boto:Token: None 2024-04-16T18:27:52.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw= 2024-04-16T18:27:52.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw=', 'Content-Length': '0'} 2024-04-16T18:27:52.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000716a83eb4a4d6127-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.690 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.691 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.691 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.691 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.691 DEBUG:boto:Method: GET 2024-04-16T18:27:52.691 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.691 DEBUG:boto:Data: 2024-04-16T18:27:52.691 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.691 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.691 DEBUG:boto:Params: {} 2024-04-16T18:27:52.691 DEBUG:boto:Token: None 2024-04-16T18:27:52.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw= 2024-04-16T18:27:52.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw=', 'Content-Length': '0'} 2024-04-16T18:27:52.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3a3d2d7c243467c-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.696 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.696 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.696 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.696 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.696 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.696 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.696 DEBUG:boto:Data: 2024-04-16T18:27:52.696 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.696 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.696 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.696 DEBUG:boto:Params: {} 2024-04-16T18:27:52.697 DEBUG:boto:Token: None 2024-04-16T18:27:52.697 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.697 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WyVExJBqKDUPlbyZyuFqIRtsM7w= 2024-04-16T18:27:52.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WyVExJBqKDUPlbyZyuFqIRtsM7w=', 'Content-Length': '0'} 2024-04-16T18:27:52.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c546874b1da1d785-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.700 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.700 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.700 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.700 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.700 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.700 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.701 DEBUG:boto:Data: 2024-04-16T18:27:52.701 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.701 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.701 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.701 DEBUG:boto:Params: {} 2024-04-16T18:27:52.701 DEBUG:boto:Token: None 2024-04-16T18:27:52.701 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.701 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WyVExJBqKDUPlbyZyuFqIRtsM7w= 2024-04-16T18:27:52.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WyVExJBqKDUPlbyZyuFqIRtsM7w=', 'Content-Length': '0'} 2024-04-16T18:27:52.704 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042f32d944a9f340c-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.705 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:27:52.705 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.705 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.705 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.705 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.705 DEBUG:boto:Method: GET 2024-04-16T18:27:52.705 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.705 DEBUG:boto:Data: 2024-04-16T18:27:52.705 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.705 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.705 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.706 DEBUG:boto:Params: {} 2024-04-16T18:27:52.706 DEBUG:boto:Token: None 2024-04-16T18:27:52.706 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.706 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw= 2024-04-16T18:27:52.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw=', 'Content-Length': '0'} 2024-04-16T18:27:52.709 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048484b5de37833e9-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.750 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.750 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.750 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.751 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.751 DEBUG:boto:Method: GET 2024-04-16T18:27:52.751 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.751 DEBUG:boto:Data: 2024-04-16T18:27:52.751 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.751 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.751 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.751 DEBUG:boto:Params: {} 2024-04-16T18:27:52.751 DEBUG:boto:Token: None 2024-04-16T18:27:52.751 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:27:52.751 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw= 2024-04-16T18:27:52.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:POMSNnP05YhpKYaT8VuF5pdmICw=', 'Content-Length': '0'} 2024-04-16T18:27:52.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008718a7e2ab0bc516-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.755 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.755 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.755 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.755 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.755 DEBUG:boto:Method: GET 2024-04-16T18:27:52.755 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:27:52.756 DEBUG:boto:Data: 2024-04-16T18:27:52.756 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.756 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.756 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.756 DEBUG:boto:Params: {} 2024-04-16T18:27:52.756 DEBUG:boto:Token: None 2024-04-16T18:27:52.756 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.756 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.760 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006f78d5c30e53963-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.760 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.760 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.760 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.761 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.761 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.761 DEBUG:boto:Method: GET 2024-04-16T18:27:52.761 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:27:52.761 DEBUG:boto:Data: 2024-04-16T18:27:52.761 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.761 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.761 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.761 DEBUG:boto:Params: {} 2024-04-16T18:27:52.761 DEBUG:boto:Token: None 2024-04-16T18:27:52.761 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.767 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000129d3072513aed57-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.810 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.811 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.811 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.811 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.811 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:27:52.811 DEBUG:boto:Data: 2024-04-16T18:27:52.811 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.811 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.811 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.811 DEBUG:boto:Params: {} 2024-04-16T18:27:52.812 DEBUG:boto:Token: None 2024-04-16T18:27:52.812 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95 2024-04-16T18:27:52.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UWs781aAZIPFCkIc3GPXtDzLKTM= 2024-04-16T18:27:52.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UWs781aAZIPFCkIc3GPXtDzLKTM=', 'Content-Length': '0'} 2024-04-16T18:27:52.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8da14d7a7b7ab4d-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.815 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.815 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.815 DEBUG:boto:Method: HEAD 2024-04-16T18:27:52.815 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:27:52.815 DEBUG:boto:Data: 2024-04-16T18:27:52.815 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.816 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.816 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.816 DEBUG:boto:Params: {} 2024-04-16T18:27:52.816 DEBUG:boto:Token: None 2024-04-16T18:27:52.816 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95 2024-04-16T18:27:52.816 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UWs781aAZIPFCkIc3GPXtDzLKTM= 2024-04-16T18:27:52.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UWs781aAZIPFCkIc3GPXtDzLKTM=', 'Content-Length': '0'} 2024-04-16T18:27:52.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006100616e3bf91f89-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.819 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:27:52.820 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.820 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.820 DEBUG:boto:Method: GET 2024-04-16T18:27:52.820 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:27:52.820 DEBUG:boto:Data: 2024-04-16T18:27:52.820 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.820 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.820 DEBUG:boto:Params: {} 2024-04-16T18:27:52.820 DEBUG:boto:Token: None 2024-04-16T18:27:52.821 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95 2024-04-16T18:27:52.821 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oq1M6ALAxgHtJFSjG97MUDnckEE= 2024-04-16T18:27:52.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oq1M6ALAxgHtJFSjG97MUDnckEE=', 'Content-Length': '0'} 2024-04-16T18:27:52.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005958ca22431d434-00661ec328-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.866 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:27:52.866 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:27:52.867 DEBUG:boto:Method: GET 2024-04-16T18:27:52.867 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:27:52.867 DEBUG:boto:Data: 2024-04-16T18:27:52.867 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.867 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.867 DEBUG:boto:Params: {} 2024-04-16T18:27:52.867 DEBUG:boto:Token: None 2024-04-16T18:27:52.867 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/obj95 2024-04-16T18:27:52.867 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:oq1M6ALAxgHtJFSjG97MUDnckEE= 2024-04-16T18:27:52.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oq1M6ALAxgHtJFSjG97MUDnckEE=', 'Content-Length': '0'} 2024-04-16T18:27:52.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026d1d16616ce6499-00661ec328-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.871 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.871 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.871 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.871 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.871 DEBUG:boto:Method: GET 2024-04-16T18:27:52.871 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:27:52.871 DEBUG:boto:Data: 2024-04-16T18:27:52.871 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.872 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:27:52.872 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.872 DEBUG:boto:Params: {} 2024-04-16T18:27:52.872 DEBUG:boto:Token: None 2024-04-16T18:27:52.872 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.872 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049280e1aa4ccf4cb-00661ec328-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.922 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.923 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:52.923 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:52.923 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.923 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:27:52.923 DEBUG:boto:Method: GET 2024-04-16T18:27:52.923 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:27:52.923 DEBUG:boto:Data: 2024-04-16T18:27:52.923 DEBUG:boto:Headers: {} 2024-04-16T18:27:52.923 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:52.923 DEBUG:boto:Port: 8000 2024-04-16T18:27:52.923 DEBUG:boto:Params: {} 2024-04-16T18:27:52.924 DEBUG:boto:Token: None 2024-04-16T18:27:52.924 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:27:52 GMT /lybtug-52/ 2024-04-16T18:27:52.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE= 2024-04-16T18:27:52.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OuNISxVEJfDI232brBAV7X1n+CE=', 'Content-Length': '0'} 2024-04-16T18:27:52.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9cac4ed3765e50a-00661ec328-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:52.931 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:27:52.931 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:27:52.931 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '37', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:27:52.931 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--num-shards', '37', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:27:52.932 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --num-shards 37 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T18:27:52.977 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:27:52.977 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:27:53.087 INFO:teuthology.orchestra.run.smithi071.stdout:tenant: 2024-04-16T18:27:53.087 INFO:teuthology.orchestra.run.smithi071.stdout:bucket name: lybtug-52 2024-04-16T18:27:53.114 INFO:teuthology.orchestra.run.smithi071.stdout:total entries: 30 2024-04-16T18:27:53.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:53.288 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:53.288 DEBUG:boto:Method: HEAD 2024-04-16T18:27:53.288 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:53.288 DEBUG:boto:Data: 2024-04-16T18:27:53.288 DEBUG:boto:Headers: {} 2024-04-16T18:27:53.288 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.288 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.288 DEBUG:boto:Params: {} 2024-04-16T18:27:53.288 DEBUG:boto:Token: None 2024-04-16T18:27:53.288 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/ 2024-04-16T18:27:53.289 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4= 2024-04-16T18:27:53.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4=', 'Content-Length': '0'} 2024-04-16T18:27:53.294 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f6c94a77574e7c51-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.295 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:27:53.295 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:27:53.295 DEBUG:boto:Method: PUT 2024-04-16T18:27:53.295 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:27:53.295 DEBUG:boto:Data: 2024-04-16T18:27:53.295 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:27:53.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.295 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.295 DEBUG:boto:Params: {} 2024-04-16T18:27:53.295 DEBUG:boto:Token: None 2024-04-16T18:27:53.296 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/obj37 2024-04-16T18:27:53.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vs7FHXO1ihl0JNdUK5aEO6FUHNg= 2024-04-16T18:27:53.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vs7FHXO1ihl0JNdUK5aEO6FUHNg='} 2024-04-16T18:27:53.307 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ffe3a083f2f3cac4-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.307 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:53.307 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:53.307 DEBUG:boto:Method: HEAD 2024-04-16T18:27:53.307 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:53.307 DEBUG:boto:Data: 2024-04-16T18:27:53.307 DEBUG:boto:Headers: {} 2024-04-16T18:27:53.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.308 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.308 DEBUG:boto:Params: {} 2024-04-16T18:27:53.308 DEBUG:boto:Token: None 2024-04-16T18:27:53.308 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/ 2024-04-16T18:27:53.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4= 2024-04-16T18:27:53.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4=', 'Content-Length': '0'} 2024-04-16T18:27:53.313 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '93'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009c2a414743a0e0e5-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.313 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:27:53.313 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:27:53.313 DEBUG:boto:Method: PUT 2024-04-16T18:27:53.313 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:27:53.313 DEBUG:boto:Data: 2024-04-16T18:27:53.313 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:27:53.314 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.314 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.314 DEBUG:boto:Params: {} 2024-04-16T18:27:53.314 DEBUG:boto:Token: None 2024-04-16T18:27:53.314 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/obj74 2024-04-16T18:27:53.314 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XZf4PXjwlRYqScunDUdGDd08aUU= 2024-04-16T18:27:53.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XZf4PXjwlRYqScunDUdGDd08aUU='} 2024-04-16T18:27:53.358 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bcb097c5e8b8fff8-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.359 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:53.359 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:53.359 DEBUG:boto:Method: HEAD 2024-04-16T18:27:53.359 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:53.359 DEBUG:boto:Data: 2024-04-16T18:27:53.359 DEBUG:boto:Headers: {} 2024-04-16T18:27:53.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.359 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.359 DEBUG:boto:Params: {} 2024-04-16T18:27:53.359 DEBUG:boto:Token: None 2024-04-16T18:27:53.359 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/ 2024-04-16T18:27:53.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4= 2024-04-16T18:27:53.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4=', 'Content-Length': '0'} 2024-04-16T18:27:53.365 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004ded8ac594ae841e-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.365 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:27:53.366 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:27:53.366 DEBUG:boto:Method: PUT 2024-04-16T18:27:53.366 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:27:53.366 DEBUG:boto:Data: 2024-04-16T18:27:53.366 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:27:53.366 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.366 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.366 DEBUG:boto:Params: {} 2024-04-16T18:27:53.366 DEBUG:boto:Token: None 2024-04-16T18:27:53.367 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/obj111 2024-04-16T18:27:53.367 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JSZzcG8e3hKXfhMU9oA4OryQ0cU= 2024-04-16T18:27:53.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JSZzcG8e3hKXfhMU9oA4OryQ0cU='} 2024-04-16T18:27:53.379 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006b14aca008517c54-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.379 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:53.379 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:53.379 DEBUG:boto:Method: HEAD 2024-04-16T18:27:53.380 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:53.380 DEBUG:boto:Data: 2024-04-16T18:27:53.380 DEBUG:boto:Headers: {} 2024-04-16T18:27:53.380 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.380 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.380 DEBUG:boto:Params: {} 2024-04-16T18:27:53.380 DEBUG:boto:Token: None 2024-04-16T18:27:53.380 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/ 2024-04-16T18:27:53.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4= 2024-04-16T18:27:53.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4=', 'Content-Length': '0'} 2024-04-16T18:27:53.386 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '99'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b3d8fd4b06595e58-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.386 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:27:53.386 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:27:53.386 DEBUG:boto:Method: PUT 2024-04-16T18:27:53.386 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:27:53.387 DEBUG:boto:Data: 2024-04-16T18:27:53.387 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:27:53.387 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.387 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.387 DEBUG:boto:Params: {} 2024-04-16T18:27:53.387 DEBUG:boto:Token: None 2024-04-16T18:27:53.387 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/obj148 2024-04-16T18:27:53.387 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eaXYDBhHCmriDdNRy6dfs3dhV0M= 2024-04-16T18:27:53.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eaXYDBhHCmriDdNRy6dfs3dhV0M='} 2024-04-16T18:27:53.430 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004ef6454dd8a84a23-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.431 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:27:53.431 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:27:53.431 DEBUG:boto:Method: HEAD 2024-04-16T18:27:53.431 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:27:53.431 DEBUG:boto:Data: 2024-04-16T18:27:53.431 DEBUG:boto:Headers: {} 2024-04-16T18:27:53.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.431 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.431 DEBUG:boto:Params: {} 2024-04-16T18:27:53.431 DEBUG:boto:Token: None 2024-04-16T18:27:53.431 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/ 2024-04-16T18:27:53.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4= 2024-04-16T18:27:53.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mHGFGAimSuhZjqU2xvWpQVYMjL4=', 'Content-Length': '0'} 2024-04-16T18:27:53.438 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000032760e4bff71bd06-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.439 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:27:53.439 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:27:53.439 DEBUG:boto:Method: PUT 2024-04-16T18:27:53.439 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:27:53.439 DEBUG:boto:Data: 2024-04-16T18:27:53.439 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:27:53.439 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:27:53.439 DEBUG:boto:Port: 8000 2024-04-16T18:27:53.439 DEBUG:boto:Params: {} 2024-04-16T18:27:53.439 DEBUG:boto:Token: None 2024-04-16T18:27:53.439 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:27:53 GMT /lybtug-52/obj185 2024-04-16T18:27:53.440 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N+HVD8lwoy3CZ6kyDEwiZFMxXB8= 2024-04-16T18:27:53.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:27:53 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N+HVD8lwoy3CZ6kyDEwiZFMxXB8='} 2024-04-16T18:27:53.453 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000011b3657aeac99a5c-00661ec329-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:27:53.453 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:27:53.453 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:27:53.453 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:27:53.453 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:27:53.503 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:27:53.503 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:27:53.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:53.587+0000 7fee91d50980 1 waiting to reach latest gen 6, on 5.. 2024-04-16T18:27:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:27:58.591+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:27:58.594 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:27:58.594 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:03.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:03.595+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:03.598 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:03.598 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:08.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:08.603+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:08.602 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:08.602 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:13.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:13.607+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:13.606 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:13.606 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:18.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:18.607+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:18.610 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:18.610 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:23.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:23.610+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:23.610 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:23.610 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:28.614+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:28.614 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:28.614 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:33.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:33.618+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:33.618 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:33.619 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:38.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:38.622+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:38.624 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:38.624 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:43.626+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:43.626 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:43.626 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:48.630+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:48.631 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:48.631 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:53.633+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:53.635 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , , , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:53.635 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:58.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:28:58.637+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:28:58.639 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:28:58.639 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:03.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:03.641+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:03.643 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:03.643 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:08.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:08.645+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:08.647 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:08.647 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:13.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:13.649+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:13.652 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:13.652 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:18.657+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:18.656 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:18.656 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:23.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:23.661+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:23.661 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:23.661 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:28.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:28.664+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:28.665 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:28.665 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:33.668+0000 7fee91d50980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:33.669 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , , , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:33.669 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:38.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:38.672+0000 7fee91d50980 1 bucket sync caught up with source: 2024-04-16T18:29:38.673 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:38.673 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:29:38.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:29:38.672+0000 7fee91d50980 0 bucket checkpoint complete 2024-04-16T18:29:38.721 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:29:38.752 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:29:38.753 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:29:38.753 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:29:38.803 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:29:38.803 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:29:38.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:29:38.886+0000 7f48a3e88980 1 waiting for incremental sync to catch up: 2024-04-16T18:29:38.883 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000005.1238.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000004.483.6, 00000000006.338.6, 00000000006.431.6] 2024-04-16T18:29:38.883 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000005.1238.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000004.483.6, 00000000007.342.6, 00000000006.431.6] 2024-04-16T18:29:43.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:29:43.890+0000 7f48a3e88980 1 bucket sync caught up with source: 2024-04-16T18:29:43.886 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000005.1238.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000004.483.6, 00000000007.342.6, 00000000006.431.6] 2024-04-16T18:29:43.886 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000003.645.6, 00000000002.559.6, 00000000002.483.6, 00000000005.1238.6, 00000000004.529.6, 00000000001.961.6, 00000000001.984.6, 00000000004.483.6, 00000000007.342.6, 00000000006.431.6] 2024-04-16T18:29:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:29:43.890+0000 7f48a3e88980 0 bucket checkpoint complete 2024-04-16T18:29:43.931 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:29:43.931 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:43.931 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:43.932 DEBUG:boto:Method: HEAD 2024-04-16T18:29:43.932 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:29:43.932 DEBUG:boto:Data: 2024-04-16T18:29:43.932 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.932 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:43.932 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.932 DEBUG:boto:Params: {} 2024-04-16T18:29:43.933 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:29:43.933 DEBUG:boto:Token: None 2024-04-16T18:29:43.933 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/ 2024-04-16T18:29:43.933 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+gx3xNIq+jtGNBFSIAwTlNCgqiM= 2024-04-16T18:29:43.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+gx3xNIq+jtGNBFSIAwTlNCgqiM=', 'Content-Length': '0'} 2024-04-16T18:29:43.940 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000001d07a6ab4e825cb-00661ec397-5545-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:29:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:43.940 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:43.940 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:43.940 DEBUG:boto:Method: HEAD 2024-04-16T18:29:43.941 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:29:43.941 DEBUG:boto:Data: 2024-04-16T18:29:43.941 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.941 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:43.941 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.941 DEBUG:boto:Params: {} 2024-04-16T18:29:43.941 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:29:43.942 DEBUG:boto:Token: None 2024-04-16T18:29:43.942 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/ 2024-04-16T18:29:43.942 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+gx3xNIq+jtGNBFSIAwTlNCgqiM= 2024-04-16T18:29:43.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+gx3xNIq+jtGNBFSIAwTlNCgqiM=', 'Content-Length': '0'} 2024-04-16T18:29:43.949 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b71df93f453853e0-00661ec397-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:29:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:43.949 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:29:43.950 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:43.950 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:43.950 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.950 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.950 DEBUG:boto:Method: GET 2024-04-16T18:29:43.951 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.951 DEBUG:boto:Data: 2024-04-16T18:29:43.951 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.951 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:43.951 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.952 DEBUG:boto:Params: {} 2024-04-16T18:29:43.952 DEBUG:boto:Token: None 2024-04-16T18:29:43.952 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/?versions 2024-04-16T18:29:43.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA= 2024-04-16T18:29:43.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA=', 'Content-Length': '0'} 2024-04-16T18:29:43.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000696710dfb56ef376-00661ec397-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:43.958 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:29:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2024-04-16T18:29:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:29:43.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:29:43.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:29:43.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:29:43.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:29:43.964 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2024-04-16T18:29:43.964 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:29:43.964 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:29:43.964 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:43.964 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:43.964 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.964 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.964 DEBUG:boto:Method: GET 2024-04-16T18:29:43.965 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.965 DEBUG:boto:Data: 2024-04-16T18:29:43.965 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.965 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:43.965 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.965 DEBUG:boto:Params: {} 2024-04-16T18:29:43.965 DEBUG:boto:Token: None 2024-04-16T18:29:43.965 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/?versions 2024-04-16T18:29:43.965 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA= 2024-04-16T18:29:43.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA=', 'Content-Length': '0'} 2024-04-16T18:29:43.973 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2ba505c40fdf8e1-00661ec397-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:43.973 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:43.974 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:29:43.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:29:43.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2024-04-16T18:29:43.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:29:43.977 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:43.977 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:43.978 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.978 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.978 DEBUG:boto:Method: GET 2024-04-16T18:29:43.978 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.978 DEBUG:boto:Data: 2024-04-16T18:29:43.978 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.978 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:43.978 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.978 DEBUG:boto:Params: {} 2024-04-16T18:29:43.978 DEBUG:boto:Token: None 2024-04-16T18:29:43.979 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/?versions 2024-04-16T18:29:43.979 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA= 2024-04-16T18:29:43.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA=', 'Content-Length': '0'} 2024-04-16T18:29:43.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de2e6fd8fcabd41d-00661ec397-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:43.984 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:43.985 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:43.985 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:43.986 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.986 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.986 DEBUG:boto:Method: GET 2024-04-16T18:29:43.986 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:29:43.986 DEBUG:boto:Data: 2024-04-16T18:29:43.986 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.986 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:43.986 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.986 DEBUG:boto:Params: {} 2024-04-16T18:29:43.986 DEBUG:boto:Token: None 2024-04-16T18:29:43.987 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/?versions 2024-04-16T18:29:43.987 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA= 2024-04-16T18:29:43.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/ILnac+OAzYyfzIEyqqEE6fTCFA=', 'Content-Length': '0'} 2024-04-16T18:29:43.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004841defe1827c022-00661ec397-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:43 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:43.993 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:43.995 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:29:43.995 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:43.995 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:43.996 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:43.996 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:43.996 DEBUG:boto:Method: GET 2024-04-16T18:29:43.996 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:29:43.996 DEBUG:boto:Data: 2024-04-16T18:29:43.996 DEBUG:boto:Headers: {} 2024-04-16T18:29:43.996 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:43.996 DEBUG:boto:Port: 8000 2024-04-16T18:29:43.996 DEBUG:boto:Params: {} 2024-04-16T18:29:43.996 DEBUG:boto:Token: None 2024-04-16T18:29:43.997 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:43 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:29:43.997 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MsKoao4F2twXwp3YLBGwDSj1mXE= 2024-04-16T18:29:43.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:43 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MsKoao4F2twXwp3YLBGwDSj1mXE=', 'Content-Length': '0'} 2024-04-16T18:29:44.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1943aff2f0c4e0c-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.042 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.042 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.042 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.043 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.043 DEBUG:boto:Method: GET 2024-04-16T18:29:44.043 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.043 DEBUG:boto:Data: 2024-04-16T18:29:44.043 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.043 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.043 DEBUG:boto:Params: {} 2024-04-16T18:29:44.043 DEBUG:boto:Token: None 2024-04-16T18:29:44.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KiSVkLj+YxjvRUeio62MpwU+h6k= 2024-04-16T18:29:44.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KiSVkLj+YxjvRUeio62MpwU+h6k=', 'Content-Length': '0'} 2024-04-16T18:29:44.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091c296a3b55796d0-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.047 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.047 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.047 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.047 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.047 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.047 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.047 DEBUG:boto:Data: 2024-04-16T18:29:44.047 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.047 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.047 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.047 DEBUG:boto:Params: {} 2024-04-16T18:29:44.048 DEBUG:boto:Token: None 2024-04-16T18:29:44.048 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GtaC+SQxmgEEladYZrzTvVvQHZo= 2024-04-16T18:29:44.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GtaC+SQxmgEEladYZrzTvVvQHZo=', 'Content-Length': '0'} 2024-04-16T18:29:44.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077d75bb558302622-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.052 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.052 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.052 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.052 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.052 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.052 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.052 DEBUG:boto:Data: 2024-04-16T18:29:44.052 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.052 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.052 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.052 DEBUG:boto:Params: {} 2024-04-16T18:29:44.053 DEBUG:boto:Token: None 2024-04-16T18:29:44.053 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.053 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GtaC+SQxmgEEladYZrzTvVvQHZo= 2024-04-16T18:29:44.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GtaC+SQxmgEEladYZrzTvVvQHZo=', 'Content-Length': '0'} 2024-04-16T18:29:44.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a50497008c9b9e19-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.056 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:29:44.056 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.056 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.056 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.056 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.056 DEBUG:boto:Method: GET 2024-04-16T18:29:44.056 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.056 DEBUG:boto:Data: 2024-04-16T18:29:44.056 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.056 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.057 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.057 DEBUG:boto:Params: {} 2024-04-16T18:29:44.057 DEBUG:boto:Token: None 2024-04-16T18:29:44.057 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.057 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KiSVkLj+YxjvRUeio62MpwU+h6k= 2024-04-16T18:29:44.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KiSVkLj+YxjvRUeio62MpwU+h6k=', 'Content-Length': '0'} 2024-04-16T18:29:44.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8e7219e99068dc1-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.102 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.103 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.103 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.103 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.103 DEBUG:boto:Method: GET 2024-04-16T18:29:44.103 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.103 DEBUG:boto:Data: 2024-04-16T18:29:44.103 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.103 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.104 DEBUG:boto:Params: {} 2024-04-16T18:29:44.104 DEBUG:boto:Token: None 2024-04-16T18:29:44.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:29:44.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KiSVkLj+YxjvRUeio62MpwU+h6k= 2024-04-16T18:29:44.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KiSVkLj+YxjvRUeio62MpwU+h6k=', 'Content-Length': '0'} 2024-04-16T18:29:44.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b55bf3f8fa6fb831-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.107 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.107 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.107 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.108 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.108 DEBUG:boto:Method: GET 2024-04-16T18:29:44.108 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:29:44.108 DEBUG:boto:Data: 2024-04-16T18:29:44.108 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.108 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.108 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.108 DEBUG:boto:Params: {} 2024-04-16T18:29:44.108 DEBUG:boto:Token: None 2024-04-16T18:29:44.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.112 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000260011ab051046b4-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.158 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.159 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.159 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.159 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.159 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.159 DEBUG:boto:Method: GET 2024-04-16T18:29:44.159 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:29:44.159 DEBUG:boto:Data: 2024-04-16T18:29:44.160 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.160 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.160 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.160 DEBUG:boto:Params: {} 2024-04-16T18:29:44.160 DEBUG:boto:Token: None 2024-04-16T18:29:44.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.167 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003052f9e76dde6848-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.167 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.167 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.168 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.168 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.168 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:29:44.168 DEBUG:boto:Data: 2024-04-16T18:29:44.168 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.168 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.168 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.168 DEBUG:boto:Params: {} 2024-04-16T18:29:44.169 DEBUG:boto:Token: None 2024-04-16T18:29:44.169 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11 2024-04-16T18:29:44.169 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AsDzTqIGkMmkX1+RFD7uhmppOeY= 2024-04-16T18:29:44.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AsDzTqIGkMmkX1+RFD7uhmppOeY=', 'Content-Length': '0'} 2024-04-16T18:29:44.172 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000079f94955d297b89-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.172 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.172 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.173 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.173 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:29:44.173 DEBUG:boto:Data: 2024-04-16T18:29:44.173 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.173 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.173 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.173 DEBUG:boto:Params: {} 2024-04-16T18:29:44.173 DEBUG:boto:Token: None 2024-04-16T18:29:44.173 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11 2024-04-16T18:29:44.174 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AsDzTqIGkMmkX1+RFD7uhmppOeY= 2024-04-16T18:29:44.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AsDzTqIGkMmkX1+RFD7uhmppOeY=', 'Content-Length': '0'} 2024-04-16T18:29:44.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a4ad3ed64152e5d-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.176 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:29:44.177 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.177 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.177 DEBUG:boto:Method: GET 2024-04-16T18:29:44.177 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:29:44.177 DEBUG:boto:Data: 2024-04-16T18:29:44.177 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.177 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.177 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.177 DEBUG:boto:Params: {} 2024-04-16T18:29:44.177 DEBUG:boto:Token: None 2024-04-16T18:29:44.178 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11 2024-04-16T18:29:44.178 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VdpZmOwFUxJadXyU0pY6RW0dVIg= 2024-04-16T18:29:44.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VdpZmOwFUxJadXyU0pY6RW0dVIg=', 'Content-Length': '0'} 2024-04-16T18:29:44.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039ccf6271735dadb-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.181 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:29:44.181 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:29:44.181 DEBUG:boto:Method: GET 2024-04-16T18:29:44.181 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:29:44.181 DEBUG:boto:Data: 2024-04-16T18:29:44.181 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.181 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.182 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.182 DEBUG:boto:Params: {} 2024-04-16T18:29:44.182 DEBUG:boto:Token: None 2024-04-16T18:29:44.182 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj11 2024-04-16T18:29:44.182 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VdpZmOwFUxJadXyU0pY6RW0dVIg= 2024-04-16T18:29:44.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VdpZmOwFUxJadXyU0pY6RW0dVIg=', 'Content-Length': '0'} 2024-04-16T18:29:44.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000387d9d089bab7b32-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.186 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.186 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.186 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.186 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.187 DEBUG:boto:Method: GET 2024-04-16T18:29:44.187 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:29:44.187 DEBUG:boto:Data: 2024-04-16T18:29:44.187 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.187 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.187 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.187 DEBUG:boto:Params: {} 2024-04-16T18:29:44.187 DEBUG:boto:Token: None 2024-04-16T18:29:44.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.188 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052d81aaa081e0eb6-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.192 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.192 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.193 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.193 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.193 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:29:44.193 DEBUG:boto:Method: GET 2024-04-16T18:29:44.193 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:29:44.193 DEBUG:boto:Data: 2024-04-16T18:29:44.193 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.193 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.193 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.193 DEBUG:boto:Params: {} 2024-04-16T18:29:44.194 DEBUG:boto:Token: None 2024-04-16T18:29:44.194 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.194 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.201 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5d4436c4ddf1ca0-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.201 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.201 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2024-04-16T18:29:44.201 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.202 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.202 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.202 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.202 DEBUG:boto:Method: GET 2024-04-16T18:29:44.202 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.202 DEBUG:boto:Data: 2024-04-16T18:29:44.202 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.202 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.202 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.202 DEBUG:boto:Params: {} 2024-04-16T18:29:44.203 DEBUG:boto:Token: None 2024-04-16T18:29:44.203 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.203 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8= 2024-04-16T18:29:44.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8=', 'Content-Length': '0'} 2024-04-16T18:29:44.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:28:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a8e6919acf42369-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.246 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.246 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.247 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.247 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.247 DEBUG:boto:Method: GET 2024-04-16T18:29:44.247 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.247 DEBUG:boto:Data: 2024-04-16T18:29:44.247 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.247 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.247 DEBUG:boto:Params: {} 2024-04-16T18:29:44.247 DEBUG:boto:Token: None 2024-04-16T18:29:44.248 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8= 2024-04-16T18:29:44.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8=', 'Content-Length': '0'} 2024-04-16T18:29:44.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011002e07819b77c1-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.251 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.251 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.251 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.251 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.252 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.252 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.252 DEBUG:boto:Data: 2024-04-16T18:29:44.252 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.252 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.252 DEBUG:boto:Params: {} 2024-04-16T18:29:44.252 DEBUG:boto:Token: None 2024-04-16T18:29:44.252 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.253 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kXHgpC+9o7Niv24qLrhhoJ000P0= 2024-04-16T18:29:44.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kXHgpC+9o7Niv24qLrhhoJ000P0=', 'Content-Length': '0'} 2024-04-16T18:29:44.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:28:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080997758b4711888-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.256 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.256 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.256 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.256 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.257 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.257 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.257 DEBUG:boto:Data: 2024-04-16T18:29:44.257 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.257 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.257 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.257 DEBUG:boto:Params: {} 2024-04-16T18:29:44.257 DEBUG:boto:Token: None 2024-04-16T18:29:44.257 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.257 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kXHgpC+9o7Niv24qLrhhoJ000P0= 2024-04-16T18:29:44.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kXHgpC+9o7Niv24qLrhhoJ000P0=', 'Content-Length': '0'} 2024-04-16T18:29:44.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e48136682c56c9d-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.260 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2024-04-16T18:29:44.261 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.261 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.261 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.261 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.261 DEBUG:boto:Method: GET 2024-04-16T18:29:44.261 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.261 DEBUG:boto:Data: 2024-04-16T18:29:44.261 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.261 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.261 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.262 DEBUG:boto:Params: {} 2024-04-16T18:29:44.262 DEBUG:boto:Token: None 2024-04-16T18:29:44.262 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.262 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8= 2024-04-16T18:29:44.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8=', 'Content-Length': '0'} 2024-04-16T18:29:44.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:28:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000339cf317bbc4452d-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.265 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.265 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.266 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.266 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.266 DEBUG:boto:Method: GET 2024-04-16T18:29:44.266 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.266 DEBUG:boto:Data: 2024-04-16T18:29:44.266 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.266 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.266 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.266 DEBUG:boto:Params: {} 2024-04-16T18:29:44.267 DEBUG:boto:Token: None 2024-04-16T18:29:44.267 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:29:44.267 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8= 2024-04-16T18:29:44.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iVfbZkSGzH043ufM14wTWtzSdH8=', 'Content-Length': '0'} 2024-04-16T18:29:44.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3731e4a27ca5ad4-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.310 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.310 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.311 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.311 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.311 DEBUG:boto:Method: GET 2024-04-16T18:29:44.311 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:29:44.311 DEBUG:boto:Data: 2024-04-16T18:29:44.311 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.311 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.311 DEBUG:boto:Params: {} 2024-04-16T18:29:44.312 DEBUG:boto:Token: None 2024-04-16T18:29:44.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.316 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b359171e6eb172a-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.316 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.317 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.317 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.317 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.317 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.317 DEBUG:boto:Method: GET 2024-04-16T18:29:44.317 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:29:44.317 DEBUG:boto:Data: 2024-04-16T18:29:44.317 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.318 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.318 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.318 DEBUG:boto:Params: {} 2024-04-16T18:29:44.318 DEBUG:boto:Token: None 2024-04-16T18:29:44.318 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.318 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dac2f5e200312a35-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.325 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.326 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.326 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.326 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.326 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:29:44.326 DEBUG:boto:Data: 2024-04-16T18:29:44.326 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.326 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.326 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.327 DEBUG:boto:Params: {} 2024-04-16T18:29:44.327 DEBUG:boto:Token: None 2024-04-16T18:29:44.327 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111 2024-04-16T18:29:44.327 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gldlE6PFEDTbz4bJgT4+woXLHY4= 2024-04-16T18:29:44.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gldlE6PFEDTbz4bJgT4+woXLHY4=', 'Content-Length': '0'} 2024-04-16T18:29:44.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:28:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008590c9c5b31cb822-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.330 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.330 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.330 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.330 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:29:44.330 DEBUG:boto:Data: 2024-04-16T18:29:44.331 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.331 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.331 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.331 DEBUG:boto:Params: {} 2024-04-16T18:29:44.331 DEBUG:boto:Token: None 2024-04-16T18:29:44.331 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111 2024-04-16T18:29:44.331 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gldlE6PFEDTbz4bJgT4+woXLHY4= 2024-04-16T18:29:44.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gldlE6PFEDTbz4bJgT4+woXLHY4=', 'Content-Length': '0'} 2024-04-16T18:29:44.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f95b7b91e4b181c-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.334 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2024-04-16T18:29:44.334 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.334 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.335 DEBUG:boto:Method: GET 2024-04-16T18:29:44.335 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:29:44.335 DEBUG:boto:Data: 2024-04-16T18:29:44.335 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.335 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.335 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.335 DEBUG:boto:Params: {} 2024-04-16T18:29:44.335 DEBUG:boto:Token: None 2024-04-16T18:29:44.335 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111 2024-04-16T18:29:44.335 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8993jGVjuh4bgFrLJtyZK++/6cs= 2024-04-16T18:29:44.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8993jGVjuh4bgFrLJtyZK++/6cs=', 'Content-Length': '0'} 2024-04-16T18:29:44.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:28:57 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009dfa418f39e5cf91-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.382 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:29:44.383 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:29:44.383 DEBUG:boto:Method: GET 2024-04-16T18:29:44.383 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:29:44.383 DEBUG:boto:Data: 2024-04-16T18:29:44.383 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.383 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.383 DEBUG:boto:Params: {} 2024-04-16T18:29:44.383 DEBUG:boto:Token: None 2024-04-16T18:29:44.383 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj111 2024-04-16T18:29:44.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8993jGVjuh4bgFrLJtyZK++/6cs= 2024-04-16T18:29:44.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8993jGVjuh4bgFrLJtyZK++/6cs=', 'Content-Length': '0'} 2024-04-16T18:29:44.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd84d98c43bc97b3-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.387 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.387 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.387 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.387 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.387 DEBUG:boto:Method: GET 2024-04-16T18:29:44.387 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:29:44.387 DEBUG:boto:Data: 2024-04-16T18:29:44.387 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.388 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.388 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.388 DEBUG:boto:Params: {} 2024-04-16T18:29:44.388 DEBUG:boto:Token: None 2024-04-16T18:29:44.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d096a77d634e02e-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.392 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.393 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.393 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.393 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.393 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:29:44.393 DEBUG:boto:Method: GET 2024-04-16T18:29:44.393 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:29:44.393 DEBUG:boto:Data: 2024-04-16T18:29:44.393 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.393 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.393 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.393 DEBUG:boto:Params: {} 2024-04-16T18:29:44.394 DEBUG:boto:Token: None 2024-04-16T18:29:44.394 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.394 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.400 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca16c7501296ece7-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.442 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.443 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:29:44.443 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.443 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.443 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.443 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.443 DEBUG:boto:Method: GET 2024-04-16T18:29:44.443 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.443 DEBUG:boto:Data: 2024-04-16T18:29:44.443 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.443 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.443 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.444 DEBUG:boto:Params: {} 2024-04-16T18:29:44.444 DEBUG:boto:Token: None 2024-04-16T18:29:44.444 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk= 2024-04-16T18:29:44.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk=', 'Content-Length': '0'} 2024-04-16T18:29:44.447 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c46f3122ae4c3573-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.447 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.447 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.448 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.448 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.448 DEBUG:boto:Method: GET 2024-04-16T18:29:44.448 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.448 DEBUG:boto:Data: 2024-04-16T18:29:44.448 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.448 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.448 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.448 DEBUG:boto:Params: {} 2024-04-16T18:29:44.448 DEBUG:boto:Token: None 2024-04-16T18:29:44.449 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.449 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk= 2024-04-16T18:29:44.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk=', 'Content-Length': '0'} 2024-04-16T18:29:44.452 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000434c022dd9173546-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.452 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.452 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.452 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.452 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.452 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.452 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.453 DEBUG:boto:Data: 2024-04-16T18:29:44.453 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.453 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.453 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.453 DEBUG:boto:Params: {} 2024-04-16T18:29:44.453 DEBUG:boto:Token: None 2024-04-16T18:29:44.453 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.453 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L05KU2FtB/Y3KZ0ZATUcPT4Hgl8= 2024-04-16T18:29:44.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L05KU2FtB/Y3KZ0ZATUcPT4Hgl8=', 'Content-Length': '0'} 2024-04-16T18:29:44.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f1c39970fa2a4dc-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.457 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.457 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.457 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.457 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.457 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.457 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.457 DEBUG:boto:Data: 2024-04-16T18:29:44.457 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.457 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.457 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.457 DEBUG:boto:Params: {} 2024-04-16T18:29:44.457 DEBUG:boto:Token: None 2024-04-16T18:29:44.458 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.458 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:L05KU2FtB/Y3KZ0ZATUcPT4Hgl8= 2024-04-16T18:29:44.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:L05KU2FtB/Y3KZ0ZATUcPT4Hgl8=', 'Content-Length': '0'} 2024-04-16T18:29:44.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eaa98dbf988fbab1-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.461 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:29:44.461 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.461 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.461 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.461 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.462 DEBUG:boto:Method: GET 2024-04-16T18:29:44.462 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.462 DEBUG:boto:Data: 2024-04-16T18:29:44.462 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.462 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.462 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.462 DEBUG:boto:Params: {} 2024-04-16T18:29:44.462 DEBUG:boto:Token: None 2024-04-16T18:29:44.462 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.462 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk= 2024-04-16T18:29:44.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk=', 'Content-Length': '0'} 2024-04-16T18:29:44.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028c793536a14fdc3-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.506 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.506 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.507 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.507 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.507 DEBUG:boto:Method: GET 2024-04-16T18:29:44.507 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.507 DEBUG:boto:Data: 2024-04-16T18:29:44.507 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.507 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.507 DEBUG:boto:Params: {} 2024-04-16T18:29:44.507 DEBUG:boto:Token: None 2024-04-16T18:29:44.508 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:29:44.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk= 2024-04-16T18:29:44.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gz7eTyaEeeV0jxGRv8JWn3THwZk=', 'Content-Length': '0'} 2024-04-16T18:29:44.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3a160738103c154-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.511 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.511 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.511 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.511 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.511 DEBUG:boto:Method: GET 2024-04-16T18:29:44.512 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:29:44.512 DEBUG:boto:Data: 2024-04-16T18:29:44.512 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.512 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.512 DEBUG:boto:Params: {} 2024-04-16T18:29:44.512 DEBUG:boto:Token: None 2024-04-16T18:29:44.512 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.516 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6ab37b98b056464-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.516 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.517 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.517 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.517 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.517 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.517 DEBUG:boto:Method: GET 2024-04-16T18:29:44.517 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:29:44.517 DEBUG:boto:Data: 2024-04-16T18:29:44.517 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.517 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.517 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.518 DEBUG:boto:Params: {} 2024-04-16T18:29:44.518 DEBUG:boto:Token: None 2024-04-16T18:29:44.518 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.518 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.525 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000038a50c0cd51b58-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.570 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.571 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.571 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.571 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.571 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:29:44.571 DEBUG:boto:Data: 2024-04-16T18:29:44.571 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.571 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.571 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.571 DEBUG:boto:Params: {} 2024-04-16T18:29:44.572 DEBUG:boto:Token: None 2024-04-16T18:29:44.572 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115 2024-04-16T18:29:44.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5sAThEjGxwN9CHSis49p10R3/KA= 2024-04-16T18:29:44.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5sAThEjGxwN9CHSis49p10R3/KA=', 'Content-Length': '0'} 2024-04-16T18:29:44.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb6835187f4eb698-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.575 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.575 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.575 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.576 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:29:44.576 DEBUG:boto:Data: 2024-04-16T18:29:44.576 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.576 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.576 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.576 DEBUG:boto:Params: {} 2024-04-16T18:29:44.576 DEBUG:boto:Token: None 2024-04-16T18:29:44.576 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115 2024-04-16T18:29:44.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5sAThEjGxwN9CHSis49p10R3/KA= 2024-04-16T18:29:44.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5sAThEjGxwN9CHSis49p10R3/KA=', 'Content-Length': '0'} 2024-04-16T18:29:44.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e2bfcd75b1a0f73-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.580 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:29:44.580 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.580 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.580 DEBUG:boto:Method: GET 2024-04-16T18:29:44.580 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:29:44.580 DEBUG:boto:Data: 2024-04-16T18:29:44.580 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.581 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.581 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.581 DEBUG:boto:Params: {} 2024-04-16T18:29:44.581 DEBUG:boto:Token: None 2024-04-16T18:29:44.581 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115 2024-04-16T18:29:44.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qJd+nANGh/LYFUFYwf74UWe8RH0= 2024-04-16T18:29:44.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qJd+nANGh/LYFUFYwf74UWe8RH0=', 'Content-Length': '0'} 2024-04-16T18:29:44.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:49 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a311ec53cf1fbf17-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.585 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:29:44.585 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:29:44.585 DEBUG:boto:Method: GET 2024-04-16T18:29:44.585 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:29:44.585 DEBUG:boto:Data: 2024-04-16T18:29:44.585 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.585 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.585 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.585 DEBUG:boto:Params: {} 2024-04-16T18:29:44.585 DEBUG:boto:Token: None 2024-04-16T18:29:44.586 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj115 2024-04-16T18:29:44.586 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qJd+nANGh/LYFUFYwf74UWe8RH0= 2024-04-16T18:29:44.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qJd+nANGh/LYFUFYwf74UWe8RH0=', 'Content-Length': '0'} 2024-04-16T18:29:44.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fce970ae902ac74-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.590 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.590 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.590 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.590 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.590 DEBUG:boto:Method: GET 2024-04-16T18:29:44.590 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:29:44.590 DEBUG:boto:Data: 2024-04-16T18:29:44.590 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.590 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.591 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.591 DEBUG:boto:Params: {} 2024-04-16T18:29:44.591 DEBUG:boto:Token: None 2024-04-16T18:29:44.591 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.591 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.595 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b2feb25812ed8ef-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.638 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.639 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.639 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.639 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.639 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:29:44.639 DEBUG:boto:Method: GET 2024-04-16T18:29:44.639 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:29:44.639 DEBUG:boto:Data: 2024-04-16T18:29:44.639 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.639 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.639 DEBUG:boto:Params: {} 2024-04-16T18:29:44.640 DEBUG:boto:Token: None 2024-04-16T18:29:44.640 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.646 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1d0a5b44b1e5952-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.646 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.647 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:29:44.647 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.647 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.647 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.647 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.647 DEBUG:boto:Method: GET 2024-04-16T18:29:44.647 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.647 DEBUG:boto:Data: 2024-04-16T18:29:44.647 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.647 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.647 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.648 DEBUG:boto:Params: {} 2024-04-16T18:29:44.648 DEBUG:boto:Token: None 2024-04-16T18:29:44.648 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.648 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI= 2024-04-16T18:29:44.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI=', 'Content-Length': '0'} 2024-04-16T18:29:44.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001f35d92993b477f-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.651 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.651 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.651 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.652 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.652 DEBUG:boto:Method: GET 2024-04-16T18:29:44.652 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.652 DEBUG:boto:Data: 2024-04-16T18:29:44.652 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.652 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.652 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.652 DEBUG:boto:Params: {} 2024-04-16T18:29:44.652 DEBUG:boto:Token: None 2024-04-16T18:29:44.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI= 2024-04-16T18:29:44.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI=', 'Content-Length': '0'} 2024-04-16T18:29:44.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a82a32823415076-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.698 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.699 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.699 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.699 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.699 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.699 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.699 DEBUG:boto:Data: 2024-04-16T18:29:44.699 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.699 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.699 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.699 DEBUG:boto:Params: {} 2024-04-16T18:29:44.699 DEBUG:boto:Token: None 2024-04-16T18:29:44.700 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bHZpNuV5JmiDAnSqoilj/zJwWps= 2024-04-16T18:29:44.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bHZpNuV5JmiDAnSqoilj/zJwWps=', 'Content-Length': '0'} 2024-04-16T18:29:44.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062ed2ac5f5735cf3-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.703 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.703 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.703 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.703 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.703 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.703 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.703 DEBUG:boto:Data: 2024-04-16T18:29:44.704 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.704 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.704 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.704 DEBUG:boto:Params: {} 2024-04-16T18:29:44.704 DEBUG:boto:Token: None 2024-04-16T18:29:44.704 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bHZpNuV5JmiDAnSqoilj/zJwWps= 2024-04-16T18:29:44.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bHZpNuV5JmiDAnSqoilj/zJwWps=', 'Content-Length': '0'} 2024-04-16T18:29:44.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037ba9b6a997928be-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:29:44.708 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.708 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.708 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.708 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.708 DEBUG:boto:Method: GET 2024-04-16T18:29:44.708 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.708 DEBUG:boto:Data: 2024-04-16T18:29:44.708 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.708 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.709 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.709 DEBUG:boto:Params: {} 2024-04-16T18:29:44.709 DEBUG:boto:Token: None 2024-04-16T18:29:44.709 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.709 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI= 2024-04-16T18:29:44.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI=', 'Content-Length': '0'} 2024-04-16T18:29:44.712 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031e5ddcf8cf402f5-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.712 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.712 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.713 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.713 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.713 DEBUG:boto:Method: GET 2024-04-16T18:29:44.713 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.713 DEBUG:boto:Data: 2024-04-16T18:29:44.713 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.713 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.713 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.713 DEBUG:boto:Params: {} 2024-04-16T18:29:44.713 DEBUG:boto:Token: None 2024-04-16T18:29:44.713 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:29:44.714 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI= 2024-04-16T18:29:44.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8UWs96lUZ/20jZDs8EvSFp7gERI=', 'Content-Length': '0'} 2024-04-16T18:29:44.717 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046b1afe25c5ecc6f-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.717 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.717 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.717 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.717 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.717 DEBUG:boto:Method: GET 2024-04-16T18:29:44.718 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:29:44.718 DEBUG:boto:Data: 2024-04-16T18:29:44.718 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.718 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.718 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.718 DEBUG:boto:Params: {} 2024-04-16T18:29:44.718 DEBUG:boto:Token: None 2024-04-16T18:29:44.718 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.722 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cbbe6fb95852ef9-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.766 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.767 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.767 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.767 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.768 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.768 DEBUG:boto:Method: GET 2024-04-16T18:29:44.768 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:29:44.768 DEBUG:boto:Data: 2024-04-16T18:29:44.768 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.768 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.768 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.768 DEBUG:boto:Params: {} 2024-04-16T18:29:44.768 DEBUG:boto:Token: None 2024-04-16T18:29:44.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005694274a56efda70-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.776 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.776 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.776 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.776 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.777 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:29:44.777 DEBUG:boto:Data: 2024-04-16T18:29:44.777 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.777 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.777 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.777 DEBUG:boto:Params: {} 2024-04-16T18:29:44.777 DEBUG:boto:Token: None 2024-04-16T18:29:44.777 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116 2024-04-16T18:29:44.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VRzADTBGireowIvjpV6oX5gKMjw= 2024-04-16T18:29:44.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VRzADTBGireowIvjpV6oX5gKMjw=', 'Content-Length': '0'} 2024-04-16T18:29:44.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5d85deae89d80b5-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.781 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.781 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.781 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.781 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:29:44.781 DEBUG:boto:Data: 2024-04-16T18:29:44.781 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.781 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.781 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.782 DEBUG:boto:Params: {} 2024-04-16T18:29:44.782 DEBUG:boto:Token: None 2024-04-16T18:29:44.782 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116 2024-04-16T18:29:44.782 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VRzADTBGireowIvjpV6oX5gKMjw= 2024-04-16T18:29:44.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VRzADTBGireowIvjpV6oX5gKMjw=', 'Content-Length': '0'} 2024-04-16T18:29:44.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004df7279c82b7df4e-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.785 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:29:44.785 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.785 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.785 DEBUG:boto:Method: GET 2024-04-16T18:29:44.785 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:29:44.786 DEBUG:boto:Data: 2024-04-16T18:29:44.786 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.786 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.786 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.786 DEBUG:boto:Params: {} 2024-04-16T18:29:44.786 DEBUG:boto:Token: None 2024-04-16T18:29:44.786 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116 2024-04-16T18:29:44.786 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uvphvy1h53SK0XXJ5ofFgX0k26Q= 2024-04-16T18:29:44.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uvphvy1h53SK0XXJ5ofFgX0k26Q=', 'Content-Length': '0'} 2024-04-16T18:29:44.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:32 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074735757af0b0721-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.790 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:29:44.790 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:29:44.790 DEBUG:boto:Method: GET 2024-04-16T18:29:44.790 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:29:44.790 DEBUG:boto:Data: 2024-04-16T18:29:44.790 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.790 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.790 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.791 DEBUG:boto:Params: {} 2024-04-16T18:29:44.791 DEBUG:boto:Token: None 2024-04-16T18:29:44.791 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj116 2024-04-16T18:29:44.791 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uvphvy1h53SK0XXJ5ofFgX0k26Q= 2024-04-16T18:29:44.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uvphvy1h53SK0XXJ5ofFgX0k26Q=', 'Content-Length': '0'} 2024-04-16T18:29:44.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000423121e1468b3887-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.834 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.835 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.835 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.835 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.835 DEBUG:boto:Method: GET 2024-04-16T18:29:44.835 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:29:44.835 DEBUG:boto:Data: 2024-04-16T18:29:44.835 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.835 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.835 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.835 DEBUG:boto:Params: {} 2024-04-16T18:29:44.835 DEBUG:boto:Token: None 2024-04-16T18:29:44.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3e2365ee1396c3a-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.840 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.840 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.840 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.840 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.840 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:29:44.840 DEBUG:boto:Method: GET 2024-04-16T18:29:44.841 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:29:44.841 DEBUG:boto:Data: 2024-04-16T18:29:44.841 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.841 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.841 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.841 DEBUG:boto:Params: {} 2024-04-16T18:29:44.841 DEBUG:boto:Token: None 2024-04-16T18:29:44.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.848 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3545412fd69d238-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.848 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.848 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:29:44.848 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.848 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.849 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.849 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.849 DEBUG:boto:Method: GET 2024-04-16T18:29:44.849 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.849 DEBUG:boto:Data: 2024-04-16T18:29:44.849 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.849 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.849 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.849 DEBUG:boto:Params: {} 2024-04-16T18:29:44.849 DEBUG:boto:Token: None 2024-04-16T18:29:44.849 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.850 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20= 2024-04-16T18:29:44.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20=', 'Content-Length': '0'} 2024-04-16T18:29:44.853 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ae6726db32dc83a-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.894 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.894 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.894 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.895 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.895 DEBUG:boto:Method: GET 2024-04-16T18:29:44.895 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.895 DEBUG:boto:Data: 2024-04-16T18:29:44.895 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.895 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.895 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.895 DEBUG:boto:Params: {} 2024-04-16T18:29:44.895 DEBUG:boto:Token: None 2024-04-16T18:29:44.895 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20= 2024-04-16T18:29:44.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20=', 'Content-Length': '0'} 2024-04-16T18:29:44.898 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1afa1179c364a54-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.899 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.899 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.899 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.899 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.899 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.899 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.899 DEBUG:boto:Data: 2024-04-16T18:29:44.899 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.899 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.900 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.900 DEBUG:boto:Params: {} 2024-04-16T18:29:44.900 DEBUG:boto:Token: None 2024-04-16T18:29:44.900 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.900 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tyiHtMxFhNNDjnJI+0vZwIFvCNM= 2024-04-16T18:29:44.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tyiHtMxFhNNDjnJI+0vZwIFvCNM=', 'Content-Length': '0'} 2024-04-16T18:29:44.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6c1a477cb266fdf-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.903 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.903 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.904 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.904 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.904 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.904 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.904 DEBUG:boto:Data: 2024-04-16T18:29:44.904 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.904 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.904 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.904 DEBUG:boto:Params: {} 2024-04-16T18:29:44.904 DEBUG:boto:Token: None 2024-04-16T18:29:44.904 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.905 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tyiHtMxFhNNDjnJI+0vZwIFvCNM= 2024-04-16T18:29:44.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tyiHtMxFhNNDjnJI+0vZwIFvCNM=', 'Content-Length': '0'} 2024-04-16T18:29:44.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee8759c23611fee1-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.907 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:29:44.908 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.908 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.908 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.908 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.908 DEBUG:boto:Method: GET 2024-04-16T18:29:44.908 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.908 DEBUG:boto:Data: 2024-04-16T18:29:44.908 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.908 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.908 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.908 DEBUG:boto:Params: {} 2024-04-16T18:29:44.909 DEBUG:boto:Token: None 2024-04-16T18:29:44.909 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20= 2024-04-16T18:29:44.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20=', 'Content-Length': '0'} 2024-04-16T18:29:44.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b53ef9bda03135-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.912 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.912 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.912 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.912 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.912 DEBUG:boto:Method: GET 2024-04-16T18:29:44.912 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.912 DEBUG:boto:Data: 2024-04-16T18:29:44.913 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.913 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.913 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.913 DEBUG:boto:Params: {} 2024-04-16T18:29:44.913 DEBUG:boto:Token: None 2024-04-16T18:29:44.913 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:29:44.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20= 2024-04-16T18:29:44.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hBy+fugABPUxwSvtfDCh9x7/z20=', 'Content-Length': '0'} 2024-04-16T18:29:44.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b0e84725ffa96a7-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.958 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.958 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.959 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:44.959 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:44.959 DEBUG:boto:Method: GET 2024-04-16T18:29:44.959 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:29:44.959 DEBUG:boto:Data: 2024-04-16T18:29:44.959 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.959 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.959 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.959 DEBUG:boto:Params: {} 2024-04-16T18:29:44.959 DEBUG:boto:Token: None 2024-04-16T18:29:44.959 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.960 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078431b8f1a81e7ef-00661ec398-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.963 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.964 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:44.964 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:44.964 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:44.964 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:44.964 DEBUG:boto:Method: GET 2024-04-16T18:29:44.964 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:29:44.964 DEBUG:boto:Data: 2024-04-16T18:29:44.964 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.964 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.964 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.964 DEBUG:boto:Params: {} 2024-04-16T18:29:44.965 DEBUG:boto:Token: None 2024-04-16T18:29:44.965 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/ 2024-04-16T18:29:44.965 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI= 2024-04-16T18:29:44.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LKbwRQ9Bv7JIKRk5cJVf3tWvAXI=', 'Content-Length': '0'} 2024-04-16T18:29:44.971 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000111fd68b3dfdba0a-00661ec398-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.971 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:44.971 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.972 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.972 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.972 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:29:44.972 DEBUG:boto:Data: 2024-04-16T18:29:44.972 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.972 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.972 DEBUG:boto:Params: {} 2024-04-16T18:29:44.972 DEBUG:boto:Token: None 2024-04-16T18:29:44.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124 2024-04-16T18:29:44.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jf2YEfn4b0Vn0x80Q3Zw5lDboZs= 2024-04-16T18:29:44.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jf2YEfn4b0Vn0x80Q3Zw5lDboZs=', 'Content-Length': '0'} 2024-04-16T18:29:44.975 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c91d63202c5380f-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.976 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.976 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.976 DEBUG:boto:Method: HEAD 2024-04-16T18:29:44.976 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:29:44.976 DEBUG:boto:Data: 2024-04-16T18:29:44.976 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.976 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:44.976 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.976 DEBUG:boto:Params: {} 2024-04-16T18:29:44.976 DEBUG:boto:Token: None 2024-04-16T18:29:44.976 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124 2024-04-16T18:29:44.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jf2YEfn4b0Vn0x80Q3Zw5lDboZs= 2024-04-16T18:29:44.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jf2YEfn4b0Vn0x80Q3Zw5lDboZs=', 'Content-Length': '0'} 2024-04-16T18:29:44.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb096d10e38d0d8c-00661ec398-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:44.980 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:29:44.980 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:44.980 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:44.980 DEBUG:boto:Method: GET 2024-04-16T18:29:44.980 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:29:44.980 DEBUG:boto:Data: 2024-04-16T18:29:44.980 DEBUG:boto:Headers: {} 2024-04-16T18:29:44.980 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:44.980 DEBUG:boto:Port: 8000 2024-04-16T18:29:44.981 DEBUG:boto:Params: {} 2024-04-16T18:29:44.981 DEBUG:boto:Token: None 2024-04-16T18:29:44.981 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:44 GMT /lybtug-52/obj124 2024-04-16T18:29:44.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hz3SGl1lkThgAhQYb9GfUj27MZU= 2024-04-16T18:29:44.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:44 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hz3SGl1lkThgAhQYb9GfUj27MZU=', 'Content-Length': '0'} 2024-04-16T18:29:44.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000742b5193bf7c2c-00661ec398-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:44 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.030 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:29:45.030 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:29:45.031 DEBUG:boto:Method: GET 2024-04-16T18:29:45.031 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:29:45.031 DEBUG:boto:Data: 2024-04-16T18:29:45.031 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.031 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.031 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.031 DEBUG:boto:Params: {} 2024-04-16T18:29:45.031 DEBUG:boto:Token: None 2024-04-16T18:29:45.031 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj124 2024-04-16T18:29:45.031 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ruYHShqq8Ee0+PezggMNeqEqSiM= 2024-04-16T18:29:45.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ruYHShqq8Ee0+PezggMNeqEqSiM=', 'Content-Length': '0'} 2024-04-16T18:29:45.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008742d9c2cec3d001-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.035 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.035 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.035 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:45.035 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:45.035 DEBUG:boto:Method: GET 2024-04-16T18:29:45.035 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:29:45.035 DEBUG:boto:Data: 2024-04-16T18:29:45.035 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.035 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.036 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.036 DEBUG:boto:Params: {} 2024-04-16T18:29:45.036 DEBUG:boto:Token: None 2024-04-16T18:29:45.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.036 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000471be6cb552be736-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.040 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.041 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.041 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.041 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:45.041 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:29:45.041 DEBUG:boto:Method: GET 2024-04-16T18:29:45.041 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:29:45.041 DEBUG:boto:Data: 2024-04-16T18:29:45.041 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.041 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.041 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.041 DEBUG:boto:Params: {} 2024-04-16T18:29:45.042 DEBUG:boto:Token: None 2024-04-16T18:29:45.042 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.042 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.048 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ecbe21d3ba4d644-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.090 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:29:45.091 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.091 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.091 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.091 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.091 DEBUG:boto:Method: GET 2024-04-16T18:29:45.091 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.091 DEBUG:boto:Data: 2024-04-16T18:29:45.091 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.092 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.092 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.092 DEBUG:boto:Params: {} 2024-04-16T18:29:45.092 DEBUG:boto:Token: None 2024-04-16T18:29:45.092 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8= 2024-04-16T18:29:45.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8=', 'Content-Length': '0'} 2024-04-16T18:29:45.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d120c4b03844aec-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.096 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.096 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.096 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.096 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.096 DEBUG:boto:Method: GET 2024-04-16T18:29:45.096 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.096 DEBUG:boto:Data: 2024-04-16T18:29:45.096 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.096 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.096 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.096 DEBUG:boto:Params: {} 2024-04-16T18:29:45.097 DEBUG:boto:Token: None 2024-04-16T18:29:45.097 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8= 2024-04-16T18:29:45.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8=', 'Content-Length': '0'} 2024-04-16T18:29:45.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b795581f7d46611a-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.100 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.100 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.100 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.100 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.100 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.101 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.101 DEBUG:boto:Data: 2024-04-16T18:29:45.101 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.101 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.101 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.101 DEBUG:boto:Params: {} 2024-04-16T18:29:45.101 DEBUG:boto:Token: None 2024-04-16T18:29:45.101 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.101 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3Agaj6jDbOXzPdXmOadiZjtkCrI= 2024-04-16T18:29:45.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3Agaj6jDbOXzPdXmOadiZjtkCrI=', 'Content-Length': '0'} 2024-04-16T18:29:45.104 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000619cb122e5f187d6-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.104 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.104 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.104 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.104 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.105 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.105 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.105 DEBUG:boto:Data: 2024-04-16T18:29:45.105 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.105 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.105 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.105 DEBUG:boto:Params: {} 2024-04-16T18:29:45.105 DEBUG:boto:Token: None 2024-04-16T18:29:45.105 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.105 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3Agaj6jDbOXzPdXmOadiZjtkCrI= 2024-04-16T18:29:45.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3Agaj6jDbOXzPdXmOadiZjtkCrI=', 'Content-Length': '0'} 2024-04-16T18:29:45.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000892094ffcfac288d-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.108 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:29:45.109 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.109 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.109 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.109 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.109 DEBUG:boto:Method: GET 2024-04-16T18:29:45.109 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.109 DEBUG:boto:Data: 2024-04-16T18:29:45.109 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.109 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.109 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.109 DEBUG:boto:Params: {} 2024-04-16T18:29:45.110 DEBUG:boto:Token: None 2024-04-16T18:29:45.110 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.110 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8= 2024-04-16T18:29:45.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8=', 'Content-Length': '0'} 2024-04-16T18:29:45.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba6b39a9c183758c-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.154 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.154 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.155 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.155 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.155 DEBUG:boto:Method: GET 2024-04-16T18:29:45.155 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.155 DEBUG:boto:Data: 2024-04-16T18:29:45.155 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.155 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.155 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.155 DEBUG:boto:Params: {} 2024-04-16T18:29:45.155 DEBUG:boto:Token: None 2024-04-16T18:29:45.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:29:45.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8= 2024-04-16T18:29:45.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5B/mdjormZFocfttNx6A+Y6JKf8=', 'Content-Length': '0'} 2024-04-16T18:29:45.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096432516cb65e352-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.159 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.159 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.159 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.159 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.159 DEBUG:boto:Method: GET 2024-04-16T18:29:45.159 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:29:45.159 DEBUG:boto:Data: 2024-04-16T18:29:45.159 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.160 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.160 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.160 DEBUG:boto:Params: {} 2024-04-16T18:29:45.160 DEBUG:boto:Token: None 2024-04-16T18:29:45.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040a12c45283f63d5-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.164 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.164 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.164 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.165 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.165 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.165 DEBUG:boto:Method: GET 2024-04-16T18:29:45.165 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:29:45.165 DEBUG:boto:Data: 2024-04-16T18:29:45.165 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.165 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.165 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.165 DEBUG:boto:Params: {} 2024-04-16T18:29:45.165 DEBUG:boto:Token: None 2024-04-16T18:29:45.165 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.166 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.172 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d7a2d762a963ea5-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.214 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.215 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.215 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.215 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.215 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:29:45.215 DEBUG:boto:Data: 2024-04-16T18:29:45.215 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.215 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.215 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.215 DEBUG:boto:Params: {} 2024-04-16T18:29:45.215 DEBUG:boto:Token: None 2024-04-16T18:29:45.216 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145 2024-04-16T18:29:45.216 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vEDq/KEeMxLbaF+QtjeTwBkg/VY= 2024-04-16T18:29:45.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vEDq/KEeMxLbaF+QtjeTwBkg/VY=', 'Content-Length': '0'} 2024-04-16T18:29:45.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088d6747d4b9bfb08-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.219 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.219 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.219 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.219 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:29:45.219 DEBUG:boto:Data: 2024-04-16T18:29:45.219 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.219 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.219 DEBUG:boto:Params: {} 2024-04-16T18:29:45.220 DEBUG:boto:Token: None 2024-04-16T18:29:45.220 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145 2024-04-16T18:29:45.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vEDq/KEeMxLbaF+QtjeTwBkg/VY= 2024-04-16T18:29:45.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vEDq/KEeMxLbaF+QtjeTwBkg/VY=', 'Content-Length': '0'} 2024-04-16T18:29:45.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f94bcde5da4a0855-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.223 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:29:45.223 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.223 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.223 DEBUG:boto:Method: GET 2024-04-16T18:29:45.224 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:29:45.224 DEBUG:boto:Data: 2024-04-16T18:29:45.224 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.224 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.224 DEBUG:boto:Params: {} 2024-04-16T18:29:45.224 DEBUG:boto:Token: None 2024-04-16T18:29:45.224 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145 2024-04-16T18:29:45.224 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:is6V7HhJYqe32KRyLKAs/A/3jRc= 2024-04-16T18:29:45.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:is6V7HhJYqe32KRyLKAs/A/3jRc=', 'Content-Length': '0'} 2024-04-16T18:29:45.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f281227dadfdfba8-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.227 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:29:45.227 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:29:45.228 DEBUG:boto:Method: GET 2024-04-16T18:29:45.228 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:29:45.228 DEBUG:boto:Data: 2024-04-16T18:29:45.228 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.228 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.228 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.228 DEBUG:boto:Params: {} 2024-04-16T18:29:45.228 DEBUG:boto:Token: None 2024-04-16T18:29:45.228 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj145 2024-04-16T18:29:45.229 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:is6V7HhJYqe32KRyLKAs/A/3jRc= 2024-04-16T18:29:45.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:is6V7HhJYqe32KRyLKAs/A/3jRc=', 'Content-Length': '0'} 2024-04-16T18:29:45.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000042e5e7e6fc25201-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.232 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.232 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.232 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.232 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.232 DEBUG:boto:Method: GET 2024-04-16T18:29:45.232 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:29:45.232 DEBUG:boto:Data: 2024-04-16T18:29:45.232 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.232 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.233 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.233 DEBUG:boto:Params: {} 2024-04-16T18:29:45.233 DEBUG:boto:Token: None 2024-04-16T18:29:45.233 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.233 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.237 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020d28d893b4aa990-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.278 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.279 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.279 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.279 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.279 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:29:45.279 DEBUG:boto:Method: GET 2024-04-16T18:29:45.279 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:29:45.279 DEBUG:boto:Data: 2024-04-16T18:29:45.279 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.279 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.279 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.279 DEBUG:boto:Params: {} 2024-04-16T18:29:45.279 DEBUG:boto:Token: None 2024-04-16T18:29:45.280 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.286 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8622c1d28f6f569-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.286 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.286 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2024-04-16T18:29:45.287 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.287 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.287 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.287 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.287 DEBUG:boto:Method: GET 2024-04-16T18:29:45.287 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.287 DEBUG:boto:Data: 2024-04-16T18:29:45.287 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.287 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.287 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.287 DEBUG:boto:Params: {} 2024-04-16T18:29:45.288 DEBUG:boto:Token: None 2024-04-16T18:29:45.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es= 2024-04-16T18:29:45.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es=', 'Content-Length': '0'} 2024-04-16T18:29:45.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a80a3e2550c59f3f-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.291 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.291 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.292 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.292 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.292 DEBUG:boto:Method: GET 2024-04-16T18:29:45.292 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.292 DEBUG:boto:Data: 2024-04-16T18:29:45.292 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.292 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.292 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.292 DEBUG:boto:Params: {} 2024-04-16T18:29:45.292 DEBUG:boto:Token: None 2024-04-16T18:29:45.293 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.293 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es= 2024-04-16T18:29:45.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es=', 'Content-Length': '0'} 2024-04-16T18:29:45.296 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d6fd471dc0b7773-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.338 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.338 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.339 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.339 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.339 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.339 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.339 DEBUG:boto:Data: 2024-04-16T18:29:45.339 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.339 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.339 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.339 DEBUG:boto:Params: {} 2024-04-16T18:29:45.339 DEBUG:boto:Token: None 2024-04-16T18:29:45.339 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QSpKP5MGcfsB5e27Lfx15fJcOdM= 2024-04-16T18:29:45.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QSpKP5MGcfsB5e27Lfx15fJcOdM=', 'Content-Length': '0'} 2024-04-16T18:29:45.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b2a8a8fdf347d9c-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.343 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.343 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.343 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.343 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.343 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.343 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.343 DEBUG:boto:Data: 2024-04-16T18:29:45.343 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.344 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.344 DEBUG:boto:Params: {} 2024-04-16T18:29:45.344 DEBUG:boto:Token: None 2024-04-16T18:29:45.344 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QSpKP5MGcfsB5e27Lfx15fJcOdM= 2024-04-16T18:29:45.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QSpKP5MGcfsB5e27Lfx15fJcOdM=', 'Content-Length': '0'} 2024-04-16T18:29:45.348 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1431dfb75c0ce27-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2024-04-16T18:29:45.348 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.348 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.348 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.348 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.348 DEBUG:boto:Method: GET 2024-04-16T18:29:45.348 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.348 DEBUG:boto:Data: 2024-04-16T18:29:45.349 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.349 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.349 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.349 DEBUG:boto:Params: {} 2024-04-16T18:29:45.349 DEBUG:boto:Token: None 2024-04-16T18:29:45.349 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es= 2024-04-16T18:29:45.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es=', 'Content-Length': '0'} 2024-04-16T18:29:45.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003aa720a0fa0290ff-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.352 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.352 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.352 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.353 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.353 DEBUG:boto:Method: GET 2024-04-16T18:29:45.353 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.353 DEBUG:boto:Data: 2024-04-16T18:29:45.353 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.353 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.353 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.353 DEBUG:boto:Params: {} 2024-04-16T18:29:45.353 DEBUG:boto:Token: None 2024-04-16T18:29:45.353 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:29:45.353 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es= 2024-04-16T18:29:45.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:waYTv+PBUG+kbcosVfeqSWxC8es=', 'Content-Length': '0'} 2024-04-16T18:29:45.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e650fa2a0375f5b-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.357 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.357 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.357 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.357 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.357 DEBUG:boto:Method: GET 2024-04-16T18:29:45.357 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:29:45.357 DEBUG:boto:Data: 2024-04-16T18:29:45.357 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.357 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.357 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.358 DEBUG:boto:Params: {} 2024-04-16T18:29:45.358 DEBUG:boto:Token: None 2024-04-16T18:29:45.358 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.358 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.362 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030956f56032be5fd-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.402 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.402 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.403 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.403 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.403 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.403 DEBUG:boto:Method: GET 2024-04-16T18:29:45.403 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:29:45.403 DEBUG:boto:Data: 2024-04-16T18:29:45.403 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.403 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.403 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.403 DEBUG:boto:Params: {} 2024-04-16T18:29:45.404 DEBUG:boto:Token: None 2024-04-16T18:29:45.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbc419f95ffda088-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.411 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.411 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.411 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.411 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.412 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:29:45.412 DEBUG:boto:Data: 2024-04-16T18:29:45.412 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.412 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.412 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.412 DEBUG:boto:Params: {} 2024-04-16T18:29:45.412 DEBUG:boto:Token: None 2024-04-16T18:29:45.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148 2024-04-16T18:29:45.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:giMRmOn+2ehhJWB90vjdkCLg2aY= 2024-04-16T18:29:45.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:giMRmOn+2ehhJWB90vjdkCLg2aY=', 'Content-Length': '0'} 2024-04-16T18:29:45.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7c5fbd26f1a4c0a-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.416 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.416 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.416 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.416 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:29:45.416 DEBUG:boto:Data: 2024-04-16T18:29:45.416 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.416 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.417 DEBUG:boto:Params: {} 2024-04-16T18:29:45.417 DEBUG:boto:Token: None 2024-04-16T18:29:45.417 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148 2024-04-16T18:29:45.417 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:giMRmOn+2ehhJWB90vjdkCLg2aY= 2024-04-16T18:29:45.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:giMRmOn+2ehhJWB90vjdkCLg2aY=', 'Content-Length': '0'} 2024-04-16T18:29:45.421 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f349bcb38ee43212-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.421 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2024-04-16T18:29:45.421 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.421 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.421 DEBUG:boto:Method: GET 2024-04-16T18:29:45.421 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:29:45.421 DEBUG:boto:Data: 2024-04-16T18:29:45.421 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.422 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.422 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.422 DEBUG:boto:Params: {} 2024-04-16T18:29:45.422 DEBUG:boto:Token: None 2024-04-16T18:29:45.422 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148 2024-04-16T18:29:45.422 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Y+Sh7yCQD+9jIUqSGky0bR2qeLk= 2024-04-16T18:29:45.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Y+Sh7yCQD+9jIUqSGky0bR2qeLk=', 'Content-Length': '0'} 2024-04-16T18:29:45.425 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9512451370d5166-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.425 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:29:45.425 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:29:45.425 DEBUG:boto:Method: GET 2024-04-16T18:29:45.426 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:29:45.426 DEBUG:boto:Data: 2024-04-16T18:29:45.426 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.426 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.426 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.426 DEBUG:boto:Params: {} 2024-04-16T18:29:45.426 DEBUG:boto:Token: None 2024-04-16T18:29:45.426 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj148 2024-04-16T18:29:45.426 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Y+Sh7yCQD+9jIUqSGky0bR2qeLk= 2024-04-16T18:29:45.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Y+Sh7yCQD+9jIUqSGky0bR2qeLk=', 'Content-Length': '0'} 2024-04-16T18:29:45.429 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a440bfdb1ce73bf9-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.470 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.470 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.471 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.471 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.471 DEBUG:boto:Method: GET 2024-04-16T18:29:45.471 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:29:45.471 DEBUG:boto:Data: 2024-04-16T18:29:45.471 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.471 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.471 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.471 DEBUG:boto:Params: {} 2024-04-16T18:29:45.471 DEBUG:boto:Token: None 2024-04-16T18:29:45.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6407fe7501b899d-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.476 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.476 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.476 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.476 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.476 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:29:45.477 DEBUG:boto:Method: GET 2024-04-16T18:29:45.477 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:29:45.477 DEBUG:boto:Data: 2024-04-16T18:29:45.477 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.477 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.477 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.477 DEBUG:boto:Params: {} 2024-04-16T18:29:45.477 DEBUG:boto:Token: None 2024-04-16T18:29:45.477 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.477 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f5ed35c9793184e-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.484 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.484 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:29:45.484 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.484 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.485 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.485 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.485 DEBUG:boto:Method: GET 2024-04-16T18:29:45.485 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.485 DEBUG:boto:Data: 2024-04-16T18:29:45.485 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.485 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.485 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.485 DEBUG:boto:Params: {} 2024-04-16T18:29:45.485 DEBUG:boto:Token: None 2024-04-16T18:29:45.485 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.486 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE= 2024-04-16T18:29:45.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE=', 'Content-Length': '0'} 2024-04-16T18:29:45.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007744b8352422628c-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.530 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.530 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.531 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.531 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.531 DEBUG:boto:Method: GET 2024-04-16T18:29:45.531 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.531 DEBUG:boto:Data: 2024-04-16T18:29:45.531 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.531 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.531 DEBUG:boto:Params: {} 2024-04-16T18:29:45.531 DEBUG:boto:Token: None 2024-04-16T18:29:45.531 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.532 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE= 2024-04-16T18:29:45.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE=', 'Content-Length': '0'} 2024-04-16T18:29:45.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004143450f49c9ecdf-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.535 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.535 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.535 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.535 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.535 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.535 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.535 DEBUG:boto:Data: 2024-04-16T18:29:45.535 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.536 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.536 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.536 DEBUG:boto:Params: {} 2024-04-16T18:29:45.536 DEBUG:boto:Token: None 2024-04-16T18:29:45.536 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BSI5CDLW+vJcyOAY/g8os7Qgvsk= 2024-04-16T18:29:45.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BSI5CDLW+vJcyOAY/g8os7Qgvsk=', 'Content-Length': '0'} 2024-04-16T18:29:45.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff2c49b06c867665-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.540 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.540 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.540 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.540 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.540 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.540 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.540 DEBUG:boto:Data: 2024-04-16T18:29:45.540 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.540 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.540 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.540 DEBUG:boto:Params: {} 2024-04-16T18:29:45.541 DEBUG:boto:Token: None 2024-04-16T18:29:45.541 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.541 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BSI5CDLW+vJcyOAY/g8os7Qgvsk= 2024-04-16T18:29:45.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BSI5CDLW+vJcyOAY/g8os7Qgvsk=', 'Content-Length': '0'} 2024-04-16T18:29:45.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c60bda9d50581f5-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:29:45.544 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.544 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.544 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.544 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.545 DEBUG:boto:Method: GET 2024-04-16T18:29:45.545 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.545 DEBUG:boto:Data: 2024-04-16T18:29:45.545 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.545 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.545 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.545 DEBUG:boto:Params: {} 2024-04-16T18:29:45.545 DEBUG:boto:Token: None 2024-04-16T18:29:45.545 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.545 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE= 2024-04-16T18:29:45.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE=', 'Content-Length': '0'} 2024-04-16T18:29:45.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad9e7ccec9b0c01d-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.549 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.549 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.550 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.550 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.550 DEBUG:boto:Method: GET 2024-04-16T18:29:45.550 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.550 DEBUG:boto:Data: 2024-04-16T18:29:45.550 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.550 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.550 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.550 DEBUG:boto:Params: {} 2024-04-16T18:29:45.550 DEBUG:boto:Token: None 2024-04-16T18:29:45.551 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:29:45.551 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE= 2024-04-16T18:29:45.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/tud+y4wlPhBmTUA09p1WhMbBE=', 'Content-Length': '0'} 2024-04-16T18:29:45.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045256da423edeaed-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.594 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.594 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.595 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.595 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.595 DEBUG:boto:Method: GET 2024-04-16T18:29:45.595 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:29:45.595 DEBUG:boto:Data: 2024-04-16T18:29:45.595 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.595 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.595 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.595 DEBUG:boto:Params: {} 2024-04-16T18:29:45.595 DEBUG:boto:Token: None 2024-04-16T18:29:45.595 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.599 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007951a47e58c80cc0-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.600 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.600 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.600 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.600 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.600 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.600 DEBUG:boto:Method: GET 2024-04-16T18:29:45.600 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:29:45.600 DEBUG:boto:Data: 2024-04-16T18:29:45.601 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.601 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.601 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.601 DEBUG:boto:Params: {} 2024-04-16T18:29:45.601 DEBUG:boto:Token: None 2024-04-16T18:29:45.601 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.601 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c81e43ab55636e2f-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.608 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.608 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.608 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.608 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.608 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:29:45.608 DEBUG:boto:Data: 2024-04-16T18:29:45.609 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.609 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.609 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.609 DEBUG:boto:Params: {} 2024-04-16T18:29:45.609 DEBUG:boto:Token: None 2024-04-16T18:29:45.609 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155 2024-04-16T18:29:45.609 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sSYQsB90RDeAYrbdMyXRbAIrcbs= 2024-04-16T18:29:45.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sSYQsB90RDeAYrbdMyXRbAIrcbs=', 'Content-Length': '0'} 2024-04-16T18:29:45.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea3f5f74a42df796-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.612 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.612 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.612 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.612 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:29:45.613 DEBUG:boto:Data: 2024-04-16T18:29:45.613 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.613 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.613 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.613 DEBUG:boto:Params: {} 2024-04-16T18:29:45.613 DEBUG:boto:Token: None 2024-04-16T18:29:45.613 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155 2024-04-16T18:29:45.613 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sSYQsB90RDeAYrbdMyXRbAIrcbs= 2024-04-16T18:29:45.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sSYQsB90RDeAYrbdMyXRbAIrcbs=', 'Content-Length': '0'} 2024-04-16T18:29:45.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c50e512549c769b-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.616 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:29:45.616 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.616 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.616 DEBUG:boto:Method: GET 2024-04-16T18:29:45.616 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:29:45.616 DEBUG:boto:Data: 2024-04-16T18:29:45.617 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.617 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.617 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.617 DEBUG:boto:Params: {} 2024-04-16T18:29:45.617 DEBUG:boto:Token: None 2024-04-16T18:29:45.617 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155 2024-04-16T18:29:45.617 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fD1hjDgBXWvdsjsJ/9a0OdDQX04= 2024-04-16T18:29:45.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fD1hjDgBXWvdsjsJ/9a0OdDQX04=', 'Content-Length': '0'} 2024-04-16T18:29:45.620 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080dc597001787a18-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.662 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:29:45.662 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:29:45.663 DEBUG:boto:Method: GET 2024-04-16T18:29:45.663 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:29:45.663 DEBUG:boto:Data: 2024-04-16T18:29:45.663 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.663 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.663 DEBUG:boto:Params: {} 2024-04-16T18:29:45.663 DEBUG:boto:Token: None 2024-04-16T18:29:45.663 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj155 2024-04-16T18:29:45.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fD1hjDgBXWvdsjsJ/9a0OdDQX04= 2024-04-16T18:29:45.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fD1hjDgBXWvdsjsJ/9a0OdDQX04=', 'Content-Length': '0'} 2024-04-16T18:29:45.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000efe94819b9582f6-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.667 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.667 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.667 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.667 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.667 DEBUG:boto:Method: GET 2024-04-16T18:29:45.667 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:29:45.667 DEBUG:boto:Data: 2024-04-16T18:29:45.667 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.667 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.667 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.668 DEBUG:boto:Params: {} 2024-04-16T18:29:45.668 DEBUG:boto:Token: None 2024-04-16T18:29:45.668 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe24b999c19b4b56-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.672 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.672 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.672 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.672 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.672 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:29:45.673 DEBUG:boto:Method: GET 2024-04-16T18:29:45.673 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:29:45.673 DEBUG:boto:Data: 2024-04-16T18:29:45.673 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.673 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.673 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.673 DEBUG:boto:Params: {} 2024-04-16T18:29:45.673 DEBUG:boto:Token: None 2024-04-16T18:29:45.673 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.673 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.680 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061b8ae68c0a3902f-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.722 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:29:45.723 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.723 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.723 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.723 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.723 DEBUG:boto:Method: GET 2024-04-16T18:29:45.723 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.723 DEBUG:boto:Data: 2024-04-16T18:29:45.723 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.723 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.723 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.724 DEBUG:boto:Params: {} 2024-04-16T18:29:45.724 DEBUG:boto:Token: None 2024-04-16T18:29:45.724 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI= 2024-04-16T18:29:45.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI=', 'Content-Length': '0'} 2024-04-16T18:29:45.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002042f790980b25a8-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.727 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.727 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.728 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.728 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.728 DEBUG:boto:Method: GET 2024-04-16T18:29:45.728 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.728 DEBUG:boto:Data: 2024-04-16T18:29:45.728 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.728 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.728 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.728 DEBUG:boto:Params: {} 2024-04-16T18:29:45.728 DEBUG:boto:Token: None 2024-04-16T18:29:45.728 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI= 2024-04-16T18:29:45.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI=', 'Content-Length': '0'} 2024-04-16T18:29:45.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000131f89ecaab23920-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.732 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.732 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.732 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.732 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.732 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.732 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.732 DEBUG:boto:Data: 2024-04-16T18:29:45.732 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.732 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.733 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.733 DEBUG:boto:Params: {} 2024-04-16T18:29:45.733 DEBUG:boto:Token: None 2024-04-16T18:29:45.733 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1kxWSYeiUPbuxlR//4DUWxz8PFY= 2024-04-16T18:29:45.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1kxWSYeiUPbuxlR//4DUWxz8PFY=', 'Content-Length': '0'} 2024-04-16T18:29:45.736 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fc5e55d5393775b-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.736 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.736 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.736 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.736 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.736 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.736 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.737 DEBUG:boto:Data: 2024-04-16T18:29:45.737 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.737 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.737 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.737 DEBUG:boto:Params: {} 2024-04-16T18:29:45.737 DEBUG:boto:Token: None 2024-04-16T18:29:45.737 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.737 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1kxWSYeiUPbuxlR//4DUWxz8PFY= 2024-04-16T18:29:45.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1kxWSYeiUPbuxlR//4DUWxz8PFY=', 'Content-Length': '0'} 2024-04-16T18:29:45.741 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcd45c43409c04e4-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.741 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:29:45.741 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.741 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.741 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.741 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.741 DEBUG:boto:Method: GET 2024-04-16T18:29:45.741 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.742 DEBUG:boto:Data: 2024-04-16T18:29:45.742 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.742 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.742 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.742 DEBUG:boto:Params: {} 2024-04-16T18:29:45.742 DEBUG:boto:Token: None 2024-04-16T18:29:45.742 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.742 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI= 2024-04-16T18:29:45.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI=', 'Content-Length': '0'} 2024-04-16T18:29:45.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002a79d710a049a09-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.786 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.786 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.787 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.787 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.787 DEBUG:boto:Method: GET 2024-04-16T18:29:45.787 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.787 DEBUG:boto:Data: 2024-04-16T18:29:45.787 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.787 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.787 DEBUG:boto:Params: {} 2024-04-16T18:29:45.787 DEBUG:boto:Token: None 2024-04-16T18:29:45.787 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:29:45.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI= 2024-04-16T18:29:45.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+n4E8SkJTmvh0OW1xzom/enjREI=', 'Content-Length': '0'} 2024-04-16T18:29:45.790 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000347ebeac083764c-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.791 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.791 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.791 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.791 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.791 DEBUG:boto:Method: GET 2024-04-16T18:29:45.791 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:29:45.791 DEBUG:boto:Data: 2024-04-16T18:29:45.791 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.792 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.792 DEBUG:boto:Params: {} 2024-04-16T18:29:45.792 DEBUG:boto:Token: None 2024-04-16T18:29:45.792 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000880376f616e29c2a-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.796 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.796 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.796 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.796 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.796 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.796 DEBUG:boto:Method: GET 2024-04-16T18:29:45.797 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:29:45.797 DEBUG:boto:Data: 2024-04-16T18:29:45.797 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.797 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.797 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.797 DEBUG:boto:Params: {} 2024-04-16T18:29:45.797 DEBUG:boto:Token: None 2024-04-16T18:29:45.797 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c48df58575d3485-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.846 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.847 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.847 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.847 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.847 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:29:45.847 DEBUG:boto:Data: 2024-04-16T18:29:45.847 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.847 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.847 DEBUG:boto:Params: {} 2024-04-16T18:29:45.847 DEBUG:boto:Token: None 2024-04-16T18:29:45.847 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17 2024-04-16T18:29:45.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nPoCrHOXLkHKuFdAtSHuiHOytdE= 2024-04-16T18:29:45.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nPoCrHOXLkHKuFdAtSHuiHOytdE=', 'Content-Length': '0'} 2024-04-16T18:29:45.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a9f1a32bc66191c-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.851 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.851 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.851 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.851 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:29:45.851 DEBUG:boto:Data: 2024-04-16T18:29:45.851 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.852 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.852 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.852 DEBUG:boto:Params: {} 2024-04-16T18:29:45.852 DEBUG:boto:Token: None 2024-04-16T18:29:45.852 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17 2024-04-16T18:29:45.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nPoCrHOXLkHKuFdAtSHuiHOytdE= 2024-04-16T18:29:45.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nPoCrHOXLkHKuFdAtSHuiHOytdE=', 'Content-Length': '0'} 2024-04-16T18:29:45.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002016ba3f67e0b76b-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.855 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:29:45.856 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.856 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.856 DEBUG:boto:Method: GET 2024-04-16T18:29:45.856 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:29:45.856 DEBUG:boto:Data: 2024-04-16T18:29:45.856 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.856 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.856 DEBUG:boto:Params: {} 2024-04-16T18:29:45.856 DEBUG:boto:Token: None 2024-04-16T18:29:45.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17 2024-04-16T18:29:45.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rpYPh3eeyrd9Bpcgg3ntv9EqOLI= 2024-04-16T18:29:45.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rpYPh3eeyrd9Bpcgg3ntv9EqOLI=', 'Content-Length': '0'} 2024-04-16T18:29:45.860 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afb76856256aec49-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.860 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:29:45.860 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:29:45.860 DEBUG:boto:Method: GET 2024-04-16T18:29:45.860 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:29:45.860 DEBUG:boto:Data: 2024-04-16T18:29:45.861 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.861 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.861 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.861 DEBUG:boto:Params: {} 2024-04-16T18:29:45.861 DEBUG:boto:Token: None 2024-04-16T18:29:45.861 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj17 2024-04-16T18:29:45.861 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rpYPh3eeyrd9Bpcgg3ntv9EqOLI= 2024-04-16T18:29:45.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rpYPh3eeyrd9Bpcgg3ntv9EqOLI=', 'Content-Length': '0'} 2024-04-16T18:29:45.864 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9506f17fa1cd455-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.864 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.864 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.864 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.865 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.865 DEBUG:boto:Method: GET 2024-04-16T18:29:45.865 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:29:45.865 DEBUG:boto:Data: 2024-04-16T18:29:45.865 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.865 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.865 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.865 DEBUG:boto:Params: {} 2024-04-16T18:29:45.865 DEBUG:boto:Token: None 2024-04-16T18:29:45.865 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.865 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.869 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb524406be9b12f3-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.910 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.911 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.911 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.911 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.911 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:29:45.911 DEBUG:boto:Method: GET 2024-04-16T18:29:45.911 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:29:45.911 DEBUG:boto:Data: 2024-04-16T18:29:45.911 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.911 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.912 DEBUG:boto:Params: {} 2024-04-16T18:29:45.912 DEBUG:boto:Token: None 2024-04-16T18:29:45.912 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.918 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a7bd449056d14ea-00661ec399-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.919 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:45.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2024-04-16T18:29:45.919 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:45.919 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:45.920 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.920 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.920 DEBUG:boto:Method: GET 2024-04-16T18:29:45.920 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.920 DEBUG:boto:Data: 2024-04-16T18:29:45.920 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.920 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.920 DEBUG:boto:Params: {} 2024-04-16T18:29:45.920 DEBUG:boto:Token: None 2024-04-16T18:29:45.921 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50= 2024-04-16T18:29:45.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50=', 'Content-Length': '0'} 2024-04-16T18:29:45.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b32a5066717bd94-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.924 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:45.924 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:45.924 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.924 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.924 DEBUG:boto:Method: GET 2024-04-16T18:29:45.925 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.925 DEBUG:boto:Data: 2024-04-16T18:29:45.925 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.925 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.925 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.925 DEBUG:boto:Params: {} 2024-04-16T18:29:45.925 DEBUG:boto:Token: None 2024-04-16T18:29:45.925 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.925 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50= 2024-04-16T18:29:45.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50=', 'Content-Length': '0'} 2024-04-16T18:29:45.929 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c63517467657e20-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.970 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:45.970 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:45.971 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.971 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.971 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.971 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.971 DEBUG:boto:Data: 2024-04-16T18:29:45.971 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.971 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.971 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.971 DEBUG:boto:Params: {} 2024-04-16T18:29:45.971 DEBUG:boto:Token: None 2024-04-16T18:29:45.971 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M24fewu8sDO25IRS7HBj+eri5fc= 2024-04-16T18:29:45.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M24fewu8sDO25IRS7HBj+eri5fc=', 'Content-Length': '0'} 2024-04-16T18:29:45.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064c6205f83d7e9b7-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.975 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:45.975 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:45.975 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.975 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.975 DEBUG:boto:Method: HEAD 2024-04-16T18:29:45.975 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.975 DEBUG:boto:Data: 2024-04-16T18:29:45.975 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.975 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.975 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.975 DEBUG:boto:Params: {} 2024-04-16T18:29:45.976 DEBUG:boto:Token: None 2024-04-16T18:29:45.976 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.976 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M24fewu8sDO25IRS7HBj+eri5fc= 2024-04-16T18:29:45.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M24fewu8sDO25IRS7HBj+eri5fc=', 'Content-Length': '0'} 2024-04-16T18:29:45.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b59cc5bdb898ae4-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2024-04-16T18:29:45.979 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:45.979 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:45.980 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.980 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.980 DEBUG:boto:Method: GET 2024-04-16T18:29:45.980 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.980 DEBUG:boto:Data: 2024-04-16T18:29:45.980 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.980 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.980 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.980 DEBUG:boto:Params: {} 2024-04-16T18:29:45.980 DEBUG:boto:Token: None 2024-04-16T18:29:45.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50= 2024-04-16T18:29:45.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50=', 'Content-Length': '0'} 2024-04-16T18:29:45.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a850632ac9fe239e-00661ec399-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.984 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:45.984 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:45.984 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.984 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.984 DEBUG:boto:Method: GET 2024-04-16T18:29:45.984 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.984 DEBUG:boto:Data: 2024-04-16T18:29:45.984 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.985 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:45.985 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.985 DEBUG:boto:Params: {} 2024-04-16T18:29:45.985 DEBUG:boto:Token: None 2024-04-16T18:29:45.985 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:29:45.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50= 2024-04-16T18:29:45.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JX5SgvNYJN/JsfUfii3GyAeWP50=', 'Content-Length': '0'} 2024-04-16T18:29:45.988 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b2f9a5bd7c168c8-00661ec399-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:45.988 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:45.989 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:45.989 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:45.989 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:45.989 DEBUG:boto:Method: GET 2024-04-16T18:29:45.989 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:29:45.989 DEBUG:boto:Data: 2024-04-16T18:29:45.989 DEBUG:boto:Headers: {} 2024-04-16T18:29:45.989 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:45.989 DEBUG:boto:Port: 8000 2024-04-16T18:29:45.989 DEBUG:boto:Params: {} 2024-04-16T18:29:45.989 DEBUG:boto:Token: None 2024-04-16T18:29:45.990 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:45 GMT /lybtug-52/ 2024-04-16T18:29:45.990 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng= 2024-04-16T18:29:45.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:45 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eKymYLqXDkSSTTY5UHRg3MsHsng=', 'Content-Length': '0'} 2024-04-16T18:29:45.994 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e38722e8ebf0cc39-00661ec399-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:45 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.038 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.039 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.039 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.039 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:46.039 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:46.039 DEBUG:boto:Method: GET 2024-04-16T18:29:46.039 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:29:46.039 DEBUG:boto:Data: 2024-04-16T18:29:46.039 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.039 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.039 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.040 DEBUG:boto:Params: {} 2024-04-16T18:29:46.040 DEBUG:boto:Token: None 2024-04-16T18:29:46.040 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017aad58ec2aa4243-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.046 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.047 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:46.047 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:46.047 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.047 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:29:46.047 DEBUG:boto:Data: 2024-04-16T18:29:46.047 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.047 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.047 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.047 DEBUG:boto:Params: {} 2024-04-16T18:29:46.047 DEBUG:boto:Token: None 2024-04-16T18:29:46.048 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj185 2024-04-16T18:29:46.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sNoCjnaQ7Q4NQUxjHO507pPn7PI= 2024-04-16T18:29:46.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sNoCjnaQ7Q4NQUxjHO507pPn7PI=', 'Content-Length': '0'} 2024-04-16T18:29:46.050 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa2f7bfc1e2fdf40-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.051 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:46.051 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:46.051 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.051 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:29:46.051 DEBUG:boto:Data: 2024-04-16T18:29:46.051 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.051 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.051 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.051 DEBUG:boto:Params: {} 2024-04-16T18:29:46.052 DEBUG:boto:Token: None 2024-04-16T18:29:46.052 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj185 2024-04-16T18:29:46.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sNoCjnaQ7Q4NQUxjHO507pPn7PI= 2024-04-16T18:29:46.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sNoCjnaQ7Q4NQUxjHO507pPn7PI=', 'Content-Length': '0'} 2024-04-16T18:29:46.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7e0395fa43023ac-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.055 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2024-04-16T18:29:46.055 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:46.055 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:46.055 DEBUG:boto:Method: GET 2024-04-16T18:29:46.056 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:29:46.056 DEBUG:boto:Data: 2024-04-16T18:29:46.056 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.056 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.056 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.056 DEBUG:boto:Params: {} 2024-04-16T18:29:46.056 DEBUG:boto:Token: None 2024-04-16T18:29:46.056 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj185 2024-04-16T18:29:46.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6SpmVyK/aQKdmzGB4eIC2yF2fB4= 2024-04-16T18:29:46.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6SpmVyK/aQKdmzGB4eIC2yF2fB4=', 'Content-Length': '0'} 2024-04-16T18:29:46.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001971ddb21f5cc8d9-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.060 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:29:46.060 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:29:46.060 DEBUG:boto:Method: GET 2024-04-16T18:29:46.060 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:29:46.060 DEBUG:boto:Data: 2024-04-16T18:29:46.060 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.060 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.060 DEBUG:boto:Params: {} 2024-04-16T18:29:46.060 DEBUG:boto:Token: None 2024-04-16T18:29:46.061 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj185 2024-04-16T18:29:46.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6SpmVyK/aQKdmzGB4eIC2yF2fB4= 2024-04-16T18:29:46.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6SpmVyK/aQKdmzGB4eIC2yF2fB4=', 'Content-Length': '0'} 2024-04-16T18:29:46.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000775a9490da36e795-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.106 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.106 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.107 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:46.107 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:46.107 DEBUG:boto:Method: GET 2024-04-16T18:29:46.107 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:29:46.107 DEBUG:boto:Data: 2024-04-16T18:29:46.107 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.107 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.107 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.107 DEBUG:boto:Params: {} 2024-04-16T18:29:46.107 DEBUG:boto:Token: None 2024-04-16T18:29:46.107 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002fcc789372d0d50-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.111 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.112 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.112 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.112 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:46.112 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:29:46.112 DEBUG:boto:Method: GET 2024-04-16T18:29:46.112 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:29:46.112 DEBUG:boto:Data: 2024-04-16T18:29:46.112 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.112 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.112 DEBUG:boto:Params: {} 2024-04-16T18:29:46.113 DEBUG:boto:Token: None 2024-04-16T18:29:46.113 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.113 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.119 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8be72a8bbbc187b-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.120 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:29:46.120 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.120 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.120 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.120 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.120 DEBUG:boto:Method: GET 2024-04-16T18:29:46.120 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.121 DEBUG:boto:Data: 2024-04-16T18:29:46.121 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.121 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.121 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.121 DEBUG:boto:Params: {} 2024-04-16T18:29:46.121 DEBUG:boto:Token: None 2024-04-16T18:29:46.121 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA= 2024-04-16T18:29:46.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA=', 'Content-Length': '0'} 2024-04-16T18:29:46.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089886a4e0fd6fd29-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.167 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.167 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.167 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.167 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.167 DEBUG:boto:Method: GET 2024-04-16T18:29:46.167 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.167 DEBUG:boto:Data: 2024-04-16T18:29:46.167 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.167 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.167 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.168 DEBUG:boto:Params: {} 2024-04-16T18:29:46.168 DEBUG:boto:Token: None 2024-04-16T18:29:46.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA= 2024-04-16T18:29:46.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA=', 'Content-Length': '0'} 2024-04-16T18:29:46.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6a1827790763946-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.172 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.172 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.172 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.172 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.172 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.172 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.172 DEBUG:boto:Data: 2024-04-16T18:29:46.172 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.172 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.172 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.173 DEBUG:boto:Params: {} 2024-04-16T18:29:46.173 DEBUG:boto:Token: None 2024-04-16T18:29:46.173 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.173 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JknkOZ4TUtqPzTb3lJCsr3kJuFs= 2024-04-16T18:29:46.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JknkOZ4TUtqPzTb3lJCsr3kJuFs=', 'Content-Length': '0'} 2024-04-16T18:29:46.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007de58bd8daec411b-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.176 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.176 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.176 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.177 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.177 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.177 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.177 DEBUG:boto:Data: 2024-04-16T18:29:46.177 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.177 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.177 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.177 DEBUG:boto:Params: {} 2024-04-16T18:29:46.177 DEBUG:boto:Token: None 2024-04-16T18:29:46.177 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JknkOZ4TUtqPzTb3lJCsr3kJuFs= 2024-04-16T18:29:46.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JknkOZ4TUtqPzTb3lJCsr3kJuFs=', 'Content-Length': '0'} 2024-04-16T18:29:46.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac0d4b9a898aa7d4-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.181 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:29:46.181 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.181 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.181 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.181 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.181 DEBUG:boto:Method: GET 2024-04-16T18:29:46.181 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.181 DEBUG:boto:Data: 2024-04-16T18:29:46.181 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.181 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.181 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.182 DEBUG:boto:Params: {} 2024-04-16T18:29:46.182 DEBUG:boto:Token: None 2024-04-16T18:29:46.182 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.182 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA= 2024-04-16T18:29:46.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA=', 'Content-Length': '0'} 2024-04-16T18:29:46.185 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cb949cb9e552545-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.186 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.186 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.186 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.186 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.186 DEBUG:boto:Method: GET 2024-04-16T18:29:46.186 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.186 DEBUG:boto:Data: 2024-04-16T18:29:46.186 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.186 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.186 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.186 DEBUG:boto:Params: {} 2024-04-16T18:29:46.187 DEBUG:boto:Token: None 2024-04-16T18:29:46.187 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:29:46.187 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA= 2024-04-16T18:29:46.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJg35wC1pPpMxJ1hiBhKRoaCcXA=', 'Content-Length': '0'} 2024-04-16T18:29:46.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6e4cebab3f0dff8-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.230 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.230 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.231 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.231 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.231 DEBUG:boto:Method: GET 2024-04-16T18:29:46.231 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:29:46.231 DEBUG:boto:Data: 2024-04-16T18:29:46.231 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.231 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.231 DEBUG:boto:Params: {} 2024-04-16T18:29:46.231 DEBUG:boto:Token: None 2024-04-16T18:29:46.231 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000452149e90472e5db-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.236 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.236 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.236 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.236 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.236 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.236 DEBUG:boto:Method: GET 2024-04-16T18:29:46.236 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:29:46.236 DEBUG:boto:Data: 2024-04-16T18:29:46.237 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.237 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.237 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.237 DEBUG:boto:Params: {} 2024-04-16T18:29:46.237 DEBUG:boto:Token: None 2024-04-16T18:29:46.237 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.237 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fdb17141a2435970-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.244 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.245 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.245 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.245 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.245 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:29:46.245 DEBUG:boto:Data: 2024-04-16T18:29:46.245 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.245 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.245 DEBUG:boto:Params: {} 2024-04-16T18:29:46.246 DEBUG:boto:Token: None 2024-04-16T18:29:46.246 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19 2024-04-16T18:29:46.246 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9g2mebb7iZE31CZnsQm5ZAEg15k= 2024-04-16T18:29:46.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9g2mebb7iZE31CZnsQm5ZAEg15k=', 'Content-Length': '0'} 2024-04-16T18:29:46.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de072d8e5a668aec-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.249 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.249 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.249 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.250 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:29:46.250 DEBUG:boto:Data: 2024-04-16T18:29:46.250 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.250 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.250 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.250 DEBUG:boto:Params: {} 2024-04-16T18:29:46.250 DEBUG:boto:Token: None 2024-04-16T18:29:46.250 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19 2024-04-16T18:29:46.250 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9g2mebb7iZE31CZnsQm5ZAEg15k= 2024-04-16T18:29:46.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9g2mebb7iZE31CZnsQm5ZAEg15k=', 'Content-Length': '0'} 2024-04-16T18:29:46.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033ad2c330dad980b-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.254 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:29:46.254 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.254 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.254 DEBUG:boto:Method: GET 2024-04-16T18:29:46.254 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:29:46.254 DEBUG:boto:Data: 2024-04-16T18:29:46.254 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.254 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.254 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.255 DEBUG:boto:Params: {} 2024-04-16T18:29:46.255 DEBUG:boto:Token: None 2024-04-16T18:29:46.255 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19 2024-04-16T18:29:46.255 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PiKTm/BhgfgfX1nH3LwdGdyUkaI= 2024-04-16T18:29:46.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PiKTm/BhgfgfX1nH3LwdGdyUkaI=', 'Content-Length': '0'} 2024-04-16T18:29:46.258 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092ddc5a2703faaa9-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.298 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:29:46.298 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:29:46.299 DEBUG:boto:Method: GET 2024-04-16T18:29:46.299 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:29:46.299 DEBUG:boto:Data: 2024-04-16T18:29:46.299 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.299 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.299 DEBUG:boto:Params: {} 2024-04-16T18:29:46.299 DEBUG:boto:Token: None 2024-04-16T18:29:46.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj19 2024-04-16T18:29:46.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PiKTm/BhgfgfX1nH3LwdGdyUkaI= 2024-04-16T18:29:46.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PiKTm/BhgfgfX1nH3LwdGdyUkaI=', 'Content-Length': '0'} 2024-04-16T18:29:46.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c49b023da6f7786e-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.303 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.303 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.303 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.303 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.303 DEBUG:boto:Method: GET 2024-04-16T18:29:46.303 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:29:46.303 DEBUG:boto:Data: 2024-04-16T18:29:46.303 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.303 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.303 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.303 DEBUG:boto:Params: {} 2024-04-16T18:29:46.304 DEBUG:boto:Token: None 2024-04-16T18:29:46.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004296358e1cd46ff1-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.308 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.308 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.308 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.309 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.309 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:29:46.309 DEBUG:boto:Method: GET 2024-04-16T18:29:46.309 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:29:46.309 DEBUG:boto:Data: 2024-04-16T18:29:46.309 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.309 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.309 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.309 DEBUG:boto:Params: {} 2024-04-16T18:29:46.309 DEBUG:boto:Token: None 2024-04-16T18:29:46.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.310 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.317 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1e27aa7503dee92-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.358 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.359 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:29:46.359 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.359 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.359 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.359 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.359 DEBUG:boto:Method: GET 2024-04-16T18:29:46.359 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.359 DEBUG:boto:Data: 2024-04-16T18:29:46.360 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.360 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.360 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.360 DEBUG:boto:Params: {} 2024-04-16T18:29:46.360 DEBUG:boto:Token: None 2024-04-16T18:29:46.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA= 2024-04-16T18:29:46.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA=', 'Content-Length': '0'} 2024-04-16T18:29:46.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4adbe97c2c3f885-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.363 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.364 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.364 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.364 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.364 DEBUG:boto:Method: GET 2024-04-16T18:29:46.364 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.364 DEBUG:boto:Data: 2024-04-16T18:29:46.364 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.364 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.364 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.364 DEBUG:boto:Params: {} 2024-04-16T18:29:46.365 DEBUG:boto:Token: None 2024-04-16T18:29:46.365 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.365 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA= 2024-04-16T18:29:46.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA=', 'Content-Length': '0'} 2024-04-16T18:29:46.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e99df74f5d4ce0a-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.368 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.368 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.368 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.368 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.368 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.368 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.368 DEBUG:boto:Data: 2024-04-16T18:29:46.368 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.369 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.369 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.369 DEBUG:boto:Params: {} 2024-04-16T18:29:46.369 DEBUG:boto:Token: None 2024-04-16T18:29:46.370 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.370 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5RvxsBdsk0BVSIFP8khAxBbRNzE= 2024-04-16T18:29:46.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5RvxsBdsk0BVSIFP8khAxBbRNzE=', 'Content-Length': '0'} 2024-04-16T18:29:46.373 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f904f4ef84ec90c-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.374 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.374 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.374 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.374 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.374 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.374 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.374 DEBUG:boto:Data: 2024-04-16T18:29:46.374 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.374 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.374 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.374 DEBUG:boto:Params: {} 2024-04-16T18:29:46.375 DEBUG:boto:Token: None 2024-04-16T18:29:46.375 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.375 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5RvxsBdsk0BVSIFP8khAxBbRNzE= 2024-04-16T18:29:46.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5RvxsBdsk0BVSIFP8khAxBbRNzE=', 'Content-Length': '0'} 2024-04-16T18:29:46.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee9c191ad9fbd9b1-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:29:46.378 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.378 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.378 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.378 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.378 DEBUG:boto:Method: GET 2024-04-16T18:29:46.378 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.378 DEBUG:boto:Data: 2024-04-16T18:29:46.378 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.379 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.379 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.379 DEBUG:boto:Params: {} 2024-04-16T18:29:46.379 DEBUG:boto:Token: None 2024-04-16T18:29:46.379 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.379 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA= 2024-04-16T18:29:46.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA=', 'Content-Length': '0'} 2024-04-16T18:29:46.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5cd7e1e34f2d7fd-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.382 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.382 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.382 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.382 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.382 DEBUG:boto:Method: GET 2024-04-16T18:29:46.383 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.383 DEBUG:boto:Data: 2024-04-16T18:29:46.383 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.383 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.383 DEBUG:boto:Params: {} 2024-04-16T18:29:46.383 DEBUG:boto:Token: None 2024-04-16T18:29:46.383 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:29:46.383 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA= 2024-04-16T18:29:46.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kQpCkIlbP04sRrny6P4ZpXOQ+dA=', 'Content-Length': '0'} 2024-04-16T18:29:46.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4cc69b47d314607-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.386 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.387 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.387 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.387 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.387 DEBUG:boto:Method: GET 2024-04-16T18:29:46.387 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:29:46.387 DEBUG:boto:Data: 2024-04-16T18:29:46.387 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.387 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.387 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.387 DEBUG:boto:Params: {} 2024-04-16T18:29:46.387 DEBUG:boto:Token: None 2024-04-16T18:29:46.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004658513b871b84b-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.392 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.392 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.392 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.392 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.393 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.393 DEBUG:boto:Method: GET 2024-04-16T18:29:46.393 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:29:46.393 DEBUG:boto:Data: 2024-04-16T18:29:46.393 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.393 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.393 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.393 DEBUG:boto:Params: {} 2024-04-16T18:29:46.393 DEBUG:boto:Token: None 2024-04-16T18:29:46.393 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.393 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.400 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2bd3471dc89f16c-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.442 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.443 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.443 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.443 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.443 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:29:46.443 DEBUG:boto:Data: 2024-04-16T18:29:46.443 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.443 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.443 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.443 DEBUG:boto:Params: {} 2024-04-16T18:29:46.443 DEBUG:boto:Token: None 2024-04-16T18:29:46.444 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22 2024-04-16T18:29:46.444 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IVJi5TBasyHYRVt9f93dUA7nlSM= 2024-04-16T18:29:46.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IVJi5TBasyHYRVt9f93dUA7nlSM=', 'Content-Length': '0'} 2024-04-16T18:29:46.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e959f3c7fbc3914e-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.447 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.447 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.447 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.447 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:29:46.447 DEBUG:boto:Data: 2024-04-16T18:29:46.447 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.447 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.447 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.447 DEBUG:boto:Params: {} 2024-04-16T18:29:46.448 DEBUG:boto:Token: None 2024-04-16T18:29:46.448 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22 2024-04-16T18:29:46.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IVJi5TBasyHYRVt9f93dUA7nlSM= 2024-04-16T18:29:46.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IVJi5TBasyHYRVt9f93dUA7nlSM=', 'Content-Length': '0'} 2024-04-16T18:29:46.450 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033e4df581e734ff6-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.451 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:29:46.451 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.451 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.451 DEBUG:boto:Method: GET 2024-04-16T18:29:46.451 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:29:46.451 DEBUG:boto:Data: 2024-04-16T18:29:46.451 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.451 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.451 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.451 DEBUG:boto:Params: {} 2024-04-16T18:29:46.452 DEBUG:boto:Token: None 2024-04-16T18:29:46.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22 2024-04-16T18:29:46.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3t64BoC/oVs2OY6+s1poe3aIoWM= 2024-04-16T18:29:46.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3t64BoC/oVs2OY6+s1poe3aIoWM=', 'Content-Length': '0'} 2024-04-16T18:29:46.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6c1ca6c9e9d6539-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.498 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:29:46.498 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:29:46.499 DEBUG:boto:Method: GET 2024-04-16T18:29:46.499 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:29:46.499 DEBUG:boto:Data: 2024-04-16T18:29:46.499 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.499 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.499 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.499 DEBUG:boto:Params: {} 2024-04-16T18:29:46.499 DEBUG:boto:Token: None 2024-04-16T18:29:46.499 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj22 2024-04-16T18:29:46.500 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3t64BoC/oVs2OY6+s1poe3aIoWM= 2024-04-16T18:29:46.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3t64BoC/oVs2OY6+s1poe3aIoWM=', 'Content-Length': '0'} 2024-04-16T18:29:46.502 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000aae0194a11bbf07-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.503 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.503 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.503 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.503 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.503 DEBUG:boto:Method: GET 2024-04-16T18:29:46.503 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:29:46.503 DEBUG:boto:Data: 2024-04-16T18:29:46.503 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.503 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.503 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.504 DEBUG:boto:Params: {} 2024-04-16T18:29:46.504 DEBUG:boto:Token: None 2024-04-16T18:29:46.504 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.508 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056de5cb265b782cc-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.508 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.508 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.508 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.509 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.509 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:29:46.509 DEBUG:boto:Method: GET 2024-04-16T18:29:46.509 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:29:46.509 DEBUG:boto:Data: 2024-04-16T18:29:46.509 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.509 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.509 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.509 DEBUG:boto:Params: {} 2024-04-16T18:29:46.509 DEBUG:boto:Token: None 2024-04-16T18:29:46.509 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.510 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.517 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035ca6ea123211802-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.517 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.517 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:29:46.517 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.518 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.518 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.518 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.518 DEBUG:boto:Method: GET 2024-04-16T18:29:46.518 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.518 DEBUG:boto:Data: 2024-04-16T18:29:46.518 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.518 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.518 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.518 DEBUG:boto:Params: {} 2024-04-16T18:29:46.519 DEBUG:boto:Token: None 2024-04-16T18:29:46.519 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.519 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw= 2024-04-16T18:29:46.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw=', 'Content-Length': '0'} 2024-04-16T18:29:46.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051281bc17bcd6446-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.522 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.522 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.522 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.522 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.523 DEBUG:boto:Method: GET 2024-04-16T18:29:46.523 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.523 DEBUG:boto:Data: 2024-04-16T18:29:46.523 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.523 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.523 DEBUG:boto:Params: {} 2024-04-16T18:29:46.523 DEBUG:boto:Token: None 2024-04-16T18:29:46.523 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw= 2024-04-16T18:29:46.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw=', 'Content-Length': '0'} 2024-04-16T18:29:46.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000952e7f5aea9ec5de-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.570 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.570 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.570 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.571 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.571 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.571 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.571 DEBUG:boto:Data: 2024-04-16T18:29:46.571 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.571 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.571 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.571 DEBUG:boto:Params: {} 2024-04-16T18:29:46.571 DEBUG:boto:Token: None 2024-04-16T18:29:46.572 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pegkwpZyNLaly3QO897xWvHaT4k= 2024-04-16T18:29:46.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pegkwpZyNLaly3QO897xWvHaT4k=', 'Content-Length': '0'} 2024-04-16T18:29:46.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6eef775c9b80501-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.575 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.575 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.575 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.575 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.575 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.576 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.576 DEBUG:boto:Data: 2024-04-16T18:29:46.576 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.576 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.576 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.576 DEBUG:boto:Params: {} 2024-04-16T18:29:46.576 DEBUG:boto:Token: None 2024-04-16T18:29:46.576 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:pegkwpZyNLaly3QO897xWvHaT4k= 2024-04-16T18:29:46.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pegkwpZyNLaly3QO897xWvHaT4k=', 'Content-Length': '0'} 2024-04-16T18:29:46.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044b8c73ceb5eda12-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:29:46.579 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.579 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.580 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.580 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.580 DEBUG:boto:Method: GET 2024-04-16T18:29:46.580 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.580 DEBUG:boto:Data: 2024-04-16T18:29:46.580 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.580 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.580 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.580 DEBUG:boto:Params: {} 2024-04-16T18:29:46.580 DEBUG:boto:Token: None 2024-04-16T18:29:46.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw= 2024-04-16T18:29:46.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw=', 'Content-Length': '0'} 2024-04-16T18:29:46.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce54b4ea7d13af6a-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.626 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.627 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.627 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.627 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.627 DEBUG:boto:Method: GET 2024-04-16T18:29:46.627 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.627 DEBUG:boto:Data: 2024-04-16T18:29:46.627 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.627 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.627 DEBUG:boto:Params: {} 2024-04-16T18:29:46.628 DEBUG:boto:Token: None 2024-04-16T18:29:46.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:29:46.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw= 2024-04-16T18:29:46.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yg2dHDloAIDd1UH5fCXfTN8pClw=', 'Content-Length': '0'} 2024-04-16T18:29:46.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de2c08e4414af3ee-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.631 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.631 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.631 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.631 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.631 DEBUG:boto:Method: GET 2024-04-16T18:29:46.631 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:29:46.631 DEBUG:boto:Data: 2024-04-16T18:29:46.632 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.632 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.632 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.632 DEBUG:boto:Params: {} 2024-04-16T18:29:46.632 DEBUG:boto:Token: None 2024-04-16T18:29:46.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.633 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.636 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9cec3b3e3ed6ffe-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.636 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.637 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.637 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.637 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.637 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.637 DEBUG:boto:Method: GET 2024-04-16T18:29:46.637 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:29:46.637 DEBUG:boto:Data: 2024-04-16T18:29:46.637 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.637 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.637 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.638 DEBUG:boto:Params: {} 2024-04-16T18:29:46.638 DEBUG:boto:Token: None 2024-04-16T18:29:46.638 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.638 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.645 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062a54b349945481a-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.645 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.645 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.645 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.646 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.646 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:29:46.646 DEBUG:boto:Data: 2024-04-16T18:29:46.646 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.646 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.646 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.646 DEBUG:boto:Params: {} 2024-04-16T18:29:46.646 DEBUG:boto:Token: None 2024-04-16T18:29:46.646 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23 2024-04-16T18:29:46.646 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hIJrP68zrcF+QOyjWdrQDBoLiss= 2024-04-16T18:29:46.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hIJrP68zrcF+QOyjWdrQDBoLiss=', 'Content-Length': '0'} 2024-04-16T18:29:46.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014da294faa33f684-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.649 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.649 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.649 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.650 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:29:46.650 DEBUG:boto:Data: 2024-04-16T18:29:46.650 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.650 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.650 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.650 DEBUG:boto:Params: {} 2024-04-16T18:29:46.650 DEBUG:boto:Token: None 2024-04-16T18:29:46.650 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23 2024-04-16T18:29:46.650 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hIJrP68zrcF+QOyjWdrQDBoLiss= 2024-04-16T18:29:46.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hIJrP68zrcF+QOyjWdrQDBoLiss=', 'Content-Length': '0'} 2024-04-16T18:29:46.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067c928aeeff90249-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.653 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:29:46.653 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.654 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.654 DEBUG:boto:Method: GET 2024-04-16T18:29:46.654 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:29:46.654 DEBUG:boto:Data: 2024-04-16T18:29:46.654 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.654 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.654 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.654 DEBUG:boto:Params: {} 2024-04-16T18:29:46.654 DEBUG:boto:Token: None 2024-04-16T18:29:46.655 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23 2024-04-16T18:29:46.655 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zq4kltpYQiEvlW1IbjBR5sEUlOA= 2024-04-16T18:29:46.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zq4kltpYQiEvlW1IbjBR5sEUlOA=', 'Content-Length': '0'} 2024-04-16T18:29:46.658 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2379f8bbabc2a96-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.658 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:29:46.658 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:29:46.658 DEBUG:boto:Method: GET 2024-04-16T18:29:46.658 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:29:46.658 DEBUG:boto:Data: 2024-04-16T18:29:46.658 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.659 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.659 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.659 DEBUG:boto:Params: {} 2024-04-16T18:29:46.659 DEBUG:boto:Token: None 2024-04-16T18:29:46.659 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj23 2024-04-16T18:29:46.659 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zq4kltpYQiEvlW1IbjBR5sEUlOA= 2024-04-16T18:29:46.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zq4kltpYQiEvlW1IbjBR5sEUlOA=', 'Content-Length': '0'} 2024-04-16T18:29:46.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096079e94445c39dc-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.702 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.703 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.703 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.703 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.703 DEBUG:boto:Method: GET 2024-04-16T18:29:46.703 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:29:46.703 DEBUG:boto:Data: 2024-04-16T18:29:46.703 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.703 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.703 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.703 DEBUG:boto:Params: {} 2024-04-16T18:29:46.704 DEBUG:boto:Token: None 2024-04-16T18:29:46.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000167860689821b067-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.708 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.708 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.708 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.709 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.709 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:29:46.709 DEBUG:boto:Method: GET 2024-04-16T18:29:46.709 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:29:46.709 DEBUG:boto:Data: 2024-04-16T18:29:46.709 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.709 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.709 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.709 DEBUG:boto:Params: {} 2024-04-16T18:29:46.709 DEBUG:boto:Token: None 2024-04-16T18:29:46.710 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.710 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075d4c570f08ba6c5-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.716 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:29:46.716 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.717 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.717 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.717 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.717 DEBUG:boto:Method: GET 2024-04-16T18:29:46.717 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.717 DEBUG:boto:Data: 2024-04-16T18:29:46.717 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.717 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.717 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.717 DEBUG:boto:Params: {} 2024-04-16T18:29:46.717 DEBUG:boto:Token: None 2024-04-16T18:29:46.718 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8= 2024-04-16T18:29:46.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8=', 'Content-Length': '0'} 2024-04-16T18:29:46.721 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043460b834a24a877-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.766 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.767 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.767 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.767 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.767 DEBUG:boto:Method: GET 2024-04-16T18:29:46.767 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.767 DEBUG:boto:Data: 2024-04-16T18:29:46.767 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.767 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.767 DEBUG:boto:Params: {} 2024-04-16T18:29:46.768 DEBUG:boto:Token: None 2024-04-16T18:29:46.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8= 2024-04-16T18:29:46.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8=', 'Content-Length': '0'} 2024-04-16T18:29:46.772 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005612fb811831152c-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.772 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.772 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.772 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.772 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.773 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.773 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.773 DEBUG:boto:Data: 2024-04-16T18:29:46.773 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.773 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.773 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.773 DEBUG:boto:Params: {} 2024-04-16T18:29:46.773 DEBUG:boto:Token: None 2024-04-16T18:29:46.773 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.773 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vp3YkQ8Zk7Nlak0YVRt8rbBQLQI= 2024-04-16T18:29:46.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vp3YkQ8Zk7Nlak0YVRt8rbBQLQI=', 'Content-Length': '0'} 2024-04-16T18:29:46.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000636a125e04111763-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.777 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.777 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.777 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.777 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.777 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.777 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.777 DEBUG:boto:Data: 2024-04-16T18:29:46.778 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.778 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.778 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.778 DEBUG:boto:Params: {} 2024-04-16T18:29:46.778 DEBUG:boto:Token: None 2024-04-16T18:29:46.778 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.778 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vp3YkQ8Zk7Nlak0YVRt8rbBQLQI= 2024-04-16T18:29:46.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vp3YkQ8Zk7Nlak0YVRt8rbBQLQI=', 'Content-Length': '0'} 2024-04-16T18:29:46.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084975724c5593198-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.781 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:29:46.782 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.782 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.782 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.782 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.782 DEBUG:boto:Method: GET 2024-04-16T18:29:46.782 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.782 DEBUG:boto:Data: 2024-04-16T18:29:46.782 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.782 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.782 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.783 DEBUG:boto:Params: {} 2024-04-16T18:29:46.783 DEBUG:boto:Token: None 2024-04-16T18:29:46.783 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.783 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8= 2024-04-16T18:29:46.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8=', 'Content-Length': '0'} 2024-04-16T18:29:46.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076b2f346623927e0-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.787 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.787 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.787 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.787 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.787 DEBUG:boto:Method: GET 2024-04-16T18:29:46.787 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.787 DEBUG:boto:Data: 2024-04-16T18:29:46.787 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.788 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.788 DEBUG:boto:Params: {} 2024-04-16T18:29:46.788 DEBUG:boto:Token: None 2024-04-16T18:29:46.788 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:29:46.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8= 2024-04-16T18:29:46.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TIntRdjxKInPymPKE7aGuNMbSu8=', 'Content-Length': '0'} 2024-04-16T18:29:46.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5c0784da735ca3b-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.792 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.792 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.792 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.792 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.792 DEBUG:boto:Method: GET 2024-04-16T18:29:46.792 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:29:46.792 DEBUG:boto:Data: 2024-04-16T18:29:46.792 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.792 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.792 DEBUG:boto:Params: {} 2024-04-16T18:29:46.793 DEBUG:boto:Token: None 2024-04-16T18:29:46.793 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.793 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000266e7c73259f7cf5-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.797 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.797 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.797 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.797 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.797 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.798 DEBUG:boto:Method: GET 2024-04-16T18:29:46.798 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:29:46.798 DEBUG:boto:Data: 2024-04-16T18:29:46.798 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.798 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.798 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.798 DEBUG:boto:Params: {} 2024-04-16T18:29:46.798 DEBUG:boto:Token: None 2024-04-16T18:29:46.798 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.799 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.805 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d572a22e5c24185f-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.846 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.847 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.847 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.847 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.847 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:29:46.847 DEBUG:boto:Data: 2024-04-16T18:29:46.847 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.847 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.847 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.847 DEBUG:boto:Params: {} 2024-04-16T18:29:46.848 DEBUG:boto:Token: None 2024-04-16T18:29:46.848 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29 2024-04-16T18:29:46.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VIrACBAca8Thpk5LGtk1ZVPvcP4= 2024-04-16T18:29:46.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VIrACBAca8Thpk5LGtk1ZVPvcP4=', 'Content-Length': '0'} 2024-04-16T18:29:46.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000556a4755933abd6c-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.851 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.851 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.851 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.852 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:29:46.852 DEBUG:boto:Data: 2024-04-16T18:29:46.852 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.852 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.852 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.852 DEBUG:boto:Params: {} 2024-04-16T18:29:46.852 DEBUG:boto:Token: None 2024-04-16T18:29:46.852 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29 2024-04-16T18:29:46.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VIrACBAca8Thpk5LGtk1ZVPvcP4= 2024-04-16T18:29:46.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VIrACBAca8Thpk5LGtk1ZVPvcP4=', 'Content-Length': '0'} 2024-04-16T18:29:46.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0b13445a0fc6533-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.855 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:29:46.856 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.856 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.856 DEBUG:boto:Method: GET 2024-04-16T18:29:46.856 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:29:46.856 DEBUG:boto:Data: 2024-04-16T18:29:46.856 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.856 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.856 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.856 DEBUG:boto:Params: {} 2024-04-16T18:29:46.856 DEBUG:boto:Token: None 2024-04-16T18:29:46.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29 2024-04-16T18:29:46.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GptO3aKzq74wTGkydU8G2zizSdc= 2024-04-16T18:29:46.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GptO3aKzq74wTGkydU8G2zizSdc=', 'Content-Length': '0'} 2024-04-16T18:29:46.860 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d002591354461ffd-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.902 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:29:46.903 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:29:46.903 DEBUG:boto:Method: GET 2024-04-16T18:29:46.903 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:29:46.903 DEBUG:boto:Data: 2024-04-16T18:29:46.903 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.903 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.903 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.903 DEBUG:boto:Params: {} 2024-04-16T18:29:46.903 DEBUG:boto:Token: None 2024-04-16T18:29:46.904 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj29 2024-04-16T18:29:46.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GptO3aKzq74wTGkydU8G2zizSdc= 2024-04-16T18:29:46.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GptO3aKzq74wTGkydU8G2zizSdc=', 'Content-Length': '0'} 2024-04-16T18:29:46.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbf57d49f416e9f8-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.907 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.907 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.908 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.908 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.908 DEBUG:boto:Method: GET 2024-04-16T18:29:46.908 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:29:46.908 DEBUG:boto:Data: 2024-04-16T18:29:46.908 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.908 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.908 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.908 DEBUG:boto:Params: {} 2024-04-16T18:29:46.908 DEBUG:boto:Token: None 2024-04-16T18:29:46.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.912 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000298da0af7c2ec7e-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.912 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.913 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:46.913 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:46.913 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.913 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:29:46.913 DEBUG:boto:Method: GET 2024-04-16T18:29:46.913 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:29:46.913 DEBUG:boto:Data: 2024-04-16T18:29:46.914 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.914 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.914 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.914 DEBUG:boto:Params: {} 2024-04-16T18:29:46.914 DEBUG:boto:Token: None 2024-04-16T18:29:46.914 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/ 2024-04-16T18:29:46.914 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o= 2024-04-16T18:29:46.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fmEDEiWZrW2rn1x0uS+NtWshj6o=', 'Content-Length': '0'} 2024-04-16T18:29:46.921 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb6ae8dd821409ab-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.921 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:46.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:29:46.921 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:46.922 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:46.922 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.922 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.922 DEBUG:boto:Method: GET 2024-04-16T18:29:46.922 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.922 DEBUG:boto:Data: 2024-04-16T18:29:46.922 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.922 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.922 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.922 DEBUG:boto:Params: {} 2024-04-16T18:29:46.922 DEBUG:boto:Token: None 2024-04-16T18:29:46.923 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.923 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rV+ONC0pKoiHSjlV1m6O3Fw+0VI= 2024-04-16T18:29:46.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rV+ONC0pKoiHSjlV1m6O3Fw+0VI=', 'Content-Length': '0'} 2024-04-16T18:29:46.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000195030e2da32a63b-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.966 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:46.967 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:46.967 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.967 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.967 DEBUG:boto:Method: GET 2024-04-16T18:29:46.967 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.967 DEBUG:boto:Data: 2024-04-16T18:29:46.968 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.968 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.968 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.968 DEBUG:boto:Params: {} 2024-04-16T18:29:46.969 DEBUG:boto:Token: None 2024-04-16T18:29:46.969 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.969 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rV+ONC0pKoiHSjlV1m6O3Fw+0VI= 2024-04-16T18:29:46.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rV+ONC0pKoiHSjlV1m6O3Fw+0VI=', 'Content-Length': '0'} 2024-04-16T18:29:46.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f70cf51a8b2f38b4-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.973 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:46.973 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:46.973 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.973 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.973 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.973 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.973 DEBUG:boto:Data: 2024-04-16T18:29:46.974 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.974 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.974 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.974 DEBUG:boto:Params: {} 2024-04-16T18:29:46.974 DEBUG:boto:Token: None 2024-04-16T18:29:46.974 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.974 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7xwy//e/Z1pFDta8ZhhSblxdHZc= 2024-04-16T18:29:46.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7xwy//e/Z1pFDta8ZhhSblxdHZc=', 'Content-Length': '0'} 2024-04-16T18:29:46.977 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8c3b1c96c8e3b0b-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.978 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:46.978 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:46.978 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.978 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.978 DEBUG:boto:Method: HEAD 2024-04-16T18:29:46.978 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.979 DEBUG:boto:Data: 2024-04-16T18:29:46.979 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.979 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:46.979 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.979 DEBUG:boto:Params: {} 2024-04-16T18:29:46.979 DEBUG:boto:Token: None 2024-04-16T18:29:46.979 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.979 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7xwy//e/Z1pFDta8ZhhSblxdHZc= 2024-04-16T18:29:46.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7xwy//e/Z1pFDta8ZhhSblxdHZc=', 'Content-Length': '0'} 2024-04-16T18:29:46.982 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021f08463e345d55e-00661ec39a-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:46.982 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:29:46.983 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:46.983 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:46.983 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.983 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.983 DEBUG:boto:Method: GET 2024-04-16T18:29:46.983 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.983 DEBUG:boto:Data: 2024-04-16T18:29:46.983 DEBUG:boto:Headers: {} 2024-04-16T18:29:46.983 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:46.983 DEBUG:boto:Port: 8000 2024-04-16T18:29:46.983 DEBUG:boto:Params: {} 2024-04-16T18:29:46.984 DEBUG:boto:Token: None 2024-04-16T18:29:46.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:46 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:29:46.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:rV+ONC0pKoiHSjlV1m6O3Fw+0VI= 2024-04-16T18:29:46.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:46 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rV+ONC0pKoiHSjlV1m6O3Fw+0VI=', 'Content-Length': '0'} 2024-04-16T18:29:46.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aad7ed30e0cd5f98-00661ec39a-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:46 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.030 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:47.030 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:47.031 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:47.031 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:29:47.031 DEBUG:boto:Method: GET 2024-04-16T18:29:47.031 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:29:47.031 DEBUG:boto:Data: 2024-04-16T18:29:47.031 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.031 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.031 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.031 DEBUG:boto:Params: {} 2024-04-16T18:29:47.031 DEBUG:boto:Token: None 2024-04-16T18:29:47.032 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:29:47.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JQ23Vsr5WTwO5/dfCOoJWLLAGMQ= 2024-04-16T18:29:47.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JQ23Vsr5WTwO5/dfCOoJWLLAGMQ=', 'Content-Length': '0'} 2024-04-16T18:29:47.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055e365e2b0dffb50-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.035 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.035 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.036 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.036 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.036 DEBUG:boto:Method: GET 2024-04-16T18:29:47.036 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:29:47.036 DEBUG:boto:Data: 2024-04-16T18:29:47.036 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.036 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.036 DEBUG:boto:Params: {} 2024-04-16T18:29:47.036 DEBUG:boto:Token: None 2024-04-16T18:29:47.037 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.041 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db61388e2a0eb635-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.041 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.041 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.041 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.041 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.041 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.042 DEBUG:boto:Method: GET 2024-04-16T18:29:47.042 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:29:47.042 DEBUG:boto:Data: 2024-04-16T18:29:47.042 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.042 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.042 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.042 DEBUG:boto:Params: {} 2024-04-16T18:29:47.042 DEBUG:boto:Token: None 2024-04-16T18:29:47.042 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000177e2b145a844b2c-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.050 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.050 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:47.050 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:47.051 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.051 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:29:47.051 DEBUG:boto:Data: 2024-04-16T18:29:47.051 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.051 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.051 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.051 DEBUG:boto:Params: {} 2024-04-16T18:29:47.051 DEBUG:boto:Token: None 2024-04-16T18:29:47.051 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj31 2024-04-16T18:29:47.051 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:inWaF68NoasUHA+/upZG1V+FKkw= 2024-04-16T18:29:47.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:inWaF68NoasUHA+/upZG1V+FKkw=', 'Content-Length': '0'} 2024-04-16T18:29:47.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f94f9d384e7b2db2-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.055 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:47.055 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:47.055 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.055 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:29:47.055 DEBUG:boto:Data: 2024-04-16T18:29:47.055 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.055 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.055 DEBUG:boto:Params: {} 2024-04-16T18:29:47.055 DEBUG:boto:Token: None 2024-04-16T18:29:47.056 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj31 2024-04-16T18:29:47.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:inWaF68NoasUHA+/upZG1V+FKkw= 2024-04-16T18:29:47.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:inWaF68NoasUHA+/upZG1V+FKkw=', 'Content-Length': '0'} 2024-04-16T18:29:47.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2e6601b79c99553-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.059 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:29:47.059 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:47.060 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:47.060 DEBUG:boto:Method: GET 2024-04-16T18:29:47.060 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:29:47.060 DEBUG:boto:Data: 2024-04-16T18:29:47.060 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.060 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.060 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.060 DEBUG:boto:Params: {} 2024-04-16T18:29:47.060 DEBUG:boto:Token: None 2024-04-16T18:29:47.060 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj31 2024-04-16T18:29:47.061 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qY+loycssXjorLdK6wnfBhLNf78= 2024-04-16T18:29:47.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qY+loycssXjorLdK6wnfBhLNf78=', 'Content-Length': '0'} 2024-04-16T18:29:47.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c8ca7735a5a3979-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.106 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:29:47.107 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:29:47.107 DEBUG:boto:Method: GET 2024-04-16T18:29:47.107 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:29:47.107 DEBUG:boto:Data: 2024-04-16T18:29:47.107 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.107 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.107 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.107 DEBUG:boto:Params: {} 2024-04-16T18:29:47.107 DEBUG:boto:Token: None 2024-04-16T18:29:47.108 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj31 2024-04-16T18:29:47.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qY+loycssXjorLdK6wnfBhLNf78= 2024-04-16T18:29:47.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qY+loycssXjorLdK6wnfBhLNf78=', 'Content-Length': '0'} 2024-04-16T18:29:47.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e306ebd42acc4f5-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.111 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.111 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.111 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.111 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.112 DEBUG:boto:Method: GET 2024-04-16T18:29:47.112 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:29:47.112 DEBUG:boto:Data: 2024-04-16T18:29:47.112 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.112 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.112 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.112 DEBUG:boto:Params: {} 2024-04-16T18:29:47.112 DEBUG:boto:Token: None 2024-04-16T18:29:47.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.116 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006df778c96c59bfc9-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.116 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.117 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.117 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.117 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.117 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:29:47.117 DEBUG:boto:Method: GET 2024-04-16T18:29:47.117 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:29:47.117 DEBUG:boto:Data: 2024-04-16T18:29:47.117 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.118 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.118 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.118 DEBUG:boto:Params: {} 2024-04-16T18:29:47.118 DEBUG:boto:Token: None 2024-04-16T18:29:47.118 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.118 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3100fecbace4a7e-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.166 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.167 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:29:47.167 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.167 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.167 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.167 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.167 DEBUG:boto:Method: GET 2024-04-16T18:29:47.167 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.168 DEBUG:boto:Data: 2024-04-16T18:29:47.168 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.168 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.168 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.168 DEBUG:boto:Params: {} 2024-04-16T18:29:47.168 DEBUG:boto:Token: None 2024-04-16T18:29:47.168 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.168 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8= 2024-04-16T18:29:47.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8=', 'Content-Length': '0'} 2024-04-16T18:29:47.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9afa007abab7808-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.218 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.219 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.219 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.219 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.219 DEBUG:boto:Method: GET 2024-04-16T18:29:47.219 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.219 DEBUG:boto:Data: 2024-04-16T18:29:47.219 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.219 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.220 DEBUG:boto:Params: {} 2024-04-16T18:29:47.220 DEBUG:boto:Token: None 2024-04-16T18:29:47.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8= 2024-04-16T18:29:47.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8=', 'Content-Length': '0'} 2024-04-16T18:29:47.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0a5d2adcac0609f-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.224 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.224 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.224 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.224 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.224 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.224 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.225 DEBUG:boto:Data: 2024-04-16T18:29:47.225 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.225 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.225 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.225 DEBUG:boto:Params: {} 2024-04-16T18:29:47.225 DEBUG:boto:Token: None 2024-04-16T18:29:47.225 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sKnpkl0aH4nDZhBoiocKfXoLC6U= 2024-04-16T18:29:47.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sKnpkl0aH4nDZhBoiocKfXoLC6U=', 'Content-Length': '0'} 2024-04-16T18:29:47.228 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000780b4ced7ba38887-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.229 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.229 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.229 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.229 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.229 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.229 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.230 DEBUG:boto:Data: 2024-04-16T18:29:47.230 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.230 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.230 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.230 DEBUG:boto:Params: {} 2024-04-16T18:29:47.230 DEBUG:boto:Token: None 2024-04-16T18:29:47.230 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.230 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sKnpkl0aH4nDZhBoiocKfXoLC6U= 2024-04-16T18:29:47.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sKnpkl0aH4nDZhBoiocKfXoLC6U=', 'Content-Length': '0'} 2024-04-16T18:29:47.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eed10664feb54f4b-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.233 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:29:47.234 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.234 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.234 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.234 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.234 DEBUG:boto:Method: GET 2024-04-16T18:29:47.234 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.234 DEBUG:boto:Data: 2024-04-16T18:29:47.234 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.235 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.235 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.235 DEBUG:boto:Params: {} 2024-04-16T18:29:47.235 DEBUG:boto:Token: None 2024-04-16T18:29:47.235 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.235 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8= 2024-04-16T18:29:47.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8=', 'Content-Length': '0'} 2024-04-16T18:29:47.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb059f91904af54b-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.282 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.283 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.283 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.283 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.283 DEBUG:boto:Method: GET 2024-04-16T18:29:47.283 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.283 DEBUG:boto:Data: 2024-04-16T18:29:47.283 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.283 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.283 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.283 DEBUG:boto:Params: {} 2024-04-16T18:29:47.284 DEBUG:boto:Token: None 2024-04-16T18:29:47.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:29:47.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8= 2024-04-16T18:29:47.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0a1Aw9JzKYybKAPYxXlhYGPnKw8=', 'Content-Length': '0'} 2024-04-16T18:29:47.287 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1c2131011de6c54-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.287 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.287 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.287 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.288 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.288 DEBUG:boto:Method: GET 2024-04-16T18:29:47.288 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:29:47.288 DEBUG:boto:Data: 2024-04-16T18:29:47.288 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.288 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.288 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.288 DEBUG:boto:Params: {} 2024-04-16T18:29:47.288 DEBUG:boto:Token: None 2024-04-16T18:29:47.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006bf5cdcb7d5e564e-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.292 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.293 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.293 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.293 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.293 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.293 DEBUG:boto:Method: GET 2024-04-16T18:29:47.293 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:29:47.293 DEBUG:boto:Data: 2024-04-16T18:29:47.293 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.294 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.294 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.294 DEBUG:boto:Params: {} 2024-04-16T18:29:47.294 DEBUG:boto:Token: None 2024-04-16T18:29:47.294 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.294 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.301 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ebe645ca864a9024-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.301 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.301 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.301 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.301 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.302 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:29:47.302 DEBUG:boto:Data: 2024-04-16T18:29:47.302 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.302 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.302 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.302 DEBUG:boto:Params: {} 2024-04-16T18:29:47.302 DEBUG:boto:Token: None 2024-04-16T18:29:47.302 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33 2024-04-16T18:29:47.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FRWOy8mGxiij5I4+uT85Wr6WqDA= 2024-04-16T18:29:47.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FRWOy8mGxiij5I4+uT85Wr6WqDA=', 'Content-Length': '0'} 2024-04-16T18:29:47.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031a88af519d4400d-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.305 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.305 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.306 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.306 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:29:47.306 DEBUG:boto:Data: 2024-04-16T18:29:47.306 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.306 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.306 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.306 DEBUG:boto:Params: {} 2024-04-16T18:29:47.306 DEBUG:boto:Token: None 2024-04-16T18:29:47.306 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33 2024-04-16T18:29:47.306 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FRWOy8mGxiij5I4+uT85Wr6WqDA= 2024-04-16T18:29:47.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FRWOy8mGxiij5I4+uT85Wr6WqDA=', 'Content-Length': '0'} 2024-04-16T18:29:47.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b708e39e7521ebe-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.310 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:29:47.310 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.310 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.310 DEBUG:boto:Method: GET 2024-04-16T18:29:47.310 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:29:47.310 DEBUG:boto:Data: 2024-04-16T18:29:47.310 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.311 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.311 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.311 DEBUG:boto:Params: {} 2024-04-16T18:29:47.311 DEBUG:boto:Token: None 2024-04-16T18:29:47.311 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33 2024-04-16T18:29:47.311 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:quPO+/sTRE1DmVjzmZfzLkWjBXI= 2024-04-16T18:29:47.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:quPO+/sTRE1DmVjzmZfzLkWjBXI=', 'Content-Length': '0'} 2024-04-16T18:29:47.314 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084549f370df17e2d-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.354 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:29:47.355 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:29:47.355 DEBUG:boto:Method: GET 2024-04-16T18:29:47.355 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:29:47.355 DEBUG:boto:Data: 2024-04-16T18:29:47.355 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.355 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.355 DEBUG:boto:Params: {} 2024-04-16T18:29:47.355 DEBUG:boto:Token: None 2024-04-16T18:29:47.356 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj33 2024-04-16T18:29:47.356 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:quPO+/sTRE1DmVjzmZfzLkWjBXI= 2024-04-16T18:29:47.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:quPO+/sTRE1DmVjzmZfzLkWjBXI=', 'Content-Length': '0'} 2024-04-16T18:29:47.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e2e683358bed6c5-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.359 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.359 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.359 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.359 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.359 DEBUG:boto:Method: GET 2024-04-16T18:29:47.359 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:29:47.360 DEBUG:boto:Data: 2024-04-16T18:29:47.360 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.360 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.360 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.360 DEBUG:boto:Params: {} 2024-04-16T18:29:47.360 DEBUG:boto:Token: None 2024-04-16T18:29:47.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071908027455afea6-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.406 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.407 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.407 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.407 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.407 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:29:47.407 DEBUG:boto:Method: GET 2024-04-16T18:29:47.407 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:29:47.407 DEBUG:boto:Data: 2024-04-16T18:29:47.407 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.407 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.407 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.408 DEBUG:boto:Params: {} 2024-04-16T18:29:47.408 DEBUG:boto:Token: None 2024-04-16T18:29:47.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.408 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000885d15808ceb500f-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.415 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.415 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:29:47.415 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.415 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.415 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.415 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.415 DEBUG:boto:Method: GET 2024-04-16T18:29:47.415 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.415 DEBUG:boto:Data: 2024-04-16T18:29:47.416 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.416 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.416 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.416 DEBUG:boto:Params: {} 2024-04-16T18:29:47.416 DEBUG:boto:Token: None 2024-04-16T18:29:47.416 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc= 2024-04-16T18:29:47.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc=', 'Content-Length': '0'} 2024-04-16T18:29:47.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a50da396b0aa037e-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.462 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.463 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.463 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.463 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.463 DEBUG:boto:Method: GET 2024-04-16T18:29:47.463 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.463 DEBUG:boto:Data: 2024-04-16T18:29:47.463 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.463 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.463 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.463 DEBUG:boto:Params: {} 2024-04-16T18:29:47.464 DEBUG:boto:Token: None 2024-04-16T18:29:47.464 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.464 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc= 2024-04-16T18:29:47.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc=', 'Content-Length': '0'} 2024-04-16T18:29:47.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba5e6ae22fff9954-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.467 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.467 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.467 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.468 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.468 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.468 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.468 DEBUG:boto:Data: 2024-04-16T18:29:47.468 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.468 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.468 DEBUG:boto:Params: {} 2024-04-16T18:29:47.468 DEBUG:boto:Token: None 2024-04-16T18:29:47.468 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3f2yMxrDbUdKPp1UYdL1sCSG5gk= 2024-04-16T18:29:47.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3f2yMxrDbUdKPp1UYdL1sCSG5gk=', 'Content-Length': '0'} 2024-04-16T18:29:47.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af7f3daad75915a2-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.472 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.472 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.472 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.472 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.472 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.472 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.472 DEBUG:boto:Data: 2024-04-16T18:29:47.472 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.472 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.472 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.473 DEBUG:boto:Params: {} 2024-04-16T18:29:47.473 DEBUG:boto:Token: None 2024-04-16T18:29:47.473 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.473 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3f2yMxrDbUdKPp1UYdL1sCSG5gk= 2024-04-16T18:29:47.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3f2yMxrDbUdKPp1UYdL1sCSG5gk=', 'Content-Length': '0'} 2024-04-16T18:29:47.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3416fbb0d1a663c-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.476 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:29:47.476 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.476 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.476 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.477 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.477 DEBUG:boto:Method: GET 2024-04-16T18:29:47.477 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.477 DEBUG:boto:Data: 2024-04-16T18:29:47.477 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.477 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.477 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.477 DEBUG:boto:Params: {} 2024-04-16T18:29:47.477 DEBUG:boto:Token: None 2024-04-16T18:29:47.477 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.477 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc= 2024-04-16T18:29:47.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc=', 'Content-Length': '0'} 2024-04-16T18:29:47.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b598824288c0762e-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.522 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.522 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.523 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.523 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.523 DEBUG:boto:Method: GET 2024-04-16T18:29:47.523 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.523 DEBUG:boto:Data: 2024-04-16T18:29:47.523 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.523 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.523 DEBUG:boto:Params: {} 2024-04-16T18:29:47.523 DEBUG:boto:Token: None 2024-04-16T18:29:47.524 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:29:47.524 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc= 2024-04-16T18:29:47.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+FIUfbTfgSzh3nz3geeoSD4STLc=', 'Content-Length': '0'} 2024-04-16T18:29:47.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fc2a6bd49978d81-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.527 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.527 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.527 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.527 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.528 DEBUG:boto:Method: GET 2024-04-16T18:29:47.528 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:29:47.528 DEBUG:boto:Data: 2024-04-16T18:29:47.528 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.528 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.528 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.528 DEBUG:boto:Params: {} 2024-04-16T18:29:47.528 DEBUG:boto:Token: None 2024-04-16T18:29:47.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.532 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d3422bededb90e1-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.532 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.532 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.532 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.533 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.533 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.533 DEBUG:boto:Method: GET 2024-04-16T18:29:47.533 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:29:47.533 DEBUG:boto:Data: 2024-04-16T18:29:47.533 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.533 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.533 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.533 DEBUG:boto:Params: {} 2024-04-16T18:29:47.533 DEBUG:boto:Token: None 2024-04-16T18:29:47.533 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.540 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a8de022e6b668a8-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.540 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.541 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.541 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.541 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.541 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:29:47.541 DEBUG:boto:Data: 2024-04-16T18:29:47.541 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.541 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.541 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.541 DEBUG:boto:Params: {} 2024-04-16T18:29:47.542 DEBUG:boto:Token: None 2024-04-16T18:29:47.542 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34 2024-04-16T18:29:47.542 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kYPVeJQTj3dAlrug6bL8E+1nnXo= 2024-04-16T18:29:47.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kYPVeJQTj3dAlrug6bL8E+1nnXo=', 'Content-Length': '0'} 2024-04-16T18:29:47.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4e50d2d9ff7721a-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.545 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.545 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.545 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.545 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:29:47.545 DEBUG:boto:Data: 2024-04-16T18:29:47.545 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.545 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.545 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.546 DEBUG:boto:Params: {} 2024-04-16T18:29:47.546 DEBUG:boto:Token: None 2024-04-16T18:29:47.546 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34 2024-04-16T18:29:47.546 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kYPVeJQTj3dAlrug6bL8E+1nnXo= 2024-04-16T18:29:47.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kYPVeJQTj3dAlrug6bL8E+1nnXo=', 'Content-Length': '0'} 2024-04-16T18:29:47.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a92a9a6533550ae7-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:29:47.549 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.549 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.549 DEBUG:boto:Method: GET 2024-04-16T18:29:47.549 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:29:47.550 DEBUG:boto:Data: 2024-04-16T18:29:47.550 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.550 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.550 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.550 DEBUG:boto:Params: {} 2024-04-16T18:29:47.550 DEBUG:boto:Token: None 2024-04-16T18:29:47.550 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34 2024-04-16T18:29:47.550 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dg3fhoYhr5+3GZkB5UgkRGDpAQ4= 2024-04-16T18:29:47.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dg3fhoYhr5+3GZkB5UgkRGDpAQ4=', 'Content-Length': '0'} 2024-04-16T18:29:47.553 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:16 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085832a28b67f3db2-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.598 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:29:47.599 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:29:47.599 DEBUG:boto:Method: GET 2024-04-16T18:29:47.599 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:29:47.599 DEBUG:boto:Data: 2024-04-16T18:29:47.599 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.599 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.599 DEBUG:boto:Params: {} 2024-04-16T18:29:47.599 DEBUG:boto:Token: None 2024-04-16T18:29:47.600 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj34 2024-04-16T18:29:47.600 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dg3fhoYhr5+3GZkB5UgkRGDpAQ4= 2024-04-16T18:29:47.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dg3fhoYhr5+3GZkB5UgkRGDpAQ4=', 'Content-Length': '0'} 2024-04-16T18:29:47.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f8c7744ffa4318c-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.603 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.604 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.604 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.604 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.604 DEBUG:boto:Method: GET 2024-04-16T18:29:47.604 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:29:47.604 DEBUG:boto:Data: 2024-04-16T18:29:47.604 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.604 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.604 DEBUG:boto:Params: {} 2024-04-16T18:29:47.605 DEBUG:boto:Token: None 2024-04-16T18:29:47.605 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000678997af56661807-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.650 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.651 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.651 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.651 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.651 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:29:47.651 DEBUG:boto:Method: GET 2024-04-16T18:29:47.651 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:29:47.651 DEBUG:boto:Data: 2024-04-16T18:29:47.651 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.652 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.652 DEBUG:boto:Params: {} 2024-04-16T18:29:47.652 DEBUG:boto:Token: None 2024-04-16T18:29:47.652 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066f9bbbf0f44b414-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.659 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.659 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2024-04-16T18:29:47.659 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.660 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.660 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.660 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.660 DEBUG:boto:Method: GET 2024-04-16T18:29:47.660 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.660 DEBUG:boto:Data: 2024-04-16T18:29:47.660 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.660 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.660 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.660 DEBUG:boto:Params: {} 2024-04-16T18:29:47.660 DEBUG:boto:Token: None 2024-04-16T18:29:47.661 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.661 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA= 2024-04-16T18:29:47.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA=', 'Content-Length': '0'} 2024-04-16T18:29:47.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:29:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d93f819fcbb3c794-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.706 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.706 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.707 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.707 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.707 DEBUG:boto:Method: GET 2024-04-16T18:29:47.707 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.707 DEBUG:boto:Data: 2024-04-16T18:29:47.707 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.707 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.707 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.707 DEBUG:boto:Params: {} 2024-04-16T18:29:47.707 DEBUG:boto:Token: None 2024-04-16T18:29:47.708 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.708 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA= 2024-04-16T18:29:47.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA=', 'Content-Length': '0'} 2024-04-16T18:29:47.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000422c1971b1c5b9c7-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.711 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.711 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.712 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.712 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.712 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.712 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.712 DEBUG:boto:Data: 2024-04-16T18:29:47.712 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.712 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.712 DEBUG:boto:Params: {} 2024-04-16T18:29:47.712 DEBUG:boto:Token: None 2024-04-16T18:29:47.713 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jidxfaZOtcViFKaUW9cyg5CV3co= 2024-04-16T18:29:47.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jidxfaZOtcViFKaUW9cyg5CV3co=', 'Content-Length': '0'} 2024-04-16T18:29:47.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:29:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008313370b38badc2b-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.716 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.717 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.717 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.717 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.717 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.717 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.717 DEBUG:boto:Data: 2024-04-16T18:29:47.717 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.717 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.717 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.718 DEBUG:boto:Params: {} 2024-04-16T18:29:47.718 DEBUG:boto:Token: None 2024-04-16T18:29:47.718 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.718 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jidxfaZOtcViFKaUW9cyg5CV3co= 2024-04-16T18:29:47.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jidxfaZOtcViFKaUW9cyg5CV3co=', 'Content-Length': '0'} 2024-04-16T18:29:47.721 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b7a3e239f23d73f-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.722 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2024-04-16T18:29:47.722 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.722 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.722 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.722 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.722 DEBUG:boto:Method: GET 2024-04-16T18:29:47.722 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.722 DEBUG:boto:Data: 2024-04-16T18:29:47.722 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.722 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.722 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.723 DEBUG:boto:Params: {} 2024-04-16T18:29:47.723 DEBUG:boto:Token: None 2024-04-16T18:29:47.723 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.723 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA= 2024-04-16T18:29:47.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA=', 'Content-Length': '0'} 2024-04-16T18:29:47.726 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:29:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007609e13c2d1ed24f-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.770 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.770 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.771 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.771 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.771 DEBUG:boto:Method: GET 2024-04-16T18:29:47.771 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.771 DEBUG:boto:Data: 2024-04-16T18:29:47.771 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.772 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.772 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.772 DEBUG:boto:Params: {} 2024-04-16T18:29:47.772 DEBUG:boto:Token: None 2024-04-16T18:29:47.772 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:29:47.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA= 2024-04-16T18:29:47.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Yf9/W2tDyDNoChawPdWV3jBiEgA=', 'Content-Length': '0'} 2024-04-16T18:29:47.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037bcf2da4ce4bf21-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.776 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.776 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.776 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.776 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.777 DEBUG:boto:Method: GET 2024-04-16T18:29:47.777 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:29:47.777 DEBUG:boto:Data: 2024-04-16T18:29:47.777 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.777 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.777 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.777 DEBUG:boto:Params: {} 2024-04-16T18:29:47.777 DEBUG:boto:Token: None 2024-04-16T18:29:47.777 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.781 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b70157bc964b9ae4-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.781 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.782 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.782 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.782 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.782 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.782 DEBUG:boto:Method: GET 2024-04-16T18:29:47.782 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:29:47.783 DEBUG:boto:Data: 2024-04-16T18:29:47.783 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.783 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.783 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.783 DEBUG:boto:Params: {} 2024-04-16T18:29:47.783 DEBUG:boto:Token: None 2024-04-16T18:29:47.783 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.783 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.790 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000724a1afa753bb962-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.790 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.791 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.791 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.791 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.791 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:29:47.791 DEBUG:boto:Data: 2024-04-16T18:29:47.791 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.791 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.791 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.791 DEBUG:boto:Params: {} 2024-04-16T18:29:47.792 DEBUG:boto:Token: None 2024-04-16T18:29:47.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37 2024-04-16T18:29:47.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sM+16DXVzQttEhz9dOyRvCKazK8= 2024-04-16T18:29:47.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sM+16DXVzQttEhz9dOyRvCKazK8=', 'Content-Length': '0'} 2024-04-16T18:29:47.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:29:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002380f357bfccc581-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.795 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.795 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.795 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.795 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:29:47.796 DEBUG:boto:Data: 2024-04-16T18:29:47.796 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.796 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.796 DEBUG:boto:Params: {} 2024-04-16T18:29:47.796 DEBUG:boto:Token: None 2024-04-16T18:29:47.796 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37 2024-04-16T18:29:47.796 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sM+16DXVzQttEhz9dOyRvCKazK8= 2024-04-16T18:29:47.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sM+16DXVzQttEhz9dOyRvCKazK8=', 'Content-Length': '0'} 2024-04-16T18:29:47.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d96e366e3fc22d5-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.799 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2024-04-16T18:29:47.799 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.800 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.800 DEBUG:boto:Method: GET 2024-04-16T18:29:47.800 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:29:47.800 DEBUG:boto:Data: 2024-04-16T18:29:47.800 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.800 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.800 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.800 DEBUG:boto:Params: {} 2024-04-16T18:29:47.800 DEBUG:boto:Token: None 2024-04-16T18:29:47.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37 2024-04-16T18:29:47.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eIuVRXztAqzt7hoJyLengj85I/A= 2024-04-16T18:29:47.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eIuVRXztAqzt7hoJyLengj85I/A=', 'Content-Length': '0'} 2024-04-16T18:29:47.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:29:36 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c787bb0c220678d-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.850 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:29:47.850 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:29:47.850 DEBUG:boto:Method: GET 2024-04-16T18:29:47.851 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:29:47.851 DEBUG:boto:Data: 2024-04-16T18:29:47.851 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.851 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.851 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.851 DEBUG:boto:Params: {} 2024-04-16T18:29:47.851 DEBUG:boto:Token: None 2024-04-16T18:29:47.851 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj37 2024-04-16T18:29:47.851 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eIuVRXztAqzt7hoJyLengj85I/A= 2024-04-16T18:29:47.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eIuVRXztAqzt7hoJyLengj85I/A=', 'Content-Length': '0'} 2024-04-16T18:29:47.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b30697a81fa8739-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.855 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.855 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.855 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.855 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.855 DEBUG:boto:Method: GET 2024-04-16T18:29:47.855 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:29:47.855 DEBUG:boto:Data: 2024-04-16T18:29:47.855 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.855 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.855 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.855 DEBUG:boto:Params: {} 2024-04-16T18:29:47.856 DEBUG:boto:Token: None 2024-04-16T18:29:47.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b34fc9ab2945918b-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.860 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.860 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.860 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.860 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.860 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:29:47.860 DEBUG:boto:Method: GET 2024-04-16T18:29:47.860 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:29:47.861 DEBUG:boto:Data: 2024-04-16T18:29:47.861 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.861 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.861 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.861 DEBUG:boto:Params: {} 2024-04-16T18:29:47.861 DEBUG:boto:Token: None 2024-04-16T18:29:47.862 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.869 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000995fa25445b892ca-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.869 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:29:47.870 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:47.870 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:47.870 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.870 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.870 DEBUG:boto:Method: GET 2024-04-16T18:29:47.870 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.870 DEBUG:boto:Data: 2024-04-16T18:29:47.870 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.870 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.870 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.870 DEBUG:boto:Params: {} 2024-04-16T18:29:47.871 DEBUG:boto:Token: None 2024-04-16T18:29:47.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.871 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs= 2024-04-16T18:29:47.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs=', 'Content-Length': '0'} 2024-04-16T18:29:47.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005005b5780d84355d-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.914 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:47.914 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:47.915 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.915 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.915 DEBUG:boto:Method: GET 2024-04-16T18:29:47.915 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.915 DEBUG:boto:Data: 2024-04-16T18:29:47.915 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.915 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.915 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.915 DEBUG:boto:Params: {} 2024-04-16T18:29:47.915 DEBUG:boto:Token: None 2024-04-16T18:29:47.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs= 2024-04-16T18:29:47.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs=', 'Content-Length': '0'} 2024-04-16T18:29:47.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c460ae4a3d25994f-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.919 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:47.919 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:47.920 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.920 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.920 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.920 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.920 DEBUG:boto:Data: 2024-04-16T18:29:47.920 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.920 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.920 DEBUG:boto:Params: {} 2024-04-16T18:29:47.920 DEBUG:boto:Token: None 2024-04-16T18:29:47.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.921 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lEDCLV18uW8aQebyvjdpwNrxJuc= 2024-04-16T18:29:47.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lEDCLV18uW8aQebyvjdpwNrxJuc=', 'Content-Length': '0'} 2024-04-16T18:29:47.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000011752693fe9d01e-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.924 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:47.924 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:47.924 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.924 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.924 DEBUG:boto:Method: HEAD 2024-04-16T18:29:47.924 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.924 DEBUG:boto:Data: 2024-04-16T18:29:47.924 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.924 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.924 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.924 DEBUG:boto:Params: {} 2024-04-16T18:29:47.925 DEBUG:boto:Token: None 2024-04-16T18:29:47.925 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.925 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lEDCLV18uW8aQebyvjdpwNrxJuc= 2024-04-16T18:29:47.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lEDCLV18uW8aQebyvjdpwNrxJuc=', 'Content-Length': '0'} 2024-04-16T18:29:47.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063861cc9e5811d12-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:29:47.928 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:47.928 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:47.928 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.928 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.929 DEBUG:boto:Method: GET 2024-04-16T18:29:47.929 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.929 DEBUG:boto:Data: 2024-04-16T18:29:47.929 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.929 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.929 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.929 DEBUG:boto:Params: {} 2024-04-16T18:29:47.930 DEBUG:boto:Token: None 2024-04-16T18:29:47.930 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.930 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs= 2024-04-16T18:29:47.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs=', 'Content-Length': '0'} 2024-04-16T18:29:47.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002564afb25503891a-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.978 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:47.979 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:47.979 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.979 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.979 DEBUG:boto:Method: GET 2024-04-16T18:29:47.979 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.979 DEBUG:boto:Data: 2024-04-16T18:29:47.979 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.979 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.979 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.979 DEBUG:boto:Params: {} 2024-04-16T18:29:47.979 DEBUG:boto:Token: None 2024-04-16T18:29:47.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:29:47.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs= 2024-04-16T18:29:47.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/PCLbRz1pSyDPiQ2hEL/ssqOmYs=', 'Content-Length': '0'} 2024-04-16T18:29:47.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fae9d76230d37d7b-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.983 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.983 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.983 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:47.983 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:47.983 DEBUG:boto:Method: GET 2024-04-16T18:29:47.983 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:29:47.984 DEBUG:boto:Data: 2024-04-16T18:29:47.984 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.984 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:47.984 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.984 DEBUG:boto:Params: {} 2024-04-16T18:29:47.984 DEBUG:boto:Token: None 2024-04-16T18:29:47.984 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.988 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b5f099d2b64ce2a-00661ec39b-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:47 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:47.988 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:47.988 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:47.988 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:47.989 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:47.989 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:47.989 DEBUG:boto:Method: GET 2024-04-16T18:29:47.989 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:29:47.989 DEBUG:boto:Data: 2024-04-16T18:29:47.989 DEBUG:boto:Headers: {} 2024-04-16T18:29:47.989 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:47.989 DEBUG:boto:Port: 8000 2024-04-16T18:29:47.989 DEBUG:boto:Params: {} 2024-04-16T18:29:47.989 DEBUG:boto:Token: None 2024-04-16T18:29:47.990 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:47 GMT /lybtug-52/ 2024-04-16T18:29:47.990 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4= 2024-04-16T18:29:47.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:47 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:07mQSAZLg7PAyZHx4KJrky5XkS4=', 'Content-Length': '0'} 2024-04-16T18:29:47.997 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015c9126eb08e7915-00661ec39b-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.038 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.039 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:48.039 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:48.039 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.039 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:29:48.039 DEBUG:boto:Data: 2024-04-16T18:29:48.039 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.039 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.039 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.039 DEBUG:boto:Params: {} 2024-04-16T18:29:48.040 DEBUG:boto:Token: None 2024-04-16T18:29:48.040 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj38 2024-04-16T18:29:48.040 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:idE642bRsyMiFlHNIY8cHwzZBMc= 2024-04-16T18:29:48.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:idE642bRsyMiFlHNIY8cHwzZBMc=', 'Content-Length': '0'} 2024-04-16T18:29:48.043 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026c29636471db8a9-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.043 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:48.043 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:48.043 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.043 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:29:48.044 DEBUG:boto:Data: 2024-04-16T18:29:48.044 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.044 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.044 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.044 DEBUG:boto:Params: {} 2024-04-16T18:29:48.044 DEBUG:boto:Token: None 2024-04-16T18:29:48.044 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj38 2024-04-16T18:29:48.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:idE642bRsyMiFlHNIY8cHwzZBMc= 2024-04-16T18:29:48.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:idE642bRsyMiFlHNIY8cHwzZBMc=', 'Content-Length': '0'} 2024-04-16T18:29:48.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000887c63689ad58d3f-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:29:48.047 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:48.047 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:48.047 DEBUG:boto:Method: GET 2024-04-16T18:29:48.048 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:29:48.048 DEBUG:boto:Data: 2024-04-16T18:29:48.048 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.048 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.048 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.048 DEBUG:boto:Params: {} 2024-04-16T18:29:48.048 DEBUG:boto:Token: None 2024-04-16T18:29:48.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj38 2024-04-16T18:29:48.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mup/1KUo+N6lnn07PIfd+Nto8iI= 2024-04-16T18:29:48.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mup/1KUo+N6lnn07PIfd+Nto8iI=', 'Content-Length': '0'} 2024-04-16T18:29:48.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dcb0b9763a23d47-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.094 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:29:48.095 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:29:48.095 DEBUG:boto:Method: GET 2024-04-16T18:29:48.095 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:29:48.095 DEBUG:boto:Data: 2024-04-16T18:29:48.095 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.095 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.095 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.095 DEBUG:boto:Params: {} 2024-04-16T18:29:48.095 DEBUG:boto:Token: None 2024-04-16T18:29:48.096 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj38 2024-04-16T18:29:48.096 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Mup/1KUo+N6lnn07PIfd+Nto8iI= 2024-04-16T18:29:48.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Mup/1KUo+N6lnn07PIfd+Nto8iI=', 'Content-Length': '0'} 2024-04-16T18:29:48.098 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c4237840d8f03fa-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.099 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.099 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.099 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:48.099 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:48.099 DEBUG:boto:Method: GET 2024-04-16T18:29:48.099 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:29:48.099 DEBUG:boto:Data: 2024-04-16T18:29:48.099 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.100 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.100 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.100 DEBUG:boto:Params: {} 2024-04-16T18:29:48.100 DEBUG:boto:Token: None 2024-04-16T18:29:48.100 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.100 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.104 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d5ee793adf01cf8-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.146 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.147 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.147 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.147 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:48.147 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:29:48.147 DEBUG:boto:Method: GET 2024-04-16T18:29:48.147 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:29:48.147 DEBUG:boto:Data: 2024-04-16T18:29:48.147 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.147 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.147 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.147 DEBUG:boto:Params: {} 2024-04-16T18:29:48.148 DEBUG:boto:Token: None 2024-04-16T18:29:48.148 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.154 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003c8c72155efb811-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.155 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:29:48.155 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.155 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.155 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.155 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.155 DEBUG:boto:Method: GET 2024-04-16T18:29:48.155 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.156 DEBUG:boto:Data: 2024-04-16T18:29:48.156 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.156 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.156 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.156 DEBUG:boto:Params: {} 2024-04-16T18:29:48.156 DEBUG:boto:Token: None 2024-04-16T18:29:48.156 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.156 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU= 2024-04-16T18:29:48.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU=', 'Content-Length': '0'} 2024-04-16T18:29:48.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048d9cbbf0238f388-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.203 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.203 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.203 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.203 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.203 DEBUG:boto:Method: GET 2024-04-16T18:29:48.203 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.203 DEBUG:boto:Data: 2024-04-16T18:29:48.203 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.203 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.204 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.204 DEBUG:boto:Params: {} 2024-04-16T18:29:48.204 DEBUG:boto:Token: None 2024-04-16T18:29:48.204 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU= 2024-04-16T18:29:48.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU=', 'Content-Length': '0'} 2024-04-16T18:29:48.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000049f85b0f7867e8e-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.207 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.208 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.208 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.208 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.208 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.208 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.208 DEBUG:boto:Data: 2024-04-16T18:29:48.208 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.208 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.208 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.208 DEBUG:boto:Params: {} 2024-04-16T18:29:48.208 DEBUG:boto:Token: None 2024-04-16T18:29:48.209 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.209 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LasA1dhmKsaLNJEUiPd0FkAHh8w= 2024-04-16T18:29:48.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LasA1dhmKsaLNJEUiPd0FkAHh8w=', 'Content-Length': '0'} 2024-04-16T18:29:48.212 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d2c2cbe89aaca23-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.212 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.212 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.213 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.213 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.213 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.213 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.213 DEBUG:boto:Data: 2024-04-16T18:29:48.213 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.213 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.213 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.213 DEBUG:boto:Params: {} 2024-04-16T18:29:48.213 DEBUG:boto:Token: None 2024-04-16T18:29:48.214 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.214 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LasA1dhmKsaLNJEUiPd0FkAHh8w= 2024-04-16T18:29:48.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LasA1dhmKsaLNJEUiPd0FkAHh8w=', 'Content-Length': '0'} 2024-04-16T18:29:48.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000416d9d0b010d4fe3-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.217 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:29:48.217 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.217 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.217 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.217 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.218 DEBUG:boto:Method: GET 2024-04-16T18:29:48.218 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.218 DEBUG:boto:Data: 2024-04-16T18:29:48.218 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.218 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.218 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.218 DEBUG:boto:Params: {} 2024-04-16T18:29:48.218 DEBUG:boto:Token: None 2024-04-16T18:29:48.218 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.218 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU= 2024-04-16T18:29:48.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU=', 'Content-Length': '0'} 2024-04-16T18:29:48.222 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a29c8af01c5bad5c-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.262 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.262 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.263 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.263 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.263 DEBUG:boto:Method: GET 2024-04-16T18:29:48.263 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.263 DEBUG:boto:Data: 2024-04-16T18:29:48.263 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.263 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.263 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.263 DEBUG:boto:Params: {} 2024-04-16T18:29:48.264 DEBUG:boto:Token: None 2024-04-16T18:29:48.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:29:48.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU= 2024-04-16T18:29:48.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sZtz6TyPR+lEQ1obK1vcCbcTFsU=', 'Content-Length': '0'} 2024-04-16T18:29:48.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ef682aa5820cf42-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.268 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.268 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.268 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.268 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.268 DEBUG:boto:Method: GET 2024-04-16T18:29:48.268 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:29:48.268 DEBUG:boto:Data: 2024-04-16T18:29:48.268 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.268 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.269 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.269 DEBUG:boto:Params: {} 2024-04-16T18:29:48.269 DEBUG:boto:Token: None 2024-04-16T18:29:48.269 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.269 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2e837c60bae3c87-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.272 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.273 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.273 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.273 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.273 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.273 DEBUG:boto:Method: GET 2024-04-16T18:29:48.273 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:29:48.274 DEBUG:boto:Data: 2024-04-16T18:29:48.274 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.274 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.274 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.274 DEBUG:boto:Params: {} 2024-04-16T18:29:48.274 DEBUG:boto:Token: None 2024-04-16T18:29:48.274 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.274 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.280 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009d919cd70f5e9a7-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.322 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.323 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.323 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.323 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.323 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:29:48.323 DEBUG:boto:Data: 2024-04-16T18:29:48.323 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.323 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.323 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.323 DEBUG:boto:Params: {} 2024-04-16T18:29:48.324 DEBUG:boto:Token: None 2024-04-16T18:29:48.324 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44 2024-04-16T18:29:48.324 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:osUgfH3PqLYcfsHabKX3KvnLtEQ= 2024-04-16T18:29:48.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:osUgfH3PqLYcfsHabKX3KvnLtEQ=', 'Content-Length': '0'} 2024-04-16T18:29:48.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000675c1f7784bcf14e-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.328 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.328 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.328 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.328 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:29:48.328 DEBUG:boto:Data: 2024-04-16T18:29:48.328 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.328 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.329 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.329 DEBUG:boto:Params: {} 2024-04-16T18:29:48.329 DEBUG:boto:Token: None 2024-04-16T18:29:48.329 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44 2024-04-16T18:29:48.329 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:osUgfH3PqLYcfsHabKX3KvnLtEQ= 2024-04-16T18:29:48.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:osUgfH3PqLYcfsHabKX3KvnLtEQ=', 'Content-Length': '0'} 2024-04-16T18:29:48.332 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1a7775f2c817596-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.332 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:29:48.332 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.333 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.333 DEBUG:boto:Method: GET 2024-04-16T18:29:48.333 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:29:48.333 DEBUG:boto:Data: 2024-04-16T18:29:48.333 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.333 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.333 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.333 DEBUG:boto:Params: {} 2024-04-16T18:29:48.333 DEBUG:boto:Token: None 2024-04-16T18:29:48.333 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44 2024-04-16T18:29:48.333 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tt6B8csmCZ3Crceu+aaUb0omH+g= 2024-04-16T18:29:48.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tt6B8csmCZ3Crceu+aaUb0omH+g=', 'Content-Length': '0'} 2024-04-16T18:29:48.336 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000894fc611036c4c4d-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.378 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:29:48.378 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:29:48.379 DEBUG:boto:Method: GET 2024-04-16T18:29:48.379 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:29:48.379 DEBUG:boto:Data: 2024-04-16T18:29:48.379 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.379 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.379 DEBUG:boto:Params: {} 2024-04-16T18:29:48.379 DEBUG:boto:Token: None 2024-04-16T18:29:48.379 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj44 2024-04-16T18:29:48.379 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tt6B8csmCZ3Crceu+aaUb0omH+g= 2024-04-16T18:29:48.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tt6B8csmCZ3Crceu+aaUb0omH+g=', 'Content-Length': '0'} 2024-04-16T18:29:48.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000883519da8b969fea-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.383 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.383 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.383 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.383 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.383 DEBUG:boto:Method: GET 2024-04-16T18:29:48.383 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:29:48.383 DEBUG:boto:Data: 2024-04-16T18:29:48.383 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.383 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.384 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.384 DEBUG:boto:Params: {} 2024-04-16T18:29:48.384 DEBUG:boto:Token: None 2024-04-16T18:29:48.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.387 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a53d06b7fe16cfde-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.430 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.431 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.431 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.431 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.431 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:29:48.431 DEBUG:boto:Method: GET 2024-04-16T18:29:48.431 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:29:48.431 DEBUG:boto:Data: 2024-04-16T18:29:48.431 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.432 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.432 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.432 DEBUG:boto:Params: {} 2024-04-16T18:29:48.432 DEBUG:boto:Token: None 2024-04-16T18:29:48.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f78aa73481a802b9-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.439 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.439 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:29:48.439 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.440 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.440 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.440 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.440 DEBUG:boto:Method: GET 2024-04-16T18:29:48.440 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.440 DEBUG:boto:Data: 2024-04-16T18:29:48.440 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.440 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.440 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.440 DEBUG:boto:Params: {} 2024-04-16T18:29:48.441 DEBUG:boto:Token: None 2024-04-16T18:29:48.441 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.441 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY= 2024-04-16T18:29:48.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY=', 'Content-Length': '0'} 2024-04-16T18:29:48.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc54547a579e8d66-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.486 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.486 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.487 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.487 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.487 DEBUG:boto:Method: GET 2024-04-16T18:29:48.487 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.487 DEBUG:boto:Data: 2024-04-16T18:29:48.487 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.487 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.487 DEBUG:boto:Params: {} 2024-04-16T18:29:48.487 DEBUG:boto:Token: None 2024-04-16T18:29:48.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY= 2024-04-16T18:29:48.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY=', 'Content-Length': '0'} 2024-04-16T18:29:48.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001339821b7c3403d1-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.491 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.491 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.492 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.492 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.492 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.492 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.492 DEBUG:boto:Data: 2024-04-16T18:29:48.492 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.492 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.492 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.492 DEBUG:boto:Params: {} 2024-04-16T18:29:48.492 DEBUG:boto:Token: None 2024-04-16T18:29:48.493 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X9SVNCYtG0+eTWhu59hfrki+ylU= 2024-04-16T18:29:48.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X9SVNCYtG0+eTWhu59hfrki+ylU=', 'Content-Length': '0'} 2024-04-16T18:29:48.496 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000567e7e493b317d10-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.496 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.496 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.496 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.496 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.496 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.496 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.496 DEBUG:boto:Data: 2024-04-16T18:29:48.497 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.497 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.497 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.497 DEBUG:boto:Params: {} 2024-04-16T18:29:48.497 DEBUG:boto:Token: None 2024-04-16T18:29:48.497 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X9SVNCYtG0+eTWhu59hfrki+ylU= 2024-04-16T18:29:48.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X9SVNCYtG0+eTWhu59hfrki+ylU=', 'Content-Length': '0'} 2024-04-16T18:29:48.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c66f7efc6a48eb1-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.500 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:29:48.501 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.501 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.501 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.501 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.501 DEBUG:boto:Method: GET 2024-04-16T18:29:48.501 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.501 DEBUG:boto:Data: 2024-04-16T18:29:48.501 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.501 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.501 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.502 DEBUG:boto:Params: {} 2024-04-16T18:29:48.502 DEBUG:boto:Token: None 2024-04-16T18:29:48.502 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.502 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY= 2024-04-16T18:29:48.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY=', 'Content-Length': '0'} 2024-04-16T18:29:48.505 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f8237e8be42d382-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.546 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.547 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.547 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.547 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.547 DEBUG:boto:Method: GET 2024-04-16T18:29:48.547 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.547 DEBUG:boto:Data: 2024-04-16T18:29:48.547 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.547 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.547 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.547 DEBUG:boto:Params: {} 2024-04-16T18:29:48.547 DEBUG:boto:Token: None 2024-04-16T18:29:48.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:29:48.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY= 2024-04-16T18:29:48.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JxNlsS8hsQsIUs5DDHgSjqlurpY=', 'Content-Length': '0'} 2024-04-16T18:29:48.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000424d7eb4d23e4847-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.551 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.551 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.551 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.551 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.551 DEBUG:boto:Method: GET 2024-04-16T18:29:48.551 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:29:48.551 DEBUG:boto:Data: 2024-04-16T18:29:48.552 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.552 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.552 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.552 DEBUG:boto:Params: {} 2024-04-16T18:29:48.552 DEBUG:boto:Token: None 2024-04-16T18:29:48.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a729691c02c46ebf-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.556 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.556 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.556 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.557 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.557 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.557 DEBUG:boto:Method: GET 2024-04-16T18:29:48.557 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:29:48.557 DEBUG:boto:Data: 2024-04-16T18:29:48.557 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.557 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.557 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.557 DEBUG:boto:Params: {} 2024-04-16T18:29:48.557 DEBUG:boto:Token: None 2024-04-16T18:29:48.557 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.558 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd35665104cdd5ae-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.606 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.607 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.607 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.607 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.607 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:29:48.607 DEBUG:boto:Data: 2024-04-16T18:29:48.607 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.607 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.607 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.607 DEBUG:boto:Params: {} 2024-04-16T18:29:48.607 DEBUG:boto:Token: None 2024-04-16T18:29:48.608 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46 2024-04-16T18:29:48.608 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l7iYhunB3l3mwlJMkSZn/DjDUhg= 2024-04-16T18:29:48.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l7iYhunB3l3mwlJMkSZn/DjDUhg=', 'Content-Length': '0'} 2024-04-16T18:29:48.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d333b7c003fdc6a0-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.611 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.611 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.611 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.611 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:29:48.611 DEBUG:boto:Data: 2024-04-16T18:29:48.611 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.612 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.612 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.612 DEBUG:boto:Params: {} 2024-04-16T18:29:48.612 DEBUG:boto:Token: None 2024-04-16T18:29:48.612 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46 2024-04-16T18:29:48.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:l7iYhunB3l3mwlJMkSZn/DjDUhg= 2024-04-16T18:29:48.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l7iYhunB3l3mwlJMkSZn/DjDUhg=', 'Content-Length': '0'} 2024-04-16T18:29:48.615 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e1a847fc57a3ab7-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.615 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:29:48.615 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.615 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.616 DEBUG:boto:Method: GET 2024-04-16T18:29:48.616 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:29:48.616 DEBUG:boto:Data: 2024-04-16T18:29:48.616 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.616 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.616 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.616 DEBUG:boto:Params: {} 2024-04-16T18:29:48.616 DEBUG:boto:Token: None 2024-04-16T18:29:48.616 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46 2024-04-16T18:29:48.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8iiILrsV0ZXg0VjKIo3kM0+u/sk= 2024-04-16T18:29:48.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8iiILrsV0ZXg0VjKIo3kM0+u/sk=', 'Content-Length': '0'} 2024-04-16T18:29:48.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:14 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5873146a74e1944-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.666 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:29:48.666 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:29:48.667 DEBUG:boto:Method: GET 2024-04-16T18:29:48.667 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:29:48.667 DEBUG:boto:Data: 2024-04-16T18:29:48.667 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.667 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.667 DEBUG:boto:Params: {} 2024-04-16T18:29:48.667 DEBUG:boto:Token: None 2024-04-16T18:29:48.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj46 2024-04-16T18:29:48.667 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8iiILrsV0ZXg0VjKIo3kM0+u/sk= 2024-04-16T18:29:48.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8iiILrsV0ZXg0VjKIo3kM0+u/sk=', 'Content-Length': '0'} 2024-04-16T18:29:48.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b9ef002f0c399e8-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.671 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.671 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.671 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.671 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.671 DEBUG:boto:Method: GET 2024-04-16T18:29:48.671 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:29:48.671 DEBUG:boto:Data: 2024-04-16T18:29:48.671 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.672 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.672 DEBUG:boto:Params: {} 2024-04-16T18:29:48.672 DEBUG:boto:Token: None 2024-04-16T18:29:48.672 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.675 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005fce97116907086d-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.718 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.719 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.719 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.719 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.719 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:29:48.719 DEBUG:boto:Method: GET 2024-04-16T18:29:48.719 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:29:48.719 DEBUG:boto:Data: 2024-04-16T18:29:48.719 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.719 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.720 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.720 DEBUG:boto:Params: {} 2024-04-16T18:29:48.720 DEBUG:boto:Token: None 2024-04-16T18:29:48.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.727 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060e707d3bbcb2946-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.727 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.727 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:29:48.727 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.728 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.728 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.728 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.728 DEBUG:boto:Method: GET 2024-04-16T18:29:48.728 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.728 DEBUG:boto:Data: 2024-04-16T18:29:48.728 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.728 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.728 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.728 DEBUG:boto:Params: {} 2024-04-16T18:29:48.729 DEBUG:boto:Token: None 2024-04-16T18:29:48.729 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.729 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k= 2024-04-16T18:29:48.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k=', 'Content-Length': '0'} 2024-04-16T18:29:48.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089883c5a78461a46-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.774 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.774 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.775 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.775 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.775 DEBUG:boto:Method: GET 2024-04-16T18:29:48.775 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.775 DEBUG:boto:Data: 2024-04-16T18:29:48.775 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.775 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.775 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.775 DEBUG:boto:Params: {} 2024-04-16T18:29:48.775 DEBUG:boto:Token: None 2024-04-16T18:29:48.775 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k= 2024-04-16T18:29:48.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k=', 'Content-Length': '0'} 2024-04-16T18:29:48.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e63d589af216ac1b-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.779 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.779 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.779 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.779 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.779 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.779 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.779 DEBUG:boto:Data: 2024-04-16T18:29:48.779 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.780 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.780 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.780 DEBUG:boto:Params: {} 2024-04-16T18:29:48.780 DEBUG:boto:Token: None 2024-04-16T18:29:48.780 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.780 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gv8dv9U2lpNmvX7DsLsChFwG+z0= 2024-04-16T18:29:48.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gv8dv9U2lpNmvX7DsLsChFwG+z0=', 'Content-Length': '0'} 2024-04-16T18:29:48.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008124943a0c6ed6c6-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.783 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.783 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.783 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.783 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.784 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.784 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.784 DEBUG:boto:Data: 2024-04-16T18:29:48.784 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.784 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.784 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.784 DEBUG:boto:Params: {} 2024-04-16T18:29:48.784 DEBUG:boto:Token: None 2024-04-16T18:29:48.784 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.784 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Gv8dv9U2lpNmvX7DsLsChFwG+z0= 2024-04-16T18:29:48.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Gv8dv9U2lpNmvX7DsLsChFwG+z0=', 'Content-Length': '0'} 2024-04-16T18:29:48.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cc005217221eee9-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:29:48.788 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.788 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.788 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.788 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.788 DEBUG:boto:Method: GET 2024-04-16T18:29:48.788 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.788 DEBUG:boto:Data: 2024-04-16T18:29:48.788 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.789 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.789 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.789 DEBUG:boto:Params: {} 2024-04-16T18:29:48.789 DEBUG:boto:Token: None 2024-04-16T18:29:48.789 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.789 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k= 2024-04-16T18:29:48.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k=', 'Content-Length': '0'} 2024-04-16T18:29:48.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000995c582915d0dfb3-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.834 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.835 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.835 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.835 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.835 DEBUG:boto:Method: GET 2024-04-16T18:29:48.835 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.835 DEBUG:boto:Data: 2024-04-16T18:29:48.835 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.835 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.835 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.835 DEBUG:boto:Params: {} 2024-04-16T18:29:48.836 DEBUG:boto:Token: None 2024-04-16T18:29:48.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:29:48.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k= 2024-04-16T18:29:48.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lSbO0+1TcV9w2JZ36RiPSh64o/k=', 'Content-Length': '0'} 2024-04-16T18:29:48.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006206ec1260cf1ffe-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.839 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.840 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.840 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:48.840 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:48.840 DEBUG:boto:Method: GET 2024-04-16T18:29:48.840 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:29:48.840 DEBUG:boto:Data: 2024-04-16T18:29:48.840 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.840 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.840 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.840 DEBUG:boto:Params: {} 2024-04-16T18:29:48.840 DEBUG:boto:Token: None 2024-04-16T18:29:48.841 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.841 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f3736f3c7172d63-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.845 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.845 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.845 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.845 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:48.845 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:48.846 DEBUG:boto:Method: GET 2024-04-16T18:29:48.846 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:29:48.846 DEBUG:boto:Data: 2024-04-16T18:29:48.846 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.846 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.846 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.846 DEBUG:boto:Params: {} 2024-04-16T18:29:48.846 DEBUG:boto:Token: None 2024-04-16T18:29:48.846 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.846 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.853 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a3523972494e7f4-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.894 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:48.895 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.895 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.895 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.895 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:29:48.896 DEBUG:boto:Data: 2024-04-16T18:29:48.896 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.896 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.896 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.896 DEBUG:boto:Params: {} 2024-04-16T18:29:48.896 DEBUG:boto:Token: None 2024-04-16T18:29:48.896 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51 2024-04-16T18:29:48.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JFGdHeindxUJ+HwnCq9OcGY7xAI= 2024-04-16T18:29:48.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JFGdHeindxUJ+HwnCq9OcGY7xAI=', 'Content-Length': '0'} 2024-04-16T18:29:48.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000970aea433f0aa87c-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.900 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.900 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.900 DEBUG:boto:Method: HEAD 2024-04-16T18:29:48.900 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:29:48.900 DEBUG:boto:Data: 2024-04-16T18:29:48.900 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.901 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.901 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.901 DEBUG:boto:Params: {} 2024-04-16T18:29:48.901 DEBUG:boto:Token: None 2024-04-16T18:29:48.901 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51 2024-04-16T18:29:48.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JFGdHeindxUJ+HwnCq9OcGY7xAI= 2024-04-16T18:29:48.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JFGdHeindxUJ+HwnCq9OcGY7xAI=', 'Content-Length': '0'} 2024-04-16T18:29:48.905 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d682373fa3ece35f-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.905 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:29:48.905 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.905 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.905 DEBUG:boto:Method: GET 2024-04-16T18:29:48.905 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:29:48.906 DEBUG:boto:Data: 2024-04-16T18:29:48.906 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.906 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.906 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.906 DEBUG:boto:Params: {} 2024-04-16T18:29:48.906 DEBUG:boto:Token: None 2024-04-16T18:29:48.906 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51 2024-04-16T18:29:48.907 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kJ/tqIhQr+XV9PHKY6daxFwOUEA= 2024-04-16T18:29:48.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kJ/tqIhQr+XV9PHKY6daxFwOUEA=', 'Content-Length': '0'} 2024-04-16T18:29:48.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:21:21 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8a1e2d8b5e3684d-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.954 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:29:48.955 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:29:48.955 DEBUG:boto:Method: GET 2024-04-16T18:29:48.955 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:29:48.955 DEBUG:boto:Data: 2024-04-16T18:29:48.955 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.955 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:48.955 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.955 DEBUG:boto:Params: {} 2024-04-16T18:29:48.956 DEBUG:boto:Token: None 2024-04-16T18:29:48.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/obj51 2024-04-16T18:29:48.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kJ/tqIhQr+XV9PHKY6daxFwOUEA= 2024-04-16T18:29:48.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kJ/tqIhQr+XV9PHKY6daxFwOUEA=', 'Content-Length': '0'} 2024-04-16T18:29:48.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c165d6fa04756921-00661ec39c-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:48.959 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:48.959 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:48.959 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:48.960 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:48.960 DEBUG:boto:Method: GET 2024-04-16T18:29:48.960 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:29:48.960 DEBUG:boto:Data: 2024-04-16T18:29:48.960 DEBUG:boto:Headers: {} 2024-04-16T18:29:48.960 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:48.960 DEBUG:boto:Port: 8000 2024-04-16T18:29:48.960 DEBUG:boto:Params: {} 2024-04-16T18:29:48.960 DEBUG:boto:Token: None 2024-04-16T18:29:48.960 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:48 GMT /lybtug-52/ 2024-04-16T18:29:48.961 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4= 2024-04-16T18:29:48.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:48 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:87ZFVoEm6z6hX0/pzNgs/Qzo8h4=', 'Content-Length': '0'} 2024-04-16T18:29:48.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b34c65e9ec5bb15-00661ec39c-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:48 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.006 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.007 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.007 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.007 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:49.007 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:29:49.007 DEBUG:boto:Method: GET 2024-04-16T18:29:49.007 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:29:49.008 DEBUG:boto:Data: 2024-04-16T18:29:49.008 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.008 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.008 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.008 DEBUG:boto:Params: {} 2024-04-16T18:29:49.008 DEBUG:boto:Token: None 2024-04-16T18:29:49.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.015 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed0561464458b57d-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.016 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.016 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:29:49.016 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.016 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.016 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.016 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.016 DEBUG:boto:Method: GET 2024-04-16T18:29:49.016 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.017 DEBUG:boto:Data: 2024-04-16T18:29:49.017 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.017 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.017 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.017 DEBUG:boto:Params: {} 2024-04-16T18:29:49.017 DEBUG:boto:Token: None 2024-04-16T18:29:49.017 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.017 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo= 2024-04-16T18:29:49.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo=', 'Content-Length': '0'} 2024-04-16T18:29:49.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1ba6c5dc55b3742-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.062 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.063 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.063 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.063 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.063 DEBUG:boto:Method: GET 2024-04-16T18:29:49.063 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.063 DEBUG:boto:Data: 2024-04-16T18:29:49.063 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.063 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.063 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.063 DEBUG:boto:Params: {} 2024-04-16T18:29:49.063 DEBUG:boto:Token: None 2024-04-16T18:29:49.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo= 2024-04-16T18:29:49.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo=', 'Content-Length': '0'} 2024-04-16T18:29:49.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7a1f51d95425049-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.067 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.067 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.067 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.067 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.067 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.068 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.068 DEBUG:boto:Data: 2024-04-16T18:29:49.068 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.068 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.068 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.068 DEBUG:boto:Params: {} 2024-04-16T18:29:49.068 DEBUG:boto:Token: None 2024-04-16T18:29:49.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Sikca4H1C1Cn0AkHH9sWndmdxEs= 2024-04-16T18:29:49.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sikca4H1C1Cn0AkHH9sWndmdxEs=', 'Content-Length': '0'} 2024-04-16T18:29:49.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aafdef7adef5e723-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.071 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.072 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.072 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.072 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.072 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.072 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.072 DEBUG:boto:Data: 2024-04-16T18:29:49.072 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.072 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.072 DEBUG:boto:Params: {} 2024-04-16T18:29:49.072 DEBUG:boto:Token: None 2024-04-16T18:29:49.073 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.073 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Sikca4H1C1Cn0AkHH9sWndmdxEs= 2024-04-16T18:29:49.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sikca4H1C1Cn0AkHH9sWndmdxEs=', 'Content-Length': '0'} 2024-04-16T18:29:49.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b57093d310be59-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:29:49.076 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.076 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.076 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.076 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.076 DEBUG:boto:Method: GET 2024-04-16T18:29:49.076 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.076 DEBUG:boto:Data: 2024-04-16T18:29:49.076 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.077 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.077 DEBUG:boto:Params: {} 2024-04-16T18:29:49.077 DEBUG:boto:Token: None 2024-04-16T18:29:49.077 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.077 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo= 2024-04-16T18:29:49.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo=', 'Content-Length': '0'} 2024-04-16T18:29:49.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9a93a8523e0bb41-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.126 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.127 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.127 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.127 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.127 DEBUG:boto:Method: GET 2024-04-16T18:29:49.127 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.127 DEBUG:boto:Data: 2024-04-16T18:29:49.127 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.127 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.127 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.128 DEBUG:boto:Params: {} 2024-04-16T18:29:49.128 DEBUG:boto:Token: None 2024-04-16T18:29:49.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:29:49.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo= 2024-04-16T18:29:49.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vOrFgf4I54zKpLnrcA1lKAQySAo=', 'Content-Length': '0'} 2024-04-16T18:29:49.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2bb9cca4b7a089f-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.131 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.131 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.132 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.132 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.132 DEBUG:boto:Method: GET 2024-04-16T18:29:49.132 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:29:49.132 DEBUG:boto:Data: 2024-04-16T18:29:49.132 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.132 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.132 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.132 DEBUG:boto:Params: {} 2024-04-16T18:29:49.132 DEBUG:boto:Token: None 2024-04-16T18:29:49.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.133 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b83b39c70fb252af-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.137 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.137 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.137 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.137 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.137 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.138 DEBUG:boto:Method: GET 2024-04-16T18:29:49.138 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:29:49.138 DEBUG:boto:Data: 2024-04-16T18:29:49.138 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.138 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.138 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.138 DEBUG:boto:Params: {} 2024-04-16T18:29:49.138 DEBUG:boto:Token: None 2024-04-16T18:29:49.138 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.139 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.145 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000608dd9cbff760440-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.190 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.191 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.191 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.191 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.191 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:29:49.191 DEBUG:boto:Data: 2024-04-16T18:29:49.191 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.191 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.192 DEBUG:boto:Params: {} 2024-04-16T18:29:49.192 DEBUG:boto:Token: None 2024-04-16T18:29:49.192 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55 2024-04-16T18:29:49.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AgRlHQV0vOfWIkIcUSQeTqpujrw= 2024-04-16T18:29:49.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AgRlHQV0vOfWIkIcUSQeTqpujrw=', 'Content-Length': '0'} 2024-04-16T18:29:49.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c01d10f668181ea7-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.196 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.196 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.196 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.196 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:29:49.196 DEBUG:boto:Data: 2024-04-16T18:29:49.196 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.196 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.196 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.196 DEBUG:boto:Params: {} 2024-04-16T18:29:49.196 DEBUG:boto:Token: None 2024-04-16T18:29:49.197 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55 2024-04-16T18:29:49.197 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AgRlHQV0vOfWIkIcUSQeTqpujrw= 2024-04-16T18:29:49.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AgRlHQV0vOfWIkIcUSQeTqpujrw=', 'Content-Length': '0'} 2024-04-16T18:29:49.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053c74f0db5126094-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.200 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:29:49.200 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.200 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.200 DEBUG:boto:Method: GET 2024-04-16T18:29:49.200 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:29:49.200 DEBUG:boto:Data: 2024-04-16T18:29:49.200 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.201 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.201 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.201 DEBUG:boto:Params: {} 2024-04-16T18:29:49.201 DEBUG:boto:Token: None 2024-04-16T18:29:49.201 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55 2024-04-16T18:29:49.201 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IZUf+qlpPMZuLMXtpC74EVuEruE= 2024-04-16T18:29:49.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IZUf+qlpPMZuLMXtpC74EVuEruE=', 'Content-Length': '0'} 2024-04-16T18:29:49.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:05 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cd0439e46cdb476-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.246 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:29:49.247 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:29:49.247 DEBUG:boto:Method: GET 2024-04-16T18:29:49.247 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:29:49.247 DEBUG:boto:Data: 2024-04-16T18:29:49.247 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.247 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.247 DEBUG:boto:Params: {} 2024-04-16T18:29:49.247 DEBUG:boto:Token: None 2024-04-16T18:29:49.248 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj55 2024-04-16T18:29:49.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IZUf+qlpPMZuLMXtpC74EVuEruE= 2024-04-16T18:29:49.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IZUf+qlpPMZuLMXtpC74EVuEruE=', 'Content-Length': '0'} 2024-04-16T18:29:49.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040194c7979734e51-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.251 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.251 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.251 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.251 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.252 DEBUG:boto:Method: GET 2024-04-16T18:29:49.252 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:29:49.252 DEBUG:boto:Data: 2024-04-16T18:29:49.252 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.252 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.252 DEBUG:boto:Params: {} 2024-04-16T18:29:49.252 DEBUG:boto:Token: None 2024-04-16T18:29:49.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005713e957dc2adb5a-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.298 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.299 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.299 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.299 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.299 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:29:49.299 DEBUG:boto:Method: GET 2024-04-16T18:29:49.299 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:29:49.299 DEBUG:boto:Data: 2024-04-16T18:29:49.300 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.300 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.300 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.300 DEBUG:boto:Params: {} 2024-04-16T18:29:49.300 DEBUG:boto:Token: None 2024-04-16T18:29:49.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.300 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007bec52f8c0c72e55-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.308 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.308 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:29:49.308 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.308 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.308 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.308 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.309 DEBUG:boto:Method: GET 2024-04-16T18:29:49.309 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.309 DEBUG:boto:Data: 2024-04-16T18:29:49.309 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.309 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.309 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.309 DEBUG:boto:Params: {} 2024-04-16T18:29:49.309 DEBUG:boto:Token: None 2024-04-16T18:29:49.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY= 2024-04-16T18:29:49.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY=', 'Content-Length': '0'} 2024-04-16T18:29:49.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a7df7877e143014-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.358 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.358 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.359 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.359 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.359 DEBUG:boto:Method: GET 2024-04-16T18:29:49.359 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.359 DEBUG:boto:Data: 2024-04-16T18:29:49.359 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.359 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.359 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.359 DEBUG:boto:Params: {} 2024-04-16T18:29:49.359 DEBUG:boto:Token: None 2024-04-16T18:29:49.360 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY= 2024-04-16T18:29:49.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY=', 'Content-Length': '0'} 2024-04-16T18:29:49.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000138401e7b1225915-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.363 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.363 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.363 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.363 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.363 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.363 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.364 DEBUG:boto:Data: 2024-04-16T18:29:49.364 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.364 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.364 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.364 DEBUG:boto:Params: {} 2024-04-16T18:29:49.364 DEBUG:boto:Token: None 2024-04-16T18:29:49.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vzLk0+WR63F+xMuWAPF7wuJxr4s= 2024-04-16T18:29:49.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vzLk0+WR63F+xMuWAPF7wuJxr4s=', 'Content-Length': '0'} 2024-04-16T18:29:49.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091a0b09dd2d64897-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.368 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.368 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.368 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.368 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.368 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.368 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.368 DEBUG:boto:Data: 2024-04-16T18:29:49.368 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.368 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.369 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.369 DEBUG:boto:Params: {} 2024-04-16T18:29:49.369 DEBUG:boto:Token: None 2024-04-16T18:29:49.369 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vzLk0+WR63F+xMuWAPF7wuJxr4s= 2024-04-16T18:29:49.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vzLk0+WR63F+xMuWAPF7wuJxr4s=', 'Content-Length': '0'} 2024-04-16T18:29:49.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c7179d24a2553c4-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:29:49.372 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.372 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.373 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.373 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.373 DEBUG:boto:Method: GET 2024-04-16T18:29:49.373 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.373 DEBUG:boto:Data: 2024-04-16T18:29:49.373 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.373 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.373 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.373 DEBUG:boto:Params: {} 2024-04-16T18:29:49.373 DEBUG:boto:Token: None 2024-04-16T18:29:49.373 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.374 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY= 2024-04-16T18:29:49.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY=', 'Content-Length': '0'} 2024-04-16T18:29:49.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e4e6c9c2441448a-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.418 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.418 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.419 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.419 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.419 DEBUG:boto:Method: GET 2024-04-16T18:29:49.419 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.419 DEBUG:boto:Data: 2024-04-16T18:29:49.419 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.419 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.419 DEBUG:boto:Params: {} 2024-04-16T18:29:49.419 DEBUG:boto:Token: None 2024-04-16T18:29:49.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:29:49.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY= 2024-04-16T18:29:49.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3rFKhGVq1lXD0kA2+M9invwZdDY=', 'Content-Length': '0'} 2024-04-16T18:29:49.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044a947e578ba85dd-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.423 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.423 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.423 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.424 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.424 DEBUG:boto:Method: GET 2024-04-16T18:29:49.424 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:29:49.424 DEBUG:boto:Data: 2024-04-16T18:29:49.424 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.424 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.424 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.424 DEBUG:boto:Params: {} 2024-04-16T18:29:49.424 DEBUG:boto:Token: None 2024-04-16T18:29:49.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.425 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.429 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088a3957db4cac679-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.429 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.429 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.429 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.429 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.430 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.430 DEBUG:boto:Method: GET 2024-04-16T18:29:49.430 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:29:49.430 DEBUG:boto:Data: 2024-04-16T18:29:49.430 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.430 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.430 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.430 DEBUG:boto:Params: {} 2024-04-16T18:29:49.430 DEBUG:boto:Token: None 2024-04-16T18:29:49.430 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.431 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a88bc8fa4059516-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.478 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.479 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.479 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.479 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.479 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:29:49.479 DEBUG:boto:Data: 2024-04-16T18:29:49.479 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.479 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.479 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.480 DEBUG:boto:Params: {} 2024-04-16T18:29:49.480 DEBUG:boto:Token: None 2024-04-16T18:29:49.480 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57 2024-04-16T18:29:49.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lGo+GWaCYq7I1yB3clc/7Yz1ULQ= 2024-04-16T18:29:49.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lGo+GWaCYq7I1yB3clc/7Yz1ULQ=', 'Content-Length': '0'} 2024-04-16T18:29:49.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b64e0b4aa2592b38-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.483 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.483 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.484 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.484 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:29:49.484 DEBUG:boto:Data: 2024-04-16T18:29:49.484 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.484 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.484 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.484 DEBUG:boto:Params: {} 2024-04-16T18:29:49.484 DEBUG:boto:Token: None 2024-04-16T18:29:49.485 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57 2024-04-16T18:29:49.485 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lGo+GWaCYq7I1yB3clc/7Yz1ULQ= 2024-04-16T18:29:49.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lGo+GWaCYq7I1yB3clc/7Yz1ULQ=', 'Content-Length': '0'} 2024-04-16T18:29:49.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000860ff05568998e45-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.488 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:29:49.488 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.488 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.488 DEBUG:boto:Method: GET 2024-04-16T18:29:49.489 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:29:49.489 DEBUG:boto:Data: 2024-04-16T18:29:49.489 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.489 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.489 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.489 DEBUG:boto:Params: {} 2024-04-16T18:29:49.489 DEBUG:boto:Token: None 2024-04-16T18:29:49.489 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57 2024-04-16T18:29:49.489 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0Dw+Z1NHVGuS8lalNYjoO2Xtr34= 2024-04-16T18:29:49.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0Dw+Z1NHVGuS8lalNYjoO2Xtr34=', 'Content-Length': '0'} 2024-04-16T18:29:49.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004eafd4d5b8be1d85-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.538 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:29:49.539 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:29:49.539 DEBUG:boto:Method: GET 2024-04-16T18:29:49.539 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:29:49.539 DEBUG:boto:Data: 2024-04-16T18:29:49.539 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.539 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.539 DEBUG:boto:Params: {} 2024-04-16T18:29:49.539 DEBUG:boto:Token: None 2024-04-16T18:29:49.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj57 2024-04-16T18:29:49.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0Dw+Z1NHVGuS8lalNYjoO2Xtr34= 2024-04-16T18:29:49.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0Dw+Z1NHVGuS8lalNYjoO2Xtr34=', 'Content-Length': '0'} 2024-04-16T18:29:49.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f737f649193920c-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.543 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.543 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.543 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.543 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.544 DEBUG:boto:Method: GET 2024-04-16T18:29:49.544 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:29:49.544 DEBUG:boto:Data: 2024-04-16T18:29:49.544 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.544 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.544 DEBUG:boto:Params: {} 2024-04-16T18:29:49.544 DEBUG:boto:Token: None 2024-04-16T18:29:49.544 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098fd9bd1000d65e2-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.594 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.595 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.595 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.595 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.595 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:29:49.595 DEBUG:boto:Method: GET 2024-04-16T18:29:49.595 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:29:49.595 DEBUG:boto:Data: 2024-04-16T18:29:49.595 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.596 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.596 DEBUG:boto:Params: {} 2024-04-16T18:29:49.596 DEBUG:boto:Token: None 2024-04-16T18:29:49.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abe7c8484c7f7a54-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.603 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.603 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:29:49.603 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.603 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.603 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.603 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.604 DEBUG:boto:Method: GET 2024-04-16T18:29:49.604 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.604 DEBUG:boto:Data: 2024-04-16T18:29:49.604 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.604 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.604 DEBUG:boto:Params: {} 2024-04-16T18:29:49.604 DEBUG:boto:Token: None 2024-04-16T18:29:49.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0= 2024-04-16T18:29:49.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0=', 'Content-Length': '0'} 2024-04-16T18:29:49.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b9e609487365397-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.654 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.654 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.655 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.655 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.655 DEBUG:boto:Method: GET 2024-04-16T18:29:49.655 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.655 DEBUG:boto:Data: 2024-04-16T18:29:49.655 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.655 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.655 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.655 DEBUG:boto:Params: {} 2024-04-16T18:29:49.655 DEBUG:boto:Token: None 2024-04-16T18:29:49.655 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0= 2024-04-16T18:29:49.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0=', 'Content-Length': '0'} 2024-04-16T18:29:49.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a363baf9438ce3c-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.659 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.659 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.659 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.659 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.659 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.660 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.660 DEBUG:boto:Data: 2024-04-16T18:29:49.660 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.660 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.660 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.660 DEBUG:boto:Params: {} 2024-04-16T18:29:49.660 DEBUG:boto:Token: None 2024-04-16T18:29:49.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5Z/8aOSNsPwxdxi0ykMOwyw+/F8= 2024-04-16T18:29:49.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5Z/8aOSNsPwxdxi0ykMOwyw+/F8=', 'Content-Length': '0'} 2024-04-16T18:29:49.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000affa269fe015cbf2-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.664 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.664 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.664 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.664 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.664 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.664 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.664 DEBUG:boto:Data: 2024-04-16T18:29:49.664 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.664 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.664 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.665 DEBUG:boto:Params: {} 2024-04-16T18:29:49.665 DEBUG:boto:Token: None 2024-04-16T18:29:49.665 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.665 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5Z/8aOSNsPwxdxi0ykMOwyw+/F8= 2024-04-16T18:29:49.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5Z/8aOSNsPwxdxi0ykMOwyw+/F8=', 'Content-Length': '0'} 2024-04-16T18:29:49.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033f62d6ffe6a2af8-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.668 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:29:49.668 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.668 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.668 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.669 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.669 DEBUG:boto:Method: GET 2024-04-16T18:29:49.669 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.669 DEBUG:boto:Data: 2024-04-16T18:29:49.669 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.669 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.669 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.669 DEBUG:boto:Params: {} 2024-04-16T18:29:49.669 DEBUG:boto:Token: None 2024-04-16T18:29:49.669 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0= 2024-04-16T18:29:49.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0=', 'Content-Length': '0'} 2024-04-16T18:29:49.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005db844b8e26fef34-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.714 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.715 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.715 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.715 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.715 DEBUG:boto:Method: GET 2024-04-16T18:29:49.715 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.715 DEBUG:boto:Data: 2024-04-16T18:29:49.715 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.715 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.715 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.715 DEBUG:boto:Params: {} 2024-04-16T18:29:49.715 DEBUG:boto:Token: None 2024-04-16T18:29:49.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:29:49.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0= 2024-04-16T18:29:49.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zeIsRrxAkPMlHmR6TFRFCuHUlJ0=', 'Content-Length': '0'} 2024-04-16T18:29:49.719 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083d4fcf399040ba7-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.719 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.719 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.719 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.719 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.719 DEBUG:boto:Method: GET 2024-04-16T18:29:49.719 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:29:49.720 DEBUG:boto:Data: 2024-04-16T18:29:49.720 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.720 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.720 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.720 DEBUG:boto:Params: {} 2024-04-16T18:29:49.720 DEBUG:boto:Token: None 2024-04-16T18:29:49.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.724 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001b703f025eb36c8-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.724 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.725 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.725 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.725 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.725 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.725 DEBUG:boto:Method: GET 2024-04-16T18:29:49.725 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:29:49.725 DEBUG:boto:Data: 2024-04-16T18:29:49.725 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.726 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.726 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.726 DEBUG:boto:Params: {} 2024-04-16T18:29:49.726 DEBUG:boto:Token: None 2024-04-16T18:29:49.726 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.726 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.733 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000633efa7eea64d21b-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.778 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.779 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.779 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.779 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.780 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:29:49.780 DEBUG:boto:Data: 2024-04-16T18:29:49.780 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.780 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.780 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.780 DEBUG:boto:Params: {} 2024-04-16T18:29:49.780 DEBUG:boto:Token: None 2024-04-16T18:29:49.781 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58 2024-04-16T18:29:49.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6rdoQPvL1TyguSlGoSrhER/Q80A= 2024-04-16T18:29:49.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6rdoQPvL1TyguSlGoSrhER/Q80A=', 'Content-Length': '0'} 2024-04-16T18:29:49.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc851d939ca11df0-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.785 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.785 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.785 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.785 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:29:49.785 DEBUG:boto:Data: 2024-04-16T18:29:49.785 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.785 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.785 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.785 DEBUG:boto:Params: {} 2024-04-16T18:29:49.786 DEBUG:boto:Token: None 2024-04-16T18:29:49.786 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58 2024-04-16T18:29:49.786 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6rdoQPvL1TyguSlGoSrhER/Q80A= 2024-04-16T18:29:49.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6rdoQPvL1TyguSlGoSrhER/Q80A=', 'Content-Length': '0'} 2024-04-16T18:29:49.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d355450238f9a37c-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.789 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:29:49.789 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.789 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.789 DEBUG:boto:Method: GET 2024-04-16T18:29:49.789 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:29:49.789 DEBUG:boto:Data: 2024-04-16T18:29:49.789 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.790 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.790 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.790 DEBUG:boto:Params: {} 2024-04-16T18:29:49.790 DEBUG:boto:Token: None 2024-04-16T18:29:49.790 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58 2024-04-16T18:29:49.790 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ia1zVOdnPfiSdkR7WX9Uje6SSD8= 2024-04-16T18:29:49.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ia1zVOdnPfiSdkR7WX9Uje6SSD8=', 'Content-Length': '0'} 2024-04-16T18:29:49.793 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6b23222ff2d2fb4-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.834 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:29:49.835 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:29:49.835 DEBUG:boto:Method: GET 2024-04-16T18:29:49.835 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:29:49.835 DEBUG:boto:Data: 2024-04-16T18:29:49.835 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.835 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.836 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.836 DEBUG:boto:Params: {} 2024-04-16T18:29:49.836 DEBUG:boto:Token: None 2024-04-16T18:29:49.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj58 2024-04-16T18:29:49.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ia1zVOdnPfiSdkR7WX9Uje6SSD8= 2024-04-16T18:29:49.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ia1zVOdnPfiSdkR7WX9Uje6SSD8=', 'Content-Length': '0'} 2024-04-16T18:29:49.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cc57793bd46607a-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.840 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.840 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.841 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.841 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.841 DEBUG:boto:Method: GET 2024-04-16T18:29:49.841 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:29:49.841 DEBUG:boto:Data: 2024-04-16T18:29:49.841 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.841 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.841 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.841 DEBUG:boto:Params: {} 2024-04-16T18:29:49.842 DEBUG:boto:Token: None 2024-04-16T18:29:49.842 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.846 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018f44d4cba2d1a81-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.886 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.887 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:49.887 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:49.887 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.887 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:29:49.887 DEBUG:boto:Method: GET 2024-04-16T18:29:49.887 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:29:49.888 DEBUG:boto:Data: 2024-04-16T18:29:49.888 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.888 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.888 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.888 DEBUG:boto:Params: {} 2024-04-16T18:29:49.888 DEBUG:boto:Token: None 2024-04-16T18:29:49.888 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/ 2024-04-16T18:29:49.888 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50= 2024-04-16T18:29:49.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3SKzjHIRdi5pF73aACmK7Wltz50=', 'Content-Length': '0'} 2024-04-16T18:29:49.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000006e8178ac35ef8-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.896 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:49.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:29:49.896 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:49.896 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:49.896 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.897 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.897 DEBUG:boto:Method: GET 2024-04-16T18:29:49.897 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.897 DEBUG:boto:Data: 2024-04-16T18:29:49.897 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.897 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.897 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.897 DEBUG:boto:Params: {} 2024-04-16T18:29:49.897 DEBUG:boto:Token: None 2024-04-16T18:29:49.897 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.898 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8lMPInr2+yJbM5MNdDOpWWjnCvY= 2024-04-16T18:29:49.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8lMPInr2+yJbM5MNdDOpWWjnCvY=', 'Content-Length': '0'} 2024-04-16T18:29:49.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de7640f2516e5246-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.942 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:49.942 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:49.943 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.943 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.943 DEBUG:boto:Method: GET 2024-04-16T18:29:49.943 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.943 DEBUG:boto:Data: 2024-04-16T18:29:49.943 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.943 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.943 DEBUG:boto:Params: {} 2024-04-16T18:29:49.943 DEBUG:boto:Token: None 2024-04-16T18:29:49.943 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8lMPInr2+yJbM5MNdDOpWWjnCvY= 2024-04-16T18:29:49.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8lMPInr2+yJbM5MNdDOpWWjnCvY=', 'Content-Length': '0'} 2024-04-16T18:29:49.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e6c5c067e6bb74c-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.947 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:49.947 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:49.947 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.947 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.947 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.947 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.947 DEBUG:boto:Data: 2024-04-16T18:29:49.948 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.948 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.948 DEBUG:boto:Params: {} 2024-04-16T18:29:49.948 DEBUG:boto:Token: None 2024-04-16T18:29:49.948 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HXxJsilS5wsdYeQ4N/SwG1VcGrI= 2024-04-16T18:29:49.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HXxJsilS5wsdYeQ4N/SwG1VcGrI=', 'Content-Length': '0'} 2024-04-16T18:29:49.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db28d467be820f5a-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.951 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:49.952 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:49.952 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.952 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.952 DEBUG:boto:Method: HEAD 2024-04-16T18:29:49.952 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.952 DEBUG:boto:Data: 2024-04-16T18:29:49.952 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.952 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:49.952 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.952 DEBUG:boto:Params: {} 2024-04-16T18:29:49.952 DEBUG:boto:Token: None 2024-04-16T18:29:49.953 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.953 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HXxJsilS5wsdYeQ4N/SwG1VcGrI= 2024-04-16T18:29:49.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HXxJsilS5wsdYeQ4N/SwG1VcGrI=', 'Content-Length': '0'} 2024-04-16T18:29:49.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026bd29d9ff19520c-00661ec39d-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:49.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:29:49.956 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:49.956 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:49.956 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.956 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.956 DEBUG:boto:Method: GET 2024-04-16T18:29:49.956 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.956 DEBUG:boto:Data: 2024-04-16T18:29:49.956 DEBUG:boto:Headers: {} 2024-04-16T18:29:49.956 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:49.956 DEBUG:boto:Port: 8000 2024-04-16T18:29:49.957 DEBUG:boto:Params: {} 2024-04-16T18:29:49.957 DEBUG:boto:Token: None 2024-04-16T18:29:49.957 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:49 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:29:49.957 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8lMPInr2+yJbM5MNdDOpWWjnCvY= 2024-04-16T18:29:49.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:49 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8lMPInr2+yJbM5MNdDOpWWjnCvY=', 'Content-Length': '0'} 2024-04-16T18:29:49.960 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa0049db69e47e85-00661ec39d-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:49 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.006 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:50.006 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:50.007 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:50.007 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:29:50.007 DEBUG:boto:Method: GET 2024-04-16T18:29:50.007 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:29:50.007 DEBUG:boto:Data: 2024-04-16T18:29:50.007 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.007 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.007 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.007 DEBUG:boto:Params: {} 2024-04-16T18:29:50.007 DEBUG:boto:Token: None 2024-04-16T18:29:50.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:29:50.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1yj8LghIVGgiJUWmsUQE1vwTzkY= 2024-04-16T18:29:50.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1yj8LghIVGgiJUWmsUQE1vwTzkY=', 'Content-Length': '0'} 2024-04-16T18:29:50.011 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035f8eeedfc399574-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.012 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.012 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.012 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.012 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.012 DEBUG:boto:Method: GET 2024-04-16T18:29:50.012 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:29:50.012 DEBUG:boto:Data: 2024-04-16T18:29:50.012 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.012 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.012 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.012 DEBUG:boto:Params: {} 2024-04-16T18:29:50.013 DEBUG:boto:Token: None 2024-04-16T18:29:50.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff26c9f61a935c9c-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.017 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.017 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.017 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.017 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.017 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.017 DEBUG:boto:Method: GET 2024-04-16T18:29:50.018 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:29:50.018 DEBUG:boto:Data: 2024-04-16T18:29:50.018 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.018 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.018 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.018 DEBUG:boto:Params: {} 2024-04-16T18:29:50.018 DEBUG:boto:Token: None 2024-04-16T18:29:50.018 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.018 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e1c6d9b004ec2cf-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.066 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.067 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:50.067 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:50.067 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.067 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:29:50.067 DEBUG:boto:Data: 2024-04-16T18:29:50.067 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.067 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.068 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.068 DEBUG:boto:Params: {} 2024-04-16T18:29:50.068 DEBUG:boto:Token: None 2024-04-16T18:29:50.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj62 2024-04-16T18:29:50.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AjuwxqCor4N30Y+02aEDIgMImbU= 2024-04-16T18:29:50.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AjuwxqCor4N30Y+02aEDIgMImbU=', 'Content-Length': '0'} 2024-04-16T18:29:50.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e7fe6c8c412f8a1-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.071 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:50.071 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:50.071 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.071 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:29:50.072 DEBUG:boto:Data: 2024-04-16T18:29:50.072 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.072 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.072 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.072 DEBUG:boto:Params: {} 2024-04-16T18:29:50.072 DEBUG:boto:Token: None 2024-04-16T18:29:50.072 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj62 2024-04-16T18:29:50.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AjuwxqCor4N30Y+02aEDIgMImbU= 2024-04-16T18:29:50.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AjuwxqCor4N30Y+02aEDIgMImbU=', 'Content-Length': '0'} 2024-04-16T18:29:50.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0661e085ee61c26-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.075 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:29:50.075 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:50.075 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:50.075 DEBUG:boto:Method: GET 2024-04-16T18:29:50.075 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:29:50.076 DEBUG:boto:Data: 2024-04-16T18:29:50.076 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.076 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.076 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.076 DEBUG:boto:Params: {} 2024-04-16T18:29:50.076 DEBUG:boto:Token: None 2024-04-16T18:29:50.076 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj62 2024-04-16T18:29:50.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UoZA5m+MxbpzOHHGsA25FDdKG1g= 2024-04-16T18:29:50.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UoZA5m+MxbpzOHHGsA25FDdKG1g=', 'Content-Length': '0'} 2024-04-16T18:29:50.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:37 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d56e18b9887d7d88-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.122 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:29:50.123 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:29:50.123 DEBUG:boto:Method: GET 2024-04-16T18:29:50.123 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:29:50.123 DEBUG:boto:Data: 2024-04-16T18:29:50.123 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.123 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.123 DEBUG:boto:Params: {} 2024-04-16T18:29:50.123 DEBUG:boto:Token: None 2024-04-16T18:29:50.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj62 2024-04-16T18:29:50.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UoZA5m+MxbpzOHHGsA25FDdKG1g= 2024-04-16T18:29:50.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UoZA5m+MxbpzOHHGsA25FDdKG1g=', 'Content-Length': '0'} 2024-04-16T18:29:50.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b79d70cf243f9403-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.127 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.127 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.127 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.127 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.127 DEBUG:boto:Method: GET 2024-04-16T18:29:50.128 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:29:50.128 DEBUG:boto:Data: 2024-04-16T18:29:50.128 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.128 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.128 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.128 DEBUG:boto:Params: {} 2024-04-16T18:29:50.128 DEBUG:boto:Token: None 2024-04-16T18:29:50.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003dc8336b4e2fa23e-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.174 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.175 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.175 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.175 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.175 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:29:50.175 DEBUG:boto:Method: GET 2024-04-16T18:29:50.175 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:29:50.175 DEBUG:boto:Data: 2024-04-16T18:29:50.176 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.176 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.176 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.176 DEBUG:boto:Params: {} 2024-04-16T18:29:50.176 DEBUG:boto:Token: None 2024-04-16T18:29:50.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.183 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082d1885f895c9bd5-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.183 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.184 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:29:50.184 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.184 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.184 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.184 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.184 DEBUG:boto:Method: GET 2024-04-16T18:29:50.184 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.184 DEBUG:boto:Data: 2024-04-16T18:29:50.184 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.185 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.185 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.185 DEBUG:boto:Params: {} 2024-04-16T18:29:50.185 DEBUG:boto:Token: None 2024-04-16T18:29:50.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA= 2024-04-16T18:29:50.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA=', 'Content-Length': '0'} 2024-04-16T18:29:50.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b4e6c2a660657a0-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.234 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.235 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.235 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.235 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.235 DEBUG:boto:Method: GET 2024-04-16T18:29:50.235 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.235 DEBUG:boto:Data: 2024-04-16T18:29:50.235 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.235 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.236 DEBUG:boto:Params: {} 2024-04-16T18:29:50.236 DEBUG:boto:Token: None 2024-04-16T18:29:50.236 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA= 2024-04-16T18:29:50.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA=', 'Content-Length': '0'} 2024-04-16T18:29:50.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053705a049c8baefb-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.242 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.242 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.242 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.242 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.242 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.243 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.243 DEBUG:boto:Data: 2024-04-16T18:29:50.243 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.243 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.243 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.243 DEBUG:boto:Params: {} 2024-04-16T18:29:50.243 DEBUG:boto:Token: None 2024-04-16T18:29:50.243 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.243 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:92yJguJglNgRssOHkYCI3vTRMgo= 2024-04-16T18:29:50.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:92yJguJglNgRssOHkYCI3vTRMgo=', 'Content-Length': '0'} 2024-04-16T18:29:50.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052939f33e924fc54-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.246 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.247 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.247 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.247 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.247 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.247 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.247 DEBUG:boto:Data: 2024-04-16T18:29:50.247 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.247 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.247 DEBUG:boto:Params: {} 2024-04-16T18:29:50.247 DEBUG:boto:Token: None 2024-04-16T18:29:50.248 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.248 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:92yJguJglNgRssOHkYCI3vTRMgo= 2024-04-16T18:29:50.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:92yJguJglNgRssOHkYCI3vTRMgo=', 'Content-Length': '0'} 2024-04-16T18:29:50.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f93bca05a3ad4985-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:29:50.251 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.251 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.251 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.251 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.252 DEBUG:boto:Method: GET 2024-04-16T18:29:50.252 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.252 DEBUG:boto:Data: 2024-04-16T18:29:50.252 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.252 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.252 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.252 DEBUG:boto:Params: {} 2024-04-16T18:29:50.252 DEBUG:boto:Token: None 2024-04-16T18:29:50.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA= 2024-04-16T18:29:50.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA=', 'Content-Length': '0'} 2024-04-16T18:29:50.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4a7253c20623404-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.298 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.298 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.299 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.299 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.299 DEBUG:boto:Method: GET 2024-04-16T18:29:50.299 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.299 DEBUG:boto:Data: 2024-04-16T18:29:50.299 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.299 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.299 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.299 DEBUG:boto:Params: {} 2024-04-16T18:29:50.299 DEBUG:boto:Token: None 2024-04-16T18:29:50.299 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:29:50.299 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA= 2024-04-16T18:29:50.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yKOolonCZrQM4e4ccXO+16STiNA=', 'Content-Length': '0'} 2024-04-16T18:29:50.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067d3c535e123565a-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.303 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.303 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.303 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.303 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.303 DEBUG:boto:Method: GET 2024-04-16T18:29:50.303 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:29:50.303 DEBUG:boto:Data: 2024-04-16T18:29:50.303 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.304 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.304 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.304 DEBUG:boto:Params: {} 2024-04-16T18:29:50.304 DEBUG:boto:Token: None 2024-04-16T18:29:50.304 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000717ccc50aaa1fd6b-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.350 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.351 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.351 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.351 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.351 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.351 DEBUG:boto:Method: GET 2024-04-16T18:29:50.351 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:29:50.351 DEBUG:boto:Data: 2024-04-16T18:29:50.351 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.351 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.351 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.351 DEBUG:boto:Params: {} 2024-04-16T18:29:50.352 DEBUG:boto:Token: None 2024-04-16T18:29:50.352 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8bf68a019e18d10-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.358 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.359 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.359 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.359 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.359 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:29:50.359 DEBUG:boto:Data: 2024-04-16T18:29:50.359 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.359 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.359 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.359 DEBUG:boto:Params: {} 2024-04-16T18:29:50.360 DEBUG:boto:Token: None 2024-04-16T18:29:50.360 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68 2024-04-16T18:29:50.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4ufqolWfF/7tWXHYDNxMuk0BP60= 2024-04-16T18:29:50.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4ufqolWfF/7tWXHYDNxMuk0BP60=', 'Content-Length': '0'} 2024-04-16T18:29:50.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009051560d568e27fa-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.363 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.363 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.363 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.363 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:29:50.363 DEBUG:boto:Data: 2024-04-16T18:29:50.363 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.363 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.364 DEBUG:boto:Params: {} 2024-04-16T18:29:50.364 DEBUG:boto:Token: None 2024-04-16T18:29:50.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68 2024-04-16T18:29:50.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4ufqolWfF/7tWXHYDNxMuk0BP60= 2024-04-16T18:29:50.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4ufqolWfF/7tWXHYDNxMuk0BP60=', 'Content-Length': '0'} 2024-04-16T18:29:50.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee1df12ba714bf74-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:29:50.367 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.367 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.367 DEBUG:boto:Method: GET 2024-04-16T18:29:50.367 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:29:50.367 DEBUG:boto:Data: 2024-04-16T18:29:50.368 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.368 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.368 DEBUG:boto:Params: {} 2024-04-16T18:29:50.368 DEBUG:boto:Token: None 2024-04-16T18:29:50.368 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68 2024-04-16T18:29:50.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LFiQ3naYaVBjx5VWxCcIMPaisWw= 2024-04-16T18:29:50.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LFiQ3naYaVBjx5VWxCcIMPaisWw=', 'Content-Length': '0'} 2024-04-16T18:29:50.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:20:39 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000997b77780bd559a9-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.418 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:29:50.418 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:29:50.419 DEBUG:boto:Method: GET 2024-04-16T18:29:50.419 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:29:50.419 DEBUG:boto:Data: 2024-04-16T18:29:50.419 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.419 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.419 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.419 DEBUG:boto:Params: {} 2024-04-16T18:29:50.419 DEBUG:boto:Token: None 2024-04-16T18:29:50.419 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj68 2024-04-16T18:29:50.419 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LFiQ3naYaVBjx5VWxCcIMPaisWw= 2024-04-16T18:29:50.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LFiQ3naYaVBjx5VWxCcIMPaisWw=', 'Content-Length': '0'} 2024-04-16T18:29:50.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8cd060d555f7326-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.423 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.423 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.423 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.423 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.423 DEBUG:boto:Method: GET 2024-04-16T18:29:50.424 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:29:50.424 DEBUG:boto:Data: 2024-04-16T18:29:50.424 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.424 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.424 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.424 DEBUG:boto:Params: {} 2024-04-16T18:29:50.424 DEBUG:boto:Token: None 2024-04-16T18:29:50.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.428 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000203b1b2f9003a75-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.470 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.471 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.471 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.471 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.471 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:29:50.471 DEBUG:boto:Method: GET 2024-04-16T18:29:50.471 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:29:50.471 DEBUG:boto:Data: 2024-04-16T18:29:50.471 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.472 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.472 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.472 DEBUG:boto:Params: {} 2024-04-16T18:29:50.472 DEBUG:boto:Token: None 2024-04-16T18:29:50.472 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.472 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.478 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c4a1eea9b3f8f99-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.478 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:29:50.479 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.479 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.479 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.479 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.479 DEBUG:boto:Method: GET 2024-04-16T18:29:50.479 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.480 DEBUG:boto:Data: 2024-04-16T18:29:50.480 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.480 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.480 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.480 DEBUG:boto:Params: {} 2024-04-16T18:29:50.480 DEBUG:boto:Token: None 2024-04-16T18:29:50.480 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w= 2024-04-16T18:29:50.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w=', 'Content-Length': '0'} 2024-04-16T18:29:50.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000326c9f28a8d87239-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.526 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.527 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.527 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.527 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.527 DEBUG:boto:Method: GET 2024-04-16T18:29:50.527 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.527 DEBUG:boto:Data: 2024-04-16T18:29:50.527 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.527 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.527 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.527 DEBUG:boto:Params: {} 2024-04-16T18:29:50.527 DEBUG:boto:Token: None 2024-04-16T18:29:50.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w= 2024-04-16T18:29:50.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w=', 'Content-Length': '0'} 2024-04-16T18:29:50.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3d1c9d9b4f15a98-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.531 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.531 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.531 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.532 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.532 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.532 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.532 DEBUG:boto:Data: 2024-04-16T18:29:50.532 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.532 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.532 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.532 DEBUG:boto:Params: {} 2024-04-16T18:29:50.532 DEBUG:boto:Token: None 2024-04-16T18:29:50.532 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.533 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iPqF2TYOTvMP+Clj4h5Y8lqk0s4= 2024-04-16T18:29:50.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iPqF2TYOTvMP+Clj4h5Y8lqk0s4=', 'Content-Length': '0'} 2024-04-16T18:29:50.536 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000383131806a8dcd94-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.536 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.536 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.536 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.536 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.536 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.536 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.537 DEBUG:boto:Data: 2024-04-16T18:29:50.537 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.537 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.537 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.537 DEBUG:boto:Params: {} 2024-04-16T18:29:50.537 DEBUG:boto:Token: None 2024-04-16T18:29:50.537 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.537 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:iPqF2TYOTvMP+Clj4h5Y8lqk0s4= 2024-04-16T18:29:50.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iPqF2TYOTvMP+Clj4h5Y8lqk0s4=', 'Content-Length': '0'} 2024-04-16T18:29:50.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0d465d0ad82d885-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.541 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:29:50.541 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.541 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.541 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.541 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.541 DEBUG:boto:Method: GET 2024-04-16T18:29:50.541 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.541 DEBUG:boto:Data: 2024-04-16T18:29:50.541 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.542 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.542 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.542 DEBUG:boto:Params: {} 2024-04-16T18:29:50.542 DEBUG:boto:Token: None 2024-04-16T18:29:50.542 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.542 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w= 2024-04-16T18:29:50.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w=', 'Content-Length': '0'} 2024-04-16T18:29:50.545 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf35a73d5a1a354f-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.586 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.586 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.587 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.587 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.587 DEBUG:boto:Method: GET 2024-04-16T18:29:50.587 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.587 DEBUG:boto:Data: 2024-04-16T18:29:50.587 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.587 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.587 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.587 DEBUG:boto:Params: {} 2024-04-16T18:29:50.587 DEBUG:boto:Token: None 2024-04-16T18:29:50.588 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:29:50.588 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w= 2024-04-16T18:29:50.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yT+OKuntaWQcaUuvQUoz5UGr85w=', 'Content-Length': '0'} 2024-04-16T18:29:50.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee85f532ff35c08a-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.591 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.591 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.591 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.591 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.592 DEBUG:boto:Method: GET 2024-04-16T18:29:50.592 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:29:50.592 DEBUG:boto:Data: 2024-04-16T18:29:50.592 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.592 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.592 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.592 DEBUG:boto:Params: {} 2024-04-16T18:29:50.592 DEBUG:boto:Token: None 2024-04-16T18:29:50.592 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.592 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b668e0a9e54ab76f-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.596 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.597 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.597 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.597 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.597 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.597 DEBUG:boto:Method: GET 2024-04-16T18:29:50.597 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:29:50.597 DEBUG:boto:Data: 2024-04-16T18:29:50.597 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.597 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.597 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.598 DEBUG:boto:Params: {} 2024-04-16T18:29:50.598 DEBUG:boto:Token: None 2024-04-16T18:29:50.598 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.598 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.605 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000258fd859774b0032-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.605 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.605 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.605 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.606 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.606 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:29:50.606 DEBUG:boto:Data: 2024-04-16T18:29:50.606 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.606 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.606 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.606 DEBUG:boto:Params: {} 2024-04-16T18:29:50.606 DEBUG:boto:Token: None 2024-04-16T18:29:50.606 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69 2024-04-16T18:29:50.606 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9d+8jxMRdNknGXWIqZMxgUiUAQg= 2024-04-16T18:29:50.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9d+8jxMRdNknGXWIqZMxgUiUAQg=', 'Content-Length': '0'} 2024-04-16T18:29:50.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000a81a2835fa49e2-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.610 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.610 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.610 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.610 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:29:50.611 DEBUG:boto:Data: 2024-04-16T18:29:50.611 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.611 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.611 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.611 DEBUG:boto:Params: {} 2024-04-16T18:29:50.611 DEBUG:boto:Token: None 2024-04-16T18:29:50.611 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69 2024-04-16T18:29:50.611 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9d+8jxMRdNknGXWIqZMxgUiUAQg= 2024-04-16T18:29:50.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9d+8jxMRdNknGXWIqZMxgUiUAQg=', 'Content-Length': '0'} 2024-04-16T18:29:50.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000803b4f30eaad0497-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.614 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:29:50.615 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.615 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.615 DEBUG:boto:Method: GET 2024-04-16T18:29:50.615 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:29:50.615 DEBUG:boto:Data: 2024-04-16T18:29:50.615 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.615 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.615 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.615 DEBUG:boto:Params: {} 2024-04-16T18:29:50.615 DEBUG:boto:Token: None 2024-04-16T18:29:50.615 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69 2024-04-16T18:29:50.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EjhS68yKIMO2R7kady5XFcw2iL8= 2024-04-16T18:29:50.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EjhS68yKIMO2R7kady5XFcw2iL8=', 'Content-Length': '0'} 2024-04-16T18:29:50.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:24:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000898fb9104f410744-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.662 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:29:50.662 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:29:50.663 DEBUG:boto:Method: GET 2024-04-16T18:29:50.663 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:29:50.663 DEBUG:boto:Data: 2024-04-16T18:29:50.663 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.663 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.663 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.663 DEBUG:boto:Params: {} 2024-04-16T18:29:50.663 DEBUG:boto:Token: None 2024-04-16T18:29:50.663 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj69 2024-04-16T18:29:50.663 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EjhS68yKIMO2R7kady5XFcw2iL8= 2024-04-16T18:29:50.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EjhS68yKIMO2R7kady5XFcw2iL8=', 'Content-Length': '0'} 2024-04-16T18:29:50.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b7d67377e129092-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.667 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.667 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.667 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.667 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.667 DEBUG:boto:Method: GET 2024-04-16T18:29:50.667 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:29:50.668 DEBUG:boto:Data: 2024-04-16T18:29:50.668 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.668 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.668 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.668 DEBUG:boto:Params: {} 2024-04-16T18:29:50.668 DEBUG:boto:Token: None 2024-04-16T18:29:50.668 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d6619b0bea95ff8-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.714 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.715 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.715 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.715 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.715 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:29:50.715 DEBUG:boto:Method: GET 2024-04-16T18:29:50.715 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:29:50.715 DEBUG:boto:Data: 2024-04-16T18:29:50.715 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.716 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.716 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.716 DEBUG:boto:Params: {} 2024-04-16T18:29:50.716 DEBUG:boto:Token: None 2024-04-16T18:29:50.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.722 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db7776b6124b25dc-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.722 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2024-04-16T18:29:50.723 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.723 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.723 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.723 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.723 DEBUG:boto:Method: GET 2024-04-16T18:29:50.723 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.723 DEBUG:boto:Data: 2024-04-16T18:29:50.723 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.724 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.724 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.724 DEBUG:boto:Params: {} 2024-04-16T18:29:50.724 DEBUG:boto:Token: None 2024-04-16T18:29:50.724 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU= 2024-04-16T18:29:50.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU=', 'Content-Length': '0'} 2024-04-16T18:29:50.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a74987557b8497ff-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.770 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.770 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.771 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.771 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.771 DEBUG:boto:Method: GET 2024-04-16T18:29:50.771 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.771 DEBUG:boto:Data: 2024-04-16T18:29:50.771 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.771 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.771 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.771 DEBUG:boto:Params: {} 2024-04-16T18:29:50.771 DEBUG:boto:Token: None 2024-04-16T18:29:50.771 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU= 2024-04-16T18:29:50.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU=', 'Content-Length': '0'} 2024-04-16T18:29:50.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003dd88f48b1ec88d6-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.775 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.775 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.775 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.775 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.776 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.776 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.776 DEBUG:boto:Data: 2024-04-16T18:29:50.776 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.776 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.776 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.776 DEBUG:boto:Params: {} 2024-04-16T18:29:50.776 DEBUG:boto:Token: None 2024-04-16T18:29:50.776 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aVMY956I4892VA0wabX5CQxn6zM= 2024-04-16T18:29:50.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aVMY956I4892VA0wabX5CQxn6zM=', 'Content-Length': '0'} 2024-04-16T18:29:50.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f05a29d99d7e2e0b-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.779 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.780 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.780 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.780 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.780 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.780 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.780 DEBUG:boto:Data: 2024-04-16T18:29:50.780 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.780 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.780 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.780 DEBUG:boto:Params: {} 2024-04-16T18:29:50.780 DEBUG:boto:Token: None 2024-04-16T18:29:50.781 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aVMY956I4892VA0wabX5CQxn6zM= 2024-04-16T18:29:50.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aVMY956I4892VA0wabX5CQxn6zM=', 'Content-Length': '0'} 2024-04-16T18:29:50.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0d5299911eb09fb-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.784 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2024-04-16T18:29:50.784 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.784 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.784 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.784 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.785 DEBUG:boto:Method: GET 2024-04-16T18:29:50.785 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.785 DEBUG:boto:Data: 2024-04-16T18:29:50.785 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.785 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.785 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.785 DEBUG:boto:Params: {} 2024-04-16T18:29:50.785 DEBUG:boto:Token: None 2024-04-16T18:29:50.785 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.785 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU= 2024-04-16T18:29:50.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU=', 'Content-Length': '0'} 2024-04-16T18:29:50.788 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6b05a534d364357-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.830 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.831 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.831 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.831 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.831 DEBUG:boto:Method: GET 2024-04-16T18:29:50.831 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.831 DEBUG:boto:Data: 2024-04-16T18:29:50.831 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.831 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.831 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.831 DEBUG:boto:Params: {} 2024-04-16T18:29:50.831 DEBUG:boto:Token: None 2024-04-16T18:29:50.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:29:50.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU= 2024-04-16T18:29:50.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tsHAicAiBLrEBCvk4Is9/bFF3jU=', 'Content-Length': '0'} 2024-04-16T18:29:50.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2cfed22b6768bc9-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.835 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.835 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.835 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.835 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.836 DEBUG:boto:Method: GET 2024-04-16T18:29:50.836 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:29:50.836 DEBUG:boto:Data: 2024-04-16T18:29:50.836 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.836 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.836 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.836 DEBUG:boto:Params: {} 2024-04-16T18:29:50.836 DEBUG:boto:Token: None 2024-04-16T18:29:50.836 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.836 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041a8b51fcf74c38c-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.840 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.840 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.841 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.841 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.841 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.841 DEBUG:boto:Method: GET 2024-04-16T18:29:50.841 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:29:50.841 DEBUG:boto:Data: 2024-04-16T18:29:50.841 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.841 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.841 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.841 DEBUG:boto:Params: {} 2024-04-16T18:29:50.841 DEBUG:boto:Token: None 2024-04-16T18:29:50.842 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.848 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e532ac32aec75b54-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.848 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.848 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.848 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.849 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.849 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:29:50.849 DEBUG:boto:Data: 2024-04-16T18:29:50.849 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.849 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.849 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.849 DEBUG:boto:Params: {} 2024-04-16T18:29:50.849 DEBUG:boto:Token: None 2024-04-16T18:29:50.849 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74 2024-04-16T18:29:50.849 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fEKUYTDQOl9BzoO1+3Kod1G+sfA= 2024-04-16T18:29:50.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fEKUYTDQOl9BzoO1+3Kod1G+sfA=', 'Content-Length': '0'} 2024-04-16T18:29:50.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002aea0f372d723a36-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.853 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.853 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.853 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.853 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:29:50.853 DEBUG:boto:Data: 2024-04-16T18:29:50.853 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.853 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.853 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.853 DEBUG:boto:Params: {} 2024-04-16T18:29:50.853 DEBUG:boto:Token: None 2024-04-16T18:29:50.853 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74 2024-04-16T18:29:50.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:fEKUYTDQOl9BzoO1+3Kod1G+sfA= 2024-04-16T18:29:50.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fEKUYTDQOl9BzoO1+3Kod1G+sfA=', 'Content-Length': '0'} 2024-04-16T18:29:50.856 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092170f29076ebe3e-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.857 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2024-04-16T18:29:50.857 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.857 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.857 DEBUG:boto:Method: GET 2024-04-16T18:29:50.857 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:29:50.857 DEBUG:boto:Data: 2024-04-16T18:29:50.857 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.857 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.857 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.858 DEBUG:boto:Params: {} 2024-04-16T18:29:50.858 DEBUG:boto:Token: None 2024-04-16T18:29:50.858 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74 2024-04-16T18:29:50.858 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kzkc50a/Xz+hWGRxNSGbd0YZ+74= 2024-04-16T18:29:50.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kzkc50a/Xz+hWGRxNSGbd0YZ+74=', 'Content-Length': '0'} 2024-04-16T18:29:50.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:58 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ed31cc2d0c07769-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.902 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:29:50.903 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:29:50.903 DEBUG:boto:Method: GET 2024-04-16T18:29:50.903 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:29:50.903 DEBUG:boto:Data: 2024-04-16T18:29:50.903 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.903 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.903 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.903 DEBUG:boto:Params: {} 2024-04-16T18:29:50.903 DEBUG:boto:Token: None 2024-04-16T18:29:50.903 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj74 2024-04-16T18:29:50.904 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Kzkc50a/Xz+hWGRxNSGbd0YZ+74= 2024-04-16T18:29:50.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kzkc50a/Xz+hWGRxNSGbd0YZ+74=', 'Content-Length': '0'} 2024-04-16T18:29:50.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a3f528efd8e6d80-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.907 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.907 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.907 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.907 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.907 DEBUG:boto:Method: GET 2024-04-16T18:29:50.908 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:29:50.908 DEBUG:boto:Data: 2024-04-16T18:29:50.908 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.908 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.908 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.908 DEBUG:boto:Params: {} 2024-04-16T18:29:50.908 DEBUG:boto:Token: None 2024-04-16T18:29:50.908 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.912 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac2ad15e2d865cf5-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.912 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.912 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:50.912 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:50.912 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.912 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:29:50.913 DEBUG:boto:Method: GET 2024-04-16T18:29:50.913 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:29:50.913 DEBUG:boto:Data: 2024-04-16T18:29:50.913 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.913 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.913 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.913 DEBUG:boto:Params: {} 2024-04-16T18:29:50.913 DEBUG:boto:Token: None 2024-04-16T18:29:50.913 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/ 2024-04-16T18:29:50.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8= 2024-04-16T18:29:50.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y0Zi2+QrTuvS2dISfT+Mr8II/k8=', 'Content-Length': '0'} 2024-04-16T18:29:50.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e7252236d149bb3-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.920 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:50.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:29:50.921 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:50.921 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:50.921 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.921 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.921 DEBUG:boto:Method: GET 2024-04-16T18:29:50.921 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.921 DEBUG:boto:Data: 2024-04-16T18:29:50.921 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.921 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.921 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.921 DEBUG:boto:Params: {} 2024-04-16T18:29:50.922 DEBUG:boto:Token: None 2024-04-16T18:29:50.922 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.922 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPR2+WlslQPtVa1+O6R3pp0IA2I= 2024-04-16T18:29:50.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPR2+WlslQPtVa1+O6R3pp0IA2I=', 'Content-Length': '0'} 2024-04-16T18:29:50.925 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e819a179b50d99d4-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.966 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:50.967 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:50.967 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.967 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.967 DEBUG:boto:Method: GET 2024-04-16T18:29:50.967 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.967 DEBUG:boto:Data: 2024-04-16T18:29:50.967 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.967 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.967 DEBUG:boto:Params: {} 2024-04-16T18:29:50.967 DEBUG:boto:Token: None 2024-04-16T18:29:50.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPR2+WlslQPtVa1+O6R3pp0IA2I= 2024-04-16T18:29:50.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPR2+WlslQPtVa1+O6R3pp0IA2I=', 'Content-Length': '0'} 2024-04-16T18:29:50.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b1afa7023f660a9-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.971 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:50.971 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:50.971 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.972 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.972 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.972 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.972 DEBUG:boto:Data: 2024-04-16T18:29:50.972 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.972 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.972 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.972 DEBUG:boto:Params: {} 2024-04-16T18:29:50.972 DEBUG:boto:Token: None 2024-04-16T18:29:50.972 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.972 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/r3nbL5/HdpbJCIUTuXqoaFEjQs= 2024-04-16T18:29:50.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/r3nbL5/HdpbJCIUTuXqoaFEjQs=', 'Content-Length': '0'} 2024-04-16T18:29:50.975 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005417e4e8890c818f-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.976 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:50.976 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:50.976 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.976 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.976 DEBUG:boto:Method: HEAD 2024-04-16T18:29:50.976 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.976 DEBUG:boto:Data: 2024-04-16T18:29:50.976 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.976 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:50.976 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.976 DEBUG:boto:Params: {} 2024-04-16T18:29:50.977 DEBUG:boto:Token: None 2024-04-16T18:29:50.977 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/r3nbL5/HdpbJCIUTuXqoaFEjQs= 2024-04-16T18:29:50.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/r3nbL5/HdpbJCIUTuXqoaFEjQs=', 'Content-Length': '0'} 2024-04-16T18:29:50.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075acf558858cf607-00661ec39e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:50.980 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:29:50.980 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:50.980 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:50.980 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.980 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.980 DEBUG:boto:Method: GET 2024-04-16T18:29:50.980 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.981 DEBUG:boto:Data: 2024-04-16T18:29:50.981 DEBUG:boto:Headers: {} 2024-04-16T18:29:50.981 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:50.981 DEBUG:boto:Port: 8000 2024-04-16T18:29:50.981 DEBUG:boto:Params: {} 2024-04-16T18:29:50.981 DEBUG:boto:Token: None 2024-04-16T18:29:50.981 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:50 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:29:50.981 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yPR2+WlslQPtVa1+O6R3pp0IA2I= 2024-04-16T18:29:50.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:50 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yPR2+WlslQPtVa1+O6R3pp0IA2I=', 'Content-Length': '0'} 2024-04-16T18:29:50.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000530e1ac8b9c9f4e4-00661ec39e-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:50 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.026 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:51.026 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:51.027 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:51.027 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:29:51.027 DEBUG:boto:Method: GET 2024-04-16T18:29:51.027 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:29:51.027 DEBUG:boto:Data: 2024-04-16T18:29:51.027 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.027 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.027 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.027 DEBUG:boto:Params: {} 2024-04-16T18:29:51.027 DEBUG:boto:Token: None 2024-04-16T18:29:51.027 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:29:51.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dEoGrvgXL83o5NC6DGIIFDd3/ng= 2024-04-16T18:29:51.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dEoGrvgXL83o5NC6DGIIFDd3/ng=', 'Content-Length': '0'} 2024-04-16T18:29:51.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003688cc32c409dafa-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.031 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.031 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.031 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.031 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.031 DEBUG:boto:Method: GET 2024-04-16T18:29:51.032 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:29:51.032 DEBUG:boto:Data: 2024-04-16T18:29:51.032 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.032 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.032 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.032 DEBUG:boto:Params: {} 2024-04-16T18:29:51.032 DEBUG:boto:Token: None 2024-04-16T18:29:51.032 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.032 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6dd6eb92badb6e2-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.036 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.037 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.037 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.037 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.037 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.037 DEBUG:boto:Method: GET 2024-04-16T18:29:51.037 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:29:51.037 DEBUG:boto:Data: 2024-04-16T18:29:51.037 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.037 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.037 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.037 DEBUG:boto:Params: {} 2024-04-16T18:29:51.037 DEBUG:boto:Token: None 2024-04-16T18:29:51.038 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.038 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001949e0b59d587233-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.086 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.087 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:51.087 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:51.087 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.087 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:29:51.087 DEBUG:boto:Data: 2024-04-16T18:29:51.087 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.087 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.087 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.087 DEBUG:boto:Params: {} 2024-04-16T18:29:51.088 DEBUG:boto:Token: None 2024-04-16T18:29:51.088 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj76 2024-04-16T18:29:51.088 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vdwvo2Ec8YwoQD89xzU+A7S6ZMM= 2024-04-16T18:29:51.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vdwvo2Ec8YwoQD89xzU+A7S6ZMM=', 'Content-Length': '0'} 2024-04-16T18:29:51.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2110367cdac49f7-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.091 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:51.091 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:51.091 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.092 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:29:51.092 DEBUG:boto:Data: 2024-04-16T18:29:51.092 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.092 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.092 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.092 DEBUG:boto:Params: {} 2024-04-16T18:29:51.092 DEBUG:boto:Token: None 2024-04-16T18:29:51.092 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj76 2024-04-16T18:29:51.092 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vdwvo2Ec8YwoQD89xzU+A7S6ZMM= 2024-04-16T18:29:51.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vdwvo2Ec8YwoQD89xzU+A7S6ZMM=', 'Content-Length': '0'} 2024-04-16T18:29:51.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f3ae3858973665c-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.095 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:29:51.096 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:51.096 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:51.096 DEBUG:boto:Method: GET 2024-04-16T18:29:51.096 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:29:51.096 DEBUG:boto:Data: 2024-04-16T18:29:51.096 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.096 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.096 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.096 DEBUG:boto:Params: {} 2024-04-16T18:29:51.096 DEBUG:boto:Token: None 2024-04-16T18:29:51.097 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj76 2024-04-16T18:29:51.097 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IqY5BjmCgew355S4DSm4yZeP0ig= 2024-04-16T18:29:51.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IqY5BjmCgew355S4DSm4yZeP0ig=', 'Content-Length': '0'} 2024-04-16T18:29:51.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b560c0aaf0ae2ef-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.142 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:29:51.142 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:29:51.143 DEBUG:boto:Method: GET 2024-04-16T18:29:51.143 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:29:51.143 DEBUG:boto:Data: 2024-04-16T18:29:51.143 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.143 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.143 DEBUG:boto:Params: {} 2024-04-16T18:29:51.143 DEBUG:boto:Token: None 2024-04-16T18:29:51.143 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj76 2024-04-16T18:29:51.143 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:IqY5BjmCgew355S4DSm4yZeP0ig= 2024-04-16T18:29:51.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IqY5BjmCgew355S4DSm4yZeP0ig=', 'Content-Length': '0'} 2024-04-16T18:29:51.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080e3e13f5e24a8f5-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.147 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.147 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.147 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.147 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.147 DEBUG:boto:Method: GET 2024-04-16T18:29:51.147 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:29:51.147 DEBUG:boto:Data: 2024-04-16T18:29:51.147 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.147 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.148 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.148 DEBUG:boto:Params: {} 2024-04-16T18:29:51.148 DEBUG:boto:Token: None 2024-04-16T18:29:51.148 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.148 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc557e29d0495839-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.194 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.195 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.195 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.195 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.195 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:29:51.195 DEBUG:boto:Method: GET 2024-04-16T18:29:51.195 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:29:51.195 DEBUG:boto:Data: 2024-04-16T18:29:51.195 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.195 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.195 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.196 DEBUG:boto:Params: {} 2024-04-16T18:29:51.196 DEBUG:boto:Token: None 2024-04-16T18:29:51.196 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000968c8b2fefdc31d7-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.203 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:29:51.203 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.203 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.204 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.204 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.204 DEBUG:boto:Method: GET 2024-04-16T18:29:51.204 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.204 DEBUG:boto:Data: 2024-04-16T18:29:51.204 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.204 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.204 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.204 DEBUG:boto:Params: {} 2024-04-16T18:29:51.204 DEBUG:boto:Token: None 2024-04-16T18:29:51.205 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.205 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4= 2024-04-16T18:29:51.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4=', 'Content-Length': '0'} 2024-04-16T18:29:51.208 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ebcd29502577f3f-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.250 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.250 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.251 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.251 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.251 DEBUG:boto:Method: GET 2024-04-16T18:29:51.251 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.251 DEBUG:boto:Data: 2024-04-16T18:29:51.251 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.251 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.251 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.251 DEBUG:boto:Params: {} 2024-04-16T18:29:51.251 DEBUG:boto:Token: None 2024-04-16T18:29:51.251 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4= 2024-04-16T18:29:51.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4=', 'Content-Length': '0'} 2024-04-16T18:29:51.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d489883bd081241-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.255 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.255 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.255 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.255 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.256 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.256 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.256 DEBUG:boto:Data: 2024-04-16T18:29:51.256 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.256 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.256 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.256 DEBUG:boto:Params: {} 2024-04-16T18:29:51.256 DEBUG:boto:Token: None 2024-04-16T18:29:51.256 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.256 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yGFKwk6jOCqzQs2eDwb7ELiDtE4= 2024-04-16T18:29:51.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yGFKwk6jOCqzQs2eDwb7ELiDtE4=', 'Content-Length': '0'} 2024-04-16T18:29:51.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ecff25c21216f54f-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.262 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.262 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.262 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.262 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.262 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.262 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.262 DEBUG:boto:Data: 2024-04-16T18:29:51.262 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.262 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.262 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.263 DEBUG:boto:Params: {} 2024-04-16T18:29:51.263 DEBUG:boto:Token: None 2024-04-16T18:29:51.263 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.263 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yGFKwk6jOCqzQs2eDwb7ELiDtE4= 2024-04-16T18:29:51.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yGFKwk6jOCqzQs2eDwb7ELiDtE4=', 'Content-Length': '0'} 2024-04-16T18:29:51.266 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a86b5a979e68bf1-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.266 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:29:51.266 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.267 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.267 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.267 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.267 DEBUG:boto:Method: GET 2024-04-16T18:29:51.267 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.267 DEBUG:boto:Data: 2024-04-16T18:29:51.267 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.267 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.267 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.267 DEBUG:boto:Params: {} 2024-04-16T18:29:51.267 DEBUG:boto:Token: None 2024-04-16T18:29:51.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4= 2024-04-16T18:29:51.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4=', 'Content-Length': '0'} 2024-04-16T18:29:51.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098a04fd6a734813f-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.314 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.315 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.315 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.315 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.315 DEBUG:boto:Method: GET 2024-04-16T18:29:51.315 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.315 DEBUG:boto:Data: 2024-04-16T18:29:51.315 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.315 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.315 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.315 DEBUG:boto:Params: {} 2024-04-16T18:29:51.316 DEBUG:boto:Token: None 2024-04-16T18:29:51.316 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:29:51.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4= 2024-04-16T18:29:51.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WqYRd9qFnsB1bdxOkv+6Qt7HNw4=', 'Content-Length': '0'} 2024-04-16T18:29:51.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092f8144e6e1c6c7c-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.320 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.320 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.320 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.320 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.320 DEBUG:boto:Method: GET 2024-04-16T18:29:51.320 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:29:51.320 DEBUG:boto:Data: 2024-04-16T18:29:51.320 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.320 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.320 DEBUG:boto:Params: {} 2024-04-16T18:29:51.320 DEBUG:boto:Token: None 2024-04-16T18:29:51.321 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.321 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093d841d4e63569c3-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.325 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.325 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.325 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.325 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.326 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.326 DEBUG:boto:Method: GET 2024-04-16T18:29:51.326 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:29:51.326 DEBUG:boto:Data: 2024-04-16T18:29:51.326 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.326 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.326 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.326 DEBUG:boto:Params: {} 2024-04-16T18:29:51.326 DEBUG:boto:Token: None 2024-04-16T18:29:51.326 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.326 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.333 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000efb3d76b513c0f95-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.374 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.375 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.375 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.375 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.375 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:29:51.375 DEBUG:boto:Data: 2024-04-16T18:29:51.375 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.375 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.375 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.375 DEBUG:boto:Params: {} 2024-04-16T18:29:51.376 DEBUG:boto:Token: None 2024-04-16T18:29:51.376 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85 2024-04-16T18:29:51.376 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixRBOdJmSUjlrG1DvZgbN874moY= 2024-04-16T18:29:51.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixRBOdJmSUjlrG1DvZgbN874moY=', 'Content-Length': '0'} 2024-04-16T18:29:51.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ab751290786d47f-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.379 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.379 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.379 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.379 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:29:51.379 DEBUG:boto:Data: 2024-04-16T18:29:51.379 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.379 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.379 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.379 DEBUG:boto:Params: {} 2024-04-16T18:29:51.380 DEBUG:boto:Token: None 2024-04-16T18:29:51.380 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85 2024-04-16T18:29:51.380 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ixRBOdJmSUjlrG1DvZgbN874moY= 2024-04-16T18:29:51.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ixRBOdJmSUjlrG1DvZgbN874moY=', 'Content-Length': '0'} 2024-04-16T18:29:51.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a807cef9912e3a2a-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.383 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:29:51.383 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.383 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.383 DEBUG:boto:Method: GET 2024-04-16T18:29:51.383 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:29:51.384 DEBUG:boto:Data: 2024-04-16T18:29:51.384 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.384 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.384 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.384 DEBUG:boto:Params: {} 2024-04-16T18:29:51.384 DEBUG:boto:Token: None 2024-04-16T18:29:51.384 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85 2024-04-16T18:29:51.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jCeKyiBtZ2cYFA5cia1QGxks87o= 2024-04-16T18:29:51.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jCeKyiBtZ2cYFA5cia1QGxks87o=', 'Content-Length': '0'} 2024-04-16T18:29:51.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:22:09 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000704721a15ac71898-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.430 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:29:51.430 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:29:51.431 DEBUG:boto:Method: GET 2024-04-16T18:29:51.431 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:29:51.431 DEBUG:boto:Data: 2024-04-16T18:29:51.431 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.431 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.431 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.431 DEBUG:boto:Params: {} 2024-04-16T18:29:51.431 DEBUG:boto:Token: None 2024-04-16T18:29:51.431 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj85 2024-04-16T18:29:51.431 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jCeKyiBtZ2cYFA5cia1QGxks87o= 2024-04-16T18:29:51.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jCeKyiBtZ2cYFA5cia1QGxks87o=', 'Content-Length': '0'} 2024-04-16T18:29:51.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb594ca2ba0b1203-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.435 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.435 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.435 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.435 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.435 DEBUG:boto:Method: GET 2024-04-16T18:29:51.435 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:29:51.435 DEBUG:boto:Data: 2024-04-16T18:29:51.435 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.435 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.435 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.435 DEBUG:boto:Params: {} 2024-04-16T18:29:51.436 DEBUG:boto:Token: None 2024-04-16T18:29:51.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006427dcf2599762ed-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.482 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.483 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.483 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.483 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.483 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:29:51.483 DEBUG:boto:Method: GET 2024-04-16T18:29:51.483 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:29:51.483 DEBUG:boto:Data: 2024-04-16T18:29:51.483 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.483 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.483 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.483 DEBUG:boto:Params: {} 2024-04-16T18:29:51.484 DEBUG:boto:Token: None 2024-04-16T18:29:51.484 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005205fcdbc6210653-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.491 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.491 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:29:51.492 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.492 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.492 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.492 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.492 DEBUG:boto:Method: GET 2024-04-16T18:29:51.492 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.492 DEBUG:boto:Data: 2024-04-16T18:29:51.492 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.492 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.492 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.492 DEBUG:boto:Params: {} 2024-04-16T18:29:51.493 DEBUG:boto:Token: None 2024-04-16T18:29:51.493 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw= 2024-04-16T18:29:51.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw=', 'Content-Length': '0'} 2024-04-16T18:29:51.496 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000737323c8ca99f1c4-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.538 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.539 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.539 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.539 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.539 DEBUG:boto:Method: GET 2024-04-16T18:29:51.539 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.539 DEBUG:boto:Data: 2024-04-16T18:29:51.539 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.539 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.539 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.539 DEBUG:boto:Params: {} 2024-04-16T18:29:51.539 DEBUG:boto:Token: None 2024-04-16T18:29:51.540 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.540 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw= 2024-04-16T18:29:51.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw=', 'Content-Length': '0'} 2024-04-16T18:29:51.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfa80122df280ee5-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.543 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.543 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.543 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.543 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.543 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.543 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.543 DEBUG:boto:Data: 2024-04-16T18:29:51.544 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.544 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.544 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.544 DEBUG:boto:Params: {} 2024-04-16T18:29:51.544 DEBUG:boto:Token: None 2024-04-16T18:29:51.544 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.544 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wci3qcQzE/XoIE5t8GMMVdzbroA= 2024-04-16T18:29:51.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wci3qcQzE/XoIE5t8GMMVdzbroA=', 'Content-Length': '0'} 2024-04-16T18:29:51.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053d968077d154545-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.548 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.548 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.548 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.548 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.548 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.548 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.548 DEBUG:boto:Data: 2024-04-16T18:29:51.548 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.548 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.548 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.548 DEBUG:boto:Params: {} 2024-04-16T18:29:51.549 DEBUG:boto:Token: None 2024-04-16T18:29:51.549 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.549 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Wci3qcQzE/XoIE5t8GMMVdzbroA= 2024-04-16T18:29:51.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wci3qcQzE/XoIE5t8GMMVdzbroA=', 'Content-Length': '0'} 2024-04-16T18:29:51.552 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f83a408f8ff6e2b-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.552 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:29:51.552 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.552 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.552 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.552 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.553 DEBUG:boto:Method: GET 2024-04-16T18:29:51.553 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.553 DEBUG:boto:Data: 2024-04-16T18:29:51.553 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.553 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.553 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.553 DEBUG:boto:Params: {} 2024-04-16T18:29:51.553 DEBUG:boto:Token: None 2024-04-16T18:29:51.553 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.553 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw= 2024-04-16T18:29:51.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw=', 'Content-Length': '0'} 2024-04-16T18:29:51.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074cf5d10e09259f9-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.598 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.598 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.598 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.599 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.599 DEBUG:boto:Method: GET 2024-04-16T18:29:51.599 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.599 DEBUG:boto:Data: 2024-04-16T18:29:51.599 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.599 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.599 DEBUG:boto:Params: {} 2024-04-16T18:29:51.599 DEBUG:boto:Token: None 2024-04-16T18:29:51.599 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:29:51.599 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw= 2024-04-16T18:29:51.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gjpXREA/T8ysOHa2Ca1PrQnZwgw=', 'Content-Length': '0'} 2024-04-16T18:29:51.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd883e624151e3b2-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.603 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.603 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.603 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.603 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.603 DEBUG:boto:Method: GET 2024-04-16T18:29:51.603 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:29:51.603 DEBUG:boto:Data: 2024-04-16T18:29:51.604 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.604 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.604 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.604 DEBUG:boto:Params: {} 2024-04-16T18:29:51.604 DEBUG:boto:Token: None 2024-04-16T18:29:51.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd644c5d32150d0c-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.608 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.608 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.608 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.608 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.608 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.609 DEBUG:boto:Method: GET 2024-04-16T18:29:51.609 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:29:51.609 DEBUG:boto:Data: 2024-04-16T18:29:51.609 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.609 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.609 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.609 DEBUG:boto:Params: {} 2024-04-16T18:29:51.609 DEBUG:boto:Token: None 2024-04-16T18:29:51.609 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.609 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.616 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0d79bfa5327bd07-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.658 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.658 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.659 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.659 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.659 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:29:51.659 DEBUG:boto:Data: 2024-04-16T18:29:51.659 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.659 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.659 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.659 DEBUG:boto:Params: {} 2024-04-16T18:29:51.659 DEBUG:boto:Token: None 2024-04-16T18:29:51.659 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87 2024-04-16T18:29:51.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+uQtzktZUsglAaKnAhfssTeOb/c= 2024-04-16T18:29:51.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+uQtzktZUsglAaKnAhfssTeOb/c=', 'Content-Length': '0'} 2024-04-16T18:29:51.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025d1da3e51577a02-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.663 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.663 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.663 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.663 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:29:51.663 DEBUG:boto:Data: 2024-04-16T18:29:51.664 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.664 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.664 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.664 DEBUG:boto:Params: {} 2024-04-16T18:29:51.664 DEBUG:boto:Token: None 2024-04-16T18:29:51.664 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87 2024-04-16T18:29:51.664 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+uQtzktZUsglAaKnAhfssTeOb/c= 2024-04-16T18:29:51.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+uQtzktZUsglAaKnAhfssTeOb/c=', 'Content-Length': '0'} 2024-04-16T18:29:51.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095b6c47ccd63283e-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.668 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:29:51.668 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.668 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.668 DEBUG:boto:Method: GET 2024-04-16T18:29:51.668 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:29:51.668 DEBUG:boto:Data: 2024-04-16T18:29:51.668 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.668 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.668 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.668 DEBUG:boto:Params: {} 2024-04-16T18:29:51.669 DEBUG:boto:Token: None 2024-04-16T18:29:51.669 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87 2024-04-16T18:29:51.669 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vZsQ5vy1Aq8TJWB5eV4r2RQ0oyA= 2024-04-16T18:29:51.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vZsQ5vy1Aq8TJWB5eV4r2RQ0oyA=', 'Content-Length': '0'} 2024-04-16T18:29:51.673 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:26:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b93425373839d39-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.714 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:29:51.714 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:29:51.715 DEBUG:boto:Method: GET 2024-04-16T18:29:51.715 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:29:51.715 DEBUG:boto:Data: 2024-04-16T18:29:51.715 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.715 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.715 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.715 DEBUG:boto:Params: {} 2024-04-16T18:29:51.715 DEBUG:boto:Token: None 2024-04-16T18:29:51.715 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj87 2024-04-16T18:29:51.715 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vZsQ5vy1Aq8TJWB5eV4r2RQ0oyA= 2024-04-16T18:29:51.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vZsQ5vy1Aq8TJWB5eV4r2RQ0oyA=', 'Content-Length': '0'} 2024-04-16T18:29:51.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fee8f295e37cb8f7-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.718 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.719 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.719 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.719 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.719 DEBUG:boto:Method: GET 2024-04-16T18:29:51.719 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:29:51.719 DEBUG:boto:Data: 2024-04-16T18:29:51.719 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.719 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.719 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.719 DEBUG:boto:Params: {} 2024-04-16T18:29:51.720 DEBUG:boto:Token: None 2024-04-16T18:29:51.720 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.720 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000be3f28094fc287d-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.766 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.767 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.767 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.767 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.767 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:29:51.767 DEBUG:boto:Method: GET 2024-04-16T18:29:51.767 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:29:51.767 DEBUG:boto:Data: 2024-04-16T18:29:51.767 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.767 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.767 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.768 DEBUG:boto:Params: {} 2024-04-16T18:29:51.768 DEBUG:boto:Token: None 2024-04-16T18:29:51.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008031fdab23442377-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.776 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.776 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:29:51.776 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.776 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.776 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.776 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.777 DEBUG:boto:Method: GET 2024-04-16T18:29:51.777 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.777 DEBUG:boto:Data: 2024-04-16T18:29:51.777 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.777 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.777 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.777 DEBUG:boto:Params: {} 2024-04-16T18:29:51.777 DEBUG:boto:Token: None 2024-04-16T18:29:51.777 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.777 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8= 2024-04-16T18:29:51.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8=', 'Content-Length': '0'} 2024-04-16T18:29:51.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032c33a38933a7532-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.822 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.823 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.823 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.823 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.823 DEBUG:boto:Method: GET 2024-04-16T18:29:51.823 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.823 DEBUG:boto:Data: 2024-04-16T18:29:51.823 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.823 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.823 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.823 DEBUG:boto:Params: {} 2024-04-16T18:29:51.823 DEBUG:boto:Token: None 2024-04-16T18:29:51.824 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.824 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8= 2024-04-16T18:29:51.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8=', 'Content-Length': '0'} 2024-04-16T18:29:51.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bf998c0cf524fba-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.827 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.827 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.827 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.827 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.827 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.827 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.827 DEBUG:boto:Data: 2024-04-16T18:29:51.827 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.828 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.828 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.828 DEBUG:boto:Params: {} 2024-04-16T18:29:51.828 DEBUG:boto:Token: None 2024-04-16T18:29:51.828 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GAA6lOkzHd+eEcLZKOcxOybK+9U= 2024-04-16T18:29:51.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GAA6lOkzHd+eEcLZKOcxOybK+9U=', 'Content-Length': '0'} 2024-04-16T18:29:51.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000486178b1ccf108a5-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.831 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.831 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.832 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.832 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.832 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.832 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.832 DEBUG:boto:Data: 2024-04-16T18:29:51.832 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.832 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.832 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.832 DEBUG:boto:Params: {} 2024-04-16T18:29:51.832 DEBUG:boto:Token: None 2024-04-16T18:29:51.832 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.833 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:GAA6lOkzHd+eEcLZKOcxOybK+9U= 2024-04-16T18:29:51.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GAA6lOkzHd+eEcLZKOcxOybK+9U=', 'Content-Length': '0'} 2024-04-16T18:29:51.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f19c5aa6d999733d-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.835 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:29:51.836 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.836 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.836 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.836 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.836 DEBUG:boto:Method: GET 2024-04-16T18:29:51.836 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.836 DEBUG:boto:Data: 2024-04-16T18:29:51.836 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.836 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.836 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.836 DEBUG:boto:Params: {} 2024-04-16T18:29:51.836 DEBUG:boto:Token: None 2024-04-16T18:29:51.837 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8= 2024-04-16T18:29:51.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8=', 'Content-Length': '0'} 2024-04-16T18:29:51.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021019e8cde9a5f43-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.886 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.886 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.886 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.886 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.887 DEBUG:boto:Method: GET 2024-04-16T18:29:51.887 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.887 DEBUG:boto:Data: 2024-04-16T18:29:51.887 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.887 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.887 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.887 DEBUG:boto:Params: {} 2024-04-16T18:29:51.887 DEBUG:boto:Token: None 2024-04-16T18:29:51.887 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:29:51.887 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8= 2024-04-16T18:29:51.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+iwu87SzYSEyWUtO5VqvvD8xKj8=', 'Content-Length': '0'} 2024-04-16T18:29:51.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008678331be0e82deb-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.891 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.891 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.891 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:51.891 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:51.891 DEBUG:boto:Method: GET 2024-04-16T18:29:51.891 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:29:51.891 DEBUG:boto:Data: 2024-04-16T18:29:51.891 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.891 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.892 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.892 DEBUG:boto:Params: {} 2024-04-16T18:29:51.892 DEBUG:boto:Token: None 2024-04-16T18:29:51.892 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.892 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.895 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8d9dbce4a76adee-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.896 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.896 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:51.896 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:51.896 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:51.896 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:51.896 DEBUG:boto:Method: GET 2024-04-16T18:29:51.896 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:29:51.896 DEBUG:boto:Data: 2024-04-16T18:29:51.897 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.897 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.897 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.897 DEBUG:boto:Params: {} 2024-04-16T18:29:51.897 DEBUG:boto:Token: None 2024-04-16T18:29:51.897 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/ 2024-04-16T18:29:51.897 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU= 2024-04-16T18:29:51.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M4ZHBPcPoKGv4M1eVajWEUviypU=', 'Content-Length': '0'} 2024-04-16T18:29:51.903 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095b75b9d1ae4377e-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.946 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:51.946 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.947 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.947 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.947 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:29:51.947 DEBUG:boto:Data: 2024-04-16T18:29:51.947 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.947 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.947 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.947 DEBUG:boto:Params: {} 2024-04-16T18:29:51.947 DEBUG:boto:Token: None 2024-04-16T18:29:51.947 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92 2024-04-16T18:29:51.947 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5H/Tinfu9b+KRWdprUGdh7UCyWc= 2024-04-16T18:29:51.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5H/Tinfu9b+KRWdprUGdh7UCyWc=', 'Content-Length': '0'} 2024-04-16T18:29:51.950 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ff00ce471ff06bc-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.951 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.951 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.951 DEBUG:boto:Method: HEAD 2024-04-16T18:29:51.951 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:29:51.951 DEBUG:boto:Data: 2024-04-16T18:29:51.951 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.951 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:51.951 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.951 DEBUG:boto:Params: {} 2024-04-16T18:29:51.952 DEBUG:boto:Token: None 2024-04-16T18:29:51.952 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92 2024-04-16T18:29:51.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5H/Tinfu9b+KRWdprUGdh7UCyWc= 2024-04-16T18:29:51.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5H/Tinfu9b+KRWdprUGdh7UCyWc=', 'Content-Length': '0'} 2024-04-16T18:29:51.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eda55230cadc9b10-00661ec39f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:51.955 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:29:51.955 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:51.955 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:51.955 DEBUG:boto:Method: GET 2024-04-16T18:29:51.955 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:29:51.955 DEBUG:boto:Data: 2024-04-16T18:29:51.955 DEBUG:boto:Headers: {} 2024-04-16T18:29:51.955 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:51.955 DEBUG:boto:Port: 8000 2024-04-16T18:29:51.955 DEBUG:boto:Params: {} 2024-04-16T18:29:51.955 DEBUG:boto:Token: None 2024-04-16T18:29:51.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:51 GMT /lybtug-52/obj92 2024-04-16T18:29:51.956 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:FfoD3IIvaNqqEahv3xcpqdqCWQE= 2024-04-16T18:29:51.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:FfoD3IIvaNqqEahv3xcpqdqCWQE=', 'Content-Length': '0'} 2024-04-16T18:29:51.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:25:15 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008df8781d2f4bec8c-00661ec39f-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.002 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:29:52.002 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:29:52.003 DEBUG:boto:Method: GET 2024-04-16T18:29:52.003 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:29:52.003 DEBUG:boto:Data: 2024-04-16T18:29:52.003 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.003 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.003 DEBUG:boto:Params: {} 2024-04-16T18:29:52.003 DEBUG:boto:Token: None 2024-04-16T18:29:52.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj92 2024-04-16T18:29:52.003 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M9E8c3IBwy/wPSjCgAjaXZmR4RQ= 2024-04-16T18:29:52.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M9E8c3IBwy/wPSjCgAjaXZmR4RQ=', 'Content-Length': '0'} 2024-04-16T18:29:52.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9854aa5d5be5810-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.007 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.007 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.007 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:52.007 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:52.007 DEBUG:boto:Method: GET 2024-04-16T18:29:52.007 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:29:52.007 DEBUG:boto:Data: 2024-04-16T18:29:52.007 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.007 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.008 DEBUG:boto:Params: {} 2024-04-16T18:29:52.008 DEBUG:boto:Token: None 2024-04-16T18:29:52.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb30990e7a6176e3-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.054 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.055 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.055 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.055 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:52.055 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:29:52.055 DEBUG:boto:Method: GET 2024-04-16T18:29:52.055 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:29:52.055 DEBUG:boto:Data: 2024-04-16T18:29:52.055 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.055 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.055 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.056 DEBUG:boto:Params: {} 2024-04-16T18:29:52.056 DEBUG:boto:Token: None 2024-04-16T18:29:52.056 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.062 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be5206067ccde1c9-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.063 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.063 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:29:52.063 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.063 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.063 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.063 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.063 DEBUG:boto:Method: GET 2024-04-16T18:29:52.064 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.064 DEBUG:boto:Data: 2024-04-16T18:29:52.064 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.064 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.064 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.064 DEBUG:boto:Params: {} 2024-04-16T18:29:52.064 DEBUG:boto:Token: None 2024-04-16T18:29:52.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4= 2024-04-16T18:29:52.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4=', 'Content-Length': '0'} 2024-04-16T18:29:52.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d115b95c8e3a16b-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.110 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.110 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.111 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.111 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.111 DEBUG:boto:Method: GET 2024-04-16T18:29:52.111 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.111 DEBUG:boto:Data: 2024-04-16T18:29:52.111 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.111 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.111 DEBUG:boto:Params: {} 2024-04-16T18:29:52.111 DEBUG:boto:Token: None 2024-04-16T18:29:52.111 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4= 2024-04-16T18:29:52.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4=', 'Content-Length': '0'} 2024-04-16T18:29:52.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067c0555469449a97-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.115 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.115 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.115 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.115 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.115 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.115 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.115 DEBUG:boto:Data: 2024-04-16T18:29:52.115 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.116 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.116 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.116 DEBUG:boto:Params: {} 2024-04-16T18:29:52.116 DEBUG:boto:Token: None 2024-04-16T18:29:52.116 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8tScdnP3X+xfYOEnyztfzqnA9EA= 2024-04-16T18:29:52.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8tScdnP3X+xfYOEnyztfzqnA9EA=', 'Content-Length': '0'} 2024-04-16T18:29:52.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1f642f1d573a1a0-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.119 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.119 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.119 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.119 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.119 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.119 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.120 DEBUG:boto:Data: 2024-04-16T18:29:52.120 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.120 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.120 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.120 DEBUG:boto:Params: {} 2024-04-16T18:29:52.120 DEBUG:boto:Token: None 2024-04-16T18:29:52.120 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.120 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8tScdnP3X+xfYOEnyztfzqnA9EA= 2024-04-16T18:29:52.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8tScdnP3X+xfYOEnyztfzqnA9EA=', 'Content-Length': '0'} 2024-04-16T18:29:52.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c7f06ea3d446b5a-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.123 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:29:52.123 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.123 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.123 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.123 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.124 DEBUG:boto:Method: GET 2024-04-16T18:29:52.124 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.124 DEBUG:boto:Data: 2024-04-16T18:29:52.124 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.124 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.124 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.124 DEBUG:boto:Params: {} 2024-04-16T18:29:52.124 DEBUG:boto:Token: None 2024-04-16T18:29:52.124 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.124 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4= 2024-04-16T18:29:52.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4=', 'Content-Length': '0'} 2024-04-16T18:29:52.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cfe001abb7aecba-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.170 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.170 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.171 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.171 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.171 DEBUG:boto:Method: GET 2024-04-16T18:29:52.171 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.171 DEBUG:boto:Data: 2024-04-16T18:29:52.171 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.171 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.171 DEBUG:boto:Params: {} 2024-04-16T18:29:52.171 DEBUG:boto:Token: None 2024-04-16T18:29:52.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:29:52.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4= 2024-04-16T18:29:52.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G4tds/LcoHGmIqYCXB14ePifYP4=', 'Content-Length': '0'} 2024-04-16T18:29:52.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a269d0c4986d9a15-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.175 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.175 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.175 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.176 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.176 DEBUG:boto:Method: GET 2024-04-16T18:29:52.176 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:29:52.176 DEBUG:boto:Data: 2024-04-16T18:29:52.176 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.176 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.176 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.176 DEBUG:boto:Params: {} 2024-04-16T18:29:52.176 DEBUG:boto:Token: None 2024-04-16T18:29:52.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc6e330a7bfc82b7-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.180 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.181 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.181 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.181 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.181 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.181 DEBUG:boto:Method: GET 2024-04-16T18:29:52.181 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:29:52.181 DEBUG:boto:Data: 2024-04-16T18:29:52.181 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.181 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.181 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.181 DEBUG:boto:Params: {} 2024-04-16T18:29:52.182 DEBUG:boto:Token: None 2024-04-16T18:29:52.182 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.182 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080c032061129d757-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.230 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.231 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.231 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.231 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.231 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:29:52.231 DEBUG:boto:Data: 2024-04-16T18:29:52.231 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.231 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.231 DEBUG:boto:Params: {} 2024-04-16T18:29:52.231 DEBUG:boto:Token: None 2024-04-16T18:29:52.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93 2024-04-16T18:29:52.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PbHr1xHsLPX2vEaCnXI7XK87sCQ= 2024-04-16T18:29:52.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PbHr1xHsLPX2vEaCnXI7XK87sCQ=', 'Content-Length': '0'} 2024-04-16T18:29:52.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a12c5879481a307c-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.235 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.235 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.235 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.235 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:29:52.235 DEBUG:boto:Data: 2024-04-16T18:29:52.235 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.235 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.235 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.235 DEBUG:boto:Params: {} 2024-04-16T18:29:52.235 DEBUG:boto:Token: None 2024-04-16T18:29:52.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93 2024-04-16T18:29:52.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PbHr1xHsLPX2vEaCnXI7XK87sCQ= 2024-04-16T18:29:52.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PbHr1xHsLPX2vEaCnXI7XK87sCQ=', 'Content-Length': '0'} 2024-04-16T18:29:52.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d38e3b10ef6056f-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:29:52.239 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.239 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.239 DEBUG:boto:Method: GET 2024-04-16T18:29:52.239 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:29:52.239 DEBUG:boto:Data: 2024-04-16T18:29:52.239 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.239 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.239 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.239 DEBUG:boto:Params: {} 2024-04-16T18:29:52.240 DEBUG:boto:Token: None 2024-04-16T18:29:52.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93 2024-04-16T18:29:52.240 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CnSghxTEhyAyNsJbKCUvtDRTa74= 2024-04-16T18:29:52.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CnSghxTEhyAyNsJbKCUvtDRTa74=', 'Content-Length': '0'} 2024-04-16T18:29:52.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:27:31 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f43f0b8c5d2608c8-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.286 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:29:52.286 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:29:52.287 DEBUG:boto:Method: GET 2024-04-16T18:29:52.287 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:29:52.287 DEBUG:boto:Data: 2024-04-16T18:29:52.287 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.287 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.287 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.287 DEBUG:boto:Params: {} 2024-04-16T18:29:52.288 DEBUG:boto:Token: None 2024-04-16T18:29:52.288 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj93 2024-04-16T18:29:52.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CnSghxTEhyAyNsJbKCUvtDRTa74= 2024-04-16T18:29:52.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CnSghxTEhyAyNsJbKCUvtDRTa74=', 'Content-Length': '0'} 2024-04-16T18:29:52.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db8128f30b4da0e2-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.291 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.291 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.292 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.292 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.292 DEBUG:boto:Method: GET 2024-04-16T18:29:52.292 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:29:52.292 DEBUG:boto:Data: 2024-04-16T18:29:52.292 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.292 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.292 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.292 DEBUG:boto:Params: {} 2024-04-16T18:29:52.292 DEBUG:boto:Token: None 2024-04-16T18:29:52.293 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.293 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043a0af8746461dfc-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.338 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.339 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.339 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.339 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.339 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:29:52.339 DEBUG:boto:Method: GET 2024-04-16T18:29:52.339 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:29:52.339 DEBUG:boto:Data: 2024-04-16T18:29:52.339 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.340 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.340 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.340 DEBUG:boto:Params: {} 2024-04-16T18:29:52.340 DEBUG:boto:Token: None 2024-04-16T18:29:52.340 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.346 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000299f7490a0c1a6b-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.347 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.347 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:29:52.347 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.347 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.347 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.347 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.348 DEBUG:boto:Method: GET 2024-04-16T18:29:52.348 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.348 DEBUG:boto:Data: 2024-04-16T18:29:52.348 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.348 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.348 DEBUG:boto:Params: {} 2024-04-16T18:29:52.348 DEBUG:boto:Token: None 2024-04-16T18:29:52.348 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.348 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0= 2024-04-16T18:29:52.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0=', 'Content-Length': '0'} 2024-04-16T18:29:52.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff6a42b3b4dd3c28-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.394 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.395 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.395 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.395 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.395 DEBUG:boto:Method: GET 2024-04-16T18:29:52.395 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.395 DEBUG:boto:Data: 2024-04-16T18:29:52.395 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.395 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.395 DEBUG:boto:Params: {} 2024-04-16T18:29:52.396 DEBUG:boto:Token: None 2024-04-16T18:29:52.396 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0= 2024-04-16T18:29:52.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0=', 'Content-Length': '0'} 2024-04-16T18:29:52.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6b4fa278c55e23a-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.399 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.399 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.399 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.399 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.399 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.399 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.400 DEBUG:boto:Data: 2024-04-16T18:29:52.400 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.400 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.400 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.400 DEBUG:boto:Params: {} 2024-04-16T18:29:52.400 DEBUG:boto:Token: None 2024-04-16T18:29:52.400 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6jI4EsTbJsMYtwdeqTvzh4tppzk= 2024-04-16T18:29:52.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6jI4EsTbJsMYtwdeqTvzh4tppzk=', 'Content-Length': '0'} 2024-04-16T18:29:52.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027ea828f0a999693-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.403 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.403 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.403 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.403 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.404 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.404 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.404 DEBUG:boto:Data: 2024-04-16T18:29:52.404 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.404 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.404 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.404 DEBUG:boto:Params: {} 2024-04-16T18:29:52.404 DEBUG:boto:Token: None 2024-04-16T18:29:52.404 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6jI4EsTbJsMYtwdeqTvzh4tppzk= 2024-04-16T18:29:52.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6jI4EsTbJsMYtwdeqTvzh4tppzk=', 'Content-Length': '0'} 2024-04-16T18:29:52.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006031c6adc70fe190-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.407 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:29:52.407 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.407 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.408 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.408 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.408 DEBUG:boto:Method: GET 2024-04-16T18:29:52.408 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.408 DEBUG:boto:Data: 2024-04-16T18:29:52.408 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.408 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.408 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.408 DEBUG:boto:Params: {} 2024-04-16T18:29:52.408 DEBUG:boto:Token: None 2024-04-16T18:29:52.408 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.409 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0= 2024-04-16T18:29:52.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0=', 'Content-Length': '0'} 2024-04-16T18:29:52.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9f3e518cebf7447-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.454 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.455 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.455 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.455 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.455 DEBUG:boto:Method: GET 2024-04-16T18:29:52.455 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.455 DEBUG:boto:Data: 2024-04-16T18:29:52.455 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.455 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.455 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.455 DEBUG:boto:Params: {} 2024-04-16T18:29:52.455 DEBUG:boto:Token: None 2024-04-16T18:29:52.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:29:52.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0= 2024-04-16T18:29:52.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KKc8y7gyklNlJ7nXRw4b5PwDfx0=', 'Content-Length': '0'} 2024-04-16T18:29:52.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006632dfb193303c1-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.459 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.459 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.459 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.459 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.460 DEBUG:boto:Method: GET 2024-04-16T18:29:52.460 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:29:52.460 DEBUG:boto:Data: 2024-04-16T18:29:52.460 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.460 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.460 DEBUG:boto:Params: {} 2024-04-16T18:29:52.460 DEBUG:boto:Token: None 2024-04-16T18:29:52.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb3a28fac1cbcb53-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.464 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.464 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.464 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.465 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.465 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.465 DEBUG:boto:Method: GET 2024-04-16T18:29:52.465 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:29:52.465 DEBUG:boto:Data: 2024-04-16T18:29:52.465 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.465 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.465 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.465 DEBUG:boto:Params: {} 2024-04-16T18:29:52.465 DEBUG:boto:Token: None 2024-04-16T18:29:52.465 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.466 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.473 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000802447aa9b2b45c4-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.514 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.515 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.515 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.515 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.515 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:29:52.515 DEBUG:boto:Data: 2024-04-16T18:29:52.515 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.515 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.516 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.516 DEBUG:boto:Params: {} 2024-04-16T18:29:52.516 DEBUG:boto:Token: None 2024-04-16T18:29:52.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95 2024-04-16T18:29:52.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U1yr7ba3O7xn+9dGa51p3x2783A= 2024-04-16T18:29:52.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U1yr7ba3O7xn+9dGa51p3x2783A=', 'Content-Length': '0'} 2024-04-16T18:29:52.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000790a677acff79ef4-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.519 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.520 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.520 DEBUG:boto:Method: HEAD 2024-04-16T18:29:52.520 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:29:52.520 DEBUG:boto:Data: 2024-04-16T18:29:52.520 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.520 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.520 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.520 DEBUG:boto:Params: {} 2024-04-16T18:29:52.520 DEBUG:boto:Token: None 2024-04-16T18:29:52.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95 2024-04-16T18:29:52.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U1yr7ba3O7xn+9dGa51p3x2783A= 2024-04-16T18:29:52.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U1yr7ba3O7xn+9dGa51p3x2783A=', 'Content-Length': '0'} 2024-04-16T18:29:52.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2bca3ac324c1ed6-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.524 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:29:52.524 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.524 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.524 DEBUG:boto:Method: GET 2024-04-16T18:29:52.524 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:29:52.524 DEBUG:boto:Data: 2024-04-16T18:29:52.524 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.524 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.525 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.525 DEBUG:boto:Params: {} 2024-04-16T18:29:52.525 DEBUG:boto:Token: None 2024-04-16T18:29:52.525 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95 2024-04-16T18:29:52.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ri2W+fTsiOHrevsx8IlQwOjWyeI= 2024-04-16T18:29:52.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ri2W+fTsiOHrevsx8IlQwOjWyeI=', 'Content-Length': '0'} 2024-04-16T18:29:52.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:23:04 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db46a40c88614285-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.574 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:29:52.574 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:29:52.575 DEBUG:boto:Method: GET 2024-04-16T18:29:52.575 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:29:52.575 DEBUG:boto:Data: 2024-04-16T18:29:52.575 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.575 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.575 DEBUG:boto:Params: {} 2024-04-16T18:29:52.575 DEBUG:boto:Token: None 2024-04-16T18:29:52.575 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/obj95 2024-04-16T18:29:52.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ri2W+fTsiOHrevsx8IlQwOjWyeI= 2024-04-16T18:29:52.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ri2W+fTsiOHrevsx8IlQwOjWyeI=', 'Content-Length': '0'} 2024-04-16T18:29:52.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df43e688c89c341a-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.579 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.579 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.579 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.579 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.579 DEBUG:boto:Method: GET 2024-04-16T18:29:52.579 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:29:52.579 DEBUG:boto:Data: 2024-04-16T18:29:52.579 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.579 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:29:52.579 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.580 DEBUG:boto:Params: {} 2024-04-16T18:29:52.580 DEBUG:boto:Token: None 2024-04-16T18:29:52.580 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.580 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.583 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e2b6fdbd2ce46f7-00661ec3a0-5545-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.626 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.627 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:29:52.627 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:29:52.627 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.627 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:29:52.627 DEBUG:boto:Method: GET 2024-04-16T18:29:52.627 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:29:52.627 DEBUG:boto:Data: 2024-04-16T18:29:52.627 DEBUG:boto:Headers: {} 2024-04-16T18:29:52.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:29:52.628 DEBUG:boto:Port: 8000 2024-04-16T18:29:52.628 DEBUG:boto:Params: {} 2024-04-16T18:29:52.628 DEBUG:boto:Token: None 2024-04-16T18:29:52.628 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:29:52 GMT /lybtug-52/ 2024-04-16T18:29:52.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o= 2024-04-16T18:29:52.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:29:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OT8jMfRAo2Lt1jwBaiCxETtkj4o=', 'Content-Length': '0'} 2024-04-16T18:29:52.635 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004588e27125729595-00661ec3a0-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:29:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:29:52.635 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:29:52.636 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:29:52.636 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2024-04-16T18:29:52.636 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T18:29:52.638 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:29:52.636+0000 7fd89ff93640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 132331) UID: 0 2024-04-16T18:29:52.638 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:29:52.636+0000 7fd8a37dba40 -1 shutting down 2024-04-16T18:29:53.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T18:29:58.736 INFO:tasks.rgw.c2.client.0:Stopped 2024-04-16T18:29:58.736 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:29:58.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:29:58.736 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket list --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --max-entries 10000 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:29:58.782 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:29:58.782 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:29:58.891 INFO:teuthology.orchestra.run.smithi122.stdout:[{"name":"obj11","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:00.237172Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_fgQ5eClR3YffgB9ISU6S31rjRRxCqYs","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj111","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.377102Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_U9h1aJGMA6E17DfCtjCV1veKiOuyyAv","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj115","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:23:25.802945Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_kKuEhuSz6gUhRr9ikCvoMVEggIGFaLc","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj116","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:25:26.853472Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_T8l-pWTSJ2etufuHtCwW0iAkdvCyyS7","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj124","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:26:49.340484Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_i_X0e07hSsplPq5ac8yPBK-BpBr95t0","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj145","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:25:26.904602Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_gEbYfcyFgXId9x6Al3ZVc9I0W5-3n6A","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj148","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:27:53.397807Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_I_jGLjf64w2ohQkoB85LK2aoUWnTcrX","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj155","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:26:49.392862Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_704qEZAQrXiTiz-Dyio3jvvlaYVbFBH","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj17","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:12.390828Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_MHFVOa3CYDBzXyo4Uus7yq3iKBei2R_","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj185","instance":"","ver":{"pool":9,"epoch":16},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:27:53.450035Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_-Uryut3_uf3hvBoGnbC6fj2gNpcawCl","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj19","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:22:20.871403Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_jU2IsPpfQZAUbl5nxnFNpoT4PVkdD74","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj22","instance":"","ver":{"pool":9,"epoch":38},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:20:00.257626Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_QZSXzCckyzBAKIuUp1wL_e8YMhQJfDf","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj23","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:23:25.664581Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_69EPA8IWANnk6NsJJcBDnL-fm1pd_lA","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj29","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:25:26.766409Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_l3Q7niuREDmFUEjwm3rni4vtI-sfyuB","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj31","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:26:49.246767Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_5RrnrIDMSMU6UlYu3zc1Ar8wNcJkmk5","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj33","instance":"","ver":{"pool":9,"epoch":2},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:20:00.309065Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_BQBk9owld4yB5sGXXKXPPlicH6iZoww","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj34","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:20:12.407283Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_cyHTcAItvo7Frl_YdQYu6bRK5uBIasU","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj37","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:27:53.306174Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_gO7eXcYFt3rwiU0LNN7yptUi-mlW-Y8","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj38","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:22:20.888430Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_wf5uDX_SplidwbctKnuk2rUZG35iIc1","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj44","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:00.328043Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_1PemvgyBSFl4cE_E5G4OatRQqXTrUo1","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj46","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:23:25.714804Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_Yhg4kYS8NZjwWzs7Ts1Y3cSCFkVku8D","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj51","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:12.458478Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_6wneq808iIkJv34TznpM0KhLjsiqaDm","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj55","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:00.377847Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_1CrTl6WihWgeahNO-bMzDl9LxTeZoUS","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj57","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:22:20.901936Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_go4YpMv8NGjxl0JlXgjpa22B0OOHtme","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj58","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:25:26.782716Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_IDbu0p-Lh3nlKQh3BX_IizBuXWc1eow","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj62","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:26:49.266169Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_H0SUdUp7l1izinvjR7nh56iFkpM07Pz","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj68","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:12.476177Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_egjOZJv1NOOaiT22IrgfaBbPTalGq4u","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj69","instance":"","ver":{"pool":9,"epoch":15},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:23:25.732337Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_dL2pXNPlq1fZU0TozPAVE0VdbGzZNvv","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj74","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:27:53.324563Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_f0vcQZqESv5xWPcZDuLF6PFxgnQe4pK","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj76","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:22:20.954658Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_K0CWNYOraqP-rjt7_GzbWHnGa1cBni8","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj85","instance":"","ver":{"pool":9,"epoch":3},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:12.526435Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_gsD0IBdBLFsNYLXudxLocb6C7sQOHz_","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj87","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:25:26.833550Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_Wrky8GhcxnYIxThsPcxB8CYUk6m_HH0","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj92","instance":"","ver":{"pool":9,"epoch":4},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:23:25.782803Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_bwiIR2Dw6tDusAsrU6UNn0p2MYe3vBk","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj93","instance":"","ver":{"pool":9,"epoch":2},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:26:49.317259Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_xnkRqr80ujmMUD9SKqCz2gM7QU8Por1","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj95","instance":"","ver":{"pool":9,"epoch":12},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:22:20.973293Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_2vhENRm0g8RnMM4M5BZ0qf9JP-ZvvKg","flags":0,"pending_map":[],"versioned_epoch":0}] 2024-04-16T18:29:58.891 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:58.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:58.892 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj11 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:29:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:29:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:29:59.140 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj111', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.140 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj111', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.140 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj111 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:29:59.226 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:29:59.226 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:29:59.442 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj115', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj115', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.442 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj115 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:29:59.483 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:29:59.483 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:29:59.701 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj116', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.701 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj116', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.702 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj116 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:29:59.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2024-04-16T18:29:59.784 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:29:59.784 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:29:59.994 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj124', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.994 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj124', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:29:59.994 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj124 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:00.079 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:00.079 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:00.264 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj145', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:00.264 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj145', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:00.264 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj145 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:00.348 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:00.348 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:00.571 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj148', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:00.571 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj148', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:00.572 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj148 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:00.657 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:00.657 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:00.872 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj155', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:00.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj155', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:00.873 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj155 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:00.957 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:00.957 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:01.190 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:01.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:01.190 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj17 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:01.499 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj185', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:01.499 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj185', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:01.500 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj185 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:01.798 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:01.798 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:01.798 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj19 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:01.886 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:02.098 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj22', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.098 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj22', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.099 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj22 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:02.184 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:02.184 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:02.412 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj23', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.412 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj23', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.412 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj23 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:02.498 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:02.498 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:02.720 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj29', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.720 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj29', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.721 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj29 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:02.763 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:02.763 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:02.982 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj31', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.982 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj31', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:02.982 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj31 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:03.069 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:03.069 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:03.300 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj33', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:03.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj33', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:03.301 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj33 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:03.385 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:03.385 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:03.594 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj34', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:03.594 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj34', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:03.594 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj34 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:03.682 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:03.682 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:03.895 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj37', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:03.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj37', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:03.895 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj37 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:03.982 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:03.982 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:04.205 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj38', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:04.205 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj38', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:04.205 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj38 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:04.289 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:04.289 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:04.518 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:04.518 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:04.519 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj44 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:04.605 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:04.605 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:04.819 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:04.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:04.819 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj46 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:04.905 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:04.905 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:05.129 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:05.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:05.130 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj51 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:05.431 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj55', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:05.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj55', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:05.432 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj55 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:05.743 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj57', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:05.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj57', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:05.744 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj57 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:06.048 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj58', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.048 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj58', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.048 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj58 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:06.133 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:06.133 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:06.357 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.357 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.357 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj62 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:06.441 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:06.441 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:06.665 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.665 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.665 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj68 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:06.748 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:06.748 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:06.983 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.983 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:06.984 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj69 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:07.069 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:07.069 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:07.289 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj74', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:07.290 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj74', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:07.290 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj74 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:07.372 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:07.372 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:07.590 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj76', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:07.590 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj76', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:07.590 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj76 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:07.679 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:07.679 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:07.904 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj85', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:07.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj85', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:07.904 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj85 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:07.946 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:07.946 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:08.161 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj87', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:08.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj87', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:08.162 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj87 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:08.250 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:08.250 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:08.479 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj92', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:08.479 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj92', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:08.479 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj92 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:08.565 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:08.565 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:08.792 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj93', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:08.792 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj93', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:08.792 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj93 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:08.878 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:08.878 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:09.103 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj95', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.103 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--object', 'obj95', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.103 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --object obj95 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:09.191 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:09.191 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:09.413 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'trim', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'trim', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.414 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog trim --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:09.454 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:09.569 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.569 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.569 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync init --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:09.648 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:09.648 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:09.758 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'run', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.759 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'run', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:30:09.759 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync run --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:30:09.841 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:09.841 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.007+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: init. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.007+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: waiting on bucket lease 2024-04-16T18:30:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.019+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: status after acquiring the lock is: init 2024-04-16T18:30:10.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.039+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bucket for full sync 2024-04-16T18:30:10.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj11 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj11 op=0 op_state=1 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj11[0] 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj111 op=0 op_state=1 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: : injecting data sync error on key=obj111 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj115 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj115 op=0 op_state=1 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj115[0] 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj116 op=0 op_state=1 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj116[0] 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj124 op=0 op_state=1 2024-04-16T18:30:10.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj124[0] 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj145 op=0 op_state=1 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj145[0] 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj148 op=0 op_state=1 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj148[0] 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj155 op=0 op_state=1 2024-04-16T18:30:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: : injecting data sync error on key=obj155 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj17 op=0 op_state=1 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj17[0] 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.051+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj185 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj185 op=0 op_state=1 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj185[0] 2024-04-16T18:30:10.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj19 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj19 op=0 op_state=1 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj19[0] 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj22 op=0 op_state=1 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: : injecting data sync error on key=obj22 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj23 2024-04-16T18:30:10.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj23 op=0 op_state=1 2024-04-16T18:30:10.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj23[0] 2024-04-16T18:30:10.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj29 op=0 op_state=1 2024-04-16T18:30:10.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: : injecting data sync error on key=obj29 2024-04-16T18:30:10.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj31 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj31 op=0 op_state=1 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj31[0] 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj33 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj33 op=0 op_state=1 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj33[0] 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj34 op=0 op_state=1 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: : injecting data sync error on key=obj34 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj37 2024-04-16T18:30:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj37 op=0 op_state=1 2024-04-16T18:30:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj37[0] 2024-04-16T18:30:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj38 2024-04-16T18:30:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj38 op=0 op_state=1 2024-04-16T18:30:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj38[0] 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj44 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj44 op=0 op_state=1 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj44[0] 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj46 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj46 op=0 op_state=1 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj46[0] 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj51 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj51 op=0 op_state=1 2024-04-16T18:30:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj51[0] 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj55 op=0 op_state=1 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj55[0] 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj57 op=0 op_state=1 2024-04-16T18:30:10.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.055+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj57[0] 2024-04-16T18:30:10.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj58 2024-04-16T18:30:10.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj58 op=0 op_state=1 2024-04-16T18:30:10.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj58[0] 2024-04-16T18:30:10.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj62 op=0 op_state=1 2024-04-16T18:30:10.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.059+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj62[0] 2024-04-16T18:30:10.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.111+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: success 2024-04-16T18:30:10.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.111+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.111+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj68 op=0 op_state=1 2024-04-16T18:30:10.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.111+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj68[0] 2024-04-16T18:30:10.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: success 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: success 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj69 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj69 op=0 op_state=1 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj69[0] 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj74 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj74 op=0 op_state=1 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: success 2024-04-16T18:30:10.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj74[0] 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj76 op=0 op_state=1 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj76[0] 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: success 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj85 op=0 op_state=1 2024-04-16T18:30:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.119+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj85[0] 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: success 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj87 op=0 op_state=1 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj87[0] 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: success 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj92 2024-04-16T18:30:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj92 op=0 op_state=1 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj92[0] 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: success 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj93 op=0 op_state=1 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj93[0] 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: success 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: success 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: success 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj95 2024-04-16T18:30:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj95 op=0 op_state=1 2024-04-16T18:30:10.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj95[0] 2024-04-16T18:30:10.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: success 2024-04-16T18:30:10.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.123+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: success 2024-04-16T18:30:10.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.127+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: success 2024-04-16T18:30:10.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.131+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: success 2024-04-16T18:30:10.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.135+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: success 2024-04-16T18:30:10.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.135+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: success 2024-04-16T18:30:10.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.135+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: success 2024-04-16T18:30:10.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.139+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: success 2024-04-16T18:30:10.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.147+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: success 2024-04-16T18:30:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.147+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: success 2024-04-16T18:30:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.151+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: success 2024-04-16T18:30:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.155+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: success 2024-04-16T18:30:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.155+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: success 2024-04-16T18:30:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.155+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: success 2024-04-16T18:30:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.159+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: success 2024-04-16T18:30:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.159+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: success 2024-04-16T18:30:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.175+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: success 2024-04-16T18:30:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.175+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: success 2024-04-16T18:30:10.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.179+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: success 2024-04-16T18:30:10.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.179+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj11 2024-04-16T18:30:10.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.183+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: backing out with sync_status=-5 2024-04-16T18:30:10.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.183+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: ERROR: full sync failed. error: -5 2024-04-16T18:30:10.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.187+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: full. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.187+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: waiting on bucket lease 2024-04-16T18:30:10.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.195+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: status after acquiring the lock is: full 2024-04-16T18:30:10.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.211+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bucket for full sync 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj11 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj11 op=0 op_state=1 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj11[0] 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj111 op=0 op_state=1 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj111[0] 2024-04-16T18:30:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj115 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj115 op=0 op_state=1 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj115[0] 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj116 op=0 op_state=1 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj116[0] 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj124 op=0 op_state=1 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj124[0] 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj145 op=0 op_state=1 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj145[0] 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj148 op=0 op_state=1 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj148[0] 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj155 op=0 op_state=1 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj155[0] 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj17 op=0 op_state=1 2024-04-16T18:30:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj17[0] 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj185 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj185 op=0 op_state=1 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj185[0] 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj19 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj19 op=0 op_state=1 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj19[0] 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj22 op=0 op_state=1 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj22[0] 2024-04-16T18:30:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj23 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj23 op=0 op_state=1 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj23[0] 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj29 op=0 op_state=1 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj29[0] 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj31 2024-04-16T18:30:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj31 op=0 op_state=1 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj31[0] 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj33 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj33 op=0 op_state=1 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj33[0] 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj34 op=0 op_state=1 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj34[0] 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj37 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj37 op=0 op_state=1 2024-04-16T18:30:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj37[0] 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj38 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj38 op=0 op_state=1 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj38[0] 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj44 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj44 op=0 op_state=1 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj44[0] 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj46 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj46 op=0 op_state=1 2024-04-16T18:30:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.223+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj46[0] 2024-04-16T18:30:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.227+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: success 2024-04-16T18:30:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.227+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj51 2024-04-16T18:30:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.227+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj51 op=0 op_state=1 2024-04-16T18:30:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.227+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj51[0] 2024-04-16T18:30:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: success 2024-04-16T18:30:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj55 op=0 op_state=1 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj55[0] 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: success 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: success 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj57 op=0 op_state=1 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj57[0] 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj58 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj58 op=0 op_state=1 2024-04-16T18:30:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj58[0] 2024-04-16T18:30:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: success 2024-04-16T18:30:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj62 op=0 op_state=1 2024-04-16T18:30:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj62[0] 2024-04-16T18:30:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: success 2024-04-16T18:30:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj68 op=0 op_state=1 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: : injecting data sync error on key=obj68 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: success 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj69 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj69 op=0 op_state=1 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj69[0] 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: success 2024-04-16T18:30:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj74 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj74 op=0 op_state=1 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.231+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj74[0] 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj76 op=0 op_state=1 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: : injecting data sync error on key=obj76 2024-04-16T18:30:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: success 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: success 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj85 op=0 op_state=1 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj85[0] 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj87 op=0 op_state=1 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: : injecting data sync error on key=obj87 2024-04-16T18:30:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: success 2024-04-16T18:30:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj92 2024-04-16T18:30:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj92 op=0 op_state=1 2024-04-16T18:30:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj92[0] 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: success 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: success 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj93 op=0 op_state=1 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj93[0] 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj95 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj95 op=0 op_state=1 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.235+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj95[0] 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: success 2024-04-16T18:30:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: success 2024-04-16T18:30:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: success 2024-04-16T18:30:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj116 2024-04-16T18:30:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: success 2024-04-16T18:30:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: success 2024-04-16T18:30:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: success 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: success 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: success 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: success 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: success 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: success 2024-04-16T18:30:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.239+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: success 2024-04-16T18:30:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: success 2024-04-16T18:30:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj23 2024-04-16T18:30:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: success 2024-04-16T18:30:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: success 2024-04-16T18:30:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: success 2024-04-16T18:30:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: success 2024-04-16T18:30:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.243+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: success 2024-04-16T18:30:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.251+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: success 2024-04-16T18:30:10.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.251+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj62 2024-04-16T18:30:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.251+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: backing out with sync_status=-5 2024-04-16T18:30:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.251+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: ERROR: full sync failed. error: -5 2024-04-16T18:30:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.255+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: full. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.255+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: waiting on bucket lease 2024-04-16T18:30:10.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.307+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: status after acquiring the lock is: full 2024-04-16T18:30:10.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.327+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bucket for full sync 2024-04-16T18:30:10.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj11 2024-04-16T18:30:10.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj11 op=0 op_state=1 2024-04-16T18:30:10.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj11[0] 2024-04-16T18:30:10.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj111 op=0 op_state=1 2024-04-16T18:30:10.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj111[0] 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj115 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj115 op=0 op_state=1 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj115[0] 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj116 op=0 op_state=1 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: : injecting data sync error on key=obj116 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj124 op=0 op_state=1 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj124[0] 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.339+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj145 op=0 op_state=1 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj145[0] 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj148 op=0 op_state=1 2024-04-16T18:30:10.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj148[0] 2024-04-16T18:30:10.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj155 op=0 op_state=1 2024-04-16T18:30:10.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: : injecting data sync error on key=obj155 2024-04-16T18:30:10.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj17 op=0 op_state=1 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj17[0] 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj185 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj185 op=0 op_state=1 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj185[0] 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj19 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj19 op=0 op_state=1 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj19[0] 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj22 op=0 op_state=1 2024-04-16T18:30:10.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj22[0] 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj23 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj23 op=0 op_state=1 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj23[0] 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj29 op=0 op_state=1 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj29[0] 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj31 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj31 op=0 op_state=1 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj31[0] 2024-04-16T18:30:10.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj33 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj33 op=0 op_state=1 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj33[0] 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj34 op=0 op_state=1 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj34[0] 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj37 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj37 op=0 op_state=1 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj37[0] 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj38 2024-04-16T18:30:10.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj38 op=0 op_state=1 2024-04-16T18:30:10.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj38[0] 2024-04-16T18:30:10.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj44 2024-04-16T18:30:10.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj44 op=0 op_state=1 2024-04-16T18:30:10.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj44[0] 2024-04-16T18:30:10.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj46 2024-04-16T18:30:10.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj46 op=0 op_state=1 2024-04-16T18:30:10.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.343+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj46[0] 2024-04-16T18:30:10.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj51 2024-04-16T18:30:10.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj51 op=0 op_state=1 2024-04-16T18:30:10.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj51[0] 2024-04-16T18:30:10.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj55 op=0 op_state=1 2024-04-16T18:30:10.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj55[0] 2024-04-16T18:30:10.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: success 2024-04-16T18:30:10.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj57 op=0 op_state=1 2024-04-16T18:30:10.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj57[0] 2024-04-16T18:30:10.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: success 2024-04-16T18:30:10.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj58 2024-04-16T18:30:10.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj58 op=0 op_state=1 2024-04-16T18:30:10.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.347+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj58[0] 2024-04-16T18:30:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: success 2024-04-16T18:30:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: success 2024-04-16T18:30:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj62 op=0 op_state=1 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: : injecting data sync error on key=obj62 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj68 op=0 op_state=1 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj68[0] 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: success 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj69 2024-04-16T18:30:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj69 op=0 op_state=1 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj69[0] 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: success 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.351+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj74 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj74 op=0 op_state=1 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj74[0] 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: success 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj76 op=0 op_state=1 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj76[0] 2024-04-16T18:30:10.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: success 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj85 op=0 op_state=1 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: : injecting data sync error on key=obj85 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: success 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: success 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj87 op=0 op_state=1 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: : injecting data sync error on key=obj87 2024-04-16T18:30:10.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj92 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj92 op=0 op_state=1 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj92[0] 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: success 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: success 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: success 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: success 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: success 2024-04-16T18:30:10.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj93 op=0 op_state=1 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: : injecting data sync error on key=obj93 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: success 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj95 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj95 op=0 op_state=1 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj95[0] 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: success 2024-04-16T18:30:10.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: success 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: success 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.355+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: success 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: success 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: success 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: success 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: success 2024-04-16T18:30:10.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: success 2024-04-16T18:30:10.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: success 2024-04-16T18:30:10.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.359+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: success 2024-04-16T18:30:10.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.363+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: success 2024-04-16T18:30:10.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.363+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: success 2024-04-16T18:30:10.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.363+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj115 2024-04-16T18:30:10.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.367+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: backing out with sync_status=-5 2024-04-16T18:30:10.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.367+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: ERROR: full sync failed. error: -5 2024-04-16T18:30:10.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.371+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: full. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.371+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: waiting on bucket lease 2024-04-16T18:30:10.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.423+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: status after acquiring the lock is: full 2024-04-16T18:30:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.443+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bucket for full sync 2024-04-16T18:30:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj11 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj11 op=0 op_state=1 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj11[0] 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj111 op=0 op_state=1 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: : injecting data sync error on key=obj111 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj115 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj115 op=0 op_state=1 2024-04-16T18:30:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj115[0] 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj116 op=0 op_state=1 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj116[0] 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj124 op=0 op_state=1 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: : injecting data sync error on key=obj124 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj145 op=0 op_state=1 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj145[0] 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj148 op=0 op_state=1 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: : injecting data sync error on key=obj148 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj155 op=0 op_state=1 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj155[0] 2024-04-16T18:30:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj17 op=0 op_state=1 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: : injecting data sync error on key=obj17 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj185 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj185 op=0 op_state=1 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj185[0] 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj19 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj19 op=0 op_state=1 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj19[0] 2024-04-16T18:30:10.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj22 op=0 op_state=1 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj22[0] 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj23 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj23 op=0 op_state=1 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj23[0] 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj29 op=0 op_state=1 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj29[0] 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj31 2024-04-16T18:30:10.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj31 op=0 op_state=1 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj31[0] 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj33 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj33 op=0 op_state=1 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj33[0] 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj34 op=0 op_state=1 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj34[0] 2024-04-16T18:30:10.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.459+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj37 2024-04-16T18:30:10.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj37 op=0 op_state=1 2024-04-16T18:30:10.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj37[0] 2024-04-16T18:30:10.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj38 2024-04-16T18:30:10.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj38 op=0 op_state=1 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj38[0] 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj44 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj44 op=0 op_state=1 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj44[0] 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj46 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj46 op=0 op_state=1 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj46[0] 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj51 2024-04-16T18:30:10.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj51 op=0 op_state=1 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj51[0] 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj55 op=0 op_state=1 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: : injecting data sync error on key=obj55 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj57 op=0 op_state=1 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: : injecting data sync error on key=obj57 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj58 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj58 op=0 op_state=1 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.463+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj58[0] 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: success 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: success 2024-04-16T18:30:10.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj62 op=0 op_state=1 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj62[0] 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj68 op=0 op_state=1 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj68[0] 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj69 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj69 op=0 op_state=1 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj69[0] 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj74 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj74 op=0 op_state=1 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj74[0] 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: success 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj76 op=0 op_state=1 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj76[0] 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: success 2024-04-16T18:30:10.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj85 op=0 op_state=1 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.467+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj85[0] 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: success 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj87 op=0 op_state=1 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj87[0] 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: success 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj92 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj92 op=0 op_state=1 2024-04-16T18:30:10.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj92[0] 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: success 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: success 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj93 op=0 op_state=1 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj93[0] 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj95 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj95 op=0 op_state=1 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj95[0] 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: success 2024-04-16T18:30:10.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: success 2024-04-16T18:30:10.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: success 2024-04-16T18:30:10.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: success 2024-04-16T18:30:10.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.471+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: success 2024-04-16T18:30:10.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.475+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.475+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.475+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.475+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.475+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.479+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.479+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.479+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.479+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: success 2024-04-16T18:30:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.479+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj11 2024-04-16T18:30:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.483+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: backing out with sync_status=-5 2024-04-16T18:30:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.483+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: ERROR: full sync failed. error: -5 2024-04-16T18:30:10.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.487+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: full. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.487+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: waiting on bucket lease 2024-04-16T18:30:10.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.539+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: status after acquiring the lock is: full 2024-04-16T18:30:10.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.559+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bucket for full sync 2024-04-16T18:30:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj11 2024-04-16T18:30:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj11 op=0 op_state=1 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj11[0] 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj111 op=0 op_state=1 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj111[0] 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj115 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj115 op=0 op_state=1 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj115[0] 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj116 op=0 op_state=1 2024-04-16T18:30:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj116[0] 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj124 op=0 op_state=1 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj124[0] 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj145 op=0 op_state=1 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: : injecting data sync error on key=obj145 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj148 op=0 op_state=1 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj148[0] 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj155 op=0 op_state=1 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj155[0] 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj17 op=0 op_state=1 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj17[0] 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj185 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj185 op=0 op_state=1 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj185[0] 2024-04-16T18:30:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj19 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj19 op=0 op_state=1 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj19[0] 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj22 op=0 op_state=1 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj22[0] 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj23 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj23 op=0 op_state=1 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj23[0] 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj29 op=0 op_state=1 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: : injecting data sync error on key=obj29 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj31 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj31 op=0 op_state=1 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj31[0] 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj33 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj33 op=0 op_state=1 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj33[0] 2024-04-16T18:30:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj34 op=0 op_state=1 2024-04-16T18:30:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj34[0] 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.575+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj37 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj37 op=0 op_state=1 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj37[0] 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj38 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj38 op=0 op_state=1 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj38[0] 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj44 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj44 op=0 op_state=1 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj44[0] 2024-04-16T18:30:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj46 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj46 op=0 op_state=1 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj46[0] 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj51 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj51 op=0 op_state=1 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj51[0] 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj55 op=0 op_state=1 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.579+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj55[0] 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: success 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj57 op=0 op_state=1 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj57[0] 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: success 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj58 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj58 op=0 op_state=1 2024-04-16T18:30:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj58[0] 2024-04-16T18:30:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: success 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: success 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: success 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj62 op=0 op_state=1 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj62[0] 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj68 op=0 op_state=1 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj68[0] 2024-04-16T18:30:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj69 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj69 op=0 op_state=1 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.583+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj69[0] 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: success 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: success 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: success 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj74 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj74 op=0 op_state=1 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj74[0] 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj76 op=0 op_state=1 2024-04-16T18:30:10.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj76[0] 2024-04-16T18:30:10.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: success 2024-04-16T18:30:10.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj85 op=0 op_state=1 2024-04-16T18:30:10.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: : injecting data sync error on key=obj85 2024-04-16T18:30:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: failed, retcode=-5 ((5) Input/output error) 2024-04-16T18:30:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 0 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: ERROR: failed to sync object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj87 op=0 op_state=1 2024-04-16T18:30:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj87[0] 2024-04-16T18:30:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: success 2024-04-16T18:30:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj124 2024-04-16T18:30:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: success 2024-04-16T18:30:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj92 2024-04-16T18:30:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj92 op=0 op_state=1 2024-04-16T18:30:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.587+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj92[0] 2024-04-16T18:30:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: success 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: success 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj93 op=0 op_state=1 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj93[0] 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: success 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj95 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj95 op=0 op_state=1 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj95[0] 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: success 2024-04-16T18:30:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: a sync operation returned error 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.591+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: success 2024-04-16T18:30:10.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: success 2024-04-16T18:30:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: success 2024-04-16T18:30:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: success 2024-04-16T18:30:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: success 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: success 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: success 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: success 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.595+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: success 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.599+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: success 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.599+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: backing out with sync_status=-5 2024-04-16T18:30:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.599+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: ERROR: full sync failed. error: -5 2024-04-16T18:30:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.603+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: full. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.603+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: waiting on bucket lease 2024-04-16T18:30:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.655+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: status after acquiring the lock is: full 2024-04-16T18:30:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.675+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bucket for full sync 2024-04-16T18:30:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj11 2024-04-16T18:30:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj11 op=0 op_state=1 2024-04-16T18:30:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj11[0] 2024-04-16T18:30:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj111 2024-04-16T18:30:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj111 op=0 op_state=1 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj111[0] 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj115 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj115 op=0 op_state=1 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj115[0] 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj116 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj116 op=0 op_state=1 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj116[0] 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj124 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj124 op=0 op_state=1 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj124[0] 2024-04-16T18:30:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj145 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj145 op=0 op_state=1 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj145[0] 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj148 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj148 op=0 op_state=1 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj148[0] 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj155 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj155 op=0 op_state=1 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj155[0] 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj17 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj17 op=0 op_state=1 2024-04-16T18:30:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj17[0] 2024-04-16T18:30:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj185 2024-04-16T18:30:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj185 op=0 op_state=1 2024-04-16T18:30:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj185[0] 2024-04-16T18:30:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj19 2024-04-16T18:30:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj19 op=0 op_state=1 2024-04-16T18:30:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj19[0] 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj22 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj22 op=0 op_state=1 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj22[0] 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj23 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj23 op=0 op_state=1 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj23[0] 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj29 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj29 op=0 op_state=1 2024-04-16T18:30:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj29[0] 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj31 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj31 op=0 op_state=1 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj31[0] 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj33 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj33 op=0 op_state=1 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj33[0] 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj34 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj34 op=0 op_state=1 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj34[0] 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj37 2024-04-16T18:30:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj37 op=0 op_state=1 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj37[0] 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj38 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj38 op=0 op_state=1 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj38[0] 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj44 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj44 op=0 op_state=1 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj44[0] 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj46 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj46 op=0 op_state=1 2024-04-16T18:30:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.731+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj46[0] 2024-04-16T18:30:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj115]: success 2024-04-16T18:30:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj111]: success 2024-04-16T18:30:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj116]: success 2024-04-16T18:30:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj51 2024-04-16T18:30:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj51 op=0 op_state=1 2024-04-16T18:30:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj51[0] 2024-04-16T18:30:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj55 2024-04-16T18:30:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj55 op=0 op_state=1 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj55[0] 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj57 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj57 op=0 op_state=1 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj57[0] 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj155]: success 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj58 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj58 op=0 op_state=1 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj58[0] 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj11]: success 2024-04-16T18:30:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj62 2024-04-16T18:30:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj62 op=0 op_state=1 2024-04-16T18:30:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj19]: success 2024-04-16T18:30:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj62[0] 2024-04-16T18:30:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj68 2024-04-16T18:30:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj68 op=0 op_state=1 2024-04-16T18:30:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.739+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj68[0] 2024-04-16T18:30:10.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj185]: success 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj29]: success 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj31]: success 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj46]: success 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj69 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj69 op=0 op_state=1 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj69[0] 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj74 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj74 op=0 op_state=1 2024-04-16T18:30:10.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj74[0] 2024-04-16T18:30:10.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj76 2024-04-16T18:30:10.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj76 op=0 op_state=1 2024-04-16T18:30:10.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj76[0] 2024-04-16T18:30:10.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj37]: success 2024-04-16T18:30:10.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj85 2024-04-16T18:30:10.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj85 op=0 op_state=1 2024-04-16T18:30:10.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj85[0] 2024-04-16T18:30:10.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj87 2024-04-16T18:30:10.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj87 op=0 op_state=1 2024-04-16T18:30:10.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj87[0] 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj44]: success 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj148]: success 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj92 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj92 op=0 op_state=1 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj92[0] 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj93 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj93 op=0 op_state=1 2024-04-16T18:30:10.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj93[0] 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj23]: success 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj38]: success 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: [full sync] syncing object: lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0/obj95 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b=bucket sync single entry (source_zone=054102a3-8d94-4596-8b3b-6341eab312e1) b= log_entry=obj95 op=0 op_state=1 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: bucket sync: sync obj: 054102a3-8d94-4596-8b3b-6341eab312e1/:lybtug-52[054102a3-8d94-4596-8b3b-6341eab312e1.4804.21])/obj95[0] 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj34]: success 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.743+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj22]: success 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.747+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj124]: success 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.747+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj124 2024-04-16T18:30:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.747+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj33]: success 2024-04-16T18:30:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.747+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj55]: success 2024-04-16T18:30:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj58]: success 2024-04-16T18:30:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj57]: success 2024-04-16T18:30:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj68]: success 2024-04-16T18:30:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj145]: success 2024-04-16T18:30:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj51]: success 2024-04-16T18:30:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj62]: success 2024-04-16T18:30:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj17]: success 2024-04-16T18:30:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj76]: success 2024-04-16T18:30:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj74]: success 2024-04-16T18:30:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.751+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj93]: success 2024-04-16T18:30:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj87]: success 2024-04-16T18:30:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj85]: success 2024-04-16T18:30:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj69]: success 2024-04-16T18:30:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj87 2024-04-16T18:30:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj95]: success 2024-04-16T18:30:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:entry[obj92]: success 2024-04-16T18:30:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.755+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: updating marker oid=bucket.full-sync-status.054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21 marker=obj95 2024-04-16T18:30:10.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.759+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: set bucket state=incremental 2024-04-16T18:30:10.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.763+0000 7f19c62fd980 5 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:full_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: bucket status objv={r=_csTmBkUGHazPm_P8NdwY13K:151,w=:0} 2024-04-16T18:30:10.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.767+0000 7f19c62fd980 10 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: WARNING: requested sync of past generation 0 < 6, returning success 2024-04-16T18:30:10.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.775+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.775+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:1]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.775+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:4]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.775+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:2]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.775+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: sync status for source bucket shard: 2 2024-04-16T18:30:10.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.775+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:0]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:5]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:1]: sync status for source bucket shard: 2 2024-04-16T18:30:10.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:1]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:1]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:4]: sync status for source bucket shard: 2 2024-04-16T18:30:10.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:4]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:4]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:2]: sync status for source bucket shard: 2 2024-04-16T18:30:10.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:2]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:2]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:6]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:3]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:5]: sync status for source bucket shard: 2 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:5]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:5]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:7]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:6]: sync status for source bucket shard: 2 2024-04-16T18:30:10.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:6]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:6]: listing bilog for incremental sync; position=00000000002.83.6 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:8]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:3]: sync status for source bucket shard: 2 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:3]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:3]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:7]: sync status for source bucket shard: 2 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.779+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:7]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:7]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:9]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:10]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:8]: sync status for source bucket shard: 2 2024-04-16T18:30:10.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:8]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:8]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:11]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:9]: sync status for source bucket shard: 2 2024-04-16T18:30:10.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:9]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:9]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:12]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:10]: sync status for source bucket shard: 2 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:10]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:10]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:13]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:11]: sync status for source bucket shard: 2 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:11]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:11]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:14]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:12]: sync status for source bucket shard: 2 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:12]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:12]: listing bilog for incremental sync; position=00000000002.81.6 2024-04-16T18:30:10.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:15]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:14]: sync status for source bucket shard: 2 2024-04-16T18:30:10.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.783+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:14]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:14]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:16]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:13]: sync status for source bucket shard: 2 2024-04-16T18:30:10.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:13]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:13]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:17]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:19]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:15]: sync status for source bucket shard: 2 2024-04-16T18:30:10.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:15]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:15]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:20]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:16]: sync status for source bucket shard: 2 2024-04-16T18:30:10.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:16]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:16]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:21]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:17]: sync status for source bucket shard: 2 2024-04-16T18:30:10.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:17]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:17]: listing bilog for incremental sync; position=00000000002.159.6 2024-04-16T18:30:10.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:19]: sync status for source bucket shard: 2 2024-04-16T18:30:10.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:19]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:19]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:20]: sync status for source bucket shard: 2 2024-04-16T18:30:10.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:20]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:20]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.787+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:22]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:23]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:21]: sync status for source bucket shard: 2 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:21]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:21]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:25]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:24]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:22]: sync status for source bucket shard: 2 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:22]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:22]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:23]: sync status for source bucket shard: 2 2024-04-16T18:30:10.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:23]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:23]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:25]: sync status for source bucket shard: 2 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:25]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:25]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:26]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:24]: sync status for source bucket shard: 2 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:27]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:24]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:24]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:28]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:29]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:26]: sync status for source bucket shard: 2 2024-04-16T18:30:10.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.791+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:26]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:26]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:30]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:18]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:27]: sync status for source bucket shard: 2 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:27]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:27]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:28]: sync status for source bucket shard: 2 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:31]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:28]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:28]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:29]: sync status for source bucket shard: 2 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:29]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:29]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:32]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:30]: sync status for source bucket shard: 2 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:30]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:30]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:33]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:18]: sync status for source bucket shard: 2 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:18]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:18]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:34]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:31]: sync status for source bucket shard: 2 2024-04-16T18:30:10.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:31]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:31]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:36]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:35]: sync status for source bucket: incremental. lease is: not taken. stop indications is: 0 2024-04-16T18:30:10.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:32]: sync status for source bucket shard: 2 2024-04-16T18:30:10.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.795+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:32]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:32]: listing bilog for incremental sync; position=00000000001.53.6 2024-04-16T18:30:10.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:36]: sync status for source bucket shard: 2 2024-04-16T18:30:10.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:36]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:36]: listing bilog for incremental sync; position=00000000001.98.6 2024-04-16T18:30:10.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:34]: sync status for source bucket shard: 2 2024-04-16T18:30:10.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:34]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:34]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:33]: sync status for source bucket shard: 2 2024-04-16T18:30:10.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:33]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:33]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:35]: sync status for source bucket shard: 2 2024-04-16T18:30:10.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:10.799+0000 7f19c62fd980 20 RGW-SYNC:bucket[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21<-lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:35]:inc_sync[lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21:35]: listing bilog for incremental sync; position= 2024-04-16T18:30:10.912 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'list', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--max-entries', '10000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:10.912 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'list', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--max-entries', '10000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:10.912 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket list --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --max-entries 10000 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:10.957 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:10.957 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:11.064 INFO:teuthology.orchestra.run.smithi071.stdout:[{"name":"obj11","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:20:00.237172Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15048585085549124383","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj111","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:27:53.377102Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.7715981864926805076","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj115","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:23:25.802945Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6218543018970983402","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj116","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:25:26.853472Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2925059832229808770","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj124","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:26:49.340484Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15585524890135659897","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj145","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:25:26.904602Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.4684596270171826372","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj148","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.397807Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.5689811379878906403","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj155","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:26:49.392862Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2482291167386272197","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj17","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:12.390828Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2252280313833029433","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj185","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.450035Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1275474698075478620","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj19","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:22:20.871403Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.11629121535673071090","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj22","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:20:00.257626Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.3133044097907854457","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj23","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:23:25.664581Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.16618771069852409686","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj29","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:25:26.766409Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17740967203672950191","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj31","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:26:49.246767Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.13939750285080369137","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj33","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:00.309065Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.1671788961684431482","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj34","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:20:12.407283Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15867562824846839825","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj37","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.306174Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.18438757787962165956","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj38","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:22:20.888430Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2315008106278844429","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj44","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:00.328043Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.3542996355204485727","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj46","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:23:25.714804Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2388773711748696897","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj51","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:12.458478Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.4043139877701837521","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj55","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:00.377847Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.10749734705266288713","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj57","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:22:20.901936Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.4345151050549957310","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj58","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:25:26.782716Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15588284110387865913","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj62","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:26:49.266169Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.974113629449544127","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj68","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:12.476177Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6675146936156308698","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj69","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:23:25.732337Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.5676869889308184003","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj74","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:27:53.324563Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.13596534151300317176","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj76","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:22:20.954658Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15791149183813234963","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj85","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:12.526435Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.16291806910261604854","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj87","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:25:26.833550Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.17986397381064747975","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj92","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:23:25.782803Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15192481628349403358","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj93","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:26:49.317259Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.13660566367733892003","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj95","instance":"","ver":{"pool":9,"epoch":0},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:22:20.973293Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.6440445776255876545","flags":0,"pending_map":[],"versioned_epoch":0}] 2024-04-16T18:30:11.065 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:11.065 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'lybtug-52', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:11.065 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket list --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket lybtug-52 --max-entries 10000 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:11.115 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:11.115 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:11.222 INFO:teuthology.orchestra.run.smithi122.stdout:[{"name":"obj11","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:20:00.237172Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_SamGi9QqDkFEwBLcujxRpSj5U-m_Z5m","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj111","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.377102Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_Q6AEWodi5r9IOYvJsx0vJMe8_TX3zPI","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj115","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:23:25.802945Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_OG824mC7nIicH2smFLZIf8MLEhI0Cwe","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj116","instance":"","ver":{"pool":9,"epoch":5},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:25:26.853472Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_c-e0fIoTKME-ks9F3Px7ioYq6XtKVDd","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj124","instance":"","ver":{"pool":9,"epoch":12},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:26:49.340484Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_FAHaws4d7MLcTPc06tywWyNVCXxmDf1","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj145","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:25:26.904602Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_CeAp_ujiCUe7uv6IUi-0zePlwUyj68-","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj148","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:27:53.397807Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_K3gak_M46hEsVnJonA8uHgy8KWoXZ4e","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj155","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:26:49.392862Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_my_KCosF-p-h3Ni-nEPCFi5F-uWyrBK","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj17","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:12.390828Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_bQju_E0byYU2misQ56mLbzZ7OlaAuLo","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj185","instance":"","ver":{"pool":9,"epoch":19},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.450035Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_8EHLtq6OKRgIV8XTUGirTPN55wu-tiQ","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj19","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:22:20.871403Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_Uf3LuinQdORCkmX_pKaQ-4HUy6stlPN","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj22","instance":"","ver":{"pool":9,"epoch":40},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:20:00.257626Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_Hn3lyXKI221DYGSgs7TujlQdphXguOP","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj23","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:23:25.664581Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_vaHxbafuIwhGS66wYJkbb2CHbFX2lZk","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj29","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:25:26.766409Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lnjInR5RB2bA7K-qJdcthmwesxhJHuy","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj31","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:26:49.246767Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_47_PNSGvrZGgvjnhX-TkFh7bL9rot8z","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj33","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:00.309065Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_U7-mrAIWErwnB8i0POrnAODsh6jQ4qI","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj34","instance":"","ver":{"pool":9,"epoch":16},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:12.407283Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_BcdQ2wyM6eTk3TH4sw-eNz1TecxFkGp","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj37","instance":"","ver":{"pool":9,"epoch":13},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:27:53.306174Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_c0pg-Q3Mko4wg7YA3P55AEQM7oQLUN6","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj38","instance":"","ver":{"pool":9,"epoch":13},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:22:20.888430Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_-Fq-URvAYw5_XiZYglxrLblcyGATaEU","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj44","instance":"","ver":{"pool":9,"epoch":14},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:20:00.328043Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_l4vT69F0dIM5s5PaG7AJRhU-XkeCcHe","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj46","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:23:25.714804Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_bXgYFZ0hW39rmq3OLtAMNfRH4LRw1xb","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj51","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:20:12.458478Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_F5TpMdhm8yCqk0O4ByjAsUniCkJQO8J","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj55","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:20:00.377847Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_UbRvXLgbqla5O7lt8ZvWE8vJuFGQEB0","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj57","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:22:20.901936Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_fHXAjdGuLMWb-hGvcYgSxyTUFo3zAM0","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj58","instance":"","ver":{"pool":9,"epoch":11},"locator":"","exists":true,"meta":{"category":1,"size":6,"mtime":"2024-04-16T18:25:26.782716Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_p3BC7w0CUGYGCm-kPCL78zqGxqaWmI7","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj62","instance":"","ver":{"pool":9,"epoch":10},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:26:49.266169Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_vtqmydoM22dufeP2I2aq1TSP0G-z5Jy","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj68","instance":"","ver":{"pool":9,"epoch":13},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:20:12.476177Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_9WihJZutGTdJ8HiYbdE-4afnr88yCq6","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj69","instance":"","ver":{"pool":9,"epoch":20},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:23:25.732337Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_d1-pE4HRdAL_gQzhvTA8XGWGSqaCAjz","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj74","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:27:53.324563Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_tZAHNliP08Anj_N4Une1nFL68PBm3v7","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj76","instance":"","ver":{"pool":9,"epoch":8},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:22:20.954658Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_i-6rylMgTXCx308iV7leA3Xng43x-Gi","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj85","instance":"","ver":{"pool":9,"epoch":7},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:20:12.526435Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_8UtZ_Q6lk-4SAnObOvZ_s1WW8OAFdUg","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj87","instance":"","ver":{"pool":9,"epoch":9},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:25:26.833550Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_qOQH3zQcNMxmPLSQY7lc5N641vr4Ypi","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj92","instance":"","ver":{"pool":9,"epoch":12},"locator":"","exists":true,"meta":{"category":1,"size":16,"mtime":"2024-04-16T18:23:25.782803Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_9yvPJItOHCqkSG32qJzKZl5_kXVj8Oo","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj93","instance":"","ver":{"pool":9,"epoch":6},"locator":"","exists":true,"meta":{"category":1,"size":5,"mtime":"2024-04-16T18:26:49.317259Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_lnFEPJkCsidz-gko3M9wBJMl8E3ZMY5","flags":0,"pending_map":[],"versioned_epoch":0},{"name":"obj95","instance":"","ver":{"pool":9,"epoch":15},"locator":"","exists":true,"meta":{"category":1,"size":3,"mtime":"2024-04-16T18:22:20.973293Z","etag":"acbd18db4cc2f85cedef654fccc4a4d8","storage_class":"","owner":"RGW11111111111111111","owner_display_name":"","content_type":"application/octet-stream","accounted_size":3,"user_data":"","appendable":false},"tag":"_kcwhF0Z9nerbcE9D67t0OPBqO372LEW","flags":0,"pending_map":[],"versioned_epoch":0}] 2024-04-16T18:30:11.223 INFO:tasks.rgw.c2.client.0:Restarting daemon 2024-04-16T18:30:11.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T18:30:11.225 INFO:tasks.rgw.c2.client.0:Started 2024-04-16T18:30:11.225 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:11.234 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T18:30:11.234 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:30:11.234 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:30:11.234 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T18:30:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:curl: (7) Failed to connect to smithi122.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T18:30:11.498 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:30:11.499+0000 7f204d29ea40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T18:30:12.236 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:12.247 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:30:12.247 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:30:12.249 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 116k 0 --:--:-- --:--:-- --:--:-- 182k 2024-04-16T18:30:12.249 INFO:teuthology.orchestra.run.smithi122.stdout:anonymous 2024-04-16T18:30:12.250 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-52 2024-04-16T18:30:12.250 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:30:12.250 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:30:12.250 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:30:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:12.427+0000 7f70b8756980 1 bucket sync caught up with source: 2024-04-16T18:30:12.429 INFO:teuthology.orchestra.run.smithi122.stderr: local status: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:30:12.429 INFO:teuthology.orchestra.run.smithi122.stderr: remote markers: [, , , , , , 00000000002.83.6, , , , , , 00000000002.81.6, , , , , 00000000002.159.6, , , , , , , , , , , , , , , 00000000001.53.6, , , , 00000000001.98.6] 2024-04-16T18:30:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:12.427+0000 7f70b8756980 0 bucket checkpoint complete 2024-04-16T18:30:12.461 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-52 2024-04-16T18:30:12.461 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:30:12.461 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-52', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:30:12.461 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-52 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T18:30:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:12.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:30:12.597+0000 7f571337f980 1 bucket sync caught up with source: 2024-04-16T18:30:12.593 INFO:teuthology.orchestra.run.smithi071.stderr: local status: [, 00000000009.661.6, 00000000006.580.6, 00000000006.495.6, 00000000015.1260.6, 00000000012.552.6, 00000000003.972.6, 00000000003.995.6, 00000000012.501.6, 00000000021.372.6, 00000000018.457.6] 2024-04-16T18:30:12.593 INFO:teuthology.orchestra.run.smithi071.stderr: remote markers: [, 00000000009.661.6, 00000000006.580.6, 00000000006.495.6, 00000000015.1260.6, 00000000012.552.6, 00000000003.972.6, 00000000003.995.6, 00000000012.501.6, 00000000021.372.6, 00000000018.457.6] 2024-04-16T18:30:12.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T18:30:12.597+0000 7f571337f980 0 bucket checkpoint complete 2024-04-16T18:30:12.624 INFO:tasks.rgw_multi.zone_rados:comparing bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:30:12.624 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:12.624 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:12.624 DEBUG:boto:Method: HEAD 2024-04-16T18:30:12.624 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:30:12.624 DEBUG:boto:Data: 2024-04-16T18:30:12.624 DEBUG:boto:Headers: {} 2024-04-16T18:30:12.624 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:12.625 DEBUG:boto:Port: 8000 2024-04-16T18:30:12.625 DEBUG:boto:Params: {} 2024-04-16T18:30:12.625 DEBUG:boto:Token: None 2024-04-16T18:30:12.625 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:12 GMT /lybtug-52/ 2024-04-16T18:30:12.625 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ssZ6eogCeS6pmXUFYbtHavXYx7Y= 2024-04-16T18:30:12.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ssZ6eogCeS6pmXUFYbtHavXYx7Y=', 'Content-Length': '0'} 2024-04-16T18:30:12.626 DEBUG:boto:encountered RemoteDisconnected exception, reconnecting 2024-04-16T18:30:12.626 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:30:13.035 DEBUG:boto:Token: None 2024-04-16T18:30:13.035 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:12 GMT /lybtug-52/ 2024-04-16T18:30:13.035 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ssZ6eogCeS6pmXUFYbtHavXYx7Y= 2024-04-16T18:30:13.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:12 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ssZ6eogCeS6pmXUFYbtHavXYx7Y=', 'Content-Length': '0'} 2024-04-16T18:30:13.046 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001a117aea7af3e3ac-00661ec3b5-6022-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.046 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.047 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.047 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.047 DEBUG:boto:Path: /lybtug-52/ 2024-04-16T18:30:13.047 DEBUG:boto:Data: 2024-04-16T18:30:13.047 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.047 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.047 DEBUG:boto:Params: {} 2024-04-16T18:30:13.047 DEBUG:boto:Token: None 2024-04-16T18:30:13.047 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V/mYv5vB004eExQUqroohD1xOm8= 2024-04-16T18:30:13.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V/mYv5vB004eExQUqroohD1xOm8=', 'Content-Length': '0'} 2024-04-16T18:30:13.054 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003c20ea93398e0a1c-00661ec3b5-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.054 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2024-04-16T18:30:13.054 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.054 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.054 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.054 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.055 DEBUG:boto:Method: GET 2024-04-16T18:30:13.055 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.055 DEBUG:boto:Data: 2024-04-16T18:30:13.055 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.055 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.055 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.055 DEBUG:boto:Params: {} 2024-04-16T18:30:13.055 DEBUG:boto:Token: None 2024-04-16T18:30:13.055 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/?versions 2024-04-16T18:30:13.055 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI= 2024-04-16T18:30:13.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI=', 'Content-Length': '0'} 2024-04-16T18:30:13.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a11c8354a974e65f-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.061 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:30:13.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2024-04-16T18:30:13.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:30:13.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2024-04-16T18:30:13.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2024-04-16T18:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:30:13.067 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2024-04-16T18:30:13.067 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.067 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.067 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.067 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.067 DEBUG:boto:Method: GET 2024-04-16T18:30:13.067 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.067 DEBUG:boto:Data: 2024-04-16T18:30:13.067 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.067 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.068 DEBUG:boto:Params: {} 2024-04-16T18:30:13.068 DEBUG:boto:Token: None 2024-04-16T18:30:13.068 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/?versions 2024-04-16T18:30:13.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI= 2024-04-16T18:30:13.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI=', 'Content-Length': '0'} 2024-04-16T18:30:13.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000881021e4960410a8-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.076 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.077 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2024-04-16T18:30:13.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2024-04-16T18:30:13.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2024-04-16T18:30:13.080 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2024-04-16T18:30:13.081 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.081 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.081 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.081 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.081 DEBUG:boto:Method: GET 2024-04-16T18:30:13.081 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.081 DEBUG:boto:Data: 2024-04-16T18:30:13.081 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.081 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.082 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.082 DEBUG:boto:Params: {} 2024-04-16T18:30:13.082 DEBUG:boto:Token: None 2024-04-16T18:30:13.082 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/?versions 2024-04-16T18:30:13.082 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI= 2024-04-16T18:30:13.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI=', 'Content-Length': '0'} 2024-04-16T18:30:13.087 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d99038e4b0d110d9-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.087 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.089 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.089 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.089 DEBUG:boto:path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.089 DEBUG:boto:auth_path=/lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.089 DEBUG:boto:Method: GET 2024-04-16T18:30:13.089 DEBUG:boto:Path: /lybtug-52/?versions&max-keys=999 2024-04-16T18:30:13.089 DEBUG:boto:Data: 2024-04-16T18:30:13.090 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.090 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.090 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.090 DEBUG:boto:Params: {} 2024-04-16T18:30:13.090 DEBUG:boto:Token: None 2024-04-16T18:30:13.090 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/?versions 2024-04-16T18:30:13.090 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI= 2024-04-16T18:30:13.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2eh9I12V4wFc+B829H8T2wGyiTI=', 'Content-Length': '0'} 2024-04-16T18:30:13.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003af469719c9c90b9-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.100 DEBUG:boto:b'lybtug-52999falseobj11nulltrue2024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj111nulltrue2024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj115nulltrue2024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj116nulltrue2024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj124nulltrue2024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj145nulltrue2024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj148nulltrue2024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj155nulltrue2024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj17nulltrue2024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj185nulltrue2024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj19nulltrue2024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj22nulltrue2024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj23nulltrue2024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj29nulltrue2024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj31nulltrue2024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj33nulltrue2024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj34nulltrue2024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj37nulltrue2024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj38nulltrue2024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj44nulltrue2024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj46nulltrue2024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj51nulltrue2024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj55nulltrue2024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj57nulltrue2024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj58nulltrue2024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj62nulltrue2024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj68nulltrue2024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj69nulltrue2024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj74nulltrue2024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj76nulltrue2024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj85nulltrue2024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj87nulltrue2024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj92nulltrue2024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj93nulltrue2024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj95nulltrue2024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:30:13.102 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.102 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.102 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.102 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.102 DEBUG:boto:Method: GET 2024-04-16T18:30:13.102 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.102 DEBUG:boto:Data: 2024-04-16T18:30:13.102 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.102 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.102 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.103 DEBUG:boto:Params: {} 2024-04-16T18:30:13.103 DEBUG:boto:Token: None 2024-04-16T18:30:13.103 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.103 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q= 2024-04-16T18:30:13.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q=', 'Content-Length': '0'} 2024-04-16T18:30:13.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004254fe442aaf964a-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.150 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.150 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.150 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.151 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.151 DEBUG:boto:Method: GET 2024-04-16T18:30:13.151 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.151 DEBUG:boto:Data: 2024-04-16T18:30:13.151 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.151 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.151 DEBUG:boto:Params: {} 2024-04-16T18:30:13.151 DEBUG:boto:Token: None 2024-04-16T18:30:13.152 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q= 2024-04-16T18:30:13.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q=', 'Content-Length': '0'} 2024-04-16T18:30:13.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f36c5706cddd88d9-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.155 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.155 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.155 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.155 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.156 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.156 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.156 DEBUG:boto:Data: 2024-04-16T18:30:13.156 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.156 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.156 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.156 DEBUG:boto:Params: {} 2024-04-16T18:30:13.156 DEBUG:boto:Token: None 2024-04-16T18:30:13.156 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.157 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ctGeOW6koLnRPuhrf5yrOxZkXNc= 2024-04-16T18:30:13.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ctGeOW6koLnRPuhrf5yrOxZkXNc=', 'Content-Length': '0'} 2024-04-16T18:30:13.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdd9a698e12dd6dd-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.160 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.160 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.160 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.160 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.160 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.160 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.160 DEBUG:boto:Data: 2024-04-16T18:30:13.160 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.160 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.161 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.161 DEBUG:boto:Params: {} 2024-04-16T18:30:13.161 DEBUG:boto:Token: None 2024-04-16T18:30:13.161 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.161 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ctGeOW6koLnRPuhrf5yrOxZkXNc= 2024-04-16T18:30:13.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ctGeOW6koLnRPuhrf5yrOxZkXNc=', 'Content-Length': '0'} 2024-04-16T18:30:13.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f90fca5b13209f4-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.164 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:30:13.165 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.165 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.165 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.165 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.165 DEBUG:boto:Method: GET 2024-04-16T18:30:13.165 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.165 DEBUG:boto:Data: 2024-04-16T18:30:13.165 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.165 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.165 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.165 DEBUG:boto:Params: {} 2024-04-16T18:30:13.166 DEBUG:boto:Token: None 2024-04-16T18:30:13.166 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.166 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q= 2024-04-16T18:30:13.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q=', 'Content-Length': '0'} 2024-04-16T18:30:13.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003795442c730936ac-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.210 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.210 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.210 DEBUG:boto:path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.211 DEBUG:boto:auth_path=/lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.211 DEBUG:boto:Method: GET 2024-04-16T18:30:13.211 DEBUG:boto:Path: /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.211 DEBUG:boto:Data: 2024-04-16T18:30:13.211 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.211 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.211 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.211 DEBUG:boto:Params: {} 2024-04-16T18:30:13.211 DEBUG:boto:Token: None 2024-04-16T18:30:13.211 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11?versionId=null 2024-04-16T18:30:13.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q= 2024-04-16T18:30:13.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mZ6N/No0kmryOeKZ8on1ptE535Q=', 'Content-Length': '0'} 2024-04-16T18:30:13.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af017926d76ebaf9-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.215 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.215 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.215 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.216 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.216 DEBUG:boto:Method: GET 2024-04-16T18:30:13.216 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:30:13.216 DEBUG:boto:Data: 2024-04-16T18:30:13.216 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.216 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.216 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.216 DEBUG:boto:Params: {} 2024-04-16T18:30:13.216 DEBUG:boto:Token: None 2024-04-16T18:30:13.216 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.217 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.220 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd3cc656cd13b00a-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.262 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.263 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.263 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.263 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.263 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.263 DEBUG:boto:Method: GET 2024-04-16T18:30:13.263 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:30:13.263 DEBUG:boto:Data: 2024-04-16T18:30:13.263 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.263 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.264 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.264 DEBUG:boto:Params: {} 2024-04-16T18:30:13.264 DEBUG:boto:Token: None 2024-04-16T18:30:13.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb3d1692623951b0-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.271 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.271 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.272 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.272 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.272 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:30:13.272 DEBUG:boto:Data: 2024-04-16T18:30:13.272 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.272 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.272 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.272 DEBUG:boto:Params: {} 2024-04-16T18:30:13.272 DEBUG:boto:Token: None 2024-04-16T18:30:13.272 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11 2024-04-16T18:30:13.272 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x8MRC8dQKmhIJPI1NJjwTmSJ1A0= 2024-04-16T18:30:13.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x8MRC8dQKmhIJPI1NJjwTmSJ1A0=', 'Content-Length': '0'} 2024-04-16T18:30:13.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000132a19d615644f2a-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.276 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.276 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.276 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.276 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:30:13.276 DEBUG:boto:Data: 2024-04-16T18:30:13.277 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.277 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.277 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.277 DEBUG:boto:Params: {} 2024-04-16T18:30:13.277 DEBUG:boto:Token: None 2024-04-16T18:30:13.277 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11 2024-04-16T18:30:13.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:x8MRC8dQKmhIJPI1NJjwTmSJ1A0= 2024-04-16T18:30:13.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:x8MRC8dQKmhIJPI1NJjwTmSJ1A0=', 'Content-Length': '0'} 2024-04-16T18:30:13.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047c14cb9282428ac-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.280 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2024-04-16T18:30:13.280 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.280 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.281 DEBUG:boto:Method: GET 2024-04-16T18:30:13.281 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:30:13.281 DEBUG:boto:Data: 2024-04-16T18:30:13.281 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.281 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.281 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.281 DEBUG:boto:Params: {} 2024-04-16T18:30:13.281 DEBUG:boto:Token: None 2024-04-16T18:30:13.281 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11 2024-04-16T18:30:13.281 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WWVSxGv6FlwIRebVm6ioW0JpJC8= 2024-04-16T18:30:13.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WWVSxGv6FlwIRebVm6ioW0JpJC8=', 'Content-Length': '0'} 2024-04-16T18:30:13.284 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000489cb4822dac04a7-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.285 DEBUG:boto:path=/lybtug-52/obj11 2024-04-16T18:30:13.285 DEBUG:boto:auth_path=/lybtug-52/obj11 2024-04-16T18:30:13.285 DEBUG:boto:Method: GET 2024-04-16T18:30:13.285 DEBUG:boto:Path: /lybtug-52/obj11 2024-04-16T18:30:13.285 DEBUG:boto:Data: 2024-04-16T18:30:13.285 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.285 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.285 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.285 DEBUG:boto:Params: {} 2024-04-16T18:30:13.286 DEBUG:boto:Token: None 2024-04-16T18:30:13.286 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj11 2024-04-16T18:30:13.286 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WWVSxGv6FlwIRebVm6ioW0JpJC8= 2024-04-16T18:30:13.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WWVSxGv6FlwIRebVm6ioW0JpJC8=', 'Content-Length': '0'} 2024-04-16T18:30:13.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0ce77e9c30f30c8-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.330 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.330 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.331 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.331 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.331 DEBUG:boto:Method: GET 2024-04-16T18:30:13.331 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:30:13.331 DEBUG:boto:Data: 2024-04-16T18:30:13.331 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.331 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.331 DEBUG:boto:Params: {} 2024-04-16T18:30:13.331 DEBUG:boto:Token: None 2024-04-16T18:30:13.331 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086190fe4166298ff-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.335 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.336 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.336 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.336 DEBUG:boto:path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.336 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj11 2024-04-16T18:30:13.336 DEBUG:boto:Method: GET 2024-04-16T18:30:13.336 DEBUG:boto:Path: /lybtug-52/?prefix=obj11 2024-04-16T18:30:13.336 DEBUG:boto:Data: 2024-04-16T18:30:13.336 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.337 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.337 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.337 DEBUG:boto:Params: {} 2024-04-16T18:30:13.337 DEBUG:boto:Token: None 2024-04-16T18:30:13.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c241632d9cb44ba-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.344 DEBUG:boto:b'lybtug-52obj111000falseobj112024-04-16T18:20:00.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2024-04-16T18:30:13.344 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.344 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.345 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.345 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.345 DEBUG:boto:Method: GET 2024-04-16T18:30:13.345 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.345 DEBUG:boto:Data: 2024-04-16T18:30:13.345 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.345 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.346 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.346 DEBUG:boto:Params: {} 2024-04-16T18:30:13.346 DEBUG:boto:Token: None 2024-04-16T18:30:13.346 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.346 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU= 2024-04-16T18:30:13.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU=', 'Content-Length': '0'} 2024-04-16T18:30:13.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018ae06a12fb2ea26-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.390 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.390 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.391 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.391 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.391 DEBUG:boto:Method: GET 2024-04-16T18:30:13.391 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.391 DEBUG:boto:Data: 2024-04-16T18:30:13.391 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.391 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.391 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.391 DEBUG:boto:Params: {} 2024-04-16T18:30:13.391 DEBUG:boto:Token: None 2024-04-16T18:30:13.392 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.392 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU= 2024-04-16T18:30:13.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU=', 'Content-Length': '0'} 2024-04-16T18:30:13.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8f7b9ea9bc177ca-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.395 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.395 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.395 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.395 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.395 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.396 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.396 DEBUG:boto:Data: 2024-04-16T18:30:13.396 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.396 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.396 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.396 DEBUG:boto:Params: {} 2024-04-16T18:30:13.396 DEBUG:boto:Token: None 2024-04-16T18:30:13.396 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tbB6L4XR/V5j88rR+ZZxpiGoNlI= 2024-04-16T18:30:13.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tbB6L4XR/V5j88rR+ZZxpiGoNlI=', 'Content-Length': '0'} 2024-04-16T18:30:13.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000734a688c6703dc9d-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.400 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.400 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.400 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.400 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.400 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.400 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.400 DEBUG:boto:Data: 2024-04-16T18:30:13.400 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.400 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.400 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.400 DEBUG:boto:Params: {} 2024-04-16T18:30:13.401 DEBUG:boto:Token: None 2024-04-16T18:30:13.401 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.401 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tbB6L4XR/V5j88rR+ZZxpiGoNlI= 2024-04-16T18:30:13.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tbB6L4XR/V5j88rR+ZZxpiGoNlI=', 'Content-Length': '0'} 2024-04-16T18:30:13.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b287e33e24932e3c-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2024-04-16T18:30:13.404 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.404 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.404 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.404 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.404 DEBUG:boto:Method: GET 2024-04-16T18:30:13.404 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.404 DEBUG:boto:Data: 2024-04-16T18:30:13.405 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.405 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.405 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.405 DEBUG:boto:Params: {} 2024-04-16T18:30:13.405 DEBUG:boto:Token: None 2024-04-16T18:30:13.405 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.405 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU= 2024-04-16T18:30:13.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU=', 'Content-Length': '0'} 2024-04-16T18:30:13.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a9baab9795dfbb7-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.409 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.409 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.409 DEBUG:boto:path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.409 DEBUG:boto:auth_path=/lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.409 DEBUG:boto:Method: GET 2024-04-16T18:30:13.409 DEBUG:boto:Path: /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.409 DEBUG:boto:Data: 2024-04-16T18:30:13.409 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.409 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.409 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.409 DEBUG:boto:Params: {} 2024-04-16T18:30:13.410 DEBUG:boto:Token: None 2024-04-16T18:30:13.410 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111?versionId=null 2024-04-16T18:30:13.410 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU= 2024-04-16T18:30:13.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HPaEmKmZQUnWbMb1FbiqWPCJGAU=', 'Content-Length': '0'} 2024-04-16T18:30:13.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063a200eab34df1b9-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.454 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.455 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.455 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.455 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.455 DEBUG:boto:Method: GET 2024-04-16T18:30:13.455 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:30:13.455 DEBUG:boto:Data: 2024-04-16T18:30:13.455 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.455 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.455 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.455 DEBUG:boto:Params: {} 2024-04-16T18:30:13.455 DEBUG:boto:Token: None 2024-04-16T18:30:13.456 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.456 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.459 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d78fb33c58453296-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.459 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.460 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.460 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.460 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.460 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.460 DEBUG:boto:Method: GET 2024-04-16T18:30:13.460 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:30:13.460 DEBUG:boto:Data: 2024-04-16T18:30:13.460 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.460 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.460 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.460 DEBUG:boto:Params: {} 2024-04-16T18:30:13.461 DEBUG:boto:Token: None 2024-04-16T18:30:13.461 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.461 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008db922e634622565-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.468 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.468 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.468 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.468 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.468 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:30:13.468 DEBUG:boto:Data: 2024-04-16T18:30:13.469 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.469 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.469 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.469 DEBUG:boto:Params: {} 2024-04-16T18:30:13.469 DEBUG:boto:Token: None 2024-04-16T18:30:13.469 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111 2024-04-16T18:30:13.469 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xo+yIqIBnNERZ23MIaGnK9tOUfI= 2024-04-16T18:30:13.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xo+yIqIBnNERZ23MIaGnK9tOUfI=', 'Content-Length': '0'} 2024-04-16T18:30:13.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5a015218b199b62-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.473 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.473 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.473 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.473 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:30:13.473 DEBUG:boto:Data: 2024-04-16T18:30:13.473 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.473 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.473 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.473 DEBUG:boto:Params: {} 2024-04-16T18:30:13.473 DEBUG:boto:Token: None 2024-04-16T18:30:13.473 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111 2024-04-16T18:30:13.474 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:xo+yIqIBnNERZ23MIaGnK9tOUfI= 2024-04-16T18:30:13.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xo+yIqIBnNERZ23MIaGnK9tOUfI=', 'Content-Length': '0'} 2024-04-16T18:30:13.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002576ff1eb8f3040a-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.477 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2024-04-16T18:30:13.477 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.477 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.477 DEBUG:boto:Method: GET 2024-04-16T18:30:13.477 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:30:13.477 DEBUG:boto:Data: 2024-04-16T18:30:13.477 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.477 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.477 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.478 DEBUG:boto:Params: {} 2024-04-16T18:30:13.478 DEBUG:boto:Token: None 2024-04-16T18:30:13.478 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111 2024-04-16T18:30:13.478 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TlDgyLs1XPoalzD6Rw7STJnjT90= 2024-04-16T18:30:13.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TlDgyLs1XPoalzD6Rw7STJnjT90=', 'Content-Length': '0'} 2024-04-16T18:30:13.481 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006314d246cba089c0-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.522 DEBUG:boto:path=/lybtug-52/obj111 2024-04-16T18:30:13.522 DEBUG:boto:auth_path=/lybtug-52/obj111 2024-04-16T18:30:13.523 DEBUG:boto:Method: GET 2024-04-16T18:30:13.523 DEBUG:boto:Path: /lybtug-52/obj111 2024-04-16T18:30:13.523 DEBUG:boto:Data: 2024-04-16T18:30:13.523 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.523 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.523 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.523 DEBUG:boto:Params: {} 2024-04-16T18:30:13.523 DEBUG:boto:Token: None 2024-04-16T18:30:13.523 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj111 2024-04-16T18:30:13.523 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:TlDgyLs1XPoalzD6Rw7STJnjT90= 2024-04-16T18:30:13.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TlDgyLs1XPoalzD6Rw7STJnjT90=', 'Content-Length': '0'} 2024-04-16T18:30:13.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052f3a34fa7003f29-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.527 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.527 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.527 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.527 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.527 DEBUG:boto:Method: GET 2024-04-16T18:30:13.527 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:30:13.527 DEBUG:boto:Data: 2024-04-16T18:30:13.527 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.527 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.528 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.528 DEBUG:boto:Params: {} 2024-04-16T18:30:13.528 DEBUG:boto:Token: None 2024-04-16T18:30:13.528 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.528 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.532 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb1d2f39059da530-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.532 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.532 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.533 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.533 DEBUG:boto:path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.533 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj111 2024-04-16T18:30:13.533 DEBUG:boto:Method: GET 2024-04-16T18:30:13.533 DEBUG:boto:Path: /lybtug-52/?prefix=obj111 2024-04-16T18:30:13.533 DEBUG:boto:Data: 2024-04-16T18:30:13.533 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.533 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.533 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.533 DEBUG:boto:Params: {} 2024-04-16T18:30:13.533 DEBUG:boto:Token: None 2024-04-16T18:30:13.534 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.534 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.541 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000047644b3cf57fc9c-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.582 DEBUG:boto:b'lybtug-52obj1111000falseobj1112024-04-16T18:27:53.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.583 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:30:13.583 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.583 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.583 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.583 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.583 DEBUG:boto:Method: GET 2024-04-16T18:30:13.583 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.583 DEBUG:boto:Data: 2024-04-16T18:30:13.583 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.583 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.584 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.584 DEBUG:boto:Params: {} 2024-04-16T18:30:13.584 DEBUG:boto:Token: None 2024-04-16T18:30:13.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc= 2024-04-16T18:30:13.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc=', 'Content-Length': '0'} 2024-04-16T18:30:13.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055cd44dad79356e9-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.588 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.589 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.589 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.589 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.589 DEBUG:boto:Method: GET 2024-04-16T18:30:13.589 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.589 DEBUG:boto:Data: 2024-04-16T18:30:13.589 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.589 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.589 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.589 DEBUG:boto:Params: {} 2024-04-16T18:30:13.589 DEBUG:boto:Token: None 2024-04-16T18:30:13.589 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.590 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc= 2024-04-16T18:30:13.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc=', 'Content-Length': '0'} 2024-04-16T18:30:13.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd91fb3cffc5a645-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.593 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.593 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.593 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.593 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.593 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.593 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.594 DEBUG:boto:Data: 2024-04-16T18:30:13.594 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.594 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.594 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.594 DEBUG:boto:Params: {} 2024-04-16T18:30:13.594 DEBUG:boto:Token: None 2024-04-16T18:30:13.594 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.594 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R9RDlZYppXJAsAMYV9J20Tv3Vxw= 2024-04-16T18:30:13.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R9RDlZYppXJAsAMYV9J20Tv3Vxw=', 'Content-Length': '0'} 2024-04-16T18:30:13.598 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e2fb1bd21bdb9e4-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.598 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.598 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.598 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.598 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.598 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.598 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.598 DEBUG:boto:Data: 2024-04-16T18:30:13.599 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.599 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.599 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.599 DEBUG:boto:Params: {} 2024-04-16T18:30:13.599 DEBUG:boto:Token: None 2024-04-16T18:30:13.599 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.599 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:R9RDlZYppXJAsAMYV9J20Tv3Vxw= 2024-04-16T18:30:13.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:R9RDlZYppXJAsAMYV9J20Tv3Vxw=', 'Content-Length': '0'} 2024-04-16T18:30:13.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085e0e5d8c8a28ef2-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.602 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:30:13.603 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.603 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.603 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.603 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.603 DEBUG:boto:Method: GET 2024-04-16T18:30:13.603 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.603 DEBUG:boto:Data: 2024-04-16T18:30:13.603 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.603 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.603 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.603 DEBUG:boto:Params: {} 2024-04-16T18:30:13.603 DEBUG:boto:Token: None 2024-04-16T18:30:13.604 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.604 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc= 2024-04-16T18:30:13.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc=', 'Content-Length': '0'} 2024-04-16T18:30:13.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0962762619292e3-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.607 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.607 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.608 DEBUG:boto:path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.608 DEBUG:boto:auth_path=/lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.608 DEBUG:boto:Method: GET 2024-04-16T18:30:13.608 DEBUG:boto:Path: /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.608 DEBUG:boto:Data: 2024-04-16T18:30:13.608 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.608 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.608 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.608 DEBUG:boto:Params: {} 2024-04-16T18:30:13.608 DEBUG:boto:Token: None 2024-04-16T18:30:13.608 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115?versionId=null 2024-04-16T18:30:13.609 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc= 2024-04-16T18:30:13.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N6Vm9JlA6+qp+wPAFAuBwFM1ejc=', 'Content-Length': '0'} 2024-04-16T18:30:13.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad72092a7daa24e2-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.612 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.612 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.612 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.612 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.612 DEBUG:boto:Method: GET 2024-04-16T18:30:13.612 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:30:13.612 DEBUG:boto:Data: 2024-04-16T18:30:13.612 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.612 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.612 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.612 DEBUG:boto:Params: {} 2024-04-16T18:30:13.613 DEBUG:boto:Token: None 2024-04-16T18:30:13.613 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.613 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.616 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000388462bf94cd471f-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.658 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.659 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.659 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.659 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.659 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.659 DEBUG:boto:Method: GET 2024-04-16T18:30:13.659 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:30:13.659 DEBUG:boto:Data: 2024-04-16T18:30:13.659 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.660 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.660 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.660 DEBUG:boto:Params: {} 2024-04-16T18:30:13.660 DEBUG:boto:Token: None 2024-04-16T18:30:13.660 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.667 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050f11409c2f24283-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.667 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.667 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.667 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.667 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.668 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:30:13.668 DEBUG:boto:Data: 2024-04-16T18:30:13.668 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.668 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.668 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.668 DEBUG:boto:Params: {} 2024-04-16T18:30:13.668 DEBUG:boto:Token: None 2024-04-16T18:30:13.668 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115 2024-04-16T18:30:13.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qxc+mHiT5/hrpgx4RFRAFXeBjlM= 2024-04-16T18:30:13.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qxc+mHiT5/hrpgx4RFRAFXeBjlM=', 'Content-Length': '0'} 2024-04-16T18:30:13.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7a538dcc1dea282-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.672 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.672 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.672 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.672 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:30:13.672 DEBUG:boto:Data: 2024-04-16T18:30:13.672 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.672 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.673 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.673 DEBUG:boto:Params: {} 2024-04-16T18:30:13.673 DEBUG:boto:Token: None 2024-04-16T18:30:13.673 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115 2024-04-16T18:30:13.673 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qxc+mHiT5/hrpgx4RFRAFXeBjlM= 2024-04-16T18:30:13.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qxc+mHiT5/hrpgx4RFRAFXeBjlM=', 'Content-Length': '0'} 2024-04-16T18:30:13.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001728edbca37cc90a-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.676 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2024-04-16T18:30:13.677 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.677 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.677 DEBUG:boto:Method: GET 2024-04-16T18:30:13.677 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:30:13.677 DEBUG:boto:Data: 2024-04-16T18:30:13.677 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.677 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.677 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.677 DEBUG:boto:Params: {} 2024-04-16T18:30:13.677 DEBUG:boto:Token: None 2024-04-16T18:30:13.678 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115 2024-04-16T18:30:13.678 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJLfjiYbug6FKa5dxhvPaliyY+w= 2024-04-16T18:30:13.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJLfjiYbug6FKa5dxhvPaliyY+w=', 'Content-Length': '0'} 2024-04-16T18:30:13.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a650b2235e8f0bb-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.681 DEBUG:boto:path=/lybtug-52/obj115 2024-04-16T18:30:13.681 DEBUG:boto:auth_path=/lybtug-52/obj115 2024-04-16T18:30:13.682 DEBUG:boto:Method: GET 2024-04-16T18:30:13.682 DEBUG:boto:Path: /lybtug-52/obj115 2024-04-16T18:30:13.682 DEBUG:boto:Data: 2024-04-16T18:30:13.682 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.682 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.682 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.682 DEBUG:boto:Params: {} 2024-04-16T18:30:13.682 DEBUG:boto:Token: None 2024-04-16T18:30:13.682 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj115 2024-04-16T18:30:13.682 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MJLfjiYbug6FKa5dxhvPaliyY+w= 2024-04-16T18:30:13.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MJLfjiYbug6FKa5dxhvPaliyY+w=', 'Content-Length': '0'} 2024-04-16T18:30:13.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6248cf0bab286a0-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.726 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.726 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.727 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.727 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.727 DEBUG:boto:Method: GET 2024-04-16T18:30:13.727 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:30:13.727 DEBUG:boto:Data: 2024-04-16T18:30:13.727 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.727 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.727 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.727 DEBUG:boto:Params: {} 2024-04-16T18:30:13.727 DEBUG:boto:Token: None 2024-04-16T18:30:13.727 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.727 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.731 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba0a8d6aad6d6df3-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.732 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.732 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.732 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.732 DEBUG:boto:path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.732 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj115 2024-04-16T18:30:13.732 DEBUG:boto:Method: GET 2024-04-16T18:30:13.732 DEBUG:boto:Path: /lybtug-52/?prefix=obj115 2024-04-16T18:30:13.732 DEBUG:boto:Data: 2024-04-16T18:30:13.733 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.733 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.733 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.733 DEBUG:boto:Params: {} 2024-04-16T18:30:13.733 DEBUG:boto:Token: None 2024-04-16T18:30:13.733 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.733 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b4348668f8127ce-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.740 DEBUG:boto:b'lybtug-52obj1151000falseobj1152024-04-16T18:23:25.802Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:30:13.741 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.741 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.741 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.741 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.741 DEBUG:boto:Method: GET 2024-04-16T18:30:13.741 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.741 DEBUG:boto:Data: 2024-04-16T18:30:13.741 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.741 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.741 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.742 DEBUG:boto:Params: {} 2024-04-16T18:30:13.742 DEBUG:boto:Token: None 2024-04-16T18:30:13.742 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.742 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM= 2024-04-16T18:30:13.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM=', 'Content-Length': '0'} 2024-04-16T18:30:13.746 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051c2f5c4e6203468-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.786 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.786 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.787 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.787 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.787 DEBUG:boto:Method: GET 2024-04-16T18:30:13.787 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.787 DEBUG:boto:Data: 2024-04-16T18:30:13.787 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.787 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.787 DEBUG:boto:Params: {} 2024-04-16T18:30:13.787 DEBUG:boto:Token: None 2024-04-16T18:30:13.787 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM= 2024-04-16T18:30:13.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM=', 'Content-Length': '0'} 2024-04-16T18:30:13.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022886626c7b3e158-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.791 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.791 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.791 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.792 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.792 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.792 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.792 DEBUG:boto:Data: 2024-04-16T18:30:13.792 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.792 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.792 DEBUG:boto:Params: {} 2024-04-16T18:30:13.792 DEBUG:boto:Token: None 2024-04-16T18:30:13.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.793 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Tm2gqjAlxzzV/xWoh7s/pLIj19o= 2024-04-16T18:30:13.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Tm2gqjAlxzzV/xWoh7s/pLIj19o=', 'Content-Length': '0'} 2024-04-16T18:30:13.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4b27d3571ad4817-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.796 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.796 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.796 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.796 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.796 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.796 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.796 DEBUG:boto:Data: 2024-04-16T18:30:13.797 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.797 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.797 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.797 DEBUG:boto:Params: {} 2024-04-16T18:30:13.797 DEBUG:boto:Token: None 2024-04-16T18:30:13.797 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Tm2gqjAlxzzV/xWoh7s/pLIj19o= 2024-04-16T18:30:13.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Tm2gqjAlxzzV/xWoh7s/pLIj19o=', 'Content-Length': '0'} 2024-04-16T18:30:13.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd567d6f7d332021-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:30:13.801 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.801 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.801 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.801 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.801 DEBUG:boto:Method: GET 2024-04-16T18:30:13.801 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.801 DEBUG:boto:Data: 2024-04-16T18:30:13.801 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.801 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.801 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.801 DEBUG:boto:Params: {} 2024-04-16T18:30:13.802 DEBUG:boto:Token: None 2024-04-16T18:30:13.802 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.802 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM= 2024-04-16T18:30:13.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM=', 'Content-Length': '0'} 2024-04-16T18:30:13.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0b7d50fe72e2d34-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.805 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.805 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.805 DEBUG:boto:path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.806 DEBUG:boto:auth_path=/lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.806 DEBUG:boto:Method: GET 2024-04-16T18:30:13.806 DEBUG:boto:Path: /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.806 DEBUG:boto:Data: 2024-04-16T18:30:13.806 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.806 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.806 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.806 DEBUG:boto:Params: {} 2024-04-16T18:30:13.806 DEBUG:boto:Token: None 2024-04-16T18:30:13.806 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116?versionId=null 2024-04-16T18:30:13.806 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM= 2024-04-16T18:30:13.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v4/mxokrZCos+tPnlVForcTt2cM=', 'Content-Length': '0'} 2024-04-16T18:30:13.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ba353d5711ed6fd-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.850 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.850 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.851 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.851 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.851 DEBUG:boto:Method: GET 2024-04-16T18:30:13.851 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:30:13.851 DEBUG:boto:Data: 2024-04-16T18:30:13.851 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.851 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.851 DEBUG:boto:Params: {} 2024-04-16T18:30:13.851 DEBUG:boto:Token: None 2024-04-16T18:30:13.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.855 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7bcfd3072345ee3-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.855 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.855 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.855 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.856 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.856 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.856 DEBUG:boto:Method: GET 2024-04-16T18:30:13.856 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:30:13.856 DEBUG:boto:Data: 2024-04-16T18:30:13.856 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.856 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.856 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.856 DEBUG:boto:Params: {} 2024-04-16T18:30:13.856 DEBUG:boto:Token: None 2024-04-16T18:30:13.856 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.856 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.863 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007de3f49e8a13dc56-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.863 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.864 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.864 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.864 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.864 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:30:13.864 DEBUG:boto:Data: 2024-04-16T18:30:13.864 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.864 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.864 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.864 DEBUG:boto:Params: {} 2024-04-16T18:30:13.864 DEBUG:boto:Token: None 2024-04-16T18:30:13.865 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116 2024-04-16T18:30:13.865 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qek0pejCf8xHQ/e8XLYBNmDSlhc= 2024-04-16T18:30:13.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qek0pejCf8xHQ/e8XLYBNmDSlhc=', 'Content-Length': '0'} 2024-04-16T18:30:13.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb312cb26619a402-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.868 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.868 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.868 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.868 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:30:13.868 DEBUG:boto:Data: 2024-04-16T18:30:13.868 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.868 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.869 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.869 DEBUG:boto:Params: {} 2024-04-16T18:30:13.869 DEBUG:boto:Token: None 2024-04-16T18:30:13.869 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116 2024-04-16T18:30:13.869 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Qek0pejCf8xHQ/e8XLYBNmDSlhc= 2024-04-16T18:30:13.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qek0pejCf8xHQ/e8XLYBNmDSlhc=', 'Content-Length': '0'} 2024-04-16T18:30:13.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acdcfe690e1403da-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.872 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2024-04-16T18:30:13.873 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.873 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.873 DEBUG:boto:Method: GET 2024-04-16T18:30:13.873 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:30:13.873 DEBUG:boto:Data: 2024-04-16T18:30:13.873 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.873 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.873 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.873 DEBUG:boto:Params: {} 2024-04-16T18:30:13.873 DEBUG:boto:Token: None 2024-04-16T18:30:13.874 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116 2024-04-16T18:30:13.874 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B8cEVEB54u54sA1wrGRunyohaqI= 2024-04-16T18:30:13.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B8cEVEB54u54sA1wrGRunyohaqI=', 'Content-Length': '0'} 2024-04-16T18:30:13.877 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099fc1b1f04c31d73-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.918 DEBUG:boto:path=/lybtug-52/obj116 2024-04-16T18:30:13.918 DEBUG:boto:auth_path=/lybtug-52/obj116 2024-04-16T18:30:13.919 DEBUG:boto:Method: GET 2024-04-16T18:30:13.919 DEBUG:boto:Path: /lybtug-52/obj116 2024-04-16T18:30:13.919 DEBUG:boto:Data: 2024-04-16T18:30:13.919 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.919 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.919 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.919 DEBUG:boto:Params: {} 2024-04-16T18:30:13.919 DEBUG:boto:Token: None 2024-04-16T18:30:13.919 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj116 2024-04-16T18:30:13.919 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B8cEVEB54u54sA1wrGRunyohaqI= 2024-04-16T18:30:13.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B8cEVEB54u54sA1wrGRunyohaqI=', 'Content-Length': '0'} 2024-04-16T18:30:13.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016502ffe7bbd739a-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.925 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.925 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.925 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.925 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.925 DEBUG:boto:Method: GET 2024-04-16T18:30:13.925 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:30:13.925 DEBUG:boto:Data: 2024-04-16T18:30:13.925 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.926 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.926 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.926 DEBUG:boto:Params: {} 2024-04-16T18:30:13.926 DEBUG:boto:Token: None 2024-04-16T18:30:13.926 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.926 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b735a033e67db425-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.930 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.930 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:13.930 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:13.931 DEBUG:boto:path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.931 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj116 2024-04-16T18:30:13.931 DEBUG:boto:Method: GET 2024-04-16T18:30:13.931 DEBUG:boto:Path: /lybtug-52/?prefix=obj116 2024-04-16T18:30:13.931 DEBUG:boto:Data: 2024-04-16T18:30:13.931 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.931 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.931 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.931 DEBUG:boto:Params: {} 2024-04-16T18:30:13.931 DEBUG:boto:Token: None 2024-04-16T18:30:13.932 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/ 2024-04-16T18:30:13.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs= 2024-04-16T18:30:13.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cVuqdBG/1bk+NRY3m1+e1hHoHWs=', 'Content-Length': '0'} 2024-04-16T18:30:13.938 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075d5adc7e7b354ec-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.978 DEBUG:boto:b'lybtug-52obj1161000falseobj1162024-04-16T18:25:26.853Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:13.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:30:13.979 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:13.979 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:13.979 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.979 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.979 DEBUG:boto:Method: GET 2024-04-16T18:30:13.979 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.979 DEBUG:boto:Data: 2024-04-16T18:30:13.979 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.979 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.980 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.980 DEBUG:boto:Params: {} 2024-04-16T18:30:13.980 DEBUG:boto:Token: None 2024-04-16T18:30:13.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oi0bcLi4ZuR7TNZvFEOi+CHybQU= 2024-04-16T18:30:13.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oi0bcLi4ZuR7TNZvFEOi+CHybQU=', 'Content-Length': '0'} 2024-04-16T18:30:13.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f42efc94a0fdba6-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.984 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:13.984 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:13.984 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.984 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.984 DEBUG:boto:Method: GET 2024-04-16T18:30:13.984 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.984 DEBUG:boto:Data: 2024-04-16T18:30:13.984 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.984 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.984 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.984 DEBUG:boto:Params: {} 2024-04-16T18:30:13.984 DEBUG:boto:Token: None 2024-04-16T18:30:13.985 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.985 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oi0bcLi4ZuR7TNZvFEOi+CHybQU= 2024-04-16T18:30:13.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oi0bcLi4ZuR7TNZvFEOi+CHybQU=', 'Content-Length': '0'} 2024-04-16T18:30:13.988 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d631f8f9fcdd5276-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.988 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:13.988 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:13.988 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.988 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.988 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.988 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.989 DEBUG:boto:Data: 2024-04-16T18:30:13.989 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.989 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.989 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.989 DEBUG:boto:Params: {} 2024-04-16T18:30:13.989 DEBUG:boto:Token: None 2024-04-16T18:30:13.989 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6LKp5XXS1nYLLC/BAnhICPqs/ig= 2024-04-16T18:30:13.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6LKp5XXS1nYLLC/BAnhICPqs/ig=', 'Content-Length': '0'} 2024-04-16T18:30:13.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000474184ee9ff7c820-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.992 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:13.992 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:13.992 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.992 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.992 DEBUG:boto:Method: HEAD 2024-04-16T18:30:13.993 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.993 DEBUG:boto:Data: 2024-04-16T18:30:13.993 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.993 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:13.993 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.993 DEBUG:boto:Params: {} 2024-04-16T18:30:13.993 DEBUG:boto:Token: None 2024-04-16T18:30:13.993 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.993 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6LKp5XXS1nYLLC/BAnhICPqs/ig= 2024-04-16T18:30:13.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6LKp5XXS1nYLLC/BAnhICPqs/ig=', 'Content-Length': '0'} 2024-04-16T18:30:13.996 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d9ee4805edd3545-00661ec3b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:13.996 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:30:13.996 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:13.996 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:13.997 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.997 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.997 DEBUG:boto:Method: GET 2024-04-16T18:30:13.997 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.997 DEBUG:boto:Data: 2024-04-16T18:30:13.997 DEBUG:boto:Headers: {} 2024-04-16T18:30:13.997 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:13.997 DEBUG:boto:Port: 8000 2024-04-16T18:30:13.997 DEBUG:boto:Params: {} 2024-04-16T18:30:13.997 DEBUG:boto:Token: None 2024-04-16T18:30:13.997 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:13 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:30:13.998 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Oi0bcLi4ZuR7TNZvFEOi+CHybQU= 2024-04-16T18:30:13.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:13 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oi0bcLi4ZuR7TNZvFEOi+CHybQU=', 'Content-Length': '0'} 2024-04-16T18:30:14.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000237bb2278dec5d9c-00661ec3b5-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:13 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.042 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:14.042 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:14.043 DEBUG:boto:path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:14.043 DEBUG:boto:auth_path=/lybtug-52/obj124?versionId=null 2024-04-16T18:30:14.043 DEBUG:boto:Method: GET 2024-04-16T18:30:14.043 DEBUG:boto:Path: /lybtug-52/obj124?versionId=null 2024-04-16T18:30:14.043 DEBUG:boto:Data: 2024-04-16T18:30:14.043 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.043 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.043 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.043 DEBUG:boto:Params: {} 2024-04-16T18:30:14.043 DEBUG:boto:Token: None 2024-04-16T18:30:14.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj124?versionId=null 2024-04-16T18:30:14.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:odOYLtoTl/t+SWN6XXHoojKuFcg= 2024-04-16T18:30:14.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:odOYLtoTl/t+SWN6XXHoojKuFcg=', 'Content-Length': '0'} 2024-04-16T18:30:14.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000198eca6b74dbe170-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.047 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.047 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.047 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.047 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.047 DEBUG:boto:Method: GET 2024-04-16T18:30:14.047 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:30:14.047 DEBUG:boto:Data: 2024-04-16T18:30:14.047 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.047 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.047 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.047 DEBUG:boto:Params: {} 2024-04-16T18:30:14.048 DEBUG:boto:Token: None 2024-04-16T18:30:14.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d6eebaa5ca8852f0-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.052 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.052 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.052 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.052 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.052 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.052 DEBUG:boto:Method: GET 2024-04-16T18:30:14.052 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:30:14.052 DEBUG:boto:Data: 2024-04-16T18:30:14.052 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.053 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.053 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.053 DEBUG:boto:Params: {} 2024-04-16T18:30:14.053 DEBUG:boto:Token: None 2024-04-16T18:30:14.053 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.053 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.060 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000265650fd40658ac0-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.106 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.107 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:14.107 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:14.107 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.107 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:30:14.107 DEBUG:boto:Data: 2024-04-16T18:30:14.107 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.107 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.107 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.107 DEBUG:boto:Params: {} 2024-04-16T18:30:14.107 DEBUG:boto:Token: None 2024-04-16T18:30:14.108 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj124 2024-04-16T18:30:14.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/7HjloJAX6WFQmn9M/3uzFMxbx4= 2024-04-16T18:30:14.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/7HjloJAX6WFQmn9M/3uzFMxbx4=', 'Content-Length': '0'} 2024-04-16T18:30:14.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e061c90a6627f32e-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.111 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:14.111 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:14.111 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.111 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:30:14.111 DEBUG:boto:Data: 2024-04-16T18:30:14.111 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.112 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.112 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.112 DEBUG:boto:Params: {} 2024-04-16T18:30:14.112 DEBUG:boto:Token: None 2024-04-16T18:30:14.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj124 2024-04-16T18:30:14.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/7HjloJAX6WFQmn9M/3uzFMxbx4= 2024-04-16T18:30:14.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/7HjloJAX6WFQmn9M/3uzFMxbx4=', 'Content-Length': '0'} 2024-04-16T18:30:14.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000430e7c5ccbd4939d-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.115 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2024-04-16T18:30:14.115 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:14.115 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:14.116 DEBUG:boto:Method: GET 2024-04-16T18:30:14.116 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:30:14.116 DEBUG:boto:Data: 2024-04-16T18:30:14.116 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.116 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.116 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.116 DEBUG:boto:Params: {} 2024-04-16T18:30:14.116 DEBUG:boto:Token: None 2024-04-16T18:30:14.116 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj124 2024-04-16T18:30:14.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c/i19DY44dqib8YqlKnvpWsuvg8= 2024-04-16T18:30:14.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c/i19DY44dqib8YqlKnvpWsuvg8=', 'Content-Length': '0'} 2024-04-16T18:30:14.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cabeb31730c7ea19-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.120 DEBUG:boto:path=/lybtug-52/obj124 2024-04-16T18:30:14.120 DEBUG:boto:auth_path=/lybtug-52/obj124 2024-04-16T18:30:14.120 DEBUG:boto:Method: GET 2024-04-16T18:30:14.120 DEBUG:boto:Path: /lybtug-52/obj124 2024-04-16T18:30:14.120 DEBUG:boto:Data: 2024-04-16T18:30:14.120 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.120 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.120 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.121 DEBUG:boto:Params: {} 2024-04-16T18:30:14.121 DEBUG:boto:Token: None 2024-04-16T18:30:14.121 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj124 2024-04-16T18:30:14.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:c/i19DY44dqib8YqlKnvpWsuvg8= 2024-04-16T18:30:14.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:c/i19DY44dqib8YqlKnvpWsuvg8=', 'Content-Length': '0'} 2024-04-16T18:30:14.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025e96aa2670a6302-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.124 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.124 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.125 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.125 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.125 DEBUG:boto:Method: GET 2024-04-16T18:30:14.125 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:30:14.125 DEBUG:boto:Data: 2024-04-16T18:30:14.125 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.125 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.125 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.125 DEBUG:boto:Params: {} 2024-04-16T18:30:14.125 DEBUG:boto:Token: None 2024-04-16T18:30:14.126 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.126 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b7b0ae708271243-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.170 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.171 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.171 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.171 DEBUG:boto:path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.171 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj124 2024-04-16T18:30:14.171 DEBUG:boto:Method: GET 2024-04-16T18:30:14.171 DEBUG:boto:Path: /lybtug-52/?prefix=obj124 2024-04-16T18:30:14.171 DEBUG:boto:Data: 2024-04-16T18:30:14.171 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.171 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.171 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.172 DEBUG:boto:Params: {} 2024-04-16T18:30:14.172 DEBUG:boto:Token: None 2024-04-16T18:30:14.172 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.172 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.178 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006aa1a5420a4c40b2-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.178 DEBUG:boto:b'lybtug-52obj1241000falseobj1242024-04-16T18:26:49.340Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.179 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:30:14.179 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.179 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.179 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.179 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.179 DEBUG:boto:Method: GET 2024-04-16T18:30:14.179 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.179 DEBUG:boto:Data: 2024-04-16T18:30:14.179 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.180 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.180 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.180 DEBUG:boto:Params: {} 2024-04-16T18:30:14.180 DEBUG:boto:Token: None 2024-04-16T18:30:14.180 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.180 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos= 2024-04-16T18:30:14.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos=', 'Content-Length': '0'} 2024-04-16T18:30:14.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1f54ca277221233-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.184 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.184 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.184 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.184 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.184 DEBUG:boto:Method: GET 2024-04-16T18:30:14.184 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.185 DEBUG:boto:Data: 2024-04-16T18:30:14.185 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.185 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.185 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.185 DEBUG:boto:Params: {} 2024-04-16T18:30:14.185 DEBUG:boto:Token: None 2024-04-16T18:30:14.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos= 2024-04-16T18:30:14.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos=', 'Content-Length': '0'} 2024-04-16T18:30:14.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000040c027cd0beb349-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.230 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.231 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.231 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.231 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.231 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.231 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.231 DEBUG:boto:Data: 2024-04-16T18:30:14.231 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.231 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.231 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.231 DEBUG:boto:Params: {} 2024-04-16T18:30:14.232 DEBUG:boto:Token: None 2024-04-16T18:30:14.232 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:79bGA6WYTGNSkjq0DOPbC29lIqs= 2024-04-16T18:30:14.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:79bGA6WYTGNSkjq0DOPbC29lIqs=', 'Content-Length': '0'} 2024-04-16T18:30:14.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d64b4cabfa039a04-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.235 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.235 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.235 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.235 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.235 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.235 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.235 DEBUG:boto:Data: 2024-04-16T18:30:14.235 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.236 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.236 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.236 DEBUG:boto:Params: {} 2024-04-16T18:30:14.236 DEBUG:boto:Token: None 2024-04-16T18:30:14.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:79bGA6WYTGNSkjq0DOPbC29lIqs= 2024-04-16T18:30:14.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:79bGA6WYTGNSkjq0DOPbC29lIqs=', 'Content-Length': '0'} 2024-04-16T18:30:14.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee30cce9d2fac019-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:30:14.239 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.240 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.240 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.240 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.240 DEBUG:boto:Method: GET 2024-04-16T18:30:14.240 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.240 DEBUG:boto:Data: 2024-04-16T18:30:14.240 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.240 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.240 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.240 DEBUG:boto:Params: {} 2024-04-16T18:30:14.240 DEBUG:boto:Token: None 2024-04-16T18:30:14.240 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos= 2024-04-16T18:30:14.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos=', 'Content-Length': '0'} 2024-04-16T18:30:14.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a2ea1fe8e60bda1-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.244 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.244 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.244 DEBUG:boto:path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.244 DEBUG:boto:auth_path=/lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.244 DEBUG:boto:Method: GET 2024-04-16T18:30:14.244 DEBUG:boto:Path: /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.244 DEBUG:boto:Data: 2024-04-16T18:30:14.244 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.244 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.244 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.244 DEBUG:boto:Params: {} 2024-04-16T18:30:14.245 DEBUG:boto:Token: None 2024-04-16T18:30:14.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145?versionId=null 2024-04-16T18:30:14.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos= 2024-04-16T18:30:14.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:25nCYp05QzJ/Pr/C7yfssffFpos=', 'Content-Length': '0'} 2024-04-16T18:30:14.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058cb6f7fc4de1cad-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.248 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.248 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.248 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.248 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.248 DEBUG:boto:Method: GET 2024-04-16T18:30:14.248 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:30:14.248 DEBUG:boto:Data: 2024-04-16T18:30:14.249 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.249 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.249 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.249 DEBUG:boto:Params: {} 2024-04-16T18:30:14.249 DEBUG:boto:Token: None 2024-04-16T18:30:14.249 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.249 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.253 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009395d25053f05fb2-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.294 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.295 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.295 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.295 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.295 DEBUG:boto:Method: GET 2024-04-16T18:30:14.295 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:30:14.295 DEBUG:boto:Data: 2024-04-16T18:30:14.295 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.295 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.295 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.296 DEBUG:boto:Params: {} 2024-04-16T18:30:14.296 DEBUG:boto:Token: None 2024-04-16T18:30:14.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.302 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d6f9f03d8a325746-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.302 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.303 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.303 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.303 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.303 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:30:14.303 DEBUG:boto:Data: 2024-04-16T18:30:14.303 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.303 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.303 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.303 DEBUG:boto:Params: {} 2024-04-16T18:30:14.303 DEBUG:boto:Token: None 2024-04-16T18:30:14.304 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145 2024-04-16T18:30:14.304 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bbHhSUoxB4PMJDTI4YbkObHYfGw= 2024-04-16T18:30:14.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bbHhSUoxB4PMJDTI4YbkObHYfGw=', 'Content-Length': '0'} 2024-04-16T18:30:14.306 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cd48bdc1903a90e-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.307 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.307 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.307 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.307 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:30:14.307 DEBUG:boto:Data: 2024-04-16T18:30:14.307 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.307 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.307 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.307 DEBUG:boto:Params: {} 2024-04-16T18:30:14.308 DEBUG:boto:Token: None 2024-04-16T18:30:14.308 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145 2024-04-16T18:30:14.308 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bbHhSUoxB4PMJDTI4YbkObHYfGw= 2024-04-16T18:30:14.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bbHhSUoxB4PMJDTI4YbkObHYfGw=', 'Content-Length': '0'} 2024-04-16T18:30:14.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c661f36377dfa13d-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.311 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2024-04-16T18:30:14.311 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.311 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.311 DEBUG:boto:Method: GET 2024-04-16T18:30:14.311 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:30:14.311 DEBUG:boto:Data: 2024-04-16T18:30:14.311 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.312 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.312 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.312 DEBUG:boto:Params: {} 2024-04-16T18:30:14.312 DEBUG:boto:Token: None 2024-04-16T18:30:14.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145 2024-04-16T18:30:14.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LwoMZp04MHFmyieJGc3jOn99x+U= 2024-04-16T18:30:14.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LwoMZp04MHFmyieJGc3jOn99x+U=', 'Content-Length': '0'} 2024-04-16T18:30:14.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee18ce1504a4176e-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.315 DEBUG:boto:path=/lybtug-52/obj145 2024-04-16T18:30:14.315 DEBUG:boto:auth_path=/lybtug-52/obj145 2024-04-16T18:30:14.315 DEBUG:boto:Method: GET 2024-04-16T18:30:14.315 DEBUG:boto:Path: /lybtug-52/obj145 2024-04-16T18:30:14.315 DEBUG:boto:Data: 2024-04-16T18:30:14.316 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.316 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.316 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.316 DEBUG:boto:Params: {} 2024-04-16T18:30:14.316 DEBUG:boto:Token: None 2024-04-16T18:30:14.316 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj145 2024-04-16T18:30:14.316 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LwoMZp04MHFmyieJGc3jOn99x+U= 2024-04-16T18:30:14.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LwoMZp04MHFmyieJGc3jOn99x+U=', 'Content-Length': '0'} 2024-04-16T18:30:14.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2a2e44dddec5ae9-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.362 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.362 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.363 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.363 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.363 DEBUG:boto:Method: GET 2024-04-16T18:30:14.363 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:30:14.363 DEBUG:boto:Data: 2024-04-16T18:30:14.363 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.363 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.363 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.363 DEBUG:boto:Params: {} 2024-04-16T18:30:14.363 DEBUG:boto:Token: None 2024-04-16T18:30:14.364 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.367 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed972b0108b7b721-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.368 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.368 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.368 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.368 DEBUG:boto:path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.368 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj145 2024-04-16T18:30:14.368 DEBUG:boto:Method: GET 2024-04-16T18:30:14.368 DEBUG:boto:Path: /lybtug-52/?prefix=obj145 2024-04-16T18:30:14.369 DEBUG:boto:Data: 2024-04-16T18:30:14.369 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.369 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.369 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.369 DEBUG:boto:Params: {} 2024-04-16T18:30:14.369 DEBUG:boto:Token: None 2024-04-16T18:30:14.369 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002437a880f35160bc-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.376 DEBUG:boto:b'lybtug-52obj1451000falseobj1452024-04-16T18:25:26.904Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.377 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2024-04-16T18:30:14.377 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.377 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.377 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.377 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.377 DEBUG:boto:Method: GET 2024-04-16T18:30:14.377 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.377 DEBUG:boto:Data: 2024-04-16T18:30:14.377 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.378 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.378 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.378 DEBUG:boto:Params: {} 2024-04-16T18:30:14.378 DEBUG:boto:Token: None 2024-04-16T18:30:14.378 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.378 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M= 2024-04-16T18:30:14.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M=', 'Content-Length': '0'} 2024-04-16T18:30:14.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003df9b486b19006e4-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.422 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.422 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.423 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.423 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.423 DEBUG:boto:Method: GET 2024-04-16T18:30:14.423 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.423 DEBUG:boto:Data: 2024-04-16T18:30:14.423 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.423 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.423 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.423 DEBUG:boto:Params: {} 2024-04-16T18:30:14.423 DEBUG:boto:Token: None 2024-04-16T18:30:14.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M= 2024-04-16T18:30:14.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M=', 'Content-Length': '0'} 2024-04-16T18:30:14.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d01bf5dcf6e88ed6-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.427 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.427 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.427 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.427 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.427 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.428 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.428 DEBUG:boto:Data: 2024-04-16T18:30:14.428 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.428 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.428 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.428 DEBUG:boto:Params: {} 2024-04-16T18:30:14.428 DEBUG:boto:Token: None 2024-04-16T18:30:14.428 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LCxPV25ywP8WM3Ud2Tfsp4zmVh8= 2024-04-16T18:30:14.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LCxPV25ywP8WM3Ud2Tfsp4zmVh8=', 'Content-Length': '0'} 2024-04-16T18:30:14.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042214b2f271a1303-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.432 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.432 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.432 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.432 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.432 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.432 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.432 DEBUG:boto:Data: 2024-04-16T18:30:14.432 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.433 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.433 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.433 DEBUG:boto:Params: {} 2024-04-16T18:30:14.433 DEBUG:boto:Token: None 2024-04-16T18:30:14.433 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.433 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LCxPV25ywP8WM3Ud2Tfsp4zmVh8= 2024-04-16T18:30:14.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LCxPV25ywP8WM3Ud2Tfsp4zmVh8=', 'Content-Length': '0'} 2024-04-16T18:30:14.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009887336e4668d170-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.436 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2024-04-16T18:30:14.436 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.436 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.436 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.436 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.437 DEBUG:boto:Method: GET 2024-04-16T18:30:14.437 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.437 DEBUG:boto:Data: 2024-04-16T18:30:14.437 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.437 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.437 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.437 DEBUG:boto:Params: {} 2024-04-16T18:30:14.437 DEBUG:boto:Token: None 2024-04-16T18:30:14.437 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M= 2024-04-16T18:30:14.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M=', 'Content-Length': '0'} 2024-04-16T18:30:14.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020e086e465ee4eec-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.441 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.441 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.441 DEBUG:boto:path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.441 DEBUG:boto:auth_path=/lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.441 DEBUG:boto:Method: GET 2024-04-16T18:30:14.441 DEBUG:boto:Path: /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.441 DEBUG:boto:Data: 2024-04-16T18:30:14.441 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.441 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.441 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.442 DEBUG:boto:Params: {} 2024-04-16T18:30:14.442 DEBUG:boto:Token: None 2024-04-16T18:30:14.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148?versionId=null 2024-04-16T18:30:14.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M= 2024-04-16T18:30:14.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v/a3oyQGXpJompfXCYBEN1iwe5M=', 'Content-Length': '0'} 2024-04-16T18:30:14.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e57a20aa54082ca8-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.486 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.486 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.487 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.487 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.487 DEBUG:boto:Method: GET 2024-04-16T18:30:14.487 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:30:14.487 DEBUG:boto:Data: 2024-04-16T18:30:14.487 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.487 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.487 DEBUG:boto:Params: {} 2024-04-16T18:30:14.487 DEBUG:boto:Token: None 2024-04-16T18:30:14.488 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d150c7865776e792-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.492 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.492 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.492 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.492 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.492 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.492 DEBUG:boto:Method: GET 2024-04-16T18:30:14.493 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:30:14.493 DEBUG:boto:Data: 2024-04-16T18:30:14.493 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.493 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.493 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.493 DEBUG:boto:Params: {} 2024-04-16T18:30:14.493 DEBUG:boto:Token: None 2024-04-16T18:30:14.493 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.493 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.500 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004212c47eb3fad26f-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.500 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.500 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.500 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.501 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.501 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:30:14.501 DEBUG:boto:Data: 2024-04-16T18:30:14.501 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.501 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.501 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.501 DEBUG:boto:Params: {} 2024-04-16T18:30:14.501 DEBUG:boto:Token: None 2024-04-16T18:30:14.501 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148 2024-04-16T18:30:14.502 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dia0QDBTgSSJeKNUZdGG/Oy6NmA= 2024-04-16T18:30:14.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dia0QDBTgSSJeKNUZdGG/Oy6NmA=', 'Content-Length': '0'} 2024-04-16T18:30:14.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f58904d6173cde4-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.504 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.505 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.505 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.505 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:30:14.505 DEBUG:boto:Data: 2024-04-16T18:30:14.505 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.505 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.505 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.505 DEBUG:boto:Params: {} 2024-04-16T18:30:14.505 DEBUG:boto:Token: None 2024-04-16T18:30:14.506 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148 2024-04-16T18:30:14.506 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Dia0QDBTgSSJeKNUZdGG/Oy6NmA= 2024-04-16T18:30:14.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Dia0QDBTgSSJeKNUZdGG/Oy6NmA=', 'Content-Length': '0'} 2024-04-16T18:30:14.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087d259c866c1e451-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.509 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2024-04-16T18:30:14.509 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.509 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.509 DEBUG:boto:Method: GET 2024-04-16T18:30:14.509 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:30:14.509 DEBUG:boto:Data: 2024-04-16T18:30:14.509 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.509 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.509 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.509 DEBUG:boto:Params: {} 2024-04-16T18:30:14.510 DEBUG:boto:Token: None 2024-04-16T18:30:14.510 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148 2024-04-16T18:30:14.510 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/YL9yfiMA0Vpch15Z4hX+IFI6s= 2024-04-16T18:30:14.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/YL9yfiMA0Vpch15Z4hX+IFI6s=', 'Content-Length': '0'} 2024-04-16T18:30:14.513 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2da690ef6e8d693-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.554 DEBUG:boto:path=/lybtug-52/obj148 2024-04-16T18:30:14.554 DEBUG:boto:auth_path=/lybtug-52/obj148 2024-04-16T18:30:14.555 DEBUG:boto:Method: GET 2024-04-16T18:30:14.555 DEBUG:boto:Path: /lybtug-52/obj148 2024-04-16T18:30:14.555 DEBUG:boto:Data: 2024-04-16T18:30:14.555 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.555 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.555 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.555 DEBUG:boto:Params: {} 2024-04-16T18:30:14.555 DEBUG:boto:Token: None 2024-04-16T18:30:14.555 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj148 2024-04-16T18:30:14.555 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:P/YL9yfiMA0Vpch15Z4hX+IFI6s= 2024-04-16T18:30:14.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:P/YL9yfiMA0Vpch15Z4hX+IFI6s=', 'Content-Length': '0'} 2024-04-16T18:30:14.558 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc0e379be294594d-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.559 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.559 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.559 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.559 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.559 DEBUG:boto:Method: GET 2024-04-16T18:30:14.559 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:30:14.559 DEBUG:boto:Data: 2024-04-16T18:30:14.559 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.559 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.560 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.560 DEBUG:boto:Params: {} 2024-04-16T18:30:14.560 DEBUG:boto:Token: None 2024-04-16T18:30:14.560 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.560 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069fcfca7986c73e0-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.564 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.565 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.565 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.565 DEBUG:boto:path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.565 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj148 2024-04-16T18:30:14.565 DEBUG:boto:Method: GET 2024-04-16T18:30:14.565 DEBUG:boto:Path: /lybtug-52/?prefix=obj148 2024-04-16T18:30:14.565 DEBUG:boto:Data: 2024-04-16T18:30:14.565 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.565 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.565 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.566 DEBUG:boto:Params: {} 2024-04-16T18:30:14.566 DEBUG:boto:Token: None 2024-04-16T18:30:14.566 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.566 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.573 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002a1513c4f3373b0-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.614 DEBUG:boto:b'lybtug-52obj1481000falseobj1482024-04-16T18:27:53.397Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.615 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:30:14.615 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.615 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.615 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.615 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.615 DEBUG:boto:Method: GET 2024-04-16T18:30:14.615 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.615 DEBUG:boto:Data: 2024-04-16T18:30:14.615 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.615 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.615 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.616 DEBUG:boto:Params: {} 2024-04-16T18:30:14.616 DEBUG:boto:Token: None 2024-04-16T18:30:14.616 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo= 2024-04-16T18:30:14.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo=', 'Content-Length': '0'} 2024-04-16T18:30:14.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a639027d38c366d2-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.620 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.620 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.620 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.620 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.620 DEBUG:boto:Method: GET 2024-04-16T18:30:14.620 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.620 DEBUG:boto:Data: 2024-04-16T18:30:14.621 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.621 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.621 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.621 DEBUG:boto:Params: {} 2024-04-16T18:30:14.621 DEBUG:boto:Token: None 2024-04-16T18:30:14.621 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.621 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo= 2024-04-16T18:30:14.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo=', 'Content-Length': '0'} 2024-04-16T18:30:14.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000608b83474c82f2cb-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.624 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.625 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.625 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.625 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.625 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.625 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.625 DEBUG:boto:Data: 2024-04-16T18:30:14.625 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.625 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.625 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.625 DEBUG:boto:Params: {} 2024-04-16T18:30:14.626 DEBUG:boto:Token: None 2024-04-16T18:30:14.626 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.626 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ozR8w26940qskFoRwIovKr5lj9U= 2024-04-16T18:30:14.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ozR8w26940qskFoRwIovKr5lj9U=', 'Content-Length': '0'} 2024-04-16T18:30:14.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4138785ec013293-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.629 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.629 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.629 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.630 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.630 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.630 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.630 DEBUG:boto:Data: 2024-04-16T18:30:14.630 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.630 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.630 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.630 DEBUG:boto:Params: {} 2024-04-16T18:30:14.630 DEBUG:boto:Token: None 2024-04-16T18:30:14.630 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.631 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ozR8w26940qskFoRwIovKr5lj9U= 2024-04-16T18:30:14.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ozR8w26940qskFoRwIovKr5lj9U=', 'Content-Length': '0'} 2024-04-16T18:30:14.633 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c965730b215b353-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.634 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:30:14.634 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.634 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.634 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.634 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.634 DEBUG:boto:Method: GET 2024-04-16T18:30:14.634 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.634 DEBUG:boto:Data: 2024-04-16T18:30:14.634 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.635 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.635 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.635 DEBUG:boto:Params: {} 2024-04-16T18:30:14.635 DEBUG:boto:Token: None 2024-04-16T18:30:14.635 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.635 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo= 2024-04-16T18:30:14.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo=', 'Content-Length': '0'} 2024-04-16T18:30:14.638 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b9c1c21dd86ec64-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.638 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.639 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.639 DEBUG:boto:path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.639 DEBUG:boto:auth_path=/lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.639 DEBUG:boto:Method: GET 2024-04-16T18:30:14.639 DEBUG:boto:Path: /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.639 DEBUG:boto:Data: 2024-04-16T18:30:14.639 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.639 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.639 DEBUG:boto:Params: {} 2024-04-16T18:30:14.639 DEBUG:boto:Token: None 2024-04-16T18:30:14.640 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155?versionId=null 2024-04-16T18:30:14.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo= 2024-04-16T18:30:14.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+MGS7unCc5RTffc67AmeSIn0qQo=', 'Content-Length': '0'} 2024-04-16T18:30:14.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003db3b3c8caf295b0-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.643 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.643 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.643 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.643 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.644 DEBUG:boto:Method: GET 2024-04-16T18:30:14.644 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:30:14.644 DEBUG:boto:Data: 2024-04-16T18:30:14.644 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.644 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.644 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.644 DEBUG:boto:Params: {} 2024-04-16T18:30:14.644 DEBUG:boto:Token: None 2024-04-16T18:30:14.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.648 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008961601a0a556df9-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.690 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.691 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.691 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.691 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.691 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.691 DEBUG:boto:Method: GET 2024-04-16T18:30:14.691 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:30:14.691 DEBUG:boto:Data: 2024-04-16T18:30:14.691 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.692 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.692 DEBUG:boto:Params: {} 2024-04-16T18:30:14.692 DEBUG:boto:Token: None 2024-04-16T18:30:14.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.698 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078aee137dfce8638-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.699 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.699 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.699 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.699 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.699 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:30:14.699 DEBUG:boto:Data: 2024-04-16T18:30:14.699 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.699 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.700 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.700 DEBUG:boto:Params: {} 2024-04-16T18:30:14.700 DEBUG:boto:Token: None 2024-04-16T18:30:14.700 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155 2024-04-16T18:30:14.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wUtikL2VeuFIW7A+Ddmtkpxg5Vo= 2024-04-16T18:30:14.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wUtikL2VeuFIW7A+Ddmtkpxg5Vo=', 'Content-Length': '0'} 2024-04-16T18:30:14.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b37e70d79661690-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.704 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.704 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.704 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.704 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:30:14.704 DEBUG:boto:Data: 2024-04-16T18:30:14.704 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.705 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.705 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.705 DEBUG:boto:Params: {} 2024-04-16T18:30:14.705 DEBUG:boto:Token: None 2024-04-16T18:30:14.705 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155 2024-04-16T18:30:14.705 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:wUtikL2VeuFIW7A+Ddmtkpxg5Vo= 2024-04-16T18:30:14.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wUtikL2VeuFIW7A+Ddmtkpxg5Vo=', 'Content-Length': '0'} 2024-04-16T18:30:14.709 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e81c0be7723f5fa-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.709 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2024-04-16T18:30:14.709 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.709 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.710 DEBUG:boto:Method: GET 2024-04-16T18:30:14.710 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:30:14.710 DEBUG:boto:Data: 2024-04-16T18:30:14.710 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.710 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.710 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.711 DEBUG:boto:Params: {} 2024-04-16T18:30:14.711 DEBUG:boto:Token: None 2024-04-16T18:30:14.711 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155 2024-04-16T18:30:14.711 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2JTm3pvcxvc8CIVcASkpfU8Jq/w= 2024-04-16T18:30:14.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2JTm3pvcxvc8CIVcASkpfU8Jq/w=', 'Content-Length': '0'} 2024-04-16T18:30:14.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027d3f7297ac05aa5-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.715 DEBUG:boto:path=/lybtug-52/obj155 2024-04-16T18:30:14.715 DEBUG:boto:auth_path=/lybtug-52/obj155 2024-04-16T18:30:14.715 DEBUG:boto:Method: GET 2024-04-16T18:30:14.715 DEBUG:boto:Path: /lybtug-52/obj155 2024-04-16T18:30:14.716 DEBUG:boto:Data: 2024-04-16T18:30:14.716 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.716 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.716 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.716 DEBUG:boto:Params: {} 2024-04-16T18:30:14.716 DEBUG:boto:Token: None 2024-04-16T18:30:14.717 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj155 2024-04-16T18:30:14.717 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2JTm3pvcxvc8CIVcASkpfU8Jq/w= 2024-04-16T18:30:14.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2JTm3pvcxvc8CIVcASkpfU8Jq/w=', 'Content-Length': '0'} 2024-04-16T18:30:14.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000679714ccc1525b1c-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.762 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.763 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.763 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.763 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.763 DEBUG:boto:Method: GET 2024-04-16T18:30:14.763 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:30:14.763 DEBUG:boto:Data: 2024-04-16T18:30:14.763 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.764 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.764 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.764 DEBUG:boto:Params: {} 2024-04-16T18:30:14.764 DEBUG:boto:Token: None 2024-04-16T18:30:14.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.765 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046734c105d417b7b-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.769 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.769 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.769 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.769 DEBUG:boto:path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.769 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj155 2024-04-16T18:30:14.769 DEBUG:boto:Method: GET 2024-04-16T18:30:14.770 DEBUG:boto:Path: /lybtug-52/?prefix=obj155 2024-04-16T18:30:14.770 DEBUG:boto:Data: 2024-04-16T18:30:14.770 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.770 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.770 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.770 DEBUG:boto:Params: {} 2024-04-16T18:30:14.770 DEBUG:boto:Token: None 2024-04-16T18:30:14.770 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.770 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.777 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f110dead50d6f00f-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.777 DEBUG:boto:b'lybtug-52obj1551000falseobj1552024-04-16T18:26:49.392Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.777 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:30:14.777 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.778 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.778 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.778 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.778 DEBUG:boto:Method: GET 2024-04-16T18:30:14.778 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.778 DEBUG:boto:Data: 2024-04-16T18:30:14.778 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.778 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.778 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.778 DEBUG:boto:Params: {} 2024-04-16T18:30:14.779 DEBUG:boto:Token: None 2024-04-16T18:30:14.779 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.779 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM= 2024-04-16T18:30:14.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM=', 'Content-Length': '0'} 2024-04-16T18:30:14.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf2a603891801ed1-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.826 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.826 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.827 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.827 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.827 DEBUG:boto:Method: GET 2024-04-16T18:30:14.827 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.827 DEBUG:boto:Data: 2024-04-16T18:30:14.827 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.827 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.827 DEBUG:boto:Params: {} 2024-04-16T18:30:14.827 DEBUG:boto:Token: None 2024-04-16T18:30:14.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.828 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM= 2024-04-16T18:30:14.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM=', 'Content-Length': '0'} 2024-04-16T18:30:14.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce5df863bd481b6d-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.831 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.831 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.831 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.831 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.832 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.832 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.832 DEBUG:boto:Data: 2024-04-16T18:30:14.832 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.832 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.832 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.832 DEBUG:boto:Params: {} 2024-04-16T18:30:14.832 DEBUG:boto:Token: None 2024-04-16T18:30:14.832 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bQEBMWfLyZw16SK1y0HbrTI2P/M= 2024-04-16T18:30:14.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bQEBMWfLyZw16SK1y0HbrTI2P/M=', 'Content-Length': '0'} 2024-04-16T18:30:14.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c5dda9339fd1512-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.836 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.836 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.836 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.836 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.836 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.836 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.836 DEBUG:boto:Data: 2024-04-16T18:30:14.837 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.837 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.837 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.837 DEBUG:boto:Params: {} 2024-04-16T18:30:14.837 DEBUG:boto:Token: None 2024-04-16T18:30:14.837 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.837 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bQEBMWfLyZw16SK1y0HbrTI2P/M= 2024-04-16T18:30:14.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bQEBMWfLyZw16SK1y0HbrTI2P/M=', 'Content-Length': '0'} 2024-04-16T18:30:14.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b271edc4e19d4e2c-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.841 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:30:14.841 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.841 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.841 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.841 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.841 DEBUG:boto:Method: GET 2024-04-16T18:30:14.841 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.841 DEBUG:boto:Data: 2024-04-16T18:30:14.841 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.842 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.842 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.842 DEBUG:boto:Params: {} 2024-04-16T18:30:14.842 DEBUG:boto:Token: None 2024-04-16T18:30:14.842 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.842 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM= 2024-04-16T18:30:14.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM=', 'Content-Length': '0'} 2024-04-16T18:30:14.845 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d861b8f68e02efa-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.846 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.846 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.846 DEBUG:boto:path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.846 DEBUG:boto:auth_path=/lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.846 DEBUG:boto:Method: GET 2024-04-16T18:30:14.846 DEBUG:boto:Path: /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.846 DEBUG:boto:Data: 2024-04-16T18:30:14.847 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.847 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.847 DEBUG:boto:Params: {} 2024-04-16T18:30:14.847 DEBUG:boto:Token: None 2024-04-16T18:30:14.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17?versionId=null 2024-04-16T18:30:14.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM= 2024-04-16T18:30:14.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BofHTfeXuekbBdac6Hy27/IYiyM=', 'Content-Length': '0'} 2024-04-16T18:30:14.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2e6490a2df4934b-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.894 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.894 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.895 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.895 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.895 DEBUG:boto:Method: GET 2024-04-16T18:30:14.895 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:30:14.895 DEBUG:boto:Data: 2024-04-16T18:30:14.895 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.895 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.895 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.895 DEBUG:boto:Params: {} 2024-04-16T18:30:14.895 DEBUG:boto:Token: None 2024-04-16T18:30:14.896 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.896 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096386e914ce6d030-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.900 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.900 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.900 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.900 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.900 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.900 DEBUG:boto:Method: GET 2024-04-16T18:30:14.901 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:30:14.901 DEBUG:boto:Data: 2024-04-16T18:30:14.901 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.901 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.901 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.901 DEBUG:boto:Params: {} 2024-04-16T18:30:14.901 DEBUG:boto:Token: None 2024-04-16T18:30:14.901 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.901 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.907 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc2e418530e9615e-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.908 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.908 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.908 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.908 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.908 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:30:14.908 DEBUG:boto:Data: 2024-04-16T18:30:14.908 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.908 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.909 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.909 DEBUG:boto:Params: {} 2024-04-16T18:30:14.909 DEBUG:boto:Token: None 2024-04-16T18:30:14.909 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17 2024-04-16T18:30:14.909 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PTty2ULUqEiv4ebk5EGhptOhxd4= 2024-04-16T18:30:14.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PTty2ULUqEiv4ebk5EGhptOhxd4=', 'Content-Length': '0'} 2024-04-16T18:30:14.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000711c691b3c0130b7-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.912 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.912 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.913 DEBUG:boto:Method: HEAD 2024-04-16T18:30:14.913 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:30:14.913 DEBUG:boto:Data: 2024-04-16T18:30:14.913 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.913 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.913 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.913 DEBUG:boto:Params: {} 2024-04-16T18:30:14.913 DEBUG:boto:Token: None 2024-04-16T18:30:14.913 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17 2024-04-16T18:30:14.913 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PTty2ULUqEiv4ebk5EGhptOhxd4= 2024-04-16T18:30:14.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PTty2ULUqEiv4ebk5EGhptOhxd4=', 'Content-Length': '0'} 2024-04-16T18:30:14.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9db00254a64072b-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.916 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2024-04-16T18:30:14.917 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.917 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.917 DEBUG:boto:Method: GET 2024-04-16T18:30:14.917 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:30:14.917 DEBUG:boto:Data: 2024-04-16T18:30:14.917 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.917 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.917 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.917 DEBUG:boto:Params: {} 2024-04-16T18:30:14.917 DEBUG:boto:Token: None 2024-04-16T18:30:14.918 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17 2024-04-16T18:30:14.918 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pQL6sH/oi9GVb4nitux2BIuw4U= 2024-04-16T18:30:14.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pQL6sH/oi9GVb4nitux2BIuw4U=', 'Content-Length': '0'} 2024-04-16T18:30:14.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f895bd81fb461eb2-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.962 DEBUG:boto:path=/lybtug-52/obj17 2024-04-16T18:30:14.962 DEBUG:boto:auth_path=/lybtug-52/obj17 2024-04-16T18:30:14.963 DEBUG:boto:Method: GET 2024-04-16T18:30:14.963 DEBUG:boto:Path: /lybtug-52/obj17 2024-04-16T18:30:14.963 DEBUG:boto:Data: 2024-04-16T18:30:14.963 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.963 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.963 DEBUG:boto:Params: {} 2024-04-16T18:30:14.963 DEBUG:boto:Token: None 2024-04-16T18:30:14.963 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/obj17 2024-04-16T18:30:14.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9pQL6sH/oi9GVb4nitux2BIuw4U= 2024-04-16T18:30:14.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9pQL6sH/oi9GVb4nitux2BIuw4U=', 'Content-Length': '0'} 2024-04-16T18:30:14.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7a3e16d51bca9a6-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.967 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.967 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.967 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.967 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.967 DEBUG:boto:Method: GET 2024-04-16T18:30:14.967 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:30:14.968 DEBUG:boto:Data: 2024-04-16T18:30:14.968 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.968 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:14.968 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.968 DEBUG:boto:Params: {} 2024-04-16T18:30:14.968 DEBUG:boto:Token: None 2024-04-16T18:30:14.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1815397bd2c5953-00661ec3b6-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:14.972 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:14.972 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:14.972 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:14.972 DEBUG:boto:path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.972 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj17 2024-04-16T18:30:14.972 DEBUG:boto:Method: GET 2024-04-16T18:30:14.973 DEBUG:boto:Path: /lybtug-52/?prefix=obj17 2024-04-16T18:30:14.973 DEBUG:boto:Data: 2024-04-16T18:30:14.973 DEBUG:boto:Headers: {} 2024-04-16T18:30:14.973 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:14.973 DEBUG:boto:Port: 8000 2024-04-16T18:30:14.973 DEBUG:boto:Params: {} 2024-04-16T18:30:14.973 DEBUG:boto:Token: None 2024-04-16T18:30:14.973 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:14 GMT /lybtug-52/ 2024-04-16T18:30:14.973 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I= 2024-04-16T18:30:14.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZPan7nBvhFsNAZgHNJVEp5fyT5I=', 'Content-Length': '0'} 2024-04-16T18:30:14.980 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc63398cbb24982f-00661ec3b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.022 DEBUG:boto:b'lybtug-52obj171000falseobj172024-04-16T18:20:12.390Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2024-04-16T18:30:15.023 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.023 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.023 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.023 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.023 DEBUG:boto:Method: GET 2024-04-16T18:30:15.023 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.023 DEBUG:boto:Data: 2024-04-16T18:30:15.023 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.023 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.023 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.024 DEBUG:boto:Params: {} 2024-04-16T18:30:15.024 DEBUG:boto:Token: None 2024-04-16T18:30:15.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4= 2024-04-16T18:30:15.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4=', 'Content-Length': '0'} 2024-04-16T18:30:15.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc40e8e5a1f3d2c6-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.028 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.028 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.028 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.028 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.028 DEBUG:boto:Method: GET 2024-04-16T18:30:15.028 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.028 DEBUG:boto:Data: 2024-04-16T18:30:15.028 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.028 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.029 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.029 DEBUG:boto:Params: {} 2024-04-16T18:30:15.029 DEBUG:boto:Token: None 2024-04-16T18:30:15.029 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4= 2024-04-16T18:30:15.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4=', 'Content-Length': '0'} 2024-04-16T18:30:15.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049a39d7702e0bb87-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.033 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.033 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.033 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.033 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.033 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.033 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.033 DEBUG:boto:Data: 2024-04-16T18:30:15.033 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.033 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.033 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.034 DEBUG:boto:Params: {} 2024-04-16T18:30:15.034 DEBUG:boto:Token: None 2024-04-16T18:30:15.034 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.034 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vthsUhCglVOuXl/xT/wIgic5iBY= 2024-04-16T18:30:15.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vthsUhCglVOuXl/xT/wIgic5iBY=', 'Content-Length': '0'} 2024-04-16T18:30:15.037 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074b03e25eac128eb-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.037 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.037 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.037 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.038 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.038 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.038 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.038 DEBUG:boto:Data: 2024-04-16T18:30:15.038 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.038 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.038 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.038 DEBUG:boto:Params: {} 2024-04-16T18:30:15.038 DEBUG:boto:Token: None 2024-04-16T18:30:15.038 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.038 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vthsUhCglVOuXl/xT/wIgic5iBY= 2024-04-16T18:30:15.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vthsUhCglVOuXl/xT/wIgic5iBY=', 'Content-Length': '0'} 2024-04-16T18:30:15.042 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000336d4ec7cce9d5dc-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.042 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2024-04-16T18:30:15.042 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.042 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.042 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.042 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.042 DEBUG:boto:Method: GET 2024-04-16T18:30:15.042 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.043 DEBUG:boto:Data: 2024-04-16T18:30:15.043 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.043 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.043 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.043 DEBUG:boto:Params: {} 2024-04-16T18:30:15.043 DEBUG:boto:Token: None 2024-04-16T18:30:15.043 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.043 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4= 2024-04-16T18:30:15.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4=', 'Content-Length': '0'} 2024-04-16T18:30:15.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c8c482e897e0e77-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.047 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.047 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.047 DEBUG:boto:path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.047 DEBUG:boto:auth_path=/lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.047 DEBUG:boto:Method: GET 2024-04-16T18:30:15.047 DEBUG:boto:Path: /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.047 DEBUG:boto:Data: 2024-04-16T18:30:15.047 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.047 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.047 DEBUG:boto:Params: {} 2024-04-16T18:30:15.047 DEBUG:boto:Token: None 2024-04-16T18:30:15.048 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185?versionId=null 2024-04-16T18:30:15.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4= 2024-04-16T18:30:15.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gbucwG9Vfc0xVYHkvPMD38G1nb4=', 'Content-Length': '0'} 2024-04-16T18:30:15.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da6b0a67e947aa97-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.051 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.051 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.051 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.051 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.052 DEBUG:boto:Method: GET 2024-04-16T18:30:15.052 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:30:15.052 DEBUG:boto:Data: 2024-04-16T18:30:15.052 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.052 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.052 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.052 DEBUG:boto:Params: {} 2024-04-16T18:30:15.052 DEBUG:boto:Token: None 2024-04-16T18:30:15.052 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.052 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bded03d5af7c6ddd-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.102 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.103 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.103 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.103 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.103 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.103 DEBUG:boto:Method: GET 2024-04-16T18:30:15.103 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:30:15.103 DEBUG:boto:Data: 2024-04-16T18:30:15.103 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.103 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.103 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.103 DEBUG:boto:Params: {} 2024-04-16T18:30:15.104 DEBUG:boto:Token: None 2024-04-16T18:30:15.104 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000273b00528838fae2-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.111 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.111 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.111 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.111 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.111 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:30:15.112 DEBUG:boto:Data: 2024-04-16T18:30:15.112 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.112 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.112 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.112 DEBUG:boto:Params: {} 2024-04-16T18:30:15.112 DEBUG:boto:Token: None 2024-04-16T18:30:15.112 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185 2024-04-16T18:30:15.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tSk0ir9uDk6YuAre7hIq33lWStM= 2024-04-16T18:30:15.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tSk0ir9uDk6YuAre7hIq33lWStM=', 'Content-Length': '0'} 2024-04-16T18:30:15.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000635688f4ff1aa3f5-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.116 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.116 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.116 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.116 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:30:15.116 DEBUG:boto:Data: 2024-04-16T18:30:15.116 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.116 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.116 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.116 DEBUG:boto:Params: {} 2024-04-16T18:30:15.116 DEBUG:boto:Token: None 2024-04-16T18:30:15.117 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185 2024-04-16T18:30:15.117 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tSk0ir9uDk6YuAre7hIq33lWStM= 2024-04-16T18:30:15.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tSk0ir9uDk6YuAre7hIq33lWStM=', 'Content-Length': '0'} 2024-04-16T18:30:15.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084cdb29525da31d5-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2024-04-16T18:30:15.120 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.120 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.120 DEBUG:boto:Method: GET 2024-04-16T18:30:15.120 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:30:15.121 DEBUG:boto:Data: 2024-04-16T18:30:15.121 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.121 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.121 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.121 DEBUG:boto:Params: {} 2024-04-16T18:30:15.121 DEBUG:boto:Token: None 2024-04-16T18:30:15.121 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185 2024-04-16T18:30:15.121 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z55lt7oX3UvKivXEgf3FTk85tiY= 2024-04-16T18:30:15.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z55lt7oX3UvKivXEgf3FTk85tiY=', 'Content-Length': '0'} 2024-04-16T18:30:15.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007856e1973e8f7ff9-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.124 DEBUG:boto:path=/lybtug-52/obj185 2024-04-16T18:30:15.125 DEBUG:boto:auth_path=/lybtug-52/obj185 2024-04-16T18:30:15.125 DEBUG:boto:Method: GET 2024-04-16T18:30:15.125 DEBUG:boto:Path: /lybtug-52/obj185 2024-04-16T18:30:15.125 DEBUG:boto:Data: 2024-04-16T18:30:15.125 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.125 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.125 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.125 DEBUG:boto:Params: {} 2024-04-16T18:30:15.125 DEBUG:boto:Token: None 2024-04-16T18:30:15.125 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj185 2024-04-16T18:30:15.126 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Z55lt7oX3UvKivXEgf3FTk85tiY= 2024-04-16T18:30:15.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Z55lt7oX3UvKivXEgf3FTk85tiY=', 'Content-Length': '0'} 2024-04-16T18:30:15.129 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfed845da567814b-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.170 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.170 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.170 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.171 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.171 DEBUG:boto:Method: GET 2024-04-16T18:30:15.171 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:30:15.171 DEBUG:boto:Data: 2024-04-16T18:30:15.171 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.171 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.171 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.171 DEBUG:boto:Params: {} 2024-04-16T18:30:15.171 DEBUG:boto:Token: None 2024-04-16T18:30:15.171 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.171 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ce5ef7d3c04841ce-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.176 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.176 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.176 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.176 DEBUG:boto:path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.176 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj185 2024-04-16T18:30:15.176 DEBUG:boto:Method: GET 2024-04-16T18:30:15.177 DEBUG:boto:Path: /lybtug-52/?prefix=obj185 2024-04-16T18:30:15.177 DEBUG:boto:Data: 2024-04-16T18:30:15.177 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.177 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.177 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.177 DEBUG:boto:Params: {} 2024-04-16T18:30:15.177 DEBUG:boto:Token: None 2024-04-16T18:30:15.177 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.177 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.184 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000077f61f5d9f33acd-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.184 DEBUG:boto:b'lybtug-52obj1851000falseobj1852024-04-16T18:27:53.450Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.184 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:30:15.184 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.184 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.184 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.185 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.185 DEBUG:boto:Method: GET 2024-04-16T18:30:15.185 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.185 DEBUG:boto:Data: 2024-04-16T18:30:15.185 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.185 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.185 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.185 DEBUG:boto:Params: {} 2024-04-16T18:30:15.185 DEBUG:boto:Token: None 2024-04-16T18:30:15.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug= 2024-04-16T18:30:15.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug=', 'Content-Length': '0'} 2024-04-16T18:30:15.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ce7a593e4b1feac-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.230 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.230 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.231 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.231 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.231 DEBUG:boto:Method: GET 2024-04-16T18:30:15.231 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.231 DEBUG:boto:Data: 2024-04-16T18:30:15.231 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.231 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.231 DEBUG:boto:Params: {} 2024-04-16T18:30:15.231 DEBUG:boto:Token: None 2024-04-16T18:30:15.231 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug= 2024-04-16T18:30:15.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug=', 'Content-Length': '0'} 2024-04-16T18:30:15.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1c4d55a1c157d8c-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.235 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.235 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.235 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.235 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.235 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.235 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.235 DEBUG:boto:Data: 2024-04-16T18:30:15.235 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.236 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.236 DEBUG:boto:Params: {} 2024-04-16T18:30:15.236 DEBUG:boto:Token: None 2024-04-16T18:30:15.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hc9PJZ3P6kc5nqM0h0EzAnGymBo= 2024-04-16T18:30:15.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hc9PJZ3P6kc5nqM0h0EzAnGymBo=', 'Content-Length': '0'} 2024-04-16T18:30:15.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd5cb2712c1e78bc-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.239 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.240 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.240 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.240 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.240 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.240 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.240 DEBUG:boto:Data: 2024-04-16T18:30:15.240 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.240 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.240 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.240 DEBUG:boto:Params: {} 2024-04-16T18:30:15.240 DEBUG:boto:Token: None 2024-04-16T18:30:15.241 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hc9PJZ3P6kc5nqM0h0EzAnGymBo= 2024-04-16T18:30:15.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hc9PJZ3P6kc5nqM0h0EzAnGymBo=', 'Content-Length': '0'} 2024-04-16T18:30:15.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a66e4a518bebe7f6-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.244 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:30:15.244 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.244 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.244 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.244 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.244 DEBUG:boto:Method: GET 2024-04-16T18:30:15.244 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.245 DEBUG:boto:Data: 2024-04-16T18:30:15.245 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.245 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.245 DEBUG:boto:Params: {} 2024-04-16T18:30:15.245 DEBUG:boto:Token: None 2024-04-16T18:30:15.245 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.245 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug= 2024-04-16T18:30:15.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug=', 'Content-Length': '0'} 2024-04-16T18:30:15.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000902f17a7c6ba2e00-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.249 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.249 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.249 DEBUG:boto:path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.249 DEBUG:boto:auth_path=/lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.249 DEBUG:boto:Method: GET 2024-04-16T18:30:15.249 DEBUG:boto:Path: /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.249 DEBUG:boto:Data: 2024-04-16T18:30:15.249 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.249 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.249 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.249 DEBUG:boto:Params: {} 2024-04-16T18:30:15.250 DEBUG:boto:Token: None 2024-04-16T18:30:15.250 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19?versionId=null 2024-04-16T18:30:15.250 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug= 2024-04-16T18:30:15.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YEH88h3RrggZuK6pL7+LcmhQkug=', 'Content-Length': '0'} 2024-04-16T18:30:15.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad3e639280ff7ebd-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.294 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.294 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.294 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.295 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.295 DEBUG:boto:Method: GET 2024-04-16T18:30:15.295 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:30:15.295 DEBUG:boto:Data: 2024-04-16T18:30:15.295 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.295 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.295 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.295 DEBUG:boto:Params: {} 2024-04-16T18:30:15.295 DEBUG:boto:Token: None 2024-04-16T18:30:15.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000844e2f4040c9bbc0-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.300 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.300 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.300 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.301 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.301 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.301 DEBUG:boto:Method: GET 2024-04-16T18:30:15.301 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:30:15.301 DEBUG:boto:Data: 2024-04-16T18:30:15.301 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.301 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.301 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.301 DEBUG:boto:Params: {} 2024-04-16T18:30:15.301 DEBUG:boto:Token: None 2024-04-16T18:30:15.301 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a08dea0087c69785-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.308 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.308 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.308 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.308 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.309 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:30:15.309 DEBUG:boto:Data: 2024-04-16T18:30:15.309 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.309 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.309 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.309 DEBUG:boto:Params: {} 2024-04-16T18:30:15.309 DEBUG:boto:Token: None 2024-04-16T18:30:15.309 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19 2024-04-16T18:30:15.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V3Tykdo/d74aWtLqakpTLbAdUUc= 2024-04-16T18:30:15.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V3Tykdo/d74aWtLqakpTLbAdUUc=', 'Content-Length': '0'} 2024-04-16T18:30:15.312 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002995b5e4a15b2f80-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.312 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.313 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.313 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.313 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:30:15.313 DEBUG:boto:Data: 2024-04-16T18:30:15.313 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.313 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.313 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.313 DEBUG:boto:Params: {} 2024-04-16T18:30:15.313 DEBUG:boto:Token: None 2024-04-16T18:30:15.313 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19 2024-04-16T18:30:15.314 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:V3Tykdo/d74aWtLqakpTLbAdUUc= 2024-04-16T18:30:15.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:V3Tykdo/d74aWtLqakpTLbAdUUc=', 'Content-Length': '0'} 2024-04-16T18:30:15.317 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e54a2a173aff788-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.317 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2024-04-16T18:30:15.317 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.317 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.317 DEBUG:boto:Method: GET 2024-04-16T18:30:15.317 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:30:15.317 DEBUG:boto:Data: 2024-04-16T18:30:15.317 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.318 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.318 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.318 DEBUG:boto:Params: {} 2024-04-16T18:30:15.318 DEBUG:boto:Token: None 2024-04-16T18:30:15.318 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19 2024-04-16T18:30:15.318 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ikxqe7+sjazMpyFUNfahHcZ+Z0M= 2024-04-16T18:30:15.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ikxqe7+sjazMpyFUNfahHcZ+Z0M=', 'Content-Length': '0'} 2024-04-16T18:30:15.321 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000679ec96434f6156f-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.362 DEBUG:boto:path=/lybtug-52/obj19 2024-04-16T18:30:15.363 DEBUG:boto:auth_path=/lybtug-52/obj19 2024-04-16T18:30:15.363 DEBUG:boto:Method: GET 2024-04-16T18:30:15.363 DEBUG:boto:Path: /lybtug-52/obj19 2024-04-16T18:30:15.363 DEBUG:boto:Data: 2024-04-16T18:30:15.363 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.363 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.363 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.363 DEBUG:boto:Params: {} 2024-04-16T18:30:15.363 DEBUG:boto:Token: None 2024-04-16T18:30:15.363 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj19 2024-04-16T18:30:15.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ikxqe7+sjazMpyFUNfahHcZ+Z0M= 2024-04-16T18:30:15.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ikxqe7+sjazMpyFUNfahHcZ+Z0M=', 'Content-Length': '0'} 2024-04-16T18:30:15.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f18e64b4b0b54395-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.367 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.367 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.367 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.367 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.367 DEBUG:boto:Method: GET 2024-04-16T18:30:15.367 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:30:15.368 DEBUG:boto:Data: 2024-04-16T18:30:15.368 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.368 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.368 DEBUG:boto:Params: {} 2024-04-16T18:30:15.368 DEBUG:boto:Token: None 2024-04-16T18:30:15.368 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.372 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c62e947a1f74b509-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.373 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.373 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.373 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.373 DEBUG:boto:path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.373 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj19 2024-04-16T18:30:15.373 DEBUG:boto:Method: GET 2024-04-16T18:30:15.373 DEBUG:boto:Path: /lybtug-52/?prefix=obj19 2024-04-16T18:30:15.374 DEBUG:boto:Data: 2024-04-16T18:30:15.374 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.374 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.374 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.374 DEBUG:boto:Params: {} 2024-04-16T18:30:15.374 DEBUG:boto:Token: None 2024-04-16T18:30:15.374 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.374 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7e349f4568efc85-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.422 DEBUG:boto:b'lybtug-52obj191000falseobj192024-04-16T18:22:20.871Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.422 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:30:15.423 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.423 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.423 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.423 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.423 DEBUG:boto:Method: GET 2024-04-16T18:30:15.423 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.423 DEBUG:boto:Data: 2024-04-16T18:30:15.423 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.423 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.423 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.423 DEBUG:boto:Params: {} 2024-04-16T18:30:15.424 DEBUG:boto:Token: None 2024-04-16T18:30:15.424 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.424 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU= 2024-04-16T18:30:15.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU=', 'Content-Length': '0'} 2024-04-16T18:30:15.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002d232ec60d1c0c2-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.427 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.428 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.428 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.428 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.428 DEBUG:boto:Method: GET 2024-04-16T18:30:15.428 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.428 DEBUG:boto:Data: 2024-04-16T18:30:15.428 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.428 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.428 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.428 DEBUG:boto:Params: {} 2024-04-16T18:30:15.428 DEBUG:boto:Token: None 2024-04-16T18:30:15.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.429 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU= 2024-04-16T18:30:15.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU=', 'Content-Length': '0'} 2024-04-16T18:30:15.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000101977e52fb7295b-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.432 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.432 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.432 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.432 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.432 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.432 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.432 DEBUG:boto:Data: 2024-04-16T18:30:15.432 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.432 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.432 DEBUG:boto:Params: {} 2024-04-16T18:30:15.433 DEBUG:boto:Token: None 2024-04-16T18:30:15.433 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.433 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XzI64p6TO9w3nJNiE/aZ8gKBoeQ= 2024-04-16T18:30:15.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XzI64p6TO9w3nJNiE/aZ8gKBoeQ=', 'Content-Length': '0'} 2024-04-16T18:30:15.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002da5ae99fa3df8a2-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.436 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.436 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.436 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.436 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.436 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.436 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.437 DEBUG:boto:Data: 2024-04-16T18:30:15.437 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.437 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.437 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.437 DEBUG:boto:Params: {} 2024-04-16T18:30:15.437 DEBUG:boto:Token: None 2024-04-16T18:30:15.437 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.437 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:XzI64p6TO9w3nJNiE/aZ8gKBoeQ= 2024-04-16T18:30:15.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XzI64p6TO9w3nJNiE/aZ8gKBoeQ=', 'Content-Length': '0'} 2024-04-16T18:30:15.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d3f6f134a2f30ea-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.440 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:30:15.440 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.441 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.441 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.441 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.441 DEBUG:boto:Method: GET 2024-04-16T18:30:15.441 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.441 DEBUG:boto:Data: 2024-04-16T18:30:15.441 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.441 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.441 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.441 DEBUG:boto:Params: {} 2024-04-16T18:30:15.441 DEBUG:boto:Token: None 2024-04-16T18:30:15.442 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.442 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU= 2024-04-16T18:30:15.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU=', 'Content-Length': '0'} 2024-04-16T18:30:15.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002912c53d7a5c29b4-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.486 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.487 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.487 DEBUG:boto:path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.487 DEBUG:boto:auth_path=/lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.487 DEBUG:boto:Method: GET 2024-04-16T18:30:15.487 DEBUG:boto:Path: /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.487 DEBUG:boto:Data: 2024-04-16T18:30:15.487 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.487 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.487 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.487 DEBUG:boto:Params: {} 2024-04-16T18:30:15.487 DEBUG:boto:Token: None 2024-04-16T18:30:15.487 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22?versionId=null 2024-04-16T18:30:15.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU= 2024-04-16T18:30:15.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EdqT7APEkBQSdJNAHLm3ctTSaEU=', 'Content-Length': '0'} 2024-04-16T18:30:15.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbcd7d7e70a0cfe4-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.491 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.491 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.491 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.491 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.491 DEBUG:boto:Method: GET 2024-04-16T18:30:15.491 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:30:15.491 DEBUG:boto:Data: 2024-04-16T18:30:15.491 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.491 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.491 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.492 DEBUG:boto:Params: {} 2024-04-16T18:30:15.492 DEBUG:boto:Token: None 2024-04-16T18:30:15.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.496 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000677a34cc72599aff-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.496 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.496 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.496 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.496 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.497 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.497 DEBUG:boto:Method: GET 2024-04-16T18:30:15.497 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:30:15.497 DEBUG:boto:Data: 2024-04-16T18:30:15.497 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.497 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.497 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.497 DEBUG:boto:Params: {} 2024-04-16T18:30:15.497 DEBUG:boto:Token: None 2024-04-16T18:30:15.497 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000335e86dd38570c84-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.546 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.547 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.547 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.547 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.547 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:30:15.547 DEBUG:boto:Data: 2024-04-16T18:30:15.547 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.547 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.547 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.547 DEBUG:boto:Params: {} 2024-04-16T18:30:15.547 DEBUG:boto:Token: None 2024-04-16T18:30:15.547 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22 2024-04-16T18:30:15.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:biag7fOxZSJn2gPsy12FUWFB8kI= 2024-04-16T18:30:15.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:biag7fOxZSJn2gPsy12FUWFB8kI=', 'Content-Length': '0'} 2024-04-16T18:30:15.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000981521f2c67d90c8-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.551 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.551 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.551 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.552 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:30:15.552 DEBUG:boto:Data: 2024-04-16T18:30:15.552 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.552 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.552 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.552 DEBUG:boto:Params: {} 2024-04-16T18:30:15.552 DEBUG:boto:Token: None 2024-04-16T18:30:15.552 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22 2024-04-16T18:30:15.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:biag7fOxZSJn2gPsy12FUWFB8kI= 2024-04-16T18:30:15.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:biag7fOxZSJn2gPsy12FUWFB8kI=', 'Content-Length': '0'} 2024-04-16T18:30:15.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000677007ebba717eea-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.556 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2024-04-16T18:30:15.556 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.556 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.556 DEBUG:boto:Method: GET 2024-04-16T18:30:15.556 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:30:15.556 DEBUG:boto:Data: 2024-04-16T18:30:15.556 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.556 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.556 DEBUG:boto:Params: {} 2024-04-16T18:30:15.557 DEBUG:boto:Token: None 2024-04-16T18:30:15.557 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22 2024-04-16T18:30:15.557 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5NCUfAUd3eoP9YghanzjN08D1Jg= 2024-04-16T18:30:15.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5NCUfAUd3eoP9YghanzjN08D1Jg=', 'Content-Length': '0'} 2024-04-16T18:30:15.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038e5aac5f8c76ce4-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.560 DEBUG:boto:path=/lybtug-52/obj22 2024-04-16T18:30:15.560 DEBUG:boto:auth_path=/lybtug-52/obj22 2024-04-16T18:30:15.560 DEBUG:boto:Method: GET 2024-04-16T18:30:15.560 DEBUG:boto:Path: /lybtug-52/obj22 2024-04-16T18:30:15.560 DEBUG:boto:Data: 2024-04-16T18:30:15.560 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.560 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.560 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.560 DEBUG:boto:Params: {} 2024-04-16T18:30:15.561 DEBUG:boto:Token: None 2024-04-16T18:30:15.561 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj22 2024-04-16T18:30:15.561 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:5NCUfAUd3eoP9YghanzjN08D1Jg= 2024-04-16T18:30:15.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5NCUfAUd3eoP9YghanzjN08D1Jg=', 'Content-Length': '0'} 2024-04-16T18:30:15.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a73abf1de6ca5165-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.564 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.564 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.564 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.564 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.565 DEBUG:boto:Method: GET 2024-04-16T18:30:15.565 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:30:15.565 DEBUG:boto:Data: 2024-04-16T18:30:15.565 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.565 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.565 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.565 DEBUG:boto:Params: {} 2024-04-16T18:30:15.565 DEBUG:boto:Token: None 2024-04-16T18:30:15.565 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.565 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.569 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d98b11cc8c523a3e-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.610 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.611 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.611 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.611 DEBUG:boto:path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.611 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj22 2024-04-16T18:30:15.611 DEBUG:boto:Method: GET 2024-04-16T18:30:15.611 DEBUG:boto:Path: /lybtug-52/?prefix=obj22 2024-04-16T18:30:15.611 DEBUG:boto:Data: 2024-04-16T18:30:15.611 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.611 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.611 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.611 DEBUG:boto:Params: {} 2024-04-16T18:30:15.612 DEBUG:boto:Token: None 2024-04-16T18:30:15.612 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000664824aefba1d8be-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.619 DEBUG:boto:b'lybtug-52obj221000falseobj222024-04-16T18:20:00.257Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:30:15.619 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.619 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.619 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.619 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.619 DEBUG:boto:Method: GET 2024-04-16T18:30:15.619 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.620 DEBUG:boto:Data: 2024-04-16T18:30:15.620 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.620 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.620 DEBUG:boto:Params: {} 2024-04-16T18:30:15.620 DEBUG:boto:Token: None 2024-04-16T18:30:15.620 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.620 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY= 2024-04-16T18:30:15.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY=', 'Content-Length': '0'} 2024-04-16T18:30:15.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007c94369bdacc2a1-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.624 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.624 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.624 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.624 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.624 DEBUG:boto:Method: GET 2024-04-16T18:30:15.624 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.624 DEBUG:boto:Data: 2024-04-16T18:30:15.624 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.624 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.625 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.625 DEBUG:boto:Params: {} 2024-04-16T18:30:15.625 DEBUG:boto:Token: None 2024-04-16T18:30:15.625 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.625 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY= 2024-04-16T18:30:15.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY=', 'Content-Length': '0'} 2024-04-16T18:30:15.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e980243c919930a2-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.670 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.671 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.671 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.671 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.671 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.671 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.671 DEBUG:boto:Data: 2024-04-16T18:30:15.671 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.671 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.671 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.671 DEBUG:boto:Params: {} 2024-04-16T18:30:15.671 DEBUG:boto:Token: None 2024-04-16T18:30:15.671 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.672 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hbdONOA7EMyKaCTGFtkzt7UjvLo= 2024-04-16T18:30:15.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hbdONOA7EMyKaCTGFtkzt7UjvLo=', 'Content-Length': '0'} 2024-04-16T18:30:15.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d37daa7a70936055-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.676 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.676 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.677 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.677 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.677 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.677 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.677 DEBUG:boto:Data: 2024-04-16T18:30:15.677 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.677 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.677 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.677 DEBUG:boto:Params: {} 2024-04-16T18:30:15.677 DEBUG:boto:Token: None 2024-04-16T18:30:15.677 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.677 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hbdONOA7EMyKaCTGFtkzt7UjvLo= 2024-04-16T18:30:15.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hbdONOA7EMyKaCTGFtkzt7UjvLo=', 'Content-Length': '0'} 2024-04-16T18:30:15.680 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fed106c09eea743-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.681 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:30:15.681 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.681 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.681 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.681 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.681 DEBUG:boto:Method: GET 2024-04-16T18:30:15.681 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.681 DEBUG:boto:Data: 2024-04-16T18:30:15.681 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.681 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.681 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.681 DEBUG:boto:Params: {} 2024-04-16T18:30:15.682 DEBUG:boto:Token: None 2024-04-16T18:30:15.682 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.682 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY= 2024-04-16T18:30:15.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY=', 'Content-Length': '0'} 2024-04-16T18:30:15.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ee2884cd93b2e96-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.685 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.685 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.685 DEBUG:boto:path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.685 DEBUG:boto:auth_path=/lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.685 DEBUG:boto:Method: GET 2024-04-16T18:30:15.685 DEBUG:boto:Path: /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.685 DEBUG:boto:Data: 2024-04-16T18:30:15.686 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.686 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.686 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.686 DEBUG:boto:Params: {} 2024-04-16T18:30:15.686 DEBUG:boto:Token: None 2024-04-16T18:30:15.686 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23?versionId=null 2024-04-16T18:30:15.686 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY= 2024-04-16T18:30:15.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3aLXZOfwi4+44HyJdwi65oq0dLY=', 'Content-Length': '0'} 2024-04-16T18:30:15.689 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009560ada326cefee1-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.689 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.690 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.690 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.690 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.690 DEBUG:boto:Method: GET 2024-04-16T18:30:15.690 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:30:15.690 DEBUG:boto:Data: 2024-04-16T18:30:15.690 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.690 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.690 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.690 DEBUG:boto:Params: {} 2024-04-16T18:30:15.690 DEBUG:boto:Token: None 2024-04-16T18:30:15.690 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.691 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009809ab3e656ee30f-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.695 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.695 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.695 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.695 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.695 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.695 DEBUG:boto:Method: GET 2024-04-16T18:30:15.696 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:30:15.696 DEBUG:boto:Data: 2024-04-16T18:30:15.696 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.696 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.696 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.696 DEBUG:boto:Params: {} 2024-04-16T18:30:15.696 DEBUG:boto:Token: None 2024-04-16T18:30:15.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.696 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079ba20f9eeaf3b5e-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.703 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.704 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.704 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.704 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.704 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:30:15.704 DEBUG:boto:Data: 2024-04-16T18:30:15.704 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.704 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.704 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.704 DEBUG:boto:Params: {} 2024-04-16T18:30:15.704 DEBUG:boto:Token: None 2024-04-16T18:30:15.705 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23 2024-04-16T18:30:15.705 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ak9H46uskTZl0yM4O7IPc2LUL4= 2024-04-16T18:30:15.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ak9H46uskTZl0yM4O7IPc2LUL4=', 'Content-Length': '0'} 2024-04-16T18:30:15.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ebc85d29afa6ba4-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.708 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.708 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.708 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.708 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:30:15.708 DEBUG:boto:Data: 2024-04-16T18:30:15.708 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.708 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.708 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.708 DEBUG:boto:Params: {} 2024-04-16T18:30:15.709 DEBUG:boto:Token: None 2024-04-16T18:30:15.709 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23 2024-04-16T18:30:15.709 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6ak9H46uskTZl0yM4O7IPc2LUL4= 2024-04-16T18:30:15.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6ak9H46uskTZl0yM4O7IPc2LUL4=', 'Content-Length': '0'} 2024-04-16T18:30:15.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bfee14504f3cb4f-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.712 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2024-04-16T18:30:15.712 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.712 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.712 DEBUG:boto:Method: GET 2024-04-16T18:30:15.712 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:30:15.712 DEBUG:boto:Data: 2024-04-16T18:30:15.712 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.712 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.712 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.712 DEBUG:boto:Params: {} 2024-04-16T18:30:15.713 DEBUG:boto:Token: None 2024-04-16T18:30:15.713 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23 2024-04-16T18:30:15.713 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HH0FODBKq9wbF1IdDfsQbFn79W8= 2024-04-16T18:30:15.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HH0FODBKq9wbF1IdDfsQbFn79W8=', 'Content-Length': '0'} 2024-04-16T18:30:15.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f6ff82c425cd3d9-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.758 DEBUG:boto:path=/lybtug-52/obj23 2024-04-16T18:30:15.759 DEBUG:boto:auth_path=/lybtug-52/obj23 2024-04-16T18:30:15.759 DEBUG:boto:Method: GET 2024-04-16T18:30:15.759 DEBUG:boto:Path: /lybtug-52/obj23 2024-04-16T18:30:15.759 DEBUG:boto:Data: 2024-04-16T18:30:15.759 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.759 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.759 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.759 DEBUG:boto:Params: {} 2024-04-16T18:30:15.759 DEBUG:boto:Token: None 2024-04-16T18:30:15.759 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj23 2024-04-16T18:30:15.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:HH0FODBKq9wbF1IdDfsQbFn79W8= 2024-04-16T18:30:15.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HH0FODBKq9wbF1IdDfsQbFn79W8=', 'Content-Length': '0'} 2024-04-16T18:30:15.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab00d8215cda5ac7-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.763 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.763 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.763 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.763 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.763 DEBUG:boto:Method: GET 2024-04-16T18:30:15.763 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:30:15.763 DEBUG:boto:Data: 2024-04-16T18:30:15.764 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.764 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.764 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.764 DEBUG:boto:Params: {} 2024-04-16T18:30:15.764 DEBUG:boto:Token: None 2024-04-16T18:30:15.764 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000024379be3ecd922b4-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.768 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.768 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.769 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.769 DEBUG:boto:path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.769 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj23 2024-04-16T18:30:15.769 DEBUG:boto:Method: GET 2024-04-16T18:30:15.769 DEBUG:boto:Path: /lybtug-52/?prefix=obj23 2024-04-16T18:30:15.769 DEBUG:boto:Data: 2024-04-16T18:30:15.769 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.769 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.769 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.769 DEBUG:boto:Params: {} 2024-04-16T18:30:15.769 DEBUG:boto:Token: None 2024-04-16T18:30:15.770 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.770 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7c480260c633642-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.776 DEBUG:boto:b'lybtug-52obj231000falseobj232024-04-16T18:23:25.664Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.777 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:30:15.777 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.777 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.777 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.777 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.777 DEBUG:boto:Method: GET 2024-04-16T18:30:15.777 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.777 DEBUG:boto:Data: 2024-04-16T18:30:15.778 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.778 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.778 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.778 DEBUG:boto:Params: {} 2024-04-16T18:30:15.778 DEBUG:boto:Token: None 2024-04-16T18:30:15.778 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.778 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY= 2024-04-16T18:30:15.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY=', 'Content-Length': '0'} 2024-04-16T18:30:15.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031ed53e6e08a2c48-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.782 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.782 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.782 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.782 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.782 DEBUG:boto:Method: GET 2024-04-16T18:30:15.783 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.783 DEBUG:boto:Data: 2024-04-16T18:30:15.783 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.783 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.783 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.783 DEBUG:boto:Params: {} 2024-04-16T18:30:15.783 DEBUG:boto:Token: None 2024-04-16T18:30:15.783 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.783 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY= 2024-04-16T18:30:15.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY=', 'Content-Length': '0'} 2024-04-16T18:30:15.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000161ea5c993b26115-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.786 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.787 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.787 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.787 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.787 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.787 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.787 DEBUG:boto:Data: 2024-04-16T18:30:15.787 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.787 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.787 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.787 DEBUG:boto:Params: {} 2024-04-16T18:30:15.787 DEBUG:boto:Token: None 2024-04-16T18:30:15.788 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uMU7qC6rkqcooi9h5mvEM+ac/Us= 2024-04-16T18:30:15.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uMU7qC6rkqcooi9h5mvEM+ac/Us=', 'Content-Length': '0'} 2024-04-16T18:30:15.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e4c195f12fd4ff6-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.791 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.791 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.791 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.791 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.791 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.792 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.792 DEBUG:boto:Data: 2024-04-16T18:30:15.792 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.792 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.792 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.792 DEBUG:boto:Params: {} 2024-04-16T18:30:15.792 DEBUG:boto:Token: None 2024-04-16T18:30:15.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:uMU7qC6rkqcooi9h5mvEM+ac/Us= 2024-04-16T18:30:15.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uMU7qC6rkqcooi9h5mvEM+ac/Us=', 'Content-Length': '0'} 2024-04-16T18:30:15.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065852a1ffd0e3da0-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:30:15.796 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.796 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.796 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.796 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.796 DEBUG:boto:Method: GET 2024-04-16T18:30:15.796 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.796 DEBUG:boto:Data: 2024-04-16T18:30:15.796 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.796 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.796 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.796 DEBUG:boto:Params: {} 2024-04-16T18:30:15.797 DEBUG:boto:Token: None 2024-04-16T18:30:15.797 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY= 2024-04-16T18:30:15.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY=', 'Content-Length': '0'} 2024-04-16T18:30:15.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f61936768a2c3052-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.842 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.842 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.843 DEBUG:boto:path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.843 DEBUG:boto:auth_path=/lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.843 DEBUG:boto:Method: GET 2024-04-16T18:30:15.843 DEBUG:boto:Path: /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.843 DEBUG:boto:Data: 2024-04-16T18:30:15.843 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.843 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.843 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.843 DEBUG:boto:Params: {} 2024-04-16T18:30:15.843 DEBUG:boto:Token: None 2024-04-16T18:30:15.844 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29?versionId=null 2024-04-16T18:30:15.844 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY= 2024-04-16T18:30:15.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QH+0dag046GJSoV1cQpVmiEzjyY=', 'Content-Length': '0'} 2024-04-16T18:30:15.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8991200307946c7-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.847 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.847 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.847 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.847 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.847 DEBUG:boto:Method: GET 2024-04-16T18:30:15.847 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:30:15.848 DEBUG:boto:Data: 2024-04-16T18:30:15.848 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.848 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.848 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.848 DEBUG:boto:Params: {} 2024-04-16T18:30:15.848 DEBUG:boto:Token: None 2024-04-16T18:30:15.848 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.848 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be5dd1758950be91-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.852 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.853 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.853 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.853 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.853 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.853 DEBUG:boto:Method: GET 2024-04-16T18:30:15.853 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:30:15.853 DEBUG:boto:Data: 2024-04-16T18:30:15.853 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.853 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.853 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.854 DEBUG:boto:Params: {} 2024-04-16T18:30:15.854 DEBUG:boto:Token: None 2024-04-16T18:30:15.854 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.854 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.861 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000836c06fd44cfce7a-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.861 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.861 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.861 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.861 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.862 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:30:15.862 DEBUG:boto:Data: 2024-04-16T18:30:15.862 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.862 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.862 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.862 DEBUG:boto:Params: {} 2024-04-16T18:30:15.862 DEBUG:boto:Token: None 2024-04-16T18:30:15.862 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29 2024-04-16T18:30:15.862 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BucKMS5fDejOZRspJoy07uV2ys8= 2024-04-16T18:30:15.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BucKMS5fDejOZRspJoy07uV2ys8=', 'Content-Length': '0'} 2024-04-16T18:30:15.865 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af2ccef110dbe589-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.865 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.866 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.866 DEBUG:boto:Method: HEAD 2024-04-16T18:30:15.866 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:30:15.866 DEBUG:boto:Data: 2024-04-16T18:30:15.866 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.866 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.866 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.866 DEBUG:boto:Params: {} 2024-04-16T18:30:15.866 DEBUG:boto:Token: None 2024-04-16T18:30:15.866 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29 2024-04-16T18:30:15.866 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:BucKMS5fDejOZRspJoy07uV2ys8= 2024-04-16T18:30:15.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BucKMS5fDejOZRspJoy07uV2ys8=', 'Content-Length': '0'} 2024-04-16T18:30:15.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075df7bbaeeec3634-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2024-04-16T18:30:15.870 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.870 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.870 DEBUG:boto:Method: GET 2024-04-16T18:30:15.870 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:30:15.870 DEBUG:boto:Data: 2024-04-16T18:30:15.870 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.870 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.870 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.870 DEBUG:boto:Params: {} 2024-04-16T18:30:15.870 DEBUG:boto:Token: None 2024-04-16T18:30:15.871 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29 2024-04-16T18:30:15.871 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ykVQDrTgjKZ4rsG61EuSv8E1VnY= 2024-04-16T18:30:15.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ykVQDrTgjKZ4rsG61EuSv8E1VnY=', 'Content-Length': '0'} 2024-04-16T18:30:15.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed30947d6c0aea35-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.914 DEBUG:boto:path=/lybtug-52/obj29 2024-04-16T18:30:15.914 DEBUG:boto:auth_path=/lybtug-52/obj29 2024-04-16T18:30:15.915 DEBUG:boto:Method: GET 2024-04-16T18:30:15.915 DEBUG:boto:Path: /lybtug-52/obj29 2024-04-16T18:30:15.915 DEBUG:boto:Data: 2024-04-16T18:30:15.915 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.915 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.915 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.915 DEBUG:boto:Params: {} 2024-04-16T18:30:15.915 DEBUG:boto:Token: None 2024-04-16T18:30:15.915 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj29 2024-04-16T18:30:15.915 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ykVQDrTgjKZ4rsG61EuSv8E1VnY= 2024-04-16T18:30:15.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ykVQDrTgjKZ4rsG61EuSv8E1VnY=', 'Content-Length': '0'} 2024-04-16T18:30:15.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cb3af477ea40952-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.919 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.919 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.919 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.919 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.919 DEBUG:boto:Method: GET 2024-04-16T18:30:15.919 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:30:15.919 DEBUG:boto:Data: 2024-04-16T18:30:15.920 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.920 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.920 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.920 DEBUG:boto:Params: {} 2024-04-16T18:30:15.920 DEBUG:boto:Token: None 2024-04-16T18:30:15.920 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.924 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b73941ff436853d-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.966 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.967 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:15.967 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:15.967 DEBUG:boto:path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.967 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj29 2024-04-16T18:30:15.967 DEBUG:boto:Method: GET 2024-04-16T18:30:15.967 DEBUG:boto:Path: /lybtug-52/?prefix=obj29 2024-04-16T18:30:15.967 DEBUG:boto:Data: 2024-04-16T18:30:15.967 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.967 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:15.967 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.967 DEBUG:boto:Params: {} 2024-04-16T18:30:15.968 DEBUG:boto:Token: None 2024-04-16T18:30:15.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/ 2024-04-16T18:30:15.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg= 2024-04-16T18:30:15.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Yj9ZZFc/WzDC6+utTqhvAyiZRg=', 'Content-Length': '0'} 2024-04-16T18:30:15.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4c296c5d36ab46e-00661ec3b7-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:15.975 DEBUG:boto:b'lybtug-52obj291000falseobj292024-04-16T18:25:26.766Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:15.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:30:15.976 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:15.976 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:15.976 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:15.976 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:15.976 DEBUG:boto:Method: GET 2024-04-16T18:30:15.976 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:30:15.976 DEBUG:boto:Data: 2024-04-16T18:30:15.976 DEBUG:boto:Headers: {} 2024-04-16T18:30:15.976 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:15.976 DEBUG:boto:Port: 8000 2024-04-16T18:30:15.977 DEBUG:boto:Params: {} 2024-04-16T18:30:15.977 DEBUG:boto:Token: None 2024-04-16T18:30:15.977 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:15 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:30:15.977 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:WukBcGWUL3HsfYyZa4CL7KkLRXY= 2024-04-16T18:30:15.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WukBcGWUL3HsfYyZa4CL7KkLRXY=', 'Content-Length': '0'} 2024-04-16T18:30:15.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d19bc3e6cc493d44-00661ec3b7-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.022 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.023 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.023 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.023 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.023 DEBUG:boto:Method: GET 2024-04-16T18:30:16.023 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.023 DEBUG:boto:Data: 2024-04-16T18:30:16.023 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.023 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.023 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.023 DEBUG:boto:Params: {} 2024-04-16T18:30:16.024 DEBUG:boto:Token: None 2024-04-16T18:30:16.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M5N6uFAIxWwYbP0Ajk13oQ64tcM= 2024-04-16T18:30:16.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M5N6uFAIxWwYbP0Ajk13oQ64tcM=', 'Content-Length': '0'} 2024-04-16T18:30:16.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4d0217fc551fc40-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.027 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.028 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.028 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.028 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.028 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.028 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.028 DEBUG:boto:Data: 2024-04-16T18:30:16.028 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.028 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.028 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.028 DEBUG:boto:Params: {} 2024-04-16T18:30:16.028 DEBUG:boto:Token: None 2024-04-16T18:30:16.029 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.029 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VCdvBew3mr22EpG4bNPECYKwIFI= 2024-04-16T18:30:16.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VCdvBew3mr22EpG4bNPECYKwIFI=', 'Content-Length': '0'} 2024-04-16T18:30:16.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fb4b2468177fad3-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.032 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.032 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.032 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.032 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.032 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.033 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.033 DEBUG:boto:Data: 2024-04-16T18:30:16.033 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.033 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.033 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.033 DEBUG:boto:Params: {} 2024-04-16T18:30:16.033 DEBUG:boto:Token: None 2024-04-16T18:30:16.033 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.033 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:VCdvBew3mr22EpG4bNPECYKwIFI= 2024-04-16T18:30:16.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VCdvBew3mr22EpG4bNPECYKwIFI=', 'Content-Length': '0'} 2024-04-16T18:30:16.036 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4169a89ff9fff64-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.037 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:30:16.037 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.037 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.037 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.037 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.037 DEBUG:boto:Method: GET 2024-04-16T18:30:16.037 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.037 DEBUG:boto:Data: 2024-04-16T18:30:16.037 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.038 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.038 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.038 DEBUG:boto:Params: {} 2024-04-16T18:30:16.038 DEBUG:boto:Token: None 2024-04-16T18:30:16.038 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.038 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M5N6uFAIxWwYbP0Ajk13oQ64tcM= 2024-04-16T18:30:16.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M5N6uFAIxWwYbP0Ajk13oQ64tcM=', 'Content-Length': '0'} 2024-04-16T18:30:16.041 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009272be908cb16907-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.082 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.082 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.083 DEBUG:boto:path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.083 DEBUG:boto:auth_path=/lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.083 DEBUG:boto:Method: GET 2024-04-16T18:30:16.083 DEBUG:boto:Path: /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.083 DEBUG:boto:Data: 2024-04-16T18:30:16.083 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.083 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.083 DEBUG:boto:Params: {} 2024-04-16T18:30:16.083 DEBUG:boto:Token: None 2024-04-16T18:30:16.084 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31?versionId=null 2024-04-16T18:30:16.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:M5N6uFAIxWwYbP0Ajk13oQ64tcM= 2024-04-16T18:30:16.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M5N6uFAIxWwYbP0Ajk13oQ64tcM=', 'Content-Length': '0'} 2024-04-16T18:30:16.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096ac8e3b5aabf5cb-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.087 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.088 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.088 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.088 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.088 DEBUG:boto:Method: GET 2024-04-16T18:30:16.088 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:30:16.088 DEBUG:boto:Data: 2024-04-16T18:30:16.088 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.088 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.088 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.089 DEBUG:boto:Params: {} 2024-04-16T18:30:16.089 DEBUG:boto:Token: None 2024-04-16T18:30:16.089 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.089 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c42aca8e9a21b24-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.093 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.093 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.094 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.094 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.094 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.094 DEBUG:boto:Method: GET 2024-04-16T18:30:16.094 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:30:16.094 DEBUG:boto:Data: 2024-04-16T18:30:16.094 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.094 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.094 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.095 DEBUG:boto:Params: {} 2024-04-16T18:30:16.095 DEBUG:boto:Token: None 2024-04-16T18:30:16.095 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.095 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.102 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000379b45e287a5d74e-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.102 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.102 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.103 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.103 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.103 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:30:16.103 DEBUG:boto:Data: 2024-04-16T18:30:16.103 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.103 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.103 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.103 DEBUG:boto:Params: {} 2024-04-16T18:30:16.103 DEBUG:boto:Token: None 2024-04-16T18:30:16.103 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31 2024-04-16T18:30:16.104 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MesKULxbxLxH/cMdobPA3zg8S4Q= 2024-04-16T18:30:16.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MesKULxbxLxH/cMdobPA3zg8S4Q=', 'Content-Length': '0'} 2024-04-16T18:30:16.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e4908c2409880d9-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.107 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.107 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.107 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.107 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:30:16.107 DEBUG:boto:Data: 2024-04-16T18:30:16.108 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.108 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.108 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.108 DEBUG:boto:Params: {} 2024-04-16T18:30:16.108 DEBUG:boto:Token: None 2024-04-16T18:30:16.108 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31 2024-04-16T18:30:16.108 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MesKULxbxLxH/cMdobPA3zg8S4Q= 2024-04-16T18:30:16.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MesKULxbxLxH/cMdobPA3zg8S4Q=', 'Content-Length': '0'} 2024-04-16T18:30:16.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a088cf4fe9a44d55-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.111 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2024-04-16T18:30:16.111 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.111 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.112 DEBUG:boto:Method: GET 2024-04-16T18:30:16.112 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:30:16.112 DEBUG:boto:Data: 2024-04-16T18:30:16.112 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.112 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.112 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.112 DEBUG:boto:Params: {} 2024-04-16T18:30:16.112 DEBUG:boto:Token: None 2024-04-16T18:30:16.112 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31 2024-04-16T18:30:16.112 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EN2/gaV2+BdtJNdJZEZRwOgnKjs= 2024-04-16T18:30:16.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EN2/gaV2+BdtJNdJZEZRwOgnKjs=', 'Content-Length': '0'} 2024-04-16T18:30:16.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030bcfe1b4caa3bbd-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.158 DEBUG:boto:path=/lybtug-52/obj31 2024-04-16T18:30:16.158 DEBUG:boto:auth_path=/lybtug-52/obj31 2024-04-16T18:30:16.159 DEBUG:boto:Method: GET 2024-04-16T18:30:16.159 DEBUG:boto:Path: /lybtug-52/obj31 2024-04-16T18:30:16.159 DEBUG:boto:Data: 2024-04-16T18:30:16.159 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.159 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.159 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.159 DEBUG:boto:Params: {} 2024-04-16T18:30:16.159 DEBUG:boto:Token: None 2024-04-16T18:30:16.159 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj31 2024-04-16T18:30:16.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EN2/gaV2+BdtJNdJZEZRwOgnKjs= 2024-04-16T18:30:16.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EN2/gaV2+BdtJNdJZEZRwOgnKjs=', 'Content-Length': '0'} 2024-04-16T18:30:16.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6cf498baa91534c-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.163 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.163 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.163 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.163 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.163 DEBUG:boto:Method: GET 2024-04-16T18:30:16.163 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:30:16.163 DEBUG:boto:Data: 2024-04-16T18:30:16.163 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.164 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.164 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.164 DEBUG:boto:Params: {} 2024-04-16T18:30:16.164 DEBUG:boto:Token: None 2024-04-16T18:30:16.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005979091fcca9659f-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.168 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.168 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.168 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.168 DEBUG:boto:path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.169 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj31 2024-04-16T18:30:16.169 DEBUG:boto:Method: GET 2024-04-16T18:30:16.169 DEBUG:boto:Path: /lybtug-52/?prefix=obj31 2024-04-16T18:30:16.169 DEBUG:boto:Data: 2024-04-16T18:30:16.169 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.169 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.169 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.169 DEBUG:boto:Params: {} 2024-04-16T18:30:16.169 DEBUG:boto:Token: None 2024-04-16T18:30:16.169 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.170 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8689f2bd7b45110-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.218 DEBUG:boto:b'lybtug-52obj311000falseobj312024-04-16T18:26:49.246Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.219 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:30:16.219 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.219 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.219 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.219 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.219 DEBUG:boto:Method: GET 2024-04-16T18:30:16.219 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.219 DEBUG:boto:Data: 2024-04-16T18:30:16.219 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.219 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.220 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.220 DEBUG:boto:Params: {} 2024-04-16T18:30:16.220 DEBUG:boto:Token: None 2024-04-16T18:30:16.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I= 2024-04-16T18:30:16.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I=', 'Content-Length': '0'} 2024-04-16T18:30:16.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f02cb50fcd974f1e-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.266 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.267 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.267 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.267 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.267 DEBUG:boto:Method: GET 2024-04-16T18:30:16.267 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.267 DEBUG:boto:Data: 2024-04-16T18:30:16.267 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.267 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.267 DEBUG:boto:Params: {} 2024-04-16T18:30:16.268 DEBUG:boto:Token: None 2024-04-16T18:30:16.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I= 2024-04-16T18:30:16.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I=', 'Content-Length': '0'} 2024-04-16T18:30:16.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afc8f752a6a976d8-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.271 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.271 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.271 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.272 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.272 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.272 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.272 DEBUG:boto:Data: 2024-04-16T18:30:16.272 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.272 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.272 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.272 DEBUG:boto:Params: {} 2024-04-16T18:30:16.272 DEBUG:boto:Token: None 2024-04-16T18:30:16.272 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yBKFVAl0h5OUDefKoK751eYGRHg= 2024-04-16T18:30:16.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yBKFVAl0h5OUDefKoK751eYGRHg=', 'Content-Length': '0'} 2024-04-16T18:30:16.275 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b22e36df277c7c0-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.276 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.276 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.276 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.276 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.276 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.276 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.276 DEBUG:boto:Data: 2024-04-16T18:30:16.276 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.276 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.276 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.277 DEBUG:boto:Params: {} 2024-04-16T18:30:16.277 DEBUG:boto:Token: None 2024-04-16T18:30:16.277 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.277 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yBKFVAl0h5OUDefKoK751eYGRHg= 2024-04-16T18:30:16.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yBKFVAl0h5OUDefKoK751eYGRHg=', 'Content-Length': '0'} 2024-04-16T18:30:16.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081f13ba001406d14-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.280 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:30:16.280 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.280 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.281 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.281 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.281 DEBUG:boto:Method: GET 2024-04-16T18:30:16.281 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.281 DEBUG:boto:Data: 2024-04-16T18:30:16.281 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.281 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.281 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.281 DEBUG:boto:Params: {} 2024-04-16T18:30:16.282 DEBUG:boto:Token: None 2024-04-16T18:30:16.282 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.282 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I= 2024-04-16T18:30:16.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I=', 'Content-Length': '0'} 2024-04-16T18:30:16.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0e3a47e7efe00e1-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.326 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.326 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.326 DEBUG:boto:path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.326 DEBUG:boto:auth_path=/lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.327 DEBUG:boto:Method: GET 2024-04-16T18:30:16.327 DEBUG:boto:Path: /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.327 DEBUG:boto:Data: 2024-04-16T18:30:16.327 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.327 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.327 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.327 DEBUG:boto:Params: {} 2024-04-16T18:30:16.327 DEBUG:boto:Token: None 2024-04-16T18:30:16.327 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33?versionId=null 2024-04-16T18:30:16.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I= 2024-04-16T18:30:16.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7PKT8agOEySSqa7motwd5sT/E3I=', 'Content-Length': '0'} 2024-04-16T18:30:16.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000785f7143228e0bba-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.331 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.331 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.331 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.331 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.331 DEBUG:boto:Method: GET 2024-04-16T18:30:16.331 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:30:16.331 DEBUG:boto:Data: 2024-04-16T18:30:16.331 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.331 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.331 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.332 DEBUG:boto:Params: {} 2024-04-16T18:30:16.332 DEBUG:boto:Token: None 2024-04-16T18:30:16.332 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.332 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.336 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000406cbba443747215-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.336 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.336 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.337 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.337 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.337 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.337 DEBUG:boto:Method: GET 2024-04-16T18:30:16.337 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:30:16.337 DEBUG:boto:Data: 2024-04-16T18:30:16.337 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.337 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.337 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.337 DEBUG:boto:Params: {} 2024-04-16T18:30:16.337 DEBUG:boto:Token: None 2024-04-16T18:30:16.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.338 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.345 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c2f11309b6924da-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.345 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.345 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.345 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.345 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.345 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:30:16.346 DEBUG:boto:Data: 2024-04-16T18:30:16.346 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.346 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.346 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.346 DEBUG:boto:Params: {} 2024-04-16T18:30:16.346 DEBUG:boto:Token: None 2024-04-16T18:30:16.346 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33 2024-04-16T18:30:16.346 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jlP1cJSpd8VllubjU5gFO/6+hCc= 2024-04-16T18:30:16.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jlP1cJSpd8VllubjU5gFO/6+hCc=', 'Content-Length': '0'} 2024-04-16T18:30:16.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a47067cd0dae845-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.349 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.349 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.349 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.350 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:30:16.350 DEBUG:boto:Data: 2024-04-16T18:30:16.350 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.350 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.350 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.350 DEBUG:boto:Params: {} 2024-04-16T18:30:16.350 DEBUG:boto:Token: None 2024-04-16T18:30:16.350 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33 2024-04-16T18:30:16.350 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jlP1cJSpd8VllubjU5gFO/6+hCc= 2024-04-16T18:30:16.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jlP1cJSpd8VllubjU5gFO/6+hCc=', 'Content-Length': '0'} 2024-04-16T18:30:16.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022d39c955a4c2916-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.354 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2024-04-16T18:30:16.354 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.354 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.354 DEBUG:boto:Method: GET 2024-04-16T18:30:16.354 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:30:16.354 DEBUG:boto:Data: 2024-04-16T18:30:16.354 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.354 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.354 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.354 DEBUG:boto:Params: {} 2024-04-16T18:30:16.355 DEBUG:boto:Token: None 2024-04-16T18:30:16.355 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33 2024-04-16T18:30:16.355 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6B6E4gvnHeDKv1KlN5Fkth073/w= 2024-04-16T18:30:16.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6B6E4gvnHeDKv1KlN5Fkth073/w=', 'Content-Length': '0'} 2024-04-16T18:30:16.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c21cd0acdb3ffd0-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.398 DEBUG:boto:path=/lybtug-52/obj33 2024-04-16T18:30:16.398 DEBUG:boto:auth_path=/lybtug-52/obj33 2024-04-16T18:30:16.399 DEBUG:boto:Method: GET 2024-04-16T18:30:16.399 DEBUG:boto:Path: /lybtug-52/obj33 2024-04-16T18:30:16.399 DEBUG:boto:Data: 2024-04-16T18:30:16.399 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.399 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.399 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.399 DEBUG:boto:Params: {} 2024-04-16T18:30:16.399 DEBUG:boto:Token: None 2024-04-16T18:30:16.399 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj33 2024-04-16T18:30:16.399 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6B6E4gvnHeDKv1KlN5Fkth073/w= 2024-04-16T18:30:16.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6B6E4gvnHeDKv1KlN5Fkth073/w=', 'Content-Length': '0'} 2024-04-16T18:30:16.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e97b2aa2046efa74-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.403 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.403 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.403 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.403 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.403 DEBUG:boto:Method: GET 2024-04-16T18:30:16.403 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:30:16.403 DEBUG:boto:Data: 2024-04-16T18:30:16.403 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.403 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.403 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.403 DEBUG:boto:Params: {} 2024-04-16T18:30:16.404 DEBUG:boto:Token: None 2024-04-16T18:30:16.404 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.404 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003eb02eafc6d25279-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.450 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.451 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.451 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.451 DEBUG:boto:path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.451 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj33 2024-04-16T18:30:16.451 DEBUG:boto:Method: GET 2024-04-16T18:30:16.451 DEBUG:boto:Path: /lybtug-52/?prefix=obj33 2024-04-16T18:30:16.451 DEBUG:boto:Data: 2024-04-16T18:30:16.451 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.451 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.451 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.451 DEBUG:boto:Params: {} 2024-04-16T18:30:16.451 DEBUG:boto:Token: None 2024-04-16T18:30:16.452 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.452 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.458 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b7a937aaae6f064-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.459 DEBUG:boto:b'lybtug-52obj331000falseobj332024-04-16T18:20:00.309Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:30:16.459 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.459 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.459 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.459 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.459 DEBUG:boto:Method: GET 2024-04-16T18:30:16.459 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.460 DEBUG:boto:Data: 2024-04-16T18:30:16.460 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.460 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.460 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.460 DEBUG:boto:Params: {} 2024-04-16T18:30:16.460 DEBUG:boto:Token: None 2024-04-16T18:30:16.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE= 2024-04-16T18:30:16.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE=', 'Content-Length': '0'} 2024-04-16T18:30:16.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ea4bc67080988a1-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.506 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.506 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.507 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.507 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.507 DEBUG:boto:Method: GET 2024-04-16T18:30:16.507 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.507 DEBUG:boto:Data: 2024-04-16T18:30:16.507 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.507 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.507 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.507 DEBUG:boto:Params: {} 2024-04-16T18:30:16.507 DEBUG:boto:Token: None 2024-04-16T18:30:16.507 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE= 2024-04-16T18:30:16.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE=', 'Content-Length': '0'} 2024-04-16T18:30:16.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b26144c0d64f66f3-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.511 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.511 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.511 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.511 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.511 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.511 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.511 DEBUG:boto:Data: 2024-04-16T18:30:16.512 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.512 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.512 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.512 DEBUG:boto:Params: {} 2024-04-16T18:30:16.512 DEBUG:boto:Token: None 2024-04-16T18:30:16.512 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.512 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UZeea5whS1Lj4N6qvrrR8CSz9js= 2024-04-16T18:30:16.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UZeea5whS1Lj4N6qvrrR8CSz9js=', 'Content-Length': '0'} 2024-04-16T18:30:16.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d5aee62e446e78f-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.515 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.515 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.515 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.515 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.516 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.516 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.516 DEBUG:boto:Data: 2024-04-16T18:30:16.516 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.516 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.516 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.516 DEBUG:boto:Params: {} 2024-04-16T18:30:16.516 DEBUG:boto:Token: None 2024-04-16T18:30:16.516 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UZeea5whS1Lj4N6qvrrR8CSz9js= 2024-04-16T18:30:16.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UZeea5whS1Lj4N6qvrrR8CSz9js=', 'Content-Length': '0'} 2024-04-16T18:30:16.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b15441ed6023d73-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.519 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:30:16.520 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.520 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.520 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.520 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.520 DEBUG:boto:Method: GET 2024-04-16T18:30:16.520 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.520 DEBUG:boto:Data: 2024-04-16T18:30:16.520 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.520 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.520 DEBUG:boto:Params: {} 2024-04-16T18:30:16.521 DEBUG:boto:Token: None 2024-04-16T18:30:16.521 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.521 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE= 2024-04-16T18:30:16.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE=', 'Content-Length': '0'} 2024-04-16T18:30:16.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab6646f9e1e52f44-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.566 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.566 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.567 DEBUG:boto:path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.567 DEBUG:boto:auth_path=/lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.567 DEBUG:boto:Method: GET 2024-04-16T18:30:16.567 DEBUG:boto:Path: /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.567 DEBUG:boto:Data: 2024-04-16T18:30:16.567 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.567 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.567 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.567 DEBUG:boto:Params: {} 2024-04-16T18:30:16.567 DEBUG:boto:Token: None 2024-04-16T18:30:16.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34?versionId=null 2024-04-16T18:30:16.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE= 2024-04-16T18:30:16.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qUrjLTewEQuUIEf3jhl3C8rOTdE=', 'Content-Length': '0'} 2024-04-16T18:30:16.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000346e71c1ac239f82-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.571 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.571 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.571 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.571 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.571 DEBUG:boto:Method: GET 2024-04-16T18:30:16.571 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:30:16.571 DEBUG:boto:Data: 2024-04-16T18:30:16.572 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.572 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.572 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.572 DEBUG:boto:Params: {} 2024-04-16T18:30:16.572 DEBUG:boto:Token: None 2024-04-16T18:30:16.572 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.572 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fbcf6f09efd7957-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.576 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.576 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.577 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.577 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.577 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.577 DEBUG:boto:Method: GET 2024-04-16T18:30:16.577 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:30:16.577 DEBUG:boto:Data: 2024-04-16T18:30:16.577 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.577 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.577 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.577 DEBUG:boto:Params: {} 2024-04-16T18:30:16.577 DEBUG:boto:Token: None 2024-04-16T18:30:16.578 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.578 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd7d1331e6fd85b8-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.584 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.585 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.585 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.585 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.585 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:30:16.585 DEBUG:boto:Data: 2024-04-16T18:30:16.585 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.585 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.585 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.585 DEBUG:boto:Params: {} 2024-04-16T18:30:16.586 DEBUG:boto:Token: None 2024-04-16T18:30:16.586 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34 2024-04-16T18:30:16.586 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:19rn9g2ljDbLo8nXS9K1ICTz/ms= 2024-04-16T18:30:16.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:19rn9g2ljDbLo8nXS9K1ICTz/ms=', 'Content-Length': '0'} 2024-04-16T18:30:16.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b3d8183190e7a93-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.589 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.589 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.589 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.589 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:30:16.590 DEBUG:boto:Data: 2024-04-16T18:30:16.590 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.590 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.590 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.590 DEBUG:boto:Params: {} 2024-04-16T18:30:16.590 DEBUG:boto:Token: None 2024-04-16T18:30:16.590 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34 2024-04-16T18:30:16.590 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:19rn9g2ljDbLo8nXS9K1ICTz/ms= 2024-04-16T18:30:16.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:19rn9g2ljDbLo8nXS9K1ICTz/ms=', 'Content-Length': '0'} 2024-04-16T18:30:16.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081f0ee22d982ce4c-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.593 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2024-04-16T18:30:16.594 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.594 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.594 DEBUG:boto:Method: GET 2024-04-16T18:30:16.594 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:30:16.594 DEBUG:boto:Data: 2024-04-16T18:30:16.594 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.594 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.594 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.594 DEBUG:boto:Params: {} 2024-04-16T18:30:16.594 DEBUG:boto:Token: None 2024-04-16T18:30:16.595 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34 2024-04-16T18:30:16.595 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eQmKB4FUI8yjr2a0JB+sZYg6t34= 2024-04-16T18:30:16.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eQmKB4FUI8yjr2a0JB+sZYg6t34=', 'Content-Length': '0'} 2024-04-16T18:30:16.598 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b40681875e456d6-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.638 DEBUG:boto:path=/lybtug-52/obj34 2024-04-16T18:30:16.638 DEBUG:boto:auth_path=/lybtug-52/obj34 2024-04-16T18:30:16.639 DEBUG:boto:Method: GET 2024-04-16T18:30:16.639 DEBUG:boto:Path: /lybtug-52/obj34 2024-04-16T18:30:16.639 DEBUG:boto:Data: 2024-04-16T18:30:16.639 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.639 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.639 DEBUG:boto:Params: {} 2024-04-16T18:30:16.639 DEBUG:boto:Token: None 2024-04-16T18:30:16.639 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj34 2024-04-16T18:30:16.639 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:eQmKB4FUI8yjr2a0JB+sZYg6t34= 2024-04-16T18:30:16.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eQmKB4FUI8yjr2a0JB+sZYg6t34=', 'Content-Length': '0'} 2024-04-16T18:30:16.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4dbc4f69d53c90b-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.642 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.643 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.643 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.643 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.643 DEBUG:boto:Method: GET 2024-04-16T18:30:16.643 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:30:16.643 DEBUG:boto:Data: 2024-04-16T18:30:16.643 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.643 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.643 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.643 DEBUG:boto:Params: {} 2024-04-16T18:30:16.643 DEBUG:boto:Token: None 2024-04-16T18:30:16.644 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.647 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004af684ffd15283d7-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.690 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.691 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.691 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.691 DEBUG:boto:path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.691 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj34 2024-04-16T18:30:16.691 DEBUG:boto:Method: GET 2024-04-16T18:30:16.691 DEBUG:boto:Path: /lybtug-52/?prefix=obj34 2024-04-16T18:30:16.691 DEBUG:boto:Data: 2024-04-16T18:30:16.691 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.691 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.691 DEBUG:boto:Params: {} 2024-04-16T18:30:16.692 DEBUG:boto:Token: None 2024-04-16T18:30:16.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.699 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093d8e5c63a496213-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.700 DEBUG:boto:b'lybtug-52obj341000falseobj342024-04-16T18:20:12.407Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.700 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2024-04-16T18:30:16.700 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.700 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.700 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.700 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.700 DEBUG:boto:Method: GET 2024-04-16T18:30:16.700 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.701 DEBUG:boto:Data: 2024-04-16T18:30:16.701 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.701 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.701 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.701 DEBUG:boto:Params: {} 2024-04-16T18:30:16.701 DEBUG:boto:Token: None 2024-04-16T18:30:16.701 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.701 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM= 2024-04-16T18:30:16.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM=', 'Content-Length': '0'} 2024-04-16T18:30:16.705 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000bbd4e2d43bb35f-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.746 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.746 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.746 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.747 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.747 DEBUG:boto:Method: GET 2024-04-16T18:30:16.747 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.747 DEBUG:boto:Data: 2024-04-16T18:30:16.747 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.747 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.747 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.747 DEBUG:boto:Params: {} 2024-04-16T18:30:16.747 DEBUG:boto:Token: None 2024-04-16T18:30:16.747 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.748 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM= 2024-04-16T18:30:16.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM=', 'Content-Length': '0'} 2024-04-16T18:30:16.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fc9106d53878ce0-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.751 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.751 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.751 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.752 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.752 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.752 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.752 DEBUG:boto:Data: 2024-04-16T18:30:16.752 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.752 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.752 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.752 DEBUG:boto:Params: {} 2024-04-16T18:30:16.752 DEBUG:boto:Token: None 2024-04-16T18:30:16.752 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.752 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YXZkAOhmToplSbr7sWJETxCj88Q= 2024-04-16T18:30:16.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YXZkAOhmToplSbr7sWJETxCj88Q=', 'Content-Length': '0'} 2024-04-16T18:30:16.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007aae8a54473f0002-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.756 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.756 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.756 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.757 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.757 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.757 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.757 DEBUG:boto:Data: 2024-04-16T18:30:16.757 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.757 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.757 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.757 DEBUG:boto:Params: {} 2024-04-16T18:30:16.757 DEBUG:boto:Token: None 2024-04-16T18:30:16.757 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.757 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YXZkAOhmToplSbr7sWJETxCj88Q= 2024-04-16T18:30:16.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YXZkAOhmToplSbr7sWJETxCj88Q=', 'Content-Length': '0'} 2024-04-16T18:30:16.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee7d7a2025c77403-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.761 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2024-04-16T18:30:16.761 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.761 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.761 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.761 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.761 DEBUG:boto:Method: GET 2024-04-16T18:30:16.761 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.761 DEBUG:boto:Data: 2024-04-16T18:30:16.761 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.761 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.762 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.762 DEBUG:boto:Params: {} 2024-04-16T18:30:16.762 DEBUG:boto:Token: None 2024-04-16T18:30:16.762 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.762 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM= 2024-04-16T18:30:16.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM=', 'Content-Length': '0'} 2024-04-16T18:30:16.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd004256c4088134-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.806 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.806 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.807 DEBUG:boto:path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.807 DEBUG:boto:auth_path=/lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.807 DEBUG:boto:Method: GET 2024-04-16T18:30:16.807 DEBUG:boto:Path: /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.807 DEBUG:boto:Data: 2024-04-16T18:30:16.807 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.807 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.807 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.807 DEBUG:boto:Params: {} 2024-04-16T18:30:16.807 DEBUG:boto:Token: None 2024-04-16T18:30:16.807 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37?versionId=null 2024-04-16T18:30:16.808 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM= 2024-04-16T18:30:16.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:N3IXEYhPNp3stPCEixtXQU45NAM=', 'Content-Length': '0'} 2024-04-16T18:30:16.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000475e223b4cbb65da-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.811 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.811 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.811 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.811 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.811 DEBUG:boto:Method: GET 2024-04-16T18:30:16.811 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:30:16.812 DEBUG:boto:Data: 2024-04-16T18:30:16.812 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.812 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.812 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.812 DEBUG:boto:Params: {} 2024-04-16T18:30:16.812 DEBUG:boto:Token: None 2024-04-16T18:30:16.812 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.812 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.816 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058e87bae2a7f6dbc-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.816 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.816 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.817 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.817 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.817 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.817 DEBUG:boto:Method: GET 2024-04-16T18:30:16.817 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:30:16.817 DEBUG:boto:Data: 2024-04-16T18:30:16.817 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.817 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.817 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.817 DEBUG:boto:Params: {} 2024-04-16T18:30:16.817 DEBUG:boto:Token: None 2024-04-16T18:30:16.817 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.818 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000521ddf2e05d8681f-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.823 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.824 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.824 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.824 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.824 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:30:16.824 DEBUG:boto:Data: 2024-04-16T18:30:16.824 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.824 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.824 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.824 DEBUG:boto:Params: {} 2024-04-16T18:30:16.825 DEBUG:boto:Token: None 2024-04-16T18:30:16.825 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37 2024-04-16T18:30:16.825 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EO5/q0T+o6NiSMaDymhsGTqnERo= 2024-04-16T18:30:16.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EO5/q0T+o6NiSMaDymhsGTqnERo=', 'Content-Length': '0'} 2024-04-16T18:30:16.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005342925aa223f9a8-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.828 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.828 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.828 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.828 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:30:16.829 DEBUG:boto:Data: 2024-04-16T18:30:16.829 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.829 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.829 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.829 DEBUG:boto:Params: {} 2024-04-16T18:30:16.829 DEBUG:boto:Token: None 2024-04-16T18:30:16.829 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37 2024-04-16T18:30:16.829 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EO5/q0T+o6NiSMaDymhsGTqnERo= 2024-04-16T18:30:16.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EO5/q0T+o6NiSMaDymhsGTqnERo=', 'Content-Length': '0'} 2024-04-16T18:30:16.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6d6a94092aba677-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.833 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2024-04-16T18:30:16.833 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.833 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.833 DEBUG:boto:Method: GET 2024-04-16T18:30:16.833 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:30:16.833 DEBUG:boto:Data: 2024-04-16T18:30:16.833 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.833 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.833 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.833 DEBUG:boto:Params: {} 2024-04-16T18:30:16.833 DEBUG:boto:Token: None 2024-04-16T18:30:16.834 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37 2024-04-16T18:30:16.834 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nl4khZ8yMiwoJ7OHKlO8P3HadKI= 2024-04-16T18:30:16.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nl4khZ8yMiwoJ7OHKlO8P3HadKI=', 'Content-Length': '0'} 2024-04-16T18:30:16.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000723bf2ce329556f6-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.878 DEBUG:boto:path=/lybtug-52/obj37 2024-04-16T18:30:16.878 DEBUG:boto:auth_path=/lybtug-52/obj37 2024-04-16T18:30:16.879 DEBUG:boto:Method: GET 2024-04-16T18:30:16.879 DEBUG:boto:Path: /lybtug-52/obj37 2024-04-16T18:30:16.879 DEBUG:boto:Data: 2024-04-16T18:30:16.879 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.879 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.879 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.879 DEBUG:boto:Params: {} 2024-04-16T18:30:16.879 DEBUG:boto:Token: None 2024-04-16T18:30:16.879 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj37 2024-04-16T18:30:16.879 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nl4khZ8yMiwoJ7OHKlO8P3HadKI= 2024-04-16T18:30:16.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nl4khZ8yMiwoJ7OHKlO8P3HadKI=', 'Content-Length': '0'} 2024-04-16T18:30:16.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000513c2979449e838b-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.883 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.883 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.883 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.883 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.883 DEBUG:boto:Method: GET 2024-04-16T18:30:16.883 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:30:16.883 DEBUG:boto:Data: 2024-04-16T18:30:16.883 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.883 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.884 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.884 DEBUG:boto:Params: {} 2024-04-16T18:30:16.884 DEBUG:boto:Token: None 2024-04-16T18:30:16.884 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.884 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045826ecf566942ba-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.888 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.888 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:16.888 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:16.888 DEBUG:boto:path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.888 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj37 2024-04-16T18:30:16.889 DEBUG:boto:Method: GET 2024-04-16T18:30:16.889 DEBUG:boto:Path: /lybtug-52/?prefix=obj37 2024-04-16T18:30:16.889 DEBUG:boto:Data: 2024-04-16T18:30:16.889 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.889 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.889 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.889 DEBUG:boto:Params: {} 2024-04-16T18:30:16.889 DEBUG:boto:Token: None 2024-04-16T18:30:16.889 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/ 2024-04-16T18:30:16.889 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA= 2024-04-16T18:30:16.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Jo71IELoCLpqpYoC1+Cyr8LNUtA=', 'Content-Length': '0'} 2024-04-16T18:30:16.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a57f634706a2be78-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.896 DEBUG:boto:b'lybtug-52obj371000falseobj372024-04-16T18:27:53.306Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:16.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:30:16.897 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:16.897 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:16.897 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.897 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.897 DEBUG:boto:Method: GET 2024-04-16T18:30:16.897 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.897 DEBUG:boto:Data: 2024-04-16T18:30:16.897 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.897 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.897 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.897 DEBUG:boto:Params: {} 2024-04-16T18:30:16.898 DEBUG:boto:Token: None 2024-04-16T18:30:16.898 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.898 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ddIHlBuz758wINBoSeT/D58y8AU= 2024-04-16T18:30:16.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ddIHlBuz758wINBoSeT/D58y8AU=', 'Content-Length': '0'} 2024-04-16T18:30:16.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005dc3661c97bdc228-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.942 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:16.943 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:16.943 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.943 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.943 DEBUG:boto:Method: GET 2024-04-16T18:30:16.943 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.943 DEBUG:boto:Data: 2024-04-16T18:30:16.943 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.943 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.943 DEBUG:boto:Params: {} 2024-04-16T18:30:16.943 DEBUG:boto:Token: None 2024-04-16T18:30:16.944 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ddIHlBuz758wINBoSeT/D58y8AU= 2024-04-16T18:30:16.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ddIHlBuz758wINBoSeT/D58y8AU=', 'Content-Length': '0'} 2024-04-16T18:30:16.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000622a27cbf28e5041-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.947 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:16.947 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:16.947 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.947 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.947 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.947 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.947 DEBUG:boto:Data: 2024-04-16T18:30:16.947 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.947 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.948 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.948 DEBUG:boto:Params: {} 2024-04-16T18:30:16.948 DEBUG:boto:Token: None 2024-04-16T18:30:16.948 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0BlCpn9MEkPydFUuFaTCPSt+fw= 2024-04-16T18:30:16.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0BlCpn9MEkPydFUuFaTCPSt+fw=', 'Content-Length': '0'} 2024-04-16T18:30:16.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035e146a45a45539d-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.951 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:16.951 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:16.951 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.951 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.952 DEBUG:boto:Method: HEAD 2024-04-16T18:30:16.952 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.952 DEBUG:boto:Data: 2024-04-16T18:30:16.952 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.952 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:16.952 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.952 DEBUG:boto:Params: {} 2024-04-16T18:30:16.952 DEBUG:boto:Token: None 2024-04-16T18:30:16.952 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.952 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:v0BlCpn9MEkPydFUuFaTCPSt+fw= 2024-04-16T18:30:16.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0BlCpn9MEkPydFUuFaTCPSt+fw=', 'Content-Length': '0'} 2024-04-16T18:30:16.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a5827489997cde7-00661ec3b8-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:16.955 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:30:16.955 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:16.956 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:16.956 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.956 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.956 DEBUG:boto:Method: GET 2024-04-16T18:30:16.956 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.956 DEBUG:boto:Data: 2024-04-16T18:30:16.956 DEBUG:boto:Headers: {} 2024-04-16T18:30:16.956 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:16.956 DEBUG:boto:Port: 8000 2024-04-16T18:30:16.956 DEBUG:boto:Params: {} 2024-04-16T18:30:16.956 DEBUG:boto:Token: None 2024-04-16T18:30:16.956 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:16 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:30:16.957 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ddIHlBuz758wINBoSeT/D58y8AU= 2024-04-16T18:30:16.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ddIHlBuz758wINBoSeT/D58y8AU=', 'Content-Length': '0'} 2024-04-16T18:30:16.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a92c67b09fe8053c-00661ec3b8-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.002 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:17.002 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:17.002 DEBUG:boto:path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:17.003 DEBUG:boto:auth_path=/lybtug-52/obj38?versionId=null 2024-04-16T18:30:17.003 DEBUG:boto:Method: GET 2024-04-16T18:30:17.003 DEBUG:boto:Path: /lybtug-52/obj38?versionId=null 2024-04-16T18:30:17.003 DEBUG:boto:Data: 2024-04-16T18:30:17.003 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.003 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.003 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.003 DEBUG:boto:Params: {} 2024-04-16T18:30:17.003 DEBUG:boto:Token: None 2024-04-16T18:30:17.003 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj38?versionId=null 2024-04-16T18:30:17.004 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:LxnCWTKPx6k0reBYoJ4SLCksa0A= 2024-04-16T18:30:17.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LxnCWTKPx6k0reBYoJ4SLCksa0A=', 'Content-Length': '0'} 2024-04-16T18:30:17.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000352e8c7db159d045-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.007 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.007 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.007 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.007 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.007 DEBUG:boto:Method: GET 2024-04-16T18:30:17.007 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:30:17.007 DEBUG:boto:Data: 2024-04-16T18:30:17.007 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.007 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.007 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.007 DEBUG:boto:Params: {} 2024-04-16T18:30:17.008 DEBUG:boto:Token: None 2024-04-16T18:30:17.008 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.008 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c5a45c563cda0ea-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.012 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.012 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.012 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.012 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.012 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.012 DEBUG:boto:Method: GET 2024-04-16T18:30:17.012 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:30:17.013 DEBUG:boto:Data: 2024-04-16T18:30:17.013 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.013 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.013 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.013 DEBUG:boto:Params: {} 2024-04-16T18:30:17.013 DEBUG:boto:Token: None 2024-04-16T18:30:17.013 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.013 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa143d562540d128-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.062 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.063 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:17.063 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:17.063 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.063 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:30:17.063 DEBUG:boto:Data: 2024-04-16T18:30:17.063 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.063 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.063 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.063 DEBUG:boto:Params: {} 2024-04-16T18:30:17.063 DEBUG:boto:Token: None 2024-04-16T18:30:17.064 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj38 2024-04-16T18:30:17.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t8++n7zcnhRw6QC/jbRm3vByO9I= 2024-04-16T18:30:17.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t8++n7zcnhRw6QC/jbRm3vByO9I=', 'Content-Length': '0'} 2024-04-16T18:30:17.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efc3d49684683f13-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.067 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:17.067 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:17.067 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.067 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:30:17.067 DEBUG:boto:Data: 2024-04-16T18:30:17.067 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.067 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.067 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.067 DEBUG:boto:Params: {} 2024-04-16T18:30:17.067 DEBUG:boto:Token: None 2024-04-16T18:30:17.068 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj38 2024-04-16T18:30:17.068 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:t8++n7zcnhRw6QC/jbRm3vByO9I= 2024-04-16T18:30:17.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:t8++n7zcnhRw6QC/jbRm3vByO9I=', 'Content-Length': '0'} 2024-04-16T18:30:17.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd17d7647c201369-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2024-04-16T18:30:17.071 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:17.071 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:17.071 DEBUG:boto:Method: GET 2024-04-16T18:30:17.071 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:30:17.071 DEBUG:boto:Data: 2024-04-16T18:30:17.071 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.071 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.071 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.071 DEBUG:boto:Params: {} 2024-04-16T18:30:17.071 DEBUG:boto:Token: None 2024-04-16T18:30:17.072 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj38 2024-04-16T18:30:17.072 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mGgjjCGTnM1VbiLEzcRZZDUehAA= 2024-04-16T18:30:17.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mGgjjCGTnM1VbiLEzcRZZDUehAA=', 'Content-Length': '0'} 2024-04-16T18:30:17.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e02643f9dfdc0aac-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.122 DEBUG:boto:path=/lybtug-52/obj38 2024-04-16T18:30:17.122 DEBUG:boto:auth_path=/lybtug-52/obj38 2024-04-16T18:30:17.123 DEBUG:boto:Method: GET 2024-04-16T18:30:17.123 DEBUG:boto:Path: /lybtug-52/obj38 2024-04-16T18:30:17.123 DEBUG:boto:Data: 2024-04-16T18:30:17.123 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.123 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.123 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.123 DEBUG:boto:Params: {} 2024-04-16T18:30:17.123 DEBUG:boto:Token: None 2024-04-16T18:30:17.123 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj38 2024-04-16T18:30:17.123 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:mGgjjCGTnM1VbiLEzcRZZDUehAA= 2024-04-16T18:30:17.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mGgjjCGTnM1VbiLEzcRZZDUehAA=', 'Content-Length': '0'} 2024-04-16T18:30:17.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067d72b949f81492e-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.127 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.127 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.127 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.127 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.127 DEBUG:boto:Method: GET 2024-04-16T18:30:17.127 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:30:17.127 DEBUG:boto:Data: 2024-04-16T18:30:17.127 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.127 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.127 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.127 DEBUG:boto:Params: {} 2024-04-16T18:30:17.128 DEBUG:boto:Token: None 2024-04-16T18:30:17.128 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.128 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ca05c33958a0484-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.174 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.175 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.175 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.175 DEBUG:boto:path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.175 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj38 2024-04-16T18:30:17.175 DEBUG:boto:Method: GET 2024-04-16T18:30:17.175 DEBUG:boto:Path: /lybtug-52/?prefix=obj38 2024-04-16T18:30:17.175 DEBUG:boto:Data: 2024-04-16T18:30:17.175 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.175 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.175 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.175 DEBUG:boto:Params: {} 2024-04-16T18:30:17.176 DEBUG:boto:Token: None 2024-04-16T18:30:17.176 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.176 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.183 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086f8b8e34a83f09a-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.183 DEBUG:boto:b'lybtug-52obj381000falseobj382024-04-16T18:22:20.888Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.184 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:30:17.184 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.184 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.184 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.184 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.184 DEBUG:boto:Method: GET 2024-04-16T18:30:17.184 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.185 DEBUG:boto:Data: 2024-04-16T18:30:17.185 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.185 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.185 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.185 DEBUG:boto:Params: {} 2024-04-16T18:30:17.185 DEBUG:boto:Token: None 2024-04-16T18:30:17.185 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.185 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc= 2024-04-16T18:30:17.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc=', 'Content-Length': '0'} 2024-04-16T18:30:17.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1d6f4d90771290b-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.230 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.230 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.231 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.231 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.231 DEBUG:boto:Method: GET 2024-04-16T18:30:17.231 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.231 DEBUG:boto:Data: 2024-04-16T18:30:17.231 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.231 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.231 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.231 DEBUG:boto:Params: {} 2024-04-16T18:30:17.231 DEBUG:boto:Token: None 2024-04-16T18:30:17.232 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.232 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc= 2024-04-16T18:30:17.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc=', 'Content-Length': '0'} 2024-04-16T18:30:17.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3c7d00c173afca7-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.235 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.235 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.235 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.235 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.235 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.236 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.236 DEBUG:boto:Data: 2024-04-16T18:30:17.236 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.236 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.236 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.236 DEBUG:boto:Params: {} 2024-04-16T18:30:17.236 DEBUG:boto:Token: None 2024-04-16T18:30:17.236 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.236 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3tT0QiOKpGoU9z1VxqlbPVCihpI= 2024-04-16T18:30:17.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3tT0QiOKpGoU9z1VxqlbPVCihpI=', 'Content-Length': '0'} 2024-04-16T18:30:17.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078928c55d82e56c7-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.239 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.240 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.240 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.240 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.240 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.240 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.240 DEBUG:boto:Data: 2024-04-16T18:30:17.240 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.240 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.240 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.240 DEBUG:boto:Params: {} 2024-04-16T18:30:17.240 DEBUG:boto:Token: None 2024-04-16T18:30:17.241 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.241 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:3tT0QiOKpGoU9z1VxqlbPVCihpI= 2024-04-16T18:30:17.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3tT0QiOKpGoU9z1VxqlbPVCihpI=', 'Content-Length': '0'} 2024-04-16T18:30:17.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080930113e40d998f-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.244 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:30:17.244 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.244 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.245 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.245 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.245 DEBUG:boto:Method: GET 2024-04-16T18:30:17.245 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.245 DEBUG:boto:Data: 2024-04-16T18:30:17.245 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.245 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.245 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.245 DEBUG:boto:Params: {} 2024-04-16T18:30:17.245 DEBUG:boto:Token: None 2024-04-16T18:30:17.246 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.246 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc= 2024-04-16T18:30:17.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc=', 'Content-Length': '0'} 2024-04-16T18:30:17.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006144ee6ee3f12374-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.290 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.290 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.291 DEBUG:boto:path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.291 DEBUG:boto:auth_path=/lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.291 DEBUG:boto:Method: GET 2024-04-16T18:30:17.291 DEBUG:boto:Path: /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.291 DEBUG:boto:Data: 2024-04-16T18:30:17.291 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.291 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.291 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.291 DEBUG:boto:Params: {} 2024-04-16T18:30:17.291 DEBUG:boto:Token: None 2024-04-16T18:30:17.292 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44?versionId=null 2024-04-16T18:30:17.292 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc= 2024-04-16T18:30:17.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hmYjbvr+r5lmzaKUNw/+C0LiGPc=', 'Content-Length': '0'} 2024-04-16T18:30:17.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1a57dba692f38c2-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.295 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.295 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.295 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.295 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.295 DEBUG:boto:Method: GET 2024-04-16T18:30:17.295 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:30:17.296 DEBUG:boto:Data: 2024-04-16T18:30:17.296 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.296 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.296 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.296 DEBUG:boto:Params: {} 2024-04-16T18:30:17.296 DEBUG:boto:Token: None 2024-04-16T18:30:17.296 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.296 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd6d163433e86a3b-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.300 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.300 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.301 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.301 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.301 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.301 DEBUG:boto:Method: GET 2024-04-16T18:30:17.301 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:30:17.301 DEBUG:boto:Data: 2024-04-16T18:30:17.301 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.301 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.301 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.301 DEBUG:boto:Params: {} 2024-04-16T18:30:17.301 DEBUG:boto:Token: None 2024-04-16T18:30:17.302 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.302 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2445c2462ebc209-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.350 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.351 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.351 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.351 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.351 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:30:17.351 DEBUG:boto:Data: 2024-04-16T18:30:17.351 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.351 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.352 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.352 DEBUG:boto:Params: {} 2024-04-16T18:30:17.352 DEBUG:boto:Token: None 2024-04-16T18:30:17.352 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44 2024-04-16T18:30:17.352 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/6F6ytTGuFXmBFkChnQdnNASi4c= 2024-04-16T18:30:17.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/6F6ytTGuFXmBFkChnQdnNASi4c=', 'Content-Length': '0'} 2024-04-16T18:30:17.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008124630773338507-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.356 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.356 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.356 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.356 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:30:17.356 DEBUG:boto:Data: 2024-04-16T18:30:17.356 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.356 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.357 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.357 DEBUG:boto:Params: {} 2024-04-16T18:30:17.357 DEBUG:boto:Token: None 2024-04-16T18:30:17.357 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44 2024-04-16T18:30:17.357 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/6F6ytTGuFXmBFkChnQdnNASi4c= 2024-04-16T18:30:17.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/6F6ytTGuFXmBFkChnQdnNASi4c=', 'Content-Length': '0'} 2024-04-16T18:30:17.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000537f0366217589b9-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.360 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2024-04-16T18:30:17.360 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.360 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.360 DEBUG:boto:Method: GET 2024-04-16T18:30:17.361 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:30:17.361 DEBUG:boto:Data: 2024-04-16T18:30:17.361 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.361 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.361 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.361 DEBUG:boto:Params: {} 2024-04-16T18:30:17.361 DEBUG:boto:Token: None 2024-04-16T18:30:17.361 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44 2024-04-16T18:30:17.361 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bfjTvOz24qvw+3U0dU78n6mBvko= 2024-04-16T18:30:17.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bfjTvOz24qvw+3U0dU78n6mBvko=', 'Content-Length': '0'} 2024-04-16T18:30:17.365 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001a6d854daee0944-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.406 DEBUG:boto:path=/lybtug-52/obj44 2024-04-16T18:30:17.406 DEBUG:boto:auth_path=/lybtug-52/obj44 2024-04-16T18:30:17.407 DEBUG:boto:Method: GET 2024-04-16T18:30:17.407 DEBUG:boto:Path: /lybtug-52/obj44 2024-04-16T18:30:17.407 DEBUG:boto:Data: 2024-04-16T18:30:17.407 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.407 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.407 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.407 DEBUG:boto:Params: {} 2024-04-16T18:30:17.407 DEBUG:boto:Token: None 2024-04-16T18:30:17.407 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj44 2024-04-16T18:30:17.407 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:bfjTvOz24qvw+3U0dU78n6mBvko= 2024-04-16T18:30:17.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bfjTvOz24qvw+3U0dU78n6mBvko=', 'Content-Length': '0'} 2024-04-16T18:30:17.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bf104e0e92aea59-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.411 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.411 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.411 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.411 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.411 DEBUG:boto:Method: GET 2024-04-16T18:30:17.412 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:30:17.412 DEBUG:boto:Data: 2024-04-16T18:30:17.412 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.412 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.412 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.412 DEBUG:boto:Params: {} 2024-04-16T18:30:17.412 DEBUG:boto:Token: None 2024-04-16T18:30:17.412 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.416 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068c9d8e59457d691-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.458 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.459 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.459 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.459 DEBUG:boto:path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.459 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj44 2024-04-16T18:30:17.459 DEBUG:boto:Method: GET 2024-04-16T18:30:17.459 DEBUG:boto:Path: /lybtug-52/?prefix=obj44 2024-04-16T18:30:17.459 DEBUG:boto:Data: 2024-04-16T18:30:17.459 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.459 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.459 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.459 DEBUG:boto:Params: {} 2024-04-16T18:30:17.460 DEBUG:boto:Token: None 2024-04-16T18:30:17.460 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.460 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.466 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002674a4454dc775af-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.467 DEBUG:boto:b'lybtug-52obj441000falseobj442024-04-16T18:20:00.328Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:30:17.467 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.467 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.467 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.467 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.467 DEBUG:boto:Method: GET 2024-04-16T18:30:17.468 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.468 DEBUG:boto:Data: 2024-04-16T18:30:17.468 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.468 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.468 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.468 DEBUG:boto:Params: {} 2024-04-16T18:30:17.468 DEBUG:boto:Token: None 2024-04-16T18:30:17.468 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.468 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y= 2024-04-16T18:30:17.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y=', 'Content-Length': '0'} 2024-04-16T18:30:17.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002da8f3b2a78b7b3a-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.514 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.514 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.515 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.515 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.515 DEBUG:boto:Method: GET 2024-04-16T18:30:17.515 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.515 DEBUG:boto:Data: 2024-04-16T18:30:17.515 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.515 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.515 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.515 DEBUG:boto:Params: {} 2024-04-16T18:30:17.515 DEBUG:boto:Token: None 2024-04-16T18:30:17.516 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.516 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y= 2024-04-16T18:30:17.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y=', 'Content-Length': '0'} 2024-04-16T18:30:17.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000160df80805f62295-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.519 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.519 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.519 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.519 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.519 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.519 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.520 DEBUG:boto:Data: 2024-04-16T18:30:17.520 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.520 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.520 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.520 DEBUG:boto:Params: {} 2024-04-16T18:30:17.520 DEBUG:boto:Token: None 2024-04-16T18:30:17.520 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.520 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n1afjuUszQ3Za2PWzgMasV9NMIc= 2024-04-16T18:30:17.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n1afjuUszQ3Za2PWzgMasV9NMIc=', 'Content-Length': '0'} 2024-04-16T18:30:17.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000080df141f67cccfd-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.523 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.524 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.524 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.524 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.524 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.524 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.524 DEBUG:boto:Data: 2024-04-16T18:30:17.524 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.524 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.524 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.524 DEBUG:boto:Params: {} 2024-04-16T18:30:17.525 DEBUG:boto:Token: None 2024-04-16T18:30:17.525 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.525 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:n1afjuUszQ3Za2PWzgMasV9NMIc= 2024-04-16T18:30:17.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:n1afjuUszQ3Za2PWzgMasV9NMIc=', 'Content-Length': '0'} 2024-04-16T18:30:17.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000432cb88c4d5e47f7-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.528 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:30:17.528 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.528 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.528 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.529 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.529 DEBUG:boto:Method: GET 2024-04-16T18:30:17.529 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.529 DEBUG:boto:Data: 2024-04-16T18:30:17.529 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.529 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.529 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.529 DEBUG:boto:Params: {} 2024-04-16T18:30:17.529 DEBUG:boto:Token: None 2024-04-16T18:30:17.529 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.529 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y= 2024-04-16T18:30:17.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y=', 'Content-Length': '0'} 2024-04-16T18:30:17.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3da7f904c7296f1-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.574 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.575 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.575 DEBUG:boto:path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.575 DEBUG:boto:auth_path=/lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.575 DEBUG:boto:Method: GET 2024-04-16T18:30:17.575 DEBUG:boto:Path: /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.575 DEBUG:boto:Data: 2024-04-16T18:30:17.575 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.575 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.575 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.576 DEBUG:boto:Params: {} 2024-04-16T18:30:17.576 DEBUG:boto:Token: None 2024-04-16T18:30:17.576 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46?versionId=null 2024-04-16T18:30:17.576 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y= 2024-04-16T18:30:17.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EDBKy2Jh6IhHy73zdjB/Mj8dk2Y=', 'Content-Length': '0'} 2024-04-16T18:30:17.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d9ae574323205ba-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.579 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.580 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.580 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.580 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.580 DEBUG:boto:Method: GET 2024-04-16T18:30:17.580 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:30:17.580 DEBUG:boto:Data: 2024-04-16T18:30:17.580 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.580 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.580 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.580 DEBUG:boto:Params: {} 2024-04-16T18:30:17.581 DEBUG:boto:Token: None 2024-04-16T18:30:17.581 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.581 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac6495424d76a918-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.584 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.585 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.585 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.585 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.585 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.585 DEBUG:boto:Method: GET 2024-04-16T18:30:17.585 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:30:17.585 DEBUG:boto:Data: 2024-04-16T18:30:17.585 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.586 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.586 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.586 DEBUG:boto:Params: {} 2024-04-16T18:30:17.586 DEBUG:boto:Token: None 2024-04-16T18:30:17.586 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.586 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004db86929d5203a75-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.634 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.635 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.635 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.635 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.635 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:30:17.635 DEBUG:boto:Data: 2024-04-16T18:30:17.635 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.635 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.635 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.635 DEBUG:boto:Params: {} 2024-04-16T18:30:17.635 DEBUG:boto:Token: None 2024-04-16T18:30:17.636 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46 2024-04-16T18:30:17.636 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vRVu7Bcm/sGJXSRNc64kbB8PbW0= 2024-04-16T18:30:17.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vRVu7Bcm/sGJXSRNc64kbB8PbW0=', 'Content-Length': '0'} 2024-04-16T18:30:17.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df2636070eb4ea0d-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.639 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.639 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.639 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.640 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:30:17.640 DEBUG:boto:Data: 2024-04-16T18:30:17.640 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.640 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.640 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.640 DEBUG:boto:Params: {} 2024-04-16T18:30:17.640 DEBUG:boto:Token: None 2024-04-16T18:30:17.640 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46 2024-04-16T18:30:17.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:vRVu7Bcm/sGJXSRNc64kbB8PbW0= 2024-04-16T18:30:17.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vRVu7Bcm/sGJXSRNc64kbB8PbW0=', 'Content-Length': '0'} 2024-04-16T18:30:17.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cd59742a2266830-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.644 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2024-04-16T18:30:17.644 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.644 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.644 DEBUG:boto:Method: GET 2024-04-16T18:30:17.644 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:30:17.644 DEBUG:boto:Data: 2024-04-16T18:30:17.644 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.644 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.645 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.645 DEBUG:boto:Params: {} 2024-04-16T18:30:17.645 DEBUG:boto:Token: None 2024-04-16T18:30:17.645 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46 2024-04-16T18:30:17.645 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8kczapVmQEjiAmn2XS4HyYOYlWE= 2024-04-16T18:30:17.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8kczapVmQEjiAmn2XS4HyYOYlWE=', 'Content-Length': '0'} 2024-04-16T18:30:17.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6c96bc74862b4ca-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.690 DEBUG:boto:path=/lybtug-52/obj46 2024-04-16T18:30:17.691 DEBUG:boto:auth_path=/lybtug-52/obj46 2024-04-16T18:30:17.691 DEBUG:boto:Method: GET 2024-04-16T18:30:17.691 DEBUG:boto:Path: /lybtug-52/obj46 2024-04-16T18:30:17.691 DEBUG:boto:Data: 2024-04-16T18:30:17.691 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.691 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.691 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.691 DEBUG:boto:Params: {} 2024-04-16T18:30:17.691 DEBUG:boto:Token: None 2024-04-16T18:30:17.692 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj46 2024-04-16T18:30:17.692 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:8kczapVmQEjiAmn2XS4HyYOYlWE= 2024-04-16T18:30:17.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8kczapVmQEjiAmn2XS4HyYOYlWE=', 'Content-Length': '0'} 2024-04-16T18:30:17.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4197d7f9a994b47-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.695 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.695 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.695 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.696 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.696 DEBUG:boto:Method: GET 2024-04-16T18:30:17.696 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:30:17.696 DEBUG:boto:Data: 2024-04-16T18:30:17.696 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.696 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.696 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.696 DEBUG:boto:Params: {} 2024-04-16T18:30:17.696 DEBUG:boto:Token: None 2024-04-16T18:30:17.696 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.697 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009404c5123a3bd238-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.742 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.743 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.743 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.743 DEBUG:boto:path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.743 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj46 2024-04-16T18:30:17.743 DEBUG:boto:Method: GET 2024-04-16T18:30:17.743 DEBUG:boto:Path: /lybtug-52/?prefix=obj46 2024-04-16T18:30:17.743 DEBUG:boto:Data: 2024-04-16T18:30:17.744 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.744 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.744 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.744 DEBUG:boto:Params: {} 2024-04-16T18:30:17.744 DEBUG:boto:Token: None 2024-04-16T18:30:17.744 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.744 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb68915faea0f32c-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.752 DEBUG:boto:b'lybtug-52obj461000falseobj462024-04-16T18:23:25.714Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:30:17.753 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.753 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.753 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.753 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.753 DEBUG:boto:Method: GET 2024-04-16T18:30:17.753 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.753 DEBUG:boto:Data: 2024-04-16T18:30:17.753 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.754 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.754 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.754 DEBUG:boto:Params: {} 2024-04-16T18:30:17.754 DEBUG:boto:Token: None 2024-04-16T18:30:17.754 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.754 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI= 2024-04-16T18:30:17.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI=', 'Content-Length': '0'} 2024-04-16T18:30:17.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069ff9ac5f017c434-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.798 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.799 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.799 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.799 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.799 DEBUG:boto:Method: GET 2024-04-16T18:30:17.799 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.799 DEBUG:boto:Data: 2024-04-16T18:30:17.799 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.799 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.799 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.799 DEBUG:boto:Params: {} 2024-04-16T18:30:17.799 DEBUG:boto:Token: None 2024-04-16T18:30:17.800 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.800 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI= 2024-04-16T18:30:17.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI=', 'Content-Length': '0'} 2024-04-16T18:30:17.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c95b5b59742a9366-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.803 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.804 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.804 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.804 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.804 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.804 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.804 DEBUG:boto:Data: 2024-04-16T18:30:17.804 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.804 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.804 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.804 DEBUG:boto:Params: {} 2024-04-16T18:30:17.804 DEBUG:boto:Token: None 2024-04-16T18:30:17.805 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.805 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D49bhwIONKnluNzXF0ByXdrfj3g= 2024-04-16T18:30:17.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D49bhwIONKnluNzXF0ByXdrfj3g=', 'Content-Length': '0'} 2024-04-16T18:30:17.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f269635f805462e4-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.808 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.808 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.808 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.808 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.808 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.808 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.809 DEBUG:boto:Data: 2024-04-16T18:30:17.809 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.809 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.809 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.809 DEBUG:boto:Params: {} 2024-04-16T18:30:17.809 DEBUG:boto:Token: None 2024-04-16T18:30:17.809 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.809 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:D49bhwIONKnluNzXF0ByXdrfj3g= 2024-04-16T18:30:17.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:D49bhwIONKnluNzXF0ByXdrfj3g=', 'Content-Length': '0'} 2024-04-16T18:30:17.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7ba8d10f22039be-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:30:17.813 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.813 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.813 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.813 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.813 DEBUG:boto:Method: GET 2024-04-16T18:30:17.813 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.813 DEBUG:boto:Data: 2024-04-16T18:30:17.813 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.813 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.814 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.814 DEBUG:boto:Params: {} 2024-04-16T18:30:17.814 DEBUG:boto:Token: None 2024-04-16T18:30:17.814 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.814 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI= 2024-04-16T18:30:17.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI=', 'Content-Length': '0'} 2024-04-16T18:30:17.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022152aca2f0462a5-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.858 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.858 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.859 DEBUG:boto:path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.859 DEBUG:boto:auth_path=/lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.859 DEBUG:boto:Method: GET 2024-04-16T18:30:17.859 DEBUG:boto:Path: /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.859 DEBUG:boto:Data: 2024-04-16T18:30:17.859 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.859 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.859 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.859 DEBUG:boto:Params: {} 2024-04-16T18:30:17.859 DEBUG:boto:Token: None 2024-04-16T18:30:17.860 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51?versionId=null 2024-04-16T18:30:17.860 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI= 2024-04-16T18:30:17.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B/97raJ979WQl9kBjWIPSKmXXDI=', 'Content-Length': '0'} 2024-04-16T18:30:17.863 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3adbb8ed736306c-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.863 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.863 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.863 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:17.863 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:17.863 DEBUG:boto:Method: GET 2024-04-16T18:30:17.864 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:30:17.864 DEBUG:boto:Data: 2024-04-16T18:30:17.864 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.864 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.864 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.864 DEBUG:boto:Params: {} 2024-04-16T18:30:17.864 DEBUG:boto:Token: None 2024-04-16T18:30:17.864 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.864 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de13095d6f4f01ce-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.868 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.869 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.869 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.869 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:17.869 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:17.869 DEBUG:boto:Method: GET 2024-04-16T18:30:17.869 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:30:17.869 DEBUG:boto:Data: 2024-04-16T18:30:17.869 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.869 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.870 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.870 DEBUG:boto:Params: {} 2024-04-16T18:30:17.870 DEBUG:boto:Token: None 2024-04-16T18:30:17.870 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.870 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.877 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d43b1f0319f8ea0c-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.918 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:17.919 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.919 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.919 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.919 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:30:17.919 DEBUG:boto:Data: 2024-04-16T18:30:17.919 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.919 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.919 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.920 DEBUG:boto:Params: {} 2024-04-16T18:30:17.920 DEBUG:boto:Token: None 2024-04-16T18:30:17.920 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51 2024-04-16T18:30:17.920 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g46NuPp+hhuDURVgDbAcwNUnalg= 2024-04-16T18:30:17.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g46NuPp+hhuDURVgDbAcwNUnalg=', 'Content-Length': '0'} 2024-04-16T18:30:17.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c98828436df98d38-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.923 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.923 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.923 DEBUG:boto:Method: HEAD 2024-04-16T18:30:17.923 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:30:17.923 DEBUG:boto:Data: 2024-04-16T18:30:17.924 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.924 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.924 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.924 DEBUG:boto:Params: {} 2024-04-16T18:30:17.924 DEBUG:boto:Token: None 2024-04-16T18:30:17.924 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51 2024-04-16T18:30:17.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:g46NuPp+hhuDURVgDbAcwNUnalg= 2024-04-16T18:30:17.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g46NuPp+hhuDURVgDbAcwNUnalg=', 'Content-Length': '0'} 2024-04-16T18:30:17.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3087274f3056797-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2024-04-16T18:30:17.928 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.928 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.928 DEBUG:boto:Method: GET 2024-04-16T18:30:17.928 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:30:17.928 DEBUG:boto:Data: 2024-04-16T18:30:17.929 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.929 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.929 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.929 DEBUG:boto:Params: {} 2024-04-16T18:30:17.929 DEBUG:boto:Token: None 2024-04-16T18:30:17.929 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51 2024-04-16T18:30:17.929 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jyNNDUy4D4wgTQw+sy6IHcGd2/U= 2024-04-16T18:30:17.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jyNNDUy4D4wgTQw+sy6IHcGd2/U=', 'Content-Length': '0'} 2024-04-16T18:30:17.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c5c565d72e53b74-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.974 DEBUG:boto:path=/lybtug-52/obj51 2024-04-16T18:30:17.974 DEBUG:boto:auth_path=/lybtug-52/obj51 2024-04-16T18:30:17.975 DEBUG:boto:Method: GET 2024-04-16T18:30:17.975 DEBUG:boto:Path: /lybtug-52/obj51 2024-04-16T18:30:17.975 DEBUG:boto:Data: 2024-04-16T18:30:17.975 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.975 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:17.975 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.975 DEBUG:boto:Params: {} 2024-04-16T18:30:17.975 DEBUG:boto:Token: None 2024-04-16T18:30:17.975 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/obj51 2024-04-16T18:30:17.975 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jyNNDUy4D4wgTQw+sy6IHcGd2/U= 2024-04-16T18:30:17.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jyNNDUy4D4wgTQw+sy6IHcGd2/U=', 'Content-Length': '0'} 2024-04-16T18:30:17.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019f3eb87d2090995-00661ec3b9-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:17.979 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:17.979 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:17.979 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:17.979 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:17.979 DEBUG:boto:Method: GET 2024-04-16T18:30:17.979 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:30:17.980 DEBUG:boto:Data: 2024-04-16T18:30:17.980 DEBUG:boto:Headers: {} 2024-04-16T18:30:17.980 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:17.980 DEBUG:boto:Port: 8000 2024-04-16T18:30:17.980 DEBUG:boto:Params: {} 2024-04-16T18:30:17.980 DEBUG:boto:Token: None 2024-04-16T18:30:17.980 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:17 GMT /lybtug-52/ 2024-04-16T18:30:17.980 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU= 2024-04-16T18:30:17.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nBuCd4192HEBplS4E9NJwsiHSQU=', 'Content-Length': '0'} 2024-04-16T18:30:17.984 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002908cd41dd94c877-00661ec3b9-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.026 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.027 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.027 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.027 DEBUG:boto:path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:18.027 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj51 2024-04-16T18:30:18.027 DEBUG:boto:Method: GET 2024-04-16T18:30:18.027 DEBUG:boto:Path: /lybtug-52/?prefix=obj51 2024-04-16T18:30:18.027 DEBUG:boto:Data: 2024-04-16T18:30:18.027 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.027 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.027 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.028 DEBUG:boto:Params: {} 2024-04-16T18:30:18.028 DEBUG:boto:Token: None 2024-04-16T18:30:18.028 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.028 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.035 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069240c4756b5959c-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.035 DEBUG:boto:b'lybtug-52obj511000falseobj512024-04-16T18:20:12.458Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.035 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:30:18.035 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.035 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.035 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.036 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.036 DEBUG:boto:Method: GET 2024-04-16T18:30:18.036 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.036 DEBUG:boto:Data: 2024-04-16T18:30:18.036 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.036 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.036 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.036 DEBUG:boto:Params: {} 2024-04-16T18:30:18.036 DEBUG:boto:Token: None 2024-04-16T18:30:18.036 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.037 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro= 2024-04-16T18:30:18.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro=', 'Content-Length': '0'} 2024-04-16T18:30:18.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033e18202e4b7d661-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.082 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.083 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.083 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.083 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.083 DEBUG:boto:Method: GET 2024-04-16T18:30:18.083 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.083 DEBUG:boto:Data: 2024-04-16T18:30:18.083 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.083 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.083 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.083 DEBUG:boto:Params: {} 2024-04-16T18:30:18.084 DEBUG:boto:Token: None 2024-04-16T18:30:18.084 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro= 2024-04-16T18:30:18.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro=', 'Content-Length': '0'} 2024-04-16T18:30:18.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f809da9ebdbc5f85-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.088 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.088 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.088 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.088 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.088 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.088 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.088 DEBUG:boto:Data: 2024-04-16T18:30:18.088 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.088 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.088 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.089 DEBUG:boto:Params: {} 2024-04-16T18:30:18.089 DEBUG:boto:Token: None 2024-04-16T18:30:18.089 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.089 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U/gzHK+goTSIdZ9tWV3YmO34Mmo= 2024-04-16T18:30:18.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U/gzHK+goTSIdZ9tWV3YmO34Mmo=', 'Content-Length': '0'} 2024-04-16T18:30:18.092 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ea14edf65ff995c-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.092 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.092 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.092 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.092 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.093 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.093 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.093 DEBUG:boto:Data: 2024-04-16T18:30:18.093 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.093 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.093 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.093 DEBUG:boto:Params: {} 2024-04-16T18:30:18.093 DEBUG:boto:Token: None 2024-04-16T18:30:18.093 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.094 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:U/gzHK+goTSIdZ9tWV3YmO34Mmo= 2024-04-16T18:30:18.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:U/gzHK+goTSIdZ9tWV3YmO34Mmo=', 'Content-Length': '0'} 2024-04-16T18:30:18.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec328e2c25616121-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:30:18.097 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.097 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.097 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.097 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.098 DEBUG:boto:Method: GET 2024-04-16T18:30:18.098 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.098 DEBUG:boto:Data: 2024-04-16T18:30:18.098 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.098 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.098 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.098 DEBUG:boto:Params: {} 2024-04-16T18:30:18.098 DEBUG:boto:Token: None 2024-04-16T18:30:18.098 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.099 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro= 2024-04-16T18:30:18.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro=', 'Content-Length': '0'} 2024-04-16T18:30:18.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ad9f4fc2b601ecf-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.142 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.142 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.143 DEBUG:boto:path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.143 DEBUG:boto:auth_path=/lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.143 DEBUG:boto:Method: GET 2024-04-16T18:30:18.143 DEBUG:boto:Path: /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.143 DEBUG:boto:Data: 2024-04-16T18:30:18.143 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.143 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.143 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.143 DEBUG:boto:Params: {} 2024-04-16T18:30:18.143 DEBUG:boto:Token: None 2024-04-16T18:30:18.144 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55?versionId=null 2024-04-16T18:30:18.144 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro= 2024-04-16T18:30:18.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cekzSvGPdpfBsf2J+ZDea77sbro=', 'Content-Length': '0'} 2024-04-16T18:30:18.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfb4adddff3641b6-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.147 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.148 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.148 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.148 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.148 DEBUG:boto:Method: GET 2024-04-16T18:30:18.148 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:30:18.148 DEBUG:boto:Data: 2024-04-16T18:30:18.148 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.148 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.148 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.148 DEBUG:boto:Params: {} 2024-04-16T18:30:18.149 DEBUG:boto:Token: None 2024-04-16T18:30:18.149 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.149 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051d5175e328f35b4-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.153 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.153 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.153 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.154 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.154 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.154 DEBUG:boto:Method: GET 2024-04-16T18:30:18.154 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:30:18.154 DEBUG:boto:Data: 2024-04-16T18:30:18.154 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.154 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.154 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.154 DEBUG:boto:Params: {} 2024-04-16T18:30:18.155 DEBUG:boto:Token: None 2024-04-16T18:30:18.155 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.155 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.162 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000623a04ca153a6d63-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.202 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.203 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.203 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.203 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.203 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:30:18.203 DEBUG:boto:Data: 2024-04-16T18:30:18.203 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.203 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.203 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.203 DEBUG:boto:Params: {} 2024-04-16T18:30:18.203 DEBUG:boto:Token: None 2024-04-16T18:30:18.204 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55 2024-04-16T18:30:18.204 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dqJhACPAIqqNM+vo/bPqO9HhR+0= 2024-04-16T18:30:18.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dqJhACPAIqqNM+vo/bPqO9HhR+0=', 'Content-Length': '0'} 2024-04-16T18:30:18.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7040f219aede0cf-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.207 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.207 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.207 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.207 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:30:18.207 DEBUG:boto:Data: 2024-04-16T18:30:18.207 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.207 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.207 DEBUG:boto:Params: {} 2024-04-16T18:30:18.208 DEBUG:boto:Token: None 2024-04-16T18:30:18.208 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55 2024-04-16T18:30:18.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dqJhACPAIqqNM+vo/bPqO9HhR+0= 2024-04-16T18:30:18.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dqJhACPAIqqNM+vo/bPqO9HhR+0=', 'Content-Length': '0'} 2024-04-16T18:30:18.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d0f80a1d54f7ae1-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.211 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2024-04-16T18:30:18.211 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.211 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.211 DEBUG:boto:Method: GET 2024-04-16T18:30:18.211 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:30:18.211 DEBUG:boto:Data: 2024-04-16T18:30:18.211 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.212 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.212 DEBUG:boto:Params: {} 2024-04-16T18:30:18.212 DEBUG:boto:Token: None 2024-04-16T18:30:18.212 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55 2024-04-16T18:30:18.212 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZvQ2AW0RuyiDXpNhbuLYaWIPg8s= 2024-04-16T18:30:18.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZvQ2AW0RuyiDXpNhbuLYaWIPg8s=', 'Content-Length': '0'} 2024-04-16T18:30:18.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003533c7e0e040a3e8-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.258 DEBUG:boto:path=/lybtug-52/obj55 2024-04-16T18:30:18.259 DEBUG:boto:auth_path=/lybtug-52/obj55 2024-04-16T18:30:18.259 DEBUG:boto:Method: GET 2024-04-16T18:30:18.259 DEBUG:boto:Path: /lybtug-52/obj55 2024-04-16T18:30:18.259 DEBUG:boto:Data: 2024-04-16T18:30:18.259 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.259 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.259 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.259 DEBUG:boto:Params: {} 2024-04-16T18:30:18.259 DEBUG:boto:Token: None 2024-04-16T18:30:18.259 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj55 2024-04-16T18:30:18.260 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZvQ2AW0RuyiDXpNhbuLYaWIPg8s= 2024-04-16T18:30:18.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZvQ2AW0RuyiDXpNhbuLYaWIPg8s=', 'Content-Length': '0'} 2024-04-16T18:30:18.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:00 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbfa37d360e9323c-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.263 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.263 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.263 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.263 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.264 DEBUG:boto:Method: GET 2024-04-16T18:30:18.264 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:30:18.264 DEBUG:boto:Data: 2024-04-16T18:30:18.264 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.264 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.264 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.264 DEBUG:boto:Params: {} 2024-04-16T18:30:18.264 DEBUG:boto:Token: None 2024-04-16T18:30:18.264 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.264 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c335a364e030a5d-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.310 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.311 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.311 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.311 DEBUG:boto:path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.311 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj55 2024-04-16T18:30:18.311 DEBUG:boto:Method: GET 2024-04-16T18:30:18.311 DEBUG:boto:Path: /lybtug-52/?prefix=obj55 2024-04-16T18:30:18.311 DEBUG:boto:Data: 2024-04-16T18:30:18.311 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.311 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.311 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.311 DEBUG:boto:Params: {} 2024-04-16T18:30:18.312 DEBUG:boto:Token: None 2024-04-16T18:30:18.312 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.312 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fba363b8d94652ba-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.319 DEBUG:boto:b'lybtug-52obj551000falseobj552024-04-16T18:20:00.377Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.319 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:30:18.319 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.319 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.319 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.319 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.319 DEBUG:boto:Method: GET 2024-04-16T18:30:18.320 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.320 DEBUG:boto:Data: 2024-04-16T18:30:18.320 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.320 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.320 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.320 DEBUG:boto:Params: {} 2024-04-16T18:30:18.320 DEBUG:boto:Token: None 2024-04-16T18:30:18.320 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.320 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ= 2024-04-16T18:30:18.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ=', 'Content-Length': '0'} 2024-04-16T18:30:18.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000928f291f1f6f21d5-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.366 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.366 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.367 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.367 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.367 DEBUG:boto:Method: GET 2024-04-16T18:30:18.367 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.367 DEBUG:boto:Data: 2024-04-16T18:30:18.367 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.367 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.367 DEBUG:boto:Params: {} 2024-04-16T18:30:18.367 DEBUG:boto:Token: None 2024-04-16T18:30:18.367 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.368 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ= 2024-04-16T18:30:18.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ=', 'Content-Length': '0'} 2024-04-16T18:30:18.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd380aca9fbc7315-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.371 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.371 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.371 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.371 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.371 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.371 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.371 DEBUG:boto:Data: 2024-04-16T18:30:18.371 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.372 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.372 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.372 DEBUG:boto:Params: {} 2024-04-16T18:30:18.372 DEBUG:boto:Token: None 2024-04-16T18:30:18.372 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.372 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CJhgo1ZKW2RmbHTPlecLG5xiesQ= 2024-04-16T18:30:18.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CJhgo1ZKW2RmbHTPlecLG5xiesQ=', 'Content-Length': '0'} 2024-04-16T18:30:18.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000913ef259b2383ae5-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.376 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.376 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.376 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.376 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.376 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.376 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.376 DEBUG:boto:Data: 2024-04-16T18:30:18.376 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.376 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.377 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.377 DEBUG:boto:Params: {} 2024-04-16T18:30:18.377 DEBUG:boto:Token: None 2024-04-16T18:30:18.377 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.377 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CJhgo1ZKW2RmbHTPlecLG5xiesQ= 2024-04-16T18:30:18.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CJhgo1ZKW2RmbHTPlecLG5xiesQ=', 'Content-Length': '0'} 2024-04-16T18:30:18.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c62db91eaf0e59f-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:30:18.380 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.380 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.380 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.380 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.380 DEBUG:boto:Method: GET 2024-04-16T18:30:18.380 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.380 DEBUG:boto:Data: 2024-04-16T18:30:18.381 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.381 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.381 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.381 DEBUG:boto:Params: {} 2024-04-16T18:30:18.381 DEBUG:boto:Token: None 2024-04-16T18:30:18.381 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.381 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ= 2024-04-16T18:30:18.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ=', 'Content-Length': '0'} 2024-04-16T18:30:18.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001999a9bd0847238a-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.426 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.427 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.427 DEBUG:boto:path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.427 DEBUG:boto:auth_path=/lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.427 DEBUG:boto:Method: GET 2024-04-16T18:30:18.427 DEBUG:boto:Path: /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.427 DEBUG:boto:Data: 2024-04-16T18:30:18.427 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.427 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.427 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.427 DEBUG:boto:Params: {} 2024-04-16T18:30:18.427 DEBUG:boto:Token: None 2024-04-16T18:30:18.428 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57?versionId=null 2024-04-16T18:30:18.428 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ= 2024-04-16T18:30:18.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9kKJlLw/X3hFY8m8tfDdqPpH4LQ=', 'Content-Length': '0'} 2024-04-16T18:30:18.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fad85c0f39aa5e6b-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.431 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.431 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.431 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.431 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.431 DEBUG:boto:Method: GET 2024-04-16T18:30:18.431 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:30:18.431 DEBUG:boto:Data: 2024-04-16T18:30:18.432 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.432 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.432 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.432 DEBUG:boto:Params: {} 2024-04-16T18:30:18.432 DEBUG:boto:Token: None 2024-04-16T18:30:18.432 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.432 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be2fc3066a9d83a6-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.436 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.437 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.437 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.437 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.437 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.437 DEBUG:boto:Method: GET 2024-04-16T18:30:18.437 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:30:18.437 DEBUG:boto:Data: 2024-04-16T18:30:18.437 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.437 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.437 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.437 DEBUG:boto:Params: {} 2024-04-16T18:30:18.438 DEBUG:boto:Token: None 2024-04-16T18:30:18.438 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.438 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e2b2c0e1007f15a-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.486 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.487 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.487 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.487 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.487 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:30:18.487 DEBUG:boto:Data: 2024-04-16T18:30:18.487 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.487 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.487 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.487 DEBUG:boto:Params: {} 2024-04-16T18:30:18.487 DEBUG:boto:Token: None 2024-04-16T18:30:18.488 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57 2024-04-16T18:30:18.488 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j3JxlYzBF8Hj/z3H3JLJ8KlQukA= 2024-04-16T18:30:18.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j3JxlYzBF8Hj/z3H3JLJ8KlQukA=', 'Content-Length': '0'} 2024-04-16T18:30:18.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035e4f5bebdf080ca-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.491 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.491 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.491 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.491 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:30:18.491 DEBUG:boto:Data: 2024-04-16T18:30:18.492 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.492 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.492 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.492 DEBUG:boto:Params: {} 2024-04-16T18:30:18.492 DEBUG:boto:Token: None 2024-04-16T18:30:18.492 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57 2024-04-16T18:30:18.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:j3JxlYzBF8Hj/z3H3JLJ8KlQukA= 2024-04-16T18:30:18.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:j3JxlYzBF8Hj/z3H3JLJ8KlQukA=', 'Content-Length': '0'} 2024-04-16T18:30:18.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e05aebc1e0efe2b2-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2024-04-16T18:30:18.496 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.496 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.496 DEBUG:boto:Method: GET 2024-04-16T18:30:18.496 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:30:18.496 DEBUG:boto:Data: 2024-04-16T18:30:18.496 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.496 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.496 DEBUG:boto:Params: {} 2024-04-16T18:30:18.496 DEBUG:boto:Token: None 2024-04-16T18:30:18.496 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57 2024-04-16T18:30:18.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jRvFo1oz83O9RJzCKnwIagdrpT0= 2024-04-16T18:30:18.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jRvFo1oz83O9RJzCKnwIagdrpT0=', 'Content-Length': '0'} 2024-04-16T18:30:18.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c00ba76afa0d39c2-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.542 DEBUG:boto:path=/lybtug-52/obj57 2024-04-16T18:30:18.542 DEBUG:boto:auth_path=/lybtug-52/obj57 2024-04-16T18:30:18.543 DEBUG:boto:Method: GET 2024-04-16T18:30:18.543 DEBUG:boto:Path: /lybtug-52/obj57 2024-04-16T18:30:18.543 DEBUG:boto:Data: 2024-04-16T18:30:18.543 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.543 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.543 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.543 DEBUG:boto:Params: {} 2024-04-16T18:30:18.543 DEBUG:boto:Token: None 2024-04-16T18:30:18.543 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj57 2024-04-16T18:30:18.543 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:jRvFo1oz83O9RJzCKnwIagdrpT0= 2024-04-16T18:30:18.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jRvFo1oz83O9RJzCKnwIagdrpT0=', 'Content-Length': '0'} 2024-04-16T18:30:18.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c885480c2a731ad-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.547 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.547 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.547 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.547 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.547 DEBUG:boto:Method: GET 2024-04-16T18:30:18.547 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:30:18.547 DEBUG:boto:Data: 2024-04-16T18:30:18.547 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.547 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.547 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.547 DEBUG:boto:Params: {} 2024-04-16T18:30:18.548 DEBUG:boto:Token: None 2024-04-16T18:30:18.548 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.548 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.552 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abd08cb06249b9f2-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.594 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.595 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.595 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.595 DEBUG:boto:path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.595 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj57 2024-04-16T18:30:18.595 DEBUG:boto:Method: GET 2024-04-16T18:30:18.595 DEBUG:boto:Path: /lybtug-52/?prefix=obj57 2024-04-16T18:30:18.595 DEBUG:boto:Data: 2024-04-16T18:30:18.595 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.595 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.595 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.595 DEBUG:boto:Params: {} 2024-04-16T18:30:18.596 DEBUG:boto:Token: None 2024-04-16T18:30:18.596 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.596 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.603 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000214634028ae8f5a6-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.604 DEBUG:boto:b'lybtug-52obj571000falseobj572024-04-16T18:22:20.901Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:30:18.604 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.604 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.604 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.604 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.604 DEBUG:boto:Method: GET 2024-04-16T18:30:18.604 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.605 DEBUG:boto:Data: 2024-04-16T18:30:18.605 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.605 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.605 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.605 DEBUG:boto:Params: {} 2024-04-16T18:30:18.605 DEBUG:boto:Token: None 2024-04-16T18:30:18.605 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.605 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM= 2024-04-16T18:30:18.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM=', 'Content-Length': '0'} 2024-04-16T18:30:18.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e62d943ddc20111-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.650 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.650 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.651 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.651 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.651 DEBUG:boto:Method: GET 2024-04-16T18:30:18.651 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.651 DEBUG:boto:Data: 2024-04-16T18:30:18.651 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.651 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.651 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.651 DEBUG:boto:Params: {} 2024-04-16T18:30:18.651 DEBUG:boto:Token: None 2024-04-16T18:30:18.651 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.652 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM= 2024-04-16T18:30:18.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM=', 'Content-Length': '0'} 2024-04-16T18:30:18.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095da7e291fb9d81a-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.655 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.655 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.655 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.655 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.655 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.655 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.655 DEBUG:boto:Data: 2024-04-16T18:30:18.655 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.656 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.656 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.656 DEBUG:boto:Params: {} 2024-04-16T18:30:18.656 DEBUG:boto:Token: None 2024-04-16T18:30:18.656 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.656 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Vz0M4GkX80lfKuhuA1DIoLnr9s= 2024-04-16T18:30:18.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Vz0M4GkX80lfKuhuA1DIoLnr9s=', 'Content-Length': '0'} 2024-04-16T18:30:18.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000792e54473c86b7b6-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.659 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.659 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.659 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.660 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.660 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.660 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.660 DEBUG:boto:Data: 2024-04-16T18:30:18.660 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.660 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.660 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.660 DEBUG:boto:Params: {} 2024-04-16T18:30:18.660 DEBUG:boto:Token: None 2024-04-16T18:30:18.660 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.660 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6Vz0M4GkX80lfKuhuA1DIoLnr9s= 2024-04-16T18:30:18.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6Vz0M4GkX80lfKuhuA1DIoLnr9s=', 'Content-Length': '0'} 2024-04-16T18:30:18.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2981877947f9122-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.664 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:30:18.664 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.664 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.664 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.664 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.664 DEBUG:boto:Method: GET 2024-04-16T18:30:18.664 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.664 DEBUG:boto:Data: 2024-04-16T18:30:18.664 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.664 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.664 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.665 DEBUG:boto:Params: {} 2024-04-16T18:30:18.665 DEBUG:boto:Token: None 2024-04-16T18:30:18.665 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.665 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM= 2024-04-16T18:30:18.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM=', 'Content-Length': '0'} 2024-04-16T18:30:18.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfc5b9683d0b35ab-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.710 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.710 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.710 DEBUG:boto:path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.711 DEBUG:boto:auth_path=/lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.711 DEBUG:boto:Method: GET 2024-04-16T18:30:18.711 DEBUG:boto:Path: /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.711 DEBUG:boto:Data: 2024-04-16T18:30:18.711 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.711 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.711 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.711 DEBUG:boto:Params: {} 2024-04-16T18:30:18.711 DEBUG:boto:Token: None 2024-04-16T18:30:18.711 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58?versionId=null 2024-04-16T18:30:18.712 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM= 2024-04-16T18:30:18.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CF+0nE2oT/bJRFCUJAlKfdeujDM=', 'Content-Length': '0'} 2024-04-16T18:30:18.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6749c60c9bd4d52-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.715 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.715 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.715 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.715 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.715 DEBUG:boto:Method: GET 2024-04-16T18:30:18.715 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:30:18.715 DEBUG:boto:Data: 2024-04-16T18:30:18.715 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.715 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.715 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.715 DEBUG:boto:Params: {} 2024-04-16T18:30:18.716 DEBUG:boto:Token: None 2024-04-16T18:30:18.716 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.716 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3fd4f6e26e20c33-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.720 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.720 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.720 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.720 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.720 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.720 DEBUG:boto:Method: GET 2024-04-16T18:30:18.720 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:30:18.720 DEBUG:boto:Data: 2024-04-16T18:30:18.721 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.721 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.721 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.721 DEBUG:boto:Params: {} 2024-04-16T18:30:18.721 DEBUG:boto:Token: None 2024-04-16T18:30:18.721 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.721 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.727 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011d2a56371860d31-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.770 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.771 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.771 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.771 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.771 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:30:18.771 DEBUG:boto:Data: 2024-04-16T18:30:18.771 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.771 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.771 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.772 DEBUG:boto:Params: {} 2024-04-16T18:30:18.772 DEBUG:boto:Token: None 2024-04-16T18:30:18.772 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58 2024-04-16T18:30:18.772 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vi/AdR3TnWPO5zDGDjElA53fO98= 2024-04-16T18:30:18.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vi/AdR3TnWPO5zDGDjElA53fO98=', 'Content-Length': '0'} 2024-04-16T18:30:18.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003353978c1cb72a69-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.775 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.776 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.776 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.776 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:30:18.776 DEBUG:boto:Data: 2024-04-16T18:30:18.776 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.776 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.776 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.776 DEBUG:boto:Params: {} 2024-04-16T18:30:18.776 DEBUG:boto:Token: None 2024-04-16T18:30:18.776 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58 2024-04-16T18:30:18.776 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Vi/AdR3TnWPO5zDGDjElA53fO98= 2024-04-16T18:30:18.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Vi/AdR3TnWPO5zDGDjElA53fO98=', 'Content-Length': '0'} 2024-04-16T18:30:18.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9452e2e14a4423c-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.780 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2024-04-16T18:30:18.780 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.780 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.780 DEBUG:boto:Method: GET 2024-04-16T18:30:18.780 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:30:18.780 DEBUG:boto:Data: 2024-04-16T18:30:18.780 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.780 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.780 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.781 DEBUG:boto:Params: {} 2024-04-16T18:30:18.781 DEBUG:boto:Token: None 2024-04-16T18:30:18.781 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58 2024-04-16T18:30:18.781 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sNudz9ESjeZb3eurJFEH0Ty3hns= 2024-04-16T18:30:18.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sNudz9ESjeZb3eurJFEH0Ty3hns=', 'Content-Length': '0'} 2024-04-16T18:30:18.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003264181d4946c803-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.826 DEBUG:boto:path=/lybtug-52/obj58 2024-04-16T18:30:18.826 DEBUG:boto:auth_path=/lybtug-52/obj58 2024-04-16T18:30:18.827 DEBUG:boto:Method: GET 2024-04-16T18:30:18.827 DEBUG:boto:Path: /lybtug-52/obj58 2024-04-16T18:30:18.827 DEBUG:boto:Data: 2024-04-16T18:30:18.827 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.827 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.827 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.827 DEBUG:boto:Params: {} 2024-04-16T18:30:18.827 DEBUG:boto:Token: None 2024-04-16T18:30:18.827 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj58 2024-04-16T18:30:18.827 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sNudz9ESjeZb3eurJFEH0Ty3hns= 2024-04-16T18:30:18.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sNudz9ESjeZb3eurJFEH0Ty3hns=', 'Content-Length': '0'} 2024-04-16T18:30:18.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee14c982f5240f36-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.831 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.831 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.831 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.831 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.831 DEBUG:boto:Method: GET 2024-04-16T18:30:18.831 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:30:18.831 DEBUG:boto:Data: 2024-04-16T18:30:18.831 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.831 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.831 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.832 DEBUG:boto:Params: {} 2024-04-16T18:30:18.832 DEBUG:boto:Token: None 2024-04-16T18:30:18.832 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.832 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eedca711f54a4859-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.878 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.879 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.879 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.879 DEBUG:boto:path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.879 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj58 2024-04-16T18:30:18.879 DEBUG:boto:Method: GET 2024-04-16T18:30:18.879 DEBUG:boto:Path: /lybtug-52/?prefix=obj58 2024-04-16T18:30:18.879 DEBUG:boto:Data: 2024-04-16T18:30:18.879 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.880 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.880 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.880 DEBUG:boto:Params: {} 2024-04-16T18:30:18.880 DEBUG:boto:Token: None 2024-04-16T18:30:18.880 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/ 2024-04-16T18:30:18.880 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI= 2024-04-16T18:30:18.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aXTCqem7l82h5VdjBHU+fU4L/kI=', 'Content-Length': '0'} 2024-04-16T18:30:18.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005663a885f80ca7f1-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.888 DEBUG:boto:b'lybtug-52obj581000falseobj582024-04-16T18:25:26.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:18.888 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:30:18.888 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:18.888 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:18.888 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.888 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.888 DEBUG:boto:Method: GET 2024-04-16T18:30:18.889 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.889 DEBUG:boto:Data: 2024-04-16T18:30:18.889 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.889 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.889 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.889 DEBUG:boto:Params: {} 2024-04-16T18:30:18.889 DEBUG:boto:Token: None 2024-04-16T18:30:18.889 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.889 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8= 2024-04-16T18:30:18.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8=', 'Content-Length': '0'} 2024-04-16T18:30:18.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006527d3039e7f7983-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.934 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:18.934 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:18.934 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.935 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.935 DEBUG:boto:Method: GET 2024-04-16T18:30:18.935 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.935 DEBUG:boto:Data: 2024-04-16T18:30:18.935 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.935 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.935 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.935 DEBUG:boto:Params: {} 2024-04-16T18:30:18.935 DEBUG:boto:Token: None 2024-04-16T18:30:18.935 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.935 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8= 2024-04-16T18:30:18.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8=', 'Content-Length': '0'} 2024-04-16T18:30:18.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084129af132ac6fad-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.939 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:18.939 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:18.939 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.939 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.939 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.939 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.939 DEBUG:boto:Data: 2024-04-16T18:30:18.939 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.939 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.939 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.939 DEBUG:boto:Params: {} 2024-04-16T18:30:18.940 DEBUG:boto:Token: None 2024-04-16T18:30:18.940 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.940 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CHVAybTJ3qAL2alpdlv4Ol+o/BE= 2024-04-16T18:30:18.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CHVAybTJ3qAL2alpdlv4Ol+o/BE=', 'Content-Length': '0'} 2024-04-16T18:30:18.942 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012d25ebf974bb6cf-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.943 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:18.943 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:18.943 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.943 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.943 DEBUG:boto:Method: HEAD 2024-04-16T18:30:18.943 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.943 DEBUG:boto:Data: 2024-04-16T18:30:18.943 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.943 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.943 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.944 DEBUG:boto:Params: {} 2024-04-16T18:30:18.944 DEBUG:boto:Token: None 2024-04-16T18:30:18.944 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.944 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:CHVAybTJ3qAL2alpdlv4Ol+o/BE= 2024-04-16T18:30:18.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CHVAybTJ3qAL2alpdlv4Ol+o/BE=', 'Content-Length': '0'} 2024-04-16T18:30:18.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e18ef24b5952242d-00661ec3ba-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:30:18.947 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:18.947 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:18.947 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.947 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.947 DEBUG:boto:Method: GET 2024-04-16T18:30:18.947 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.947 DEBUG:boto:Data: 2024-04-16T18:30:18.947 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.948 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:18.948 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.948 DEBUG:boto:Params: {} 2024-04-16T18:30:18.948 DEBUG:boto:Token: None 2024-04-16T18:30:18.948 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.948 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8= 2024-04-16T18:30:18.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8=', 'Content-Length': '0'} 2024-04-16T18:30:18.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2662d0cba991ee1-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.994 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:18.994 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:18.994 DEBUG:boto:path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.995 DEBUG:boto:auth_path=/lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.995 DEBUG:boto:Method: GET 2024-04-16T18:30:18.995 DEBUG:boto:Path: /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.995 DEBUG:boto:Data: 2024-04-16T18:30:18.995 DEBUG:boto:Headers: {} 2024-04-16T18:30:18.995 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:18.995 DEBUG:boto:Port: 8000 2024-04-16T18:30:18.995 DEBUG:boto:Params: {} 2024-04-16T18:30:18.995 DEBUG:boto:Token: None 2024-04-16T18:30:18.995 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:18 GMT /lybtug-52/obj62?versionId=null 2024-04-16T18:30:18.995 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8= 2024-04-16T18:30:18.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Rql1ZdEdvHh53Z43GP/5kFH5tD8=', 'Content-Length': '0'} 2024-04-16T18:30:18.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bff1df3a4128487c-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:18.999 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:18.999 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:18.999 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:18.999 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:18.999 DEBUG:boto:Method: GET 2024-04-16T18:30:18.999 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:30:19.000 DEBUG:boto:Data: 2024-04-16T18:30:19.000 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.000 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.000 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.000 DEBUG:boto:Params: {} 2024-04-16T18:30:19.000 DEBUG:boto:Token: None 2024-04-16T18:30:19.000 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.000 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6da9d53b9b745a0-00661ec3ba-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.004 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.004 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.005 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.005 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:19.005 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:19.005 DEBUG:boto:Method: GET 2024-04-16T18:30:19.005 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:30:19.005 DEBUG:boto:Data: 2024-04-16T18:30:19.005 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.005 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.005 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.005 DEBUG:boto:Params: {} 2024-04-16T18:30:19.005 DEBUG:boto:Token: None 2024-04-16T18:30:19.006 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.006 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ed2cdcf2842c8e0-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.054 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.055 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:19.055 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:19.055 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.055 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:30:19.055 DEBUG:boto:Data: 2024-04-16T18:30:19.055 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.055 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.055 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.055 DEBUG:boto:Params: {} 2024-04-16T18:30:19.055 DEBUG:boto:Token: None 2024-04-16T18:30:19.056 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj62 2024-04-16T18:30:19.056 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I1aLvxcMrlqYrgvQnku0f58uE98= 2024-04-16T18:30:19.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I1aLvxcMrlqYrgvQnku0f58uE98=', 'Content-Length': '0'} 2024-04-16T18:30:19.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009826c9aa546af265-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.059 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:19.059 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:19.059 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.059 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:30:19.060 DEBUG:boto:Data: 2024-04-16T18:30:19.060 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.060 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.060 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.060 DEBUG:boto:Params: {} 2024-04-16T18:30:19.060 DEBUG:boto:Token: None 2024-04-16T18:30:19.060 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj62 2024-04-16T18:30:19.060 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:I1aLvxcMrlqYrgvQnku0f58uE98= 2024-04-16T18:30:19.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I1aLvxcMrlqYrgvQnku0f58uE98=', 'Content-Length': '0'} 2024-04-16T18:30:19.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f562204da4e8ccbc-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.063 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2024-04-16T18:30:19.063 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:19.063 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:19.063 DEBUG:boto:Method: GET 2024-04-16T18:30:19.064 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:30:19.064 DEBUG:boto:Data: 2024-04-16T18:30:19.064 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.064 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.064 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.064 DEBUG:boto:Params: {} 2024-04-16T18:30:19.064 DEBUG:boto:Token: None 2024-04-16T18:30:19.064 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj62 2024-04-16T18:30:19.064 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RxvyW7//2vWOgoTGBxJr+irY/Yw= 2024-04-16T18:30:19.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RxvyW7//2vWOgoTGBxJr+irY/Yw=', 'Content-Length': '0'} 2024-04-16T18:30:19.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088e2da992771d32d-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.110 DEBUG:boto:path=/lybtug-52/obj62 2024-04-16T18:30:19.110 DEBUG:boto:auth_path=/lybtug-52/obj62 2024-04-16T18:30:19.110 DEBUG:boto:Method: GET 2024-04-16T18:30:19.111 DEBUG:boto:Path: /lybtug-52/obj62 2024-04-16T18:30:19.111 DEBUG:boto:Data: 2024-04-16T18:30:19.111 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.111 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.111 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.111 DEBUG:boto:Params: {} 2024-04-16T18:30:19.111 DEBUG:boto:Token: None 2024-04-16T18:30:19.111 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj62 2024-04-16T18:30:19.111 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RxvyW7//2vWOgoTGBxJr+irY/Yw= 2024-04-16T18:30:19.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RxvyW7//2vWOgoTGBxJr+irY/Yw=', 'Content-Length': '0'} 2024-04-16T18:30:19.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a350278b398051a-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.115 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.115 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.115 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:19.115 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:19.115 DEBUG:boto:Method: GET 2024-04-16T18:30:19.115 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:30:19.115 DEBUG:boto:Data: 2024-04-16T18:30:19.115 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.115 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.115 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.115 DEBUG:boto:Params: {} 2024-04-16T18:30:19.115 DEBUG:boto:Token: None 2024-04-16T18:30:19.116 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.116 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5239578f1c8d68e-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.162 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.163 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.163 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.163 DEBUG:boto:path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:19.163 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj62 2024-04-16T18:30:19.163 DEBUG:boto:Method: GET 2024-04-16T18:30:19.163 DEBUG:boto:Path: /lybtug-52/?prefix=obj62 2024-04-16T18:30:19.163 DEBUG:boto:Data: 2024-04-16T18:30:19.163 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.163 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.163 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.163 DEBUG:boto:Params: {} 2024-04-16T18:30:19.164 DEBUG:boto:Token: None 2024-04-16T18:30:19.164 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.164 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.171 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b97f29f6313e1037-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.171 DEBUG:boto:b'lybtug-52obj621000falseobj622024-04-16T18:26:49.266Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.172 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:30:19.172 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.172 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.172 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.172 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.172 DEBUG:boto:Method: GET 2024-04-16T18:30:19.172 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.172 DEBUG:boto:Data: 2024-04-16T18:30:19.172 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.173 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.173 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.173 DEBUG:boto:Params: {} 2024-04-16T18:30:19.173 DEBUG:boto:Token: None 2024-04-16T18:30:19.173 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.173 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8= 2024-04-16T18:30:19.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8=', 'Content-Length': '0'} 2024-04-16T18:30:19.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a5dba56f573b506-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.218 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.219 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.219 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.219 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.219 DEBUG:boto:Method: GET 2024-04-16T18:30:19.219 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.219 DEBUG:boto:Data: 2024-04-16T18:30:19.219 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.219 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.219 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.219 DEBUG:boto:Params: {} 2024-04-16T18:30:19.219 DEBUG:boto:Token: None 2024-04-16T18:30:19.220 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.220 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8= 2024-04-16T18:30:19.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8=', 'Content-Length': '0'} 2024-04-16T18:30:19.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001133a9df68dccc66-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.224 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.224 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.224 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.224 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.224 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.224 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.224 DEBUG:boto:Data: 2024-04-16T18:30:19.224 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.224 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.224 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.224 DEBUG:boto:Params: {} 2024-04-16T18:30:19.224 DEBUG:boto:Token: None 2024-04-16T18:30:19.225 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.225 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o8VqDjvhKEZ0kx213TdHOhF8qKU= 2024-04-16T18:30:19.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o8VqDjvhKEZ0kx213TdHOhF8qKU=', 'Content-Length': '0'} 2024-04-16T18:30:19.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003be192a0bd3d8642-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.228 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.228 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.228 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.228 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.228 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.228 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.228 DEBUG:boto:Data: 2024-04-16T18:30:19.228 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.228 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.228 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.229 DEBUG:boto:Params: {} 2024-04-16T18:30:19.229 DEBUG:boto:Token: None 2024-04-16T18:30:19.229 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.229 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o8VqDjvhKEZ0kx213TdHOhF8qKU= 2024-04-16T18:30:19.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o8VqDjvhKEZ0kx213TdHOhF8qKU=', 'Content-Length': '0'} 2024-04-16T18:30:19.232 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021e6a524d32e101e-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.233 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:30:19.233 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.233 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.233 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.233 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.233 DEBUG:boto:Method: GET 2024-04-16T18:30:19.233 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.233 DEBUG:boto:Data: 2024-04-16T18:30:19.233 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.233 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.233 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.233 DEBUG:boto:Params: {} 2024-04-16T18:30:19.234 DEBUG:boto:Token: None 2024-04-16T18:30:19.234 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.234 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8= 2024-04-16T18:30:19.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8=', 'Content-Length': '0'} 2024-04-16T18:30:19.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019040d514dfce95e-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.278 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.278 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.279 DEBUG:boto:path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.279 DEBUG:boto:auth_path=/lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.279 DEBUG:boto:Method: GET 2024-04-16T18:30:19.279 DEBUG:boto:Path: /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.279 DEBUG:boto:Data: 2024-04-16T18:30:19.279 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.279 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.279 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.279 DEBUG:boto:Params: {} 2024-04-16T18:30:19.279 DEBUG:boto:Token: None 2024-04-16T18:30:19.279 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68?versionId=null 2024-04-16T18:30:19.280 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8= 2024-04-16T18:30:19.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2Fq3Nq4xv+MwsHuVJWIEDoRxn8=', 'Content-Length': '0'} 2024-04-16T18:30:19.283 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000573edb6c627552f3-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.283 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.283 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.283 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.283 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.283 DEBUG:boto:Method: GET 2024-04-16T18:30:19.283 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:30:19.283 DEBUG:boto:Data: 2024-04-16T18:30:19.283 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.284 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.284 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.284 DEBUG:boto:Params: {} 2024-04-16T18:30:19.284 DEBUG:boto:Token: None 2024-04-16T18:30:19.284 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.284 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001bf28d7fb285c551-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.288 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.288 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.288 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.288 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.288 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.288 DEBUG:boto:Method: GET 2024-04-16T18:30:19.288 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:30:19.289 DEBUG:boto:Data: 2024-04-16T18:30:19.289 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.289 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.289 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.289 DEBUG:boto:Params: {} 2024-04-16T18:30:19.289 DEBUG:boto:Token: None 2024-04-16T18:30:19.289 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.289 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa15ed4582c4f93f-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.338 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.338 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.339 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.339 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.339 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:30:19.339 DEBUG:boto:Data: 2024-04-16T18:30:19.339 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.339 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.339 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.339 DEBUG:boto:Params: {} 2024-04-16T18:30:19.339 DEBUG:boto:Token: None 2024-04-16T18:30:19.339 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68 2024-04-16T18:30:19.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zptg4mY3dWpWoz/Dcqw6aRDCOnE= 2024-04-16T18:30:19.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zptg4mY3dWpWoz/Dcqw6aRDCOnE=', 'Content-Length': '0'} 2024-04-16T18:30:19.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036c0f13b8f37e928-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.343 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.343 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.343 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.343 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:30:19.343 DEBUG:boto:Data: 2024-04-16T18:30:19.343 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.343 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.343 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.343 DEBUG:boto:Params: {} 2024-04-16T18:30:19.343 DEBUG:boto:Token: None 2024-04-16T18:30:19.344 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68 2024-04-16T18:30:19.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zptg4mY3dWpWoz/Dcqw6aRDCOnE= 2024-04-16T18:30:19.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zptg4mY3dWpWoz/Dcqw6aRDCOnE=', 'Content-Length': '0'} 2024-04-16T18:30:19.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090c25ae5ccb45883-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2024-04-16T18:30:19.348 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.348 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.348 DEBUG:boto:Method: GET 2024-04-16T18:30:19.348 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:30:19.348 DEBUG:boto:Data: 2024-04-16T18:30:19.348 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.348 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.348 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.348 DEBUG:boto:Params: {} 2024-04-16T18:30:19.349 DEBUG:boto:Token: None 2024-04-16T18:30:19.349 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68 2024-04-16T18:30:19.349 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yh47+ywKXcRLZFJ2ryun83lEqWs= 2024-04-16T18:30:19.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yh47+ywKXcRLZFJ2ryun83lEqWs=', 'Content-Length': '0'} 2024-04-16T18:30:19.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000156a2aac0486b155-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.394 DEBUG:boto:path=/lybtug-52/obj68 2024-04-16T18:30:19.394 DEBUG:boto:auth_path=/lybtug-52/obj68 2024-04-16T18:30:19.395 DEBUG:boto:Method: GET 2024-04-16T18:30:19.395 DEBUG:boto:Path: /lybtug-52/obj68 2024-04-16T18:30:19.395 DEBUG:boto:Data: 2024-04-16T18:30:19.395 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.395 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.395 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.395 DEBUG:boto:Params: {} 2024-04-16T18:30:19.395 DEBUG:boto:Token: None 2024-04-16T18:30:19.395 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj68 2024-04-16T18:30:19.396 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yh47+ywKXcRLZFJ2ryun83lEqWs= 2024-04-16T18:30:19.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yh47+ywKXcRLZFJ2ryun83lEqWs=', 'Content-Length': '0'} 2024-04-16T18:30:19.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001724506445a303f0-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.399 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.399 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.399 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.399 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.399 DEBUG:boto:Method: GET 2024-04-16T18:30:19.399 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:30:19.399 DEBUG:boto:Data: 2024-04-16T18:30:19.400 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.400 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.400 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.400 DEBUG:boto:Params: {} 2024-04-16T18:30:19.400 DEBUG:boto:Token: None 2024-04-16T18:30:19.400 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.400 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0ed725f862ca05d-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.446 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.447 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.447 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.447 DEBUG:boto:path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.447 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj68 2024-04-16T18:30:19.447 DEBUG:boto:Method: GET 2024-04-16T18:30:19.447 DEBUG:boto:Path: /lybtug-52/?prefix=obj68 2024-04-16T18:30:19.447 DEBUG:boto:Data: 2024-04-16T18:30:19.447 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.447 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.448 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.448 DEBUG:boto:Params: {} 2024-04-16T18:30:19.448 DEBUG:boto:Token: None 2024-04-16T18:30:19.448 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.448 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f45d526fe42cf12a-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.455 DEBUG:boto:b'lybtug-52obj681000falseobj682024-04-16T18:20:12.476Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.455 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:30:19.455 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.456 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.456 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.456 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.456 DEBUG:boto:Method: GET 2024-04-16T18:30:19.456 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.456 DEBUG:boto:Data: 2024-04-16T18:30:19.456 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.456 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.456 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.456 DEBUG:boto:Params: {} 2024-04-16T18:30:19.456 DEBUG:boto:Token: None 2024-04-16T18:30:19.457 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.457 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk= 2024-04-16T18:30:19.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk=', 'Content-Length': '0'} 2024-04-16T18:30:19.460 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e26bac508c56e2b-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.502 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.502 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.503 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.503 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.503 DEBUG:boto:Method: GET 2024-04-16T18:30:19.503 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.503 DEBUG:boto:Data: 2024-04-16T18:30:19.503 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.503 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.503 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.503 DEBUG:boto:Params: {} 2024-04-16T18:30:19.503 DEBUG:boto:Token: None 2024-04-16T18:30:19.503 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.504 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk= 2024-04-16T18:30:19.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk=', 'Content-Length': '0'} 2024-04-16T18:30:19.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a957bbc2d8796e8-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.507 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.507 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.507 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.507 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.507 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.507 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.508 DEBUG:boto:Data: 2024-04-16T18:30:19.508 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.508 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.508 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.508 DEBUG:boto:Params: {} 2024-04-16T18:30:19.508 DEBUG:boto:Token: None 2024-04-16T18:30:19.508 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.508 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2aWlrrs91HPClo+9lt7E6SbPgsU= 2024-04-16T18:30:19.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2aWlrrs91HPClo+9lt7E6SbPgsU=', 'Content-Length': '0'} 2024-04-16T18:30:19.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057689d464cb9ea22-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.512 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.512 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.512 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.512 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.512 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.512 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.512 DEBUG:boto:Data: 2024-04-16T18:30:19.512 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.512 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.512 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.512 DEBUG:boto:Params: {} 2024-04-16T18:30:19.513 DEBUG:boto:Token: None 2024-04-16T18:30:19.513 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.513 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2aWlrrs91HPClo+9lt7E6SbPgsU= 2024-04-16T18:30:19.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2aWlrrs91HPClo+9lt7E6SbPgsU=', 'Content-Length': '0'} 2024-04-16T18:30:19.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1219b2bc620cf8a-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.516 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:30:19.516 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.516 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.516 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.516 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.516 DEBUG:boto:Method: GET 2024-04-16T18:30:19.517 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.517 DEBUG:boto:Data: 2024-04-16T18:30:19.517 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.517 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.517 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.517 DEBUG:boto:Params: {} 2024-04-16T18:30:19.517 DEBUG:boto:Token: None 2024-04-16T18:30:19.517 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.517 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk= 2024-04-16T18:30:19.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk=', 'Content-Length': '0'} 2024-04-16T18:30:19.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec2afeb2c4c6bb7a-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.562 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.562 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.562 DEBUG:boto:path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.563 DEBUG:boto:auth_path=/lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.563 DEBUG:boto:Method: GET 2024-04-16T18:30:19.563 DEBUG:boto:Path: /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.563 DEBUG:boto:Data: 2024-04-16T18:30:19.563 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.563 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.563 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.563 DEBUG:boto:Params: {} 2024-04-16T18:30:19.563 DEBUG:boto:Token: None 2024-04-16T18:30:19.563 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69?versionId=null 2024-04-16T18:30:19.563 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk= 2024-04-16T18:30:19.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZzVHnBoHwhFUB4DoYXVPYk5o3Zk=', 'Content-Length': '0'} 2024-04-16T18:30:19.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d28fc4386f12e39-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.567 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.567 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.567 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.567 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.567 DEBUG:boto:Method: GET 2024-04-16T18:30:19.567 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:30:19.568 DEBUG:boto:Data: 2024-04-16T18:30:19.568 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.568 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.568 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.568 DEBUG:boto:Params: {} 2024-04-16T18:30:19.568 DEBUG:boto:Token: None 2024-04-16T18:30:19.568 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.568 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000191aa66c64ddfc43-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.572 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.572 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.572 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.572 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.572 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.572 DEBUG:boto:Method: GET 2024-04-16T18:30:19.573 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:30:19.573 DEBUG:boto:Data: 2024-04-16T18:30:19.573 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.573 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.573 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.573 DEBUG:boto:Params: {} 2024-04-16T18:30:19.573 DEBUG:boto:Token: None 2024-04-16T18:30:19.573 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.573 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005b730a264d73c56-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.622 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.623 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.623 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.623 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.623 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:30:19.623 DEBUG:boto:Data: 2024-04-16T18:30:19.623 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.623 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.623 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.623 DEBUG:boto:Params: {} 2024-04-16T18:30:19.623 DEBUG:boto:Token: None 2024-04-16T18:30:19.623 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69 2024-04-16T18:30:19.624 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2nQd8YS4BYrDIJysBURCD9qY7tU= 2024-04-16T18:30:19.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2nQd8YS4BYrDIJysBURCD9qY7tU=', 'Content-Length': '0'} 2024-04-16T18:30:19.626 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074b6f9015e517ee5-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.627 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.627 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.627 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.627 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:30:19.627 DEBUG:boto:Data: 2024-04-16T18:30:19.627 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.627 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.627 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.627 DEBUG:boto:Params: {} 2024-04-16T18:30:19.627 DEBUG:boto:Token: None 2024-04-16T18:30:19.628 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69 2024-04-16T18:30:19.628 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2nQd8YS4BYrDIJysBURCD9qY7tU= 2024-04-16T18:30:19.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2nQd8YS4BYrDIJysBURCD9qY7tU=', 'Content-Length': '0'} 2024-04-16T18:30:19.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f644766c386aa36-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2024-04-16T18:30:19.631 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.631 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.631 DEBUG:boto:Method: GET 2024-04-16T18:30:19.631 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:30:19.631 DEBUG:boto:Data: 2024-04-16T18:30:19.631 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.631 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.632 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.632 DEBUG:boto:Params: {} 2024-04-16T18:30:19.632 DEBUG:boto:Token: None 2024-04-16T18:30:19.632 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69 2024-04-16T18:30:19.632 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Zv9A/gGqE4UthlRxMsCLaT96OXg= 2024-04-16T18:30:19.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Zv9A/gGqE4UthlRxMsCLaT96OXg=', 'Content-Length': '0'} 2024-04-16T18:30:19.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044034ac7d8390086-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.678 DEBUG:boto:path=/lybtug-52/obj69 2024-04-16T18:30:19.678 DEBUG:boto:auth_path=/lybtug-52/obj69 2024-04-16T18:30:19.678 DEBUG:boto:Method: GET 2024-04-16T18:30:19.679 DEBUG:boto:Path: /lybtug-52/obj69 2024-04-16T18:30:19.679 DEBUG:boto:Data: 2024-04-16T18:30:19.679 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.679 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.679 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.679 DEBUG:boto:Params: {} 2024-04-16T18:30:19.679 DEBUG:boto:Token: None 2024-04-16T18:30:19.679 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj69 2024-04-16T18:30:19.679 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Zv9A/gGqE4UthlRxMsCLaT96OXg= 2024-04-16T18:30:19.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Zv9A/gGqE4UthlRxMsCLaT96OXg=', 'Content-Length': '0'} 2024-04-16T18:30:19.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004219aee73fee0e2b-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.683 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.683 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.683 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.683 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.683 DEBUG:boto:Method: GET 2024-04-16T18:30:19.684 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:30:19.684 DEBUG:boto:Data: 2024-04-16T18:30:19.684 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.684 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.684 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.684 DEBUG:boto:Params: {} 2024-04-16T18:30:19.684 DEBUG:boto:Token: None 2024-04-16T18:30:19.684 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.684 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.688 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089b8210c88254beb-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.730 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.730 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.731 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.731 DEBUG:boto:path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.731 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj69 2024-04-16T18:30:19.731 DEBUG:boto:Method: GET 2024-04-16T18:30:19.731 DEBUG:boto:Path: /lybtug-52/?prefix=obj69 2024-04-16T18:30:19.731 DEBUG:boto:Data: 2024-04-16T18:30:19.731 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.731 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.731 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.731 DEBUG:boto:Params: {} 2024-04-16T18:30:19.731 DEBUG:boto:Token: None 2024-04-16T18:30:19.732 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.732 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b37f91e42a8a9d36-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.738 DEBUG:boto:b'lybtug-52obj691000falseobj692024-04-16T18:23:25.732Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.738 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2024-04-16T18:30:19.738 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.739 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.739 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.739 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.739 DEBUG:boto:Method: GET 2024-04-16T18:30:19.739 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.739 DEBUG:boto:Data: 2024-04-16T18:30:19.739 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.739 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.739 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.739 DEBUG:boto:Params: {} 2024-04-16T18:30:19.739 DEBUG:boto:Token: None 2024-04-16T18:30:19.740 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.740 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40= 2024-04-16T18:30:19.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40=', 'Content-Length': '0'} 2024-04-16T18:30:19.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000729b97943f8c8c3a-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.786 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.787 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.787 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.787 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.787 DEBUG:boto:Method: GET 2024-04-16T18:30:19.787 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.787 DEBUG:boto:Data: 2024-04-16T18:30:19.787 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.787 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.787 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.787 DEBUG:boto:Params: {} 2024-04-16T18:30:19.787 DEBUG:boto:Token: None 2024-04-16T18:30:19.788 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.788 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40= 2024-04-16T18:30:19.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40=', 'Content-Length': '0'} 2024-04-16T18:30:19.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d643ac6f10d0115a-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.791 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.791 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.791 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.791 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.791 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.791 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.791 DEBUG:boto:Data: 2024-04-16T18:30:19.792 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.792 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.792 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.792 DEBUG:boto:Params: {} 2024-04-16T18:30:19.792 DEBUG:boto:Token: None 2024-04-16T18:30:19.792 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.792 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AhRL3y1j0iWWI14lpheOd4vbyuI= 2024-04-16T18:30:19.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AhRL3y1j0iWWI14lpheOd4vbyuI=', 'Content-Length': '0'} 2024-04-16T18:30:19.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9e0dc254baaa5c9-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.796 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.796 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.796 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.796 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.796 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.796 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.796 DEBUG:boto:Data: 2024-04-16T18:30:19.796 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.796 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.796 DEBUG:boto:Params: {} 2024-04-16T18:30:19.797 DEBUG:boto:Token: None 2024-04-16T18:30:19.797 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AhRL3y1j0iWWI14lpheOd4vbyuI= 2024-04-16T18:30:19.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AhRL3y1j0iWWI14lpheOd4vbyuI=', 'Content-Length': '0'} 2024-04-16T18:30:19.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057f8c2ad39886f7c-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2024-04-16T18:30:19.800 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.800 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.800 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.800 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.800 DEBUG:boto:Method: GET 2024-04-16T18:30:19.800 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.801 DEBUG:boto:Data: 2024-04-16T18:30:19.801 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.801 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.801 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.801 DEBUG:boto:Params: {} 2024-04-16T18:30:19.801 DEBUG:boto:Token: None 2024-04-16T18:30:19.801 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.801 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40= 2024-04-16T18:30:19.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40=', 'Content-Length': '0'} 2024-04-16T18:30:19.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e724db80ad1aac7-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.846 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.846 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.846 DEBUG:boto:path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.847 DEBUG:boto:auth_path=/lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.847 DEBUG:boto:Method: GET 2024-04-16T18:30:19.847 DEBUG:boto:Path: /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.847 DEBUG:boto:Data: 2024-04-16T18:30:19.847 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.847 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.847 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.847 DEBUG:boto:Params: {} 2024-04-16T18:30:19.847 DEBUG:boto:Token: None 2024-04-16T18:30:19.847 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74?versionId=null 2024-04-16T18:30:19.847 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40= 2024-04-16T18:30:19.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tVRBwyKax5m1u8rpGTV5olEOk40=', 'Content-Length': '0'} 2024-04-16T18:30:19.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000963f23d2bd97b9a6-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.851 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.851 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.851 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:19.851 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:19.851 DEBUG:boto:Method: GET 2024-04-16T18:30:19.851 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:30:19.851 DEBUG:boto:Data: 2024-04-16T18:30:19.851 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.851 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.852 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.852 DEBUG:boto:Params: {} 2024-04-16T18:30:19.852 DEBUG:boto:Token: None 2024-04-16T18:30:19.852 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.852 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ee622dbd25e6197-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.856 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.856 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.856 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.857 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:19.857 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:19.857 DEBUG:boto:Method: GET 2024-04-16T18:30:19.857 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:30:19.857 DEBUG:boto:Data: 2024-04-16T18:30:19.857 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.857 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.857 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.857 DEBUG:boto:Params: {} 2024-04-16T18:30:19.857 DEBUG:boto:Token: None 2024-04-16T18:30:19.857 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.857 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf32789c1851c40c-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.906 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:19.907 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.907 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.907 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.907 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:30:19.907 DEBUG:boto:Data: 2024-04-16T18:30:19.907 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.907 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.907 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.907 DEBUG:boto:Params: {} 2024-04-16T18:30:19.907 DEBUG:boto:Token: None 2024-04-16T18:30:19.907 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74 2024-04-16T18:30:19.908 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0tIZMMyBpxjBZ1TF82A3sVQBtfw= 2024-04-16T18:30:19.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0tIZMMyBpxjBZ1TF82A3sVQBtfw=', 'Content-Length': '0'} 2024-04-16T18:30:19.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cac5cd0bd51dc9f3-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.911 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.911 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.911 DEBUG:boto:Method: HEAD 2024-04-16T18:30:19.911 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:30:19.911 DEBUG:boto:Data: 2024-04-16T18:30:19.911 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.911 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.912 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.912 DEBUG:boto:Params: {} 2024-04-16T18:30:19.912 DEBUG:boto:Token: None 2024-04-16T18:30:19.912 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74 2024-04-16T18:30:19.912 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:0tIZMMyBpxjBZ1TF82A3sVQBtfw= 2024-04-16T18:30:19.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0tIZMMyBpxjBZ1TF82A3sVQBtfw=', 'Content-Length': '0'} 2024-04-16T18:30:19.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037ccf0f8008d4c38-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2024-04-16T18:30:19.915 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.916 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.916 DEBUG:boto:Method: GET 2024-04-16T18:30:19.916 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:30:19.916 DEBUG:boto:Data: 2024-04-16T18:30:19.916 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.916 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.916 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.916 DEBUG:boto:Params: {} 2024-04-16T18:30:19.916 DEBUG:boto:Token: None 2024-04-16T18:30:19.916 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74 2024-04-16T18:30:19.916 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xv3uXkn0NIMJDkvRwtvpMBMxR7M= 2024-04-16T18:30:19.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xv3uXkn0NIMJDkvRwtvpMBMxR7M=', 'Content-Length': '0'} 2024-04-16T18:30:19.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a76f25c1c675a75d-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.962 DEBUG:boto:path=/lybtug-52/obj74 2024-04-16T18:30:19.962 DEBUG:boto:auth_path=/lybtug-52/obj74 2024-04-16T18:30:19.962 DEBUG:boto:Method: GET 2024-04-16T18:30:19.963 DEBUG:boto:Path: /lybtug-52/obj74 2024-04-16T18:30:19.963 DEBUG:boto:Data: 2024-04-16T18:30:19.963 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.963 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:19.963 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.963 DEBUG:boto:Params: {} 2024-04-16T18:30:19.963 DEBUG:boto:Token: None 2024-04-16T18:30:19.963 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/obj74 2024-04-16T18:30:19.963 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Xv3uXkn0NIMJDkvRwtvpMBMxR7M= 2024-04-16T18:30:19.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xv3uXkn0NIMJDkvRwtvpMBMxR7M=', 'Content-Length': '0'} 2024-04-16T18:30:19.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:27:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcdcaa9fc31ca549-00661ec3bb-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:19.967 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:19.967 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:19.967 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:19.967 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:19.967 DEBUG:boto:Method: GET 2024-04-16T18:30:19.967 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:30:19.967 DEBUG:boto:Data: 2024-04-16T18:30:19.967 DEBUG:boto:Headers: {} 2024-04-16T18:30:19.967 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:19.967 DEBUG:boto:Port: 8000 2024-04-16T18:30:19.968 DEBUG:boto:Params: {} 2024-04-16T18:30:19.968 DEBUG:boto:Token: None 2024-04-16T18:30:19.968 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:19 GMT /lybtug-52/ 2024-04-16T18:30:19.968 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU= 2024-04-16T18:30:19.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:19 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2kFuc6ANLO4ycSNgJA/VHQRllVU=', 'Content-Length': '0'} 2024-04-16T18:30:19.971 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000229c9ac02495f2-00661ec3bb-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:19 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.014 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.015 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.015 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.015 DEBUG:boto:path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:20.015 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj74 2024-04-16T18:30:20.015 DEBUG:boto:Method: GET 2024-04-16T18:30:20.015 DEBUG:boto:Path: /lybtug-52/?prefix=obj74 2024-04-16T18:30:20.015 DEBUG:boto:Data: 2024-04-16T18:30:20.015 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.015 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.015 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.015 DEBUG:boto:Params: {} 2024-04-16T18:30:20.016 DEBUG:boto:Token: None 2024-04-16T18:30:20.016 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.016 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.022 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f2360d2dac6ba0d-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.022 DEBUG:boto:b'lybtug-52obj741000falseobj742024-04-16T18:27:53.324Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:30:20.023 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.023 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.023 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.023 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.023 DEBUG:boto:Method: GET 2024-04-16T18:30:20.023 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.023 DEBUG:boto:Data: 2024-04-16T18:30:20.023 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.023 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.023 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.023 DEBUG:boto:Params: {} 2024-04-16T18:30:20.024 DEBUG:boto:Token: None 2024-04-16T18:30:20.024 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.024 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY= 2024-04-16T18:30:20.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY=', 'Content-Length': '0'} 2024-04-16T18:30:20.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005fa980dae3f0b906-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.070 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.070 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.071 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.071 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.071 DEBUG:boto:Method: GET 2024-04-16T18:30:20.071 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.071 DEBUG:boto:Data: 2024-04-16T18:30:20.071 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.071 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.071 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.071 DEBUG:boto:Params: {} 2024-04-16T18:30:20.071 DEBUG:boto:Token: None 2024-04-16T18:30:20.071 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.071 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY= 2024-04-16T18:30:20.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY=', 'Content-Length': '0'} 2024-04-16T18:30:20.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c1f7cd0ddf9b7f8-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.075 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.075 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.075 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.075 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.075 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.075 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.075 DEBUG:boto:Data: 2024-04-16T18:30:20.075 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.075 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.075 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.075 DEBUG:boto:Params: {} 2024-04-16T18:30:20.075 DEBUG:boto:Token: None 2024-04-16T18:30:20.076 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.076 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywsVxaNmbpX0eyP0K6gL2q406dE= 2024-04-16T18:30:20.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywsVxaNmbpX0eyP0K6gL2q406dE=', 'Content-Length': '0'} 2024-04-16T18:30:20.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c784d586a5ace985-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.079 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.079 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.079 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.079 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.079 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.079 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.079 DEBUG:boto:Data: 2024-04-16T18:30:20.079 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.079 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.080 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.080 DEBUG:boto:Params: {} 2024-04-16T18:30:20.080 DEBUG:boto:Token: None 2024-04-16T18:30:20.080 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.080 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ywsVxaNmbpX0eyP0K6gL2q406dE= 2024-04-16T18:30:20.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ywsVxaNmbpX0eyP0K6gL2q406dE=', 'Content-Length': '0'} 2024-04-16T18:30:20.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b05ba8f8e1f90510-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.083 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:30:20.083 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.083 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.083 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.083 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.083 DEBUG:boto:Method: GET 2024-04-16T18:30:20.083 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.084 DEBUG:boto:Data: 2024-04-16T18:30:20.084 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.084 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.084 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.084 DEBUG:boto:Params: {} 2024-04-16T18:30:20.084 DEBUG:boto:Token: None 2024-04-16T18:30:20.084 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.084 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY= 2024-04-16T18:30:20.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY=', 'Content-Length': '0'} 2024-04-16T18:30:20.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b58bc7bb019fb7c-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.130 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.130 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.131 DEBUG:boto:path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.131 DEBUG:boto:auth_path=/lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.131 DEBUG:boto:Method: GET 2024-04-16T18:30:20.131 DEBUG:boto:Path: /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.131 DEBUG:boto:Data: 2024-04-16T18:30:20.131 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.131 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.131 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.131 DEBUG:boto:Params: {} 2024-04-16T18:30:20.131 DEBUG:boto:Token: None 2024-04-16T18:30:20.132 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76?versionId=null 2024-04-16T18:30:20.132 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY= 2024-04-16T18:30:20.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zdxRpgtQY5K2FXWugOsDxaURHrY=', 'Content-Length': '0'} 2024-04-16T18:30:20.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ff9b883a79b5504-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.135 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.135 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.135 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.135 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.135 DEBUG:boto:Method: GET 2024-04-16T18:30:20.135 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:30:20.136 DEBUG:boto:Data: 2024-04-16T18:30:20.136 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.136 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.136 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.136 DEBUG:boto:Params: {} 2024-04-16T18:30:20.136 DEBUG:boto:Token: None 2024-04-16T18:30:20.136 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.136 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e7ad7678bbd9183-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.140 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.140 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.141 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.141 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.141 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.141 DEBUG:boto:Method: GET 2024-04-16T18:30:20.141 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:30:20.141 DEBUG:boto:Data: 2024-04-16T18:30:20.141 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.141 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.141 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.141 DEBUG:boto:Params: {} 2024-04-16T18:30:20.141 DEBUG:boto:Token: None 2024-04-16T18:30:20.142 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.142 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002274b3fa2dd59ae7-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.190 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.191 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.191 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.191 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.191 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:30:20.191 DEBUG:boto:Data: 2024-04-16T18:30:20.191 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.191 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.191 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.191 DEBUG:boto:Params: {} 2024-04-16T18:30:20.192 DEBUG:boto:Token: None 2024-04-16T18:30:20.192 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76 2024-04-16T18:30:20.192 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sWlN2kMukuhsGuHHtg6kivdjtBc= 2024-04-16T18:30:20.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sWlN2kMukuhsGuHHtg6kivdjtBc=', 'Content-Length': '0'} 2024-04-16T18:30:20.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063b89cd5cba044be-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.195 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.195 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.195 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.195 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:30:20.195 DEBUG:boto:Data: 2024-04-16T18:30:20.196 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.196 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.196 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.196 DEBUG:boto:Params: {} 2024-04-16T18:30:20.196 DEBUG:boto:Token: None 2024-04-16T18:30:20.196 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76 2024-04-16T18:30:20.196 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:sWlN2kMukuhsGuHHtg6kivdjtBc= 2024-04-16T18:30:20.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:sWlN2kMukuhsGuHHtg6kivdjtBc=', 'Content-Length': '0'} 2024-04-16T18:30:20.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2bb5c910a8ac007-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.200 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2024-04-16T18:30:20.200 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.200 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.200 DEBUG:boto:Method: GET 2024-04-16T18:30:20.200 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:30:20.200 DEBUG:boto:Data: 2024-04-16T18:30:20.200 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.200 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.200 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.200 DEBUG:boto:Params: {} 2024-04-16T18:30:20.201 DEBUG:boto:Token: None 2024-04-16T18:30:20.201 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76 2024-04-16T18:30:20.201 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y5kOKr2CiD+FSUHi/HqOpRGRbX8= 2024-04-16T18:30:20.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y5kOKr2CiD+FSUHi/HqOpRGRbX8=', 'Content-Length': '0'} 2024-04-16T18:30:20.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004be208a419c3bfd6-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.246 DEBUG:boto:path=/lybtug-52/obj76 2024-04-16T18:30:20.246 DEBUG:boto:auth_path=/lybtug-52/obj76 2024-04-16T18:30:20.246 DEBUG:boto:Method: GET 2024-04-16T18:30:20.247 DEBUG:boto:Path: /lybtug-52/obj76 2024-04-16T18:30:20.247 DEBUG:boto:Data: 2024-04-16T18:30:20.247 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.247 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.247 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.247 DEBUG:boto:Params: {} 2024-04-16T18:30:20.247 DEBUG:boto:Token: None 2024-04-16T18:30:20.247 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj76 2024-04-16T18:30:20.247 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:y5kOKr2CiD+FSUHi/HqOpRGRbX8= 2024-04-16T18:30:20.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y5kOKr2CiD+FSUHi/HqOpRGRbX8=', 'Content-Length': '0'} 2024-04-16T18:30:20.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d253ad8a1e01301e-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.251 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.251 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.251 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.251 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.251 DEBUG:boto:Method: GET 2024-04-16T18:30:20.251 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:30:20.251 DEBUG:boto:Data: 2024-04-16T18:30:20.251 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.251 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.251 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.251 DEBUG:boto:Params: {} 2024-04-16T18:30:20.252 DEBUG:boto:Token: None 2024-04-16T18:30:20.252 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.252 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9eaad9c34b0c52a-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.298 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.299 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.299 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.300 DEBUG:boto:path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.300 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj76 2024-04-16T18:30:20.300 DEBUG:boto:Method: GET 2024-04-16T18:30:20.300 DEBUG:boto:Path: /lybtug-52/?prefix=obj76 2024-04-16T18:30:20.300 DEBUG:boto:Data: 2024-04-16T18:30:20.300 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.300 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.300 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.300 DEBUG:boto:Params: {} 2024-04-16T18:30:20.300 DEBUG:boto:Token: None 2024-04-16T18:30:20.300 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.301 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b839484490b26dc-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.307 DEBUG:boto:b'lybtug-52obj761000falseobj762024-04-16T18:22:20.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.308 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:30:20.308 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.308 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.308 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.308 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.308 DEBUG:boto:Method: GET 2024-04-16T18:30:20.308 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.308 DEBUG:boto:Data: 2024-04-16T18:30:20.308 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.308 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.309 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.309 DEBUG:boto:Params: {} 2024-04-16T18:30:20.309 DEBUG:boto:Token: None 2024-04-16T18:30:20.309 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.309 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y= 2024-04-16T18:30:20.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y=', 'Content-Length': '0'} 2024-04-16T18:30:20.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000430b58a32fa4537b-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.354 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.354 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.355 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.355 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.355 DEBUG:boto:Method: GET 2024-04-16T18:30:20.355 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.355 DEBUG:boto:Data: 2024-04-16T18:30:20.355 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.355 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.355 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.355 DEBUG:boto:Params: {} 2024-04-16T18:30:20.355 DEBUG:boto:Token: None 2024-04-16T18:30:20.355 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.355 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y= 2024-04-16T18:30:20.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y=', 'Content-Length': '0'} 2024-04-16T18:30:20.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7a1d4bb24a6de5f-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.359 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.359 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.359 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.359 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.359 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.359 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.359 DEBUG:boto:Data: 2024-04-16T18:30:20.359 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.360 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.360 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.360 DEBUG:boto:Params: {} 2024-04-16T18:30:20.360 DEBUG:boto:Token: None 2024-04-16T18:30:20.360 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.360 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kap9pN/Da4x8obcnHi4Pw/nsfT0= 2024-04-16T18:30:20.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kap9pN/Da4x8obcnHi4Pw/nsfT0=', 'Content-Length': '0'} 2024-04-16T18:30:20.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022bb65dcef4bee28-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.363 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.363 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.364 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.364 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.364 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.364 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.364 DEBUG:boto:Data: 2024-04-16T18:30:20.364 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.364 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.364 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.364 DEBUG:boto:Params: {} 2024-04-16T18:30:20.364 DEBUG:boto:Token: None 2024-04-16T18:30:20.364 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.364 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:kap9pN/Da4x8obcnHi4Pw/nsfT0= 2024-04-16T18:30:20.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kap9pN/Da4x8obcnHi4Pw/nsfT0=', 'Content-Length': '0'} 2024-04-16T18:30:20.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c90d625677340f38-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.368 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:30:20.368 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.368 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.368 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.368 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.368 DEBUG:boto:Method: GET 2024-04-16T18:30:20.368 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.368 DEBUG:boto:Data: 2024-04-16T18:30:20.368 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.368 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.368 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.368 DEBUG:boto:Params: {} 2024-04-16T18:30:20.369 DEBUG:boto:Token: None 2024-04-16T18:30:20.369 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.369 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y= 2024-04-16T18:30:20.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y=', 'Content-Length': '0'} 2024-04-16T18:30:20.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004420c80ff7db511a-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.414 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.414 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.415 DEBUG:boto:path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.415 DEBUG:boto:auth_path=/lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.415 DEBUG:boto:Method: GET 2024-04-16T18:30:20.415 DEBUG:boto:Path: /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.415 DEBUG:boto:Data: 2024-04-16T18:30:20.415 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.415 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.415 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.415 DEBUG:boto:Params: {} 2024-04-16T18:30:20.415 DEBUG:boto:Token: None 2024-04-16T18:30:20.415 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85?versionId=null 2024-04-16T18:30:20.416 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y= 2024-04-16T18:30:20.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:YaVj+4YZjFsMOH2+m9c5xsHgo0Y=', 'Content-Length': '0'} 2024-04-16T18:30:20.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006008339574c5d2d0-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.419 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.419 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.419 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.419 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.419 DEBUG:boto:Method: GET 2024-04-16T18:30:20.419 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:30:20.419 DEBUG:boto:Data: 2024-04-16T18:30:20.420 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.420 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.420 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.420 DEBUG:boto:Params: {} 2024-04-16T18:30:20.420 DEBUG:boto:Token: None 2024-04-16T18:30:20.420 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.420 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa407cd33686c9d0-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.424 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.425 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.425 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.425 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.425 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.425 DEBUG:boto:Method: GET 2024-04-16T18:30:20.425 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:30:20.425 DEBUG:boto:Data: 2024-04-16T18:30:20.425 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.425 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.425 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.426 DEBUG:boto:Params: {} 2024-04-16T18:30:20.426 DEBUG:boto:Token: None 2024-04-16T18:30:20.426 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.426 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.433 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ce2039d292ead244-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.474 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.475 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.475 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.475 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.475 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:30:20.475 DEBUG:boto:Data: 2024-04-16T18:30:20.475 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.475 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.475 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.475 DEBUG:boto:Params: {} 2024-04-16T18:30:20.475 DEBUG:boto:Token: None 2024-04-16T18:30:20.476 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85 2024-04-16T18:30:20.476 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Mim9svhCdC1hkPm6l3Ym+lKUHs= 2024-04-16T18:30:20.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Mim9svhCdC1hkPm6l3Ym+lKUHs=', 'Content-Length': '0'} 2024-04-16T18:30:20.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004118d9b688989b9f-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.479 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.479 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.479 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.479 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:30:20.479 DEBUG:boto:Data: 2024-04-16T18:30:20.479 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.479 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.479 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.479 DEBUG:boto:Params: {} 2024-04-16T18:30:20.479 DEBUG:boto:Token: None 2024-04-16T18:30:20.480 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85 2024-04-16T18:30:20.480 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1Mim9svhCdC1hkPm6l3Ym+lKUHs= 2024-04-16T18:30:20.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1Mim9svhCdC1hkPm6l3Ym+lKUHs=', 'Content-Length': '0'} 2024-04-16T18:30:20.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016675e605f2d0206-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.483 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2024-04-16T18:30:20.483 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.483 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.483 DEBUG:boto:Method: GET 2024-04-16T18:30:20.484 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:30:20.484 DEBUG:boto:Data: 2024-04-16T18:30:20.484 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.484 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.484 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.484 DEBUG:boto:Params: {} 2024-04-16T18:30:20.484 DEBUG:boto:Token: None 2024-04-16T18:30:20.484 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85 2024-04-16T18:30:20.484 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dBiTYAe1FXglgxP1IkV05/IA0L0= 2024-04-16T18:30:20.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dBiTYAe1FXglgxP1IkV05/IA0L0=', 'Content-Length': '0'} 2024-04-16T18:30:20.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019bab6377b376e95-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.530 DEBUG:boto:path=/lybtug-52/obj85 2024-04-16T18:30:20.530 DEBUG:boto:auth_path=/lybtug-52/obj85 2024-04-16T18:30:20.530 DEBUG:boto:Method: GET 2024-04-16T18:30:20.531 DEBUG:boto:Path: /lybtug-52/obj85 2024-04-16T18:30:20.531 DEBUG:boto:Data: 2024-04-16T18:30:20.531 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.531 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.531 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.531 DEBUG:boto:Params: {} 2024-04-16T18:30:20.531 DEBUG:boto:Token: None 2024-04-16T18:30:20.531 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj85 2024-04-16T18:30:20.531 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:dBiTYAe1FXglgxP1IkV05/IA0L0= 2024-04-16T18:30:20.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dBiTYAe1FXglgxP1IkV05/IA0L0=', 'Content-Length': '0'} 2024-04-16T18:30:20.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:20:12 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000881c5c1ddb578ce-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.535 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.535 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.535 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.535 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.535 DEBUG:boto:Method: GET 2024-04-16T18:30:20.535 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:30:20.535 DEBUG:boto:Data: 2024-04-16T18:30:20.535 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.535 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.535 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.535 DEBUG:boto:Params: {} 2024-04-16T18:30:20.535 DEBUG:boto:Token: None 2024-04-16T18:30:20.536 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.536 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.539 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a422e1354b556a9-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.582 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.583 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.583 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.583 DEBUG:boto:path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.583 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj85 2024-04-16T18:30:20.583 DEBUG:boto:Method: GET 2024-04-16T18:30:20.583 DEBUG:boto:Path: /lybtug-52/?prefix=obj85 2024-04-16T18:30:20.583 DEBUG:boto:Data: 2024-04-16T18:30:20.583 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.583 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.583 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.583 DEBUG:boto:Params: {} 2024-04-16T18:30:20.583 DEBUG:boto:Token: None 2024-04-16T18:30:20.584 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.584 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c64c84d07a1871a9-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.590 DEBUG:boto:b'lybtug-52obj851000falseobj852024-04-16T18:20:12.526Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.590 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:30:20.590 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.590 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.591 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.591 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.591 DEBUG:boto:Method: GET 2024-04-16T18:30:20.591 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.591 DEBUG:boto:Data: 2024-04-16T18:30:20.591 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.591 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.591 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.591 DEBUG:boto:Params: {} 2024-04-16T18:30:20.591 DEBUG:boto:Token: None 2024-04-16T18:30:20.591 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.591 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68= 2024-04-16T18:30:20.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68=', 'Content-Length': '0'} 2024-04-16T18:30:20.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df7ec6a87a511bf1-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.638 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.639 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.639 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.639 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.639 DEBUG:boto:Method: GET 2024-04-16T18:30:20.639 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.639 DEBUG:boto:Data: 2024-04-16T18:30:20.639 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.639 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.639 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.639 DEBUG:boto:Params: {} 2024-04-16T18:30:20.640 DEBUG:boto:Token: None 2024-04-16T18:30:20.640 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.640 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68= 2024-04-16T18:30:20.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68=', 'Content-Length': '0'} 2024-04-16T18:30:20.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4fe6735005dc1c1-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.643 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.643 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.643 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.643 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.644 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.644 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.644 DEBUG:boto:Data: 2024-04-16T18:30:20.644 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.644 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.644 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.644 DEBUG:boto:Params: {} 2024-04-16T18:30:20.644 DEBUG:boto:Token: None 2024-04-16T18:30:20.644 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.644 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O8aXWhGZNse1JGxy9IK3z5H5Qso= 2024-04-16T18:30:20.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O8aXWhGZNse1JGxy9IK3z5H5Qso=', 'Content-Length': '0'} 2024-04-16T18:30:20.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af962c3d9a0f0392-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.648 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.648 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.648 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.648 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.648 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.649 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.649 DEBUG:boto:Data: 2024-04-16T18:30:20.649 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.649 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.649 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.649 DEBUG:boto:Params: {} 2024-04-16T18:30:20.649 DEBUG:boto:Token: None 2024-04-16T18:30:20.649 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.649 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:O8aXWhGZNse1JGxy9IK3z5H5Qso= 2024-04-16T18:30:20.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O8aXWhGZNse1JGxy9IK3z5H5Qso=', 'Content-Length': '0'} 2024-04-16T18:30:20.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cba9065a5d409bb-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.652 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:30:20.653 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.653 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.653 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.653 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.653 DEBUG:boto:Method: GET 2024-04-16T18:30:20.653 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.653 DEBUG:boto:Data: 2024-04-16T18:30:20.653 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.653 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.653 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.653 DEBUG:boto:Params: {} 2024-04-16T18:30:20.654 DEBUG:boto:Token: None 2024-04-16T18:30:20.654 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.654 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68= 2024-04-16T18:30:20.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68=', 'Content-Length': '0'} 2024-04-16T18:30:20.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bd7e0cd04704077-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.698 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.698 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.699 DEBUG:boto:path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.699 DEBUG:boto:auth_path=/lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.699 DEBUG:boto:Method: GET 2024-04-16T18:30:20.699 DEBUG:boto:Path: /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.699 DEBUG:boto:Data: 2024-04-16T18:30:20.699 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.699 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.699 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.699 DEBUG:boto:Params: {} 2024-04-16T18:30:20.699 DEBUG:boto:Token: None 2024-04-16T18:30:20.699 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87?versionId=null 2024-04-16T18:30:20.700 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68= 2024-04-16T18:30:20.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7w5F4uGWTpz9H6TVvpjouJKDG68=', 'Content-Length': '0'} 2024-04-16T18:30:20.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a14e9fb5dc803a9-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.703 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.703 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.703 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.703 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.703 DEBUG:boto:Method: GET 2024-04-16T18:30:20.704 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:30:20.704 DEBUG:boto:Data: 2024-04-16T18:30:20.704 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.704 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.704 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.704 DEBUG:boto:Params: {} 2024-04-16T18:30:20.704 DEBUG:boto:Token: None 2024-04-16T18:30:20.704 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.704 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.708 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005daa8916d773cc52-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.708 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.709 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.709 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.709 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.709 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.709 DEBUG:boto:Method: GET 2024-04-16T18:30:20.709 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:30:20.709 DEBUG:boto:Data: 2024-04-16T18:30:20.709 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.710 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.710 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.710 DEBUG:boto:Params: {} 2024-04-16T18:30:20.710 DEBUG:boto:Token: None 2024-04-16T18:30:20.710 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.710 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.717 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000379349cb6f5733a0-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.758 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.759 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.759 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.759 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.759 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:30:20.759 DEBUG:boto:Data: 2024-04-16T18:30:20.759 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.759 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.759 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.759 DEBUG:boto:Params: {} 2024-04-16T18:30:20.759 DEBUG:boto:Token: None 2024-04-16T18:30:20.760 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87 2024-04-16T18:30:20.760 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PC5vDSCgYrdtw7gmNGiq8JUJW/k= 2024-04-16T18:30:20.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PC5vDSCgYrdtw7gmNGiq8JUJW/k=', 'Content-Length': '0'} 2024-04-16T18:30:20.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ab1901774110081-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.763 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.763 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.763 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.763 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:30:20.763 DEBUG:boto:Data: 2024-04-16T18:30:20.763 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.764 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.764 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.764 DEBUG:boto:Params: {} 2024-04-16T18:30:20.764 DEBUG:boto:Token: None 2024-04-16T18:30:20.764 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87 2024-04-16T18:30:20.764 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:PC5vDSCgYrdtw7gmNGiq8JUJW/k= 2024-04-16T18:30:20.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:PC5vDSCgYrdtw7gmNGiq8JUJW/k=', 'Content-Length': '0'} 2024-04-16T18:30:20.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008654304de91035e0-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.767 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2024-04-16T18:30:20.767 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.767 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.768 DEBUG:boto:Method: GET 2024-04-16T18:30:20.768 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:30:20.768 DEBUG:boto:Data: 2024-04-16T18:30:20.768 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.768 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.768 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.768 DEBUG:boto:Params: {} 2024-04-16T18:30:20.768 DEBUG:boto:Token: None 2024-04-16T18:30:20.768 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87 2024-04-16T18:30:20.768 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tKDjwoECOVCFSHVORMpnyUmBYks= 2024-04-16T18:30:20.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tKDjwoECOVCFSHVORMpnyUmBYks=', 'Content-Length': '0'} 2024-04-16T18:30:20.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090966c56c4ee3600-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.814 DEBUG:boto:path=/lybtug-52/obj87 2024-04-16T18:30:20.814 DEBUG:boto:auth_path=/lybtug-52/obj87 2024-04-16T18:30:20.815 DEBUG:boto:Method: GET 2024-04-16T18:30:20.815 DEBUG:boto:Path: /lybtug-52/obj87 2024-04-16T18:30:20.815 DEBUG:boto:Data: 2024-04-16T18:30:20.815 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.815 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.815 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.815 DEBUG:boto:Params: {} 2024-04-16T18:30:20.815 DEBUG:boto:Token: None 2024-04-16T18:30:20.815 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj87 2024-04-16T18:30:20.815 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:tKDjwoECOVCFSHVORMpnyUmBYks= 2024-04-16T18:30:20.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tKDjwoECOVCFSHVORMpnyUmBYks=', 'Content-Length': '0'} 2024-04-16T18:30:20.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:25:26 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df2d3d1fa95e80a9-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.819 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.819 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.819 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.819 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.819 DEBUG:boto:Method: GET 2024-04-16T18:30:20.819 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:30:20.819 DEBUG:boto:Data: 2024-04-16T18:30:20.819 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.820 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.820 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.820 DEBUG:boto:Params: {} 2024-04-16T18:30:20.820 DEBUG:boto:Token: None 2024-04-16T18:30:20.820 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.820 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.824 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000794191bc9a895a9b-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.866 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.867 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.867 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.867 DEBUG:boto:path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.867 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj87 2024-04-16T18:30:20.867 DEBUG:boto:Method: GET 2024-04-16T18:30:20.867 DEBUG:boto:Path: /lybtug-52/?prefix=obj87 2024-04-16T18:30:20.867 DEBUG:boto:Data: 2024-04-16T18:30:20.867 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.867 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.867 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.867 DEBUG:boto:Params: {} 2024-04-16T18:30:20.868 DEBUG:boto:Token: None 2024-04-16T18:30:20.868 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.868 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.874 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d208049250abb1c6-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.875 DEBUG:boto:b'lybtug-52obj871000falseobj872024-04-16T18:25:26.833Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:30:20.875 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:20.875 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:20.875 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.875 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.875 DEBUG:boto:Method: GET 2024-04-16T18:30:20.876 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.876 DEBUG:boto:Data: 2024-04-16T18:30:20.876 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.876 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.876 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.876 DEBUG:boto:Params: {} 2024-04-16T18:30:20.876 DEBUG:boto:Token: None 2024-04-16T18:30:20.876 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.876 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk= 2024-04-16T18:30:20.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk=', 'Content-Length': '0'} 2024-04-16T18:30:20.880 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2e2dcacac82c3a1-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.922 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:20.922 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:20.922 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.923 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.923 DEBUG:boto:Method: GET 2024-04-16T18:30:20.923 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.923 DEBUG:boto:Data: 2024-04-16T18:30:20.923 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.923 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.923 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.923 DEBUG:boto:Params: {} 2024-04-16T18:30:20.923 DEBUG:boto:Token: None 2024-04-16T18:30:20.923 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.924 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk= 2024-04-16T18:30:20.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk=', 'Content-Length': '0'} 2024-04-16T18:30:20.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065e7b9be8909dd56-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.927 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:20.927 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:20.927 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.927 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.927 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.927 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.927 DEBUG:boto:Data: 2024-04-16T18:30:20.927 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.927 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.927 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.928 DEBUG:boto:Params: {} 2024-04-16T18:30:20.928 DEBUG:boto:Token: None 2024-04-16T18:30:20.928 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.928 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/FSWlqAFGsxgtcMB44l8Zqgnw9Q= 2024-04-16T18:30:20.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/FSWlqAFGsxgtcMB44l8Zqgnw9Q=', 'Content-Length': '0'} 2024-04-16T18:30:20.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fbf61bb9c55d68f-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.931 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:20.931 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:20.931 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.931 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.931 DEBUG:boto:Method: HEAD 2024-04-16T18:30:20.931 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.931 DEBUG:boto:Data: 2024-04-16T18:30:20.932 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.932 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.932 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.932 DEBUG:boto:Params: {} 2024-04-16T18:30:20.932 DEBUG:boto:Token: None 2024-04-16T18:30:20.932 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.932 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:/FSWlqAFGsxgtcMB44l8Zqgnw9Q= 2024-04-16T18:30:20.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/FSWlqAFGsxgtcMB44l8Zqgnw9Q=', 'Content-Length': '0'} 2024-04-16T18:30:20.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072f6940b690a6bfd-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.936 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:30:20.936 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:20.936 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:20.936 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.936 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.936 DEBUG:boto:Method: GET 2024-04-16T18:30:20.936 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.936 DEBUG:boto:Data: 2024-04-16T18:30:20.936 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.936 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.937 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.937 DEBUG:boto:Params: {} 2024-04-16T18:30:20.937 DEBUG:boto:Token: None 2024-04-16T18:30:20.937 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.937 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk= 2024-04-16T18:30:20.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk=', 'Content-Length': '0'} 2024-04-16T18:30:20.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a0e92303d06d8f6-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.982 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:20.982 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:20.983 DEBUG:boto:path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.983 DEBUG:boto:auth_path=/lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.983 DEBUG:boto:Method: GET 2024-04-16T18:30:20.983 DEBUG:boto:Path: /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.983 DEBUG:boto:Data: 2024-04-16T18:30:20.983 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.983 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.983 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.983 DEBUG:boto:Params: {} 2024-04-16T18:30:20.983 DEBUG:boto:Token: None 2024-04-16T18:30:20.983 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/obj92?versionId=null 2024-04-16T18:30:20.984 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk= 2024-04-16T18:30:20.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lArN5a/evoktvTfkcPnh7RVqDnk=', 'Content-Length': '0'} 2024-04-16T18:30:20.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e537d7505555ea60-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.987 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.987 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.988 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:20.988 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:20.988 DEBUG:boto:Method: GET 2024-04-16T18:30:20.988 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:30:20.988 DEBUG:boto:Data: 2024-04-16T18:30:20.988 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.988 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:20.988 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.988 DEBUG:boto:Params: {} 2024-04-16T18:30:20.988 DEBUG:boto:Token: None 2024-04-16T18:30:20.988 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.989 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:20.992 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005c8fd108598dfda-00661ec3bc-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:20 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:20.992 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:20.993 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:20.993 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:20.993 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:20.993 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:20.993 DEBUG:boto:Method: GET 2024-04-16T18:30:20.993 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:30:20.993 DEBUG:boto:Data: 2024-04-16T18:30:20.993 DEBUG:boto:Headers: {} 2024-04-16T18:30:20.993 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:20.993 DEBUG:boto:Port: 8000 2024-04-16T18:30:20.993 DEBUG:boto:Params: {} 2024-04-16T18:30:20.994 DEBUG:boto:Token: None 2024-04-16T18:30:20.994 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:20 GMT /lybtug-52/ 2024-04-16T18:30:20.994 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU= 2024-04-16T18:30:20.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:20 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Hin9ifLNYkUv0yks1oRTzAfLEZU=', 'Content-Length': '0'} 2024-04-16T18:30:21.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034e411827ee390c5-00661ec3bc-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.042 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.043 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:21.043 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:21.043 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.043 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:30:21.043 DEBUG:boto:Data: 2024-04-16T18:30:21.043 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.043 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.043 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.044 DEBUG:boto:Params: {} 2024-04-16T18:30:21.044 DEBUG:boto:Token: None 2024-04-16T18:30:21.044 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj92 2024-04-16T18:30:21.044 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ePCb2f2/spmOpcS4xRPEZxJ5VGY= 2024-04-16T18:30:21.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ePCb2f2/spmOpcS4xRPEZxJ5VGY=', 'Content-Length': '0'} 2024-04-16T18:30:21.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094c5390fed3812c4-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.047 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:21.047 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:21.047 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.047 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:30:21.047 DEBUG:boto:Data: 2024-04-16T18:30:21.048 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.048 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.048 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.048 DEBUG:boto:Params: {} 2024-04-16T18:30:21.048 DEBUG:boto:Token: None 2024-04-16T18:30:21.048 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj92 2024-04-16T18:30:21.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:ePCb2f2/spmOpcS4xRPEZxJ5VGY= 2024-04-16T18:30:21.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ePCb2f2/spmOpcS4xRPEZxJ5VGY=', 'Content-Length': '0'} 2024-04-16T18:30:21.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7439cac32baaa99-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.051 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2024-04-16T18:30:21.052 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:21.052 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:21.052 DEBUG:boto:Method: GET 2024-04-16T18:30:21.052 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:30:21.052 DEBUG:boto:Data: 2024-04-16T18:30:21.052 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.052 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.052 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.052 DEBUG:boto:Params: {} 2024-04-16T18:30:21.052 DEBUG:boto:Token: None 2024-04-16T18:30:21.053 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj92 2024-04-16T18:30:21.053 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1+JTJn4zGZ3/wLyGDBZiHy/LShs= 2024-04-16T18:30:21.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1+JTJn4zGZ3/wLyGDBZiHy/LShs=', 'Content-Length': '0'} 2024-04-16T18:30:21.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6fc172364b8a3a5-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.100 DEBUG:boto:path=/lybtug-52/obj92 2024-04-16T18:30:21.100 DEBUG:boto:auth_path=/lybtug-52/obj92 2024-04-16T18:30:21.100 DEBUG:boto:Method: GET 2024-04-16T18:30:21.100 DEBUG:boto:Path: /lybtug-52/obj92 2024-04-16T18:30:21.100 DEBUG:boto:Data: 2024-04-16T18:30:21.100 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.100 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.100 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.100 DEBUG:boto:Params: {} 2024-04-16T18:30:21.101 DEBUG:boto:Token: None 2024-04-16T18:30:21.101 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj92 2024-04-16T18:30:21.101 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:1+JTJn4zGZ3/wLyGDBZiHy/LShs= 2024-04-16T18:30:21.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1+JTJn4zGZ3/wLyGDBZiHy/LShs=', 'Content-Length': '0'} 2024-04-16T18:30:21.104 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:23:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1686f71793ab298-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.104 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.104 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.104 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:21.104 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:21.105 DEBUG:boto:Method: GET 2024-04-16T18:30:21.105 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:30:21.105 DEBUG:boto:Data: 2024-04-16T18:30:21.105 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.105 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.105 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.105 DEBUG:boto:Params: {} 2024-04-16T18:30:21.105 DEBUG:boto:Token: None 2024-04-16T18:30:21.105 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.105 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.109 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053309eeeed639a03-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.150 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.151 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.151 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.151 DEBUG:boto:path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:21.151 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj92 2024-04-16T18:30:21.151 DEBUG:boto:Method: GET 2024-04-16T18:30:21.151 DEBUG:boto:Path: /lybtug-52/?prefix=obj92 2024-04-16T18:30:21.151 DEBUG:boto:Data: 2024-04-16T18:30:21.151 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.151 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.151 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.152 DEBUG:boto:Params: {} 2024-04-16T18:30:21.152 DEBUG:boto:Token: None 2024-04-16T18:30:21.152 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.152 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.158 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003eba851f140a5d3f-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.158 DEBUG:boto:b'lybtug-52obj921000falseobj922024-04-16T18:23:25.782Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.159 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:30:21.159 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.159 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.159 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.159 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.159 DEBUG:boto:Method: GET 2024-04-16T18:30:21.159 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.159 DEBUG:boto:Data: 2024-04-16T18:30:21.160 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.160 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.160 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.160 DEBUG:boto:Params: {} 2024-04-16T18:30:21.160 DEBUG:boto:Token: None 2024-04-16T18:30:21.160 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.160 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc= 2024-04-16T18:30:21.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc=', 'Content-Length': '0'} 2024-04-16T18:30:21.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003caa4bb9adfad34a-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.206 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.206 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.207 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.207 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.207 DEBUG:boto:Method: GET 2024-04-16T18:30:21.207 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.207 DEBUG:boto:Data: 2024-04-16T18:30:21.207 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.207 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.207 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.207 DEBUG:boto:Params: {} 2024-04-16T18:30:21.207 DEBUG:boto:Token: None 2024-04-16T18:30:21.207 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.208 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc= 2024-04-16T18:30:21.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc=', 'Content-Length': '0'} 2024-04-16T18:30:21.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b303e856bf0b037-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.211 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.211 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.212 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.212 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.212 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.212 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.212 DEBUG:boto:Data: 2024-04-16T18:30:21.212 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.212 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.212 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.212 DEBUG:boto:Params: {} 2024-04-16T18:30:21.212 DEBUG:boto:Token: None 2024-04-16T18:30:21.212 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.213 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ucq4wN8nCM0A/C7o6sCtnWHKbSo= 2024-04-16T18:30:21.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ucq4wN8nCM0A/C7o6sCtnWHKbSo=', 'Content-Length': '0'} 2024-04-16T18:30:21.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae1581875382876c-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.216 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.216 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.216 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.216 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.216 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.216 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.216 DEBUG:boto:Data: 2024-04-16T18:30:21.217 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.217 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.217 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.217 DEBUG:boto:Params: {} 2024-04-16T18:30:21.217 DEBUG:boto:Token: None 2024-04-16T18:30:21.217 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.217 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Ucq4wN8nCM0A/C7o6sCtnWHKbSo= 2024-04-16T18:30:21.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ucq4wN8nCM0A/C7o6sCtnWHKbSo=', 'Content-Length': '0'} 2024-04-16T18:30:21.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b761fb0ea5ce08a1-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.221 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:30:21.221 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.221 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.221 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.221 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.221 DEBUG:boto:Method: GET 2024-04-16T18:30:21.221 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.221 DEBUG:boto:Data: 2024-04-16T18:30:21.221 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.222 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.222 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.222 DEBUG:boto:Params: {} 2024-04-16T18:30:21.222 DEBUG:boto:Token: None 2024-04-16T18:30:21.222 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.222 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc= 2024-04-16T18:30:21.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc=', 'Content-Length': '0'} 2024-04-16T18:30:21.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000214c291c990e1d30-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.266 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.266 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.267 DEBUG:boto:path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.267 DEBUG:boto:auth_path=/lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.267 DEBUG:boto:Method: GET 2024-04-16T18:30:21.267 DEBUG:boto:Path: /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.267 DEBUG:boto:Data: 2024-04-16T18:30:21.267 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.267 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.267 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.267 DEBUG:boto:Params: {} 2024-04-16T18:30:21.267 DEBUG:boto:Token: None 2024-04-16T18:30:21.268 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93?versionId=null 2024-04-16T18:30:21.268 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc= 2024-04-16T18:30:21.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AtafqMEUp0uE0Cir/gScrutdYSc=', 'Content-Length': '0'} 2024-04-16T18:30:21.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a8fda6a2a5ab3b7-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.271 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.271 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.272 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.272 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.272 DEBUG:boto:Method: GET 2024-04-16T18:30:21.272 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:30:21.272 DEBUG:boto:Data: 2024-04-16T18:30:21.272 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.272 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.272 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.272 DEBUG:boto:Params: {} 2024-04-16T18:30:21.272 DEBUG:boto:Token: None 2024-04-16T18:30:21.272 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.276 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0be6fb7ea3aea34-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.276 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.277 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.277 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.277 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.277 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.277 DEBUG:boto:Method: GET 2024-04-16T18:30:21.277 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:30:21.277 DEBUG:boto:Data: 2024-04-16T18:30:21.277 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.278 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.278 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.278 DEBUG:boto:Params: {} 2024-04-16T18:30:21.278 DEBUG:boto:Token: None 2024-04-16T18:30:21.278 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.278 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.285 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f6be15d49825f68-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.326 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.326 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.327 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.327 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.327 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:30:21.327 DEBUG:boto:Data: 2024-04-16T18:30:21.327 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.327 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.327 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.327 DEBUG:boto:Params: {} 2024-04-16T18:30:21.327 DEBUG:boto:Token: None 2024-04-16T18:30:21.328 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93 2024-04-16T18:30:21.328 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q0I1y6aZ9AvZcQ35EJjh21jSM/8= 2024-04-16T18:30:21.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q0I1y6aZ9AvZcQ35EJjh21jSM/8=', 'Content-Length': '0'} 2024-04-16T18:30:21.331 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000706beedcba6f9ccb-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.332 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.332 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.332 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.332 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:30:21.332 DEBUG:boto:Data: 2024-04-16T18:30:21.332 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.332 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.332 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.332 DEBUG:boto:Params: {} 2024-04-16T18:30:21.333 DEBUG:boto:Token: None 2024-04-16T18:30:21.333 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93 2024-04-16T18:30:21.333 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Q0I1y6aZ9AvZcQ35EJjh21jSM/8= 2024-04-16T18:30:21.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Q0I1y6aZ9AvZcQ35EJjh21jSM/8=', 'Content-Length': '0'} 2024-04-16T18:30:21.336 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bbb36d6512f0231-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.336 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2024-04-16T18:30:21.336 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.336 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.336 DEBUG:boto:Method: GET 2024-04-16T18:30:21.336 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:30:21.336 DEBUG:boto:Data: 2024-04-16T18:30:21.336 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.337 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.337 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.337 DEBUG:boto:Params: {} 2024-04-16T18:30:21.337 DEBUG:boto:Token: None 2024-04-16T18:30:21.337 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93 2024-04-16T18:30:21.337 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MB4uQye53LMxBg03IoqqC9v5K6g= 2024-04-16T18:30:21.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MB4uQye53LMxBg03IoqqC9v5K6g=', 'Content-Length': '0'} 2024-04-16T18:30:21.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c4e582245bf8028-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.382 DEBUG:boto:path=/lybtug-52/obj93 2024-04-16T18:30:21.383 DEBUG:boto:auth_path=/lybtug-52/obj93 2024-04-16T18:30:21.383 DEBUG:boto:Method: GET 2024-04-16T18:30:21.383 DEBUG:boto:Path: /lybtug-52/obj93 2024-04-16T18:30:21.383 DEBUG:boto:Data: 2024-04-16T18:30:21.383 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.383 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.383 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.383 DEBUG:boto:Params: {} 2024-04-16T18:30:21.383 DEBUG:boto:Token: None 2024-04-16T18:30:21.383 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj93 2024-04-16T18:30:21.384 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:MB4uQye53LMxBg03IoqqC9v5K6g= 2024-04-16T18:30:21.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MB4uQye53LMxBg03IoqqC9v5K6g=', 'Content-Length': '0'} 2024-04-16T18:30:21.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:26:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ac82a658b8e7e8c-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.387 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.387 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.387 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.387 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.387 DEBUG:boto:Method: GET 2024-04-16T18:30:21.387 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:30:21.387 DEBUG:boto:Data: 2024-04-16T18:30:21.387 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.387 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.387 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.387 DEBUG:boto:Params: {} 2024-04-16T18:30:21.388 DEBUG:boto:Token: None 2024-04-16T18:30:21.388 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.388 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e9696e2a6bf90b5-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.434 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.435 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.435 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.435 DEBUG:boto:path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.435 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj93 2024-04-16T18:30:21.435 DEBUG:boto:Method: GET 2024-04-16T18:30:21.435 DEBUG:boto:Path: /lybtug-52/?prefix=obj93 2024-04-16T18:30:21.435 DEBUG:boto:Data: 2024-04-16T18:30:21.436 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.436 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.436 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.436 DEBUG:boto:Params: {} 2024-04-16T18:30:21.436 DEBUG:boto:Token: None 2024-04-16T18:30:21.436 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.436 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3caeca5afc3cd89-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.443 DEBUG:boto:b'lybtug-52obj931000falseobj932024-04-16T18:26:49.317Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.443 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:30:21.443 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.443 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.444 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.444 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.444 DEBUG:boto:Method: GET 2024-04-16T18:30:21.444 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.444 DEBUG:boto:Data: 2024-04-16T18:30:21.444 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.444 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.444 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.444 DEBUG:boto:Params: {} 2024-04-16T18:30:21.444 DEBUG:boto:Token: None 2024-04-16T18:30:21.445 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.445 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A= 2024-04-16T18:30:21.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A=', 'Content-Length': '0'} 2024-04-16T18:30:21.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001558e264278af473-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.491 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.491 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.491 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.491 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.491 DEBUG:boto:Method: GET 2024-04-16T18:30:21.491 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.491 DEBUG:boto:Data: 2024-04-16T18:30:21.491 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.491 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.491 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.492 DEBUG:boto:Params: {} 2024-04-16T18:30:21.492 DEBUG:boto:Token: None 2024-04-16T18:30:21.492 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.492 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A= 2024-04-16T18:30:21.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A=', 'Content-Length': '0'} 2024-04-16T18:30:21.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007531efba6e1f05c-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.495 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.496 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.496 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.496 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.496 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.496 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.496 DEBUG:boto:Data: 2024-04-16T18:30:21.496 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.496 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.496 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.496 DEBUG:boto:Params: {} 2024-04-16T18:30:21.497 DEBUG:boto:Token: None 2024-04-16T18:30:21.497 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.497 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmMVytv8x7gLfP29wqQ7/jgn0E4= 2024-04-16T18:30:21.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmMVytv8x7gLfP29wqQ7/jgn0E4=', 'Content-Length': '0'} 2024-04-16T18:30:21.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fd639f113951fae-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.500 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.500 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.500 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.500 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.501 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.501 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.501 DEBUG:boto:Data: 2024-04-16T18:30:21.501 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.501 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.501 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.501 DEBUG:boto:Params: {} 2024-04-16T18:30:21.501 DEBUG:boto:Token: None 2024-04-16T18:30:21.501 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.501 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:lmMVytv8x7gLfP29wqQ7/jgn0E4= 2024-04-16T18:30:21.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lmMVytv8x7gLfP29wqQ7/jgn0E4=', 'Content-Length': '0'} 2024-04-16T18:30:21.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000400d55c0d6bd9370-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.505 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:30:21.505 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.505 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.505 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.505 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.505 DEBUG:boto:Method: GET 2024-04-16T18:30:21.505 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.505 DEBUG:boto:Data: 2024-04-16T18:30:21.505 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.506 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.506 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.506 DEBUG:boto:Params: {} 2024-04-16T18:30:21.506 DEBUG:boto:Token: None 2024-04-16T18:30:21.506 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.506 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A= 2024-04-16T18:30:21.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A=', 'Content-Length': '0'} 2024-04-16T18:30:21.509 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004881fda77a6159e3-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.550 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.550 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.551 DEBUG:boto:path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.551 DEBUG:boto:auth_path=/lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.551 DEBUG:boto:Method: GET 2024-04-16T18:30:21.551 DEBUG:boto:Path: /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.551 DEBUG:boto:Data: 2024-04-16T18:30:21.551 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.551 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.551 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.551 DEBUG:boto:Params: {} 2024-04-16T18:30:21.551 DEBUG:boto:Token: None 2024-04-16T18:30:21.552 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95?versionId=null 2024-04-16T18:30:21.552 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A= 2024-04-16T18:30:21.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nf3yngNJ2vzscJ3KfD731IfkU7A=', 'Content-Length': '0'} 2024-04-16T18:30:21.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2983f32085ca56d-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.555 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.555 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.555 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.555 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.555 DEBUG:boto:Method: GET 2024-04-16T18:30:21.555 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:30:21.556 DEBUG:boto:Data: 2024-04-16T18:30:21.556 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.556 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.556 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.556 DEBUG:boto:Params: {} 2024-04-16T18:30:21.556 DEBUG:boto:Token: None 2024-04-16T18:30:21.556 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.556 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.560 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec457db4db6db9cf-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.560 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.561 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.561 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.561 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.561 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.561 DEBUG:boto:Method: GET 2024-04-16T18:30:21.561 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:30:21.561 DEBUG:boto:Data: 2024-04-16T18:30:21.561 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.561 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.561 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.562 DEBUG:boto:Params: {} 2024-04-16T18:30:21.562 DEBUG:boto:Token: None 2024-04-16T18:30:21.562 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.562 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.569 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e83df4ba5627ac1e-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.610 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.611 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.611 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.611 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.611 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:30:21.611 DEBUG:boto:Data: 2024-04-16T18:30:21.611 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.611 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.611 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.611 DEBUG:boto:Params: {} 2024-04-16T18:30:21.612 DEBUG:boto:Token: None 2024-04-16T18:30:21.612 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95 2024-04-16T18:30:21.612 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X1rrfMiUSsXsqruRsPzx8uEat1A= 2024-04-16T18:30:21.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X1rrfMiUSsXsqruRsPzx8uEat1A=', 'Content-Length': '0'} 2024-04-16T18:30:21.615 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000970faeb030ef3703-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.615 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.615 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.615 DEBUG:boto:Method: HEAD 2024-04-16T18:30:21.615 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:30:21.615 DEBUG:boto:Data: 2024-04-16T18:30:21.616 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.616 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.616 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.616 DEBUG:boto:Params: {} 2024-04-16T18:30:21.616 DEBUG:boto:Token: None 2024-04-16T18:30:21.616 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95 2024-04-16T18:30:21.616 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:X1rrfMiUSsXsqruRsPzx8uEat1A= 2024-04-16T18:30:21.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:X1rrfMiUSsXsqruRsPzx8uEat1A=', 'Content-Length': '0'} 2024-04-16T18:30:21.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fe7e6a864114895-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2024-04-16T18:30:21.620 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.620 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.620 DEBUG:boto:Method: GET 2024-04-16T18:30:21.620 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:30:21.620 DEBUG:boto:Data: 2024-04-16T18:30:21.620 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.620 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.620 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.620 DEBUG:boto:Params: {} 2024-04-16T18:30:21.621 DEBUG:boto:Token: None 2024-04-16T18:30:21.621 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95 2024-04-16T18:30:21.621 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RpmbPFbNqs5sAEdjFI/APTunozY= 2024-04-16T18:30:21.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RpmbPFbNqs5sAEdjFI/APTunozY=', 'Content-Length': '0'} 2024-04-16T18:30:21.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-52:054102a3-8d94-4596-8b3b-6341eab312e1.4804.21'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:30:10 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018c93764a1830d45-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.666 DEBUG:boto:path=/lybtug-52/obj95 2024-04-16T18:30:21.667 DEBUG:boto:auth_path=/lybtug-52/obj95 2024-04-16T18:30:21.667 DEBUG:boto:Method: GET 2024-04-16T18:30:21.667 DEBUG:boto:Path: /lybtug-52/obj95 2024-04-16T18:30:21.667 DEBUG:boto:Data: 2024-04-16T18:30:21.667 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.667 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.667 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.667 DEBUG:boto:Params: {} 2024-04-16T18:30:21.667 DEBUG:boto:Token: None 2024-04-16T18:30:21.667 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/obj95 2024-04-16T18:30:21.668 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:RpmbPFbNqs5sAEdjFI/APTunozY= 2024-04-16T18:30:21.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RpmbPFbNqs5sAEdjFI/APTunozY=', 'Content-Length': '0'} 2024-04-16T18:30:21.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:22:20 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1315ee87bd92ba6-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.672 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.672 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.672 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.672 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.672 DEBUG:boto:Method: GET 2024-04-16T18:30:21.672 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:30:21.672 DEBUG:boto:Data: 2024-04-16T18:30:21.672 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.672 DEBUG:boto:Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T18:30:21.673 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.673 DEBUG:boto:Params: {} 2024-04-16T18:30:21.673 DEBUG:boto:Token: None 2024-04-16T18:30:21.673 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.673 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.677 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000feeb8d68b97772f8-00661ec3bd-6022-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.719 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.721 DEBUG:boto:path=/lybtug-52/ 2024-04-16T18:30:21.721 DEBUG:boto:auth_path=/lybtug-52/ 2024-04-16T18:30:21.722 DEBUG:boto:path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.722 DEBUG:boto:auth_path=/lybtug-52/?prefix=obj95 2024-04-16T18:30:21.722 DEBUG:boto:Method: GET 2024-04-16T18:30:21.722 DEBUG:boto:Path: /lybtug-52/?prefix=obj95 2024-04-16T18:30:21.722 DEBUG:boto:Data: 2024-04-16T18:30:21.722 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.722 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.722 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.722 DEBUG:boto:Params: {} 2024-04-16T18:30:21.723 DEBUG:boto:Token: None 2024-04-16T18:30:21.723 DEBUG:boto:StringToSign: GET Tue, 16 Apr 2024 18:30:21 GMT /lybtug-52/ 2024-04-16T18:30:21.724 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI= 2024-04-16T18:30:21.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UlfgQOXwMJBWjOmyNVzjimkySnI=', 'Content-Length': '0'} 2024-04-16T18:30:21.730 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000805b544cd68bd2d0-00661ec3bd-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.731 DEBUG:boto:b'lybtug-52obj951000falseobj952024-04-16T18:22:20.973Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T18:30:21.731 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=lybtug-52 zones={test-zone2, test-zone1} 2024-04-16T18:30:21.732 INFO:tasks.rgw_multisite_tests:Create several generations of objects, trash them, then run bucket sync init ... ok 2024-04-16T18:30:21.737 DEBUG:boto:path=/lybtug-53/ 2024-04-16T18:30:21.737 DEBUG:boto:auth_path=/lybtug-53/ 2024-04-16T18:30:21.737 DEBUG:boto:Method: PUT 2024-04-16T18:30:21.737 DEBUG:boto:Path: /lybtug-53/ 2024-04-16T18:30:21.737 DEBUG:boto:Data: 2024-04-16T18:30:21.738 DEBUG:boto:Headers: {} 2024-04-16T18:30:21.738 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:30:21.738 DEBUG:boto:Port: 8000 2024-04-16T18:30:21.738 DEBUG:boto:Params: {} 2024-04-16T18:30:21.738 DEBUG:boto:Token: None 2024-04-16T18:30:21.738 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:30:21 GMT /lybtug-53/ 2024-04-16T18:30:21.738 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:6foZE5avGfrCC32f0KyliHdyaac= 2024-04-16T18:30:21.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6foZE5avGfrCC32f0KyliHdyaac=', 'Content-Length': '0'} 2024-04-16T18:30:21.775 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005fa82e70366266bc-00661ec3bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:30:21.776 DEBUG:rgw_multi.tests:created bucket=lybtug-53 2024-04-16T18:30:21.776 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:21.776 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:21.776 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:21.828 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:21.828 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:21.899 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:30:21.931 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713292221.776788_782.1","last_update":"2024-04-16T18:30:21.776788Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:30:21.932 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:21.932 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:30:21.932 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:21.932 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:21.932 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:21.975 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:21.975 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:22.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:22.047+0000 7fd302a1e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:22.051+0000 7fd302a1e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:22.085 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:22Z"} 2024-04-16T18:30:22.085 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:22Z"} 2024-04-16T18:30:22.085 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:22.085 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:22.085 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:22.085 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:27.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:27.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:27.087 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:27.129 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:27.129 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:27.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:27.199+0000 7f664ac60980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:27.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:27.203+0000 7f664ac60980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:27.236 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:27Z"} 2024-04-16T18:30:27.236 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:27Z"} 2024-04-16T18:30:27.236 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:27.236 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:27.236 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:27.236 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:32.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:32.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:32.237 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:32.280 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:32.280 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:32.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:32.351+0000 7f98ceb0d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:32.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:32.355+0000 7f98ceb0d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:32.388 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:32Z"} 2024-04-16T18:30:32.389 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:32Z"} 2024-04-16T18:30:32.389 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:32.389 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:32.389 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:32.389 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:37.391 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:37.391 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:37.391 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:37.435 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:37.435 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:37.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:37.507+0000 7fb5509fc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:37.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:37.507+0000 7fb5509fc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:37.543 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:37Z"} 2024-04-16T18:30:37.543 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:37Z"} 2024-04-16T18:30:37.543 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:37.544 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:37.544 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:37.544 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:42.545 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:42.545 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:42.545 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:42.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:42.658+0000 7f4311f79980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:42.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:42.658+0000 7f4311f79980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:42.696 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:42Z"} 2024-04-16T18:30:42.696 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:42Z"} 2024-04-16T18:30:42.697 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:42.697 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:42.697 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:42.697 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:47.698 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:47.698 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:47.699 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:47.742 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:47.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:47.810+0000 7f009966d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:47.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:47.814+0000 7f009966d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:47.849 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:47Z"} 2024-04-16T18:30:47.850 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:47Z"} 2024-04-16T18:30:47.850 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:47.850 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:47.850 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:47.850 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:52.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:52.851 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:52.852 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:52.962+0000 7f7ab37bd980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:52.962+0000 7f7ab37bd980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:52.999 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:52Z"} 2024-04-16T18:30:52.999 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:52Z"} 2024-04-16T18:30:52.999 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:53.000 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:53.000 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:53.000 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:30:58.000 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:58.000 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:30:58.001 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:30:58.043 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:30:58.043 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:30:58.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:58.110+0000 7fb60c360980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:58.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:30:58.114+0000 7fb60c360980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:30:58.148 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:58Z"} 2024-04-16T18:30:58.148 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:58Z"} 2024-04-16T18:30:58.148 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:30:58.148 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:58.148 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:30:58.149 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:03.149 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:03.150 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:03.150 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:03.194 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:03.194 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:03.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:03.262+0000 7f169c760980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:03.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:03.262+0000 7f169c760980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:03.299 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:03Z"} 2024-04-16T18:31:03.299 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:03Z"} 2024-04-16T18:31:03.300 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:03.300 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:03.300 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:03.300 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:08.301 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:08.301 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:08.301 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:08.344 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:08.344 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:08.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:08.414+0000 7f0f4f1d0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:08.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:08.418+0000 7f0f4f1d0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:08.456 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:08Z"} 2024-04-16T18:31:08.456 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:08Z"} 2024-04-16T18:31:08.456 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:08.456 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:08.456 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:08.457 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:13.457 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:13.457 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:13.458 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:13.500 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:13.500 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:13.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:13.570+0000 7fc0600c5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:13.570+0000 7fc0600c5980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:13.608 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:13Z"} 2024-04-16T18:31:13.608 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:13Z"} 2024-04-16T18:31:13.609 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:13.609 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:13.609 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:13.609 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:18.610 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:18.610 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:18.611 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:18.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:18.726+0000 7fba24108980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:18.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:18.730+0000 7fba24108980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:18.766 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:18Z"} 2024-04-16T18:31:18.766 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:18Z"} 2024-04-16T18:31:18.766 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:18.767 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:18.767 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:18.767 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:23.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:23.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:23.768 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:23.881+0000 7f55c1fd3980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:23.881+0000 7f55c1fd3980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:23.918 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:23Z"} 2024-04-16T18:31:23.918 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:23Z"} 2024-04-16T18:31:23.918 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:23.919 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:23.919 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:23.919 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:28.920 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:28.920 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:28.920 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:28.964 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:28.964 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:29.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:29.033+0000 7f8f573b0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:29.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:29.033+0000 7f8f573b0980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:29.072 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:29Z"} 2024-04-16T18:31:29.072 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:29Z"} 2024-04-16T18:31:29.073 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:29.073 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:29.073 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:29.073 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:34.074 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:34.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:34.074 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:34.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:34.185+0000 7f51f0a81980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:34.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:34.189+0000 7f51f0a81980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:34.229 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:34Z"} 2024-04-16T18:31:34.229 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:34Z"} 2024-04-16T18:31:34.229 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:34.230 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:34.230 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:34.230 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:39.230 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:39.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:39.231 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:39.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:39.345+0000 7f269caf1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:39.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:39.349+0000 7f269caf1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:39.385 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:39Z"} 2024-04-16T18:31:39.385 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:39Z"} 2024-04-16T18:31:39.385 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:39.386 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:39.386 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:39.386 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:44.387 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:44.387 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:44.387 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:44.431 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:44.431 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:44.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:44.501+0000 7f97db4d9980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:44.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:44.501+0000 7f97db4d9980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:44.539 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:44Z"} 2024-04-16T18:31:44.539 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:44Z"} 2024-04-16T18:31:44.539 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:44.539 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:44.539 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:44.539 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:49.540 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:49.540 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:49.540 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:49.584 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:49.584 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:49.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:49.657+0000 7fe47ef6d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:49.661+0000 7fe47ef6d980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:49.694 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:49Z"} 2024-04-16T18:31:49.694 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:49Z"} 2024-04-16T18:31:49.694 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:49.694 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:49.694 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:49.695 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:54.695 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:54.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:54.696 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:54.740 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:54.740 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:54.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:54.813+0000 7fb19fcee980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:54.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:54.813+0000 7fb19fcee980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:54.848 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:54Z"} 2024-04-16T18:31:54.848 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:54Z"} 2024-04-16T18:31:54.848 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:31:54.848 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:54.849 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:31:54.849 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T18:31:59.850 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:59.850 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:31:59.851 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:31:59.894 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:31:59.894 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:31:59.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:59.968+0000 7f0153b35980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:31:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:31:59.968+0000 7f0153b35980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:32:00.007 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.756655_779.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.756655Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:59Z"} 2024-04-16T18:32:00.007 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.756655_779.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.756655Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:59Z"} 2024-04-16T18:32:00.007 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.756655_779.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.756655Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:32:00.008 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:32:00.008 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.756655_779.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:32:00.008 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292221.756655_779.1 2024-04-16T18:32:05.008 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:32:05.009 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:32:05.009 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:32:05.051 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:32:05.051 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:32:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:05.120+0000 7fa3bdade980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:32:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:05.124+0000 7fa3bdade980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:32:05.159 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:32:05Z"} 2024-04-16T18:32:05.159 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:32:05Z"} 2024-04-16T18:32:05.159 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:32:05.160 DEBUG:rgw_multi.tests:log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:32:05.160 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:32:05.160 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2024-04-16T18:32:05.160 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2024-04-16T18:32:05.160 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T18:32:05.203 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:32:05.204+0000 7f2049a56640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 172624) UID: 0 2024-04-16T18:32:05.203 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:32:05.204+0000 7f204d29ea40 -1 shutting down 2024-04-16T18:32:05.723 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T18:32:11.262 INFO:tasks.rgw.c2.client.0:Stopped 2024-04-16T18:32:11.263 DEBUG:boto:path=/lybtug-53/ 2024-04-16T18:32:11.263 DEBUG:boto:auth_path=/lybtug-53/ 2024-04-16T18:32:11.263 DEBUG:boto:Method: HEAD 2024-04-16T18:32:11.263 DEBUG:boto:Path: /lybtug-53/ 2024-04-16T18:32:11.263 DEBUG:boto:Data: 2024-04-16T18:32:11.263 DEBUG:boto:Headers: {} 2024-04-16T18:32:11.263 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.263 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.264 DEBUG:boto:Params: {} 2024-04-16T18:32:11.264 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:32:11.264 DEBUG:boto:Token: None 2024-04-16T18:32:11.265 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/ 2024-04-16T18:32:11.265 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T18:32:11.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T18:32:11.271 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000357c52ce1450970-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.272 DEBUG:boto:path=/lybtug-53/obj11 2024-04-16T18:32:11.272 DEBUG:boto:auth_path=/lybtug-53/obj11 2024-04-16T18:32:11.272 DEBUG:boto:Method: PUT 2024-04-16T18:32:11.272 DEBUG:boto:Path: /lybtug-53/obj11 2024-04-16T18:32:11.272 DEBUG:boto:Data: 2024-04-16T18:32:11.272 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:32:11.272 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.273 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.273 DEBUG:boto:Params: {} 2024-04-16T18:32:11.273 DEBUG:boto:Token: None 2024-04-16T18:32:11.273 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/obj11 2024-04-16T18:32:11.273 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:o4cV6vQ+EimVnLYacefuo9kDQMk= 2024-04-16T18:32:11.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o4cV6vQ+EimVnLYacefuo9kDQMk='} 2024-04-16T18:32:11.287 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000556044d3f10dbf73-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.287 DEBUG:boto:path=/lybtug-53/ 2024-04-16T18:32:11.287 DEBUG:boto:auth_path=/lybtug-53/ 2024-04-16T18:32:11.287 DEBUG:boto:Method: HEAD 2024-04-16T18:32:11.287 DEBUG:boto:Path: /lybtug-53/ 2024-04-16T18:32:11.287 DEBUG:boto:Data: 2024-04-16T18:32:11.288 DEBUG:boto:Headers: {} 2024-04-16T18:32:11.288 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.288 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.288 DEBUG:boto:Params: {} 2024-04-16T18:32:11.288 DEBUG:boto:Token: None 2024-04-16T18:32:11.288 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/ 2024-04-16T18:32:11.288 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T18:32:11.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T18:32:11.293 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009c755159c28cfe0b-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.294 DEBUG:boto:path=/lybtug-53/obj22 2024-04-16T18:32:11.294 DEBUG:boto:auth_path=/lybtug-53/obj22 2024-04-16T18:32:11.294 DEBUG:boto:Method: PUT 2024-04-16T18:32:11.294 DEBUG:boto:Path: /lybtug-53/obj22 2024-04-16T18:32:11.294 DEBUG:boto:Data: 2024-04-16T18:32:11.294 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:32:11.294 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.294 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.295 DEBUG:boto:Params: {} 2024-04-16T18:32:11.295 DEBUG:boto:Token: None 2024-04-16T18:32:11.295 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/obj22 2024-04-16T18:32:11.295 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:JRUE6BT4lE86I8Jt1/XmAILTJFY= 2024-04-16T18:32:11.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JRUE6BT4lE86I8Jt1/XmAILTJFY='} 2024-04-16T18:32:11.338 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000029059ff0bc78fa64-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.339 DEBUG:boto:path=/lybtug-53/ 2024-04-16T18:32:11.339 DEBUG:boto:auth_path=/lybtug-53/ 2024-04-16T18:32:11.339 DEBUG:boto:Method: HEAD 2024-04-16T18:32:11.339 DEBUG:boto:Path: /lybtug-53/ 2024-04-16T18:32:11.339 DEBUG:boto:Data: 2024-04-16T18:32:11.339 DEBUG:boto:Headers: {} 2024-04-16T18:32:11.339 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.339 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.339 DEBUG:boto:Params: {} 2024-04-16T18:32:11.340 DEBUG:boto:Token: None 2024-04-16T18:32:11.340 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/ 2024-04-16T18:32:11.340 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T18:32:11.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T18:32:11.343 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0dd1f6b07023ca8-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.343 DEBUG:boto:path=/lybtug-53/obj33 2024-04-16T18:32:11.344 DEBUG:boto:auth_path=/lybtug-53/obj33 2024-04-16T18:32:11.344 DEBUG:boto:Method: PUT 2024-04-16T18:32:11.344 DEBUG:boto:Path: /lybtug-53/obj33 2024-04-16T18:32:11.344 DEBUG:boto:Data: 2024-04-16T18:32:11.344 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:32:11.344 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.344 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.344 DEBUG:boto:Params: {} 2024-04-16T18:32:11.344 DEBUG:boto:Token: None 2024-04-16T18:32:11.344 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/obj33 2024-04-16T18:32:11.344 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:7lvHXQrcShBVnuNlCKh4RGu2uuQ= 2024-04-16T18:32:11.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7lvHXQrcShBVnuNlCKh4RGu2uuQ='} 2024-04-16T18:32:11.361 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007598d797b482dace-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.361 DEBUG:boto:path=/lybtug-53/ 2024-04-16T18:32:11.361 DEBUG:boto:auth_path=/lybtug-53/ 2024-04-16T18:32:11.361 DEBUG:boto:Method: HEAD 2024-04-16T18:32:11.361 DEBUG:boto:Path: /lybtug-53/ 2024-04-16T18:32:11.362 DEBUG:boto:Data: 2024-04-16T18:32:11.362 DEBUG:boto:Headers: {} 2024-04-16T18:32:11.362 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.362 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.362 DEBUG:boto:Params: {} 2024-04-16T18:32:11.362 DEBUG:boto:Token: None 2024-04-16T18:32:11.362 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/ 2024-04-16T18:32:11.362 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T18:32:11.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T18:32:11.366 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000005830cdda77757d4-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.366 DEBUG:boto:path=/lybtug-53/obj44 2024-04-16T18:32:11.366 DEBUG:boto:auth_path=/lybtug-53/obj44 2024-04-16T18:32:11.366 DEBUG:boto:Method: PUT 2024-04-16T18:32:11.366 DEBUG:boto:Path: /lybtug-53/obj44 2024-04-16T18:32:11.366 DEBUG:boto:Data: 2024-04-16T18:32:11.366 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:32:11.367 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.367 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.367 DEBUG:boto:Params: {} 2024-04-16T18:32:11.367 DEBUG:boto:Token: None 2024-04-16T18:32:11.367 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/obj44 2024-04-16T18:32:11.367 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:4XgQWi9X9M9a4yboUrA40TNCzQY= 2024-04-16T18:32:11.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4XgQWi9X9M9a4yboUrA40TNCzQY='} 2024-04-16T18:32:11.410 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d5d94926014102d3-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.411 DEBUG:boto:path=/lybtug-53/ 2024-04-16T18:32:11.411 DEBUG:boto:auth_path=/lybtug-53/ 2024-04-16T18:32:11.411 DEBUG:boto:Method: HEAD 2024-04-16T18:32:11.411 DEBUG:boto:Path: /lybtug-53/ 2024-04-16T18:32:11.411 DEBUG:boto:Data: 2024-04-16T18:32:11.411 DEBUG:boto:Headers: {} 2024-04-16T18:32:11.411 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.411 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.411 DEBUG:boto:Params: {} 2024-04-16T18:32:11.412 DEBUG:boto:Token: None 2024-04-16T18:32:11.412 DEBUG:boto:StringToSign: HEAD Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/ 2024-04-16T18:32:11.412 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T18:32:11.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T18:32:11.416 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004c4f195b88d87bd1-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.416 DEBUG:boto:path=/lybtug-53/obj55 2024-04-16T18:32:11.416 DEBUG:boto:auth_path=/lybtug-53/obj55 2024-04-16T18:32:11.416 DEBUG:boto:Method: PUT 2024-04-16T18:32:11.416 DEBUG:boto:Path: /lybtug-53/obj55 2024-04-16T18:32:11.416 DEBUG:boto:Data: 2024-04-16T18:32:11.416 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T18:32:11.416 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:32:11.417 DEBUG:boto:Port: 8000 2024-04-16T18:32:11.417 DEBUG:boto:Params: {} 2024-04-16T18:32:11.417 DEBUG:boto:Token: None 2024-04-16T18:32:11.417 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Tue, 16 Apr 2024 18:32:11 GMT /lybtug-53/obj55 2024-04-16T18:32:11.417 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yzkELej/F6miexxYRuT8z8Fu1Jg= 2024-04-16T18:32:11.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yzkELej/F6miexxYRuT8z8Fu1Jg='} 2024-04-16T18:32:11.426 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009dfcd4808bd5dde3-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:32:11.426 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:32:11.426 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T18:32:11.426 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync init --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T18:32:11.472 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:32:11.472 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:32:11.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:11.548+0000 7f5f14290980 5 RGW-SYNC:data: waiting on data sync status lease 2024-04-16T18:32:11.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:11.552+0000 7f5f14290980 5 RGW-SYNC:data: acquired data sync status lease 2024-04-16T18:32:11.603 INFO:tasks.rgw.c2.client.0:Restarting daemon 2024-04-16T18:32:11.604 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T18:32:11.605 INFO:tasks.rgw.c2.client.0:Started 2024-04-16T18:32:11.606 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:32:11.614 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:32:11.614 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:32:11.616 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-16T18:32:11.616 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T18:32:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:curl: (7) Failed to connect to smithi122.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T18:32:11.630 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2024-04-16T18:32:11.850 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:32:11.848+0000 7f30fcfe9a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T18:32:12.617 DEBUG:teuthology.orchestra.run.smithi122:> curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T18:32:12.629 INFO:teuthology.orchestra.run.smithi122.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T18:32:12.629 INFO:teuthology.orchestra.run.smithi122.stderr: Dload Upload Total Spent Left Speed 2024-04-16T18:32:12.630 INFO:teuthology.orchestra.run.smithi122.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 98008 0 --:--:-- --:--:-- --:--:-- 182k 2024-04-16T18:32:12.631 INFO:teuthology.orchestra.run.smithi122.stdout:anonymous 2024-04-16T18:32:12.632 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-53 2024-04-16T18:32:12.632 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-53', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:32:12.632 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-53', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T18:32:12.632 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-53 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T18:32:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:32:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:32:12.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:12.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:14.337 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:32:14.336+0000 7f2fe5579640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T18:32:17.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:17.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:22.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:22.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:27.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:32.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:32.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:37.808+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:42.807+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:47.807+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:52.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:52.807+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:32:57.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:32:57.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:02.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:02.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:07.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:07.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:12.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:12.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:17.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:17.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:22.814+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:27.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:27.814+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:32.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:32.814+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:37.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:37.818+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:42.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:42.818+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:45.740 INFO:tasks.rgw.c2.client.0.smithi122.stdout:*** Caught signal (Segmentation fault) ** 2024-04-16T18:33:45.740 INFO:tasks.rgw.c2.client.0.smithi122.stdout: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T18:33:45.742 INFO:tasks.rgw.c2.client.0.smithi122.stdout: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T18:33:45.742 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T18:33:45.742 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T18:33:45.742 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout:2024-04-16T18:33:45.738+0000 7f2fd8d60640 -1 *** Caught signal (Segmentation fault) ** 2024-04-16T18:33:45.743 INFO:tasks.rgw.c2.client.0.smithi122.stdout: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T18:33:45.744 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T18:33:45.745 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T18:33:45.745 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T18:33:45.745 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T18:33:45.745 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T18:33:45.745 INFO:tasks.rgw.c2.client.0.smithi122.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T18:33:45.745 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2024-04-16T18:33:45.755 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 0> 2024-04-16T18:33:45.738+0000 7f2fd8d60640 -1 *** Caught signal (Segmentation fault) ** 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T18:33:45.756 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T18:33:45.757 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T18:33:45.757 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T18:33:45.757 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T18:33:45.757 INFO:tasks.rgw.c2.client.0.smithi122.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T18:33:45.757 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2024-04-16T18:33:45.765 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 0> 2024-04-16T18:33:45.738+0000 7f2fd8d60640 -1 *** Caught signal (Segmentation fault) ** 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T18:33:45.766 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T18:33:45.767 INFO:tasks.rgw.c2.client.0.smithi122.stdout: 2024-04-16T18:33:45.783 INFO:tasks.rgw.c2.client.0.smithi122.stderr:daemon-helper: command crashed with signal 11 2024-04-16T18:33:47.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T18:33:47.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:47.818+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:52.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:52.822+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:53.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~6s 2024-04-16T18:33:57.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:33:57.822+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:33:59.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~12s 2024-04-16T18:34:02.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:02.822+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:05.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~18s 2024-04-16T18:34:07.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:07.821+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:11.305 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~24s 2024-04-16T18:34:12.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:12.825+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:17.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~30s 2024-04-16T18:34:17.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:17.825+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:22.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:22.829+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:23.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~35s 2024-04-16T18:34:27.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:27.829+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:29.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~41s 2024-04-16T18:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:32.833+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:34.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~47s 2024-04-16T18:34:37.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:37.833+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:40.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~53s 2024-04-16T18:34:42.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:42.833+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:46.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~59s 2024-04-16T18:34:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:47.837+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:52.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~65s 2024-04-16T18:34:52.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:52.836+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:57.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:34:57.836+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:34:58.572 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~71s 2024-04-16T18:35:02.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:02.840+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:04.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~77s 2024-04-16T18:35:07.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:07.840+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:10.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~83s 2024-04-16T18:35:12.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:12.840+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:16.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~89s 2024-04-16T18:35:17.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:17.844+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:22.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~95s 2024-04-16T18:35:22.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:22.844+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:27.844+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:28.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~100s 2024-04-16T18:35:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:32.848+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:34.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~106s 2024-04-16T18:35:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:37.847+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:39.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~112s 2024-04-16T18:35:42.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:42.847+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:45.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~118s 2024-04-16T18:35:47.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:47.851+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:51.733 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~124s 2024-04-16T18:35:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:52.851+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:35:57.641 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~130s 2024-04-16T18:35:57.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:35:57.851+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:02.855+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:03.549 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~136s 2024-04-16T18:36:07.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:07.855+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:09.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~142s 2024-04-16T18:36:12.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:12.855+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:15.364 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~148s 2024-04-16T18:36:17.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:17.859+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:21.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~154s 2024-04-16T18:36:22.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:22.858+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:27.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~160s 2024-04-16T18:36:27.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:27.858+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:32.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:32.858+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:33.088 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~165s 2024-04-16T18:36:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:37.862+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:38.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~171s 2024-04-16T18:36:42.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:42.862+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:44.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~177s 2024-04-16T18:36:47.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:47.866+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:50.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~183s 2024-04-16T18:36:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:52.866+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:36:56.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~189s 2024-04-16T18:36:57.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:36:57.866+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:37:02.629 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~195s 2024-04-16T18:37:02.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:02.870+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T18:37:07.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:07.869+0000 7f2217e66980 -1 bucket checkpoint timed out waiting to reach incremental sync 2024-04-16T18:37:07.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:07.869+0000 7f2217e66980 0 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T18:37:07.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:07.869+0000 7f2217e66980 -1 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T18:37:07.927 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-16T18:37:08.159 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_full_sync_after_data_sync_init ... FAIL 2024-04-16T18:37:08.160 DEBUG:boto:path=/lybtug-54/ 2024-04-16T18:37:08.160 DEBUG:boto:auth_path=/lybtug-54/ 2024-04-16T18:37:08.161 DEBUG:boto:Method: PUT 2024-04-16T18:37:08.161 DEBUG:boto:Path: /lybtug-54/ 2024-04-16T18:37:08.161 DEBUG:boto:Data: 2024-04-16T18:37:08.161 DEBUG:boto:Headers: {} 2024-04-16T18:37:08.161 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:37:08.161 DEBUG:boto:Port: 8000 2024-04-16T18:37:08.161 DEBUG:boto:Params: {} 2024-04-16T18:37:08.162 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:37:08.162 DEBUG:boto:Token: None 2024-04-16T18:37:08.162 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:37:08 GMT /lybtug-54/ 2024-04-16T18:37:08.162 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yLMV8/o+zR5KVj1vqZ2aQaig4Ag= 2024-04-16T18:37:08.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:37:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yLMV8/o+zR5KVj1vqZ2aQaig4Ag=', 'Content-Length': '0'} 2024-04-16T18:37:08.205 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003b537801dc8a76ba-00661ec554-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:37:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:37:08.205 DEBUG:rgw_multi.tests:created bucket=lybtug-54 2024-04-16T18:37:08.205 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:08.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:08.206 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:08.247 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:08.247 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:08.316 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:37:08.350 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713292628.204217_1458.1","last_update":"2024-04-16T18:37:08.204217Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:37:08.350 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:08.350 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:37:08.351 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:08.351 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:08.351 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:08.395 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:08.395 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:08.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:08.461+0000 7f88411e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:08.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:08.465+0000 7f88411e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:08.501 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:08Z"} 2024-04-16T18:37:08.501 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:08Z"} 2024-04-16T18:37:08.501 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:08.501 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:08.501 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:08.502 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:08.766 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~201s 2024-04-16T18:37:13.502 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:13.502 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:13.503 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:13.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:13.617+0000 7f8f72442980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:13.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:13.617+0000 7f8f72442980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:13.657 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:13Z"} 2024-04-16T18:37:13.657 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:13Z"} 2024-04-16T18:37:13.657 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:13.657 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:13.657 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:13.657 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:14.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~207s 2024-04-16T18:37:18.658 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:18.659 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:18.659 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:18.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:18.769+0000 7f12cac5e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:18.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:18.773+0000 7f12cac5e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:18.810 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:18Z"} 2024-04-16T18:37:18.810 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:18Z"} 2024-04-16T18:37:18.810 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:18.810 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:18.810 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:18.811 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:20.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~213s 2024-04-16T18:37:23.811 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:23.812 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:23.812 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:23.856 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:23.857 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:23.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:23.925+0000 7f3eededb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:23.929+0000 7f3eededb980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:23.965 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:23Z"} 2024-04-16T18:37:23.965 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:23Z"} 2024-04-16T18:37:23.966 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:23.966 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:23.966 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:23.966 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:26.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~219s 2024-04-16T18:37:28.967 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:28.967 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:28.967 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:29.011 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:29.081+0000 7f27114ba980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:29.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:29.081+0000 7f27114ba980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:29.120 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:29Z"} 2024-04-16T18:37:29.120 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:29Z"} 2024-04-16T18:37:29.120 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:29.120 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:29.121 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:29.121 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:32.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~225s 2024-04-16T18:37:34.121 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:34.121 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:34.122 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:34.166 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:34.166 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:34.241+0000 7f824035e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:34.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:34.241+0000 7f824035e980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:34.276 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:34Z"} 2024-04-16T18:37:34.276 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:34Z"} 2024-04-16T18:37:34.276 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:34.276 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:34.276 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:34.276 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:38.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~231s 2024-04-16T18:37:39.277 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:39.278 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:39.278 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:39.322 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:39.322 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:39.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:39.397+0000 7fe399049980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:39.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:39.397+0000 7fe399049980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:39.431 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:39Z"} 2024-04-16T18:37:39.431 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:39Z"} 2024-04-16T18:37:39.432 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:39.432 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:39.432 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:39.432 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:44.216 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~237s 2024-04-16T18:37:44.434 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:44.434 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:44.434 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:44.476 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:44.477 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:44.549+0000 7f440f7dd980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:44.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:44.553+0000 7f440f7dd980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:44.585 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:44Z"} 2024-04-16T18:37:44.585 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:44Z"} 2024-04-16T18:37:44.585 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:44.585 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:44.586 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:44.586 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:49.587 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:49.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:49.587 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:49.630 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:49.630 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:49.704+0000 7f94d9adc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:49.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:49.704+0000 7f94d9adc980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:49.740 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:49Z"} 2024-04-16T18:37:49.740 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:49Z"} 2024-04-16T18:37:49.740 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:49.740 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:49.740 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:49.740 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:50.125 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~242s 2024-04-16T18:37:54.742 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:54.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:54.742 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:54.785 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:54.785 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:37:54.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:54.860+0000 7f1cb5885980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:54.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:37:54.864+0000 7f1cb5885980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:37:54.896 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:54Z"} 2024-04-16T18:37:54.896 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:54Z"} 2024-04-16T18:37:54.897 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:37:54.897 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:54.897 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:37:54.897 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:37:56.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~248s 2024-04-16T18:37:59.898 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:59.898 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:37:59.898 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:37:59.937 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:37:59.937 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:00.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:00.008+0000 7f4491fe4980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:00.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:00.008+0000 7f4491fe4980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:00.044 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:00Z"} 2024-04-16T18:38:00.044 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:00Z"} 2024-04-16T18:38:00.044 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:00.044 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:00.044 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:00.045 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:01.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~254s 2024-04-16T18:38:05.046 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:05.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:05.046 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:05.089 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:05.089 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:05.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:05.164+0000 7f14e9ad6980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:05.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:05.164+0000 7f14e9ad6980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:05.200 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:05Z"} 2024-04-16T18:38:05.200 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:05Z"} 2024-04-16T18:38:05.201 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:05.201 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:05.201 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:05.201 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:07.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~260s 2024-04-16T18:38:10.202 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:10.202 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:10.203 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:10.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:10.316+0000 7f87f6831980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:10.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:10.320+0000 7f87f6831980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:10.352 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:10Z"} 2024-04-16T18:38:10.353 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:10Z"} 2024-04-16T18:38:10.353 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:10.353 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:10.353 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:10.353 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:13.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~266s 2024-04-16T18:38:15.354 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:15.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:15.354 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:15.398 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:15.398 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:15.472+0000 7f8242973980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:15.472+0000 7f8242973980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:15.508 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:15Z"} 2024-04-16T18:38:15.508 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:15Z"} 2024-04-16T18:38:15.508 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:15.508 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:15.508 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:15.508 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:19.664 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~272s 2024-04-16T18:38:20.509 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:20.509 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:20.509 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:20.553 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:20.553 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:20.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:20.624+0000 7f3ef41f3980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:20.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:20.628+0000 7f3ef41f3980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:20.662 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:20Z"} 2024-04-16T18:38:20.662 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:20Z"} 2024-04-16T18:38:20.662 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:20.662 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:20.663 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:20.663 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:25.573 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~278s 2024-04-16T18:38:25.663 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:25.664 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:25.664 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:25.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:25.780+0000 7fefb5766980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:25.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:25.784+0000 7fefb5766980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:25.818 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:25Z"} 2024-04-16T18:38:25.818 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:25Z"} 2024-04-16T18:38:25.819 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:25.819 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:25.819 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:25.819 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:30.820 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:30.820 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:30.820 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:30.864 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:30.865 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:30.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:30.932+0000 7f9a3d29b980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:30.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:30.936+0000 7f9a3d29b980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:30.971 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:30Z"} 2024-04-16T18:38:30.971 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:30Z"} 2024-04-16T18:38:30.971 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:30.971 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:30.972 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:30.972 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:31.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~284s 2024-04-16T18:38:35.973 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:35.974 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:35.974 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:36.017 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:36.018 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:36.087+0000 7f81be26a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:36.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:36.091+0000 7f81be26a980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:36.125 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:36Z"} 2024-04-16T18:38:36.126 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:36Z"} 2024-04-16T18:38:36.126 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:36.126 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:36.127 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:36.127 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:37.386 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~290s 2024-04-16T18:38:41.128 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:41.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:41.128 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:41.172 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:41.172 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:41.243+0000 7f64db882980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:41.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:41.247+0000 7f64db882980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:41.282 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:41Z"} 2024-04-16T18:38:41.283 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:41Z"} 2024-04-16T18:38:41.283 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:41.283 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:41.283 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:41.283 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:43.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~296s 2024-04-16T18:38:46.284 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:46.285 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:46.285 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:46.329 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:46.329 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:38:46.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:46.399+0000 7fd3ba063980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:46.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:38:46.403+0000 7fd3ba063980 20 RGW-SYNC:meta: read sync status 2024-04-16T18:38:46.438 INFO:teuthology.orchestra.run.smithi122.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:46Z"} 2024-04-16T18:38:46.438 DEBUG:rgw_multi.tests:current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:46Z"} 2024-04-16T18:38:46.439 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T18:38:46.439 DEBUG:rgw_multi.tests:log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:46.439 DEBUG:rgw_multi.tests:sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:38:46.439 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T18:38:49.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~302s 2024-04-16T18:38:49.200 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-16T18:38:50.106 INFO:tasks.ceph.c2.osd.0:Sent signal 15 2024-04-16T18:38:50.107 INFO:tasks.ceph.c2.osd.1:Sent signal 15 2024-04-16T18:38:50.107 INFO:tasks.ceph.c2.osd.2:Sent signal 15 2024-04-16T18:38:50.107 INFO:tasks.ceph.c2.osd.3:Sent signal 15 2024-04-16T18:38:50.107 INFO:tasks.ceph.c2.osd.4:Sent signal 15 2024-04-16T18:38:50.107 INFO:tasks.ceph.c2.osd.5:Sent signal 15 2024-04-16T18:38:50.108 INFO:tasks.ceph.c2.mon.a:Sent signal 15 2024-04-16T18:38:50.108 INFO:tasks.rgw.c2.client.1:Sent signal 15 2024-04-16T18:38:50.108 INFO:tasks.ceph.c2.mgr.x:Sent signal 15 2024-04-16T18:38:50.109 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T18:38:50.111+0000 7fc2f06bc640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 3 (PID: 22806) UID: 0 2024-04-16T18:38:50.109 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T18:38:50.111+0000 7fc2f06bc640 -1 osd.3 55 *** Got signal Terminated *** 2024-04-16T18:38:50.109 INFO:tasks.ceph.c2.osd.3.smithi188.stderr:2024-04-16T18:38:50.111+0000 7fc2f06bc640 -1 osd.3 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T18:38:50.110 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f7274e81640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 4 (PID: 22808) UID: 0 2024-04-16T18:38:50.110 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f7274e81640 -1 osd.4 55 *** Got signal Terminated *** 2024-04-16T18:38:50.110 INFO:tasks.ceph.c2.osd.4.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f7274e81640 -1 osd.4 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T18:38:50.110 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f7b1f6d9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 5 (PID: 22810) UID: 0 2024-04-16T18:38:50.110 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f7b1f6d9640 -1 osd.5 55 *** Got signal Terminated *** 2024-04-16T18:38:50.110 INFO:tasks.ceph.c2.osd.5.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f7b1f6d9640 -1 osd.5 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T18:38:50.111 INFO:tasks.rgw.c2.client.1.smithi188.stdout:2024-04-16T18:38:50.111+0000 7f45dda82640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground (PID: 27407) UID: 0 2024-04-16T18:38:50.111 INFO:tasks.rgw.c2.client.1.smithi188.stdout:2024-04-16T18:38:50.111+0000 7f45e12caa40 -1 shutting down 2024-04-16T18:38:50.111 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f8a58d0d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mgr -f --cluster c2 -i x (PID: 22626) UID: 0 2024-04-16T18:38:50.111 INFO:tasks.ceph.c2.mgr.x.smithi188.stderr:2024-04-16T18:38:50.111+0000 7f8a58d0d640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-16T18:38:50.111 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f0fe888e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 0 (PID: 22818) UID: 0 2024-04-16T18:38:50.112 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f0fe888e640 -1 osd.0 55 *** Got signal Terminated *** 2024-04-16T18:38:50.112 INFO:tasks.ceph.c2.osd.0.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f0fe888e640 -1 osd.0 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T18:38:50.112 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T18:38:50.107+0000 7ff34e1ce640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 1 (PID: 22820) UID: 0 2024-04-16T18:38:50.112 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T18:38:50.107+0000 7ff34e1ce640 -1 osd.1 55 *** Got signal Terminated *** 2024-04-16T18:38:50.112 INFO:tasks.ceph.c2.osd.1.smithi122.stderr:2024-04-16T18:38:50.107+0000 7ff34e1ce640 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T18:38:50.113 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f21bd318640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 2 (PID: 22823) UID: 0 2024-04-16T18:38:50.113 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f21bd318640 -1 osd.2 55 *** Got signal Terminated *** 2024-04-16T18:38:50.113 INFO:tasks.ceph.c2.osd.2.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f21bd318640 -1 osd.2 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T18:38:50.113 INFO:tasks.ceph.c2.mon.a.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f1c8184e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster c2 -i a (PID: 22745) UID: 0 2024-04-16T18:38:50.113 INFO:tasks.ceph.c2.mon.a.smithi122.stderr:2024-04-16T18:38:50.107+0000 7f1c8184e640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T18:38:51.440 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:51.440 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:38:51.440 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:38:51.484 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:38:51.484 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:43:51.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:43:51.485+0000 7f6ad2a9e980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T18:43:51.487 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T18:43:51.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T18:43:51.517 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_resharded_bucket_full_sync_after_data_sync_init ... FAIL 2024-04-16T18:43:51.519 DEBUG:boto:path=/lybtug-55/ 2024-04-16T18:43:51.519 DEBUG:boto:auth_path=/lybtug-55/ 2024-04-16T18:43:51.519 DEBUG:boto:Method: PUT 2024-04-16T18:43:51.520 DEBUG:boto:Path: /lybtug-55/ 2024-04-16T18:43:51.520 DEBUG:boto:Data: 2024-04-16T18:43:51.520 DEBUG:boto:Headers: {} 2024-04-16T18:43:51.520 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:43:51.520 DEBUG:boto:Port: 8000 2024-04-16T18:43:51.520 DEBUG:boto:Params: {} 2024-04-16T18:43:51.521 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:43:51.521 DEBUG:boto:Token: None 2024-04-16T18:43:51.521 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:43:51 GMT /lybtug-55/ 2024-04-16T18:43:51.522 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:b0oKjtTpPECT1KW6ilGsd7mWoWs= 2024-04-16T18:43:51.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:43:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b0oKjtTpPECT1KW6ilGsd7mWoWs=', 'Content-Length': '0'} 2024-04-16T18:43:51.571 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000053beebcd67dbae66-00661ec6e7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:43:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:43:51.571 DEBUG:rgw_multi.tests:created bucket=lybtug-55 2024-04-16T18:43:51.571 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:43:51.572 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:43:51.572 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:43:51.613 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:43:51.613 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:43:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:43:51.716 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293031.566554_2750.1","last_update":"2024-04-16T18:43:51.566554Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:43:51.716 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293031.566554_2750.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:43:51.716 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:43:51.717 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:43:51.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:43:51.717 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:43:51.760 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:43:51.760 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:48:51.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:48:51.762+0000 7f8323156980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T18:48:51.766 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T18:48:51.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T18:48:51.793 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_incremental_sync_after_data_sync_init ... FAIL 2024-04-16T18:48:51.795 DEBUG:boto:path=/lybtug-56/ 2024-04-16T18:48:51.795 DEBUG:boto:auth_path=/lybtug-56/ 2024-04-16T18:48:51.795 DEBUG:boto:Method: PUT 2024-04-16T18:48:51.795 DEBUG:boto:Path: /lybtug-56/ 2024-04-16T18:48:51.795 DEBUG:boto:Data: 2024-04-16T18:48:51.795 DEBUG:boto:Headers: {} 2024-04-16T18:48:51.796 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:48:51.796 DEBUG:boto:Port: 8000 2024-04-16T18:48:51.796 DEBUG:boto:Params: {} 2024-04-16T18:48:51.796 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:48:51.797 DEBUG:boto:Token: None 2024-04-16T18:48:51.797 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:48:51 GMT /lybtug-56/ 2024-04-16T18:48:51.797 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:yQtFeD55GNMtqawLOjNCuk7wsbk= 2024-04-16T18:48:51.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:48:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yQtFeD55GNMtqawLOjNCuk7wsbk=', 'Content-Length': '0'} 2024-04-16T18:48:51.844 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000026b70e215d25a38b-00661ec813-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:48:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:48:51.844 DEBUG:rgw_multi.tests:created bucket=lybtug-56 2024-04-16T18:48:51.844 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:48:51.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:48:51.845 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:48:51.886 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:48:51.887 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:48:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:48:51.990 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293331.839346_3712.1","last_update":"2024-04-16T18:48:51.839346Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:48:51.990 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293331.839346_3712.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:48:51.991 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:48:51.991 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:48:51.991 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:48:51.991 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:48:52.036 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:48:52.036 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:53:52.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:53:52.038+0000 7f77efe64980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T18:53:52.042 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T18:53:52.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T18:53:52.045 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_resharded_bucket_incremental_sync_latest_after_data_sync_init ... FAIL 2024-04-16T18:53:52.046 DEBUG:boto:path=/lybtug-57/ 2024-04-16T18:53:52.046 DEBUG:boto:auth_path=/lybtug-57/ 2024-04-16T18:53:52.046 DEBUG:boto:Method: PUT 2024-04-16T18:53:52.047 DEBUG:boto:Path: /lybtug-57/ 2024-04-16T18:53:52.047 DEBUG:boto:Data: 2024-04-16T18:53:52.047 DEBUG:boto:Headers: {} 2024-04-16T18:53:52.047 DEBUG:boto:Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T18:53:52.047 DEBUG:boto:Port: 8000 2024-04-16T18:53:52.047 DEBUG:boto:Params: {} 2024-04-16T18:53:52.047 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T18:53:52.047 DEBUG:boto:Token: None 2024-04-16T18:53:52.048 DEBUG:boto:StringToSign: PUT Tue, 16 Apr 2024 18:53:52 GMT /lybtug-57/ 2024-04-16T18:53:52.048 DEBUG:boto:Signature: AWS 076QSHZWMHBH668NR2NY:J22F+Ny5fHdciI1m+mDg3WcpRS8= 2024-04-16T18:53:52.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:53:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J22F+Ny5fHdciI1m+mDg3WcpRS8=', 'Content-Length': '0'} 2024-04-16T18:53:52.090 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000bef2c458eca91dc7-00661ec940-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:53:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T18:53:52.090 DEBUG:rgw_multi.tests:created bucket=lybtug-57 2024-04-16T18:53:52.090 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:53:52.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:53:52.091 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:53:52.133 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:53:52.133 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:53:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:53:52.237 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:53:52.237 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:53:52.237 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:53:52.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:53:52.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:53:52.237 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:53:52.281 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:53:52.281 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:58:52.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T18:58:52.283+0000 7fddcf221980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T18:58:52.286 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T18:58:52.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T18:58:52.298 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_resharded_bucket_incremental_sync_oldest_after_data_sync_init ... FAIL 2024-04-16T18:58:52.299 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:58:52.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:58:52.300 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:58:52.342 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:58:52.342 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T18:58:52.412 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T18:58:52.446 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T18:58:52.446 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T18:58:52.447 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T18:58:52.447 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:58:52.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T18:58:52.447 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T18:58:52.490 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T18:58:52.490 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:03:52.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:03:52.492+0000 7feb0b37d980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:03:52.522 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:03:52.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:03:52.525 INFO:tasks.rgw_multisite_tests:test_sync_policy_config_zonegroup: ... FAIL 2024-04-16T19:03:52.526 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:03:52.527 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:03:52.527 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:03:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:03:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:03:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:03:52.674 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:03:52.675 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:03:52.675 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T19:03:52.675 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:03:52.675 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:03:52.675 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:03:52.717 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:03:52.717 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:08:52.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:08:52.718+0000 7fc5aed8d980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:08:52.753 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:08:52.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:08:52.765 INFO:tasks.rgw_multisite_tests:test_sync_flow_symmetrical_zonegroup_all: ... FAIL 2024-04-16T19:08:52.767 INFO:tasks.rgw_multisite_tests:test_sync_flow_symmetrical_zonegroup_select: ... SKIP: test_sync_flow_symmetrical_zonegroup_select skipped. Requires 3 or more zones in master zonegroup. 2024-04-16T19:08:52.767 INFO:tasks.rgw_multisite_tests:test_sync_flow_directional_zonegroup_select: ... SKIP: test_sync_flow_symmetrical_zonegroup_select skipped. Requires 3 or more zones in master zonegroup. 2024-04-16T19:08:52.768 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:08:52.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:08:52.768 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:08:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:08:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:08:52.879 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:08:52.913 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:08:52.928 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:08:52.928 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T19:08:52.929 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:08:52.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:08:52.929 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:08:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:08:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:13:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:13:52.975+0000 7f8a96cd4980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:13:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:13:53.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:13:53.094 INFO:tasks.rgw_multisite_tests:test_sync_single_bucket: ... FAIL 2024-04-16T19:13:53.096 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:13:53.097 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:13:53.097 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:13:53.141 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:13:53.142 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:13:53.213 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:13:53.246 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:13:53.246 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:13:53.247 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T19:13:53.247 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:13:53.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:13:53.247 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:13:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:13:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:18:53.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:18:53.295+0000 7efeb3b59980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:18:53.319 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:18:53.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:18:53.368 INFO:tasks.rgw_multisite_tests:test_sync_different_buckets: ... FAIL 2024-04-16T19:18:53.370 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:18:53.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:18:53.371 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:18:53.414 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:18:53.414 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:18:53.486 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:18:53.520 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:18:53.520 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:18:53.520 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T19:18:53.520 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:18:53.521 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:18:53.521 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:18:53.564 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:18:53.564 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:23:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:23:53.564+0000 7f800624b980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:23:53.568 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:23:53.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:23:53.571 INFO:tasks.rgw_multisite_tests:test_sync_multiple_buckets_to_single: ... FAIL 2024-04-16T19:23:53.573 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:23:53.573 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:23:53.573 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:23:53.615 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:23:53.616 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:23:53.687 INFO:teuthology.orchestra.run.smithi071.stderr:No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:23:53.736 INFO:teuthology.orchestra.run.smithi071.stdout:[{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:23:53.737 DEBUG:rgw_multi.tests:master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:23:53.737 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2024-04-16T19:23:53.737 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:23:53.737 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:23:53.737 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:23:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:23:53.781 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:28:53.782+0000 7f9bb1899980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:53.805 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:53.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:28:53.807 INFO:tasks.rgw_multisite_tests:test_sync_single_bucket_to_multiple: ... FAIL 2024-04-16T19:28:53.809 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_create_rgw_down ... SKIP: test_bucket_create_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.809 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_remove_rgw_down ... SKIP: test_bucket_remove_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.810 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_sync_rgw_down ... SKIP: test_object_sync_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.810 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_delete_rgw_down ... SKIP: test_object_delete_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.810 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_concurrent_versioned_object_incremental_sync_rgw_down ... SKIP: test_concurrent_versioned_object_incremental_sync_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.811 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_suspended_delete_marker_full_sync_rgw_down ... SKIP: test_suspended_delete_marker_full_sync_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.811 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_acl_rgw_down ... SKIP: test_bucket_acl_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.811 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_enable_right_after_disable_rgw_down ... SKIP: test_bucket_sync_enable_right_after_disable_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.812 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multipart_object_sync_rgw_down ... SKIP: test_multipart_object_sync_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.812 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_run_basic_incremental_rgw_down ... SKIP: test_bucket_sync_run_basic_incremental_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.813 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_role_sync_rgw_down ... SKIP: test_role_sync_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.813 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_full_sync_after_data_sync_init_rgw_down ... SKIP: test_bucket_full_sync_after_data_sync_init_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.813 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_sync_policy_config_zonegroup_rgw_down ... SKIP: test_sync_policy_config_zonegroup_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.814 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_sync_flow_symmetrical_zonegroup_all_rgw_down ... SKIP: test_sync_flow_symmetrical_zonegroup_all_rgw_down skipped. More than one rgw needed in any one or multiple zone(s). 2024-04-16T19:28:53.814 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.814 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:53.814 INFO:tasks.rgw_multisite_tests:ERROR: rgw_multi.tests.test_topic_notification_sync 2024-04-16T19:28:53.814 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 1788, in test_topic_notification_sync 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: zone_conn.create_notification(bucket.name, [topic_conf]) 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw_multi/zone_rados.py", line 160, in create_notification 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: return self.s3_client.put_bucket_notification_configuration( 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/client.py", line 553, in _api_call 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: return self._make_api_call(operation_name, kwargs) 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/botocore/client.py", line 1009, in _make_api_call 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests: raise error_class(parsed_response, operation_name) 2024-04-16T19:28:53.815 INFO:tasks.rgw_multisite_tests:botocore.exceptions.ClientError: An error occurred (ConcurrentModification) when calling the PutBucketNotificationConfiguration operation: None 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.816 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:36.755+0000 7f986d511980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:36.759+0000 7f986d511980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:36Z"} 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713289749.894324_100.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: finish meta checkpoint for zone=test-zone2 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: create topic zone=test-zone1 name=topic-lybtug-1 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.sns.CreateTopic: calling handler 2024-04-16T19:28:53.817 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'topic-lybtug-1', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'http://kaboom:9999', 'Attributes.entry.2.key': 'persistent', 'Attributes.entry.2.value': 'true'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler > 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event choose-signer.sns.CreateTopic: calling handler 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Calculating signature using v4 auth. 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: CanonicalRequest: 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:POST 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:/ 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:content-type:application/x-www-form-urlencoded; charset=utf-8 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:host:smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.818 INFO:tasks.rgw_multisite_tests:x-amz-date:20240416T181541Z 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:content-type;host;x-amz-date 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:6c1d177bfdd7e7dab29cb103da0c6e232c1871d8a55783668dd7f45c9a0b19fe 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: StringToSign: 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:AWS4-HMAC-SHA256 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:20240416T181541Z 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:20240416/test-zonegroup/sns/aws4_request 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:55362b45b03625ff9d7e750b901af6938205a68622f0f50ab5f5ef31a78c7c1f 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Signature: 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:c3d20d0790720573331d33d6ecdb5aea941c86d3dcf2f640835da70e864f7bf4 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Sending http request: 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000dd0f74954f23a6f4-00661ec04d-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '337', 'Date': 'Tue, 16 Apr 2024 18:15:41 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T19:28:53.819 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response body: 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:b'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1054102a3-8d94-4596-8b3b-6341eab312e1.4804.15929078591659419380' 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.sns.CreateTopic: calling handler 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.retryhandler: DEBUG: No retry needed. 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: create topic zone=test-zone2 name=topic-lybtug-2 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.sns.CreateTopic: calling handler 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.CreateTopic: calling handler 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'topic-lybtug-2', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'http://kaboom:9999', 'Attributes.entry.2.key': 'persistent', 'Attributes.entry.2.value': 'true'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler > 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event choose-signer.sns.CreateTopic: calling handler 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Calculating signature using v4 auth. 2024-04-16T19:28:53.820 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: CanonicalRequest: 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:POST 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:/ 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:content-type:application/x-www-form-urlencoded; charset=utf-8 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:host:smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:x-amz-date:20240416T181541Z 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:content-type;host;x-amz-date 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:6774c3c7b156ceb70c08415911657cb1e511c662663d2f736ba5e37479485e4a 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: StringToSign: 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:AWS4-HMAC-SHA256 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:20240416T181541Z 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:20240416/test-zonegroup/sns/aws4_request 2024-04-16T19:28:53.821 INFO:tasks.rgw_multisite_tests:7e4f2589375c36f2378655630022d45f32b1a3bc382b5aee3e82c0b3e591acac 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Signature: 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:ed878a170108552237d6bde4dace52b6a9bc7bc31c1b143dd50a35cb8f382050 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.CreateTopic: calling handler 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Sending http request: 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000eeaa0cec01c84729-00661ec04d-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '337', 'Date': 'Tue, 16 Apr 2024 18:15:41 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response body: 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:b'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2bd830684-b604-4841-ab52-8c82ee812281.5397.17197572334849378089' 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.sns.CreateTopic: calling handler 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:botocore.retryhandler: DEBUG: No retry needed. 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: topic_arns: ['arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1', 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2'] 2024-04-16T19:28:53.822 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713291176.246608_485.1","last_update":"2024-04-16T18:12:56.246608Z"},{"marker":"1_1713291341.856004_360.1","last_update":"2024-04-16T18:15:41.856004Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713291176.246608_485.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.823 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:42.135+0000 7f5b3a4e7980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:42.135+0000 7f5b3a4e7980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291176.246608_485.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:12:56.246608Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291176.246608_485.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:12:56.246608Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291341.856004_360.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:41.856004Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713291176.246608_485.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713291176.246608_485.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: finish meta checkpoint for zone=test-zone2 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.sns.ListTopics: calling handler 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000'} 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.ListTopics: calling handler 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.ListTopics: calling handler 2024-04-16T19:28:53.824 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Making request for OperationModel(name=ListTopics) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListTopics', 'Version': '2010-03-31'}, 'url': 'http://smithi071.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.ListTopics: calling handler > 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event choose-signer.sns.ListTopics: calling handler 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Calculating signature using v4 auth. 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: CanonicalRequest: 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:POST 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:/ 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:content-type:application/x-www-form-urlencoded; charset=utf-8 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:host:smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:x-amz-date:20240416T181542Z 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:content-type;host;x-amz-date 2024-04-16T19:28:53.825 INFO:tasks.rgw_multisite_tests:55e772bd3d832d8507965af84ab07fd17d9be3cbb788151ceb0d7fc2be442a54 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: StringToSign: 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:AWS4-HMAC-SHA256 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:20240416T181542Z 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:20240416/test-zonegroup/sns/aws4_request 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:08169e34f610bd48de5ab8908a2e55f23f608b87dc9166dc2494236a8fbb4909 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Signature: 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:f4c58d1fa15b8be2a932b3685ab7f4cbdf8f71ba5aea0e99cf92b35d615bc2ce 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.ListTopics: calling handler 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Sending http request: 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000d6fd106ba491703b-00661ec04e-4804-test-zone1', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1415', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response body: 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:b'RGW11111111111111111topic-lybtug-1http://kaboom:9999Version=2010-03-31topic-lybtug-1FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1RGW11111111111111111topic-lybtug-2http://kaboom:9999Version=2010-03-31&rgwx-uid=RGW11111111111111111&rgwx-zonegroup=test-zonegrouptopic-lybtug-2FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2054102a3-8d94-4596-8b3b-6341eab312e1.4804.15491556347732914235' 2024-04-16T19:28:53.826 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.sns.ListTopics: calling handler 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.retryhandler: DEBUG: No retry needed. 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: topics for zone=test-zone1 = [{'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1'}, {'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2'}] 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.sns.ListTopics: calling handler 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Calling endpoint provider with parameters: {'Region': 'test-zonegroup', 'UseDualStack': False, 'UseFIPS': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000'} 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.ListTopics: calling handler 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.sns.ListTopics: calling handler 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Making request for OperationModel(name=ListTopics) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': {'Action': 'ListTopics', 'Version': '2010-03-31'}, 'url': 'http://smithi122.front.sepia.ceph.com:8000/', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': None}} 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.ListTopics: calling handler > 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event choose-signer.sns.ListTopics: calling handler 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Calculating signature using v4 auth. 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: CanonicalRequest: 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:POST 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests:/ 2024-04-16T19:28:53.827 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:content-type:application/x-www-form-urlencoded; charset=utf-8 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:host:smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:x-amz-date:20240416T181542Z 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:content-type;host;x-amz-date 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:55e772bd3d832d8507965af84ab07fd17d9be3cbb788151ceb0d7fc2be442a54 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: StringToSign: 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:AWS4-HMAC-SHA256 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:20240416T181542Z 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:20240416/test-zonegroup/sns/aws4_request 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:d88226fa1ea80c48fb7710f4ea147582ec6b3a4ef1b34e2c496b9a7f3d692c61 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Signature: 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:4f274ce1b9862c6bdd52f5b5c3dc6d78567c0a9c7870c0ee80bf91a1ea0d9113 2024-04-16T19:28:53.828 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.sns.ListTopics: calling handler 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Sending http request: 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx000008006102a1c087900-00661ec04e-5397-test-zone2', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '1414', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response body: 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:b'RGW11111111111111111topic-lybtug-1http://kaboom:9999Version=2010-03-31topic-lybtug-1FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1RGW11111111111111111topic-lybtug-2http://kaboom:9999Version=2010-03-31&rgwx-uid=RGW11111111111111111&rgwx-zonegroup=test-zonegrouptopic-lybtug-2FalseTrueNoneNoneNonearn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2bd830684-b604-4841-ab52-8c82ee812281.5397.9225078659760027904' 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.sns.ListTopics: calling handler 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:botocore.retryhandler: DEBUG: No retry needed. 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: topics for zone=test-zone2 = [{'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1'}, {'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2'}] 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-49/ 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-49/ 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-49/ 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.829 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:42 GMT 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:/lybtug-49/ 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Ogbg+rIpjzNXT51rB2PzI3GYhsY= 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ogbg+rIpjzNXT51rB2PzI3GYhsY=', 'Content-Length': '0'} 2024-04-16T19:28:53.830 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx0000002c3f7561436c558-00661ec04e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:42 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: created bucket=lybtug-49 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713291342.220251_500.1","last_update":"2024-04-16T18:15:42.220251Z"},{"marker":"1_1713291341.856004_360.1","last_update":"2024-04-16T18:15:41.856004Z"},{"marker":"1_1713291336.175369_271.1","last_update":"2024-04-16T18:15:36.175369Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713291342.220251_500.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:53.831 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:42.483+0000 7f8351580980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:42.487+0000 7f8351580980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291342.220251_500.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:42.220251Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291342.220251_500.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:42.220251Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291341.856004_360.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:41.856004Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291336.175369_271.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:36.175369Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:42Z"} 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291342.220251_500.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:42.220251Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291341.856004_360.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:41.856004Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291336.175369_271.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:36.175369Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713291342.220251_500.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713291342.220251_500.1', 1: '1_1713291341.856004_360.1', 2: '1_1713291336.175369_271.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: finish meta checkpoint for zone=test-zone2 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: creating bucket notification for zone=test-zone1 name=bn-lybtug-1 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.832 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-endpoint-resolution.s3: calling handler 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-endpoint-resolution.s3: calling handler > 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Calling endpoint provider with parameters: {'Bucket': 'lybtug-49', 'Region': 'test-zonegroup', 'UseFIPS': False, 'UseDualStack': False, 'Endpoint': 'http://smithi071.front.sepia.ceph.com:8000', 'ForcePathStyle': True, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True, 'UseS3ExpressControlEndpoint': True} 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Endpoint provider result: http://smithi071.front.sepia.ceph.com:8000/lybtug-49 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None" 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Selected auth type "v4" as "v4" with signing context params: {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True} 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Making request for OperationModel(name=PutBucketNotificationConfiguration) with params: {'url_path': '?notification', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': b'bn-lybtug-1arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1s3:ObjectCreated:*', 'auth_path': '/lybtug-49/', 'url': 'http://smithi071.front.sepia.ceph.com:8000/lybtug-49?notification', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': 'v4', 's3_redirect': {'redirected': False, 'bucket': 'lybtug-49', 'params': {'Bucket': 'lybtug-49', 'NotificationConfiguration': {'TopicConfigurations': [{'Id': 'bn-lybtug-1', 'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-1', 'Events': ['s3:ObjectCreated:*']}]}}}, 'S3Express': {'bucket_name': 'lybtug-49'}, 'signing': {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'test-zonegroup'}]}}} 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.833 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-sign.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-sign.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Calculating signature using v4 auth. 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: CanonicalRequest: 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:/lybtug-49 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:notification= 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:host:smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:x-amz-content-sha256:5e41bdd252e04b646cefb30bc3c736f83ae11b3ef056e4074db0a6801d176148 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:x-amz-date:20240416T181542Z 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:host;x-amz-content-sha256;x-amz-date 2024-04-16T19:28:53.834 INFO:tasks.rgw_multisite_tests:5e41bdd252e04b646cefb30bc3c736f83ae11b3ef056e4074db0a6801d176148 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: StringToSign: 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:AWS4-HMAC-SHA256 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:20240416T181542Z 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:20240416/test-zonegroup/s3/aws4_request 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:316cba3cf9ca68e2e6debbd9a10452564dde96e82032104c6cad78f91e089ff6 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Signature: 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:5211ece45ad4ec8557eeab0330a0275e93cab0872f22b6eac67639e61ab7d442 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Sending http request: 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response headers: {'x-amz-request-id': 'tx00000b7c26786192e4869-00661ec04e-4804-test-zone1', 'Server': 'Ceph Object Gateway (squid)', 'Content-Length': '0', 'Date': 'Tue, 16 Apr 2024 18:15:42 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response body: 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:b'' 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.835 INFO:tasks.rgw_multisite_tests:botocore.retryhandler: DEBUG: No retry needed. 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: creating bucket notification for zone=test-zone2 name=bn-lybtug-2 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-parameter-build.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-endpoint-resolution.s3: calling handler 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-endpoint-resolution.s3: calling handler > 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Calling endpoint provider with parameters: {'Bucket': 'lybtug-49', 'Region': 'test-zonegroup', 'UseFIPS': False, 'UseDualStack': False, 'Endpoint': 'http://smithi122.front.sepia.ceph.com:8000', 'ForcePathStyle': True, 'Accelerate': False, 'UseGlobalEndpoint': False, 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True, 'UseS3ExpressControlEndpoint': True} 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Endpoint provider result: http://smithi122.front.sepia.ceph.com:8000/lybtug-49 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Selecting from endpoint provider's list of auth schemes: "sigv4". User selected auth scheme is: "None" 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.regions: DEBUG: Selected auth type "v4" as "v4" with signing context params: {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True} 2024-04-16T19:28:53.836 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-call.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Making request for OperationModel(name=PutBucketNotificationConfiguration) with params: {'url_path': '?notification', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.34.8 md/Botocore#1.34.8 ua/2.0 os/linux#5.4.0-166-generic md/arch#x86_64 lang/python#3.8.10 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.8'}, 'body': b'bn-lybtug-2arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2s3:ObjectCreated:*', 'auth_path': '/lybtug-49/', 'url': 'http://smithi122.front.sepia.ceph.com:8000/lybtug-49?notification', 'context': {'client_region': 'test-zonegroup', 'client_config': , 'has_streaming_input': False, 'auth_type': 'v4', 's3_redirect': {'redirected': False, 'bucket': 'lybtug-49', 'params': {'Bucket': 'lybtug-49', 'NotificationConfiguration': {'TopicConfigurations': [{'Id': 'bn-lybtug-2', 'TopicArn': 'arn:aws:sns:test-zonegroup:RGW11111111111111111:topic-lybtug-2', 'Events': ['s3:ObjectCreated:*']}]}}}, 'S3Express': {'bucket_name': 'lybtug-49'}, 'signing': {'region': 'test-zonegroup', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'test-zonegroup'}]}}} 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event choose-signer.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-sign.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event before-sign.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Calculating signature using v4 auth. 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: CanonicalRequest: 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:/lybtug-49 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:notification= 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:host:smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.837 INFO:tasks.rgw_multisite_tests:x-amz-content-sha256:c629530fae5787b7ee540d6c6306e59500adcac685bdbaa3124682d4eb30f587 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:x-amz-date:20240416T181542Z 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:host;x-amz-content-sha256;x-amz-date 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:c629530fae5787b7ee540d6c6306e59500adcac685bdbaa3124682d4eb30f587 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: StringToSign: 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:AWS4-HMAC-SHA256 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:20240416T181542Z 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:20240416/test-zonegroup/s3/aws4_request 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:01ed2c74f221e494d438ab0286aa9e81f2634244eb9f18cecd30358d1902b019 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:botocore.auth: DEBUG: Signature: 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:2d4e32af45ef0d0395fcc373d1b315e90b02a84776aee15d84b887165d1d8f0a 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event request-created.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.838 INFO:tasks.rgw_multisite_tests:botocore.endpoint: DEBUG: Sending http request: 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response headers: {'Content-Length': '261', 'x-amz-request-id': 'tx00000f31bf77b15bd460e-00661ec04e-5397-test-zone2', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Server': 'Ceph Object Gateway (squid)', 'Date': 'Tue, 16 Apr 2024 18:15:43 GMT', 'Connection': 'Keep-Alive'} 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:botocore.parsers: DEBUG: Response body: 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:b'ConcurrentModificationlybtug-49tx00000f31bf77b15bd460e-00661ec04e-5397-test-zone25397-test-zone2-test-zonegroup' 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:botocore.retryhandler: DEBUG: No retry needed. 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:botocore.hooks: DEBUG: Event needs-retry.s3.PutBucketNotificationConfiguration: calling handler > 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_multi_object_delete 2024-04-16T19:28:53.839 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:53.840 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 880, in test_multi_object_delete 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: zone_bucket_checkpoint(target_conn.zone, source_conn.zone, bucket.name) 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 390, in zone_bucket_checkpoint 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: target_zone.cluster.admin(cmd, debug_rgw=1) 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw_multisite.py", line 197, in admin 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests: assert r == 0 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: create bucket zone=test-zone1 name=lybtug-11 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.841 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:36 GMT 2024-04-16T19:28:53.842 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:1G7WmcqS9DJH0kZBehjdo1BMkd0= 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1G7WmcqS9DJH0kZBehjdo1BMkd0=', 'Content-Length': '0'} 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx00000299693dd133182d6-00661eba30-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: create bucket zone=test-zone2 name=lybtug-12 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.843 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:36 GMT 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bQD9LZ+WD+9ItTqXemlug8QVF/U= 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bQD9LZ+WD+9ItTqXemlug8QVF/U=', 'Content-Length': '0'} 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx00000c8d4ceaa73485b46-00661eba30-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:36 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.844 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:36 GMT 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+4nI2cfescxoCQNgVybKJmkjjMA= 2024-04-16T19:28:53.845 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:36 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+4nI2cfescxoCQNgVybKJmkjjMA=', 'Content-Length': '0'} 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d56e9bb04ba7bdef-00661eba30-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj1 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj1 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj1 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.846 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj1 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:INvo8xgnJQpOp3mxVp9On1G6eTg= 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:INvo8xgnJQpOp3mxVp9On1G6eTg='} 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003fb9fb8f177deaa5-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.847 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b2be6a979b991ad3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj2 2024-04-16T19:28:53.848 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj2 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj2 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.849 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj2 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:f3kCL/ITxfPGSoqD9X0pTss/foE= 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:f3kCL/ITxfPGSoqD9X0pTss/foE='} 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bcbfeda5c61972c8-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.850 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cd8d97435f58a758-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj3 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj3 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj3 2024-04-16T19:28:53.851 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj3 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:8QnnPiS5DI7DvFe7Q0G5NRLgMko= 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8QnnPiS5DI7DvFe7Q0G5NRLgMko='} 2024-04-16T19:28:53.852 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006728f057c4d3f1d9-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.853 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a6acd8f7a1001632-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj4 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj4 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj4 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.854 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj4 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9sbutZVG3Ts+UM6lrDr3e7+dq1s= 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9sbutZVG3Ts+UM6lrDr3e7+dq1s='} 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005e140dfd74f62adf-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.855 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.856 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ac0da4903729ff4b-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj5 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj5 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj5 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.857 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj5 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:xF2QLJRFucQVqvjs5AISxDTvvm0= 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xF2QLJRFucQVqvjs5AISxDTvvm0='} 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c81ad494128c8c86-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.858 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002896d6ee6b96d95f-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj6 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj6 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj6 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.859 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj6 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:G0IYHPmx1nuoM8UuqPOkvdyiMSg= 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G0IYHPmx1nuoM8UuqPOkvdyiMSg='} 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fae36a1fe50ee0bb-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.860 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.861 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000109c75f15d94feff-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj7 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj7 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj7 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.862 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj7 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bdqon9w5AFwbIsEGmU/Th7mkZF4= 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bdqon9w5AFwbIsEGmU/Th7mkZF4='} 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005fdaae6a9c7894d9-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.863 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.864 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000652dfaffc6bcfc61-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj8 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj8 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj8 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.865 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj8 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:tM1zL++4AQJbhxhsTZPJoV79Gwc= 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:tM1zL++4AQJbhxhsTZPJoV79Gwc='} 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000de4df081249d3e73-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.866 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000199e6af2f8859f2b-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj9 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj9 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.867 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj9 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj9 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.868 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:dThEF7O+q7D6fpC5E+EMkaB584s= 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dThEF7O+q7D6fpC5E+EMkaB584s='} 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a603cb0aee0209a8-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.869 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a8dafb5bddef76fc-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj10 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj10 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj10 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.870 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj10 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:AvePScqxATQPCvOtmnlK4HCJlcY= 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AvePScqxATQPCvOtmnlK4HCJlcY='} 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000492476cecd5754cf-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.871 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.872 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009f87e60327feb6b1-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj11 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj11 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj11 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.873 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj11 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Ch/0xxJfLbL+SZnkcvBeTGHGXaA= 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ch/0xxJfLbL+SZnkcvBeTGHGXaA='} 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000066d3cad71a0658dd-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.874 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009dfe4edc4de00ed7-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj12 2024-04-16T19:28:53.875 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj12 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj12 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj12 2024-04-16T19:28:53.876 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:DfRtWdpKh+sZCDPItgpRJFqv0dg= 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DfRtWdpKh+sZCDPItgpRJFqv0dg='} 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d98a68fbb5bebc79-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.877 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000000da18c3d9b77588-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj13 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj13 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj13 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.878 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj13 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Sa8PeFn7yU2nv5HtbT3vopvQb9Y= 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Sa8PeFn7yU2nv5HtbT3vopvQb9Y='} 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000086d684f2400c99cd-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.879 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.880 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f82b4bd3e45537f9-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj14 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj14 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj14 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.881 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj14 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:evB/39NHU64CyzNb8E14FONRWRE= 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:evB/39NHU64CyzNb8E14FONRWRE='} 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc5d7a0b887c9c59-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.882 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a32f60f56a521a64-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj15 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj15 2024-04-16T19:28:53.883 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj15 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj15 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:I5wslan6/JElJqWvxeKYTsf+4TA= 2024-04-16T19:28:53.884 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I5wslan6/JElJqWvxeKYTsf+4TA='} 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000064029d9cbf6a5390-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.885 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002f957a443ec6d7c2-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj16 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj16 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj16 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.886 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj16 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:GoETj96Lrbb2f4gv6Fcc32V1RZo= 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:GoETj96Lrbb2f4gv6Fcc32V1RZo='} 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005548bef2e0bee951-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.887 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cec63116b06c1896-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.888 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj17 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj17 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj17 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj17 2024-04-16T19:28:53.889 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:me1hVUYxMp88mzZKMxHeZQmX5Pw= 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:me1hVUYxMp88mzZKMxHeZQmX5Pw='} 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000092301b145fc5b2eb-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.890 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004c07fcc310a82df3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj18 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj18 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.891 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj18 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj18 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Wr5FNon9mGeiiB2orsUQBdENxHA= 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Wr5FNon9mGeiiB2orsUQBdENxHA='} 2024-04-16T19:28:53.892 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c4a1af5c937c606b-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.893 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '108'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000543970cd3d5520e4-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj19 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj19 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj19 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.894 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj19 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+SORG6Zu5yx9fvPAODwtcaUAEwo= 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+SORG6Zu5yx9fvPAODwtcaUAEwo='} 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b7db5e13668ff5ea-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.895 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.896 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000078a470ca0541bc61-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj20 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj20 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj20 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.897 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj20 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ZkE1gTVR7UarDXwAPV2f3iWvKBA= 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZkE1gTVR7UarDXwAPV2f3iWvKBA='} 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003dcaeac8b2cd7bc8-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.898 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000e485fd929f81006-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj21 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj21 2024-04-16T19:28:53.899 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj21 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj21 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.900 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:3XcYtcsMEKG2Ndt6e43JhB4D3T4= 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:3XcYtcsMEKG2Ndt6e43JhB4D3T4='} 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005b04a8003152359a-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.901 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000d6824abaadeab79-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj22 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj22 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj22 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.902 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj22 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+55DjsEPpu0v0TFtmQNeFIX9Yj8= 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+55DjsEPpu0v0TFtmQNeFIX9Yj8='} 2024-04-16T19:28:53.903 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000208ca2b65feacf4a-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.904 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '132'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000051b78a348ac82b17-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj23 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj23 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj23 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.905 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj23 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:kcJUutwtrqGuEC/hZo087+ce2W0= 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kcJUutwtrqGuEC/hZo087+ce2W0='} 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006cee05114966e1e3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.906 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.907 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '138'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000af3f7f9026a45092-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj24 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj24 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj24 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.908 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj24 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:0Io2owG7509qiM8EjpC8PQCUlww= 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0Io2owG7509qiM8EjpC8PQCUlww='} 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000062c36e860e0e3248-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.909 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.910 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '144'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000059c59a501116edcd-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj25 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj25 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj25 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.911 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj25 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:apc+/MysuMt/Sm9EZQpYdPJHr3A= 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:apc+/MysuMt/Sm9EZQpYdPJHr3A='} 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009354657e465774bb-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.912 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c4494081b93243c2-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj26 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj26 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.913 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj26 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj26 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:I5doxI9wVVye8jw+F53XRY+cynQ= 2024-04-16T19:28:53.914 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:I5doxI9wVVye8jw+F53XRY+cynQ='} 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000ae13908654a22c4-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.915 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '156'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000325cf4699ff159f3-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj27 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj27 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj27 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.916 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj27 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SnSuX9KiFXyGSsc8uYlsQgmfUHg= 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SnSuX9KiFXyGSsc8uYlsQgmfUHg='} 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000017e41a052140b03f-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.917 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.918 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003fe76d0ee5b4f82e-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj28 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj28 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj28 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.919 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj28 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:g+uGI1z/un9nqYAUtaL7/HrQahQ= 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g+uGI1z/un9nqYAUtaL7/HrQahQ='} 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008a54faf378fd7e52-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.920 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '168'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b07c907c41f43bb5-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj29 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj29 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj29 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.921 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj29 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:cdhZqxTE5M/50AXFqynLGpxrJPw= 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:cdhZqxTE5M/50AXFqynLGpxrJPw='} 2024-04-16T19:28:53.922 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000738ff7419325bc36-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.923 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000713060d5798afa8f-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj30 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj30 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj30 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.924 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj30 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:MmPUD9GK4Yhh3jY0IFlnhH7ZkZE= 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:MmPUD9GK4Yhh3jY0IFlnhH7ZkZE='} 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d9e8da8aa527bd30-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.925 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.926 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e27564638851a9af-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj31 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj31 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj31 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.927 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj31 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gO6fFoWp3CkcyLqKbNLpGD6zLaQ= 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gO6fFoWp3CkcyLqKbNLpGD6zLaQ='} 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005d8ce63aa48c249c-00661eba31-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:37 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.928 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:37 GMT 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM= 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:37 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aJZjDQ+rWAwrZIm8hg8voYvC9mM=', 'Content-Length': '0'} 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '186'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000baf7a586454c7959-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj32 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj32 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj32 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.929 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj32 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:nSJXONsxbWbP5dggGtmfSWG4hX4= 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nSJXONsxbWbP5dggGtmfSWG4hX4='} 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000053e8086f7d64db64-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.930 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.931 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '192'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000062b5011f27ea32e-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj33 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj33 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj33 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.932 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj33 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+OmWgNPuAPt2Tu1wMXyBHXTciu0= 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+OmWgNPuAPt2Tu1wMXyBHXTciu0='} 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000032e355110a42153-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.933 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '198'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003b7b523a4cba74b8-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj34 2024-04-16T19:28:53.934 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj34 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj34 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj34 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.935 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:jWbZAPuTyoRnfZxADwO1cQtyvtg= 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jWbZAPuTyoRnfZxADwO1cQtyvtg='} 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000033f1ebfbcfb26d5-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.936 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '204'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eb4932ad781b3221-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj35 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj35 2024-04-16T19:28:53.937 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.938 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj35 2024-04-16T19:28:53.938 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.938 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.938 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.938 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj35 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5UrE8rZHMsVivEMGEyvlYN624EQ= 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5UrE8rZHMsVivEMGEyvlYN624EQ='} 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e91d66a1ee085214-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.939 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.940 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '210'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c84f03d3d5cfb396-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj36 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj36 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj36 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.941 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj36 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:lcOQc8jSHyWsUQbMYtTPRKxKW2A= 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lcOQc8jSHyWsUQbMYtTPRKxKW2A='} 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b33d50fbb02ce03d-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.942 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '36'), ('X-RGW-Bytes-Used', '216'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066e6a1b6c36cd40b-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj37 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj37 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj37 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.943 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj37 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:XA5wa9RO68SvrNhKTtrmxlNampk= 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XA5wa9RO68SvrNhKTtrmxlNampk='} 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c1602d6e9d100448-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.944 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.945 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '222'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f50e80b8c213adfa-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj38 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj38 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj38 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.946 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj38 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:DZAhoytqZa+MlCf0ltR5AkPVUYI= 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DZAhoytqZa+MlCf0ltR5AkPVUYI='} 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc60bd14b4bbd99c-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.947 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '228'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008558f53791349cdd-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj39 2024-04-16T19:28:53.948 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj39 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj39 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj39 2024-04-16T19:28:53.949 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:BgryOXDWZCC5Tw6iL5yIwoHYKow= 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:BgryOXDWZCC5Tw6iL5yIwoHYKow='} 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000aa2e1a2a48b9c9e3-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.950 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '39'), ('X-RGW-Bytes-Used', '234'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b95192304f65decc-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj40 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj40 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj40 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.951 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj40 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:IU6ydphsCCc72GlaYfFiCvC06oA= 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IU6ydphsCCc72GlaYfFiCvC06oA='} 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008471487ec0da0375-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.952 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.953 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fb0bae1d8b7d40ab-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj41 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj41 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj41 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.954 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj41 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ZMn0jhZIbwGTtr+RdBp3FPn1yzA= 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZMn0jhZIbwGTtr+RdBp3FPn1yzA='} 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007701e755dfefa32c-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.955 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '41'), ('X-RGW-Bytes-Used', '246'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000a0940aaa2c43cba-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj42 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj42 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.956 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj42 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj42 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+GmPJavXsp3TlitVaBio8l+T/50= 2024-04-16T19:28:53.957 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+GmPJavXsp3TlitVaBio8l+T/50='} 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000c9f920fcddeb739-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.958 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '252'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b5d714269f7e4e56-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj43 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj43 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj43 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.959 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj43 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iEwlnTuoEhozkenpmnrLuv2y95M= 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iEwlnTuoEhozkenpmnrLuv2y95M='} 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ed9b0dfe86ff0cec-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.960 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '43'), ('X-RGW-Bytes-Used', '258'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000798e2f2b653eaa00-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.961 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj44 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj44 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj44 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.962 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj44 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:0cGSFZrkyomi5eD4FrrNQv3q9YE= 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0cGSFZrkyomi5eD4FrrNQv3q9YE='} 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004deb9fde98ec6856-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.963 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '44'), ('X-RGW-Bytes-Used', '264'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050b5b141ce3b5c76-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj45 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj45 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj45 2024-04-16T19:28:53.964 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj45 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Ld29GMPp6mGph4y/TwWN5HtPH/Y= 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Ld29GMPp6mGph4y/TwWN5HtPH/Y='} 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f94320fa674b89b9-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.965 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.966 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '270'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005be30bb7bea1f009-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj46 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj46 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj46 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.967 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj46 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:G7YUIIc/PbkafIVyUI8tDfvNP0o= 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:G7YUIIc/PbkafIVyUI8tDfvNP0o='} 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000041b8d3d213d2fab6-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.968 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '46'), ('X-RGW-Bytes-Used', '276'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000083267c889adc2b06-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.969 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj47 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj47 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj47 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.970 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj47 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:6YFPKG9UijJYjMJj8SWbIyJwgvE= 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6YFPKG9UijJYjMJj8SWbIyJwgvE='} 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000043bd1b19d9892df2-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.971 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '47'), ('X-RGW-Bytes-Used', '282'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bb9e5d24c573a955-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj48 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj48 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj48 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.972 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj48 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CxrUgzvMKChWJzYN4VX3FprBJnQ= 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CxrUgzvMKChWJzYN4VX3FprBJnQ='} 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fe4865108ca8c0e1-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/ 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/ 2024-04-16T19:28:53.973 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/ 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:/lybtug-11/ 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.974 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E= 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bIVM+oEoA1kQWmTW5PgdtrzFn6E=', 'Content-Length': '0'} 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '48'), ('X-RGW-Bytes-Used', '288'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000354404913400807e-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-11/obj49 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-11/obj49 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-11/obj49 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.975 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:/lybtug-11/obj49 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Nmbb1GDSmAGbP4XVt6NRQ0l3N2k= 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Nmbb1GDSmAGbP4XVt6NRQ0l3N2k='} 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000054358e9c3f76abb6-00661eba32-4353-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.976 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000080486a6223627db-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj1 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj1 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj1 2024-04-16T19:28:53.977 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj1 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:erC5YB3G4fN1EmxF1CkZB+s/poQ= 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:erC5YB3G4fN1EmxF1CkZB+s/poQ='} 2024-04-16T19:28:53.978 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000060921bc1ae71b5bb-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.979 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fc5de81100e99ce9-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj2 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj2 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj2 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.980 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj2 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oCMc5d37rBC1ez90PzzVeS6lcsA= 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oCMc5d37rBC1ez90PzzVeS6lcsA='} 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009a2dfbc399227c59-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.981 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050980cde1a8aab6f-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj3 2024-04-16T19:28:53.982 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj3 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj3 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj3 2024-04-16T19:28:53.983 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9w0RWRIEP8BHR42DwiIMtzg/ZaQ= 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9w0RWRIEP8BHR42DwiIMtzg/ZaQ='} 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b06eb23b25367837-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.984 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a6afe447b0ab4b55-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj4 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj4 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj4 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.985 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj4 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:z/njFxMvvB21eYeX3c6+kjcNAjE= 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:z/njFxMvvB21eYeX3c6+kjcNAjE='} 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000034c14f09ab536ea7-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.986 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.987 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000140249a9745b5dab-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj5 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj5 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj5 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.988 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj5 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:4lpw7UymqwwH/b6nARG5Nx8dQFM= 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4lpw7UymqwwH/b6nARG5Nx8dQFM='} 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ca025750975ece77-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.989 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009b93633364607282-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj6 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj6 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj6 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.990 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj6 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:KB3/eHWkpWbNh7I7rb2ync5m/fI= 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KB3/eHWkpWbNh7I7rb2ync5m/fI='} 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ce891a0d5bf9d4be-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.991 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.992 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000286b55b89be4dd96-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj7 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj7 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj7 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.993 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj7 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:C+RP5TgEVTQ7pyr4sgU/ro6xAOY= 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:C+RP5TgEVTQ7pyr4sgU/ro6xAOY='} 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008b29d2c920f62363-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.994 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e483a227ea0ad272-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj8 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj8 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj8 2024-04-16T19:28:53.995 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj8 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:o7IQg0n4QVt50wxo30E4d+2qqvY= 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o7IQg0n4QVt50wxo30E4d+2qqvY='} 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006d7f876c692b636c-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.996 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:53.997 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea57af861c2e113e-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj9 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj9 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj9 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:53.998 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj9 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:zlQJdT2GGVBCCA+UHpgVWEbPrWY= 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:zlQJdT2GGVBCCA+UHpgVWEbPrWY='} 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008ac957b1a61587b5-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:53.999 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c6edd59d351272eb-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj10 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj10 2024-04-16T19:28:54.000 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj10 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj10 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:JDNJXQAOuCYUeihGaMlclEqYkkw= 2024-04-16T19:28:54.001 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JDNJXQAOuCYUeihGaMlclEqYkkw='} 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000017deb5cd5d691009-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.002 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ca61a069f551fcb8-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj11 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj11 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj11 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.003 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj11 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:nmPyPwyWuRWV+AqvQU4bfiY6hA0= 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nmPyPwyWuRWV+AqvQU4bfiY6hA0='} 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000932b6909414e9bbf-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.004 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008cd0d9b893cab4d0-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.005 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj12 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj12 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj12 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.006 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj12 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:VTENEmyPuGlLlBZQKZcqVsLZUKA= 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VTENEmyPuGlLlBZQKZcqVsLZUKA='} 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004c5d5b60d8c061f7-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.007 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY= 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oA+Qyvh3imJIsZFSi5aOl9KtQTY=', 'Content-Length': '0'} 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006566105b09e0fb8e-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:38 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj13 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj13 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj13 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.008 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:38 GMT 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj13 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ks25KBP7NfBbU62ZpsxUCY9jDmc= 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:38 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ks25KBP7NfBbU62ZpsxUCY9jDmc='} 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ec4dc29a991314ae-00661eba32-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.009 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.010 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000437677509f4dfa90-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj14 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj14 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj14 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.011 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj14 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:h405Y96cJgCmYdrPlnXx+m2u5PU= 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:h405Y96cJgCmYdrPlnXx+m2u5PU='} 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001084fc3ad3836ca6-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.012 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e4f245a8a7b1192e-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj15 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj15 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.013 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj15 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj15 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.014 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:8HO35itNVKVDNmTY7SwyaCeKeQw= 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8HO35itNVKVDNmTY7SwyaCeKeQw='} 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004b8f3c4ca343e7b6-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.015 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001945170028be4337-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj16 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj16 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj16 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.016 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj16 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:O3AZMA91Jkb+S4AQ5VaIHz5FfTs= 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O3AZMA91Jkb+S4AQ5VaIHz5FfTs='} 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000de352dbcf1ca640c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.017 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.018 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000098146b01e8ab5c1d-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj17 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj17 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj17 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.019 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj17 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a05ogLfDYDBkQZrdFW836hgCEM0= 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a05ogLfDYDBkQZrdFW836hgCEM0='} 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cb151fb28356e1e5-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.020 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e9a1525895c8d8dc-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj18 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj18 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.021 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj18 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj18 2024-04-16T19:28:54.022 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:kM0a58UXDxt8inrdrJpCRWGaXiU= 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kM0a58UXDxt8inrdrJpCRWGaXiU='} 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cfe9bc2434da3428-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.023 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '108'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000041915185d0ba2af0-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj19 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj19 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj19 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.024 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj19 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:beroYqmoXy6yynzh2fyqu7wHCNM= 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:beroYqmoXy6yynzh2fyqu7wHCNM='} 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ef5d81e78a84cb14-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.025 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.026 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '114'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008398dd8f43928adc-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj20 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj20 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj20 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.027 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj20 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:4uII/FhM4P82igOXnXlg3oykXQ4= 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4uII/FhM4P82igOXnXlg3oykXQ4='} 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000931bc5b33c3462d3-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.028 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '120'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ad86e4943003d67-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj21 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj21 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj21 2024-04-16T19:28:54.029 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj21 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:H9h9TvqyGxphQ/qKTuIB8hXqDWg= 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:H9h9TvqyGxphQ/qKTuIB8hXqDWg='} 2024-04-16T19:28:54.030 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f49b6c483d5a337f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.031 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '126'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dcbd023d86cb912f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj22 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj22 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj22 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.032 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj22 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9jP+eL2jQeYj7tjNF6nZKGYpJr4= 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9jP+eL2jQeYj7tjNF6nZKGYpJr4='} 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d1e7bafcc1aeebee-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.033 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '132'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f7f6fe6b119a1c69-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.034 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj23 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj23 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj23 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj23 2024-04-16T19:28:54.035 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:UsOO3o618dGANrfPjzp371cljGw= 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:UsOO3o618dGANrfPjzp371cljGw='} 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000a121842aba556b3-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.036 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '138'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ffe60c5ab0e102c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj24 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj24 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj24 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.037 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj24 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:qfZIwvEr00NXUVH8986B9+mIjJc= 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qfZIwvEr00NXUVH8986B9+mIjJc='} 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001008d77528cf78ea-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.038 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.039 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '144'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000076fd282d70bea7f2-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj25 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj25 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj25 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.040 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj25 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:O5prrqsNBgjudXXo9cetZDeMBO8= 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:O5prrqsNBgjudXXo9cetZDeMBO8='} 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000004022a60c44fe727-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.041 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '150'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000ba578b38a990bfb-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.042 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj26 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj26 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj26 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj26 2024-04-16T19:28:54.043 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OXR1VgKiCFWJSZ59D4tCxK2bAkY= 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OXR1VgKiCFWJSZ59D4tCxK2bAkY='} 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c146bfc0929645e2-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.044 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '156'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ab6f7c039ab2608c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj27 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj27 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj27 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.045 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj27 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5UwC3G6KrKM4plEDh+FWXuMZnY4= 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5UwC3G6KrKM4plEDh+FWXuMZnY4='} 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000eb3e607b6b6ad9a2-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.046 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.047 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '162'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a599df499bbfdecc-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj28 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj28 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj28 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.048 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj28 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:4t5qmsa+9k1lCWN+0oRFvoxHX84= 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4t5qmsa+9k1lCWN+0oRFvoxHX84='} 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000064b1b4cd8ae51b6e-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.049 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.050 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '168'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000253fb594de5623ed-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj29 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj29 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj29 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.051 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj29 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:6f/CsabvlTcnYBwoFnDQvco40Fw= 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6f/CsabvlTcnYBwoFnDQvco40Fw='} 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000076eb802062a56d1e-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.052 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '174'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b23be95cadbe8e3f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj30 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj30 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj30 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.053 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj30 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:K4o+j5fNVnRHrouyhD7ObCGxBAE= 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:K4o+j5fNVnRHrouyhD7ObCGxBAE='} 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001bb2f64c15c4b5ad-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.054 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.055 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '180'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000051d70031ad40bd59-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj31 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj31 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj31 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.056 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj31 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+buhP34IIsgxHxtErADz8eqruf4= 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+buhP34IIsgxHxtErADz8eqruf4='} 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000bd9306ac08ecc15c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.057 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '186'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000096b94c8d63beef8c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj32 2024-04-16T19:28:54.058 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj32 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj32 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj32 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.059 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:8zgihM9SxdBHDWIJplhjTlBCctw= 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8zgihM9SxdBHDWIJplhjTlBCctw='} 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a14118696c517bf1-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.060 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '192'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fba722c0451c6169-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj33 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj33 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj33 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.061 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj33 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gu8Sv2z2FO3H76TRGXbSRxTvuek= 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gu8Sv2z2FO3H76TRGXbSRxTvuek='} 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005c29d62b8cb19034-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.062 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.063 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '198'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000405cb94c0da0bc95-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj34 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj34 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj34 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.064 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj34 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:lnr25DzwqtPlbz3hSkGtDA01qgg= 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lnr25DzwqtPlbz3hSkGtDA01qgg='} 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009fb1e4d6b1889a7f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.065 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '204'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000215e2105d01c019c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj35 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj35 2024-04-16T19:28:54.066 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj35 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj35 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.067 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:lBuKujZGBqelRsRsir6OcSDmxKE= 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lBuKujZGBqelRsRsir6OcSDmxKE='} 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000af0d6bbd7805de06-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.068 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '210'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000035f5a6bb4d54df58-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj36 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj36 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj36 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.069 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj36 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:KNdsxJRdaTu/yT7E4yPJye3oH2o= 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KNdsxJRdaTu/yT7E4yPJye3oH2o='} 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000063b04fd489d27f6c-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.070 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.071 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '36'), ('X-RGW-Bytes-Used', '216'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ce4192012cae1b57-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj37 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj37 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj37 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.072 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj37 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:/E9fB8/K3Fjcjnfz6Lgz4++qDTA= 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/E9fB8/K3Fjcjnfz6Lgz4++qDTA='} 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c1169d7a632249cf-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.073 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '37'), ('X-RGW-Bytes-Used', '222'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000054f3a7d73d532887-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj38 2024-04-16T19:28:54.074 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj38 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj38 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj38 2024-04-16T19:28:54.075 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Bajf2c54tI6vxQBNOxzVqTBl+Xk= 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Bajf2c54tI6vxQBNOxzVqTBl+Xk='} 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fbd9178163cbe466-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.076 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '38'), ('X-RGW-Bytes-Used', '228'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006c055fc2798fecc4-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj39 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj39 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj39 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.077 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj39 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:6BK+U+NMclQLMTjDxBTsilLSCco= 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6BK+U+NMclQLMTjDxBTsilLSCco='} 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001cc78fd4f049f1bd-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.078 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.079 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '39'), ('X-RGW-Bytes-Used', '234'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ee4918c648d2836f-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj40 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj40 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj40 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.080 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj40 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:M+ZyCE/nlAkQwpySMzsb/F76lSc= 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:M+ZyCE/nlAkQwpySMzsb/F76lSc='} 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003a6f1505bbdee4f8-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.081 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '40'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007508a346400e062b-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj41 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj41 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.082 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj41 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj41 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:/oWMmpoVcge5Pi/nvHJ7NLWYmi8= 2024-04-16T19:28:54.083 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/oWMmpoVcge5Pi/nvHJ7NLWYmi8='} 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004fe8b6c7aece44ba-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.084 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '41'), ('X-RGW-Bytes-Used', '246'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009664f04fa0288b49-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj42 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj42 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj42 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.085 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj42 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9QE7OO+3z4u21SlnOr/g/Hv06pk= 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9QE7OO+3z4u21SlnOr/g/Hv06pk='} 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b715d47398094cdf-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.086 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '42'), ('X-RGW-Bytes-Used', '252'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e22f45f6127ad540-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.087 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj43 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj43 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj43 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.088 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj43 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:/C7b9SxlyB0A0+nl1RbKY8fcbOc= 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:/C7b9SxlyB0A0+nl1RbKY8fcbOc='} 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004d40b6f9f258281a-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:39 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.089 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:39 GMT 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY= 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:39 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:d4clTQnZTZbiXE6p48MP/oZlymY=', 'Content-Length': '0'} 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '43'), ('X-RGW-Bytes-Used', '258'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d741353817187778-00661eba33-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj44 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj44 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj44 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.090 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj44 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ImFIyRmCm+FhnqR2GmII83H1Nnw= 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ImFIyRmCm+FhnqR2GmII83H1Nnw='} 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009bc9aff4bd59a729-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.091 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.092 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '44'), ('X-RGW-Bytes-Used', '264'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000323333e3365adf67-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj45 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj45 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj45 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.093 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj45 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:WfKRfrDnz/QX0j4XNy9PNBsWWzo= 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WfKRfrDnz/QX0j4XNy9PNBsWWzo='} 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001720b0bb69e36212-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.094 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '45'), ('X-RGW-Bytes-Used', '270'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000018d50b3cb255d1c1-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj46 2024-04-16T19:28:54.095 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj46 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj46 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj46 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.096 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:+J/x07gmqU7xzSk6he3oJIu/yWo= 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:+J/x07gmqU7xzSk6he3oJIu/yWo='} 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000900ebb6e2e2dd71a-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.097 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '46'), ('X-RGW-Bytes-Used', '276'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b1a6c7f3b28a2a4c-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj47 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj47 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj47 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.098 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj47 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:AJg8CkuJpOwz0Vr/VrxND108N6E= 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:AJg8CkuJpOwz0Vr/VrxND108N6E='} 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000b1f5c8a436baaf2-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.099 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T19:28:54.100 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '47'), ('X-RGW-Bytes-Used', '282'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009e3f23378d48440d-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj48 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj48 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj48 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.101 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj48 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:w6BkmtTKXFifRjTOUWsqey83cQk= 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:w6BkmtTKXFifRjTOUWsqey83cQk='} 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000090598eff8e15047-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/ 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/ 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/ 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.102 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:/lybtug-12/ 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0= 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:gqfbYb1JrpDYbe20Tff3+07DFf0=', 'Content-Length': '0'} 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '48'), ('X-RGW-Bytes-Used', '288'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008f8ac638818e949c-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-12/obj49 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-12/obj49 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-12/obj49 2024-04-16T19:28:54.103 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:qPXxZ/RPSWTmyZje6CcRDA== 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 17:49:40 GMT 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:/lybtug-12/obj49 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:brTjrxfGqV7qAcziz1q1P/IaIyQ= 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 17:49:40 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:brTjrxfGqV7qAcziz1q1P/IaIyQ='} 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008a831b4da0e0e685-00661eba34-4335-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 17:49:40 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.104 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713289776.972768_45.1","last_update":"2024-04-16T17:49:36.972768Z"},{"marker":"1_1713289749.894324_100.1","last_update":"2024-04-16T17:49:09.894324Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713289776.972768_45.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:49:40.447+0000 7faba102d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.105 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:49:40.447+0000 7faba102d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289776.972768_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:36.972768Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:40Z"} 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713289776.972768_45.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T17:49:36.972768Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713289749.894324_100.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T17:49:09.894324Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T17:49:40Z"} 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713289776.972768_45.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T17:49:36.972768Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713289749.894324_100.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T17:49:09.894324Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}] 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713289776.972768_45.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713289776.972768_45.1', 1: '1_1713289749.894324_100.1', 2: '', 3: ''} 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: finish meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-11', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-11', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-11 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:49:40.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:49:45.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.106 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:49:50.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:49:55.663+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:00.667+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:05.667+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:10.667+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:15.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:20.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:25.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:30.670+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:35.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:40.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:45.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:50.674+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:50:55.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:00.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:05.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.107 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:10.677+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:15.681+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:20.681+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:25.685+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:30.685+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:35.689+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:40.689+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:45.688+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:50.688+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:51:55.692+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:00.692+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:05.692+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:10.696+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:15.696+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:20.696+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:25.695+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.108 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:30.699+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:35.699+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:40.699+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:45.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:50.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:52:55.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:00.703+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:05.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:10.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:15.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:20.707+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:25.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:30.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:35.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:40.711+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:45.715+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.109 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:50.715+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:53:55.714+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:00.718+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:05.718+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:10.718+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:15.722+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:20.722+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:25.722+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:30.726+0000 7f52084e9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:35.726+0000 7f52084e9980 -1 bucket checkpoint timed out waiting to reach incremental sync 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:35.726+0000 7f52084e9980 0 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T17:54:35.726+0000 7f52084e9980 -1 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 110 2024-04-16T19:28:54.110 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.111 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.111 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.111 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_bucket_reshard_index_log_trim 2024-04-16T19:28:54.111 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 1998, in test_bucket_reshard_index_log_trim 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests: assert(len(json_obj_5['layout']['logs']) == 1) 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.112 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: create bucket zone=test-zone1 name=lybtug-50 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.113 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:uvhsFsGYw4ICde9McLjlrHMnkdg= 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:uvhsFsGYw4ICde9McLjlrHMnkdg=', 'Content-Length': '0'} 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx00000615ed1d6aacb179f-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.114 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b23e300758907cb0-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.115 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5thL707kPxD7kJii1Vpgz2cqPyE= 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5thL707kPxD7kJii1Vpgz2cqPyE='} 2024-04-16T19:28:54.116 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d4f0a17730323df4-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.117 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007a223f0b542ae158-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.118 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9syNMp1yVQR0SM+A4Ezlh3in78c= 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9syNMp1yVQR0SM+A4Ezlh3in78c='} 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001dde57b76928a1d4-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.119 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009ac16bb7360a73d3-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.120 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.121 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:vbCO5WoKisUgAkk0p8aTdcTGTFs= 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vbCO5WoKisUgAkk0p8aTdcTGTFs='} 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b1f0ed1038f6bbfb-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.122 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo= 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TCnw9963AvFInNxU4YX108mflJo=', 'Content-Length': '0'} 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ecaa10b96fbb8703-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.123 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:15:58 GMT 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:kHxBMgSUbPDcTq6g5dA/pts8Kt0= 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:15:58 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kHxBMgSUbPDcTq6g5dA/pts8Kt0='} 2024-04-16T19:28:54.124 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001c3bcc8d42f5e803-00661ec05e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713291358.216212_532.1","last_update":"2024-04-16T18:15:58.216212Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713291358.216212_532.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.125 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:58.623+0000 7fd3501be980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:58.623+0000 7fd3501be980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291358.216212_532.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:58.216212Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:58Z"} 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713291358.216212_532.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:15:58.216212Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:15:58Z"} 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713291358.216212_532.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:15:58.216212Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713291358.216212_532.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713291358.216212_532.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: finish meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-50 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.126 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:15:58.827+0000 7fad9dacb980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:03.830+0000 7fad9dacb980 1 bucket sync caught up with source: 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, 00000000001.55.6, , , , 00000000001.67.6, , , 00000000001.46.6, , 00000000001.53.6] 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, 00000000001.55.6, , , , 00000000001.67.6, , , 00000000001.46.6, , 00000000001.53.6] 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:03.830+0000 7fad9dacb980 0 bucket checkpoint complete 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-50 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:16:04.006+0000 7fb7e31d9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:16:09.006+0000 7fb7e31d9980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:16:14.006+0000 7fb7e31d9980 1 bucket sync caught up with source: 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: local status: [, 00000000001.901.6, , , , 00000000001.1167.6, , , 00000000001.306.6, , 00000000001.407.6] 2024-04-16T19:28:54.127 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: remote markers: [, 00000000001.901.6, , , , 00000000001.1167.6, , , 00000000001.306.6, , 00000000001.407.6] 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:16:14.006+0000 7fb7e31d9980 0 bucket checkpoint complete 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: INFO: comparing bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.128 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0= 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0=', 'Content-Length': '0'} 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000250fba0e42c19b5b-00661ec06e-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.129 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0= 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IY/Y2Z9MyRrvu69hbjparJmfSg0=', 'Content-Length': '0'} 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ccd602ab393da906-00661ec06e-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: bucket1 objects: 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.130 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.131 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000489c48a5765b016a-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=a 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=b 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=c 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=d 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: bucket2 objects: 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.132 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T19:28:54.133 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067e9cd392d9b304e-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=a 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=b 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=c 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=d 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.134 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a76fa31306ba956d-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.135 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.136 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA= 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:shj4ZwQ2yx4HPItLJu8BTSDNgrA=', 'Content-Length': '0'} 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6174e676fde57f9-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.137 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T19:28:54.138 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a49399ff1a7ab6b6-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.139 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013d5812e7cd278ef-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.140 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY= 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY=', 'Content-Length': '0'} 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000637dd4dd264e4d2d-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.141 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.142 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY= 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SJfX6ig/pockgPDbi6pKQiRudvY=', 'Content-Length': '0'} 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004820bd4c9a562e31-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.143 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004982d41bb25cb472-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.144 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.145 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg= 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a13CbST1x4P7Afm3POV4vhg9uTg=', 'Content-Length': '0'} 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df21cbd0cb1452c7-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.146 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d4ff0da215c30d9-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.147 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.148 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ded40dd8ec6580c7-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.149 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE= 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE=', 'Content-Length': '0'} 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b63a7e689d61f9-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.150 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE= 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:vc6545qtcAqt9+l1oFgrMAn7YYE=', 'Content-Length': '0'} 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5db143facb468d2-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.151 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.152 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I= 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I=', 'Content-Length': '0'} 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a3ed1953b6978e1-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.153 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I= 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:khkPY3JHyxDZ7j5HmhNY6hG7b0I=', 'Content-Length': '0'} 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d72aca4917f6b16-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.154 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.155 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001229000d3ae140ae-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.156 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab16a75e8c34797f-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.157 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T19:28:54.158 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001000939febb5de50-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.159 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0576bc2956d65f9-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.160 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg= 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg=', 'Content-Length': '0'} 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bf4fc1e91d52cb3-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.161 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.162 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg= 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:dciecWCt/92UJ99Rrf7GmY+KgXg=', 'Content-Length': '0'} 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd8eec98ee657eda-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.163 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ccff8d860766ff0-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.164 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.165 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE= 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wSdkoq4ZVMQ9NeTD3QJeWTH65hE=', 'Content-Length': '0'} 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071ea13b6b5c1d357-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.166 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000492702019b4eaa90-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.167 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.168 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5cb1932fa9d2967-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.169 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ= 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001dc45b379570665f-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.170 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ= 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iYGu+bTxuWmf+qMhhZMR9qNp2bQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d640d94f9df5ebbc-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.171 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.172 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI= 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI=', 'Content-Length': '0'} 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2894c2a5dac881b-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.173 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI= 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HgzdB56lCo/WpBaqrX9ffMhBmFI=', 'Content-Length': '0'} 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008aac2e293166aca2-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.174 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.175 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f406332fb602772-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.176 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a8eb04d9177f1f9-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.177 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.178 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f4d54b91eb7d911-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.179 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d44ff7844ace201-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.180 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM= 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM=', 'Content-Length': '0'} 2024-04-16T19:28:54.181 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbd3ddefb5d3cbd2-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.182 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM= 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:imZFBaE3Qu/YHdS859GWWM9+bHM=', 'Content-Length': '0'} 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a56d300931273479-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.183 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T19:28:54.184 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001813b39a0a9a12dc-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.185 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4= 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rfxTsTVgsvYlgyyVq/6uWQLqcs4=', 'Content-Length': '0'} 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e05a3b25865d019a-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.186 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4db10df60f805ce-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.187 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.188 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079f217cea22b942a-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.189 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40= 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40=', 'Content-Length': '0'} 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c9085e09649e986-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.190 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40= 2024-04-16T19:28:54.191 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:duIlulKh+RIA5+IJB7Kg/Dyxq40=', 'Content-Length': '0'} 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b5eacee7bddbed1-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.192 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E= 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E=', 'Content-Length': '0'} 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013ce4732480fc210-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.193 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E= 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:oJr4quK0S+MoHzaJcw/8hsNTQ7E=', 'Content-Length': '0'} 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002ba4a669dec3993-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.194 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.195 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0e55afa5fddf315-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.196 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3d1ded25fb2e5fc-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.197 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.198 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000839e19fcf3024cae-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.199 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007267426868d6694c-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.200 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.201 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI= 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI=', 'Content-Length': '0'} 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef8d4e310ddf7504-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.202 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI= 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:en/dYYnxFVPl/e77Kt0VW1zjKUI=', 'Content-Length': '0'} 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef595552e67feaa3-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.203 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.204 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f32c21b3e01f92a-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.205 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU= 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:pLRgWDy2fAwerY52/B8zvwXQvYU=', 'Content-Length': '0'} 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baba412cc933aabb-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.206 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.207 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd39172da960b0b4-00661ec06e-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.208 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4= 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:iInCMhkFpsUl1bBDURE2MXrl8z4=', 'Content-Length': '0'} 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008be175c3ce717008-00661ec06e-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:14 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.209 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:14 GMT 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wdl6mP8v9hYo9lyKZtVtP+FfDoY= 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:14 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wdl6mP8v9hYo9lyKZtVtP+FfDoY=', 'Content-Length': '0'} 2024-04-16T19:28:54.210 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e68fdaa4c27b1fae-00661ec06f-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:15 GMT 2024-04-16T19:28:54.211 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:u1lpxb5EdR/jqPlCQ+hva1SLtW8= 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:u1lpxb5EdR/jqPlCQ+hva1SLtW8=', 'Content-Length': '0'} 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009007d94e267a19d3-00661ec06f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.212 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:15 GMT 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM= 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM=', 'Content-Length': '0'} 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000836c1340cb19e47d-00661ec06f-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.213 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:15 GMT 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM= 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:qjyaPrVdKNh9BQBQASfCToGq1fM=', 'Content-Length': '0'} 2024-04-16T19:28:54.214 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000659d1c1dc4cee97f-00661ec06f-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.215 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:15 GMT 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY= 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY=', 'Content-Length': '0'} 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f55706da73ad987-00661ec06f-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.216 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:15 GMT 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY= 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:15 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Kr8roDyt/9bD6o3NODb1NOdoNKY=', 'Content-Length': '0'} 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fcc88e79b2f0fcd-00661ec06f-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:15 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.217 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: INFO: success, bucket identical: bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: {"layout":{"resharding":"None","current_index":{"gen":0,"layout":{"type":"Normal","normal":{"num_shards":11,"hash_type":"Mod"}}},"logs":[{"gen":0,"layout":{"type":"InIndex","in_index":{"gen":0,"layout":{"num_shards":11,"hash_type":"Mod"}}}}]}} 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-50', '--gen', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-50', '--gen', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-50 --gen 0 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"op_id":"1#00000000001.55.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.15343941463733059060","op":"write","object":"a","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.232829449Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]},{"op_id":"5#00000000001.67.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2034444563773777923","op":"write","object":"d","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.362254267Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]},{"op_id":"8#00000000001.46.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.12822008793054755835","op":"write","object":"c","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.312027972Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]},{"op_id":"10#00000000001.53.6","op_tag":"054102a3-8d94-4596-8b3b-6341eab312e1.4804.2152254117184578004","op":"write","object":"b","instance":"","state":"complete","index_ver":1,"timestamp":"2024-04-16T18:15:58.290803125Z","ver":{"pool":9,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19"}]}] 2024-04-16T19:28:54.218 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket lybtug-50 --num-shards 13 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: tenant: 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: bucket name: lybtug-50 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: total entries: 4 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: {"layout":{"resharding":"None","current_index":{"gen":1,"layout":{"type":"Normal","normal":{"num_shards":13,"hash_type":"Mod"}}},"logs":[{"gen":0,"layout":{"type":"InIndex","in_index":{"gen":0,"layout":{"num_shards":11,"hash_type":"Mod"}}}},{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}}]}} 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'lybtug-50', '--gen', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'lybtug-50', '--gen', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.219 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket lybtug-50 --gen 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [] 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.220 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008cbf16b4f7dd8b3b-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.221 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ruYWUxSP8YhsmFw1G/B34FCsbQg= 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ruYWUxSP8YhsmFw1G/B34FCsbQg='} 2024-04-16T19:28:54.222 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d6f20914fc9fcc09-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.223 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000548abd2aeee90cbe-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.224 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:l5a9P60fXC4jP9HpVd833oxmAwY= 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:l5a9P60fXC4jP9HpVd833oxmAwY='} 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003e81fcd115f30c79-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.225 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000040dc6d300b986900-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.226 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.227 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Aw1cLLxzhiGUw1Jt3csiSZmZeVI= 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Aw1cLLxzhiGUw1Jt3csiSZmZeVI='} 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dad40ab842caf3c8-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.228 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ= 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CXVXs3yBrF3GKZE1zqiiqGX2TlQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000496ae517ec22d2d7-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.229 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:16:16 GMT 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:B8nHYSG6uRZvjFXauDZowtv9Qsw= 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:16:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:B8nHYSG6uRZvjFXauDZowtv9Qsw='} 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000689ebcc5787d5525-00661ec070-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-50 2024-04-16T19:28:54.230 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:16.302+0000 7fb9de2a8980 1 waiting to reach latest gen 1, on 0.. 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:21.306+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:26.310+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:31.310+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.231 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:36.314+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:41.318+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:46.318+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:51.317+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:16:56.321+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:01.321+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.232 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , , , , , , ] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:06.325+0000 7fb9de2a8980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , , , , , , 00000000002.36.6, , , , 00000000001.32.6, ] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:11.325+0000 7fb9de2a8980 1 bucket sync caught up with source: 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: local status: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: remote markers: [, , 00000000001.26.6, , , , , 00000000002.36.6, , , 00000000001.62.6, 00000000001.32.6, ] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:11.325+0000 7fb9de2a8980 0 bucket checkpoint complete 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-50 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:17:11.505+0000 7fdd6454c980 1 waiting for incremental sync to catch up: 2024-04-16T19:28:54.233 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: local status: [, 00000000002.905.6, , , , 00000000001.1167.6, , , 00000000001.306.6, , 00000000002.413.6] 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: remote markers: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:17:16.505+0000 7fdd6454c980 1 bucket sync caught up with source: 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: local status: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: remote markers: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:17:16.505+0000 7fdd6454c980 0 bucket checkpoint complete 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: INFO: comparing bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.234 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM= 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM=', 'Content-Length': '0'} 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000010dcc596d8edb598-00661ec0ac-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.235 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.236 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM= 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LPqAuWn3oEFjSQ55sQqxx+fZeaM=', 'Content-Length': '0'} 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000024f6b9107005997b-00661ec0ac-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: bucket1 objects: 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.237 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ee9633d3d403e2b-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.238 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=a 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=b 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=c 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=d 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=e 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=f 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=g 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=h 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: bucket2 objects: 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.239 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005205633bf757be12-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.240 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=a 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=b 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=c 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=d 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=e 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=f 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=g 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=h 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.241 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T19:28:54.242 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067061c63e84d12d3-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.243 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps= 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k9g1vyFKxnt5KgsK9HAvqLrUbps=', 'Content-Length': '0'} 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c46d528db8db9f5f-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.244 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.245 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4f97346761b2cb3-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.246 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000417f45a78f7022b2-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.247 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U= 2024-04-16T19:28:54.248 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U=', 'Content-Length': '0'} 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003e407f64ffd74bf-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.249 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U= 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Xqj6U6d64vgjMVVoklOKt59qH1U=', 'Content-Length': '0'} 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dd39912c6800d45-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.250 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T19:28:54.251 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001127a8c240d2a2f8-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.252 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk= 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Oc85j+0szYGEHKFbGwJurWlE1jk=', 'Content-Length': '0'} 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f6993ac9b179f79-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.253 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6d6161c07361489-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.254 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.255 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b983e0239bd401e3-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.256 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4= 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4=', 'Content-Length': '0'} 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4129ecfb4a97016-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.257 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4= 2024-04-16T19:28:54.258 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wmZ2ig4Iz8txXtcC5hJWtWm00L4=', 'Content-Length': '0'} 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5ab46c088b9ac44-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.259 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs= 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs=', 'Content-Length': '0'} 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d50d4f6565a1fa9-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.260 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs= 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:lgfH694tbAF8VKiXKOC7DidkhLs=', 'Content-Length': '0'} 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000571bfba6ffde9338-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.261 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.262 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9d313c37a2430ee-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.263 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3103312ecd3ef4b-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.264 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.265 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f863d55ce30e14cd-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.266 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ffba2674ad80d544-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.267 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.268 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo= 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo=', 'Content-Length': '0'} 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002baa3a5dbb13679c-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.269 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo= 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9to8vbANJODI/URtsOg9VmoOgjo=', 'Content-Length': '0'} 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb93e3f7666a59a3-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.270 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.271 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057f6b799ad464157-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.272 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8= 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:0B8oQhtq2hjyTKLjz4t/5lshnv8=', 'Content-Length': '0'} 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7133b53c2edeb42-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.273 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.274 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000fa7687430d7df3c-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.275 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68= 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ThaWXgvx2LRFaleFddOBCteuU68=', 'Content-Length': '0'} 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056d3ea1b8a0f1e1b-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.276 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc= 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc=', 'Content-Length': '0'} 2024-04-16T19:28:54.277 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004572b6a5c6a10bd-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.278 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc= 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjA4aKBVfLzyTiv2AUKF+d2I0Jc=', 'Content-Length': '0'} 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae7865510aa87c90-00661ec0ac-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.279 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:16 GMT 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:QhfYwKKsEuB1tmy+SZ6IZuxYDgw= 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:16 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:QhfYwKKsEuB1tmy+SZ6IZuxYDgw=', 'Content-Length': '0'} 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8c6c26a3ef3cf12-00661ec0ac-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:16 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.280 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:IxfhkbeBvpbvkTLoMQnR5CHiq9Q= 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:IxfhkbeBvpbvkTLoMQnR5CHiq9Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079d16a625ffaed2b-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.281 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.282 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000792eb5b3ee7d6693-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.283 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9058d4db2ec3b76-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.284 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.285 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cca416191edd493-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.286 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d161c5038e2c6d9-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.287 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.288 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE= 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE=', 'Content-Length': '0'} 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009da4d00d681d79a5-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.289 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE= 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A/P/5TxIG8PLFzSatlOWUIWcMpE=', 'Content-Length': '0'} 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc9b8a889cddfbd9-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.290 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.291 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007848327eb6b75955-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.292 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98= 2024-04-16T19:28:54.293 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:Qd17Ixe2VftppNoytj94xwi3C98=', 'Content-Length': '0'} 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fdefc191176b0af-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.294 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012d96947b3b85a26-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.295 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.296 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036abfc28cfc09a3a-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.297 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0= 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0=', 'Content-Length': '0'} 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af4b4cb8b40957d3-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.298 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.299 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0= 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wWEzocLm1KPJIj0M/c5LZMPdaK0=', 'Content-Length': '0'} 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000976a5c5eb205b751-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.300 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4= 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4=', 'Content-Length': '0'} 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0b33ce54bfbcd47-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.301 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.302 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4= 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RjGjGBWukuW8jpHgoBYaDRyEVu4=', 'Content-Length': '0'} 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b0ccda0beeeec10-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.303 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c3272cb9755088c-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.304 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.305 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a91e3ff25d7a4e42-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.306 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.307 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000691bce239c8774d1-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.308 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b9137d94790dc7c-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.309 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.310 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c= 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c=', 'Content-Length': '0'} 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be03ad3eceba214e-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.311 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c= 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nlm644/ah3ZoSh7wONKsl0ofg5c=', 'Content-Length': '0'} 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bbb22c0c6e33a12-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.312 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.313 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bab675b3d6fbeebd-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.314 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU= 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:ZlQst+U/0FRbvx0WR2EmenEyoOU=', 'Content-Length': '0'} 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e49a464506915b91-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.315 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.316 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000040c642602d4aac9-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.317 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7ae7eddc35ebdb6-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.318 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.319 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc= 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc=', 'Content-Length': '0'} 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4974f16e1390c33-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.320 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc= 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SzEnw91g7O06rA2jQFBNGFZFYmc=', 'Content-Length': '0'} 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000957fae678d29c54e-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.321 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8= 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8=', 'Content-Length': '0'} 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cade9a280d3e6824-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.322 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.323 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8= 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:rZ+UI7G7PsO2LvwV2q0vcdzUHX8=', 'Content-Length': '0'} 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f585a00f598c00f5-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.324 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b808f2b546cbd2ed-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.325 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.326 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063fd38737b171ab1-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=e 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.327 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4296b5745e993c8-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.328 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.329 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f832eb25cd103bb5-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.330 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM= 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM=', 'Content-Length': '0'} 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000581f7cb195ec8fa-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.331 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.332 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM= 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:nLOPCqEzxHe0jVkFRSN1+yBoqcM=', 'Content-Length': '0'} 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b68e0852735367ca-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=e 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.333 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000441e1c83194c9355-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.334 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.335 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM= 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:EoT6T+ozBYl79Fv16UMft/GRpuM=', 'Content-Length': '0'} 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089959a83d3524d2d-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.336 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018e297a8a51bad07-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.337 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.338 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009da02acc863571d6-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.339 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM= 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM=', 'Content-Length': '0'} 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf183f5ec5735360-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.340 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM= 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7MMzE7nw8R+8CS3ybrC1wJTJQJM=', 'Content-Length': '0'} 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b23fff7046528f3-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=e 2024-04-16T19:28:54.341 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.342 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI= 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI=', 'Content-Length': '0'} 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002627e91df3d4671b-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.343 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI= 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:1p1gqgHWUBuEKYQB0hU5/b8pvlI=', 'Content-Length': '0'} 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b423da786b2b964c-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.344 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.345 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dac45b11b75ffa17-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.346 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a7c42fe1e33a3d3-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=f 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.347 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.348 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000647b14d90740c109-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.349 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fef2035d89b1ead1-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.350 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.351 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s= 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s=', 'Content-Length': '0'} 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038912aed22eeae79-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.352 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s= 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LX/8cbL3+RlfJrOSySpQy1nRW8s=', 'Content-Length': '0'} 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a12c70c254b58b2-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=f 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.353 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.354 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008262f880d5c7e7ed-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.355 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg= 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SRaANw/j5Y0mcq5M3IQnjOxdzzg=', 'Content-Length': '0'} 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020696a23cfc97d4d-00661ec0ad-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.356 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.357 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:17 GMT 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0= 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:17 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7fwWANinKnaWRj6T0W+I3p87HE0=', 'Content-Length': '0'} 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007db1a0703a73bb31-00661ec0ad-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:17 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.358 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0cfd0f7f40518d5-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.359 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.360 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU= 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU=', 'Content-Length': '0'} 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d4a18f582318937-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.361 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU= 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:2Hw83HNze++AYsKhHB7xm6P6ufU=', 'Content-Length': '0'} 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2396b9ee6ae3cf6-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=f 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.362 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw= 2024-04-16T19:28:54.363 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw=', 'Content-Length': '0'} 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4ceb83cac0b9159-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.364 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw= 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a3F2FWz2pkifWUb2Tbaly+mouGw=', 'Content-Length': '0'} 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021f96a1e7366acd0-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.365 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000749c8b43d085cd83-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.366 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.367 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b53210aeff2990b-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=g 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.368 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b4b802de66cf1b8-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.369 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.370 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe6a3540e3f77ee3-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.371 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg= 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg=', 'Content-Length': '0'} 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2df72848c33ac77-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.372 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.373 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg= 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:032OZSqifG3urf652xs54FpIuKg=', 'Content-Length': '0'} 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002bf4138a9a9084c-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=g 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.374 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055fc90d799218396-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.375 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.376 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA= 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:TdlATznk+0RLOZuwT9l9SblUmFA=', 'Content-Length': '0'} 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baa67b5fa5449665-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.377 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b30700ca55709edc-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.378 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.379 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001743687a37f33c6c-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.380 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM= 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM=', 'Content-Length': '0'} 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e6d3636ae51385a-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.381 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM= 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:mlusiM2a4svpNM2gJJwtd5ulyDM=', 'Content-Length': '0'} 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5a0c03b02d92dab-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=g 2024-04-16T19:28:54.382 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.383 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8= 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8=', 'Content-Length': '0'} 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000309cfab461ab8ad6-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.384 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8= 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:wNYNbvoTOf0RAXZ/GHGTYJih1U8=', 'Content-Length': '0'} 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090cbf17f378a2bc6-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.385 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006435e0da09a43c0c-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.386 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.387 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf1fe7ec92ea22be-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=h 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.388 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T19:28:54.389 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2b2f70ebc2be26b-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.390 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5d8333032b3653c-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.391 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q= 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca060d8c7eb22fd3-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.392 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.393 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q= 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s/B9lkLhLpjPhwtrmMku4/GRE8Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c03b698e783382b6-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=h 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.394 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T19:28:54.395 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1ae2e2cddfcd164-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.396 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8= 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:LfTM15paKP3HB+RwqezRLj+skN8=', 'Content-Length': '0'} 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ec3a85523b3e16d-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.397 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.398 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1a85348bd49d5a5-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.399 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed6e2ed003d1bdcd-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.400 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg= 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg=', 'Content-Length': '0'} 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004328659a15ac77ff-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.401 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.402 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg= 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:aKh45XvVeXmmJNL9Tx/H4T4KUrg=', 'Content-Length': '0'} 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000aeec3e0d7410ee7-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=h 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.403 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY= 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY=', 'Content-Length': '0'} 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081bf062ac161d869-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.404 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY= 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:kZf3l+N4t5UZrMx7bxIi4sKV+WY=', 'Content-Length': '0'} 2024-04-16T19:28:54.405 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000415fbbf2e2dab53f-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.406 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058a3676c3d29cc61-00661ec0ae-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.407 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:18 GMT 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.408 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI= 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:18 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bzn80HdG5UnmXSlZI4SG0F0jxqI=', 'Content-Length': '0'} 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066eb2b689732b784-00661ec0ae-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:18 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: INFO: success, bucket identical: bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'lybtug-50', '--num-shards', '15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket lybtug-50 --num-shards 15 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: tenant: 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: bucket name: lybtug-50 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: total entries: 8 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.409 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: {"layout":{"resharding":"None","current_index":{"gen":2,"layout":{"type":"Normal","normal":{"num_shards":15,"hash_type":"Mod"}}},"logs":[{"gen":0,"layout":{"type":"InIndex","in_index":{"gen":0,"layout":{"num_shards":11,"hash_type":"Mod"}}}},{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}},{"gen":2,"layout":{"type":"InIndex","in_index":{"gen":2,"layout":{"num_shards":15,"hash_type":"Mod"}}}}]}} 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-50 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:19.221+0000 7fde8b1ef980 1 waiting to reach latest gen 2, on 1.. 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:24.229+0000 7fde8b1ef980 1 bucket sync caught up with empty source 2024-04-16T19:28:54.410 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:17:24.229+0000 7fde8b1ef980 0 bucket checkpoint complete 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone2 target=test-zone1 bucket=lybtug-50 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-50', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket lybtug-50 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:17:24.405+0000 7f72223d0980 1 bucket sync caught up with source: 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: local status: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: remote markers: [, 00000000002.905.6, , , 00000000001.1015.6, 00000000001.1167.6, , , 00000000002.310.6, , 00000000002.413.6] 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: 2024-04-16T18:17:24.405+0000 7f72223d0980 0 bucket checkpoint complete 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: INFO: comparing bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.411 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM= 2024-04-16T19:28:54.412 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM=', 'Content-Length': '0'} 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0439cf64ce0e8de-00661ec0b4-5397-test-zone2'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/ 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.413 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM= 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:bP37J+7X3QMTqm+cU9RPurHu/bM=', 'Content-Length': '0'} 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001b2541f1698401cd-00661ec0b4-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: bucket1 objects: 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.414 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db991537d01f3bed-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.415 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=a 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=b 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=c 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=d 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=e 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=f 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=g 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=h 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: bucket2 objects: 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.416 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T19:28:54.417 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c73e0eefd9983c96-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=a 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=b 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=c 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=d 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=e 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=f 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=g 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: o=h 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.418 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.419 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055ee618352cfd940-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?versions&max-keys=999 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.420 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:/lybtug-50/?versions 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8= 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:y35QKbt1uBw70CaTD9n0kPFNMz8=', 'Content-Length': '0'} 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe2c690732990985-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50999falseanulltrue2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalbnulltrue2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalcnulltrue2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normaldnulltrue2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalenulltrue2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalfnulltrue2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalgnulltrue2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normalhnulltrue2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.421 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.422 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000624f87421d358c29-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.423 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5fb2b8ec1e7de41-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.424 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.425 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ= 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fa4af6a4b5e85fb-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.426 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ= 2024-04-16T19:28:54.427 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b2AaZEAAv26MuGzp752SjMt67JQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2d0f1719bf6aa82-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.428 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5998a2f701bfde0-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a?versionId=null 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a?versionId=null 2024-04-16T19:28:54.429 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:/lybtug-50/a?versionId=null 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.430 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE= 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HNAVOS6ulFOV7OVoBsnk3p8K6ZE=', 'Content-Length': '0'} 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd8282fc10c4e8a9-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.431 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073cd4418032fb5f6-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.432 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.433 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8abead73ae0a159-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.434 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU= 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU=', 'Content-Length': '0'} 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f32eb342b13ea44c-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.435 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU= 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:8ooRT+joKudWxjbZW+w1IRCxkjU=', 'Content-Length': '0'} 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004733b289894f47e0-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=a 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.436 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.437 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU= 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU=', 'Content-Length': '0'} 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7a0cdb71dc38990-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/a 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/a 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/a 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.438 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:/lybtug-50/a 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU= 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:HUc//sQ4S07mraJpKki+jXYT5XU=', 'Content-Length': '0'} 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006eb8018829fd954-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.439 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c1df529351ee20d-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.440 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=a 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=a 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.441 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000697fa6f751585cda-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50a1000falsea2024-04-16T18:15:58.232Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.442 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T19:28:54.443 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efa4836074cc2357-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.444 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083294fb68dbdc055-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.445 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M= 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M=', 'Content-Length': '0'} 2024-04-16T19:28:54.446 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e823fa3f03785376-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.447 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M= 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:T0BwDF+xVQP6KSDtoy6OhTFK7+M=', 'Content-Length': '0'} 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005226a851074f7546-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.448 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2f1120c55ef05f9-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.449 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b?versionId=null 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b?versionId=null 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.450 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:/lybtug-50/b?versionId=null 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY= 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:v0dSg9XEBD9YGgfyhX7CV1mT2uY=', 'Content-Length': '0'} 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3f046c4f7ae09b4-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.451 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b7694677e39306b-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.452 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.453 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d12053607e05e264-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.454 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto= 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto=', 'Content-Length': '0'} 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a30c9f0d970c96d6-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.455 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto= 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:WxGosEbtAUdT2Hvkm1UOG5Tymto=', 'Content-Length': '0'} 2024-04-16T19:28:54.456 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007786b506fb94abde-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=b 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.457 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ= 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b8d76cb1ab0404d-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/b 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/b 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/b 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.458 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:/lybtug-50/b 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ= 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OE+oXduR2Sw+BRz9+ZZCBzf45UQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000acbf5d77dc2a384-00661ec0b4-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.459 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:24 GMT 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.460 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A= 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:24 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:k2/9DpwIIaMXXndQWJST8YYQj6A=', 'Content-Length': '0'} 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068acff4f48a8ed72-00661ec0b4-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:24 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=b 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=b 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.461 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9d378196d988fab-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50b1000falseb2024-04-16T18:15:58.290Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.462 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.463 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f271b74245aff31-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.464 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4851165a660adc8-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.465 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.466 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE= 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE=', 'Content-Length': '0'} 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fc56d2515ebcd85-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.467 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE= 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:9uqNhvgQ3yJ5JeovcyN/MjV0VOE=', 'Content-Length': '0'} 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000057a86c0c85fbc69-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.468 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T19:28:54.469 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f31e2142b1d5014-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c?versionId=null 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c?versionId=null 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.470 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:/lybtug-50/c?versionId=null 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64= 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:s+03AEN/b/T22pGLQP5Cd8TqQ64=', 'Content-Length': '0'} 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d0e2f69085cbe64-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.471 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ca3c7da969f4285-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.472 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.473 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0f26b4892fcd554-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.474 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8= 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8=', 'Content-Length': '0'} 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa876ee9b8eba0c1-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.475 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8= 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J+mXNrwIHRK0nmU3rudcb8TqKN8=', 'Content-Length': '0'} 2024-04-16T19:28:54.476 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008233ac211406f5a2-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=c 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.477 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo= 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo=', 'Content-Length': '0'} 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000304c37ae768718c7-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/c 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/c 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/c 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.478 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:/lybtug-50/c 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo= 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:meXZ0gZX+Ijx0ZyaNGSelWzzIyo=', 'Content-Length': '0'} 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077f43b9e271d1a9a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.479 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.480 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be2056ea81d5ef7e-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=c 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=c 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.481 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000979a62a2531c1185-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50c1000falsec2024-04-16T18:15:58.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.482 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.483 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069c3ee23558073d8-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.484 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a44123dad02fe254-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.485 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0= 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0=', 'Content-Length': '0'} 2024-04-16T19:28:54.486 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb8bdd2ec89fca26-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.487 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0= 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:m4Qs6ox/Z3s6LkUq2jAnIZMUwg0=', 'Content-Length': '0'} 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095e35904aa47afbc-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.488 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040ee4b293558fe0c-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.489 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d?versionId=null 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d?versionId=null 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.490 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:/lybtug-50/d?versionId=null 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw= 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:xm0AJl57kULZ+6hFZ+dg67Thlqw=', 'Content-Length': '0'} 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004570202c72e2252-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.491 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bbc1a4ae5d4aad6-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.492 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.493 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083e1de40b40792f6-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.494 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A= 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A=', 'Content-Length': '0'} 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009523773fc2489d05-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.495 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A= 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:g1/mD5X03ANECJhmmz5Us/1611A=', 'Content-Length': '0'} 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030756fa89a163535-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=d 2024-04-16T19:28:54.496 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.497 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI= 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI=', 'Content-Length': '0'} 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:16:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026315591c90cdb07-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/d 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/d 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/d 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.498 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:/lybtug-50/d 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI= 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:hbZpCHUODKZD6M9avzz7Z9DLtYI=', 'Content-Length': '0'} 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:15:58 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1b898e2359a8a66-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.499 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075b2e603916c087b-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.500 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=d 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=d 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.501 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f3a6df08d27935a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50d1000falsed2024-04-16T18:15:58.362Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=e 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.502 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b787651555ea85b-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.503 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.504 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba99448b5e63ff6a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.505 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I= 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I=', 'Content-Length': '0'} 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fde0f8dd82651f13-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.506 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.507 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I= 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6NNL2Xydg2L0UWm/j9fHTzoUL5I=', 'Content-Length': '0'} 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab3a0c37330a69b8-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=e 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.508 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b57bec5ce1e6ce-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e?versionId=null 2024-04-16T19:28:54.509 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e?versionId=null 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:/lybtug-50/e?versionId=null 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo= 2024-04-16T19:28:54.510 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:CrkHB0BrYeP/BFyGki2RaqgB9Xo=', 'Content-Length': '0'} 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b38fb2116fab32ee-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.511 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d118acbc11996f8-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.512 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014fa6998b46c2865-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.513 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.514 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg= 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg=', 'Content-Length': '0'} 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072fff4d3b846ff7c-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.515 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg= 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:RLxynzH/EN6kuifhQkgiWbhR4Vg=', 'Content-Length': '0'} 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000654242232d00b739-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=e 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.516 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk= 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk=', 'Content-Length': '0'} 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fc49cf56b28ba5a-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.517 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/e 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/e 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/e 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.518 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:/lybtug-50/e 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk= 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:XjZNcyVEI63pJsEnktfr0KE/stk=', 'Content-Length': '0'} 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab6c17a25d59b97f-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.519 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000008398197cdf0466-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.520 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=e 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=e 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.521 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001875c40ad55ca6eb-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50e1000falsee2024-04-16T18:16:16.054Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=f 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.522 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb01df5223d4e46c-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.523 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.524 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080cd6a32d8bad9c1-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.525 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8= 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8=', 'Content-Length': '0'} 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc14391bd71537ea-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.526 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.527 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8= 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:a9+B8rI6fYsFkGY6aB6gw4LLzK8=', 'Content-Length': '0'} 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030f1738433e8a5a7-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=f 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.528 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026680b0457b6e5d8-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f?versionId=null 2024-04-16T19:28:54.529 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f?versionId=null 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:/lybtug-50/f?versionId=null 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.530 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ= 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:SLpTb8LNVKMKc5fu3nzH/urf7QQ=', 'Content-Length': '0'} 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021eb30cb27f4a1c4-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.531 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b33c61bdb57b3ff6-00661ec0b5-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.532 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:25 GMT 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY= 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:25 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:r+rlRGyXz7CfzfgqCXD+9+63mZY=', 'Content-Length': '0'} 2024-04-16T19:28:54.533 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2dec0afd100d99a-00661ec0b5-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:25 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.534 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE= 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE=', 'Content-Length': '0'} 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a0904fa64285ebc-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.535 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE= 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KxMb/NzpW1bo1+OI7SV48wWq7TE=', 'Content-Length': '0'} 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000874bfe9c7f44050f-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=f 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.536 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U= 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U=', 'Content-Length': '0'} 2024-04-16T19:28:54.537 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:02 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd4b2457cb54ec81-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/f 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/f 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/f 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.538 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:/lybtug-50/f 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U= 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:A1giFXwifWsR8Ixjaj9EjjqQt6U=', 'Content-Length': '0'} 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000523c42661b66be6e-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.539 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1ac1cd86e85e384-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.540 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=f 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=f 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.541 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f09985119b7c347c-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50f1000falsef2024-04-16T18:16:16.068Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=g 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.542 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d127041d6c8a5bc-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.543 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.544 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1e02e17e26702b4-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.545 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk= 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk=', 'Content-Length': '0'} 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f5034db783df87e-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.546 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.547 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk= 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:DSMyTFByqsXVao4B3KlDydrsLhk=', 'Content-Length': '0'} 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a54c2b74c243e3ea-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=g 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.548 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b35cffa06fbbbe2-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g?versionId=null 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g?versionId=null 2024-04-16T19:28:54.549 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:/lybtug-50/g?versionId=null 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso= 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:OhW0sTkOkMhqx//s2KGm14x0sso=', 'Content-Length': '0'} 2024-04-16T19:28:54.550 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a4b78bc389bc5cc-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.551 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037f8ea3b0dd0a2c3-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.552 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057f8d4cd42d40cbc-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.553 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.554 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M= 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M=', 'Content-Length': '0'} 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000252ef69447c1ac37-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.555 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M= 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:jVlHkWH/OHrcfdxHPdXZV+r4d3M=', 'Content-Length': '0'} 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fae2ddab1d06a98e-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=g 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.556 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw= 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw=', 'Content-Length': '0'} 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d1b7b645a1874d5-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/g 2024-04-16T19:28:54.557 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/g 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/g 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:/lybtug-50/g 2024-04-16T19:28:54.558 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw= 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:eLG7Zrn0zIhsLtjYWFXYS8Uf5rw=', 'Content-Length': '0'} 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001416c27874183696-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.559 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d33986b67e93bd67-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=g 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.560 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=g 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.561 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b94ff15a799fdeee-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50g1000falseg2024-04-16T18:16:16.115Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=h 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.562 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033f59013adb658ba-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.563 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e39e41e4c77ab529-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.564 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.565 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E= 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E=', 'Content-Length': '0'} 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000505518a94439367e-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.566 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E= 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:URUJUsGPERDlWA2vr5zBMHtla3E=', 'Content-Length': '0'} 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1ffb9aecff403f3-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=h 2024-04-16T19:28:54.567 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.568 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d31c4b08809de2c-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h?versionId=null 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h?versionId=null 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.569 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:/lybtug-50/h?versionId=null 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q= 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:VDqXsitlr2vHEAT2J3Tvd81pi6Q=', 'Content-Length': '0'} 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca677ea7f7bdf4aa-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.570 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.571 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045be7cfea307ab33-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.572 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac4ff7ed3afeaf71-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.573 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.574 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk= 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk=', 'Content-Length': '0'} 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063cf2fce5f5bea2a-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.575 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk= 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:KbqslJKJTv7Rl2zAade0PL7dEgk=', 'Content-Length': '0'} 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abb3849dfec3ecfb-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: DEBUG: comparing key name=h 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.576 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8= 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8=', 'Content-Length': '0'} 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('x-amz-replication-status', 'REPLICA'), ('x-rgw-replicated-from', '054102a3-8d94-4596-8b3b-6341eab312e1:lybtug-50:054102a3-8d94-4596-8b3b-6341eab312e1.4804.19'), ('x-rgw-replicated-at', 'Tue, 16 Apr 2024 18:17:07 GMT'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b64a7540cba10ae-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/h 2024-04-16T19:28:54.577 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/h 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/h 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:/lybtug-50/h 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.578 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8= 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:fIcTEGX2fLNvW/AOgguIIoujyD8=', 'Content-Length': '0'} 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Tue, 16 Apr 2024 18:16:16 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005394276f7902d10e-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.579 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d31771ca0404ab67-00661ec0b6-5397-test-zone2'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/ 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/ 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-50/?prefix=h 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: GET 2024-04-16T19:28:54.580 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-50/?prefix=h 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:GET 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:17:26 GMT 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:/lybtug-50/ 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0= 2024-04-16T19:28:54.581 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:17:26 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:5hAtSrrkbNPSfbNnN2pkEGYlfZ0=', 'Content-Length': '0'} 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084d5a1d7b7946296-00661ec0b6-4804-test-zone1'), ('Content-Type', 'application/xml'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:17:26 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:boto: DEBUG: b'lybtug-50h1000falseh2024-04-16T18:16:16.132Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDRGW11111111111111111Normal' 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.zone_rados: INFO: success, bucket identical: bucket=lybtug-50 zones={test-zone2, test-zone1} 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.053+0000 7fbdc43fc640 -1 req 18081045207925834297 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-43:054102a3-8d94-4596-8b3b-6341eab312e1.4804.12,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-43:054102a3-8d94-4596-8b3b-6341eab312e1.4804.12,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.065+0000 7fbdccc0d640 -1 req 14660751659862165158 0.003999925s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-25:054102a3-8d94-4596-8b3b-6341eab312e1.4353.29,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-25:054102a3-8d94-4596-8b3b-6341eab312e1.4353.29,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.149+0000 7fbdeb44a640 -1 req 26479994857397418 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-34:054102a3-8d94-4596-8b3b-6341eab312e1.4804.3,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-34:054102a3-8d94-4596-8b3b-6341eab312e1.4804.3,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.149+0000 7fbde1c37640 -1 req 4080226723044830306 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-24:054102a3-8d94-4596-8b3b-6341eab312e1.4353.28,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-24:054102a3-8d94-4596-8b3b-6341eab312e1.4353.28,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.149+0000 7fbdf6460640 -1 req 7259517271128369061 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-1:054102a3-8d94-4596-8b3b-6341eab312e1.4353.1,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.173+0000 7fbe07c83640 -1 req 13089633187912110970 0.007999849s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-21:054102a3-8d94-4596-8b3b-6341eab312e1.4353.25,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-21:054102a3-8d94-4596-8b3b-6341eab312e1.4353.25,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.582 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.201+0000 7fbe0e490640 -1 req 1862940798833270506 0.007999849s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-33:054102a3-8d94-4596-8b3b-6341eab312e1.4804.2,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-33:054102a3-8d94-4596-8b3b-6341eab312e1.4804.2,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: {"layout":{"resharding":"None","current_index":{"gen":2,"layout":{"type":"Normal","normal":{"num_shards":15,"hash_type":"Mod"}}},"logs":[{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}},{"gen":2,"layout":{"type":"InIndex","in_index":{"gen":2,"layout":{"num_shards":15,"hash_type":"Mod"}}}}]}} 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.857+0000 7fbe22cb9640 -1 req 5988557027030468213 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-4:054102a3-8d94-4596-8b3b-6341eab312e1.4353.4,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.861+0000 7fbe454fe640 -1 req 6899803751102318845 0.007999849s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-26:054102a3-8d94-4596-8b3b-6341eab312e1.4353.30,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-26:054102a3-8d94-4596-8b3b-6341eab312e1.4353.30,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.873+0000 7fbe68544640 -1 req 572665038004721884 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-32:054102a3-8d94-4596-8b3b-6341eab312e1.4804.1,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-32:054102a3-8d94-4596-8b3b-6341eab312e1.4804.1,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.583 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.925+0000 7fbd7e370640 -1 req 15925822944464962007 0.003999924s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-35:054102a3-8d94-4596-8b3b-6341eab312e1.4804.4,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-35:054102a3-8d94-4596-8b3b-6341eab312e1.4804.4,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:17:27.985+0000 7fbd8337a640 -1 req 14826523632275738640 0.003999925s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=lybtug-23:054102a3-8d94-4596-8b3b-6341eab312e1.4353.27,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=lybtug-23:054102a3-8d94-4596-8b3b-6341eab312e1.4353.27,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} returned ret=-2 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'lybtug-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket lybtug-50 --cluster c1 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: {"layout":{"resharding":"None","current_index":{"gen":2,"layout":{"type":"Normal","normal":{"num_shards":15,"hash_type":"Mod"}}},"logs":[{"gen":1,"layout":{"type":"InIndex","in_index":{"gen":1,"layout":{"num_shards":13,"hash_type":"Mod"}}}},{"gen":2,"layout":{"type":"InIndex","in_index":{"gen":2,"layout":{"num_shards":15,"hash_type":"Mod"}}}}]}} 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.584 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_bucket_full_sync_after_data_sync_init 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2160, in test_bucket_full_sync_after_data_sync_init 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: zonegroup_bucket_checkpoint(zonegroup_conns, bucket.name) 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 398, in zonegroup_bucket_checkpoint 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: zone_bucket_checkpoint(target_conn.zone, source_conn.zone, bucket_name) 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 390, in zone_bucket_checkpoint 2024-04-16T19:28:54.585 INFO:tasks.rgw_multisite_tests: target_zone.cluster.admin(cmd, debug_rgw=1) 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw_multisite.py", line 197, in admin 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests: assert r == 0 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/ 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/ 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/ 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.586 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:30:21 GMT 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:/lybtug-53/ 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:6foZE5avGfrCC32f0KyliHdyaac= 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:30:21 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:6foZE5avGfrCC32f0KyliHdyaac=', 'Content-Length': '0'} 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx000005fa82e70366266bc-00661ec3bd-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:30:21 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: created bucket=lybtug-53 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.587 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713292221.776788_782.1","last_update":"2024-04-16T18:30:21.776788Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:22.047+0000 7fd302a1e980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:22.051+0000 7fd302a1e980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:22Z"} 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:22Z"} 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.588 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:27.199+0000 7f664ac60980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:27.203+0000 7f664ac60980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:27Z"} 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:27Z"} 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.589 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:32.351+0000 7f98ceb0d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:32.355+0000 7f98ceb0d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:32Z"} 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:32Z"} 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.590 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:37.507+0000 7fb5509fc980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:37.507+0000 7fb5509fc980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:37Z"} 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:37Z"} 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.591 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:42.658+0000 7f4311f79980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:42.658+0000 7f4311f79980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:42Z"} 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:42Z"} 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:47.810+0000 7f009966d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:47.814+0000 7f009966d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.592 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:47Z"} 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:47Z"} 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:52.962+0000 7f7ab37bd980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:52.962+0000 7f7ab37bd980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:52Z"} 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:52Z"} 2024-04-16T19:28:54.593 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:58.110+0000 7fb60c360980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:30:58.114+0000 7fb60c360980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:58Z"} 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:30:58Z"} 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.594 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:03.262+0000 7f169c760980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:03.262+0000 7f169c760980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:03Z"} 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:03Z"} 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.595 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:08.414+0000 7f0f4f1d0980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:08.418+0000 7f0f4f1d0980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:08Z"} 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:08Z"} 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.596 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:13.570+0000 7fc0600c5980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:13.570+0000 7fc0600c5980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:13Z"} 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:13Z"} 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:18.726+0000 7fba24108980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.597 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:18.730+0000 7fba24108980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:18Z"} 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:18Z"} 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:23.881+0000 7f55c1fd3980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:23.881+0000 7f55c1fd3980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:23Z"} 2024-04-16T19:28:54.598 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:23Z"} 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:29.033+0000 7f8f573b0980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:29.033+0000 7f8f573b0980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:29Z"} 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:29Z"} 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.599 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:34.185+0000 7f51f0a81980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:34.189+0000 7f51f0a81980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:34Z"} 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:34Z"} 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.600 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:39.345+0000 7f269caf1980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:39.349+0000 7f269caf1980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:39Z"} 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:39Z"} 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.601 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:44.501+0000 7f97db4d9980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:44.501+0000 7f97db4d9980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:44Z"} 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:44Z"} 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.602 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:49.657+0000 7fe47ef6d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:49.661+0000 7fe47ef6d980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:49Z"} 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:49Z"} 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:54.813+0000 7fb19fcee980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.603 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:54.813+0000 7fb19fcee980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:54Z"} 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292073.136324_727.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:27:53.136324Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:54Z"} 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292073.136324_727.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:27:53.136324Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292073.136324_727.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292073.136324_727.1 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:59.968+0000 7f0153b35980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:31:59.968+0000 7f0153b35980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.756655_779.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.756655Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:59Z"} 2024-04-16T19:28:54.604 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.756655_779.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.756655Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:31:59Z"} 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.756655_779.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.756655Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.756655_779.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292221.776788_782.1 target=1_1713292221.756655_779.1 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:05.120+0000 7fa3bdade980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:05.124+0000 7fa3bdade980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:32:05Z"} 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:32:05Z"} 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.605 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: finish meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0: DEBUG: waiting for process to exit 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: INFO: waiting for 300 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:32:05.204+0000 7f2049a56640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 172624) UID: 0 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:32:05.204+0000 7f204d29ea40 -1 shutting down 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0: INFO: Stopped 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/ 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/ 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/ 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.606 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:/lybtug-53/ 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000357c52ce1450970-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/obj11 2024-04-16T19:28:54.607 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/obj11 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/obj11 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:/lybtug-53/obj11 2024-04-16T19:28:54.608 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:o4cV6vQ+EimVnLYacefuo9kDQMk= 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:o4cV6vQ+EimVnLYacefuo9kDQMk='} 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000556044d3f10dbf73-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/ 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/ 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/ 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.609 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:/lybtug-53/ 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009c755159c28cfe0b-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/obj22 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/obj22 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/obj22 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.610 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:/lybtug-53/obj22 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:JRUE6BT4lE86I8Jt1/XmAILTJFY= 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:JRUE6BT4lE86I8Jt1/XmAILTJFY='} 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000029059ff0bc78fa64-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/ 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/ 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.611 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/ 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:/lybtug-53/ 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T19:28:54.612 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0dd1f6b07023ca8-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/obj33 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/obj33 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/obj33 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.613 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:/lybtug-53/obj33 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:7lvHXQrcShBVnuNlCKh4RGu2uuQ= 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:7lvHXQrcShBVnuNlCKh4RGu2uuQ='} 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007598d797b482dace-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/ 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/ 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/ 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.614 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:/lybtug-53/ 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000005830cdda77757d4-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/obj44 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/obj44 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/obj44 2024-04-16T19:28:54.615 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:/lybtug-53/obj44 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:4XgQWi9X9M9a4yboUrA40TNCzQY= 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:4XgQWi9X9M9a4yboUrA40TNCzQY='} 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d5d94926014102d3-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.616 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/ 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/ 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: HEAD 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/ 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:HEAD 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:/lybtug-53/ 2024-04-16T19:28:54.617 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc= 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:38ac2PiNlOgx7EH1cGv1PFM/LWc=', 'Content-Length': '0'} 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004c4f195b88d87bd1-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-53/obj55 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-53/obj55 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-53/obj55 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.618 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:rL0Y20zC+Fzt72VPzMSk2A== 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:application/octet-stream 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:32:11 GMT 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:/lybtug-53/obj55 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:yzkELej/F6miexxYRuT8z8Fu1Jg= 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Tue, 16 Apr 2024 18:32:11 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yzkELej/F6miexxYRuT8z8Fu1Jg='} 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009dfcd4808bd5dde3-00661ec42b-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Date', 'Tue, 16 Apr 2024 18:32:11 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['data', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync init --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:11.548+0000 7f5f14290980 5 RGW-SYNC:data: waiting on data sync status lease 2024-04-16T19:28:54.619 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:11.552+0000 7f5f14290980 5 RGW-SYNC:data: acquired data sync status lease 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0: INFO: Restarting daemon 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0: INFO: Started 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: Dload Upload Total Spent Left Speed 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 7 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: curl: (7) Failed to connect to smithi122.front.sepia.ceph.com port 8000 after 0 ms: Connection refused 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 has been restored 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:32:11.848+0000 7f30fcfe9a40 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > curl http://smithi122.front.sepia.ceph.com:8000 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: Dload Upload Total Spent Left Speed 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 98008 0 --:--:-- --:--:-- --:--:-- 182k 2024-04-16T19:28:54.620 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: anonymous 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: bucket checkpoint: source=test-zone1 target=test-zone2 bucket=lybtug-53 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-53', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'lybtug-53', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket lybtug-53 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:12.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:32:14.336+0000 7f2fe5579640 -1 rgw async rados processor: ERROR: _send_request(): get_sync_policy_handler() returned -2 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:17.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:22.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:27.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:32.804+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.621 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:37.808+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:42.807+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:47.807+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:52.807+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:32:57.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:02.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:07.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:12.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:17.811+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:22.814+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:27.814+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:32.814+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:37.818+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:42.818+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.622 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: *** Caught signal (Segmentation fault) ** 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T19:28:54.623 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T18:33:45.738+0000 7f2fd8d60640 -1 *** Caught signal (Segmentation fault) ** 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T19:28:54.624 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 0> 2024-04-16T18:33:45.738+0000 7f2fd8d60640 -1 *** Caught signal (Segmentation fault) ** 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T19:28:54.625 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 0> 2024-04-16T18:33:45.738+0000 7f2fd8d60640 -1 *** Caught signal (Segmentation fault) ** 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: in thread 7f2fd8d60640 thread_name:data-sync 2024-04-16T19:28:54.626 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: ceph version 19.0.0-3089-gb0c7b917 (b0c7b917267c471208541fdeb652f5e897d00086) squid (dev) 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f30ff788520] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2: radosgw(+0x118e619) [0x557e0523d619] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 3: radosgw(+0x504aa2) [0x557e045b3aa2] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 4: radosgw(+0x9b6869) [0x557e04a65869] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 5: radosgw(+0xb8c8e7) [0x557e04c3b8e7] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x98) [0x557e047c3fd8] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x35b) [0x557e047d404b] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 8: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0xad) [0x557e047dbf5d] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 9: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x4a2) [0x557e04c4d9f2] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 10: radosgw(+0x8fd9a8) [0x557e049ac9a8] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 11: (RGWRadosThread::Worker::entry()+0xb3) [0x557e0499cfe3] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 12: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f30ff7dab43] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f30ff86ca00] 2024-04-16T19:28:54.627 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stdout: INFO: 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi122.stderr: INFO: daemon-helper: command crashed with signal 11 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~0s 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:47.818+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:52.822+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~6s 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:33:57.822+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~12s 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:02.822+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~18s 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:07.821+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~24s 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:12.825+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~30s 2024-04-16T19:28:54.628 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:17.825+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:22.829+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~35s 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:27.829+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~41s 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:32.833+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~47s 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:37.833+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~53s 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:42.833+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~59s 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:47.837+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~65s 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:52.836+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:34:57.836+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.629 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~71s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:02.840+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~77s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:07.840+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~83s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:12.840+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~89s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:17.844+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~95s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:22.844+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:27.844+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~100s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:32.848+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~106s 2024-04-16T19:28:54.630 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:37.847+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~112s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:42.847+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~118s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:47.851+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~124s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:52.851+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~130s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:35:57.851+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:02.855+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~136s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:07.855+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~142s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:12.855+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~148s 2024-04-16T19:28:54.631 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:17.859+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~154s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:22.858+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~160s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:27.858+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:32.858+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~165s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:37.862+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~171s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:42.862+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~177s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:47.866+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~183s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:52.866+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~189s 2024-04-16T19:28:54.632 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:36:57.866+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~195s 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:02.870+0000 7f2217e66980 1 waiting to reach incremental sync.. 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:07.869+0000 7f2217e66980 -1 bucket checkpoint timed out waiting to reach incremental sync 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:07.869+0000 7f2217e66980 0 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:07.869+0000 7f2217e66980 -1 bucket sync checkpoint failed: (110) Connection timed out 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 110 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_resharded_bucket_full_sync_after_data_sync_init 2024-04-16T19:28:54.633 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2174, in test_resharded_bucket_full_sync_after_data_sync_init 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.634 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-54/ 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-54/ 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-54/ 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.635 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:37:08 GMT 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:/lybtug-54/ 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:yLMV8/o+zR5KVj1vqZ2aQaig4Ag= 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:37:08 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yLMV8/o+zR5KVj1vqZ2aQaig4Ag=', 'Content-Length': '0'} 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx000003b537801dc8a76ba-00661ec554-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:37:08 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: created bucket=lybtug-54 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.636 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713292628.204217_1458.1","last_update":"2024-04-16T18:37:08.204217Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:08.461+0000 7f88411e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:08.465+0000 7f88411e1980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:08Z"} 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:08Z"} 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.637 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~201s 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:13.617+0000 7f8f72442980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:13.617+0000 7f8f72442980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:13Z"} 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:13Z"} 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.638 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~207s 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:18.769+0000 7f12cac5e980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:18.773+0000 7f12cac5e980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:18Z"} 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:18Z"} 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.639 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~213s 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:23.925+0000 7f3eededb980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:23.929+0000 7f3eededb980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:23Z"} 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:23Z"} 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.640 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~219s 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:29.081+0000 7f27114ba980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:29.081+0000 7f27114ba980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:29Z"} 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:29Z"} 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.641 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~225s 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:34.241+0000 7f824035e980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:34.241+0000 7f824035e980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:34Z"} 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:34Z"} 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.642 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~231s 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:39.397+0000 7fe399049980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:39.397+0000 7fe399049980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:39Z"} 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:39Z"} 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~237s 2024-04-16T19:28:54.643 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:44.549+0000 7f440f7dd980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:44.553+0000 7f440f7dd980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:44Z"} 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:44Z"} 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.644 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:49.704+0000 7f94d9adc980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:49.704+0000 7f94d9adc980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:49Z"} 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:49Z"} 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~242s 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.645 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:54.860+0000 7f1cb5885980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:37:54.864+0000 7f1cb5885980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:54Z"} 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:37:54Z"} 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~248s 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.646 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:00.008+0000 7f4491fe4980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:00.008+0000 7f4491fe4980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:00Z"} 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:00Z"} 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~254s 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.647 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:05.164+0000 7f14e9ad6980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:05.164+0000 7f14e9ad6980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:05Z"} 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:05Z"} 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~260s 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.648 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:10.316+0000 7f87f6831980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:10.320+0000 7f87f6831980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:10Z"} 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:10Z"} 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~266s 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.649 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:15.472+0000 7f8242973980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:15.472+0000 7f8242973980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:15Z"} 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:15Z"} 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~272s 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.650 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:20.624+0000 7f3ef41f3980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:20.628+0000 7f3ef41f3980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:20Z"} 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:20Z"} 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~278s 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.651 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:25.780+0000 7fefb5766980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:25.784+0000 7fefb5766980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:25Z"} 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:25Z"} 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:30.932+0000 7f9a3d29b980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:30.936+0000 7f9a3d29b980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.652 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:30Z"} 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:30Z"} 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~284s 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:36.087+0000 7f81be26a980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:36.091+0000 7f81be26a980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:36Z"} 2024-04-16T19:28:54.653 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:36Z"} 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~290s 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:41.243+0000 7f64db882980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:41.247+0000 7f64db882980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:41Z"} 2024-04-16T19:28:54.654 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:41Z"} 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~296s 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:46.399+0000 7fd3ba063980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:38:46.403+0000 7fd3ba063980 20 RGW-SYNC:meta: read sync status 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stdout: INFO: {"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:46Z"} 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: current meta sync status={"sync_status":{"info":{"status":"sync","num_shards":4,"period":"8c713f9b-78f8-4e05-979f-375a77da9531","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1713292221.776788_782.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2024-04-16T18:30:21.776788Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1713291352.392860_404.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:52.392860Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1713291351.746265_283.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:51.746265Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1713291335.360921_269.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2024-04-16T18:15:35.360921Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1},"current_time":"2024-04-16T18:38:46Z"} 2024-04-16T19:28:54.655 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1713292221.776788_782.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2024-04-16T18:30:21.776788Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1713291352.392860_404.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:52.392860Z', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1713291351.746265_283.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:51.746265Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1713291335.360921_269.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2024-04-16T18:15:35.360921Z', 'realm_epoch': 2}}] 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: log_status={0: '1_1713292628.204217_1458.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: sync_status={0: '1_1713292221.776788_782.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: WARNING: zone test-zone2 behind master: shard=0 master=1_1713292628.204217_1458.1 target=1_1713292221.776788_782.1 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: daemon c2.rgw.client.0 is failed for ~302s 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.daemonwatchdog.daemon_watchdog: INFO: BARK! unmounting mounts and killing all daemons 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.0: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.1: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.2: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.3: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.4: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.5: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.mon.a: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.1: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.mgr.x: INFO: Sent signal 15 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.3.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7fc2f06bc640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 3 (PID: 22806) UID: 0 2024-04-16T19:28:54.656 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.3.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7fc2f06bc640 -1 osd.3 55 *** Got signal Terminated *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.3.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7fc2f06bc640 -1 osd.3 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.4.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f7274e81640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 4 (PID: 22808) UID: 0 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.4.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f7274e81640 -1 osd.4 55 *** Got signal Terminated *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.4.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f7274e81640 -1 osd.4 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.5.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f7b1f6d9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 5 (PID: 22810) UID: 0 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.5.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f7b1f6d9640 -1 osd.5 55 *** Got signal Terminated *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.5.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f7b1f6d9640 -1 osd.5 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.1.smithi188.stdout: INFO: 2024-04-16T18:38:50.111+0000 7f45dda82640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground (PID: 27407) UID: 0 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.1.smithi188.stdout: INFO: 2024-04-16T18:38:50.111+0000 7f45e12caa40 -1 shutting down 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.mgr.x.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f8a58d0d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mgr -f --cluster c2 -i x (PID: 22626) UID: 0 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.mgr.x.smithi188.stderr: INFO: 2024-04-16T18:38:50.111+0000 7f8a58d0d640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.0.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f0fe888e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 0 (PID: 22818) UID: 0 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.0.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f0fe888e640 -1 osd.0 55 *** Got signal Terminated *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.0.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f0fe888e640 -1 osd.0 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T19:28:54.657 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.1.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7ff34e1ce640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 1 (PID: 22820) UID: 0 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.1.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7ff34e1ce640 -1 osd.1 55 *** Got signal Terminated *** 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.1.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7ff34e1ce640 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.2.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f21bd318640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster c2 -i 2 (PID: 22823) UID: 0 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.2.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f21bd318640 -1 osd.2 55 *** Got signal Terminated *** 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.osd.2.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f21bd318640 -1 osd.2 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.mon.a.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f1c8184e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster c2 -i a (PID: 22745) UID: 0 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.ceph.c2.mon.a.smithi122.stderr: INFO: 2024-04-16T18:38:50.107+0000 7f1c8184e640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:43:51.485+0000 7f6ad2a9e980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.658 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_bucket_incremental_sync_after_data_sync_init 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2214, in test_bucket_incremental_sync_after_data_sync_init 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.659 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-55/ 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-55/ 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-55/ 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.660 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:43:51 GMT 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:/lybtug-55/ 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:b0oKjtTpPECT1KW6ilGsd7mWoWs= 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:43:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:b0oKjtTpPECT1KW6ilGsd7mWoWs=', 'Content-Length': '0'} 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx0000053beebcd67dbae66-00661ec6e7-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:43:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: created bucket=lybtug-55 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.661 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293031.566554_2750.1","last_update":"2024-04-16T18:43:51.566554Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293031.566554_2750.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:48:51.762+0000 7f8323156980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.662 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_resharded_bucket_incremental_sync_latest_after_data_sync_init 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2253, in test_resharded_bucket_incremental_sync_latest_after_data_sync_init 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.663 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-56/ 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-56/ 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-56/ 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.664 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:48:51 GMT 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:/lybtug-56/ 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:yQtFeD55GNMtqawLOjNCuk7wsbk= 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:48:51 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:yQtFeD55GNMtqawLOjNCuk7wsbk=', 'Content-Length': '0'} 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx0000026b70e215d25a38b-00661ec813-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:48:51 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: created bucket=lybtug-56 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.665 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293331.839346_3712.1","last_update":"2024-04-16T18:48:51.839346Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293331.839346_3712.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:53:52.038+0000 7f77efe64980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.666 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests:FAIL: rgw_multi.tests.test_resharded_bucket_incremental_sync_oldest_after_data_sync_init 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2302, in test_resharded_bucket_incremental_sync_oldest_after_data_sync_init 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.667 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: path=/lybtug-57/ 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: auth_path=/lybtug-57/ 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Method: PUT 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Path: /lybtug-57/ 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Data: 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Headers: {} 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Host: smithi071.front.sepia.ceph.com:8000 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Port: 8000 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Params: {} 2024-04-16T19:28:54.668 INFO:tasks.rgw_multisite_tests:boto: DEBUG: establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Token: None 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:boto: DEBUG: StringToSign: 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:PUT 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:Tue, 16 Apr 2024 18:53:52 GMT 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:/lybtug-57/ 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Signature: 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:AWS 076QSHZWMHBH668NR2NY:J22F+Ny5fHdciI1m+mDg3WcpRS8= 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-166-generic', 'Date': 'Tue, 16 Apr 2024 18:53:52 GMT', 'Authorization': 'AWS 076QSHZWMHBH668NR2NY:J22F+Ny5fHdciI1m+mDg3WcpRS8=', 'Content-Length': '0'} 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:boto: DEBUG: Response headers: [('x-amz-request-id', 'tx00000bef2c458eca91dc7-00661ec940-4804-test-zone1'), ('Server', 'Ceph Object Gateway (squid)'), ('Content-Length', '0'), ('Date', 'Tue, 16 Apr 2024 18:53:52 GMT'), ('Connection', 'Keep-Alive')] 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: created bucket=lybtug-57 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.669 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T18:58:52.283+0000 7fddcf221980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.670 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests:FAIL: test_sync_policy_config_zonegroup: 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2461, in test_sync_policy_config_zonegroup 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.671 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.672 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T19:03:52.492+0000 7feb0b37d980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:FAIL: test_sync_flow_symmetrical_zonegroup_all: 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.673 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2534, in test_sync_flow_symmetrical_zonegroup_all 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.674 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T19:08:52.718+0000 7fc5aed8d980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.675 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests:FAIL: test_sync_single_bucket: 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2783, in test_sync_single_bucket 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.676 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.677 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T19:13:52.975+0000 7f8a96cd4980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:FAIL: test_sync_different_buckets: 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.678 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 2901, in test_sync_different_buckets 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.679 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T19:18:53.295+0000 7efeb3b59980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.680 INFO:tasks.rgw_multisite_tests:FAIL: test_sync_multiple_buckets_to_single: 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 3048, in test_sync_multiple_buckets_to_single 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.681 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.682 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T19:23:53.564+0000 7f800624b980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:====================================================================== 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:FAIL: test_sync_single_bucket_to_multiple: 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last): 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/nose/case.py", line 198, in runTest 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: self.test(*self.arg) 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 3170, in test_sync_single_bucket_to_multiple 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: zonegroup_meta_checkpoint(zonegroup) 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 211, in zonegroup_meta_checkpoint 2024-04-16T19:28:54.683 INFO:tasks.rgw_multisite_tests: zone_meta_checkpoint(zone, meta_master_zone, master_status) 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 186, in zone_meta_checkpoint 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests: global_status, period, realm_epoch, num_shards, sync_status = meta_sync_status(zone) 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/../src/test/rgw/rgw_multi/tests.py", line 143, in meta_sync_status 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests: assert(retcode == 2) # ENOENT 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:AssertionError: 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:-------------------- >> begin captured logging << -------------------- 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stderr: INFO: No --period given, using current period=8c713f9b-78f8-4e05-979f-375a77da9531 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi071.stdout: INFO: [{"marker":"1_1713293632.085053_4667.1","last_update":"2024-04-16T18:53:52.085053Z"},{"marker":"1_1713291352.392860_404.1","last_update":"2024-04-16T18:15:52.392860Z"},{"marker":"1_1713291351.746265_283.1","last_update":"2024-04-16T18:15:51.746265Z"},{"marker":"1_1713291335.360921_269.1","last_update":"2024-04-16T18:15:35.360921Z"}] 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: DEBUG: master meta markers={0: '1_1713293632.085053_4667.1', 1: '1_1713291352.392860_404.1', 2: '1_1713291351.746265_283.1', 3: '1_1713291335.360921_269.1'} 2024-04-16T19:28:54.684 INFO:tasks.rgw_multisite_tests:rgw_multi.tests: INFO: starting meta checkpoint for zone=test-zone2 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: INFO: rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:tasks.util.rgw: DEBUG: rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122: DEBUG: > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setuser ceph since I am not root 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: ignoring --setgroup ceph since I am not root 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: 2024-04-16T19:28:53.782+0000 7f9bb1899980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run.smithi122.stderr: INFO: failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:teuthology.orchestra.run: DEBUG: got remote process result: 1 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:--------------------- >> end captured logging << --------------------- 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:---------------------------------------------------------------------- 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests:Ran 65 tests in 6021.512s 2024-04-16T19:28:54.685 INFO:tasks.rgw_multisite_tests: 2024-04-16T19:28:54.686 INFO:tasks.rgw_multisite_tests:FAILED (SKIP=18, errors=1, failures=13) 2024-04-16T19:28:54.686 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 123, in __enter__ self.begin() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw_multisite_tests.py", line 96, in begin assert nose.run(argv=argv, config=conf), 'rgw multisite test failures' AssertionError: rgw multisite test failures 2024-04-16T19:28:57.715 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=80b86bedc2c34da6a0f9464df81b3fa2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 123, in __enter__ self.begin() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw_multisite_tests.py", line 96, in begin assert nose.run(argv=argv, config=conf), 'rgw multisite test failures' AssertionError: rgw multisite test failures 2024-04-16T19:28:57.717 DEBUG:teuthology.run_tasks:Unwinding manager rgw-multisite-tests 2024-04-16T19:28:57.727 DEBUG:teuthology.run_tasks:Unwinding manager rgw-multisite 2024-04-16T19:28:57.753 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-16T19:28:57.761 DEBUG:tasks.rgw.c1.client.0:waiting for process to exit 2024-04-16T19:28:57.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:28:57.766 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T19:28:57.759+0000 7f59da0b8640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 31601) UID: 0 2024-04-16T19:28:57.766 INFO:tasks.rgw.c1.client.0.smithi071.stdout:2024-04-16T19:28:57.759+0000 7f59dd900a40 -1 shutting down 2024-04-16T19:28:58.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 is failed for ~0s 2024-04-16T19:29:03.863 INFO:tasks.rgw.c1.client.0:Stopped 2024-04-16T19:29:03.865 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock 2024-04-16T19:29:03.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/vault-root-token 2024-04-16T19:29:03.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-16T19:29:03.986 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['gc', 'process', '--include-all'] 2024-04-16T19:29:03.986 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'gc', 'process', '--include-all'] 2024-04-16T19:29:03.986 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 gc process --include-all 2024-04-16T19:29:04.067 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:29:04.067 INFO:teuthology.orchestra.run.smithi071.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:29:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.103+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.103+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T19:29:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.103+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=110 2024-04-16T19:29:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 realm test-realm 750bb778-ce9e-43d8-aa1d-59a6a3ecda80 2024-04-16T19:29:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=110 2024-04-16T19:29:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=10 2024-04-16T19:29:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=1380 2024-04-16T19:29:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T19:29:04.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.107+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=110 2024-04-16T19:29:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T19:29:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=0 bl.length=1165 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 zone test-zone1 found 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 4 Realm: test-realm (750bb778-ce9e-43d8-aa1d-59a6a3ecda80) 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 4 ZoneGroup: test-zonegroup (test-zonegroup) 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 4 Zone: test-zone1 (054102a3-8d94-4596-8b3b-6341eab312e1) 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 4 using period configuration: 8c713f9b-78f8-4e05-979f-375a77da9531:10 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 period zonegroup name test-zonegroup 2024-04-16T19:29:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 using current period zonegroup test-zonegroup 2024-04-16T19:29:04.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:04.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:04.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 init(): pipe_map (zone=054102a3-8d94-4596-8b3b-6341eab312e1 bucket=*): adding potential pipe: {id=all,s={b=*,z=},d={b=*,z=}} 2024-04-16T19:29:04.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 reflect(): flow manager (bucket=): adding source pipe: {s={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0},d={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0}} 2024-04-16T19:29:04.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 reflect(): flow manager (bucket=): adding dest pipe: {s={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} 2024-04-16T19:29:04.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados->read ofs=0 len=0 2024-04-16T19:29:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 init(): pipe_map (zone=bd830684-b604-4841-ab52-8c82ee812281 bucket=*): adding potential pipe: {id=all,s={b=*,z=},d={b=*,z=}} 2024-04-16T19:29:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 reflect(): flow manager (bucket=): adding source pipe: {s={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} 2024-04-16T19:29:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 reflect(): flow manager (bucket=): adding dest pipe: {s={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0},d={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0}} 2024-04-16T19:29:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 reflect(): flow manager (bucket=): adding source pipe: {s={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0},d={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0}} 2024-04-16T19:29:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 reflect(): flow manager (bucket=): adding dest pipe: {s={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} 2024-04-16T19:29:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 started sync module instance, tier type = 2024-04-16T19:29:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 generating connection object for zone test-zone2 id bd830684-b604-4841-ab52-8c82ee812281 2024-04-16T19:29:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.111+0000 7fdd080e6980 20 started zone id=054102a3-8d94-4596-8b3b-6341eab312e1 (name=test-zone1) with tier type = 2024-04-16T19:29:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=0 2024-04-16T19:29:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=1 2024-04-16T19:29:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=2 2024-04-16T19:29:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=3 2024-04-16T19:29:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=4 2024-04-16T19:29:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=5 2024-04-16T19:29:04.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=6 2024-04-16T19:29:04.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 20 add_watcher() i=7 2024-04-16T19:29:04.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.131+0000 7fdd080e6980 2 all 8 watchers are set, enabling cache 2024-04-16T19:29:04.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.135+0000 7fdbb77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T19:29:04.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.135+0000 7fdd080e6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T19:29:04.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.135+0000 7fdd080e6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T19:29:04.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.247+0000 7fdbb67fc640 20 reqs_thread_entry: start 2024-04-16T19:29:04.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.247+0000 7fdd080e6980 20 init_complete bucket index max shards: 11 2024-04-16T19:29:04.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.247+0000 7fdd080e6980 20 Filter name: none 2024-04-16T19:29:04.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.247+0000 7fdb27fff640 20 reqs_thread_entry: start 2024-04-16T19:29:04.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.247+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-16T19:29:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.251+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.251+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.251+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.255+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-16T19:29:04.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.259+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.259+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.259+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.259+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-16T19:29:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.263+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.267+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.267+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.271+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-16T19:29:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.271+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.275+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.275+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.279+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-16T19:29:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.283+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.283+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.283+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.287+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-16T19:29:04.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.291+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.295+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.295+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.295+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-16T19:29:04.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.299+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.303+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.303+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.307+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-16T19:29:04.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.311+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.311+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.311+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.315+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-16T19:29:04.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.319+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.323+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.323+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.327+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-16T19:29:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.331+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.331+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.335+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.335+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-16T19:29:04.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.339+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.339+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.343+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.343+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-16T19:29:04.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.347+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.347+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.351+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.351+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-16T19:29:04.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.355+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.359+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.359+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.363+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-16T19:29:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.367+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.367+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.367+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.371+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-16T19:29:04.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.375+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.375+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.379+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.379+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-16T19:29:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.383+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.387+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.387+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.391+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-16T19:29:04.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.395+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.395+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.399+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.399+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-16T19:29:04.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.403+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.407+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.407+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.411+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-16T19:29:04.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.415+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.415+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.415+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.419+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-16T19:29:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.423+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.427+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.427+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.427+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-16T19:29:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.435+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.435+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.435+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.439+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-16T19:29:04.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.443+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.443+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.443+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.447+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-16T19:29:04.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.451+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.451+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.451+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.455+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-16T19:29:04.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.459+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.463+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.463+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.467+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-16T19:29:04.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.471+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.471+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.471+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.475+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-16T19:29:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.479+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.483+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.483+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.487+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-16T19:29:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.487+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.491+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.491+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.495+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-16T19:29:04.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.499+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.499+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.503+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.503+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-16T19:29:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.507+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.511+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.511+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.515+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-16T19:29:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.519+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.519+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.519+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.523+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-16T19:29:04.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.527+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.531+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.531+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.535+0000 7fdd080e6980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-16T19:29:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.539+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.539+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:04.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.539+0000 7fdd080e6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.547+0000 7fdbb77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T19:29:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.551+0000 7fdd080e6980 20 remove_watcher() i=0 2024-04-16T19:29:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.551+0000 7fdd080e6980 2 removed watcher, disabling cache 2024-04-16T19:29:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.551+0000 7fdd080e6980 20 remove_watcher() i=1 2024-04-16T19:29:04.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.555+0000 7fdd080e6980 20 remove_watcher() i=2 2024-04-16T19:29:04.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.555+0000 7fdd080e6980 20 remove_watcher() i=3 2024-04-16T19:29:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.559+0000 7fdd080e6980 20 remove_watcher() i=4 2024-04-16T19:29:04.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.563+0000 7fdd080e6980 20 remove_watcher() i=5 2024-04-16T19:29:04.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.567+0000 7fdd080e6980 20 remove_watcher() i=6 2024-04-16T19:29:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T19:29:04.567+0000 7fdd080e6980 20 remove_watcher() i=7 2024-04-16T19:29:04.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 has been restored 2024-04-16T19:29:04.588 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2024-04-16T19:29:04.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:29:04.589 INFO:tasks.rgw.c1.client.1.smithi083.stdout:2024-04-16T19:29:04.586+0000 7fa70c091640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground (PID: 27592) UID: 0 2024-04-16T19:29:04.589 INFO:tasks.rgw.c1.client.1.smithi083.stdout:2024-04-16T19:29:04.586+0000 7fa70f8d9a40 -1 shutting down 2024-04-16T19:29:10.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.1 is failed for ~0s 2024-04-16T19:29:10.690 INFO:tasks.rgw.c1.client.1:Stopped 2024-04-16T19:29:10.690 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock 2024-04-16T19:29:10.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/vault-root-token 2024-04-16T19:29:10.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-16T19:29:10.807 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['gc', 'process', '--include-all'] 2024-04-16T19:29:10.808 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'gc', 'process', '--include-all'] 2024-04-16T19:29:10.808 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 gc process --include-all 2024-04-16T19:29:10.887 INFO:teuthology.orchestra.run.smithi083.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:29:10.887 INFO:teuthology.orchestra.run.smithi083.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:29:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.930+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T19:29:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=110 2024-04-16T19:29:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 realm test-realm 750bb778-ce9e-43d8-aa1d-59a6a3ecda80 2024-04-16T19:29:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=110 2024-04-16T19:29:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.934+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=10 2024-04-16T19:29:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=1380 2024-04-16T19:29:10.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T19:29:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=110 2024-04-16T19:29:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.938+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=46 2024-04-16T19:29:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados_obj.operate() r=0 bl.length=1165 2024-04-16T19:29:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 zone test-zone1 found 2024-04-16T19:29:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 4 Realm: test-realm (750bb778-ce9e-43d8-aa1d-59a6a3ecda80) 2024-04-16T19:29:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 4 ZoneGroup: test-zonegroup (test-zonegroup) 2024-04-16T19:29:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 4 Zone: test-zone1 (054102a3-8d94-4596-8b3b-6341eab312e1) 2024-04-16T19:29:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 4 using period configuration: 8c713f9b-78f8-4e05-979f-375a77da9531:10 2024-04-16T19:29:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 period zonegroup name test-zonegroup 2024-04-16T19:29:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 using current period zonegroup test-zonegroup 2024-04-16T19:29:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 init(): pipe_map (zone=054102a3-8d94-4596-8b3b-6341eab312e1 bucket=*): adding potential pipe: {id=all,s={b=*,z=},d={b=*,z=}} 2024-04-16T19:29:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 reflect(): flow manager (bucket=): adding source pipe: {s={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0},d={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0}} 2024-04-16T19:29:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 reflect(): flow manager (bucket=): adding dest pipe: {s={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} 2024-04-16T19:29:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados->read ofs=0 len=0 2024-04-16T19:29:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-16T19:29:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 init(): pipe_map (zone=bd830684-b604-4841-ab52-8c82ee812281 bucket=*): adding potential pipe: {id=all,s={b=*,z=},d={b=*,z=}} 2024-04-16T19:29:10.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 reflect(): flow manager (bucket=): adding source pipe: {s={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} 2024-04-16T19:29:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 reflect(): flow manager (bucket=): adding dest pipe: {s={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0},d={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0}} 2024-04-16T19:29:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 reflect(): flow manager (bucket=): adding source pipe: {s={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0},d={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0}} 2024-04-16T19:29:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 reflect(): flow manager (bucket=): adding dest pipe: {s={b=*,z=054102a3-8d94-4596-8b3b-6341eab312e1,az=0},d={b=*,z=bd830684-b604-4841-ab52-8c82ee812281,az=0}} 2024-04-16T19:29:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 started sync module instance, tier type = 2024-04-16T19:29:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 generating connection object for zone test-zone2 id bd830684-b604-4841-ab52-8c82ee812281 2024-04-16T19:29:10.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.942+0000 7f74702ad980 20 started zone id=054102a3-8d94-4596-8b3b-6341eab312e1 (name=test-zone1) with tier type = 2024-04-16T19:29:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=0 2024-04-16T19:29:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=1 2024-04-16T19:29:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=2 2024-04-16T19:29:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=3 2024-04-16T19:29:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=4 2024-04-16T19:29:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=5 2024-04-16T19:29:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.962+0000 7f74702ad980 20 add_watcher() i=6 2024-04-16T19:29:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.966+0000 7f74702ad980 20 add_watcher() i=7 2024-04-16T19:29:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.966+0000 7f74702ad980 2 all 8 watchers are set, enabling cache 2024-04-16T19:29:10.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.970+0000 7f731f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T19:29:10.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.970+0000 7f74702ad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-16T19:29:10.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:10.970+0000 7f74702ad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-16T19:29:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.074+0000 7f731e7fc640 20 reqs_thread_entry: start 2024-04-16T19:29:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.074+0000 7f74702ad980 20 init_complete bucket index max shards: 11 2024-04-16T19:29:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.074+0000 7f74702ad980 20 Filter name: none 2024-04-16T19:29:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.074+0000 7f728ffff640 20 reqs_thread_entry: start 2024-04-16T19:29:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.074+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-16T19:29:11.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.078+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.078+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.082+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.082+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-16T19:29:11.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.086+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.086+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.086+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.090+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-16T19:29:11.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.094+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.094+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.094+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.098+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-16T19:29:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.102+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.102+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.102+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.106+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-16T19:29:11.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.106+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.110+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.110+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.110+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-16T19:29:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.114+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.118+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.118+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.118+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-16T19:29:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.122+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.126+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.126+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.126+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-16T19:29:11.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.130+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.130+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.134+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.134+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-16T19:29:11.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.138+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.138+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.138+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.142+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-16T19:29:11.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.146+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.146+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.146+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.150+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-16T19:29:11.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.154+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.154+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.158+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.158+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-16T19:29:11.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.162+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.162+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.166+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.166+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-16T19:29:11.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.170+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.170+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.170+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.174+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-16T19:29:11.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.178+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.178+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.178+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.182+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-16T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.186+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.186+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.186+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.190+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-16T19:29:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.194+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.194+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.194+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.194+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-16T19:29:11.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.198+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.202+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.202+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.206+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-16T19:29:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.206+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.210+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.210+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.210+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-16T19:29:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.214+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.218+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.218+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.222+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-16T19:29:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.226+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.226+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.230+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.234+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-16T19:29:11.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.234+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.238+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.238+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.242+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-16T19:29:11.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.246+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.246+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.250+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.250+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-16T19:29:11.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.254+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.258+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.258+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.262+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-16T19:29:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.262+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.266+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.266+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.270+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-16T19:29:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.274+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.274+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.274+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.278+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-16T19:29:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.282+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.282+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.282+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.286+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-16T19:29:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.290+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.290+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.294+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.294+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-16T19:29:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.298+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.298+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.302+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.302+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-16T19:29:11.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.306+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.310+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.310+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.310+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-16T19:29:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.314+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.318+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.318+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.322+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-16T19:29:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.322+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.326+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.326+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.330+0000 7f74702ad980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-16T19:29:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.334+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.334+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-16T19:29:11.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.338+0000 7f74702ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-16T19:29:11.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.342+0000 7f731f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-16T19:29:11.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.346+0000 7f74702ad980 20 remove_watcher() i=0 2024-04-16T19:29:11.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.346+0000 7f74702ad980 2 removed watcher, disabling cache 2024-04-16T19:29:11.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.350+0000 7f74702ad980 20 remove_watcher() i=1 2024-04-16T19:29:11.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.354+0000 7f74702ad980 20 remove_watcher() i=2 2024-04-16T19:29:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.358+0000 7f74702ad980 20 remove_watcher() i=3 2024-04-16T19:29:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.358+0000 7f74702ad980 20 remove_watcher() i=4 2024-04-16T19:29:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.362+0000 7f74702ad980 20 remove_watcher() i=5 2024-04-16T19:29:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.362+0000 7f74702ad980 20 remove_watcher() i=6 2024-04-16T19:29:11.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-16T19:29:11.366+0000 7f74702ad980 20 remove_watcher() i=7 2024-04-16T19:29:11.390 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2024-04-16T19:29:11.390 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:29:11.391 INFO:tasks.rgw.c2.client.0:Stopped 2024-04-16T19:29:11.391 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock 2024-04-16T19:29:11.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/vault-root-token 2024-04-16T19:29:11.452 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-16T19:29:11.508 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['gc', 'process', '--include-all'] 2024-04-16T19:29:11.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'gc', 'process', '--include-all'] 2024-04-16T19:29:11.508 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all 2024-04-16T19:29:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setuser ceph since I am not root 2024-04-16T19:29:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:ignoring --setgroup ceph since I am not root 2024-04-16T19:29:16.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.1 has been restored 2024-04-16T19:34:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T19:34:11.590+0000 7ff23bb69980 0 monclient(hunting): authenticate timed out after 300 2024-04-16T19:34:11.592 INFO:teuthology.orchestra.run.smithi122.stderr:failed to fetch mon config (--no-mon-config to skip) 2024-04-16T19:34:11.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:34:11.594 ERROR:teuthology.run_tasks:Manager failed: rgw Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 556, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 364, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 269, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' 2024-04-16T19:34:11.657 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-16T19:34:11.744 INFO:tasks.ceph.ceph_manager.c2:waiting for clean 2024-04-16T19:34:11.745 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2024-04-16T19:36:11.774 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-16T19:36:11.775 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 556, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 364, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 269, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json' 2024-04-16T19:36:11.868 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-16T19:36:11.868 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-16T19:36:11.869 DEBUG:tasks.ceph.c2.osd.0:waiting for process to exit 2024-04-16T19:36:11.869 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.873 INFO:tasks.ceph.c2.osd.0:Stopped 2024-04-16T19:36:11.873 DEBUG:tasks.ceph.c2.osd.1:waiting for process to exit 2024-04-16T19:36:11.874 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.874 INFO:tasks.ceph.c2.osd.1:Stopped 2024-04-16T19:36:11.874 DEBUG:tasks.ceph.c2.osd.2:waiting for process to exit 2024-04-16T19:36:11.875 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.875 INFO:tasks.ceph.c2.osd.2:Stopped 2024-04-16T19:36:11.875 DEBUG:tasks.ceph.c2.osd.3:waiting for process to exit 2024-04-16T19:36:11.875 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.876 INFO:tasks.ceph.c2.osd.3:Stopped 2024-04-16T19:36:11.876 DEBUG:tasks.ceph.c2.osd.4:waiting for process to exit 2024-04-16T19:36:11.876 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.876 INFO:tasks.ceph.c2.osd.4:Stopped 2024-04-16T19:36:11.877 DEBUG:tasks.ceph.c2.osd.5:waiting for process to exit 2024-04-16T19:36:11.877 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.877 INFO:tasks.ceph.c2.osd.5:Stopped 2024-04-16T19:36:11.877 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-16T19:36:11.878 DEBUG:tasks.ceph.c2.mgr.x:waiting for process to exit 2024-04-16T19:36:11.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.878 INFO:tasks.ceph.c2.mgr.x:Stopped 2024-04-16T19:36:11.878 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-16T19:36:11.879 DEBUG:tasks.ceph.c2.mon.a:waiting for process to exit 2024-04-16T19:36:11.879 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:36:11.879 INFO:tasks.ceph.c2.mon.a:Stopped 2024-04-16T19:36:11.880 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-16T19:36:11.880 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c2.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-04-16T19:36:11.899 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c2-0 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T19:36:11.900 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/c2-0 2024-04-16T19:36:12.146 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c2-1 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T19:36:12.146 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/c2-1 2024-04-16T19:36:12.311 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c2-2 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T19:36:12.312 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/c2-2 2024-04-16T19:36:12.504 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c2-3 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T19:36:12.504 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/c2-3 2024-04-16T19:36:12.626 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c2-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T19:36:12.627 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/c2-4 2024-04-16T19:36:12.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c2-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T19:36:12.800 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/c2-5 2024-04-16T19:36:12.974 INFO:tasks.ceph:Archiving mon data... 2024-04-16T19:36:12.976 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/c2-a to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/data/c2.mon.a.tgz 2024-04-16T19:36:12.977 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-04-16T19:36:12.983 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.q1cHBCQEwf 2024-04-16T19:36:12.984 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/c2-a -- . > /tmp/tmp.q1cHBCQEwf 2024-04-16T19:36:13.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.q1cHBCQEwf 2024-04-16T19:36:13.185 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.q1cHBCQEwf is 323KB 2024-04-16T19:36:13.291 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.q1cHBCQEwf 2024-04-16T19:36:13.297 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-16T19:36:13.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/c2.conf /etc/ceph/c2.keyring /home/ubuntu/cephtest/c2.data /home/ubuntu/cephtest/c2.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:36:13.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/c2.conf /etc/ceph/c2.keyring /home/ubuntu/cephtest/c2.data /home/ubuntu/cephtest/c2.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:36:13.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/c2.conf /etc/ceph/c2.keyring /home/ubuntu/cephtest/c2.data /home/ubuntu/cephtest/c2.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:36:13.342 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/c2.conf /etc/ceph/c2.keyring /home/ubuntu/cephtest/c2.data /home/ubuntu/cephtest/c2.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:36:13.358 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 556, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 364, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 269, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json' 2024-04-16T19:36:13.366 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-16T19:36:13.377 INFO:tasks.ceph.ceph_manager.c1:waiting for clean 2024-04-16T19:36:13.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:36:13.776 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:36:13.810 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:36:13.816 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3334,"stamp":"2024-04-16T19:36:12.315071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39759,"ondisk_log_size":39759,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":354032,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":343862,"kb_avail":628791568,"statfs":{"total":644245094400,"available":643882565632,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":352115360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008536"},"pg_stats":[{"pgid":"9.35","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147548+0000","last_change":"2024-04-16T17:47:55.160868+0000","last_active":"2024-04-16T19:31:51.147548+0000","last_peered":"2024-04-16T19:31:51.147548+0000","last_clean":"2024-04-16T19:31:51.147548+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:31:51.147548+0000","last_undegraded":"2024-04-16T19:31:51.147548+0000","last_fullsized":"2024-04-16T19:31:51.147548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176615+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:33:36.176615+0000","last_peered":"2024-04-16T19:33:36.176615+0000","last_clean":"2024-04-16T19:33:36.176615+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:33:36.176615+0000","last_undegraded":"2024-04-16T19:33:36.176615+0000","last_fullsized":"2024-04-16T19:33:36.176615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:51.288503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153219+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:32:26.153219+0000","last_peered":"2024-04-16T19:32:26.153219+0000","last_clean":"2024-04-16T19:32:26.153219+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:32:26.153219+0000","last_undegraded":"2024-04-16T19:32:26.153219+0000","last_fullsized":"2024-04-16T19:32:26.153219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'14","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143970+0000","last_change":"2024-04-16T17:47:55.173885+0000","last_active":"2024-04-16T19:31:51.143970+0000","last_peered":"2024-04-16T19:31:51.143970+0000","last_clean":"2024-04-16T19:31:51.143970+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:31:51.143970+0000","last_undegraded":"2024-04-16T19:31:51.143970+0000","last_fullsized":"2024-04-16T19:31:51.143970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:50.786411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174452+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:33:36.174452+0000","last_peered":"2024-04-16T19:33:36.174452+0000","last_clean":"2024-04-16T19:33:36.174452+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:33:36.174452+0000","last_undegraded":"2024-04-16T19:33:36.174452+0000","last_fullsized":"2024-04-16T19:33:36.174452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.158347+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:32:26.158347+0000","last_peered":"2024-04-16T19:32:26.158347+0000","last_clean":"2024-04-16T19:32:26.158347+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:32:26.158347+0000","last_undegraded":"2024-04-16T19:32:26.158347+0000","last_fullsized":"2024-04-16T19:32:26.158347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'20","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152906+0000","last_change":"2024-04-16T17:47:55.156902+0000","last_active":"2024-04-16T19:32:26.152906+0000","last_peered":"2024-04-16T19:32:26.152906+0000","last_clean":"2024-04-16T19:32:26.152906+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:32:26.152906+0000","last_undegraded":"2024-04-16T19:32:26.152906+0000","last_fullsized":"2024-04-16T19:32:26.152906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:40.884246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173569+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:33:36.173569+0000","last_peered":"2024-04-16T19:33:36.173569+0000","last_clean":"2024-04-16T19:33:36.173569+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:33:36.173569+0000","last_undegraded":"2024-04-16T19:33:36.173569+0000","last_fullsized":"2024-04-16T19:33:36.173569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152536+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:32:26.152536+0000","last_peered":"2024-04-16T19:32:26.152536+0000","last_clean":"2024-04-16T19:32:26.152536+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:32:26.152536+0000","last_undegraded":"2024-04-16T19:32:26.152536+0000","last_fullsized":"2024-04-16T19:32:26.152536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:32.905270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147603+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:31:51.147603+0000","last_peered":"2024-04-16T19:31:51.147603+0000","last_clean":"2024-04-16T19:31:51.147603+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:31:51.147603+0000","last_undegraded":"2024-04-16T19:31:51.147603+0000","last_fullsized":"2024-04-16T19:31:51.147603+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:10.585096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174129+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:33:46.174129+0000","last_peered":"2024-04-16T19:33:46.174129+0000","last_clean":"2024-04-16T19:33:46.174129+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:33:46.174129+0000","last_undegraded":"2024-04-16T19:33:46.174129+0000","last_fullsized":"2024-04-16T19:33:46.174129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:53.592351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155669+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:32:26.155669+0000","last_peered":"2024-04-16T19:32:26.155669+0000","last_clean":"2024-04-16T19:32:26.155669+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:32:26.155669+0000","last_undegraded":"2024-04-16T19:32:26.155669+0000","last_fullsized":"2024-04-16T19:32:26.155669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'4","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159892+0000","last_change":"2024-04-16T17:47:55.159892+0000","last_active":"2024-04-16T19:32:56.159892+0000","last_peered":"2024-04-16T19:32:56.159892+0000","last_clean":"2024-04-16T19:32:56.159892+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:32:56.159892+0000","last_undegraded":"2024-04-16T19:32:56.159892+0000","last_fullsized":"2024-04-16T19:32:56.159892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:58.461789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159155+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:32:36.159155+0000","last_peered":"2024-04-16T19:32:36.159155+0000","last_clean":"2024-04-16T19:32:36.159155+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:32:36.159155+0000","last_undegraded":"2024-04-16T19:32:36.159155+0000","last_fullsized":"2024-04-16T19:32:36.159155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175465+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:33:51.175465+0000","last_peered":"2024-04-16T19:33:51.175465+0000","last_clean":"2024-04-16T19:33:51.175465+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:33:51.175465+0000","last_undegraded":"2024-04-16T19:33:51.175465+0000","last_fullsized":"2024-04-16T19:33:51.175465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'11","reported_seq":309,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163313+0000","last_change":"2024-04-16T17:47:55.163604+0000","last_active":"2024-04-16T19:32:56.163313+0000","last_peered":"2024-04-16T19:32:56.163313+0000","last_clean":"2024-04-16T19:32:56.163313+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:32:56.163313+0000","last_undegraded":"2024-04-16T19:32:56.163313+0000","last_fullsized":"2024-04-16T19:32:56.163313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151934+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:32:21.151934+0000","last_peered":"2024-04-16T19:32:21.151934+0000","last_clean":"2024-04-16T19:32:21.151934+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:32:21.151934+0000","last_undegraded":"2024-04-16T19:32:21.151934+0000","last_fullsized":"2024-04-16T19:32:21.151934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171180+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:33:41.171180+0000","last_peered":"2024-04-16T19:33:41.171180+0000","last_clean":"2024-04-16T19:33:41.171180+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:33:41.171180+0000","last_undegraded":"2024-04-16T19:33:41.171180+0000","last_fullsized":"2024-04-16T19:33:41.171180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:47.915323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'12","reported_seq":268,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.148697+0000","last_change":"2024-04-16T17:47:55.161748+0000","last_active":"2024-04-16T19:31:51.148697+0000","last_peered":"2024-04-16T19:31:51.148697+0000","last_clean":"2024-04-16T19:31:51.148697+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:31:51.148697+0000","last_undegraded":"2024-04-16T19:31:51.148697+0000","last_fullsized":"2024-04-16T19:31:51.148697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157014+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:32:21.157014+0000","last_peered":"2024-04-16T19:32:21.157014+0000","last_clean":"2024-04-16T19:32:21.157014+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:32:21.157014+0000","last_undegraded":"2024-04-16T19:32:21.157014+0000","last_fullsized":"2024-04-16T19:32:21.157014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:14.655507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172868+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:33:41.172868+0000","last_peered":"2024-04-16T19:33:41.172868+0000","last_clean":"2024-04-16T19:33:41.172868+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:33:41.172868+0000","last_undegraded":"2024-04-16T19:33:41.172868+0000","last_fullsized":"2024-04-16T19:33:41.172868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'8","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168449+0000","last_change":"2024-04-16T17:47:55.164548+0000","last_active":"2024-04-16T19:33:31.168449+0000","last_peered":"2024-04-16T19:33:31.168449+0000","last_clean":"2024-04-16T19:33:31.168449+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:33:31.168449+0000","last_undegraded":"2024-04-16T19:33:31.168449+0000","last_fullsized":"2024-04-16T19:33:31.168449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:27.967153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138549+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:32:21.138549+0000","last_peered":"2024-04-16T19:32:21.138549+0000","last_clean":"2024-04-16T19:32:21.138549+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:32:21.138549+0000","last_undegraded":"2024-04-16T19:32:21.138549+0000","last_fullsized":"2024-04-16T19:32:21.138549+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:19:39.583925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157481+0000","last_change":"2024-04-16T17:47:23.254199+0000","last_active":"2024-04-16T19:33:41.157481+0000","last_peered":"2024-04-16T19:33:41.157481+0000","last_clean":"2024-04-16T19:33:41.157481+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:33:41.157481+0000","last_undegraded":"2024-04-16T19:33:41.157481+0000","last_fullsized":"2024-04-16T19:33:41.157481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:11.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"48'5","reported_seq":300,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148127+0000","last_change":"2024-04-16T17:47:55.184230+0000","last_active":"2024-04-16T19:32:06.148127+0000","last_peered":"2024-04-16T19:32:06.148127+0000","last_clean":"2024-04-16T19:32:06.148127+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:32:06.148127+0000","last_undegraded":"2024-04-16T19:32:06.148127+0000","last_fullsized":"2024-04-16T19:32:06.148127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152428+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:32:26.152428+0000","last_peered":"2024-04-16T19:32:26.152428+0000","last_clean":"2024-04-16T19:32:26.152428+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:32:26.152428+0000","last_undegraded":"2024-04-16T19:32:26.152428+0000","last_fullsized":"2024-04-16T19:32:26.152428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150380+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:32:01.150380+0000","last_peered":"2024-04-16T19:32:01.150380+0000","last_clean":"2024-04-16T19:32:01.150380+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:32:01.150380+0000","last_undegraded":"2024-04-16T19:32:01.150380+0000","last_fullsized":"2024-04-16T19:32:01.150380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'14","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159666+0000","last_change":"2024-04-16T17:47:55.163042+0000","last_active":"2024-04-16T19:32:31.159666+0000","last_peered":"2024-04-16T19:32:31.159666+0000","last_clean":"2024-04-16T19:32:31.159666+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:32:31.159666+0000","last_undegraded":"2024-04-16T19:32:31.159666+0000","last_fullsized":"2024-04-16T19:32:31.159666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153260+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:32:26.153260+0000","last_peered":"2024-04-16T19:32:26.153260+0000","last_clean":"2024-04-16T19:32:26.153260+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:32:26.153260+0000","last_undegraded":"2024-04-16T19:32:26.153260+0000","last_fullsized":"2024-04-16T19:32:26.153260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178113+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:33:41.178113+0000","last_peered":"2024-04-16T19:33:41.178113+0000","last_clean":"2024-04-16T19:33:41.178113+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:33:41.178113+0000","last_undegraded":"2024-04-16T19:33:41.178113+0000","last_fullsized":"2024-04-16T19:33:41.178113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'8","reported_seq":270,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135460+0000","last_change":"2024-04-16T17:47:55.163584+0000","last_active":"2024-04-16T19:32:06.135460+0000","last_peered":"2024-04-16T19:32:06.135460+0000","last_clean":"2024-04-16T19:32:06.135460+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:32:06.135460+0000","last_undegraded":"2024-04-16T19:32:06.135460+0000","last_fullsized":"2024-04-16T19:32:06.135460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:28.473272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159805+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:32:51.159805+0000","last_peered":"2024-04-16T19:32:51.159805+0000","last_clean":"2024-04-16T19:32:51.159805+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:32:51.159805+0000","last_undegraded":"2024-04-16T19:32:51.159805+0000","last_fullsized":"2024-04-16T19:32:51.159805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172776+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:33:41.172776+0000","last_peered":"2024-04-16T19:33:41.172776+0000","last_clean":"2024-04-16T19:33:41.172776+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:33:41.172776+0000","last_undegraded":"2024-04-16T19:33:41.172776+0000","last_fullsized":"2024-04-16T19:33:41.172776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:01.960480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"50'5","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135378+0000","last_change":"2024-04-16T17:47:55.164609+0000","last_active":"2024-04-16T19:32:06.135378+0000","last_peered":"2024-04-16T19:32:06.135378+0000","last_clean":"2024-04-16T19:32:06.135378+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:32:06.135378+0000","last_undegraded":"2024-04-16T19:32:06.135378+0000","last_fullsized":"2024-04-16T19:32:06.135378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:56.378878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158570+0000","last_change":"2024-04-16T17:47:23.254226+0000","last_active":"2024-04-16T19:33:46.158570+0000","last_peered":"2024-04-16T19:33:46.158570+0000","last_clean":"2024-04-16T19:33:46.158570+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:33:46.158570+0000","last_undegraded":"2024-04-16T19:33:46.158570+0000","last_fullsized":"2024-04-16T19:33:46.158570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:06.511950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171122+0000","last_change":"2024-04-16T17:47:32.284109+0000","last_active":"2024-04-16T19:33:41.171122+0000","last_peered":"2024-04-16T19:33:41.171122+0000","last_clean":"2024-04-16T19:33:41.171122+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:33:41.171122+0000","last_undegraded":"2024-04-16T19:33:41.171122+0000","last_fullsized":"2024-04-16T19:33:41.171122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'5","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166165+0000","last_change":"2024-04-16T17:47:55.172087+0000","last_active":"2024-04-16T19:32:56.166165+0000","last_peered":"2024-04-16T19:32:56.166165+0000","last_clean":"2024-04-16T19:32:56.166165+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:32:56.166165+0000","last_undegraded":"2024-04-16T19:32:56.166165+0000","last_fullsized":"2024-04-16T19:32:56.166165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158622+0000","last_change":"2024-04-16T17:47:23.266132+0000","last_active":"2024-04-16T19:33:46.158622+0000","last_peered":"2024-04-16T19:33:46.158622+0000","last_clean":"2024-04-16T19:33:46.158622+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:33:46.158622+0000","last_undegraded":"2024-04-16T19:33:46.158622+0000","last_fullsized":"2024-04-16T19:33:46.158622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:09.592666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.358504+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:31:31.358504+0000","last_peered":"2024-04-16T19:31:31.358504+0000","last_clean":"2024-04-16T19:31:31.358504+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:31:31.358504+0000","last_undegraded":"2024-04-16T19:31:31.358504+0000","last_fullsized":"2024-04-16T19:31:31.358504+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:02.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'10","reported_seq":265,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146504+0000","last_change":"2024-04-16T17:47:55.158785+0000","last_active":"2024-04-16T19:32:56.146504+0000","last_peered":"2024-04-16T19:32:56.146504+0000","last_clean":"2024-04-16T19:32:56.146504+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:32:56.146504+0000","last_undegraded":"2024-04-16T19:32:56.146504+0000","last_fullsized":"2024-04-16T19:32:56.146504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:39.387454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172430+0000","last_change":"2024-04-16T17:47:23.257289+0000","last_active":"2024-04-16T19:33:46.172430+0000","last_peered":"2024-04-16T19:33:46.172430+0000","last_clean":"2024-04-16T19:33:46.172430+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:33:46.172430+0000","last_undegraded":"2024-04-16T19:33:46.172430+0000","last_fullsized":"2024-04-16T19:33:46.172430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163511+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:32:56.163511+0000","last_peered":"2024-04-16T19:32:56.163511+0000","last_clean":"2024-04-16T19:32:56.163511+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:32:56.163511+0000","last_undegraded":"2024-04-16T19:32:56.163511+0000","last_fullsized":"2024-04-16T19:32:56.163511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:29:20.877336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151839+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:32:06.151839+0000","last_peered":"2024-04-16T19:32:06.151839+0000","last_clean":"2024-04-16T19:32:06.151839+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:32:06.151839+0000","last_undegraded":"2024-04-16T19:32:06.151839+0000","last_fullsized":"2024-04-16T19:32:06.151839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:23:28.148511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170388+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:33:11.170388+0000","last_peered":"2024-04-16T19:33:11.170388+0000","last_clean":"2024-04-16T19:33:11.170388+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:33:11.170388+0000","last_undegraded":"2024-04-16T19:33:11.170388+0000","last_fullsized":"2024-04-16T19:33:11.170388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175004+0000","last_change":"2024-04-16T17:48:27.308777+0000","last_active":"2024-04-16T19:33:31.175004+0000","last_peered":"2024-04-16T19:33:31.175004+0000","last_clean":"2024-04-16T19:33:31.175004+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:33:31.175004+0000","last_undegraded":"2024-04-16T19:33:31.175004+0000","last_fullsized":"2024-04-16T19:33:31.175004+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'9","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164889+0000","last_change":"2024-04-16T17:48:01.266271+0000","last_active":"2024-04-16T19:33:01.164889+0000","last_peered":"2024-04-16T19:33:01.164889+0000","last_clean":"2024-04-16T19:33:01.164889+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:33:01.164889+0000","last_undegraded":"2024-04-16T19:33:01.164889+0000","last_fullsized":"2024-04-16T19:33:01.164889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161254+0000","last_change":"2024-04-16T17:47:59.181005+0000","last_active":"2024-04-16T19:33:01.161254+0000","last_peered":"2024-04-16T19:33:01.161254+0000","last_clean":"2024-04-16T19:33:01.161254+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:33:01.161254+0000","last_undegraded":"2024-04-16T19:33:01.161254+0000","last_fullsized":"2024-04-16T19:33:01.161254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:34.728189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'547","reported_seq":799,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167615+0000","last_change":"2024-04-16T17:47:57.177580+0000","last_active":"2024-04-16T19:33:01.167615+0000","last_peered":"2024-04-16T19:33:01.167615+0000","last_clean":"2024-04-16T19:33:01.167615+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:33:01.167615+0000","last_undegraded":"2024-04-16T19:33:01.167615+0000","last_fullsized":"2024-04-16T19:33:01.167615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'7","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138635+0000","last_change":"2024-04-16T17:47:55.165841+0000","last_active":"2024-04-16T19:32:21.138635+0000","last_peered":"2024-04-16T19:32:21.138635+0000","last_clean":"2024-04-16T19:32:21.138635+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:32:21.138635+0000","last_undegraded":"2024-04-16T19:32:21.138635+0000","last_fullsized":"2024-04-16T19:32:21.138635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:07:11.773810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.139718+0000","last_change":"2024-04-16T17:47:39.326260+0000","last_active":"2024-04-16T19:31:36.139718+0000","last_peered":"2024-04-16T19:31:36.139718+0000","last_clean":"2024-04-16T19:31:36.139718+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:31:36.139718+0000","last_undegraded":"2024-04-16T19:31:36.139718+0000","last_fullsized":"2024-04-16T19:31:36.139718+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:58.230179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164460+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:32:56.164460+0000","last_peered":"2024-04-16T19:32:56.164460+0000","last_clean":"2024-04-16T19:32:56.164460+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:32:56.164460+0000","last_undegraded":"2024-04-16T19:32:56.164460+0000","last_fullsized":"2024-04-16T19:32:56.164460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159202+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:32:36.159202+0000","last_peered":"2024-04-16T19:32:36.159202+0000","last_clean":"2024-04-16T19:32:36.159202+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:32:36.159202+0000","last_undegraded":"2024-04-16T19:32:36.159202+0000","last_fullsized":"2024-04-16T19:32:36.159202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":768,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.370756+0000","last_change":"2024-04-16T17:47:33.303155+0000","last_active":"2024-04-16T19:31:31.370756+0000","last_peered":"2024-04-16T19:31:31.370756+0000","last_clean":"2024-04-16T19:31:31.370756+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:31:31.370756+0000","last_undegraded":"2024-04-16T19:31:31.370756+0000","last_fullsized":"2024-04-16T19:31:31.370756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:14.676250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367632+0000","last_change":"2024-04-16T17:47:35.314855+0000","last_active":"2024-04-16T19:31:31.367632+0000","last_peered":"2024-04-16T19:31:31.367632+0000","last_clean":"2024-04-16T19:31:31.367632+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:31:31.367632+0000","last_undegraded":"2024-04-16T19:31:31.367632+0000","last_fullsized":"2024-04-16T19:31:31.367632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140046+0000","last_change":"2024-04-16T17:47:37.332945+0000","last_active":"2024-04-16T19:31:36.140046+0000","last_peered":"2024-04-16T19:31:36.140046+0000","last_clean":"2024-04-16T19:31:36.140046+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:31:36.140046+0000","last_undegraded":"2024-04-16T19:31:36.140046+0000","last_fullsized":"2024-04-16T19:31:36.140046+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152563+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:33:21.152563+0000","last_peered":"2024-04-16T19:33:21.152563+0000","last_clean":"2024-04-16T19:33:21.152563+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:33:21.152563+0000","last_undegraded":"2024-04-16T19:33:21.152563+0000","last_fullsized":"2024-04-16T19:33:21.152563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:24.843140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'83","reported_seq":1420,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161248+0000","last_change":"2024-04-16T17:48:34.319654+0000","last_active":"2024-04-16T19:32:46.161248+0000","last_peered":"2024-04-16T19:32:46.161248+0000","last_clean":"2024-04-16T19:32:46.161248+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:32:46.161248+0000","last_undegraded":"2024-04-16T19:32:46.161248+0000","last_fullsized":"2024-04-16T19:32:46.161248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:13.391814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169963+0000","last_change":"2024-04-16T17:48:27.303999+0000","last_active":"2024-04-16T19:33:31.169963+0000","last_peered":"2024-04-16T19:33:31.169963+0000","last_clean":"2024-04-16T19:33:31.169963+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:33:31.169963+0000","last_undegraded":"2024-04-16T19:33:31.169963+0000","last_fullsized":"2024-04-16T19:33:31.169963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'5","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164928+0000","last_change":"2024-04-16T17:48:01.266699+0000","last_active":"2024-04-16T19:33:01.164928+0000","last_peered":"2024-04-16T19:33:01.164928+0000","last_clean":"2024-04-16T19:33:01.164928+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:33:01.164928+0000","last_undegraded":"2024-04-16T19:33:01.164928+0000","last_fullsized":"2024-04-16T19:33:01.164928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162337+0000","last_change":"2024-04-16T17:47:59.191306+0000","last_active":"2024-04-16T19:33:01.162337+0000","last_peered":"2024-04-16T19:33:01.162337+0000","last_clean":"2024-04-16T19:33:01.162337+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:33:01.162337+0000","last_undegraded":"2024-04-16T19:33:01.162337+0000","last_fullsized":"2024-04-16T19:33:01.162337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7326","reported_seq":20595,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147895+0000","last_change":"2024-04-16T17:47:57.179815+0000","last_active":"2024-04-16T19:33:01.147895+0000","last_peered":"2024-04-16T19:33:01.147895+0000","last_clean":"2024-04-16T19:33:01.147895+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:33:01.147895+0000","last_undegraded":"2024-04-16T19:33:01.147895+0000","last_fullsized":"2024-04-16T19:33:01.147895+0000","mapping_epoch":34,"log_start":"53'5500","ondisk_log_start":"53'5500","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1826,"log_dups_size":3000,"ondisk_log_size":1826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:36.013179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'19","reported_seq":354,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143591+0000","last_change":"2024-04-16T17:47:55.178555+0000","last_active":"2024-04-16T19:31:51.143591+0000","last_peered":"2024-04-16T19:31:51.143591+0000","last_clean":"2024-04-16T19:31:51.143591+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:31:51.143591+0000","last_undegraded":"2024-04-16T19:31:51.143591+0000","last_fullsized":"2024-04-16T19:31:51.143591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146395+0000","last_change":"2024-04-16T17:47:39.333211+0000","last_active":"2024-04-16T19:32:01.146395+0000","last_peered":"2024-04-16T19:32:01.146395+0000","last_clean":"2024-04-16T19:32:01.146395+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:32:01.146395+0000","last_undegraded":"2024-04-16T19:32:01.146395+0000","last_fullsized":"2024-04-16T19:32:01.146395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174179+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:33:46.174179+0000","last_peered":"2024-04-16T19:33:46.174179+0000","last_clean":"2024-04-16T19:33:46.174179+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:33:46.174179+0000","last_undegraded":"2024-04-16T19:33:46.174179+0000","last_fullsized":"2024-04-16T19:33:46.174179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154341+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:32:21.154341+0000","last_peered":"2024-04-16T19:32:21.154341+0000","last_clean":"2024-04-16T19:32:21.154341+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:32:21.154341+0000","last_undegraded":"2024-04-16T19:32:21.154341+0000","last_fullsized":"2024-04-16T19:32:21.154341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"48'5","reported_seq":359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167357+0000","last_change":"2024-04-16T17:47:33.310538+0000","last_active":"2024-04-16T19:33:11.167357+0000","last_peered":"2024-04-16T19:33:11.167357+0000","last_clean":"2024-04-16T19:33:11.167357+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:33:11.167357+0000","last_undegraded":"2024-04-16T19:33:11.167357+0000","last_fullsized":"2024-04-16T19:33:11.167357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'72","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361573+0000","last_change":"2024-04-16T17:47:35.305717+0000","last_active":"2024-04-16T19:31:31.361573+0000","last_peered":"2024-04-16T19:31:31.361573+0000","last_clean":"2024-04-16T19:31:31.361573+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:31:31.361573+0000","last_undegraded":"2024-04-16T19:31:31.361573+0000","last_fullsized":"2024-04-16T19:31:31.361573+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:27.152861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140105+0000","last_change":"2024-04-16T17:47:37.326918+0000","last_active":"2024-04-16T19:31:36.140105+0000","last_peered":"2024-04-16T19:31:36.140105+0000","last_clean":"2024-04-16T19:31:36.140105+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:31:36.140105+0000","last_undegraded":"2024-04-16T19:31:36.140105+0000","last_fullsized":"2024-04-16T19:31:36.140105+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:19.938869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166042+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:33:21.166042+0000","last_peered":"2024-04-16T19:33:21.166042+0000","last_clean":"2024-04-16T19:33:21.166042+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:33:21.166042+0000","last_undegraded":"2024-04-16T19:33:21.166042+0000","last_fullsized":"2024-04-16T19:33:21.166042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:35.987275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'56","reported_seq":876,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173425+0000","last_change":"2024-04-16T17:48:34.333949+0000","last_active":"2024-04-16T19:33:36.173425+0000","last_peered":"2024-04-16T19:33:36.173425+0000","last_clean":"2024-04-16T19:33:36.173425+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:33:36.173425+0000","last_undegraded":"2024-04-16T19:33:36.173425+0000","last_fullsized":"2024-04-16T19:33:36.173425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156598+0000","last_change":"2024-04-16T17:48:27.299066+0000","last_active":"2024-04-16T19:32:41.156598+0000","last_peered":"2024-04-16T19:32:41.156598+0000","last_clean":"2024-04-16T19:32:41.156598+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:32:41.156598+0000","last_undegraded":"2024-04-16T19:32:41.156598+0000","last_fullsized":"2024-04-16T19:32:41.156598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:12.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'16","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155319+0000","last_change":"2024-04-16T17:48:01.239365+0000","last_active":"2024-04-16T19:32:36.155319+0000","last_peered":"2024-04-16T19:32:36.155319+0000","last_clean":"2024-04-16T19:32:36.155319+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:32:36.155319+0000","last_undegraded":"2024-04-16T19:32:36.155319+0000","last_fullsized":"2024-04-16T19:32:36.155319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:31.169600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156956+0000","last_change":"2024-04-16T17:47:59.192949+0000","last_active":"2024-04-16T19:32:31.156956+0000","last_peered":"2024-04-16T19:32:31.156956+0000","last_clean":"2024-04-16T19:32:31.156956+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:32:31.156956+0000","last_undegraded":"2024-04-16T19:32:31.156956+0000","last_fullsized":"2024-04-16T19:32:31.156956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3595","reported_seq":7189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132670+0000","last_change":"2024-04-16T17:47:57.176567+0000","last_active":"2024-04-16T19:31:56.132670+0000","last_peered":"2024-04-16T19:31:56.132670+0000","last_clean":"2024-04-16T19:31:56.132670+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:31:56.132670+0000","last_undegraded":"2024-04-16T19:31:56.132670+0000","last_fullsized":"2024-04-16T19:31:56.132670+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1795,"log_dups_size":1800,"ondisk_log_size":1795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:10.013659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'9","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163598+0000","last_change":"2024-04-16T17:47:55.178252+0000","last_active":"2024-04-16T19:32:56.163598+0000","last_peered":"2024-04-16T19:32:56.163598+0000","last_clean":"2024-04-16T19:32:56.163598+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:32:56.163598+0000","last_undegraded":"2024-04-16T19:32:56.163598+0000","last_fullsized":"2024-04-16T19:32:56.163598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159596+0000","last_change":"2024-04-16T17:47:39.334187+0000","last_active":"2024-04-16T19:32:41.159596+0000","last_peered":"2024-04-16T19:32:41.159596+0000","last_clean":"2024-04-16T19:32:41.159596+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:32:41.159596+0000","last_undegraded":"2024-04-16T19:32:41.159596+0000","last_fullsized":"2024-04-16T19:32:41.159596+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176178+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:33:46.176178+0000","last_peered":"2024-04-16T19:33:46.176178+0000","last_clean":"2024-04-16T19:33:46.176178+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:33:46.176178+0000","last_undegraded":"2024-04-16T19:33:46.176178+0000","last_fullsized":"2024-04-16T19:33:46.176178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156989+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:32:21.156989+0000","last_peered":"2024-04-16T19:32:21.156989+0000","last_clean":"2024-04-16T19:32:21.156989+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:32:21.156989+0000","last_undegraded":"2024-04-16T19:32:21.156989+0000","last_fullsized":"2024-04-16T19:32:21.156989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"48'11","reported_seq":605,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.343972+0000","last_change":"2024-04-16T17:47:33.300240+0000","last_active":"2024-04-16T19:31:31.343972+0000","last_peered":"2024-04-16T19:31:31.343972+0000","last_clean":"2024-04-16T19:31:31.343972+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:31:31.343972+0000","last_undegraded":"2024-04-16T19:31:31.343972+0000","last_fullsized":"2024-04-16T19:31:31.343972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:30.896735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'120","reported_seq":290,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.149082+0000","last_change":"2024-04-16T17:47:35.314675+0000","last_active":"2024-04-16T19:33:06.149082+0000","last_peered":"2024-04-16T19:33:06.149082+0000","last_clean":"2024-04-16T19:33:06.149082+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:33:06.149082+0000","last_undegraded":"2024-04-16T19:33:06.149082+0000","last_fullsized":"2024-04-16T19:33:06.149082+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:43.962803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'12","reported_seq":1332,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144472+0000","last_change":"2024-04-16T17:47:37.333045+0000","last_active":"2024-04-16T19:31:36.144472+0000","last_peered":"2024-04-16T19:31:36.144472+0000","last_clean":"2024-04-16T19:31:36.144472+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:31:36.144472+0000","last_undegraded":"2024-04-16T19:31:36.144472+0000","last_fullsized":"2024-04-16T19:31:36.144472+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:40.632643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170471+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:33:21.170471+0000","last_peered":"2024-04-16T19:33:21.170471+0000","last_clean":"2024-04-16T19:33:21.170471+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:33:21.170471+0000","last_undegraded":"2024-04-16T19:33:21.170471+0000","last_fullsized":"2024-04-16T19:33:21.170471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:20.115696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173520+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:33:36.173520+0000","last_peered":"2024-04-16T19:33:36.173520+0000","last_clean":"2024-04-16T19:33:36.173520+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:33:36.173520+0000","last_undegraded":"2024-04-16T19:33:36.173520+0000","last_fullsized":"2024-04-16T19:33:36.173520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:58.360796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.162482+0000","last_change":"2024-04-16T17:48:27.297318+0000","last_active":"2024-04-16T19:33:06.162482+0000","last_peered":"2024-04-16T19:33:06.162482+0000","last_clean":"2024-04-16T19:33:06.162482+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:33:06.162482+0000","last_undegraded":"2024-04-16T19:33:06.162482+0000","last_fullsized":"2024-04-16T19:33:06.162482+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:20:02.104358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'19","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154557+0000","last_change":"2024-04-16T17:48:01.256079+0000","last_active":"2024-04-16T19:32:11.154557+0000","last_peered":"2024-04-16T19:32:11.154557+0000","last_clean":"2024-04-16T19:32:11.154557+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:32:11.154557+0000","last_undegraded":"2024-04-16T19:32:11.154557+0000","last_fullsized":"2024-04-16T19:32:11.154557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132631+0000","last_change":"2024-04-16T17:47:59.183129+0000","last_active":"2024-04-16T19:31:56.132631+0000","last_peered":"2024-04-16T19:31:56.132631+0000","last_clean":"2024-04-16T19:31:56.132631+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:31:56.132631+0000","last_undegraded":"2024-04-16T19:31:56.132631+0000","last_fullsized":"2024-04-16T19:31:56.132631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:48.953458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'385","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156502+0000","last_change":"2024-04-16T17:47:57.184276+0000","last_active":"2024-04-16T19:32:26.156502+0000","last_peered":"2024-04-16T19:32:26.156502+0000","last_clean":"2024-04-16T19:32:26.156502+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:32:26.156502+0000","last_undegraded":"2024-04-16T19:32:26.156502+0000","last_fullsized":"2024-04-16T19:32:26.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:59.219214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'6","reported_seq":173,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164223+0000","last_change":"2024-04-16T17:47:55.161902+0000","last_active":"2024-04-16T19:32:56.164223+0000","last_peered":"2024-04-16T19:32:56.164223+0000","last_clean":"2024-04-16T19:32:56.164223+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:32:56.164223+0000","last_undegraded":"2024-04-16T19:32:56.164223+0000","last_fullsized":"2024-04-16T19:32:56.164223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:47.909979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156406+0000","last_change":"2024-04-16T17:47:39.699810+0000","last_active":"2024-04-16T19:32:41.156406+0000","last_peered":"2024-04-16T19:32:41.156406+0000","last_clean":"2024-04-16T19:32:41.156406+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:32:41.156406+0000","last_undegraded":"2024-04-16T19:32:41.156406+0000","last_fullsized":"2024-04-16T19:32:41.156406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:40.561497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176260+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:33:46.176260+0000","last_peered":"2024-04-16T19:33:46.176260+0000","last_clean":"2024-04-16T19:33:46.176260+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:33:46.176260+0000","last_undegraded":"2024-04-16T19:33:46.176260+0000","last_fullsized":"2024-04-16T19:33:46.176260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151772+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:32:21.151772+0000","last_peered":"2024-04-16T19:32:21.151772+0000","last_clean":"2024-04-16T19:32:21.151772+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:32:21.151772+0000","last_undegraded":"2024-04-16T19:32:21.151772+0000","last_fullsized":"2024-04-16T19:32:21.151772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374413+0000","last_change":"2024-04-16T17:47:33.305749+0000","last_active":"2024-04-16T19:31:31.374413+0000","last_peered":"2024-04-16T19:31:31.374413+0000","last_clean":"2024-04-16T19:31:31.374413+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:31:31.374413+0000","last_undegraded":"2024-04-16T19:31:31.374413+0000","last_fullsized":"2024-04-16T19:31:31.374413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:15.902753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374347+0000","last_change":"2024-04-16T17:47:35.314832+0000","last_active":"2024-04-16T19:31:31.374347+0000","last_peered":"2024-04-16T19:31:31.374347+0000","last_clean":"2024-04-16T19:31:31.374347+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:31:31.374347+0000","last_undegraded":"2024-04-16T19:31:31.374347+0000","last_fullsized":"2024-04-16T19:31:31.374347+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:36.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'12","reported_seq":1334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168147+0000","last_change":"2024-04-16T17:47:37.334246+0000","last_active":"2024-04-16T19:33:11.168147+0000","last_peered":"2024-04-16T19:33:11.168147+0000","last_clean":"2024-04-16T19:33:11.168147+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:33:11.168147+0000","last_undegraded":"2024-04-16T19:33:11.168147+0000","last_fullsized":"2024-04-16T19:33:11.168147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:21.558307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147648+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:32:06.147648+0000","last_peered":"2024-04-16T19:32:06.147648+0000","last_clean":"2024-04-16T19:32:06.147648+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:32:06.147648+0000","last_undegraded":"2024-04-16T19:32:06.147648+0000","last_fullsized":"2024-04-16T19:32:06.147648+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:13:48.347861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'164","reported_seq":1853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163697+0000","last_change":"2024-04-16T17:48:34.332673+0000","last_active":"2024-04-16T19:32:46.163697+0000","last_peered":"2024-04-16T19:32:46.163697+0000","last_clean":"2024-04-16T19:32:46.163697+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:32:46.163697+0000","last_undegraded":"2024-04-16T19:32:46.163697+0000","last_fullsized":"2024-04-16T19:32:46.163697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162327+0000","last_change":"2024-04-16T17:48:27.306930+0000","last_active":"2024-04-16T19:32:41.162327+0000","last_peered":"2024-04-16T19:32:41.162327+0000","last_clean":"2024-04-16T19:32:41.162327+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:32:41.162327+0000","last_undegraded":"2024-04-16T19:32:41.162327+0000","last_fullsized":"2024-04-16T19:32:41.162327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'10","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.154948+0000","last_change":"2024-04-16T17:48:01.252700+0000","last_active":"2024-04-16T19:32:36.154948+0000","last_peered":"2024-04-16T19:32:36.154948+0000","last_clean":"2024-04-16T19:32:36.154948+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:32:36.154948+0000","last_undegraded":"2024-04-16T19:32:36.154948+0000","last_fullsized":"2024-04-16T19:32:36.154948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:40.367478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146809+0000","last_change":"2024-04-16T17:47:59.200630+0000","last_active":"2024-04-16T19:32:01.146809+0000","last_peered":"2024-04-16T19:32:01.146809+0000","last_clean":"2024-04-16T19:32:01.146809+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:32:01.146809+0000","last_undegraded":"2024-04-16T19:32:01.146809+0000","last_fullsized":"2024-04-16T19:32:01.146809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1087","reported_seq":1742,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146730+0000","last_change":"2024-04-16T17:47:57.186883+0000","last_active":"2024-04-16T19:32:01.146730+0000","last_peered":"2024-04-16T19:32:01.146730+0000","last_clean":"2024-04-16T19:32:01.146730+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:32:01.146730+0000","last_undegraded":"2024-04-16T19:32:01.146730+0000","last_fullsized":"2024-04-16T19:32:01.146730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:14.402693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"51'4","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159908+0000","last_change":"2024-04-16T17:47:55.177129+0000","last_active":"2024-04-16T19:32:41.159908+0000","last_peered":"2024-04-16T19:32:41.159908+0000","last_clean":"2024-04-16T19:32:41.159908+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:32:41.159908+0000","last_undegraded":"2024-04-16T19:32:41.159908+0000","last_fullsized":"2024-04-16T19:32:41.159908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:31.401461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176098+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:33:36.176098+0000","last_peered":"2024-04-16T19:33:36.176098+0000","last_clean":"2024-04-16T19:33:36.176098+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:33:36.176098+0000","last_undegraded":"2024-04-16T19:33:36.176098+0000","last_fullsized":"2024-04-16T19:33:36.176098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.170137+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:33:31.170137+0000","last_peered":"2024-04-16T19:33:31.170137+0000","last_clean":"2024-04-16T19:33:31.170137+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:33:31.170137+0000","last_undegraded":"2024-04-16T19:33:31.170137+0000","last_fullsized":"2024-04-16T19:33:31.170137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.173925+0000","last_change":"2024-04-16T17:47:33.309390+0000","last_active":"2024-04-16T19:33:46.173925+0000","last_peered":"2024-04-16T19:33:46.173925+0000","last_clean":"2024-04-16T19:33:46.173925+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:33:46.173925+0000","last_undegraded":"2024-04-16T19:33:46.173925+0000","last_fullsized":"2024-04-16T19:33:46.173925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'200","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146472+0000","last_change":"2024-04-16T17:47:35.314414+0000","last_active":"2024-04-16T19:32:01.146472+0000","last_peered":"2024-04-16T19:32:01.146472+0000","last_clean":"2024-04-16T19:32:01.146472+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:32:01.146472+0000","last_undegraded":"2024-04-16T19:32:01.146472+0000","last_fullsized":"2024-04-16T19:32:01.146472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171681+0000","last_change":"2024-04-16T17:47:37.335122+0000","last_active":"2024-04-16T19:33:26.171681+0000","last_peered":"2024-04-16T19:33:26.171681+0000","last_clean":"2024-04-16T19:33:26.171681+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:33:26.171681+0000","last_undegraded":"2024-04-16T19:33:26.171681+0000","last_fullsized":"2024-04-16T19:33:26.171681+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:38.549732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176783+0000","last_change":"2024-04-16T17:47:39.339833+0000","last_active":"2024-04-16T19:33:56.176783+0000","last_peered":"2024-04-16T19:33:56.176783+0000","last_clean":"2024-04-16T19:33:56.176783+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:33:56.176783+0000","last_undegraded":"2024-04-16T19:33:56.176783+0000","last_fullsized":"2024-04-16T19:33:56.176783+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135246+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:32:06.135246+0000","last_peered":"2024-04-16T19:32:06.135246+0000","last_clean":"2024-04-16T19:32:06.135246+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:32:06.135246+0000","last_undegraded":"2024-04-16T19:32:06.135246+0000","last_fullsized":"2024-04-16T19:32:06.135246+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:13:30.436128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1187,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161075+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:32:46.161075+0000","last_peered":"2024-04-16T19:32:46.161075+0000","last_clean":"2024-04-16T19:32:46.161075+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:32:46.161075+0000","last_undegraded":"2024-04-16T19:32:46.161075+0000","last_fullsized":"2024-04-16T19:32:46.161075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168743+0000","last_change":"2024-04-16T17:48:27.299718+0000","last_active":"2024-04-16T19:33:06.168743+0000","last_peered":"2024-04-16T19:33:06.168743+0000","last_clean":"2024-04-16T19:33:06.168743+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:33:06.168743+0000","last_undegraded":"2024-04-16T19:33:06.168743+0000","last_fullsized":"2024-04-16T19:33:06.168743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'2","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154067+0000","last_change":"2024-04-16T17:48:01.253979+0000","last_active":"2024-04-16T19:32:11.154067+0000","last_peered":"2024-04-16T19:32:11.154067+0000","last_clean":"2024-04-16T19:32:11.154067+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:32:11.154067+0000","last_undegraded":"2024-04-16T19:32:11.154067+0000","last_fullsized":"2024-04-16T19:32:11.154067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149567+0000","last_change":"2024-04-16T17:47:59.193745+0000","last_active":"2024-04-16T19:32:01.149567+0000","last_peered":"2024-04-16T19:32:01.149567+0000","last_clean":"2024-04-16T19:32:01.149567+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:32:01.149567+0000","last_undegraded":"2024-04-16T19:32:01.149567+0000","last_fullsized":"2024-04-16T19:32:01.149567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1173","reported_seq":1516,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151419+0000","last_change":"2024-04-16T17:47:57.191812+0000","last_active":"2024-04-16T19:32:01.151419+0000","last_peered":"2024-04-16T19:32:01.151419+0000","last_clean":"2024-04-16T19:32:01.151419+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:32:01.151419+0000","last_undegraded":"2024-04-16T19:32:01.151419+0000","last_fullsized":"2024-04-16T19:32:01.151419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1173,"log_dups_size":0,"ondisk_log_size":1173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'14","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151697+0000","last_change":"2024-04-16T17:47:55.163005+0000","last_active":"2024-04-16T19:32:06.151697+0000","last_peered":"2024-04-16T19:32:06.151697+0000","last_clean":"2024-04-16T19:32:06.151697+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:32:06.151697+0000","last_undegraded":"2024-04-16T19:32:06.151697+0000","last_fullsized":"2024-04-16T19:32:06.151697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169773+0000","last_change":"2024-04-16T17:47:23.266202+0000","last_active":"2024-04-16T19:33:36.169773+0000","last_peered":"2024-04-16T19:33:36.169773+0000","last_clean":"2024-04-16T19:33:36.169773+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:33:36.169773+0000","last_undegraded":"2024-04-16T19:33:36.169773+0000","last_fullsized":"2024-04-16T19:33:36.169773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163890+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:33:06.163890+0000","last_peered":"2024-04-16T19:33:06.163890+0000","last_clean":"2024-04-16T19:33:06.163890+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:33:06.163890+0000","last_undegraded":"2024-04-16T19:33:06.163890+0000","last_fullsized":"2024-04-16T19:33:06.163890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:45.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"31'1","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158405+0000","last_change":"2024-04-16T17:47:33.302299+0000","last_active":"2024-04-16T19:33:46.158405+0000","last_peered":"2024-04-16T19:33:46.158405+0000","last_clean":"2024-04-16T19:33:46.158405+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:33:46.158405+0000","last_undegraded":"2024-04-16T19:33:46.158405+0000","last_fullsized":"2024-04-16T19:33:46.158405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:23.313435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'80","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170395+0000","last_change":"2024-04-16T17:47:35.321698+0000","last_active":"2024-04-16T19:33:21.170395+0000","last_peered":"2024-04-16T19:33:21.170395+0000","last_clean":"2024-04-16T19:33:21.170395+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:33:21.170395+0000","last_undegraded":"2024-04-16T19:33:21.170395+0000","last_fullsized":"2024-04-16T19:33:21.170395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:02.350103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136445+0000","last_change":"2024-04-16T17:47:37.325461+0000","last_active":"2024-04-16T19:32:11.136445+0000","last_peered":"2024-04-16T19:32:11.136445+0000","last_clean":"2024-04-16T19:32:11.136445+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:32:11.136445+0000","last_undegraded":"2024-04-16T19:32:11.136445+0000","last_fullsized":"2024-04-16T19:32:11.136445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:18.621208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152703+0000","last_change":"2024-04-16T17:47:39.323337+0000","last_active":"2024-04-16T19:32:26.152703+0000","last_peered":"2024-04-16T19:32:26.152703+0000","last_clean":"2024-04-16T19:32:26.152703+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:32:26.152703+0000","last_undegraded":"2024-04-16T19:32:26.152703+0000","last_fullsized":"2024-04-16T19:32:26.152703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:12.655890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152382+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:32:06.152382+0000","last_peered":"2024-04-16T19:32:06.152382+0000","last_clean":"2024-04-16T19:32:06.152382+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:32:06.152382+0000","last_undegraded":"2024-04-16T19:32:06.152382+0000","last_fullsized":"2024-04-16T19:32:06.152382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:58.661997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'70","reported_seq":901,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.148912+0000","last_change":"2024-04-16T17:48:34.345575+0000","last_active":"2024-04-16T19:33:06.148912+0000","last_peered":"2024-04-16T19:33:06.148912+0000","last_clean":"2024-04-16T19:33:06.148912+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:33:06.148912+0000","last_undegraded":"2024-04-16T19:33:06.148912+0000","last_fullsized":"2024-04-16T19:33:06.148912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:51.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159686+0000","last_change":"2024-04-16T17:48:27.295648+0000","last_active":"2024-04-16T19:32:41.159686+0000","last_peered":"2024-04-16T19:32:41.159686+0000","last_clean":"2024-04-16T19:32:41.159686+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:32:41.159686+0000","last_undegraded":"2024-04-16T19:32:41.159686+0000","last_fullsized":"2024-04-16T19:32:41.159686+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154259+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:32:11.154259+0000","last_peered":"2024-04-16T19:32:11.154259+0000","last_clean":"2024-04-16T19:32:11.154259+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:32:11.154259+0000","last_undegraded":"2024-04-16T19:32:11.154259+0000","last_fullsized":"2024-04-16T19:32:11.154259+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:20.025368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.157655+0000","last_change":"2024-04-16T17:47:59.197037+0000","last_active":"2024-04-16T19:32:46.157655+0000","last_peered":"2024-04-16T19:32:46.157655+0000","last_clean":"2024-04-16T19:32:46.157655+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:32:46.157655+0000","last_undegraded":"2024-04-16T19:32:46.157655+0000","last_fullsized":"2024-04-16T19:32:46.157655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:54.818691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1049","reported_seq":6110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152019+0000","last_change":"2024-04-16T17:47:57.189817+0000","last_active":"2024-04-16T19:32:11.152019+0000","last_peered":"2024-04-16T19:32:11.152019+0000","last_clean":"2024-04-16T19:32:11.152019+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:32:11.152019+0000","last_undegraded":"2024-04-16T19:32:11.152019+0000","last_fullsized":"2024-04-16T19:32:11.152019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'4","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145438+0000","last_change":"2024-04-16T17:47:55.182984+0000","last_active":"2024-04-16T19:31:56.145438+0000","last_peered":"2024-04-16T19:31:56.145438+0000","last_clean":"2024-04-16T19:31:56.145438+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:31:56.145438+0000","last_undegraded":"2024-04-16T19:31:56.145438+0000","last_fullsized":"2024-04-16T19:31:56.145438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:55.129164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178050+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:34:01.178050+0000","last_peered":"2024-04-16T19:34:01.178050+0000","last_clean":"2024-04-16T19:34:01.178050+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:34:01.178050+0000","last_undegraded":"2024-04-16T19:34:01.178050+0000","last_fullsized":"2024-04-16T19:34:01.178050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.169031+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:33:06.169031+0000","last_peered":"2024-04-16T19:33:06.169031+0000","last_clean":"2024-04-16T19:33:06.169031+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:33:06.169031+0000","last_undegraded":"2024-04-16T19:33:06.169031+0000","last_fullsized":"2024-04-16T19:33:06.169031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152471+0000","last_change":"2024-04-16T17:47:33.302725+0000","last_active":"2024-04-16T19:33:21.152471+0000","last_peered":"2024-04-16T19:33:21.152471+0000","last_clean":"2024-04-16T19:33:21.152471+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:33:21.152471+0000","last_undegraded":"2024-04-16T19:33:21.152471+0000","last_fullsized":"2024-04-16T19:33:21.152471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:42.697302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'152","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179125+0000","last_change":"2024-04-16T17:47:35.317742+0000","last_active":"2024-04-16T19:33:46.179125+0000","last_peered":"2024-04-16T19:33:46.179125+0000","last_clean":"2024-04-16T19:33:46.179125+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:33:46.179125+0000","last_undegraded":"2024-04-16T19:33:46.179125+0000","last_fullsized":"2024-04-16T19:33:46.179125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'12","reported_seq":1338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177186+0000","last_change":"2024-04-16T17:47:37.318010+0000","last_active":"2024-04-16T19:33:51.177186+0000","last_peered":"2024-04-16T19:33:51.177186+0000","last_clean":"2024-04-16T19:33:51.177186+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:33:51.177186+0000","last_undegraded":"2024-04-16T19:33:51.177186+0000","last_fullsized":"2024-04-16T19:33:51.177186+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177479+0000","last_change":"2024-04-16T17:47:39.328909+0000","last_active":"2024-04-16T19:33:51.177479+0000","last_peered":"2024-04-16T19:33:51.177479+0000","last_clean":"2024-04-16T19:33:51.177479+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:33:51.177479+0000","last_undegraded":"2024-04-16T19:33:51.177479+0000","last_fullsized":"2024-04-16T19:33:51.177479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:54.351646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152331+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:33:21.152331+0000","last_peered":"2024-04-16T19:33:21.152331+0000","last_clean":"2024-04-16T19:33:21.152331+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:33:21.152331+0000","last_undegraded":"2024-04-16T19:33:21.152331+0000","last_fullsized":"2024-04-16T19:33:21.152331+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:16.718022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'74","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161693+0000","last_change":"2024-04-16T17:48:34.327024+0000","last_active":"2024-04-16T19:32:46.161693+0000","last_peered":"2024-04-16T19:32:46.161693+0000","last_clean":"2024-04-16T19:32:46.161693+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:32:46.161693+0000","last_undegraded":"2024-04-16T19:32:46.161693+0000","last_fullsized":"2024-04-16T19:32:46.161693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:00.683240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157359+0000","last_change":"2024-04-16T17:48:27.301787+0000","last_active":"2024-04-16T19:32:41.157359+0000","last_peered":"2024-04-16T19:32:41.157359+0000","last_clean":"2024-04-16T19:32:41.157359+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:32:41.157359+0000","last_undegraded":"2024-04-16T19:32:41.157359+0000","last_fullsized":"2024-04-16T19:32:41.157359+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'17","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.151912+0000","last_change":"2024-04-16T17:48:01.243297+0000","last_active":"2024-04-16T19:32:11.151912+0000","last_peered":"2024-04-16T19:32:11.151912+0000","last_clean":"2024-04-16T19:32:11.151912+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:32:11.151912+0000","last_undegraded":"2024-04-16T19:32:11.151912+0000","last_fullsized":"2024-04-16T19:32:11.151912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149205+0000","last_change":"2024-04-16T17:47:59.193938+0000","last_active":"2024-04-16T19:32:11.149205+0000","last_peered":"2024-04-16T19:32:11.149205+0000","last_clean":"2024-04-16T19:32:11.149205+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:32:11.149205+0000","last_undegraded":"2024-04-16T19:32:11.149205+0000","last_fullsized":"2024-04-16T19:32:11.149205+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'703","reported_seq":1434,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.143345+0000","last_change":"2024-04-16T17:47:57.173194+0000","last_active":"2024-04-16T19:32:41.143345+0000","last_peered":"2024-04-16T19:32:41.143345+0000","last_clean":"2024-04-16T19:32:41.143345+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:32:41.143345+0000","last_undegraded":"2024-04-16T19:32:41.143345+0000","last_fullsized":"2024-04-16T19:32:41.143345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:02.892691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'14","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145388+0000","last_change":"2024-04-16T17:47:55.181545+0000","last_active":"2024-04-16T19:31:56.145388+0000","last_peered":"2024-04-16T19:31:56.145388+0000","last_clean":"2024-04-16T19:31:56.145388+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:31:56.145388+0000","last_undegraded":"2024-04-16T19:31:56.145388+0000","last_fullsized":"2024-04-16T19:31:56.145388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167271+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:33:11.167271+0000","last_peered":"2024-04-16T19:33:11.167271+0000","last_clean":"2024-04-16T19:33:11.167271+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:33:11.167271+0000","last_undegraded":"2024-04-16T19:33:11.167271+0000","last_fullsized":"2024-04-16T19:33:11.167271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163761+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:33:06.163761+0000","last_peered":"2024-04-16T19:33:06.163761+0000","last_clean":"2024-04-16T19:33:06.163761+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:33:06.163761+0000","last_undegraded":"2024-04-16T19:33:06.163761+0000","last_fullsized":"2024-04-16T19:33:06.163761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135219+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:32:06.135219+0000","last_peered":"2024-04-16T19:32:06.135219+0000","last_clean":"2024-04-16T19:32:06.135219+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:32:06.135219+0000","last_undegraded":"2024-04-16T19:32:06.135219+0000","last_fullsized":"2024-04-16T19:32:06.135219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:21.347240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'88","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175899+0000","last_change":"2024-04-16T17:47:35.317589+0000","last_active":"2024-04-16T19:33:46.175899+0000","last_peered":"2024-04-16T19:33:46.175899+0000","last_clean":"2024-04-16T19:33:46.175899+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:33:46.175899+0000","last_undegraded":"2024-04-16T19:33:46.175899+0000","last_fullsized":"2024-04-16T19:33:46.175899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159758+0000","last_change":"2024-04-16T17:47:37.330541+0000","last_active":"2024-04-16T19:33:51.159758+0000","last_peered":"2024-04-16T19:33:51.159758+0000","last_clean":"2024-04-16T19:33:51.159758+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:33:51.159758+0000","last_undegraded":"2024-04-16T19:33:51.159758+0000","last_fullsized":"2024-04-16T19:33:51.159758+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:07.207646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178646+0000","last_change":"2024-04-16T17:47:39.338681+0000","last_active":"2024-04-16T19:33:51.178646+0000","last_peered":"2024-04-16T19:33:51.178646+0000","last_clean":"2024-04-16T19:33:51.178646+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:33:51.178646+0000","last_undegraded":"2024-04-16T19:33:51.178646+0000","last_fullsized":"2024-04-16T19:33:51.178646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168291+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:33:11.168291+0000","last_peered":"2024-04-16T19:33:11.168291+0000","last_clean":"2024-04-16T19:33:11.168291+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:33:11.168291+0000","last_undegraded":"2024-04-16T19:33:11.168291+0000","last_fullsized":"2024-04-16T19:33:11.168291+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:26.664630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'40","reported_seq":973,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158409+0000","last_change":"2024-04-16T17:48:34.333313+0000","last_active":"2024-04-16T19:32:36.158409+0000","last_peered":"2024-04-16T19:32:36.158409+0000","last_clean":"2024-04-16T19:32:36.158409+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:32:36.158409+0000","last_undegraded":"2024-04-16T19:32:36.158409+0000","last_fullsized":"2024-04-16T19:32:36.158409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:06:58.890753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164925+0000","last_change":"2024-04-16T17:48:27.304408+0000","last_active":"2024-04-16T19:33:16.164925+0000","last_peered":"2024-04-16T19:33:16.164925+0000","last_clean":"2024-04-16T19:33:16.164925+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:33:16.164925+0000","last_undegraded":"2024-04-16T19:33:16.164925+0000","last_fullsized":"2024-04-16T19:33:16.164925+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:27:10.178784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'9","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149161+0000","last_change":"2024-04-16T17:48:01.259988+0000","last_active":"2024-04-16T19:32:11.149161+0000","last_peered":"2024-04-16T19:32:11.149161+0000","last_clean":"2024-04-16T19:32:11.149161+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:32:11.149161+0000","last_undegraded":"2024-04-16T19:32:11.149161+0000","last_fullsized":"2024-04-16T19:32:11.149161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'555","reported_seq":2287,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154329+0000","last_change":"2024-04-16T17:47:59.181942+0000","last_active":"2024-04-16T19:32:11.154329+0000","last_peered":"2024-04-16T19:32:11.154329+0000","last_clean":"2024-04-16T19:32:11.154329+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:32:11.154329+0000","last_undegraded":"2024-04-16T19:32:11.154329+0000","last_fullsized":"2024-04-16T19:32:11.154329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:25.688721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'901","reported_seq":1313,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152857+0000","last_change":"2024-04-16T17:47:57.174998+0000","last_active":"2024-04-16T19:32:11.152857+0000","last_peered":"2024-04-16T19:32:11.152857+0000","last_clean":"2024-04-16T19:32:11.152857+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:32:11.152857+0000","last_undegraded":"2024-04-16T19:32:11.152857+0000","last_fullsized":"2024-04-16T19:32:11.152857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:14.325870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'14","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150617+0000","last_change":"2024-04-16T17:47:55.155349+0000","last_active":"2024-04-16T19:32:06.150617+0000","last_peered":"2024-04-16T19:32:06.150617+0000","last_clean":"2024-04-16T19:32:06.150617+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:32:06.150617+0000","last_undegraded":"2024-04-16T19:32:06.150617+0000","last_fullsized":"2024-04-16T19:32:06.150617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163638+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:33:11.163638+0000","last_peered":"2024-04-16T19:33:11.163638+0000","last_clean":"2024-04-16T19:33:11.163638+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:33:11.163638+0000","last_undegraded":"2024-04-16T19:33:11.163638+0000","last_fullsized":"2024-04-16T19:33:11.163638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166835+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:33:06.166835+0000","last_peered":"2024-04-16T19:33:06.166835+0000","last_clean":"2024-04-16T19:33:06.166835+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:33:06.166835+0000","last_undegraded":"2024-04-16T19:33:06.166835+0000","last_fullsized":"2024-04-16T19:33:06.166835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":175,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152538+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:33:21.152538+0000","last_peered":"2024-04-16T19:33:21.152538+0000","last_clean":"2024-04-16T19:33:21.152538+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:33:21.152538+0000","last_undegraded":"2024-04-16T19:33:21.152538+0000","last_fullsized":"2024-04-16T19:33:21.152538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:05.646107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'104","reported_seq":273,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166177+0000","last_change":"2024-04-16T17:47:35.307146+0000","last_active":"2024-04-16T19:33:21.166177+0000","last_peered":"2024-04-16T19:33:21.166177+0000","last_clean":"2024-04-16T19:33:21.166177+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:33:21.166177+0000","last_undegraded":"2024-04-16T19:33:21.166177+0000","last_fullsized":"2024-04-16T19:33:21.166177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:28.729299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145847+0000","last_change":"2024-04-16T17:47:37.324514+0000","last_active":"2024-04-16T19:31:46.145847+0000","last_peered":"2024-04-16T19:31:46.145847+0000","last_clean":"2024-04-16T19:31:46.145847+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:31:46.145847+0000","last_undegraded":"2024-04-16T19:31:46.145847+0000","last_fullsized":"2024-04-16T19:31:46.145847+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183327+0000","last_change":"2024-04-16T17:47:39.340900+0000","last_active":"2024-04-16T19:34:11.183327+0000","last_peered":"2024-04-16T19:34:11.183327+0000","last_clean":"2024-04-16T19:34:11.183327+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:34:11.183327+0000","last_undegraded":"2024-04-16T19:34:11.183327+0000","last_fullsized":"2024-04-16T19:34:11.183327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.644408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167437+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:33:11.167437+0000","last_peered":"2024-04-16T19:33:11.167437+0000","last_clean":"2024-04-16T19:33:11.167437+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:33:11.167437+0000","last_undegraded":"2024-04-16T19:33:11.167437+0000","last_fullsized":"2024-04-16T19:33:11.167437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156402+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:32:36.156402+0000","last_peered":"2024-04-16T19:32:36.156402+0000","last_clean":"2024-04-16T19:32:36.156402+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:32:36.156402+0000","last_undegraded":"2024-04-16T19:32:36.156402+0000","last_fullsized":"2024-04-16T19:32:36.156402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157486+0000","last_change":"2024-04-16T17:48:27.308234+0000","last_active":"2024-04-16T19:32:41.157486+0000","last_peered":"2024-04-16T19:32:41.157486+0000","last_clean":"2024-04-16T19:32:41.157486+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:32:41.157486+0000","last_undegraded":"2024-04-16T19:32:41.157486+0000","last_fullsized":"2024-04-16T19:32:41.157486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'135","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163865+0000","last_change":"2024-04-16T17:48:01.254153+0000","last_active":"2024-04-16T19:32:46.163865+0000","last_peered":"2024-04-16T19:32:46.163865+0000","last_clean":"2024-04-16T19:32:46.163865+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:32:46.163865+0000","last_undegraded":"2024-04-16T19:32:46.163865+0000","last_fullsized":"2024-04-16T19:32:46.163865+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136391+0000","last_change":"2024-04-16T17:47:59.174180+0000","last_active":"2024-04-16T19:32:11.136391+0000","last_peered":"2024-04-16T19:32:11.136391+0000","last_clean":"2024-04-16T19:32:11.136391+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:32:11.136391+0000","last_undegraded":"2024-04-16T19:32:11.136391+0000","last_fullsized":"2024-04-16T19:32:11.136391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:29.135288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'679","reported_seq":1066,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136485+0000","last_change":"2024-04-16T17:47:57.182491+0000","last_active":"2024-04-16T19:32:11.136485+0000","last_peered":"2024-04-16T19:32:11.136485+0000","last_clean":"2024-04-16T19:32:11.136485+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:32:11.136485+0000","last_undegraded":"2024-04-16T19:32:11.136485+0000","last_fullsized":"2024-04-16T19:32:11.136485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:14.757156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'6","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156433+0000","last_change":"2024-04-16T17:47:55.158851+0000","last_active":"2024-04-16T19:32:26.156433+0000","last_peered":"2024-04-16T19:32:26.156433+0000","last_clean":"2024-04-16T19:32:26.156433+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:32:26.156433+0000","last_undegraded":"2024-04-16T19:32:26.156433+0000","last_fullsized":"2024-04-16T19:32:26.156433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:39:56.856045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170332+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:33:11.170332+0000","last_peered":"2024-04-16T19:33:11.170332+0000","last_clean":"2024-04-16T19:33:11.170332+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:33:11.170332+0000","last_undegraded":"2024-04-16T19:33:11.170332+0000","last_fullsized":"2024-04-16T19:33:11.170332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:31.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.149879+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:31:56.149879+0000","last_peered":"2024-04-16T19:31:56.149879+0000","last_clean":"2024-04-16T19:31:56.149879+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:31:56.149879+0000","last_undegraded":"2024-04-16T19:31:56.149879+0000","last_fullsized":"2024-04-16T19:31:56.149879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:06.495990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"48'13","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170506+0000","last_change":"2024-04-16T17:47:33.309159+0000","last_active":"2024-04-16T19:33:21.170506+0000","last_peered":"2024-04-16T19:33:21.170506+0000","last_clean":"2024-04-16T19:33:21.170506+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:33:21.170506+0000","last_undegraded":"2024-04-16T19:33:21.170506+0000","last_fullsized":"2024-04-16T19:33:21.170506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:26.709263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'152","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160039+0000","last_change":"2024-04-16T17:47:35.305022+0000","last_active":"2024-04-16T19:33:51.160039+0000","last_peered":"2024-04-16T19:33:51.160039+0000","last_clean":"2024-04-16T19:33:51.160039+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:33:51.160039+0000","last_undegraded":"2024-04-16T19:33:51.160039+0000","last_fullsized":"2024-04-16T19:33:51.160039+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:03.908445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.153692+0000","last_change":"2024-04-16T17:47:37.331019+0000","last_active":"2024-04-16T19:33:26.153692+0000","last_peered":"2024-04-16T19:33:26.153692+0000","last_clean":"2024-04-16T19:33:26.153692+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:33:26.153692+0000","last_undegraded":"2024-04-16T19:33:26.153692+0000","last_fullsized":"2024-04-16T19:33:26.153692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:18.849817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179728+0000","last_change":"2024-04-16T17:47:39.331395+0000","last_active":"2024-04-16T19:33:56.179728+0000","last_peered":"2024-04-16T19:33:56.179728+0000","last_clean":"2024-04-16T19:33:56.179728+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:33:56.179728+0000","last_undegraded":"2024-04-16T19:33:56.179728+0000","last_fullsized":"2024-04-16T19:33:56.179728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:12.149771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160965+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:32:36.160965+0000","last_peered":"2024-04-16T19:32:36.160965+0000","last_clean":"2024-04-16T19:32:36.160965+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:32:36.160965+0000","last_undegraded":"2024-04-16T19:32:36.160965+0000","last_fullsized":"2024-04-16T19:32:36.160965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'86","reported_seq":1117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163783+0000","last_change":"2024-04-16T17:48:34.326754+0000","last_active":"2024-04-16T19:32:46.163783+0000","last_peered":"2024-04-16T19:32:46.163783+0000","last_clean":"2024-04-16T19:32:46.163783+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:32:46.163783+0000","last_undegraded":"2024-04-16T19:32:46.163783+0000","last_fullsized":"2024-04-16T19:32:46.163783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159819+0000","last_change":"2024-04-16T17:48:27.298815+0000","last_active":"2024-04-16T19:32:31.159819+0000","last_peered":"2024-04-16T19:32:31.159819+0000","last_clean":"2024-04-16T19:32:31.159819+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:32:31.159819+0000","last_undegraded":"2024-04-16T19:32:31.159819+0000","last_fullsized":"2024-04-16T19:32:31.159819+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"48'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150226+0000","last_change":"2024-04-16T17:48:01.246604+0000","last_active":"2024-04-16T19:32:01.150226+0000","last_peered":"2024-04-16T19:32:01.150226+0000","last_clean":"2024-04-16T19:32:01.150226+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:32:01.150226+0000","last_undegraded":"2024-04-16T19:32:01.150226+0000","last_fullsized":"2024-04-16T19:32:01.150226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:16.195310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148847+0000","last_change":"2024-04-16T17:47:59.180320+0000","last_active":"2024-04-16T19:32:11.148847+0000","last_peered":"2024-04-16T19:32:11.148847+0000","last_clean":"2024-04-16T19:32:11.148847+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:32:11.148847+0000","last_undegraded":"2024-04-16T19:32:11.148847+0000","last_fullsized":"2024-04-16T19:32:11.148847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:16.107500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1483","reported_seq":5477,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153145+0000","last_change":"2024-04-16T17:47:57.186127+0000","last_active":"2024-04-16T19:32:26.153145+0000","last_peered":"2024-04-16T19:32:26.153145+0000","last_clean":"2024-04-16T19:32:26.153145+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:32:26.153145+0000","last_undegraded":"2024-04-16T19:32:26.153145+0000","last_fullsized":"2024-04-16T19:32:26.153145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1483,"log_dups_size":0,"ondisk_log_size":1483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:02.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'8","reported_seq":224,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152628+0000","last_change":"2024-04-16T17:47:55.161973+0000","last_active":"2024-04-16T19:32:06.152628+0000","last_peered":"2024-04-16T19:32:06.152628+0000","last_clean":"2024-04-16T19:32:06.152628+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:32:06.152628+0000","last_undegraded":"2024-04-16T19:32:06.152628+0000","last_fullsized":"2024-04-16T19:32:06.152628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163747+0000","last_change":"2024-04-16T17:47:23.266018+0000","last_active":"2024-04-16T19:33:11.163747+0000","last_peered":"2024-04-16T19:33:11.163747+0000","last_clean":"2024-04-16T19:33:11.163747+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:33:11.163747+0000","last_undegraded":"2024-04-16T19:33:11.163747+0000","last_fullsized":"2024-04-16T19:33:11.163747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168338+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:33:31.168338+0000","last_peered":"2024-04-16T19:33:31.168338+0000","last_clean":"2024-04-16T19:33:31.168338+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:33:31.168338+0000","last_undegraded":"2024-04-16T19:33:31.168338+0000","last_fullsized":"2024-04-16T19:33:31.168338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174370+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:33:46.174370+0000","last_peered":"2024-04-16T19:33:46.174370+0000","last_clean":"2024-04-16T19:33:46.174370+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:33:46.174370+0000","last_undegraded":"2024-04-16T19:33:46.174370+0000","last_fullsized":"2024-04-16T19:33:46.174370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'120","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170532+0000","last_change":"2024-04-16T17:47:35.319180+0000","last_active":"2024-04-16T19:33:21.170532+0000","last_peered":"2024-04-16T19:33:21.170532+0000","last_clean":"2024-04-16T19:33:21.170532+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:33:21.170532+0000","last_undegraded":"2024-04-16T19:33:21.170532+0000","last_fullsized":"2024-04-16T19:33:21.170532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:28.081965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.167247+0000","last_change":"2024-04-16T17:47:37.326971+0000","last_active":"2024-04-16T19:33:26.167247+0000","last_peered":"2024-04-16T19:33:26.167247+0000","last_clean":"2024-04-16T19:33:26.167247+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:33:26.167247+0000","last_undegraded":"2024-04-16T19:33:26.167247+0000","last_fullsized":"2024-04-16T19:33:26.167247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:04.366184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174929+0000","last_change":"2024-04-16T17:47:39.325433+0000","last_active":"2024-04-16T19:33:56.174929+0000","last_peered":"2024-04-16T19:33:56.174929+0000","last_clean":"2024-04-16T19:33:56.174929+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:33:56.174929+0000","last_undegraded":"2024-04-16T19:33:56.174929+0000","last_fullsized":"2024-04-16T19:33:56.174929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:50.427879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151752+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:32:06.151752+0000","last_peered":"2024-04-16T19:32:06.151752+0000","last_clean":"2024-04-16T19:32:06.151752+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:32:06.151752+0000","last_undegraded":"2024-04-16T19:32:06.151752+0000","last_fullsized":"2024-04-16T19:32:06.151752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:24.908570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'90","reported_seq":977,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.169718+0000","last_change":"2024-04-16T17:48:34.333850+0000","last_active":"2024-04-16T19:33:21.169718+0000","last_peered":"2024-04-16T19:33:21.169718+0000","last_clean":"2024-04-16T19:33:21.169718+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:33:21.169718+0000","last_undegraded":"2024-04-16T19:33:21.169718+0000","last_fullsized":"2024-04-16T19:33:21.169718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:31.546580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159785+0000","last_change":"2024-04-16T17:48:27.299707+0000","last_active":"2024-04-16T19:32:31.159785+0000","last_peered":"2024-04-16T19:32:31.159785+0000","last_clean":"2024-04-16T19:32:31.159785+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:32:31.159785+0000","last_undegraded":"2024-04-16T19:32:31.159785+0000","last_fullsized":"2024-04-16T19:32:31.159785+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'5","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150341+0000","last_change":"2024-04-16T17:48:01.258518+0000","last_active":"2024-04-16T19:32:01.150341+0000","last_peered":"2024-04-16T19:32:01.150341+0000","last_clean":"2024-04-16T19:32:01.150341+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:32:01.150341+0000","last_undegraded":"2024-04-16T19:32:01.150341+0000","last_fullsized":"2024-04-16T19:32:01.150341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:25.260545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'555","reported_seq":1941,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157758+0000","last_change":"2024-04-16T17:47:59.184752+0000","last_active":"2024-04-16T19:32:31.157758+0000","last_peered":"2024-04-16T19:32:31.157758+0000","last_clean":"2024-04-16T19:32:31.157758+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:32:31.157758+0000","last_undegraded":"2024-04-16T19:32:31.157758+0000","last_fullsized":"2024-04-16T19:32:31.157758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:36.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'651","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148802+0000","last_change":"2024-04-16T17:47:57.193900+0000","last_active":"2024-04-16T19:32:11.148802+0000","last_peered":"2024-04-16T19:32:11.148802+0000","last_clean":"2024-04-16T19:32:11.148802+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:32:11.148802+0000","last_undegraded":"2024-04-16T19:32:11.148802+0000","last_fullsized":"2024-04-16T19:32:11.148802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:12.186417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'12","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151428+0000","last_change":"2024-04-16T17:47:55.161557+0000","last_active":"2024-04-16T19:32:06.151428+0000","last_peered":"2024-04-16T19:32:06.151428+0000","last_clean":"2024-04-16T19:32:06.151428+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:32:06.151428+0000","last_undegraded":"2024-04-16T19:32:06.151428+0000","last_fullsized":"2024-04-16T19:32:06.151428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:51.923908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.176037+0000","last_change":"2024-04-16T17:47:23.261436+0000","last_active":"2024-04-16T19:33:41.176037+0000","last_peered":"2024-04-16T19:33:41.176037+0000","last_clean":"2024-04-16T19:33:41.176037+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:33:41.176037+0000","last_undegraded":"2024-04-16T19:33:41.176037+0000","last_fullsized":"2024-04-16T19:33:41.176037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154630+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:33:31.154630+0000","last_peered":"2024-04-16T19:33:31.154630+0000","last_clean":"2024-04-16T19:33:31.154630+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:33:31.154630+0000","last_undegraded":"2024-04-16T19:33:31.154630+0000","last_fullsized":"2024-04-16T19:33:31.154630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:35.902429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172709+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:33:21.172709+0000","last_peered":"2024-04-16T19:33:21.172709+0000","last_clean":"2024-04-16T19:33:21.172709+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:33:21.172709+0000","last_undegraded":"2024-04-16T19:33:21.172709+0000","last_fullsized":"2024-04-16T19:33:21.172709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:26.061929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'176","reported_seq":381,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167670+0000","last_change":"2024-04-16T17:47:35.317949+0000","last_active":"2024-04-16T19:33:21.167670+0000","last_peered":"2024-04-16T19:33:21.167670+0000","last_clean":"2024-04-16T19:33:21.167670+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:33:21.167670+0000","last_undegraded":"2024-04-16T19:33:21.167670+0000","last_fullsized":"2024-04-16T19:33:21.167670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.173767+0000","last_change":"2024-04-16T17:47:37.332636+0000","last_active":"2024-04-16T19:33:26.173767+0000","last_peered":"2024-04-16T19:33:26.173767+0000","last_clean":"2024-04-16T19:33:26.173767+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:33:26.173767+0000","last_undegraded":"2024-04-16T19:33:26.173767+0000","last_fullsized":"2024-04-16T19:33:26.173767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179806+0000","last_change":"2024-04-16T17:47:39.335342+0000","last_active":"2024-04-16T19:33:56.179806+0000","last_peered":"2024-04-16T19:33:56.179806+0000","last_clean":"2024-04-16T19:33:56.179806+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:33:56.179806+0000","last_undegraded":"2024-04-16T19:33:56.179806+0000","last_fullsized":"2024-04-16T19:33:56.179806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131831+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:31:51.131831+0000","last_peered":"2024-04-16T19:31:51.131831+0000","last_clean":"2024-04-16T19:31:51.131831+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:31:51.131831+0000","last_undegraded":"2024-04-16T19:31:51.131831+0000","last_fullsized":"2024-04-16T19:31:51.131831+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:43.281248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'118","reported_seq":1502,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156628+0000","last_change":"2024-04-16T17:48:34.329015+0000","last_active":"2024-04-16T19:32:41.156628+0000","last_peered":"2024-04-16T19:32:41.156628+0000","last_clean":"2024-04-16T19:32:41.156628+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:32:41.156628+0000","last_undegraded":"2024-04-16T19:32:41.156628+0000","last_fullsized":"2024-04-16T19:32:41.156628+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:38.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150062+0000","last_change":"2024-04-16T17:48:27.296987+0000","last_active":"2024-04-16T19:32:01.150062+0000","last_peered":"2024-04-16T19:32:01.150062+0000","last_clean":"2024-04-16T19:32:01.150062+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:32:01.150062+0000","last_undegraded":"2024-04-16T19:32:01.150062+0000","last_fullsized":"2024-04-16T19:32:01.150062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:13.235223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'8","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172452+0000","last_change":"2024-04-16T17:48:01.240484+0000","last_active":"2024-04-16T19:33:46.172452+0000","last_peered":"2024-04-16T19:33:46.172452+0000","last_clean":"2024-04-16T19:33:46.172452+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:33:46.172452+0000","last_undegraded":"2024-04-16T19:33:46.172452+0000","last_fullsized":"2024-04-16T19:33:46.172452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:38.257398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157906+0000","last_change":"2024-04-16T17:47:59.184121+0000","last_active":"2024-04-16T19:32:31.157906+0000","last_peered":"2024-04-16T19:32:31.157906+0000","last_clean":"2024-04-16T19:32:31.157906+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:32:31.157906+0000","last_undegraded":"2024-04-16T19:32:31.157906+0000","last_fullsized":"2024-04-16T19:32:31.157906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:46.708915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'783","reported_seq":43142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174072+0000","last_change":"2024-04-16T17:47:57.186095+0000","last_active":"2024-04-16T19:33:46.174072+0000","last_peered":"2024-04-16T19:33:46.174072+0000","last_clean":"2024-04-16T19:33:46.174072+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:33:46.174072+0000","last_undegraded":"2024-04-16T19:33:46.174072+0000","last_fullsized":"2024-04-16T19:33:46.174072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:50.249328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'9","reported_seq":208,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132707+0000","last_change":"2024-04-16T17:47:55.143458+0000","last_active":"2024-04-16T19:31:56.132707+0000","last_peered":"2024-04-16T19:31:56.132707+0000","last_clean":"2024-04-16T19:31:56.132707+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:31:56.132707+0000","last_undegraded":"2024-04-16T19:31:56.132707+0000","last_fullsized":"2024-04-16T19:31:56.132707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:53:52.793161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154404+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:32:11.154404+0000","last_peered":"2024-04-16T19:32:11.154404+0000","last_clean":"2024-04-16T19:32:11.154404+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:32:11.154404+0000","last_undegraded":"2024-04-16T19:32:11.154404+0000","last_fullsized":"2024-04-16T19:32:11.154404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:43.276218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171281+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:33:16.171281+0000","last_peered":"2024-04-16T19:33:16.171281+0000","last_clean":"2024-04-16T19:33:16.171281+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:33:16.171281+0000","last_undegraded":"2024-04-16T19:33:16.171281+0000","last_fullsized":"2024-04-16T19:33:16.171281+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174137+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:33:36.174137+0000","last_peered":"2024-04-16T19:33:36.174137+0000","last_clean":"2024-04-16T19:33:36.174137+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:33:36.174137+0000","last_undegraded":"2024-04-16T19:33:36.174137+0000","last_fullsized":"2024-04-16T19:33:36.174137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:33:14.972217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173777+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:33:51.173777+0000","last_peered":"2024-04-16T19:33:51.173777+0000","last_clean":"2024-04-16T19:33:51.173777+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:33:51.173777+0000","last_undegraded":"2024-04-16T19:33:51.173777+0000","last_fullsized":"2024-04-16T19:33:51.173777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:14.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'72","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174316+0000","last_change":"2024-04-16T17:47:35.310088+0000","last_active":"2024-04-16T19:33:46.174316+0000","last_peered":"2024-04-16T19:33:46.174316+0000","last_clean":"2024-04-16T19:33:46.174316+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:33:46.174316+0000","last_undegraded":"2024-04-16T19:33:46.174316+0000","last_fullsized":"2024-04-16T19:33:46.174316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177457+0000","last_change":"2024-04-16T17:47:37.327822+0000","last_active":"2024-04-16T19:33:51.177457+0000","last_peered":"2024-04-16T19:33:51.177457+0000","last_clean":"2024-04-16T19:33:51.177457+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:33:51.177457+0000","last_undegraded":"2024-04-16T19:33:51.177457+0000","last_fullsized":"2024-04-16T19:33:51.177457+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175307+0000","last_change":"2024-04-16T17:47:39.341107+0000","last_active":"2024-04-16T19:33:51.175307+0000","last_peered":"2024-04-16T19:33:51.175307+0000","last_clean":"2024-04-16T19:33:51.175307+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:33:51.175307+0000","last_undegraded":"2024-04-16T19:33:51.175307+0000","last_fullsized":"2024-04-16T19:33:51.175307+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157259+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:32:21.157259+0000","last_peered":"2024-04-16T19:32:21.157259+0000","last_clean":"2024-04-16T19:32:21.157259+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:32:21.157259+0000","last_undegraded":"2024-04-16T19:32:21.157259+0000","last_fullsized":"2024-04-16T19:32:21.157259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'100","reported_seq":1745,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147877+0000","last_change":"2024-04-16T17:48:34.333902+0000","last_active":"2024-04-16T19:32:06.147877+0000","last_peered":"2024-04-16T19:32:06.147877+0000","last_clean":"2024-04-16T19:32:06.147877+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:32:06.147877+0000","last_undegraded":"2024-04-16T19:32:06.147877+0000","last_fullsized":"2024-04-16T19:32:06.147877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:26.562009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146546+0000","last_change":"2024-04-16T17:48:27.296497+0000","last_active":"2024-04-16T19:32:01.146546+0000","last_peered":"2024-04-16T19:32:01.146546+0000","last_clean":"2024-04-16T19:32:01.146546+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:32:01.146546+0000","last_undegraded":"2024-04-16T19:32:01.146546+0000","last_fullsized":"2024-04-16T19:32:01.146546+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:59:05.786900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'8","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176076+0000","last_change":"2024-04-16T17:48:01.261918+0000","last_active":"2024-04-16T19:33:46.176076+0000","last_peered":"2024-04-16T19:33:46.176076+0000","last_clean":"2024-04-16T19:33:46.176076+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:33:46.176076+0000","last_undegraded":"2024-04-16T19:33:46.176076+0000","last_fullsized":"2024-04-16T19:33:46.176076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158533+0000","last_change":"2024-04-16T17:47:59.182311+0000","last_active":"2024-04-16T19:33:46.158533+0000","last_peered":"2024-04-16T19:33:46.158533+0000","last_clean":"2024-04-16T19:33:46.158533+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:33:46.158533+0000","last_undegraded":"2024-04-16T19:33:46.158533+0000","last_fullsized":"2024-04-16T19:33:46.158533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:15.733801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'897","reported_seq":3791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155083+0000","last_change":"2024-04-16T17:47:57.178953+0000","last_active":"2024-04-16T19:32:36.155083+0000","last_peered":"2024-04-16T19:32:36.155083+0000","last_clean":"2024-04-16T19:32:36.155083+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:32:36.155083+0000","last_undegraded":"2024-04-16T19:32:36.155083+0000","last_fullsized":"2024-04-16T19:32:36.155083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:27.120398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'29","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148129+0000","last_change":"2024-04-16T17:47:55.165769+0000","last_active":"2024-04-16T19:31:56.148129+0000","last_peered":"2024-04-16T19:31:56.148129+0000","last_clean":"2024-04-16T19:31:56.148129+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:31:56.148129+0000","last_undegraded":"2024-04-16T19:31:56.148129+0000","last_fullsized":"2024-04-16T19:31:56.148129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175855+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:33:41.175855+0000","last_peered":"2024-04-16T19:33:41.175855+0000","last_clean":"2024-04-16T19:33:41.175855+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:33:41.175855+0000","last_undegraded":"2024-04-16T19:33:41.175855+0000","last_fullsized":"2024-04-16T19:33:41.175855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159749+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:32:51.159749+0000","last_peered":"2024-04-16T19:32:51.159749+0000","last_clean":"2024-04-16T19:32:51.159749+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:32:51.159749+0000","last_undegraded":"2024-04-16T19:32:51.159749+0000","last_fullsized":"2024-04-16T19:32:51.159749+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169877+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:33:36.169877+0000","last_peered":"2024-04-16T19:33:36.169877+0000","last_clean":"2024-04-16T19:33:36.169877+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:33:36.169877+0000","last_undegraded":"2024-04-16T19:33:36.169877+0000","last_fullsized":"2024-04-16T19:33:36.169877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154521+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:32:21.154521+0000","last_peered":"2024-04-16T19:32:21.154521+0000","last_clean":"2024-04-16T19:32:21.154521+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:32:21.154521+0000","last_undegraded":"2024-04-16T19:32:21.154521+0000","last_fullsized":"2024-04-16T19:32:21.154521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'144","reported_seq":330,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177257+0000","last_change":"2024-04-16T17:47:35.314035+0000","last_active":"2024-04-16T19:33:46.177257+0000","last_peered":"2024-04-16T19:33:46.177257+0000","last_clean":"2024-04-16T19:33:46.177257+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:33:46.177257+0000","last_undegraded":"2024-04-16T19:33:46.177257+0000","last_fullsized":"2024-04-16T19:33:46.177257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181110+0000","last_change":"2024-04-16T17:47:37.334598+0000","last_active":"2024-04-16T19:33:51.181110+0000","last_peered":"2024-04-16T19:33:51.181110+0000","last_clean":"2024-04-16T19:33:51.181110+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:33:51.181110+0000","last_undegraded":"2024-04-16T19:33:51.181110+0000","last_fullsized":"2024-04-16T19:33:51.181110+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:24.856058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175255+0000","last_change":"2024-04-16T17:47:39.339775+0000","last_active":"2024-04-16T19:33:51.175255+0000","last_peered":"2024-04-16T19:33:51.175255+0000","last_clean":"2024-04-16T19:33:51.175255+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:33:51.175255+0000","last_undegraded":"2024-04-16T19:33:51.175255+0000","last_fullsized":"2024-04-16T19:33:51.175255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143941+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:31:51.143941+0000","last_peered":"2024-04-16T19:31:51.143941+0000","last_clean":"2024-04-16T19:31:51.143941+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:31:51.143941+0000","last_undegraded":"2024-04-16T19:31:51.143941+0000","last_fullsized":"2024-04-16T19:31:51.143941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:26.624319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148088+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:32:06.148088+0000","last_peered":"2024-04-16T19:32:06.148088+0000","last_clean":"2024-04-16T19:32:06.148088+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:32:06.148088+0000","last_undegraded":"2024-04-16T19:32:06.148088+0000","last_fullsized":"2024-04-16T19:32:06.148088+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146697+0000","last_change":"2024-04-16T17:48:27.307048+0000","last_active":"2024-04-16T19:32:01.146697+0000","last_peered":"2024-04-16T19:32:01.146697+0000","last_clean":"2024-04-16T19:32:01.146697+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:32:01.146697+0000","last_undegraded":"2024-04-16T19:32:01.146697+0000","last_fullsized":"2024-04-16T19:32:01.146697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:19.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'33","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152888+0000","last_change":"2024-04-16T17:48:01.250054+0000","last_active":"2024-04-16T19:32:06.152888+0000","last_peered":"2024-04-16T19:32:06.152888+0000","last_clean":"2024-04-16T19:32:06.152888+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:32:06.152888+0000","last_undegraded":"2024-04-16T19:32:06.152888+0000","last_fullsized":"2024-04-16T19:32:06.152888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133916+0000","last_change":"2024-04-16T17:47:59.190987+0000","last_active":"2024-04-16T19:32:01.133916+0000","last_peered":"2024-04-16T19:32:01.133916+0000","last_clean":"2024-04-16T19:32:01.133916+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:32:01.133916+0000","last_undegraded":"2024-04-16T19:32:01.133916+0000","last_fullsized":"2024-04-16T19:32:01.133916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:11.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'264","reported_seq":640,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151154+0000","last_change":"2024-04-16T17:47:57.189816+0000","last_active":"2024-04-16T19:32:01.151154+0000","last_peered":"2024-04-16T19:32:01.151154+0000","last_clean":"2024-04-16T19:32:01.151154+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:32:01.151154+0000","last_undegraded":"2024-04-16T19:32:01.151154+0000","last_fullsized":"2024-04-16T19:32:01.151154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'10","reported_seq":156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159575+0000","last_change":"2024-04-16T17:47:55.161898+0000","last_active":"2024-04-16T19:32:31.159575+0000","last_peered":"2024-04-16T19:32:31.159575+0000","last_clean":"2024-04-16T19:32:31.159575+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:32:31.159575+0000","last_undegraded":"2024-04-16T19:32:31.159575+0000","last_fullsized":"2024-04-16T19:32:31.159575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174563+0000","last_change":"2024-04-16T17:47:23.265711+0000","last_active":"2024-04-16T19:33:36.174563+0000","last_peered":"2024-04-16T19:33:36.174563+0000","last_clean":"2024-04-16T19:33:36.174563+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:33:36.174563+0000","last_undegraded":"2024-04-16T19:33:36.174563+0000","last_fullsized":"2024-04-16T19:33:36.174563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.158718+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:32:51.158718+0000","last_peered":"2024-04-16T19:32:51.158718+0000","last_clean":"2024-04-16T19:32:51.158718+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:32:51.158718+0000","last_undegraded":"2024-04-16T19:32:51.158718+0000","last_fullsized":"2024-04-16T19:32:51.158718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178254+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:33:51.178254+0000","last_peered":"2024-04-16T19:33:51.178254+0000","last_clean":"2024-04-16T19:33:51.178254+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:33:51.178254+0000","last_undegraded":"2024-04-16T19:33:51.178254+0000","last_fullsized":"2024-04-16T19:33:51.178254+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:49.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"48'4","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158694+0000","last_change":"2024-04-16T17:47:33.301081+0000","last_active":"2024-04-16T19:33:46.158694+0000","last_peered":"2024-04-16T19:33:46.158694+0000","last_clean":"2024-04-16T19:33:46.158694+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:33:46.158694+0000","last_undegraded":"2024-04-16T19:33:46.158694+0000","last_fullsized":"2024-04-16T19:33:46.158694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:05.482314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'151","reported_seq":1555,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149286+0000","last_change":"2024-04-16T17:47:35.316795+0000","last_active":"2024-04-16T19:32:11.149286+0000","last_peered":"2024-04-16T19:32:11.149286+0000","last_clean":"2024-04-16T19:32:11.149286+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:32:11.149286+0000","last_undegraded":"2024-04-16T19:32:11.149286+0000","last_fullsized":"2024-04-16T19:32:11.149286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178652+0000","last_change":"2024-04-16T17:47:37.326201+0000","last_active":"2024-04-16T19:33:56.178652+0000","last_peered":"2024-04-16T19:33:56.178652+0000","last_clean":"2024-04-16T19:33:56.178652+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:33:56.178652+0000","last_undegraded":"2024-04-16T19:33:56.178652+0000","last_fullsized":"2024-04-16T19:33:56.178652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170822+0000","last_change":"2024-04-16T17:47:39.332609+0000","last_active":"2024-04-16T19:33:26.170822+0000","last_peered":"2024-04-16T19:33:26.170822+0000","last_clean":"2024-04-16T19:33:26.170822+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:33:26.170822+0000","last_undegraded":"2024-04-16T19:33:26.170822+0000","last_fullsized":"2024-04-16T19:33:26.170822+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143874+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:31:51.143874+0000","last_peered":"2024-04-16T19:31:51.143874+0000","last_clean":"2024-04-16T19:31:51.143874+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:31:51.143874+0000","last_undegraded":"2024-04-16T19:31:51.143874+0000","last_fullsized":"2024-04-16T19:31:51.143874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:45.404216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'76","reported_seq":1158,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150775+0000","last_change":"2024-04-16T17:48:34.318880+0000","last_active":"2024-04-16T19:32:06.150775+0000","last_peered":"2024-04-16T19:32:06.150775+0000","last_clean":"2024-04-16T19:32:06.150775+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:32:06.150775+0000","last_undegraded":"2024-04-16T19:32:06.150775+0000","last_fullsized":"2024-04-16T19:32:06.150775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153591+0000","last_change":"2024-04-16T17:48:27.299238+0000","last_active":"2024-04-16T19:32:26.153591+0000","last_peered":"2024-04-16T19:32:26.153591+0000","last_clean":"2024-04-16T19:32:26.153591+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:32:26.153591+0000","last_undegraded":"2024-04-16T19:32:26.153591+0000","last_fullsized":"2024-04-16T19:32:26.153591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:49.437897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'6","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176200+0000","last_change":"2024-04-16T17:48:01.256444+0000","last_active":"2024-04-16T19:33:46.176200+0000","last_peered":"2024-04-16T19:33:46.176200+0000","last_clean":"2024-04-16T19:33:46.176200+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:33:46.176200+0000","last_undegraded":"2024-04-16T19:33:46.176200+0000","last_fullsized":"2024-04-16T19:33:46.176200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'555","reported_seq":1932,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133972+0000","last_change":"2024-04-16T17:47:59.183633+0000","last_active":"2024-04-16T19:32:01.133972+0000","last_peered":"2024-04-16T19:32:01.133972+0000","last_clean":"2024-04-16T19:32:01.133972+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:32:01.133972+0000","last_undegraded":"2024-04-16T19:32:01.133972+0000","last_fullsized":"2024-04-16T19:32:01.133972+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:14.201307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'306","reported_seq":623,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149338+0000","last_change":"2024-04-16T17:47:57.177185+0000","last_active":"2024-04-16T19:32:01.149338+0000","last_peered":"2024-04-16T19:32:01.149338+0000","last_clean":"2024-04-16T19:32:01.149338+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:32:01.149338+0000","last_undegraded":"2024-04-16T19:32:01.149338+0000","last_fullsized":"2024-04-16T19:32:01.149338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:18.465537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"48'5","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174804+0000","last_change":"2024-04-16T17:47:55.178154+0000","last_active":"2024-04-16T19:33:41.174804+0000","last_peered":"2024-04-16T19:33:41.174804+0000","last_clean":"2024-04-16T19:33:41.174804+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:33:41.174804+0000","last_undegraded":"2024-04-16T19:33:41.174804+0000","last_fullsized":"2024-04-16T19:33:41.174804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173663+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:33:36.173663+0000","last_peered":"2024-04-16T19:33:36.173663+0000","last_clean":"2024-04-16T19:33:36.173663+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:33:36.173663+0000","last_undegraded":"2024-04-16T19:33:36.173663+0000","last_fullsized":"2024-04-16T19:33:36.173663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164327+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:32:56.164327+0000","last_peered":"2024-04-16T19:32:56.164327+0000","last_clean":"2024-04-16T19:32:56.164327+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:32:56.164327+0000","last_undegraded":"2024-04-16T19:32:56.164327+0000","last_fullsized":"2024-04-16T19:32:56.164327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171318+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:33:16.171318+0000","last_peered":"2024-04-16T19:33:16.171318+0000","last_clean":"2024-04-16T19:33:16.171318+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:33:16.171318+0000","last_undegraded":"2024-04-16T19:33:16.171318+0000","last_fullsized":"2024-04-16T19:33:16.171318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:05:07.906618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173252+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:33:36.173252+0000","last_peered":"2024-04-16T19:33:36.173252+0000","last_clean":"2024-04-16T19:33:36.173252+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:33:36.173252+0000","last_undegraded":"2024-04-16T19:33:36.173252+0000","last_fullsized":"2024-04-16T19:33:36.173252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:30.892245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":504,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174348+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:33:46.174348+0000","last_peered":"2024-04-16T19:33:46.174348+0000","last_clean":"2024-04-16T19:33:46.174348+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:33:46.174348+0000","last_undegraded":"2024-04-16T19:33:46.174348+0000","last_fullsized":"2024-04-16T19:33:46.174348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'177","reported_seq":1612,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178544+0000","last_change":"2024-04-16T17:47:35.312413+0000","last_active":"2024-04-16T19:33:51.178544+0000","last_peered":"2024-04-16T19:33:51.178544+0000","last_clean":"2024-04-16T19:33:51.178544+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:33:51.178544+0000","last_undegraded":"2024-04-16T19:33:51.178544+0000","last_fullsized":"2024-04-16T19:33:51.178544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152741+0000","last_change":"2024-04-16T17:47:37.327792+0000","last_active":"2024-04-16T19:32:26.152741+0000","last_peered":"2024-04-16T19:32:26.152741+0000","last_clean":"2024-04-16T19:32:26.152741+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:32:26.152741+0000","last_undegraded":"2024-04-16T19:32:26.152741+0000","last_fullsized":"2024-04-16T19:32:26.152741+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:43.802094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152938+0000","last_change":"2024-04-16T17:47:39.700262+0000","last_active":"2024-04-16T19:32:11.152938+0000","last_peered":"2024-04-16T19:32:11.152938+0000","last_clean":"2024-04-16T19:32:11.152938+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:32:11.152938+0000","last_undegraded":"2024-04-16T19:32:11.152938+0000","last_fullsized":"2024-04-16T19:32:11.152938+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:26.867823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157247+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:33:41.157247+0000","last_peered":"2024-04-16T19:33:41.157247+0000","last_clean":"2024-04-16T19:33:41.157247+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:33:41.157247+0000","last_undegraded":"2024-04-16T19:33:41.157247+0000","last_fullsized":"2024-04-16T19:33:41.157247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.324013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150723+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:32:06.150723+0000","last_peered":"2024-04-16T19:32:06.150723+0000","last_clean":"2024-04-16T19:32:06.150723+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:32:06.150723+0000","last_undegraded":"2024-04-16T19:32:06.150723+0000","last_fullsized":"2024-04-16T19:32:06.150723+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:02.642376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140845+0000","last_change":"2024-04-16T17:48:27.304274+0000","last_active":"2024-04-16T19:32:31.140845+0000","last_peered":"2024-04-16T19:32:31.140845+0000","last_clean":"2024-04-16T19:32:31.140845+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:32:31.140845+0000","last_undegraded":"2024-04-16T19:32:31.140845+0000","last_fullsized":"2024-04-16T19:32:31.140845+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:19.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'14","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133996+0000","last_change":"2024-04-16T17:48:01.251987+0000","last_active":"2024-04-16T19:32:01.133996+0000","last_peered":"2024-04-16T19:32:01.133996+0000","last_clean":"2024-04-16T19:32:01.133996+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:32:01.133996+0000","last_undegraded":"2024-04-16T19:32:01.133996+0000","last_fullsized":"2024-04-16T19:32:01.133996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:53.985409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'555","reported_seq":1923,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172522+0000","last_change":"2024-04-16T17:47:59.188607+0000","last_active":"2024-04-16T19:33:46.172522+0000","last_peered":"2024-04-16T19:33:46.172522+0000","last_clean":"2024-04-16T19:33:46.172522+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:33:46.172522+0000","last_undegraded":"2024-04-16T19:33:46.172522+0000","last_fullsized":"2024-04-16T19:33:46.172522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:09.833400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'514","reported_seq":1190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152969+0000","last_change":"2024-04-16T17:47:57.187984+0000","last_active":"2024-04-16T19:32:06.152969+0000","last_peered":"2024-04-16T19:32:06.152969+0000","last_clean":"2024-04-16T19:32:06.152969+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:32:06.152969+0000","last_undegraded":"2024-04-16T19:32:06.152969+0000","last_fullsized":"2024-04-16T19:32:06.152969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:09.942121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"51'3","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157358+0000","last_change":"2024-04-16T17:47:55.173104+0000","last_active":"2024-04-16T19:33:41.157358+0000","last_peered":"2024-04-16T19:33:41.157358+0000","last_clean":"2024-04-16T19:33:41.157358+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:33:41.157358+0000","last_undegraded":"2024-04-16T19:33:41.157358+0000","last_fullsized":"2024-04-16T19:33:41.157358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:46.009020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.170986+0000","last_change":"2024-04-16T17:47:23.266373+0000","last_active":"2024-04-16T19:33:41.170986+0000","last_peered":"2024-04-16T19:33:41.170986+0000","last_clean":"2024-04-16T19:33:41.170986+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:33:41.170986+0000","last_undegraded":"2024-04-16T19:33:41.170986+0000","last_fullsized":"2024-04-16T19:33:41.170986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:11.306509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158724+0000","last_change":"2024-04-16T17:47:32.273580+0000","last_active":"2024-04-16T19:33:46.158724+0000","last_peered":"2024-04-16T19:33:46.158724+0000","last_clean":"2024-04-16T19:33:46.158724+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:33:46.158724+0000","last_undegraded":"2024-04-16T19:33:46.158724+0000","last_fullsized":"2024-04-16T19:33:46.158724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:28.727683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164407+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:32:56.164407+0000","last_peered":"2024-04-16T19:32:56.164407+0000","last_clean":"2024-04-16T19:32:56.164407+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:32:56.164407+0000","last_undegraded":"2024-04-16T19:32:56.164407+0000","last_fullsized":"2024-04-16T19:32:56.164407+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163230+0000","last_change":"2024-04-16T17:47:33.300039+0000","last_active":"2024-04-16T19:34:06.163230+0000","last_peered":"2024-04-16T19:34:06.163230+0000","last_clean":"2024-04-16T19:34:06.163230+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:34:06.163230+0000","last_undegraded":"2024-04-16T19:34:06.163230+0000","last_fullsized":"2024-04-16T19:34:06.163230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:42.946001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'144","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173676+0000","last_change":"2024-04-16T17:47:35.305470+0000","last_active":"2024-04-16T19:33:51.173676+0000","last_peered":"2024-04-16T19:33:51.173676+0000","last_clean":"2024-04-16T19:33:51.173676+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:33:51.173676+0000","last_undegraded":"2024-04-16T19:33:51.173676+0000","last_fullsized":"2024-04-16T19:33:51.173676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:00.096888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.161004+0000","last_change":"2024-04-16T17:47:37.334307+0000","last_active":"2024-04-16T19:33:56.161004+0000","last_peered":"2024-04-16T19:33:56.161004+0000","last_clean":"2024-04-16T19:33:56.161004+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:33:56.161004+0000","last_undegraded":"2024-04-16T19:33:56.161004+0000","last_fullsized":"2024-04-16T19:33:56.161004+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:58.517922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168901+0000","last_change":"2024-04-16T17:47:39.343779+0000","last_active":"2024-04-16T19:33:26.168901+0000","last_peered":"2024-04-16T19:33:26.168901+0000","last_clean":"2024-04-16T19:33:26.168901+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:33:26.168901+0000","last_undegraded":"2024-04-16T19:33:26.168901+0000","last_fullsized":"2024-04-16T19:33:26.168901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:43:28.188142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131712+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:31:51.131712+0000","last_peered":"2024-04-16T19:31:51.131712+0000","last_clean":"2024-04-16T19:31:51.131712+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:31:51.131712+0000","last_undegraded":"2024-04-16T19:31:51.131712+0000","last_fullsized":"2024-04-16T19:31:51.131712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:49:14.730835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'122","reported_seq":1763,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170106+0000","last_change":"2024-04-16T17:48:34.337192+0000","last_active":"2024-04-16T19:33:11.170106+0000","last_peered":"2024-04-16T19:33:11.170106+0000","last_clean":"2024-04-16T19:33:11.170106+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:33:11.170106+0000","last_undegraded":"2024-04-16T19:33:11.170106+0000","last_fullsized":"2024-04-16T19:33:11.170106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.154528+0000","last_change":"2024-04-16T17:48:27.301617+0000","last_active":"2024-04-16T19:32:31.154528+0000","last_peered":"2024-04-16T19:32:31.154528+0000","last_clean":"2024-04-16T19:32:31.154528+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:32:31.154528+0000","last_undegraded":"2024-04-16T19:32:31.154528+0000","last_fullsized":"2024-04-16T19:32:31.154528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'16","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133844+0000","last_change":"2024-04-16T17:48:01.234685+0000","last_active":"2024-04-16T19:32:01.133844+0000","last_peered":"2024-04-16T19:32:01.133844+0000","last_clean":"2024-04-16T19:32:01.133844+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:32:01.133844+0000","last_undegraded":"2024-04-16T19:32:01.133844+0000","last_fullsized":"2024-04-16T19:32:01.133844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:02.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152930+0000","last_change":"2024-04-16T17:47:59.185616+0000","last_active":"2024-04-16T19:32:06.152930+0000","last_peered":"2024-04-16T19:32:06.152930+0000","last_clean":"2024-04-16T19:32:06.152930+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:32:06.152930+0000","last_undegraded":"2024-04-16T19:32:06.152930+0000","last_fullsized":"2024-04-16T19:32:06.152930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:24:43.650159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1159","reported_seq":1411,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172544+0000","last_change":"2024-04-16T17:47:57.188806+0000","last_active":"2024-04-16T19:33:46.172544+0000","last_peered":"2024-04-16T19:33:46.172544+0000","last_clean":"2024-04-16T19:33:46.172544+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:33:46.172544+0000","last_undegraded":"2024-04-16T19:33:46.172544+0000","last_fullsized":"2024-04-16T19:33:46.172544+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1159,"log_dups_size":0,"ondisk_log_size":1159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:00.199310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"48'18","reported_seq":399,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177873+0000","last_change":"2024-04-16T17:47:55.164040+0000","last_active":"2024-04-16T19:33:41.177873+0000","last_peered":"2024-04-16T19:33:41.177873+0000","last_clean":"2024-04-16T19:33:41.177873+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:33:41.177873+0000","last_undegraded":"2024-04-16T19:33:41.177873+0000","last_fullsized":"2024-04-16T19:33:41.177873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.179804+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:34:01.179804+0000","last_peered":"2024-04-16T19:34:01.179804+0000","last_clean":"2024-04-16T19:34:01.179804+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:34:01.179804+0000","last_undegraded":"2024-04-16T19:34:01.179804+0000","last_fullsized":"2024-04-16T19:34:01.179804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154908+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:33:31.154908+0000","last_peered":"2024-04-16T19:33:31.154908+0000","last_clean":"2024-04-16T19:33:31.154908+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:33:31.154908+0000","last_undegraded":"2024-04-16T19:33:31.154908+0000","last_fullsized":"2024-04-16T19:33:31.154908+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:55:03.538702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168590+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:33:31.168590+0000","last_peered":"2024-04-16T19:33:31.168590+0000","last_clean":"2024-04-16T19:33:31.168590+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:33:31.168590+0000","last_undegraded":"2024-04-16T19:33:31.168590+0000","last_fullsized":"2024-04-16T19:33:31.168590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180686+0000","last_change":"2024-04-16T17:47:33.300948+0000","last_active":"2024-04-16T19:33:51.180686+0000","last_peered":"2024-04-16T19:33:51.180686+0000","last_clean":"2024-04-16T19:33:51.180686+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:33:51.180686+0000","last_undegraded":"2024-04-16T19:33:51.180686+0000","last_fullsized":"2024-04-16T19:33:51.180686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'120","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180791+0000","last_change":"2024-04-16T17:47:35.313794+0000","last_active":"2024-04-16T19:33:51.180791+0000","last_peered":"2024-04-16T19:33:51.180791+0000","last_clean":"2024-04-16T19:33:51.180791+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:33:51.180791+0000","last_undegraded":"2024-04-16T19:33:51.180791+0000","last_fullsized":"2024-04-16T19:33:51.180791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178567+0000","last_change":"2024-04-16T17:47:37.329509+0000","last_active":"2024-04-16T19:33:56.178567+0000","last_peered":"2024-04-16T19:33:56.178567+0000","last_clean":"2024-04-16T19:33:56.178567+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:33:56.178567+0000","last_undegraded":"2024-04-16T19:33:56.178567+0000","last_fullsized":"2024-04-16T19:33:56.178567+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170908+0000","last_change":"2024-04-16T17:47:39.697737+0000","last_active":"2024-04-16T19:33:26.170908+0000","last_peered":"2024-04-16T19:33:26.170908+0000","last_clean":"2024-04-16T19:33:26.170908+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:33:26.170908+0000","last_undegraded":"2024-04-16T19:33:26.170908+0000","last_fullsized":"2024-04-16T19:33:26.170908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.150046+0000","last_change":"2024-04-16T18:10:50.537157+0000","last_active":"2024-04-16T19:33:11.150046+0000","last_peered":"2024-04-16T19:33:11.150046+0000","last_clean":"2024-04-16T19:33:11.150046+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:33:11.150046+0000","last_undegraded":"2024-04-16T19:33:11.150046+0000","last_fullsized":"2024-04-16T19:33:11.150046+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:13.931584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1282,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156075+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:32:36.156075+0000","last_peered":"2024-04-16T19:32:36.156075+0000","last_clean":"2024-04-16T19:32:36.156075+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:32:36.156075+0000","last_undegraded":"2024-04-16T19:32:36.156075+0000","last_fullsized":"2024-04-16T19:32:36.156075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:20.310434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147837+0000","last_change":"2024-04-16T17:48:27.305636+0000","last_active":"2024-04-16T19:33:01.147837+0000","last_peered":"2024-04-16T19:33:01.147837+0000","last_clean":"2024-04-16T19:33:01.147837+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:33:01.147837+0000","last_undegraded":"2024-04-16T19:33:01.147837+0000","last_fullsized":"2024-04-16T19:33:01.147837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:06.425749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'13","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175996+0000","last_change":"2024-04-16T17:48:01.266136+0000","last_active":"2024-04-16T19:33:46.175996+0000","last_peered":"2024-04-16T19:33:46.175996+0000","last_clean":"2024-04-16T19:33:46.175996+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:33:46.175996+0000","last_undegraded":"2024-04-16T19:33:46.175996+0000","last_fullsized":"2024-04-16T19:33:46.175996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177121+0000","last_change":"2024-04-16T17:47:59.188856+0000","last_active":"2024-04-16T19:33:46.177121+0000","last_peered":"2024-04-16T19:33:46.177121+0000","last_clean":"2024-04-16T19:33:46.177121+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:33:46.177121+0000","last_undegraded":"2024-04-16T19:33:46.177121+0000","last_fullsized":"2024-04-16T19:33:46.177121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:35.433741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1314","reported_seq":1867,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172479+0000","last_change":"2024-04-16T17:47:57.192362+0000","last_active":"2024-04-16T19:33:46.172479+0000","last_peered":"2024-04-16T19:33:46.172479+0000","last_clean":"2024-04-16T19:33:46.172479+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:33:46.172479+0000","last_undegraded":"2024-04-16T19:33:46.172479+0000","last_fullsized":"2024-04-16T19:33:46.172479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1314,"log_dups_size":0,"ondisk_log_size":1314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:14.446218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'4","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178048+0000","last_change":"2024-04-16T17:47:55.156911+0000","last_active":"2024-04-16T19:33:41.178048+0000","last_peered":"2024-04-16T19:33:41.178048+0000","last_clean":"2024-04-16T19:33:41.178048+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:33:41.178048+0000","last_undegraded":"2024-04-16T19:33:41.178048+0000","last_fullsized":"2024-04-16T19:33:41.178048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174726+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:33:41.174726+0000","last_peered":"2024-04-16T19:33:41.174726+0000","last_clean":"2024-04-16T19:33:41.174726+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:33:41.174726+0000","last_undegraded":"2024-04-16T19:33:41.174726+0000","last_fullsized":"2024-04-16T19:33:41.174726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174314+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:33:36.174314+0000","last_peered":"2024-04-16T19:33:36.174314+0000","last_clean":"2024-04-16T19:33:36.174314+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:33:36.174314+0000","last_undegraded":"2024-04-16T19:33:36.174314+0000","last_fullsized":"2024-04-16T19:33:36.174314+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:22.291956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163913+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:32:46.163913+0000","last_peered":"2024-04-16T19:32:46.163913+0000","last_clean":"2024-04-16T19:32:46.163913+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:32:46.163913+0000","last_undegraded":"2024-04-16T19:32:46.163913+0000","last_fullsized":"2024-04-16T19:32:46.163913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175384+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:33:51.175384+0000","last_peered":"2024-04-16T19:33:51.175384+0000","last_clean":"2024-04-16T19:33:51.175384+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:33:51.175384+0000","last_undegraded":"2024-04-16T19:33:51.175384+0000","last_fullsized":"2024-04-16T19:33:51.175384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'168","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178373+0000","last_change":"2024-04-16T17:47:35.321995+0000","last_active":"2024-04-16T19:33:51.178373+0000","last_peered":"2024-04-16T19:33:51.178373+0000","last_clean":"2024-04-16T19:33:51.178373+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:33:51.178373+0000","last_undegraded":"2024-04-16T19:33:51.178373+0000","last_fullsized":"2024-04-16T19:33:51.178373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:17.922789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'12","reported_seq":1342,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179297+0000","last_change":"2024-04-16T17:47:37.323586+0000","last_active":"2024-04-16T19:34:06.179297+0000","last_peered":"2024-04-16T19:34:06.179297+0000","last_clean":"2024-04-16T19:34:06.179297+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:34:06.179297+0000","last_undegraded":"2024-04-16T19:34:06.179297+0000","last_fullsized":"2024-04-16T19:34:06.179297+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145872+0000","last_change":"2024-04-16T17:47:39.698152+0000","last_active":"2024-04-16T19:31:46.145872+0000","last_peered":"2024-04-16T19:31:46.145872+0000","last_clean":"2024-04-16T19:31:46.145872+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:31:46.145872+0000","last_undegraded":"2024-04-16T19:31:46.145872+0000","last_fullsized":"2024-04-16T19:31:46.145872+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:08.010344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.164914+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:33:11.164914+0000","last_peered":"2024-04-16T19:33:11.164914+0000","last_clean":"2024-04-16T19:33:11.164914+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:33:11.164914+0000","last_undegraded":"2024-04-16T19:33:11.164914+0000","last_fullsized":"2024-04-16T19:33:11.164914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:36:23.318927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158153+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:32:36.158153+0000","last_peered":"2024-04-16T19:32:36.158153+0000","last_clean":"2024-04-16T19:32:36.158153+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:32:36.158153+0000","last_undegraded":"2024-04-16T19:32:36.158153+0000","last_fullsized":"2024-04-16T19:32:36.158153+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151256+0000","last_change":"2024-04-16T17:48:27.298996+0000","last_active":"2024-04-16T19:32:01.151256+0000","last_peered":"2024-04-16T19:32:01.151256+0000","last_clean":"2024-04-16T19:32:01.151256+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:32:01.151256+0000","last_undegraded":"2024-04-16T19:32:01.151256+0000","last_fullsized":"2024-04-16T19:32:01.151256+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"51'4","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140996+0000","last_change":"2024-04-16T17:48:01.252079+0000","last_active":"2024-04-16T19:32:31.140996+0000","last_peered":"2024-04-16T19:32:31.140996+0000","last_clean":"2024-04-16T19:32:31.140996+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:32:31.140996+0000","last_undegraded":"2024-04-16T19:32:31.140996+0000","last_fullsized":"2024-04-16T19:32:31.140996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:57.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'555","reported_seq":1994,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158653+0000","last_change":"2024-04-16T17:47:59.197276+0000","last_active":"2024-04-16T19:33:46.158653+0000","last_peered":"2024-04-16T19:33:46.158653+0000","last_clean":"2024-04-16T19:33:46.158653+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:33:46.158653+0000","last_undegraded":"2024-04-16T19:33:46.158653+0000","last_fullsized":"2024-04-16T19:33:46.158653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:16.461108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'553","reported_seq":822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172334+0000","last_change":"2024-04-16T17:47:57.191520+0000","last_active":"2024-04-16T19:33:46.172334+0000","last_peered":"2024-04-16T19:33:46.172334+0000","last_clean":"2024-04-16T19:33:46.172334+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:33:46.172334+0000","last_undegraded":"2024-04-16T19:33:46.172334+0000","last_fullsized":"2024-04-16T19:33:46.172334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:25.845115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'14","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144097+0000","last_change":"2024-04-16T17:47:55.178734+0000","last_active":"2024-04-16T19:31:51.144097+0000","last_peered":"2024-04-16T19:31:51.144097+0000","last_clean":"2024-04-16T19:31:51.144097+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:31:51.144097+0000","last_undegraded":"2024-04-16T19:31:51.144097+0000","last_fullsized":"2024-04-16T19:31:51.144097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175692+0000","last_change":"2024-04-16T17:47:23.265822+0000","last_active":"2024-04-16T19:33:41.175692+0000","last_peered":"2024-04-16T19:33:41.175692+0000","last_clean":"2024-04-16T19:33:41.175692+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:33:41.175692+0000","last_undegraded":"2024-04-16T19:33:41.175692+0000","last_fullsized":"2024-04-16T19:33:41.175692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178216+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:34:01.178216+0000","last_peered":"2024-04-16T19:34:01.178216+0000","last_clean":"2024-04-16T19:34:01.178216+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:34:01.178216+0000","last_undegraded":"2024-04-16T19:34:01.178216+0000","last_fullsized":"2024-04-16T19:34:01.178216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.158816+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:32:46.158816+0000","last_peered":"2024-04-16T19:32:46.158816+0000","last_clean":"2024-04-16T19:32:46.158816+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:32:46.158816+0000","last_undegraded":"2024-04-16T19:32:46.158816+0000","last_fullsized":"2024-04-16T19:32:46.158816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177291+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:33:51.177291+0000","last_peered":"2024-04-16T19:33:51.177291+0000","last_clean":"2024-04-16T19:33:51.177291+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:33:51.177291+0000","last_undegraded":"2024-04-16T19:33:51.177291+0000","last_fullsized":"2024-04-16T19:33:51.177291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'160","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182468+0000","last_change":"2024-04-16T17:47:35.312410+0000","last_active":"2024-04-16T19:34:11.182468+0000","last_peered":"2024-04-16T19:34:11.182468+0000","last_clean":"2024-04-16T19:34:11.182468+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:34:11.182468+0000","last_undegraded":"2024-04-16T19:34:11.182468+0000","last_fullsized":"2024-04-16T19:34:11.182468+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174883+0000","last_change":"2024-04-16T17:47:37.322938+0000","last_active":"2024-04-16T19:33:56.174883+0000","last_peered":"2024-04-16T19:33:56.174883+0000","last_clean":"2024-04-16T19:33:56.174883+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:33:56.174883+0000","last_undegraded":"2024-04-16T19:33:56.174883+0000","last_fullsized":"2024-04-16T19:33:56.174883+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:33.277719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170994+0000","last_change":"2024-04-16T17:47:39.699806+0000","last_active":"2024-04-16T19:33:26.170994+0000","last_peered":"2024-04-16T19:33:26.170994+0000","last_clean":"2024-04-16T19:33:26.170994+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:33:26.170994+0000","last_undegraded":"2024-04-16T19:33:26.170994+0000","last_fullsized":"2024-04-16T19:33:26.170994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164695+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:33:01.164695+0000","last_peered":"2024-04-16T19:33:01.164695+0000","last_clean":"2024-04-16T19:33:01.164695+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:33:01.164695+0000","last_undegraded":"2024-04-16T19:33:01.164695+0000","last_fullsized":"2024-04-16T19:33:01.164695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1098,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168822+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:33:06.168822+0000","last_peered":"2024-04-16T19:33:06.168822+0000","last_clean":"2024-04-16T19:33:06.168822+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:33:06.168822+0000","last_undegraded":"2024-04-16T19:33:06.168822+0000","last_fullsized":"2024-04-16T19:33:06.168822+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:38.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172871+0000","last_change":"2024-04-16T17:48:27.307428+0000","last_active":"2024-04-16T19:33:31.172871+0000","last_peered":"2024-04-16T19:33:31.172871+0000","last_clean":"2024-04-16T19:33:31.172871+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:33:31.172871+0000","last_undegraded":"2024-04-16T19:33:31.172871+0000","last_fullsized":"2024-04-16T19:33:31.172871+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:33.963133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161215+0000","last_change":"2024-04-16T17:48:01.240690+0000","last_active":"2024-04-16T19:33:01.161215+0000","last_peered":"2024-04-16T19:33:01.161215+0000","last_clean":"2024-04-16T19:33:01.161215+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:33:01.161215+0000","last_undegraded":"2024-04-16T19:33:01.161215+0000","last_fullsized":"2024-04-16T19:33:01.161215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:42.135971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162598+0000","last_change":"2024-04-16T17:47:59.192669+0000","last_active":"2024-04-16T19:33:01.162598+0000","last_peered":"2024-04-16T19:33:01.162598+0000","last_clean":"2024-04-16T19:33:01.162598+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:33:01.162598+0000","last_undegraded":"2024-04-16T19:33:01.162598+0000","last_fullsized":"2024-04-16T19:33:01.162598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'850","reported_seq":1151,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161341+0000","last_change":"2024-04-16T17:47:57.193698+0000","last_active":"2024-04-16T19:33:01.161341+0000","last_peered":"2024-04-16T19:33:01.161341+0000","last_clean":"2024-04-16T19:33:01.161341+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:33:01.161341+0000","last_undegraded":"2024-04-16T19:33:01.161341+0000","last_fullsized":"2024-04-16T19:33:01.161341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:15.692887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'16","reported_seq":234,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.178756+0000","last_change":"2024-04-16T17:49:32.398153+0000","last_active":"2024-04-16T19:34:11.178756+0000","last_peered":"2024-04-16T19:34:11.178756+0000","last_clean":"2024-04-16T19:34:11.178756+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:34:11.178756+0000","last_undegraded":"2024-04-16T19:34:11.178756+0000","last_fullsized":"2024-04-16T19:34:11.178756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:45.378036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178826+0000","last_change":"2024-04-16T17:47:39.336834+0000","last_active":"2024-04-16T19:33:56.178826+0000","last_peered":"2024-04-16T19:33:56.178826+0000","last_clean":"2024-04-16T19:33:56.178826+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:33:56.178826+0000","last_undegraded":"2024-04-16T19:33:56.178826+0000","last_fullsized":"2024-04-16T19:33:56.178826+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:09.285627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152606+0000","last_change":"2024-04-16T17:47:23.257996+0000","last_active":"2024-04-16T19:32:26.152606+0000","last_peered":"2024-04-16T19:32:26.152606+0000","last_clean":"2024-04-16T19:32:26.152606+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:32:26.152606+0000","last_undegraded":"2024-04-16T19:32:26.152606+0000","last_fullsized":"2024-04-16T19:32:26.152606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166359+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:32:56.166359+0000","last_peered":"2024-04-16T19:32:56.166359+0000","last_clean":"2024-04-16T19:32:56.166359+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:32:56.166359+0000","last_undegraded":"2024-04-16T19:32:56.166359+0000","last_fullsized":"2024-04-16T19:32:56.166359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"48'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159071+0000","last_change":"2024-04-16T17:47:33.297781+0000","last_active":"2024-04-16T19:32:36.159071+0000","last_peered":"2024-04-16T19:32:36.159071+0000","last_clean":"2024-04-16T19:32:36.159071+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:32:36.159071+0000","last_undegraded":"2024-04-16T19:32:36.159071+0000","last_fullsized":"2024-04-16T19:32:36.159071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:51.904305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142052+0000","last_change":"2024-04-16T17:47:35.310182+0000","last_active":"2024-04-16T19:32:36.142052+0000","last_peered":"2024-04-16T19:32:36.142052+0000","last_clean":"2024-04-16T19:32:36.142052+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:32:36.142052+0000","last_undegraded":"2024-04-16T19:32:36.142052+0000","last_fullsized":"2024-04-16T19:32:36.142052+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:57:05.362295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160301+0000","last_change":"2024-04-16T17:47:37.334854+0000","last_active":"2024-04-16T19:32:41.160301+0000","last_peered":"2024-04-16T19:32:41.160301+0000","last_clean":"2024-04-16T19:32:41.160301+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:32:41.160301+0000","last_undegraded":"2024-04-16T19:32:41.160301+0000","last_fullsized":"2024-04-16T19:32:41.160301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:24.862379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168290+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:33:31.168290+0000","last_peered":"2024-04-16T19:33:31.168290+0000","last_clean":"2024-04-16T19:33:31.168290+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:33:31.168290+0000","last_undegraded":"2024-04-16T19:33:31.168290+0000","last_fullsized":"2024-04-16T19:33:31.168290+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:11.558629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'55","reported_seq":671,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157685+0000","last_change":"2024-04-16T17:48:34.343212+0000","last_active":"2024-04-16T19:32:31.157685+0000","last_peered":"2024-04-16T19:32:31.157685+0000","last_clean":"2024-04-16T19:32:31.157685+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:32:31.157685+0000","last_undegraded":"2024-04-16T19:32:31.157685+0000","last_fullsized":"2024-04-16T19:32:31.157685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:28:18.790386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168495+0000","last_change":"2024-04-16T17:48:27.301479+0000","last_active":"2024-04-16T19:33:31.168495+0000","last_peered":"2024-04-16T19:33:31.168495+0000","last_clean":"2024-04-16T19:33:31.168495+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:33:31.168495+0000","last_undegraded":"2024-04-16T19:33:31.168495+0000","last_fullsized":"2024-04-16T19:33:31.168495+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:04.776813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'9","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161137+0000","last_change":"2024-04-16T17:48:01.250254+0000","last_active":"2024-04-16T19:33:01.161137+0000","last_peered":"2024-04-16T19:33:01.161137+0000","last_clean":"2024-04-16T19:33:01.161137+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:33:01.161137+0000","last_undegraded":"2024-04-16T19:33:01.161137+0000","last_fullsized":"2024-04-16T19:33:01.161137+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:53.163721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167775+0000","last_change":"2024-04-16T17:47:59.176369+0000","last_active":"2024-04-16T19:33:01.167775+0000","last_peered":"2024-04-16T19:33:01.167775+0000","last_clean":"2024-04-16T19:33:01.167775+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:33:01.167775+0000","last_undegraded":"2024-04-16T19:33:01.167775+0000","last_fullsized":"2024-04-16T19:33:01.167775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:52.164568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'592","reported_seq":2279,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162562+0000","last_change":"2024-04-16T17:47:57.185970+0000","last_active":"2024-04-16T19:33:01.162562+0000","last_peered":"2024-04-16T19:33:01.162562+0000","last_clean":"2024-04-16T19:33:01.162562+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:33:01.162562+0000","last_undegraded":"2024-04-16T19:33:01.162562+0000","last_fullsized":"2024-04-16T19:33:01.162562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:33.418851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"51'2","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151881+0000","last_change":"2024-04-16T17:47:55.163430+0000","last_active":"2024-04-16T19:32:06.151881+0000","last_peered":"2024-04-16T19:32:06.151881+0000","last_clean":"2024-04-16T19:32:06.151881+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:32:06.151881+0000","last_undegraded":"2024-04-16T19:32:06.151881+0000","last_fullsized":"2024-04-16T19:32:06.151881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183840+0000","last_change":"2024-04-16T17:47:39.338758+0000","last_active":"2024-04-16T19:34:11.183840+0000","last_peered":"2024-04-16T19:34:11.183840+0000","last_clean":"2024-04-16T19:34:11.183840+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:34:11.183840+0000","last_undegraded":"2024-04-16T19:34:11.183840+0000","last_fullsized":"2024-04-16T19:34:11.183840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156609+0000","last_change":"2024-04-16T17:47:23.260667+0000","last_active":"2024-04-16T19:32:26.156609+0000","last_peered":"2024-04-16T19:32:26.156609+0000","last_clean":"2024-04-16T19:32:26.156609+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:32:26.156609+0000","last_undegraded":"2024-04-16T19:32:26.156609+0000","last_fullsized":"2024-04-16T19:32:26.156609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177933+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:33:41.177933+0000","last_peered":"2024-04-16T19:33:41.177933+0000","last_clean":"2024-04-16T19:33:41.177933+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:33:41.177933+0000","last_undegraded":"2024-04-16T19:33:41.177933+0000","last_fullsized":"2024-04-16T19:33:41.177933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"48'1","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159182+0000","last_change":"2024-04-16T17:47:33.297748+0000","last_active":"2024-04-16T19:32:36.159182+0000","last_peered":"2024-04-16T19:32:36.159182+0000","last_clean":"2024-04-16T19:32:36.159182+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:32:36.159182+0000","last_undegraded":"2024-04-16T19:32:36.159182+0000","last_fullsized":"2024-04-16T19:32:36.159182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:36:58.076852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'50","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158106+0000","last_change":"2024-04-16T17:47:35.314682+0000","last_active":"2024-04-16T19:32:36.158106+0000","last_peered":"2024-04-16T19:32:36.158106+0000","last_clean":"2024-04-16T19:32:36.158106+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:32:36.158106+0000","last_undegraded":"2024-04-16T19:32:36.158106+0000","last_fullsized":"2024-04-16T19:32:36.158106+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162296+0000","last_change":"2024-04-16T17:47:37.327586+0000","last_active":"2024-04-16T19:32:41.162296+0000","last_peered":"2024-04-16T19:32:41.162296+0000","last_clean":"2024-04-16T19:32:41.162296+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:32:41.162296+0000","last_undegraded":"2024-04-16T19:32:41.162296+0000","last_fullsized":"2024-04-16T19:32:41.162296+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167526+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:33:21.167526+0000","last_peered":"2024-04-16T19:33:21.167526+0000","last_clean":"2024-04-16T19:33:21.167526+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:33:21.167526+0000","last_undegraded":"2024-04-16T19:33:21.167526+0000","last_fullsized":"2024-04-16T19:33:21.167526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:07.939676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'99","reported_seq":1687,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174223+0000","last_change":"2024-04-16T17:48:34.338854+0000","last_active":"2024-04-16T19:33:36.174223+0000","last_peered":"2024-04-16T19:33:36.174223+0000","last_clean":"2024-04-16T19:33:36.174223+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:33:36.174223+0000","last_undegraded":"2024-04-16T19:33:36.174223+0000","last_fullsized":"2024-04-16T19:33:36.174223+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:41.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153090+0000","last_change":"2024-04-16T17:48:27.301694+0000","last_active":"2024-04-16T19:32:26.153090+0000","last_peered":"2024-04-16T19:32:26.153090+0000","last_clean":"2024-04-16T19:32:26.153090+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:32:26.153090+0000","last_undegraded":"2024-04-16T19:32:26.153090+0000","last_fullsized":"2024-04-16T19:32:26.153090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'13","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156916+0000","last_change":"2024-04-16T17:48:01.248700+0000","last_active":"2024-04-16T19:32:31.156916+0000","last_peered":"2024-04-16T19:32:31.156916+0000","last_clean":"2024-04-16T19:32:31.156916+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:32:31.156916+0000","last_undegraded":"2024-04-16T19:32:31.156916+0000","last_fullsized":"2024-04-16T19:32:31.156916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'20","reported_seq":1421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153482+0000","last_change":"2024-04-16T17:47:59.196995+0000","last_active":"2024-04-16T19:32:26.153482+0000","last_peered":"2024-04-16T19:32:26.153482+0000","last_clean":"2024-04-16T19:32:26.153482+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:32:26.153482+0000","last_undegraded":"2024-04-16T19:32:26.153482+0000","last_fullsized":"2024-04-16T19:32:26.153482+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.685479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'6983","reported_seq":10557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154510+0000","last_change":"2024-04-16T17:47:57.195602+0000","last_active":"2024-04-16T19:32:11.154510+0000","last_peered":"2024-04-16T19:32:11.154510+0000","last_clean":"2024-04-16T19:32:11.154510+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:32:11.154510+0000","last_undegraded":"2024-04-16T19:32:11.154510+0000","last_fullsized":"2024-04-16T19:32:11.154510+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1783,"log_dups_size":3000,"ondisk_log_size":1783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:48:37.455011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'6","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163653+0000","last_change":"2024-04-16T17:47:55.162215+0000","last_active":"2024-04-16T19:32:56.163653+0000","last_peered":"2024-04-16T19:32:56.163653+0000","last_clean":"2024-04-16T19:32:56.163653+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:32:56.163653+0000","last_undegraded":"2024-04-16T19:32:56.163653+0000","last_fullsized":"2024-04-16T19:32:56.163653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159959+0000","last_change":"2024-04-16T17:47:39.329121+0000","last_active":"2024-04-16T19:32:41.159959+0000","last_peered":"2024-04-16T19:32:41.159959+0000","last_clean":"2024-04-16T19:32:41.159959+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:32:41.159959+0000","last_undegraded":"2024-04-16T19:32:41.159959+0000","last_fullsized":"2024-04-16T19:32:41.159959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139757+0000","last_change":"2024-04-16T17:47:23.269620+0000","last_active":"2024-04-16T19:32:26.139757+0000","last_peered":"2024-04-16T19:32:26.139757+0000","last_clean":"2024-04-16T19:32:26.139757+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:32:26.139757+0000","last_undegraded":"2024-04-16T19:32:26.139757+0000","last_fullsized":"2024-04-16T19:32:26.139757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:43.201589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172841+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:33:41.172841+0000","last_peered":"2024-04-16T19:33:41.172841+0000","last_clean":"2024-04-16T19:33:41.172841+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:33:41.172841+0000","last_undegraded":"2024-04-16T19:33:41.172841+0000","last_fullsized":"2024-04-16T19:33:41.172841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155004+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:32:36.155004+0000","last_peered":"2024-04-16T19:32:36.155004+0000","last_clean":"2024-04-16T19:32:36.155004+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:32:36.155004+0000","last_undegraded":"2024-04-16T19:32:36.155004+0000","last_fullsized":"2024-04-16T19:32:36.155004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:14.585091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142192+0000","last_change":"2024-04-16T17:47:35.310781+0000","last_active":"2024-04-16T19:32:36.142192+0000","last_peered":"2024-04-16T19:32:36.142192+0000","last_clean":"2024-04-16T19:32:36.142192+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:32:36.142192+0000","last_undegraded":"2024-04-16T19:32:36.142192+0000","last_fullsized":"2024-04-16T19:32:36.142192+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:20.569781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157441+0000","last_change":"2024-04-16T17:47:37.322773+0000","last_active":"2024-04-16T19:32:41.157441+0000","last_peered":"2024-04-16T19:32:41.157441+0000","last_clean":"2024-04-16T19:32:41.157441+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:32:41.157441+0000","last_undegraded":"2024-04-16T19:32:41.157441+0000","last_fullsized":"2024-04-16T19:32:41.157441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172568+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:33:21.172568+0000","last_peered":"2024-04-16T19:33:21.172568+0000","last_clean":"2024-04-16T19:33:21.172568+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:33:21.172568+0000","last_undegraded":"2024-04-16T19:33:21.172568+0000","last_fullsized":"2024-04-16T19:33:21.172568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147896+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:32:06.147896+0000","last_peered":"2024-04-16T19:32:06.147896+0000","last_clean":"2024-04-16T19:32:06.147896+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:32:06.147896+0000","last_undegraded":"2024-04-16T19:32:06.147896+0000","last_fullsized":"2024-04-16T19:32:06.147896+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:46:24.982845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146589+0000","last_change":"2024-04-16T17:48:27.293279+0000","last_active":"2024-04-16T19:32:56.146589+0000","last_peered":"2024-04-16T19:32:56.146589+0000","last_clean":"2024-04-16T19:32:56.146589+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:32:56.146589+0000","last_undegraded":"2024-04-16T19:32:56.146589+0000","last_fullsized":"2024-04-16T19:32:56.146589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:44.442181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"48'8","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148529+0000","last_change":"2024-04-16T17:48:01.237052+0000","last_active":"2024-04-16T19:31:56.148529+0000","last_peered":"2024-04-16T19:31:56.148529+0000","last_clean":"2024-04-16T19:31:56.148529+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:31:56.148529+0000","last_undegraded":"2024-04-16T19:31:56.148529+0000","last_fullsized":"2024-04-16T19:31:56.148529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:22.523578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'555","reported_seq":1956,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136598+0000","last_change":"2024-04-16T17:47:59.187458+0000","last_active":"2024-04-16T19:32:11.136598+0000","last_peered":"2024-04-16T19:32:11.136598+0000","last_clean":"2024-04-16T19:32:11.136598+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:32:11.136598+0000","last_undegraded":"2024-04-16T19:32:11.136598+0000","last_fullsized":"2024-04-16T19:32:11.136598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:17.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'544","reported_seq":775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158452+0000","last_change":"2024-04-16T17:47:57.184810+0000","last_active":"2024-04-16T19:32:36.158452+0000","last_peered":"2024-04-16T19:32:36.158452+0000","last_clean":"2024-04-16T19:32:36.158452+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:32:36.158452+0000","last_undegraded":"2024-04-16T19:32:36.158452+0000","last_fullsized":"2024-04-16T19:32:36.158452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"51'4","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163711+0000","last_change":"2024-04-16T17:47:55.183376+0000","last_active":"2024-04-16T19:32:56.163711+0000","last_peered":"2024-04-16T19:32:56.163711+0000","last_clean":"2024-04-16T19:32:56.163711+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:32:56.163711+0000","last_undegraded":"2024-04-16T19:32:56.163711+0000","last_fullsized":"2024-04-16T19:32:56.163711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157536+0000","last_change":"2024-04-16T17:47:39.339617+0000","last_active":"2024-04-16T19:32:41.157536+0000","last_peered":"2024-04-16T19:32:41.157536+0000","last_clean":"2024-04-16T19:32:41.157536+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:32:41.157536+0000","last_undegraded":"2024-04-16T19:32:41.157536+0000","last_fullsized":"2024-04-16T19:32:41.157536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156357+0000","last_change":"2024-04-16T17:47:23.263370+0000","last_active":"2024-04-16T19:32:26.156357+0000","last_peered":"2024-04-16T19:32:26.156357+0000","last_clean":"2024-04-16T19:32:26.156357+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:32:26.156357+0000","last_undegraded":"2024-04-16T19:32:26.156357+0000","last_fullsized":"2024-04-16T19:32:26.156357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174981+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:33:41.174981+0000","last_peered":"2024-04-16T19:33:41.174981+0000","last_clean":"2024-04-16T19:33:41.174981+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:33:41.174981+0000","last_undegraded":"2024-04-16T19:33:41.174981+0000","last_fullsized":"2024-04-16T19:33:41.174981+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160793+0000","last_change":"2024-04-16T17:47:33.307304+0000","last_active":"2024-04-16T19:32:36.160793+0000","last_peered":"2024-04-16T19:32:36.160793+0000","last_clean":"2024-04-16T19:32:36.160793+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:32:36.160793+0000","last_undegraded":"2024-04-16T19:32:36.160793+0000","last_fullsized":"2024-04-16T19:32:36.160793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156191+0000","last_change":"2024-04-16T17:47:35.311451+0000","last_active":"2024-04-16T19:32:36.156191+0000","last_peered":"2024-04-16T19:32:36.156191+0000","last_clean":"2024-04-16T19:32:36.156191+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:32:36.156191+0000","last_undegraded":"2024-04-16T19:32:36.156191+0000","last_fullsized":"2024-04-16T19:32:36.156191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159764+0000","last_change":"2024-04-16T17:47:37.326333+0000","last_active":"2024-04-16T19:32:41.159764+0000","last_peered":"2024-04-16T19:32:41.159764+0000","last_clean":"2024-04-16T19:32:41.159764+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:32:41.159764+0000","last_undegraded":"2024-04-16T19:32:41.159764+0000","last_fullsized":"2024-04-16T19:32:41.159764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167717+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:33:01.167717+0000","last_peered":"2024-04-16T19:33:01.167717+0000","last_clean":"2024-04-16T19:33:01.167717+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:33:01.167717+0000","last_undegraded":"2024-04-16T19:33:01.167717+0000","last_fullsized":"2024-04-16T19:33:01.167717+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'89","reported_seq":694,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159707+0000","last_change":"2024-04-16T17:48:34.328117+0000","last_active":"2024-04-16T19:32:31.159707+0000","last_peered":"2024-04-16T19:32:31.159707+0000","last_clean":"2024-04-16T19:32:31.159707+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:32:31.159707+0000","last_undegraded":"2024-04-16T19:32:31.159707+0000","last_fullsized":"2024-04-16T19:32:31.159707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155893+0000","last_change":"2024-04-16T17:48:27.295608+0000","last_active":"2024-04-16T19:32:26.155893+0000","last_peered":"2024-04-16T19:32:26.155893+0000","last_clean":"2024-04-16T19:32:26.155893+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:32:26.155893+0000","last_undegraded":"2024-04-16T19:32:26.155893+0000","last_fullsized":"2024-04-16T19:32:26.155893+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'12","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145278+0000","last_change":"2024-04-16T17:48:01.246592+0000","last_active":"2024-04-16T19:31:56.145278+0000","last_peered":"2024-04-16T19:31:56.145278+0000","last_clean":"2024-04-16T19:31:56.145278+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:31:56.145278+0000","last_undegraded":"2024-04-16T19:31:56.145278+0000","last_fullsized":"2024-04-16T19:31:56.145278+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:06.492939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164653+0000","last_change":"2024-04-16T17:47:59.178449+0000","last_active":"2024-04-16T19:33:01.164653+0000","last_peered":"2024-04-16T19:33:01.164653+0000","last_clean":"2024-04-16T19:33:01.164653+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:33:01.164653+0000","last_undegraded":"2024-04-16T19:33:01.164653+0000","last_fullsized":"2024-04-16T19:33:01.164653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'457","reported_seq":4410,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162217+0000","last_change":"2024-04-16T17:47:57.181693+0000","last_active":"2024-04-16T19:33:01.162217+0000","last_peered":"2024-04-16T19:33:01.162217+0000","last_clean":"2024-04-16T19:33:01.162217+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:33:01.162217+0000","last_undegraded":"2024-04-16T19:33:01.162217+0000","last_fullsized":"2024-04-16T19:33:01.162217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:44.694025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'6","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161031+0000","last_change":"2024-04-16T17:47:55.174874+0000","last_active":"2024-04-16T19:32:56.161031+0000","last_peered":"2024-04-16T19:32:56.161031+0000","last_clean":"2024-04-16T19:32:56.161031+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:32:56.161031+0000","last_undegraded":"2024-04-16T19:32:56.161031+0000","last_fullsized":"2024-04-16T19:32:56.161031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178208+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:33:41.178208+0000","last_peered":"2024-04-16T19:33:41.178208+0000","last_clean":"2024-04-16T19:33:41.178208+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:33:41.178208+0000","last_undegraded":"2024-04-16T19:33:41.178208+0000","last_fullsized":"2024-04-16T19:33:41.178208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:37.592633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157093+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:33:41.157093+0000","last_peered":"2024-04-16T19:33:41.157093+0000","last_clean":"2024-04-16T19:33:41.157093+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:33:41.157093+0000","last_undegraded":"2024-04-16T19:33:41.157093+0000","last_fullsized":"2024-04-16T19:33:41.157093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:08.399471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182918+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:34:11.182918+0000","last_peered":"2024-04-16T19:34:11.182918+0000","last_clean":"2024-04-16T19:34:11.182918+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:34:11.182918+0000","last_undegraded":"2024-04-16T19:34:11.182918+0000","last_fullsized":"2024-04-16T19:34:11.182918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'88","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159309+0000","last_change":"2024-04-16T17:47:35.319036+0000","last_active":"2024-04-16T19:32:36.159309+0000","last_peered":"2024-04-16T19:32:36.159309+0000","last_clean":"2024-04-16T19:32:36.159309+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:32:36.159309+0000","last_undegraded":"2024-04-16T19:32:36.159309+0000","last_fullsized":"2024-04-16T19:32:36.159309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162487+0000","last_change":"2024-04-16T17:47:37.330847+0000","last_active":"2024-04-16T19:32:41.162487+0000","last_peered":"2024-04-16T19:32:41.162487+0000","last_clean":"2024-04-16T19:32:41.162487+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:32:41.162487+0000","last_undegraded":"2024-04-16T19:32:41.162487+0000","last_fullsized":"2024-04-16T19:32:41.162487+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:10.138006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160442+0000","last_change":"2024-04-16T17:47:39.701654+0000","last_active":"2024-04-16T19:32:41.160442+0000","last_peered":"2024-04-16T19:32:41.160442+0000","last_clean":"2024-04-16T19:32:41.160442+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:32:41.160442+0000","last_undegraded":"2024-04-16T19:32:41.160442+0000","last_fullsized":"2024-04-16T19:32:41.160442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162629+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:33:01.162629+0000","last_peered":"2024-04-16T19:33:01.162629+0000","last_clean":"2024-04-16T19:33:01.162629+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:33:01.162629+0000","last_undegraded":"2024-04-16T19:33:01.162629+0000","last_fullsized":"2024-04-16T19:33:01.162629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:28:06.366066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'71","reported_seq":966,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143796+0000","last_change":"2024-04-16T17:48:34.313054+0000","last_active":"2024-04-16T19:31:51.143796+0000","last_peered":"2024-04-16T19:31:51.143796+0000","last_clean":"2024-04-16T19:31:51.143796+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:31:51.143796+0000","last_undegraded":"2024-04-16T19:31:51.143796+0000","last_fullsized":"2024-04-16T19:31:51.143796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:11.966580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153539+0000","last_change":"2024-04-16T17:48:27.309014+0000","last_active":"2024-04-16T19:32:26.153539+0000","last_peered":"2024-04-16T19:32:26.153539+0000","last_clean":"2024-04-16T19:32:26.153539+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:32:26.153539+0000","last_undegraded":"2024-04-16T19:32:26.153539+0000","last_fullsized":"2024-04-16T19:32:26.153539+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'4","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145295+0000","last_change":"2024-04-16T17:48:01.251229+0000","last_active":"2024-04-16T19:31:56.145295+0000","last_peered":"2024-04-16T19:31:56.145295+0000","last_clean":"2024-04-16T19:31:56.145295+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:31:56.145295+0000","last_undegraded":"2024-04-16T19:31:56.145295+0000","last_fullsized":"2024-04-16T19:31:56.145295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147763+0000","last_change":"2024-04-16T17:47:59.174249+0000","last_active":"2024-04-16T19:33:01.147763+0000","last_peered":"2024-04-16T19:33:01.147763+0000","last_clean":"2024-04-16T19:33:01.147763+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:33:01.147763+0000","last_undegraded":"2024-04-16T19:33:01.147763+0000","last_fullsized":"2024-04-16T19:33:01.147763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:54.272783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11370","reported_seq":61791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161030+0000","last_change":"2024-04-16T17:47:57.192920+0000","last_active":"2024-04-16T19:33:01.161030+0000","last_peered":"2024-04-16T19:33:01.161030+0000","last_clean":"2024-04-16T19:33:01.161030+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:33:01.161030+0000","last_undegraded":"2024-04-16T19:33:01.161030+0000","last_fullsized":"2024-04-16T19:33:01.161030+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1670,"log_dups_size":3000,"ondisk_log_size":1670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:13.718911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'12","reported_seq":338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166321+0000","last_change":"2024-04-16T17:47:55.178367+0000","last_active":"2024-04-16T19:32:56.166321+0000","last_peered":"2024-04-16T19:32:56.166321+0000","last_clean":"2024-04-16T19:32:56.166321+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:32:56.166321+0000","last_undegraded":"2024-04-16T19:32:56.166321+0000","last_fullsized":"2024-04-16T19:32:56.166321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.180159+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:34:01.180159+0000","last_peered":"2024-04-16T19:34:01.180159+0000","last_clean":"2024-04-16T19:34:01.180159+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:34:01.180159+0000","last_undegraded":"2024-04-16T19:34:01.180159+0000","last_fullsized":"2024-04-16T19:34:01.180159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145890+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:31:46.145890+0000","last_peered":"2024-04-16T19:31:46.145890+0000","last_clean":"2024-04-16T19:31:46.145890+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:31:46.145890+0000","last_undegraded":"2024-04-16T19:31:46.145890+0000","last_fullsized":"2024-04-16T19:31:46.145890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181077+0000","last_change":"2024-04-16T17:47:33.296653+0000","last_active":"2024-04-16T19:33:51.181077+0000","last_peered":"2024-04-16T19:33:51.181077+0000","last_clean":"2024-04-16T19:33:51.181077+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:33:51.181077+0000","last_undegraded":"2024-04-16T19:33:51.181077+0000","last_fullsized":"2024-04-16T19:33:51.181077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:00.330068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'152","reported_seq":340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160930+0000","last_change":"2024-04-16T17:47:35.317123+0000","last_active":"2024-04-16T19:32:36.160930+0000","last_peered":"2024-04-16T19:32:36.160930+0000","last_clean":"2024-04-16T19:32:36.160930+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:32:36.160930+0000","last_undegraded":"2024-04-16T19:32:36.160930+0000","last_fullsized":"2024-04-16T19:32:36.160930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160571+0000","last_change":"2024-04-16T17:47:37.334019+0000","last_active":"2024-04-16T19:32:41.160571+0000","last_peered":"2024-04-16T19:32:41.160571+0000","last_clean":"2024-04-16T19:32:41.160571+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:32:41.160571+0000","last_undegraded":"2024-04-16T19:32:41.160571+0000","last_fullsized":"2024-04-16T19:32:41.160571+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:07.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162425+0000","last_change":"2024-04-16T17:47:39.699556+0000","last_active":"2024-04-16T19:32:41.162425+0000","last_peered":"2024-04-16T19:32:41.162425+0000","last_clean":"2024-04-16T19:32:41.162425+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:32:41.162425+0000","last_undegraded":"2024-04-16T19:32:41.162425+0000","last_fullsized":"2024-04-16T19:32:41.162425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165020+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:33:01.165020+0000","last_peered":"2024-04-16T19:33:01.165020+0000","last_clean":"2024-04-16T19:33:01.165020+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:33:01.165020+0000","last_undegraded":"2024-04-16T19:33:01.165020+0000","last_fullsized":"2024-04-16T19:33:01.165020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'103","reported_seq":1819,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158185+0000","last_change":"2024-04-16T17:48:34.337561+0000","last_active":"2024-04-16T19:32:31.158185+0000","last_peered":"2024-04-16T19:32:31.158185+0000","last_clean":"2024-04-16T19:32:31.158185+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:32:31.158185+0000","last_undegraded":"2024-04-16T19:32:31.158185+0000","last_fullsized":"2024-04-16T19:32:31.158185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:01.320649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145725+0000","last_change":"2024-04-16T17:48:27.293905+0000","last_active":"2024-04-16T19:31:46.145725+0000","last_peered":"2024-04-16T19:31:46.145725+0000","last_clean":"2024-04-16T19:31:46.145725+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:31:46.145725+0000","last_undegraded":"2024-04-16T19:31:46.145725+0000","last_fullsized":"2024-04-16T19:31:46.145725+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'8","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145241+0000","last_change":"2024-04-16T17:48:01.248531+0000","last_active":"2024-04-16T19:31:56.145241+0000","last_peered":"2024-04-16T19:31:56.145241+0000","last_clean":"2024-04-16T19:31:56.145241+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:31:56.145241+0000","last_undegraded":"2024-04-16T19:31:56.145241+0000","last_fullsized":"2024-04-16T19:31:56.145241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167662+0000","last_change":"2024-04-16T17:47:59.187755+0000","last_active":"2024-04-16T19:33:01.167662+0000","last_peered":"2024-04-16T19:33:01.167662+0000","last_clean":"2024-04-16T19:33:01.167662+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:33:01.167662+0000","last_undegraded":"2024-04-16T19:33:01.167662+0000","last_fullsized":"2024-04-16T19:33:01.167662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:25.460755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'196","reported_seq":421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165564+0000","last_change":"2024-04-16T17:47:57.195138+0000","last_active":"2024-04-16T19:33:01.165564+0000","last_peered":"2024-04-16T19:33:01.165564+0000","last_clean":"2024-04-16T19:33:01.165564+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:33:01.165564+0000","last_undegraded":"2024-04-16T19:33:01.165564+0000","last_fullsized":"2024-04-16T19:33:01.165564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:36.309438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'12","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159825+0000","last_change":"2024-04-16T17:47:55.160472+0000","last_active":"2024-04-16T19:32:56.159825+0000","last_peered":"2024-04-16T19:32:56.159825+0000","last_clean":"2024-04-16T19:32:56.159825+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:32:56.159825+0000","last_undegraded":"2024-04-16T19:32:56.159825+0000","last_fullsized":"2024-04-16T19:32:56.159825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:00.875301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155799+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:32:26.155799+0000","last_peered":"2024-04-16T19:32:26.155799+0000","last_clean":"2024-04-16T19:32:26.155799+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:32:26.155799+0000","last_undegraded":"2024-04-16T19:32:26.155799+0000","last_fullsized":"2024-04-16T19:32:26.155799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151302+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:32:21.151302+0000","last_peered":"2024-04-16T19:32:21.151302+0000","last_clean":"2024-04-16T19:32:21.151302+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:32:21.151302+0000","last_undegraded":"2024-04-16T19:32:21.151302+0000","last_fullsized":"2024-04-16T19:32:21.151302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:44.777444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"48'5","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155232+0000","last_change":"2024-04-16T17:47:33.306460+0000","last_active":"2024-04-16T19:32:36.155232+0000","last_peered":"2024-04-16T19:32:36.155232+0000","last_clean":"2024-04-16T19:32:36.155232+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:32:36.155232+0000","last_undegraded":"2024-04-16T19:32:36.155232+0000","last_fullsized":"2024-04-16T19:32:36.155232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:15.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'104","reported_seq":277,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160070+0000","last_change":"2024-04-16T17:47:35.314274+0000","last_active":"2024-04-16T19:33:51.160070+0000","last_peered":"2024-04-16T19:33:51.160070+0000","last_clean":"2024-04-16T19:33:51.160070+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:33:51.160070+0000","last_undegraded":"2024-04-16T19:33:51.160070+0000","last_fullsized":"2024-04-16T19:33:51.160070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:58.087539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183865+0000","last_change":"2024-04-16T17:47:37.331871+0000","last_active":"2024-04-16T19:34:11.183865+0000","last_peered":"2024-04-16T19:34:11.183865+0000","last_clean":"2024-04-16T19:34:11.183865+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:34:11.183865+0000","last_undegraded":"2024-04-16T19:34:11.183865+0000","last_fullsized":"2024-04-16T19:34:11.183865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:10.644125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162350+0000","last_change":"2024-04-16T17:47:39.323414+0000","last_active":"2024-04-16T19:32:41.162350+0000","last_peered":"2024-04-16T19:32:41.162350+0000","last_clean":"2024-04-16T19:32:41.162350+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:32:41.162350+0000","last_undegraded":"2024-04-16T19:32:41.162350+0000","last_fullsized":"2024-04-16T19:32:41.162350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177321+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:33:46.177321+0000","last_peered":"2024-04-16T19:33:46.177321+0000","last_clean":"2024-04-16T19:33:46.177321+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:33:46.177321+0000","last_undegraded":"2024-04-16T19:33:46.177321+0000","last_fullsized":"2024-04-16T19:33:46.177321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:41.778594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'68","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158123+0000","last_change":"2024-04-16T17:48:34.343716+0000","last_active":"2024-04-16T19:32:31.158123+0000","last_peered":"2024-04-16T19:32:31.158123+0000","last_clean":"2024-04-16T19:32:31.158123+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:32:31.158123+0000","last_undegraded":"2024-04-16T19:32:31.158123+0000","last_fullsized":"2024-04-16T19:32:31.158123+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:48.210902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152776+0000","last_change":"2024-04-16T17:48:27.305044+0000","last_active":"2024-04-16T19:32:26.152776+0000","last_peered":"2024-04-16T19:32:26.152776+0000","last_clean":"2024-04-16T19:32:26.152776+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:32:26.152776+0000","last_undegraded":"2024-04-16T19:32:26.152776+0000","last_fullsized":"2024-04-16T19:32:26.152776+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:17.183220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"48'1","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168780+0000","last_change":"2024-04-16T17:48:01.254059+0000","last_active":"2024-04-16T19:33:26.168780+0000","last_peered":"2024-04-16T19:33:26.168780+0000","last_clean":"2024-04-16T19:33:26.168780+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:33:26.168780+0000","last_undegraded":"2024-04-16T19:33:26.168780+0000","last_fullsized":"2024-04-16T19:33:26.168780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162464+0000","last_change":"2024-04-16T17:47:59.195638+0000","last_active":"2024-04-16T19:33:01.162464+0000","last_peered":"2024-04-16T19:33:01.162464+0000","last_clean":"2024-04-16T19:33:01.162464+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:33:01.162464+0000","last_undegraded":"2024-04-16T19:33:01.162464+0000","last_fullsized":"2024-04-16T19:33:01.162464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:57.426064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'488","reported_seq":3937,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162374+0000","last_change":"2024-04-16T17:47:57.174503+0000","last_active":"2024-04-16T19:33:01.162374+0000","last_peered":"2024-04-16T19:33:01.162374+0000","last_clean":"2024-04-16T19:33:01.162374+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:33:01.162374+0000","last_undegraded":"2024-04-16T19:33:01.162374+0000","last_fullsized":"2024-04-16T19:33:01.162374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:34.199358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'27","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166245+0000","last_change":"2024-04-16T17:47:55.169780+0000","last_active":"2024-04-16T19:32:56.166245+0000","last_peered":"2024-04-16T19:32:56.166245+0000","last_clean":"2024-04-16T19:32:56.166245+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:32:56.166245+0000","last_undegraded":"2024-04-16T19:32:56.166245+0000","last_fullsized":"2024-04-16T19:32:56.166245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139882+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:32:26.139882+0000","last_peered":"2024-04-16T19:32:26.139882+0000","last_clean":"2024-04-16T19:32:26.139882+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:32:26.139882+0000","last_undegraded":"2024-04-16T19:32:26.139882+0000","last_fullsized":"2024-04-16T19:32:26.139882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:43:58.984431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154419+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:32:21.154419+0000","last_peered":"2024-04-16T19:32:21.154419+0000","last_clean":"2024-04-16T19:32:21.154419+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:32:21.154419+0000","last_undegraded":"2024-04-16T19:32:21.154419+0000","last_fullsized":"2024-04-16T19:32:21.154419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"33'3","reported_seq":802,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159282+0000","last_change":"2024-04-16T17:47:33.297794+0000","last_active":"2024-04-16T19:32:36.159282+0000","last_peered":"2024-04-16T19:32:36.159282+0000","last_clean":"2024-04-16T19:32:36.159282+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:32:36.159282+0000","last_undegraded":"2024-04-16T19:32:36.159282+0000","last_fullsized":"2024-04-16T19:32:36.159282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'120","reported_seq":298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186858+0000","last_change":"2024-04-16T17:47:35.319657+0000","last_active":"2024-04-16T19:34:11.186858+0000","last_peered":"2024-04-16T19:34:11.186858+0000","last_clean":"2024-04-16T19:34:11.186858+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:34:11.186858+0000","last_undegraded":"2024-04-16T19:34:11.186858+0000","last_fullsized":"2024-04-16T19:34:11.186858+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:08.628044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.177090+0000","last_change":"2024-04-16T17:47:37.331215+0000","last_active":"2024-04-16T19:33:56.177090+0000","last_peered":"2024-04-16T19:33:56.177090+0000","last_clean":"2024-04-16T19:33:56.177090+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:33:56.177090+0000","last_undegraded":"2024-04-16T19:33:56.177090+0000","last_fullsized":"2024-04-16T19:33:56.177090+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162223+0000","last_change":"2024-04-16T17:47:39.332386+0000","last_active":"2024-04-16T19:32:41.162223+0000","last_peered":"2024-04-16T19:32:41.162223+0000","last_clean":"2024-04-16T19:32:41.162223+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:32:41.162223+0000","last_undegraded":"2024-04-16T19:32:41.162223+0000","last_fullsized":"2024-04-16T19:32:41.162223+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.153850+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:32:31.153850+0000","last_peered":"2024-04-16T19:32:31.153850+0000","last_clean":"2024-04-16T19:32:31.153850+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:32:31.153850+0000","last_undegraded":"2024-04-16T19:32:31.153850+0000","last_fullsized":"2024-04-16T19:32:31.153850+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:59:18.189132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'86","reported_seq":1557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169733+0000","last_change":"2024-04-16T17:48:34.318628+0000","last_active":"2024-04-16T19:33:36.169733+0000","last_peered":"2024-04-16T19:33:36.169733+0000","last_clean":"2024-04-16T19:33:36.169733+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:33:36.169733+0000","last_undegraded":"2024-04-16T19:33:36.169733+0000","last_fullsized":"2024-04-16T19:33:36.169733+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:15:00.721377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172016+0000","last_change":"2024-04-16T17:48:27.299691+0000","last_active":"2024-04-16T19:33:31.172016+0000","last_peered":"2024-04-16T19:33:31.172016+0000","last_clean":"2024-04-16T19:33:31.172016+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:33:31.172016+0000","last_undegraded":"2024-04-16T19:33:31.172016+0000","last_fullsized":"2024-04-16T19:33:31.172016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:40:17.511736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'6","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165634+0000","last_change":"2024-04-16T17:48:01.246789+0000","last_active":"2024-04-16T19:33:01.165634+0000","last_peered":"2024-04-16T19:33:01.165634+0000","last_clean":"2024-04-16T19:33:01.165634+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:33:01.165634+0000","last_undegraded":"2024-04-16T19:33:01.165634+0000","last_fullsized":"2024-04-16T19:33:01.165634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:47.722149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'555","reported_seq":1953,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176329+0000","last_change":"2024-04-16T17:47:59.182365+0000","last_active":"2024-04-16T19:33:36.176329+0000","last_peered":"2024-04-16T19:33:36.176329+0000","last_clean":"2024-04-16T19:33:36.176329+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:33:36.176329+0000","last_undegraded":"2024-04-16T19:33:36.176329+0000","last_fullsized":"2024-04-16T19:33:36.176329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'492","reported_seq":710,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148336+0000","last_change":"2024-04-16T17:47:57.180106+0000","last_active":"2024-04-16T19:31:56.148336+0000","last_peered":"2024-04-16T19:31:56.148336+0000","last_clean":"2024-04-16T19:31:56.148336+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:31:56.148336+0000","last_undegraded":"2024-04-16T19:31:56.148336+0000","last_fullsized":"2024-04-16T19:31:56.148336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'4","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144008+0000","last_change":"2024-04-16T17:47:55.169894+0000","last_active":"2024-04-16T19:31:51.144008+0000","last_peered":"2024-04-16T19:31:51.144008+0000","last_clean":"2024-04-16T19:31:51.144008+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:31:51.144008+0000","last_undegraded":"2024-04-16T19:31:51.144008+0000","last_fullsized":"2024-04-16T19:31:51.144008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:24.954941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152629+0000","last_change":"2024-04-16T17:47:23.266570+0000","last_active":"2024-04-16T19:32:26.152629+0000","last_peered":"2024-04-16T19:32:26.152629+0000","last_clean":"2024-04-16T19:32:26.152629+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:32:26.152629+0000","last_undegraded":"2024-04-16T19:32:26.152629+0000","last_fullsized":"2024-04-16T19:32:26.152629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:55.441583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152031+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:32:21.152031+0000","last_peered":"2024-04-16T19:32:21.152031+0000","last_clean":"2024-04-16T19:32:21.152031+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:32:21.152031+0000","last_undegraded":"2024-04-16T19:32:21.152031+0000","last_fullsized":"2024-04-16T19:32:21.152031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155126+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:32:36.155126+0000","last_peered":"2024-04-16T19:32:36.155126+0000","last_clean":"2024-04-16T19:32:36.155126+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:32:36.155126+0000","last_undegraded":"2024-04-16T19:32:36.155126+0000","last_fullsized":"2024-04-16T19:32:36.155126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:54.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'152","reported_seq":341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145026+0000","last_change":"2024-04-16T17:47:35.310214+0000","last_active":"2024-04-16T19:31:56.145026+0000","last_peered":"2024-04-16T19:31:56.145026+0000","last_clean":"2024-04-16T19:31:56.145026+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:31:56.145026+0000","last_undegraded":"2024-04-16T19:31:56.145026+0000","last_fullsized":"2024-04-16T19:31:56.145026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.142943+0000","last_change":"2024-04-16T17:47:37.328751+0000","last_active":"2024-04-16T19:31:36.142943+0000","last_peered":"2024-04-16T19:31:36.142943+0000","last_clean":"2024-04-16T19:31:36.142943+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:31:36.142943+0000","last_undegraded":"2024-04-16T19:31:36.142943+0000","last_fullsized":"2024-04-16T19:31:36.142943+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:30:39.392016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144646+0000","last_change":"2024-04-16T17:47:39.336993+0000","last_active":"2024-04-16T19:31:36.144646+0000","last_peered":"2024-04-16T19:31:36.144646+0000","last_clean":"2024-04-16T19:31:36.144646+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:31:36.144646+0000","last_undegraded":"2024-04-16T19:31:36.144646+0000","last_fullsized":"2024-04-16T19:31:36.144646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140940+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:32:31.140940+0000","last_peered":"2024-04-16T19:32:31.140940+0000","last_clean":"2024-04-16T19:32:31.140940+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:32:31.140940+0000","last_undegraded":"2024-04-16T19:32:31.140940+0000","last_fullsized":"2024-04-16T19:32:31.140940+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:16.152440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'99","reported_seq":1431,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174016+0000","last_change":"2024-04-16T17:48:34.341651+0000","last_active":"2024-04-16T19:33:36.174016+0000","last_peered":"2024-04-16T19:33:36.174016+0000","last_clean":"2024-04-16T19:33:36.174016+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:33:36.174016+0000","last_undegraded":"2024-04-16T19:33:36.174016+0000","last_fullsized":"2024-04-16T19:33:36.174016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:45.319846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169918+0000","last_change":"2024-04-16T17:48:27.306641+0000","last_active":"2024-04-16T19:33:31.169918+0000","last_peered":"2024-04-16T19:33:31.169918+0000","last_clean":"2024-04-16T19:33:31.169918+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:33:31.169918+0000","last_undegraded":"2024-04-16T19:33:31.169918+0000","last_fullsized":"2024-04-16T19:33:31.169918+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'7","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164768+0000","last_change":"2024-04-16T17:48:01.263221+0000","last_active":"2024-04-16T19:33:01.164768+0000","last_peered":"2024-04-16T19:33:01.164768+0000","last_clean":"2024-04-16T19:33:01.164768+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:33:01.164768+0000","last_undegraded":"2024-04-16T19:33:01.164768+0000","last_fullsized":"2024-04-16T19:33:01.164768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'555","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132909+0000","last_change":"2024-04-16T17:47:59.183413+0000","last_active":"2024-04-16T19:31:56.132909+0000","last_peered":"2024-04-16T19:31:56.132909+0000","last_clean":"2024-04-16T19:31:56.132909+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:31:56.132909+0000","last_undegraded":"2024-04-16T19:31:56.132909+0000","last_fullsized":"2024-04-16T19:31:56.132909+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:27.860005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'230","reported_seq":460,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172962+0000","last_change":"2024-04-16T17:47:57.173077+0000","last_active":"2024-04-16T19:33:31.172962+0000","last_peered":"2024-04-16T19:33:31.172962+0000","last_clean":"2024-04-16T19:33:31.172962+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:33:31.172962+0000","last_undegraded":"2024-04-16T19:33:31.172962+0000","last_fullsized":"2024-04-16T19:33:31.172962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:49.105681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168703+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:33:31.168703+0000","last_peered":"2024-04-16T19:33:31.168703+0000","last_clean":"2024-04-16T19:33:31.168703+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:33:31.168703+0000","last_undegraded":"2024-04-16T19:33:31.168703+0000","last_fullsized":"2024-04-16T19:33:31.168703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:11.770314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152568+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:32:26.152568+0000","last_peered":"2024-04-16T19:32:26.152568+0000","last_clean":"2024-04-16T19:32:26.152568+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:32:26.152568+0000","last_undegraded":"2024-04-16T19:32:26.152568+0000","last_fullsized":"2024-04-16T19:32:26.152568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:32.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152102+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:32:21.152102+0000","last_peered":"2024-04-16T19:32:21.152102+0000","last_clean":"2024-04-16T19:32:21.152102+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:32:21.152102+0000","last_undegraded":"2024-04-16T19:32:21.152102+0000","last_fullsized":"2024-04-16T19:32:21.152102+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156238+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:32:36.156238+0000","last_peered":"2024-04-16T19:32:36.156238+0000","last_clean":"2024-04-16T19:32:36.156238+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:32:36.156238+0000","last_undegraded":"2024-04-16T19:32:36.156238+0000","last_fullsized":"2024-04-16T19:32:36.156238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367224+0000","last_change":"2024-04-16T17:47:35.303292+0000","last_active":"2024-04-16T19:31:31.367224+0000","last_peered":"2024-04-16T19:31:31.367224+0000","last_clean":"2024-04-16T19:31:31.367224+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:31:31.367224+0000","last_undegraded":"2024-04-16T19:31:31.367224+0000","last_fullsized":"2024-04-16T19:31:31.367224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154187+0000","last_change":"2024-04-16T17:47:37.331029+0000","last_active":"2024-04-16T19:32:11.154187+0000","last_peered":"2024-04-16T19:32:11.154187+0000","last_clean":"2024-04-16T19:32:11.154187+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:32:11.154187+0000","last_undegraded":"2024-04-16T19:32:11.154187+0000","last_fullsized":"2024-04-16T19:32:11.154187+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.143665+0000","last_change":"2024-04-16T17:47:39.700695+0000","last_active":"2024-04-16T19:31:36.143665+0000","last_peered":"2024-04-16T19:31:36.143665+0000","last_clean":"2024-04-16T19:31:36.143665+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:31:36.143665+0000","last_undegraded":"2024-04-16T19:31:36.143665+0000","last_fullsized":"2024-04-16T19:31:36.143665+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157016+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:32:31.157016+0000","last_peered":"2024-04-16T19:32:31.157016+0000","last_clean":"2024-04-16T19:32:31.157016+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:32:31.157016+0000","last_undegraded":"2024-04-16T19:32:31.157016+0000","last_fullsized":"2024-04-16T19:32:31.157016+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'96","reported_seq":919,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156060+0000","last_change":"2024-04-16T17:48:34.324041+0000","last_active":"2024-04-16T19:33:36.156060+0000","last_peered":"2024-04-16T19:33:36.156060+0000","last_clean":"2024-04-16T19:33:36.156060+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:33:36.156060+0000","last_undegraded":"2024-04-16T19:33:36.156060+0000","last_fullsized":"2024-04-16T19:33:36.156060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:18:27.393972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172731+0000","last_change":"2024-04-16T17:48:27.307648+0000","last_active":"2024-04-16T19:33:31.172731+0000","last_peered":"2024-04-16T19:33:31.172731+0000","last_clean":"2024-04-16T19:33:31.172731+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:33:31.172731+0000","last_undegraded":"2024-04-16T19:33:31.172731+0000","last_fullsized":"2024-04-16T19:33:31.172731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:20.265988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'7","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162293+0000","last_change":"2024-04-16T17:48:01.248401+0000","last_active":"2024-04-16T19:33:01.162293+0000","last_peered":"2024-04-16T19:33:01.162293+0000","last_clean":"2024-04-16T19:33:01.162293+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:33:01.162293+0000","last_undegraded":"2024-04-16T19:33:01.162293+0000","last_fullsized":"2024-04-16T19:33:01.162293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145357+0000","last_change":"2024-04-16T17:47:59.199961+0000","last_active":"2024-04-16T19:31:56.145357+0000","last_peered":"2024-04-16T19:31:56.145357+0000","last_clean":"2024-04-16T19:31:56.145357+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:31:56.145357+0000","last_undegraded":"2024-04-16T19:31:56.145357+0000","last_fullsized":"2024-04-16T19:31:56.145357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:44.427887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'129","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150185+0000","last_change":"2024-04-16T17:47:57.189447+0000","last_active":"2024-04-16T19:31:56.150185+0000","last_peered":"2024-04-16T19:31:56.150185+0000","last_clean":"2024-04-16T19:31:56.150185+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:31:56.150185+0000","last_undegraded":"2024-04-16T19:31:56.150185+0000","last_fullsized":"2024-04-16T19:31:56.150185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:55.411174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'3","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171761+0000","last_change":"2024-04-16T17:47:55.160684+0000","last_active":"2024-04-16T19:33:26.171761+0000","last_peered":"2024-04-16T19:33:26.171761+0000","last_clean":"2024-04-16T19:33:26.171761+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:33:26.171761+0000","last_undegraded":"2024-04-16T19:33:26.171761+0000","last_fullsized":"2024-04-16T19:33:26.171761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148167+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:31:56.148167+0000","last_peered":"2024-04-16T19:31:56.148167+0000","last_clean":"2024-04-16T19:31:56.148167+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:31:56.148167+0000","last_undegraded":"2024-04-16T19:31:56.148167+0000","last_fullsized":"2024-04-16T19:31:56.148167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159976+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:33:51.159976+0000","last_peered":"2024-04-16T19:33:51.159976+0000","last_clean":"2024-04-16T19:33:51.159976+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:33:51.159976+0000","last_undegraded":"2024-04-16T19:33:51.159976+0000","last_fullsized":"2024-04-16T19:33:51.159976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:50:28.662489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361145+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:31:31.361145+0000","last_peered":"2024-04-16T19:31:31.361145+0000","last_clean":"2024-04-16T19:31:31.361145+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:31:31.361145+0000","last_undegraded":"2024-04-16T19:31:31.361145+0000","last_fullsized":"2024-04-16T19:31:31.361145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:03.171529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'32","reported_seq":160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158251+0000","last_change":"2024-04-16T17:47:35.317051+0000","last_active":"2024-04-16T19:32:36.158251+0000","last_peered":"2024-04-16T19:32:36.158251+0000","last_clean":"2024-04-16T19:32:36.158251+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:32:36.158251+0000","last_undegraded":"2024-04-16T19:32:36.158251+0000","last_fullsized":"2024-04-16T19:32:36.158251+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162398+0000","last_change":"2024-04-16T17:47:37.320351+0000","last_active":"2024-04-16T19:32:41.162398+0000","last_peered":"2024-04-16T19:32:41.162398+0000","last_clean":"2024-04-16T19:32:41.162398+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:32:41.162398+0000","last_undegraded":"2024-04-16T19:32:41.162398+0000","last_fullsized":"2024-04-16T19:32:41.162398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144501+0000","last_change":"2024-04-16T17:47:39.329617+0000","last_active":"2024-04-16T19:31:36.144501+0000","last_peered":"2024-04-16T19:31:36.144501+0000","last_clean":"2024-04-16T19:31:36.144501+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:31:36.144501+0000","last_undegraded":"2024-04-16T19:31:36.144501+0000","last_fullsized":"2024-04-16T19:31:36.144501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157983+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:32:31.157983+0000","last_peered":"2024-04-16T19:32:31.157983+0000","last_clean":"2024-04-16T19:32:31.157983+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:32:31.157983+0000","last_undegraded":"2024-04-16T19:32:31.157983+0000","last_fullsized":"2024-04-16T19:32:31.157983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:49:40.225240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'126","reported_seq":1450,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156003+0000","last_change":"2024-04-16T17:48:34.338811+0000","last_active":"2024-04-16T19:33:36.156003+0000","last_peered":"2024-04-16T19:33:36.156003+0000","last_clean":"2024-04-16T19:33:36.156003+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:33:36.156003+0000","last_undegraded":"2024-04-16T19:33:36.156003+0000","last_fullsized":"2024-04-16T19:33:36.156003+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:09:10.563495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172826+0000","last_change":"2024-04-16T17:48:27.288374+0000","last_active":"2024-04-16T19:33:31.172826+0000","last_peered":"2024-04-16T19:33:31.172826+0000","last_clean":"2024-04-16T19:33:31.172826+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:33:31.172826+0000","last_undegraded":"2024-04-16T19:33:31.172826+0000","last_fullsized":"2024-04-16T19:33:31.172826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:11.751579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164610+0000","last_change":"2024-04-16T17:48:01.263690+0000","last_active":"2024-04-16T19:33:01.164610+0000","last_peered":"2024-04-16T19:33:01.164610+0000","last_clean":"2024-04-16T19:33:01.164610+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:33:01.164610+0000","last_undegraded":"2024-04-16T19:33:01.164610+0000","last_fullsized":"2024-04-16T19:33:01.164610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:47.176953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145421+0000","last_change":"2024-04-16T17:47:59.199357+0000","last_active":"2024-04-16T19:31:56.145421+0000","last_peered":"2024-04-16T19:31:56.145421+0000","last_clean":"2024-04-16T19:31:56.145421+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:31:56.145421+0000","last_undegraded":"2024-04-16T19:31:56.145421+0000","last_fullsized":"2024-04-16T19:31:56.145421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:42.077363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11055","reported_seq":18133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150080+0000","last_change":"2024-04-16T17:47:57.184044+0000","last_active":"2024-04-16T19:31:56.150080+0000","last_peered":"2024-04-16T19:31:56.150080+0000","last_clean":"2024-04-16T19:31:56.150080+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:31:56.150080+0000","last_undegraded":"2024-04-16T19:31:56.150080+0000","last_fullsized":"2024-04-16T19:31:56.150080+0000","mapping_epoch":34,"log_start":"53'9200","ondisk_log_start":"53'9200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1855,"log_dups_size":3000,"ondisk_log_size":1855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'6","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143908+0000","last_change":"2024-04-16T17:47:55.159410+0000","last_active":"2024-04-16T19:31:51.143908+0000","last_peered":"2024-04-16T19:31:51.143908+0000","last_clean":"2024-04-16T19:31:51.143908+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:31:51.143908+0000","last_undegraded":"2024-04-16T19:31:51.143908+0000","last_fullsized":"2024-04-16T19:31:51.143908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:42.685205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179184+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:33:46.179184+0000","last_peered":"2024-04-16T19:33:46.179184+0000","last_clean":"2024-04-16T19:33:46.179184+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:33:46.179184+0000","last_undegraded":"2024-04-16T19:33:46.179184+0000","last_fullsized":"2024-04-16T19:33:46.179184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:54.182792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173750+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:33:36.173750+0000","last_peered":"2024-04-16T19:33:36.173750+0000","last_clean":"2024-04-16T19:33:36.173750+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:33:36.173750+0000","last_undegraded":"2024-04-16T19:33:36.173750+0000","last_fullsized":"2024-04-16T19:33:36.173750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149851+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:32:01.149851+0000","last_peered":"2024-04-16T19:32:01.149851+0000","last_clean":"2024-04-16T19:32:01.149851+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:32:01.149851+0000","last_undegraded":"2024-04-16T19:32:01.149851+0000","last_fullsized":"2024-04-16T19:32:01.149851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:53.316949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'80","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155149+0000","last_change":"2024-04-16T17:47:35.308166+0000","last_active":"2024-04-16T19:32:36.155149+0000","last_peered":"2024-04-16T19:32:36.155149+0000","last_clean":"2024-04-16T19:32:36.155149+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:32:36.155149+0000","last_undegraded":"2024-04-16T19:32:36.155149+0000","last_fullsized":"2024-04-16T19:32:36.155149+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:53.924341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159795+0000","last_change":"2024-04-16T17:47:37.321988+0000","last_active":"2024-04-16T19:32:41.159795+0000","last_peered":"2024-04-16T19:32:41.159795+0000","last_clean":"2024-04-16T19:32:41.159795+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:32:41.159795+0000","last_undegraded":"2024-04-16T19:32:41.159795+0000","last_fullsized":"2024-04-16T19:32:41.159795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164797+0000","last_change":"2024-04-16T17:47:39.326823+0000","last_active":"2024-04-16T19:33:16.164797+0000","last_peered":"2024-04-16T19:33:16.164797+0000","last_clean":"2024-04-16T19:33:16.164797+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:33:16.164797+0000","last_undegraded":"2024-04-16T19:33:16.164797+0000","last_fullsized":"2024-04-16T19:33:16.164797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:26.874144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'22","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157286+0000","last_change":"2024-04-16T17:47:55.158595+0000","last_active":"2024-04-16T19:33:41.157286+0000","last_peered":"2024-04-16T19:33:41.157286+0000","last_clean":"2024-04-16T19:33:41.157286+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:33:41.157286+0000","last_undegraded":"2024-04-16T19:33:41.157286+0000","last_fullsized":"2024-04-16T19:33:41.157286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:20.899420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158764+0000","last_change":"2024-04-16T17:47:32.278320+0000","last_active":"2024-04-16T19:33:46.158764+0000","last_peered":"2024-04-16T19:33:46.158764+0000","last_clean":"2024-04-16T19:33:46.158764+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:33:46.158764+0000","last_undegraded":"2024-04-16T19:33:46.158764+0000","last_fullsized":"2024-04-16T19:33:46.158764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:32.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169629+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:33:36.169629+0000","last_peered":"2024-04-16T19:33:36.169629+0000","last_clean":"2024-04-16T19:33:36.169629+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:33:36.169629+0000","last_undegraded":"2024-04-16T19:33:36.169629+0000","last_fullsized":"2024-04-16T19:33:36.169629+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'3","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172656+0000","last_change":"2024-04-16T17:47:55.175820+0000","last_active":"2024-04-16T19:33:41.172656+0000","last_peered":"2024-04-16T19:33:41.172656+0000","last_clean":"2024-04-16T19:33:41.172656+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:33:41.172656+0000","last_undegraded":"2024-04-16T19:33:41.172656+0000","last_fullsized":"2024-04-16T19:33:41.172656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174508+0000","last_change":"2024-04-16T17:47:23.260359+0000","last_active":"2024-04-16T19:33:36.174508+0000","last_peered":"2024-04-16T19:33:36.174508+0000","last_clean":"2024-04-16T19:33:36.174508+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:33:36.174508+0000","last_undegraded":"2024-04-16T19:33:36.174508+0000","last_fullsized":"2024-04-16T19:33:36.174508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:31.902037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.155944+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:33:36.155944+0000","last_peered":"2024-04-16T19:33:36.155944+0000","last_clean":"2024-04-16T19:33:36.155944+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:33:36.155944+0000","last_undegraded":"2024-04-16T19:33:36.155944+0000","last_fullsized":"2024-04-16T19:33:36.155944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:39.513573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'7","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151820+0000","last_change":"2024-04-16T17:47:55.178677+0000","last_active":"2024-04-16T19:32:21.151820+0000","last_peered":"2024-04-16T19:32:21.151820+0000","last_clean":"2024-04-16T19:32:21.151820+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:32:21.151820+0000","last_undegraded":"2024-04-16T19:32:21.151820+0000","last_fullsized":"2024-04-16T19:32:21.151820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.171376+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:33:36.171376+0000","last_peered":"2024-04-16T19:33:36.171376+0000","last_clean":"2024-04-16T19:33:36.171376+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:33:36.171376+0000","last_undegraded":"2024-04-16T19:33:36.171376+0000","last_fullsized":"2024-04-16T19:33:36.171376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176213+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:33:36.176213+0000","last_peered":"2024-04-16T19:33:36.176213+0000","last_clean":"2024-04-16T19:33:36.176213+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:33:36.176213+0000","last_undegraded":"2024-04-16T19:33:36.176213+0000","last_fullsized":"2024-04-16T19:33:36.176213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'6","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177717+0000","last_change":"2024-04-16T17:47:55.156061+0000","last_active":"2024-04-16T19:33:41.177717+0000","last_peered":"2024-04-16T19:33:41.177717+0000","last_clean":"2024-04-16T19:33:41.177717+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:33:41.177717+0000","last_undegraded":"2024-04-16T19:33:41.177717+0000","last_fullsized":"2024-04-16T19:33:41.177717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174362+0000","last_change":"2024-04-16T17:47:23.261639+0000","last_active":"2024-04-16T19:33:36.174362+0000","last_peered":"2024-04-16T19:33:36.174362+0000","last_clean":"2024-04-16T19:33:36.174362+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:33:36.174362+0000","last_undegraded":"2024-04-16T19:33:36.174362+0000","last_fullsized":"2024-04-16T19:33:36.174362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150697+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:32:06.150697+0000","last_peered":"2024-04-16T19:32:06.150697+0000","last_clean":"2024-04-16T19:32:06.150697+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:32:06.150697+0000","last_undegraded":"2024-04-16T19:32:06.150697+0000","last_fullsized":"2024-04-16T19:32:06.150697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"48'3","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161266+0000","last_change":"2024-04-16T17:47:55.181215+0000","last_active":"2024-04-16T19:32:56.161266+0000","last_peered":"2024-04-16T19:32:56.161266+0000","last_clean":"2024-04-16T19:32:56.161266+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:32:56.161266+0000","last_undegraded":"2024-04-16T19:32:56.161266+0000","last_fullsized":"2024-04-16T19:32:56.161266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135310+0000","last_change":"2024-04-16T17:47:23.253450+0000","last_active":"2024-04-16T19:32:06.135310+0000","last_peered":"2024-04-16T19:32:06.135310+0000","last_clean":"2024-04-16T19:32:06.135310+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:32:06.135310+0000","last_undegraded":"2024-04-16T19:32:06.135310+0000","last_fullsized":"2024-04-16T19:32:06.135310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:44.749396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172146+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:33:31.172146+0000","last_peered":"2024-04-16T19:33:31.172146+0000","last_clean":"2024-04-16T19:33:31.172146+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:33:31.172146+0000","last_undegraded":"2024-04-16T19:33:31.172146+0000","last_fullsized":"2024-04-16T19:33:31.172146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"48'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146820+0000","last_change":"2024-04-16T17:47:55.165525+0000","last_active":"2024-04-16T19:32:56.146820+0000","last_peered":"2024-04-16T19:32:56.146820+0000","last_clean":"2024-04-16T19:32:56.146820+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:32:56.146820+0000","last_undegraded":"2024-04-16T19:32:56.146820+0000","last_fullsized":"2024-04-16T19:32:56.146820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:20:06.245260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171037+0000","last_change":"2024-04-16T17:47:23.257600+0000","last_active":"2024-04-16T19:33:41.171037+0000","last_peered":"2024-04-16T19:33:41.171037+0000","last_clean":"2024-04-16T19:33:41.171037+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:33:41.171037+0000","last_undegraded":"2024-04-16T19:33:41.171037+0000","last_fullsized":"2024-04-16T19:33:41.171037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:11.143975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.173036+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:33:31.173036+0000","last_peered":"2024-04-16T19:33:31.173036+0000","last_clean":"2024-04-16T19:33:31.173036+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:33:31.173036+0000","last_undegraded":"2024-04-16T19:33:31.173036+0000","last_fullsized":"2024-04-16T19:33:31.173036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'40","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175919+0000","last_change":"2024-04-16T17:47:55.160972+0000","last_active":"2024-04-16T19:33:41.175919+0000","last_peered":"2024-04-16T19:33:41.175919+0000","last_clean":"2024-04-16T19:33:41.175919+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:33:41.175919+0000","last_undegraded":"2024-04-16T19:33:41.175919+0000","last_fullsized":"2024-04-16T19:33:41.175919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159832+0000","last_change":"2024-04-16T17:47:32.281080+0000","last_active":"2024-04-16T19:33:51.159832+0000","last_peered":"2024-04-16T19:33:51.159832+0000","last_clean":"2024-04-16T19:33:51.159832+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:33:51.159832+0000","last_undegraded":"2024-04-16T19:33:51.159832+0000","last_fullsized":"2024-04-16T19:33:51.159832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:58.918776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175091+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:33:31.175091+0000","last_peered":"2024-04-16T19:33:31.175091+0000","last_clean":"2024-04-16T19:33:31.175091+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:33:31.175091+0000","last_undegraded":"2024-04-16T19:33:31.175091+0000","last_fullsized":"2024-04-16T19:33:31.175091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171222+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:33:41.171222+0000","last_peered":"2024-04-16T19:33:41.171222+0000","last_clean":"2024-04-16T19:33:41.171222+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:33:41.171222+0000","last_undegraded":"2024-04-16T19:33:41.171222+0000","last_fullsized":"2024-04-16T19:33:41.171222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:26.938403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177647+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:33:41.177647+0000","last_peered":"2024-04-16T19:33:41.177647+0000","last_clean":"2024-04-16T19:33:41.177647+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:33:41.177647+0000","last_undegraded":"2024-04-16T19:33:41.177647+0000","last_fullsized":"2024-04-16T19:33:41.177647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154939+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:33:31.154939+0000","last_peered":"2024-04-16T19:33:31.154939+0000","last_clean":"2024-04-16T19:33:31.154939+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:33:31.154939+0000","last_undegraded":"2024-04-16T19:33:31.154939+0000","last_fullsized":"2024-04-16T19:33:31.154939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:12.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'15","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152554+0000","last_change":"2024-04-16T17:47:55.170559+0000","last_active":"2024-04-16T19:32:06.152554+0000","last_peered":"2024-04-16T19:32:06.152554+0000","last_clean":"2024-04-16T19:32:06.152554+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:32:06.152554+0000","last_undegraded":"2024-04-16T19:32:06.152554+0000","last_fullsized":"2024-04-16T19:32:06.152554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155962+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:32:26.155962+0000","last_peered":"2024-04-16T19:32:26.155962+0000","last_clean":"2024-04-16T19:32:26.155962+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:32:26.155962+0000","last_undegraded":"2024-04-16T19:32:26.155962+0000","last_fullsized":"2024-04-16T19:32:26.155962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176559+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:33:56.176559+0000","last_peered":"2024-04-16T19:33:56.176559+0000","last_clean":"2024-04-16T19:33:56.176559+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:33:56.176559+0000","last_undegraded":"2024-04-16T19:33:56.176559+0000","last_fullsized":"2024-04-16T19:33:56.176559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:44.519947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'18","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142162+0000","last_change":"2024-04-16T17:47:55.177250+0000","last_active":"2024-04-16T19:32:36.142162+0000","last_peered":"2024-04-16T19:32:36.142162+0000","last_clean":"2024-04-16T19:32:36.142162+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:32:36.142162+0000","last_undegraded":"2024-04-16T19:32:36.142162+0000","last_fullsized":"2024-04-16T19:32:36.142162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:42.691526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152860+0000","last_change":"2024-04-16T17:47:23.264252+0000","last_active":"2024-04-16T19:32:26.152860+0000","last_peered":"2024-04-16T19:32:26.152860+0000","last_clean":"2024-04-16T19:32:26.152860+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:32:26.152860+0000","last_undegraded":"2024-04-16T19:32:26.152860+0000","last_fullsized":"2024-04-16T19:32:26.152860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166011+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:33:06.166011+0000","last_peered":"2024-04-16T19:33:06.166011+0000","last_clean":"2024-04-16T19:33:06.166011+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:33:06.166011+0000","last_undegraded":"2024-04-16T19:33:06.166011+0000","last_fullsized":"2024-04-16T19:33:06.166011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'1","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150517+0000","last_change":"2024-04-16T17:47:55.168092+0000","last_active":"2024-04-16T19:32:06.150517+0000","last_peered":"2024-04-16T19:32:06.150517+0000","last_clean":"2024-04-16T19:32:06.150517+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:32:06.150517+0000","last_undegraded":"2024-04-16T19:32:06.150517+0000","last_fullsized":"2024-04-16T19:32:06.150517+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157395+0000","last_change":"2024-04-16T17:47:23.265700+0000","last_active":"2024-04-16T19:33:41.157395+0000","last_peered":"2024-04-16T19:33:41.157395+0000","last_clean":"2024-04-16T19:33:41.157395+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:33:41.157395+0000","last_undegraded":"2024-04-16T19:33:41.157395+0000","last_fullsized":"2024-04-16T19:33:41.157395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:24:05.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163673+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:33:06.163673+0000","last_peered":"2024-04-16T19:33:06.163673+0000","last_clean":"2024-04-16T19:33:06.163673+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:33:06.163673+0000","last_undegraded":"2024-04-16T19:33:06.163673+0000","last_fullsized":"2024-04-16T19:33:06.163673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'18","reported_seq":243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135182+0000","last_change":"2024-04-16T17:47:55.147785+0000","last_active":"2024-04-16T19:32:06.135182+0000","last_peered":"2024-04-16T19:32:06.135182+0000","last_clean":"2024-04-16T19:32:06.135182+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:32:06.135182+0000","last_undegraded":"2024-04-16T19:32:06.135182+0000","last_fullsized":"2024-04-16T19:32:06.135182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:26.488442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170252+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:33:11.170252+0000","last_peered":"2024-04-16T19:33:11.170252+0000","last_clean":"2024-04-16T19:33:11.170252+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:33:11.170252+0000","last_undegraded":"2024-04-16T19:33:11.170252+0000","last_fullsized":"2024-04-16T19:33:11.170252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166111+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:33:06.166111+0000","last_peered":"2024-04-16T19:33:06.166111+0000","last_clean":"2024-04-16T19:33:06.166111+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:33:06.166111+0000","last_undegraded":"2024-04-16T19:33:06.166111+0000","last_fullsized":"2024-04-16T19:33:06.166111+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145325+0000","last_change":"2024-04-16T17:47:55.183714+0000","last_active":"2024-04-16T19:31:56.145325+0000","last_peered":"2024-04-16T19:31:56.145325+0000","last_clean":"2024-04-16T19:31:56.145325+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:31:56.145325+0000","last_undegraded":"2024-04-16T19:31:56.145325+0000","last_fullsized":"2024-04-16T19:31:56.145325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163558+0000","last_change":"2024-04-16T17:47:23.265903+0000","last_active":"2024-04-16T19:33:11.163558+0000","last_peered":"2024-04-16T19:33:11.163558+0000","last_clean":"2024-04-16T19:33:11.163558+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:33:11.163558+0000","last_undegraded":"2024-04-16T19:33:11.163558+0000","last_fullsized":"2024-04-16T19:33:11.163558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:55.605376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168971+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:33:06.168971+0000","last_peered":"2024-04-16T19:33:06.168971+0000","last_clean":"2024-04-16T19:33:06.168971+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:33:06.168971+0000","last_undegraded":"2024-04-16T19:33:06.168971+0000","last_fullsized":"2024-04-16T19:33:06.168971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:14.412040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"48'2","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132984+0000","last_change":"2024-04-16T17:47:55.162693+0000","last_active":"2024-04-16T19:31:56.132984+0000","last_peered":"2024-04-16T19:31:56.132984+0000","last_clean":"2024-04-16T19:31:56.132984+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:31:56.132984+0000","last_undegraded":"2024-04-16T19:31:56.132984+0000","last_fullsized":"2024-04-16T19:31:56.132984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:26:14.047560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167227+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:33:11.167227+0000","last_peered":"2024-04-16T19:33:11.167227+0000","last_clean":"2024-04-16T19:33:11.167227+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:33:11.167227+0000","last_undegraded":"2024-04-16T19:33:11.167227+0000","last_fullsized":"2024-04-16T19:33:11.167227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177980+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:33:41.177980+0000","last_peered":"2024-04-16T19:33:41.177980+0000","last_clean":"2024-04-16T19:33:41.177980+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:33:41.177980+0000","last_undegraded":"2024-04-16T19:33:41.177980+0000","last_fullsized":"2024-04-16T19:33:41.177980+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'3","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132884+0000","last_change":"2024-04-16T17:47:55.178234+0000","last_active":"2024-04-16T19:31:56.132884+0000","last_peered":"2024-04-16T19:31:56.132884+0000","last_clean":"2024-04-16T19:31:56.132884+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:31:56.132884+0000","last_undegraded":"2024-04-16T19:31:56.132884+0000","last_fullsized":"2024-04-16T19:31:56.132884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:17.561808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167956+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:33:11.167956+0000","last_peered":"2024-04-16T19:33:11.167956+0000","last_clean":"2024-04-16T19:33:11.167956+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:33:11.167956+0000","last_undegraded":"2024-04-16T19:33:11.167956+0000","last_fullsized":"2024-04-16T19:33:11.167956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138794+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:32:21.138794+0000","last_peered":"2024-04-16T19:32:21.138794+0000","last_clean":"2024-04-16T19:32:21.138794+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:32:21.138794+0000","last_undegraded":"2024-04-16T19:32:21.138794+0000","last_fullsized":"2024-04-16T19:32:21.138794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:40.793750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'6","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148370+0000","last_change":"2024-04-16T17:47:55.177975+0000","last_active":"2024-04-16T19:31:56.148370+0000","last_peered":"2024-04-16T19:31:56.148370+0000","last_clean":"2024-04-16T19:31:56.148370+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:31:56.148370+0000","last_undegraded":"2024-04-16T19:31:56.148370+0000","last_fullsized":"2024-04-16T19:31:56.148370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:32.181968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151087+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:32:01.151087+0000","last_peered":"2024-04-16T19:32:01.151087+0000","last_clean":"2024-04-16T19:32:01.151087+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:32:01.151087+0000","last_undegraded":"2024-04-16T19:32:01.151087+0000","last_fullsized":"2024-04-16T19:32:01.151087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152209+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:32:21.152209+0000","last_peered":"2024-04-16T19:32:21.152209+0000","last_clean":"2024-04-16T19:32:21.152209+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:32:21.152209+0000","last_undegraded":"2024-04-16T19:32:21.152209+0000","last_fullsized":"2024-04-16T19:32:21.152209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'10","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163428+0000","last_change":"2024-04-16T17:47:55.156240+0000","last_active":"2024-04-16T19:32:56.163428+0000","last_peered":"2024-04-16T19:32:56.163428+0000","last_clean":"2024-04-16T19:32:56.163428+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:32:56.163428+0000","last_undegraded":"2024-04-16T19:32:56.163428+0000","last_fullsized":"2024-04-16T19:32:56.163428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156891+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:32:21.156891+0000","last_peered":"2024-04-16T19:32:21.156891+0000","last_clean":"2024-04-16T19:32:21.156891+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:32:21.156891+0000","last_undegraded":"2024-04-16T19:32:21.156891+0000","last_fullsized":"2024-04-16T19:32:21.156891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148441+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:31:56.148441+0000","last_peered":"2024-04-16T19:31:56.148441+0000","last_clean":"2024-04-16T19:31:56.148441+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:31:56.148441+0000","last_undegraded":"2024-04-16T19:31:56.148441+0000","last_fullsized":"2024-04-16T19:31:56.148441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'14","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146712+0000","last_change":"2024-04-16T17:47:55.164923+0000","last_active":"2024-04-16T19:32:56.146712+0000","last_peered":"2024-04-16T19:32:56.146712+0000","last_clean":"2024-04-16T19:32:56.146712+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:32:56.146712+0000","last_undegraded":"2024-04-16T19:32:56.146712+0000","last_fullsized":"2024-04-16T19:32:56.146712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:25.251453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.155204+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:32:21.155204+0000","last_peered":"2024-04-16T19:32:21.155204+0000","last_clean":"2024-04-16T19:32:21.155204+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:32:21.155204+0000","last_undegraded":"2024-04-16T19:32:21.155204+0000","last_fullsized":"2024-04-16T19:32:21.155204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:30.655498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174246+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:33:46.174246+0000","last_peered":"2024-04-16T19:33:46.174246+0000","last_clean":"2024-04-16T19:33:46.174246+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:33:46.174246+0000","last_undegraded":"2024-04-16T19:33:46.174246+0000","last_fullsized":"2024-04-16T19:33:46.174246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2844,"ondisk_log_size":2844,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3706,"ondisk_log_size":3706,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":623,"ondisk_log_size":623,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27355,"ondisk_log_size":27355,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4460,"ondisk_log_size":4460,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":437,"ondisk_log_size":437,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641575,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46584,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811016,"statfs":{"total":107374182400,"available":107326480384,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641574,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641574,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":99932,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":104757668,"statfs":{"total":107374182400,"available":107271852032,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641574,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46572,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811028,"statfs":{"total":107374182400,"available":107326492672,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641575,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":55408,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":104802192,"statfs":{"total":107374182400,"available":107317444608,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641575,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104829060,"statfs":{"total":107374182400,"available":107344957440,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:36:13.841 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:36:14.217 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:36:14.236 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:36:14.244 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3334,"stamp":"2024-04-16T19:36:12.315071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39759,"ondisk_log_size":39759,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":354032,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":343862,"kb_avail":628791568,"statfs":{"total":644245094400,"available":643882565632,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":352115360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008536"},"pg_stats":[{"pgid":"9.35","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147548+0000","last_change":"2024-04-16T17:47:55.160868+0000","last_active":"2024-04-16T19:31:51.147548+0000","last_peered":"2024-04-16T19:31:51.147548+0000","last_clean":"2024-04-16T19:31:51.147548+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:31:51.147548+0000","last_undegraded":"2024-04-16T19:31:51.147548+0000","last_fullsized":"2024-04-16T19:31:51.147548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176615+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:33:36.176615+0000","last_peered":"2024-04-16T19:33:36.176615+0000","last_clean":"2024-04-16T19:33:36.176615+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:33:36.176615+0000","last_undegraded":"2024-04-16T19:33:36.176615+0000","last_fullsized":"2024-04-16T19:33:36.176615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:51.288503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153219+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:32:26.153219+0000","last_peered":"2024-04-16T19:32:26.153219+0000","last_clean":"2024-04-16T19:32:26.153219+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:32:26.153219+0000","last_undegraded":"2024-04-16T19:32:26.153219+0000","last_fullsized":"2024-04-16T19:32:26.153219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'14","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143970+0000","last_change":"2024-04-16T17:47:55.173885+0000","last_active":"2024-04-16T19:31:51.143970+0000","last_peered":"2024-04-16T19:31:51.143970+0000","last_clean":"2024-04-16T19:31:51.143970+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:31:51.143970+0000","last_undegraded":"2024-04-16T19:31:51.143970+0000","last_fullsized":"2024-04-16T19:31:51.143970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:50.786411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174452+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:33:36.174452+0000","last_peered":"2024-04-16T19:33:36.174452+0000","last_clean":"2024-04-16T19:33:36.174452+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:33:36.174452+0000","last_undegraded":"2024-04-16T19:33:36.174452+0000","last_fullsized":"2024-04-16T19:33:36.174452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.158347+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:32:26.158347+0000","last_peered":"2024-04-16T19:32:26.158347+0000","last_clean":"2024-04-16T19:32:26.158347+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:32:26.158347+0000","last_undegraded":"2024-04-16T19:32:26.158347+0000","last_fullsized":"2024-04-16T19:32:26.158347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'20","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152906+0000","last_change":"2024-04-16T17:47:55.156902+0000","last_active":"2024-04-16T19:32:26.152906+0000","last_peered":"2024-04-16T19:32:26.152906+0000","last_clean":"2024-04-16T19:32:26.152906+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:32:26.152906+0000","last_undegraded":"2024-04-16T19:32:26.152906+0000","last_fullsized":"2024-04-16T19:32:26.152906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:40.884246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173569+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:33:36.173569+0000","last_peered":"2024-04-16T19:33:36.173569+0000","last_clean":"2024-04-16T19:33:36.173569+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:33:36.173569+0000","last_undegraded":"2024-04-16T19:33:36.173569+0000","last_fullsized":"2024-04-16T19:33:36.173569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152536+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:32:26.152536+0000","last_peered":"2024-04-16T19:32:26.152536+0000","last_clean":"2024-04-16T19:32:26.152536+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:32:26.152536+0000","last_undegraded":"2024-04-16T19:32:26.152536+0000","last_fullsized":"2024-04-16T19:32:26.152536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:32.905270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147603+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:31:51.147603+0000","last_peered":"2024-04-16T19:31:51.147603+0000","last_clean":"2024-04-16T19:31:51.147603+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:31:51.147603+0000","last_undegraded":"2024-04-16T19:31:51.147603+0000","last_fullsized":"2024-04-16T19:31:51.147603+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:10.585096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174129+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:33:46.174129+0000","last_peered":"2024-04-16T19:33:46.174129+0000","last_clean":"2024-04-16T19:33:46.174129+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:33:46.174129+0000","last_undegraded":"2024-04-16T19:33:46.174129+0000","last_fullsized":"2024-04-16T19:33:46.174129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:53.592351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155669+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:32:26.155669+0000","last_peered":"2024-04-16T19:32:26.155669+0000","last_clean":"2024-04-16T19:32:26.155669+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:32:26.155669+0000","last_undegraded":"2024-04-16T19:32:26.155669+0000","last_fullsized":"2024-04-16T19:32:26.155669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'4","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159892+0000","last_change":"2024-04-16T17:47:55.159892+0000","last_active":"2024-04-16T19:32:56.159892+0000","last_peered":"2024-04-16T19:32:56.159892+0000","last_clean":"2024-04-16T19:32:56.159892+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:32:56.159892+0000","last_undegraded":"2024-04-16T19:32:56.159892+0000","last_fullsized":"2024-04-16T19:32:56.159892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:58.461789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159155+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:32:36.159155+0000","last_peered":"2024-04-16T19:32:36.159155+0000","last_clean":"2024-04-16T19:32:36.159155+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:32:36.159155+0000","last_undegraded":"2024-04-16T19:32:36.159155+0000","last_fullsized":"2024-04-16T19:32:36.159155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175465+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:33:51.175465+0000","last_peered":"2024-04-16T19:33:51.175465+0000","last_clean":"2024-04-16T19:33:51.175465+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:33:51.175465+0000","last_undegraded":"2024-04-16T19:33:51.175465+0000","last_fullsized":"2024-04-16T19:33:51.175465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'11","reported_seq":309,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163313+0000","last_change":"2024-04-16T17:47:55.163604+0000","last_active":"2024-04-16T19:32:56.163313+0000","last_peered":"2024-04-16T19:32:56.163313+0000","last_clean":"2024-04-16T19:32:56.163313+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:32:56.163313+0000","last_undegraded":"2024-04-16T19:32:56.163313+0000","last_fullsized":"2024-04-16T19:32:56.163313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151934+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:32:21.151934+0000","last_peered":"2024-04-16T19:32:21.151934+0000","last_clean":"2024-04-16T19:32:21.151934+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:32:21.151934+0000","last_undegraded":"2024-04-16T19:32:21.151934+0000","last_fullsized":"2024-04-16T19:32:21.151934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171180+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:33:41.171180+0000","last_peered":"2024-04-16T19:33:41.171180+0000","last_clean":"2024-04-16T19:33:41.171180+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:33:41.171180+0000","last_undegraded":"2024-04-16T19:33:41.171180+0000","last_fullsized":"2024-04-16T19:33:41.171180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:47.915323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'12","reported_seq":268,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.148697+0000","last_change":"2024-04-16T17:47:55.161748+0000","last_active":"2024-04-16T19:31:51.148697+0000","last_peered":"2024-04-16T19:31:51.148697+0000","last_clean":"2024-04-16T19:31:51.148697+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:31:51.148697+0000","last_undegraded":"2024-04-16T19:31:51.148697+0000","last_fullsized":"2024-04-16T19:31:51.148697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157014+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:32:21.157014+0000","last_peered":"2024-04-16T19:32:21.157014+0000","last_clean":"2024-04-16T19:32:21.157014+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:32:21.157014+0000","last_undegraded":"2024-04-16T19:32:21.157014+0000","last_fullsized":"2024-04-16T19:32:21.157014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:14.655507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172868+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:33:41.172868+0000","last_peered":"2024-04-16T19:33:41.172868+0000","last_clean":"2024-04-16T19:33:41.172868+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:33:41.172868+0000","last_undegraded":"2024-04-16T19:33:41.172868+0000","last_fullsized":"2024-04-16T19:33:41.172868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'8","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168449+0000","last_change":"2024-04-16T17:47:55.164548+0000","last_active":"2024-04-16T19:33:31.168449+0000","last_peered":"2024-04-16T19:33:31.168449+0000","last_clean":"2024-04-16T19:33:31.168449+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:33:31.168449+0000","last_undegraded":"2024-04-16T19:33:31.168449+0000","last_fullsized":"2024-04-16T19:33:31.168449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:27.967153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138549+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:32:21.138549+0000","last_peered":"2024-04-16T19:32:21.138549+0000","last_clean":"2024-04-16T19:32:21.138549+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:32:21.138549+0000","last_undegraded":"2024-04-16T19:32:21.138549+0000","last_fullsized":"2024-04-16T19:32:21.138549+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:19:39.583925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157481+0000","last_change":"2024-04-16T17:47:23.254199+0000","last_active":"2024-04-16T19:33:41.157481+0000","last_peered":"2024-04-16T19:33:41.157481+0000","last_clean":"2024-04-16T19:33:41.157481+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:33:41.157481+0000","last_undegraded":"2024-04-16T19:33:41.157481+0000","last_fullsized":"2024-04-16T19:33:41.157481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:11.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"48'5","reported_seq":300,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148127+0000","last_change":"2024-04-16T17:47:55.184230+0000","last_active":"2024-04-16T19:32:06.148127+0000","last_peered":"2024-04-16T19:32:06.148127+0000","last_clean":"2024-04-16T19:32:06.148127+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:32:06.148127+0000","last_undegraded":"2024-04-16T19:32:06.148127+0000","last_fullsized":"2024-04-16T19:32:06.148127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152428+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:32:26.152428+0000","last_peered":"2024-04-16T19:32:26.152428+0000","last_clean":"2024-04-16T19:32:26.152428+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:32:26.152428+0000","last_undegraded":"2024-04-16T19:32:26.152428+0000","last_fullsized":"2024-04-16T19:32:26.152428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150380+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:32:01.150380+0000","last_peered":"2024-04-16T19:32:01.150380+0000","last_clean":"2024-04-16T19:32:01.150380+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:32:01.150380+0000","last_undegraded":"2024-04-16T19:32:01.150380+0000","last_fullsized":"2024-04-16T19:32:01.150380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'14","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159666+0000","last_change":"2024-04-16T17:47:55.163042+0000","last_active":"2024-04-16T19:32:31.159666+0000","last_peered":"2024-04-16T19:32:31.159666+0000","last_clean":"2024-04-16T19:32:31.159666+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:32:31.159666+0000","last_undegraded":"2024-04-16T19:32:31.159666+0000","last_fullsized":"2024-04-16T19:32:31.159666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153260+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:32:26.153260+0000","last_peered":"2024-04-16T19:32:26.153260+0000","last_clean":"2024-04-16T19:32:26.153260+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:32:26.153260+0000","last_undegraded":"2024-04-16T19:32:26.153260+0000","last_fullsized":"2024-04-16T19:32:26.153260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178113+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:33:41.178113+0000","last_peered":"2024-04-16T19:33:41.178113+0000","last_clean":"2024-04-16T19:33:41.178113+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:33:41.178113+0000","last_undegraded":"2024-04-16T19:33:41.178113+0000","last_fullsized":"2024-04-16T19:33:41.178113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'8","reported_seq":270,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135460+0000","last_change":"2024-04-16T17:47:55.163584+0000","last_active":"2024-04-16T19:32:06.135460+0000","last_peered":"2024-04-16T19:32:06.135460+0000","last_clean":"2024-04-16T19:32:06.135460+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:32:06.135460+0000","last_undegraded":"2024-04-16T19:32:06.135460+0000","last_fullsized":"2024-04-16T19:32:06.135460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:28.473272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159805+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:32:51.159805+0000","last_peered":"2024-04-16T19:32:51.159805+0000","last_clean":"2024-04-16T19:32:51.159805+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:32:51.159805+0000","last_undegraded":"2024-04-16T19:32:51.159805+0000","last_fullsized":"2024-04-16T19:32:51.159805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172776+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:33:41.172776+0000","last_peered":"2024-04-16T19:33:41.172776+0000","last_clean":"2024-04-16T19:33:41.172776+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:33:41.172776+0000","last_undegraded":"2024-04-16T19:33:41.172776+0000","last_fullsized":"2024-04-16T19:33:41.172776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:01.960480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"50'5","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135378+0000","last_change":"2024-04-16T17:47:55.164609+0000","last_active":"2024-04-16T19:32:06.135378+0000","last_peered":"2024-04-16T19:32:06.135378+0000","last_clean":"2024-04-16T19:32:06.135378+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:32:06.135378+0000","last_undegraded":"2024-04-16T19:32:06.135378+0000","last_fullsized":"2024-04-16T19:32:06.135378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:56.378878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158570+0000","last_change":"2024-04-16T17:47:23.254226+0000","last_active":"2024-04-16T19:33:46.158570+0000","last_peered":"2024-04-16T19:33:46.158570+0000","last_clean":"2024-04-16T19:33:46.158570+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:33:46.158570+0000","last_undegraded":"2024-04-16T19:33:46.158570+0000","last_fullsized":"2024-04-16T19:33:46.158570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:06.511950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171122+0000","last_change":"2024-04-16T17:47:32.284109+0000","last_active":"2024-04-16T19:33:41.171122+0000","last_peered":"2024-04-16T19:33:41.171122+0000","last_clean":"2024-04-16T19:33:41.171122+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:33:41.171122+0000","last_undegraded":"2024-04-16T19:33:41.171122+0000","last_fullsized":"2024-04-16T19:33:41.171122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'5","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166165+0000","last_change":"2024-04-16T17:47:55.172087+0000","last_active":"2024-04-16T19:32:56.166165+0000","last_peered":"2024-04-16T19:32:56.166165+0000","last_clean":"2024-04-16T19:32:56.166165+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:32:56.166165+0000","last_undegraded":"2024-04-16T19:32:56.166165+0000","last_fullsized":"2024-04-16T19:32:56.166165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158622+0000","last_change":"2024-04-16T17:47:23.266132+0000","last_active":"2024-04-16T19:33:46.158622+0000","last_peered":"2024-04-16T19:33:46.158622+0000","last_clean":"2024-04-16T19:33:46.158622+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:33:46.158622+0000","last_undegraded":"2024-04-16T19:33:46.158622+0000","last_fullsized":"2024-04-16T19:33:46.158622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:09.592666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.358504+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:31:31.358504+0000","last_peered":"2024-04-16T19:31:31.358504+0000","last_clean":"2024-04-16T19:31:31.358504+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:31:31.358504+0000","last_undegraded":"2024-04-16T19:31:31.358504+0000","last_fullsized":"2024-04-16T19:31:31.358504+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:02.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'10","reported_seq":265,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146504+0000","last_change":"2024-04-16T17:47:55.158785+0000","last_active":"2024-04-16T19:32:56.146504+0000","last_peered":"2024-04-16T19:32:56.146504+0000","last_clean":"2024-04-16T19:32:56.146504+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:32:56.146504+0000","last_undegraded":"2024-04-16T19:32:56.146504+0000","last_fullsized":"2024-04-16T19:32:56.146504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:39.387454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172430+0000","last_change":"2024-04-16T17:47:23.257289+0000","last_active":"2024-04-16T19:33:46.172430+0000","last_peered":"2024-04-16T19:33:46.172430+0000","last_clean":"2024-04-16T19:33:46.172430+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:33:46.172430+0000","last_undegraded":"2024-04-16T19:33:46.172430+0000","last_fullsized":"2024-04-16T19:33:46.172430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163511+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:32:56.163511+0000","last_peered":"2024-04-16T19:32:56.163511+0000","last_clean":"2024-04-16T19:32:56.163511+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:32:56.163511+0000","last_undegraded":"2024-04-16T19:32:56.163511+0000","last_fullsized":"2024-04-16T19:32:56.163511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:29:20.877336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151839+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:32:06.151839+0000","last_peered":"2024-04-16T19:32:06.151839+0000","last_clean":"2024-04-16T19:32:06.151839+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:32:06.151839+0000","last_undegraded":"2024-04-16T19:32:06.151839+0000","last_fullsized":"2024-04-16T19:32:06.151839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:23:28.148511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170388+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:33:11.170388+0000","last_peered":"2024-04-16T19:33:11.170388+0000","last_clean":"2024-04-16T19:33:11.170388+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:33:11.170388+0000","last_undegraded":"2024-04-16T19:33:11.170388+0000","last_fullsized":"2024-04-16T19:33:11.170388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175004+0000","last_change":"2024-04-16T17:48:27.308777+0000","last_active":"2024-04-16T19:33:31.175004+0000","last_peered":"2024-04-16T19:33:31.175004+0000","last_clean":"2024-04-16T19:33:31.175004+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:33:31.175004+0000","last_undegraded":"2024-04-16T19:33:31.175004+0000","last_fullsized":"2024-04-16T19:33:31.175004+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'9","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164889+0000","last_change":"2024-04-16T17:48:01.266271+0000","last_active":"2024-04-16T19:33:01.164889+0000","last_peered":"2024-04-16T19:33:01.164889+0000","last_clean":"2024-04-16T19:33:01.164889+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:33:01.164889+0000","last_undegraded":"2024-04-16T19:33:01.164889+0000","last_fullsized":"2024-04-16T19:33:01.164889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161254+0000","last_change":"2024-04-16T17:47:59.181005+0000","last_active":"2024-04-16T19:33:01.161254+0000","last_peered":"2024-04-16T19:33:01.161254+0000","last_clean":"2024-04-16T19:33:01.161254+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:33:01.161254+0000","last_undegraded":"2024-04-16T19:33:01.161254+0000","last_fullsized":"2024-04-16T19:33:01.161254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:34.728189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'547","reported_seq":799,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167615+0000","last_change":"2024-04-16T17:47:57.177580+0000","last_active":"2024-04-16T19:33:01.167615+0000","last_peered":"2024-04-16T19:33:01.167615+0000","last_clean":"2024-04-16T19:33:01.167615+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:33:01.167615+0000","last_undegraded":"2024-04-16T19:33:01.167615+0000","last_fullsized":"2024-04-16T19:33:01.167615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'7","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138635+0000","last_change":"2024-04-16T17:47:55.165841+0000","last_active":"2024-04-16T19:32:21.138635+0000","last_peered":"2024-04-16T19:32:21.138635+0000","last_clean":"2024-04-16T19:32:21.138635+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:32:21.138635+0000","last_undegraded":"2024-04-16T19:32:21.138635+0000","last_fullsized":"2024-04-16T19:32:21.138635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:07:11.773810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.139718+0000","last_change":"2024-04-16T17:47:39.326260+0000","last_active":"2024-04-16T19:31:36.139718+0000","last_peered":"2024-04-16T19:31:36.139718+0000","last_clean":"2024-04-16T19:31:36.139718+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:31:36.139718+0000","last_undegraded":"2024-04-16T19:31:36.139718+0000","last_fullsized":"2024-04-16T19:31:36.139718+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:58.230179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164460+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:32:56.164460+0000","last_peered":"2024-04-16T19:32:56.164460+0000","last_clean":"2024-04-16T19:32:56.164460+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:32:56.164460+0000","last_undegraded":"2024-04-16T19:32:56.164460+0000","last_fullsized":"2024-04-16T19:32:56.164460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159202+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:32:36.159202+0000","last_peered":"2024-04-16T19:32:36.159202+0000","last_clean":"2024-04-16T19:32:36.159202+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:32:36.159202+0000","last_undegraded":"2024-04-16T19:32:36.159202+0000","last_fullsized":"2024-04-16T19:32:36.159202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":768,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.370756+0000","last_change":"2024-04-16T17:47:33.303155+0000","last_active":"2024-04-16T19:31:31.370756+0000","last_peered":"2024-04-16T19:31:31.370756+0000","last_clean":"2024-04-16T19:31:31.370756+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:31:31.370756+0000","last_undegraded":"2024-04-16T19:31:31.370756+0000","last_fullsized":"2024-04-16T19:31:31.370756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:14.676250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367632+0000","last_change":"2024-04-16T17:47:35.314855+0000","last_active":"2024-04-16T19:31:31.367632+0000","last_peered":"2024-04-16T19:31:31.367632+0000","last_clean":"2024-04-16T19:31:31.367632+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:31:31.367632+0000","last_undegraded":"2024-04-16T19:31:31.367632+0000","last_fullsized":"2024-04-16T19:31:31.367632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140046+0000","last_change":"2024-04-16T17:47:37.332945+0000","last_active":"2024-04-16T19:31:36.140046+0000","last_peered":"2024-04-16T19:31:36.140046+0000","last_clean":"2024-04-16T19:31:36.140046+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:31:36.140046+0000","last_undegraded":"2024-04-16T19:31:36.140046+0000","last_fullsized":"2024-04-16T19:31:36.140046+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152563+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:33:21.152563+0000","last_peered":"2024-04-16T19:33:21.152563+0000","last_clean":"2024-04-16T19:33:21.152563+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:33:21.152563+0000","last_undegraded":"2024-04-16T19:33:21.152563+0000","last_fullsized":"2024-04-16T19:33:21.152563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:24.843140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'83","reported_seq":1420,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161248+0000","last_change":"2024-04-16T17:48:34.319654+0000","last_active":"2024-04-16T19:32:46.161248+0000","last_peered":"2024-04-16T19:32:46.161248+0000","last_clean":"2024-04-16T19:32:46.161248+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:32:46.161248+0000","last_undegraded":"2024-04-16T19:32:46.161248+0000","last_fullsized":"2024-04-16T19:32:46.161248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:13.391814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169963+0000","last_change":"2024-04-16T17:48:27.303999+0000","last_active":"2024-04-16T19:33:31.169963+0000","last_peered":"2024-04-16T19:33:31.169963+0000","last_clean":"2024-04-16T19:33:31.169963+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:33:31.169963+0000","last_undegraded":"2024-04-16T19:33:31.169963+0000","last_fullsized":"2024-04-16T19:33:31.169963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'5","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164928+0000","last_change":"2024-04-16T17:48:01.266699+0000","last_active":"2024-04-16T19:33:01.164928+0000","last_peered":"2024-04-16T19:33:01.164928+0000","last_clean":"2024-04-16T19:33:01.164928+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:33:01.164928+0000","last_undegraded":"2024-04-16T19:33:01.164928+0000","last_fullsized":"2024-04-16T19:33:01.164928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162337+0000","last_change":"2024-04-16T17:47:59.191306+0000","last_active":"2024-04-16T19:33:01.162337+0000","last_peered":"2024-04-16T19:33:01.162337+0000","last_clean":"2024-04-16T19:33:01.162337+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:33:01.162337+0000","last_undegraded":"2024-04-16T19:33:01.162337+0000","last_fullsized":"2024-04-16T19:33:01.162337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7326","reported_seq":20595,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147895+0000","last_change":"2024-04-16T17:47:57.179815+0000","last_active":"2024-04-16T19:33:01.147895+0000","last_peered":"2024-04-16T19:33:01.147895+0000","last_clean":"2024-04-16T19:33:01.147895+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:33:01.147895+0000","last_undegraded":"2024-04-16T19:33:01.147895+0000","last_fullsized":"2024-04-16T19:33:01.147895+0000","mapping_epoch":34,"log_start":"53'5500","ondisk_log_start":"53'5500","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1826,"log_dups_size":3000,"ondisk_log_size":1826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:36.013179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'19","reported_seq":354,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143591+0000","last_change":"2024-04-16T17:47:55.178555+0000","last_active":"2024-04-16T19:31:51.143591+0000","last_peered":"2024-04-16T19:31:51.143591+0000","last_clean":"2024-04-16T19:31:51.143591+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:31:51.143591+0000","last_undegraded":"2024-04-16T19:31:51.143591+0000","last_fullsized":"2024-04-16T19:31:51.143591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146395+0000","last_change":"2024-04-16T17:47:39.333211+0000","last_active":"2024-04-16T19:32:01.146395+0000","last_peered":"2024-04-16T19:32:01.146395+0000","last_clean":"2024-04-16T19:32:01.146395+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:32:01.146395+0000","last_undegraded":"2024-04-16T19:32:01.146395+0000","last_fullsized":"2024-04-16T19:32:01.146395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174179+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:33:46.174179+0000","last_peered":"2024-04-16T19:33:46.174179+0000","last_clean":"2024-04-16T19:33:46.174179+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:33:46.174179+0000","last_undegraded":"2024-04-16T19:33:46.174179+0000","last_fullsized":"2024-04-16T19:33:46.174179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154341+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:32:21.154341+0000","last_peered":"2024-04-16T19:32:21.154341+0000","last_clean":"2024-04-16T19:32:21.154341+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:32:21.154341+0000","last_undegraded":"2024-04-16T19:32:21.154341+0000","last_fullsized":"2024-04-16T19:32:21.154341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"48'5","reported_seq":359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167357+0000","last_change":"2024-04-16T17:47:33.310538+0000","last_active":"2024-04-16T19:33:11.167357+0000","last_peered":"2024-04-16T19:33:11.167357+0000","last_clean":"2024-04-16T19:33:11.167357+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:33:11.167357+0000","last_undegraded":"2024-04-16T19:33:11.167357+0000","last_fullsized":"2024-04-16T19:33:11.167357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'72","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361573+0000","last_change":"2024-04-16T17:47:35.305717+0000","last_active":"2024-04-16T19:31:31.361573+0000","last_peered":"2024-04-16T19:31:31.361573+0000","last_clean":"2024-04-16T19:31:31.361573+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:31:31.361573+0000","last_undegraded":"2024-04-16T19:31:31.361573+0000","last_fullsized":"2024-04-16T19:31:31.361573+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:27.152861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140105+0000","last_change":"2024-04-16T17:47:37.326918+0000","last_active":"2024-04-16T19:31:36.140105+0000","last_peered":"2024-04-16T19:31:36.140105+0000","last_clean":"2024-04-16T19:31:36.140105+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:31:36.140105+0000","last_undegraded":"2024-04-16T19:31:36.140105+0000","last_fullsized":"2024-04-16T19:31:36.140105+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:19.938869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166042+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:33:21.166042+0000","last_peered":"2024-04-16T19:33:21.166042+0000","last_clean":"2024-04-16T19:33:21.166042+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:33:21.166042+0000","last_undegraded":"2024-04-16T19:33:21.166042+0000","last_fullsized":"2024-04-16T19:33:21.166042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:35.987275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'56","reported_seq":876,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173425+0000","last_change":"2024-04-16T17:48:34.333949+0000","last_active":"2024-04-16T19:33:36.173425+0000","last_peered":"2024-04-16T19:33:36.173425+0000","last_clean":"2024-04-16T19:33:36.173425+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:33:36.173425+0000","last_undegraded":"2024-04-16T19:33:36.173425+0000","last_fullsized":"2024-04-16T19:33:36.173425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156598+0000","last_change":"2024-04-16T17:48:27.299066+0000","last_active":"2024-04-16T19:32:41.156598+0000","last_peered":"2024-04-16T19:32:41.156598+0000","last_clean":"2024-04-16T19:32:41.156598+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:32:41.156598+0000","last_undegraded":"2024-04-16T19:32:41.156598+0000","last_fullsized":"2024-04-16T19:32:41.156598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:12.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'16","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155319+0000","last_change":"2024-04-16T17:48:01.239365+0000","last_active":"2024-04-16T19:32:36.155319+0000","last_peered":"2024-04-16T19:32:36.155319+0000","last_clean":"2024-04-16T19:32:36.155319+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:32:36.155319+0000","last_undegraded":"2024-04-16T19:32:36.155319+0000","last_fullsized":"2024-04-16T19:32:36.155319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:31.169600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156956+0000","last_change":"2024-04-16T17:47:59.192949+0000","last_active":"2024-04-16T19:32:31.156956+0000","last_peered":"2024-04-16T19:32:31.156956+0000","last_clean":"2024-04-16T19:32:31.156956+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:32:31.156956+0000","last_undegraded":"2024-04-16T19:32:31.156956+0000","last_fullsized":"2024-04-16T19:32:31.156956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3595","reported_seq":7189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132670+0000","last_change":"2024-04-16T17:47:57.176567+0000","last_active":"2024-04-16T19:31:56.132670+0000","last_peered":"2024-04-16T19:31:56.132670+0000","last_clean":"2024-04-16T19:31:56.132670+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:31:56.132670+0000","last_undegraded":"2024-04-16T19:31:56.132670+0000","last_fullsized":"2024-04-16T19:31:56.132670+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1795,"log_dups_size":1800,"ondisk_log_size":1795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:10.013659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'9","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163598+0000","last_change":"2024-04-16T17:47:55.178252+0000","last_active":"2024-04-16T19:32:56.163598+0000","last_peered":"2024-04-16T19:32:56.163598+0000","last_clean":"2024-04-16T19:32:56.163598+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:32:56.163598+0000","last_undegraded":"2024-04-16T19:32:56.163598+0000","last_fullsized":"2024-04-16T19:32:56.163598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159596+0000","last_change":"2024-04-16T17:47:39.334187+0000","last_active":"2024-04-16T19:32:41.159596+0000","last_peered":"2024-04-16T19:32:41.159596+0000","last_clean":"2024-04-16T19:32:41.159596+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:32:41.159596+0000","last_undegraded":"2024-04-16T19:32:41.159596+0000","last_fullsized":"2024-04-16T19:32:41.159596+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176178+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:33:46.176178+0000","last_peered":"2024-04-16T19:33:46.176178+0000","last_clean":"2024-04-16T19:33:46.176178+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:33:46.176178+0000","last_undegraded":"2024-04-16T19:33:46.176178+0000","last_fullsized":"2024-04-16T19:33:46.176178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156989+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:32:21.156989+0000","last_peered":"2024-04-16T19:32:21.156989+0000","last_clean":"2024-04-16T19:32:21.156989+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:32:21.156989+0000","last_undegraded":"2024-04-16T19:32:21.156989+0000","last_fullsized":"2024-04-16T19:32:21.156989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"48'11","reported_seq":605,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.343972+0000","last_change":"2024-04-16T17:47:33.300240+0000","last_active":"2024-04-16T19:31:31.343972+0000","last_peered":"2024-04-16T19:31:31.343972+0000","last_clean":"2024-04-16T19:31:31.343972+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:31:31.343972+0000","last_undegraded":"2024-04-16T19:31:31.343972+0000","last_fullsized":"2024-04-16T19:31:31.343972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:30.896735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'120","reported_seq":290,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.149082+0000","last_change":"2024-04-16T17:47:35.314675+0000","last_active":"2024-04-16T19:33:06.149082+0000","last_peered":"2024-04-16T19:33:06.149082+0000","last_clean":"2024-04-16T19:33:06.149082+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:33:06.149082+0000","last_undegraded":"2024-04-16T19:33:06.149082+0000","last_fullsized":"2024-04-16T19:33:06.149082+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:43.962803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'12","reported_seq":1332,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144472+0000","last_change":"2024-04-16T17:47:37.333045+0000","last_active":"2024-04-16T19:31:36.144472+0000","last_peered":"2024-04-16T19:31:36.144472+0000","last_clean":"2024-04-16T19:31:36.144472+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:31:36.144472+0000","last_undegraded":"2024-04-16T19:31:36.144472+0000","last_fullsized":"2024-04-16T19:31:36.144472+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:40.632643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170471+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:33:21.170471+0000","last_peered":"2024-04-16T19:33:21.170471+0000","last_clean":"2024-04-16T19:33:21.170471+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:33:21.170471+0000","last_undegraded":"2024-04-16T19:33:21.170471+0000","last_fullsized":"2024-04-16T19:33:21.170471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:20.115696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173520+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:33:36.173520+0000","last_peered":"2024-04-16T19:33:36.173520+0000","last_clean":"2024-04-16T19:33:36.173520+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:33:36.173520+0000","last_undegraded":"2024-04-16T19:33:36.173520+0000","last_fullsized":"2024-04-16T19:33:36.173520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:58.360796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.162482+0000","last_change":"2024-04-16T17:48:27.297318+0000","last_active":"2024-04-16T19:33:06.162482+0000","last_peered":"2024-04-16T19:33:06.162482+0000","last_clean":"2024-04-16T19:33:06.162482+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:33:06.162482+0000","last_undegraded":"2024-04-16T19:33:06.162482+0000","last_fullsized":"2024-04-16T19:33:06.162482+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:20:02.104358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'19","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154557+0000","last_change":"2024-04-16T17:48:01.256079+0000","last_active":"2024-04-16T19:32:11.154557+0000","last_peered":"2024-04-16T19:32:11.154557+0000","last_clean":"2024-04-16T19:32:11.154557+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:32:11.154557+0000","last_undegraded":"2024-04-16T19:32:11.154557+0000","last_fullsized":"2024-04-16T19:32:11.154557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132631+0000","last_change":"2024-04-16T17:47:59.183129+0000","last_active":"2024-04-16T19:31:56.132631+0000","last_peered":"2024-04-16T19:31:56.132631+0000","last_clean":"2024-04-16T19:31:56.132631+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:31:56.132631+0000","last_undegraded":"2024-04-16T19:31:56.132631+0000","last_fullsized":"2024-04-16T19:31:56.132631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:48.953458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'385","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156502+0000","last_change":"2024-04-16T17:47:57.184276+0000","last_active":"2024-04-16T19:32:26.156502+0000","last_peered":"2024-04-16T19:32:26.156502+0000","last_clean":"2024-04-16T19:32:26.156502+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:32:26.156502+0000","last_undegraded":"2024-04-16T19:32:26.156502+0000","last_fullsized":"2024-04-16T19:32:26.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:59.219214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'6","reported_seq":173,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164223+0000","last_change":"2024-04-16T17:47:55.161902+0000","last_active":"2024-04-16T19:32:56.164223+0000","last_peered":"2024-04-16T19:32:56.164223+0000","last_clean":"2024-04-16T19:32:56.164223+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:32:56.164223+0000","last_undegraded":"2024-04-16T19:32:56.164223+0000","last_fullsized":"2024-04-16T19:32:56.164223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:47.909979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156406+0000","last_change":"2024-04-16T17:47:39.699810+0000","last_active":"2024-04-16T19:32:41.156406+0000","last_peered":"2024-04-16T19:32:41.156406+0000","last_clean":"2024-04-16T19:32:41.156406+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:32:41.156406+0000","last_undegraded":"2024-04-16T19:32:41.156406+0000","last_fullsized":"2024-04-16T19:32:41.156406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:40.561497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176260+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:33:46.176260+0000","last_peered":"2024-04-16T19:33:46.176260+0000","last_clean":"2024-04-16T19:33:46.176260+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:33:46.176260+0000","last_undegraded":"2024-04-16T19:33:46.176260+0000","last_fullsized":"2024-04-16T19:33:46.176260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151772+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:32:21.151772+0000","last_peered":"2024-04-16T19:32:21.151772+0000","last_clean":"2024-04-16T19:32:21.151772+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:32:21.151772+0000","last_undegraded":"2024-04-16T19:32:21.151772+0000","last_fullsized":"2024-04-16T19:32:21.151772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374413+0000","last_change":"2024-04-16T17:47:33.305749+0000","last_active":"2024-04-16T19:31:31.374413+0000","last_peered":"2024-04-16T19:31:31.374413+0000","last_clean":"2024-04-16T19:31:31.374413+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:31:31.374413+0000","last_undegraded":"2024-04-16T19:31:31.374413+0000","last_fullsized":"2024-04-16T19:31:31.374413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:15.902753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374347+0000","last_change":"2024-04-16T17:47:35.314832+0000","last_active":"2024-04-16T19:31:31.374347+0000","last_peered":"2024-04-16T19:31:31.374347+0000","last_clean":"2024-04-16T19:31:31.374347+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:31:31.374347+0000","last_undegraded":"2024-04-16T19:31:31.374347+0000","last_fullsized":"2024-04-16T19:31:31.374347+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:36.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'12","reported_seq":1334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168147+0000","last_change":"2024-04-16T17:47:37.334246+0000","last_active":"2024-04-16T19:33:11.168147+0000","last_peered":"2024-04-16T19:33:11.168147+0000","last_clean":"2024-04-16T19:33:11.168147+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:33:11.168147+0000","last_undegraded":"2024-04-16T19:33:11.168147+0000","last_fullsized":"2024-04-16T19:33:11.168147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:21.558307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147648+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:32:06.147648+0000","last_peered":"2024-04-16T19:32:06.147648+0000","last_clean":"2024-04-16T19:32:06.147648+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:32:06.147648+0000","last_undegraded":"2024-04-16T19:32:06.147648+0000","last_fullsized":"2024-04-16T19:32:06.147648+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:13:48.347861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'164","reported_seq":1853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163697+0000","last_change":"2024-04-16T17:48:34.332673+0000","last_active":"2024-04-16T19:32:46.163697+0000","last_peered":"2024-04-16T19:32:46.163697+0000","last_clean":"2024-04-16T19:32:46.163697+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:32:46.163697+0000","last_undegraded":"2024-04-16T19:32:46.163697+0000","last_fullsized":"2024-04-16T19:32:46.163697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162327+0000","last_change":"2024-04-16T17:48:27.306930+0000","last_active":"2024-04-16T19:32:41.162327+0000","last_peered":"2024-04-16T19:32:41.162327+0000","last_clean":"2024-04-16T19:32:41.162327+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:32:41.162327+0000","last_undegraded":"2024-04-16T19:32:41.162327+0000","last_fullsized":"2024-04-16T19:32:41.162327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'10","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.154948+0000","last_change":"2024-04-16T17:48:01.252700+0000","last_active":"2024-04-16T19:32:36.154948+0000","last_peered":"2024-04-16T19:32:36.154948+0000","last_clean":"2024-04-16T19:32:36.154948+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:32:36.154948+0000","last_undegraded":"2024-04-16T19:32:36.154948+0000","last_fullsized":"2024-04-16T19:32:36.154948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:40.367478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146809+0000","last_change":"2024-04-16T17:47:59.200630+0000","last_active":"2024-04-16T19:32:01.146809+0000","last_peered":"2024-04-16T19:32:01.146809+0000","last_clean":"2024-04-16T19:32:01.146809+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:32:01.146809+0000","last_undegraded":"2024-04-16T19:32:01.146809+0000","last_fullsized":"2024-04-16T19:32:01.146809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1087","reported_seq":1742,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146730+0000","last_change":"2024-04-16T17:47:57.186883+0000","last_active":"2024-04-16T19:32:01.146730+0000","last_peered":"2024-04-16T19:32:01.146730+0000","last_clean":"2024-04-16T19:32:01.146730+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:32:01.146730+0000","last_undegraded":"2024-04-16T19:32:01.146730+0000","last_fullsized":"2024-04-16T19:32:01.146730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:14.402693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"51'4","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159908+0000","last_change":"2024-04-16T17:47:55.177129+0000","last_active":"2024-04-16T19:32:41.159908+0000","last_peered":"2024-04-16T19:32:41.159908+0000","last_clean":"2024-04-16T19:32:41.159908+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:32:41.159908+0000","last_undegraded":"2024-04-16T19:32:41.159908+0000","last_fullsized":"2024-04-16T19:32:41.159908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:31.401461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176098+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:33:36.176098+0000","last_peered":"2024-04-16T19:33:36.176098+0000","last_clean":"2024-04-16T19:33:36.176098+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:33:36.176098+0000","last_undegraded":"2024-04-16T19:33:36.176098+0000","last_fullsized":"2024-04-16T19:33:36.176098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.170137+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:33:31.170137+0000","last_peered":"2024-04-16T19:33:31.170137+0000","last_clean":"2024-04-16T19:33:31.170137+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:33:31.170137+0000","last_undegraded":"2024-04-16T19:33:31.170137+0000","last_fullsized":"2024-04-16T19:33:31.170137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.173925+0000","last_change":"2024-04-16T17:47:33.309390+0000","last_active":"2024-04-16T19:33:46.173925+0000","last_peered":"2024-04-16T19:33:46.173925+0000","last_clean":"2024-04-16T19:33:46.173925+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:33:46.173925+0000","last_undegraded":"2024-04-16T19:33:46.173925+0000","last_fullsized":"2024-04-16T19:33:46.173925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'200","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146472+0000","last_change":"2024-04-16T17:47:35.314414+0000","last_active":"2024-04-16T19:32:01.146472+0000","last_peered":"2024-04-16T19:32:01.146472+0000","last_clean":"2024-04-16T19:32:01.146472+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:32:01.146472+0000","last_undegraded":"2024-04-16T19:32:01.146472+0000","last_fullsized":"2024-04-16T19:32:01.146472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171681+0000","last_change":"2024-04-16T17:47:37.335122+0000","last_active":"2024-04-16T19:33:26.171681+0000","last_peered":"2024-04-16T19:33:26.171681+0000","last_clean":"2024-04-16T19:33:26.171681+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:33:26.171681+0000","last_undegraded":"2024-04-16T19:33:26.171681+0000","last_fullsized":"2024-04-16T19:33:26.171681+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:38.549732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176783+0000","last_change":"2024-04-16T17:47:39.339833+0000","last_active":"2024-04-16T19:33:56.176783+0000","last_peered":"2024-04-16T19:33:56.176783+0000","last_clean":"2024-04-16T19:33:56.176783+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:33:56.176783+0000","last_undegraded":"2024-04-16T19:33:56.176783+0000","last_fullsized":"2024-04-16T19:33:56.176783+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135246+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:32:06.135246+0000","last_peered":"2024-04-16T19:32:06.135246+0000","last_clean":"2024-04-16T19:32:06.135246+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:32:06.135246+0000","last_undegraded":"2024-04-16T19:32:06.135246+0000","last_fullsized":"2024-04-16T19:32:06.135246+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:13:30.436128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1187,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161075+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:32:46.161075+0000","last_peered":"2024-04-16T19:32:46.161075+0000","last_clean":"2024-04-16T19:32:46.161075+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:32:46.161075+0000","last_undegraded":"2024-04-16T19:32:46.161075+0000","last_fullsized":"2024-04-16T19:32:46.161075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168743+0000","last_change":"2024-04-16T17:48:27.299718+0000","last_active":"2024-04-16T19:33:06.168743+0000","last_peered":"2024-04-16T19:33:06.168743+0000","last_clean":"2024-04-16T19:33:06.168743+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:33:06.168743+0000","last_undegraded":"2024-04-16T19:33:06.168743+0000","last_fullsized":"2024-04-16T19:33:06.168743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'2","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154067+0000","last_change":"2024-04-16T17:48:01.253979+0000","last_active":"2024-04-16T19:32:11.154067+0000","last_peered":"2024-04-16T19:32:11.154067+0000","last_clean":"2024-04-16T19:32:11.154067+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:32:11.154067+0000","last_undegraded":"2024-04-16T19:32:11.154067+0000","last_fullsized":"2024-04-16T19:32:11.154067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149567+0000","last_change":"2024-04-16T17:47:59.193745+0000","last_active":"2024-04-16T19:32:01.149567+0000","last_peered":"2024-04-16T19:32:01.149567+0000","last_clean":"2024-04-16T19:32:01.149567+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:32:01.149567+0000","last_undegraded":"2024-04-16T19:32:01.149567+0000","last_fullsized":"2024-04-16T19:32:01.149567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1173","reported_seq":1516,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151419+0000","last_change":"2024-04-16T17:47:57.191812+0000","last_active":"2024-04-16T19:32:01.151419+0000","last_peered":"2024-04-16T19:32:01.151419+0000","last_clean":"2024-04-16T19:32:01.151419+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:32:01.151419+0000","last_undegraded":"2024-04-16T19:32:01.151419+0000","last_fullsized":"2024-04-16T19:32:01.151419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1173,"log_dups_size":0,"ondisk_log_size":1173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'14","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151697+0000","last_change":"2024-04-16T17:47:55.163005+0000","last_active":"2024-04-16T19:32:06.151697+0000","last_peered":"2024-04-16T19:32:06.151697+0000","last_clean":"2024-04-16T19:32:06.151697+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:32:06.151697+0000","last_undegraded":"2024-04-16T19:32:06.151697+0000","last_fullsized":"2024-04-16T19:32:06.151697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169773+0000","last_change":"2024-04-16T17:47:23.266202+0000","last_active":"2024-04-16T19:33:36.169773+0000","last_peered":"2024-04-16T19:33:36.169773+0000","last_clean":"2024-04-16T19:33:36.169773+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:33:36.169773+0000","last_undegraded":"2024-04-16T19:33:36.169773+0000","last_fullsized":"2024-04-16T19:33:36.169773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163890+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:33:06.163890+0000","last_peered":"2024-04-16T19:33:06.163890+0000","last_clean":"2024-04-16T19:33:06.163890+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:33:06.163890+0000","last_undegraded":"2024-04-16T19:33:06.163890+0000","last_fullsized":"2024-04-16T19:33:06.163890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:45.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"31'1","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158405+0000","last_change":"2024-04-16T17:47:33.302299+0000","last_active":"2024-04-16T19:33:46.158405+0000","last_peered":"2024-04-16T19:33:46.158405+0000","last_clean":"2024-04-16T19:33:46.158405+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:33:46.158405+0000","last_undegraded":"2024-04-16T19:33:46.158405+0000","last_fullsized":"2024-04-16T19:33:46.158405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:23.313435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'80","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170395+0000","last_change":"2024-04-16T17:47:35.321698+0000","last_active":"2024-04-16T19:33:21.170395+0000","last_peered":"2024-04-16T19:33:21.170395+0000","last_clean":"2024-04-16T19:33:21.170395+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:33:21.170395+0000","last_undegraded":"2024-04-16T19:33:21.170395+0000","last_fullsized":"2024-04-16T19:33:21.170395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:02.350103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136445+0000","last_change":"2024-04-16T17:47:37.325461+0000","last_active":"2024-04-16T19:32:11.136445+0000","last_peered":"2024-04-16T19:32:11.136445+0000","last_clean":"2024-04-16T19:32:11.136445+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:32:11.136445+0000","last_undegraded":"2024-04-16T19:32:11.136445+0000","last_fullsized":"2024-04-16T19:32:11.136445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:18.621208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152703+0000","last_change":"2024-04-16T17:47:39.323337+0000","last_active":"2024-04-16T19:32:26.152703+0000","last_peered":"2024-04-16T19:32:26.152703+0000","last_clean":"2024-04-16T19:32:26.152703+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:32:26.152703+0000","last_undegraded":"2024-04-16T19:32:26.152703+0000","last_fullsized":"2024-04-16T19:32:26.152703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:12.655890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152382+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:32:06.152382+0000","last_peered":"2024-04-16T19:32:06.152382+0000","last_clean":"2024-04-16T19:32:06.152382+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:32:06.152382+0000","last_undegraded":"2024-04-16T19:32:06.152382+0000","last_fullsized":"2024-04-16T19:32:06.152382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:58.661997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'70","reported_seq":901,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.148912+0000","last_change":"2024-04-16T17:48:34.345575+0000","last_active":"2024-04-16T19:33:06.148912+0000","last_peered":"2024-04-16T19:33:06.148912+0000","last_clean":"2024-04-16T19:33:06.148912+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:33:06.148912+0000","last_undegraded":"2024-04-16T19:33:06.148912+0000","last_fullsized":"2024-04-16T19:33:06.148912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:51.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159686+0000","last_change":"2024-04-16T17:48:27.295648+0000","last_active":"2024-04-16T19:32:41.159686+0000","last_peered":"2024-04-16T19:32:41.159686+0000","last_clean":"2024-04-16T19:32:41.159686+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:32:41.159686+0000","last_undegraded":"2024-04-16T19:32:41.159686+0000","last_fullsized":"2024-04-16T19:32:41.159686+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154259+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:32:11.154259+0000","last_peered":"2024-04-16T19:32:11.154259+0000","last_clean":"2024-04-16T19:32:11.154259+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:32:11.154259+0000","last_undegraded":"2024-04-16T19:32:11.154259+0000","last_fullsized":"2024-04-16T19:32:11.154259+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:20.025368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.157655+0000","last_change":"2024-04-16T17:47:59.197037+0000","last_active":"2024-04-16T19:32:46.157655+0000","last_peered":"2024-04-16T19:32:46.157655+0000","last_clean":"2024-04-16T19:32:46.157655+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:32:46.157655+0000","last_undegraded":"2024-04-16T19:32:46.157655+0000","last_fullsized":"2024-04-16T19:32:46.157655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:54.818691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1049","reported_seq":6110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152019+0000","last_change":"2024-04-16T17:47:57.189817+0000","last_active":"2024-04-16T19:32:11.152019+0000","last_peered":"2024-04-16T19:32:11.152019+0000","last_clean":"2024-04-16T19:32:11.152019+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:32:11.152019+0000","last_undegraded":"2024-04-16T19:32:11.152019+0000","last_fullsized":"2024-04-16T19:32:11.152019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'4","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145438+0000","last_change":"2024-04-16T17:47:55.182984+0000","last_active":"2024-04-16T19:31:56.145438+0000","last_peered":"2024-04-16T19:31:56.145438+0000","last_clean":"2024-04-16T19:31:56.145438+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:31:56.145438+0000","last_undegraded":"2024-04-16T19:31:56.145438+0000","last_fullsized":"2024-04-16T19:31:56.145438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:55.129164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178050+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:34:01.178050+0000","last_peered":"2024-04-16T19:34:01.178050+0000","last_clean":"2024-04-16T19:34:01.178050+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:34:01.178050+0000","last_undegraded":"2024-04-16T19:34:01.178050+0000","last_fullsized":"2024-04-16T19:34:01.178050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.169031+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:33:06.169031+0000","last_peered":"2024-04-16T19:33:06.169031+0000","last_clean":"2024-04-16T19:33:06.169031+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:33:06.169031+0000","last_undegraded":"2024-04-16T19:33:06.169031+0000","last_fullsized":"2024-04-16T19:33:06.169031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152471+0000","last_change":"2024-04-16T17:47:33.302725+0000","last_active":"2024-04-16T19:33:21.152471+0000","last_peered":"2024-04-16T19:33:21.152471+0000","last_clean":"2024-04-16T19:33:21.152471+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:33:21.152471+0000","last_undegraded":"2024-04-16T19:33:21.152471+0000","last_fullsized":"2024-04-16T19:33:21.152471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:42.697302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'152","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179125+0000","last_change":"2024-04-16T17:47:35.317742+0000","last_active":"2024-04-16T19:33:46.179125+0000","last_peered":"2024-04-16T19:33:46.179125+0000","last_clean":"2024-04-16T19:33:46.179125+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:33:46.179125+0000","last_undegraded":"2024-04-16T19:33:46.179125+0000","last_fullsized":"2024-04-16T19:33:46.179125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'12","reported_seq":1338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177186+0000","last_change":"2024-04-16T17:47:37.318010+0000","last_active":"2024-04-16T19:33:51.177186+0000","last_peered":"2024-04-16T19:33:51.177186+0000","last_clean":"2024-04-16T19:33:51.177186+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:33:51.177186+0000","last_undegraded":"2024-04-16T19:33:51.177186+0000","last_fullsized":"2024-04-16T19:33:51.177186+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177479+0000","last_change":"2024-04-16T17:47:39.328909+0000","last_active":"2024-04-16T19:33:51.177479+0000","last_peered":"2024-04-16T19:33:51.177479+0000","last_clean":"2024-04-16T19:33:51.177479+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:33:51.177479+0000","last_undegraded":"2024-04-16T19:33:51.177479+0000","last_fullsized":"2024-04-16T19:33:51.177479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:54.351646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152331+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:33:21.152331+0000","last_peered":"2024-04-16T19:33:21.152331+0000","last_clean":"2024-04-16T19:33:21.152331+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:33:21.152331+0000","last_undegraded":"2024-04-16T19:33:21.152331+0000","last_fullsized":"2024-04-16T19:33:21.152331+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:16.718022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'74","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161693+0000","last_change":"2024-04-16T17:48:34.327024+0000","last_active":"2024-04-16T19:32:46.161693+0000","last_peered":"2024-04-16T19:32:46.161693+0000","last_clean":"2024-04-16T19:32:46.161693+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:32:46.161693+0000","last_undegraded":"2024-04-16T19:32:46.161693+0000","last_fullsized":"2024-04-16T19:32:46.161693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:00.683240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157359+0000","last_change":"2024-04-16T17:48:27.301787+0000","last_active":"2024-04-16T19:32:41.157359+0000","last_peered":"2024-04-16T19:32:41.157359+0000","last_clean":"2024-04-16T19:32:41.157359+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:32:41.157359+0000","last_undegraded":"2024-04-16T19:32:41.157359+0000","last_fullsized":"2024-04-16T19:32:41.157359+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'17","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.151912+0000","last_change":"2024-04-16T17:48:01.243297+0000","last_active":"2024-04-16T19:32:11.151912+0000","last_peered":"2024-04-16T19:32:11.151912+0000","last_clean":"2024-04-16T19:32:11.151912+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:32:11.151912+0000","last_undegraded":"2024-04-16T19:32:11.151912+0000","last_fullsized":"2024-04-16T19:32:11.151912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149205+0000","last_change":"2024-04-16T17:47:59.193938+0000","last_active":"2024-04-16T19:32:11.149205+0000","last_peered":"2024-04-16T19:32:11.149205+0000","last_clean":"2024-04-16T19:32:11.149205+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:32:11.149205+0000","last_undegraded":"2024-04-16T19:32:11.149205+0000","last_fullsized":"2024-04-16T19:32:11.149205+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'703","reported_seq":1434,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.143345+0000","last_change":"2024-04-16T17:47:57.173194+0000","last_active":"2024-04-16T19:32:41.143345+0000","last_peered":"2024-04-16T19:32:41.143345+0000","last_clean":"2024-04-16T19:32:41.143345+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:32:41.143345+0000","last_undegraded":"2024-04-16T19:32:41.143345+0000","last_fullsized":"2024-04-16T19:32:41.143345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:02.892691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'14","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145388+0000","last_change":"2024-04-16T17:47:55.181545+0000","last_active":"2024-04-16T19:31:56.145388+0000","last_peered":"2024-04-16T19:31:56.145388+0000","last_clean":"2024-04-16T19:31:56.145388+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:31:56.145388+0000","last_undegraded":"2024-04-16T19:31:56.145388+0000","last_fullsized":"2024-04-16T19:31:56.145388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167271+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:33:11.167271+0000","last_peered":"2024-04-16T19:33:11.167271+0000","last_clean":"2024-04-16T19:33:11.167271+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:33:11.167271+0000","last_undegraded":"2024-04-16T19:33:11.167271+0000","last_fullsized":"2024-04-16T19:33:11.167271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163761+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:33:06.163761+0000","last_peered":"2024-04-16T19:33:06.163761+0000","last_clean":"2024-04-16T19:33:06.163761+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:33:06.163761+0000","last_undegraded":"2024-04-16T19:33:06.163761+0000","last_fullsized":"2024-04-16T19:33:06.163761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135219+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:32:06.135219+0000","last_peered":"2024-04-16T19:32:06.135219+0000","last_clean":"2024-04-16T19:32:06.135219+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:32:06.135219+0000","last_undegraded":"2024-04-16T19:32:06.135219+0000","last_fullsized":"2024-04-16T19:32:06.135219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:21.347240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'88","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175899+0000","last_change":"2024-04-16T17:47:35.317589+0000","last_active":"2024-04-16T19:33:46.175899+0000","last_peered":"2024-04-16T19:33:46.175899+0000","last_clean":"2024-04-16T19:33:46.175899+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:33:46.175899+0000","last_undegraded":"2024-04-16T19:33:46.175899+0000","last_fullsized":"2024-04-16T19:33:46.175899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159758+0000","last_change":"2024-04-16T17:47:37.330541+0000","last_active":"2024-04-16T19:33:51.159758+0000","last_peered":"2024-04-16T19:33:51.159758+0000","last_clean":"2024-04-16T19:33:51.159758+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:33:51.159758+0000","last_undegraded":"2024-04-16T19:33:51.159758+0000","last_fullsized":"2024-04-16T19:33:51.159758+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:07.207646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178646+0000","last_change":"2024-04-16T17:47:39.338681+0000","last_active":"2024-04-16T19:33:51.178646+0000","last_peered":"2024-04-16T19:33:51.178646+0000","last_clean":"2024-04-16T19:33:51.178646+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:33:51.178646+0000","last_undegraded":"2024-04-16T19:33:51.178646+0000","last_fullsized":"2024-04-16T19:33:51.178646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168291+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:33:11.168291+0000","last_peered":"2024-04-16T19:33:11.168291+0000","last_clean":"2024-04-16T19:33:11.168291+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:33:11.168291+0000","last_undegraded":"2024-04-16T19:33:11.168291+0000","last_fullsized":"2024-04-16T19:33:11.168291+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:26.664630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'40","reported_seq":973,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158409+0000","last_change":"2024-04-16T17:48:34.333313+0000","last_active":"2024-04-16T19:32:36.158409+0000","last_peered":"2024-04-16T19:32:36.158409+0000","last_clean":"2024-04-16T19:32:36.158409+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:32:36.158409+0000","last_undegraded":"2024-04-16T19:32:36.158409+0000","last_fullsized":"2024-04-16T19:32:36.158409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:06:58.890753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164925+0000","last_change":"2024-04-16T17:48:27.304408+0000","last_active":"2024-04-16T19:33:16.164925+0000","last_peered":"2024-04-16T19:33:16.164925+0000","last_clean":"2024-04-16T19:33:16.164925+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:33:16.164925+0000","last_undegraded":"2024-04-16T19:33:16.164925+0000","last_fullsized":"2024-04-16T19:33:16.164925+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:27:10.178784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'9","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149161+0000","last_change":"2024-04-16T17:48:01.259988+0000","last_active":"2024-04-16T19:32:11.149161+0000","last_peered":"2024-04-16T19:32:11.149161+0000","last_clean":"2024-04-16T19:32:11.149161+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:32:11.149161+0000","last_undegraded":"2024-04-16T19:32:11.149161+0000","last_fullsized":"2024-04-16T19:32:11.149161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'555","reported_seq":2287,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154329+0000","last_change":"2024-04-16T17:47:59.181942+0000","last_active":"2024-04-16T19:32:11.154329+0000","last_peered":"2024-04-16T19:32:11.154329+0000","last_clean":"2024-04-16T19:32:11.154329+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:32:11.154329+0000","last_undegraded":"2024-04-16T19:32:11.154329+0000","last_fullsized":"2024-04-16T19:32:11.154329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:25.688721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'901","reported_seq":1313,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152857+0000","last_change":"2024-04-16T17:47:57.174998+0000","last_active":"2024-04-16T19:32:11.152857+0000","last_peered":"2024-04-16T19:32:11.152857+0000","last_clean":"2024-04-16T19:32:11.152857+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:32:11.152857+0000","last_undegraded":"2024-04-16T19:32:11.152857+0000","last_fullsized":"2024-04-16T19:32:11.152857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:14.325870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'14","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150617+0000","last_change":"2024-04-16T17:47:55.155349+0000","last_active":"2024-04-16T19:32:06.150617+0000","last_peered":"2024-04-16T19:32:06.150617+0000","last_clean":"2024-04-16T19:32:06.150617+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:32:06.150617+0000","last_undegraded":"2024-04-16T19:32:06.150617+0000","last_fullsized":"2024-04-16T19:32:06.150617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163638+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:33:11.163638+0000","last_peered":"2024-04-16T19:33:11.163638+0000","last_clean":"2024-04-16T19:33:11.163638+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:33:11.163638+0000","last_undegraded":"2024-04-16T19:33:11.163638+0000","last_fullsized":"2024-04-16T19:33:11.163638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166835+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:33:06.166835+0000","last_peered":"2024-04-16T19:33:06.166835+0000","last_clean":"2024-04-16T19:33:06.166835+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:33:06.166835+0000","last_undegraded":"2024-04-16T19:33:06.166835+0000","last_fullsized":"2024-04-16T19:33:06.166835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":175,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152538+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:33:21.152538+0000","last_peered":"2024-04-16T19:33:21.152538+0000","last_clean":"2024-04-16T19:33:21.152538+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:33:21.152538+0000","last_undegraded":"2024-04-16T19:33:21.152538+0000","last_fullsized":"2024-04-16T19:33:21.152538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:05.646107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'104","reported_seq":273,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166177+0000","last_change":"2024-04-16T17:47:35.307146+0000","last_active":"2024-04-16T19:33:21.166177+0000","last_peered":"2024-04-16T19:33:21.166177+0000","last_clean":"2024-04-16T19:33:21.166177+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:33:21.166177+0000","last_undegraded":"2024-04-16T19:33:21.166177+0000","last_fullsized":"2024-04-16T19:33:21.166177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:28.729299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145847+0000","last_change":"2024-04-16T17:47:37.324514+0000","last_active":"2024-04-16T19:31:46.145847+0000","last_peered":"2024-04-16T19:31:46.145847+0000","last_clean":"2024-04-16T19:31:46.145847+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:31:46.145847+0000","last_undegraded":"2024-04-16T19:31:46.145847+0000","last_fullsized":"2024-04-16T19:31:46.145847+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183327+0000","last_change":"2024-04-16T17:47:39.340900+0000","last_active":"2024-04-16T19:34:11.183327+0000","last_peered":"2024-04-16T19:34:11.183327+0000","last_clean":"2024-04-16T19:34:11.183327+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:34:11.183327+0000","last_undegraded":"2024-04-16T19:34:11.183327+0000","last_fullsized":"2024-04-16T19:34:11.183327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.644408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167437+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:33:11.167437+0000","last_peered":"2024-04-16T19:33:11.167437+0000","last_clean":"2024-04-16T19:33:11.167437+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:33:11.167437+0000","last_undegraded":"2024-04-16T19:33:11.167437+0000","last_fullsized":"2024-04-16T19:33:11.167437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156402+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:32:36.156402+0000","last_peered":"2024-04-16T19:32:36.156402+0000","last_clean":"2024-04-16T19:32:36.156402+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:32:36.156402+0000","last_undegraded":"2024-04-16T19:32:36.156402+0000","last_fullsized":"2024-04-16T19:32:36.156402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157486+0000","last_change":"2024-04-16T17:48:27.308234+0000","last_active":"2024-04-16T19:32:41.157486+0000","last_peered":"2024-04-16T19:32:41.157486+0000","last_clean":"2024-04-16T19:32:41.157486+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:32:41.157486+0000","last_undegraded":"2024-04-16T19:32:41.157486+0000","last_fullsized":"2024-04-16T19:32:41.157486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'135","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163865+0000","last_change":"2024-04-16T17:48:01.254153+0000","last_active":"2024-04-16T19:32:46.163865+0000","last_peered":"2024-04-16T19:32:46.163865+0000","last_clean":"2024-04-16T19:32:46.163865+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:32:46.163865+0000","last_undegraded":"2024-04-16T19:32:46.163865+0000","last_fullsized":"2024-04-16T19:32:46.163865+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136391+0000","last_change":"2024-04-16T17:47:59.174180+0000","last_active":"2024-04-16T19:32:11.136391+0000","last_peered":"2024-04-16T19:32:11.136391+0000","last_clean":"2024-04-16T19:32:11.136391+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:32:11.136391+0000","last_undegraded":"2024-04-16T19:32:11.136391+0000","last_fullsized":"2024-04-16T19:32:11.136391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:29.135288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'679","reported_seq":1066,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136485+0000","last_change":"2024-04-16T17:47:57.182491+0000","last_active":"2024-04-16T19:32:11.136485+0000","last_peered":"2024-04-16T19:32:11.136485+0000","last_clean":"2024-04-16T19:32:11.136485+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:32:11.136485+0000","last_undegraded":"2024-04-16T19:32:11.136485+0000","last_fullsized":"2024-04-16T19:32:11.136485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:14.757156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'6","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156433+0000","last_change":"2024-04-16T17:47:55.158851+0000","last_active":"2024-04-16T19:32:26.156433+0000","last_peered":"2024-04-16T19:32:26.156433+0000","last_clean":"2024-04-16T19:32:26.156433+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:32:26.156433+0000","last_undegraded":"2024-04-16T19:32:26.156433+0000","last_fullsized":"2024-04-16T19:32:26.156433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:39:56.856045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170332+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:33:11.170332+0000","last_peered":"2024-04-16T19:33:11.170332+0000","last_clean":"2024-04-16T19:33:11.170332+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:33:11.170332+0000","last_undegraded":"2024-04-16T19:33:11.170332+0000","last_fullsized":"2024-04-16T19:33:11.170332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:31.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.149879+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:31:56.149879+0000","last_peered":"2024-04-16T19:31:56.149879+0000","last_clean":"2024-04-16T19:31:56.149879+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:31:56.149879+0000","last_undegraded":"2024-04-16T19:31:56.149879+0000","last_fullsized":"2024-04-16T19:31:56.149879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:06.495990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"48'13","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170506+0000","last_change":"2024-04-16T17:47:33.309159+0000","last_active":"2024-04-16T19:33:21.170506+0000","last_peered":"2024-04-16T19:33:21.170506+0000","last_clean":"2024-04-16T19:33:21.170506+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:33:21.170506+0000","last_undegraded":"2024-04-16T19:33:21.170506+0000","last_fullsized":"2024-04-16T19:33:21.170506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:26.709263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'152","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160039+0000","last_change":"2024-04-16T17:47:35.305022+0000","last_active":"2024-04-16T19:33:51.160039+0000","last_peered":"2024-04-16T19:33:51.160039+0000","last_clean":"2024-04-16T19:33:51.160039+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:33:51.160039+0000","last_undegraded":"2024-04-16T19:33:51.160039+0000","last_fullsized":"2024-04-16T19:33:51.160039+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:03.908445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.153692+0000","last_change":"2024-04-16T17:47:37.331019+0000","last_active":"2024-04-16T19:33:26.153692+0000","last_peered":"2024-04-16T19:33:26.153692+0000","last_clean":"2024-04-16T19:33:26.153692+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:33:26.153692+0000","last_undegraded":"2024-04-16T19:33:26.153692+0000","last_fullsized":"2024-04-16T19:33:26.153692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:18.849817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179728+0000","last_change":"2024-04-16T17:47:39.331395+0000","last_active":"2024-04-16T19:33:56.179728+0000","last_peered":"2024-04-16T19:33:56.179728+0000","last_clean":"2024-04-16T19:33:56.179728+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:33:56.179728+0000","last_undegraded":"2024-04-16T19:33:56.179728+0000","last_fullsized":"2024-04-16T19:33:56.179728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:12.149771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160965+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:32:36.160965+0000","last_peered":"2024-04-16T19:32:36.160965+0000","last_clean":"2024-04-16T19:32:36.160965+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:32:36.160965+0000","last_undegraded":"2024-04-16T19:32:36.160965+0000","last_fullsized":"2024-04-16T19:32:36.160965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'86","reported_seq":1117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163783+0000","last_change":"2024-04-16T17:48:34.326754+0000","last_active":"2024-04-16T19:32:46.163783+0000","last_peered":"2024-04-16T19:32:46.163783+0000","last_clean":"2024-04-16T19:32:46.163783+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:32:46.163783+0000","last_undegraded":"2024-04-16T19:32:46.163783+0000","last_fullsized":"2024-04-16T19:32:46.163783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159819+0000","last_change":"2024-04-16T17:48:27.298815+0000","last_active":"2024-04-16T19:32:31.159819+0000","last_peered":"2024-04-16T19:32:31.159819+0000","last_clean":"2024-04-16T19:32:31.159819+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:32:31.159819+0000","last_undegraded":"2024-04-16T19:32:31.159819+0000","last_fullsized":"2024-04-16T19:32:31.159819+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"48'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150226+0000","last_change":"2024-04-16T17:48:01.246604+0000","last_active":"2024-04-16T19:32:01.150226+0000","last_peered":"2024-04-16T19:32:01.150226+0000","last_clean":"2024-04-16T19:32:01.150226+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:32:01.150226+0000","last_undegraded":"2024-04-16T19:32:01.150226+0000","last_fullsized":"2024-04-16T19:32:01.150226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:16.195310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148847+0000","last_change":"2024-04-16T17:47:59.180320+0000","last_active":"2024-04-16T19:32:11.148847+0000","last_peered":"2024-04-16T19:32:11.148847+0000","last_clean":"2024-04-16T19:32:11.148847+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:32:11.148847+0000","last_undegraded":"2024-04-16T19:32:11.148847+0000","last_fullsized":"2024-04-16T19:32:11.148847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:16.107500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1483","reported_seq":5477,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153145+0000","last_change":"2024-04-16T17:47:57.186127+0000","last_active":"2024-04-16T19:32:26.153145+0000","last_peered":"2024-04-16T19:32:26.153145+0000","last_clean":"2024-04-16T19:32:26.153145+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:32:26.153145+0000","last_undegraded":"2024-04-16T19:32:26.153145+0000","last_fullsized":"2024-04-16T19:32:26.153145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1483,"log_dups_size":0,"ondisk_log_size":1483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:02.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'8","reported_seq":224,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152628+0000","last_change":"2024-04-16T17:47:55.161973+0000","last_active":"2024-04-16T19:32:06.152628+0000","last_peered":"2024-04-16T19:32:06.152628+0000","last_clean":"2024-04-16T19:32:06.152628+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:32:06.152628+0000","last_undegraded":"2024-04-16T19:32:06.152628+0000","last_fullsized":"2024-04-16T19:32:06.152628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163747+0000","last_change":"2024-04-16T17:47:23.266018+0000","last_active":"2024-04-16T19:33:11.163747+0000","last_peered":"2024-04-16T19:33:11.163747+0000","last_clean":"2024-04-16T19:33:11.163747+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:33:11.163747+0000","last_undegraded":"2024-04-16T19:33:11.163747+0000","last_fullsized":"2024-04-16T19:33:11.163747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168338+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:33:31.168338+0000","last_peered":"2024-04-16T19:33:31.168338+0000","last_clean":"2024-04-16T19:33:31.168338+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:33:31.168338+0000","last_undegraded":"2024-04-16T19:33:31.168338+0000","last_fullsized":"2024-04-16T19:33:31.168338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174370+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:33:46.174370+0000","last_peered":"2024-04-16T19:33:46.174370+0000","last_clean":"2024-04-16T19:33:46.174370+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:33:46.174370+0000","last_undegraded":"2024-04-16T19:33:46.174370+0000","last_fullsized":"2024-04-16T19:33:46.174370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'120","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170532+0000","last_change":"2024-04-16T17:47:35.319180+0000","last_active":"2024-04-16T19:33:21.170532+0000","last_peered":"2024-04-16T19:33:21.170532+0000","last_clean":"2024-04-16T19:33:21.170532+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:33:21.170532+0000","last_undegraded":"2024-04-16T19:33:21.170532+0000","last_fullsized":"2024-04-16T19:33:21.170532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:28.081965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.167247+0000","last_change":"2024-04-16T17:47:37.326971+0000","last_active":"2024-04-16T19:33:26.167247+0000","last_peered":"2024-04-16T19:33:26.167247+0000","last_clean":"2024-04-16T19:33:26.167247+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:33:26.167247+0000","last_undegraded":"2024-04-16T19:33:26.167247+0000","last_fullsized":"2024-04-16T19:33:26.167247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:04.366184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174929+0000","last_change":"2024-04-16T17:47:39.325433+0000","last_active":"2024-04-16T19:33:56.174929+0000","last_peered":"2024-04-16T19:33:56.174929+0000","last_clean":"2024-04-16T19:33:56.174929+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:33:56.174929+0000","last_undegraded":"2024-04-16T19:33:56.174929+0000","last_fullsized":"2024-04-16T19:33:56.174929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:50.427879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151752+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:32:06.151752+0000","last_peered":"2024-04-16T19:32:06.151752+0000","last_clean":"2024-04-16T19:32:06.151752+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:32:06.151752+0000","last_undegraded":"2024-04-16T19:32:06.151752+0000","last_fullsized":"2024-04-16T19:32:06.151752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:24.908570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'90","reported_seq":977,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.169718+0000","last_change":"2024-04-16T17:48:34.333850+0000","last_active":"2024-04-16T19:33:21.169718+0000","last_peered":"2024-04-16T19:33:21.169718+0000","last_clean":"2024-04-16T19:33:21.169718+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:33:21.169718+0000","last_undegraded":"2024-04-16T19:33:21.169718+0000","last_fullsized":"2024-04-16T19:33:21.169718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:31.546580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159785+0000","last_change":"2024-04-16T17:48:27.299707+0000","last_active":"2024-04-16T19:32:31.159785+0000","last_peered":"2024-04-16T19:32:31.159785+0000","last_clean":"2024-04-16T19:32:31.159785+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:32:31.159785+0000","last_undegraded":"2024-04-16T19:32:31.159785+0000","last_fullsized":"2024-04-16T19:32:31.159785+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'5","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150341+0000","last_change":"2024-04-16T17:48:01.258518+0000","last_active":"2024-04-16T19:32:01.150341+0000","last_peered":"2024-04-16T19:32:01.150341+0000","last_clean":"2024-04-16T19:32:01.150341+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:32:01.150341+0000","last_undegraded":"2024-04-16T19:32:01.150341+0000","last_fullsized":"2024-04-16T19:32:01.150341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:25.260545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'555","reported_seq":1941,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157758+0000","last_change":"2024-04-16T17:47:59.184752+0000","last_active":"2024-04-16T19:32:31.157758+0000","last_peered":"2024-04-16T19:32:31.157758+0000","last_clean":"2024-04-16T19:32:31.157758+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:32:31.157758+0000","last_undegraded":"2024-04-16T19:32:31.157758+0000","last_fullsized":"2024-04-16T19:32:31.157758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:36.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'651","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148802+0000","last_change":"2024-04-16T17:47:57.193900+0000","last_active":"2024-04-16T19:32:11.148802+0000","last_peered":"2024-04-16T19:32:11.148802+0000","last_clean":"2024-04-16T19:32:11.148802+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:32:11.148802+0000","last_undegraded":"2024-04-16T19:32:11.148802+0000","last_fullsized":"2024-04-16T19:32:11.148802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:12.186417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'12","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151428+0000","last_change":"2024-04-16T17:47:55.161557+0000","last_active":"2024-04-16T19:32:06.151428+0000","last_peered":"2024-04-16T19:32:06.151428+0000","last_clean":"2024-04-16T19:32:06.151428+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:32:06.151428+0000","last_undegraded":"2024-04-16T19:32:06.151428+0000","last_fullsized":"2024-04-16T19:32:06.151428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:51.923908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.176037+0000","last_change":"2024-04-16T17:47:23.261436+0000","last_active":"2024-04-16T19:33:41.176037+0000","last_peered":"2024-04-16T19:33:41.176037+0000","last_clean":"2024-04-16T19:33:41.176037+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:33:41.176037+0000","last_undegraded":"2024-04-16T19:33:41.176037+0000","last_fullsized":"2024-04-16T19:33:41.176037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154630+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:33:31.154630+0000","last_peered":"2024-04-16T19:33:31.154630+0000","last_clean":"2024-04-16T19:33:31.154630+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:33:31.154630+0000","last_undegraded":"2024-04-16T19:33:31.154630+0000","last_fullsized":"2024-04-16T19:33:31.154630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:35.902429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172709+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:33:21.172709+0000","last_peered":"2024-04-16T19:33:21.172709+0000","last_clean":"2024-04-16T19:33:21.172709+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:33:21.172709+0000","last_undegraded":"2024-04-16T19:33:21.172709+0000","last_fullsized":"2024-04-16T19:33:21.172709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:26.061929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'176","reported_seq":381,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167670+0000","last_change":"2024-04-16T17:47:35.317949+0000","last_active":"2024-04-16T19:33:21.167670+0000","last_peered":"2024-04-16T19:33:21.167670+0000","last_clean":"2024-04-16T19:33:21.167670+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:33:21.167670+0000","last_undegraded":"2024-04-16T19:33:21.167670+0000","last_fullsized":"2024-04-16T19:33:21.167670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.173767+0000","last_change":"2024-04-16T17:47:37.332636+0000","last_active":"2024-04-16T19:33:26.173767+0000","last_peered":"2024-04-16T19:33:26.173767+0000","last_clean":"2024-04-16T19:33:26.173767+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:33:26.173767+0000","last_undegraded":"2024-04-16T19:33:26.173767+0000","last_fullsized":"2024-04-16T19:33:26.173767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179806+0000","last_change":"2024-04-16T17:47:39.335342+0000","last_active":"2024-04-16T19:33:56.179806+0000","last_peered":"2024-04-16T19:33:56.179806+0000","last_clean":"2024-04-16T19:33:56.179806+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:33:56.179806+0000","last_undegraded":"2024-04-16T19:33:56.179806+0000","last_fullsized":"2024-04-16T19:33:56.179806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131831+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:31:51.131831+0000","last_peered":"2024-04-16T19:31:51.131831+0000","last_clean":"2024-04-16T19:31:51.131831+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:31:51.131831+0000","last_undegraded":"2024-04-16T19:31:51.131831+0000","last_fullsized":"2024-04-16T19:31:51.131831+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:43.281248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'118","reported_seq":1502,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156628+0000","last_change":"2024-04-16T17:48:34.329015+0000","last_active":"2024-04-16T19:32:41.156628+0000","last_peered":"2024-04-16T19:32:41.156628+0000","last_clean":"2024-04-16T19:32:41.156628+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:32:41.156628+0000","last_undegraded":"2024-04-16T19:32:41.156628+0000","last_fullsized":"2024-04-16T19:32:41.156628+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:38.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150062+0000","last_change":"2024-04-16T17:48:27.296987+0000","last_active":"2024-04-16T19:32:01.150062+0000","last_peered":"2024-04-16T19:32:01.150062+0000","last_clean":"2024-04-16T19:32:01.150062+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:32:01.150062+0000","last_undegraded":"2024-04-16T19:32:01.150062+0000","last_fullsized":"2024-04-16T19:32:01.150062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:13.235223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'8","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172452+0000","last_change":"2024-04-16T17:48:01.240484+0000","last_active":"2024-04-16T19:33:46.172452+0000","last_peered":"2024-04-16T19:33:46.172452+0000","last_clean":"2024-04-16T19:33:46.172452+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:33:46.172452+0000","last_undegraded":"2024-04-16T19:33:46.172452+0000","last_fullsized":"2024-04-16T19:33:46.172452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:38.257398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157906+0000","last_change":"2024-04-16T17:47:59.184121+0000","last_active":"2024-04-16T19:32:31.157906+0000","last_peered":"2024-04-16T19:32:31.157906+0000","last_clean":"2024-04-16T19:32:31.157906+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:32:31.157906+0000","last_undegraded":"2024-04-16T19:32:31.157906+0000","last_fullsized":"2024-04-16T19:32:31.157906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:46.708915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'783","reported_seq":43142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174072+0000","last_change":"2024-04-16T17:47:57.186095+0000","last_active":"2024-04-16T19:33:46.174072+0000","last_peered":"2024-04-16T19:33:46.174072+0000","last_clean":"2024-04-16T19:33:46.174072+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:33:46.174072+0000","last_undegraded":"2024-04-16T19:33:46.174072+0000","last_fullsized":"2024-04-16T19:33:46.174072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:50.249328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'9","reported_seq":208,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132707+0000","last_change":"2024-04-16T17:47:55.143458+0000","last_active":"2024-04-16T19:31:56.132707+0000","last_peered":"2024-04-16T19:31:56.132707+0000","last_clean":"2024-04-16T19:31:56.132707+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:31:56.132707+0000","last_undegraded":"2024-04-16T19:31:56.132707+0000","last_fullsized":"2024-04-16T19:31:56.132707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:53:52.793161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154404+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:32:11.154404+0000","last_peered":"2024-04-16T19:32:11.154404+0000","last_clean":"2024-04-16T19:32:11.154404+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:32:11.154404+0000","last_undegraded":"2024-04-16T19:32:11.154404+0000","last_fullsized":"2024-04-16T19:32:11.154404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:43.276218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171281+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:33:16.171281+0000","last_peered":"2024-04-16T19:33:16.171281+0000","last_clean":"2024-04-16T19:33:16.171281+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:33:16.171281+0000","last_undegraded":"2024-04-16T19:33:16.171281+0000","last_fullsized":"2024-04-16T19:33:16.171281+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174137+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:33:36.174137+0000","last_peered":"2024-04-16T19:33:36.174137+0000","last_clean":"2024-04-16T19:33:36.174137+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:33:36.174137+0000","last_undegraded":"2024-04-16T19:33:36.174137+0000","last_fullsized":"2024-04-16T19:33:36.174137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:33:14.972217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173777+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:33:51.173777+0000","last_peered":"2024-04-16T19:33:51.173777+0000","last_clean":"2024-04-16T19:33:51.173777+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:33:51.173777+0000","last_undegraded":"2024-04-16T19:33:51.173777+0000","last_fullsized":"2024-04-16T19:33:51.173777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:14.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'72","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174316+0000","last_change":"2024-04-16T17:47:35.310088+0000","last_active":"2024-04-16T19:33:46.174316+0000","last_peered":"2024-04-16T19:33:46.174316+0000","last_clean":"2024-04-16T19:33:46.174316+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:33:46.174316+0000","last_undegraded":"2024-04-16T19:33:46.174316+0000","last_fullsized":"2024-04-16T19:33:46.174316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177457+0000","last_change":"2024-04-16T17:47:37.327822+0000","last_active":"2024-04-16T19:33:51.177457+0000","last_peered":"2024-04-16T19:33:51.177457+0000","last_clean":"2024-04-16T19:33:51.177457+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:33:51.177457+0000","last_undegraded":"2024-04-16T19:33:51.177457+0000","last_fullsized":"2024-04-16T19:33:51.177457+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175307+0000","last_change":"2024-04-16T17:47:39.341107+0000","last_active":"2024-04-16T19:33:51.175307+0000","last_peered":"2024-04-16T19:33:51.175307+0000","last_clean":"2024-04-16T19:33:51.175307+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:33:51.175307+0000","last_undegraded":"2024-04-16T19:33:51.175307+0000","last_fullsized":"2024-04-16T19:33:51.175307+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157259+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:32:21.157259+0000","last_peered":"2024-04-16T19:32:21.157259+0000","last_clean":"2024-04-16T19:32:21.157259+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:32:21.157259+0000","last_undegraded":"2024-04-16T19:32:21.157259+0000","last_fullsized":"2024-04-16T19:32:21.157259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'100","reported_seq":1745,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147877+0000","last_change":"2024-04-16T17:48:34.333902+0000","last_active":"2024-04-16T19:32:06.147877+0000","last_peered":"2024-04-16T19:32:06.147877+0000","last_clean":"2024-04-16T19:32:06.147877+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:32:06.147877+0000","last_undegraded":"2024-04-16T19:32:06.147877+0000","last_fullsized":"2024-04-16T19:32:06.147877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:26.562009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146546+0000","last_change":"2024-04-16T17:48:27.296497+0000","last_active":"2024-04-16T19:32:01.146546+0000","last_peered":"2024-04-16T19:32:01.146546+0000","last_clean":"2024-04-16T19:32:01.146546+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:32:01.146546+0000","last_undegraded":"2024-04-16T19:32:01.146546+0000","last_fullsized":"2024-04-16T19:32:01.146546+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:59:05.786900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'8","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176076+0000","last_change":"2024-04-16T17:48:01.261918+0000","last_active":"2024-04-16T19:33:46.176076+0000","last_peered":"2024-04-16T19:33:46.176076+0000","last_clean":"2024-04-16T19:33:46.176076+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:33:46.176076+0000","last_undegraded":"2024-04-16T19:33:46.176076+0000","last_fullsized":"2024-04-16T19:33:46.176076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158533+0000","last_change":"2024-04-16T17:47:59.182311+0000","last_active":"2024-04-16T19:33:46.158533+0000","last_peered":"2024-04-16T19:33:46.158533+0000","last_clean":"2024-04-16T19:33:46.158533+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:33:46.158533+0000","last_undegraded":"2024-04-16T19:33:46.158533+0000","last_fullsized":"2024-04-16T19:33:46.158533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:15.733801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'897","reported_seq":3791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155083+0000","last_change":"2024-04-16T17:47:57.178953+0000","last_active":"2024-04-16T19:32:36.155083+0000","last_peered":"2024-04-16T19:32:36.155083+0000","last_clean":"2024-04-16T19:32:36.155083+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:32:36.155083+0000","last_undegraded":"2024-04-16T19:32:36.155083+0000","last_fullsized":"2024-04-16T19:32:36.155083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:27.120398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'29","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148129+0000","last_change":"2024-04-16T17:47:55.165769+0000","last_active":"2024-04-16T19:31:56.148129+0000","last_peered":"2024-04-16T19:31:56.148129+0000","last_clean":"2024-04-16T19:31:56.148129+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:31:56.148129+0000","last_undegraded":"2024-04-16T19:31:56.148129+0000","last_fullsized":"2024-04-16T19:31:56.148129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175855+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:33:41.175855+0000","last_peered":"2024-04-16T19:33:41.175855+0000","last_clean":"2024-04-16T19:33:41.175855+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:33:41.175855+0000","last_undegraded":"2024-04-16T19:33:41.175855+0000","last_fullsized":"2024-04-16T19:33:41.175855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159749+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:32:51.159749+0000","last_peered":"2024-04-16T19:32:51.159749+0000","last_clean":"2024-04-16T19:32:51.159749+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:32:51.159749+0000","last_undegraded":"2024-04-16T19:32:51.159749+0000","last_fullsized":"2024-04-16T19:32:51.159749+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169877+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:33:36.169877+0000","last_peered":"2024-04-16T19:33:36.169877+0000","last_clean":"2024-04-16T19:33:36.169877+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:33:36.169877+0000","last_undegraded":"2024-04-16T19:33:36.169877+0000","last_fullsized":"2024-04-16T19:33:36.169877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154521+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:32:21.154521+0000","last_peered":"2024-04-16T19:32:21.154521+0000","last_clean":"2024-04-16T19:32:21.154521+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:32:21.154521+0000","last_undegraded":"2024-04-16T19:32:21.154521+0000","last_fullsized":"2024-04-16T19:32:21.154521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'144","reported_seq":330,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177257+0000","last_change":"2024-04-16T17:47:35.314035+0000","last_active":"2024-04-16T19:33:46.177257+0000","last_peered":"2024-04-16T19:33:46.177257+0000","last_clean":"2024-04-16T19:33:46.177257+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:33:46.177257+0000","last_undegraded":"2024-04-16T19:33:46.177257+0000","last_fullsized":"2024-04-16T19:33:46.177257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181110+0000","last_change":"2024-04-16T17:47:37.334598+0000","last_active":"2024-04-16T19:33:51.181110+0000","last_peered":"2024-04-16T19:33:51.181110+0000","last_clean":"2024-04-16T19:33:51.181110+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:33:51.181110+0000","last_undegraded":"2024-04-16T19:33:51.181110+0000","last_fullsized":"2024-04-16T19:33:51.181110+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:24.856058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175255+0000","last_change":"2024-04-16T17:47:39.339775+0000","last_active":"2024-04-16T19:33:51.175255+0000","last_peered":"2024-04-16T19:33:51.175255+0000","last_clean":"2024-04-16T19:33:51.175255+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:33:51.175255+0000","last_undegraded":"2024-04-16T19:33:51.175255+0000","last_fullsized":"2024-04-16T19:33:51.175255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143941+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:31:51.143941+0000","last_peered":"2024-04-16T19:31:51.143941+0000","last_clean":"2024-04-16T19:31:51.143941+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:31:51.143941+0000","last_undegraded":"2024-04-16T19:31:51.143941+0000","last_fullsized":"2024-04-16T19:31:51.143941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:26.624319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148088+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:32:06.148088+0000","last_peered":"2024-04-16T19:32:06.148088+0000","last_clean":"2024-04-16T19:32:06.148088+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:32:06.148088+0000","last_undegraded":"2024-04-16T19:32:06.148088+0000","last_fullsized":"2024-04-16T19:32:06.148088+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146697+0000","last_change":"2024-04-16T17:48:27.307048+0000","last_active":"2024-04-16T19:32:01.146697+0000","last_peered":"2024-04-16T19:32:01.146697+0000","last_clean":"2024-04-16T19:32:01.146697+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:32:01.146697+0000","last_undegraded":"2024-04-16T19:32:01.146697+0000","last_fullsized":"2024-04-16T19:32:01.146697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:19.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'33","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152888+0000","last_change":"2024-04-16T17:48:01.250054+0000","last_active":"2024-04-16T19:32:06.152888+0000","last_peered":"2024-04-16T19:32:06.152888+0000","last_clean":"2024-04-16T19:32:06.152888+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:32:06.152888+0000","last_undegraded":"2024-04-16T19:32:06.152888+0000","last_fullsized":"2024-04-16T19:32:06.152888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133916+0000","last_change":"2024-04-16T17:47:59.190987+0000","last_active":"2024-04-16T19:32:01.133916+0000","last_peered":"2024-04-16T19:32:01.133916+0000","last_clean":"2024-04-16T19:32:01.133916+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:32:01.133916+0000","last_undegraded":"2024-04-16T19:32:01.133916+0000","last_fullsized":"2024-04-16T19:32:01.133916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:11.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'264","reported_seq":640,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151154+0000","last_change":"2024-04-16T17:47:57.189816+0000","last_active":"2024-04-16T19:32:01.151154+0000","last_peered":"2024-04-16T19:32:01.151154+0000","last_clean":"2024-04-16T19:32:01.151154+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:32:01.151154+0000","last_undegraded":"2024-04-16T19:32:01.151154+0000","last_fullsized":"2024-04-16T19:32:01.151154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'10","reported_seq":156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159575+0000","last_change":"2024-04-16T17:47:55.161898+0000","last_active":"2024-04-16T19:32:31.159575+0000","last_peered":"2024-04-16T19:32:31.159575+0000","last_clean":"2024-04-16T19:32:31.159575+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:32:31.159575+0000","last_undegraded":"2024-04-16T19:32:31.159575+0000","last_fullsized":"2024-04-16T19:32:31.159575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174563+0000","last_change":"2024-04-16T17:47:23.265711+0000","last_active":"2024-04-16T19:33:36.174563+0000","last_peered":"2024-04-16T19:33:36.174563+0000","last_clean":"2024-04-16T19:33:36.174563+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:33:36.174563+0000","last_undegraded":"2024-04-16T19:33:36.174563+0000","last_fullsized":"2024-04-16T19:33:36.174563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.158718+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:32:51.158718+0000","last_peered":"2024-04-16T19:32:51.158718+0000","last_clean":"2024-04-16T19:32:51.158718+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:32:51.158718+0000","last_undegraded":"2024-04-16T19:32:51.158718+0000","last_fullsized":"2024-04-16T19:32:51.158718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178254+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:33:51.178254+0000","last_peered":"2024-04-16T19:33:51.178254+0000","last_clean":"2024-04-16T19:33:51.178254+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:33:51.178254+0000","last_undegraded":"2024-04-16T19:33:51.178254+0000","last_fullsized":"2024-04-16T19:33:51.178254+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:49.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"48'4","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158694+0000","last_change":"2024-04-16T17:47:33.301081+0000","last_active":"2024-04-16T19:33:46.158694+0000","last_peered":"2024-04-16T19:33:46.158694+0000","last_clean":"2024-04-16T19:33:46.158694+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:33:46.158694+0000","last_undegraded":"2024-04-16T19:33:46.158694+0000","last_fullsized":"2024-04-16T19:33:46.158694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:05.482314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'151","reported_seq":1555,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149286+0000","last_change":"2024-04-16T17:47:35.316795+0000","last_active":"2024-04-16T19:32:11.149286+0000","last_peered":"2024-04-16T19:32:11.149286+0000","last_clean":"2024-04-16T19:32:11.149286+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:32:11.149286+0000","last_undegraded":"2024-04-16T19:32:11.149286+0000","last_fullsized":"2024-04-16T19:32:11.149286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178652+0000","last_change":"2024-04-16T17:47:37.326201+0000","last_active":"2024-04-16T19:33:56.178652+0000","last_peered":"2024-04-16T19:33:56.178652+0000","last_clean":"2024-04-16T19:33:56.178652+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:33:56.178652+0000","last_undegraded":"2024-04-16T19:33:56.178652+0000","last_fullsized":"2024-04-16T19:33:56.178652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170822+0000","last_change":"2024-04-16T17:47:39.332609+0000","last_active":"2024-04-16T19:33:26.170822+0000","last_peered":"2024-04-16T19:33:26.170822+0000","last_clean":"2024-04-16T19:33:26.170822+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:33:26.170822+0000","last_undegraded":"2024-04-16T19:33:26.170822+0000","last_fullsized":"2024-04-16T19:33:26.170822+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143874+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:31:51.143874+0000","last_peered":"2024-04-16T19:31:51.143874+0000","last_clean":"2024-04-16T19:31:51.143874+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:31:51.143874+0000","last_undegraded":"2024-04-16T19:31:51.143874+0000","last_fullsized":"2024-04-16T19:31:51.143874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:45.404216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'76","reported_seq":1158,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150775+0000","last_change":"2024-04-16T17:48:34.318880+0000","last_active":"2024-04-16T19:32:06.150775+0000","last_peered":"2024-04-16T19:32:06.150775+0000","last_clean":"2024-04-16T19:32:06.150775+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:32:06.150775+0000","last_undegraded":"2024-04-16T19:32:06.150775+0000","last_fullsized":"2024-04-16T19:32:06.150775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153591+0000","last_change":"2024-04-16T17:48:27.299238+0000","last_active":"2024-04-16T19:32:26.153591+0000","last_peered":"2024-04-16T19:32:26.153591+0000","last_clean":"2024-04-16T19:32:26.153591+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:32:26.153591+0000","last_undegraded":"2024-04-16T19:32:26.153591+0000","last_fullsized":"2024-04-16T19:32:26.153591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:49.437897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'6","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176200+0000","last_change":"2024-04-16T17:48:01.256444+0000","last_active":"2024-04-16T19:33:46.176200+0000","last_peered":"2024-04-16T19:33:46.176200+0000","last_clean":"2024-04-16T19:33:46.176200+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:33:46.176200+0000","last_undegraded":"2024-04-16T19:33:46.176200+0000","last_fullsized":"2024-04-16T19:33:46.176200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'555","reported_seq":1932,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133972+0000","last_change":"2024-04-16T17:47:59.183633+0000","last_active":"2024-04-16T19:32:01.133972+0000","last_peered":"2024-04-16T19:32:01.133972+0000","last_clean":"2024-04-16T19:32:01.133972+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:32:01.133972+0000","last_undegraded":"2024-04-16T19:32:01.133972+0000","last_fullsized":"2024-04-16T19:32:01.133972+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:14.201307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'306","reported_seq":623,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149338+0000","last_change":"2024-04-16T17:47:57.177185+0000","last_active":"2024-04-16T19:32:01.149338+0000","last_peered":"2024-04-16T19:32:01.149338+0000","last_clean":"2024-04-16T19:32:01.149338+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:32:01.149338+0000","last_undegraded":"2024-04-16T19:32:01.149338+0000","last_fullsized":"2024-04-16T19:32:01.149338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:18.465537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"48'5","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174804+0000","last_change":"2024-04-16T17:47:55.178154+0000","last_active":"2024-04-16T19:33:41.174804+0000","last_peered":"2024-04-16T19:33:41.174804+0000","last_clean":"2024-04-16T19:33:41.174804+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:33:41.174804+0000","last_undegraded":"2024-04-16T19:33:41.174804+0000","last_fullsized":"2024-04-16T19:33:41.174804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173663+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:33:36.173663+0000","last_peered":"2024-04-16T19:33:36.173663+0000","last_clean":"2024-04-16T19:33:36.173663+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:33:36.173663+0000","last_undegraded":"2024-04-16T19:33:36.173663+0000","last_fullsized":"2024-04-16T19:33:36.173663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164327+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:32:56.164327+0000","last_peered":"2024-04-16T19:32:56.164327+0000","last_clean":"2024-04-16T19:32:56.164327+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:32:56.164327+0000","last_undegraded":"2024-04-16T19:32:56.164327+0000","last_fullsized":"2024-04-16T19:32:56.164327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171318+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:33:16.171318+0000","last_peered":"2024-04-16T19:33:16.171318+0000","last_clean":"2024-04-16T19:33:16.171318+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:33:16.171318+0000","last_undegraded":"2024-04-16T19:33:16.171318+0000","last_fullsized":"2024-04-16T19:33:16.171318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:05:07.906618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173252+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:33:36.173252+0000","last_peered":"2024-04-16T19:33:36.173252+0000","last_clean":"2024-04-16T19:33:36.173252+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:33:36.173252+0000","last_undegraded":"2024-04-16T19:33:36.173252+0000","last_fullsized":"2024-04-16T19:33:36.173252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:30.892245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":504,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174348+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:33:46.174348+0000","last_peered":"2024-04-16T19:33:46.174348+0000","last_clean":"2024-04-16T19:33:46.174348+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:33:46.174348+0000","last_undegraded":"2024-04-16T19:33:46.174348+0000","last_fullsized":"2024-04-16T19:33:46.174348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'177","reported_seq":1612,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178544+0000","last_change":"2024-04-16T17:47:35.312413+0000","last_active":"2024-04-16T19:33:51.178544+0000","last_peered":"2024-04-16T19:33:51.178544+0000","last_clean":"2024-04-16T19:33:51.178544+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:33:51.178544+0000","last_undegraded":"2024-04-16T19:33:51.178544+0000","last_fullsized":"2024-04-16T19:33:51.178544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152741+0000","last_change":"2024-04-16T17:47:37.327792+0000","last_active":"2024-04-16T19:32:26.152741+0000","last_peered":"2024-04-16T19:32:26.152741+0000","last_clean":"2024-04-16T19:32:26.152741+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:32:26.152741+0000","last_undegraded":"2024-04-16T19:32:26.152741+0000","last_fullsized":"2024-04-16T19:32:26.152741+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:43.802094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152938+0000","last_change":"2024-04-16T17:47:39.700262+0000","last_active":"2024-04-16T19:32:11.152938+0000","last_peered":"2024-04-16T19:32:11.152938+0000","last_clean":"2024-04-16T19:32:11.152938+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:32:11.152938+0000","last_undegraded":"2024-04-16T19:32:11.152938+0000","last_fullsized":"2024-04-16T19:32:11.152938+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:26.867823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157247+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:33:41.157247+0000","last_peered":"2024-04-16T19:33:41.157247+0000","last_clean":"2024-04-16T19:33:41.157247+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:33:41.157247+0000","last_undegraded":"2024-04-16T19:33:41.157247+0000","last_fullsized":"2024-04-16T19:33:41.157247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.324013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150723+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:32:06.150723+0000","last_peered":"2024-04-16T19:32:06.150723+0000","last_clean":"2024-04-16T19:32:06.150723+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:32:06.150723+0000","last_undegraded":"2024-04-16T19:32:06.150723+0000","last_fullsized":"2024-04-16T19:32:06.150723+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:02.642376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140845+0000","last_change":"2024-04-16T17:48:27.304274+0000","last_active":"2024-04-16T19:32:31.140845+0000","last_peered":"2024-04-16T19:32:31.140845+0000","last_clean":"2024-04-16T19:32:31.140845+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:32:31.140845+0000","last_undegraded":"2024-04-16T19:32:31.140845+0000","last_fullsized":"2024-04-16T19:32:31.140845+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:19.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'14","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133996+0000","last_change":"2024-04-16T17:48:01.251987+0000","last_active":"2024-04-16T19:32:01.133996+0000","last_peered":"2024-04-16T19:32:01.133996+0000","last_clean":"2024-04-16T19:32:01.133996+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:32:01.133996+0000","last_undegraded":"2024-04-16T19:32:01.133996+0000","last_fullsized":"2024-04-16T19:32:01.133996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:53.985409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'555","reported_seq":1923,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172522+0000","last_change":"2024-04-16T17:47:59.188607+0000","last_active":"2024-04-16T19:33:46.172522+0000","last_peered":"2024-04-16T19:33:46.172522+0000","last_clean":"2024-04-16T19:33:46.172522+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:33:46.172522+0000","last_undegraded":"2024-04-16T19:33:46.172522+0000","last_fullsized":"2024-04-16T19:33:46.172522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:09.833400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'514","reported_seq":1190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152969+0000","last_change":"2024-04-16T17:47:57.187984+0000","last_active":"2024-04-16T19:32:06.152969+0000","last_peered":"2024-04-16T19:32:06.152969+0000","last_clean":"2024-04-16T19:32:06.152969+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:32:06.152969+0000","last_undegraded":"2024-04-16T19:32:06.152969+0000","last_fullsized":"2024-04-16T19:32:06.152969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:09.942121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"51'3","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157358+0000","last_change":"2024-04-16T17:47:55.173104+0000","last_active":"2024-04-16T19:33:41.157358+0000","last_peered":"2024-04-16T19:33:41.157358+0000","last_clean":"2024-04-16T19:33:41.157358+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:33:41.157358+0000","last_undegraded":"2024-04-16T19:33:41.157358+0000","last_fullsized":"2024-04-16T19:33:41.157358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:46.009020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.170986+0000","last_change":"2024-04-16T17:47:23.266373+0000","last_active":"2024-04-16T19:33:41.170986+0000","last_peered":"2024-04-16T19:33:41.170986+0000","last_clean":"2024-04-16T19:33:41.170986+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:33:41.170986+0000","last_undegraded":"2024-04-16T19:33:41.170986+0000","last_fullsized":"2024-04-16T19:33:41.170986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:11.306509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158724+0000","last_change":"2024-04-16T17:47:32.273580+0000","last_active":"2024-04-16T19:33:46.158724+0000","last_peered":"2024-04-16T19:33:46.158724+0000","last_clean":"2024-04-16T19:33:46.158724+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:33:46.158724+0000","last_undegraded":"2024-04-16T19:33:46.158724+0000","last_fullsized":"2024-04-16T19:33:46.158724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:28.727683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164407+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:32:56.164407+0000","last_peered":"2024-04-16T19:32:56.164407+0000","last_clean":"2024-04-16T19:32:56.164407+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:32:56.164407+0000","last_undegraded":"2024-04-16T19:32:56.164407+0000","last_fullsized":"2024-04-16T19:32:56.164407+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163230+0000","last_change":"2024-04-16T17:47:33.300039+0000","last_active":"2024-04-16T19:34:06.163230+0000","last_peered":"2024-04-16T19:34:06.163230+0000","last_clean":"2024-04-16T19:34:06.163230+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:34:06.163230+0000","last_undegraded":"2024-04-16T19:34:06.163230+0000","last_fullsized":"2024-04-16T19:34:06.163230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:42.946001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'144","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173676+0000","last_change":"2024-04-16T17:47:35.305470+0000","last_active":"2024-04-16T19:33:51.173676+0000","last_peered":"2024-04-16T19:33:51.173676+0000","last_clean":"2024-04-16T19:33:51.173676+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:33:51.173676+0000","last_undegraded":"2024-04-16T19:33:51.173676+0000","last_fullsized":"2024-04-16T19:33:51.173676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:00.096888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.161004+0000","last_change":"2024-04-16T17:47:37.334307+0000","last_active":"2024-04-16T19:33:56.161004+0000","last_peered":"2024-04-16T19:33:56.161004+0000","last_clean":"2024-04-16T19:33:56.161004+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:33:56.161004+0000","last_undegraded":"2024-04-16T19:33:56.161004+0000","last_fullsized":"2024-04-16T19:33:56.161004+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:58.517922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168901+0000","last_change":"2024-04-16T17:47:39.343779+0000","last_active":"2024-04-16T19:33:26.168901+0000","last_peered":"2024-04-16T19:33:26.168901+0000","last_clean":"2024-04-16T19:33:26.168901+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:33:26.168901+0000","last_undegraded":"2024-04-16T19:33:26.168901+0000","last_fullsized":"2024-04-16T19:33:26.168901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:43:28.188142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131712+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:31:51.131712+0000","last_peered":"2024-04-16T19:31:51.131712+0000","last_clean":"2024-04-16T19:31:51.131712+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:31:51.131712+0000","last_undegraded":"2024-04-16T19:31:51.131712+0000","last_fullsized":"2024-04-16T19:31:51.131712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:49:14.730835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'122","reported_seq":1763,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170106+0000","last_change":"2024-04-16T17:48:34.337192+0000","last_active":"2024-04-16T19:33:11.170106+0000","last_peered":"2024-04-16T19:33:11.170106+0000","last_clean":"2024-04-16T19:33:11.170106+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:33:11.170106+0000","last_undegraded":"2024-04-16T19:33:11.170106+0000","last_fullsized":"2024-04-16T19:33:11.170106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.154528+0000","last_change":"2024-04-16T17:48:27.301617+0000","last_active":"2024-04-16T19:32:31.154528+0000","last_peered":"2024-04-16T19:32:31.154528+0000","last_clean":"2024-04-16T19:32:31.154528+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:32:31.154528+0000","last_undegraded":"2024-04-16T19:32:31.154528+0000","last_fullsized":"2024-04-16T19:32:31.154528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'16","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133844+0000","last_change":"2024-04-16T17:48:01.234685+0000","last_active":"2024-04-16T19:32:01.133844+0000","last_peered":"2024-04-16T19:32:01.133844+0000","last_clean":"2024-04-16T19:32:01.133844+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:32:01.133844+0000","last_undegraded":"2024-04-16T19:32:01.133844+0000","last_fullsized":"2024-04-16T19:32:01.133844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:02.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152930+0000","last_change":"2024-04-16T17:47:59.185616+0000","last_active":"2024-04-16T19:32:06.152930+0000","last_peered":"2024-04-16T19:32:06.152930+0000","last_clean":"2024-04-16T19:32:06.152930+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:32:06.152930+0000","last_undegraded":"2024-04-16T19:32:06.152930+0000","last_fullsized":"2024-04-16T19:32:06.152930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:24:43.650159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1159","reported_seq":1411,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172544+0000","last_change":"2024-04-16T17:47:57.188806+0000","last_active":"2024-04-16T19:33:46.172544+0000","last_peered":"2024-04-16T19:33:46.172544+0000","last_clean":"2024-04-16T19:33:46.172544+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:33:46.172544+0000","last_undegraded":"2024-04-16T19:33:46.172544+0000","last_fullsized":"2024-04-16T19:33:46.172544+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1159,"log_dups_size":0,"ondisk_log_size":1159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:00.199310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"48'18","reported_seq":399,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177873+0000","last_change":"2024-04-16T17:47:55.164040+0000","last_active":"2024-04-16T19:33:41.177873+0000","last_peered":"2024-04-16T19:33:41.177873+0000","last_clean":"2024-04-16T19:33:41.177873+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:33:41.177873+0000","last_undegraded":"2024-04-16T19:33:41.177873+0000","last_fullsized":"2024-04-16T19:33:41.177873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.179804+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:34:01.179804+0000","last_peered":"2024-04-16T19:34:01.179804+0000","last_clean":"2024-04-16T19:34:01.179804+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:34:01.179804+0000","last_undegraded":"2024-04-16T19:34:01.179804+0000","last_fullsized":"2024-04-16T19:34:01.179804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154908+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:33:31.154908+0000","last_peered":"2024-04-16T19:33:31.154908+0000","last_clean":"2024-04-16T19:33:31.154908+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:33:31.154908+0000","last_undegraded":"2024-04-16T19:33:31.154908+0000","last_fullsized":"2024-04-16T19:33:31.154908+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:55:03.538702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168590+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:33:31.168590+0000","last_peered":"2024-04-16T19:33:31.168590+0000","last_clean":"2024-04-16T19:33:31.168590+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:33:31.168590+0000","last_undegraded":"2024-04-16T19:33:31.168590+0000","last_fullsized":"2024-04-16T19:33:31.168590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180686+0000","last_change":"2024-04-16T17:47:33.300948+0000","last_active":"2024-04-16T19:33:51.180686+0000","last_peered":"2024-04-16T19:33:51.180686+0000","last_clean":"2024-04-16T19:33:51.180686+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:33:51.180686+0000","last_undegraded":"2024-04-16T19:33:51.180686+0000","last_fullsized":"2024-04-16T19:33:51.180686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'120","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180791+0000","last_change":"2024-04-16T17:47:35.313794+0000","last_active":"2024-04-16T19:33:51.180791+0000","last_peered":"2024-04-16T19:33:51.180791+0000","last_clean":"2024-04-16T19:33:51.180791+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:33:51.180791+0000","last_undegraded":"2024-04-16T19:33:51.180791+0000","last_fullsized":"2024-04-16T19:33:51.180791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178567+0000","last_change":"2024-04-16T17:47:37.329509+0000","last_active":"2024-04-16T19:33:56.178567+0000","last_peered":"2024-04-16T19:33:56.178567+0000","last_clean":"2024-04-16T19:33:56.178567+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:33:56.178567+0000","last_undegraded":"2024-04-16T19:33:56.178567+0000","last_fullsized":"2024-04-16T19:33:56.178567+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170908+0000","last_change":"2024-04-16T17:47:39.697737+0000","last_active":"2024-04-16T19:33:26.170908+0000","last_peered":"2024-04-16T19:33:26.170908+0000","last_clean":"2024-04-16T19:33:26.170908+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:33:26.170908+0000","last_undegraded":"2024-04-16T19:33:26.170908+0000","last_fullsized":"2024-04-16T19:33:26.170908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.150046+0000","last_change":"2024-04-16T18:10:50.537157+0000","last_active":"2024-04-16T19:33:11.150046+0000","last_peered":"2024-04-16T19:33:11.150046+0000","last_clean":"2024-04-16T19:33:11.150046+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:33:11.150046+0000","last_undegraded":"2024-04-16T19:33:11.150046+0000","last_fullsized":"2024-04-16T19:33:11.150046+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:13.931584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1282,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156075+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:32:36.156075+0000","last_peered":"2024-04-16T19:32:36.156075+0000","last_clean":"2024-04-16T19:32:36.156075+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:32:36.156075+0000","last_undegraded":"2024-04-16T19:32:36.156075+0000","last_fullsized":"2024-04-16T19:32:36.156075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:20.310434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147837+0000","last_change":"2024-04-16T17:48:27.305636+0000","last_active":"2024-04-16T19:33:01.147837+0000","last_peered":"2024-04-16T19:33:01.147837+0000","last_clean":"2024-04-16T19:33:01.147837+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:33:01.147837+0000","last_undegraded":"2024-04-16T19:33:01.147837+0000","last_fullsized":"2024-04-16T19:33:01.147837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:06.425749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'13","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175996+0000","last_change":"2024-04-16T17:48:01.266136+0000","last_active":"2024-04-16T19:33:46.175996+0000","last_peered":"2024-04-16T19:33:46.175996+0000","last_clean":"2024-04-16T19:33:46.175996+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:33:46.175996+0000","last_undegraded":"2024-04-16T19:33:46.175996+0000","last_fullsized":"2024-04-16T19:33:46.175996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177121+0000","last_change":"2024-04-16T17:47:59.188856+0000","last_active":"2024-04-16T19:33:46.177121+0000","last_peered":"2024-04-16T19:33:46.177121+0000","last_clean":"2024-04-16T19:33:46.177121+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:33:46.177121+0000","last_undegraded":"2024-04-16T19:33:46.177121+0000","last_fullsized":"2024-04-16T19:33:46.177121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:35.433741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1314","reported_seq":1867,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172479+0000","last_change":"2024-04-16T17:47:57.192362+0000","last_active":"2024-04-16T19:33:46.172479+0000","last_peered":"2024-04-16T19:33:46.172479+0000","last_clean":"2024-04-16T19:33:46.172479+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:33:46.172479+0000","last_undegraded":"2024-04-16T19:33:46.172479+0000","last_fullsized":"2024-04-16T19:33:46.172479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1314,"log_dups_size":0,"ondisk_log_size":1314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:14.446218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'4","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178048+0000","last_change":"2024-04-16T17:47:55.156911+0000","last_active":"2024-04-16T19:33:41.178048+0000","last_peered":"2024-04-16T19:33:41.178048+0000","last_clean":"2024-04-16T19:33:41.178048+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:33:41.178048+0000","last_undegraded":"2024-04-16T19:33:41.178048+0000","last_fullsized":"2024-04-16T19:33:41.178048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174726+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:33:41.174726+0000","last_peered":"2024-04-16T19:33:41.174726+0000","last_clean":"2024-04-16T19:33:41.174726+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:33:41.174726+0000","last_undegraded":"2024-04-16T19:33:41.174726+0000","last_fullsized":"2024-04-16T19:33:41.174726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174314+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:33:36.174314+0000","last_peered":"2024-04-16T19:33:36.174314+0000","last_clean":"2024-04-16T19:33:36.174314+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:33:36.174314+0000","last_undegraded":"2024-04-16T19:33:36.174314+0000","last_fullsized":"2024-04-16T19:33:36.174314+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:22.291956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163913+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:32:46.163913+0000","last_peered":"2024-04-16T19:32:46.163913+0000","last_clean":"2024-04-16T19:32:46.163913+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:32:46.163913+0000","last_undegraded":"2024-04-16T19:32:46.163913+0000","last_fullsized":"2024-04-16T19:32:46.163913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175384+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:33:51.175384+0000","last_peered":"2024-04-16T19:33:51.175384+0000","last_clean":"2024-04-16T19:33:51.175384+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:33:51.175384+0000","last_undegraded":"2024-04-16T19:33:51.175384+0000","last_fullsized":"2024-04-16T19:33:51.175384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'168","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178373+0000","last_change":"2024-04-16T17:47:35.321995+0000","last_active":"2024-04-16T19:33:51.178373+0000","last_peered":"2024-04-16T19:33:51.178373+0000","last_clean":"2024-04-16T19:33:51.178373+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:33:51.178373+0000","last_undegraded":"2024-04-16T19:33:51.178373+0000","last_fullsized":"2024-04-16T19:33:51.178373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:17.922789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'12","reported_seq":1342,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179297+0000","last_change":"2024-04-16T17:47:37.323586+0000","last_active":"2024-04-16T19:34:06.179297+0000","last_peered":"2024-04-16T19:34:06.179297+0000","last_clean":"2024-04-16T19:34:06.179297+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:34:06.179297+0000","last_undegraded":"2024-04-16T19:34:06.179297+0000","last_fullsized":"2024-04-16T19:34:06.179297+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145872+0000","last_change":"2024-04-16T17:47:39.698152+0000","last_active":"2024-04-16T19:31:46.145872+0000","last_peered":"2024-04-16T19:31:46.145872+0000","last_clean":"2024-04-16T19:31:46.145872+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:31:46.145872+0000","last_undegraded":"2024-04-16T19:31:46.145872+0000","last_fullsized":"2024-04-16T19:31:46.145872+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:08.010344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.164914+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:33:11.164914+0000","last_peered":"2024-04-16T19:33:11.164914+0000","last_clean":"2024-04-16T19:33:11.164914+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:33:11.164914+0000","last_undegraded":"2024-04-16T19:33:11.164914+0000","last_fullsized":"2024-04-16T19:33:11.164914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:36:23.318927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158153+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:32:36.158153+0000","last_peered":"2024-04-16T19:32:36.158153+0000","last_clean":"2024-04-16T19:32:36.158153+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:32:36.158153+0000","last_undegraded":"2024-04-16T19:32:36.158153+0000","last_fullsized":"2024-04-16T19:32:36.158153+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151256+0000","last_change":"2024-04-16T17:48:27.298996+0000","last_active":"2024-04-16T19:32:01.151256+0000","last_peered":"2024-04-16T19:32:01.151256+0000","last_clean":"2024-04-16T19:32:01.151256+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:32:01.151256+0000","last_undegraded":"2024-04-16T19:32:01.151256+0000","last_fullsized":"2024-04-16T19:32:01.151256+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"51'4","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140996+0000","last_change":"2024-04-16T17:48:01.252079+0000","last_active":"2024-04-16T19:32:31.140996+0000","last_peered":"2024-04-16T19:32:31.140996+0000","last_clean":"2024-04-16T19:32:31.140996+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:32:31.140996+0000","last_undegraded":"2024-04-16T19:32:31.140996+0000","last_fullsized":"2024-04-16T19:32:31.140996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:57.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'555","reported_seq":1994,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158653+0000","last_change":"2024-04-16T17:47:59.197276+0000","last_active":"2024-04-16T19:33:46.158653+0000","last_peered":"2024-04-16T19:33:46.158653+0000","last_clean":"2024-04-16T19:33:46.158653+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:33:46.158653+0000","last_undegraded":"2024-04-16T19:33:46.158653+0000","last_fullsized":"2024-04-16T19:33:46.158653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:16.461108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'553","reported_seq":822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172334+0000","last_change":"2024-04-16T17:47:57.191520+0000","last_active":"2024-04-16T19:33:46.172334+0000","last_peered":"2024-04-16T19:33:46.172334+0000","last_clean":"2024-04-16T19:33:46.172334+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:33:46.172334+0000","last_undegraded":"2024-04-16T19:33:46.172334+0000","last_fullsized":"2024-04-16T19:33:46.172334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:25.845115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'14","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144097+0000","last_change":"2024-04-16T17:47:55.178734+0000","last_active":"2024-04-16T19:31:51.144097+0000","last_peered":"2024-04-16T19:31:51.144097+0000","last_clean":"2024-04-16T19:31:51.144097+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:31:51.144097+0000","last_undegraded":"2024-04-16T19:31:51.144097+0000","last_fullsized":"2024-04-16T19:31:51.144097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175692+0000","last_change":"2024-04-16T17:47:23.265822+0000","last_active":"2024-04-16T19:33:41.175692+0000","last_peered":"2024-04-16T19:33:41.175692+0000","last_clean":"2024-04-16T19:33:41.175692+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:33:41.175692+0000","last_undegraded":"2024-04-16T19:33:41.175692+0000","last_fullsized":"2024-04-16T19:33:41.175692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178216+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:34:01.178216+0000","last_peered":"2024-04-16T19:34:01.178216+0000","last_clean":"2024-04-16T19:34:01.178216+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:34:01.178216+0000","last_undegraded":"2024-04-16T19:34:01.178216+0000","last_fullsized":"2024-04-16T19:34:01.178216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.158816+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:32:46.158816+0000","last_peered":"2024-04-16T19:32:46.158816+0000","last_clean":"2024-04-16T19:32:46.158816+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:32:46.158816+0000","last_undegraded":"2024-04-16T19:32:46.158816+0000","last_fullsized":"2024-04-16T19:32:46.158816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177291+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:33:51.177291+0000","last_peered":"2024-04-16T19:33:51.177291+0000","last_clean":"2024-04-16T19:33:51.177291+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:33:51.177291+0000","last_undegraded":"2024-04-16T19:33:51.177291+0000","last_fullsized":"2024-04-16T19:33:51.177291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'160","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182468+0000","last_change":"2024-04-16T17:47:35.312410+0000","last_active":"2024-04-16T19:34:11.182468+0000","last_peered":"2024-04-16T19:34:11.182468+0000","last_clean":"2024-04-16T19:34:11.182468+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:34:11.182468+0000","last_undegraded":"2024-04-16T19:34:11.182468+0000","last_fullsized":"2024-04-16T19:34:11.182468+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174883+0000","last_change":"2024-04-16T17:47:37.322938+0000","last_active":"2024-04-16T19:33:56.174883+0000","last_peered":"2024-04-16T19:33:56.174883+0000","last_clean":"2024-04-16T19:33:56.174883+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:33:56.174883+0000","last_undegraded":"2024-04-16T19:33:56.174883+0000","last_fullsized":"2024-04-16T19:33:56.174883+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:33.277719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170994+0000","last_change":"2024-04-16T17:47:39.699806+0000","last_active":"2024-04-16T19:33:26.170994+0000","last_peered":"2024-04-16T19:33:26.170994+0000","last_clean":"2024-04-16T19:33:26.170994+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:33:26.170994+0000","last_undegraded":"2024-04-16T19:33:26.170994+0000","last_fullsized":"2024-04-16T19:33:26.170994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164695+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:33:01.164695+0000","last_peered":"2024-04-16T19:33:01.164695+0000","last_clean":"2024-04-16T19:33:01.164695+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:33:01.164695+0000","last_undegraded":"2024-04-16T19:33:01.164695+0000","last_fullsized":"2024-04-16T19:33:01.164695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1098,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168822+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:33:06.168822+0000","last_peered":"2024-04-16T19:33:06.168822+0000","last_clean":"2024-04-16T19:33:06.168822+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:33:06.168822+0000","last_undegraded":"2024-04-16T19:33:06.168822+0000","last_fullsized":"2024-04-16T19:33:06.168822+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:38.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172871+0000","last_change":"2024-04-16T17:48:27.307428+0000","last_active":"2024-04-16T19:33:31.172871+0000","last_peered":"2024-04-16T19:33:31.172871+0000","last_clean":"2024-04-16T19:33:31.172871+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:33:31.172871+0000","last_undegraded":"2024-04-16T19:33:31.172871+0000","last_fullsized":"2024-04-16T19:33:31.172871+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:33.963133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161215+0000","last_change":"2024-04-16T17:48:01.240690+0000","last_active":"2024-04-16T19:33:01.161215+0000","last_peered":"2024-04-16T19:33:01.161215+0000","last_clean":"2024-04-16T19:33:01.161215+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:33:01.161215+0000","last_undegraded":"2024-04-16T19:33:01.161215+0000","last_fullsized":"2024-04-16T19:33:01.161215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:42.135971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162598+0000","last_change":"2024-04-16T17:47:59.192669+0000","last_active":"2024-04-16T19:33:01.162598+0000","last_peered":"2024-04-16T19:33:01.162598+0000","last_clean":"2024-04-16T19:33:01.162598+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:33:01.162598+0000","last_undegraded":"2024-04-16T19:33:01.162598+0000","last_fullsized":"2024-04-16T19:33:01.162598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'850","reported_seq":1151,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161341+0000","last_change":"2024-04-16T17:47:57.193698+0000","last_active":"2024-04-16T19:33:01.161341+0000","last_peered":"2024-04-16T19:33:01.161341+0000","last_clean":"2024-04-16T19:33:01.161341+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:33:01.161341+0000","last_undegraded":"2024-04-16T19:33:01.161341+0000","last_fullsized":"2024-04-16T19:33:01.161341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:15.692887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'16","reported_seq":234,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.178756+0000","last_change":"2024-04-16T17:49:32.398153+0000","last_active":"2024-04-16T19:34:11.178756+0000","last_peered":"2024-04-16T19:34:11.178756+0000","last_clean":"2024-04-16T19:34:11.178756+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:34:11.178756+0000","last_undegraded":"2024-04-16T19:34:11.178756+0000","last_fullsized":"2024-04-16T19:34:11.178756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:45.378036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178826+0000","last_change":"2024-04-16T17:47:39.336834+0000","last_active":"2024-04-16T19:33:56.178826+0000","last_peered":"2024-04-16T19:33:56.178826+0000","last_clean":"2024-04-16T19:33:56.178826+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:33:56.178826+0000","last_undegraded":"2024-04-16T19:33:56.178826+0000","last_fullsized":"2024-04-16T19:33:56.178826+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:09.285627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152606+0000","last_change":"2024-04-16T17:47:23.257996+0000","last_active":"2024-04-16T19:32:26.152606+0000","last_peered":"2024-04-16T19:32:26.152606+0000","last_clean":"2024-04-16T19:32:26.152606+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:32:26.152606+0000","last_undegraded":"2024-04-16T19:32:26.152606+0000","last_fullsized":"2024-04-16T19:32:26.152606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166359+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:32:56.166359+0000","last_peered":"2024-04-16T19:32:56.166359+0000","last_clean":"2024-04-16T19:32:56.166359+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:32:56.166359+0000","last_undegraded":"2024-04-16T19:32:56.166359+0000","last_fullsized":"2024-04-16T19:32:56.166359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"48'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159071+0000","last_change":"2024-04-16T17:47:33.297781+0000","last_active":"2024-04-16T19:32:36.159071+0000","last_peered":"2024-04-16T19:32:36.159071+0000","last_clean":"2024-04-16T19:32:36.159071+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:32:36.159071+0000","last_undegraded":"2024-04-16T19:32:36.159071+0000","last_fullsized":"2024-04-16T19:32:36.159071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:51.904305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142052+0000","last_change":"2024-04-16T17:47:35.310182+0000","last_active":"2024-04-16T19:32:36.142052+0000","last_peered":"2024-04-16T19:32:36.142052+0000","last_clean":"2024-04-16T19:32:36.142052+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:32:36.142052+0000","last_undegraded":"2024-04-16T19:32:36.142052+0000","last_fullsized":"2024-04-16T19:32:36.142052+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:57:05.362295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160301+0000","last_change":"2024-04-16T17:47:37.334854+0000","last_active":"2024-04-16T19:32:41.160301+0000","last_peered":"2024-04-16T19:32:41.160301+0000","last_clean":"2024-04-16T19:32:41.160301+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:32:41.160301+0000","last_undegraded":"2024-04-16T19:32:41.160301+0000","last_fullsized":"2024-04-16T19:32:41.160301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:24.862379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168290+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:33:31.168290+0000","last_peered":"2024-04-16T19:33:31.168290+0000","last_clean":"2024-04-16T19:33:31.168290+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:33:31.168290+0000","last_undegraded":"2024-04-16T19:33:31.168290+0000","last_fullsized":"2024-04-16T19:33:31.168290+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:11.558629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'55","reported_seq":671,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157685+0000","last_change":"2024-04-16T17:48:34.343212+0000","last_active":"2024-04-16T19:32:31.157685+0000","last_peered":"2024-04-16T19:32:31.157685+0000","last_clean":"2024-04-16T19:32:31.157685+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:32:31.157685+0000","last_undegraded":"2024-04-16T19:32:31.157685+0000","last_fullsized":"2024-04-16T19:32:31.157685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:28:18.790386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168495+0000","last_change":"2024-04-16T17:48:27.301479+0000","last_active":"2024-04-16T19:33:31.168495+0000","last_peered":"2024-04-16T19:33:31.168495+0000","last_clean":"2024-04-16T19:33:31.168495+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:33:31.168495+0000","last_undegraded":"2024-04-16T19:33:31.168495+0000","last_fullsized":"2024-04-16T19:33:31.168495+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:04.776813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'9","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161137+0000","last_change":"2024-04-16T17:48:01.250254+0000","last_active":"2024-04-16T19:33:01.161137+0000","last_peered":"2024-04-16T19:33:01.161137+0000","last_clean":"2024-04-16T19:33:01.161137+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:33:01.161137+0000","last_undegraded":"2024-04-16T19:33:01.161137+0000","last_fullsized":"2024-04-16T19:33:01.161137+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:53.163721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167775+0000","last_change":"2024-04-16T17:47:59.176369+0000","last_active":"2024-04-16T19:33:01.167775+0000","last_peered":"2024-04-16T19:33:01.167775+0000","last_clean":"2024-04-16T19:33:01.167775+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:33:01.167775+0000","last_undegraded":"2024-04-16T19:33:01.167775+0000","last_fullsized":"2024-04-16T19:33:01.167775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:52.164568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'592","reported_seq":2279,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162562+0000","last_change":"2024-04-16T17:47:57.185970+0000","last_active":"2024-04-16T19:33:01.162562+0000","last_peered":"2024-04-16T19:33:01.162562+0000","last_clean":"2024-04-16T19:33:01.162562+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:33:01.162562+0000","last_undegraded":"2024-04-16T19:33:01.162562+0000","last_fullsized":"2024-04-16T19:33:01.162562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:33.418851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"51'2","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151881+0000","last_change":"2024-04-16T17:47:55.163430+0000","last_active":"2024-04-16T19:32:06.151881+0000","last_peered":"2024-04-16T19:32:06.151881+0000","last_clean":"2024-04-16T19:32:06.151881+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:32:06.151881+0000","last_undegraded":"2024-04-16T19:32:06.151881+0000","last_fullsized":"2024-04-16T19:32:06.151881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183840+0000","last_change":"2024-04-16T17:47:39.338758+0000","last_active":"2024-04-16T19:34:11.183840+0000","last_peered":"2024-04-16T19:34:11.183840+0000","last_clean":"2024-04-16T19:34:11.183840+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:34:11.183840+0000","last_undegraded":"2024-04-16T19:34:11.183840+0000","last_fullsized":"2024-04-16T19:34:11.183840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156609+0000","last_change":"2024-04-16T17:47:23.260667+0000","last_active":"2024-04-16T19:32:26.156609+0000","last_peered":"2024-04-16T19:32:26.156609+0000","last_clean":"2024-04-16T19:32:26.156609+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:32:26.156609+0000","last_undegraded":"2024-04-16T19:32:26.156609+0000","last_fullsized":"2024-04-16T19:32:26.156609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177933+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:33:41.177933+0000","last_peered":"2024-04-16T19:33:41.177933+0000","last_clean":"2024-04-16T19:33:41.177933+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:33:41.177933+0000","last_undegraded":"2024-04-16T19:33:41.177933+0000","last_fullsized":"2024-04-16T19:33:41.177933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"48'1","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159182+0000","last_change":"2024-04-16T17:47:33.297748+0000","last_active":"2024-04-16T19:32:36.159182+0000","last_peered":"2024-04-16T19:32:36.159182+0000","last_clean":"2024-04-16T19:32:36.159182+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:32:36.159182+0000","last_undegraded":"2024-04-16T19:32:36.159182+0000","last_fullsized":"2024-04-16T19:32:36.159182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:36:58.076852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'50","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158106+0000","last_change":"2024-04-16T17:47:35.314682+0000","last_active":"2024-04-16T19:32:36.158106+0000","last_peered":"2024-04-16T19:32:36.158106+0000","last_clean":"2024-04-16T19:32:36.158106+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:32:36.158106+0000","last_undegraded":"2024-04-16T19:32:36.158106+0000","last_fullsized":"2024-04-16T19:32:36.158106+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162296+0000","last_change":"2024-04-16T17:47:37.327586+0000","last_active":"2024-04-16T19:32:41.162296+0000","last_peered":"2024-04-16T19:32:41.162296+0000","last_clean":"2024-04-16T19:32:41.162296+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:32:41.162296+0000","last_undegraded":"2024-04-16T19:32:41.162296+0000","last_fullsized":"2024-04-16T19:32:41.162296+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167526+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:33:21.167526+0000","last_peered":"2024-04-16T19:33:21.167526+0000","last_clean":"2024-04-16T19:33:21.167526+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:33:21.167526+0000","last_undegraded":"2024-04-16T19:33:21.167526+0000","last_fullsized":"2024-04-16T19:33:21.167526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:07.939676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'99","reported_seq":1687,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174223+0000","last_change":"2024-04-16T17:48:34.338854+0000","last_active":"2024-04-16T19:33:36.174223+0000","last_peered":"2024-04-16T19:33:36.174223+0000","last_clean":"2024-04-16T19:33:36.174223+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:33:36.174223+0000","last_undegraded":"2024-04-16T19:33:36.174223+0000","last_fullsized":"2024-04-16T19:33:36.174223+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:41.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153090+0000","last_change":"2024-04-16T17:48:27.301694+0000","last_active":"2024-04-16T19:32:26.153090+0000","last_peered":"2024-04-16T19:32:26.153090+0000","last_clean":"2024-04-16T19:32:26.153090+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:32:26.153090+0000","last_undegraded":"2024-04-16T19:32:26.153090+0000","last_fullsized":"2024-04-16T19:32:26.153090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'13","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156916+0000","last_change":"2024-04-16T17:48:01.248700+0000","last_active":"2024-04-16T19:32:31.156916+0000","last_peered":"2024-04-16T19:32:31.156916+0000","last_clean":"2024-04-16T19:32:31.156916+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:32:31.156916+0000","last_undegraded":"2024-04-16T19:32:31.156916+0000","last_fullsized":"2024-04-16T19:32:31.156916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'20","reported_seq":1421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153482+0000","last_change":"2024-04-16T17:47:59.196995+0000","last_active":"2024-04-16T19:32:26.153482+0000","last_peered":"2024-04-16T19:32:26.153482+0000","last_clean":"2024-04-16T19:32:26.153482+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:32:26.153482+0000","last_undegraded":"2024-04-16T19:32:26.153482+0000","last_fullsized":"2024-04-16T19:32:26.153482+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.685479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'6983","reported_seq":10557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154510+0000","last_change":"2024-04-16T17:47:57.195602+0000","last_active":"2024-04-16T19:32:11.154510+0000","last_peered":"2024-04-16T19:32:11.154510+0000","last_clean":"2024-04-16T19:32:11.154510+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:32:11.154510+0000","last_undegraded":"2024-04-16T19:32:11.154510+0000","last_fullsized":"2024-04-16T19:32:11.154510+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1783,"log_dups_size":3000,"ondisk_log_size":1783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:48:37.455011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'6","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163653+0000","last_change":"2024-04-16T17:47:55.162215+0000","last_active":"2024-04-16T19:32:56.163653+0000","last_peered":"2024-04-16T19:32:56.163653+0000","last_clean":"2024-04-16T19:32:56.163653+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:32:56.163653+0000","last_undegraded":"2024-04-16T19:32:56.163653+0000","last_fullsized":"2024-04-16T19:32:56.163653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159959+0000","last_change":"2024-04-16T17:47:39.329121+0000","last_active":"2024-04-16T19:32:41.159959+0000","last_peered":"2024-04-16T19:32:41.159959+0000","last_clean":"2024-04-16T19:32:41.159959+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:32:41.159959+0000","last_undegraded":"2024-04-16T19:32:41.159959+0000","last_fullsized":"2024-04-16T19:32:41.159959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139757+0000","last_change":"2024-04-16T17:47:23.269620+0000","last_active":"2024-04-16T19:32:26.139757+0000","last_peered":"2024-04-16T19:32:26.139757+0000","last_clean":"2024-04-16T19:32:26.139757+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:32:26.139757+0000","last_undegraded":"2024-04-16T19:32:26.139757+0000","last_fullsized":"2024-04-16T19:32:26.139757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:43.201589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172841+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:33:41.172841+0000","last_peered":"2024-04-16T19:33:41.172841+0000","last_clean":"2024-04-16T19:33:41.172841+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:33:41.172841+0000","last_undegraded":"2024-04-16T19:33:41.172841+0000","last_fullsized":"2024-04-16T19:33:41.172841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155004+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:32:36.155004+0000","last_peered":"2024-04-16T19:32:36.155004+0000","last_clean":"2024-04-16T19:32:36.155004+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:32:36.155004+0000","last_undegraded":"2024-04-16T19:32:36.155004+0000","last_fullsized":"2024-04-16T19:32:36.155004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:14.585091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142192+0000","last_change":"2024-04-16T17:47:35.310781+0000","last_active":"2024-04-16T19:32:36.142192+0000","last_peered":"2024-04-16T19:32:36.142192+0000","last_clean":"2024-04-16T19:32:36.142192+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:32:36.142192+0000","last_undegraded":"2024-04-16T19:32:36.142192+0000","last_fullsized":"2024-04-16T19:32:36.142192+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:20.569781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157441+0000","last_change":"2024-04-16T17:47:37.322773+0000","last_active":"2024-04-16T19:32:41.157441+0000","last_peered":"2024-04-16T19:32:41.157441+0000","last_clean":"2024-04-16T19:32:41.157441+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:32:41.157441+0000","last_undegraded":"2024-04-16T19:32:41.157441+0000","last_fullsized":"2024-04-16T19:32:41.157441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172568+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:33:21.172568+0000","last_peered":"2024-04-16T19:33:21.172568+0000","last_clean":"2024-04-16T19:33:21.172568+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:33:21.172568+0000","last_undegraded":"2024-04-16T19:33:21.172568+0000","last_fullsized":"2024-04-16T19:33:21.172568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147896+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:32:06.147896+0000","last_peered":"2024-04-16T19:32:06.147896+0000","last_clean":"2024-04-16T19:32:06.147896+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:32:06.147896+0000","last_undegraded":"2024-04-16T19:32:06.147896+0000","last_fullsized":"2024-04-16T19:32:06.147896+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:46:24.982845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146589+0000","last_change":"2024-04-16T17:48:27.293279+0000","last_active":"2024-04-16T19:32:56.146589+0000","last_peered":"2024-04-16T19:32:56.146589+0000","last_clean":"2024-04-16T19:32:56.146589+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:32:56.146589+0000","last_undegraded":"2024-04-16T19:32:56.146589+0000","last_fullsized":"2024-04-16T19:32:56.146589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:44.442181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"48'8","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148529+0000","last_change":"2024-04-16T17:48:01.237052+0000","last_active":"2024-04-16T19:31:56.148529+0000","last_peered":"2024-04-16T19:31:56.148529+0000","last_clean":"2024-04-16T19:31:56.148529+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:31:56.148529+0000","last_undegraded":"2024-04-16T19:31:56.148529+0000","last_fullsized":"2024-04-16T19:31:56.148529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:22.523578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'555","reported_seq":1956,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136598+0000","last_change":"2024-04-16T17:47:59.187458+0000","last_active":"2024-04-16T19:32:11.136598+0000","last_peered":"2024-04-16T19:32:11.136598+0000","last_clean":"2024-04-16T19:32:11.136598+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:32:11.136598+0000","last_undegraded":"2024-04-16T19:32:11.136598+0000","last_fullsized":"2024-04-16T19:32:11.136598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:17.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'544","reported_seq":775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158452+0000","last_change":"2024-04-16T17:47:57.184810+0000","last_active":"2024-04-16T19:32:36.158452+0000","last_peered":"2024-04-16T19:32:36.158452+0000","last_clean":"2024-04-16T19:32:36.158452+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:32:36.158452+0000","last_undegraded":"2024-04-16T19:32:36.158452+0000","last_fullsized":"2024-04-16T19:32:36.158452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"51'4","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163711+0000","last_change":"2024-04-16T17:47:55.183376+0000","last_active":"2024-04-16T19:32:56.163711+0000","last_peered":"2024-04-16T19:32:56.163711+0000","last_clean":"2024-04-16T19:32:56.163711+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:32:56.163711+0000","last_undegraded":"2024-04-16T19:32:56.163711+0000","last_fullsized":"2024-04-16T19:32:56.163711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157536+0000","last_change":"2024-04-16T17:47:39.339617+0000","last_active":"2024-04-16T19:32:41.157536+0000","last_peered":"2024-04-16T19:32:41.157536+0000","last_clean":"2024-04-16T19:32:41.157536+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:32:41.157536+0000","last_undegraded":"2024-04-16T19:32:41.157536+0000","last_fullsized":"2024-04-16T19:32:41.157536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156357+0000","last_change":"2024-04-16T17:47:23.263370+0000","last_active":"2024-04-16T19:32:26.156357+0000","last_peered":"2024-04-16T19:32:26.156357+0000","last_clean":"2024-04-16T19:32:26.156357+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:32:26.156357+0000","last_undegraded":"2024-04-16T19:32:26.156357+0000","last_fullsized":"2024-04-16T19:32:26.156357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174981+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:33:41.174981+0000","last_peered":"2024-04-16T19:33:41.174981+0000","last_clean":"2024-04-16T19:33:41.174981+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:33:41.174981+0000","last_undegraded":"2024-04-16T19:33:41.174981+0000","last_fullsized":"2024-04-16T19:33:41.174981+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160793+0000","last_change":"2024-04-16T17:47:33.307304+0000","last_active":"2024-04-16T19:32:36.160793+0000","last_peered":"2024-04-16T19:32:36.160793+0000","last_clean":"2024-04-16T19:32:36.160793+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:32:36.160793+0000","last_undegraded":"2024-04-16T19:32:36.160793+0000","last_fullsized":"2024-04-16T19:32:36.160793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156191+0000","last_change":"2024-04-16T17:47:35.311451+0000","last_active":"2024-04-16T19:32:36.156191+0000","last_peered":"2024-04-16T19:32:36.156191+0000","last_clean":"2024-04-16T19:32:36.156191+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:32:36.156191+0000","last_undegraded":"2024-04-16T19:32:36.156191+0000","last_fullsized":"2024-04-16T19:32:36.156191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159764+0000","last_change":"2024-04-16T17:47:37.326333+0000","last_active":"2024-04-16T19:32:41.159764+0000","last_peered":"2024-04-16T19:32:41.159764+0000","last_clean":"2024-04-16T19:32:41.159764+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:32:41.159764+0000","last_undegraded":"2024-04-16T19:32:41.159764+0000","last_fullsized":"2024-04-16T19:32:41.159764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167717+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:33:01.167717+0000","last_peered":"2024-04-16T19:33:01.167717+0000","last_clean":"2024-04-16T19:33:01.167717+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:33:01.167717+0000","last_undegraded":"2024-04-16T19:33:01.167717+0000","last_fullsized":"2024-04-16T19:33:01.167717+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'89","reported_seq":694,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159707+0000","last_change":"2024-04-16T17:48:34.328117+0000","last_active":"2024-04-16T19:32:31.159707+0000","last_peered":"2024-04-16T19:32:31.159707+0000","last_clean":"2024-04-16T19:32:31.159707+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:32:31.159707+0000","last_undegraded":"2024-04-16T19:32:31.159707+0000","last_fullsized":"2024-04-16T19:32:31.159707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155893+0000","last_change":"2024-04-16T17:48:27.295608+0000","last_active":"2024-04-16T19:32:26.155893+0000","last_peered":"2024-04-16T19:32:26.155893+0000","last_clean":"2024-04-16T19:32:26.155893+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:32:26.155893+0000","last_undegraded":"2024-04-16T19:32:26.155893+0000","last_fullsized":"2024-04-16T19:32:26.155893+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'12","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145278+0000","last_change":"2024-04-16T17:48:01.246592+0000","last_active":"2024-04-16T19:31:56.145278+0000","last_peered":"2024-04-16T19:31:56.145278+0000","last_clean":"2024-04-16T19:31:56.145278+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:31:56.145278+0000","last_undegraded":"2024-04-16T19:31:56.145278+0000","last_fullsized":"2024-04-16T19:31:56.145278+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:06.492939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164653+0000","last_change":"2024-04-16T17:47:59.178449+0000","last_active":"2024-04-16T19:33:01.164653+0000","last_peered":"2024-04-16T19:33:01.164653+0000","last_clean":"2024-04-16T19:33:01.164653+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:33:01.164653+0000","last_undegraded":"2024-04-16T19:33:01.164653+0000","last_fullsized":"2024-04-16T19:33:01.164653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'457","reported_seq":4410,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162217+0000","last_change":"2024-04-16T17:47:57.181693+0000","last_active":"2024-04-16T19:33:01.162217+0000","last_peered":"2024-04-16T19:33:01.162217+0000","last_clean":"2024-04-16T19:33:01.162217+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:33:01.162217+0000","last_undegraded":"2024-04-16T19:33:01.162217+0000","last_fullsized":"2024-04-16T19:33:01.162217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:44.694025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'6","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161031+0000","last_change":"2024-04-16T17:47:55.174874+0000","last_active":"2024-04-16T19:32:56.161031+0000","last_peered":"2024-04-16T19:32:56.161031+0000","last_clean":"2024-04-16T19:32:56.161031+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:32:56.161031+0000","last_undegraded":"2024-04-16T19:32:56.161031+0000","last_fullsized":"2024-04-16T19:32:56.161031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178208+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:33:41.178208+0000","last_peered":"2024-04-16T19:33:41.178208+0000","last_clean":"2024-04-16T19:33:41.178208+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:33:41.178208+0000","last_undegraded":"2024-04-16T19:33:41.178208+0000","last_fullsized":"2024-04-16T19:33:41.178208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:37.592633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157093+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:33:41.157093+0000","last_peered":"2024-04-16T19:33:41.157093+0000","last_clean":"2024-04-16T19:33:41.157093+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:33:41.157093+0000","last_undegraded":"2024-04-16T19:33:41.157093+0000","last_fullsized":"2024-04-16T19:33:41.157093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:08.399471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182918+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:34:11.182918+0000","last_peered":"2024-04-16T19:34:11.182918+0000","last_clean":"2024-04-16T19:34:11.182918+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:34:11.182918+0000","last_undegraded":"2024-04-16T19:34:11.182918+0000","last_fullsized":"2024-04-16T19:34:11.182918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'88","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159309+0000","last_change":"2024-04-16T17:47:35.319036+0000","last_active":"2024-04-16T19:32:36.159309+0000","last_peered":"2024-04-16T19:32:36.159309+0000","last_clean":"2024-04-16T19:32:36.159309+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:32:36.159309+0000","last_undegraded":"2024-04-16T19:32:36.159309+0000","last_fullsized":"2024-04-16T19:32:36.159309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162487+0000","last_change":"2024-04-16T17:47:37.330847+0000","last_active":"2024-04-16T19:32:41.162487+0000","last_peered":"2024-04-16T19:32:41.162487+0000","last_clean":"2024-04-16T19:32:41.162487+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:32:41.162487+0000","last_undegraded":"2024-04-16T19:32:41.162487+0000","last_fullsized":"2024-04-16T19:32:41.162487+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:10.138006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160442+0000","last_change":"2024-04-16T17:47:39.701654+0000","last_active":"2024-04-16T19:32:41.160442+0000","last_peered":"2024-04-16T19:32:41.160442+0000","last_clean":"2024-04-16T19:32:41.160442+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:32:41.160442+0000","last_undegraded":"2024-04-16T19:32:41.160442+0000","last_fullsized":"2024-04-16T19:32:41.160442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162629+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:33:01.162629+0000","last_peered":"2024-04-16T19:33:01.162629+0000","last_clean":"2024-04-16T19:33:01.162629+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:33:01.162629+0000","last_undegraded":"2024-04-16T19:33:01.162629+0000","last_fullsized":"2024-04-16T19:33:01.162629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:28:06.366066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'71","reported_seq":966,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143796+0000","last_change":"2024-04-16T17:48:34.313054+0000","last_active":"2024-04-16T19:31:51.143796+0000","last_peered":"2024-04-16T19:31:51.143796+0000","last_clean":"2024-04-16T19:31:51.143796+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:31:51.143796+0000","last_undegraded":"2024-04-16T19:31:51.143796+0000","last_fullsized":"2024-04-16T19:31:51.143796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:11.966580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153539+0000","last_change":"2024-04-16T17:48:27.309014+0000","last_active":"2024-04-16T19:32:26.153539+0000","last_peered":"2024-04-16T19:32:26.153539+0000","last_clean":"2024-04-16T19:32:26.153539+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:32:26.153539+0000","last_undegraded":"2024-04-16T19:32:26.153539+0000","last_fullsized":"2024-04-16T19:32:26.153539+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'4","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145295+0000","last_change":"2024-04-16T17:48:01.251229+0000","last_active":"2024-04-16T19:31:56.145295+0000","last_peered":"2024-04-16T19:31:56.145295+0000","last_clean":"2024-04-16T19:31:56.145295+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:31:56.145295+0000","last_undegraded":"2024-04-16T19:31:56.145295+0000","last_fullsized":"2024-04-16T19:31:56.145295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147763+0000","last_change":"2024-04-16T17:47:59.174249+0000","last_active":"2024-04-16T19:33:01.147763+0000","last_peered":"2024-04-16T19:33:01.147763+0000","last_clean":"2024-04-16T19:33:01.147763+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:33:01.147763+0000","last_undegraded":"2024-04-16T19:33:01.147763+0000","last_fullsized":"2024-04-16T19:33:01.147763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:54.272783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11370","reported_seq":61791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161030+0000","last_change":"2024-04-16T17:47:57.192920+0000","last_active":"2024-04-16T19:33:01.161030+0000","last_peered":"2024-04-16T19:33:01.161030+0000","last_clean":"2024-04-16T19:33:01.161030+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:33:01.161030+0000","last_undegraded":"2024-04-16T19:33:01.161030+0000","last_fullsized":"2024-04-16T19:33:01.161030+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1670,"log_dups_size":3000,"ondisk_log_size":1670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:13.718911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'12","reported_seq":338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166321+0000","last_change":"2024-04-16T17:47:55.178367+0000","last_active":"2024-04-16T19:32:56.166321+0000","last_peered":"2024-04-16T19:32:56.166321+0000","last_clean":"2024-04-16T19:32:56.166321+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:32:56.166321+0000","last_undegraded":"2024-04-16T19:32:56.166321+0000","last_fullsized":"2024-04-16T19:32:56.166321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.180159+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:34:01.180159+0000","last_peered":"2024-04-16T19:34:01.180159+0000","last_clean":"2024-04-16T19:34:01.180159+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:34:01.180159+0000","last_undegraded":"2024-04-16T19:34:01.180159+0000","last_fullsized":"2024-04-16T19:34:01.180159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145890+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:31:46.145890+0000","last_peered":"2024-04-16T19:31:46.145890+0000","last_clean":"2024-04-16T19:31:46.145890+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:31:46.145890+0000","last_undegraded":"2024-04-16T19:31:46.145890+0000","last_fullsized":"2024-04-16T19:31:46.145890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181077+0000","last_change":"2024-04-16T17:47:33.296653+0000","last_active":"2024-04-16T19:33:51.181077+0000","last_peered":"2024-04-16T19:33:51.181077+0000","last_clean":"2024-04-16T19:33:51.181077+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:33:51.181077+0000","last_undegraded":"2024-04-16T19:33:51.181077+0000","last_fullsized":"2024-04-16T19:33:51.181077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:00.330068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'152","reported_seq":340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160930+0000","last_change":"2024-04-16T17:47:35.317123+0000","last_active":"2024-04-16T19:32:36.160930+0000","last_peered":"2024-04-16T19:32:36.160930+0000","last_clean":"2024-04-16T19:32:36.160930+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:32:36.160930+0000","last_undegraded":"2024-04-16T19:32:36.160930+0000","last_fullsized":"2024-04-16T19:32:36.160930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160571+0000","last_change":"2024-04-16T17:47:37.334019+0000","last_active":"2024-04-16T19:32:41.160571+0000","last_peered":"2024-04-16T19:32:41.160571+0000","last_clean":"2024-04-16T19:32:41.160571+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:32:41.160571+0000","last_undegraded":"2024-04-16T19:32:41.160571+0000","last_fullsized":"2024-04-16T19:32:41.160571+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:07.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162425+0000","last_change":"2024-04-16T17:47:39.699556+0000","last_active":"2024-04-16T19:32:41.162425+0000","last_peered":"2024-04-16T19:32:41.162425+0000","last_clean":"2024-04-16T19:32:41.162425+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:32:41.162425+0000","last_undegraded":"2024-04-16T19:32:41.162425+0000","last_fullsized":"2024-04-16T19:32:41.162425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165020+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:33:01.165020+0000","last_peered":"2024-04-16T19:33:01.165020+0000","last_clean":"2024-04-16T19:33:01.165020+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:33:01.165020+0000","last_undegraded":"2024-04-16T19:33:01.165020+0000","last_fullsized":"2024-04-16T19:33:01.165020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'103","reported_seq":1819,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158185+0000","last_change":"2024-04-16T17:48:34.337561+0000","last_active":"2024-04-16T19:32:31.158185+0000","last_peered":"2024-04-16T19:32:31.158185+0000","last_clean":"2024-04-16T19:32:31.158185+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:32:31.158185+0000","last_undegraded":"2024-04-16T19:32:31.158185+0000","last_fullsized":"2024-04-16T19:32:31.158185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:01.320649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145725+0000","last_change":"2024-04-16T17:48:27.293905+0000","last_active":"2024-04-16T19:31:46.145725+0000","last_peered":"2024-04-16T19:31:46.145725+0000","last_clean":"2024-04-16T19:31:46.145725+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:31:46.145725+0000","last_undegraded":"2024-04-16T19:31:46.145725+0000","last_fullsized":"2024-04-16T19:31:46.145725+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'8","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145241+0000","last_change":"2024-04-16T17:48:01.248531+0000","last_active":"2024-04-16T19:31:56.145241+0000","last_peered":"2024-04-16T19:31:56.145241+0000","last_clean":"2024-04-16T19:31:56.145241+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:31:56.145241+0000","last_undegraded":"2024-04-16T19:31:56.145241+0000","last_fullsized":"2024-04-16T19:31:56.145241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167662+0000","last_change":"2024-04-16T17:47:59.187755+0000","last_active":"2024-04-16T19:33:01.167662+0000","last_peered":"2024-04-16T19:33:01.167662+0000","last_clean":"2024-04-16T19:33:01.167662+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:33:01.167662+0000","last_undegraded":"2024-04-16T19:33:01.167662+0000","last_fullsized":"2024-04-16T19:33:01.167662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:25.460755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'196","reported_seq":421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165564+0000","last_change":"2024-04-16T17:47:57.195138+0000","last_active":"2024-04-16T19:33:01.165564+0000","last_peered":"2024-04-16T19:33:01.165564+0000","last_clean":"2024-04-16T19:33:01.165564+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:33:01.165564+0000","last_undegraded":"2024-04-16T19:33:01.165564+0000","last_fullsized":"2024-04-16T19:33:01.165564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:36.309438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'12","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159825+0000","last_change":"2024-04-16T17:47:55.160472+0000","last_active":"2024-04-16T19:32:56.159825+0000","last_peered":"2024-04-16T19:32:56.159825+0000","last_clean":"2024-04-16T19:32:56.159825+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:32:56.159825+0000","last_undegraded":"2024-04-16T19:32:56.159825+0000","last_fullsized":"2024-04-16T19:32:56.159825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:00.875301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155799+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:32:26.155799+0000","last_peered":"2024-04-16T19:32:26.155799+0000","last_clean":"2024-04-16T19:32:26.155799+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:32:26.155799+0000","last_undegraded":"2024-04-16T19:32:26.155799+0000","last_fullsized":"2024-04-16T19:32:26.155799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151302+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:32:21.151302+0000","last_peered":"2024-04-16T19:32:21.151302+0000","last_clean":"2024-04-16T19:32:21.151302+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:32:21.151302+0000","last_undegraded":"2024-04-16T19:32:21.151302+0000","last_fullsized":"2024-04-16T19:32:21.151302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:44.777444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"48'5","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155232+0000","last_change":"2024-04-16T17:47:33.306460+0000","last_active":"2024-04-16T19:32:36.155232+0000","last_peered":"2024-04-16T19:32:36.155232+0000","last_clean":"2024-04-16T19:32:36.155232+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:32:36.155232+0000","last_undegraded":"2024-04-16T19:32:36.155232+0000","last_fullsized":"2024-04-16T19:32:36.155232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:15.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'104","reported_seq":277,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160070+0000","last_change":"2024-04-16T17:47:35.314274+0000","last_active":"2024-04-16T19:33:51.160070+0000","last_peered":"2024-04-16T19:33:51.160070+0000","last_clean":"2024-04-16T19:33:51.160070+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:33:51.160070+0000","last_undegraded":"2024-04-16T19:33:51.160070+0000","last_fullsized":"2024-04-16T19:33:51.160070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:58.087539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183865+0000","last_change":"2024-04-16T17:47:37.331871+0000","last_active":"2024-04-16T19:34:11.183865+0000","last_peered":"2024-04-16T19:34:11.183865+0000","last_clean":"2024-04-16T19:34:11.183865+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:34:11.183865+0000","last_undegraded":"2024-04-16T19:34:11.183865+0000","last_fullsized":"2024-04-16T19:34:11.183865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:10.644125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162350+0000","last_change":"2024-04-16T17:47:39.323414+0000","last_active":"2024-04-16T19:32:41.162350+0000","last_peered":"2024-04-16T19:32:41.162350+0000","last_clean":"2024-04-16T19:32:41.162350+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:32:41.162350+0000","last_undegraded":"2024-04-16T19:32:41.162350+0000","last_fullsized":"2024-04-16T19:32:41.162350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177321+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:33:46.177321+0000","last_peered":"2024-04-16T19:33:46.177321+0000","last_clean":"2024-04-16T19:33:46.177321+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:33:46.177321+0000","last_undegraded":"2024-04-16T19:33:46.177321+0000","last_fullsized":"2024-04-16T19:33:46.177321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:41.778594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'68","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158123+0000","last_change":"2024-04-16T17:48:34.343716+0000","last_active":"2024-04-16T19:32:31.158123+0000","last_peered":"2024-04-16T19:32:31.158123+0000","last_clean":"2024-04-16T19:32:31.158123+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:32:31.158123+0000","last_undegraded":"2024-04-16T19:32:31.158123+0000","last_fullsized":"2024-04-16T19:32:31.158123+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:48.210902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152776+0000","last_change":"2024-04-16T17:48:27.305044+0000","last_active":"2024-04-16T19:32:26.152776+0000","last_peered":"2024-04-16T19:32:26.152776+0000","last_clean":"2024-04-16T19:32:26.152776+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:32:26.152776+0000","last_undegraded":"2024-04-16T19:32:26.152776+0000","last_fullsized":"2024-04-16T19:32:26.152776+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:17.183220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"48'1","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168780+0000","last_change":"2024-04-16T17:48:01.254059+0000","last_active":"2024-04-16T19:33:26.168780+0000","last_peered":"2024-04-16T19:33:26.168780+0000","last_clean":"2024-04-16T19:33:26.168780+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:33:26.168780+0000","last_undegraded":"2024-04-16T19:33:26.168780+0000","last_fullsized":"2024-04-16T19:33:26.168780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162464+0000","last_change":"2024-04-16T17:47:59.195638+0000","last_active":"2024-04-16T19:33:01.162464+0000","last_peered":"2024-04-16T19:33:01.162464+0000","last_clean":"2024-04-16T19:33:01.162464+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:33:01.162464+0000","last_undegraded":"2024-04-16T19:33:01.162464+0000","last_fullsized":"2024-04-16T19:33:01.162464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:57.426064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'488","reported_seq":3937,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162374+0000","last_change":"2024-04-16T17:47:57.174503+0000","last_active":"2024-04-16T19:33:01.162374+0000","last_peered":"2024-04-16T19:33:01.162374+0000","last_clean":"2024-04-16T19:33:01.162374+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:33:01.162374+0000","last_undegraded":"2024-04-16T19:33:01.162374+0000","last_fullsized":"2024-04-16T19:33:01.162374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:34.199358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'27","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166245+0000","last_change":"2024-04-16T17:47:55.169780+0000","last_active":"2024-04-16T19:32:56.166245+0000","last_peered":"2024-04-16T19:32:56.166245+0000","last_clean":"2024-04-16T19:32:56.166245+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:32:56.166245+0000","last_undegraded":"2024-04-16T19:32:56.166245+0000","last_fullsized":"2024-04-16T19:32:56.166245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139882+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:32:26.139882+0000","last_peered":"2024-04-16T19:32:26.139882+0000","last_clean":"2024-04-16T19:32:26.139882+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:32:26.139882+0000","last_undegraded":"2024-04-16T19:32:26.139882+0000","last_fullsized":"2024-04-16T19:32:26.139882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:43:58.984431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154419+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:32:21.154419+0000","last_peered":"2024-04-16T19:32:21.154419+0000","last_clean":"2024-04-16T19:32:21.154419+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:32:21.154419+0000","last_undegraded":"2024-04-16T19:32:21.154419+0000","last_fullsized":"2024-04-16T19:32:21.154419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"33'3","reported_seq":802,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159282+0000","last_change":"2024-04-16T17:47:33.297794+0000","last_active":"2024-04-16T19:32:36.159282+0000","last_peered":"2024-04-16T19:32:36.159282+0000","last_clean":"2024-04-16T19:32:36.159282+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:32:36.159282+0000","last_undegraded":"2024-04-16T19:32:36.159282+0000","last_fullsized":"2024-04-16T19:32:36.159282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'120","reported_seq":298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186858+0000","last_change":"2024-04-16T17:47:35.319657+0000","last_active":"2024-04-16T19:34:11.186858+0000","last_peered":"2024-04-16T19:34:11.186858+0000","last_clean":"2024-04-16T19:34:11.186858+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:34:11.186858+0000","last_undegraded":"2024-04-16T19:34:11.186858+0000","last_fullsized":"2024-04-16T19:34:11.186858+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:08.628044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.177090+0000","last_change":"2024-04-16T17:47:37.331215+0000","last_active":"2024-04-16T19:33:56.177090+0000","last_peered":"2024-04-16T19:33:56.177090+0000","last_clean":"2024-04-16T19:33:56.177090+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:33:56.177090+0000","last_undegraded":"2024-04-16T19:33:56.177090+0000","last_fullsized":"2024-04-16T19:33:56.177090+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162223+0000","last_change":"2024-04-16T17:47:39.332386+0000","last_active":"2024-04-16T19:32:41.162223+0000","last_peered":"2024-04-16T19:32:41.162223+0000","last_clean":"2024-04-16T19:32:41.162223+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:32:41.162223+0000","last_undegraded":"2024-04-16T19:32:41.162223+0000","last_fullsized":"2024-04-16T19:32:41.162223+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.153850+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:32:31.153850+0000","last_peered":"2024-04-16T19:32:31.153850+0000","last_clean":"2024-04-16T19:32:31.153850+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:32:31.153850+0000","last_undegraded":"2024-04-16T19:32:31.153850+0000","last_fullsized":"2024-04-16T19:32:31.153850+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:59:18.189132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'86","reported_seq":1557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169733+0000","last_change":"2024-04-16T17:48:34.318628+0000","last_active":"2024-04-16T19:33:36.169733+0000","last_peered":"2024-04-16T19:33:36.169733+0000","last_clean":"2024-04-16T19:33:36.169733+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:33:36.169733+0000","last_undegraded":"2024-04-16T19:33:36.169733+0000","last_fullsized":"2024-04-16T19:33:36.169733+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:15:00.721377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172016+0000","last_change":"2024-04-16T17:48:27.299691+0000","last_active":"2024-04-16T19:33:31.172016+0000","last_peered":"2024-04-16T19:33:31.172016+0000","last_clean":"2024-04-16T19:33:31.172016+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:33:31.172016+0000","last_undegraded":"2024-04-16T19:33:31.172016+0000","last_fullsized":"2024-04-16T19:33:31.172016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:40:17.511736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'6","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165634+0000","last_change":"2024-04-16T17:48:01.246789+0000","last_active":"2024-04-16T19:33:01.165634+0000","last_peered":"2024-04-16T19:33:01.165634+0000","last_clean":"2024-04-16T19:33:01.165634+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:33:01.165634+0000","last_undegraded":"2024-04-16T19:33:01.165634+0000","last_fullsized":"2024-04-16T19:33:01.165634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:47.722149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'555","reported_seq":1953,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176329+0000","last_change":"2024-04-16T17:47:59.182365+0000","last_active":"2024-04-16T19:33:36.176329+0000","last_peered":"2024-04-16T19:33:36.176329+0000","last_clean":"2024-04-16T19:33:36.176329+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:33:36.176329+0000","last_undegraded":"2024-04-16T19:33:36.176329+0000","last_fullsized":"2024-04-16T19:33:36.176329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'492","reported_seq":710,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148336+0000","last_change":"2024-04-16T17:47:57.180106+0000","last_active":"2024-04-16T19:31:56.148336+0000","last_peered":"2024-04-16T19:31:56.148336+0000","last_clean":"2024-04-16T19:31:56.148336+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:31:56.148336+0000","last_undegraded":"2024-04-16T19:31:56.148336+0000","last_fullsized":"2024-04-16T19:31:56.148336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'4","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144008+0000","last_change":"2024-04-16T17:47:55.169894+0000","last_active":"2024-04-16T19:31:51.144008+0000","last_peered":"2024-04-16T19:31:51.144008+0000","last_clean":"2024-04-16T19:31:51.144008+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:31:51.144008+0000","last_undegraded":"2024-04-16T19:31:51.144008+0000","last_fullsized":"2024-04-16T19:31:51.144008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:24.954941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152629+0000","last_change":"2024-04-16T17:47:23.266570+0000","last_active":"2024-04-16T19:32:26.152629+0000","last_peered":"2024-04-16T19:32:26.152629+0000","last_clean":"2024-04-16T19:32:26.152629+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:32:26.152629+0000","last_undegraded":"2024-04-16T19:32:26.152629+0000","last_fullsized":"2024-04-16T19:32:26.152629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:55.441583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152031+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:32:21.152031+0000","last_peered":"2024-04-16T19:32:21.152031+0000","last_clean":"2024-04-16T19:32:21.152031+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:32:21.152031+0000","last_undegraded":"2024-04-16T19:32:21.152031+0000","last_fullsized":"2024-04-16T19:32:21.152031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155126+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:32:36.155126+0000","last_peered":"2024-04-16T19:32:36.155126+0000","last_clean":"2024-04-16T19:32:36.155126+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:32:36.155126+0000","last_undegraded":"2024-04-16T19:32:36.155126+0000","last_fullsized":"2024-04-16T19:32:36.155126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:54.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'152","reported_seq":341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145026+0000","last_change":"2024-04-16T17:47:35.310214+0000","last_active":"2024-04-16T19:31:56.145026+0000","last_peered":"2024-04-16T19:31:56.145026+0000","last_clean":"2024-04-16T19:31:56.145026+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:31:56.145026+0000","last_undegraded":"2024-04-16T19:31:56.145026+0000","last_fullsized":"2024-04-16T19:31:56.145026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.142943+0000","last_change":"2024-04-16T17:47:37.328751+0000","last_active":"2024-04-16T19:31:36.142943+0000","last_peered":"2024-04-16T19:31:36.142943+0000","last_clean":"2024-04-16T19:31:36.142943+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:31:36.142943+0000","last_undegraded":"2024-04-16T19:31:36.142943+0000","last_fullsized":"2024-04-16T19:31:36.142943+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:30:39.392016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144646+0000","last_change":"2024-04-16T17:47:39.336993+0000","last_active":"2024-04-16T19:31:36.144646+0000","last_peered":"2024-04-16T19:31:36.144646+0000","last_clean":"2024-04-16T19:31:36.144646+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:31:36.144646+0000","last_undegraded":"2024-04-16T19:31:36.144646+0000","last_fullsized":"2024-04-16T19:31:36.144646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140940+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:32:31.140940+0000","last_peered":"2024-04-16T19:32:31.140940+0000","last_clean":"2024-04-16T19:32:31.140940+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:32:31.140940+0000","last_undegraded":"2024-04-16T19:32:31.140940+0000","last_fullsized":"2024-04-16T19:32:31.140940+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:16.152440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'99","reported_seq":1431,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174016+0000","last_change":"2024-04-16T17:48:34.341651+0000","last_active":"2024-04-16T19:33:36.174016+0000","last_peered":"2024-04-16T19:33:36.174016+0000","last_clean":"2024-04-16T19:33:36.174016+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:33:36.174016+0000","last_undegraded":"2024-04-16T19:33:36.174016+0000","last_fullsized":"2024-04-16T19:33:36.174016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:45.319846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169918+0000","last_change":"2024-04-16T17:48:27.306641+0000","last_active":"2024-04-16T19:33:31.169918+0000","last_peered":"2024-04-16T19:33:31.169918+0000","last_clean":"2024-04-16T19:33:31.169918+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:33:31.169918+0000","last_undegraded":"2024-04-16T19:33:31.169918+0000","last_fullsized":"2024-04-16T19:33:31.169918+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'7","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164768+0000","last_change":"2024-04-16T17:48:01.263221+0000","last_active":"2024-04-16T19:33:01.164768+0000","last_peered":"2024-04-16T19:33:01.164768+0000","last_clean":"2024-04-16T19:33:01.164768+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:33:01.164768+0000","last_undegraded":"2024-04-16T19:33:01.164768+0000","last_fullsized":"2024-04-16T19:33:01.164768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'555","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132909+0000","last_change":"2024-04-16T17:47:59.183413+0000","last_active":"2024-04-16T19:31:56.132909+0000","last_peered":"2024-04-16T19:31:56.132909+0000","last_clean":"2024-04-16T19:31:56.132909+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:31:56.132909+0000","last_undegraded":"2024-04-16T19:31:56.132909+0000","last_fullsized":"2024-04-16T19:31:56.132909+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:27.860005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'230","reported_seq":460,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172962+0000","last_change":"2024-04-16T17:47:57.173077+0000","last_active":"2024-04-16T19:33:31.172962+0000","last_peered":"2024-04-16T19:33:31.172962+0000","last_clean":"2024-04-16T19:33:31.172962+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:33:31.172962+0000","last_undegraded":"2024-04-16T19:33:31.172962+0000","last_fullsized":"2024-04-16T19:33:31.172962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:49.105681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168703+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:33:31.168703+0000","last_peered":"2024-04-16T19:33:31.168703+0000","last_clean":"2024-04-16T19:33:31.168703+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:33:31.168703+0000","last_undegraded":"2024-04-16T19:33:31.168703+0000","last_fullsized":"2024-04-16T19:33:31.168703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:11.770314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152568+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:32:26.152568+0000","last_peered":"2024-04-16T19:32:26.152568+0000","last_clean":"2024-04-16T19:32:26.152568+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:32:26.152568+0000","last_undegraded":"2024-04-16T19:32:26.152568+0000","last_fullsized":"2024-04-16T19:32:26.152568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:32.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152102+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:32:21.152102+0000","last_peered":"2024-04-16T19:32:21.152102+0000","last_clean":"2024-04-16T19:32:21.152102+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:32:21.152102+0000","last_undegraded":"2024-04-16T19:32:21.152102+0000","last_fullsized":"2024-04-16T19:32:21.152102+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156238+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:32:36.156238+0000","last_peered":"2024-04-16T19:32:36.156238+0000","last_clean":"2024-04-16T19:32:36.156238+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:32:36.156238+0000","last_undegraded":"2024-04-16T19:32:36.156238+0000","last_fullsized":"2024-04-16T19:32:36.156238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367224+0000","last_change":"2024-04-16T17:47:35.303292+0000","last_active":"2024-04-16T19:31:31.367224+0000","last_peered":"2024-04-16T19:31:31.367224+0000","last_clean":"2024-04-16T19:31:31.367224+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:31:31.367224+0000","last_undegraded":"2024-04-16T19:31:31.367224+0000","last_fullsized":"2024-04-16T19:31:31.367224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154187+0000","last_change":"2024-04-16T17:47:37.331029+0000","last_active":"2024-04-16T19:32:11.154187+0000","last_peered":"2024-04-16T19:32:11.154187+0000","last_clean":"2024-04-16T19:32:11.154187+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:32:11.154187+0000","last_undegraded":"2024-04-16T19:32:11.154187+0000","last_fullsized":"2024-04-16T19:32:11.154187+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.143665+0000","last_change":"2024-04-16T17:47:39.700695+0000","last_active":"2024-04-16T19:31:36.143665+0000","last_peered":"2024-04-16T19:31:36.143665+0000","last_clean":"2024-04-16T19:31:36.143665+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:31:36.143665+0000","last_undegraded":"2024-04-16T19:31:36.143665+0000","last_fullsized":"2024-04-16T19:31:36.143665+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157016+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:32:31.157016+0000","last_peered":"2024-04-16T19:32:31.157016+0000","last_clean":"2024-04-16T19:32:31.157016+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:32:31.157016+0000","last_undegraded":"2024-04-16T19:32:31.157016+0000","last_fullsized":"2024-04-16T19:32:31.157016+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'96","reported_seq":919,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156060+0000","last_change":"2024-04-16T17:48:34.324041+0000","last_active":"2024-04-16T19:33:36.156060+0000","last_peered":"2024-04-16T19:33:36.156060+0000","last_clean":"2024-04-16T19:33:36.156060+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:33:36.156060+0000","last_undegraded":"2024-04-16T19:33:36.156060+0000","last_fullsized":"2024-04-16T19:33:36.156060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:18:27.393972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172731+0000","last_change":"2024-04-16T17:48:27.307648+0000","last_active":"2024-04-16T19:33:31.172731+0000","last_peered":"2024-04-16T19:33:31.172731+0000","last_clean":"2024-04-16T19:33:31.172731+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:33:31.172731+0000","last_undegraded":"2024-04-16T19:33:31.172731+0000","last_fullsized":"2024-04-16T19:33:31.172731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:20.265988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'7","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162293+0000","last_change":"2024-04-16T17:48:01.248401+0000","last_active":"2024-04-16T19:33:01.162293+0000","last_peered":"2024-04-16T19:33:01.162293+0000","last_clean":"2024-04-16T19:33:01.162293+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:33:01.162293+0000","last_undegraded":"2024-04-16T19:33:01.162293+0000","last_fullsized":"2024-04-16T19:33:01.162293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145357+0000","last_change":"2024-04-16T17:47:59.199961+0000","last_active":"2024-04-16T19:31:56.145357+0000","last_peered":"2024-04-16T19:31:56.145357+0000","last_clean":"2024-04-16T19:31:56.145357+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:31:56.145357+0000","last_undegraded":"2024-04-16T19:31:56.145357+0000","last_fullsized":"2024-04-16T19:31:56.145357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:44.427887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'129","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150185+0000","last_change":"2024-04-16T17:47:57.189447+0000","last_active":"2024-04-16T19:31:56.150185+0000","last_peered":"2024-04-16T19:31:56.150185+0000","last_clean":"2024-04-16T19:31:56.150185+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:31:56.150185+0000","last_undegraded":"2024-04-16T19:31:56.150185+0000","last_fullsized":"2024-04-16T19:31:56.150185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:55.411174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'3","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171761+0000","last_change":"2024-04-16T17:47:55.160684+0000","last_active":"2024-04-16T19:33:26.171761+0000","last_peered":"2024-04-16T19:33:26.171761+0000","last_clean":"2024-04-16T19:33:26.171761+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:33:26.171761+0000","last_undegraded":"2024-04-16T19:33:26.171761+0000","last_fullsized":"2024-04-16T19:33:26.171761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148167+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:31:56.148167+0000","last_peered":"2024-04-16T19:31:56.148167+0000","last_clean":"2024-04-16T19:31:56.148167+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:31:56.148167+0000","last_undegraded":"2024-04-16T19:31:56.148167+0000","last_fullsized":"2024-04-16T19:31:56.148167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159976+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:33:51.159976+0000","last_peered":"2024-04-16T19:33:51.159976+0000","last_clean":"2024-04-16T19:33:51.159976+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:33:51.159976+0000","last_undegraded":"2024-04-16T19:33:51.159976+0000","last_fullsized":"2024-04-16T19:33:51.159976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:50:28.662489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361145+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:31:31.361145+0000","last_peered":"2024-04-16T19:31:31.361145+0000","last_clean":"2024-04-16T19:31:31.361145+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:31:31.361145+0000","last_undegraded":"2024-04-16T19:31:31.361145+0000","last_fullsized":"2024-04-16T19:31:31.361145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:03.171529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'32","reported_seq":160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158251+0000","last_change":"2024-04-16T17:47:35.317051+0000","last_active":"2024-04-16T19:32:36.158251+0000","last_peered":"2024-04-16T19:32:36.158251+0000","last_clean":"2024-04-16T19:32:36.158251+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:32:36.158251+0000","last_undegraded":"2024-04-16T19:32:36.158251+0000","last_fullsized":"2024-04-16T19:32:36.158251+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162398+0000","last_change":"2024-04-16T17:47:37.320351+0000","last_active":"2024-04-16T19:32:41.162398+0000","last_peered":"2024-04-16T19:32:41.162398+0000","last_clean":"2024-04-16T19:32:41.162398+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:32:41.162398+0000","last_undegraded":"2024-04-16T19:32:41.162398+0000","last_fullsized":"2024-04-16T19:32:41.162398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144501+0000","last_change":"2024-04-16T17:47:39.329617+0000","last_active":"2024-04-16T19:31:36.144501+0000","last_peered":"2024-04-16T19:31:36.144501+0000","last_clean":"2024-04-16T19:31:36.144501+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:31:36.144501+0000","last_undegraded":"2024-04-16T19:31:36.144501+0000","last_fullsized":"2024-04-16T19:31:36.144501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157983+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:32:31.157983+0000","last_peered":"2024-04-16T19:32:31.157983+0000","last_clean":"2024-04-16T19:32:31.157983+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:32:31.157983+0000","last_undegraded":"2024-04-16T19:32:31.157983+0000","last_fullsized":"2024-04-16T19:32:31.157983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:49:40.225240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'126","reported_seq":1450,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156003+0000","last_change":"2024-04-16T17:48:34.338811+0000","last_active":"2024-04-16T19:33:36.156003+0000","last_peered":"2024-04-16T19:33:36.156003+0000","last_clean":"2024-04-16T19:33:36.156003+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:33:36.156003+0000","last_undegraded":"2024-04-16T19:33:36.156003+0000","last_fullsized":"2024-04-16T19:33:36.156003+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:09:10.563495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172826+0000","last_change":"2024-04-16T17:48:27.288374+0000","last_active":"2024-04-16T19:33:31.172826+0000","last_peered":"2024-04-16T19:33:31.172826+0000","last_clean":"2024-04-16T19:33:31.172826+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:33:31.172826+0000","last_undegraded":"2024-04-16T19:33:31.172826+0000","last_fullsized":"2024-04-16T19:33:31.172826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:11.751579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164610+0000","last_change":"2024-04-16T17:48:01.263690+0000","last_active":"2024-04-16T19:33:01.164610+0000","last_peered":"2024-04-16T19:33:01.164610+0000","last_clean":"2024-04-16T19:33:01.164610+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:33:01.164610+0000","last_undegraded":"2024-04-16T19:33:01.164610+0000","last_fullsized":"2024-04-16T19:33:01.164610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:47.176953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145421+0000","last_change":"2024-04-16T17:47:59.199357+0000","last_active":"2024-04-16T19:31:56.145421+0000","last_peered":"2024-04-16T19:31:56.145421+0000","last_clean":"2024-04-16T19:31:56.145421+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:31:56.145421+0000","last_undegraded":"2024-04-16T19:31:56.145421+0000","last_fullsized":"2024-04-16T19:31:56.145421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:42.077363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11055","reported_seq":18133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150080+0000","last_change":"2024-04-16T17:47:57.184044+0000","last_active":"2024-04-16T19:31:56.150080+0000","last_peered":"2024-04-16T19:31:56.150080+0000","last_clean":"2024-04-16T19:31:56.150080+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:31:56.150080+0000","last_undegraded":"2024-04-16T19:31:56.150080+0000","last_fullsized":"2024-04-16T19:31:56.150080+0000","mapping_epoch":34,"log_start":"53'9200","ondisk_log_start":"53'9200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1855,"log_dups_size":3000,"ondisk_log_size":1855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'6","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143908+0000","last_change":"2024-04-16T17:47:55.159410+0000","last_active":"2024-04-16T19:31:51.143908+0000","last_peered":"2024-04-16T19:31:51.143908+0000","last_clean":"2024-04-16T19:31:51.143908+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:31:51.143908+0000","last_undegraded":"2024-04-16T19:31:51.143908+0000","last_fullsized":"2024-04-16T19:31:51.143908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:42.685205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179184+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:33:46.179184+0000","last_peered":"2024-04-16T19:33:46.179184+0000","last_clean":"2024-04-16T19:33:46.179184+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:33:46.179184+0000","last_undegraded":"2024-04-16T19:33:46.179184+0000","last_fullsized":"2024-04-16T19:33:46.179184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:54.182792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173750+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:33:36.173750+0000","last_peered":"2024-04-16T19:33:36.173750+0000","last_clean":"2024-04-16T19:33:36.173750+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:33:36.173750+0000","last_undegraded":"2024-04-16T19:33:36.173750+0000","last_fullsized":"2024-04-16T19:33:36.173750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149851+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:32:01.149851+0000","last_peered":"2024-04-16T19:32:01.149851+0000","last_clean":"2024-04-16T19:32:01.149851+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:32:01.149851+0000","last_undegraded":"2024-04-16T19:32:01.149851+0000","last_fullsized":"2024-04-16T19:32:01.149851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:53.316949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'80","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155149+0000","last_change":"2024-04-16T17:47:35.308166+0000","last_active":"2024-04-16T19:32:36.155149+0000","last_peered":"2024-04-16T19:32:36.155149+0000","last_clean":"2024-04-16T19:32:36.155149+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:32:36.155149+0000","last_undegraded":"2024-04-16T19:32:36.155149+0000","last_fullsized":"2024-04-16T19:32:36.155149+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:53.924341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159795+0000","last_change":"2024-04-16T17:47:37.321988+0000","last_active":"2024-04-16T19:32:41.159795+0000","last_peered":"2024-04-16T19:32:41.159795+0000","last_clean":"2024-04-16T19:32:41.159795+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:32:41.159795+0000","last_undegraded":"2024-04-16T19:32:41.159795+0000","last_fullsized":"2024-04-16T19:32:41.159795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164797+0000","last_change":"2024-04-16T17:47:39.326823+0000","last_active":"2024-04-16T19:33:16.164797+0000","last_peered":"2024-04-16T19:33:16.164797+0000","last_clean":"2024-04-16T19:33:16.164797+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:33:16.164797+0000","last_undegraded":"2024-04-16T19:33:16.164797+0000","last_fullsized":"2024-04-16T19:33:16.164797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:26.874144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'22","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157286+0000","last_change":"2024-04-16T17:47:55.158595+0000","last_active":"2024-04-16T19:33:41.157286+0000","last_peered":"2024-04-16T19:33:41.157286+0000","last_clean":"2024-04-16T19:33:41.157286+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:33:41.157286+0000","last_undegraded":"2024-04-16T19:33:41.157286+0000","last_fullsized":"2024-04-16T19:33:41.157286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:20.899420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158764+0000","last_change":"2024-04-16T17:47:32.278320+0000","last_active":"2024-04-16T19:33:46.158764+0000","last_peered":"2024-04-16T19:33:46.158764+0000","last_clean":"2024-04-16T19:33:46.158764+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:33:46.158764+0000","last_undegraded":"2024-04-16T19:33:46.158764+0000","last_fullsized":"2024-04-16T19:33:46.158764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:32.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169629+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:33:36.169629+0000","last_peered":"2024-04-16T19:33:36.169629+0000","last_clean":"2024-04-16T19:33:36.169629+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:33:36.169629+0000","last_undegraded":"2024-04-16T19:33:36.169629+0000","last_fullsized":"2024-04-16T19:33:36.169629+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'3","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172656+0000","last_change":"2024-04-16T17:47:55.175820+0000","last_active":"2024-04-16T19:33:41.172656+0000","last_peered":"2024-04-16T19:33:41.172656+0000","last_clean":"2024-04-16T19:33:41.172656+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:33:41.172656+0000","last_undegraded":"2024-04-16T19:33:41.172656+0000","last_fullsized":"2024-04-16T19:33:41.172656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174508+0000","last_change":"2024-04-16T17:47:23.260359+0000","last_active":"2024-04-16T19:33:36.174508+0000","last_peered":"2024-04-16T19:33:36.174508+0000","last_clean":"2024-04-16T19:33:36.174508+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:33:36.174508+0000","last_undegraded":"2024-04-16T19:33:36.174508+0000","last_fullsized":"2024-04-16T19:33:36.174508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:31.902037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.155944+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:33:36.155944+0000","last_peered":"2024-04-16T19:33:36.155944+0000","last_clean":"2024-04-16T19:33:36.155944+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:33:36.155944+0000","last_undegraded":"2024-04-16T19:33:36.155944+0000","last_fullsized":"2024-04-16T19:33:36.155944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:39.513573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'7","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151820+0000","last_change":"2024-04-16T17:47:55.178677+0000","last_active":"2024-04-16T19:32:21.151820+0000","last_peered":"2024-04-16T19:32:21.151820+0000","last_clean":"2024-04-16T19:32:21.151820+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:32:21.151820+0000","last_undegraded":"2024-04-16T19:32:21.151820+0000","last_fullsized":"2024-04-16T19:32:21.151820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.171376+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:33:36.171376+0000","last_peered":"2024-04-16T19:33:36.171376+0000","last_clean":"2024-04-16T19:33:36.171376+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:33:36.171376+0000","last_undegraded":"2024-04-16T19:33:36.171376+0000","last_fullsized":"2024-04-16T19:33:36.171376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176213+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:33:36.176213+0000","last_peered":"2024-04-16T19:33:36.176213+0000","last_clean":"2024-04-16T19:33:36.176213+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:33:36.176213+0000","last_undegraded":"2024-04-16T19:33:36.176213+0000","last_fullsized":"2024-04-16T19:33:36.176213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'6","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177717+0000","last_change":"2024-04-16T17:47:55.156061+0000","last_active":"2024-04-16T19:33:41.177717+0000","last_peered":"2024-04-16T19:33:41.177717+0000","last_clean":"2024-04-16T19:33:41.177717+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:33:41.177717+0000","last_undegraded":"2024-04-16T19:33:41.177717+0000","last_fullsized":"2024-04-16T19:33:41.177717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174362+0000","last_change":"2024-04-16T17:47:23.261639+0000","last_active":"2024-04-16T19:33:36.174362+0000","last_peered":"2024-04-16T19:33:36.174362+0000","last_clean":"2024-04-16T19:33:36.174362+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:33:36.174362+0000","last_undegraded":"2024-04-16T19:33:36.174362+0000","last_fullsized":"2024-04-16T19:33:36.174362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150697+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:32:06.150697+0000","last_peered":"2024-04-16T19:32:06.150697+0000","last_clean":"2024-04-16T19:32:06.150697+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:32:06.150697+0000","last_undegraded":"2024-04-16T19:32:06.150697+0000","last_fullsized":"2024-04-16T19:32:06.150697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"48'3","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161266+0000","last_change":"2024-04-16T17:47:55.181215+0000","last_active":"2024-04-16T19:32:56.161266+0000","last_peered":"2024-04-16T19:32:56.161266+0000","last_clean":"2024-04-16T19:32:56.161266+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:32:56.161266+0000","last_undegraded":"2024-04-16T19:32:56.161266+0000","last_fullsized":"2024-04-16T19:32:56.161266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135310+0000","last_change":"2024-04-16T17:47:23.253450+0000","last_active":"2024-04-16T19:32:06.135310+0000","last_peered":"2024-04-16T19:32:06.135310+0000","last_clean":"2024-04-16T19:32:06.135310+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:32:06.135310+0000","last_undegraded":"2024-04-16T19:32:06.135310+0000","last_fullsized":"2024-04-16T19:32:06.135310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:44.749396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172146+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:33:31.172146+0000","last_peered":"2024-04-16T19:33:31.172146+0000","last_clean":"2024-04-16T19:33:31.172146+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:33:31.172146+0000","last_undegraded":"2024-04-16T19:33:31.172146+0000","last_fullsized":"2024-04-16T19:33:31.172146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"48'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146820+0000","last_change":"2024-04-16T17:47:55.165525+0000","last_active":"2024-04-16T19:32:56.146820+0000","last_peered":"2024-04-16T19:32:56.146820+0000","last_clean":"2024-04-16T19:32:56.146820+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:32:56.146820+0000","last_undegraded":"2024-04-16T19:32:56.146820+0000","last_fullsized":"2024-04-16T19:32:56.146820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:20:06.245260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171037+0000","last_change":"2024-04-16T17:47:23.257600+0000","last_active":"2024-04-16T19:33:41.171037+0000","last_peered":"2024-04-16T19:33:41.171037+0000","last_clean":"2024-04-16T19:33:41.171037+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:33:41.171037+0000","last_undegraded":"2024-04-16T19:33:41.171037+0000","last_fullsized":"2024-04-16T19:33:41.171037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:11.143975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.173036+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:33:31.173036+0000","last_peered":"2024-04-16T19:33:31.173036+0000","last_clean":"2024-04-16T19:33:31.173036+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:33:31.173036+0000","last_undegraded":"2024-04-16T19:33:31.173036+0000","last_fullsized":"2024-04-16T19:33:31.173036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'40","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175919+0000","last_change":"2024-04-16T17:47:55.160972+0000","last_active":"2024-04-16T19:33:41.175919+0000","last_peered":"2024-04-16T19:33:41.175919+0000","last_clean":"2024-04-16T19:33:41.175919+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:33:41.175919+0000","last_undegraded":"2024-04-16T19:33:41.175919+0000","last_fullsized":"2024-04-16T19:33:41.175919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159832+0000","last_change":"2024-04-16T17:47:32.281080+0000","last_active":"2024-04-16T19:33:51.159832+0000","last_peered":"2024-04-16T19:33:51.159832+0000","last_clean":"2024-04-16T19:33:51.159832+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:33:51.159832+0000","last_undegraded":"2024-04-16T19:33:51.159832+0000","last_fullsized":"2024-04-16T19:33:51.159832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:58.918776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175091+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:33:31.175091+0000","last_peered":"2024-04-16T19:33:31.175091+0000","last_clean":"2024-04-16T19:33:31.175091+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:33:31.175091+0000","last_undegraded":"2024-04-16T19:33:31.175091+0000","last_fullsized":"2024-04-16T19:33:31.175091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171222+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:33:41.171222+0000","last_peered":"2024-04-16T19:33:41.171222+0000","last_clean":"2024-04-16T19:33:41.171222+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:33:41.171222+0000","last_undegraded":"2024-04-16T19:33:41.171222+0000","last_fullsized":"2024-04-16T19:33:41.171222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:26.938403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177647+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:33:41.177647+0000","last_peered":"2024-04-16T19:33:41.177647+0000","last_clean":"2024-04-16T19:33:41.177647+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:33:41.177647+0000","last_undegraded":"2024-04-16T19:33:41.177647+0000","last_fullsized":"2024-04-16T19:33:41.177647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154939+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:33:31.154939+0000","last_peered":"2024-04-16T19:33:31.154939+0000","last_clean":"2024-04-16T19:33:31.154939+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:33:31.154939+0000","last_undegraded":"2024-04-16T19:33:31.154939+0000","last_fullsized":"2024-04-16T19:33:31.154939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:12.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'15","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152554+0000","last_change":"2024-04-16T17:47:55.170559+0000","last_active":"2024-04-16T19:32:06.152554+0000","last_peered":"2024-04-16T19:32:06.152554+0000","last_clean":"2024-04-16T19:32:06.152554+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:32:06.152554+0000","last_undegraded":"2024-04-16T19:32:06.152554+0000","last_fullsized":"2024-04-16T19:32:06.152554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155962+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:32:26.155962+0000","last_peered":"2024-04-16T19:32:26.155962+0000","last_clean":"2024-04-16T19:32:26.155962+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:32:26.155962+0000","last_undegraded":"2024-04-16T19:32:26.155962+0000","last_fullsized":"2024-04-16T19:32:26.155962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176559+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:33:56.176559+0000","last_peered":"2024-04-16T19:33:56.176559+0000","last_clean":"2024-04-16T19:33:56.176559+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:33:56.176559+0000","last_undegraded":"2024-04-16T19:33:56.176559+0000","last_fullsized":"2024-04-16T19:33:56.176559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:44.519947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'18","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142162+0000","last_change":"2024-04-16T17:47:55.177250+0000","last_active":"2024-04-16T19:32:36.142162+0000","last_peered":"2024-04-16T19:32:36.142162+0000","last_clean":"2024-04-16T19:32:36.142162+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:32:36.142162+0000","last_undegraded":"2024-04-16T19:32:36.142162+0000","last_fullsized":"2024-04-16T19:32:36.142162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:42.691526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152860+0000","last_change":"2024-04-16T17:47:23.264252+0000","last_active":"2024-04-16T19:32:26.152860+0000","last_peered":"2024-04-16T19:32:26.152860+0000","last_clean":"2024-04-16T19:32:26.152860+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:32:26.152860+0000","last_undegraded":"2024-04-16T19:32:26.152860+0000","last_fullsized":"2024-04-16T19:32:26.152860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166011+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:33:06.166011+0000","last_peered":"2024-04-16T19:33:06.166011+0000","last_clean":"2024-04-16T19:33:06.166011+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:33:06.166011+0000","last_undegraded":"2024-04-16T19:33:06.166011+0000","last_fullsized":"2024-04-16T19:33:06.166011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'1","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150517+0000","last_change":"2024-04-16T17:47:55.168092+0000","last_active":"2024-04-16T19:32:06.150517+0000","last_peered":"2024-04-16T19:32:06.150517+0000","last_clean":"2024-04-16T19:32:06.150517+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:32:06.150517+0000","last_undegraded":"2024-04-16T19:32:06.150517+0000","last_fullsized":"2024-04-16T19:32:06.150517+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157395+0000","last_change":"2024-04-16T17:47:23.265700+0000","last_active":"2024-04-16T19:33:41.157395+0000","last_peered":"2024-04-16T19:33:41.157395+0000","last_clean":"2024-04-16T19:33:41.157395+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:33:41.157395+0000","last_undegraded":"2024-04-16T19:33:41.157395+0000","last_fullsized":"2024-04-16T19:33:41.157395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:24:05.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163673+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:33:06.163673+0000","last_peered":"2024-04-16T19:33:06.163673+0000","last_clean":"2024-04-16T19:33:06.163673+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:33:06.163673+0000","last_undegraded":"2024-04-16T19:33:06.163673+0000","last_fullsized":"2024-04-16T19:33:06.163673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'18","reported_seq":243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135182+0000","last_change":"2024-04-16T17:47:55.147785+0000","last_active":"2024-04-16T19:32:06.135182+0000","last_peered":"2024-04-16T19:32:06.135182+0000","last_clean":"2024-04-16T19:32:06.135182+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:32:06.135182+0000","last_undegraded":"2024-04-16T19:32:06.135182+0000","last_fullsized":"2024-04-16T19:32:06.135182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:26.488442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170252+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:33:11.170252+0000","last_peered":"2024-04-16T19:33:11.170252+0000","last_clean":"2024-04-16T19:33:11.170252+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:33:11.170252+0000","last_undegraded":"2024-04-16T19:33:11.170252+0000","last_fullsized":"2024-04-16T19:33:11.170252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166111+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:33:06.166111+0000","last_peered":"2024-04-16T19:33:06.166111+0000","last_clean":"2024-04-16T19:33:06.166111+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:33:06.166111+0000","last_undegraded":"2024-04-16T19:33:06.166111+0000","last_fullsized":"2024-04-16T19:33:06.166111+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145325+0000","last_change":"2024-04-16T17:47:55.183714+0000","last_active":"2024-04-16T19:31:56.145325+0000","last_peered":"2024-04-16T19:31:56.145325+0000","last_clean":"2024-04-16T19:31:56.145325+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:31:56.145325+0000","last_undegraded":"2024-04-16T19:31:56.145325+0000","last_fullsized":"2024-04-16T19:31:56.145325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163558+0000","last_change":"2024-04-16T17:47:23.265903+0000","last_active":"2024-04-16T19:33:11.163558+0000","last_peered":"2024-04-16T19:33:11.163558+0000","last_clean":"2024-04-16T19:33:11.163558+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:33:11.163558+0000","last_undegraded":"2024-04-16T19:33:11.163558+0000","last_fullsized":"2024-04-16T19:33:11.163558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:55.605376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168971+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:33:06.168971+0000","last_peered":"2024-04-16T19:33:06.168971+0000","last_clean":"2024-04-16T19:33:06.168971+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:33:06.168971+0000","last_undegraded":"2024-04-16T19:33:06.168971+0000","last_fullsized":"2024-04-16T19:33:06.168971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:14.412040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"48'2","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132984+0000","last_change":"2024-04-16T17:47:55.162693+0000","last_active":"2024-04-16T19:31:56.132984+0000","last_peered":"2024-04-16T19:31:56.132984+0000","last_clean":"2024-04-16T19:31:56.132984+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:31:56.132984+0000","last_undegraded":"2024-04-16T19:31:56.132984+0000","last_fullsized":"2024-04-16T19:31:56.132984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:26:14.047560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167227+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:33:11.167227+0000","last_peered":"2024-04-16T19:33:11.167227+0000","last_clean":"2024-04-16T19:33:11.167227+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:33:11.167227+0000","last_undegraded":"2024-04-16T19:33:11.167227+0000","last_fullsized":"2024-04-16T19:33:11.167227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177980+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:33:41.177980+0000","last_peered":"2024-04-16T19:33:41.177980+0000","last_clean":"2024-04-16T19:33:41.177980+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:33:41.177980+0000","last_undegraded":"2024-04-16T19:33:41.177980+0000","last_fullsized":"2024-04-16T19:33:41.177980+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'3","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132884+0000","last_change":"2024-04-16T17:47:55.178234+0000","last_active":"2024-04-16T19:31:56.132884+0000","last_peered":"2024-04-16T19:31:56.132884+0000","last_clean":"2024-04-16T19:31:56.132884+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:31:56.132884+0000","last_undegraded":"2024-04-16T19:31:56.132884+0000","last_fullsized":"2024-04-16T19:31:56.132884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:17.561808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167956+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:33:11.167956+0000","last_peered":"2024-04-16T19:33:11.167956+0000","last_clean":"2024-04-16T19:33:11.167956+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:33:11.167956+0000","last_undegraded":"2024-04-16T19:33:11.167956+0000","last_fullsized":"2024-04-16T19:33:11.167956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138794+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:32:21.138794+0000","last_peered":"2024-04-16T19:32:21.138794+0000","last_clean":"2024-04-16T19:32:21.138794+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:32:21.138794+0000","last_undegraded":"2024-04-16T19:32:21.138794+0000","last_fullsized":"2024-04-16T19:32:21.138794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:40.793750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'6","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148370+0000","last_change":"2024-04-16T17:47:55.177975+0000","last_active":"2024-04-16T19:31:56.148370+0000","last_peered":"2024-04-16T19:31:56.148370+0000","last_clean":"2024-04-16T19:31:56.148370+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:31:56.148370+0000","last_undegraded":"2024-04-16T19:31:56.148370+0000","last_fullsized":"2024-04-16T19:31:56.148370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:32.181968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151087+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:32:01.151087+0000","last_peered":"2024-04-16T19:32:01.151087+0000","last_clean":"2024-04-16T19:32:01.151087+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:32:01.151087+0000","last_undegraded":"2024-04-16T19:32:01.151087+0000","last_fullsized":"2024-04-16T19:32:01.151087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152209+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:32:21.152209+0000","last_peered":"2024-04-16T19:32:21.152209+0000","last_clean":"2024-04-16T19:32:21.152209+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:32:21.152209+0000","last_undegraded":"2024-04-16T19:32:21.152209+0000","last_fullsized":"2024-04-16T19:32:21.152209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'10","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163428+0000","last_change":"2024-04-16T17:47:55.156240+0000","last_active":"2024-04-16T19:32:56.163428+0000","last_peered":"2024-04-16T19:32:56.163428+0000","last_clean":"2024-04-16T19:32:56.163428+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:32:56.163428+0000","last_undegraded":"2024-04-16T19:32:56.163428+0000","last_fullsized":"2024-04-16T19:32:56.163428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156891+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:32:21.156891+0000","last_peered":"2024-04-16T19:32:21.156891+0000","last_clean":"2024-04-16T19:32:21.156891+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:32:21.156891+0000","last_undegraded":"2024-04-16T19:32:21.156891+0000","last_fullsized":"2024-04-16T19:32:21.156891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148441+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:31:56.148441+0000","last_peered":"2024-04-16T19:31:56.148441+0000","last_clean":"2024-04-16T19:31:56.148441+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:31:56.148441+0000","last_undegraded":"2024-04-16T19:31:56.148441+0000","last_fullsized":"2024-04-16T19:31:56.148441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'14","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146712+0000","last_change":"2024-04-16T17:47:55.164923+0000","last_active":"2024-04-16T19:32:56.146712+0000","last_peered":"2024-04-16T19:32:56.146712+0000","last_clean":"2024-04-16T19:32:56.146712+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:32:56.146712+0000","last_undegraded":"2024-04-16T19:32:56.146712+0000","last_fullsized":"2024-04-16T19:32:56.146712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:25.251453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.155204+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:32:21.155204+0000","last_peered":"2024-04-16T19:32:21.155204+0000","last_clean":"2024-04-16T19:32:21.155204+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:32:21.155204+0000","last_undegraded":"2024-04-16T19:32:21.155204+0000","last_fullsized":"2024-04-16T19:32:21.155204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:30.655498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174246+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:33:46.174246+0000","last_peered":"2024-04-16T19:33:46.174246+0000","last_clean":"2024-04-16T19:33:46.174246+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:33:46.174246+0000","last_undegraded":"2024-04-16T19:33:46.174246+0000","last_fullsized":"2024-04-16T19:33:46.174246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2844,"ondisk_log_size":2844,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3706,"ondisk_log_size":3706,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":623,"ondisk_log_size":623,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27355,"ondisk_log_size":27355,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4460,"ondisk_log_size":4460,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":437,"ondisk_log_size":437,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641575,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46584,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811016,"statfs":{"total":107374182400,"available":107326480384,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641574,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641574,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":99932,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":104757668,"statfs":{"total":107374182400,"available":107271852032,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641574,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46572,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811028,"statfs":{"total":107374182400,"available":107326492672,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641575,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":55408,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":104802192,"statfs":{"total":107374182400,"available":107317444608,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641575,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104829060,"statfs":{"total":107374182400,"available":107344957440,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:36:14.336 INFO:tasks.ceph.ceph_manager.c1:clean! 2024-04-16T19:36:14.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:36:14.715 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:36:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:36:14.758 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3335,"stamp":"2024-04-16T19:36:14.317911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39759,"ondisk_log_size":39759,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":354032,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":343862,"kb_avail":628791568,"statfs":{"total":644245094400,"available":643882565632,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":352115360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010663"},"pg_stats":[{"pgid":"9.35","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147548+0000","last_change":"2024-04-16T17:47:55.160868+0000","last_active":"2024-04-16T19:31:51.147548+0000","last_peered":"2024-04-16T19:31:51.147548+0000","last_clean":"2024-04-16T19:31:51.147548+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:31:51.147548+0000","last_undegraded":"2024-04-16T19:31:51.147548+0000","last_fullsized":"2024-04-16T19:31:51.147548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176615+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:33:36.176615+0000","last_peered":"2024-04-16T19:33:36.176615+0000","last_clean":"2024-04-16T19:33:36.176615+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:33:36.176615+0000","last_undegraded":"2024-04-16T19:33:36.176615+0000","last_fullsized":"2024-04-16T19:33:36.176615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:51.288503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153219+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:32:26.153219+0000","last_peered":"2024-04-16T19:32:26.153219+0000","last_clean":"2024-04-16T19:32:26.153219+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:32:26.153219+0000","last_undegraded":"2024-04-16T19:32:26.153219+0000","last_fullsized":"2024-04-16T19:32:26.153219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'14","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143970+0000","last_change":"2024-04-16T17:47:55.173885+0000","last_active":"2024-04-16T19:31:51.143970+0000","last_peered":"2024-04-16T19:31:51.143970+0000","last_clean":"2024-04-16T19:31:51.143970+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:31:51.143970+0000","last_undegraded":"2024-04-16T19:31:51.143970+0000","last_fullsized":"2024-04-16T19:31:51.143970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:50.786411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174452+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:33:36.174452+0000","last_peered":"2024-04-16T19:33:36.174452+0000","last_clean":"2024-04-16T19:33:36.174452+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:33:36.174452+0000","last_undegraded":"2024-04-16T19:33:36.174452+0000","last_fullsized":"2024-04-16T19:33:36.174452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.158347+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:32:26.158347+0000","last_peered":"2024-04-16T19:32:26.158347+0000","last_clean":"2024-04-16T19:32:26.158347+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:32:26.158347+0000","last_undegraded":"2024-04-16T19:32:26.158347+0000","last_fullsized":"2024-04-16T19:32:26.158347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'20","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152906+0000","last_change":"2024-04-16T17:47:55.156902+0000","last_active":"2024-04-16T19:32:26.152906+0000","last_peered":"2024-04-16T19:32:26.152906+0000","last_clean":"2024-04-16T19:32:26.152906+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:32:26.152906+0000","last_undegraded":"2024-04-16T19:32:26.152906+0000","last_fullsized":"2024-04-16T19:32:26.152906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:40.884246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173569+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:33:36.173569+0000","last_peered":"2024-04-16T19:33:36.173569+0000","last_clean":"2024-04-16T19:33:36.173569+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:33:36.173569+0000","last_undegraded":"2024-04-16T19:33:36.173569+0000","last_fullsized":"2024-04-16T19:33:36.173569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152536+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:32:26.152536+0000","last_peered":"2024-04-16T19:32:26.152536+0000","last_clean":"2024-04-16T19:32:26.152536+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:32:26.152536+0000","last_undegraded":"2024-04-16T19:32:26.152536+0000","last_fullsized":"2024-04-16T19:32:26.152536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:32.905270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147603+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:31:51.147603+0000","last_peered":"2024-04-16T19:31:51.147603+0000","last_clean":"2024-04-16T19:31:51.147603+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:31:51.147603+0000","last_undegraded":"2024-04-16T19:31:51.147603+0000","last_fullsized":"2024-04-16T19:31:51.147603+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:10.585096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174129+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:33:46.174129+0000","last_peered":"2024-04-16T19:33:46.174129+0000","last_clean":"2024-04-16T19:33:46.174129+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:33:46.174129+0000","last_undegraded":"2024-04-16T19:33:46.174129+0000","last_fullsized":"2024-04-16T19:33:46.174129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:53.592351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155669+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:32:26.155669+0000","last_peered":"2024-04-16T19:32:26.155669+0000","last_clean":"2024-04-16T19:32:26.155669+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:32:26.155669+0000","last_undegraded":"2024-04-16T19:32:26.155669+0000","last_fullsized":"2024-04-16T19:32:26.155669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'4","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159892+0000","last_change":"2024-04-16T17:47:55.159892+0000","last_active":"2024-04-16T19:32:56.159892+0000","last_peered":"2024-04-16T19:32:56.159892+0000","last_clean":"2024-04-16T19:32:56.159892+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:32:56.159892+0000","last_undegraded":"2024-04-16T19:32:56.159892+0000","last_fullsized":"2024-04-16T19:32:56.159892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:58.461789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159155+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:32:36.159155+0000","last_peered":"2024-04-16T19:32:36.159155+0000","last_clean":"2024-04-16T19:32:36.159155+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:32:36.159155+0000","last_undegraded":"2024-04-16T19:32:36.159155+0000","last_fullsized":"2024-04-16T19:32:36.159155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175465+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:33:51.175465+0000","last_peered":"2024-04-16T19:33:51.175465+0000","last_clean":"2024-04-16T19:33:51.175465+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:33:51.175465+0000","last_undegraded":"2024-04-16T19:33:51.175465+0000","last_fullsized":"2024-04-16T19:33:51.175465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'11","reported_seq":309,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163313+0000","last_change":"2024-04-16T17:47:55.163604+0000","last_active":"2024-04-16T19:32:56.163313+0000","last_peered":"2024-04-16T19:32:56.163313+0000","last_clean":"2024-04-16T19:32:56.163313+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:32:56.163313+0000","last_undegraded":"2024-04-16T19:32:56.163313+0000","last_fullsized":"2024-04-16T19:32:56.163313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151934+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:32:21.151934+0000","last_peered":"2024-04-16T19:32:21.151934+0000","last_clean":"2024-04-16T19:32:21.151934+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:32:21.151934+0000","last_undegraded":"2024-04-16T19:32:21.151934+0000","last_fullsized":"2024-04-16T19:32:21.151934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171180+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:33:41.171180+0000","last_peered":"2024-04-16T19:33:41.171180+0000","last_clean":"2024-04-16T19:33:41.171180+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:33:41.171180+0000","last_undegraded":"2024-04-16T19:33:41.171180+0000","last_fullsized":"2024-04-16T19:33:41.171180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:47.915323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'12","reported_seq":268,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.148697+0000","last_change":"2024-04-16T17:47:55.161748+0000","last_active":"2024-04-16T19:31:51.148697+0000","last_peered":"2024-04-16T19:31:51.148697+0000","last_clean":"2024-04-16T19:31:51.148697+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:31:51.148697+0000","last_undegraded":"2024-04-16T19:31:51.148697+0000","last_fullsized":"2024-04-16T19:31:51.148697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157014+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:32:21.157014+0000","last_peered":"2024-04-16T19:32:21.157014+0000","last_clean":"2024-04-16T19:32:21.157014+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:32:21.157014+0000","last_undegraded":"2024-04-16T19:32:21.157014+0000","last_fullsized":"2024-04-16T19:32:21.157014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:14.655507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172868+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:33:41.172868+0000","last_peered":"2024-04-16T19:33:41.172868+0000","last_clean":"2024-04-16T19:33:41.172868+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:33:41.172868+0000","last_undegraded":"2024-04-16T19:33:41.172868+0000","last_fullsized":"2024-04-16T19:33:41.172868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'8","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168449+0000","last_change":"2024-04-16T17:47:55.164548+0000","last_active":"2024-04-16T19:33:31.168449+0000","last_peered":"2024-04-16T19:33:31.168449+0000","last_clean":"2024-04-16T19:33:31.168449+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:33:31.168449+0000","last_undegraded":"2024-04-16T19:33:31.168449+0000","last_fullsized":"2024-04-16T19:33:31.168449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:27.967153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138549+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:32:21.138549+0000","last_peered":"2024-04-16T19:32:21.138549+0000","last_clean":"2024-04-16T19:32:21.138549+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:32:21.138549+0000","last_undegraded":"2024-04-16T19:32:21.138549+0000","last_fullsized":"2024-04-16T19:32:21.138549+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:19:39.583925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157481+0000","last_change":"2024-04-16T17:47:23.254199+0000","last_active":"2024-04-16T19:33:41.157481+0000","last_peered":"2024-04-16T19:33:41.157481+0000","last_clean":"2024-04-16T19:33:41.157481+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:33:41.157481+0000","last_undegraded":"2024-04-16T19:33:41.157481+0000","last_fullsized":"2024-04-16T19:33:41.157481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:11.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"48'5","reported_seq":300,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148127+0000","last_change":"2024-04-16T17:47:55.184230+0000","last_active":"2024-04-16T19:32:06.148127+0000","last_peered":"2024-04-16T19:32:06.148127+0000","last_clean":"2024-04-16T19:32:06.148127+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:32:06.148127+0000","last_undegraded":"2024-04-16T19:32:06.148127+0000","last_fullsized":"2024-04-16T19:32:06.148127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152428+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:32:26.152428+0000","last_peered":"2024-04-16T19:32:26.152428+0000","last_clean":"2024-04-16T19:32:26.152428+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:32:26.152428+0000","last_undegraded":"2024-04-16T19:32:26.152428+0000","last_fullsized":"2024-04-16T19:32:26.152428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150380+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:32:01.150380+0000","last_peered":"2024-04-16T19:32:01.150380+0000","last_clean":"2024-04-16T19:32:01.150380+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:32:01.150380+0000","last_undegraded":"2024-04-16T19:32:01.150380+0000","last_fullsized":"2024-04-16T19:32:01.150380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'14","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159666+0000","last_change":"2024-04-16T17:47:55.163042+0000","last_active":"2024-04-16T19:32:31.159666+0000","last_peered":"2024-04-16T19:32:31.159666+0000","last_clean":"2024-04-16T19:32:31.159666+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:32:31.159666+0000","last_undegraded":"2024-04-16T19:32:31.159666+0000","last_fullsized":"2024-04-16T19:32:31.159666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153260+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:32:26.153260+0000","last_peered":"2024-04-16T19:32:26.153260+0000","last_clean":"2024-04-16T19:32:26.153260+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:32:26.153260+0000","last_undegraded":"2024-04-16T19:32:26.153260+0000","last_fullsized":"2024-04-16T19:32:26.153260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178113+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:33:41.178113+0000","last_peered":"2024-04-16T19:33:41.178113+0000","last_clean":"2024-04-16T19:33:41.178113+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:33:41.178113+0000","last_undegraded":"2024-04-16T19:33:41.178113+0000","last_fullsized":"2024-04-16T19:33:41.178113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'8","reported_seq":270,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135460+0000","last_change":"2024-04-16T17:47:55.163584+0000","last_active":"2024-04-16T19:32:06.135460+0000","last_peered":"2024-04-16T19:32:06.135460+0000","last_clean":"2024-04-16T19:32:06.135460+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:32:06.135460+0000","last_undegraded":"2024-04-16T19:32:06.135460+0000","last_fullsized":"2024-04-16T19:32:06.135460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:28.473272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159805+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:32:51.159805+0000","last_peered":"2024-04-16T19:32:51.159805+0000","last_clean":"2024-04-16T19:32:51.159805+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:32:51.159805+0000","last_undegraded":"2024-04-16T19:32:51.159805+0000","last_fullsized":"2024-04-16T19:32:51.159805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172776+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:33:41.172776+0000","last_peered":"2024-04-16T19:33:41.172776+0000","last_clean":"2024-04-16T19:33:41.172776+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:33:41.172776+0000","last_undegraded":"2024-04-16T19:33:41.172776+0000","last_fullsized":"2024-04-16T19:33:41.172776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:01.960480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"50'5","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135378+0000","last_change":"2024-04-16T17:47:55.164609+0000","last_active":"2024-04-16T19:32:06.135378+0000","last_peered":"2024-04-16T19:32:06.135378+0000","last_clean":"2024-04-16T19:32:06.135378+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:32:06.135378+0000","last_undegraded":"2024-04-16T19:32:06.135378+0000","last_fullsized":"2024-04-16T19:32:06.135378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:56.378878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158570+0000","last_change":"2024-04-16T17:47:23.254226+0000","last_active":"2024-04-16T19:33:46.158570+0000","last_peered":"2024-04-16T19:33:46.158570+0000","last_clean":"2024-04-16T19:33:46.158570+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:33:46.158570+0000","last_undegraded":"2024-04-16T19:33:46.158570+0000","last_fullsized":"2024-04-16T19:33:46.158570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:06.511950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171122+0000","last_change":"2024-04-16T17:47:32.284109+0000","last_active":"2024-04-16T19:33:41.171122+0000","last_peered":"2024-04-16T19:33:41.171122+0000","last_clean":"2024-04-16T19:33:41.171122+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:33:41.171122+0000","last_undegraded":"2024-04-16T19:33:41.171122+0000","last_fullsized":"2024-04-16T19:33:41.171122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'5","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166165+0000","last_change":"2024-04-16T17:47:55.172087+0000","last_active":"2024-04-16T19:32:56.166165+0000","last_peered":"2024-04-16T19:32:56.166165+0000","last_clean":"2024-04-16T19:32:56.166165+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:32:56.166165+0000","last_undegraded":"2024-04-16T19:32:56.166165+0000","last_fullsized":"2024-04-16T19:32:56.166165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158622+0000","last_change":"2024-04-16T17:47:23.266132+0000","last_active":"2024-04-16T19:33:46.158622+0000","last_peered":"2024-04-16T19:33:46.158622+0000","last_clean":"2024-04-16T19:33:46.158622+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:33:46.158622+0000","last_undegraded":"2024-04-16T19:33:46.158622+0000","last_fullsized":"2024-04-16T19:33:46.158622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:09.592666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.358504+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:31:31.358504+0000","last_peered":"2024-04-16T19:31:31.358504+0000","last_clean":"2024-04-16T19:31:31.358504+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:31:31.358504+0000","last_undegraded":"2024-04-16T19:31:31.358504+0000","last_fullsized":"2024-04-16T19:31:31.358504+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:02.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'10","reported_seq":265,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146504+0000","last_change":"2024-04-16T17:47:55.158785+0000","last_active":"2024-04-16T19:32:56.146504+0000","last_peered":"2024-04-16T19:32:56.146504+0000","last_clean":"2024-04-16T19:32:56.146504+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:32:56.146504+0000","last_undegraded":"2024-04-16T19:32:56.146504+0000","last_fullsized":"2024-04-16T19:32:56.146504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:39.387454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172430+0000","last_change":"2024-04-16T17:47:23.257289+0000","last_active":"2024-04-16T19:33:46.172430+0000","last_peered":"2024-04-16T19:33:46.172430+0000","last_clean":"2024-04-16T19:33:46.172430+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:33:46.172430+0000","last_undegraded":"2024-04-16T19:33:46.172430+0000","last_fullsized":"2024-04-16T19:33:46.172430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163511+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:32:56.163511+0000","last_peered":"2024-04-16T19:32:56.163511+0000","last_clean":"2024-04-16T19:32:56.163511+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:32:56.163511+0000","last_undegraded":"2024-04-16T19:32:56.163511+0000","last_fullsized":"2024-04-16T19:32:56.163511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:29:20.877336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151839+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:32:06.151839+0000","last_peered":"2024-04-16T19:32:06.151839+0000","last_clean":"2024-04-16T19:32:06.151839+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:32:06.151839+0000","last_undegraded":"2024-04-16T19:32:06.151839+0000","last_fullsized":"2024-04-16T19:32:06.151839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:23:28.148511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170388+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:33:11.170388+0000","last_peered":"2024-04-16T19:33:11.170388+0000","last_clean":"2024-04-16T19:33:11.170388+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:33:11.170388+0000","last_undegraded":"2024-04-16T19:33:11.170388+0000","last_fullsized":"2024-04-16T19:33:11.170388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175004+0000","last_change":"2024-04-16T17:48:27.308777+0000","last_active":"2024-04-16T19:33:31.175004+0000","last_peered":"2024-04-16T19:33:31.175004+0000","last_clean":"2024-04-16T19:33:31.175004+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:33:31.175004+0000","last_undegraded":"2024-04-16T19:33:31.175004+0000","last_fullsized":"2024-04-16T19:33:31.175004+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'9","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164889+0000","last_change":"2024-04-16T17:48:01.266271+0000","last_active":"2024-04-16T19:33:01.164889+0000","last_peered":"2024-04-16T19:33:01.164889+0000","last_clean":"2024-04-16T19:33:01.164889+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:33:01.164889+0000","last_undegraded":"2024-04-16T19:33:01.164889+0000","last_fullsized":"2024-04-16T19:33:01.164889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161254+0000","last_change":"2024-04-16T17:47:59.181005+0000","last_active":"2024-04-16T19:33:01.161254+0000","last_peered":"2024-04-16T19:33:01.161254+0000","last_clean":"2024-04-16T19:33:01.161254+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:33:01.161254+0000","last_undegraded":"2024-04-16T19:33:01.161254+0000","last_fullsized":"2024-04-16T19:33:01.161254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:34.728189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'547","reported_seq":799,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167615+0000","last_change":"2024-04-16T17:47:57.177580+0000","last_active":"2024-04-16T19:33:01.167615+0000","last_peered":"2024-04-16T19:33:01.167615+0000","last_clean":"2024-04-16T19:33:01.167615+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:33:01.167615+0000","last_undegraded":"2024-04-16T19:33:01.167615+0000","last_fullsized":"2024-04-16T19:33:01.167615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'7","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138635+0000","last_change":"2024-04-16T17:47:55.165841+0000","last_active":"2024-04-16T19:32:21.138635+0000","last_peered":"2024-04-16T19:32:21.138635+0000","last_clean":"2024-04-16T19:32:21.138635+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:32:21.138635+0000","last_undegraded":"2024-04-16T19:32:21.138635+0000","last_fullsized":"2024-04-16T19:32:21.138635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:07:11.773810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.139718+0000","last_change":"2024-04-16T17:47:39.326260+0000","last_active":"2024-04-16T19:31:36.139718+0000","last_peered":"2024-04-16T19:31:36.139718+0000","last_clean":"2024-04-16T19:31:36.139718+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:31:36.139718+0000","last_undegraded":"2024-04-16T19:31:36.139718+0000","last_fullsized":"2024-04-16T19:31:36.139718+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:58.230179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164460+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:32:56.164460+0000","last_peered":"2024-04-16T19:32:56.164460+0000","last_clean":"2024-04-16T19:32:56.164460+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:32:56.164460+0000","last_undegraded":"2024-04-16T19:32:56.164460+0000","last_fullsized":"2024-04-16T19:32:56.164460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159202+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:32:36.159202+0000","last_peered":"2024-04-16T19:32:36.159202+0000","last_clean":"2024-04-16T19:32:36.159202+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:32:36.159202+0000","last_undegraded":"2024-04-16T19:32:36.159202+0000","last_fullsized":"2024-04-16T19:32:36.159202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":768,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.370756+0000","last_change":"2024-04-16T17:47:33.303155+0000","last_active":"2024-04-16T19:31:31.370756+0000","last_peered":"2024-04-16T19:31:31.370756+0000","last_clean":"2024-04-16T19:31:31.370756+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:31:31.370756+0000","last_undegraded":"2024-04-16T19:31:31.370756+0000","last_fullsized":"2024-04-16T19:31:31.370756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:14.676250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367632+0000","last_change":"2024-04-16T17:47:35.314855+0000","last_active":"2024-04-16T19:31:31.367632+0000","last_peered":"2024-04-16T19:31:31.367632+0000","last_clean":"2024-04-16T19:31:31.367632+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:31:31.367632+0000","last_undegraded":"2024-04-16T19:31:31.367632+0000","last_fullsized":"2024-04-16T19:31:31.367632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140046+0000","last_change":"2024-04-16T17:47:37.332945+0000","last_active":"2024-04-16T19:31:36.140046+0000","last_peered":"2024-04-16T19:31:36.140046+0000","last_clean":"2024-04-16T19:31:36.140046+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:31:36.140046+0000","last_undegraded":"2024-04-16T19:31:36.140046+0000","last_fullsized":"2024-04-16T19:31:36.140046+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152563+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:33:21.152563+0000","last_peered":"2024-04-16T19:33:21.152563+0000","last_clean":"2024-04-16T19:33:21.152563+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:33:21.152563+0000","last_undegraded":"2024-04-16T19:33:21.152563+0000","last_fullsized":"2024-04-16T19:33:21.152563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:24.843140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'83","reported_seq":1420,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161248+0000","last_change":"2024-04-16T17:48:34.319654+0000","last_active":"2024-04-16T19:32:46.161248+0000","last_peered":"2024-04-16T19:32:46.161248+0000","last_clean":"2024-04-16T19:32:46.161248+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:32:46.161248+0000","last_undegraded":"2024-04-16T19:32:46.161248+0000","last_fullsized":"2024-04-16T19:32:46.161248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:13.391814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169963+0000","last_change":"2024-04-16T17:48:27.303999+0000","last_active":"2024-04-16T19:33:31.169963+0000","last_peered":"2024-04-16T19:33:31.169963+0000","last_clean":"2024-04-16T19:33:31.169963+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:33:31.169963+0000","last_undegraded":"2024-04-16T19:33:31.169963+0000","last_fullsized":"2024-04-16T19:33:31.169963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'5","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164928+0000","last_change":"2024-04-16T17:48:01.266699+0000","last_active":"2024-04-16T19:33:01.164928+0000","last_peered":"2024-04-16T19:33:01.164928+0000","last_clean":"2024-04-16T19:33:01.164928+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:33:01.164928+0000","last_undegraded":"2024-04-16T19:33:01.164928+0000","last_fullsized":"2024-04-16T19:33:01.164928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162337+0000","last_change":"2024-04-16T17:47:59.191306+0000","last_active":"2024-04-16T19:33:01.162337+0000","last_peered":"2024-04-16T19:33:01.162337+0000","last_clean":"2024-04-16T19:33:01.162337+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:33:01.162337+0000","last_undegraded":"2024-04-16T19:33:01.162337+0000","last_fullsized":"2024-04-16T19:33:01.162337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7326","reported_seq":20595,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147895+0000","last_change":"2024-04-16T17:47:57.179815+0000","last_active":"2024-04-16T19:33:01.147895+0000","last_peered":"2024-04-16T19:33:01.147895+0000","last_clean":"2024-04-16T19:33:01.147895+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:33:01.147895+0000","last_undegraded":"2024-04-16T19:33:01.147895+0000","last_fullsized":"2024-04-16T19:33:01.147895+0000","mapping_epoch":34,"log_start":"53'5500","ondisk_log_start":"53'5500","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1826,"log_dups_size":3000,"ondisk_log_size":1826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:36.013179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'19","reported_seq":354,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143591+0000","last_change":"2024-04-16T17:47:55.178555+0000","last_active":"2024-04-16T19:31:51.143591+0000","last_peered":"2024-04-16T19:31:51.143591+0000","last_clean":"2024-04-16T19:31:51.143591+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:31:51.143591+0000","last_undegraded":"2024-04-16T19:31:51.143591+0000","last_fullsized":"2024-04-16T19:31:51.143591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146395+0000","last_change":"2024-04-16T17:47:39.333211+0000","last_active":"2024-04-16T19:32:01.146395+0000","last_peered":"2024-04-16T19:32:01.146395+0000","last_clean":"2024-04-16T19:32:01.146395+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:32:01.146395+0000","last_undegraded":"2024-04-16T19:32:01.146395+0000","last_fullsized":"2024-04-16T19:32:01.146395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174179+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:33:46.174179+0000","last_peered":"2024-04-16T19:33:46.174179+0000","last_clean":"2024-04-16T19:33:46.174179+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:33:46.174179+0000","last_undegraded":"2024-04-16T19:33:46.174179+0000","last_fullsized":"2024-04-16T19:33:46.174179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154341+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:32:21.154341+0000","last_peered":"2024-04-16T19:32:21.154341+0000","last_clean":"2024-04-16T19:32:21.154341+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:32:21.154341+0000","last_undegraded":"2024-04-16T19:32:21.154341+0000","last_fullsized":"2024-04-16T19:32:21.154341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"48'5","reported_seq":359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167357+0000","last_change":"2024-04-16T17:47:33.310538+0000","last_active":"2024-04-16T19:33:11.167357+0000","last_peered":"2024-04-16T19:33:11.167357+0000","last_clean":"2024-04-16T19:33:11.167357+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:33:11.167357+0000","last_undegraded":"2024-04-16T19:33:11.167357+0000","last_fullsized":"2024-04-16T19:33:11.167357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'72","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361573+0000","last_change":"2024-04-16T17:47:35.305717+0000","last_active":"2024-04-16T19:31:31.361573+0000","last_peered":"2024-04-16T19:31:31.361573+0000","last_clean":"2024-04-16T19:31:31.361573+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:31:31.361573+0000","last_undegraded":"2024-04-16T19:31:31.361573+0000","last_fullsized":"2024-04-16T19:31:31.361573+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:27.152861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140105+0000","last_change":"2024-04-16T17:47:37.326918+0000","last_active":"2024-04-16T19:31:36.140105+0000","last_peered":"2024-04-16T19:31:36.140105+0000","last_clean":"2024-04-16T19:31:36.140105+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:31:36.140105+0000","last_undegraded":"2024-04-16T19:31:36.140105+0000","last_fullsized":"2024-04-16T19:31:36.140105+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:19.938869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166042+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:33:21.166042+0000","last_peered":"2024-04-16T19:33:21.166042+0000","last_clean":"2024-04-16T19:33:21.166042+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:33:21.166042+0000","last_undegraded":"2024-04-16T19:33:21.166042+0000","last_fullsized":"2024-04-16T19:33:21.166042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:35.987275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'56","reported_seq":876,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173425+0000","last_change":"2024-04-16T17:48:34.333949+0000","last_active":"2024-04-16T19:33:36.173425+0000","last_peered":"2024-04-16T19:33:36.173425+0000","last_clean":"2024-04-16T19:33:36.173425+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:33:36.173425+0000","last_undegraded":"2024-04-16T19:33:36.173425+0000","last_fullsized":"2024-04-16T19:33:36.173425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156598+0000","last_change":"2024-04-16T17:48:27.299066+0000","last_active":"2024-04-16T19:32:41.156598+0000","last_peered":"2024-04-16T19:32:41.156598+0000","last_clean":"2024-04-16T19:32:41.156598+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:32:41.156598+0000","last_undegraded":"2024-04-16T19:32:41.156598+0000","last_fullsized":"2024-04-16T19:32:41.156598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:12.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'16","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155319+0000","last_change":"2024-04-16T17:48:01.239365+0000","last_active":"2024-04-16T19:32:36.155319+0000","last_peered":"2024-04-16T19:32:36.155319+0000","last_clean":"2024-04-16T19:32:36.155319+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:32:36.155319+0000","last_undegraded":"2024-04-16T19:32:36.155319+0000","last_fullsized":"2024-04-16T19:32:36.155319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:31.169600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156956+0000","last_change":"2024-04-16T17:47:59.192949+0000","last_active":"2024-04-16T19:32:31.156956+0000","last_peered":"2024-04-16T19:32:31.156956+0000","last_clean":"2024-04-16T19:32:31.156956+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:32:31.156956+0000","last_undegraded":"2024-04-16T19:32:31.156956+0000","last_fullsized":"2024-04-16T19:32:31.156956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3595","reported_seq":7189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132670+0000","last_change":"2024-04-16T17:47:57.176567+0000","last_active":"2024-04-16T19:31:56.132670+0000","last_peered":"2024-04-16T19:31:56.132670+0000","last_clean":"2024-04-16T19:31:56.132670+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:31:56.132670+0000","last_undegraded":"2024-04-16T19:31:56.132670+0000","last_fullsized":"2024-04-16T19:31:56.132670+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1795,"log_dups_size":1800,"ondisk_log_size":1795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:10.013659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'9","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163598+0000","last_change":"2024-04-16T17:47:55.178252+0000","last_active":"2024-04-16T19:32:56.163598+0000","last_peered":"2024-04-16T19:32:56.163598+0000","last_clean":"2024-04-16T19:32:56.163598+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:32:56.163598+0000","last_undegraded":"2024-04-16T19:32:56.163598+0000","last_fullsized":"2024-04-16T19:32:56.163598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159596+0000","last_change":"2024-04-16T17:47:39.334187+0000","last_active":"2024-04-16T19:32:41.159596+0000","last_peered":"2024-04-16T19:32:41.159596+0000","last_clean":"2024-04-16T19:32:41.159596+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:32:41.159596+0000","last_undegraded":"2024-04-16T19:32:41.159596+0000","last_fullsized":"2024-04-16T19:32:41.159596+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176178+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:33:46.176178+0000","last_peered":"2024-04-16T19:33:46.176178+0000","last_clean":"2024-04-16T19:33:46.176178+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:33:46.176178+0000","last_undegraded":"2024-04-16T19:33:46.176178+0000","last_fullsized":"2024-04-16T19:33:46.176178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156989+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:32:21.156989+0000","last_peered":"2024-04-16T19:32:21.156989+0000","last_clean":"2024-04-16T19:32:21.156989+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:32:21.156989+0000","last_undegraded":"2024-04-16T19:32:21.156989+0000","last_fullsized":"2024-04-16T19:32:21.156989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"48'11","reported_seq":605,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.343972+0000","last_change":"2024-04-16T17:47:33.300240+0000","last_active":"2024-04-16T19:31:31.343972+0000","last_peered":"2024-04-16T19:31:31.343972+0000","last_clean":"2024-04-16T19:31:31.343972+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:31:31.343972+0000","last_undegraded":"2024-04-16T19:31:31.343972+0000","last_fullsized":"2024-04-16T19:31:31.343972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:30.896735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'120","reported_seq":290,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.149082+0000","last_change":"2024-04-16T17:47:35.314675+0000","last_active":"2024-04-16T19:33:06.149082+0000","last_peered":"2024-04-16T19:33:06.149082+0000","last_clean":"2024-04-16T19:33:06.149082+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:33:06.149082+0000","last_undegraded":"2024-04-16T19:33:06.149082+0000","last_fullsized":"2024-04-16T19:33:06.149082+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:43.962803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'12","reported_seq":1332,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144472+0000","last_change":"2024-04-16T17:47:37.333045+0000","last_active":"2024-04-16T19:31:36.144472+0000","last_peered":"2024-04-16T19:31:36.144472+0000","last_clean":"2024-04-16T19:31:36.144472+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:31:36.144472+0000","last_undegraded":"2024-04-16T19:31:36.144472+0000","last_fullsized":"2024-04-16T19:31:36.144472+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:40.632643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170471+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:33:21.170471+0000","last_peered":"2024-04-16T19:33:21.170471+0000","last_clean":"2024-04-16T19:33:21.170471+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:33:21.170471+0000","last_undegraded":"2024-04-16T19:33:21.170471+0000","last_fullsized":"2024-04-16T19:33:21.170471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:20.115696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173520+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:33:36.173520+0000","last_peered":"2024-04-16T19:33:36.173520+0000","last_clean":"2024-04-16T19:33:36.173520+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:33:36.173520+0000","last_undegraded":"2024-04-16T19:33:36.173520+0000","last_fullsized":"2024-04-16T19:33:36.173520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:58.360796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.162482+0000","last_change":"2024-04-16T17:48:27.297318+0000","last_active":"2024-04-16T19:33:06.162482+0000","last_peered":"2024-04-16T19:33:06.162482+0000","last_clean":"2024-04-16T19:33:06.162482+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:33:06.162482+0000","last_undegraded":"2024-04-16T19:33:06.162482+0000","last_fullsized":"2024-04-16T19:33:06.162482+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:20:02.104358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'19","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154557+0000","last_change":"2024-04-16T17:48:01.256079+0000","last_active":"2024-04-16T19:32:11.154557+0000","last_peered":"2024-04-16T19:32:11.154557+0000","last_clean":"2024-04-16T19:32:11.154557+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:32:11.154557+0000","last_undegraded":"2024-04-16T19:32:11.154557+0000","last_fullsized":"2024-04-16T19:32:11.154557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132631+0000","last_change":"2024-04-16T17:47:59.183129+0000","last_active":"2024-04-16T19:31:56.132631+0000","last_peered":"2024-04-16T19:31:56.132631+0000","last_clean":"2024-04-16T19:31:56.132631+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:31:56.132631+0000","last_undegraded":"2024-04-16T19:31:56.132631+0000","last_fullsized":"2024-04-16T19:31:56.132631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:48.953458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'385","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156502+0000","last_change":"2024-04-16T17:47:57.184276+0000","last_active":"2024-04-16T19:32:26.156502+0000","last_peered":"2024-04-16T19:32:26.156502+0000","last_clean":"2024-04-16T19:32:26.156502+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:32:26.156502+0000","last_undegraded":"2024-04-16T19:32:26.156502+0000","last_fullsized":"2024-04-16T19:32:26.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:59.219214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'6","reported_seq":173,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164223+0000","last_change":"2024-04-16T17:47:55.161902+0000","last_active":"2024-04-16T19:32:56.164223+0000","last_peered":"2024-04-16T19:32:56.164223+0000","last_clean":"2024-04-16T19:32:56.164223+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:32:56.164223+0000","last_undegraded":"2024-04-16T19:32:56.164223+0000","last_fullsized":"2024-04-16T19:32:56.164223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:47.909979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156406+0000","last_change":"2024-04-16T17:47:39.699810+0000","last_active":"2024-04-16T19:32:41.156406+0000","last_peered":"2024-04-16T19:32:41.156406+0000","last_clean":"2024-04-16T19:32:41.156406+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:32:41.156406+0000","last_undegraded":"2024-04-16T19:32:41.156406+0000","last_fullsized":"2024-04-16T19:32:41.156406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:40.561497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176260+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:33:46.176260+0000","last_peered":"2024-04-16T19:33:46.176260+0000","last_clean":"2024-04-16T19:33:46.176260+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:33:46.176260+0000","last_undegraded":"2024-04-16T19:33:46.176260+0000","last_fullsized":"2024-04-16T19:33:46.176260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151772+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:32:21.151772+0000","last_peered":"2024-04-16T19:32:21.151772+0000","last_clean":"2024-04-16T19:32:21.151772+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:32:21.151772+0000","last_undegraded":"2024-04-16T19:32:21.151772+0000","last_fullsized":"2024-04-16T19:32:21.151772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374413+0000","last_change":"2024-04-16T17:47:33.305749+0000","last_active":"2024-04-16T19:31:31.374413+0000","last_peered":"2024-04-16T19:31:31.374413+0000","last_clean":"2024-04-16T19:31:31.374413+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:31:31.374413+0000","last_undegraded":"2024-04-16T19:31:31.374413+0000","last_fullsized":"2024-04-16T19:31:31.374413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:15.902753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374347+0000","last_change":"2024-04-16T17:47:35.314832+0000","last_active":"2024-04-16T19:31:31.374347+0000","last_peered":"2024-04-16T19:31:31.374347+0000","last_clean":"2024-04-16T19:31:31.374347+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:31:31.374347+0000","last_undegraded":"2024-04-16T19:31:31.374347+0000","last_fullsized":"2024-04-16T19:31:31.374347+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:36.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'12","reported_seq":1334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168147+0000","last_change":"2024-04-16T17:47:37.334246+0000","last_active":"2024-04-16T19:33:11.168147+0000","last_peered":"2024-04-16T19:33:11.168147+0000","last_clean":"2024-04-16T19:33:11.168147+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:33:11.168147+0000","last_undegraded":"2024-04-16T19:33:11.168147+0000","last_fullsized":"2024-04-16T19:33:11.168147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:21.558307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147648+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:32:06.147648+0000","last_peered":"2024-04-16T19:32:06.147648+0000","last_clean":"2024-04-16T19:32:06.147648+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:32:06.147648+0000","last_undegraded":"2024-04-16T19:32:06.147648+0000","last_fullsized":"2024-04-16T19:32:06.147648+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:13:48.347861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'164","reported_seq":1853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163697+0000","last_change":"2024-04-16T17:48:34.332673+0000","last_active":"2024-04-16T19:32:46.163697+0000","last_peered":"2024-04-16T19:32:46.163697+0000","last_clean":"2024-04-16T19:32:46.163697+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:32:46.163697+0000","last_undegraded":"2024-04-16T19:32:46.163697+0000","last_fullsized":"2024-04-16T19:32:46.163697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162327+0000","last_change":"2024-04-16T17:48:27.306930+0000","last_active":"2024-04-16T19:32:41.162327+0000","last_peered":"2024-04-16T19:32:41.162327+0000","last_clean":"2024-04-16T19:32:41.162327+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:32:41.162327+0000","last_undegraded":"2024-04-16T19:32:41.162327+0000","last_fullsized":"2024-04-16T19:32:41.162327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'10","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.154948+0000","last_change":"2024-04-16T17:48:01.252700+0000","last_active":"2024-04-16T19:32:36.154948+0000","last_peered":"2024-04-16T19:32:36.154948+0000","last_clean":"2024-04-16T19:32:36.154948+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:32:36.154948+0000","last_undegraded":"2024-04-16T19:32:36.154948+0000","last_fullsized":"2024-04-16T19:32:36.154948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:40.367478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146809+0000","last_change":"2024-04-16T17:47:59.200630+0000","last_active":"2024-04-16T19:32:01.146809+0000","last_peered":"2024-04-16T19:32:01.146809+0000","last_clean":"2024-04-16T19:32:01.146809+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:32:01.146809+0000","last_undegraded":"2024-04-16T19:32:01.146809+0000","last_fullsized":"2024-04-16T19:32:01.146809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1087","reported_seq":1742,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146730+0000","last_change":"2024-04-16T17:47:57.186883+0000","last_active":"2024-04-16T19:32:01.146730+0000","last_peered":"2024-04-16T19:32:01.146730+0000","last_clean":"2024-04-16T19:32:01.146730+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:32:01.146730+0000","last_undegraded":"2024-04-16T19:32:01.146730+0000","last_fullsized":"2024-04-16T19:32:01.146730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:14.402693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"51'4","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159908+0000","last_change":"2024-04-16T17:47:55.177129+0000","last_active":"2024-04-16T19:32:41.159908+0000","last_peered":"2024-04-16T19:32:41.159908+0000","last_clean":"2024-04-16T19:32:41.159908+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:32:41.159908+0000","last_undegraded":"2024-04-16T19:32:41.159908+0000","last_fullsized":"2024-04-16T19:32:41.159908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:31.401461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176098+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:33:36.176098+0000","last_peered":"2024-04-16T19:33:36.176098+0000","last_clean":"2024-04-16T19:33:36.176098+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:33:36.176098+0000","last_undegraded":"2024-04-16T19:33:36.176098+0000","last_fullsized":"2024-04-16T19:33:36.176098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.170137+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:33:31.170137+0000","last_peered":"2024-04-16T19:33:31.170137+0000","last_clean":"2024-04-16T19:33:31.170137+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:33:31.170137+0000","last_undegraded":"2024-04-16T19:33:31.170137+0000","last_fullsized":"2024-04-16T19:33:31.170137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.173925+0000","last_change":"2024-04-16T17:47:33.309390+0000","last_active":"2024-04-16T19:33:46.173925+0000","last_peered":"2024-04-16T19:33:46.173925+0000","last_clean":"2024-04-16T19:33:46.173925+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:33:46.173925+0000","last_undegraded":"2024-04-16T19:33:46.173925+0000","last_fullsized":"2024-04-16T19:33:46.173925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'200","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146472+0000","last_change":"2024-04-16T17:47:35.314414+0000","last_active":"2024-04-16T19:32:01.146472+0000","last_peered":"2024-04-16T19:32:01.146472+0000","last_clean":"2024-04-16T19:32:01.146472+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:32:01.146472+0000","last_undegraded":"2024-04-16T19:32:01.146472+0000","last_fullsized":"2024-04-16T19:32:01.146472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171681+0000","last_change":"2024-04-16T17:47:37.335122+0000","last_active":"2024-04-16T19:33:26.171681+0000","last_peered":"2024-04-16T19:33:26.171681+0000","last_clean":"2024-04-16T19:33:26.171681+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:33:26.171681+0000","last_undegraded":"2024-04-16T19:33:26.171681+0000","last_fullsized":"2024-04-16T19:33:26.171681+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:38.549732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176783+0000","last_change":"2024-04-16T17:47:39.339833+0000","last_active":"2024-04-16T19:33:56.176783+0000","last_peered":"2024-04-16T19:33:56.176783+0000","last_clean":"2024-04-16T19:33:56.176783+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:33:56.176783+0000","last_undegraded":"2024-04-16T19:33:56.176783+0000","last_fullsized":"2024-04-16T19:33:56.176783+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135246+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:32:06.135246+0000","last_peered":"2024-04-16T19:32:06.135246+0000","last_clean":"2024-04-16T19:32:06.135246+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:32:06.135246+0000","last_undegraded":"2024-04-16T19:32:06.135246+0000","last_fullsized":"2024-04-16T19:32:06.135246+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:13:30.436128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1187,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161075+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:32:46.161075+0000","last_peered":"2024-04-16T19:32:46.161075+0000","last_clean":"2024-04-16T19:32:46.161075+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:32:46.161075+0000","last_undegraded":"2024-04-16T19:32:46.161075+0000","last_fullsized":"2024-04-16T19:32:46.161075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168743+0000","last_change":"2024-04-16T17:48:27.299718+0000","last_active":"2024-04-16T19:33:06.168743+0000","last_peered":"2024-04-16T19:33:06.168743+0000","last_clean":"2024-04-16T19:33:06.168743+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:33:06.168743+0000","last_undegraded":"2024-04-16T19:33:06.168743+0000","last_fullsized":"2024-04-16T19:33:06.168743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'2","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154067+0000","last_change":"2024-04-16T17:48:01.253979+0000","last_active":"2024-04-16T19:32:11.154067+0000","last_peered":"2024-04-16T19:32:11.154067+0000","last_clean":"2024-04-16T19:32:11.154067+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:32:11.154067+0000","last_undegraded":"2024-04-16T19:32:11.154067+0000","last_fullsized":"2024-04-16T19:32:11.154067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149567+0000","last_change":"2024-04-16T17:47:59.193745+0000","last_active":"2024-04-16T19:32:01.149567+0000","last_peered":"2024-04-16T19:32:01.149567+0000","last_clean":"2024-04-16T19:32:01.149567+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:32:01.149567+0000","last_undegraded":"2024-04-16T19:32:01.149567+0000","last_fullsized":"2024-04-16T19:32:01.149567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1173","reported_seq":1516,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151419+0000","last_change":"2024-04-16T17:47:57.191812+0000","last_active":"2024-04-16T19:32:01.151419+0000","last_peered":"2024-04-16T19:32:01.151419+0000","last_clean":"2024-04-16T19:32:01.151419+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:32:01.151419+0000","last_undegraded":"2024-04-16T19:32:01.151419+0000","last_fullsized":"2024-04-16T19:32:01.151419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1173,"log_dups_size":0,"ondisk_log_size":1173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'14","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151697+0000","last_change":"2024-04-16T17:47:55.163005+0000","last_active":"2024-04-16T19:32:06.151697+0000","last_peered":"2024-04-16T19:32:06.151697+0000","last_clean":"2024-04-16T19:32:06.151697+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:32:06.151697+0000","last_undegraded":"2024-04-16T19:32:06.151697+0000","last_fullsized":"2024-04-16T19:32:06.151697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169773+0000","last_change":"2024-04-16T17:47:23.266202+0000","last_active":"2024-04-16T19:33:36.169773+0000","last_peered":"2024-04-16T19:33:36.169773+0000","last_clean":"2024-04-16T19:33:36.169773+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:33:36.169773+0000","last_undegraded":"2024-04-16T19:33:36.169773+0000","last_fullsized":"2024-04-16T19:33:36.169773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163890+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:33:06.163890+0000","last_peered":"2024-04-16T19:33:06.163890+0000","last_clean":"2024-04-16T19:33:06.163890+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:33:06.163890+0000","last_undegraded":"2024-04-16T19:33:06.163890+0000","last_fullsized":"2024-04-16T19:33:06.163890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:45.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"31'1","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158405+0000","last_change":"2024-04-16T17:47:33.302299+0000","last_active":"2024-04-16T19:33:46.158405+0000","last_peered":"2024-04-16T19:33:46.158405+0000","last_clean":"2024-04-16T19:33:46.158405+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:33:46.158405+0000","last_undegraded":"2024-04-16T19:33:46.158405+0000","last_fullsized":"2024-04-16T19:33:46.158405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:23.313435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'80","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170395+0000","last_change":"2024-04-16T17:47:35.321698+0000","last_active":"2024-04-16T19:33:21.170395+0000","last_peered":"2024-04-16T19:33:21.170395+0000","last_clean":"2024-04-16T19:33:21.170395+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:33:21.170395+0000","last_undegraded":"2024-04-16T19:33:21.170395+0000","last_fullsized":"2024-04-16T19:33:21.170395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:02.350103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136445+0000","last_change":"2024-04-16T17:47:37.325461+0000","last_active":"2024-04-16T19:32:11.136445+0000","last_peered":"2024-04-16T19:32:11.136445+0000","last_clean":"2024-04-16T19:32:11.136445+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:32:11.136445+0000","last_undegraded":"2024-04-16T19:32:11.136445+0000","last_fullsized":"2024-04-16T19:32:11.136445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:18.621208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152703+0000","last_change":"2024-04-16T17:47:39.323337+0000","last_active":"2024-04-16T19:32:26.152703+0000","last_peered":"2024-04-16T19:32:26.152703+0000","last_clean":"2024-04-16T19:32:26.152703+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:32:26.152703+0000","last_undegraded":"2024-04-16T19:32:26.152703+0000","last_fullsized":"2024-04-16T19:32:26.152703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:12.655890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152382+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:32:06.152382+0000","last_peered":"2024-04-16T19:32:06.152382+0000","last_clean":"2024-04-16T19:32:06.152382+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:32:06.152382+0000","last_undegraded":"2024-04-16T19:32:06.152382+0000","last_fullsized":"2024-04-16T19:32:06.152382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:58.661997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'70","reported_seq":901,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.148912+0000","last_change":"2024-04-16T17:48:34.345575+0000","last_active":"2024-04-16T19:33:06.148912+0000","last_peered":"2024-04-16T19:33:06.148912+0000","last_clean":"2024-04-16T19:33:06.148912+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:33:06.148912+0000","last_undegraded":"2024-04-16T19:33:06.148912+0000","last_fullsized":"2024-04-16T19:33:06.148912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:51.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159686+0000","last_change":"2024-04-16T17:48:27.295648+0000","last_active":"2024-04-16T19:32:41.159686+0000","last_peered":"2024-04-16T19:32:41.159686+0000","last_clean":"2024-04-16T19:32:41.159686+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:32:41.159686+0000","last_undegraded":"2024-04-16T19:32:41.159686+0000","last_fullsized":"2024-04-16T19:32:41.159686+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154259+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:32:11.154259+0000","last_peered":"2024-04-16T19:32:11.154259+0000","last_clean":"2024-04-16T19:32:11.154259+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:32:11.154259+0000","last_undegraded":"2024-04-16T19:32:11.154259+0000","last_fullsized":"2024-04-16T19:32:11.154259+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:20.025368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.157655+0000","last_change":"2024-04-16T17:47:59.197037+0000","last_active":"2024-04-16T19:32:46.157655+0000","last_peered":"2024-04-16T19:32:46.157655+0000","last_clean":"2024-04-16T19:32:46.157655+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:32:46.157655+0000","last_undegraded":"2024-04-16T19:32:46.157655+0000","last_fullsized":"2024-04-16T19:32:46.157655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:54.818691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1049","reported_seq":6110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152019+0000","last_change":"2024-04-16T17:47:57.189817+0000","last_active":"2024-04-16T19:32:11.152019+0000","last_peered":"2024-04-16T19:32:11.152019+0000","last_clean":"2024-04-16T19:32:11.152019+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:32:11.152019+0000","last_undegraded":"2024-04-16T19:32:11.152019+0000","last_fullsized":"2024-04-16T19:32:11.152019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'4","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145438+0000","last_change":"2024-04-16T17:47:55.182984+0000","last_active":"2024-04-16T19:31:56.145438+0000","last_peered":"2024-04-16T19:31:56.145438+0000","last_clean":"2024-04-16T19:31:56.145438+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:31:56.145438+0000","last_undegraded":"2024-04-16T19:31:56.145438+0000","last_fullsized":"2024-04-16T19:31:56.145438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:55.129164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178050+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:34:01.178050+0000","last_peered":"2024-04-16T19:34:01.178050+0000","last_clean":"2024-04-16T19:34:01.178050+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:34:01.178050+0000","last_undegraded":"2024-04-16T19:34:01.178050+0000","last_fullsized":"2024-04-16T19:34:01.178050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.169031+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:33:06.169031+0000","last_peered":"2024-04-16T19:33:06.169031+0000","last_clean":"2024-04-16T19:33:06.169031+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:33:06.169031+0000","last_undegraded":"2024-04-16T19:33:06.169031+0000","last_fullsized":"2024-04-16T19:33:06.169031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152471+0000","last_change":"2024-04-16T17:47:33.302725+0000","last_active":"2024-04-16T19:33:21.152471+0000","last_peered":"2024-04-16T19:33:21.152471+0000","last_clean":"2024-04-16T19:33:21.152471+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:33:21.152471+0000","last_undegraded":"2024-04-16T19:33:21.152471+0000","last_fullsized":"2024-04-16T19:33:21.152471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:42.697302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'152","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179125+0000","last_change":"2024-04-16T17:47:35.317742+0000","last_active":"2024-04-16T19:33:46.179125+0000","last_peered":"2024-04-16T19:33:46.179125+0000","last_clean":"2024-04-16T19:33:46.179125+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:33:46.179125+0000","last_undegraded":"2024-04-16T19:33:46.179125+0000","last_fullsized":"2024-04-16T19:33:46.179125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'12","reported_seq":1338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177186+0000","last_change":"2024-04-16T17:47:37.318010+0000","last_active":"2024-04-16T19:33:51.177186+0000","last_peered":"2024-04-16T19:33:51.177186+0000","last_clean":"2024-04-16T19:33:51.177186+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:33:51.177186+0000","last_undegraded":"2024-04-16T19:33:51.177186+0000","last_fullsized":"2024-04-16T19:33:51.177186+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177479+0000","last_change":"2024-04-16T17:47:39.328909+0000","last_active":"2024-04-16T19:33:51.177479+0000","last_peered":"2024-04-16T19:33:51.177479+0000","last_clean":"2024-04-16T19:33:51.177479+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:33:51.177479+0000","last_undegraded":"2024-04-16T19:33:51.177479+0000","last_fullsized":"2024-04-16T19:33:51.177479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:54.351646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152331+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:33:21.152331+0000","last_peered":"2024-04-16T19:33:21.152331+0000","last_clean":"2024-04-16T19:33:21.152331+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:33:21.152331+0000","last_undegraded":"2024-04-16T19:33:21.152331+0000","last_fullsized":"2024-04-16T19:33:21.152331+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:16.718022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'74","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161693+0000","last_change":"2024-04-16T17:48:34.327024+0000","last_active":"2024-04-16T19:32:46.161693+0000","last_peered":"2024-04-16T19:32:46.161693+0000","last_clean":"2024-04-16T19:32:46.161693+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:32:46.161693+0000","last_undegraded":"2024-04-16T19:32:46.161693+0000","last_fullsized":"2024-04-16T19:32:46.161693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:00.683240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157359+0000","last_change":"2024-04-16T17:48:27.301787+0000","last_active":"2024-04-16T19:32:41.157359+0000","last_peered":"2024-04-16T19:32:41.157359+0000","last_clean":"2024-04-16T19:32:41.157359+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:32:41.157359+0000","last_undegraded":"2024-04-16T19:32:41.157359+0000","last_fullsized":"2024-04-16T19:32:41.157359+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'17","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.151912+0000","last_change":"2024-04-16T17:48:01.243297+0000","last_active":"2024-04-16T19:32:11.151912+0000","last_peered":"2024-04-16T19:32:11.151912+0000","last_clean":"2024-04-16T19:32:11.151912+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:32:11.151912+0000","last_undegraded":"2024-04-16T19:32:11.151912+0000","last_fullsized":"2024-04-16T19:32:11.151912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149205+0000","last_change":"2024-04-16T17:47:59.193938+0000","last_active":"2024-04-16T19:32:11.149205+0000","last_peered":"2024-04-16T19:32:11.149205+0000","last_clean":"2024-04-16T19:32:11.149205+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:32:11.149205+0000","last_undegraded":"2024-04-16T19:32:11.149205+0000","last_fullsized":"2024-04-16T19:32:11.149205+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'703","reported_seq":1434,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.143345+0000","last_change":"2024-04-16T17:47:57.173194+0000","last_active":"2024-04-16T19:32:41.143345+0000","last_peered":"2024-04-16T19:32:41.143345+0000","last_clean":"2024-04-16T19:32:41.143345+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:32:41.143345+0000","last_undegraded":"2024-04-16T19:32:41.143345+0000","last_fullsized":"2024-04-16T19:32:41.143345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:02.892691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'14","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145388+0000","last_change":"2024-04-16T17:47:55.181545+0000","last_active":"2024-04-16T19:31:56.145388+0000","last_peered":"2024-04-16T19:31:56.145388+0000","last_clean":"2024-04-16T19:31:56.145388+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:31:56.145388+0000","last_undegraded":"2024-04-16T19:31:56.145388+0000","last_fullsized":"2024-04-16T19:31:56.145388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167271+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:33:11.167271+0000","last_peered":"2024-04-16T19:33:11.167271+0000","last_clean":"2024-04-16T19:33:11.167271+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:33:11.167271+0000","last_undegraded":"2024-04-16T19:33:11.167271+0000","last_fullsized":"2024-04-16T19:33:11.167271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163761+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:33:06.163761+0000","last_peered":"2024-04-16T19:33:06.163761+0000","last_clean":"2024-04-16T19:33:06.163761+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:33:06.163761+0000","last_undegraded":"2024-04-16T19:33:06.163761+0000","last_fullsized":"2024-04-16T19:33:06.163761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135219+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:32:06.135219+0000","last_peered":"2024-04-16T19:32:06.135219+0000","last_clean":"2024-04-16T19:32:06.135219+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:32:06.135219+0000","last_undegraded":"2024-04-16T19:32:06.135219+0000","last_fullsized":"2024-04-16T19:32:06.135219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:21.347240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'88","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175899+0000","last_change":"2024-04-16T17:47:35.317589+0000","last_active":"2024-04-16T19:33:46.175899+0000","last_peered":"2024-04-16T19:33:46.175899+0000","last_clean":"2024-04-16T19:33:46.175899+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:33:46.175899+0000","last_undegraded":"2024-04-16T19:33:46.175899+0000","last_fullsized":"2024-04-16T19:33:46.175899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159758+0000","last_change":"2024-04-16T17:47:37.330541+0000","last_active":"2024-04-16T19:33:51.159758+0000","last_peered":"2024-04-16T19:33:51.159758+0000","last_clean":"2024-04-16T19:33:51.159758+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:33:51.159758+0000","last_undegraded":"2024-04-16T19:33:51.159758+0000","last_fullsized":"2024-04-16T19:33:51.159758+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:07.207646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178646+0000","last_change":"2024-04-16T17:47:39.338681+0000","last_active":"2024-04-16T19:33:51.178646+0000","last_peered":"2024-04-16T19:33:51.178646+0000","last_clean":"2024-04-16T19:33:51.178646+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:33:51.178646+0000","last_undegraded":"2024-04-16T19:33:51.178646+0000","last_fullsized":"2024-04-16T19:33:51.178646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168291+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:33:11.168291+0000","last_peered":"2024-04-16T19:33:11.168291+0000","last_clean":"2024-04-16T19:33:11.168291+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:33:11.168291+0000","last_undegraded":"2024-04-16T19:33:11.168291+0000","last_fullsized":"2024-04-16T19:33:11.168291+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:26.664630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'40","reported_seq":973,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158409+0000","last_change":"2024-04-16T17:48:34.333313+0000","last_active":"2024-04-16T19:32:36.158409+0000","last_peered":"2024-04-16T19:32:36.158409+0000","last_clean":"2024-04-16T19:32:36.158409+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:32:36.158409+0000","last_undegraded":"2024-04-16T19:32:36.158409+0000","last_fullsized":"2024-04-16T19:32:36.158409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:06:58.890753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164925+0000","last_change":"2024-04-16T17:48:27.304408+0000","last_active":"2024-04-16T19:33:16.164925+0000","last_peered":"2024-04-16T19:33:16.164925+0000","last_clean":"2024-04-16T19:33:16.164925+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:33:16.164925+0000","last_undegraded":"2024-04-16T19:33:16.164925+0000","last_fullsized":"2024-04-16T19:33:16.164925+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:27:10.178784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'9","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149161+0000","last_change":"2024-04-16T17:48:01.259988+0000","last_active":"2024-04-16T19:32:11.149161+0000","last_peered":"2024-04-16T19:32:11.149161+0000","last_clean":"2024-04-16T19:32:11.149161+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:32:11.149161+0000","last_undegraded":"2024-04-16T19:32:11.149161+0000","last_fullsized":"2024-04-16T19:32:11.149161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'555","reported_seq":2287,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154329+0000","last_change":"2024-04-16T17:47:59.181942+0000","last_active":"2024-04-16T19:32:11.154329+0000","last_peered":"2024-04-16T19:32:11.154329+0000","last_clean":"2024-04-16T19:32:11.154329+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:32:11.154329+0000","last_undegraded":"2024-04-16T19:32:11.154329+0000","last_fullsized":"2024-04-16T19:32:11.154329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:25.688721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'901","reported_seq":1313,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152857+0000","last_change":"2024-04-16T17:47:57.174998+0000","last_active":"2024-04-16T19:32:11.152857+0000","last_peered":"2024-04-16T19:32:11.152857+0000","last_clean":"2024-04-16T19:32:11.152857+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:32:11.152857+0000","last_undegraded":"2024-04-16T19:32:11.152857+0000","last_fullsized":"2024-04-16T19:32:11.152857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:14.325870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'14","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150617+0000","last_change":"2024-04-16T17:47:55.155349+0000","last_active":"2024-04-16T19:32:06.150617+0000","last_peered":"2024-04-16T19:32:06.150617+0000","last_clean":"2024-04-16T19:32:06.150617+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:32:06.150617+0000","last_undegraded":"2024-04-16T19:32:06.150617+0000","last_fullsized":"2024-04-16T19:32:06.150617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163638+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:33:11.163638+0000","last_peered":"2024-04-16T19:33:11.163638+0000","last_clean":"2024-04-16T19:33:11.163638+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:33:11.163638+0000","last_undegraded":"2024-04-16T19:33:11.163638+0000","last_fullsized":"2024-04-16T19:33:11.163638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166835+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:33:06.166835+0000","last_peered":"2024-04-16T19:33:06.166835+0000","last_clean":"2024-04-16T19:33:06.166835+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:33:06.166835+0000","last_undegraded":"2024-04-16T19:33:06.166835+0000","last_fullsized":"2024-04-16T19:33:06.166835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":175,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152538+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:33:21.152538+0000","last_peered":"2024-04-16T19:33:21.152538+0000","last_clean":"2024-04-16T19:33:21.152538+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:33:21.152538+0000","last_undegraded":"2024-04-16T19:33:21.152538+0000","last_fullsized":"2024-04-16T19:33:21.152538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:05.646107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'104","reported_seq":273,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166177+0000","last_change":"2024-04-16T17:47:35.307146+0000","last_active":"2024-04-16T19:33:21.166177+0000","last_peered":"2024-04-16T19:33:21.166177+0000","last_clean":"2024-04-16T19:33:21.166177+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:33:21.166177+0000","last_undegraded":"2024-04-16T19:33:21.166177+0000","last_fullsized":"2024-04-16T19:33:21.166177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:28.729299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145847+0000","last_change":"2024-04-16T17:47:37.324514+0000","last_active":"2024-04-16T19:31:46.145847+0000","last_peered":"2024-04-16T19:31:46.145847+0000","last_clean":"2024-04-16T19:31:46.145847+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:31:46.145847+0000","last_undegraded":"2024-04-16T19:31:46.145847+0000","last_fullsized":"2024-04-16T19:31:46.145847+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183327+0000","last_change":"2024-04-16T17:47:39.340900+0000","last_active":"2024-04-16T19:34:11.183327+0000","last_peered":"2024-04-16T19:34:11.183327+0000","last_clean":"2024-04-16T19:34:11.183327+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:34:11.183327+0000","last_undegraded":"2024-04-16T19:34:11.183327+0000","last_fullsized":"2024-04-16T19:34:11.183327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.644408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167437+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:33:11.167437+0000","last_peered":"2024-04-16T19:33:11.167437+0000","last_clean":"2024-04-16T19:33:11.167437+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:33:11.167437+0000","last_undegraded":"2024-04-16T19:33:11.167437+0000","last_fullsized":"2024-04-16T19:33:11.167437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156402+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:32:36.156402+0000","last_peered":"2024-04-16T19:32:36.156402+0000","last_clean":"2024-04-16T19:32:36.156402+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:32:36.156402+0000","last_undegraded":"2024-04-16T19:32:36.156402+0000","last_fullsized":"2024-04-16T19:32:36.156402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157486+0000","last_change":"2024-04-16T17:48:27.308234+0000","last_active":"2024-04-16T19:32:41.157486+0000","last_peered":"2024-04-16T19:32:41.157486+0000","last_clean":"2024-04-16T19:32:41.157486+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:32:41.157486+0000","last_undegraded":"2024-04-16T19:32:41.157486+0000","last_fullsized":"2024-04-16T19:32:41.157486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'135","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163865+0000","last_change":"2024-04-16T17:48:01.254153+0000","last_active":"2024-04-16T19:32:46.163865+0000","last_peered":"2024-04-16T19:32:46.163865+0000","last_clean":"2024-04-16T19:32:46.163865+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:32:46.163865+0000","last_undegraded":"2024-04-16T19:32:46.163865+0000","last_fullsized":"2024-04-16T19:32:46.163865+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136391+0000","last_change":"2024-04-16T17:47:59.174180+0000","last_active":"2024-04-16T19:32:11.136391+0000","last_peered":"2024-04-16T19:32:11.136391+0000","last_clean":"2024-04-16T19:32:11.136391+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:32:11.136391+0000","last_undegraded":"2024-04-16T19:32:11.136391+0000","last_fullsized":"2024-04-16T19:32:11.136391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:29.135288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'679","reported_seq":1066,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136485+0000","last_change":"2024-04-16T17:47:57.182491+0000","last_active":"2024-04-16T19:32:11.136485+0000","last_peered":"2024-04-16T19:32:11.136485+0000","last_clean":"2024-04-16T19:32:11.136485+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:32:11.136485+0000","last_undegraded":"2024-04-16T19:32:11.136485+0000","last_fullsized":"2024-04-16T19:32:11.136485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:14.757156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'6","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156433+0000","last_change":"2024-04-16T17:47:55.158851+0000","last_active":"2024-04-16T19:32:26.156433+0000","last_peered":"2024-04-16T19:32:26.156433+0000","last_clean":"2024-04-16T19:32:26.156433+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:32:26.156433+0000","last_undegraded":"2024-04-16T19:32:26.156433+0000","last_fullsized":"2024-04-16T19:32:26.156433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:39:56.856045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170332+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:33:11.170332+0000","last_peered":"2024-04-16T19:33:11.170332+0000","last_clean":"2024-04-16T19:33:11.170332+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:33:11.170332+0000","last_undegraded":"2024-04-16T19:33:11.170332+0000","last_fullsized":"2024-04-16T19:33:11.170332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:31.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.149879+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:31:56.149879+0000","last_peered":"2024-04-16T19:31:56.149879+0000","last_clean":"2024-04-16T19:31:56.149879+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:31:56.149879+0000","last_undegraded":"2024-04-16T19:31:56.149879+0000","last_fullsized":"2024-04-16T19:31:56.149879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:06.495990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"48'13","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170506+0000","last_change":"2024-04-16T17:47:33.309159+0000","last_active":"2024-04-16T19:33:21.170506+0000","last_peered":"2024-04-16T19:33:21.170506+0000","last_clean":"2024-04-16T19:33:21.170506+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:33:21.170506+0000","last_undegraded":"2024-04-16T19:33:21.170506+0000","last_fullsized":"2024-04-16T19:33:21.170506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:26.709263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'152","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160039+0000","last_change":"2024-04-16T17:47:35.305022+0000","last_active":"2024-04-16T19:33:51.160039+0000","last_peered":"2024-04-16T19:33:51.160039+0000","last_clean":"2024-04-16T19:33:51.160039+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:33:51.160039+0000","last_undegraded":"2024-04-16T19:33:51.160039+0000","last_fullsized":"2024-04-16T19:33:51.160039+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:03.908445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.153692+0000","last_change":"2024-04-16T17:47:37.331019+0000","last_active":"2024-04-16T19:33:26.153692+0000","last_peered":"2024-04-16T19:33:26.153692+0000","last_clean":"2024-04-16T19:33:26.153692+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:33:26.153692+0000","last_undegraded":"2024-04-16T19:33:26.153692+0000","last_fullsized":"2024-04-16T19:33:26.153692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:18.849817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179728+0000","last_change":"2024-04-16T17:47:39.331395+0000","last_active":"2024-04-16T19:33:56.179728+0000","last_peered":"2024-04-16T19:33:56.179728+0000","last_clean":"2024-04-16T19:33:56.179728+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:33:56.179728+0000","last_undegraded":"2024-04-16T19:33:56.179728+0000","last_fullsized":"2024-04-16T19:33:56.179728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:12.149771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160965+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:32:36.160965+0000","last_peered":"2024-04-16T19:32:36.160965+0000","last_clean":"2024-04-16T19:32:36.160965+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:32:36.160965+0000","last_undegraded":"2024-04-16T19:32:36.160965+0000","last_fullsized":"2024-04-16T19:32:36.160965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'86","reported_seq":1117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163783+0000","last_change":"2024-04-16T17:48:34.326754+0000","last_active":"2024-04-16T19:32:46.163783+0000","last_peered":"2024-04-16T19:32:46.163783+0000","last_clean":"2024-04-16T19:32:46.163783+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:32:46.163783+0000","last_undegraded":"2024-04-16T19:32:46.163783+0000","last_fullsized":"2024-04-16T19:32:46.163783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159819+0000","last_change":"2024-04-16T17:48:27.298815+0000","last_active":"2024-04-16T19:32:31.159819+0000","last_peered":"2024-04-16T19:32:31.159819+0000","last_clean":"2024-04-16T19:32:31.159819+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:32:31.159819+0000","last_undegraded":"2024-04-16T19:32:31.159819+0000","last_fullsized":"2024-04-16T19:32:31.159819+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"48'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150226+0000","last_change":"2024-04-16T17:48:01.246604+0000","last_active":"2024-04-16T19:32:01.150226+0000","last_peered":"2024-04-16T19:32:01.150226+0000","last_clean":"2024-04-16T19:32:01.150226+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:32:01.150226+0000","last_undegraded":"2024-04-16T19:32:01.150226+0000","last_fullsized":"2024-04-16T19:32:01.150226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:16.195310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148847+0000","last_change":"2024-04-16T17:47:59.180320+0000","last_active":"2024-04-16T19:32:11.148847+0000","last_peered":"2024-04-16T19:32:11.148847+0000","last_clean":"2024-04-16T19:32:11.148847+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:32:11.148847+0000","last_undegraded":"2024-04-16T19:32:11.148847+0000","last_fullsized":"2024-04-16T19:32:11.148847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:16.107500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1483","reported_seq":5477,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153145+0000","last_change":"2024-04-16T17:47:57.186127+0000","last_active":"2024-04-16T19:32:26.153145+0000","last_peered":"2024-04-16T19:32:26.153145+0000","last_clean":"2024-04-16T19:32:26.153145+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:32:26.153145+0000","last_undegraded":"2024-04-16T19:32:26.153145+0000","last_fullsized":"2024-04-16T19:32:26.153145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1483,"log_dups_size":0,"ondisk_log_size":1483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:02.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'8","reported_seq":224,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152628+0000","last_change":"2024-04-16T17:47:55.161973+0000","last_active":"2024-04-16T19:32:06.152628+0000","last_peered":"2024-04-16T19:32:06.152628+0000","last_clean":"2024-04-16T19:32:06.152628+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:32:06.152628+0000","last_undegraded":"2024-04-16T19:32:06.152628+0000","last_fullsized":"2024-04-16T19:32:06.152628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163747+0000","last_change":"2024-04-16T17:47:23.266018+0000","last_active":"2024-04-16T19:33:11.163747+0000","last_peered":"2024-04-16T19:33:11.163747+0000","last_clean":"2024-04-16T19:33:11.163747+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:33:11.163747+0000","last_undegraded":"2024-04-16T19:33:11.163747+0000","last_fullsized":"2024-04-16T19:33:11.163747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168338+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:33:31.168338+0000","last_peered":"2024-04-16T19:33:31.168338+0000","last_clean":"2024-04-16T19:33:31.168338+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:33:31.168338+0000","last_undegraded":"2024-04-16T19:33:31.168338+0000","last_fullsized":"2024-04-16T19:33:31.168338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174370+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:33:46.174370+0000","last_peered":"2024-04-16T19:33:46.174370+0000","last_clean":"2024-04-16T19:33:46.174370+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:33:46.174370+0000","last_undegraded":"2024-04-16T19:33:46.174370+0000","last_fullsized":"2024-04-16T19:33:46.174370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'120","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170532+0000","last_change":"2024-04-16T17:47:35.319180+0000","last_active":"2024-04-16T19:33:21.170532+0000","last_peered":"2024-04-16T19:33:21.170532+0000","last_clean":"2024-04-16T19:33:21.170532+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:33:21.170532+0000","last_undegraded":"2024-04-16T19:33:21.170532+0000","last_fullsized":"2024-04-16T19:33:21.170532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:28.081965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.167247+0000","last_change":"2024-04-16T17:47:37.326971+0000","last_active":"2024-04-16T19:33:26.167247+0000","last_peered":"2024-04-16T19:33:26.167247+0000","last_clean":"2024-04-16T19:33:26.167247+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:33:26.167247+0000","last_undegraded":"2024-04-16T19:33:26.167247+0000","last_fullsized":"2024-04-16T19:33:26.167247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:04.366184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174929+0000","last_change":"2024-04-16T17:47:39.325433+0000","last_active":"2024-04-16T19:33:56.174929+0000","last_peered":"2024-04-16T19:33:56.174929+0000","last_clean":"2024-04-16T19:33:56.174929+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:33:56.174929+0000","last_undegraded":"2024-04-16T19:33:56.174929+0000","last_fullsized":"2024-04-16T19:33:56.174929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:50.427879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151752+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:32:06.151752+0000","last_peered":"2024-04-16T19:32:06.151752+0000","last_clean":"2024-04-16T19:32:06.151752+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:32:06.151752+0000","last_undegraded":"2024-04-16T19:32:06.151752+0000","last_fullsized":"2024-04-16T19:32:06.151752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:24.908570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'90","reported_seq":977,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.169718+0000","last_change":"2024-04-16T17:48:34.333850+0000","last_active":"2024-04-16T19:33:21.169718+0000","last_peered":"2024-04-16T19:33:21.169718+0000","last_clean":"2024-04-16T19:33:21.169718+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:33:21.169718+0000","last_undegraded":"2024-04-16T19:33:21.169718+0000","last_fullsized":"2024-04-16T19:33:21.169718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:31.546580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159785+0000","last_change":"2024-04-16T17:48:27.299707+0000","last_active":"2024-04-16T19:32:31.159785+0000","last_peered":"2024-04-16T19:32:31.159785+0000","last_clean":"2024-04-16T19:32:31.159785+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:32:31.159785+0000","last_undegraded":"2024-04-16T19:32:31.159785+0000","last_fullsized":"2024-04-16T19:32:31.159785+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'5","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150341+0000","last_change":"2024-04-16T17:48:01.258518+0000","last_active":"2024-04-16T19:32:01.150341+0000","last_peered":"2024-04-16T19:32:01.150341+0000","last_clean":"2024-04-16T19:32:01.150341+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:32:01.150341+0000","last_undegraded":"2024-04-16T19:32:01.150341+0000","last_fullsized":"2024-04-16T19:32:01.150341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:25.260545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'555","reported_seq":1941,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157758+0000","last_change":"2024-04-16T17:47:59.184752+0000","last_active":"2024-04-16T19:32:31.157758+0000","last_peered":"2024-04-16T19:32:31.157758+0000","last_clean":"2024-04-16T19:32:31.157758+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:32:31.157758+0000","last_undegraded":"2024-04-16T19:32:31.157758+0000","last_fullsized":"2024-04-16T19:32:31.157758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:36.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'651","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148802+0000","last_change":"2024-04-16T17:47:57.193900+0000","last_active":"2024-04-16T19:32:11.148802+0000","last_peered":"2024-04-16T19:32:11.148802+0000","last_clean":"2024-04-16T19:32:11.148802+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:32:11.148802+0000","last_undegraded":"2024-04-16T19:32:11.148802+0000","last_fullsized":"2024-04-16T19:32:11.148802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:12.186417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'12","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151428+0000","last_change":"2024-04-16T17:47:55.161557+0000","last_active":"2024-04-16T19:32:06.151428+0000","last_peered":"2024-04-16T19:32:06.151428+0000","last_clean":"2024-04-16T19:32:06.151428+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:32:06.151428+0000","last_undegraded":"2024-04-16T19:32:06.151428+0000","last_fullsized":"2024-04-16T19:32:06.151428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:51.923908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.176037+0000","last_change":"2024-04-16T17:47:23.261436+0000","last_active":"2024-04-16T19:33:41.176037+0000","last_peered":"2024-04-16T19:33:41.176037+0000","last_clean":"2024-04-16T19:33:41.176037+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:33:41.176037+0000","last_undegraded":"2024-04-16T19:33:41.176037+0000","last_fullsized":"2024-04-16T19:33:41.176037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154630+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:33:31.154630+0000","last_peered":"2024-04-16T19:33:31.154630+0000","last_clean":"2024-04-16T19:33:31.154630+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:33:31.154630+0000","last_undegraded":"2024-04-16T19:33:31.154630+0000","last_fullsized":"2024-04-16T19:33:31.154630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:35.902429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172709+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:33:21.172709+0000","last_peered":"2024-04-16T19:33:21.172709+0000","last_clean":"2024-04-16T19:33:21.172709+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:33:21.172709+0000","last_undegraded":"2024-04-16T19:33:21.172709+0000","last_fullsized":"2024-04-16T19:33:21.172709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:26.061929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'176","reported_seq":381,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167670+0000","last_change":"2024-04-16T17:47:35.317949+0000","last_active":"2024-04-16T19:33:21.167670+0000","last_peered":"2024-04-16T19:33:21.167670+0000","last_clean":"2024-04-16T19:33:21.167670+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:33:21.167670+0000","last_undegraded":"2024-04-16T19:33:21.167670+0000","last_fullsized":"2024-04-16T19:33:21.167670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.173767+0000","last_change":"2024-04-16T17:47:37.332636+0000","last_active":"2024-04-16T19:33:26.173767+0000","last_peered":"2024-04-16T19:33:26.173767+0000","last_clean":"2024-04-16T19:33:26.173767+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:33:26.173767+0000","last_undegraded":"2024-04-16T19:33:26.173767+0000","last_fullsized":"2024-04-16T19:33:26.173767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179806+0000","last_change":"2024-04-16T17:47:39.335342+0000","last_active":"2024-04-16T19:33:56.179806+0000","last_peered":"2024-04-16T19:33:56.179806+0000","last_clean":"2024-04-16T19:33:56.179806+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:33:56.179806+0000","last_undegraded":"2024-04-16T19:33:56.179806+0000","last_fullsized":"2024-04-16T19:33:56.179806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131831+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:31:51.131831+0000","last_peered":"2024-04-16T19:31:51.131831+0000","last_clean":"2024-04-16T19:31:51.131831+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:31:51.131831+0000","last_undegraded":"2024-04-16T19:31:51.131831+0000","last_fullsized":"2024-04-16T19:31:51.131831+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:43.281248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'118","reported_seq":1502,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156628+0000","last_change":"2024-04-16T17:48:34.329015+0000","last_active":"2024-04-16T19:32:41.156628+0000","last_peered":"2024-04-16T19:32:41.156628+0000","last_clean":"2024-04-16T19:32:41.156628+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:32:41.156628+0000","last_undegraded":"2024-04-16T19:32:41.156628+0000","last_fullsized":"2024-04-16T19:32:41.156628+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:38.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150062+0000","last_change":"2024-04-16T17:48:27.296987+0000","last_active":"2024-04-16T19:32:01.150062+0000","last_peered":"2024-04-16T19:32:01.150062+0000","last_clean":"2024-04-16T19:32:01.150062+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:32:01.150062+0000","last_undegraded":"2024-04-16T19:32:01.150062+0000","last_fullsized":"2024-04-16T19:32:01.150062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:13.235223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'8","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172452+0000","last_change":"2024-04-16T17:48:01.240484+0000","last_active":"2024-04-16T19:33:46.172452+0000","last_peered":"2024-04-16T19:33:46.172452+0000","last_clean":"2024-04-16T19:33:46.172452+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:33:46.172452+0000","last_undegraded":"2024-04-16T19:33:46.172452+0000","last_fullsized":"2024-04-16T19:33:46.172452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:38.257398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157906+0000","last_change":"2024-04-16T17:47:59.184121+0000","last_active":"2024-04-16T19:32:31.157906+0000","last_peered":"2024-04-16T19:32:31.157906+0000","last_clean":"2024-04-16T19:32:31.157906+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:32:31.157906+0000","last_undegraded":"2024-04-16T19:32:31.157906+0000","last_fullsized":"2024-04-16T19:32:31.157906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:46.708915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'783","reported_seq":43142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174072+0000","last_change":"2024-04-16T17:47:57.186095+0000","last_active":"2024-04-16T19:33:46.174072+0000","last_peered":"2024-04-16T19:33:46.174072+0000","last_clean":"2024-04-16T19:33:46.174072+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:33:46.174072+0000","last_undegraded":"2024-04-16T19:33:46.174072+0000","last_fullsized":"2024-04-16T19:33:46.174072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:50.249328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'9","reported_seq":208,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132707+0000","last_change":"2024-04-16T17:47:55.143458+0000","last_active":"2024-04-16T19:31:56.132707+0000","last_peered":"2024-04-16T19:31:56.132707+0000","last_clean":"2024-04-16T19:31:56.132707+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:31:56.132707+0000","last_undegraded":"2024-04-16T19:31:56.132707+0000","last_fullsized":"2024-04-16T19:31:56.132707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:53:52.793161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154404+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:32:11.154404+0000","last_peered":"2024-04-16T19:32:11.154404+0000","last_clean":"2024-04-16T19:32:11.154404+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:32:11.154404+0000","last_undegraded":"2024-04-16T19:32:11.154404+0000","last_fullsized":"2024-04-16T19:32:11.154404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:43.276218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171281+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:33:16.171281+0000","last_peered":"2024-04-16T19:33:16.171281+0000","last_clean":"2024-04-16T19:33:16.171281+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:33:16.171281+0000","last_undegraded":"2024-04-16T19:33:16.171281+0000","last_fullsized":"2024-04-16T19:33:16.171281+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174137+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:33:36.174137+0000","last_peered":"2024-04-16T19:33:36.174137+0000","last_clean":"2024-04-16T19:33:36.174137+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:33:36.174137+0000","last_undegraded":"2024-04-16T19:33:36.174137+0000","last_fullsized":"2024-04-16T19:33:36.174137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:33:14.972217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173777+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:33:51.173777+0000","last_peered":"2024-04-16T19:33:51.173777+0000","last_clean":"2024-04-16T19:33:51.173777+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:33:51.173777+0000","last_undegraded":"2024-04-16T19:33:51.173777+0000","last_fullsized":"2024-04-16T19:33:51.173777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:14.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'72","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174316+0000","last_change":"2024-04-16T17:47:35.310088+0000","last_active":"2024-04-16T19:33:46.174316+0000","last_peered":"2024-04-16T19:33:46.174316+0000","last_clean":"2024-04-16T19:33:46.174316+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:33:46.174316+0000","last_undegraded":"2024-04-16T19:33:46.174316+0000","last_fullsized":"2024-04-16T19:33:46.174316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177457+0000","last_change":"2024-04-16T17:47:37.327822+0000","last_active":"2024-04-16T19:33:51.177457+0000","last_peered":"2024-04-16T19:33:51.177457+0000","last_clean":"2024-04-16T19:33:51.177457+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:33:51.177457+0000","last_undegraded":"2024-04-16T19:33:51.177457+0000","last_fullsized":"2024-04-16T19:33:51.177457+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175307+0000","last_change":"2024-04-16T17:47:39.341107+0000","last_active":"2024-04-16T19:33:51.175307+0000","last_peered":"2024-04-16T19:33:51.175307+0000","last_clean":"2024-04-16T19:33:51.175307+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:33:51.175307+0000","last_undegraded":"2024-04-16T19:33:51.175307+0000","last_fullsized":"2024-04-16T19:33:51.175307+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157259+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:32:21.157259+0000","last_peered":"2024-04-16T19:32:21.157259+0000","last_clean":"2024-04-16T19:32:21.157259+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:32:21.157259+0000","last_undegraded":"2024-04-16T19:32:21.157259+0000","last_fullsized":"2024-04-16T19:32:21.157259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'100","reported_seq":1745,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147877+0000","last_change":"2024-04-16T17:48:34.333902+0000","last_active":"2024-04-16T19:32:06.147877+0000","last_peered":"2024-04-16T19:32:06.147877+0000","last_clean":"2024-04-16T19:32:06.147877+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:32:06.147877+0000","last_undegraded":"2024-04-16T19:32:06.147877+0000","last_fullsized":"2024-04-16T19:32:06.147877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:26.562009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146546+0000","last_change":"2024-04-16T17:48:27.296497+0000","last_active":"2024-04-16T19:32:01.146546+0000","last_peered":"2024-04-16T19:32:01.146546+0000","last_clean":"2024-04-16T19:32:01.146546+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:32:01.146546+0000","last_undegraded":"2024-04-16T19:32:01.146546+0000","last_fullsized":"2024-04-16T19:32:01.146546+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:59:05.786900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'8","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176076+0000","last_change":"2024-04-16T17:48:01.261918+0000","last_active":"2024-04-16T19:33:46.176076+0000","last_peered":"2024-04-16T19:33:46.176076+0000","last_clean":"2024-04-16T19:33:46.176076+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:33:46.176076+0000","last_undegraded":"2024-04-16T19:33:46.176076+0000","last_fullsized":"2024-04-16T19:33:46.176076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158533+0000","last_change":"2024-04-16T17:47:59.182311+0000","last_active":"2024-04-16T19:33:46.158533+0000","last_peered":"2024-04-16T19:33:46.158533+0000","last_clean":"2024-04-16T19:33:46.158533+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:33:46.158533+0000","last_undegraded":"2024-04-16T19:33:46.158533+0000","last_fullsized":"2024-04-16T19:33:46.158533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:15.733801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'897","reported_seq":3791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155083+0000","last_change":"2024-04-16T17:47:57.178953+0000","last_active":"2024-04-16T19:32:36.155083+0000","last_peered":"2024-04-16T19:32:36.155083+0000","last_clean":"2024-04-16T19:32:36.155083+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:32:36.155083+0000","last_undegraded":"2024-04-16T19:32:36.155083+0000","last_fullsized":"2024-04-16T19:32:36.155083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:27.120398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'29","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148129+0000","last_change":"2024-04-16T17:47:55.165769+0000","last_active":"2024-04-16T19:31:56.148129+0000","last_peered":"2024-04-16T19:31:56.148129+0000","last_clean":"2024-04-16T19:31:56.148129+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:31:56.148129+0000","last_undegraded":"2024-04-16T19:31:56.148129+0000","last_fullsized":"2024-04-16T19:31:56.148129+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175855+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:33:41.175855+0000","last_peered":"2024-04-16T19:33:41.175855+0000","last_clean":"2024-04-16T19:33:41.175855+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:33:41.175855+0000","last_undegraded":"2024-04-16T19:33:41.175855+0000","last_fullsized":"2024-04-16T19:33:41.175855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159749+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:32:51.159749+0000","last_peered":"2024-04-16T19:32:51.159749+0000","last_clean":"2024-04-16T19:32:51.159749+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:32:51.159749+0000","last_undegraded":"2024-04-16T19:32:51.159749+0000","last_fullsized":"2024-04-16T19:32:51.159749+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169877+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:33:36.169877+0000","last_peered":"2024-04-16T19:33:36.169877+0000","last_clean":"2024-04-16T19:33:36.169877+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:33:36.169877+0000","last_undegraded":"2024-04-16T19:33:36.169877+0000","last_fullsized":"2024-04-16T19:33:36.169877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154521+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:32:21.154521+0000","last_peered":"2024-04-16T19:32:21.154521+0000","last_clean":"2024-04-16T19:32:21.154521+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:32:21.154521+0000","last_undegraded":"2024-04-16T19:32:21.154521+0000","last_fullsized":"2024-04-16T19:32:21.154521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'144","reported_seq":330,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177257+0000","last_change":"2024-04-16T17:47:35.314035+0000","last_active":"2024-04-16T19:33:46.177257+0000","last_peered":"2024-04-16T19:33:46.177257+0000","last_clean":"2024-04-16T19:33:46.177257+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:33:46.177257+0000","last_undegraded":"2024-04-16T19:33:46.177257+0000","last_fullsized":"2024-04-16T19:33:46.177257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181110+0000","last_change":"2024-04-16T17:47:37.334598+0000","last_active":"2024-04-16T19:33:51.181110+0000","last_peered":"2024-04-16T19:33:51.181110+0000","last_clean":"2024-04-16T19:33:51.181110+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:33:51.181110+0000","last_undegraded":"2024-04-16T19:33:51.181110+0000","last_fullsized":"2024-04-16T19:33:51.181110+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:24.856058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175255+0000","last_change":"2024-04-16T17:47:39.339775+0000","last_active":"2024-04-16T19:33:51.175255+0000","last_peered":"2024-04-16T19:33:51.175255+0000","last_clean":"2024-04-16T19:33:51.175255+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:33:51.175255+0000","last_undegraded":"2024-04-16T19:33:51.175255+0000","last_fullsized":"2024-04-16T19:33:51.175255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143941+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:31:51.143941+0000","last_peered":"2024-04-16T19:31:51.143941+0000","last_clean":"2024-04-16T19:31:51.143941+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:31:51.143941+0000","last_undegraded":"2024-04-16T19:31:51.143941+0000","last_fullsized":"2024-04-16T19:31:51.143941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:26.624319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148088+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:32:06.148088+0000","last_peered":"2024-04-16T19:32:06.148088+0000","last_clean":"2024-04-16T19:32:06.148088+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:32:06.148088+0000","last_undegraded":"2024-04-16T19:32:06.148088+0000","last_fullsized":"2024-04-16T19:32:06.148088+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146697+0000","last_change":"2024-04-16T17:48:27.307048+0000","last_active":"2024-04-16T19:32:01.146697+0000","last_peered":"2024-04-16T19:32:01.146697+0000","last_clean":"2024-04-16T19:32:01.146697+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:32:01.146697+0000","last_undegraded":"2024-04-16T19:32:01.146697+0000","last_fullsized":"2024-04-16T19:32:01.146697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:19.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'33","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152888+0000","last_change":"2024-04-16T17:48:01.250054+0000","last_active":"2024-04-16T19:32:06.152888+0000","last_peered":"2024-04-16T19:32:06.152888+0000","last_clean":"2024-04-16T19:32:06.152888+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:32:06.152888+0000","last_undegraded":"2024-04-16T19:32:06.152888+0000","last_fullsized":"2024-04-16T19:32:06.152888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133916+0000","last_change":"2024-04-16T17:47:59.190987+0000","last_active":"2024-04-16T19:32:01.133916+0000","last_peered":"2024-04-16T19:32:01.133916+0000","last_clean":"2024-04-16T19:32:01.133916+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:32:01.133916+0000","last_undegraded":"2024-04-16T19:32:01.133916+0000","last_fullsized":"2024-04-16T19:32:01.133916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:11.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'264","reported_seq":640,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151154+0000","last_change":"2024-04-16T17:47:57.189816+0000","last_active":"2024-04-16T19:32:01.151154+0000","last_peered":"2024-04-16T19:32:01.151154+0000","last_clean":"2024-04-16T19:32:01.151154+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:32:01.151154+0000","last_undegraded":"2024-04-16T19:32:01.151154+0000","last_fullsized":"2024-04-16T19:32:01.151154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'10","reported_seq":156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159575+0000","last_change":"2024-04-16T17:47:55.161898+0000","last_active":"2024-04-16T19:32:31.159575+0000","last_peered":"2024-04-16T19:32:31.159575+0000","last_clean":"2024-04-16T19:32:31.159575+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:32:31.159575+0000","last_undegraded":"2024-04-16T19:32:31.159575+0000","last_fullsized":"2024-04-16T19:32:31.159575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174563+0000","last_change":"2024-04-16T17:47:23.265711+0000","last_active":"2024-04-16T19:33:36.174563+0000","last_peered":"2024-04-16T19:33:36.174563+0000","last_clean":"2024-04-16T19:33:36.174563+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:33:36.174563+0000","last_undegraded":"2024-04-16T19:33:36.174563+0000","last_fullsized":"2024-04-16T19:33:36.174563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.158718+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:32:51.158718+0000","last_peered":"2024-04-16T19:32:51.158718+0000","last_clean":"2024-04-16T19:32:51.158718+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:32:51.158718+0000","last_undegraded":"2024-04-16T19:32:51.158718+0000","last_fullsized":"2024-04-16T19:32:51.158718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178254+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:33:51.178254+0000","last_peered":"2024-04-16T19:33:51.178254+0000","last_clean":"2024-04-16T19:33:51.178254+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:33:51.178254+0000","last_undegraded":"2024-04-16T19:33:51.178254+0000","last_fullsized":"2024-04-16T19:33:51.178254+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:49.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"48'4","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158694+0000","last_change":"2024-04-16T17:47:33.301081+0000","last_active":"2024-04-16T19:33:46.158694+0000","last_peered":"2024-04-16T19:33:46.158694+0000","last_clean":"2024-04-16T19:33:46.158694+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:33:46.158694+0000","last_undegraded":"2024-04-16T19:33:46.158694+0000","last_fullsized":"2024-04-16T19:33:46.158694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:05.482314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'151","reported_seq":1555,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149286+0000","last_change":"2024-04-16T17:47:35.316795+0000","last_active":"2024-04-16T19:32:11.149286+0000","last_peered":"2024-04-16T19:32:11.149286+0000","last_clean":"2024-04-16T19:32:11.149286+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:32:11.149286+0000","last_undegraded":"2024-04-16T19:32:11.149286+0000","last_fullsized":"2024-04-16T19:32:11.149286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178652+0000","last_change":"2024-04-16T17:47:37.326201+0000","last_active":"2024-04-16T19:33:56.178652+0000","last_peered":"2024-04-16T19:33:56.178652+0000","last_clean":"2024-04-16T19:33:56.178652+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:33:56.178652+0000","last_undegraded":"2024-04-16T19:33:56.178652+0000","last_fullsized":"2024-04-16T19:33:56.178652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170822+0000","last_change":"2024-04-16T17:47:39.332609+0000","last_active":"2024-04-16T19:33:26.170822+0000","last_peered":"2024-04-16T19:33:26.170822+0000","last_clean":"2024-04-16T19:33:26.170822+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:33:26.170822+0000","last_undegraded":"2024-04-16T19:33:26.170822+0000","last_fullsized":"2024-04-16T19:33:26.170822+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143874+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:31:51.143874+0000","last_peered":"2024-04-16T19:31:51.143874+0000","last_clean":"2024-04-16T19:31:51.143874+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:31:51.143874+0000","last_undegraded":"2024-04-16T19:31:51.143874+0000","last_fullsized":"2024-04-16T19:31:51.143874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:45.404216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'76","reported_seq":1158,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150775+0000","last_change":"2024-04-16T17:48:34.318880+0000","last_active":"2024-04-16T19:32:06.150775+0000","last_peered":"2024-04-16T19:32:06.150775+0000","last_clean":"2024-04-16T19:32:06.150775+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:32:06.150775+0000","last_undegraded":"2024-04-16T19:32:06.150775+0000","last_fullsized":"2024-04-16T19:32:06.150775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153591+0000","last_change":"2024-04-16T17:48:27.299238+0000","last_active":"2024-04-16T19:32:26.153591+0000","last_peered":"2024-04-16T19:32:26.153591+0000","last_clean":"2024-04-16T19:32:26.153591+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:32:26.153591+0000","last_undegraded":"2024-04-16T19:32:26.153591+0000","last_fullsized":"2024-04-16T19:32:26.153591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:49.437897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'6","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176200+0000","last_change":"2024-04-16T17:48:01.256444+0000","last_active":"2024-04-16T19:33:46.176200+0000","last_peered":"2024-04-16T19:33:46.176200+0000","last_clean":"2024-04-16T19:33:46.176200+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:33:46.176200+0000","last_undegraded":"2024-04-16T19:33:46.176200+0000","last_fullsized":"2024-04-16T19:33:46.176200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'555","reported_seq":1932,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133972+0000","last_change":"2024-04-16T17:47:59.183633+0000","last_active":"2024-04-16T19:32:01.133972+0000","last_peered":"2024-04-16T19:32:01.133972+0000","last_clean":"2024-04-16T19:32:01.133972+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:32:01.133972+0000","last_undegraded":"2024-04-16T19:32:01.133972+0000","last_fullsized":"2024-04-16T19:32:01.133972+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:14.201307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'306","reported_seq":623,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149338+0000","last_change":"2024-04-16T17:47:57.177185+0000","last_active":"2024-04-16T19:32:01.149338+0000","last_peered":"2024-04-16T19:32:01.149338+0000","last_clean":"2024-04-16T19:32:01.149338+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:32:01.149338+0000","last_undegraded":"2024-04-16T19:32:01.149338+0000","last_fullsized":"2024-04-16T19:32:01.149338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:18.465537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"48'5","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174804+0000","last_change":"2024-04-16T17:47:55.178154+0000","last_active":"2024-04-16T19:33:41.174804+0000","last_peered":"2024-04-16T19:33:41.174804+0000","last_clean":"2024-04-16T19:33:41.174804+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:33:41.174804+0000","last_undegraded":"2024-04-16T19:33:41.174804+0000","last_fullsized":"2024-04-16T19:33:41.174804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173663+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:33:36.173663+0000","last_peered":"2024-04-16T19:33:36.173663+0000","last_clean":"2024-04-16T19:33:36.173663+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:33:36.173663+0000","last_undegraded":"2024-04-16T19:33:36.173663+0000","last_fullsized":"2024-04-16T19:33:36.173663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164327+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:32:56.164327+0000","last_peered":"2024-04-16T19:32:56.164327+0000","last_clean":"2024-04-16T19:32:56.164327+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:32:56.164327+0000","last_undegraded":"2024-04-16T19:32:56.164327+0000","last_fullsized":"2024-04-16T19:32:56.164327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171318+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:33:16.171318+0000","last_peered":"2024-04-16T19:33:16.171318+0000","last_clean":"2024-04-16T19:33:16.171318+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:33:16.171318+0000","last_undegraded":"2024-04-16T19:33:16.171318+0000","last_fullsized":"2024-04-16T19:33:16.171318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:05:07.906618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173252+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:33:36.173252+0000","last_peered":"2024-04-16T19:33:36.173252+0000","last_clean":"2024-04-16T19:33:36.173252+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:33:36.173252+0000","last_undegraded":"2024-04-16T19:33:36.173252+0000","last_fullsized":"2024-04-16T19:33:36.173252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:30.892245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":504,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174348+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:33:46.174348+0000","last_peered":"2024-04-16T19:33:46.174348+0000","last_clean":"2024-04-16T19:33:46.174348+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:33:46.174348+0000","last_undegraded":"2024-04-16T19:33:46.174348+0000","last_fullsized":"2024-04-16T19:33:46.174348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'177","reported_seq":1612,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178544+0000","last_change":"2024-04-16T17:47:35.312413+0000","last_active":"2024-04-16T19:33:51.178544+0000","last_peered":"2024-04-16T19:33:51.178544+0000","last_clean":"2024-04-16T19:33:51.178544+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:33:51.178544+0000","last_undegraded":"2024-04-16T19:33:51.178544+0000","last_fullsized":"2024-04-16T19:33:51.178544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152741+0000","last_change":"2024-04-16T17:47:37.327792+0000","last_active":"2024-04-16T19:32:26.152741+0000","last_peered":"2024-04-16T19:32:26.152741+0000","last_clean":"2024-04-16T19:32:26.152741+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:32:26.152741+0000","last_undegraded":"2024-04-16T19:32:26.152741+0000","last_fullsized":"2024-04-16T19:32:26.152741+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:43.802094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152938+0000","last_change":"2024-04-16T17:47:39.700262+0000","last_active":"2024-04-16T19:32:11.152938+0000","last_peered":"2024-04-16T19:32:11.152938+0000","last_clean":"2024-04-16T19:32:11.152938+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:32:11.152938+0000","last_undegraded":"2024-04-16T19:32:11.152938+0000","last_fullsized":"2024-04-16T19:32:11.152938+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:26.867823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157247+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:33:41.157247+0000","last_peered":"2024-04-16T19:33:41.157247+0000","last_clean":"2024-04-16T19:33:41.157247+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:33:41.157247+0000","last_undegraded":"2024-04-16T19:33:41.157247+0000","last_fullsized":"2024-04-16T19:33:41.157247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.324013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150723+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:32:06.150723+0000","last_peered":"2024-04-16T19:32:06.150723+0000","last_clean":"2024-04-16T19:32:06.150723+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:32:06.150723+0000","last_undegraded":"2024-04-16T19:32:06.150723+0000","last_fullsized":"2024-04-16T19:32:06.150723+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:02.642376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140845+0000","last_change":"2024-04-16T17:48:27.304274+0000","last_active":"2024-04-16T19:32:31.140845+0000","last_peered":"2024-04-16T19:32:31.140845+0000","last_clean":"2024-04-16T19:32:31.140845+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:32:31.140845+0000","last_undegraded":"2024-04-16T19:32:31.140845+0000","last_fullsized":"2024-04-16T19:32:31.140845+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:19.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'14","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133996+0000","last_change":"2024-04-16T17:48:01.251987+0000","last_active":"2024-04-16T19:32:01.133996+0000","last_peered":"2024-04-16T19:32:01.133996+0000","last_clean":"2024-04-16T19:32:01.133996+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:32:01.133996+0000","last_undegraded":"2024-04-16T19:32:01.133996+0000","last_fullsized":"2024-04-16T19:32:01.133996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:53.985409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'555","reported_seq":1923,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172522+0000","last_change":"2024-04-16T17:47:59.188607+0000","last_active":"2024-04-16T19:33:46.172522+0000","last_peered":"2024-04-16T19:33:46.172522+0000","last_clean":"2024-04-16T19:33:46.172522+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:33:46.172522+0000","last_undegraded":"2024-04-16T19:33:46.172522+0000","last_fullsized":"2024-04-16T19:33:46.172522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:09.833400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'514","reported_seq":1190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152969+0000","last_change":"2024-04-16T17:47:57.187984+0000","last_active":"2024-04-16T19:32:06.152969+0000","last_peered":"2024-04-16T19:32:06.152969+0000","last_clean":"2024-04-16T19:32:06.152969+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:32:06.152969+0000","last_undegraded":"2024-04-16T19:32:06.152969+0000","last_fullsized":"2024-04-16T19:32:06.152969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:09.942121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"51'3","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157358+0000","last_change":"2024-04-16T17:47:55.173104+0000","last_active":"2024-04-16T19:33:41.157358+0000","last_peered":"2024-04-16T19:33:41.157358+0000","last_clean":"2024-04-16T19:33:41.157358+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:33:41.157358+0000","last_undegraded":"2024-04-16T19:33:41.157358+0000","last_fullsized":"2024-04-16T19:33:41.157358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:46.009020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.170986+0000","last_change":"2024-04-16T17:47:23.266373+0000","last_active":"2024-04-16T19:33:41.170986+0000","last_peered":"2024-04-16T19:33:41.170986+0000","last_clean":"2024-04-16T19:33:41.170986+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:33:41.170986+0000","last_undegraded":"2024-04-16T19:33:41.170986+0000","last_fullsized":"2024-04-16T19:33:41.170986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:11.306509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158724+0000","last_change":"2024-04-16T17:47:32.273580+0000","last_active":"2024-04-16T19:33:46.158724+0000","last_peered":"2024-04-16T19:33:46.158724+0000","last_clean":"2024-04-16T19:33:46.158724+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:33:46.158724+0000","last_undegraded":"2024-04-16T19:33:46.158724+0000","last_fullsized":"2024-04-16T19:33:46.158724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:28.727683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164407+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:32:56.164407+0000","last_peered":"2024-04-16T19:32:56.164407+0000","last_clean":"2024-04-16T19:32:56.164407+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:32:56.164407+0000","last_undegraded":"2024-04-16T19:32:56.164407+0000","last_fullsized":"2024-04-16T19:32:56.164407+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163230+0000","last_change":"2024-04-16T17:47:33.300039+0000","last_active":"2024-04-16T19:34:06.163230+0000","last_peered":"2024-04-16T19:34:06.163230+0000","last_clean":"2024-04-16T19:34:06.163230+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:34:06.163230+0000","last_undegraded":"2024-04-16T19:34:06.163230+0000","last_fullsized":"2024-04-16T19:34:06.163230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:42.946001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'144","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173676+0000","last_change":"2024-04-16T17:47:35.305470+0000","last_active":"2024-04-16T19:33:51.173676+0000","last_peered":"2024-04-16T19:33:51.173676+0000","last_clean":"2024-04-16T19:33:51.173676+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:33:51.173676+0000","last_undegraded":"2024-04-16T19:33:51.173676+0000","last_fullsized":"2024-04-16T19:33:51.173676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:00.096888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.161004+0000","last_change":"2024-04-16T17:47:37.334307+0000","last_active":"2024-04-16T19:33:56.161004+0000","last_peered":"2024-04-16T19:33:56.161004+0000","last_clean":"2024-04-16T19:33:56.161004+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:33:56.161004+0000","last_undegraded":"2024-04-16T19:33:56.161004+0000","last_fullsized":"2024-04-16T19:33:56.161004+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:58.517922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168901+0000","last_change":"2024-04-16T17:47:39.343779+0000","last_active":"2024-04-16T19:33:26.168901+0000","last_peered":"2024-04-16T19:33:26.168901+0000","last_clean":"2024-04-16T19:33:26.168901+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:33:26.168901+0000","last_undegraded":"2024-04-16T19:33:26.168901+0000","last_fullsized":"2024-04-16T19:33:26.168901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:43:28.188142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131712+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:31:51.131712+0000","last_peered":"2024-04-16T19:31:51.131712+0000","last_clean":"2024-04-16T19:31:51.131712+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:31:51.131712+0000","last_undegraded":"2024-04-16T19:31:51.131712+0000","last_fullsized":"2024-04-16T19:31:51.131712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:49:14.730835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'122","reported_seq":1763,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170106+0000","last_change":"2024-04-16T17:48:34.337192+0000","last_active":"2024-04-16T19:33:11.170106+0000","last_peered":"2024-04-16T19:33:11.170106+0000","last_clean":"2024-04-16T19:33:11.170106+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:33:11.170106+0000","last_undegraded":"2024-04-16T19:33:11.170106+0000","last_fullsized":"2024-04-16T19:33:11.170106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.154528+0000","last_change":"2024-04-16T17:48:27.301617+0000","last_active":"2024-04-16T19:32:31.154528+0000","last_peered":"2024-04-16T19:32:31.154528+0000","last_clean":"2024-04-16T19:32:31.154528+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:32:31.154528+0000","last_undegraded":"2024-04-16T19:32:31.154528+0000","last_fullsized":"2024-04-16T19:32:31.154528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'16","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133844+0000","last_change":"2024-04-16T17:48:01.234685+0000","last_active":"2024-04-16T19:32:01.133844+0000","last_peered":"2024-04-16T19:32:01.133844+0000","last_clean":"2024-04-16T19:32:01.133844+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:32:01.133844+0000","last_undegraded":"2024-04-16T19:32:01.133844+0000","last_fullsized":"2024-04-16T19:32:01.133844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:02.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152930+0000","last_change":"2024-04-16T17:47:59.185616+0000","last_active":"2024-04-16T19:32:06.152930+0000","last_peered":"2024-04-16T19:32:06.152930+0000","last_clean":"2024-04-16T19:32:06.152930+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:32:06.152930+0000","last_undegraded":"2024-04-16T19:32:06.152930+0000","last_fullsized":"2024-04-16T19:32:06.152930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:24:43.650159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1159","reported_seq":1411,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172544+0000","last_change":"2024-04-16T17:47:57.188806+0000","last_active":"2024-04-16T19:33:46.172544+0000","last_peered":"2024-04-16T19:33:46.172544+0000","last_clean":"2024-04-16T19:33:46.172544+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:33:46.172544+0000","last_undegraded":"2024-04-16T19:33:46.172544+0000","last_fullsized":"2024-04-16T19:33:46.172544+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1159,"log_dups_size":0,"ondisk_log_size":1159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:00.199310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"48'18","reported_seq":399,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177873+0000","last_change":"2024-04-16T17:47:55.164040+0000","last_active":"2024-04-16T19:33:41.177873+0000","last_peered":"2024-04-16T19:33:41.177873+0000","last_clean":"2024-04-16T19:33:41.177873+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:33:41.177873+0000","last_undegraded":"2024-04-16T19:33:41.177873+0000","last_fullsized":"2024-04-16T19:33:41.177873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.179804+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:34:01.179804+0000","last_peered":"2024-04-16T19:34:01.179804+0000","last_clean":"2024-04-16T19:34:01.179804+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:34:01.179804+0000","last_undegraded":"2024-04-16T19:34:01.179804+0000","last_fullsized":"2024-04-16T19:34:01.179804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154908+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:33:31.154908+0000","last_peered":"2024-04-16T19:33:31.154908+0000","last_clean":"2024-04-16T19:33:31.154908+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:33:31.154908+0000","last_undegraded":"2024-04-16T19:33:31.154908+0000","last_fullsized":"2024-04-16T19:33:31.154908+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:55:03.538702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168590+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:33:31.168590+0000","last_peered":"2024-04-16T19:33:31.168590+0000","last_clean":"2024-04-16T19:33:31.168590+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:33:31.168590+0000","last_undegraded":"2024-04-16T19:33:31.168590+0000","last_fullsized":"2024-04-16T19:33:31.168590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180686+0000","last_change":"2024-04-16T17:47:33.300948+0000","last_active":"2024-04-16T19:33:51.180686+0000","last_peered":"2024-04-16T19:33:51.180686+0000","last_clean":"2024-04-16T19:33:51.180686+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:33:51.180686+0000","last_undegraded":"2024-04-16T19:33:51.180686+0000","last_fullsized":"2024-04-16T19:33:51.180686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'120","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180791+0000","last_change":"2024-04-16T17:47:35.313794+0000","last_active":"2024-04-16T19:33:51.180791+0000","last_peered":"2024-04-16T19:33:51.180791+0000","last_clean":"2024-04-16T19:33:51.180791+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:33:51.180791+0000","last_undegraded":"2024-04-16T19:33:51.180791+0000","last_fullsized":"2024-04-16T19:33:51.180791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178567+0000","last_change":"2024-04-16T17:47:37.329509+0000","last_active":"2024-04-16T19:33:56.178567+0000","last_peered":"2024-04-16T19:33:56.178567+0000","last_clean":"2024-04-16T19:33:56.178567+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:33:56.178567+0000","last_undegraded":"2024-04-16T19:33:56.178567+0000","last_fullsized":"2024-04-16T19:33:56.178567+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170908+0000","last_change":"2024-04-16T17:47:39.697737+0000","last_active":"2024-04-16T19:33:26.170908+0000","last_peered":"2024-04-16T19:33:26.170908+0000","last_clean":"2024-04-16T19:33:26.170908+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:33:26.170908+0000","last_undegraded":"2024-04-16T19:33:26.170908+0000","last_fullsized":"2024-04-16T19:33:26.170908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.150046+0000","last_change":"2024-04-16T18:10:50.537157+0000","last_active":"2024-04-16T19:33:11.150046+0000","last_peered":"2024-04-16T19:33:11.150046+0000","last_clean":"2024-04-16T19:33:11.150046+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:33:11.150046+0000","last_undegraded":"2024-04-16T19:33:11.150046+0000","last_fullsized":"2024-04-16T19:33:11.150046+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:13.931584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1282,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156075+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:32:36.156075+0000","last_peered":"2024-04-16T19:32:36.156075+0000","last_clean":"2024-04-16T19:32:36.156075+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:32:36.156075+0000","last_undegraded":"2024-04-16T19:32:36.156075+0000","last_fullsized":"2024-04-16T19:32:36.156075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:20.310434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147837+0000","last_change":"2024-04-16T17:48:27.305636+0000","last_active":"2024-04-16T19:33:01.147837+0000","last_peered":"2024-04-16T19:33:01.147837+0000","last_clean":"2024-04-16T19:33:01.147837+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:33:01.147837+0000","last_undegraded":"2024-04-16T19:33:01.147837+0000","last_fullsized":"2024-04-16T19:33:01.147837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:06.425749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'13","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175996+0000","last_change":"2024-04-16T17:48:01.266136+0000","last_active":"2024-04-16T19:33:46.175996+0000","last_peered":"2024-04-16T19:33:46.175996+0000","last_clean":"2024-04-16T19:33:46.175996+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:33:46.175996+0000","last_undegraded":"2024-04-16T19:33:46.175996+0000","last_fullsized":"2024-04-16T19:33:46.175996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177121+0000","last_change":"2024-04-16T17:47:59.188856+0000","last_active":"2024-04-16T19:33:46.177121+0000","last_peered":"2024-04-16T19:33:46.177121+0000","last_clean":"2024-04-16T19:33:46.177121+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:33:46.177121+0000","last_undegraded":"2024-04-16T19:33:46.177121+0000","last_fullsized":"2024-04-16T19:33:46.177121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:35.433741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1314","reported_seq":1867,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172479+0000","last_change":"2024-04-16T17:47:57.192362+0000","last_active":"2024-04-16T19:33:46.172479+0000","last_peered":"2024-04-16T19:33:46.172479+0000","last_clean":"2024-04-16T19:33:46.172479+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:33:46.172479+0000","last_undegraded":"2024-04-16T19:33:46.172479+0000","last_fullsized":"2024-04-16T19:33:46.172479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1314,"log_dups_size":0,"ondisk_log_size":1314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:14.446218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'4","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178048+0000","last_change":"2024-04-16T17:47:55.156911+0000","last_active":"2024-04-16T19:33:41.178048+0000","last_peered":"2024-04-16T19:33:41.178048+0000","last_clean":"2024-04-16T19:33:41.178048+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:33:41.178048+0000","last_undegraded":"2024-04-16T19:33:41.178048+0000","last_fullsized":"2024-04-16T19:33:41.178048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174726+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:33:41.174726+0000","last_peered":"2024-04-16T19:33:41.174726+0000","last_clean":"2024-04-16T19:33:41.174726+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:33:41.174726+0000","last_undegraded":"2024-04-16T19:33:41.174726+0000","last_fullsized":"2024-04-16T19:33:41.174726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174314+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:33:36.174314+0000","last_peered":"2024-04-16T19:33:36.174314+0000","last_clean":"2024-04-16T19:33:36.174314+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:33:36.174314+0000","last_undegraded":"2024-04-16T19:33:36.174314+0000","last_fullsized":"2024-04-16T19:33:36.174314+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:22.291956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163913+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:32:46.163913+0000","last_peered":"2024-04-16T19:32:46.163913+0000","last_clean":"2024-04-16T19:32:46.163913+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:32:46.163913+0000","last_undegraded":"2024-04-16T19:32:46.163913+0000","last_fullsized":"2024-04-16T19:32:46.163913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175384+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:33:51.175384+0000","last_peered":"2024-04-16T19:33:51.175384+0000","last_clean":"2024-04-16T19:33:51.175384+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:33:51.175384+0000","last_undegraded":"2024-04-16T19:33:51.175384+0000","last_fullsized":"2024-04-16T19:33:51.175384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'168","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178373+0000","last_change":"2024-04-16T17:47:35.321995+0000","last_active":"2024-04-16T19:33:51.178373+0000","last_peered":"2024-04-16T19:33:51.178373+0000","last_clean":"2024-04-16T19:33:51.178373+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:33:51.178373+0000","last_undegraded":"2024-04-16T19:33:51.178373+0000","last_fullsized":"2024-04-16T19:33:51.178373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:17.922789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'12","reported_seq":1342,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179297+0000","last_change":"2024-04-16T17:47:37.323586+0000","last_active":"2024-04-16T19:34:06.179297+0000","last_peered":"2024-04-16T19:34:06.179297+0000","last_clean":"2024-04-16T19:34:06.179297+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:34:06.179297+0000","last_undegraded":"2024-04-16T19:34:06.179297+0000","last_fullsized":"2024-04-16T19:34:06.179297+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145872+0000","last_change":"2024-04-16T17:47:39.698152+0000","last_active":"2024-04-16T19:31:46.145872+0000","last_peered":"2024-04-16T19:31:46.145872+0000","last_clean":"2024-04-16T19:31:46.145872+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:31:46.145872+0000","last_undegraded":"2024-04-16T19:31:46.145872+0000","last_fullsized":"2024-04-16T19:31:46.145872+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:08.010344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.164914+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:33:11.164914+0000","last_peered":"2024-04-16T19:33:11.164914+0000","last_clean":"2024-04-16T19:33:11.164914+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:33:11.164914+0000","last_undegraded":"2024-04-16T19:33:11.164914+0000","last_fullsized":"2024-04-16T19:33:11.164914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:36:23.318927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158153+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:32:36.158153+0000","last_peered":"2024-04-16T19:32:36.158153+0000","last_clean":"2024-04-16T19:32:36.158153+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:32:36.158153+0000","last_undegraded":"2024-04-16T19:32:36.158153+0000","last_fullsized":"2024-04-16T19:32:36.158153+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151256+0000","last_change":"2024-04-16T17:48:27.298996+0000","last_active":"2024-04-16T19:32:01.151256+0000","last_peered":"2024-04-16T19:32:01.151256+0000","last_clean":"2024-04-16T19:32:01.151256+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:32:01.151256+0000","last_undegraded":"2024-04-16T19:32:01.151256+0000","last_fullsized":"2024-04-16T19:32:01.151256+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"51'4","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140996+0000","last_change":"2024-04-16T17:48:01.252079+0000","last_active":"2024-04-16T19:32:31.140996+0000","last_peered":"2024-04-16T19:32:31.140996+0000","last_clean":"2024-04-16T19:32:31.140996+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:32:31.140996+0000","last_undegraded":"2024-04-16T19:32:31.140996+0000","last_fullsized":"2024-04-16T19:32:31.140996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:57.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'555","reported_seq":1994,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158653+0000","last_change":"2024-04-16T17:47:59.197276+0000","last_active":"2024-04-16T19:33:46.158653+0000","last_peered":"2024-04-16T19:33:46.158653+0000","last_clean":"2024-04-16T19:33:46.158653+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:33:46.158653+0000","last_undegraded":"2024-04-16T19:33:46.158653+0000","last_fullsized":"2024-04-16T19:33:46.158653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:16.461108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'553","reported_seq":822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172334+0000","last_change":"2024-04-16T17:47:57.191520+0000","last_active":"2024-04-16T19:33:46.172334+0000","last_peered":"2024-04-16T19:33:46.172334+0000","last_clean":"2024-04-16T19:33:46.172334+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:33:46.172334+0000","last_undegraded":"2024-04-16T19:33:46.172334+0000","last_fullsized":"2024-04-16T19:33:46.172334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:25.845115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'14","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144097+0000","last_change":"2024-04-16T17:47:55.178734+0000","last_active":"2024-04-16T19:31:51.144097+0000","last_peered":"2024-04-16T19:31:51.144097+0000","last_clean":"2024-04-16T19:31:51.144097+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:31:51.144097+0000","last_undegraded":"2024-04-16T19:31:51.144097+0000","last_fullsized":"2024-04-16T19:31:51.144097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175692+0000","last_change":"2024-04-16T17:47:23.265822+0000","last_active":"2024-04-16T19:33:41.175692+0000","last_peered":"2024-04-16T19:33:41.175692+0000","last_clean":"2024-04-16T19:33:41.175692+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:33:41.175692+0000","last_undegraded":"2024-04-16T19:33:41.175692+0000","last_fullsized":"2024-04-16T19:33:41.175692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178216+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:34:01.178216+0000","last_peered":"2024-04-16T19:34:01.178216+0000","last_clean":"2024-04-16T19:34:01.178216+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:34:01.178216+0000","last_undegraded":"2024-04-16T19:34:01.178216+0000","last_fullsized":"2024-04-16T19:34:01.178216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.158816+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:32:46.158816+0000","last_peered":"2024-04-16T19:32:46.158816+0000","last_clean":"2024-04-16T19:32:46.158816+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:32:46.158816+0000","last_undegraded":"2024-04-16T19:32:46.158816+0000","last_fullsized":"2024-04-16T19:32:46.158816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177291+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:33:51.177291+0000","last_peered":"2024-04-16T19:33:51.177291+0000","last_clean":"2024-04-16T19:33:51.177291+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:33:51.177291+0000","last_undegraded":"2024-04-16T19:33:51.177291+0000","last_fullsized":"2024-04-16T19:33:51.177291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'160","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182468+0000","last_change":"2024-04-16T17:47:35.312410+0000","last_active":"2024-04-16T19:34:11.182468+0000","last_peered":"2024-04-16T19:34:11.182468+0000","last_clean":"2024-04-16T19:34:11.182468+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:34:11.182468+0000","last_undegraded":"2024-04-16T19:34:11.182468+0000","last_fullsized":"2024-04-16T19:34:11.182468+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174883+0000","last_change":"2024-04-16T17:47:37.322938+0000","last_active":"2024-04-16T19:33:56.174883+0000","last_peered":"2024-04-16T19:33:56.174883+0000","last_clean":"2024-04-16T19:33:56.174883+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:33:56.174883+0000","last_undegraded":"2024-04-16T19:33:56.174883+0000","last_fullsized":"2024-04-16T19:33:56.174883+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:33.277719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170994+0000","last_change":"2024-04-16T17:47:39.699806+0000","last_active":"2024-04-16T19:33:26.170994+0000","last_peered":"2024-04-16T19:33:26.170994+0000","last_clean":"2024-04-16T19:33:26.170994+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:33:26.170994+0000","last_undegraded":"2024-04-16T19:33:26.170994+0000","last_fullsized":"2024-04-16T19:33:26.170994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164695+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:33:01.164695+0000","last_peered":"2024-04-16T19:33:01.164695+0000","last_clean":"2024-04-16T19:33:01.164695+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:33:01.164695+0000","last_undegraded":"2024-04-16T19:33:01.164695+0000","last_fullsized":"2024-04-16T19:33:01.164695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1098,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168822+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:33:06.168822+0000","last_peered":"2024-04-16T19:33:06.168822+0000","last_clean":"2024-04-16T19:33:06.168822+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:33:06.168822+0000","last_undegraded":"2024-04-16T19:33:06.168822+0000","last_fullsized":"2024-04-16T19:33:06.168822+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:38.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172871+0000","last_change":"2024-04-16T17:48:27.307428+0000","last_active":"2024-04-16T19:33:31.172871+0000","last_peered":"2024-04-16T19:33:31.172871+0000","last_clean":"2024-04-16T19:33:31.172871+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:33:31.172871+0000","last_undegraded":"2024-04-16T19:33:31.172871+0000","last_fullsized":"2024-04-16T19:33:31.172871+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:33.963133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161215+0000","last_change":"2024-04-16T17:48:01.240690+0000","last_active":"2024-04-16T19:33:01.161215+0000","last_peered":"2024-04-16T19:33:01.161215+0000","last_clean":"2024-04-16T19:33:01.161215+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:33:01.161215+0000","last_undegraded":"2024-04-16T19:33:01.161215+0000","last_fullsized":"2024-04-16T19:33:01.161215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:42.135971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162598+0000","last_change":"2024-04-16T17:47:59.192669+0000","last_active":"2024-04-16T19:33:01.162598+0000","last_peered":"2024-04-16T19:33:01.162598+0000","last_clean":"2024-04-16T19:33:01.162598+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:33:01.162598+0000","last_undegraded":"2024-04-16T19:33:01.162598+0000","last_fullsized":"2024-04-16T19:33:01.162598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'850","reported_seq":1151,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161341+0000","last_change":"2024-04-16T17:47:57.193698+0000","last_active":"2024-04-16T19:33:01.161341+0000","last_peered":"2024-04-16T19:33:01.161341+0000","last_clean":"2024-04-16T19:33:01.161341+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:33:01.161341+0000","last_undegraded":"2024-04-16T19:33:01.161341+0000","last_fullsized":"2024-04-16T19:33:01.161341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:15.692887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'16","reported_seq":234,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.178756+0000","last_change":"2024-04-16T17:49:32.398153+0000","last_active":"2024-04-16T19:34:11.178756+0000","last_peered":"2024-04-16T19:34:11.178756+0000","last_clean":"2024-04-16T19:34:11.178756+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:34:11.178756+0000","last_undegraded":"2024-04-16T19:34:11.178756+0000","last_fullsized":"2024-04-16T19:34:11.178756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:45.378036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178826+0000","last_change":"2024-04-16T17:47:39.336834+0000","last_active":"2024-04-16T19:33:56.178826+0000","last_peered":"2024-04-16T19:33:56.178826+0000","last_clean":"2024-04-16T19:33:56.178826+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:33:56.178826+0000","last_undegraded":"2024-04-16T19:33:56.178826+0000","last_fullsized":"2024-04-16T19:33:56.178826+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:09.285627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152606+0000","last_change":"2024-04-16T17:47:23.257996+0000","last_active":"2024-04-16T19:32:26.152606+0000","last_peered":"2024-04-16T19:32:26.152606+0000","last_clean":"2024-04-16T19:32:26.152606+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:32:26.152606+0000","last_undegraded":"2024-04-16T19:32:26.152606+0000","last_fullsized":"2024-04-16T19:32:26.152606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166359+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:32:56.166359+0000","last_peered":"2024-04-16T19:32:56.166359+0000","last_clean":"2024-04-16T19:32:56.166359+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:32:56.166359+0000","last_undegraded":"2024-04-16T19:32:56.166359+0000","last_fullsized":"2024-04-16T19:32:56.166359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"48'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159071+0000","last_change":"2024-04-16T17:47:33.297781+0000","last_active":"2024-04-16T19:32:36.159071+0000","last_peered":"2024-04-16T19:32:36.159071+0000","last_clean":"2024-04-16T19:32:36.159071+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:32:36.159071+0000","last_undegraded":"2024-04-16T19:32:36.159071+0000","last_fullsized":"2024-04-16T19:32:36.159071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:51.904305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142052+0000","last_change":"2024-04-16T17:47:35.310182+0000","last_active":"2024-04-16T19:32:36.142052+0000","last_peered":"2024-04-16T19:32:36.142052+0000","last_clean":"2024-04-16T19:32:36.142052+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:32:36.142052+0000","last_undegraded":"2024-04-16T19:32:36.142052+0000","last_fullsized":"2024-04-16T19:32:36.142052+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:57:05.362295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160301+0000","last_change":"2024-04-16T17:47:37.334854+0000","last_active":"2024-04-16T19:32:41.160301+0000","last_peered":"2024-04-16T19:32:41.160301+0000","last_clean":"2024-04-16T19:32:41.160301+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:32:41.160301+0000","last_undegraded":"2024-04-16T19:32:41.160301+0000","last_fullsized":"2024-04-16T19:32:41.160301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:24.862379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168290+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:33:31.168290+0000","last_peered":"2024-04-16T19:33:31.168290+0000","last_clean":"2024-04-16T19:33:31.168290+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:33:31.168290+0000","last_undegraded":"2024-04-16T19:33:31.168290+0000","last_fullsized":"2024-04-16T19:33:31.168290+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:11.558629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'55","reported_seq":671,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157685+0000","last_change":"2024-04-16T17:48:34.343212+0000","last_active":"2024-04-16T19:32:31.157685+0000","last_peered":"2024-04-16T19:32:31.157685+0000","last_clean":"2024-04-16T19:32:31.157685+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:32:31.157685+0000","last_undegraded":"2024-04-16T19:32:31.157685+0000","last_fullsized":"2024-04-16T19:32:31.157685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:28:18.790386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168495+0000","last_change":"2024-04-16T17:48:27.301479+0000","last_active":"2024-04-16T19:33:31.168495+0000","last_peered":"2024-04-16T19:33:31.168495+0000","last_clean":"2024-04-16T19:33:31.168495+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:33:31.168495+0000","last_undegraded":"2024-04-16T19:33:31.168495+0000","last_fullsized":"2024-04-16T19:33:31.168495+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:04.776813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'9","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161137+0000","last_change":"2024-04-16T17:48:01.250254+0000","last_active":"2024-04-16T19:33:01.161137+0000","last_peered":"2024-04-16T19:33:01.161137+0000","last_clean":"2024-04-16T19:33:01.161137+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:33:01.161137+0000","last_undegraded":"2024-04-16T19:33:01.161137+0000","last_fullsized":"2024-04-16T19:33:01.161137+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:53.163721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167775+0000","last_change":"2024-04-16T17:47:59.176369+0000","last_active":"2024-04-16T19:33:01.167775+0000","last_peered":"2024-04-16T19:33:01.167775+0000","last_clean":"2024-04-16T19:33:01.167775+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:33:01.167775+0000","last_undegraded":"2024-04-16T19:33:01.167775+0000","last_fullsized":"2024-04-16T19:33:01.167775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:52.164568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'592","reported_seq":2279,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162562+0000","last_change":"2024-04-16T17:47:57.185970+0000","last_active":"2024-04-16T19:33:01.162562+0000","last_peered":"2024-04-16T19:33:01.162562+0000","last_clean":"2024-04-16T19:33:01.162562+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:33:01.162562+0000","last_undegraded":"2024-04-16T19:33:01.162562+0000","last_fullsized":"2024-04-16T19:33:01.162562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:33.418851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"51'2","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.151881+0000","last_change":"2024-04-16T17:47:55.163430+0000","last_active":"2024-04-16T19:32:06.151881+0000","last_peered":"2024-04-16T19:32:06.151881+0000","last_clean":"2024-04-16T19:32:06.151881+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:32:06.151881+0000","last_undegraded":"2024-04-16T19:32:06.151881+0000","last_fullsized":"2024-04-16T19:32:06.151881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183840+0000","last_change":"2024-04-16T17:47:39.338758+0000","last_active":"2024-04-16T19:34:11.183840+0000","last_peered":"2024-04-16T19:34:11.183840+0000","last_clean":"2024-04-16T19:34:11.183840+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:34:11.183840+0000","last_undegraded":"2024-04-16T19:34:11.183840+0000","last_fullsized":"2024-04-16T19:34:11.183840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156609+0000","last_change":"2024-04-16T17:47:23.260667+0000","last_active":"2024-04-16T19:32:26.156609+0000","last_peered":"2024-04-16T19:32:26.156609+0000","last_clean":"2024-04-16T19:32:26.156609+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:32:26.156609+0000","last_undegraded":"2024-04-16T19:32:26.156609+0000","last_fullsized":"2024-04-16T19:32:26.156609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177933+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:33:41.177933+0000","last_peered":"2024-04-16T19:33:41.177933+0000","last_clean":"2024-04-16T19:33:41.177933+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:33:41.177933+0000","last_undegraded":"2024-04-16T19:33:41.177933+0000","last_fullsized":"2024-04-16T19:33:41.177933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"48'1","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159182+0000","last_change":"2024-04-16T17:47:33.297748+0000","last_active":"2024-04-16T19:32:36.159182+0000","last_peered":"2024-04-16T19:32:36.159182+0000","last_clean":"2024-04-16T19:32:36.159182+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:32:36.159182+0000","last_undegraded":"2024-04-16T19:32:36.159182+0000","last_fullsized":"2024-04-16T19:32:36.159182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:36:58.076852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'50","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158106+0000","last_change":"2024-04-16T17:47:35.314682+0000","last_active":"2024-04-16T19:32:36.158106+0000","last_peered":"2024-04-16T19:32:36.158106+0000","last_clean":"2024-04-16T19:32:36.158106+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:32:36.158106+0000","last_undegraded":"2024-04-16T19:32:36.158106+0000","last_fullsized":"2024-04-16T19:32:36.158106+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162296+0000","last_change":"2024-04-16T17:47:37.327586+0000","last_active":"2024-04-16T19:32:41.162296+0000","last_peered":"2024-04-16T19:32:41.162296+0000","last_clean":"2024-04-16T19:32:41.162296+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:32:41.162296+0000","last_undegraded":"2024-04-16T19:32:41.162296+0000","last_fullsized":"2024-04-16T19:32:41.162296+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167526+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:33:21.167526+0000","last_peered":"2024-04-16T19:33:21.167526+0000","last_clean":"2024-04-16T19:33:21.167526+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:33:21.167526+0000","last_undegraded":"2024-04-16T19:33:21.167526+0000","last_fullsized":"2024-04-16T19:33:21.167526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:07.939676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'99","reported_seq":1687,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174223+0000","last_change":"2024-04-16T17:48:34.338854+0000","last_active":"2024-04-16T19:33:36.174223+0000","last_peered":"2024-04-16T19:33:36.174223+0000","last_clean":"2024-04-16T19:33:36.174223+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:33:36.174223+0000","last_undegraded":"2024-04-16T19:33:36.174223+0000","last_fullsized":"2024-04-16T19:33:36.174223+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:41.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153090+0000","last_change":"2024-04-16T17:48:27.301694+0000","last_active":"2024-04-16T19:32:26.153090+0000","last_peered":"2024-04-16T19:32:26.153090+0000","last_clean":"2024-04-16T19:32:26.153090+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:32:26.153090+0000","last_undegraded":"2024-04-16T19:32:26.153090+0000","last_fullsized":"2024-04-16T19:32:26.153090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'13","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156916+0000","last_change":"2024-04-16T17:48:01.248700+0000","last_active":"2024-04-16T19:32:31.156916+0000","last_peered":"2024-04-16T19:32:31.156916+0000","last_clean":"2024-04-16T19:32:31.156916+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:32:31.156916+0000","last_undegraded":"2024-04-16T19:32:31.156916+0000","last_fullsized":"2024-04-16T19:32:31.156916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'20","reported_seq":1421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153482+0000","last_change":"2024-04-16T17:47:59.196995+0000","last_active":"2024-04-16T19:32:26.153482+0000","last_peered":"2024-04-16T19:32:26.153482+0000","last_clean":"2024-04-16T19:32:26.153482+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:32:26.153482+0000","last_undegraded":"2024-04-16T19:32:26.153482+0000","last_fullsized":"2024-04-16T19:32:26.153482+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.685479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'6983","reported_seq":10557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154510+0000","last_change":"2024-04-16T17:47:57.195602+0000","last_active":"2024-04-16T19:32:11.154510+0000","last_peered":"2024-04-16T19:32:11.154510+0000","last_clean":"2024-04-16T19:32:11.154510+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:32:11.154510+0000","last_undegraded":"2024-04-16T19:32:11.154510+0000","last_fullsized":"2024-04-16T19:32:11.154510+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1783,"log_dups_size":3000,"ondisk_log_size":1783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:48:37.455011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'6","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163653+0000","last_change":"2024-04-16T17:47:55.162215+0000","last_active":"2024-04-16T19:32:56.163653+0000","last_peered":"2024-04-16T19:32:56.163653+0000","last_clean":"2024-04-16T19:32:56.163653+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:32:56.163653+0000","last_undegraded":"2024-04-16T19:32:56.163653+0000","last_fullsized":"2024-04-16T19:32:56.163653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159959+0000","last_change":"2024-04-16T17:47:39.329121+0000","last_active":"2024-04-16T19:32:41.159959+0000","last_peered":"2024-04-16T19:32:41.159959+0000","last_clean":"2024-04-16T19:32:41.159959+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:32:41.159959+0000","last_undegraded":"2024-04-16T19:32:41.159959+0000","last_fullsized":"2024-04-16T19:32:41.159959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139757+0000","last_change":"2024-04-16T17:47:23.269620+0000","last_active":"2024-04-16T19:32:26.139757+0000","last_peered":"2024-04-16T19:32:26.139757+0000","last_clean":"2024-04-16T19:32:26.139757+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:32:26.139757+0000","last_undegraded":"2024-04-16T19:32:26.139757+0000","last_fullsized":"2024-04-16T19:32:26.139757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:43.201589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172841+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:33:41.172841+0000","last_peered":"2024-04-16T19:33:41.172841+0000","last_clean":"2024-04-16T19:33:41.172841+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:33:41.172841+0000","last_undegraded":"2024-04-16T19:33:41.172841+0000","last_fullsized":"2024-04-16T19:33:41.172841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155004+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:32:36.155004+0000","last_peered":"2024-04-16T19:32:36.155004+0000","last_clean":"2024-04-16T19:32:36.155004+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:32:36.155004+0000","last_undegraded":"2024-04-16T19:32:36.155004+0000","last_fullsized":"2024-04-16T19:32:36.155004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:14.585091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142192+0000","last_change":"2024-04-16T17:47:35.310781+0000","last_active":"2024-04-16T19:32:36.142192+0000","last_peered":"2024-04-16T19:32:36.142192+0000","last_clean":"2024-04-16T19:32:36.142192+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:32:36.142192+0000","last_undegraded":"2024-04-16T19:32:36.142192+0000","last_fullsized":"2024-04-16T19:32:36.142192+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:20.569781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157441+0000","last_change":"2024-04-16T17:47:37.322773+0000","last_active":"2024-04-16T19:32:41.157441+0000","last_peered":"2024-04-16T19:32:41.157441+0000","last_clean":"2024-04-16T19:32:41.157441+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:32:41.157441+0000","last_undegraded":"2024-04-16T19:32:41.157441+0000","last_fullsized":"2024-04-16T19:32:41.157441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172568+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:33:21.172568+0000","last_peered":"2024-04-16T19:33:21.172568+0000","last_clean":"2024-04-16T19:33:21.172568+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:33:21.172568+0000","last_undegraded":"2024-04-16T19:33:21.172568+0000","last_fullsized":"2024-04-16T19:33:21.172568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147896+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:32:06.147896+0000","last_peered":"2024-04-16T19:32:06.147896+0000","last_clean":"2024-04-16T19:32:06.147896+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:32:06.147896+0000","last_undegraded":"2024-04-16T19:32:06.147896+0000","last_fullsized":"2024-04-16T19:32:06.147896+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:46:24.982845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146589+0000","last_change":"2024-04-16T17:48:27.293279+0000","last_active":"2024-04-16T19:32:56.146589+0000","last_peered":"2024-04-16T19:32:56.146589+0000","last_clean":"2024-04-16T19:32:56.146589+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:32:56.146589+0000","last_undegraded":"2024-04-16T19:32:56.146589+0000","last_fullsized":"2024-04-16T19:32:56.146589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:44.442181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"48'8","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148529+0000","last_change":"2024-04-16T17:48:01.237052+0000","last_active":"2024-04-16T19:31:56.148529+0000","last_peered":"2024-04-16T19:31:56.148529+0000","last_clean":"2024-04-16T19:31:56.148529+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:31:56.148529+0000","last_undegraded":"2024-04-16T19:31:56.148529+0000","last_fullsized":"2024-04-16T19:31:56.148529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:22.523578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'555","reported_seq":1956,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136598+0000","last_change":"2024-04-16T17:47:59.187458+0000","last_active":"2024-04-16T19:32:11.136598+0000","last_peered":"2024-04-16T19:32:11.136598+0000","last_clean":"2024-04-16T19:32:11.136598+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:32:11.136598+0000","last_undegraded":"2024-04-16T19:32:11.136598+0000","last_fullsized":"2024-04-16T19:32:11.136598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:17.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'544","reported_seq":775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158452+0000","last_change":"2024-04-16T17:47:57.184810+0000","last_active":"2024-04-16T19:32:36.158452+0000","last_peered":"2024-04-16T19:32:36.158452+0000","last_clean":"2024-04-16T19:32:36.158452+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:32:36.158452+0000","last_undegraded":"2024-04-16T19:32:36.158452+0000","last_fullsized":"2024-04-16T19:32:36.158452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"51'4","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163711+0000","last_change":"2024-04-16T17:47:55.183376+0000","last_active":"2024-04-16T19:32:56.163711+0000","last_peered":"2024-04-16T19:32:56.163711+0000","last_clean":"2024-04-16T19:32:56.163711+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:32:56.163711+0000","last_undegraded":"2024-04-16T19:32:56.163711+0000","last_fullsized":"2024-04-16T19:32:56.163711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157536+0000","last_change":"2024-04-16T17:47:39.339617+0000","last_active":"2024-04-16T19:32:41.157536+0000","last_peered":"2024-04-16T19:32:41.157536+0000","last_clean":"2024-04-16T19:32:41.157536+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:32:41.157536+0000","last_undegraded":"2024-04-16T19:32:41.157536+0000","last_fullsized":"2024-04-16T19:32:41.157536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156357+0000","last_change":"2024-04-16T17:47:23.263370+0000","last_active":"2024-04-16T19:32:26.156357+0000","last_peered":"2024-04-16T19:32:26.156357+0000","last_clean":"2024-04-16T19:32:26.156357+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:32:26.156357+0000","last_undegraded":"2024-04-16T19:32:26.156357+0000","last_fullsized":"2024-04-16T19:32:26.156357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174981+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:33:41.174981+0000","last_peered":"2024-04-16T19:33:41.174981+0000","last_clean":"2024-04-16T19:33:41.174981+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:33:41.174981+0000","last_undegraded":"2024-04-16T19:33:41.174981+0000","last_fullsized":"2024-04-16T19:33:41.174981+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160793+0000","last_change":"2024-04-16T17:47:33.307304+0000","last_active":"2024-04-16T19:32:36.160793+0000","last_peered":"2024-04-16T19:32:36.160793+0000","last_clean":"2024-04-16T19:32:36.160793+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:32:36.160793+0000","last_undegraded":"2024-04-16T19:32:36.160793+0000","last_fullsized":"2024-04-16T19:32:36.160793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156191+0000","last_change":"2024-04-16T17:47:35.311451+0000","last_active":"2024-04-16T19:32:36.156191+0000","last_peered":"2024-04-16T19:32:36.156191+0000","last_clean":"2024-04-16T19:32:36.156191+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:32:36.156191+0000","last_undegraded":"2024-04-16T19:32:36.156191+0000","last_fullsized":"2024-04-16T19:32:36.156191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159764+0000","last_change":"2024-04-16T17:47:37.326333+0000","last_active":"2024-04-16T19:32:41.159764+0000","last_peered":"2024-04-16T19:32:41.159764+0000","last_clean":"2024-04-16T19:32:41.159764+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:32:41.159764+0000","last_undegraded":"2024-04-16T19:32:41.159764+0000","last_fullsized":"2024-04-16T19:32:41.159764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167717+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:33:01.167717+0000","last_peered":"2024-04-16T19:33:01.167717+0000","last_clean":"2024-04-16T19:33:01.167717+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:33:01.167717+0000","last_undegraded":"2024-04-16T19:33:01.167717+0000","last_fullsized":"2024-04-16T19:33:01.167717+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'89","reported_seq":694,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159707+0000","last_change":"2024-04-16T17:48:34.328117+0000","last_active":"2024-04-16T19:32:31.159707+0000","last_peered":"2024-04-16T19:32:31.159707+0000","last_clean":"2024-04-16T19:32:31.159707+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:32:31.159707+0000","last_undegraded":"2024-04-16T19:32:31.159707+0000","last_fullsized":"2024-04-16T19:32:31.159707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155893+0000","last_change":"2024-04-16T17:48:27.295608+0000","last_active":"2024-04-16T19:32:26.155893+0000","last_peered":"2024-04-16T19:32:26.155893+0000","last_clean":"2024-04-16T19:32:26.155893+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:32:26.155893+0000","last_undegraded":"2024-04-16T19:32:26.155893+0000","last_fullsized":"2024-04-16T19:32:26.155893+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'12","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145278+0000","last_change":"2024-04-16T17:48:01.246592+0000","last_active":"2024-04-16T19:31:56.145278+0000","last_peered":"2024-04-16T19:31:56.145278+0000","last_clean":"2024-04-16T19:31:56.145278+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:31:56.145278+0000","last_undegraded":"2024-04-16T19:31:56.145278+0000","last_fullsized":"2024-04-16T19:31:56.145278+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:06.492939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164653+0000","last_change":"2024-04-16T17:47:59.178449+0000","last_active":"2024-04-16T19:33:01.164653+0000","last_peered":"2024-04-16T19:33:01.164653+0000","last_clean":"2024-04-16T19:33:01.164653+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:33:01.164653+0000","last_undegraded":"2024-04-16T19:33:01.164653+0000","last_fullsized":"2024-04-16T19:33:01.164653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'457","reported_seq":4410,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162217+0000","last_change":"2024-04-16T17:47:57.181693+0000","last_active":"2024-04-16T19:33:01.162217+0000","last_peered":"2024-04-16T19:33:01.162217+0000","last_clean":"2024-04-16T19:33:01.162217+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:33:01.162217+0000","last_undegraded":"2024-04-16T19:33:01.162217+0000","last_fullsized":"2024-04-16T19:33:01.162217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:44.694025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'6","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161031+0000","last_change":"2024-04-16T17:47:55.174874+0000","last_active":"2024-04-16T19:32:56.161031+0000","last_peered":"2024-04-16T19:32:56.161031+0000","last_clean":"2024-04-16T19:32:56.161031+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:32:56.161031+0000","last_undegraded":"2024-04-16T19:32:56.161031+0000","last_fullsized":"2024-04-16T19:32:56.161031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178208+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:33:41.178208+0000","last_peered":"2024-04-16T19:33:41.178208+0000","last_clean":"2024-04-16T19:33:41.178208+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:33:41.178208+0000","last_undegraded":"2024-04-16T19:33:41.178208+0000","last_fullsized":"2024-04-16T19:33:41.178208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:37.592633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157093+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:33:41.157093+0000","last_peered":"2024-04-16T19:33:41.157093+0000","last_clean":"2024-04-16T19:33:41.157093+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:33:41.157093+0000","last_undegraded":"2024-04-16T19:33:41.157093+0000","last_fullsized":"2024-04-16T19:33:41.157093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:08.399471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182918+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:34:11.182918+0000","last_peered":"2024-04-16T19:34:11.182918+0000","last_clean":"2024-04-16T19:34:11.182918+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:34:11.182918+0000","last_undegraded":"2024-04-16T19:34:11.182918+0000","last_fullsized":"2024-04-16T19:34:11.182918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'88","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159309+0000","last_change":"2024-04-16T17:47:35.319036+0000","last_active":"2024-04-16T19:32:36.159309+0000","last_peered":"2024-04-16T19:32:36.159309+0000","last_clean":"2024-04-16T19:32:36.159309+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:32:36.159309+0000","last_undegraded":"2024-04-16T19:32:36.159309+0000","last_fullsized":"2024-04-16T19:32:36.159309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162487+0000","last_change":"2024-04-16T17:47:37.330847+0000","last_active":"2024-04-16T19:32:41.162487+0000","last_peered":"2024-04-16T19:32:41.162487+0000","last_clean":"2024-04-16T19:32:41.162487+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:32:41.162487+0000","last_undegraded":"2024-04-16T19:32:41.162487+0000","last_fullsized":"2024-04-16T19:32:41.162487+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:10.138006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160442+0000","last_change":"2024-04-16T17:47:39.701654+0000","last_active":"2024-04-16T19:32:41.160442+0000","last_peered":"2024-04-16T19:32:41.160442+0000","last_clean":"2024-04-16T19:32:41.160442+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:32:41.160442+0000","last_undegraded":"2024-04-16T19:32:41.160442+0000","last_fullsized":"2024-04-16T19:32:41.160442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162629+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:33:01.162629+0000","last_peered":"2024-04-16T19:33:01.162629+0000","last_clean":"2024-04-16T19:33:01.162629+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:33:01.162629+0000","last_undegraded":"2024-04-16T19:33:01.162629+0000","last_fullsized":"2024-04-16T19:33:01.162629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:28:06.366066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'71","reported_seq":966,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143796+0000","last_change":"2024-04-16T17:48:34.313054+0000","last_active":"2024-04-16T19:31:51.143796+0000","last_peered":"2024-04-16T19:31:51.143796+0000","last_clean":"2024-04-16T19:31:51.143796+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:31:51.143796+0000","last_undegraded":"2024-04-16T19:31:51.143796+0000","last_fullsized":"2024-04-16T19:31:51.143796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:11.966580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153539+0000","last_change":"2024-04-16T17:48:27.309014+0000","last_active":"2024-04-16T19:32:26.153539+0000","last_peered":"2024-04-16T19:32:26.153539+0000","last_clean":"2024-04-16T19:32:26.153539+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:32:26.153539+0000","last_undegraded":"2024-04-16T19:32:26.153539+0000","last_fullsized":"2024-04-16T19:32:26.153539+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'4","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145295+0000","last_change":"2024-04-16T17:48:01.251229+0000","last_active":"2024-04-16T19:31:56.145295+0000","last_peered":"2024-04-16T19:31:56.145295+0000","last_clean":"2024-04-16T19:31:56.145295+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:31:56.145295+0000","last_undegraded":"2024-04-16T19:31:56.145295+0000","last_fullsized":"2024-04-16T19:31:56.145295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147763+0000","last_change":"2024-04-16T17:47:59.174249+0000","last_active":"2024-04-16T19:33:01.147763+0000","last_peered":"2024-04-16T19:33:01.147763+0000","last_clean":"2024-04-16T19:33:01.147763+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:33:01.147763+0000","last_undegraded":"2024-04-16T19:33:01.147763+0000","last_fullsized":"2024-04-16T19:33:01.147763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:54.272783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11370","reported_seq":61791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161030+0000","last_change":"2024-04-16T17:47:57.192920+0000","last_active":"2024-04-16T19:33:01.161030+0000","last_peered":"2024-04-16T19:33:01.161030+0000","last_clean":"2024-04-16T19:33:01.161030+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:33:01.161030+0000","last_undegraded":"2024-04-16T19:33:01.161030+0000","last_fullsized":"2024-04-16T19:33:01.161030+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1670,"log_dups_size":3000,"ondisk_log_size":1670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:13.718911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'12","reported_seq":338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166321+0000","last_change":"2024-04-16T17:47:55.178367+0000","last_active":"2024-04-16T19:32:56.166321+0000","last_peered":"2024-04-16T19:32:56.166321+0000","last_clean":"2024-04-16T19:32:56.166321+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:32:56.166321+0000","last_undegraded":"2024-04-16T19:32:56.166321+0000","last_fullsized":"2024-04-16T19:32:56.166321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.180159+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:34:01.180159+0000","last_peered":"2024-04-16T19:34:01.180159+0000","last_clean":"2024-04-16T19:34:01.180159+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:34:01.180159+0000","last_undegraded":"2024-04-16T19:34:01.180159+0000","last_fullsized":"2024-04-16T19:34:01.180159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145890+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:31:46.145890+0000","last_peered":"2024-04-16T19:31:46.145890+0000","last_clean":"2024-04-16T19:31:46.145890+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:31:46.145890+0000","last_undegraded":"2024-04-16T19:31:46.145890+0000","last_fullsized":"2024-04-16T19:31:46.145890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181077+0000","last_change":"2024-04-16T17:47:33.296653+0000","last_active":"2024-04-16T19:33:51.181077+0000","last_peered":"2024-04-16T19:33:51.181077+0000","last_clean":"2024-04-16T19:33:51.181077+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:33:51.181077+0000","last_undegraded":"2024-04-16T19:33:51.181077+0000","last_fullsized":"2024-04-16T19:33:51.181077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:00.330068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'152","reported_seq":340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160930+0000","last_change":"2024-04-16T17:47:35.317123+0000","last_active":"2024-04-16T19:32:36.160930+0000","last_peered":"2024-04-16T19:32:36.160930+0000","last_clean":"2024-04-16T19:32:36.160930+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:32:36.160930+0000","last_undegraded":"2024-04-16T19:32:36.160930+0000","last_fullsized":"2024-04-16T19:32:36.160930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160571+0000","last_change":"2024-04-16T17:47:37.334019+0000","last_active":"2024-04-16T19:32:41.160571+0000","last_peered":"2024-04-16T19:32:41.160571+0000","last_clean":"2024-04-16T19:32:41.160571+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:32:41.160571+0000","last_undegraded":"2024-04-16T19:32:41.160571+0000","last_fullsized":"2024-04-16T19:32:41.160571+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:07.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162425+0000","last_change":"2024-04-16T17:47:39.699556+0000","last_active":"2024-04-16T19:32:41.162425+0000","last_peered":"2024-04-16T19:32:41.162425+0000","last_clean":"2024-04-16T19:32:41.162425+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:32:41.162425+0000","last_undegraded":"2024-04-16T19:32:41.162425+0000","last_fullsized":"2024-04-16T19:32:41.162425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165020+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:33:01.165020+0000","last_peered":"2024-04-16T19:33:01.165020+0000","last_clean":"2024-04-16T19:33:01.165020+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:33:01.165020+0000","last_undegraded":"2024-04-16T19:33:01.165020+0000","last_fullsized":"2024-04-16T19:33:01.165020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'103","reported_seq":1819,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158185+0000","last_change":"2024-04-16T17:48:34.337561+0000","last_active":"2024-04-16T19:32:31.158185+0000","last_peered":"2024-04-16T19:32:31.158185+0000","last_clean":"2024-04-16T19:32:31.158185+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:32:31.158185+0000","last_undegraded":"2024-04-16T19:32:31.158185+0000","last_fullsized":"2024-04-16T19:32:31.158185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:01.320649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145725+0000","last_change":"2024-04-16T17:48:27.293905+0000","last_active":"2024-04-16T19:31:46.145725+0000","last_peered":"2024-04-16T19:31:46.145725+0000","last_clean":"2024-04-16T19:31:46.145725+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:31:46.145725+0000","last_undegraded":"2024-04-16T19:31:46.145725+0000","last_fullsized":"2024-04-16T19:31:46.145725+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'8","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145241+0000","last_change":"2024-04-16T17:48:01.248531+0000","last_active":"2024-04-16T19:31:56.145241+0000","last_peered":"2024-04-16T19:31:56.145241+0000","last_clean":"2024-04-16T19:31:56.145241+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:31:56.145241+0000","last_undegraded":"2024-04-16T19:31:56.145241+0000","last_fullsized":"2024-04-16T19:31:56.145241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167662+0000","last_change":"2024-04-16T17:47:59.187755+0000","last_active":"2024-04-16T19:33:01.167662+0000","last_peered":"2024-04-16T19:33:01.167662+0000","last_clean":"2024-04-16T19:33:01.167662+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:33:01.167662+0000","last_undegraded":"2024-04-16T19:33:01.167662+0000","last_fullsized":"2024-04-16T19:33:01.167662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:25.460755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'196","reported_seq":421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165564+0000","last_change":"2024-04-16T17:47:57.195138+0000","last_active":"2024-04-16T19:33:01.165564+0000","last_peered":"2024-04-16T19:33:01.165564+0000","last_clean":"2024-04-16T19:33:01.165564+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:33:01.165564+0000","last_undegraded":"2024-04-16T19:33:01.165564+0000","last_fullsized":"2024-04-16T19:33:01.165564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:36.309438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'12","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159825+0000","last_change":"2024-04-16T17:47:55.160472+0000","last_active":"2024-04-16T19:32:56.159825+0000","last_peered":"2024-04-16T19:32:56.159825+0000","last_clean":"2024-04-16T19:32:56.159825+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:32:56.159825+0000","last_undegraded":"2024-04-16T19:32:56.159825+0000","last_fullsized":"2024-04-16T19:32:56.159825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:00.875301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155799+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:32:26.155799+0000","last_peered":"2024-04-16T19:32:26.155799+0000","last_clean":"2024-04-16T19:32:26.155799+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:32:26.155799+0000","last_undegraded":"2024-04-16T19:32:26.155799+0000","last_fullsized":"2024-04-16T19:32:26.155799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151302+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:32:21.151302+0000","last_peered":"2024-04-16T19:32:21.151302+0000","last_clean":"2024-04-16T19:32:21.151302+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:32:21.151302+0000","last_undegraded":"2024-04-16T19:32:21.151302+0000","last_fullsized":"2024-04-16T19:32:21.151302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:44.777444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"48'5","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155232+0000","last_change":"2024-04-16T17:47:33.306460+0000","last_active":"2024-04-16T19:32:36.155232+0000","last_peered":"2024-04-16T19:32:36.155232+0000","last_clean":"2024-04-16T19:32:36.155232+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:32:36.155232+0000","last_undegraded":"2024-04-16T19:32:36.155232+0000","last_fullsized":"2024-04-16T19:32:36.155232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:15.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'104","reported_seq":277,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160070+0000","last_change":"2024-04-16T17:47:35.314274+0000","last_active":"2024-04-16T19:33:51.160070+0000","last_peered":"2024-04-16T19:33:51.160070+0000","last_clean":"2024-04-16T19:33:51.160070+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:33:51.160070+0000","last_undegraded":"2024-04-16T19:33:51.160070+0000","last_fullsized":"2024-04-16T19:33:51.160070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:58.087539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183865+0000","last_change":"2024-04-16T17:47:37.331871+0000","last_active":"2024-04-16T19:34:11.183865+0000","last_peered":"2024-04-16T19:34:11.183865+0000","last_clean":"2024-04-16T19:34:11.183865+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:34:11.183865+0000","last_undegraded":"2024-04-16T19:34:11.183865+0000","last_fullsized":"2024-04-16T19:34:11.183865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:10.644125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162350+0000","last_change":"2024-04-16T17:47:39.323414+0000","last_active":"2024-04-16T19:32:41.162350+0000","last_peered":"2024-04-16T19:32:41.162350+0000","last_clean":"2024-04-16T19:32:41.162350+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:32:41.162350+0000","last_undegraded":"2024-04-16T19:32:41.162350+0000","last_fullsized":"2024-04-16T19:32:41.162350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177321+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:33:46.177321+0000","last_peered":"2024-04-16T19:33:46.177321+0000","last_clean":"2024-04-16T19:33:46.177321+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:33:46.177321+0000","last_undegraded":"2024-04-16T19:33:46.177321+0000","last_fullsized":"2024-04-16T19:33:46.177321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:41.778594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'68","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158123+0000","last_change":"2024-04-16T17:48:34.343716+0000","last_active":"2024-04-16T19:32:31.158123+0000","last_peered":"2024-04-16T19:32:31.158123+0000","last_clean":"2024-04-16T19:32:31.158123+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:32:31.158123+0000","last_undegraded":"2024-04-16T19:32:31.158123+0000","last_fullsized":"2024-04-16T19:32:31.158123+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:48.210902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152776+0000","last_change":"2024-04-16T17:48:27.305044+0000","last_active":"2024-04-16T19:32:26.152776+0000","last_peered":"2024-04-16T19:32:26.152776+0000","last_clean":"2024-04-16T19:32:26.152776+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:32:26.152776+0000","last_undegraded":"2024-04-16T19:32:26.152776+0000","last_fullsized":"2024-04-16T19:32:26.152776+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:17.183220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"48'1","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168780+0000","last_change":"2024-04-16T17:48:01.254059+0000","last_active":"2024-04-16T19:33:26.168780+0000","last_peered":"2024-04-16T19:33:26.168780+0000","last_clean":"2024-04-16T19:33:26.168780+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:33:26.168780+0000","last_undegraded":"2024-04-16T19:33:26.168780+0000","last_fullsized":"2024-04-16T19:33:26.168780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162464+0000","last_change":"2024-04-16T17:47:59.195638+0000","last_active":"2024-04-16T19:33:01.162464+0000","last_peered":"2024-04-16T19:33:01.162464+0000","last_clean":"2024-04-16T19:33:01.162464+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:33:01.162464+0000","last_undegraded":"2024-04-16T19:33:01.162464+0000","last_fullsized":"2024-04-16T19:33:01.162464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:57.426064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'488","reported_seq":3937,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162374+0000","last_change":"2024-04-16T17:47:57.174503+0000","last_active":"2024-04-16T19:33:01.162374+0000","last_peered":"2024-04-16T19:33:01.162374+0000","last_clean":"2024-04-16T19:33:01.162374+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:33:01.162374+0000","last_undegraded":"2024-04-16T19:33:01.162374+0000","last_fullsized":"2024-04-16T19:33:01.162374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:34.199358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'27","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166245+0000","last_change":"2024-04-16T17:47:55.169780+0000","last_active":"2024-04-16T19:32:56.166245+0000","last_peered":"2024-04-16T19:32:56.166245+0000","last_clean":"2024-04-16T19:32:56.166245+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:32:56.166245+0000","last_undegraded":"2024-04-16T19:32:56.166245+0000","last_fullsized":"2024-04-16T19:32:56.166245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139882+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:32:26.139882+0000","last_peered":"2024-04-16T19:32:26.139882+0000","last_clean":"2024-04-16T19:32:26.139882+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:32:26.139882+0000","last_undegraded":"2024-04-16T19:32:26.139882+0000","last_fullsized":"2024-04-16T19:32:26.139882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:43:58.984431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154419+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:32:21.154419+0000","last_peered":"2024-04-16T19:32:21.154419+0000","last_clean":"2024-04-16T19:32:21.154419+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:32:21.154419+0000","last_undegraded":"2024-04-16T19:32:21.154419+0000","last_fullsized":"2024-04-16T19:32:21.154419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"33'3","reported_seq":802,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159282+0000","last_change":"2024-04-16T17:47:33.297794+0000","last_active":"2024-04-16T19:32:36.159282+0000","last_peered":"2024-04-16T19:32:36.159282+0000","last_clean":"2024-04-16T19:32:36.159282+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:32:36.159282+0000","last_undegraded":"2024-04-16T19:32:36.159282+0000","last_fullsized":"2024-04-16T19:32:36.159282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'120","reported_seq":298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186858+0000","last_change":"2024-04-16T17:47:35.319657+0000","last_active":"2024-04-16T19:34:11.186858+0000","last_peered":"2024-04-16T19:34:11.186858+0000","last_clean":"2024-04-16T19:34:11.186858+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:34:11.186858+0000","last_undegraded":"2024-04-16T19:34:11.186858+0000","last_fullsized":"2024-04-16T19:34:11.186858+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:08.628044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.177090+0000","last_change":"2024-04-16T17:47:37.331215+0000","last_active":"2024-04-16T19:33:56.177090+0000","last_peered":"2024-04-16T19:33:56.177090+0000","last_clean":"2024-04-16T19:33:56.177090+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:33:56.177090+0000","last_undegraded":"2024-04-16T19:33:56.177090+0000","last_fullsized":"2024-04-16T19:33:56.177090+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162223+0000","last_change":"2024-04-16T17:47:39.332386+0000","last_active":"2024-04-16T19:32:41.162223+0000","last_peered":"2024-04-16T19:32:41.162223+0000","last_clean":"2024-04-16T19:32:41.162223+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:32:41.162223+0000","last_undegraded":"2024-04-16T19:32:41.162223+0000","last_fullsized":"2024-04-16T19:32:41.162223+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.153850+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:32:31.153850+0000","last_peered":"2024-04-16T19:32:31.153850+0000","last_clean":"2024-04-16T19:32:31.153850+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:32:31.153850+0000","last_undegraded":"2024-04-16T19:32:31.153850+0000","last_fullsized":"2024-04-16T19:32:31.153850+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:59:18.189132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'86","reported_seq":1557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169733+0000","last_change":"2024-04-16T17:48:34.318628+0000","last_active":"2024-04-16T19:33:36.169733+0000","last_peered":"2024-04-16T19:33:36.169733+0000","last_clean":"2024-04-16T19:33:36.169733+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:33:36.169733+0000","last_undegraded":"2024-04-16T19:33:36.169733+0000","last_fullsized":"2024-04-16T19:33:36.169733+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:15:00.721377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172016+0000","last_change":"2024-04-16T17:48:27.299691+0000","last_active":"2024-04-16T19:33:31.172016+0000","last_peered":"2024-04-16T19:33:31.172016+0000","last_clean":"2024-04-16T19:33:31.172016+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:33:31.172016+0000","last_undegraded":"2024-04-16T19:33:31.172016+0000","last_fullsized":"2024-04-16T19:33:31.172016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:40:17.511736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'6","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165634+0000","last_change":"2024-04-16T17:48:01.246789+0000","last_active":"2024-04-16T19:33:01.165634+0000","last_peered":"2024-04-16T19:33:01.165634+0000","last_clean":"2024-04-16T19:33:01.165634+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:33:01.165634+0000","last_undegraded":"2024-04-16T19:33:01.165634+0000","last_fullsized":"2024-04-16T19:33:01.165634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:47.722149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'555","reported_seq":1953,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176329+0000","last_change":"2024-04-16T17:47:59.182365+0000","last_active":"2024-04-16T19:33:36.176329+0000","last_peered":"2024-04-16T19:33:36.176329+0000","last_clean":"2024-04-16T19:33:36.176329+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:33:36.176329+0000","last_undegraded":"2024-04-16T19:33:36.176329+0000","last_fullsized":"2024-04-16T19:33:36.176329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'492","reported_seq":710,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148336+0000","last_change":"2024-04-16T17:47:57.180106+0000","last_active":"2024-04-16T19:31:56.148336+0000","last_peered":"2024-04-16T19:31:56.148336+0000","last_clean":"2024-04-16T19:31:56.148336+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:31:56.148336+0000","last_undegraded":"2024-04-16T19:31:56.148336+0000","last_fullsized":"2024-04-16T19:31:56.148336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'4","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144008+0000","last_change":"2024-04-16T17:47:55.169894+0000","last_active":"2024-04-16T19:31:51.144008+0000","last_peered":"2024-04-16T19:31:51.144008+0000","last_clean":"2024-04-16T19:31:51.144008+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:31:51.144008+0000","last_undegraded":"2024-04-16T19:31:51.144008+0000","last_fullsized":"2024-04-16T19:31:51.144008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:24.954941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152629+0000","last_change":"2024-04-16T17:47:23.266570+0000","last_active":"2024-04-16T19:32:26.152629+0000","last_peered":"2024-04-16T19:32:26.152629+0000","last_clean":"2024-04-16T19:32:26.152629+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:32:26.152629+0000","last_undegraded":"2024-04-16T19:32:26.152629+0000","last_fullsized":"2024-04-16T19:32:26.152629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:55.441583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152031+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:32:21.152031+0000","last_peered":"2024-04-16T19:32:21.152031+0000","last_clean":"2024-04-16T19:32:21.152031+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:32:21.152031+0000","last_undegraded":"2024-04-16T19:32:21.152031+0000","last_fullsized":"2024-04-16T19:32:21.152031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155126+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:32:36.155126+0000","last_peered":"2024-04-16T19:32:36.155126+0000","last_clean":"2024-04-16T19:32:36.155126+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:32:36.155126+0000","last_undegraded":"2024-04-16T19:32:36.155126+0000","last_fullsized":"2024-04-16T19:32:36.155126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:54.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'152","reported_seq":341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145026+0000","last_change":"2024-04-16T17:47:35.310214+0000","last_active":"2024-04-16T19:31:56.145026+0000","last_peered":"2024-04-16T19:31:56.145026+0000","last_clean":"2024-04-16T19:31:56.145026+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:31:56.145026+0000","last_undegraded":"2024-04-16T19:31:56.145026+0000","last_fullsized":"2024-04-16T19:31:56.145026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.142943+0000","last_change":"2024-04-16T17:47:37.328751+0000","last_active":"2024-04-16T19:31:36.142943+0000","last_peered":"2024-04-16T19:31:36.142943+0000","last_clean":"2024-04-16T19:31:36.142943+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:31:36.142943+0000","last_undegraded":"2024-04-16T19:31:36.142943+0000","last_fullsized":"2024-04-16T19:31:36.142943+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:30:39.392016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144646+0000","last_change":"2024-04-16T17:47:39.336993+0000","last_active":"2024-04-16T19:31:36.144646+0000","last_peered":"2024-04-16T19:31:36.144646+0000","last_clean":"2024-04-16T19:31:36.144646+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:31:36.144646+0000","last_undegraded":"2024-04-16T19:31:36.144646+0000","last_fullsized":"2024-04-16T19:31:36.144646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140940+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:32:31.140940+0000","last_peered":"2024-04-16T19:32:31.140940+0000","last_clean":"2024-04-16T19:32:31.140940+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:32:31.140940+0000","last_undegraded":"2024-04-16T19:32:31.140940+0000","last_fullsized":"2024-04-16T19:32:31.140940+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:16.152440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'99","reported_seq":1431,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174016+0000","last_change":"2024-04-16T17:48:34.341651+0000","last_active":"2024-04-16T19:33:36.174016+0000","last_peered":"2024-04-16T19:33:36.174016+0000","last_clean":"2024-04-16T19:33:36.174016+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:33:36.174016+0000","last_undegraded":"2024-04-16T19:33:36.174016+0000","last_fullsized":"2024-04-16T19:33:36.174016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:45.319846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169918+0000","last_change":"2024-04-16T17:48:27.306641+0000","last_active":"2024-04-16T19:33:31.169918+0000","last_peered":"2024-04-16T19:33:31.169918+0000","last_clean":"2024-04-16T19:33:31.169918+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:33:31.169918+0000","last_undegraded":"2024-04-16T19:33:31.169918+0000","last_fullsized":"2024-04-16T19:33:31.169918+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'7","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164768+0000","last_change":"2024-04-16T17:48:01.263221+0000","last_active":"2024-04-16T19:33:01.164768+0000","last_peered":"2024-04-16T19:33:01.164768+0000","last_clean":"2024-04-16T19:33:01.164768+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:33:01.164768+0000","last_undegraded":"2024-04-16T19:33:01.164768+0000","last_fullsized":"2024-04-16T19:33:01.164768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'555","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132909+0000","last_change":"2024-04-16T17:47:59.183413+0000","last_active":"2024-04-16T19:31:56.132909+0000","last_peered":"2024-04-16T19:31:56.132909+0000","last_clean":"2024-04-16T19:31:56.132909+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:31:56.132909+0000","last_undegraded":"2024-04-16T19:31:56.132909+0000","last_fullsized":"2024-04-16T19:31:56.132909+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:27.860005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'230","reported_seq":460,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172962+0000","last_change":"2024-04-16T17:47:57.173077+0000","last_active":"2024-04-16T19:33:31.172962+0000","last_peered":"2024-04-16T19:33:31.172962+0000","last_clean":"2024-04-16T19:33:31.172962+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:33:31.172962+0000","last_undegraded":"2024-04-16T19:33:31.172962+0000","last_fullsized":"2024-04-16T19:33:31.172962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:49.105681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168703+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:33:31.168703+0000","last_peered":"2024-04-16T19:33:31.168703+0000","last_clean":"2024-04-16T19:33:31.168703+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:33:31.168703+0000","last_undegraded":"2024-04-16T19:33:31.168703+0000","last_fullsized":"2024-04-16T19:33:31.168703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:11.770314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152568+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:32:26.152568+0000","last_peered":"2024-04-16T19:32:26.152568+0000","last_clean":"2024-04-16T19:32:26.152568+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:32:26.152568+0000","last_undegraded":"2024-04-16T19:32:26.152568+0000","last_fullsized":"2024-04-16T19:32:26.152568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:32.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152102+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:32:21.152102+0000","last_peered":"2024-04-16T19:32:21.152102+0000","last_clean":"2024-04-16T19:32:21.152102+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:32:21.152102+0000","last_undegraded":"2024-04-16T19:32:21.152102+0000","last_fullsized":"2024-04-16T19:32:21.152102+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156238+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:32:36.156238+0000","last_peered":"2024-04-16T19:32:36.156238+0000","last_clean":"2024-04-16T19:32:36.156238+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:32:36.156238+0000","last_undegraded":"2024-04-16T19:32:36.156238+0000","last_fullsized":"2024-04-16T19:32:36.156238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367224+0000","last_change":"2024-04-16T17:47:35.303292+0000","last_active":"2024-04-16T19:31:31.367224+0000","last_peered":"2024-04-16T19:31:31.367224+0000","last_clean":"2024-04-16T19:31:31.367224+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:31:31.367224+0000","last_undegraded":"2024-04-16T19:31:31.367224+0000","last_fullsized":"2024-04-16T19:31:31.367224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154187+0000","last_change":"2024-04-16T17:47:37.331029+0000","last_active":"2024-04-16T19:32:11.154187+0000","last_peered":"2024-04-16T19:32:11.154187+0000","last_clean":"2024-04-16T19:32:11.154187+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:32:11.154187+0000","last_undegraded":"2024-04-16T19:32:11.154187+0000","last_fullsized":"2024-04-16T19:32:11.154187+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.143665+0000","last_change":"2024-04-16T17:47:39.700695+0000","last_active":"2024-04-16T19:31:36.143665+0000","last_peered":"2024-04-16T19:31:36.143665+0000","last_clean":"2024-04-16T19:31:36.143665+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:31:36.143665+0000","last_undegraded":"2024-04-16T19:31:36.143665+0000","last_fullsized":"2024-04-16T19:31:36.143665+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157016+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:32:31.157016+0000","last_peered":"2024-04-16T19:32:31.157016+0000","last_clean":"2024-04-16T19:32:31.157016+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:32:31.157016+0000","last_undegraded":"2024-04-16T19:32:31.157016+0000","last_fullsized":"2024-04-16T19:32:31.157016+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'96","reported_seq":919,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156060+0000","last_change":"2024-04-16T17:48:34.324041+0000","last_active":"2024-04-16T19:33:36.156060+0000","last_peered":"2024-04-16T19:33:36.156060+0000","last_clean":"2024-04-16T19:33:36.156060+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:33:36.156060+0000","last_undegraded":"2024-04-16T19:33:36.156060+0000","last_fullsized":"2024-04-16T19:33:36.156060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:18:27.393972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172731+0000","last_change":"2024-04-16T17:48:27.307648+0000","last_active":"2024-04-16T19:33:31.172731+0000","last_peered":"2024-04-16T19:33:31.172731+0000","last_clean":"2024-04-16T19:33:31.172731+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:33:31.172731+0000","last_undegraded":"2024-04-16T19:33:31.172731+0000","last_fullsized":"2024-04-16T19:33:31.172731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:20.265988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'7","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162293+0000","last_change":"2024-04-16T17:48:01.248401+0000","last_active":"2024-04-16T19:33:01.162293+0000","last_peered":"2024-04-16T19:33:01.162293+0000","last_clean":"2024-04-16T19:33:01.162293+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:33:01.162293+0000","last_undegraded":"2024-04-16T19:33:01.162293+0000","last_fullsized":"2024-04-16T19:33:01.162293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145357+0000","last_change":"2024-04-16T17:47:59.199961+0000","last_active":"2024-04-16T19:31:56.145357+0000","last_peered":"2024-04-16T19:31:56.145357+0000","last_clean":"2024-04-16T19:31:56.145357+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:31:56.145357+0000","last_undegraded":"2024-04-16T19:31:56.145357+0000","last_fullsized":"2024-04-16T19:31:56.145357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:44.427887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'129","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150185+0000","last_change":"2024-04-16T17:47:57.189447+0000","last_active":"2024-04-16T19:31:56.150185+0000","last_peered":"2024-04-16T19:31:56.150185+0000","last_clean":"2024-04-16T19:31:56.150185+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:31:56.150185+0000","last_undegraded":"2024-04-16T19:31:56.150185+0000","last_fullsized":"2024-04-16T19:31:56.150185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:55.411174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'3","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171761+0000","last_change":"2024-04-16T17:47:55.160684+0000","last_active":"2024-04-16T19:33:26.171761+0000","last_peered":"2024-04-16T19:33:26.171761+0000","last_clean":"2024-04-16T19:33:26.171761+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:33:26.171761+0000","last_undegraded":"2024-04-16T19:33:26.171761+0000","last_fullsized":"2024-04-16T19:33:26.171761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148167+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:31:56.148167+0000","last_peered":"2024-04-16T19:31:56.148167+0000","last_clean":"2024-04-16T19:31:56.148167+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:31:56.148167+0000","last_undegraded":"2024-04-16T19:31:56.148167+0000","last_fullsized":"2024-04-16T19:31:56.148167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159976+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:33:51.159976+0000","last_peered":"2024-04-16T19:33:51.159976+0000","last_clean":"2024-04-16T19:33:51.159976+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:33:51.159976+0000","last_undegraded":"2024-04-16T19:33:51.159976+0000","last_fullsized":"2024-04-16T19:33:51.159976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:50:28.662489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361145+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:31:31.361145+0000","last_peered":"2024-04-16T19:31:31.361145+0000","last_clean":"2024-04-16T19:31:31.361145+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:31:31.361145+0000","last_undegraded":"2024-04-16T19:31:31.361145+0000","last_fullsized":"2024-04-16T19:31:31.361145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:03.171529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'32","reported_seq":160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158251+0000","last_change":"2024-04-16T17:47:35.317051+0000","last_active":"2024-04-16T19:32:36.158251+0000","last_peered":"2024-04-16T19:32:36.158251+0000","last_clean":"2024-04-16T19:32:36.158251+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:32:36.158251+0000","last_undegraded":"2024-04-16T19:32:36.158251+0000","last_fullsized":"2024-04-16T19:32:36.158251+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162398+0000","last_change":"2024-04-16T17:47:37.320351+0000","last_active":"2024-04-16T19:32:41.162398+0000","last_peered":"2024-04-16T19:32:41.162398+0000","last_clean":"2024-04-16T19:32:41.162398+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:32:41.162398+0000","last_undegraded":"2024-04-16T19:32:41.162398+0000","last_fullsized":"2024-04-16T19:32:41.162398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144501+0000","last_change":"2024-04-16T17:47:39.329617+0000","last_active":"2024-04-16T19:31:36.144501+0000","last_peered":"2024-04-16T19:31:36.144501+0000","last_clean":"2024-04-16T19:31:36.144501+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:31:36.144501+0000","last_undegraded":"2024-04-16T19:31:36.144501+0000","last_fullsized":"2024-04-16T19:31:36.144501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157983+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:32:31.157983+0000","last_peered":"2024-04-16T19:32:31.157983+0000","last_clean":"2024-04-16T19:32:31.157983+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:32:31.157983+0000","last_undegraded":"2024-04-16T19:32:31.157983+0000","last_fullsized":"2024-04-16T19:32:31.157983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:49:40.225240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'126","reported_seq":1450,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156003+0000","last_change":"2024-04-16T17:48:34.338811+0000","last_active":"2024-04-16T19:33:36.156003+0000","last_peered":"2024-04-16T19:33:36.156003+0000","last_clean":"2024-04-16T19:33:36.156003+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:33:36.156003+0000","last_undegraded":"2024-04-16T19:33:36.156003+0000","last_fullsized":"2024-04-16T19:33:36.156003+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:09:10.563495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172826+0000","last_change":"2024-04-16T17:48:27.288374+0000","last_active":"2024-04-16T19:33:31.172826+0000","last_peered":"2024-04-16T19:33:31.172826+0000","last_clean":"2024-04-16T19:33:31.172826+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:33:31.172826+0000","last_undegraded":"2024-04-16T19:33:31.172826+0000","last_fullsized":"2024-04-16T19:33:31.172826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:11.751579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164610+0000","last_change":"2024-04-16T17:48:01.263690+0000","last_active":"2024-04-16T19:33:01.164610+0000","last_peered":"2024-04-16T19:33:01.164610+0000","last_clean":"2024-04-16T19:33:01.164610+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:33:01.164610+0000","last_undegraded":"2024-04-16T19:33:01.164610+0000","last_fullsized":"2024-04-16T19:33:01.164610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:47.176953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145421+0000","last_change":"2024-04-16T17:47:59.199357+0000","last_active":"2024-04-16T19:31:56.145421+0000","last_peered":"2024-04-16T19:31:56.145421+0000","last_clean":"2024-04-16T19:31:56.145421+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:31:56.145421+0000","last_undegraded":"2024-04-16T19:31:56.145421+0000","last_fullsized":"2024-04-16T19:31:56.145421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:42.077363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11055","reported_seq":18133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150080+0000","last_change":"2024-04-16T17:47:57.184044+0000","last_active":"2024-04-16T19:31:56.150080+0000","last_peered":"2024-04-16T19:31:56.150080+0000","last_clean":"2024-04-16T19:31:56.150080+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:31:56.150080+0000","last_undegraded":"2024-04-16T19:31:56.150080+0000","last_fullsized":"2024-04-16T19:31:56.150080+0000","mapping_epoch":34,"log_start":"53'9200","ondisk_log_start":"53'9200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1855,"log_dups_size":3000,"ondisk_log_size":1855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'6","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143908+0000","last_change":"2024-04-16T17:47:55.159410+0000","last_active":"2024-04-16T19:31:51.143908+0000","last_peered":"2024-04-16T19:31:51.143908+0000","last_clean":"2024-04-16T19:31:51.143908+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:31:51.143908+0000","last_undegraded":"2024-04-16T19:31:51.143908+0000","last_fullsized":"2024-04-16T19:31:51.143908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:42.685205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179184+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:33:46.179184+0000","last_peered":"2024-04-16T19:33:46.179184+0000","last_clean":"2024-04-16T19:33:46.179184+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:33:46.179184+0000","last_undegraded":"2024-04-16T19:33:46.179184+0000","last_fullsized":"2024-04-16T19:33:46.179184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:54.182792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173750+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:33:36.173750+0000","last_peered":"2024-04-16T19:33:36.173750+0000","last_clean":"2024-04-16T19:33:36.173750+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:33:36.173750+0000","last_undegraded":"2024-04-16T19:33:36.173750+0000","last_fullsized":"2024-04-16T19:33:36.173750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149851+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:32:01.149851+0000","last_peered":"2024-04-16T19:32:01.149851+0000","last_clean":"2024-04-16T19:32:01.149851+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:32:01.149851+0000","last_undegraded":"2024-04-16T19:32:01.149851+0000","last_fullsized":"2024-04-16T19:32:01.149851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:53.316949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'80","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155149+0000","last_change":"2024-04-16T17:47:35.308166+0000","last_active":"2024-04-16T19:32:36.155149+0000","last_peered":"2024-04-16T19:32:36.155149+0000","last_clean":"2024-04-16T19:32:36.155149+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:32:36.155149+0000","last_undegraded":"2024-04-16T19:32:36.155149+0000","last_fullsized":"2024-04-16T19:32:36.155149+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:53.924341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159795+0000","last_change":"2024-04-16T17:47:37.321988+0000","last_active":"2024-04-16T19:32:41.159795+0000","last_peered":"2024-04-16T19:32:41.159795+0000","last_clean":"2024-04-16T19:32:41.159795+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:32:41.159795+0000","last_undegraded":"2024-04-16T19:32:41.159795+0000","last_fullsized":"2024-04-16T19:32:41.159795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164797+0000","last_change":"2024-04-16T17:47:39.326823+0000","last_active":"2024-04-16T19:33:16.164797+0000","last_peered":"2024-04-16T19:33:16.164797+0000","last_clean":"2024-04-16T19:33:16.164797+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:33:16.164797+0000","last_undegraded":"2024-04-16T19:33:16.164797+0000","last_fullsized":"2024-04-16T19:33:16.164797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:26.874144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'22","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157286+0000","last_change":"2024-04-16T17:47:55.158595+0000","last_active":"2024-04-16T19:33:41.157286+0000","last_peered":"2024-04-16T19:33:41.157286+0000","last_clean":"2024-04-16T19:33:41.157286+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:33:41.157286+0000","last_undegraded":"2024-04-16T19:33:41.157286+0000","last_fullsized":"2024-04-16T19:33:41.157286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:20.899420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158764+0000","last_change":"2024-04-16T17:47:32.278320+0000","last_active":"2024-04-16T19:33:46.158764+0000","last_peered":"2024-04-16T19:33:46.158764+0000","last_clean":"2024-04-16T19:33:46.158764+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:33:46.158764+0000","last_undegraded":"2024-04-16T19:33:46.158764+0000","last_fullsized":"2024-04-16T19:33:46.158764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:32.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169629+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:33:36.169629+0000","last_peered":"2024-04-16T19:33:36.169629+0000","last_clean":"2024-04-16T19:33:36.169629+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:33:36.169629+0000","last_undegraded":"2024-04-16T19:33:36.169629+0000","last_fullsized":"2024-04-16T19:33:36.169629+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'3","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172656+0000","last_change":"2024-04-16T17:47:55.175820+0000","last_active":"2024-04-16T19:33:41.172656+0000","last_peered":"2024-04-16T19:33:41.172656+0000","last_clean":"2024-04-16T19:33:41.172656+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:33:41.172656+0000","last_undegraded":"2024-04-16T19:33:41.172656+0000","last_fullsized":"2024-04-16T19:33:41.172656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174508+0000","last_change":"2024-04-16T17:47:23.260359+0000","last_active":"2024-04-16T19:33:36.174508+0000","last_peered":"2024-04-16T19:33:36.174508+0000","last_clean":"2024-04-16T19:33:36.174508+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:33:36.174508+0000","last_undegraded":"2024-04-16T19:33:36.174508+0000","last_fullsized":"2024-04-16T19:33:36.174508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:31.902037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.155944+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:33:36.155944+0000","last_peered":"2024-04-16T19:33:36.155944+0000","last_clean":"2024-04-16T19:33:36.155944+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:33:36.155944+0000","last_undegraded":"2024-04-16T19:33:36.155944+0000","last_fullsized":"2024-04-16T19:33:36.155944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:39.513573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'7","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151820+0000","last_change":"2024-04-16T17:47:55.178677+0000","last_active":"2024-04-16T19:32:21.151820+0000","last_peered":"2024-04-16T19:32:21.151820+0000","last_clean":"2024-04-16T19:32:21.151820+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:32:21.151820+0000","last_undegraded":"2024-04-16T19:32:21.151820+0000","last_fullsized":"2024-04-16T19:32:21.151820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.171376+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:33:36.171376+0000","last_peered":"2024-04-16T19:33:36.171376+0000","last_clean":"2024-04-16T19:33:36.171376+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:33:36.171376+0000","last_undegraded":"2024-04-16T19:33:36.171376+0000","last_fullsized":"2024-04-16T19:33:36.171376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176213+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:33:36.176213+0000","last_peered":"2024-04-16T19:33:36.176213+0000","last_clean":"2024-04-16T19:33:36.176213+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:33:36.176213+0000","last_undegraded":"2024-04-16T19:33:36.176213+0000","last_fullsized":"2024-04-16T19:33:36.176213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'6","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177717+0000","last_change":"2024-04-16T17:47:55.156061+0000","last_active":"2024-04-16T19:33:41.177717+0000","last_peered":"2024-04-16T19:33:41.177717+0000","last_clean":"2024-04-16T19:33:41.177717+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:33:41.177717+0000","last_undegraded":"2024-04-16T19:33:41.177717+0000","last_fullsized":"2024-04-16T19:33:41.177717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174362+0000","last_change":"2024-04-16T17:47:23.261639+0000","last_active":"2024-04-16T19:33:36.174362+0000","last_peered":"2024-04-16T19:33:36.174362+0000","last_clean":"2024-04-16T19:33:36.174362+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:33:36.174362+0000","last_undegraded":"2024-04-16T19:33:36.174362+0000","last_fullsized":"2024-04-16T19:33:36.174362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150697+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:32:06.150697+0000","last_peered":"2024-04-16T19:32:06.150697+0000","last_clean":"2024-04-16T19:32:06.150697+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:32:06.150697+0000","last_undegraded":"2024-04-16T19:32:06.150697+0000","last_fullsized":"2024-04-16T19:32:06.150697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"48'3","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161266+0000","last_change":"2024-04-16T17:47:55.181215+0000","last_active":"2024-04-16T19:32:56.161266+0000","last_peered":"2024-04-16T19:32:56.161266+0000","last_clean":"2024-04-16T19:32:56.161266+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:32:56.161266+0000","last_undegraded":"2024-04-16T19:32:56.161266+0000","last_fullsized":"2024-04-16T19:32:56.161266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135310+0000","last_change":"2024-04-16T17:47:23.253450+0000","last_active":"2024-04-16T19:32:06.135310+0000","last_peered":"2024-04-16T19:32:06.135310+0000","last_clean":"2024-04-16T19:32:06.135310+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:32:06.135310+0000","last_undegraded":"2024-04-16T19:32:06.135310+0000","last_fullsized":"2024-04-16T19:32:06.135310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:44.749396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172146+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:33:31.172146+0000","last_peered":"2024-04-16T19:33:31.172146+0000","last_clean":"2024-04-16T19:33:31.172146+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:33:31.172146+0000","last_undegraded":"2024-04-16T19:33:31.172146+0000","last_fullsized":"2024-04-16T19:33:31.172146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"48'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146820+0000","last_change":"2024-04-16T17:47:55.165525+0000","last_active":"2024-04-16T19:32:56.146820+0000","last_peered":"2024-04-16T19:32:56.146820+0000","last_clean":"2024-04-16T19:32:56.146820+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:32:56.146820+0000","last_undegraded":"2024-04-16T19:32:56.146820+0000","last_fullsized":"2024-04-16T19:32:56.146820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:20:06.245260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171037+0000","last_change":"2024-04-16T17:47:23.257600+0000","last_active":"2024-04-16T19:33:41.171037+0000","last_peered":"2024-04-16T19:33:41.171037+0000","last_clean":"2024-04-16T19:33:41.171037+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:33:41.171037+0000","last_undegraded":"2024-04-16T19:33:41.171037+0000","last_fullsized":"2024-04-16T19:33:41.171037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:11.143975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.173036+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:33:31.173036+0000","last_peered":"2024-04-16T19:33:31.173036+0000","last_clean":"2024-04-16T19:33:31.173036+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:33:31.173036+0000","last_undegraded":"2024-04-16T19:33:31.173036+0000","last_fullsized":"2024-04-16T19:33:31.173036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'40","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175919+0000","last_change":"2024-04-16T17:47:55.160972+0000","last_active":"2024-04-16T19:33:41.175919+0000","last_peered":"2024-04-16T19:33:41.175919+0000","last_clean":"2024-04-16T19:33:41.175919+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:33:41.175919+0000","last_undegraded":"2024-04-16T19:33:41.175919+0000","last_fullsized":"2024-04-16T19:33:41.175919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159832+0000","last_change":"2024-04-16T17:47:32.281080+0000","last_active":"2024-04-16T19:33:51.159832+0000","last_peered":"2024-04-16T19:33:51.159832+0000","last_clean":"2024-04-16T19:33:51.159832+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:33:51.159832+0000","last_undegraded":"2024-04-16T19:33:51.159832+0000","last_fullsized":"2024-04-16T19:33:51.159832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:58.918776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175091+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:33:31.175091+0000","last_peered":"2024-04-16T19:33:31.175091+0000","last_clean":"2024-04-16T19:33:31.175091+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:33:31.175091+0000","last_undegraded":"2024-04-16T19:33:31.175091+0000","last_fullsized":"2024-04-16T19:33:31.175091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171222+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:33:41.171222+0000","last_peered":"2024-04-16T19:33:41.171222+0000","last_clean":"2024-04-16T19:33:41.171222+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:33:41.171222+0000","last_undegraded":"2024-04-16T19:33:41.171222+0000","last_fullsized":"2024-04-16T19:33:41.171222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:26.938403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177647+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:33:41.177647+0000","last_peered":"2024-04-16T19:33:41.177647+0000","last_clean":"2024-04-16T19:33:41.177647+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:33:41.177647+0000","last_undegraded":"2024-04-16T19:33:41.177647+0000","last_fullsized":"2024-04-16T19:33:41.177647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154939+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:33:31.154939+0000","last_peered":"2024-04-16T19:33:31.154939+0000","last_clean":"2024-04-16T19:33:31.154939+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:33:31.154939+0000","last_undegraded":"2024-04-16T19:33:31.154939+0000","last_fullsized":"2024-04-16T19:33:31.154939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:12.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'15","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152554+0000","last_change":"2024-04-16T17:47:55.170559+0000","last_active":"2024-04-16T19:32:06.152554+0000","last_peered":"2024-04-16T19:32:06.152554+0000","last_clean":"2024-04-16T19:32:06.152554+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:32:06.152554+0000","last_undegraded":"2024-04-16T19:32:06.152554+0000","last_fullsized":"2024-04-16T19:32:06.152554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155962+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:32:26.155962+0000","last_peered":"2024-04-16T19:32:26.155962+0000","last_clean":"2024-04-16T19:32:26.155962+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:32:26.155962+0000","last_undegraded":"2024-04-16T19:32:26.155962+0000","last_fullsized":"2024-04-16T19:32:26.155962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176559+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:33:56.176559+0000","last_peered":"2024-04-16T19:33:56.176559+0000","last_clean":"2024-04-16T19:33:56.176559+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:33:56.176559+0000","last_undegraded":"2024-04-16T19:33:56.176559+0000","last_fullsized":"2024-04-16T19:33:56.176559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:44.519947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'18","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142162+0000","last_change":"2024-04-16T17:47:55.177250+0000","last_active":"2024-04-16T19:32:36.142162+0000","last_peered":"2024-04-16T19:32:36.142162+0000","last_clean":"2024-04-16T19:32:36.142162+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:32:36.142162+0000","last_undegraded":"2024-04-16T19:32:36.142162+0000","last_fullsized":"2024-04-16T19:32:36.142162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:42.691526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152860+0000","last_change":"2024-04-16T17:47:23.264252+0000","last_active":"2024-04-16T19:32:26.152860+0000","last_peered":"2024-04-16T19:32:26.152860+0000","last_clean":"2024-04-16T19:32:26.152860+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:32:26.152860+0000","last_undegraded":"2024-04-16T19:32:26.152860+0000","last_fullsized":"2024-04-16T19:32:26.152860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166011+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:33:06.166011+0000","last_peered":"2024-04-16T19:33:06.166011+0000","last_clean":"2024-04-16T19:33:06.166011+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:33:06.166011+0000","last_undegraded":"2024-04-16T19:33:06.166011+0000","last_fullsized":"2024-04-16T19:33:06.166011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'1","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150517+0000","last_change":"2024-04-16T17:47:55.168092+0000","last_active":"2024-04-16T19:32:06.150517+0000","last_peered":"2024-04-16T19:32:06.150517+0000","last_clean":"2024-04-16T19:32:06.150517+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:32:06.150517+0000","last_undegraded":"2024-04-16T19:32:06.150517+0000","last_fullsized":"2024-04-16T19:32:06.150517+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157395+0000","last_change":"2024-04-16T17:47:23.265700+0000","last_active":"2024-04-16T19:33:41.157395+0000","last_peered":"2024-04-16T19:33:41.157395+0000","last_clean":"2024-04-16T19:33:41.157395+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:33:41.157395+0000","last_undegraded":"2024-04-16T19:33:41.157395+0000","last_fullsized":"2024-04-16T19:33:41.157395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:24:05.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163673+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:33:06.163673+0000","last_peered":"2024-04-16T19:33:06.163673+0000","last_clean":"2024-04-16T19:33:06.163673+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:33:06.163673+0000","last_undegraded":"2024-04-16T19:33:06.163673+0000","last_fullsized":"2024-04-16T19:33:06.163673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'18","reported_seq":243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135182+0000","last_change":"2024-04-16T17:47:55.147785+0000","last_active":"2024-04-16T19:32:06.135182+0000","last_peered":"2024-04-16T19:32:06.135182+0000","last_clean":"2024-04-16T19:32:06.135182+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:32:06.135182+0000","last_undegraded":"2024-04-16T19:32:06.135182+0000","last_fullsized":"2024-04-16T19:32:06.135182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:26.488442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170252+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:33:11.170252+0000","last_peered":"2024-04-16T19:33:11.170252+0000","last_clean":"2024-04-16T19:33:11.170252+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:33:11.170252+0000","last_undegraded":"2024-04-16T19:33:11.170252+0000","last_fullsized":"2024-04-16T19:33:11.170252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166111+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:33:06.166111+0000","last_peered":"2024-04-16T19:33:06.166111+0000","last_clean":"2024-04-16T19:33:06.166111+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:33:06.166111+0000","last_undegraded":"2024-04-16T19:33:06.166111+0000","last_fullsized":"2024-04-16T19:33:06.166111+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145325+0000","last_change":"2024-04-16T17:47:55.183714+0000","last_active":"2024-04-16T19:31:56.145325+0000","last_peered":"2024-04-16T19:31:56.145325+0000","last_clean":"2024-04-16T19:31:56.145325+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:31:56.145325+0000","last_undegraded":"2024-04-16T19:31:56.145325+0000","last_fullsized":"2024-04-16T19:31:56.145325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163558+0000","last_change":"2024-04-16T17:47:23.265903+0000","last_active":"2024-04-16T19:33:11.163558+0000","last_peered":"2024-04-16T19:33:11.163558+0000","last_clean":"2024-04-16T19:33:11.163558+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:33:11.163558+0000","last_undegraded":"2024-04-16T19:33:11.163558+0000","last_fullsized":"2024-04-16T19:33:11.163558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:55.605376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168971+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:33:06.168971+0000","last_peered":"2024-04-16T19:33:06.168971+0000","last_clean":"2024-04-16T19:33:06.168971+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:33:06.168971+0000","last_undegraded":"2024-04-16T19:33:06.168971+0000","last_fullsized":"2024-04-16T19:33:06.168971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:14.412040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"48'2","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132984+0000","last_change":"2024-04-16T17:47:55.162693+0000","last_active":"2024-04-16T19:31:56.132984+0000","last_peered":"2024-04-16T19:31:56.132984+0000","last_clean":"2024-04-16T19:31:56.132984+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:31:56.132984+0000","last_undegraded":"2024-04-16T19:31:56.132984+0000","last_fullsized":"2024-04-16T19:31:56.132984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:26:14.047560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167227+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:33:11.167227+0000","last_peered":"2024-04-16T19:33:11.167227+0000","last_clean":"2024-04-16T19:33:11.167227+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:33:11.167227+0000","last_undegraded":"2024-04-16T19:33:11.167227+0000","last_fullsized":"2024-04-16T19:33:11.167227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177980+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:33:41.177980+0000","last_peered":"2024-04-16T19:33:41.177980+0000","last_clean":"2024-04-16T19:33:41.177980+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:33:41.177980+0000","last_undegraded":"2024-04-16T19:33:41.177980+0000","last_fullsized":"2024-04-16T19:33:41.177980+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'3","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132884+0000","last_change":"2024-04-16T17:47:55.178234+0000","last_active":"2024-04-16T19:31:56.132884+0000","last_peered":"2024-04-16T19:31:56.132884+0000","last_clean":"2024-04-16T19:31:56.132884+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:31:56.132884+0000","last_undegraded":"2024-04-16T19:31:56.132884+0000","last_fullsized":"2024-04-16T19:31:56.132884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:17.561808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167956+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:33:11.167956+0000","last_peered":"2024-04-16T19:33:11.167956+0000","last_clean":"2024-04-16T19:33:11.167956+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:33:11.167956+0000","last_undegraded":"2024-04-16T19:33:11.167956+0000","last_fullsized":"2024-04-16T19:33:11.167956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138794+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:32:21.138794+0000","last_peered":"2024-04-16T19:32:21.138794+0000","last_clean":"2024-04-16T19:32:21.138794+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:32:21.138794+0000","last_undegraded":"2024-04-16T19:32:21.138794+0000","last_fullsized":"2024-04-16T19:32:21.138794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:40.793750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'6","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148370+0000","last_change":"2024-04-16T17:47:55.177975+0000","last_active":"2024-04-16T19:31:56.148370+0000","last_peered":"2024-04-16T19:31:56.148370+0000","last_clean":"2024-04-16T19:31:56.148370+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:31:56.148370+0000","last_undegraded":"2024-04-16T19:31:56.148370+0000","last_fullsized":"2024-04-16T19:31:56.148370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:32.181968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151087+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:32:01.151087+0000","last_peered":"2024-04-16T19:32:01.151087+0000","last_clean":"2024-04-16T19:32:01.151087+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:32:01.151087+0000","last_undegraded":"2024-04-16T19:32:01.151087+0000","last_fullsized":"2024-04-16T19:32:01.151087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152209+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:32:21.152209+0000","last_peered":"2024-04-16T19:32:21.152209+0000","last_clean":"2024-04-16T19:32:21.152209+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:32:21.152209+0000","last_undegraded":"2024-04-16T19:32:21.152209+0000","last_fullsized":"2024-04-16T19:32:21.152209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'10","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163428+0000","last_change":"2024-04-16T17:47:55.156240+0000","last_active":"2024-04-16T19:32:56.163428+0000","last_peered":"2024-04-16T19:32:56.163428+0000","last_clean":"2024-04-16T19:32:56.163428+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:32:56.163428+0000","last_undegraded":"2024-04-16T19:32:56.163428+0000","last_fullsized":"2024-04-16T19:32:56.163428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156891+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:32:21.156891+0000","last_peered":"2024-04-16T19:32:21.156891+0000","last_clean":"2024-04-16T19:32:21.156891+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:32:21.156891+0000","last_undegraded":"2024-04-16T19:32:21.156891+0000","last_fullsized":"2024-04-16T19:32:21.156891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148441+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:31:56.148441+0000","last_peered":"2024-04-16T19:31:56.148441+0000","last_clean":"2024-04-16T19:31:56.148441+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:31:56.148441+0000","last_undegraded":"2024-04-16T19:31:56.148441+0000","last_fullsized":"2024-04-16T19:31:56.148441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'14","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146712+0000","last_change":"2024-04-16T17:47:55.164923+0000","last_active":"2024-04-16T19:32:56.146712+0000","last_peered":"2024-04-16T19:32:56.146712+0000","last_clean":"2024-04-16T19:32:56.146712+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:32:56.146712+0000","last_undegraded":"2024-04-16T19:32:56.146712+0000","last_fullsized":"2024-04-16T19:32:56.146712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:25.251453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.155204+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:32:21.155204+0000","last_peered":"2024-04-16T19:32:21.155204+0000","last_clean":"2024-04-16T19:32:21.155204+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:32:21.155204+0000","last_undegraded":"2024-04-16T19:32:21.155204+0000","last_fullsized":"2024-04-16T19:32:21.155204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:30.655498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174246+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:33:46.174246+0000","last_peered":"2024-04-16T19:33:46.174246+0000","last_clean":"2024-04-16T19:33:46.174246+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:33:46.174246+0000","last_undegraded":"2024-04-16T19:33:46.174246+0000","last_fullsized":"2024-04-16T19:33:46.174246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2844,"ondisk_log_size":2844,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3706,"ondisk_log_size":3706,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":623,"ondisk_log_size":623,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27355,"ondisk_log_size":27355,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4460,"ondisk_log_size":4460,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":437,"ondisk_log_size":437,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641575,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46584,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811016,"statfs":{"total":107374182400,"available":107326480384,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641574,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641574,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":99932,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":104757668,"statfs":{"total":107374182400,"available":107271852032,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641574,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46572,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811028,"statfs":{"total":107374182400,"available":107326492672,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641575,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":55408,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":104802192,"statfs":{"total":107374182400,"available":107317444608,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641575,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104829060,"statfs":{"total":107374182400,"available":107344957440,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:36:14.775 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2024-04-16T19:36:15.102 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:36:15.103 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":53,"fsid":"5a58eb97-56a6-45c2-947d-e09bf8e2f67b","created":"2024-04-16T17:46:26.316212+0000","modified":"2024-04-16T18:11:33.575135+0000","last_up_change":"2024-04-16T17:46:35.484595+0000","last_in_change":"2024-04-16T17:46:27.961326+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-16T17:46:36.260452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-16T17:46:37.085579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-04-16T17:47:19.326272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-04-16T17:47:21.557037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-04-16T17:47:31.684132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-04-16T17:47:33.392009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-04-16T17:47:35.328662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-04-16T17:47:37.385327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"test-zone1.rgw.buckets.data","create_time":"2024-04-16T17:47:53.448373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"test-zone1.rgw.log","create_time":"2024-04-16T17:47:55.421520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test-zone1.rgw.control","create_time":"2024-04-16T17:47:57.197341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test-zone1.rgw.meta","create_time":"2024-04-16T17:47:59.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test-zone1.rgw.otp","create_time":"2024-04-16T17:48:25.954675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"test-zone1.rgw.buckets.index","create_time":"2024-04-16T17:48:32.467628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"test-zone1.rgw.buckets.non-ec","create_time":"2024-04-16T18:10:49.224738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef14f41e-080b-4b2a-8e53-7c7cc39a1efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6809","nonce":2482459553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6811","nonce":2482459553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6815","nonce":2482459553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2482459553},{"type":"v1","addr":"172.21.15.71:6813","nonce":2482459553}]},"public_addr":"172.21.15.71:6809/2482459553","cluster_addr":"172.21.15.71:6811/2482459553","heartbeat_back_addr":"172.21.15.71:6815/2482459553","heartbeat_front_addr":"172.21.15.71:6813/2482459553","state":["exists","up"]},{"osd":1,"uuid":"ad347559-0237-4958-b508-679c2d02986c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6801","nonce":3255159991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6803","nonce":3255159991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6807","nonce":3255159991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3255159991},{"type":"v1","addr":"172.21.15.71:6805","nonce":3255159991}]},"public_addr":"172.21.15.71:6801/3255159991","cluster_addr":"172.21.15.71:6803/3255159991","heartbeat_back_addr":"172.21.15.71:6807/3255159991","heartbeat_front_addr":"172.21.15.71:6805/3255159991","state":["exists","up"]},{"osd":2,"uuid":"311c7137-c932-46ff-b043-8fedfac27a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6817","nonce":3171611431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6819","nonce":3171611431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6823","nonce":3171611431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3171611431},{"type":"v1","addr":"172.21.15.71:6821","nonce":3171611431}]},"public_addr":"172.21.15.71:6817/3171611431","cluster_addr":"172.21.15.71:6819/3171611431","heartbeat_back_addr":"172.21.15.71:6823/3171611431","heartbeat_front_addr":"172.21.15.71:6821/3171611431","state":["exists","up"]},{"osd":3,"uuid":"9de18877-f732-451f-9568-deb8b0cf2f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6804","nonce":3919083014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6808","nonce":3919083014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6815","nonce":3919083014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3919083014},{"type":"v1","addr":"172.21.15.83:6812","nonce":3919083014}]},"public_addr":"172.21.15.83:6804/3919083014","cluster_addr":"172.21.15.83:6808/3919083014","heartbeat_back_addr":"172.21.15.83:6815/3919083014","heartbeat_front_addr":"172.21.15.83:6812/3919083014","state":["exists","up"]},{"osd":4,"uuid":"675ac319-ba12-4f0c-930f-89812a1c1980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6817","nonce":3700970227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6819","nonce":3700970227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6823","nonce":3700970227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3700970227},{"type":"v1","addr":"172.21.15.83:6821","nonce":3700970227}]},"public_addr":"172.21.15.83:6817/3700970227","cluster_addr":"172.21.15.83:6819/3700970227","heartbeat_back_addr":"172.21.15.83:6823/3700970227","heartbeat_front_addr":"172.21.15.83:6821/3700970227","state":["exists","up"]},{"osd":5,"uuid":"aba623b1-7cf6-457d-8765-4ce5d78f4ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6801","nonce":136860477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6805","nonce":136860477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6813","nonce":136860477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":136860477},{"type":"v1","addr":"172.21.15.83:6809","nonce":136860477}]},"public_addr":"172.21.15.83:6801/136860477","cluster_addr":"172.21.15.83:6805/136860477","heartbeat_back_addr":"172.21.15.83:6813/136860477","heartbeat_front_addr":"172.21.15.83:6809/136860477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.628033+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.524646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.165496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.546615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.394696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:46:32.623608+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.13","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":0,"to":2}]},{"pgid":"3.33","mappings":[{"from":0,"to":1}]},{"pgid":"3.34","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":3,"to":1}]},{"pgid":"3.39","mappings":[{"from":0,"to":2}]},{"pgid":"3.3b","mappings":[{"from":3,"to":1}]},{"pgid":"4.21","mappings":[{"from":1,"to":2}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"9.17","mappings":[{"from":3,"to":4}]},{"pgid":"9.1e","mappings":[{"from":0,"to":4}]},{"pgid":"9.2d","mappings":[{"from":2,"to":4}]},{"pgid":"9.36","mappings":[{"from":3,"to":5}]},{"pgid":"12.2","mappings":[{"from":0,"to":5}]},{"pgid":"14.19","mappings":[{"from":0,"to":5}]},{"pgid":"14.1a","mappings":[{"from":0,"to":5}]},{"pgid":"15.17","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T19:36:16.120 INFO:tasks.ceph:Scrubbing c1.osd.0 2024-04-16T19:36:16.120 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-16T19:36:16.282 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T19:36:16.283 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T19:36:16.283 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T19:36:16.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd deep-scrub 0 2024-04-16T19:36:16.586 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 0 to deep-scrub 2024-04-16T19:36:16.605 INFO:tasks.ceph:Scrubbing c1.osd.1 2024-04-16T19:36:16.606 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-16T19:36:16.746 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T19:36:16.746 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T19:36:16.746 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T19:36:16.761 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd deep-scrub 1 2024-04-16T19:36:17.048 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 1 to deep-scrub 2024-04-16T19:36:17.067 INFO:tasks.ceph:Scrubbing c1.osd.2 2024-04-16T19:36:17.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-16T19:36:17.222 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T19:36:17.223 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T19:36:17.223 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T19:36:17.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd deep-scrub 2 2024-04-16T19:36:17.528 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 2 to deep-scrub 2024-04-16T19:36:17.546 INFO:tasks.ceph:Scrubbing c1.osd.3 2024-04-16T19:36:17.547 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-16T19:36:17.694 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T19:36:17.695 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T19:36:17.695 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T19:36:17.709 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd deep-scrub 3 2024-04-16T19:36:17.977 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 3 to deep-scrub 2024-04-16T19:36:17.995 INFO:tasks.ceph:Scrubbing c1.osd.4 2024-04-16T19:36:17.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-16T19:36:18.135 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T19:36:18.135 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T19:36:18.136 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T19:36:18.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd deep-scrub 4 2024-04-16T19:36:18.436 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 4 to deep-scrub 2024-04-16T19:36:18.453 INFO:tasks.ceph:Scrubbing c1.osd.5 2024-04-16T19:36:18.454 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-16T19:36:18.613 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T19:36:18.613 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T19:36:18.614 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T19:36:18.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd deep-scrub 5 2024-04-16T19:36:18.911 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 5 to deep-scrub 2024-04-16T19:36:18.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:36:19.300 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:36:19.321 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:36:19.323 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3337,"stamp":"2024-04-16T19:36:18.321351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39759,"ondisk_log_size":39759,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":354032,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":343862,"kb_avail":628791568,"statfs":{"total":644245094400,"available":643882565632,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":352115360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010515"},"pg_stats":[{"pgid":"9.35","version":"53'3","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147548+0000","last_change":"2024-04-16T17:47:55.160868+0000","last_active":"2024-04-16T19:31:51.147548+0000","last_peered":"2024-04-16T19:31:51.147548+0000","last_clean":"2024-04-16T19:31:51.147548+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:31:51.147548+0000","last_undegraded":"2024-04-16T19:31:51.147548+0000","last_fullsized":"2024-04-16T19:31:51.147548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176615+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:33:36.176615+0000","last_peered":"2024-04-16T19:33:36.176615+0000","last_clean":"2024-04-16T19:33:36.176615+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:33:36.176615+0000","last_undegraded":"2024-04-16T19:33:36.176615+0000","last_fullsized":"2024-04-16T19:33:36.176615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:51.288503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153219+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:32:26.153219+0000","last_peered":"2024-04-16T19:32:26.153219+0000","last_clean":"2024-04-16T19:32:26.153219+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:32:26.153219+0000","last_undegraded":"2024-04-16T19:32:26.153219+0000","last_fullsized":"2024-04-16T19:32:26.153219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'14","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143970+0000","last_change":"2024-04-16T17:47:55.173885+0000","last_active":"2024-04-16T19:31:51.143970+0000","last_peered":"2024-04-16T19:31:51.143970+0000","last_clean":"2024-04-16T19:31:51.143970+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:31:51.143970+0000","last_undegraded":"2024-04-16T19:31:51.143970+0000","last_fullsized":"2024-04-16T19:31:51.143970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:50.786411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174452+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:33:36.174452+0000","last_peered":"2024-04-16T19:33:36.174452+0000","last_clean":"2024-04-16T19:33:36.174452+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:33:36.174452+0000","last_undegraded":"2024-04-16T19:33:36.174452+0000","last_fullsized":"2024-04-16T19:33:36.174452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.158347+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:32:26.158347+0000","last_peered":"2024-04-16T19:32:26.158347+0000","last_clean":"2024-04-16T19:32:26.158347+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:32:26.158347+0000","last_undegraded":"2024-04-16T19:32:26.158347+0000","last_fullsized":"2024-04-16T19:32:26.158347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'20","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152906+0000","last_change":"2024-04-16T17:47:55.156902+0000","last_active":"2024-04-16T19:32:26.152906+0000","last_peered":"2024-04-16T19:32:26.152906+0000","last_clean":"2024-04-16T19:32:26.152906+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:32:26.152906+0000","last_undegraded":"2024-04-16T19:32:26.152906+0000","last_fullsized":"2024-04-16T19:32:26.152906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:40.884246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173569+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:33:36.173569+0000","last_peered":"2024-04-16T19:33:36.173569+0000","last_clean":"2024-04-16T19:33:36.173569+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:33:36.173569+0000","last_undegraded":"2024-04-16T19:33:36.173569+0000","last_fullsized":"2024-04-16T19:33:36.173569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152536+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:32:26.152536+0000","last_peered":"2024-04-16T19:32:26.152536+0000","last_clean":"2024-04-16T19:32:26.152536+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:32:26.152536+0000","last_undegraded":"2024-04-16T19:32:26.152536+0000","last_fullsized":"2024-04-16T19:32:26.152536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:32.905270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.147603+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:31:51.147603+0000","last_peered":"2024-04-16T19:31:51.147603+0000","last_clean":"2024-04-16T19:31:51.147603+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:31:51.147603+0000","last_undegraded":"2024-04-16T19:31:51.147603+0000","last_fullsized":"2024-04-16T19:31:51.147603+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:54:10.585096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174129+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:33:46.174129+0000","last_peered":"2024-04-16T19:33:46.174129+0000","last_clean":"2024-04-16T19:33:46.174129+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:33:46.174129+0000","last_undegraded":"2024-04-16T19:33:46.174129+0000","last_fullsized":"2024-04-16T19:33:46.174129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:53.592351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155669+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:32:26.155669+0000","last_peered":"2024-04-16T19:32:26.155669+0000","last_clean":"2024-04-16T19:32:26.155669+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:32:26.155669+0000","last_undegraded":"2024-04-16T19:32:26.155669+0000","last_fullsized":"2024-04-16T19:32:26.155669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'4","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159892+0000","last_change":"2024-04-16T17:47:55.159892+0000","last_active":"2024-04-16T19:32:56.159892+0000","last_peered":"2024-04-16T19:32:56.159892+0000","last_clean":"2024-04-16T19:32:56.159892+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:32:56.159892+0000","last_undegraded":"2024-04-16T19:32:56.159892+0000","last_fullsized":"2024-04-16T19:32:56.159892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:58.461789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159155+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:32:36.159155+0000","last_peered":"2024-04-16T19:32:36.159155+0000","last_clean":"2024-04-16T19:32:36.159155+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:32:36.159155+0000","last_undegraded":"2024-04-16T19:32:36.159155+0000","last_fullsized":"2024-04-16T19:32:36.159155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175465+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:33:51.175465+0000","last_peered":"2024-04-16T19:33:51.175465+0000","last_clean":"2024-04-16T19:33:51.175465+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:33:51.175465+0000","last_undegraded":"2024-04-16T19:33:51.175465+0000","last_fullsized":"2024-04-16T19:33:51.175465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'11","reported_seq":309,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163313+0000","last_change":"2024-04-16T17:47:55.163604+0000","last_active":"2024-04-16T19:32:56.163313+0000","last_peered":"2024-04-16T19:32:56.163313+0000","last_clean":"2024-04-16T19:32:56.163313+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:32:56.163313+0000","last_undegraded":"2024-04-16T19:32:56.163313+0000","last_fullsized":"2024-04-16T19:32:56.163313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151934+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:32:21.151934+0000","last_peered":"2024-04-16T19:32:21.151934+0000","last_clean":"2024-04-16T19:32:21.151934+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:32:21.151934+0000","last_undegraded":"2024-04-16T19:32:21.151934+0000","last_fullsized":"2024-04-16T19:32:21.151934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171180+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:33:41.171180+0000","last_peered":"2024-04-16T19:33:41.171180+0000","last_clean":"2024-04-16T19:33:41.171180+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:33:41.171180+0000","last_undegraded":"2024-04-16T19:33:41.171180+0000","last_fullsized":"2024-04-16T19:33:41.171180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:47.915323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'12","reported_seq":268,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.148697+0000","last_change":"2024-04-16T17:47:55.161748+0000","last_active":"2024-04-16T19:31:51.148697+0000","last_peered":"2024-04-16T19:31:51.148697+0000","last_clean":"2024-04-16T19:31:51.148697+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:31:51.148697+0000","last_undegraded":"2024-04-16T19:31:51.148697+0000","last_fullsized":"2024-04-16T19:31:51.148697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157014+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:32:21.157014+0000","last_peered":"2024-04-16T19:32:21.157014+0000","last_clean":"2024-04-16T19:32:21.157014+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:32:21.157014+0000","last_undegraded":"2024-04-16T19:32:21.157014+0000","last_fullsized":"2024-04-16T19:32:21.157014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:14.655507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172868+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:33:41.172868+0000","last_peered":"2024-04-16T19:33:41.172868+0000","last_clean":"2024-04-16T19:33:41.172868+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:33:41.172868+0000","last_undegraded":"2024-04-16T19:33:41.172868+0000","last_fullsized":"2024-04-16T19:33:41.172868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'8","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168449+0000","last_change":"2024-04-16T17:47:55.164548+0000","last_active":"2024-04-16T19:33:31.168449+0000","last_peered":"2024-04-16T19:33:31.168449+0000","last_clean":"2024-04-16T19:33:31.168449+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:33:31.168449+0000","last_undegraded":"2024-04-16T19:33:31.168449+0000","last_fullsized":"2024-04-16T19:33:31.168449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:27.967153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138549+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:32:21.138549+0000","last_peered":"2024-04-16T19:32:21.138549+0000","last_clean":"2024-04-16T19:32:21.138549+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:32:21.138549+0000","last_undegraded":"2024-04-16T19:32:21.138549+0000","last_fullsized":"2024-04-16T19:32:21.138549+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:19:39.583925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157481+0000","last_change":"2024-04-16T17:47:23.254199+0000","last_active":"2024-04-16T19:33:41.157481+0000","last_peered":"2024-04-16T19:33:41.157481+0000","last_clean":"2024-04-16T19:33:41.157481+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:33:41.157481+0000","last_undegraded":"2024-04-16T19:33:41.157481+0000","last_fullsized":"2024-04-16T19:33:41.157481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:25:11.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"48'5","reported_seq":301,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.181075+0000","last_change":"2024-04-16T17:47:55.184230+0000","last_active":"2024-04-16T19:34:11.181075+0000","last_peered":"2024-04-16T19:34:11.181075+0000","last_clean":"2024-04-16T19:34:11.181075+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:34:11.181075+0000","last_undegraded":"2024-04-16T19:34:11.181075+0000","last_fullsized":"2024-04-16T19:34:11.181075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152428+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:32:26.152428+0000","last_peered":"2024-04-16T19:32:26.152428+0000","last_clean":"2024-04-16T19:32:26.152428+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:32:26.152428+0000","last_undegraded":"2024-04-16T19:32:26.152428+0000","last_fullsized":"2024-04-16T19:32:26.152428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150380+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:32:01.150380+0000","last_peered":"2024-04-16T19:32:01.150380+0000","last_clean":"2024-04-16T19:32:01.150380+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:32:01.150380+0000","last_undegraded":"2024-04-16T19:32:01.150380+0000","last_fullsized":"2024-04-16T19:32:01.150380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'14","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159666+0000","last_change":"2024-04-16T17:47:55.163042+0000","last_active":"2024-04-16T19:32:31.159666+0000","last_peered":"2024-04-16T19:32:31.159666+0000","last_clean":"2024-04-16T19:32:31.159666+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:32:31.159666+0000","last_undegraded":"2024-04-16T19:32:31.159666+0000","last_fullsized":"2024-04-16T19:32:31.159666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153260+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:32:26.153260+0000","last_peered":"2024-04-16T19:32:26.153260+0000","last_clean":"2024-04-16T19:32:26.153260+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:32:26.153260+0000","last_undegraded":"2024-04-16T19:32:26.153260+0000","last_fullsized":"2024-04-16T19:32:26.153260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178113+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:33:41.178113+0000","last_peered":"2024-04-16T19:33:41.178113+0000","last_clean":"2024-04-16T19:33:41.178113+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:33:41.178113+0000","last_undegraded":"2024-04-16T19:33:41.178113+0000","last_fullsized":"2024-04-16T19:33:41.178113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'8","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.164856+0000","last_change":"2024-04-16T17:47:55.163584+0000","last_active":"2024-04-16T19:34:11.164856+0000","last_peered":"2024-04-16T19:34:11.164856+0000","last_clean":"2024-04-16T19:34:11.164856+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:34:11.164856+0000","last_undegraded":"2024-04-16T19:34:11.164856+0000","last_fullsized":"2024-04-16T19:34:11.164856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:28.473272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159805+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:32:51.159805+0000","last_peered":"2024-04-16T19:32:51.159805+0000","last_clean":"2024-04-16T19:32:51.159805+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:32:51.159805+0000","last_undegraded":"2024-04-16T19:32:51.159805+0000","last_fullsized":"2024-04-16T19:32:51.159805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172776+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:33:41.172776+0000","last_peered":"2024-04-16T19:33:41.172776+0000","last_clean":"2024-04-16T19:33:41.172776+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:33:41.172776+0000","last_undegraded":"2024-04-16T19:33:41.172776+0000","last_fullsized":"2024-04-16T19:33:41.172776+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:01.960480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"50'5","reported_seq":228,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.164794+0000","last_change":"2024-04-16T17:47:55.164609+0000","last_active":"2024-04-16T19:34:11.164794+0000","last_peered":"2024-04-16T19:34:11.164794+0000","last_clean":"2024-04-16T19:34:11.164794+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:34:11.164794+0000","last_undegraded":"2024-04-16T19:34:11.164794+0000","last_fullsized":"2024-04-16T19:34:11.164794+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:56.378878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158570+0000","last_change":"2024-04-16T17:47:23.254226+0000","last_active":"2024-04-16T19:33:46.158570+0000","last_peered":"2024-04-16T19:33:46.158570+0000","last_clean":"2024-04-16T19:33:46.158570+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:33:46.158570+0000","last_undegraded":"2024-04-16T19:33:46.158570+0000","last_fullsized":"2024-04-16T19:33:46.158570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:06.511950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171122+0000","last_change":"2024-04-16T17:47:32.284109+0000","last_active":"2024-04-16T19:33:41.171122+0000","last_peered":"2024-04-16T19:33:41.171122+0000","last_clean":"2024-04-16T19:33:41.171122+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:33:41.171122+0000","last_undegraded":"2024-04-16T19:33:41.171122+0000","last_fullsized":"2024-04-16T19:33:41.171122+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'5","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166165+0000","last_change":"2024-04-16T17:47:55.172087+0000","last_active":"2024-04-16T19:32:56.166165+0000","last_peered":"2024-04-16T19:32:56.166165+0000","last_clean":"2024-04-16T19:32:56.166165+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:32:56.166165+0000","last_undegraded":"2024-04-16T19:32:56.166165+0000","last_fullsized":"2024-04-16T19:32:56.166165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158622+0000","last_change":"2024-04-16T17:47:23.266132+0000","last_active":"2024-04-16T19:33:46.158622+0000","last_peered":"2024-04-16T19:33:46.158622+0000","last_clean":"2024-04-16T19:33:46.158622+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:33:46.158622+0000","last_undegraded":"2024-04-16T19:33:46.158622+0000","last_fullsized":"2024-04-16T19:33:46.158622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:09.592666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.358504+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:31:31.358504+0000","last_peered":"2024-04-16T19:31:31.358504+0000","last_clean":"2024-04-16T19:31:31.358504+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:31:31.358504+0000","last_undegraded":"2024-04-16T19:31:31.358504+0000","last_fullsized":"2024-04-16T19:31:31.358504+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:02.531335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'10","reported_seq":265,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146504+0000","last_change":"2024-04-16T17:47:55.158785+0000","last_active":"2024-04-16T19:32:56.146504+0000","last_peered":"2024-04-16T19:32:56.146504+0000","last_clean":"2024-04-16T19:32:56.146504+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:32:56.146504+0000","last_undegraded":"2024-04-16T19:32:56.146504+0000","last_fullsized":"2024-04-16T19:32:56.146504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:39.387454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172430+0000","last_change":"2024-04-16T17:47:23.257289+0000","last_active":"2024-04-16T19:33:46.172430+0000","last_peered":"2024-04-16T19:33:46.172430+0000","last_clean":"2024-04-16T19:33:46.172430+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:33:46.172430+0000","last_undegraded":"2024-04-16T19:33:46.172430+0000","last_fullsized":"2024-04-16T19:33:46.172430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163511+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:32:56.163511+0000","last_peered":"2024-04-16T19:32:56.163511+0000","last_clean":"2024-04-16T19:32:56.163511+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:32:56.163511+0000","last_undegraded":"2024-04-16T19:32:56.163511+0000","last_fullsized":"2024-04-16T19:32:56.163511+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:29:20.877336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183897+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:34:11.183897+0000","last_peered":"2024-04-16T19:34:11.183897+0000","last_clean":"2024-04-16T19:34:11.183897+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:34:11.183897+0000","last_undegraded":"2024-04-16T19:34:11.183897+0000","last_fullsized":"2024-04-16T19:34:11.183897+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:23:28.148511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170388+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:33:11.170388+0000","last_peered":"2024-04-16T19:33:11.170388+0000","last_clean":"2024-04-16T19:33:11.170388+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:33:11.170388+0000","last_undegraded":"2024-04-16T19:33:11.170388+0000","last_fullsized":"2024-04-16T19:33:11.170388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175004+0000","last_change":"2024-04-16T17:48:27.308777+0000","last_active":"2024-04-16T19:33:31.175004+0000","last_peered":"2024-04-16T19:33:31.175004+0000","last_clean":"2024-04-16T19:33:31.175004+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:33:31.175004+0000","last_undegraded":"2024-04-16T19:33:31.175004+0000","last_fullsized":"2024-04-16T19:33:31.175004+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'9","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164889+0000","last_change":"2024-04-16T17:48:01.266271+0000","last_active":"2024-04-16T19:33:01.164889+0000","last_peered":"2024-04-16T19:33:01.164889+0000","last_clean":"2024-04-16T19:33:01.164889+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:33:01.164889+0000","last_undegraded":"2024-04-16T19:33:01.164889+0000","last_fullsized":"2024-04-16T19:33:01.164889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161254+0000","last_change":"2024-04-16T17:47:59.181005+0000","last_active":"2024-04-16T19:33:01.161254+0000","last_peered":"2024-04-16T19:33:01.161254+0000","last_clean":"2024-04-16T19:33:01.161254+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:33:01.161254+0000","last_undegraded":"2024-04-16T19:33:01.161254+0000","last_fullsized":"2024-04-16T19:33:01.161254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:34.728189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'547","reported_seq":799,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167615+0000","last_change":"2024-04-16T17:47:57.177580+0000","last_active":"2024-04-16T19:33:01.167615+0000","last_peered":"2024-04-16T19:33:01.167615+0000","last_clean":"2024-04-16T19:33:01.167615+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:33:01.167615+0000","last_undegraded":"2024-04-16T19:33:01.167615+0000","last_fullsized":"2024-04-16T19:33:01.167615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'7","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138635+0000","last_change":"2024-04-16T17:47:55.165841+0000","last_active":"2024-04-16T19:32:21.138635+0000","last_peered":"2024-04-16T19:32:21.138635+0000","last_clean":"2024-04-16T19:32:21.138635+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:32:21.138635+0000","last_undegraded":"2024-04-16T19:32:21.138635+0000","last_fullsized":"2024-04-16T19:32:21.138635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:07:11.773810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.139718+0000","last_change":"2024-04-16T17:47:39.326260+0000","last_active":"2024-04-16T19:31:36.139718+0000","last_peered":"2024-04-16T19:31:36.139718+0000","last_clean":"2024-04-16T19:31:36.139718+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:31:36.139718+0000","last_undegraded":"2024-04-16T19:31:36.139718+0000","last_fullsized":"2024-04-16T19:31:36.139718+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:58.230179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164460+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:32:56.164460+0000","last_peered":"2024-04-16T19:32:56.164460+0000","last_clean":"2024-04-16T19:32:56.164460+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:32:56.164460+0000","last_undegraded":"2024-04-16T19:32:56.164460+0000","last_fullsized":"2024-04-16T19:32:56.164460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159202+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:32:36.159202+0000","last_peered":"2024-04-16T19:32:36.159202+0000","last_clean":"2024-04-16T19:32:36.159202+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:32:36.159202+0000","last_undegraded":"2024-04-16T19:32:36.159202+0000","last_fullsized":"2024-04-16T19:32:36.159202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":768,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.370756+0000","last_change":"2024-04-16T17:47:33.303155+0000","last_active":"2024-04-16T19:31:31.370756+0000","last_peered":"2024-04-16T19:31:31.370756+0000","last_clean":"2024-04-16T19:31:31.370756+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:31:31.370756+0000","last_undegraded":"2024-04-16T19:31:31.370756+0000","last_fullsized":"2024-04-16T19:31:31.370756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:14.676250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367632+0000","last_change":"2024-04-16T17:47:35.314855+0000","last_active":"2024-04-16T19:31:31.367632+0000","last_peered":"2024-04-16T19:31:31.367632+0000","last_clean":"2024-04-16T19:31:31.367632+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:31:31.367632+0000","last_undegraded":"2024-04-16T19:31:31.367632+0000","last_fullsized":"2024-04-16T19:31:31.367632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140046+0000","last_change":"2024-04-16T17:47:37.332945+0000","last_active":"2024-04-16T19:31:36.140046+0000","last_peered":"2024-04-16T19:31:36.140046+0000","last_clean":"2024-04-16T19:31:36.140046+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:31:36.140046+0000","last_undegraded":"2024-04-16T19:31:36.140046+0000","last_fullsized":"2024-04-16T19:31:36.140046+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152563+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:33:21.152563+0000","last_peered":"2024-04-16T19:33:21.152563+0000","last_clean":"2024-04-16T19:33:21.152563+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:33:21.152563+0000","last_undegraded":"2024-04-16T19:33:21.152563+0000","last_fullsized":"2024-04-16T19:33:21.152563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:24.843140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'83","reported_seq":1420,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161248+0000","last_change":"2024-04-16T17:48:34.319654+0000","last_active":"2024-04-16T19:32:46.161248+0000","last_peered":"2024-04-16T19:32:46.161248+0000","last_clean":"2024-04-16T19:32:46.161248+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:32:46.161248+0000","last_undegraded":"2024-04-16T19:32:46.161248+0000","last_fullsized":"2024-04-16T19:32:46.161248+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:13.391814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169963+0000","last_change":"2024-04-16T17:48:27.303999+0000","last_active":"2024-04-16T19:33:31.169963+0000","last_peered":"2024-04-16T19:33:31.169963+0000","last_clean":"2024-04-16T19:33:31.169963+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:33:31.169963+0000","last_undegraded":"2024-04-16T19:33:31.169963+0000","last_fullsized":"2024-04-16T19:33:31.169963+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'5","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164928+0000","last_change":"2024-04-16T17:48:01.266699+0000","last_active":"2024-04-16T19:33:01.164928+0000","last_peered":"2024-04-16T19:33:01.164928+0000","last_clean":"2024-04-16T19:33:01.164928+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:33:01.164928+0000","last_undegraded":"2024-04-16T19:33:01.164928+0000","last_fullsized":"2024-04-16T19:33:01.164928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162337+0000","last_change":"2024-04-16T17:47:59.191306+0000","last_active":"2024-04-16T19:33:01.162337+0000","last_peered":"2024-04-16T19:33:01.162337+0000","last_clean":"2024-04-16T19:33:01.162337+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:33:01.162337+0000","last_undegraded":"2024-04-16T19:33:01.162337+0000","last_fullsized":"2024-04-16T19:33:01.162337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7326","reported_seq":20595,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147895+0000","last_change":"2024-04-16T17:47:57.179815+0000","last_active":"2024-04-16T19:33:01.147895+0000","last_peered":"2024-04-16T19:33:01.147895+0000","last_clean":"2024-04-16T19:33:01.147895+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:33:01.147895+0000","last_undegraded":"2024-04-16T19:33:01.147895+0000","last_fullsized":"2024-04-16T19:33:01.147895+0000","mapping_epoch":34,"log_start":"53'5500","ondisk_log_start":"53'5500","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1826,"log_dups_size":3000,"ondisk_log_size":1826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:36.013179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'19","reported_seq":354,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143591+0000","last_change":"2024-04-16T17:47:55.178555+0000","last_active":"2024-04-16T19:31:51.143591+0000","last_peered":"2024-04-16T19:31:51.143591+0000","last_clean":"2024-04-16T19:31:51.143591+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:31:51.143591+0000","last_undegraded":"2024-04-16T19:31:51.143591+0000","last_fullsized":"2024-04-16T19:31:51.143591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146395+0000","last_change":"2024-04-16T17:47:39.333211+0000","last_active":"2024-04-16T19:32:01.146395+0000","last_peered":"2024-04-16T19:32:01.146395+0000","last_clean":"2024-04-16T19:32:01.146395+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:32:01.146395+0000","last_undegraded":"2024-04-16T19:32:01.146395+0000","last_fullsized":"2024-04-16T19:32:01.146395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174179+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:33:46.174179+0000","last_peered":"2024-04-16T19:33:46.174179+0000","last_clean":"2024-04-16T19:33:46.174179+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:33:46.174179+0000","last_undegraded":"2024-04-16T19:33:46.174179+0000","last_fullsized":"2024-04-16T19:33:46.174179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154341+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:32:21.154341+0000","last_peered":"2024-04-16T19:32:21.154341+0000","last_clean":"2024-04-16T19:32:21.154341+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:32:21.154341+0000","last_undegraded":"2024-04-16T19:32:21.154341+0000","last_fullsized":"2024-04-16T19:32:21.154341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"48'5","reported_seq":359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167357+0000","last_change":"2024-04-16T17:47:33.310538+0000","last_active":"2024-04-16T19:33:11.167357+0000","last_peered":"2024-04-16T19:33:11.167357+0000","last_clean":"2024-04-16T19:33:11.167357+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:33:11.167357+0000","last_undegraded":"2024-04-16T19:33:11.167357+0000","last_fullsized":"2024-04-16T19:33:11.167357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'72","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361573+0000","last_change":"2024-04-16T17:47:35.305717+0000","last_active":"2024-04-16T19:31:31.361573+0000","last_peered":"2024-04-16T19:31:31.361573+0000","last_clean":"2024-04-16T19:31:31.361573+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:31:31.361573+0000","last_undegraded":"2024-04-16T19:31:31.361573+0000","last_fullsized":"2024-04-16T19:31:31.361573+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:27.152861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.140105+0000","last_change":"2024-04-16T17:47:37.326918+0000","last_active":"2024-04-16T19:31:36.140105+0000","last_peered":"2024-04-16T19:31:36.140105+0000","last_clean":"2024-04-16T19:31:36.140105+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:31:36.140105+0000","last_undegraded":"2024-04-16T19:31:36.140105+0000","last_fullsized":"2024-04-16T19:31:36.140105+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:19.938869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166042+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:33:21.166042+0000","last_peered":"2024-04-16T19:33:21.166042+0000","last_clean":"2024-04-16T19:33:21.166042+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:33:21.166042+0000","last_undegraded":"2024-04-16T19:33:21.166042+0000","last_fullsized":"2024-04-16T19:33:21.166042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:35.987275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'56","reported_seq":876,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173425+0000","last_change":"2024-04-16T17:48:34.333949+0000","last_active":"2024-04-16T19:33:36.173425+0000","last_peered":"2024-04-16T19:33:36.173425+0000","last_clean":"2024-04-16T19:33:36.173425+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:33:36.173425+0000","last_undegraded":"2024-04-16T19:33:36.173425+0000","last_fullsized":"2024-04-16T19:33:36.173425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156598+0000","last_change":"2024-04-16T17:48:27.299066+0000","last_active":"2024-04-16T19:32:41.156598+0000","last_peered":"2024-04-16T19:32:41.156598+0000","last_clean":"2024-04-16T19:32:41.156598+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:32:41.156598+0000","last_undegraded":"2024-04-16T19:32:41.156598+0000","last_fullsized":"2024-04-16T19:32:41.156598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:12.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'16","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155319+0000","last_change":"2024-04-16T17:48:01.239365+0000","last_active":"2024-04-16T19:32:36.155319+0000","last_peered":"2024-04-16T19:32:36.155319+0000","last_clean":"2024-04-16T19:32:36.155319+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:32:36.155319+0000","last_undegraded":"2024-04-16T19:32:36.155319+0000","last_fullsized":"2024-04-16T19:32:36.155319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:31.169600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156956+0000","last_change":"2024-04-16T17:47:59.192949+0000","last_active":"2024-04-16T19:32:31.156956+0000","last_peered":"2024-04-16T19:32:31.156956+0000","last_clean":"2024-04-16T19:32:31.156956+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:32:31.156956+0000","last_undegraded":"2024-04-16T19:32:31.156956+0000","last_fullsized":"2024-04-16T19:32:31.156956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3595","reported_seq":7189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132670+0000","last_change":"2024-04-16T17:47:57.176567+0000","last_active":"2024-04-16T19:31:56.132670+0000","last_peered":"2024-04-16T19:31:56.132670+0000","last_clean":"2024-04-16T19:31:56.132670+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:31:56.132670+0000","last_undegraded":"2024-04-16T19:31:56.132670+0000","last_fullsized":"2024-04-16T19:31:56.132670+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1795,"log_dups_size":1800,"ondisk_log_size":1795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:10.013659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'9","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163598+0000","last_change":"2024-04-16T17:47:55.178252+0000","last_active":"2024-04-16T19:32:56.163598+0000","last_peered":"2024-04-16T19:32:56.163598+0000","last_clean":"2024-04-16T19:32:56.163598+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:32:56.163598+0000","last_undegraded":"2024-04-16T19:32:56.163598+0000","last_fullsized":"2024-04-16T19:32:56.163598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159596+0000","last_change":"2024-04-16T17:47:39.334187+0000","last_active":"2024-04-16T19:32:41.159596+0000","last_peered":"2024-04-16T19:32:41.159596+0000","last_clean":"2024-04-16T19:32:41.159596+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:32:41.159596+0000","last_undegraded":"2024-04-16T19:32:41.159596+0000","last_fullsized":"2024-04-16T19:32:41.159596+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176178+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:33:46.176178+0000","last_peered":"2024-04-16T19:33:46.176178+0000","last_clean":"2024-04-16T19:33:46.176178+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:33:46.176178+0000","last_undegraded":"2024-04-16T19:33:46.176178+0000","last_fullsized":"2024-04-16T19:33:46.176178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156989+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:32:21.156989+0000","last_peered":"2024-04-16T19:32:21.156989+0000","last_clean":"2024-04-16T19:32:21.156989+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:32:21.156989+0000","last_undegraded":"2024-04-16T19:32:21.156989+0000","last_fullsized":"2024-04-16T19:32:21.156989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:51.117773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"48'11","reported_seq":605,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.343972+0000","last_change":"2024-04-16T17:47:33.300240+0000","last_active":"2024-04-16T19:31:31.343972+0000","last_peered":"2024-04-16T19:31:31.343972+0000","last_clean":"2024-04-16T19:31:31.343972+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:31:31.343972+0000","last_undegraded":"2024-04-16T19:31:31.343972+0000","last_fullsized":"2024-04-16T19:31:31.343972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:30.896735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'120","reported_seq":290,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.149082+0000","last_change":"2024-04-16T17:47:35.314675+0000","last_active":"2024-04-16T19:33:06.149082+0000","last_peered":"2024-04-16T19:33:06.149082+0000","last_clean":"2024-04-16T19:33:06.149082+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:33:06.149082+0000","last_undegraded":"2024-04-16T19:33:06.149082+0000","last_fullsized":"2024-04-16T19:33:06.149082+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:43.962803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'12","reported_seq":1332,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144472+0000","last_change":"2024-04-16T17:47:37.333045+0000","last_active":"2024-04-16T19:31:36.144472+0000","last_peered":"2024-04-16T19:31:36.144472+0000","last_clean":"2024-04-16T19:31:36.144472+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:31:36.144472+0000","last_undegraded":"2024-04-16T19:31:36.144472+0000","last_fullsized":"2024-04-16T19:31:36.144472+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:40.632643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170471+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:33:21.170471+0000","last_peered":"2024-04-16T19:33:21.170471+0000","last_clean":"2024-04-16T19:33:21.170471+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:33:21.170471+0000","last_undegraded":"2024-04-16T19:33:21.170471+0000","last_fullsized":"2024-04-16T19:33:21.170471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:20.115696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173520+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:33:36.173520+0000","last_peered":"2024-04-16T19:33:36.173520+0000","last_clean":"2024-04-16T19:33:36.173520+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:33:36.173520+0000","last_undegraded":"2024-04-16T19:33:36.173520+0000","last_fullsized":"2024-04-16T19:33:36.173520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:58.360796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.162482+0000","last_change":"2024-04-16T17:48:27.297318+0000","last_active":"2024-04-16T19:33:06.162482+0000","last_peered":"2024-04-16T19:33:06.162482+0000","last_clean":"2024-04-16T19:33:06.162482+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:33:06.162482+0000","last_undegraded":"2024-04-16T19:33:06.162482+0000","last_fullsized":"2024-04-16T19:33:06.162482+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:20:02.104358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'19","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154557+0000","last_change":"2024-04-16T17:48:01.256079+0000","last_active":"2024-04-16T19:32:11.154557+0000","last_peered":"2024-04-16T19:32:11.154557+0000","last_clean":"2024-04-16T19:32:11.154557+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:32:11.154557+0000","last_undegraded":"2024-04-16T19:32:11.154557+0000","last_fullsized":"2024-04-16T19:32:11.154557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132631+0000","last_change":"2024-04-16T17:47:59.183129+0000","last_active":"2024-04-16T19:31:56.132631+0000","last_peered":"2024-04-16T19:31:56.132631+0000","last_clean":"2024-04-16T19:31:56.132631+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:31:56.132631+0000","last_undegraded":"2024-04-16T19:31:56.132631+0000","last_fullsized":"2024-04-16T19:31:56.132631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:48.953458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'385","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156502+0000","last_change":"2024-04-16T17:47:57.184276+0000","last_active":"2024-04-16T19:32:26.156502+0000","last_peered":"2024-04-16T19:32:26.156502+0000","last_clean":"2024-04-16T19:32:26.156502+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:32:26.156502+0000","last_undegraded":"2024-04-16T19:32:26.156502+0000","last_fullsized":"2024-04-16T19:32:26.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:59.219214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'6","reported_seq":173,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164223+0000","last_change":"2024-04-16T17:47:55.161902+0000","last_active":"2024-04-16T19:32:56.164223+0000","last_peered":"2024-04-16T19:32:56.164223+0000","last_clean":"2024-04-16T19:32:56.164223+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:32:56.164223+0000","last_undegraded":"2024-04-16T19:32:56.164223+0000","last_fullsized":"2024-04-16T19:32:56.164223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:47.909979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156406+0000","last_change":"2024-04-16T17:47:39.699810+0000","last_active":"2024-04-16T19:32:41.156406+0000","last_peered":"2024-04-16T19:32:41.156406+0000","last_clean":"2024-04-16T19:32:41.156406+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:32:41.156406+0000","last_undegraded":"2024-04-16T19:32:41.156406+0000","last_fullsized":"2024-04-16T19:32:41.156406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:40.561497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176260+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:33:46.176260+0000","last_peered":"2024-04-16T19:33:46.176260+0000","last_clean":"2024-04-16T19:33:46.176260+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:33:46.176260+0000","last_undegraded":"2024-04-16T19:33:46.176260+0000","last_fullsized":"2024-04-16T19:33:46.176260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151772+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:32:21.151772+0000","last_peered":"2024-04-16T19:32:21.151772+0000","last_clean":"2024-04-16T19:32:21.151772+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:32:21.151772+0000","last_undegraded":"2024-04-16T19:32:21.151772+0000","last_fullsized":"2024-04-16T19:32:21.151772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374413+0000","last_change":"2024-04-16T17:47:33.305749+0000","last_active":"2024-04-16T19:31:31.374413+0000","last_peered":"2024-04-16T19:31:31.374413+0000","last_clean":"2024-04-16T19:31:31.374413+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:31:31.374413+0000","last_undegraded":"2024-04-16T19:31:31.374413+0000","last_fullsized":"2024-04-16T19:31:31.374413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:15.902753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.374347+0000","last_change":"2024-04-16T17:47:35.314832+0000","last_active":"2024-04-16T19:31:31.374347+0000","last_peered":"2024-04-16T19:31:31.374347+0000","last_clean":"2024-04-16T19:31:31.374347+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:31:31.374347+0000","last_undegraded":"2024-04-16T19:31:31.374347+0000","last_fullsized":"2024-04-16T19:31:31.374347+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:36.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'12","reported_seq":1334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168147+0000","last_change":"2024-04-16T17:47:37.334246+0000","last_active":"2024-04-16T19:33:11.168147+0000","last_peered":"2024-04-16T19:33:11.168147+0000","last_clean":"2024-04-16T19:33:11.168147+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:33:11.168147+0000","last_undegraded":"2024-04-16T19:33:11.168147+0000","last_fullsized":"2024-04-16T19:33:11.168147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:21.558307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.178535+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:34:11.178535+0000","last_peered":"2024-04-16T19:34:11.178535+0000","last_clean":"2024-04-16T19:34:11.178535+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:34:11.178535+0000","last_undegraded":"2024-04-16T19:34:11.178535+0000","last_fullsized":"2024-04-16T19:34:11.178535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:13:48.347861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'164","reported_seq":1853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163697+0000","last_change":"2024-04-16T17:48:34.332673+0000","last_active":"2024-04-16T19:32:46.163697+0000","last_peered":"2024-04-16T19:32:46.163697+0000","last_clean":"2024-04-16T19:32:46.163697+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:32:46.163697+0000","last_undegraded":"2024-04-16T19:32:46.163697+0000","last_fullsized":"2024-04-16T19:32:46.163697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162327+0000","last_change":"2024-04-16T17:48:27.306930+0000","last_active":"2024-04-16T19:32:41.162327+0000","last_peered":"2024-04-16T19:32:41.162327+0000","last_clean":"2024-04-16T19:32:41.162327+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:32:41.162327+0000","last_undegraded":"2024-04-16T19:32:41.162327+0000","last_fullsized":"2024-04-16T19:32:41.162327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:11.441379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'10","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.154948+0000","last_change":"2024-04-16T17:48:01.252700+0000","last_active":"2024-04-16T19:32:36.154948+0000","last_peered":"2024-04-16T19:32:36.154948+0000","last_clean":"2024-04-16T19:32:36.154948+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:32:36.154948+0000","last_undegraded":"2024-04-16T19:32:36.154948+0000","last_fullsized":"2024-04-16T19:32:36.154948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:40.367478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146809+0000","last_change":"2024-04-16T17:47:59.200630+0000","last_active":"2024-04-16T19:32:01.146809+0000","last_peered":"2024-04-16T19:32:01.146809+0000","last_clean":"2024-04-16T19:32:01.146809+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:32:01.146809+0000","last_undegraded":"2024-04-16T19:32:01.146809+0000","last_fullsized":"2024-04-16T19:32:01.146809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1087","reported_seq":1742,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146730+0000","last_change":"2024-04-16T17:47:57.186883+0000","last_active":"2024-04-16T19:32:01.146730+0000","last_peered":"2024-04-16T19:32:01.146730+0000","last_clean":"2024-04-16T19:32:01.146730+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:32:01.146730+0000","last_undegraded":"2024-04-16T19:32:01.146730+0000","last_fullsized":"2024-04-16T19:32:01.146730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:14.402693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"51'4","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159908+0000","last_change":"2024-04-16T17:47:55.177129+0000","last_active":"2024-04-16T19:32:41.159908+0000","last_peered":"2024-04-16T19:32:41.159908+0000","last_clean":"2024-04-16T19:32:41.159908+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:32:41.159908+0000","last_undegraded":"2024-04-16T19:32:41.159908+0000","last_fullsized":"2024-04-16T19:32:41.159908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:31.401461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176098+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:33:36.176098+0000","last_peered":"2024-04-16T19:33:36.176098+0000","last_clean":"2024-04-16T19:33:36.176098+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:33:36.176098+0000","last_undegraded":"2024-04-16T19:33:36.176098+0000","last_fullsized":"2024-04-16T19:33:36.176098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:05.221638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.170137+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:33:31.170137+0000","last_peered":"2024-04-16T19:33:31.170137+0000","last_clean":"2024-04-16T19:33:31.170137+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:33:31.170137+0000","last_undegraded":"2024-04-16T19:33:31.170137+0000","last_fullsized":"2024-04-16T19:33:31.170137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.173925+0000","last_change":"2024-04-16T17:47:33.309390+0000","last_active":"2024-04-16T19:33:46.173925+0000","last_peered":"2024-04-16T19:33:46.173925+0000","last_clean":"2024-04-16T19:33:46.173925+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:33:46.173925+0000","last_undegraded":"2024-04-16T19:33:46.173925+0000","last_fullsized":"2024-04-16T19:33:46.173925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'200","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146472+0000","last_change":"2024-04-16T17:47:35.314414+0000","last_active":"2024-04-16T19:32:01.146472+0000","last_peered":"2024-04-16T19:32:01.146472+0000","last_clean":"2024-04-16T19:32:01.146472+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:32:01.146472+0000","last_undegraded":"2024-04-16T19:32:01.146472+0000","last_fullsized":"2024-04-16T19:32:01.146472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171681+0000","last_change":"2024-04-16T17:47:37.335122+0000","last_active":"2024-04-16T19:33:26.171681+0000","last_peered":"2024-04-16T19:33:26.171681+0000","last_clean":"2024-04-16T19:33:26.171681+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:33:26.171681+0000","last_undegraded":"2024-04-16T19:33:26.171681+0000","last_fullsized":"2024-04-16T19:33:26.171681+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:38.549732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176783+0000","last_change":"2024-04-16T17:47:39.339833+0000","last_active":"2024-04-16T19:33:56.176783+0000","last_peered":"2024-04-16T19:33:56.176783+0000","last_clean":"2024-04-16T19:33:56.176783+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:33:56.176783+0000","last_undegraded":"2024-04-16T19:33:56.176783+0000","last_fullsized":"2024-04-16T19:33:56.176783+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.164663+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:34:11.164663+0000","last_peered":"2024-04-16T19:34:11.164663+0000","last_clean":"2024-04-16T19:34:11.164663+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:34:11.164663+0000","last_undegraded":"2024-04-16T19:34:11.164663+0000","last_fullsized":"2024-04-16T19:34:11.164663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:13:30.436128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1187,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161075+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:32:46.161075+0000","last_peered":"2024-04-16T19:32:46.161075+0000","last_clean":"2024-04-16T19:32:46.161075+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:32:46.161075+0000","last_undegraded":"2024-04-16T19:32:46.161075+0000","last_fullsized":"2024-04-16T19:32:46.161075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168743+0000","last_change":"2024-04-16T17:48:27.299718+0000","last_active":"2024-04-16T19:33:06.168743+0000","last_peered":"2024-04-16T19:33:06.168743+0000","last_clean":"2024-04-16T19:33:06.168743+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:33:06.168743+0000","last_undegraded":"2024-04-16T19:33:06.168743+0000","last_fullsized":"2024-04-16T19:33:06.168743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'2","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154067+0000","last_change":"2024-04-16T17:48:01.253979+0000","last_active":"2024-04-16T19:32:11.154067+0000","last_peered":"2024-04-16T19:32:11.154067+0000","last_clean":"2024-04-16T19:32:11.154067+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:32:11.154067+0000","last_undegraded":"2024-04-16T19:32:11.154067+0000","last_fullsized":"2024-04-16T19:32:11.154067+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149567+0000","last_change":"2024-04-16T17:47:59.193745+0000","last_active":"2024-04-16T19:32:01.149567+0000","last_peered":"2024-04-16T19:32:01.149567+0000","last_clean":"2024-04-16T19:32:01.149567+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:32:01.149567+0000","last_undegraded":"2024-04-16T19:32:01.149567+0000","last_fullsized":"2024-04-16T19:32:01.149567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1173","reported_seq":1516,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151419+0000","last_change":"2024-04-16T17:47:57.191812+0000","last_active":"2024-04-16T19:32:01.151419+0000","last_peered":"2024-04-16T19:32:01.151419+0000","last_clean":"2024-04-16T19:32:01.151419+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:32:01.151419+0000","last_undegraded":"2024-04-16T19:32:01.151419+0000","last_fullsized":"2024-04-16T19:32:01.151419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1173,"log_dups_size":0,"ondisk_log_size":1173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'14","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183617+0000","last_change":"2024-04-16T17:47:55.163005+0000","last_active":"2024-04-16T19:34:11.183617+0000","last_peered":"2024-04-16T19:34:11.183617+0000","last_clean":"2024-04-16T19:34:11.183617+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:34:11.183617+0000","last_undegraded":"2024-04-16T19:34:11.183617+0000","last_fullsized":"2024-04-16T19:34:11.183617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:12:10.909699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169773+0000","last_change":"2024-04-16T17:47:23.266202+0000","last_active":"2024-04-16T19:33:36.169773+0000","last_peered":"2024-04-16T19:33:36.169773+0000","last_clean":"2024-04-16T19:33:36.169773+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:33:36.169773+0000","last_undegraded":"2024-04-16T19:33:36.169773+0000","last_fullsized":"2024-04-16T19:33:36.169773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163890+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:33:06.163890+0000","last_peered":"2024-04-16T19:33:06.163890+0000","last_clean":"2024-04-16T19:33:06.163890+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:33:06.163890+0000","last_undegraded":"2024-04-16T19:33:06.163890+0000","last_fullsized":"2024-04-16T19:33:06.163890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:37:45.902298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"31'1","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158405+0000","last_change":"2024-04-16T17:47:33.302299+0000","last_active":"2024-04-16T19:33:46.158405+0000","last_peered":"2024-04-16T19:33:46.158405+0000","last_clean":"2024-04-16T19:33:46.158405+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:33:46.158405+0000","last_undegraded":"2024-04-16T19:33:46.158405+0000","last_fullsized":"2024-04-16T19:33:46.158405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:23.313435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'80","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170395+0000","last_change":"2024-04-16T17:47:35.321698+0000","last_active":"2024-04-16T19:33:21.170395+0000","last_peered":"2024-04-16T19:33:21.170395+0000","last_clean":"2024-04-16T19:33:21.170395+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:33:21.170395+0000","last_undegraded":"2024-04-16T19:33:21.170395+0000","last_fullsized":"2024-04-16T19:33:21.170395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:02.350103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136445+0000","last_change":"2024-04-16T17:47:37.325461+0000","last_active":"2024-04-16T19:32:11.136445+0000","last_peered":"2024-04-16T19:32:11.136445+0000","last_clean":"2024-04-16T19:32:11.136445+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:32:11.136445+0000","last_undegraded":"2024-04-16T19:32:11.136445+0000","last_fullsized":"2024-04-16T19:32:11.136445+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:18.621208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152703+0000","last_change":"2024-04-16T17:47:39.323337+0000","last_active":"2024-04-16T19:32:26.152703+0000","last_peered":"2024-04-16T19:32:26.152703+0000","last_clean":"2024-04-16T19:32:26.152703+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:32:26.152703+0000","last_undegraded":"2024-04-16T19:32:26.152703+0000","last_fullsized":"2024-04-16T19:32:26.152703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:12.655890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186537+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:34:11.186537+0000","last_peered":"2024-04-16T19:34:11.186537+0000","last_clean":"2024-04-16T19:34:11.186537+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:34:11.186537+0000","last_undegraded":"2024-04-16T19:34:11.186537+0000","last_fullsized":"2024-04-16T19:34:11.186537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:58.661997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'70","reported_seq":901,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.148912+0000","last_change":"2024-04-16T17:48:34.345575+0000","last_active":"2024-04-16T19:33:06.148912+0000","last_peered":"2024-04-16T19:33:06.148912+0000","last_clean":"2024-04-16T19:33:06.148912+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:33:06.148912+0000","last_undegraded":"2024-04-16T19:33:06.148912+0000","last_fullsized":"2024-04-16T19:33:06.148912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:56:51.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159686+0000","last_change":"2024-04-16T17:48:27.295648+0000","last_active":"2024-04-16T19:32:41.159686+0000","last_peered":"2024-04-16T19:32:41.159686+0000","last_clean":"2024-04-16T19:32:41.159686+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:32:41.159686+0000","last_undegraded":"2024-04-16T19:32:41.159686+0000","last_fullsized":"2024-04-16T19:32:41.159686+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154259+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:32:11.154259+0000","last_peered":"2024-04-16T19:32:11.154259+0000","last_clean":"2024-04-16T19:32:11.154259+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:32:11.154259+0000","last_undegraded":"2024-04-16T19:32:11.154259+0000","last_fullsized":"2024-04-16T19:32:11.154259+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:20.025368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.157655+0000","last_change":"2024-04-16T17:47:59.197037+0000","last_active":"2024-04-16T19:32:46.157655+0000","last_peered":"2024-04-16T19:32:46.157655+0000","last_clean":"2024-04-16T19:32:46.157655+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:32:46.157655+0000","last_undegraded":"2024-04-16T19:32:46.157655+0000","last_fullsized":"2024-04-16T19:32:46.157655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:33:54.818691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1049","reported_seq":6110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152019+0000","last_change":"2024-04-16T17:47:57.189817+0000","last_active":"2024-04-16T19:32:11.152019+0000","last_peered":"2024-04-16T19:32:11.152019+0000","last_clean":"2024-04-16T19:32:11.152019+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:32:11.152019+0000","last_undegraded":"2024-04-16T19:32:11.152019+0000","last_fullsized":"2024-04-16T19:32:11.152019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'4","reported_seq":178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179818+0000","last_change":"2024-04-16T17:47:55.182984+0000","last_active":"2024-04-16T19:34:06.179818+0000","last_peered":"2024-04-16T19:34:06.179818+0000","last_clean":"2024-04-16T19:34:06.179818+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:34:06.179818+0000","last_undegraded":"2024-04-16T19:34:06.179818+0000","last_fullsized":"2024-04-16T19:34:06.179818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:55.129164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178050+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:34:01.178050+0000","last_peered":"2024-04-16T19:34:01.178050+0000","last_clean":"2024-04-16T19:34:01.178050+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:34:01.178050+0000","last_undegraded":"2024-04-16T19:34:01.178050+0000","last_fullsized":"2024-04-16T19:34:01.178050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.169031+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:33:06.169031+0000","last_peered":"2024-04-16T19:33:06.169031+0000","last_clean":"2024-04-16T19:33:06.169031+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:33:06.169031+0000","last_undegraded":"2024-04-16T19:33:06.169031+0000","last_fullsized":"2024-04-16T19:33:06.169031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:05.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152471+0000","last_change":"2024-04-16T17:47:33.302725+0000","last_active":"2024-04-16T19:33:21.152471+0000","last_peered":"2024-04-16T19:33:21.152471+0000","last_clean":"2024-04-16T19:33:21.152471+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:33:21.152471+0000","last_undegraded":"2024-04-16T19:33:21.152471+0000","last_fullsized":"2024-04-16T19:33:21.152471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:42.697302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'152","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179125+0000","last_change":"2024-04-16T17:47:35.317742+0000","last_active":"2024-04-16T19:33:46.179125+0000","last_peered":"2024-04-16T19:33:46.179125+0000","last_clean":"2024-04-16T19:33:46.179125+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:33:46.179125+0000","last_undegraded":"2024-04-16T19:33:46.179125+0000","last_fullsized":"2024-04-16T19:33:46.179125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'12","reported_seq":1338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177186+0000","last_change":"2024-04-16T17:47:37.318010+0000","last_active":"2024-04-16T19:33:51.177186+0000","last_peered":"2024-04-16T19:33:51.177186+0000","last_clean":"2024-04-16T19:33:51.177186+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:33:51.177186+0000","last_undegraded":"2024-04-16T19:33:51.177186+0000","last_fullsized":"2024-04-16T19:33:51.177186+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177479+0000","last_change":"2024-04-16T17:47:39.328909+0000","last_active":"2024-04-16T19:33:51.177479+0000","last_peered":"2024-04-16T19:33:51.177479+0000","last_clean":"2024-04-16T19:33:51.177479+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:33:51.177479+0000","last_undegraded":"2024-04-16T19:33:51.177479+0000","last_fullsized":"2024-04-16T19:33:51.177479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:54.351646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152331+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:33:21.152331+0000","last_peered":"2024-04-16T19:33:21.152331+0000","last_clean":"2024-04-16T19:33:21.152331+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:33:21.152331+0000","last_undegraded":"2024-04-16T19:33:21.152331+0000","last_fullsized":"2024-04-16T19:33:21.152331+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:16.718022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'74","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.161693+0000","last_change":"2024-04-16T17:48:34.327024+0000","last_active":"2024-04-16T19:32:46.161693+0000","last_peered":"2024-04-16T19:32:46.161693+0000","last_clean":"2024-04-16T19:32:46.161693+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:32:46.161693+0000","last_undegraded":"2024-04-16T19:32:46.161693+0000","last_fullsized":"2024-04-16T19:32:46.161693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:00.683240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157359+0000","last_change":"2024-04-16T17:48:27.301787+0000","last_active":"2024-04-16T19:32:41.157359+0000","last_peered":"2024-04-16T19:32:41.157359+0000","last_clean":"2024-04-16T19:32:41.157359+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:32:41.157359+0000","last_undegraded":"2024-04-16T19:32:41.157359+0000","last_fullsized":"2024-04-16T19:32:41.157359+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'17","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.151912+0000","last_change":"2024-04-16T17:48:01.243297+0000","last_active":"2024-04-16T19:32:11.151912+0000","last_peered":"2024-04-16T19:32:11.151912+0000","last_clean":"2024-04-16T19:32:11.151912+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:32:11.151912+0000","last_undegraded":"2024-04-16T19:32:11.151912+0000","last_fullsized":"2024-04-16T19:32:11.151912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:45.703782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149205+0000","last_change":"2024-04-16T17:47:59.193938+0000","last_active":"2024-04-16T19:32:11.149205+0000","last_peered":"2024-04-16T19:32:11.149205+0000","last_clean":"2024-04-16T19:32:11.149205+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:32:11.149205+0000","last_undegraded":"2024-04-16T19:32:11.149205+0000","last_fullsized":"2024-04-16T19:32:11.149205+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:36.675392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'703","reported_seq":1434,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.143345+0000","last_change":"2024-04-16T17:47:57.173194+0000","last_active":"2024-04-16T19:32:41.143345+0000","last_peered":"2024-04-16T19:32:41.143345+0000","last_clean":"2024-04-16T19:32:41.143345+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:32:41.143345+0000","last_undegraded":"2024-04-16T19:32:41.143345+0000","last_fullsized":"2024-04-16T19:32:41.143345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:02.892691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'14","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179779+0000","last_change":"2024-04-16T17:47:55.181545+0000","last_active":"2024-04-16T19:34:06.179779+0000","last_peered":"2024-04-16T19:34:06.179779+0000","last_clean":"2024-04-16T19:34:06.179779+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:34:06.179779+0000","last_undegraded":"2024-04-16T19:34:06.179779+0000","last_fullsized":"2024-04-16T19:34:06.179779+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167271+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:33:11.167271+0000","last_peered":"2024-04-16T19:33:11.167271+0000","last_clean":"2024-04-16T19:33:11.167271+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:33:11.167271+0000","last_undegraded":"2024-04-16T19:33:11.167271+0000","last_fullsized":"2024-04-16T19:33:11.167271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:36:01.244536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163761+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:33:06.163761+0000","last_peered":"2024-04-16T19:33:06.163761+0000","last_clean":"2024-04-16T19:33:06.163761+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:33:06.163761+0000","last_undegraded":"2024-04-16T19:33:06.163761+0000","last_fullsized":"2024-04-16T19:33:06.163761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.135219+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:32:06.135219+0000","last_peered":"2024-04-16T19:32:06.135219+0000","last_clean":"2024-04-16T19:32:06.135219+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:32:06.135219+0000","last_undegraded":"2024-04-16T19:32:06.135219+0000","last_fullsized":"2024-04-16T19:32:06.135219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:21.347240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'88","reported_seq":256,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175899+0000","last_change":"2024-04-16T17:47:35.317589+0000","last_active":"2024-04-16T19:33:46.175899+0000","last_peered":"2024-04-16T19:33:46.175899+0000","last_clean":"2024-04-16T19:33:46.175899+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:33:46.175899+0000","last_undegraded":"2024-04-16T19:33:46.175899+0000","last_fullsized":"2024-04-16T19:33:46.175899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159758+0000","last_change":"2024-04-16T17:47:37.330541+0000","last_active":"2024-04-16T19:33:51.159758+0000","last_peered":"2024-04-16T19:33:51.159758+0000","last_clean":"2024-04-16T19:33:51.159758+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:33:51.159758+0000","last_undegraded":"2024-04-16T19:33:51.159758+0000","last_fullsized":"2024-04-16T19:33:51.159758+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:07.207646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178646+0000","last_change":"2024-04-16T17:47:39.338681+0000","last_active":"2024-04-16T19:33:51.178646+0000","last_peered":"2024-04-16T19:33:51.178646+0000","last_clean":"2024-04-16T19:33:51.178646+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:33:51.178646+0000","last_undegraded":"2024-04-16T19:33:51.178646+0000","last_fullsized":"2024-04-16T19:33:51.178646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.168291+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:33:11.168291+0000","last_peered":"2024-04-16T19:33:11.168291+0000","last_clean":"2024-04-16T19:33:11.168291+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:33:11.168291+0000","last_undegraded":"2024-04-16T19:33:11.168291+0000","last_fullsized":"2024-04-16T19:33:11.168291+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:13:26.664630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'40","reported_seq":973,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158409+0000","last_change":"2024-04-16T17:48:34.333313+0000","last_active":"2024-04-16T19:32:36.158409+0000","last_peered":"2024-04-16T19:32:36.158409+0000","last_clean":"2024-04-16T19:32:36.158409+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:32:36.158409+0000","last_undegraded":"2024-04-16T19:32:36.158409+0000","last_fullsized":"2024-04-16T19:32:36.158409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:06:58.890753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164925+0000","last_change":"2024-04-16T17:48:27.304408+0000","last_active":"2024-04-16T19:33:16.164925+0000","last_peered":"2024-04-16T19:33:16.164925+0000","last_clean":"2024-04-16T19:33:16.164925+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:33:16.164925+0000","last_undegraded":"2024-04-16T19:33:16.164925+0000","last_fullsized":"2024-04-16T19:33:16.164925+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:27:10.178784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'9","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149161+0000","last_change":"2024-04-16T17:48:01.259988+0000","last_active":"2024-04-16T19:32:11.149161+0000","last_peered":"2024-04-16T19:32:11.149161+0000","last_clean":"2024-04-16T19:32:11.149161+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:32:11.149161+0000","last_undegraded":"2024-04-16T19:32:11.149161+0000","last_fullsized":"2024-04-16T19:32:11.149161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'555","reported_seq":2287,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154329+0000","last_change":"2024-04-16T17:47:59.181942+0000","last_active":"2024-04-16T19:32:11.154329+0000","last_peered":"2024-04-16T19:32:11.154329+0000","last_clean":"2024-04-16T19:32:11.154329+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:32:11.154329+0000","last_undegraded":"2024-04-16T19:32:11.154329+0000","last_fullsized":"2024-04-16T19:32:11.154329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:25.688721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'901","reported_seq":1313,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152857+0000","last_change":"2024-04-16T17:47:57.174998+0000","last_active":"2024-04-16T19:32:11.152857+0000","last_peered":"2024-04-16T19:32:11.152857+0000","last_clean":"2024-04-16T19:32:11.152857+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:32:11.152857+0000","last_undegraded":"2024-04-16T19:32:11.152857+0000","last_fullsized":"2024-04-16T19:32:11.152857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:14.325870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'14","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182596+0000","last_change":"2024-04-16T17:47:55.155349+0000","last_active":"2024-04-16T19:34:11.182596+0000","last_peered":"2024-04-16T19:34:11.182596+0000","last_clean":"2024-04-16T19:34:11.182596+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:34:11.182596+0000","last_undegraded":"2024-04-16T19:34:11.182596+0000","last_fullsized":"2024-04-16T19:34:11.182596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163638+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:33:11.163638+0000","last_peered":"2024-04-16T19:33:11.163638+0000","last_clean":"2024-04-16T19:33:11.163638+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:33:11.163638+0000","last_undegraded":"2024-04-16T19:33:11.163638+0000","last_fullsized":"2024-04-16T19:33:11.163638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:03.973505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166835+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:33:06.166835+0000","last_peered":"2024-04-16T19:33:06.166835+0000","last_clean":"2024-04-16T19:33:06.166835+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:33:06.166835+0000","last_undegraded":"2024-04-16T19:33:06.166835+0000","last_fullsized":"2024-04-16T19:33:06.166835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":175,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.152538+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:33:21.152538+0000","last_peered":"2024-04-16T19:33:21.152538+0000","last_clean":"2024-04-16T19:33:21.152538+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:33:21.152538+0000","last_undegraded":"2024-04-16T19:33:21.152538+0000","last_fullsized":"2024-04-16T19:33:21.152538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:05.646107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'104","reported_seq":273,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.166177+0000","last_change":"2024-04-16T17:47:35.307146+0000","last_active":"2024-04-16T19:33:21.166177+0000","last_peered":"2024-04-16T19:33:21.166177+0000","last_clean":"2024-04-16T19:33:21.166177+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:33:21.166177+0000","last_undegraded":"2024-04-16T19:33:21.166177+0000","last_fullsized":"2024-04-16T19:33:21.166177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:28.729299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.180239+0000","last_change":"2024-04-16T17:47:37.324514+0000","last_active":"2024-04-16T19:34:01.180239+0000","last_peered":"2024-04-16T19:34:01.180239+0000","last_clean":"2024-04-16T19:34:01.180239+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:34:01.180239+0000","last_undegraded":"2024-04-16T19:34:01.180239+0000","last_fullsized":"2024-04-16T19:34:01.180239+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183327+0000","last_change":"2024-04-16T17:47:39.340900+0000","last_active":"2024-04-16T19:34:11.183327+0000","last_peered":"2024-04-16T19:34:11.183327+0000","last_clean":"2024-04-16T19:34:11.183327+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:34:11.183327+0000","last_undegraded":"2024-04-16T19:34:11.183327+0000","last_fullsized":"2024-04-16T19:34:11.183327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.644408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167437+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:33:11.167437+0000","last_peered":"2024-04-16T19:33:11.167437+0000","last_clean":"2024-04-16T19:33:11.167437+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:33:11.167437+0000","last_undegraded":"2024-04-16T19:33:11.167437+0000","last_fullsized":"2024-04-16T19:33:11.167437+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156402+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:32:36.156402+0000","last_peered":"2024-04-16T19:32:36.156402+0000","last_clean":"2024-04-16T19:32:36.156402+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:32:36.156402+0000","last_undegraded":"2024-04-16T19:32:36.156402+0000","last_fullsized":"2024-04-16T19:32:36.156402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157486+0000","last_change":"2024-04-16T17:48:27.308234+0000","last_active":"2024-04-16T19:32:41.157486+0000","last_peered":"2024-04-16T19:32:41.157486+0000","last_clean":"2024-04-16T19:32:41.157486+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:32:41.157486+0000","last_undegraded":"2024-04-16T19:32:41.157486+0000","last_fullsized":"2024-04-16T19:32:41.157486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:26.381542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'135","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163865+0000","last_change":"2024-04-16T17:48:01.254153+0000","last_active":"2024-04-16T19:32:46.163865+0000","last_peered":"2024-04-16T19:32:46.163865+0000","last_clean":"2024-04-16T19:32:46.163865+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:32:46.163865+0000","last_undegraded":"2024-04-16T19:32:46.163865+0000","last_fullsized":"2024-04-16T19:32:46.163865+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136391+0000","last_change":"2024-04-16T17:47:59.174180+0000","last_active":"2024-04-16T19:32:11.136391+0000","last_peered":"2024-04-16T19:32:11.136391+0000","last_clean":"2024-04-16T19:32:11.136391+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:32:11.136391+0000","last_undegraded":"2024-04-16T19:32:11.136391+0000","last_fullsized":"2024-04-16T19:32:11.136391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:29.135288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'679","reported_seq":1066,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136485+0000","last_change":"2024-04-16T17:47:57.182491+0000","last_active":"2024-04-16T19:32:11.136485+0000","last_peered":"2024-04-16T19:32:11.136485+0000","last_clean":"2024-04-16T19:32:11.136485+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:32:11.136485+0000","last_undegraded":"2024-04-16T19:32:11.136485+0000","last_fullsized":"2024-04-16T19:32:11.136485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:12:14.757156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'6","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156433+0000","last_change":"2024-04-16T17:47:55.158851+0000","last_active":"2024-04-16T19:32:26.156433+0000","last_peered":"2024-04-16T19:32:26.156433+0000","last_clean":"2024-04-16T19:32:26.156433+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:32:26.156433+0000","last_undegraded":"2024-04-16T19:32:26.156433+0000","last_fullsized":"2024-04-16T19:32:26.156433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:39:56.856045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170332+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:33:11.170332+0000","last_peered":"2024-04-16T19:33:11.170332+0000","last_clean":"2024-04-16T19:33:11.170332+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:33:11.170332+0000","last_undegraded":"2024-04-16T19:33:11.170332+0000","last_fullsized":"2024-04-16T19:33:11.170332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:31.171367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.184927+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:34:06.184927+0000","last_peered":"2024-04-16T19:34:06.184927+0000","last_clean":"2024-04-16T19:34:06.184927+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:34:06.184927+0000","last_undegraded":"2024-04-16T19:34:06.184927+0000","last_fullsized":"2024-04-16T19:34:06.184927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:06.495990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"48'13","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170506+0000","last_change":"2024-04-16T17:47:33.309159+0000","last_active":"2024-04-16T19:33:21.170506+0000","last_peered":"2024-04-16T19:33:21.170506+0000","last_clean":"2024-04-16T19:33:21.170506+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:33:21.170506+0000","last_undegraded":"2024-04-16T19:33:21.170506+0000","last_fullsized":"2024-04-16T19:33:21.170506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:26.709263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'152","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160039+0000","last_change":"2024-04-16T17:47:35.305022+0000","last_active":"2024-04-16T19:33:51.160039+0000","last_peered":"2024-04-16T19:33:51.160039+0000","last_clean":"2024-04-16T19:33:51.160039+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:33:51.160039+0000","last_undegraded":"2024-04-16T19:33:51.160039+0000","last_fullsized":"2024-04-16T19:33:51.160039+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:03.908445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.153692+0000","last_change":"2024-04-16T17:47:37.331019+0000","last_active":"2024-04-16T19:33:26.153692+0000","last_peered":"2024-04-16T19:33:26.153692+0000","last_clean":"2024-04-16T19:33:26.153692+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:33:26.153692+0000","last_undegraded":"2024-04-16T19:33:26.153692+0000","last_fullsized":"2024-04-16T19:33:26.153692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:18.849817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179728+0000","last_change":"2024-04-16T17:47:39.331395+0000","last_active":"2024-04-16T19:33:56.179728+0000","last_peered":"2024-04-16T19:33:56.179728+0000","last_clean":"2024-04-16T19:33:56.179728+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:33:56.179728+0000","last_undegraded":"2024-04-16T19:33:56.179728+0000","last_fullsized":"2024-04-16T19:33:56.179728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:12.149771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160965+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:32:36.160965+0000","last_peered":"2024-04-16T19:32:36.160965+0000","last_clean":"2024-04-16T19:32:36.160965+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:32:36.160965+0000","last_undegraded":"2024-04-16T19:32:36.160965+0000","last_fullsized":"2024-04-16T19:32:36.160965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:26:34.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'86","reported_seq":1117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163783+0000","last_change":"2024-04-16T17:48:34.326754+0000","last_active":"2024-04-16T19:32:46.163783+0000","last_peered":"2024-04-16T19:32:46.163783+0000","last_clean":"2024-04-16T19:32:46.163783+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:32:46.163783+0000","last_undegraded":"2024-04-16T19:32:46.163783+0000","last_fullsized":"2024-04-16T19:32:46.163783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:29:47.712583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159819+0000","last_change":"2024-04-16T17:48:27.298815+0000","last_active":"2024-04-16T19:32:31.159819+0000","last_peered":"2024-04-16T19:32:31.159819+0000","last_clean":"2024-04-16T19:32:31.159819+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:32:31.159819+0000","last_undegraded":"2024-04-16T19:32:31.159819+0000","last_fullsized":"2024-04-16T19:32:31.159819+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"48'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150226+0000","last_change":"2024-04-16T17:48:01.246604+0000","last_active":"2024-04-16T19:32:01.150226+0000","last_peered":"2024-04-16T19:32:01.150226+0000","last_clean":"2024-04-16T19:32:01.150226+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:32:01.150226+0000","last_undegraded":"2024-04-16T19:32:01.150226+0000","last_fullsized":"2024-04-16T19:32:01.150226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:16.195310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148847+0000","last_change":"2024-04-16T17:47:59.180320+0000","last_active":"2024-04-16T19:32:11.148847+0000","last_peered":"2024-04-16T19:32:11.148847+0000","last_clean":"2024-04-16T19:32:11.148847+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:32:11.148847+0000","last_undegraded":"2024-04-16T19:32:11.148847+0000","last_fullsized":"2024-04-16T19:32:11.148847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:16.107500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1483","reported_seq":5477,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153145+0000","last_change":"2024-04-16T17:47:57.186127+0000","last_active":"2024-04-16T19:32:26.153145+0000","last_peered":"2024-04-16T19:32:26.153145+0000","last_clean":"2024-04-16T19:32:26.153145+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:32:26.153145+0000","last_undegraded":"2024-04-16T19:32:26.153145+0000","last_fullsized":"2024-04-16T19:32:26.153145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1483,"log_dups_size":0,"ondisk_log_size":1483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:02.434899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'8","reported_seq":225,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186717+0000","last_change":"2024-04-16T17:47:55.161973+0000","last_active":"2024-04-16T19:34:11.186717+0000","last_peered":"2024-04-16T19:34:11.186717+0000","last_clean":"2024-04-16T19:34:11.186717+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:34:11.186717+0000","last_undegraded":"2024-04-16T19:34:11.186717+0000","last_fullsized":"2024-04-16T19:34:11.186717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163747+0000","last_change":"2024-04-16T17:47:23.266018+0000","last_active":"2024-04-16T19:33:11.163747+0000","last_peered":"2024-04-16T19:33:11.163747+0000","last_clean":"2024-04-16T19:33:11.163747+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:33:11.163747+0000","last_undegraded":"2024-04-16T19:33:11.163747+0000","last_fullsized":"2024-04-16T19:33:11.163747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168338+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:33:31.168338+0000","last_peered":"2024-04-16T19:33:31.168338+0000","last_clean":"2024-04-16T19:33:31.168338+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:33:31.168338+0000","last_undegraded":"2024-04-16T19:33:31.168338+0000","last_fullsized":"2024-04-16T19:33:31.168338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174370+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:33:46.174370+0000","last_peered":"2024-04-16T19:33:46.174370+0000","last_clean":"2024-04-16T19:33:46.174370+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:33:46.174370+0000","last_undegraded":"2024-04-16T19:33:46.174370+0000","last_fullsized":"2024-04-16T19:33:46.174370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'120","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.170532+0000","last_change":"2024-04-16T17:47:35.319180+0000","last_active":"2024-04-16T19:33:21.170532+0000","last_peered":"2024-04-16T19:33:21.170532+0000","last_clean":"2024-04-16T19:33:21.170532+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:33:21.170532+0000","last_undegraded":"2024-04-16T19:33:21.170532+0000","last_fullsized":"2024-04-16T19:33:21.170532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:28.081965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'12","reported_seq":1336,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.167247+0000","last_change":"2024-04-16T17:47:37.326971+0000","last_active":"2024-04-16T19:33:26.167247+0000","last_peered":"2024-04-16T19:33:26.167247+0000","last_clean":"2024-04-16T19:33:26.167247+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:33:26.167247+0000","last_undegraded":"2024-04-16T19:33:26.167247+0000","last_fullsized":"2024-04-16T19:33:26.167247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:04.366184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174929+0000","last_change":"2024-04-16T17:47:39.325433+0000","last_active":"2024-04-16T19:33:56.174929+0000","last_peered":"2024-04-16T19:33:56.174929+0000","last_clean":"2024-04-16T19:33:56.174929+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:33:56.174929+0000","last_undegraded":"2024-04-16T19:33:56.174929+0000","last_fullsized":"2024-04-16T19:33:56.174929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:50.427879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183755+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:34:11.183755+0000","last_peered":"2024-04-16T19:34:11.183755+0000","last_clean":"2024-04-16T19:34:11.183755+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:34:11.183755+0000","last_undegraded":"2024-04-16T19:34:11.183755+0000","last_fullsized":"2024-04-16T19:34:11.183755+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:24.908570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'90","reported_seq":977,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.169718+0000","last_change":"2024-04-16T17:48:34.333850+0000","last_active":"2024-04-16T19:33:21.169718+0000","last_peered":"2024-04-16T19:33:21.169718+0000","last_clean":"2024-04-16T19:33:21.169718+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:33:21.169718+0000","last_undegraded":"2024-04-16T19:33:21.169718+0000","last_fullsized":"2024-04-16T19:33:21.169718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:31.546580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159785+0000","last_change":"2024-04-16T17:48:27.299707+0000","last_active":"2024-04-16T19:32:31.159785+0000","last_peered":"2024-04-16T19:32:31.159785+0000","last_clean":"2024-04-16T19:32:31.159785+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:32:31.159785+0000","last_undegraded":"2024-04-16T19:32:31.159785+0000","last_fullsized":"2024-04-16T19:32:31.159785+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'5","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150341+0000","last_change":"2024-04-16T17:48:01.258518+0000","last_active":"2024-04-16T19:32:01.150341+0000","last_peered":"2024-04-16T19:32:01.150341+0000","last_clean":"2024-04-16T19:32:01.150341+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:32:01.150341+0000","last_undegraded":"2024-04-16T19:32:01.150341+0000","last_fullsized":"2024-04-16T19:32:01.150341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:25.260545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'555","reported_seq":1941,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157758+0000","last_change":"2024-04-16T17:47:59.184752+0000","last_active":"2024-04-16T19:32:31.157758+0000","last_peered":"2024-04-16T19:32:31.157758+0000","last_clean":"2024-04-16T19:32:31.157758+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:32:31.157758+0000","last_undegraded":"2024-04-16T19:32:31.157758+0000","last_fullsized":"2024-04-16T19:32:31.157758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:36.214248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'651","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.148802+0000","last_change":"2024-04-16T17:47:57.193900+0000","last_active":"2024-04-16T19:32:11.148802+0000","last_peered":"2024-04-16T19:32:11.148802+0000","last_clean":"2024-04-16T19:32:11.148802+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:32:11.148802+0000","last_undegraded":"2024-04-16T19:32:11.148802+0000","last_fullsized":"2024-04-16T19:32:11.148802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:12.186417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'12","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183485+0000","last_change":"2024-04-16T17:47:55.161557+0000","last_active":"2024-04-16T19:34:11.183485+0000","last_peered":"2024-04-16T19:34:11.183485+0000","last_clean":"2024-04-16T19:34:11.183485+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:34:11.183485+0000","last_undegraded":"2024-04-16T19:34:11.183485+0000","last_fullsized":"2024-04-16T19:34:11.183485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:51.923908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.176037+0000","last_change":"2024-04-16T17:47:23.261436+0000","last_active":"2024-04-16T19:33:41.176037+0000","last_peered":"2024-04-16T19:33:41.176037+0000","last_clean":"2024-04-16T19:33:41.176037+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:33:41.176037+0000","last_undegraded":"2024-04-16T19:33:41.176037+0000","last_fullsized":"2024-04-16T19:33:41.176037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:56:53.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154630+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:33:31.154630+0000","last_peered":"2024-04-16T19:33:31.154630+0000","last_clean":"2024-04-16T19:33:31.154630+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:33:31.154630+0000","last_undegraded":"2024-04-16T19:33:31.154630+0000","last_fullsized":"2024-04-16T19:33:31.154630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:35.902429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172709+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:33:21.172709+0000","last_peered":"2024-04-16T19:33:21.172709+0000","last_clean":"2024-04-16T19:33:21.172709+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:33:21.172709+0000","last_undegraded":"2024-04-16T19:33:21.172709+0000","last_fullsized":"2024-04-16T19:33:21.172709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:06:26.061929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'176","reported_seq":381,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167670+0000","last_change":"2024-04-16T17:47:35.317949+0000","last_active":"2024-04-16T19:33:21.167670+0000","last_peered":"2024-04-16T19:33:21.167670+0000","last_clean":"2024-04-16T19:33:21.167670+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:33:21.167670+0000","last_undegraded":"2024-04-16T19:33:21.167670+0000","last_fullsized":"2024-04-16T19:33:21.167670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:51.928583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.173767+0000","last_change":"2024-04-16T17:47:37.332636+0000","last_active":"2024-04-16T19:33:26.173767+0000","last_peered":"2024-04-16T19:33:26.173767+0000","last_clean":"2024-04-16T19:33:26.173767+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:33:26.173767+0000","last_undegraded":"2024-04-16T19:33:26.173767+0000","last_fullsized":"2024-04-16T19:33:26.173767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:30:43.046155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.179806+0000","last_change":"2024-04-16T17:47:39.335342+0000","last_active":"2024-04-16T19:33:56.179806+0000","last_peered":"2024-04-16T19:33:56.179806+0000","last_clean":"2024-04-16T19:33:56.179806+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:33:56.179806+0000","last_undegraded":"2024-04-16T19:33:56.179806+0000","last_fullsized":"2024-04-16T19:33:56.179806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131831+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:31:51.131831+0000","last_peered":"2024-04-16T19:31:51.131831+0000","last_clean":"2024-04-16T19:31:51.131831+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:31:51.131831+0000","last_undegraded":"2024-04-16T19:31:51.131831+0000","last_fullsized":"2024-04-16T19:31:51.131831+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:43.281248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'118","reported_seq":1502,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.156628+0000","last_change":"2024-04-16T17:48:34.329015+0000","last_active":"2024-04-16T19:32:41.156628+0000","last_peered":"2024-04-16T19:32:41.156628+0000","last_clean":"2024-04-16T19:32:41.156628+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:32:41.156628+0000","last_undegraded":"2024-04-16T19:32:41.156628+0000","last_fullsized":"2024-04-16T19:32:41.156628+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:38.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.150062+0000","last_change":"2024-04-16T17:48:27.296987+0000","last_active":"2024-04-16T19:32:01.150062+0000","last_peered":"2024-04-16T19:32:01.150062+0000","last_clean":"2024-04-16T19:32:01.150062+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:32:01.150062+0000","last_undegraded":"2024-04-16T19:32:01.150062+0000","last_fullsized":"2024-04-16T19:32:01.150062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:07:13.235223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'8","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172452+0000","last_change":"2024-04-16T17:48:01.240484+0000","last_active":"2024-04-16T19:33:46.172452+0000","last_peered":"2024-04-16T19:33:46.172452+0000","last_clean":"2024-04-16T19:33:46.172452+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:33:46.172452+0000","last_undegraded":"2024-04-16T19:33:46.172452+0000","last_fullsized":"2024-04-16T19:33:46.172452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:38.257398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157906+0000","last_change":"2024-04-16T17:47:59.184121+0000","last_active":"2024-04-16T19:32:31.157906+0000","last_peered":"2024-04-16T19:32:31.157906+0000","last_clean":"2024-04-16T19:32:31.157906+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:32:31.157906+0000","last_undegraded":"2024-04-16T19:32:31.157906+0000","last_fullsized":"2024-04-16T19:32:31.157906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:46.708915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'783","reported_seq":43142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174072+0000","last_change":"2024-04-16T17:47:57.186095+0000","last_active":"2024-04-16T19:33:46.174072+0000","last_peered":"2024-04-16T19:33:46.174072+0000","last_clean":"2024-04-16T19:33:46.174072+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:33:46.174072+0000","last_undegraded":"2024-04-16T19:33:46.174072+0000","last_fullsized":"2024-04-16T19:33:46.174072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:50.249328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'9","reported_seq":209,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163178+0000","last_change":"2024-04-16T17:47:55.143458+0000","last_active":"2024-04-16T19:34:06.163178+0000","last_peered":"2024-04-16T19:34:06.163178+0000","last_clean":"2024-04-16T19:34:06.163178+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:34:06.163178+0000","last_undegraded":"2024-04-16T19:34:06.163178+0000","last_fullsized":"2024-04-16T19:34:06.163178+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:53:52.793161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154404+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:32:11.154404+0000","last_peered":"2024-04-16T19:32:11.154404+0000","last_clean":"2024-04-16T19:32:11.154404+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:32:11.154404+0000","last_undegraded":"2024-04-16T19:32:11.154404+0000","last_fullsized":"2024-04-16T19:32:11.154404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:43.276218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171281+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:33:16.171281+0000","last_peered":"2024-04-16T19:33:16.171281+0000","last_clean":"2024-04-16T19:33:16.171281+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:33:16.171281+0000","last_undegraded":"2024-04-16T19:33:16.171281+0000","last_fullsized":"2024-04-16T19:33:16.171281+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174137+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:33:36.174137+0000","last_peered":"2024-04-16T19:33:36.174137+0000","last_clean":"2024-04-16T19:33:36.174137+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:33:36.174137+0000","last_undegraded":"2024-04-16T19:33:36.174137+0000","last_fullsized":"2024-04-16T19:33:36.174137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:33:14.972217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173777+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:33:51.173777+0000","last_peered":"2024-04-16T19:33:51.173777+0000","last_clean":"2024-04-16T19:33:51.173777+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:33:51.173777+0000","last_undegraded":"2024-04-16T19:33:51.173777+0000","last_fullsized":"2024-04-16T19:33:51.173777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:14.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'72","reported_seq":237,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174316+0000","last_change":"2024-04-16T17:47:35.310088+0000","last_active":"2024-04-16T19:33:46.174316+0000","last_peered":"2024-04-16T19:33:46.174316+0000","last_clean":"2024-04-16T19:33:46.174316+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:33:46.174316+0000","last_undegraded":"2024-04-16T19:33:46.174316+0000","last_fullsized":"2024-04-16T19:33:46.174316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177457+0000","last_change":"2024-04-16T17:47:37.327822+0000","last_active":"2024-04-16T19:33:51.177457+0000","last_peered":"2024-04-16T19:33:51.177457+0000","last_clean":"2024-04-16T19:33:51.177457+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:33:51.177457+0000","last_undegraded":"2024-04-16T19:33:51.177457+0000","last_fullsized":"2024-04-16T19:33:51.177457+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175307+0000","last_change":"2024-04-16T17:47:39.341107+0000","last_active":"2024-04-16T19:33:51.175307+0000","last_peered":"2024-04-16T19:33:51.175307+0000","last_clean":"2024-04-16T19:33:51.175307+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:33:51.175307+0000","last_undegraded":"2024-04-16T19:33:51.175307+0000","last_fullsized":"2024-04-16T19:33:51.175307+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.157259+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:32:21.157259+0000","last_peered":"2024-04-16T19:32:21.157259+0000","last_clean":"2024-04-16T19:32:21.157259+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:32:21.157259+0000","last_undegraded":"2024-04-16T19:32:21.157259+0000","last_fullsized":"2024-04-16T19:32:21.157259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'100","reported_seq":1745,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147877+0000","last_change":"2024-04-16T17:48:34.333902+0000","last_active":"2024-04-16T19:32:06.147877+0000","last_peered":"2024-04-16T19:32:06.147877+0000","last_clean":"2024-04-16T19:32:06.147877+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:32:06.147877+0000","last_undegraded":"2024-04-16T19:32:06.147877+0000","last_fullsized":"2024-04-16T19:32:06.147877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:26.562009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146546+0000","last_change":"2024-04-16T17:48:27.296497+0000","last_active":"2024-04-16T19:32:01.146546+0000","last_peered":"2024-04-16T19:32:01.146546+0000","last_clean":"2024-04-16T19:32:01.146546+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:32:01.146546+0000","last_undegraded":"2024-04-16T19:32:01.146546+0000","last_fullsized":"2024-04-16T19:32:01.146546+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:59:05.786900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'8","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176076+0000","last_change":"2024-04-16T17:48:01.261918+0000","last_active":"2024-04-16T19:33:46.176076+0000","last_peered":"2024-04-16T19:33:46.176076+0000","last_clean":"2024-04-16T19:33:46.176076+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:33:46.176076+0000","last_undegraded":"2024-04-16T19:33:46.176076+0000","last_fullsized":"2024-04-16T19:33:46.176076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158533+0000","last_change":"2024-04-16T17:47:59.182311+0000","last_active":"2024-04-16T19:33:46.158533+0000","last_peered":"2024-04-16T19:33:46.158533+0000","last_clean":"2024-04-16T19:33:46.158533+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:33:46.158533+0000","last_undegraded":"2024-04-16T19:33:46.158533+0000","last_fullsized":"2024-04-16T19:33:46.158533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:15.733801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'897","reported_seq":3791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155083+0000","last_change":"2024-04-16T17:47:57.178953+0000","last_active":"2024-04-16T19:32:36.155083+0000","last_peered":"2024-04-16T19:32:36.155083+0000","last_clean":"2024-04-16T19:32:36.155083+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:32:36.155083+0000","last_undegraded":"2024-04-16T19:32:36.155083+0000","last_fullsized":"2024-04-16T19:32:36.155083+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:27.120398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'29","reported_seq":203,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.181200+0000","last_change":"2024-04-16T17:47:55.165769+0000","last_active":"2024-04-16T19:34:06.181200+0000","last_peered":"2024-04-16T19:34:06.181200+0000","last_clean":"2024-04-16T19:34:06.181200+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:34:06.181200+0000","last_undegraded":"2024-04-16T19:34:06.181200+0000","last_fullsized":"2024-04-16T19:34:06.181200+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175855+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:33:41.175855+0000","last_peered":"2024-04-16T19:33:41.175855+0000","last_clean":"2024-04-16T19:33:41.175855+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:33:41.175855+0000","last_undegraded":"2024-04-16T19:33:41.175855+0000","last_fullsized":"2024-04-16T19:33:41.175855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.159749+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:32:51.159749+0000","last_peered":"2024-04-16T19:32:51.159749+0000","last_clean":"2024-04-16T19:32:51.159749+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:32:51.159749+0000","last_undegraded":"2024-04-16T19:32:51.159749+0000","last_fullsized":"2024-04-16T19:32:51.159749+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169877+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:33:36.169877+0000","last_peered":"2024-04-16T19:33:36.169877+0000","last_clean":"2024-04-16T19:33:36.169877+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:33:36.169877+0000","last_undegraded":"2024-04-16T19:33:36.169877+0000","last_fullsized":"2024-04-16T19:33:36.169877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154521+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:32:21.154521+0000","last_peered":"2024-04-16T19:32:21.154521+0000","last_clean":"2024-04-16T19:32:21.154521+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:32:21.154521+0000","last_undegraded":"2024-04-16T19:32:21.154521+0000","last_fullsized":"2024-04-16T19:32:21.154521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:56.719793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'144","reported_seq":330,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177257+0000","last_change":"2024-04-16T17:47:35.314035+0000","last_active":"2024-04-16T19:33:46.177257+0000","last_peered":"2024-04-16T19:33:46.177257+0000","last_clean":"2024-04-16T19:33:46.177257+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:33:46.177257+0000","last_undegraded":"2024-04-16T19:33:46.177257+0000","last_fullsized":"2024-04-16T19:33:46.177257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181110+0000","last_change":"2024-04-16T17:47:37.334598+0000","last_active":"2024-04-16T19:33:51.181110+0000","last_peered":"2024-04-16T19:33:51.181110+0000","last_clean":"2024-04-16T19:33:51.181110+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:33:51.181110+0000","last_undegraded":"2024-04-16T19:33:51.181110+0000","last_fullsized":"2024-04-16T19:33:51.181110+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:24.856058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175255+0000","last_change":"2024-04-16T17:47:39.339775+0000","last_active":"2024-04-16T19:33:51.175255+0000","last_peered":"2024-04-16T19:33:51.175255+0000","last_clean":"2024-04-16T19:33:51.175255+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:33:51.175255+0000","last_undegraded":"2024-04-16T19:33:51.175255+0000","last_fullsized":"2024-04-16T19:33:51.175255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:25.066864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143941+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:31:51.143941+0000","last_peered":"2024-04-16T19:31:51.143941+0000","last_clean":"2024-04-16T19:31:51.143941+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:31:51.143941+0000","last_undegraded":"2024-04-16T19:31:51.143941+0000","last_fullsized":"2024-04-16T19:31:51.143941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:26.624319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.148088+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:32:06.148088+0000","last_peered":"2024-04-16T19:32:06.148088+0000","last_clean":"2024-04-16T19:32:06.148088+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:32:06.148088+0000","last_undegraded":"2024-04-16T19:32:06.148088+0000","last_fullsized":"2024-04-16T19:32:06.148088+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:33:17.617943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.146697+0000","last_change":"2024-04-16T17:48:27.307048+0000","last_active":"2024-04-16T19:32:01.146697+0000","last_peered":"2024-04-16T19:32:01.146697+0000","last_clean":"2024-04-16T19:32:01.146697+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:32:01.146697+0000","last_undegraded":"2024-04-16T19:32:01.146697+0000","last_fullsized":"2024-04-16T19:32:01.146697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:19.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'33","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152888+0000","last_change":"2024-04-16T17:48:01.250054+0000","last_active":"2024-04-16T19:32:06.152888+0000","last_peered":"2024-04-16T19:32:06.152888+0000","last_clean":"2024-04-16T19:32:06.152888+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:32:06.152888+0000","last_undegraded":"2024-04-16T19:32:06.152888+0000","last_fullsized":"2024-04-16T19:32:06.152888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133916+0000","last_change":"2024-04-16T17:47:59.190987+0000","last_active":"2024-04-16T19:32:01.133916+0000","last_peered":"2024-04-16T19:32:01.133916+0000","last_clean":"2024-04-16T19:32:01.133916+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:32:01.133916+0000","last_undegraded":"2024-04-16T19:32:01.133916+0000","last_fullsized":"2024-04-16T19:32:01.133916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:11.480554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'264","reported_seq":640,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151154+0000","last_change":"2024-04-16T17:47:57.189816+0000","last_active":"2024-04-16T19:32:01.151154+0000","last_peered":"2024-04-16T19:32:01.151154+0000","last_clean":"2024-04-16T19:32:01.151154+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:32:01.151154+0000","last_undegraded":"2024-04-16T19:32:01.151154+0000","last_fullsized":"2024-04-16T19:32:01.151154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:12.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'10","reported_seq":156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159575+0000","last_change":"2024-04-16T17:47:55.161898+0000","last_active":"2024-04-16T19:32:31.159575+0000","last_peered":"2024-04-16T19:32:31.159575+0000","last_clean":"2024-04-16T19:32:31.159575+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:32:31.159575+0000","last_undegraded":"2024-04-16T19:32:31.159575+0000","last_fullsized":"2024-04-16T19:32:31.159575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174563+0000","last_change":"2024-04-16T17:47:23.265711+0000","last_active":"2024-04-16T19:33:36.174563+0000","last_peered":"2024-04-16T19:33:36.174563+0000","last_clean":"2024-04-16T19:33:36.174563+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:33:36.174563+0000","last_undegraded":"2024-04-16T19:33:36.174563+0000","last_fullsized":"2024-04-16T19:33:36.174563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:08.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:51.158718+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:32:51.158718+0000","last_peered":"2024-04-16T19:32:51.158718+0000","last_clean":"2024-04-16T19:32:51.158718+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:32:51.158718+0000","last_undegraded":"2024-04-16T19:32:51.158718+0000","last_fullsized":"2024-04-16T19:32:51.158718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:58.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178254+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:33:51.178254+0000","last_peered":"2024-04-16T19:33:51.178254+0000","last_clean":"2024-04-16T19:33:51.178254+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:33:51.178254+0000","last_undegraded":"2024-04-16T19:33:51.178254+0000","last_fullsized":"2024-04-16T19:33:51.178254+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:49.542537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"48'4","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158694+0000","last_change":"2024-04-16T17:47:33.301081+0000","last_active":"2024-04-16T19:33:46.158694+0000","last_peered":"2024-04-16T19:33:46.158694+0000","last_clean":"2024-04-16T19:33:46.158694+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:33:46.158694+0000","last_undegraded":"2024-04-16T19:33:46.158694+0000","last_fullsized":"2024-04-16T19:33:46.158694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:05.482314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'151","reported_seq":1555,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.149286+0000","last_change":"2024-04-16T17:47:35.316795+0000","last_active":"2024-04-16T19:32:11.149286+0000","last_peered":"2024-04-16T19:32:11.149286+0000","last_clean":"2024-04-16T19:32:11.149286+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:32:11.149286+0000","last_undegraded":"2024-04-16T19:32:11.149286+0000","last_fullsized":"2024-04-16T19:32:11.149286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:25:54.202333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178652+0000","last_change":"2024-04-16T17:47:37.326201+0000","last_active":"2024-04-16T19:33:56.178652+0000","last_peered":"2024-04-16T19:33:56.178652+0000","last_clean":"2024-04-16T19:33:56.178652+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:33:56.178652+0000","last_undegraded":"2024-04-16T19:33:56.178652+0000","last_fullsized":"2024-04-16T19:33:56.178652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170822+0000","last_change":"2024-04-16T17:47:39.332609+0000","last_active":"2024-04-16T19:33:26.170822+0000","last_peered":"2024-04-16T19:33:26.170822+0000","last_clean":"2024-04-16T19:33:26.170822+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:33:26.170822+0000","last_undegraded":"2024-04-16T19:33:26.170822+0000","last_fullsized":"2024-04-16T19:33:26.170822+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143874+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:31:51.143874+0000","last_peered":"2024-04-16T19:31:51.143874+0000","last_clean":"2024-04-16T19:31:51.143874+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:31:51.143874+0000","last_undegraded":"2024-04-16T19:31:51.143874+0000","last_fullsized":"2024-04-16T19:31:51.143874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:45.404216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'76","reported_seq":1158,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150775+0000","last_change":"2024-04-16T17:48:34.318880+0000","last_active":"2024-04-16T19:32:06.150775+0000","last_peered":"2024-04-16T19:32:06.150775+0000","last_clean":"2024-04-16T19:32:06.150775+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:32:06.150775+0000","last_undegraded":"2024-04-16T19:32:06.150775+0000","last_fullsized":"2024-04-16T19:32:06.150775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:25.456330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153591+0000","last_change":"2024-04-16T17:48:27.299238+0000","last_active":"2024-04-16T19:32:26.153591+0000","last_peered":"2024-04-16T19:32:26.153591+0000","last_clean":"2024-04-16T19:32:26.153591+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:32:26.153591+0000","last_undegraded":"2024-04-16T19:32:26.153591+0000","last_fullsized":"2024-04-16T19:32:26.153591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:49.437897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'6","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.176200+0000","last_change":"2024-04-16T17:48:01.256444+0000","last_active":"2024-04-16T19:33:46.176200+0000","last_peered":"2024-04-16T19:33:46.176200+0000","last_clean":"2024-04-16T19:33:46.176200+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:33:46.176200+0000","last_undegraded":"2024-04-16T19:33:46.176200+0000","last_fullsized":"2024-04-16T19:33:46.176200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:34:59.810588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'555","reported_seq":1932,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133972+0000","last_change":"2024-04-16T17:47:59.183633+0000","last_active":"2024-04-16T19:32:01.133972+0000","last_peered":"2024-04-16T19:32:01.133972+0000","last_clean":"2024-04-16T19:32:01.133972+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:32:01.133972+0000","last_undegraded":"2024-04-16T19:32:01.133972+0000","last_fullsized":"2024-04-16T19:32:01.133972+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:14.201307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'306","reported_seq":623,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149338+0000","last_change":"2024-04-16T17:47:57.177185+0000","last_active":"2024-04-16T19:32:01.149338+0000","last_peered":"2024-04-16T19:32:01.149338+0000","last_clean":"2024-04-16T19:32:01.149338+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:32:01.149338+0000","last_undegraded":"2024-04-16T19:32:01.149338+0000","last_fullsized":"2024-04-16T19:32:01.149338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:18.465537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"48'5","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174804+0000","last_change":"2024-04-16T17:47:55.178154+0000","last_active":"2024-04-16T19:33:41.174804+0000","last_peered":"2024-04-16T19:33:41.174804+0000","last_clean":"2024-04-16T19:33:41.174804+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:33:41.174804+0000","last_undegraded":"2024-04-16T19:33:41.174804+0000","last_fullsized":"2024-04-16T19:33:41.174804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:36.060542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173663+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:33:36.173663+0000","last_peered":"2024-04-16T19:33:36.173663+0000","last_clean":"2024-04-16T19:33:36.173663+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:33:36.173663+0000","last_undegraded":"2024-04-16T19:33:36.173663+0000","last_fullsized":"2024-04-16T19:33:36.173663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164327+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:32:56.164327+0000","last_peered":"2024-04-16T19:32:56.164327+0000","last_clean":"2024-04-16T19:32:56.164327+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:32:56.164327+0000","last_undegraded":"2024-04-16T19:32:56.164327+0000","last_fullsized":"2024-04-16T19:32:56.164327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.171318+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:33:16.171318+0000","last_peered":"2024-04-16T19:33:16.171318+0000","last_clean":"2024-04-16T19:33:16.171318+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:33:16.171318+0000","last_undegraded":"2024-04-16T19:33:16.171318+0000","last_fullsized":"2024-04-16T19:33:16.171318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:05:07.906618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173252+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:33:36.173252+0000","last_peered":"2024-04-16T19:33:36.173252+0000","last_clean":"2024-04-16T19:33:36.173252+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:33:36.173252+0000","last_undegraded":"2024-04-16T19:33:36.173252+0000","last_fullsized":"2024-04-16T19:33:36.173252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:30.892245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":504,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174348+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:33:46.174348+0000","last_peered":"2024-04-16T19:33:46.174348+0000","last_clean":"2024-04-16T19:33:46.174348+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:33:46.174348+0000","last_undegraded":"2024-04-16T19:33:46.174348+0000","last_fullsized":"2024-04-16T19:33:46.174348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'177","reported_seq":1612,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178544+0000","last_change":"2024-04-16T17:47:35.312413+0000","last_active":"2024-04-16T19:33:51.178544+0000","last_peered":"2024-04-16T19:33:51.178544+0000","last_clean":"2024-04-16T19:33:51.178544+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:33:51.178544+0000","last_undegraded":"2024-04-16T19:33:51.178544+0000","last_fullsized":"2024-04-16T19:33:51.178544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152741+0000","last_change":"2024-04-16T17:47:37.327792+0000","last_active":"2024-04-16T19:32:26.152741+0000","last_peered":"2024-04-16T19:32:26.152741+0000","last_clean":"2024-04-16T19:32:26.152741+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:32:26.152741+0000","last_undegraded":"2024-04-16T19:32:26.152741+0000","last_fullsized":"2024-04-16T19:32:26.152741+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:43.802094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.152938+0000","last_change":"2024-04-16T17:47:39.700262+0000","last_active":"2024-04-16T19:32:11.152938+0000","last_peered":"2024-04-16T19:32:11.152938+0000","last_clean":"2024-04-16T19:32:11.152938+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:32:11.152938+0000","last_undegraded":"2024-04-16T19:32:11.152938+0000","last_fullsized":"2024-04-16T19:32:11.152938+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:26.867823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157247+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:33:41.157247+0000","last_peered":"2024-04-16T19:33:41.157247+0000","last_clean":"2024-04-16T19:33:41.157247+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:33:41.157247+0000","last_undegraded":"2024-04-16T19:33:41.157247+0000","last_fullsized":"2024-04-16T19:33:41.157247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.324013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.150723+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:32:06.150723+0000","last_peered":"2024-04-16T19:32:06.150723+0000","last_clean":"2024-04-16T19:32:06.150723+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:32:06.150723+0000","last_undegraded":"2024-04-16T19:32:06.150723+0000","last_fullsized":"2024-04-16T19:32:06.150723+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:02.642376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140845+0000","last_change":"2024-04-16T17:48:27.304274+0000","last_active":"2024-04-16T19:32:31.140845+0000","last_peered":"2024-04-16T19:32:31.140845+0000","last_clean":"2024-04-16T19:32:31.140845+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:32:31.140845+0000","last_undegraded":"2024-04-16T19:32:31.140845+0000","last_fullsized":"2024-04-16T19:32:31.140845+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:11:19.221992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'14","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133996+0000","last_change":"2024-04-16T17:48:01.251987+0000","last_active":"2024-04-16T19:32:01.133996+0000","last_peered":"2024-04-16T19:32:01.133996+0000","last_clean":"2024-04-16T19:32:01.133996+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:32:01.133996+0000","last_undegraded":"2024-04-16T19:32:01.133996+0000","last_fullsized":"2024-04-16T19:32:01.133996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:53.985409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'555","reported_seq":1923,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172522+0000","last_change":"2024-04-16T17:47:59.188607+0000","last_active":"2024-04-16T19:33:46.172522+0000","last_peered":"2024-04-16T19:33:46.172522+0000","last_clean":"2024-04-16T19:33:46.172522+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:33:46.172522+0000","last_undegraded":"2024-04-16T19:33:46.172522+0000","last_fullsized":"2024-04-16T19:33:46.172522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:09.833400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'514","reported_seq":1190,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152969+0000","last_change":"2024-04-16T17:47:57.187984+0000","last_active":"2024-04-16T19:32:06.152969+0000","last_peered":"2024-04-16T19:32:06.152969+0000","last_clean":"2024-04-16T19:32:06.152969+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:32:06.152969+0000","last_undegraded":"2024-04-16T19:32:06.152969+0000","last_fullsized":"2024-04-16T19:32:06.152969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:09.942121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"51'3","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157358+0000","last_change":"2024-04-16T17:47:55.173104+0000","last_active":"2024-04-16T19:33:41.157358+0000","last_peered":"2024-04-16T19:33:41.157358+0000","last_clean":"2024-04-16T19:33:41.157358+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:33:41.157358+0000","last_undegraded":"2024-04-16T19:33:41.157358+0000","last_fullsized":"2024-04-16T19:33:41.157358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:46.009020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.170986+0000","last_change":"2024-04-16T17:47:23.266373+0000","last_active":"2024-04-16T19:33:41.170986+0000","last_peered":"2024-04-16T19:33:41.170986+0000","last_clean":"2024-04-16T19:33:41.170986+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:33:41.170986+0000","last_undegraded":"2024-04-16T19:33:41.170986+0000","last_fullsized":"2024-04-16T19:33:41.170986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:10:11.306509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158724+0000","last_change":"2024-04-16T17:47:32.273580+0000","last_active":"2024-04-16T19:33:46.158724+0000","last_peered":"2024-04-16T19:33:46.158724+0000","last_clean":"2024-04-16T19:33:46.158724+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:33:46.158724+0000","last_undegraded":"2024-04-16T19:33:46.158724+0000","last_fullsized":"2024-04-16T19:33:46.158724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:28.727683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.164407+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:32:56.164407+0000","last_peered":"2024-04-16T19:32:56.164407+0000","last_clean":"2024-04-16T19:32:56.164407+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:32:56.164407+0000","last_undegraded":"2024-04-16T19:32:56.164407+0000","last_fullsized":"2024-04-16T19:32:56.164407+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163230+0000","last_change":"2024-04-16T17:47:33.300039+0000","last_active":"2024-04-16T19:34:06.163230+0000","last_peered":"2024-04-16T19:34:06.163230+0000","last_clean":"2024-04-16T19:34:06.163230+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:34:06.163230+0000","last_undegraded":"2024-04-16T19:34:06.163230+0000","last_fullsized":"2024-04-16T19:34:06.163230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:42.946001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'144","reported_seq":347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.173676+0000","last_change":"2024-04-16T17:47:35.305470+0000","last_active":"2024-04-16T19:33:51.173676+0000","last_peered":"2024-04-16T19:33:51.173676+0000","last_clean":"2024-04-16T19:33:51.173676+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:33:51.173676+0000","last_undegraded":"2024-04-16T19:33:51.173676+0000","last_fullsized":"2024-04-16T19:33:51.173676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:00.096888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.161004+0000","last_change":"2024-04-16T17:47:37.334307+0000","last_active":"2024-04-16T19:33:56.161004+0000","last_peered":"2024-04-16T19:33:56.161004+0000","last_clean":"2024-04-16T19:33:56.161004+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:33:56.161004+0000","last_undegraded":"2024-04-16T19:33:56.161004+0000","last_fullsized":"2024-04-16T19:33:56.161004+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:58.517922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168901+0000","last_change":"2024-04-16T17:47:39.343779+0000","last_active":"2024-04-16T19:33:26.168901+0000","last_peered":"2024-04-16T19:33:26.168901+0000","last_clean":"2024-04-16T19:33:26.168901+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:33:26.168901+0000","last_undegraded":"2024-04-16T19:33:26.168901+0000","last_fullsized":"2024-04-16T19:33:26.168901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:43:28.188142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.131712+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:31:51.131712+0000","last_peered":"2024-04-16T19:31:51.131712+0000","last_clean":"2024-04-16T19:31:51.131712+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:31:51.131712+0000","last_undegraded":"2024-04-16T19:31:51.131712+0000","last_fullsized":"2024-04-16T19:31:51.131712+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:49:14.730835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'122","reported_seq":1763,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170106+0000","last_change":"2024-04-16T17:48:34.337192+0000","last_active":"2024-04-16T19:33:11.170106+0000","last_peered":"2024-04-16T19:33:11.170106+0000","last_clean":"2024-04-16T19:33:11.170106+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:33:11.170106+0000","last_undegraded":"2024-04-16T19:33:11.170106+0000","last_fullsized":"2024-04-16T19:33:11.170106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.154528+0000","last_change":"2024-04-16T17:48:27.301617+0000","last_active":"2024-04-16T19:32:31.154528+0000","last_peered":"2024-04-16T19:32:31.154528+0000","last_clean":"2024-04-16T19:32:31.154528+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:32:31.154528+0000","last_undegraded":"2024-04-16T19:32:31.154528+0000","last_fullsized":"2024-04-16T19:32:31.154528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:43.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'16","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.133844+0000","last_change":"2024-04-16T17:48:01.234685+0000","last_active":"2024-04-16T19:32:01.133844+0000","last_peered":"2024-04-16T19:32:01.133844+0000","last_clean":"2024-04-16T19:32:01.133844+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:32:01.133844+0000","last_undegraded":"2024-04-16T19:32:01.133844+0000","last_fullsized":"2024-04-16T19:32:01.133844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:02.931474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.152930+0000","last_change":"2024-04-16T17:47:59.185616+0000","last_active":"2024-04-16T19:32:06.152930+0000","last_peered":"2024-04-16T19:32:06.152930+0000","last_clean":"2024-04-16T19:32:06.152930+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:32:06.152930+0000","last_undegraded":"2024-04-16T19:32:06.152930+0000","last_fullsized":"2024-04-16T19:32:06.152930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:24:43.650159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1159","reported_seq":1411,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172544+0000","last_change":"2024-04-16T17:47:57.188806+0000","last_active":"2024-04-16T19:33:46.172544+0000","last_peered":"2024-04-16T19:33:46.172544+0000","last_clean":"2024-04-16T19:33:46.172544+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:33:46.172544+0000","last_undegraded":"2024-04-16T19:33:46.172544+0000","last_fullsized":"2024-04-16T19:33:46.172544+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1159,"log_dups_size":0,"ondisk_log_size":1159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:00.199310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"48'18","reported_seq":399,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177873+0000","last_change":"2024-04-16T17:47:55.164040+0000","last_active":"2024-04-16T19:33:41.177873+0000","last_peered":"2024-04-16T19:33:41.177873+0000","last_clean":"2024-04-16T19:33:41.177873+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:33:41.177873+0000","last_undegraded":"2024-04-16T19:33:41.177873+0000","last_fullsized":"2024-04-16T19:33:41.177873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:47.088292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.179804+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:34:01.179804+0000","last_peered":"2024-04-16T19:34:01.179804+0000","last_clean":"2024-04-16T19:34:01.179804+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:34:01.179804+0000","last_undegraded":"2024-04-16T19:34:01.179804+0000","last_fullsized":"2024-04-16T19:34:01.179804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:16.425065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154908+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:33:31.154908+0000","last_peered":"2024-04-16T19:33:31.154908+0000","last_clean":"2024-04-16T19:33:31.154908+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:33:31.154908+0000","last_undegraded":"2024-04-16T19:33:31.154908+0000","last_fullsized":"2024-04-16T19:33:31.154908+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:55:03.538702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168590+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:33:31.168590+0000","last_peered":"2024-04-16T19:33:31.168590+0000","last_clean":"2024-04-16T19:33:31.168590+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:33:31.168590+0000","last_undegraded":"2024-04-16T19:33:31.168590+0000","last_fullsized":"2024-04-16T19:33:31.168590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180686+0000","last_change":"2024-04-16T17:47:33.300948+0000","last_active":"2024-04-16T19:33:51.180686+0000","last_peered":"2024-04-16T19:33:51.180686+0000","last_clean":"2024-04-16T19:33:51.180686+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:33:51.180686+0000","last_undegraded":"2024-04-16T19:33:51.180686+0000","last_fullsized":"2024-04-16T19:33:51.180686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:39.765977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'120","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.180791+0000","last_change":"2024-04-16T17:47:35.313794+0000","last_active":"2024-04-16T19:33:51.180791+0000","last_peered":"2024-04-16T19:33:51.180791+0000","last_clean":"2024-04-16T19:33:51.180791+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:33:51.180791+0000","last_undegraded":"2024-04-16T19:33:51.180791+0000","last_fullsized":"2024-04-16T19:33:51.180791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178567+0000","last_change":"2024-04-16T17:47:37.329509+0000","last_active":"2024-04-16T19:33:56.178567+0000","last_peered":"2024-04-16T19:33:56.178567+0000","last_clean":"2024-04-16T19:33:56.178567+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:33:56.178567+0000","last_undegraded":"2024-04-16T19:33:56.178567+0000","last_fullsized":"2024-04-16T19:33:56.178567+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170908+0000","last_change":"2024-04-16T17:47:39.697737+0000","last_active":"2024-04-16T19:33:26.170908+0000","last_peered":"2024-04-16T19:33:26.170908+0000","last_clean":"2024-04-16T19:33:26.170908+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:33:26.170908+0000","last_undegraded":"2024-04-16T19:33:26.170908+0000","last_fullsized":"2024-04-16T19:33:26.170908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.150046+0000","last_change":"2024-04-16T18:10:50.537157+0000","last_active":"2024-04-16T19:33:11.150046+0000","last_peered":"2024-04-16T19:33:11.150046+0000","last_clean":"2024-04-16T19:33:11.150046+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:33:11.150046+0000","last_undegraded":"2024-04-16T19:33:11.150046+0000","last_fullsized":"2024-04-16T19:33:11.150046+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:13.931584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1282,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156075+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:32:36.156075+0000","last_peered":"2024-04-16T19:32:36.156075+0000","last_clean":"2024-04-16T19:32:36.156075+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:32:36.156075+0000","last_undegraded":"2024-04-16T19:32:36.156075+0000","last_fullsized":"2024-04-16T19:32:36.156075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:20.310434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147837+0000","last_change":"2024-04-16T17:48:27.305636+0000","last_active":"2024-04-16T19:33:01.147837+0000","last_peered":"2024-04-16T19:33:01.147837+0000","last_clean":"2024-04-16T19:33:01.147837+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:33:01.147837+0000","last_undegraded":"2024-04-16T19:33:01.147837+0000","last_fullsized":"2024-04-16T19:33:01.147837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:06.425749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'13","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.175996+0000","last_change":"2024-04-16T17:48:01.266136+0000","last_active":"2024-04-16T19:33:46.175996+0000","last_peered":"2024-04-16T19:33:46.175996+0000","last_clean":"2024-04-16T19:33:46.175996+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:33:46.175996+0000","last_undegraded":"2024-04-16T19:33:46.175996+0000","last_fullsized":"2024-04-16T19:33:46.175996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:30.757791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177121+0000","last_change":"2024-04-16T17:47:59.188856+0000","last_active":"2024-04-16T19:33:46.177121+0000","last_peered":"2024-04-16T19:33:46.177121+0000","last_clean":"2024-04-16T19:33:46.177121+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:33:46.177121+0000","last_undegraded":"2024-04-16T19:33:46.177121+0000","last_fullsized":"2024-04-16T19:33:46.177121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:35.433741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1314","reported_seq":1867,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172479+0000","last_change":"2024-04-16T17:47:57.192362+0000","last_active":"2024-04-16T19:33:46.172479+0000","last_peered":"2024-04-16T19:33:46.172479+0000","last_clean":"2024-04-16T19:33:46.172479+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:33:46.172479+0000","last_undegraded":"2024-04-16T19:33:46.172479+0000","last_fullsized":"2024-04-16T19:33:46.172479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1314,"log_dups_size":0,"ondisk_log_size":1314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:14.446218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'4","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178048+0000","last_change":"2024-04-16T17:47:55.156911+0000","last_active":"2024-04-16T19:33:41.178048+0000","last_peered":"2024-04-16T19:33:41.178048+0000","last_clean":"2024-04-16T19:33:41.178048+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:33:41.178048+0000","last_undegraded":"2024-04-16T19:33:41.178048+0000","last_fullsized":"2024-04-16T19:33:41.178048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174726+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:33:41.174726+0000","last_peered":"2024-04-16T19:33:41.174726+0000","last_clean":"2024-04-16T19:33:41.174726+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:33:41.174726+0000","last_undegraded":"2024-04-16T19:33:41.174726+0000","last_fullsized":"2024-04-16T19:33:41.174726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:57.103463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174314+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:33:36.174314+0000","last_peered":"2024-04-16T19:33:36.174314+0000","last_clean":"2024-04-16T19:33:36.174314+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:33:36.174314+0000","last_undegraded":"2024-04-16T19:33:36.174314+0000","last_fullsized":"2024-04-16T19:33:36.174314+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:22.291956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.163913+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:32:46.163913+0000","last_peered":"2024-04-16T19:32:46.163913+0000","last_clean":"2024-04-16T19:32:46.163913+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:32:46.163913+0000","last_undegraded":"2024-04-16T19:32:46.163913+0000","last_fullsized":"2024-04-16T19:32:46.163913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:16:55.853307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.175384+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:33:51.175384+0000","last_peered":"2024-04-16T19:33:51.175384+0000","last_clean":"2024-04-16T19:33:51.175384+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:33:51.175384+0000","last_undegraded":"2024-04-16T19:33:51.175384+0000","last_fullsized":"2024-04-16T19:33:51.175384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'168","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.178373+0000","last_change":"2024-04-16T17:47:35.321995+0000","last_active":"2024-04-16T19:33:51.178373+0000","last_peered":"2024-04-16T19:33:51.178373+0000","last_clean":"2024-04-16T19:33:51.178373+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:33:51.178373+0000","last_undegraded":"2024-04-16T19:33:51.178373+0000","last_fullsized":"2024-04-16T19:33:51.178373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:19:17.922789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'12","reported_seq":1342,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179297+0000","last_change":"2024-04-16T17:47:37.323586+0000","last_active":"2024-04-16T19:34:06.179297+0000","last_peered":"2024-04-16T19:34:06.179297+0000","last_clean":"2024-04-16T19:34:06.179297+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:34:06.179297+0000","last_undegraded":"2024-04-16T19:34:06.179297+0000","last_fullsized":"2024-04-16T19:34:06.179297+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:30.516612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.180278+0000","last_change":"2024-04-16T17:47:39.698152+0000","last_active":"2024-04-16T19:34:01.180278+0000","last_peered":"2024-04-16T19:34:01.180278+0000","last_clean":"2024-04-16T19:34:01.180278+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:34:01.180278+0000","last_undegraded":"2024-04-16T19:34:01.180278+0000","last_fullsized":"2024-04-16T19:34:01.180278+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:08.010344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.164914+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:33:11.164914+0000","last_peered":"2024-04-16T19:33:11.164914+0000","last_clean":"2024-04-16T19:33:11.164914+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:33:11.164914+0000","last_undegraded":"2024-04-16T19:33:11.164914+0000","last_fullsized":"2024-04-16T19:33:11.164914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:36:23.318927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158153+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:32:36.158153+0000","last_peered":"2024-04-16T19:32:36.158153+0000","last_clean":"2024-04-16T19:32:36.158153+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:32:36.158153+0000","last_undegraded":"2024-04-16T19:32:36.158153+0000","last_fullsized":"2024-04-16T19:32:36.158153+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:51.468265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151256+0000","last_change":"2024-04-16T17:48:27.298996+0000","last_active":"2024-04-16T19:32:01.151256+0000","last_peered":"2024-04-16T19:32:01.151256+0000","last_clean":"2024-04-16T19:32:01.151256+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:32:01.151256+0000","last_undegraded":"2024-04-16T19:32:01.151256+0000","last_fullsized":"2024-04-16T19:32:01.151256+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"51'4","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140996+0000","last_change":"2024-04-16T17:48:01.252079+0000","last_active":"2024-04-16T19:32:31.140996+0000","last_peered":"2024-04-16T19:32:31.140996+0000","last_clean":"2024-04-16T19:32:31.140996+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:32:31.140996+0000","last_undegraded":"2024-04-16T19:32:31.140996+0000","last_fullsized":"2024-04-16T19:32:31.140996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:34:57.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'555","reported_seq":1994,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158653+0000","last_change":"2024-04-16T17:47:59.197276+0000","last_active":"2024-04-16T19:33:46.158653+0000","last_peered":"2024-04-16T19:33:46.158653+0000","last_clean":"2024-04-16T19:33:46.158653+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:33:46.158653+0000","last_undegraded":"2024-04-16T19:33:46.158653+0000","last_fullsized":"2024-04-16T19:33:46.158653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:35:16.461108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'553","reported_seq":822,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.172334+0000","last_change":"2024-04-16T17:47:57.191520+0000","last_active":"2024-04-16T19:33:46.172334+0000","last_peered":"2024-04-16T19:33:46.172334+0000","last_clean":"2024-04-16T19:33:46.172334+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:33:46.172334+0000","last_undegraded":"2024-04-16T19:33:46.172334+0000","last_fullsized":"2024-04-16T19:33:46.172334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:25.845115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'14","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144097+0000","last_change":"2024-04-16T17:47:55.178734+0000","last_active":"2024-04-16T19:31:51.144097+0000","last_peered":"2024-04-16T19:31:51.144097+0000","last_clean":"2024-04-16T19:31:51.144097+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:31:51.144097+0000","last_undegraded":"2024-04-16T19:31:51.144097+0000","last_fullsized":"2024-04-16T19:31:51.144097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:57:35.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175692+0000","last_change":"2024-04-16T17:47:23.265822+0000","last_active":"2024-04-16T19:33:41.175692+0000","last_peered":"2024-04-16T19:33:41.175692+0000","last_clean":"2024-04-16T19:33:41.175692+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:33:41.175692+0000","last_undegraded":"2024-04-16T19:33:41.175692+0000","last_fullsized":"2024-04-16T19:33:41.175692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.178216+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:34:01.178216+0000","last_peered":"2024-04-16T19:34:01.178216+0000","last_clean":"2024-04-16T19:34:01.178216+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:34:01.178216+0000","last_undegraded":"2024-04-16T19:34:01.178216+0000","last_fullsized":"2024-04-16T19:34:01.178216+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:46.158816+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:32:46.158816+0000","last_peered":"2024-04-16T19:32:46.158816+0000","last_clean":"2024-04-16T19:32:46.158816+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:32:46.158816+0000","last_undegraded":"2024-04-16T19:32:46.158816+0000","last_fullsized":"2024-04-16T19:32:46.158816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:00.243951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.177291+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:33:51.177291+0000","last_peered":"2024-04-16T19:33:51.177291+0000","last_clean":"2024-04-16T19:33:51.177291+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:33:51.177291+0000","last_undegraded":"2024-04-16T19:33:51.177291+0000","last_fullsized":"2024-04-16T19:33:51.177291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:34:47.481180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'160","reported_seq":368,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182468+0000","last_change":"2024-04-16T17:47:35.312410+0000","last_active":"2024-04-16T19:34:11.182468+0000","last_peered":"2024-04-16T19:34:11.182468+0000","last_clean":"2024-04-16T19:34:11.182468+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:34:11.182468+0000","last_undegraded":"2024-04-16T19:34:11.182468+0000","last_fullsized":"2024-04-16T19:34:11.182468+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:01.054193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.174883+0000","last_change":"2024-04-16T17:47:37.322938+0000","last_active":"2024-04-16T19:33:56.174883+0000","last_peered":"2024-04-16T19:33:56.174883+0000","last_clean":"2024-04-16T19:33:56.174883+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:33:56.174883+0000","last_undegraded":"2024-04-16T19:33:56.174883+0000","last_fullsized":"2024-04-16T19:33:56.174883+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:33.277719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.170994+0000","last_change":"2024-04-16T17:47:39.699806+0000","last_active":"2024-04-16T19:33:26.170994+0000","last_peered":"2024-04-16T19:33:26.170994+0000","last_clean":"2024-04-16T19:33:26.170994+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:33:26.170994+0000","last_undegraded":"2024-04-16T19:33:26.170994+0000","last_fullsized":"2024-04-16T19:33:26.170994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164695+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:33:01.164695+0000","last_peered":"2024-04-16T19:33:01.164695+0000","last_clean":"2024-04-16T19:33:01.164695+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:33:01.164695+0000","last_undegraded":"2024-04-16T19:33:01.164695+0000","last_fullsized":"2024-04-16T19:33:01.164695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1098,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168822+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:33:06.168822+0000","last_peered":"2024-04-16T19:33:06.168822+0000","last_clean":"2024-04-16T19:33:06.168822+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:33:06.168822+0000","last_undegraded":"2024-04-16T19:33:06.168822+0000","last_fullsized":"2024-04-16T19:33:06.168822+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:38.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172871+0000","last_change":"2024-04-16T17:48:27.307428+0000","last_active":"2024-04-16T19:33:31.172871+0000","last_peered":"2024-04-16T19:33:31.172871+0000","last_clean":"2024-04-16T19:33:31.172871+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:33:31.172871+0000","last_undegraded":"2024-04-16T19:33:31.172871+0000","last_fullsized":"2024-04-16T19:33:31.172871+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:33.963133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161215+0000","last_change":"2024-04-16T17:48:01.240690+0000","last_active":"2024-04-16T19:33:01.161215+0000","last_peered":"2024-04-16T19:33:01.161215+0000","last_clean":"2024-04-16T19:33:01.161215+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:33:01.161215+0000","last_undegraded":"2024-04-16T19:33:01.161215+0000","last_fullsized":"2024-04-16T19:33:01.161215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:42.135971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162598+0000","last_change":"2024-04-16T17:47:59.192669+0000","last_active":"2024-04-16T19:33:01.162598+0000","last_peered":"2024-04-16T19:33:01.162598+0000","last_clean":"2024-04-16T19:33:01.162598+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:33:01.162598+0000","last_undegraded":"2024-04-16T19:33:01.162598+0000","last_fullsized":"2024-04-16T19:33:01.162598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:34.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'850","reported_seq":1151,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161341+0000","last_change":"2024-04-16T17:47:57.193698+0000","last_active":"2024-04-16T19:33:01.161341+0000","last_peered":"2024-04-16T19:33:01.161341+0000","last_clean":"2024-04-16T19:33:01.161341+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:33:01.161341+0000","last_undegraded":"2024-04-16T19:33:01.161341+0000","last_fullsized":"2024-04-16T19:33:01.161341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:15.692887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'16","reported_seq":234,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.178756+0000","last_change":"2024-04-16T17:49:32.398153+0000","last_active":"2024-04-16T19:34:11.178756+0000","last_peered":"2024-04-16T19:34:11.178756+0000","last_clean":"2024-04-16T19:34:11.178756+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:34:11.178756+0000","last_undegraded":"2024-04-16T19:34:11.178756+0000","last_fullsized":"2024-04-16T19:34:11.178756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:45.378036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.178826+0000","last_change":"2024-04-16T17:47:39.336834+0000","last_active":"2024-04-16T19:33:56.178826+0000","last_peered":"2024-04-16T19:33:56.178826+0000","last_clean":"2024-04-16T19:33:56.178826+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:33:56.178826+0000","last_undegraded":"2024-04-16T19:33:56.178826+0000","last_fullsized":"2024-04-16T19:33:56.178826+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:56:09.285627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152606+0000","last_change":"2024-04-16T17:47:23.257996+0000","last_active":"2024-04-16T19:32:26.152606+0000","last_peered":"2024-04-16T19:32:26.152606+0000","last_clean":"2024-04-16T19:32:26.152606+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:32:26.152606+0000","last_undegraded":"2024-04-16T19:32:26.152606+0000","last_fullsized":"2024-04-16T19:32:26.152606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166359+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:32:56.166359+0000","last_peered":"2024-04-16T19:32:56.166359+0000","last_clean":"2024-04-16T19:32:56.166359+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:32:56.166359+0000","last_undegraded":"2024-04-16T19:32:56.166359+0000","last_fullsized":"2024-04-16T19:32:56.166359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:07.759176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"48'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159071+0000","last_change":"2024-04-16T17:47:33.297781+0000","last_active":"2024-04-16T19:32:36.159071+0000","last_peered":"2024-04-16T19:32:36.159071+0000","last_clean":"2024-04-16T19:32:36.159071+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:32:36.159071+0000","last_undegraded":"2024-04-16T19:32:36.159071+0000","last_fullsized":"2024-04-16T19:32:36.159071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:51.904305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142052+0000","last_change":"2024-04-16T17:47:35.310182+0000","last_active":"2024-04-16T19:32:36.142052+0000","last_peered":"2024-04-16T19:32:36.142052+0000","last_clean":"2024-04-16T19:32:36.142052+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:32:36.142052+0000","last_undegraded":"2024-04-16T19:32:36.142052+0000","last_fullsized":"2024-04-16T19:32:36.142052+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:57:05.362295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160301+0000","last_change":"2024-04-16T17:47:37.334854+0000","last_active":"2024-04-16T19:32:41.160301+0000","last_peered":"2024-04-16T19:32:41.160301+0000","last_clean":"2024-04-16T19:32:41.160301+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:32:41.160301+0000","last_undegraded":"2024-04-16T19:32:41.160301+0000","last_fullsized":"2024-04-16T19:32:41.160301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:24.862379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168290+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:33:31.168290+0000","last_peered":"2024-04-16T19:33:31.168290+0000","last_clean":"2024-04-16T19:33:31.168290+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:33:31.168290+0000","last_undegraded":"2024-04-16T19:33:31.168290+0000","last_fullsized":"2024-04-16T19:33:31.168290+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:11.558629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'55","reported_seq":671,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157685+0000","last_change":"2024-04-16T17:48:34.343212+0000","last_active":"2024-04-16T19:32:31.157685+0000","last_peered":"2024-04-16T19:32:31.157685+0000","last_clean":"2024-04-16T19:32:31.157685+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:32:31.157685+0000","last_undegraded":"2024-04-16T19:32:31.157685+0000","last_fullsized":"2024-04-16T19:32:31.157685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:28:18.790386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168495+0000","last_change":"2024-04-16T17:48:27.301479+0000","last_active":"2024-04-16T19:33:31.168495+0000","last_peered":"2024-04-16T19:33:31.168495+0000","last_clean":"2024-04-16T19:33:31.168495+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:33:31.168495+0000","last_undegraded":"2024-04-16T19:33:31.168495+0000","last_fullsized":"2024-04-16T19:33:31.168495+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:04.776813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'9","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161137+0000","last_change":"2024-04-16T17:48:01.250254+0000","last_active":"2024-04-16T19:33:01.161137+0000","last_peered":"2024-04-16T19:33:01.161137+0000","last_clean":"2024-04-16T19:33:01.161137+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:33:01.161137+0000","last_undegraded":"2024-04-16T19:33:01.161137+0000","last_fullsized":"2024-04-16T19:33:01.161137+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:53.163721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167775+0000","last_change":"2024-04-16T17:47:59.176369+0000","last_active":"2024-04-16T19:33:01.167775+0000","last_peered":"2024-04-16T19:33:01.167775+0000","last_clean":"2024-04-16T19:33:01.167775+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:33:01.167775+0000","last_undegraded":"2024-04-16T19:33:01.167775+0000","last_fullsized":"2024-04-16T19:33:01.167775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:43:52.164568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'592","reported_seq":2279,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162562+0000","last_change":"2024-04-16T17:47:57.185970+0000","last_active":"2024-04-16T19:33:01.162562+0000","last_peered":"2024-04-16T19:33:01.162562+0000","last_clean":"2024-04-16T19:33:01.162562+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:33:01.162562+0000","last_undegraded":"2024-04-16T19:33:01.162562+0000","last_fullsized":"2024-04-16T19:33:01.162562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:08:33.418851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"51'2","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183958+0000","last_change":"2024-04-16T17:47:55.163430+0000","last_active":"2024-04-16T19:34:11.183958+0000","last_peered":"2024-04-16T19:34:11.183958+0000","last_clean":"2024-04-16T19:34:11.183958+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:34:11.183958+0000","last_undegraded":"2024-04-16T19:34:11.183958+0000","last_fullsized":"2024-04-16T19:34:11.183958+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183840+0000","last_change":"2024-04-16T17:47:39.338758+0000","last_active":"2024-04-16T19:34:11.183840+0000","last_peered":"2024-04-16T19:34:11.183840+0000","last_clean":"2024-04-16T19:34:11.183840+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:34:11.183840+0000","last_undegraded":"2024-04-16T19:34:11.183840+0000","last_fullsized":"2024-04-16T19:34:11.183840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:46:37.213833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156609+0000","last_change":"2024-04-16T17:47:23.260667+0000","last_active":"2024-04-16T19:32:26.156609+0000","last_peered":"2024-04-16T19:32:26.156609+0000","last_clean":"2024-04-16T19:32:26.156609+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:32:26.156609+0000","last_undegraded":"2024-04-16T19:32:26.156609+0000","last_fullsized":"2024-04-16T19:32:26.156609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177933+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:33:41.177933+0000","last_peered":"2024-04-16T19:33:41.177933+0000","last_clean":"2024-04-16T19:33:41.177933+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:33:41.177933+0000","last_undegraded":"2024-04-16T19:33:41.177933+0000","last_fullsized":"2024-04-16T19:33:41.177933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"48'1","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159182+0000","last_change":"2024-04-16T17:47:33.297748+0000","last_active":"2024-04-16T19:32:36.159182+0000","last_peered":"2024-04-16T19:32:36.159182+0000","last_clean":"2024-04-16T19:32:36.159182+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:32:36.159182+0000","last_undegraded":"2024-04-16T19:32:36.159182+0000","last_fullsized":"2024-04-16T19:32:36.159182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:36:58.076852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'50","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158106+0000","last_change":"2024-04-16T17:47:35.314682+0000","last_active":"2024-04-16T19:32:36.158106+0000","last_peered":"2024-04-16T19:32:36.158106+0000","last_clean":"2024-04-16T19:32:36.158106+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:32:36.158106+0000","last_undegraded":"2024-04-16T19:32:36.158106+0000","last_fullsized":"2024-04-16T19:32:36.158106+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:40:57.716581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162296+0000","last_change":"2024-04-16T17:47:37.327586+0000","last_active":"2024-04-16T19:32:41.162296+0000","last_peered":"2024-04-16T19:32:41.162296+0000","last_clean":"2024-04-16T19:32:41.162296+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:32:41.162296+0000","last_undegraded":"2024-04-16T19:32:41.162296+0000","last_fullsized":"2024-04-16T19:32:41.162296+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:11:54.910620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.167526+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:33:21.167526+0000","last_peered":"2024-04-16T19:33:21.167526+0000","last_clean":"2024-04-16T19:33:21.167526+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:33:21.167526+0000","last_undegraded":"2024-04-16T19:33:21.167526+0000","last_fullsized":"2024-04-16T19:33:21.167526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:07.939676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'99","reported_seq":1687,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174223+0000","last_change":"2024-04-16T17:48:34.338854+0000","last_active":"2024-04-16T19:33:36.174223+0000","last_peered":"2024-04-16T19:33:36.174223+0000","last_clean":"2024-04-16T19:33:36.174223+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:33:36.174223+0000","last_undegraded":"2024-04-16T19:33:36.174223+0000","last_fullsized":"2024-04-16T19:33:36.174223+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:41.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153090+0000","last_change":"2024-04-16T17:48:27.301694+0000","last_active":"2024-04-16T19:32:26.153090+0000","last_peered":"2024-04-16T19:32:26.153090+0000","last_clean":"2024-04-16T19:32:26.153090+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:32:26.153090+0000","last_undegraded":"2024-04-16T19:32:26.153090+0000","last_fullsized":"2024-04-16T19:32:26.153090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'13","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.156916+0000","last_change":"2024-04-16T17:48:01.248700+0000","last_active":"2024-04-16T19:32:31.156916+0000","last_peered":"2024-04-16T19:32:31.156916+0000","last_clean":"2024-04-16T19:32:31.156916+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:32:31.156916+0000","last_undegraded":"2024-04-16T19:32:31.156916+0000","last_fullsized":"2024-04-16T19:32:31.156916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:27.634774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'20","reported_seq":1421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153482+0000","last_change":"2024-04-16T17:47:59.196995+0000","last_active":"2024-04-16T19:32:26.153482+0000","last_peered":"2024-04-16T19:32:26.153482+0000","last_clean":"2024-04-16T19:32:26.153482+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:32:26.153482+0000","last_undegraded":"2024-04-16T19:32:26.153482+0000","last_fullsized":"2024-04-16T19:32:26.153482+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:58:37.685479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'6983","reported_seq":10557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154510+0000","last_change":"2024-04-16T17:47:57.195602+0000","last_active":"2024-04-16T19:32:11.154510+0000","last_peered":"2024-04-16T19:32:11.154510+0000","last_clean":"2024-04-16T19:32:11.154510+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:32:11.154510+0000","last_undegraded":"2024-04-16T19:32:11.154510+0000","last_fullsized":"2024-04-16T19:32:11.154510+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1783,"log_dups_size":3000,"ondisk_log_size":1783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:48:37.455011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'6","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163653+0000","last_change":"2024-04-16T17:47:55.162215+0000","last_active":"2024-04-16T19:32:56.163653+0000","last_peered":"2024-04-16T19:32:56.163653+0000","last_clean":"2024-04-16T19:32:56.163653+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:32:56.163653+0000","last_undegraded":"2024-04-16T19:32:56.163653+0000","last_fullsized":"2024-04-16T19:32:56.163653+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:57:25.094170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159959+0000","last_change":"2024-04-16T17:47:39.329121+0000","last_active":"2024-04-16T19:32:41.159959+0000","last_peered":"2024-04-16T19:32:41.159959+0000","last_clean":"2024-04-16T19:32:41.159959+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:32:41.159959+0000","last_undegraded":"2024-04-16T19:32:41.159959+0000","last_fullsized":"2024-04-16T19:32:41.159959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139757+0000","last_change":"2024-04-16T17:47:23.269620+0000","last_active":"2024-04-16T19:32:26.139757+0000","last_peered":"2024-04-16T19:32:26.139757+0000","last_clean":"2024-04-16T19:32:26.139757+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:32:26.139757+0000","last_undegraded":"2024-04-16T19:32:26.139757+0000","last_fullsized":"2024-04-16T19:32:26.139757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:43.201589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172841+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:33:41.172841+0000","last_peered":"2024-04-16T19:33:41.172841+0000","last_clean":"2024-04-16T19:33:41.172841+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:33:41.172841+0000","last_undegraded":"2024-04-16T19:33:41.172841+0000","last_fullsized":"2024-04-16T19:33:41.172841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155004+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:32:36.155004+0000","last_peered":"2024-04-16T19:32:36.155004+0000","last_clean":"2024-04-16T19:32:36.155004+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:32:36.155004+0000","last_undegraded":"2024-04-16T19:32:36.155004+0000","last_fullsized":"2024-04-16T19:32:36.155004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:14.585091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'104","reported_seq":271,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142192+0000","last_change":"2024-04-16T17:47:35.310781+0000","last_active":"2024-04-16T19:32:36.142192+0000","last_peered":"2024-04-16T19:32:36.142192+0000","last_clean":"2024-04-16T19:32:36.142192+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:32:36.142192+0000","last_undegraded":"2024-04-16T19:32:36.142192+0000","last_fullsized":"2024-04-16T19:32:36.142192+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:20.569781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157441+0000","last_change":"2024-04-16T17:47:37.322773+0000","last_active":"2024-04-16T19:32:41.157441+0000","last_peered":"2024-04-16T19:32:41.157441+0000","last_clean":"2024-04-16T19:32:41.157441+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:32:41.157441+0000","last_undegraded":"2024-04-16T19:32:41.157441+0000","last_fullsized":"2024-04-16T19:32:41.157441+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":50,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:21.172568+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:33:21.172568+0000","last_peered":"2024-04-16T19:33:21.172568+0000","last_clean":"2024-04-16T19:33:21.172568+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:33:21.172568+0000","last_undegraded":"2024-04-16T19:33:21.172568+0000","last_fullsized":"2024-04-16T19:33:21.172568+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:06.147896+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:32:06.147896+0000","last_peered":"2024-04-16T19:32:06.147896+0000","last_clean":"2024-04-16T19:32:06.147896+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:32:06.147896+0000","last_undegraded":"2024-04-16T19:32:06.147896+0000","last_fullsized":"2024-04-16T19:32:06.147896+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:46:24.982845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146589+0000","last_change":"2024-04-16T17:48:27.293279+0000","last_active":"2024-04-16T19:32:56.146589+0000","last_peered":"2024-04-16T19:32:56.146589+0000","last_clean":"2024-04-16T19:32:56.146589+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:32:56.146589+0000","last_undegraded":"2024-04-16T19:32:56.146589+0000","last_fullsized":"2024-04-16T19:32:56.146589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:44.442181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"48'8","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148529+0000","last_change":"2024-04-16T17:48:01.237052+0000","last_active":"2024-04-16T19:31:56.148529+0000","last_peered":"2024-04-16T19:31:56.148529+0000","last_clean":"2024-04-16T19:31:56.148529+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:31:56.148529+0000","last_undegraded":"2024-04-16T19:31:56.148529+0000","last_fullsized":"2024-04-16T19:31:56.148529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:22.523578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'555","reported_seq":1956,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.136598+0000","last_change":"2024-04-16T17:47:59.187458+0000","last_active":"2024-04-16T19:32:11.136598+0000","last_peered":"2024-04-16T19:32:11.136598+0000","last_clean":"2024-04-16T19:32:11.136598+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:32:11.136598+0000","last_undegraded":"2024-04-16T19:32:11.136598+0000","last_fullsized":"2024-04-16T19:32:11.136598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:17.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'544","reported_seq":775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158452+0000","last_change":"2024-04-16T17:47:57.184810+0000","last_active":"2024-04-16T19:32:36.158452+0000","last_peered":"2024-04-16T19:32:36.158452+0000","last_clean":"2024-04-16T19:32:36.158452+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:32:36.158452+0000","last_undegraded":"2024-04-16T19:32:36.158452+0000","last_fullsized":"2024-04-16T19:32:36.158452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:43.664108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"51'4","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163711+0000","last_change":"2024-04-16T17:47:55.183376+0000","last_active":"2024-04-16T19:32:56.163711+0000","last_peered":"2024-04-16T19:32:56.163711+0000","last_clean":"2024-04-16T19:32:56.163711+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:32:56.163711+0000","last_undegraded":"2024-04-16T19:32:56.163711+0000","last_fullsized":"2024-04-16T19:32:56.163711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.157536+0000","last_change":"2024-04-16T17:47:39.339617+0000","last_active":"2024-04-16T19:32:41.157536+0000","last_peered":"2024-04-16T19:32:41.157536+0000","last_clean":"2024-04-16T19:32:41.157536+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:32:41.157536+0000","last_undegraded":"2024-04-16T19:32:41.157536+0000","last_fullsized":"2024-04-16T19:32:41.157536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:48:08.715823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.156357+0000","last_change":"2024-04-16T17:47:23.263370+0000","last_active":"2024-04-16T19:32:26.156357+0000","last_peered":"2024-04-16T19:32:26.156357+0000","last_clean":"2024-04-16T19:32:26.156357+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:32:26.156357+0000","last_undegraded":"2024-04-16T19:32:26.156357+0000","last_fullsized":"2024-04-16T19:32:26.156357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:20.856004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.174981+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:33:41.174981+0000","last_peered":"2024-04-16T19:33:41.174981+0000","last_clean":"2024-04-16T19:33:41.174981+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:33:41.174981+0000","last_undegraded":"2024-04-16T19:33:41.174981+0000","last_fullsized":"2024-04-16T19:33:41.174981+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160793+0000","last_change":"2024-04-16T17:47:33.307304+0000","last_active":"2024-04-16T19:32:36.160793+0000","last_peered":"2024-04-16T19:32:36.160793+0000","last_clean":"2024-04-16T19:32:36.160793+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:32:36.160793+0000","last_undegraded":"2024-04-16T19:32:36.160793+0000","last_fullsized":"2024-04-16T19:32:36.160793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:29.241602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156191+0000","last_change":"2024-04-16T17:47:35.311451+0000","last_active":"2024-04-16T19:32:36.156191+0000","last_peered":"2024-04-16T19:32:36.156191+0000","last_clean":"2024-04-16T19:32:36.156191+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:32:36.156191+0000","last_undegraded":"2024-04-16T19:32:36.156191+0000","last_fullsized":"2024-04-16T19:32:36.156191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:46.400033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159764+0000","last_change":"2024-04-16T17:47:37.326333+0000","last_active":"2024-04-16T19:32:41.159764+0000","last_peered":"2024-04-16T19:32:41.159764+0000","last_clean":"2024-04-16T19:32:41.159764+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:32:41.159764+0000","last_undegraded":"2024-04-16T19:32:41.159764+0000","last_fullsized":"2024-04-16T19:32:41.159764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:25:00.455171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167717+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:33:01.167717+0000","last_peered":"2024-04-16T19:33:01.167717+0000","last_clean":"2024-04-16T19:33:01.167717+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:33:01.167717+0000","last_undegraded":"2024-04-16T19:33:01.167717+0000","last_fullsized":"2024-04-16T19:33:01.167717+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:13.813272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'89","reported_seq":694,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.159707+0000","last_change":"2024-04-16T17:48:34.328117+0000","last_active":"2024-04-16T19:32:31.159707+0000","last_peered":"2024-04-16T19:32:31.159707+0000","last_clean":"2024-04-16T19:32:31.159707+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:32:31.159707+0000","last_undegraded":"2024-04-16T19:32:31.159707+0000","last_fullsized":"2024-04-16T19:32:31.159707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:32:03.507021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155893+0000","last_change":"2024-04-16T17:48:27.295608+0000","last_active":"2024-04-16T19:32:26.155893+0000","last_peered":"2024-04-16T19:32:26.155893+0000","last_clean":"2024-04-16T19:32:26.155893+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:32:26.155893+0000","last_undegraded":"2024-04-16T19:32:26.155893+0000","last_fullsized":"2024-04-16T19:32:26.155893+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:09:08.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'12","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145278+0000","last_change":"2024-04-16T17:48:01.246592+0000","last_active":"2024-04-16T19:31:56.145278+0000","last_peered":"2024-04-16T19:31:56.145278+0000","last_clean":"2024-04-16T19:31:56.145278+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:31:56.145278+0000","last_undegraded":"2024-04-16T19:31:56.145278+0000","last_fullsized":"2024-04-16T19:31:56.145278+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:06.492939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164653+0000","last_change":"2024-04-16T17:47:59.178449+0000","last_active":"2024-04-16T19:33:01.164653+0000","last_peered":"2024-04-16T19:33:01.164653+0000","last_clean":"2024-04-16T19:33:01.164653+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:33:01.164653+0000","last_undegraded":"2024-04-16T19:33:01.164653+0000","last_fullsized":"2024-04-16T19:33:01.164653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:21:23.639075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'457","reported_seq":4410,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162217+0000","last_change":"2024-04-16T17:47:57.181693+0000","last_active":"2024-04-16T19:33:01.162217+0000","last_peered":"2024-04-16T19:33:01.162217+0000","last_clean":"2024-04-16T19:33:01.162217+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:33:01.162217+0000","last_undegraded":"2024-04-16T19:33:01.162217+0000","last_fullsized":"2024-04-16T19:33:01.162217+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:40:44.694025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'6","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161031+0000","last_change":"2024-04-16T17:47:55.174874+0000","last_active":"2024-04-16T19:32:56.161031+0000","last_peered":"2024-04-16T19:32:56.161031+0000","last_clean":"2024-04-16T19:32:56.161031+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:32:56.161031+0000","last_undegraded":"2024-04-16T19:32:56.161031+0000","last_fullsized":"2024-04-16T19:32:56.161031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.178208+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:33:41.178208+0000","last_peered":"2024-04-16T19:33:41.178208+0000","last_clean":"2024-04-16T19:33:41.178208+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:33:41.178208+0000","last_undegraded":"2024-04-16T19:33:41.178208+0000","last_fullsized":"2024-04-16T19:33:41.178208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:37.592633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157093+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:33:41.157093+0000","last_peered":"2024-04-16T19:33:41.157093+0000","last_clean":"2024-04-16T19:33:41.157093+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:33:41.157093+0000","last_undegraded":"2024-04-16T19:33:41.157093+0000","last_fullsized":"2024-04-16T19:33:41.157093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:08.399471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182918+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:34:11.182918+0000","last_peered":"2024-04-16T19:34:11.182918+0000","last_clean":"2024-04-16T19:34:11.182918+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:34:11.182918+0000","last_undegraded":"2024-04-16T19:34:11.182918+0000","last_fullsized":"2024-04-16T19:34:11.182918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'88","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159309+0000","last_change":"2024-04-16T17:47:35.319036+0000","last_active":"2024-04-16T19:32:36.159309+0000","last_peered":"2024-04-16T19:32:36.159309+0000","last_clean":"2024-04-16T19:32:36.159309+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:32:36.159309+0000","last_undegraded":"2024-04-16T19:32:36.159309+0000","last_fullsized":"2024-04-16T19:32:36.159309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162487+0000","last_change":"2024-04-16T17:47:37.330847+0000","last_active":"2024-04-16T19:32:41.162487+0000","last_peered":"2024-04-16T19:32:41.162487+0000","last_clean":"2024-04-16T19:32:41.162487+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:32:41.162487+0000","last_undegraded":"2024-04-16T19:32:41.162487+0000","last_fullsized":"2024-04-16T19:32:41.162487+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:10.138006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160442+0000","last_change":"2024-04-16T17:47:39.701654+0000","last_active":"2024-04-16T19:32:41.160442+0000","last_peered":"2024-04-16T19:32:41.160442+0000","last_clean":"2024-04-16T19:32:41.160442+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:32:41.160442+0000","last_undegraded":"2024-04-16T19:32:41.160442+0000","last_fullsized":"2024-04-16T19:32:41.160442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:09.137560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162629+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:33:01.162629+0000","last_peered":"2024-04-16T19:33:01.162629+0000","last_clean":"2024-04-16T19:33:01.162629+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:33:01.162629+0000","last_undegraded":"2024-04-16T19:33:01.162629+0000","last_fullsized":"2024-04-16T19:33:01.162629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:28:06.366066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'71","reported_seq":966,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143796+0000","last_change":"2024-04-16T17:48:34.313054+0000","last_active":"2024-04-16T19:31:51.143796+0000","last_peered":"2024-04-16T19:31:51.143796+0000","last_clean":"2024-04-16T19:31:51.143796+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:31:51.143796+0000","last_undegraded":"2024-04-16T19:31:51.143796+0000","last_fullsized":"2024-04-16T19:31:51.143796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:27:11.966580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.153539+0000","last_change":"2024-04-16T17:48:27.309014+0000","last_active":"2024-04-16T19:32:26.153539+0000","last_peered":"2024-04-16T19:32:26.153539+0000","last_clean":"2024-04-16T19:32:26.153539+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:32:26.153539+0000","last_undegraded":"2024-04-16T19:32:26.153539+0000","last_fullsized":"2024-04-16T19:32:26.153539+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:11.880344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'4","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145295+0000","last_change":"2024-04-16T17:48:01.251229+0000","last_active":"2024-04-16T19:31:56.145295+0000","last_peered":"2024-04-16T19:31:56.145295+0000","last_clean":"2024-04-16T19:31:56.145295+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:31:56.145295+0000","last_undegraded":"2024-04-16T19:31:56.145295+0000","last_fullsized":"2024-04-16T19:31:56.145295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:19:27.437555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.147763+0000","last_change":"2024-04-16T17:47:59.174249+0000","last_active":"2024-04-16T19:33:01.147763+0000","last_peered":"2024-04-16T19:33:01.147763+0000","last_clean":"2024-04-16T19:33:01.147763+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:33:01.147763+0000","last_undegraded":"2024-04-16T19:33:01.147763+0000","last_fullsized":"2024-04-16T19:33:01.147763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:54.272783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11370","reported_seq":61791,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.161030+0000","last_change":"2024-04-16T17:47:57.192920+0000","last_active":"2024-04-16T19:33:01.161030+0000","last_peered":"2024-04-16T19:33:01.161030+0000","last_clean":"2024-04-16T19:33:01.161030+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:33:01.161030+0000","last_undegraded":"2024-04-16T19:33:01.161030+0000","last_fullsized":"2024-04-16T19:33:01.161030+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1670,"log_dups_size":3000,"ondisk_log_size":1670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:31:13.718911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'12","reported_seq":338,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166321+0000","last_change":"2024-04-16T17:47:55.178367+0000","last_active":"2024-04-16T19:32:56.166321+0000","last_peered":"2024-04-16T19:32:56.166321+0000","last_clean":"2024-04-16T19:32:56.166321+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:32:56.166321+0000","last_undegraded":"2024-04-16T19:32:56.166321+0000","last_fullsized":"2024-04-16T19:32:56.166321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:41.894333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:01.180159+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:34:01.180159+0000","last_peered":"2024-04-16T19:34:01.180159+0000","last_clean":"2024-04-16T19:34:01.180159+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:34:01.180159+0000","last_undegraded":"2024-04-16T19:34:01.180159+0000","last_fullsized":"2024-04-16T19:34:01.180159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:53.130798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145890+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:31:46.145890+0000","last_peered":"2024-04-16T19:31:46.145890+0000","last_clean":"2024-04-16T19:31:46.145890+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:31:46.145890+0000","last_undegraded":"2024-04-16T19:31:46.145890+0000","last_fullsized":"2024-04-16T19:31:46.145890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:56.905751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.181077+0000","last_change":"2024-04-16T17:47:33.296653+0000","last_active":"2024-04-16T19:33:51.181077+0000","last_peered":"2024-04-16T19:33:51.181077+0000","last_clean":"2024-04-16T19:33:51.181077+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:33:51.181077+0000","last_undegraded":"2024-04-16T19:33:51.181077+0000","last_fullsized":"2024-04-16T19:33:51.181077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:16:00.330068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'152","reported_seq":340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.160930+0000","last_change":"2024-04-16T17:47:35.317123+0000","last_active":"2024-04-16T19:32:36.160930+0000","last_peered":"2024-04-16T19:32:36.160930+0000","last_clean":"2024-04-16T19:32:36.160930+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:32:36.160930+0000","last_undegraded":"2024-04-16T19:32:36.160930+0000","last_fullsized":"2024-04-16T19:32:36.160930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:22.846299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.160571+0000","last_change":"2024-04-16T17:47:37.334019+0000","last_active":"2024-04-16T19:32:41.160571+0000","last_peered":"2024-04-16T19:32:41.160571+0000","last_clean":"2024-04-16T19:32:41.160571+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:32:41.160571+0000","last_undegraded":"2024-04-16T19:32:41.160571+0000","last_fullsized":"2024-04-16T19:32:41.160571+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:07.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162425+0000","last_change":"2024-04-16T17:47:39.699556+0000","last_active":"2024-04-16T19:32:41.162425+0000","last_peered":"2024-04-16T19:32:41.162425+0000","last_clean":"2024-04-16T19:32:41.162425+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:32:41.162425+0000","last_undegraded":"2024-04-16T19:32:41.162425+0000","last_fullsized":"2024-04-16T19:32:41.162425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:51.630893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165020+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:33:01.165020+0000","last_peered":"2024-04-16T19:33:01.165020+0000","last_clean":"2024-04-16T19:33:01.165020+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:33:01.165020+0000","last_undegraded":"2024-04-16T19:33:01.165020+0000","last_fullsized":"2024-04-16T19:33:01.165020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:07:42.431604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'103","reported_seq":1819,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158185+0000","last_change":"2024-04-16T17:48:34.337561+0000","last_active":"2024-04-16T19:32:31.158185+0000","last_peered":"2024-04-16T19:32:31.158185+0000","last_clean":"2024-04-16T19:32:31.158185+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:32:31.158185+0000","last_undegraded":"2024-04-16T19:32:31.158185+0000","last_fullsized":"2024-04-16T19:32:31.158185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:01.320649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:46.145725+0000","last_change":"2024-04-16T17:48:27.293905+0000","last_active":"2024-04-16T19:31:46.145725+0000","last_peered":"2024-04-16T19:31:46.145725+0000","last_clean":"2024-04-16T19:31:46.145725+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:31:46.145725+0000","last_undegraded":"2024-04-16T19:31:46.145725+0000","last_fullsized":"2024-04-16T19:31:46.145725+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:59.072103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'8","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145241+0000","last_change":"2024-04-16T17:48:01.248531+0000","last_active":"2024-04-16T19:31:56.145241+0000","last_peered":"2024-04-16T19:31:56.145241+0000","last_clean":"2024-04-16T19:31:56.145241+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:31:56.145241+0000","last_undegraded":"2024-04-16T19:31:56.145241+0000","last_fullsized":"2024-04-16T19:31:56.145241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.697705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.167662+0000","last_change":"2024-04-16T17:47:59.187755+0000","last_active":"2024-04-16T19:33:01.167662+0000","last_peered":"2024-04-16T19:33:01.167662+0000","last_clean":"2024-04-16T19:33:01.167662+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:33:01.167662+0000","last_undegraded":"2024-04-16T19:33:01.167662+0000","last_fullsized":"2024-04-16T19:33:01.167662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:25.460755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'196","reported_seq":421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165564+0000","last_change":"2024-04-16T17:47:57.195138+0000","last_active":"2024-04-16T19:33:01.165564+0000","last_peered":"2024-04-16T19:33:01.165564+0000","last_clean":"2024-04-16T19:33:01.165564+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:33:01.165564+0000","last_undegraded":"2024-04-16T19:33:01.165564+0000","last_fullsized":"2024-04-16T19:33:01.165564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:36.309438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'12","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.159825+0000","last_change":"2024-04-16T17:47:55.160472+0000","last_active":"2024-04-16T19:32:56.159825+0000","last_peered":"2024-04-16T19:32:56.159825+0000","last_clean":"2024-04-16T19:32:56.159825+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:32:56.159825+0000","last_undegraded":"2024-04-16T19:32:56.159825+0000","last_fullsized":"2024-04-16T19:32:56.159825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:00.875301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155799+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:32:26.155799+0000","last_peered":"2024-04-16T19:32:26.155799+0000","last_clean":"2024-04-16T19:32:26.155799+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:32:26.155799+0000","last_undegraded":"2024-04-16T19:32:26.155799+0000","last_fullsized":"2024-04-16T19:32:26.155799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:09.772201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151302+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:32:21.151302+0000","last_peered":"2024-04-16T19:32:21.151302+0000","last_clean":"2024-04-16T19:32:21.151302+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:32:21.151302+0000","last_undegraded":"2024-04-16T19:32:21.151302+0000","last_fullsized":"2024-04-16T19:32:21.151302+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:38:44.777444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"48'5","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155232+0000","last_change":"2024-04-16T17:47:33.306460+0000","last_active":"2024-04-16T19:32:36.155232+0000","last_peered":"2024-04-16T19:32:36.155232+0000","last_clean":"2024-04-16T19:32:36.155232+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:32:36.155232+0000","last_undegraded":"2024-04-16T19:32:36.155232+0000","last_fullsized":"2024-04-16T19:32:36.155232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:15.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'104","reported_seq":277,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.160070+0000","last_change":"2024-04-16T17:47:35.314274+0000","last_active":"2024-04-16T19:33:51.160070+0000","last_peered":"2024-04-16T19:33:51.160070+0000","last_clean":"2024-04-16T19:33:51.160070+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:33:51.160070+0000","last_undegraded":"2024-04-16T19:33:51.160070+0000","last_fullsized":"2024-04-16T19:33:51.160070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:58.087539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.183865+0000","last_change":"2024-04-16T17:47:37.331871+0000","last_active":"2024-04-16T19:34:11.183865+0000","last_peered":"2024-04-16T19:34:11.183865+0000","last_clean":"2024-04-16T19:34:11.183865+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:34:11.183865+0000","last_undegraded":"2024-04-16T19:34:11.183865+0000","last_fullsized":"2024-04-16T19:34:11.183865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:10.644125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162350+0000","last_change":"2024-04-16T17:47:39.323414+0000","last_active":"2024-04-16T19:32:41.162350+0000","last_peered":"2024-04-16T19:32:41.162350+0000","last_clean":"2024-04-16T19:32:41.162350+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:32:41.162350+0000","last_undegraded":"2024-04-16T19:32:41.162350+0000","last_fullsized":"2024-04-16T19:32:41.162350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:48:29.103797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.177321+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:33:46.177321+0000","last_peered":"2024-04-16T19:33:46.177321+0000","last_clean":"2024-04-16T19:33:46.177321+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:33:46.177321+0000","last_undegraded":"2024-04-16T19:33:46.177321+0000","last_fullsized":"2024-04-16T19:33:46.177321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:41.778594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'68","reported_seq":882,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.158123+0000","last_change":"2024-04-16T17:48:34.343716+0000","last_active":"2024-04-16T19:32:31.158123+0000","last_peered":"2024-04-16T19:32:31.158123+0000","last_clean":"2024-04-16T19:32:31.158123+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:32:31.158123+0000","last_undegraded":"2024-04-16T19:32:31.158123+0000","last_fullsized":"2024-04-16T19:32:31.158123+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:06:48.210902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152776+0000","last_change":"2024-04-16T17:48:27.305044+0000","last_active":"2024-04-16T19:32:26.152776+0000","last_peered":"2024-04-16T19:32:26.152776+0000","last_clean":"2024-04-16T19:32:26.152776+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:32:26.152776+0000","last_undegraded":"2024-04-16T19:32:26.152776+0000","last_fullsized":"2024-04-16T19:32:26.152776+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:17:17.183220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"48'1","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.168780+0000","last_change":"2024-04-16T17:48:01.254059+0000","last_active":"2024-04-16T19:33:26.168780+0000","last_peered":"2024-04-16T19:33:26.168780+0000","last_clean":"2024-04-16T19:33:26.168780+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:33:26.168780+0000","last_undegraded":"2024-04-16T19:33:26.168780+0000","last_fullsized":"2024-04-16T19:33:26.168780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162464+0000","last_change":"2024-04-16T17:47:59.195638+0000","last_active":"2024-04-16T19:33:01.162464+0000","last_peered":"2024-04-16T19:33:01.162464+0000","last_clean":"2024-04-16T19:33:01.162464+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:33:01.162464+0000","last_undegraded":"2024-04-16T19:33:01.162464+0000","last_fullsized":"2024-04-16T19:33:01.162464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:57.426064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'488","reported_seq":3937,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162374+0000","last_change":"2024-04-16T17:47:57.174503+0000","last_active":"2024-04-16T19:33:01.162374+0000","last_peered":"2024-04-16T19:33:01.162374+0000","last_clean":"2024-04-16T19:33:01.162374+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:33:01.162374+0000","last_undegraded":"2024-04-16T19:33:01.162374+0000","last_fullsized":"2024-04-16T19:33:01.162374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:34.199358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'27","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.166245+0000","last_change":"2024-04-16T17:47:55.169780+0000","last_active":"2024-04-16T19:32:56.166245+0000","last_peered":"2024-04-16T19:32:56.166245+0000","last_clean":"2024-04-16T19:32:56.166245+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:32:56.166245+0000","last_undegraded":"2024-04-16T19:32:56.166245+0000","last_fullsized":"2024-04-16T19:32:56.166245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:00:00.417510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.139882+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:32:26.139882+0000","last_peered":"2024-04-16T19:32:26.139882+0000","last_clean":"2024-04-16T19:32:26.139882+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:32:26.139882+0000","last_undegraded":"2024-04-16T19:32:26.139882+0000","last_fullsized":"2024-04-16T19:32:26.139882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:43:58.984431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.154419+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:32:21.154419+0000","last_peered":"2024-04-16T19:32:21.154419+0000","last_clean":"2024-04-16T19:32:21.154419+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:32:21.154419+0000","last_undegraded":"2024-04-16T19:32:21.154419+0000","last_fullsized":"2024-04-16T19:32:21.154419+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:21.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"33'3","reported_seq":802,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.159282+0000","last_change":"2024-04-16T17:47:33.297794+0000","last_active":"2024-04-16T19:32:36.159282+0000","last_peered":"2024-04-16T19:32:36.159282+0000","last_clean":"2024-04-16T19:32:36.159282+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:32:36.159282+0000","last_undegraded":"2024-04-16T19:32:36.159282+0000","last_fullsized":"2024-04-16T19:32:36.159282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:55:25.132826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'120","reported_seq":298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186858+0000","last_change":"2024-04-16T17:47:35.319657+0000","last_active":"2024-04-16T19:34:11.186858+0000","last_peered":"2024-04-16T19:34:11.186858+0000","last_clean":"2024-04-16T19:34:11.186858+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:34:11.186858+0000","last_undegraded":"2024-04-16T19:34:11.186858+0000","last_fullsized":"2024-04-16T19:34:11.186858+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:18:08.628044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'12","reported_seq":1340,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.177090+0000","last_change":"2024-04-16T17:47:37.331215+0000","last_active":"2024-04-16T19:33:56.177090+0000","last_peered":"2024-04-16T19:33:56.177090+0000","last_clean":"2024-04-16T19:33:56.177090+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:33:56.177090+0000","last_undegraded":"2024-04-16T19:33:56.177090+0000","last_fullsized":"2024-04-16T19:33:56.177090+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162223+0000","last_change":"2024-04-16T17:47:39.332386+0000","last_active":"2024-04-16T19:32:41.162223+0000","last_peered":"2024-04-16T19:32:41.162223+0000","last_clean":"2024-04-16T19:32:41.162223+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:32:41.162223+0000","last_undegraded":"2024-04-16T19:32:41.162223+0000","last_fullsized":"2024-04-16T19:32:41.162223+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:02.053736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.153850+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:32:31.153850+0000","last_peered":"2024-04-16T19:32:31.153850+0000","last_clean":"2024-04-16T19:32:31.153850+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:32:31.153850+0000","last_undegraded":"2024-04-16T19:32:31.153850+0000","last_fullsized":"2024-04-16T19:32:31.153850+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:59:18.189132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'86","reported_seq":1557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169733+0000","last_change":"2024-04-16T17:48:34.318628+0000","last_active":"2024-04-16T19:33:36.169733+0000","last_peered":"2024-04-16T19:33:36.169733+0000","last_clean":"2024-04-16T19:33:36.169733+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:33:36.169733+0000","last_undegraded":"2024-04-16T19:33:36.169733+0000","last_fullsized":"2024-04-16T19:33:36.169733+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:15:00.721377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172016+0000","last_change":"2024-04-16T17:48:27.299691+0000","last_active":"2024-04-16T19:33:31.172016+0000","last_peered":"2024-04-16T19:33:31.172016+0000","last_clean":"2024-04-16T19:33:31.172016+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:33:31.172016+0000","last_undegraded":"2024-04-16T19:33:31.172016+0000","last_fullsized":"2024-04-16T19:33:31.172016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:40:17.511736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'6","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.165634+0000","last_change":"2024-04-16T17:48:01.246789+0000","last_active":"2024-04-16T19:33:01.165634+0000","last_peered":"2024-04-16T19:33:01.165634+0000","last_clean":"2024-04-16T19:33:01.165634+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:33:01.165634+0000","last_undegraded":"2024-04-16T19:33:01.165634+0000","last_fullsized":"2024-04-16T19:33:01.165634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:22:47.722149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'555","reported_seq":1953,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176329+0000","last_change":"2024-04-16T17:47:59.182365+0000","last_active":"2024-04-16T19:33:36.176329+0000","last_peered":"2024-04-16T19:33:36.176329+0000","last_clean":"2024-04-16T19:33:36.176329+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:33:36.176329+0000","last_undegraded":"2024-04-16T19:33:36.176329+0000","last_fullsized":"2024-04-16T19:33:36.176329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:16:49.081799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'492","reported_seq":710,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148336+0000","last_change":"2024-04-16T17:47:57.180106+0000","last_active":"2024-04-16T19:31:56.148336+0000","last_peered":"2024-04-16T19:31:56.148336+0000","last_clean":"2024-04-16T19:31:56.148336+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:31:56.148336+0000","last_undegraded":"2024-04-16T19:31:56.148336+0000","last_fullsized":"2024-04-16T19:31:56.148336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'4","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.144008+0000","last_change":"2024-04-16T17:47:55.169894+0000","last_active":"2024-04-16T19:31:51.144008+0000","last_peered":"2024-04-16T19:31:51.144008+0000","last_clean":"2024-04-16T19:31:51.144008+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:31:51.144008+0000","last_undegraded":"2024-04-16T19:31:51.144008+0000","last_fullsized":"2024-04-16T19:31:51.144008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:24.954941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152629+0000","last_change":"2024-04-16T17:47:23.266570+0000","last_active":"2024-04-16T19:32:26.152629+0000","last_peered":"2024-04-16T19:32:26.152629+0000","last_clean":"2024-04-16T19:32:26.152629+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:32:26.152629+0000","last_undegraded":"2024-04-16T19:32:26.152629+0000","last_fullsized":"2024-04-16T19:32:26.152629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:55.441583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152031+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:32:21.152031+0000","last_peered":"2024-04-16T19:32:21.152031+0000","last_clean":"2024-04-16T19:32:21.152031+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:32:21.152031+0000","last_undegraded":"2024-04-16T19:32:21.152031+0000","last_fullsized":"2024-04-16T19:32:21.152031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155126+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:32:36.155126+0000","last_peered":"2024-04-16T19:32:36.155126+0000","last_clean":"2024-04-16T19:32:36.155126+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:32:36.155126+0000","last_undegraded":"2024-04-16T19:32:36.155126+0000","last_fullsized":"2024-04-16T19:32:36.155126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:54.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'152","reported_seq":341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145026+0000","last_change":"2024-04-16T17:47:35.310214+0000","last_active":"2024-04-16T19:31:56.145026+0000","last_peered":"2024-04-16T19:31:56.145026+0000","last_clean":"2024-04-16T19:31:56.145026+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:31:56.145026+0000","last_undegraded":"2024-04-16T19:31:56.145026+0000","last_fullsized":"2024-04-16T19:31:56.145026+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:06.643215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.142943+0000","last_change":"2024-04-16T17:47:37.328751+0000","last_active":"2024-04-16T19:31:36.142943+0000","last_peered":"2024-04-16T19:31:36.142943+0000","last_clean":"2024-04-16T19:31:36.142943+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:31:36.142943+0000","last_undegraded":"2024-04-16T19:31:36.142943+0000","last_fullsized":"2024-04-16T19:31:36.142943+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:30:39.392016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144646+0000","last_change":"2024-04-16T17:47:39.336993+0000","last_active":"2024-04-16T19:31:36.144646+0000","last_peered":"2024-04-16T19:31:36.144646+0000","last_clean":"2024-04-16T19:31:36.144646+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:31:36.144646+0000","last_undegraded":"2024-04-16T19:31:36.144646+0000","last_fullsized":"2024-04-16T19:31:36.144646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:30:55.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.140940+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:32:31.140940+0000","last_peered":"2024-04-16T19:32:31.140940+0000","last_clean":"2024-04-16T19:32:31.140940+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:32:31.140940+0000","last_undegraded":"2024-04-16T19:32:31.140940+0000","last_fullsized":"2024-04-16T19:32:31.140940+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:08:16.152440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'99","reported_seq":1431,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174016+0000","last_change":"2024-04-16T17:48:34.341651+0000","last_active":"2024-04-16T19:33:36.174016+0000","last_peered":"2024-04-16T19:33:36.174016+0000","last_clean":"2024-04-16T19:33:36.174016+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:33:36.174016+0000","last_undegraded":"2024-04-16T19:33:36.174016+0000","last_fullsized":"2024-04-16T19:33:36.174016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:45.319846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.169918+0000","last_change":"2024-04-16T17:48:27.306641+0000","last_active":"2024-04-16T19:33:31.169918+0000","last_peered":"2024-04-16T19:33:31.169918+0000","last_clean":"2024-04-16T19:33:31.169918+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:33:31.169918+0000","last_undegraded":"2024-04-16T19:33:31.169918+0000","last_fullsized":"2024-04-16T19:33:31.169918+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:02.455431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'7","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164768+0000","last_change":"2024-04-16T17:48:01.263221+0000","last_active":"2024-04-16T19:33:01.164768+0000","last_peered":"2024-04-16T19:33:01.164768+0000","last_clean":"2024-04-16T19:33:01.164768+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:33:01.164768+0000","last_undegraded":"2024-04-16T19:33:01.164768+0000","last_fullsized":"2024-04-16T19:33:01.164768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:26:38.879976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'555","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.132909+0000","last_change":"2024-04-16T17:47:59.183413+0000","last_active":"2024-04-16T19:31:56.132909+0000","last_peered":"2024-04-16T19:31:56.132909+0000","last_clean":"2024-04-16T19:31:56.132909+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:31:56.132909+0000","last_undegraded":"2024-04-16T19:31:56.132909+0000","last_fullsized":"2024-04-16T19:31:56.132909+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:27.860005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'230","reported_seq":460,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172962+0000","last_change":"2024-04-16T17:47:57.173077+0000","last_active":"2024-04-16T19:33:31.172962+0000","last_peered":"2024-04-16T19:33:31.172962+0000","last_clean":"2024-04-16T19:33:31.172962+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:33:31.172962+0000","last_undegraded":"2024-04-16T19:33:31.172962+0000","last_fullsized":"2024-04-16T19:33:31.172962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:49.105681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.168703+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:33:31.168703+0000","last_peered":"2024-04-16T19:33:31.168703+0000","last_clean":"2024-04-16T19:33:31.168703+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:33:31.168703+0000","last_undegraded":"2024-04-16T19:33:31.168703+0000","last_fullsized":"2024-04-16T19:33:31.168703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:53:11.770314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152568+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:32:26.152568+0000","last_peered":"2024-04-16T19:32:26.152568+0000","last_clean":"2024-04-16T19:32:26.152568+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:32:26.152568+0000","last_undegraded":"2024-04-16T19:32:26.152568+0000","last_fullsized":"2024-04-16T19:32:26.152568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:32.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152102+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:32:21.152102+0000","last_peered":"2024-04-16T19:32:21.152102+0000","last_clean":"2024-04-16T19:32:21.152102+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:32:21.152102+0000","last_undegraded":"2024-04-16T19:32:21.152102+0000","last_fullsized":"2024-04-16T19:32:21.152102+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:44.013747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":177,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.156238+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:32:36.156238+0000","last_peered":"2024-04-16T19:32:36.156238+0000","last_clean":"2024-04-16T19:32:36.156238+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:32:36.156238+0000","last_undegraded":"2024-04-16T19:32:36.156238+0000","last_fullsized":"2024-04-16T19:32:36.156238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:14.063717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.367224+0000","last_change":"2024-04-16T17:47:35.303292+0000","last_active":"2024-04-16T19:31:31.367224+0000","last_peered":"2024-04-16T19:31:31.367224+0000","last_clean":"2024-04-16T19:31:31.367224+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:31:31.367224+0000","last_undegraded":"2024-04-16T19:31:31.367224+0000","last_fullsized":"2024-04-16T19:31:31.367224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:53:27.611763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:11.154187+0000","last_change":"2024-04-16T17:47:37.331029+0000","last_active":"2024-04-16T19:32:11.154187+0000","last_peered":"2024-04-16T19:32:11.154187+0000","last_clean":"2024-04-16T19:32:11.154187+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:32:11.154187+0000","last_undegraded":"2024-04-16T19:32:11.154187+0000","last_fullsized":"2024-04-16T19:32:11.154187+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:22:50.167123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.143665+0000","last_change":"2024-04-16T17:47:39.700695+0000","last_active":"2024-04-16T19:31:36.143665+0000","last_peered":"2024-04-16T19:31:36.143665+0000","last_clean":"2024-04-16T19:31:36.143665+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:31:36.143665+0000","last_undegraded":"2024-04-16T19:31:36.143665+0000","last_fullsized":"2024-04-16T19:31:36.143665+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:34.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157016+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:32:31.157016+0000","last_peered":"2024-04-16T19:32:31.157016+0000","last_clean":"2024-04-16T19:32:31.157016+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:32:31.157016+0000","last_undegraded":"2024-04-16T19:32:31.157016+0000","last_fullsized":"2024-04-16T19:32:31.157016+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:05.269892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'96","reported_seq":919,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156060+0000","last_change":"2024-04-16T17:48:34.324041+0000","last_active":"2024-04-16T19:33:36.156060+0000","last_peered":"2024-04-16T19:33:36.156060+0000","last_clean":"2024-04-16T19:33:36.156060+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:33:36.156060+0000","last_undegraded":"2024-04-16T19:33:36.156060+0000","last_fullsized":"2024-04-16T19:33:36.156060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:18:27.393972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172731+0000","last_change":"2024-04-16T17:48:27.307648+0000","last_active":"2024-04-16T19:33:31.172731+0000","last_peered":"2024-04-16T19:33:31.172731+0000","last_clean":"2024-04-16T19:33:31.172731+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:33:31.172731+0000","last_undegraded":"2024-04-16T19:33:31.172731+0000","last_fullsized":"2024-04-16T19:33:31.172731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:44:20.265988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'7","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.162293+0000","last_change":"2024-04-16T17:48:01.248401+0000","last_active":"2024-04-16T19:33:01.162293+0000","last_peered":"2024-04-16T19:33:01.162293+0000","last_clean":"2024-04-16T19:33:01.162293+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:33:01.162293+0000","last_undegraded":"2024-04-16T19:33:01.162293+0000","last_fullsized":"2024-04-16T19:33:01.162293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:18:53.475406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145357+0000","last_change":"2024-04-16T17:47:59.199961+0000","last_active":"2024-04-16T19:31:56.145357+0000","last_peered":"2024-04-16T19:31:56.145357+0000","last_clean":"2024-04-16T19:31:56.145357+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:31:56.145357+0000","last_undegraded":"2024-04-16T19:31:56.145357+0000","last_fullsized":"2024-04-16T19:31:56.145357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:44.427887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'129","reported_seq":296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150185+0000","last_change":"2024-04-16T17:47:57.189447+0000","last_active":"2024-04-16T19:31:56.150185+0000","last_peered":"2024-04-16T19:31:56.150185+0000","last_clean":"2024-04-16T19:31:56.150185+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:31:56.150185+0000","last_undegraded":"2024-04-16T19:31:56.150185+0000","last_fullsized":"2024-04-16T19:31:56.150185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:55.411174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'3","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:26.171761+0000","last_change":"2024-04-16T17:47:55.160684+0000","last_active":"2024-04-16T19:33:26.171761+0000","last_peered":"2024-04-16T19:33:26.171761+0000","last_clean":"2024-04-16T19:33:26.171761+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:33:26.171761+0000","last_undegraded":"2024-04-16T19:33:26.171761+0000","last_fullsized":"2024-04-16T19:33:26.171761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:51:12.075220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148167+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:31:56.148167+0000","last_peered":"2024-04-16T19:31:56.148167+0000","last_clean":"2024-04-16T19:31:56.148167+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:31:56.148167+0000","last_undegraded":"2024-04-16T19:31:56.148167+0000","last_fullsized":"2024-04-16T19:31:56.148167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:51.847679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159976+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:33:51.159976+0000","last_peered":"2024-04-16T19:33:51.159976+0000","last_clean":"2024-04-16T19:33:51.159976+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:33:51.159976+0000","last_undegraded":"2024-04-16T19:33:51.159976+0000","last_fullsized":"2024-04-16T19:33:51.159976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:50:28.662489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:31.361145+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:31:31.361145+0000","last_peered":"2024-04-16T19:31:31.361145+0000","last_clean":"2024-04-16T19:31:31.361145+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:31:31.361145+0000","last_undegraded":"2024-04-16T19:31:31.361145+0000","last_fullsized":"2024-04-16T19:31:31.361145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:03.171529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'32","reported_seq":160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.158251+0000","last_change":"2024-04-16T17:47:35.317051+0000","last_active":"2024-04-16T19:32:36.158251+0000","last_peered":"2024-04-16T19:32:36.158251+0000","last_clean":"2024-04-16T19:32:36.158251+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:32:36.158251+0000","last_undegraded":"2024-04-16T19:32:36.158251+0000","last_fullsized":"2024-04-16T19:32:36.158251+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:26.163793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.162398+0000","last_change":"2024-04-16T17:47:37.320351+0000","last_active":"2024-04-16T19:32:41.162398+0000","last_peered":"2024-04-16T19:32:41.162398+0000","last_clean":"2024-04-16T19:32:41.162398+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:32:41.162398+0000","last_undegraded":"2024-04-16T19:32:41.162398+0000","last_fullsized":"2024-04-16T19:32:41.162398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:09:16.166643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:36.144501+0000","last_change":"2024-04-16T17:47:39.329617+0000","last_active":"2024-04-16T19:31:36.144501+0000","last_peered":"2024-04-16T19:31:36.144501+0000","last_clean":"2024-04-16T19:31:36.144501+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:31:36.144501+0000","last_undegraded":"2024-04-16T19:31:36.144501+0000","last_fullsized":"2024-04-16T19:31:36.144501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:48:34.423122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:31.157983+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:32:31.157983+0000","last_peered":"2024-04-16T19:32:31.157983+0000","last_clean":"2024-04-16T19:32:31.157983+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:32:31.157983+0000","last_undegraded":"2024-04-16T19:32:31.157983+0000","last_fullsized":"2024-04-16T19:32:31.157983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:49:40.225240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'126","reported_seq":1450,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.156003+0000","last_change":"2024-04-16T17:48:34.338811+0000","last_active":"2024-04-16T19:33:36.156003+0000","last_peered":"2024-04-16T19:33:36.156003+0000","last_clean":"2024-04-16T19:33:36.156003+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:33:36.156003+0000","last_undegraded":"2024-04-16T19:33:36.156003+0000","last_fullsized":"2024-04-16T19:33:36.156003+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:09:10.563495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172826+0000","last_change":"2024-04-16T17:48:27.288374+0000","last_active":"2024-04-16T19:33:31.172826+0000","last_peered":"2024-04-16T19:33:31.172826+0000","last_clean":"2024-04-16T19:33:31.172826+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:33:31.172826+0000","last_undegraded":"2024-04-16T19:33:31.172826+0000","last_fullsized":"2024-04-16T19:33:31.172826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:11.751579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:01.164610+0000","last_change":"2024-04-16T17:48:01.263690+0000","last_active":"2024-04-16T19:33:01.164610+0000","last_peered":"2024-04-16T19:33:01.164610+0000","last_clean":"2024-04-16T19:33:01.164610+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:33:01.164610+0000","last_undegraded":"2024-04-16T19:33:01.164610+0000","last_fullsized":"2024-04-16T19:33:01.164610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:47.176953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.145421+0000","last_change":"2024-04-16T17:47:59.199357+0000","last_active":"2024-04-16T19:31:56.145421+0000","last_peered":"2024-04-16T19:31:56.145421+0000","last_clean":"2024-04-16T19:31:56.145421+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:31:56.145421+0000","last_undegraded":"2024-04-16T19:31:56.145421+0000","last_fullsized":"2024-04-16T19:31:56.145421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:42.077363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11055","reported_seq":18133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.150080+0000","last_change":"2024-04-16T17:47:57.184044+0000","last_active":"2024-04-16T19:31:56.150080+0000","last_peered":"2024-04-16T19:31:56.150080+0000","last_clean":"2024-04-16T19:31:56.150080+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:31:56.150080+0000","last_undegraded":"2024-04-16T19:31:56.150080+0000","last_fullsized":"2024-04-16T19:31:56.150080+0000","mapping_epoch":34,"log_start":"53'9200","ondisk_log_start":"53'9200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1855,"log_dups_size":3000,"ondisk_log_size":1855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:21.202505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'6","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:51.143908+0000","last_change":"2024-04-16T17:47:55.159410+0000","last_active":"2024-04-16T19:31:51.143908+0000","last_peered":"2024-04-16T19:31:51.143908+0000","last_clean":"2024-04-16T19:31:51.143908+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:31:51.143908+0000","last_undegraded":"2024-04-16T19:31:51.143908+0000","last_fullsized":"2024-04-16T19:31:51.143908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:32:42.685205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.179184+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:33:46.179184+0000","last_peered":"2024-04-16T19:33:46.179184+0000","last_clean":"2024-04-16T19:33:46.179184+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:33:46.179184+0000","last_undegraded":"2024-04-16T19:33:46.179184+0000","last_fullsized":"2024-04-16T19:33:46.179184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:54.182792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.173750+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:33:36.173750+0000","last_peered":"2024-04-16T19:33:36.173750+0000","last_clean":"2024-04-16T19:33:36.173750+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:33:36.173750+0000","last_undegraded":"2024-04-16T19:33:36.173750+0000","last_fullsized":"2024-04-16T19:33:36.173750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:30.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.149851+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:32:01.149851+0000","last_peered":"2024-04-16T19:32:01.149851+0000","last_clean":"2024-04-16T19:32:01.149851+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:32:01.149851+0000","last_undegraded":"2024-04-16T19:32:01.149851+0000","last_fullsized":"2024-04-16T19:32:01.149851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:53.316949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'80","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.155149+0000","last_change":"2024-04-16T17:47:35.308166+0000","last_active":"2024-04-16T19:32:36.155149+0000","last_peered":"2024-04-16T19:32:36.155149+0000","last_clean":"2024-04-16T19:32:36.155149+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:32:36.155149+0000","last_undegraded":"2024-04-16T19:32:36.155149+0000","last_fullsized":"2024-04-16T19:32:36.155149+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:53.924341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:41.159795+0000","last_change":"2024-04-16T17:47:37.321988+0000","last_active":"2024-04-16T19:32:41.159795+0000","last_peered":"2024-04-16T19:32:41.159795+0000","last_clean":"2024-04-16T19:32:41.159795+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:32:41.159795+0000","last_undegraded":"2024-04-16T19:32:41.159795+0000","last_fullsized":"2024-04-16T19:32:41.159795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:19:50.343585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:16.164797+0000","last_change":"2024-04-16T17:47:39.326823+0000","last_active":"2024-04-16T19:33:16.164797+0000","last_peered":"2024-04-16T19:33:16.164797+0000","last_clean":"2024-04-16T19:33:16.164797+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:33:16.164797+0000","last_undegraded":"2024-04-16T19:33:16.164797+0000","last_fullsized":"2024-04-16T19:33:16.164797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:26.874144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'22","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157286+0000","last_change":"2024-04-16T17:47:55.158595+0000","last_active":"2024-04-16T19:33:41.157286+0000","last_peered":"2024-04-16T19:33:41.157286+0000","last_clean":"2024-04-16T19:33:41.157286+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:33:41.157286+0000","last_undegraded":"2024-04-16T19:33:41.157286+0000","last_fullsized":"2024-04-16T19:33:41.157286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:20.899420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.158764+0000","last_change":"2024-04-16T17:47:32.278320+0000","last_active":"2024-04-16T19:33:46.158764+0000","last_peered":"2024-04-16T19:33:46.158764+0000","last_clean":"2024-04-16T19:33:46.158764+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:33:46.158764+0000","last_undegraded":"2024-04-16T19:33:46.158764+0000","last_fullsized":"2024-04-16T19:33:46.158764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:32.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.169629+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:33:36.169629+0000","last_peered":"2024-04-16T19:33:36.169629+0000","last_clean":"2024-04-16T19:33:36.169629+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:33:36.169629+0000","last_undegraded":"2024-04-16T19:33:36.169629+0000","last_fullsized":"2024-04-16T19:33:36.169629+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'3","reported_seq":205,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.172656+0000","last_change":"2024-04-16T17:47:55.175820+0000","last_active":"2024-04-16T19:33:41.172656+0000","last_peered":"2024-04-16T19:33:41.172656+0000","last_clean":"2024-04-16T19:33:41.172656+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:33:41.172656+0000","last_undegraded":"2024-04-16T19:33:41.172656+0000","last_fullsized":"2024-04-16T19:33:41.172656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:15:30.695910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174508+0000","last_change":"2024-04-16T17:47:23.260359+0000","last_active":"2024-04-16T19:33:36.174508+0000","last_peered":"2024-04-16T19:33:36.174508+0000","last_clean":"2024-04-16T19:33:36.174508+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:33:36.174508+0000","last_undegraded":"2024-04-16T19:33:36.174508+0000","last_fullsized":"2024-04-16T19:33:36.174508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:31.902037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.155944+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:33:36.155944+0000","last_peered":"2024-04-16T19:33:36.155944+0000","last_clean":"2024-04-16T19:33:36.155944+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:33:36.155944+0000","last_undegraded":"2024-04-16T19:33:36.155944+0000","last_fullsized":"2024-04-16T19:33:36.155944+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:39.513573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'7","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.151820+0000","last_change":"2024-04-16T17:47:55.178677+0000","last_active":"2024-04-16T19:32:21.151820+0000","last_peered":"2024-04-16T19:32:21.151820+0000","last_clean":"2024-04-16T19:32:21.151820+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:32:21.151820+0000","last_undegraded":"2024-04-16T19:32:21.151820+0000","last_fullsized":"2024-04-16T19:32:21.151820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.171376+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:33:36.171376+0000","last_peered":"2024-04-16T19:33:36.171376+0000","last_clean":"2024-04-16T19:33:36.171376+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:33:36.171376+0000","last_undegraded":"2024-04-16T19:33:36.171376+0000","last_fullsized":"2024-04-16T19:33:36.171376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:37:07.558923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.176213+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:33:36.176213+0000","last_peered":"2024-04-16T19:33:36.176213+0000","last_clean":"2024-04-16T19:33:36.176213+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:33:36.176213+0000","last_undegraded":"2024-04-16T19:33:36.176213+0000","last_fullsized":"2024-04-16T19:33:36.176213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:40.971975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'6","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177717+0000","last_change":"2024-04-16T17:47:55.156061+0000","last_active":"2024-04-16T19:33:41.177717+0000","last_peered":"2024-04-16T19:33:41.177717+0000","last_clean":"2024-04-16T19:33:41.177717+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:33:41.177717+0000","last_undegraded":"2024-04-16T19:33:41.177717+0000","last_fullsized":"2024-04-16T19:33:41.177717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:01.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:36.174362+0000","last_change":"2024-04-16T17:47:23.261639+0000","last_active":"2024-04-16T19:33:36.174362+0000","last_peered":"2024-04-16T19:33:36.174362+0000","last_clean":"2024-04-16T19:33:36.174362+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:33:36.174362+0000","last_undegraded":"2024-04-16T19:33:36.174362+0000","last_fullsized":"2024-04-16T19:33:36.174362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:40.169520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182698+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:34:11.182698+0000","last_peered":"2024-04-16T19:34:11.182698+0000","last_clean":"2024-04-16T19:34:11.182698+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:34:11.182698+0000","last_undegraded":"2024-04-16T19:34:11.182698+0000","last_fullsized":"2024-04-16T19:34:11.182698+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:09.130950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"48'3","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.161266+0000","last_change":"2024-04-16T17:47:55.181215+0000","last_active":"2024-04-16T19:32:56.161266+0000","last_peered":"2024-04-16T19:32:56.161266+0000","last_clean":"2024-04-16T19:32:56.161266+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:32:56.161266+0000","last_undegraded":"2024-04-16T19:32:56.161266+0000","last_fullsized":"2024-04-16T19:32:56.161266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:27:04.544006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.164727+0000","last_change":"2024-04-16T17:47:23.253450+0000","last_active":"2024-04-16T19:34:11.164727+0000","last_peered":"2024-04-16T19:34:11.164727+0000","last_clean":"2024-04-16T19:34:11.164727+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:34:11.164727+0000","last_undegraded":"2024-04-16T19:34:11.164727+0000","last_fullsized":"2024-04-16T19:34:11.164727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:44.749396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.172146+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:33:31.172146+0000","last_peered":"2024-04-16T19:33:31.172146+0000","last_clean":"2024-04-16T19:33:31.172146+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:33:31.172146+0000","last_undegraded":"2024-04-16T19:33:31.172146+0000","last_fullsized":"2024-04-16T19:33:31.172146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:57.082209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"48'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146820+0000","last_change":"2024-04-16T17:47:55.165525+0000","last_active":"2024-04-16T19:32:56.146820+0000","last_peered":"2024-04-16T19:32:56.146820+0000","last_clean":"2024-04-16T19:32:56.146820+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:32:56.146820+0000","last_undegraded":"2024-04-16T19:32:56.146820+0000","last_fullsized":"2024-04-16T19:32:56.146820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:20:06.245260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171037+0000","last_change":"2024-04-16T17:47:23.257600+0000","last_active":"2024-04-16T19:33:41.171037+0000","last_peered":"2024-04-16T19:33:41.171037+0000","last_clean":"2024-04-16T19:33:41.171037+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:33:41.171037+0000","last_undegraded":"2024-04-16T19:33:41.171037+0000","last_fullsized":"2024-04-16T19:33:41.171037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:41:11.143975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.173036+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:33:31.173036+0000","last_peered":"2024-04-16T19:33:31.173036+0000","last_clean":"2024-04-16T19:33:31.173036+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:33:31.173036+0000","last_undegraded":"2024-04-16T19:33:31.173036+0000","last_fullsized":"2024-04-16T19:33:31.173036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:30:09.609840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'40","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.175919+0000","last_change":"2024-04-16T17:47:55.160972+0000","last_active":"2024-04-16T19:33:41.175919+0000","last_peered":"2024-04-16T19:33:41.175919+0000","last_clean":"2024-04-16T19:33:41.175919+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:33:41.175919+0000","last_undegraded":"2024-04-16T19:33:41.175919+0000","last_fullsized":"2024-04-16T19:33:41.175919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:05.801120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:51.159832+0000","last_change":"2024-04-16T17:47:32.281080+0000","last_active":"2024-04-16T19:33:51.159832+0000","last_peered":"2024-04-16T19:33:51.159832+0000","last_clean":"2024-04-16T19:33:51.159832+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:33:51.159832+0000","last_undegraded":"2024-04-16T19:33:51.159832+0000","last_fullsized":"2024-04-16T19:33:51.159832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:58.918776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.175091+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:33:31.175091+0000","last_peered":"2024-04-16T19:33:31.175091+0000","last_clean":"2024-04-16T19:33:31.175091+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:33:31.175091+0000","last_undegraded":"2024-04-16T19:33:31.175091+0000","last_fullsized":"2024-04-16T19:33:31.175091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.171222+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:33:41.171222+0000","last_peered":"2024-04-16T19:33:41.171222+0000","last_clean":"2024-04-16T19:33:41.171222+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:33:41.171222+0000","last_undegraded":"2024-04-16T19:33:41.171222+0000","last_fullsized":"2024-04-16T19:33:41.171222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:26.938403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177647+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:33:41.177647+0000","last_peered":"2024-04-16T19:33:41.177647+0000","last_clean":"2024-04-16T19:33:41.177647+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:33:41.177647+0000","last_undegraded":"2024-04-16T19:33:41.177647+0000","last_fullsized":"2024-04-16T19:33:41.177647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:45:44.441074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:31.154939+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:33:31.154939+0000","last_peered":"2024-04-16T19:33:31.154939+0000","last_clean":"2024-04-16T19:33:31.154939+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:33:31.154939+0000","last_undegraded":"2024-04-16T19:33:31.154939+0000","last_fullsized":"2024-04-16T19:33:31.154939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:12.925282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'15","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.186677+0000","last_change":"2024-04-16T17:47:55.170559+0000","last_active":"2024-04-16T19:34:11.186677+0000","last_peered":"2024-04-16T19:34:11.186677+0000","last_clean":"2024-04-16T19:34:11.186677+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:34:11.186677+0000","last_undegraded":"2024-04-16T19:34:11.186677+0000","last_fullsized":"2024-04-16T19:34:11.186677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:31:12.308481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.155962+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:32:26.155962+0000","last_peered":"2024-04-16T19:32:26.155962+0000","last_clean":"2024-04-16T19:32:26.155962+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:32:26.155962+0000","last_undegraded":"2024-04-16T19:32:26.155962+0000","last_fullsized":"2024-04-16T19:32:26.155962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:56.176559+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:33:56.176559+0000","last_peered":"2024-04-16T19:33:56.176559+0000","last_clean":"2024-04-16T19:33:56.176559+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:33:56.176559+0000","last_undegraded":"2024-04-16T19:33:56.176559+0000","last_fullsized":"2024-04-16T19:33:56.176559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:44.519947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'18","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:36.142162+0000","last_change":"2024-04-16T17:47:55.177250+0000","last_active":"2024-04-16T19:32:36.142162+0000","last_peered":"2024-04-16T19:32:36.142162+0000","last_clean":"2024-04-16T19:32:36.142162+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:32:36.142162+0000","last_undegraded":"2024-04-16T19:32:36.142162+0000","last_fullsized":"2024-04-16T19:32:36.142162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:10:42.691526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:26.152860+0000","last_change":"2024-04-16T17:47:23.264252+0000","last_active":"2024-04-16T19:32:26.152860+0000","last_peered":"2024-04-16T19:32:26.152860+0000","last_clean":"2024-04-16T19:32:26.152860+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:32:26.152860+0000","last_undegraded":"2024-04-16T19:32:26.152860+0000","last_fullsized":"2024-04-16T19:32:26.152860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:23.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166011+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:33:06.166011+0000","last_peered":"2024-04-16T19:33:06.166011+0000","last_clean":"2024-04-16T19:33:06.166011+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:33:06.166011+0000","last_undegraded":"2024-04-16T19:33:06.166011+0000","last_fullsized":"2024-04-16T19:33:06.166011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:38:11.259679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'1","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.182387+0000","last_change":"2024-04-16T17:47:55.168092+0000","last_active":"2024-04-16T19:34:11.182387+0000","last_peered":"2024-04-16T19:34:11.182387+0000","last_clean":"2024-04-16T19:34:11.182387+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:34:11.182387+0000","last_undegraded":"2024-04-16T19:34:11.182387+0000","last_fullsized":"2024-04-16T19:34:11.182387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:14:57.201825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.157395+0000","last_change":"2024-04-16T17:47:23.265700+0000","last_active":"2024-04-16T19:33:41.157395+0000","last_peered":"2024-04-16T19:33:41.157395+0000","last_clean":"2024-04-16T19:33:41.157395+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:33:41.157395+0000","last_undegraded":"2024-04-16T19:33:41.157395+0000","last_fullsized":"2024-04-16T19:33:41.157395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:24:05.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.163673+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:33:06.163673+0000","last_peered":"2024-04-16T19:33:06.163673+0000","last_clean":"2024-04-16T19:33:06.163673+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:33:06.163673+0000","last_undegraded":"2024-04-16T19:33:06.163673+0000","last_fullsized":"2024-04-16T19:33:06.163673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:29.889012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'18","reported_seq":244,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:11.164620+0000","last_change":"2024-04-16T17:47:55.147785+0000","last_active":"2024-04-16T19:34:11.164620+0000","last_peered":"2024-04-16T19:34:11.164620+0000","last_clean":"2024-04-16T19:34:11.164620+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:34:11.164620+0000","last_undegraded":"2024-04-16T19:34:11.164620+0000","last_fullsized":"2024-04-16T19:34:11.164620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:42:26.488442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.170252+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:33:11.170252+0000","last_peered":"2024-04-16T19:33:11.170252+0000","last_clean":"2024-04-16T19:33:11.170252+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:33:11.170252+0000","last_undegraded":"2024-04-16T19:33:11.170252+0000","last_fullsized":"2024-04-16T19:33:11.170252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:22:46.532972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.166111+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:33:06.166111+0000","last_peered":"2024-04-16T19:33:06.166111+0000","last_clean":"2024-04-16T19:33:06.166111+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:33:06.166111+0000","last_undegraded":"2024-04-16T19:33:06.166111+0000","last_fullsized":"2024-04-16T19:33:06.166111+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:38.156495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'3","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.179697+0000","last_change":"2024-04-16T17:47:55.183714+0000","last_active":"2024-04-16T19:34:06.179697+0000","last_peered":"2024-04-16T19:34:06.179697+0000","last_clean":"2024-04-16T19:34:06.179697+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:34:06.179697+0000","last_undegraded":"2024-04-16T19:34:06.179697+0000","last_fullsized":"2024-04-16T19:34:06.179697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:53:12.557365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.163558+0000","last_change":"2024-04-16T17:47:23.265903+0000","last_active":"2024-04-16T19:33:11.163558+0000","last_peered":"2024-04-16T19:33:11.163558+0000","last_clean":"2024-04-16T19:33:11.163558+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:33:11.163558+0000","last_undegraded":"2024-04-16T19:33:11.163558+0000","last_fullsized":"2024-04-16T19:33:11.163558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:55.605376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:06.168971+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:33:06.168971+0000","last_peered":"2024-04-16T19:33:06.168971+0000","last_clean":"2024-04-16T19:33:06.168971+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:33:06.168971+0000","last_undegraded":"2024-04-16T19:33:06.168971+0000","last_fullsized":"2024-04-16T19:33:06.168971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:13:14.412040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"48'2","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163612+0000","last_change":"2024-04-16T17:47:55.162693+0000","last_active":"2024-04-16T19:34:06.163612+0000","last_peered":"2024-04-16T19:34:06.163612+0000","last_clean":"2024-04-16T19:34:06.163612+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:34:06.163612+0000","last_undegraded":"2024-04-16T19:34:06.163612+0000","last_fullsized":"2024-04-16T19:34:06.163612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:26:14.047560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167227+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:33:11.167227+0000","last_peered":"2024-04-16T19:33:11.167227+0000","last_clean":"2024-04-16T19:33:11.167227+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:33:11.167227+0000","last_undegraded":"2024-04-16T19:33:11.167227+0000","last_fullsized":"2024-04-16T19:33:11.167227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:20:46.026772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:41.177980+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:33:41.177980+0000","last_peered":"2024-04-16T19:33:41.177980+0000","last_clean":"2024-04-16T19:33:41.177980+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:33:41.177980+0000","last_undegraded":"2024-04-16T19:33:41.177980+0000","last_fullsized":"2024-04-16T19:33:41.177980+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:18.842979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'3","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.163408+0000","last_change":"2024-04-16T17:47:55.178234+0000","last_active":"2024-04-16T19:34:06.163408+0000","last_peered":"2024-04-16T19:34:06.163408+0000","last_clean":"2024-04-16T19:34:06.163408+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:34:06.163408+0000","last_undegraded":"2024-04-16T19:34:06.163408+0000","last_fullsized":"2024-04-16T19:34:06.163408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:41:17.561808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:11.167956+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:33:11.167956+0000","last_peered":"2024-04-16T19:33:11.167956+0000","last_clean":"2024-04-16T19:33:11.167956+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:33:11.167956+0000","last_undegraded":"2024-04-16T19:33:11.167956+0000","last_fullsized":"2024-04-16T19:33:11.167956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:47:42.985000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.138794+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:32:21.138794+0000","last_peered":"2024-04-16T19:32:21.138794+0000","last_clean":"2024-04-16T19:32:21.138794+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:32:21.138794+0000","last_undegraded":"2024-04-16T19:32:21.138794+0000","last_fullsized":"2024-04-16T19:32:21.138794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:40.793750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'6","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:06.181429+0000","last_change":"2024-04-16T17:47:55.177975+0000","last_active":"2024-04-16T19:34:06.181429+0000","last_peered":"2024-04-16T19:34:06.181429+0000","last_clean":"2024-04-16T19:34:06.181429+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:34:06.181429+0000","last_undegraded":"2024-04-16T19:34:06.181429+0000","last_fullsized":"2024-04-16T19:34:06.181429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:38:32.181968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:01.151087+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:32:01.151087+0000","last_peered":"2024-04-16T19:32:01.151087+0000","last_clean":"2024-04-16T19:32:01.151087+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:32:01.151087+0000","last_undegraded":"2024-04-16T19:32:01.151087+0000","last_fullsized":"2024-04-16T19:32:01.151087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.152209+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:32:21.152209+0000","last_peered":"2024-04-16T19:32:21.152209+0000","last_clean":"2024-04-16T19:32:21.152209+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:32:21.152209+0000","last_undegraded":"2024-04-16T19:32:21.152209+0000","last_fullsized":"2024-04-16T19:32:21.152209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:53.428558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'10","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.163428+0000","last_change":"2024-04-16T17:47:55.156240+0000","last_active":"2024-04-16T19:32:56.163428+0000","last_peered":"2024-04-16T19:32:56.163428+0000","last_clean":"2024-04-16T19:32:56.163428+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:32:56.163428+0000","last_undegraded":"2024-04-16T19:32:56.163428+0000","last_fullsized":"2024-04-16T19:32:56.163428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:56:34.292049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.156891+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:32:21.156891+0000","last_peered":"2024-04-16T19:32:21.156891+0000","last_clean":"2024-04-16T19:32:21.156891+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:32:21.156891+0000","last_undegraded":"2024-04-16T19:32:21.156891+0000","last_fullsized":"2024-04-16T19:32:21.156891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:49.834654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:31:56.148441+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:31:56.148441+0000","last_peered":"2024-04-16T19:31:56.148441+0000","last_clean":"2024-04-16T19:31:56.148441+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:31:56.148441+0000","last_undegraded":"2024-04-16T19:31:56.148441+0000","last_fullsized":"2024-04-16T19:31:56.148441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:04.544360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'14","reported_seq":179,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:56.146712+0000","last_change":"2024-04-16T17:47:55.164923+0000","last_active":"2024-04-16T19:32:56.146712+0000","last_peered":"2024-04-16T19:32:56.146712+0000","last_clean":"2024-04-16T19:32:56.146712+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:32:56.146712+0000","last_undegraded":"2024-04-16T19:32:56.146712+0000","last_fullsized":"2024-04-16T19:32:56.146712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:59:25.251453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:32:21.155204+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:32:21.155204+0000","last_peered":"2024-04-16T19:32:21.155204+0000","last_clean":"2024-04-16T19:32:21.155204+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:32:21.155204+0000","last_undegraded":"2024-04-16T19:32:21.155204+0000","last_fullsized":"2024-04-16T19:32:21.155204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:56:30.655498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:33:46.174246+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:33:46.174246+0000","last_peered":"2024-04-16T19:33:46.174246+0000","last_clean":"2024-04-16T19:33:46.174246+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:33:46.174246+0000","last_undegraded":"2024-04-16T19:33:46.174246+0000","last_fullsized":"2024-04-16T19:33:46.174246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:36:04.772970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2844,"ondisk_log_size":2844,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3706,"ondisk_log_size":3706,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":623,"ondisk_log_size":623,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27355,"ondisk_log_size":27355,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4460,"ondisk_log_size":4460,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":437,"ondisk_log_size":437,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641576,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46584,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811016,"statfs":{"total":107374182400,"available":107326480384,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641575,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641575,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":99932,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":104757668,"statfs":{"total":107374182400,"available":107271852032,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":100137415},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641575,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46572,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811028,"statfs":{"total":107374182400,"available":107326492672,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641576,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":55408,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":104802192,"statfs":{"total":107374182400,"available":107317444608,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641576,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":104829060,"statfs":{"total":107374182400,"available":107344957440,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:36:19.344 INFO:tasks.ceph:pgid 9.35 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.345 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.345 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.345 INFO:tasks.ceph:pgid 9.34 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.345 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.345 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.345 INFO:tasks.ceph:pgid 9.37 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.346 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.346 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.346 INFO:tasks.ceph:pgid 9.36 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.346 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.346 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.347 INFO:tasks.ceph:pgid 9.31 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.347 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.347 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.347 INFO:tasks.ceph:pgid 9.30 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.347 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.348 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.348 INFO:tasks.ceph:pgid 9.33 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.348 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.348 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.349 INFO:tasks.ceph:pgid 9.32 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.349 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.349 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.349 INFO:tasks.ceph:pgid 9.3d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.349 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.350 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.350 INFO:tasks.ceph:pgid 9.3c last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.350 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.350 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.351 INFO:tasks.ceph:pgid 9.3f last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.351 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.351 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.351 INFO:tasks.ceph:pgid 9.3e last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.351 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.351 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.352 INFO:tasks.ceph:pgid 9.39 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.352 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.352 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.352 INFO:tasks.ceph:pgid 9.38 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.353 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.353 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.353 INFO:tasks.ceph:pgid 15.1f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.353 INFO:tasks.ceph:pgid 14.1e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.353 INFO:tasks.ceph:pgid 13.1d last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.354 INFO:tasks.ceph:pgid 12.1c last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.354 INFO:tasks.ceph:pgid 11.1b last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.354 INFO:tasks.ceph:pgid 10.1a last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.354 INFO:tasks.ceph:pgid 9.19 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.354 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.355 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.355 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.355 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.355 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.355 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.355 INFO:tasks.ceph:pgid 15.1e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.356 INFO:tasks.ceph:pgid 14.1f last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.356 INFO:tasks.ceph:pgid 13.1c last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.356 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.356 INFO:tasks.ceph:pgid 11.1a last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.356 INFO:tasks.ceph:pgid 10.1b last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.357 INFO:tasks.ceph:pgid 9.18 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.357 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.357 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.357 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.357 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.358 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.358 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.358 INFO:tasks.ceph:pgid 15.1d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.358 INFO:tasks.ceph:pgid 14.1c last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.358 INFO:tasks.ceph:pgid 13.1f last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.358 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.359 INFO:tasks.ceph:pgid 11.19 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.359 INFO:tasks.ceph:pgid 10.18 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.359 INFO:tasks.ceph:pgid 9.1b last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.359 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.359 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.360 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.360 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.360 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.360 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.360 INFO:tasks.ceph:pgid 15.1c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.361 INFO:tasks.ceph:pgid 14.1d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.361 INFO:tasks.ceph:pgid 13.1e last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.361 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.361 INFO:tasks.ceph:pgid 11.18 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.361 INFO:tasks.ceph:pgid 10.19 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.361 INFO:tasks.ceph:pgid 9.1a last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.362 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.362 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.362 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.362 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.362 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.363 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.363 INFO:tasks.ceph:pgid 15.3 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.363 INFO:tasks.ceph:pgid 14.2 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.363 INFO:tasks.ceph:pgid 13.1 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.363 INFO:tasks.ceph:pgid 12.0 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.364 INFO:tasks.ceph:pgid 11.7 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.364 INFO:tasks.ceph:pgid 10.6 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.364 INFO:tasks.ceph:pgid 9.5 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.364 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.364 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.365 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.365 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.365 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.365 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.365 INFO:tasks.ceph:pgid 15.2 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.365 INFO:tasks.ceph:pgid 14.3 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.366 INFO:tasks.ceph:pgid 13.0 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.366 INFO:tasks.ceph:pgid 12.1 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.366 INFO:tasks.ceph:pgid 11.6 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.366 INFO:tasks.ceph:pgid 10.7 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.366 INFO:tasks.ceph:pgid 9.4 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.367 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.367 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.367 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.367 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.367 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.368 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.368 INFO:tasks.ceph:pgid 15.1 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.368 INFO:tasks.ceph:pgid 14.0 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.368 INFO:tasks.ceph:pgid 13.3 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.368 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.369 INFO:tasks.ceph:pgid 11.5 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.369 INFO:tasks.ceph:pgid 10.4 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.369 INFO:tasks.ceph:pgid 9.7 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.369 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.369 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.369 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.370 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.370 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.370 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.370 INFO:tasks.ceph:pgid 15.0 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.371 INFO:tasks.ceph:pgid 14.1 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.371 INFO:tasks.ceph:pgid 13.2 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.371 INFO:tasks.ceph:pgid 12.3 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.371 INFO:tasks.ceph:pgid 11.4 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.371 INFO:tasks.ceph:pgid 10.5 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.372 INFO:tasks.ceph:pgid 9.6 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.372 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.372 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.372 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.372 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.373 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.373 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.373 INFO:tasks.ceph:pgid 15.7 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.373 INFO:tasks.ceph:pgid 14.6 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.373 INFO:tasks.ceph:pgid 13.5 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.374 INFO:tasks.ceph:pgid 12.4 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.374 INFO:tasks.ceph:pgid 11.3 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.374 INFO:tasks.ceph:pgid 10.2 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.374 INFO:tasks.ceph:pgid 9.1 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.374 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.375 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.375 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.375 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.375 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.375 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.375 INFO:tasks.ceph:pgid 15.6 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.376 INFO:tasks.ceph:pgid 14.7 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.376 INFO:tasks.ceph:pgid 13.4 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.376 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.376 INFO:tasks.ceph:pgid 11.2 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.376 INFO:tasks.ceph:pgid 10.3 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.377 INFO:tasks.ceph:pgid 9.0 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.377 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.377 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.377 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.377 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.377 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.378 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.378 INFO:tasks.ceph:pgid 15.5 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.378 INFO:tasks.ceph:pgid 14.4 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.378 INFO:tasks.ceph:pgid 13.7 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.378 INFO:tasks.ceph:pgid 12.6 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.379 INFO:tasks.ceph:pgid 11.1 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.379 INFO:tasks.ceph:pgid 10.0 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.379 INFO:tasks.ceph:pgid 9.3 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.379 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.379 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.379 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.380 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.380 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.380 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.380 INFO:tasks.ceph:pgid 15.4 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.380 INFO:tasks.ceph:pgid 14.5 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.381 INFO:tasks.ceph:pgid 13.6 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.381 INFO:tasks.ceph:pgid 12.7 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.381 INFO:tasks.ceph:pgid 11.0 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.381 INFO:tasks.ceph:pgid 10.1 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.381 INFO:tasks.ceph:pgid 9.2 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.382 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.382 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.382 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.382 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.382 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.383 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.383 INFO:tasks.ceph:pgid 15.e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.383 INFO:tasks.ceph:pgid 14.f last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.383 INFO:tasks.ceph:pgid 13.c last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.383 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.384 INFO:tasks.ceph:pgid 11.a last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.384 INFO:tasks.ceph:pgid 10.b last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.384 INFO:tasks.ceph:pgid 9.8 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.384 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.384 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.384 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.385 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.385 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.385 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.385 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.385 INFO:tasks.ceph:pgid 15.f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.386 INFO:tasks.ceph:pgid 14.e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.386 INFO:tasks.ceph:pgid 13.d last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.386 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.386 INFO:tasks.ceph:pgid 11.b last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.386 INFO:tasks.ceph:pgid 10.a last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.387 INFO:tasks.ceph:pgid 9.9 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.387 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.387 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.387 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.387 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.387 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.388 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.388 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.388 INFO:tasks.ceph:pgid 15.c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.388 INFO:tasks.ceph:pgid 14.d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.388 INFO:tasks.ceph:pgid 13.e last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.389 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.389 INFO:tasks.ceph:pgid 11.8 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.389 INFO:tasks.ceph:pgid 10.9 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.389 INFO:tasks.ceph:pgid 9.a last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.389 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.389 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.390 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.390 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.390 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.390 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.390 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.391 INFO:tasks.ceph:pgid 15.d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.391 INFO:tasks.ceph:pgid 14.c last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.391 INFO:tasks.ceph:pgid 13.f last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.391 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.391 INFO:tasks.ceph:pgid 11.9 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.392 INFO:tasks.ceph:pgid 10.8 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.392 INFO:tasks.ceph:pgid 9.b last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.392 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.392 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.392 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-16T17:46:37.490454+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=37, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.392 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.393 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.393 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.393 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.393 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.393 INFO:tasks.ceph:pgid 15.a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.394 INFO:tasks.ceph:pgid 14.b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.394 INFO:tasks.ceph:pgid 13.8 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.394 INFO:tasks.ceph:pgid 12.9 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.394 INFO:tasks.ceph:pgid 11.e last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.394 INFO:tasks.ceph:pgid 10.f last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.395 INFO:tasks.ceph:pgid 9.c last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.395 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.395 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.395 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.395 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.395 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.396 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.396 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.396 INFO:tasks.ceph:pgid 15.b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.396 INFO:tasks.ceph:pgid 14.a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.396 INFO:tasks.ceph:pgid 13.9 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.397 INFO:tasks.ceph:pgid 12.8 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.397 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.397 INFO:tasks.ceph:pgid 10.e last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.397 INFO:tasks.ceph:pgid 9.d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.397 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.397 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.398 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.398 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.398 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.398 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.398 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.399 INFO:tasks.ceph:pgid 15.8 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.399 INFO:tasks.ceph:pgid 14.9 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.399 INFO:tasks.ceph:pgid 13.a last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.399 INFO:tasks.ceph:pgid 12.b last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.399 INFO:tasks.ceph:pgid 11.c last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.400 INFO:tasks.ceph:pgid 10.d last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.400 INFO:tasks.ceph:pgid 9.e last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.400 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.400 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.400 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.400 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.401 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.401 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.401 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.401 INFO:tasks.ceph:pgid 15.9 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.401 INFO:tasks.ceph:pgid 14.8 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.402 INFO:tasks.ceph:pgid 13.b last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.402 INFO:tasks.ceph:pgid 12.a last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.402 INFO:tasks.ceph:pgid 11.d last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.402 INFO:tasks.ceph:pgid 10.c last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.403 INFO:tasks.ceph:pgid 9.f last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.403 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.403 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.403 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.403 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.403 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.404 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.404 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.404 INFO:tasks.ceph:pgid 15.18 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.404 INFO:tasks.ceph:pgid 14.19 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.404 INFO:tasks.ceph:pgid 13.1a last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.405 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.405 INFO:tasks.ceph:pgid 11.1c last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.405 INFO:tasks.ceph:pgid 10.1d last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.405 INFO:tasks.ceph:pgid 9.1e last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.405 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.406 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.406 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.406 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.406 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.406 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.406 INFO:tasks.ceph:pgid 15.19 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.407 INFO:tasks.ceph:pgid 14.18 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.407 INFO:tasks.ceph:pgid 13.1b last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.407 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.407 INFO:tasks.ceph:pgid 11.1d last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.407 INFO:tasks.ceph:pgid 10.1c last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.408 INFO:tasks.ceph:pgid 9.1f last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.408 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.408 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.408 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.408 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.408 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.409 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.409 INFO:tasks.ceph:pgid 15.1a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.409 INFO:tasks.ceph:pgid 14.1b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.409 INFO:tasks.ceph:pgid 13.18 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.409 INFO:tasks.ceph:pgid 12.19 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.410 INFO:tasks.ceph:pgid 11.1e last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.410 INFO:tasks.ceph:pgid 10.1f last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.410 INFO:tasks.ceph:pgid 9.1c last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.410 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.410 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.411 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.411 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.411 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.411 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.411 INFO:tasks.ceph:pgid 15.1b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.412 INFO:tasks.ceph:pgid 14.1a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.412 INFO:tasks.ceph:pgid 13.19 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.412 INFO:tasks.ceph:pgid 12.18 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.412 INFO:tasks.ceph:pgid 11.1f last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.413 INFO:tasks.ceph:pgid 10.1e last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.413 INFO:tasks.ceph:pgid 9.1d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.413 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.413 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.413 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.413 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.414 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.414 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.414 INFO:tasks.ceph:pgid 15.14 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.414 INFO:tasks.ceph:pgid 14.15 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.414 INFO:tasks.ceph:pgid 13.16 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.415 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.415 INFO:tasks.ceph:pgid 11.10 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.415 INFO:tasks.ceph:pgid 10.11 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.415 INFO:tasks.ceph:pgid 9.12 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.415 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.415 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.416 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.416 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.416 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.416 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.416 INFO:tasks.ceph:pgid 15.15 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.417 INFO:tasks.ceph:pgid 14.14 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.417 INFO:tasks.ceph:pgid 13.17 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.417 INFO:tasks.ceph:pgid 12.16 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.417 INFO:tasks.ceph:pgid 11.11 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.417 INFO:tasks.ceph:pgid 10.10 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.418 INFO:tasks.ceph:pgid 9.13 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.418 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.418 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.418 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.418 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.418 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.419 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.419 INFO:tasks.ceph:pgid 15.16 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.419 INFO:tasks.ceph:pgid 14.17 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.419 INFO:tasks.ceph:pgid 13.14 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.419 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.420 INFO:tasks.ceph:pgid 11.12 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.420 INFO:tasks.ceph:pgid 10.13 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.420 INFO:tasks.ceph:pgid 9.10 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.420 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.420 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.420 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.421 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.421 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.421 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.421 INFO:tasks.ceph:pgid 15.17 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.421 INFO:tasks.ceph:pgid 14.16 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.422 INFO:tasks.ceph:pgid 13.15 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.422 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.422 INFO:tasks.ceph:pgid 11.13 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.422 INFO:tasks.ceph:pgid 10.12 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.422 INFO:tasks.ceph:pgid 9.11 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.423 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.423 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.423 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.423 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.423 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.423 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.424 INFO:tasks.ceph:pgid 15.10 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.424 INFO:tasks.ceph:pgid 14.11 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.424 INFO:tasks.ceph:pgid 13.12 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.424 INFO:tasks.ceph:pgid 12.13 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.424 INFO:tasks.ceph:pgid 11.14 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.425 INFO:tasks.ceph:pgid 10.15 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.425 INFO:tasks.ceph:pgid 9.16 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.425 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.425 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.425 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.426 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.426 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.426 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.426 INFO:tasks.ceph:pgid 15.11 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.426 INFO:tasks.ceph:pgid 14.10 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.426 INFO:tasks.ceph:pgid 13.13 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.427 INFO:tasks.ceph:pgid 12.12 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.427 INFO:tasks.ceph:pgid 11.15 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.427 INFO:tasks.ceph:pgid 10.14 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.427 INFO:tasks.ceph:pgid 9.17 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.427 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.428 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.428 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.428 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.428 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.428 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.429 INFO:tasks.ceph:pgid 15.12 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.429 INFO:tasks.ceph:pgid 14.13 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.429 INFO:tasks.ceph:pgid 13.10 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.429 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.429 INFO:tasks.ceph:pgid 11.16 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.429 INFO:tasks.ceph:pgid 10.17 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.430 INFO:tasks.ceph:pgid 9.14 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.430 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.430 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.430 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.430 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.431 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.431 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.431 INFO:tasks.ceph:pgid 15.13 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.431 INFO:tasks.ceph:pgid 14.12 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.431 INFO:tasks.ceph:pgid 13.11 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.431 INFO:tasks.ceph:pgid 12.10 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.432 INFO:tasks.ceph:pgid 11.17 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.432 INFO:tasks.ceph:pgid 10.16 last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.432 INFO:tasks.ceph:pgid 9.15 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.432 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.432 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.433 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.433 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.433 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.433 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.433 INFO:tasks.ceph:pgid 9.2a last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.434 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.434 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.434 INFO:tasks.ceph:pgid 9.2b last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.434 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.434 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.435 INFO:tasks.ceph:pgid 9.28 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.435 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.435 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.435 INFO:tasks.ceph:pgid 9.29 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.435 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.435 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.436 INFO:tasks.ceph:pgid 9.2e last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.436 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.436 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.436 INFO:tasks.ceph:pgid 9.2f last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.436 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.437 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.437 INFO:tasks.ceph:pgid 9.2c last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.437 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.437 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.437 INFO:tasks.ceph:pgid 9.2d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.437 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.438 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.438 INFO:tasks.ceph:pgid 9.22 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.438 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.438 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.438 INFO:tasks.ceph:pgid 9.23 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.439 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.439 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.439 INFO:tasks.ceph:pgid 9.20 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.439 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.440 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.440 INFO:tasks.ceph:pgid 9.21 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.440 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.440 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.440 INFO:tasks.ceph:pgid 9.26 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.440 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.441 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.441 INFO:tasks.ceph:pgid 9.27 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.441 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.441 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.441 INFO:tasks.ceph:pgid 9.24 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.442 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.442 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.442 INFO:tasks.ceph:pgid 9.25 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.442 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.443 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.443 INFO:tasks.ceph:pgid 9.3a last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.443 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.443 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.443 INFO:tasks.ceph:pgid 9.3b last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.444 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.444 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:19.444 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-16T19:36:39.445 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:36:39.834 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:36:39.858 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:36:39.869 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3347,"stamp":"2024-04-16T19:36:38.337100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2422969,"num_omap_keys":10623,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40517,"ondisk_log_size":40517,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":341552,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":331382,"kb_avail":628804048,"statfs":{"total":644245094400,"available":643895345152,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":339335840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":898028,"num_omap_keys":3872,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010484"},"pg_stats":[{"pgid":"9.35","version":"53'6","reported_seq":151,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.507712+0000","last_change":"2024-04-16T19:36:20.507786+0000","last_active":"2024-04-16T19:36:20.507712+0000","last_peered":"2024-04-16T19:36:20.507712+0000","last_clean":"2024-04-16T19:36:20.507712+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:36:20.507712+0000","last_undegraded":"2024-04-16T19:36:20.507712+0000","last_fullsized":"2024-04-16T19:36:20.507712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.507587+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:21:12.018020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209498+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:35:41.209498+0000","last_peered":"2024-04-16T19:35:41.209498+0000","last_clean":"2024-04-16T19:35:41.209498+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:35:41.209498+0000","last_undegraded":"2024-04-16T19:35:41.209498+0000","last_fullsized":"2024-04-16T19:35:41.209498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192292+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:34:56.192292+0000","last_peered":"2024-04-16T19:34:56.192292+0000","last_clean":"2024-04-16T19:34:56.192292+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:34:56.192292+0000","last_undegraded":"2024-04-16T19:34:56.192292+0000","last_fullsized":"2024-04-16T19:34:56.192292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'14","reported_seq":292,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:16.182406+0000","last_change":"2024-04-16T17:47:55.173885+0000","last_active":"2024-04-16T19:34:16.182406+0000","last_peered":"2024-04-16T19:34:16.182406+0000","last_clean":"2024-04-16T19:34:16.182406+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:34:16.182406+0000","last_undegraded":"2024-04-16T19:34:16.182406+0000","last_fullsized":"2024-04-16T19:34:16.182406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206637+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:35:41.206637+0000","last_peered":"2024-04-16T19:35:41.206637+0000","last_clean":"2024-04-16T19:35:41.206637+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:35:41.206637+0000","last_undegraded":"2024-04-16T19:35:41.206637+0000","last_fullsized":"2024-04-16T19:35:41.206637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.197906+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:34:56.197906+0000","last_peered":"2024-04-16T19:34:56.197906+0000","last_clean":"2024-04-16T19:34:56.197906+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:34:56.197906+0000","last_undegraded":"2024-04-16T19:34:56.197906+0000","last_fullsized":"2024-04-16T19:34:56.197906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'20","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188695+0000","last_change":"2024-04-16T17:47:55.156902+0000","last_active":"2024-04-16T19:34:51.188695+0000","last_peered":"2024-04-16T19:34:51.188695+0000","last_clean":"2024-04-16T19:34:51.188695+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:34:51.188695+0000","last_undegraded":"2024-04-16T19:34:51.188695+0000","last_fullsized":"2024-04-16T19:34:51.188695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205312+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:35:41.205312+0000","last_peered":"2024-04-16T19:35:41.205312+0000","last_clean":"2024-04-16T19:35:41.205312+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:35:41.205312+0000","last_undegraded":"2024-04-16T19:35:41.205312+0000","last_fullsized":"2024-04-16T19:35:41.205312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189744+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:34:56.189744+0000","last_peered":"2024-04-16T19:34:56.189744+0000","last_clean":"2024-04-16T19:34:56.189744+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:34:56.189744+0000","last_undegraded":"2024-04-16T19:34:56.189744+0000","last_fullsized":"2024-04-16T19:34:56.189744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:16.185187+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:34:16.185187+0000","last_peered":"2024-04-16T19:34:16.185187+0000","last_clean":"2024-04-16T19:34:16.185187+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:34:16.185187+0000","last_undegraded":"2024-04-16T19:34:16.185187+0000","last_fullsized":"2024-04-16T19:34:16.185187+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.205993+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:35:46.205993+0000","last_peered":"2024-04-16T19:35:46.205993+0000","last_clean":"2024-04-16T19:35:46.205993+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:35:46.205993+0000","last_undegraded":"2024-04-16T19:35:46.205993+0000","last_fullsized":"2024-04-16T19:35:46.205993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193915+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:34:56.193915+0000","last_peered":"2024-04-16T19:34:56.193915+0000","last_clean":"2024-04-16T19:34:56.193915+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:34:56.193915+0000","last_undegraded":"2024-04-16T19:34:56.193915+0000","last_fullsized":"2024-04-16T19:34:56.193915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'8","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.663272+0000","last_change":"2024-04-16T19:36:31.663358+0000","last_active":"2024-04-16T19:36:31.663272+0000","last_peered":"2024-04-16T19:36:31.663272+0000","last_clean":"2024-04-16T19:36:31.663272+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:36:31.663272+0000","last_undegraded":"2024-04-16T19:36:31.663272+0000","last_fullsized":"2024-04-16T19:36:31.663272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.663140+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:09.078400+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197555+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:35:06.197555+0000","last_peered":"2024-04-16T19:35:06.197555+0000","last_clean":"2024-04-16T19:35:06.197555+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:35:06.197555+0000","last_undegraded":"2024-04-16T19:35:06.197555+0000","last_fullsized":"2024-04-16T19:35:06.197555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207378+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:35:51.207378+0000","last_peered":"2024-04-16T19:35:51.207378+0000","last_clean":"2024-04-16T19:35:51.207378+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:35:51.207378+0000","last_undegraded":"2024-04-16T19:35:51.207378+0000","last_fullsized":"2024-04-16T19:35:51.207378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'20","reported_seq":327,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.492651+0000","last_change":"2024-04-16T19:36:26.492760+0000","last_active":"2024-04-16T19:36:26.492651+0000","last_peered":"2024-04-16T19:36:26.492651+0000","last_clean":"2024-04-16T19:36:26.492651+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:36:26.492651+0000","last_undegraded":"2024-04-16T19:36:26.492651+0000","last_fullsized":"2024-04-16T19:36:26.492651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.492498+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:56.598246+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191143+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:34:51.191143+0000","last_peered":"2024-04-16T19:34:51.191143+0000","last_clean":"2024-04-16T19:34:51.191143+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:34:51.191143+0000","last_undegraded":"2024-04-16T19:34:51.191143+0000","last_fullsized":"2024-04-16T19:34:51.191143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202288+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:35:46.202288+0000","last_peered":"2024-04-16T19:35:46.202288+0000","last_clean":"2024-04-16T19:35:46.202288+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:35:46.202288+0000","last_undegraded":"2024-04-16T19:35:46.202288+0000","last_fullsized":"2024-04-16T19:35:46.202288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'19","reported_seq":284,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.411552+0000","last_change":"2024-04-16T19:36:31.411649+0000","last_active":"2024-04-16T19:36:31.411552+0000","last_peered":"2024-04-16T19:36:31.411552+0000","last_clean":"2024-04-16T19:36:31.411552+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:36:31.411552+0000","last_undegraded":"2024-04-16T19:36:31.411552+0000","last_fullsized":"2024-04-16T19:36:31.411552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.411418+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:18.775822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.196650+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:34:51.196650+0000","last_peered":"2024-04-16T19:34:51.196650+0000","last_clean":"2024-04-16T19:34:51.196650+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:34:51.196650+0000","last_undegraded":"2024-04-16T19:34:51.196650+0000","last_fullsized":"2024-04-16T19:34:51.196650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206153+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:35:46.206153+0000","last_peered":"2024-04-16T19:35:46.206153+0000","last_clean":"2024-04-16T19:35:46.206153+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:35:46.206153+0000","last_undegraded":"2024-04-16T19:35:46.206153+0000","last_fullsized":"2024-04-16T19:35:46.206153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'14","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.657838+0000","last_change":"2024-04-16T19:36:29.657923+0000","last_active":"2024-04-16T19:36:29.657838+0000","last_peered":"2024-04-16T19:36:29.657838+0000","last_clean":"2024-04-16T19:36:29.657838+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:36:29.657838+0000","last_undegraded":"2024-04-16T19:36:29.657838+0000","last_fullsized":"2024-04-16T19:36:29.657838+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.657764+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:42.577097+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.174105+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:34:51.174105+0000","last_peered":"2024-04-16T19:34:51.174105+0000","last_clean":"2024-04-16T19:34:51.174105+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:34:51.174105+0000","last_undegraded":"2024-04-16T19:34:51.174105+0000","last_fullsized":"2024-04-16T19:34:51.174105+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.187294+0000","last_change":"2024-04-16T17:47:23.254199+0000","last_active":"2024-04-16T19:35:46.187294+0000","last_peered":"2024-04-16T19:35:46.187294+0000","last_clean":"2024-04-16T19:35:46.187294+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:35:46.187294+0000","last_undegraded":"2024-04-16T19:35:46.187294+0000","last_fullsized":"2024-04-16T19:35:46.187294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"53'10","reported_seq":315,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.620843+0000","last_change":"2024-04-16T19:36:21.620914+0000","last_active":"2024-04-16T19:36:21.620843+0000","last_peered":"2024-04-16T19:36:21.620843+0000","last_clean":"2024-04-16T19:36:21.620843+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:36:21.620843+0000","last_undegraded":"2024-04-16T19:36:21.620843+0000","last_fullsized":"2024-04-16T19:36:21.620843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.620737+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:28.047519+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.186990+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:34:46.186990+0000","last_peered":"2024-04-16T19:34:46.186990+0000","last_clean":"2024-04-16T19:34:46.186990+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:34:46.186990+0000","last_undegraded":"2024-04-16T19:34:46.186990+0000","last_fullsized":"2024-04-16T19:34:46.186990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:26.187940+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:34:26.187940+0000","last_peered":"2024-04-16T19:34:26.187940+0000","last_clean":"2024-04-16T19:34:26.187940+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:34:26.187940+0000","last_undegraded":"2024-04-16T19:34:26.187940+0000","last_fullsized":"2024-04-16T19:34:26.187940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'18","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.400030+0000","last_change":"2024-04-16T19:36:26.400030+0000","last_active":"2024-04-16T19:36:26.400030+0000","last_peered":"2024-04-16T19:36:26.400030+0000","last_clean":"2024-04-16T19:36:26.400030+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:36:26.400030+0000","last_undegraded":"2024-04-16T19:36:26.400030+0000","last_fullsized":"2024-04-16T19:36:26.400030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.399732+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:00.453174+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.189884+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:34:46.189884+0000","last_peered":"2024-04-16T19:34:46.189884+0000","last_clean":"2024-04-16T19:34:46.189884+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:34:46.189884+0000","last_undegraded":"2024-04-16T19:34:46.189884+0000","last_fullsized":"2024-04-16T19:34:46.189884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215736+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:36:06.215736+0000","last_peered":"2024-04-16T19:36:06.215736+0000","last_clean":"2024-04-16T19:36:06.215736+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:36:06.215736+0000","last_undegraded":"2024-04-16T19:36:06.215736+0000","last_fullsized":"2024-04-16T19:36:06.215736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'8","reported_seq":272,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194482+0000","last_change":"2024-04-16T17:47:55.163584+0000","last_active":"2024-04-16T19:36:16.194482+0000","last_peered":"2024-04-16T19:36:16.194482+0000","last_clean":"2024-04-16T19:36:16.194482+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:36:16.194482+0000","last_undegraded":"2024-04-16T19:36:16.194482+0000","last_fullsized":"2024-04-16T19:36:16.194482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.196138+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:35:11.196138+0000","last_peered":"2024-04-16T19:35:11.196138+0000","last_clean":"2024-04-16T19:35:11.196138+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:35:11.196138+0000","last_undegraded":"2024-04-16T19:35:11.196138+0000","last_fullsized":"2024-04-16T19:35:11.196138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211293+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:36:06.211293+0000","last_peered":"2024-04-16T19:36:06.211293+0000","last_clean":"2024-04-16T19:36:06.211293+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:36:06.211293+0000","last_undegraded":"2024-04-16T19:36:06.211293+0000","last_fullsized":"2024-04-16T19:36:06.211293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"53'9","reported_seq":241,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.533226+0000","last_change":"2024-04-16T19:36:29.533377+0000","last_active":"2024-04-16T19:36:29.533226+0000","last_peered":"2024-04-16T19:36:29.533226+0000","last_clean":"2024-04-16T19:36:29.533226+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:36:29.533226+0000","last_undegraded":"2024-04-16T19:36:29.533226+0000","last_fullsized":"2024-04-16T19:36:29.533226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.533050+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:35:44.595573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.193282+0000","last_change":"2024-04-16T17:47:23.254226+0000","last_active":"2024-04-16T19:36:11.193282+0000","last_peered":"2024-04-16T19:36:11.193282+0000","last_clean":"2024-04-16T19:36:11.193282+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:36:11.193282+0000","last_undegraded":"2024-04-16T19:36:11.193282+0000","last_fullsized":"2024-04-16T19:36:11.193282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.207335+0000","last_change":"2024-04-16T17:47:32.284109+0000","last_active":"2024-04-16T19:36:06.207335+0000","last_peered":"2024-04-16T19:36:06.207335+0000","last_clean":"2024-04-16T19:36:06.207335+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:36:06.207335+0000","last_undegraded":"2024-04-16T19:36:06.207335+0000","last_fullsized":"2024-04-16T19:36:06.207335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'10","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498172+0000","last_change":"2024-04-16T19:36:19.498276+0000","last_active":"2024-04-16T19:36:19.498172+0000","last_peered":"2024-04-16T19:36:19.498172+0000","last_clean":"2024-04-16T19:36:19.498172+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:36:19.498172+0000","last_undegraded":"2024-04-16T19:36:19.498172+0000","last_fullsized":"2024-04-16T19:36:19.498172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498013+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:24.571217+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.193367+0000","last_change":"2024-04-16T17:47:23.266132+0000","last_active":"2024-04-16T19:36:11.193367+0000","last_peered":"2024-04-16T19:36:11.193367+0000","last_clean":"2024-04-16T19:36:11.193367+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:36:11.193367+0000","last_undegraded":"2024-04-16T19:36:11.193367+0000","last_fullsized":"2024-04-16T19:36:11.193367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048603+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:36:17.048603+0000","last_peered":"2024-04-16T19:36:17.048603+0000","last_clean":"2024-04-16T19:36:17.048603+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:36:17.048603+0000","last_undegraded":"2024-04-16T19:36:17.048603+0000","last_fullsized":"2024-04-16T19:36:17.048603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'10","reported_seq":266,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.182193+0000","last_change":"2024-04-16T17:47:55.158785+0000","last_active":"2024-04-16T19:35:26.182193+0000","last_peered":"2024-04-16T19:35:26.182193+0000","last_clean":"2024-04-16T19:35:26.182193+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:35:26.182193+0000","last_undegraded":"2024-04-16T19:35:26.182193+0000","last_fullsized":"2024-04-16T19:35:26.182193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.208583+0000","last_change":"2024-04-16T17:47:23.257289+0000","last_active":"2024-04-16T19:36:11.208583+0000","last_peered":"2024-04-16T19:36:11.208583+0000","last_clean":"2024-04-16T19:36:11.208583+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:36:11.208583+0000","last_undegraded":"2024-04-16T19:36:11.208583+0000","last_fullsized":"2024-04-16T19:36:11.208583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.198747+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:35:16.198747+0000","last_peered":"2024-04-16T19:35:16.198747+0000","last_clean":"2024-04-16T19:35:16.198747+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:35:16.198747+0000","last_undegraded":"2024-04-16T19:35:16.198747+0000","last_fullsized":"2024-04-16T19:35:16.198747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215769+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:36:16.215769+0000","last_peered":"2024-04-16T19:36:16.215769+0000","last_clean":"2024-04-16T19:36:16.215769+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:36:16.215769+0000","last_undegraded":"2024-04-16T19:36:16.215769+0000","last_fullsized":"2024-04-16T19:36:16.215769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1351,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.201950+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:35:11.201950+0000","last_peered":"2024-04-16T19:35:11.201950+0000","last_clean":"2024-04-16T19:35:11.201950+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:35:11.201950+0000","last_undegraded":"2024-04-16T19:35:11.201950+0000","last_fullsized":"2024-04-16T19:35:11.201950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.214384+0000","last_change":"2024-04-16T17:48:27.308777+0000","last_active":"2024-04-16T19:36:01.214384+0000","last_peered":"2024-04-16T19:36:01.214384+0000","last_clean":"2024-04-16T19:36:01.214384+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:36:01.214384+0000","last_undegraded":"2024-04-16T19:36:01.214384+0000","last_fullsized":"2024-04-16T19:36:01.214384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'9","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.202926+0000","last_change":"2024-04-16T17:48:01.266271+0000","last_active":"2024-04-16T19:35:31.202926+0000","last_peered":"2024-04-16T19:35:31.202926+0000","last_clean":"2024-04-16T19:35:31.202926+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:35:31.202926+0000","last_undegraded":"2024-04-16T19:35:31.202926+0000","last_fullsized":"2024-04-16T19:35:31.202926+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.617380+0000","last_change":"2024-04-16T19:36:26.617439+0000","last_active":"2024-04-16T19:36:26.617380+0000","last_peered":"2024-04-16T19:36:26.617380+0000","last_clean":"2024-04-16T19:36:26.617380+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:36:26.617380+0000","last_undegraded":"2024-04-16T19:36:26.617380+0000","last_fullsized":"2024-04-16T19:36:26.617380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.617211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:50.470545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'563","reported_seq":824,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.471001+0000","last_change":"2024-04-16T19:36:20.471131+0000","last_active":"2024-04-16T19:36:20.471001+0000","last_peered":"2024-04-16T19:36:20.471001+0000","last_clean":"2024-04-16T19:36:20.471001+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:36:20.471001+0000","last_undegraded":"2024-04-16T19:36:20.471001+0000","last_fullsized":"2024-04-16T19:36:20.471001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.470861+0000","objects_scrubbed":16,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:34.399674+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71932,"num_omap_keys":314,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'14","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.561685+0000","last_change":"2024-04-16T19:36:25.561790+0000","last_active":"2024-04-16T19:36:25.561685+0000","last_peered":"2024-04-16T19:36:25.561685+0000","last_clean":"2024-04-16T19:36:25.561685+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:36:25.561685+0000","last_undegraded":"2024-04-16T19:36:25.561685+0000","last_fullsized":"2024-04-16T19:36:25.561685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.561550+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:02:57.257953+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438495+0000","last_change":"2024-04-16T17:47:39.326260+0000","last_active":"2024-04-16T19:36:18.438495+0000","last_peered":"2024-04-16T19:36:18.438495+0000","last_clean":"2024-04-16T19:36:18.438495+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:36:18.438495+0000","last_undegraded":"2024-04-16T19:36:18.438495+0000","last_fullsized":"2024-04-16T19:36:18.438495+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.200023+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:35:16.200023+0000","last_peered":"2024-04-16T19:35:16.200023+0000","last_clean":"2024-04-16T19:35:16.200023+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:35:16.200023+0000","last_undegraded":"2024-04-16T19:35:16.200023+0000","last_fullsized":"2024-04-16T19:35:16.200023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194993+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:34:56.194993+0000","last_peered":"2024-04-16T19:34:56.194993+0000","last_clean":"2024-04-16T19:34:56.194993+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:34:56.194993+0000","last_undegraded":"2024-04-16T19:34:56.194993+0000","last_fullsized":"2024-04-16T19:34:56.194993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":770,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912468+0000","last_change":"2024-04-16T17:47:33.303155+0000","last_active":"2024-04-16T19:36:18.912468+0000","last_peered":"2024-04-16T19:36:18.912468+0000","last_clean":"2024-04-16T19:36:18.912468+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:36:18.912468+0000","last_undegraded":"2024-04-16T19:36:18.912468+0000","last_fullsized":"2024-04-16T19:36:18.912468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586459+0000","last_change":"2024-04-16T17:47:35.314855+0000","last_active":"2024-04-16T19:36:16.586459+0000","last_peered":"2024-04-16T19:36:16.586459+0000","last_clean":"2024-04-16T19:36:16.586459+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:36:16.586459+0000","last_undegraded":"2024-04-16T19:36:16.586459+0000","last_fullsized":"2024-04-16T19:36:16.586459+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912762+0000","last_change":"2024-04-16T17:47:37.332945+0000","last_active":"2024-04-16T19:36:18.912762+0000","last_peered":"2024-04-16T19:36:18.912762+0000","last_clean":"2024-04-16T19:36:18.912762+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:36:18.912762+0000","last_undegraded":"2024-04-16T19:36:18.912762+0000","last_fullsized":"2024-04-16T19:36:18.912762+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181269+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:35:21.181269+0000","last_peered":"2024-04-16T19:35:21.181269+0000","last_clean":"2024-04-16T19:35:21.181269+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:35:21.181269+0000","last_undegraded":"2024-04-16T19:35:21.181269+0000","last_fullsized":"2024-04-16T19:35:21.181269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'83","reported_seq":1421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192768+0000","last_change":"2024-04-16T17:48:34.319654+0000","last_active":"2024-04-16T19:34:51.192768+0000","last_peered":"2024-04-16T19:34:51.192768+0000","last_clean":"2024-04-16T19:34:51.192768+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:34:51.192768+0000","last_undegraded":"2024-04-16T19:34:51.192768+0000","last_fullsized":"2024-04-16T19:34:51.192768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209948+0000","last_change":"2024-04-16T17:48:27.303999+0000","last_active":"2024-04-16T19:36:01.209948+0000","last_peered":"2024-04-16T19:36:01.209948+0000","last_clean":"2024-04-16T19:36:01.209948+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:36:01.209948+0000","last_undegraded":"2024-04-16T19:36:01.209948+0000","last_fullsized":"2024-04-16T19:36:01.209948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'5","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.202968+0000","last_change":"2024-04-16T17:48:01.266699+0000","last_active":"2024-04-16T19:35:31.202968+0000","last_peered":"2024-04-16T19:35:31.202968+0000","last_clean":"2024-04-16T19:35:31.202968+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:35:31.202968+0000","last_undegraded":"2024-04-16T19:35:31.202968+0000","last_fullsized":"2024-04-16T19:35:31.202968+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.479488+0000","last_change":"2024-04-16T19:36:33.479590+0000","last_active":"2024-04-16T19:36:33.479488+0000","last_peered":"2024-04-16T19:36:33.479488+0000","last_clean":"2024-04-16T19:36:33.479488+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:36:33.479488+0000","last_undegraded":"2024-04-16T19:36:33.479488+0000","last_fullsized":"2024-04-16T19:36:33.479488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.479280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:24.835481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7349","reported_seq":20627,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.602403+0000","last_change":"2024-04-16T19:36:20.602529+0000","last_active":"2024-04-16T19:36:20.602403+0000","last_peered":"2024-04-16T19:36:20.602403+0000","last_clean":"2024-04-16T19:36:20.602403+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:36:20.602403+0000","last_undegraded":"2024-04-16T19:36:20.602403+0000","last_fullsized":"2024-04-16T19:36:20.602403+0000","mapping_epoch":34,"log_start":"53'5600","ondisk_log_start":"53'5600","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7349","last_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_deep_scrub":"53'7349","last_deep_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.602264+0000","objects_scrubbed":23,"log_size":1749,"log_dups_size":3000,"ondisk_log_size":1749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:12.702919+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72188,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'30","reported_seq":374,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.493430+0000","last_change":"2024-04-16T19:36:32.493531+0000","last_active":"2024-04-16T19:36:32.493430+0000","last_peered":"2024-04-16T19:36:32.493430+0000","last_clean":"2024-04-16T19:36:32.493430+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:36:32.493430+0000","last_undegraded":"2024-04-16T19:36:32.493430+0000","last_fullsized":"2024-04-16T19:36:32.493430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'30","last_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_deep_scrub":"53'30","last_deep_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.493294+0000","objects_scrubbed":11,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:58.339768+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:26.184791+0000","last_change":"2024-04-16T17:47:39.333211+0000","last_active":"2024-04-16T19:34:26.184791+0000","last_peered":"2024-04-16T19:34:26.184791+0000","last_clean":"2024-04-16T19:34:26.184791+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:34:26.184791+0000","last_undegraded":"2024-04-16T19:34:26.184791+0000","last_fullsized":"2024-04-16T19:34:26.184791+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212569+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:36:11.212569+0000","last_peered":"2024-04-16T19:36:11.212569+0000","last_clean":"2024-04-16T19:36:11.212569+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:36:11.212569+0000","last_undegraded":"2024-04-16T19:36:11.212569+0000","last_fullsized":"2024-04-16T19:36:11.212569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.190022+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:34:41.190022+0000","last_peered":"2024-04-16T19:34:41.190022+0000","last_clean":"2024-04-16T19:34:41.190022+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:34:41.190022+0000","last_undegraded":"2024-04-16T19:34:41.190022+0000","last_fullsized":"2024-04-16T19:34:41.190022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"48'5","reported_seq":360,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.202876+0000","last_change":"2024-04-16T17:47:33.310538+0000","last_active":"2024-04-16T19:35:31.202876+0000","last_peered":"2024-04-16T19:35:31.202876+0000","last_clean":"2024-04-16T19:35:31.202876+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:35:31.202876+0000","last_undegraded":"2024-04-16T19:35:31.202876+0000","last_fullsized":"2024-04-16T19:35:31.202876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'72","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438215+0000","last_change":"2024-04-16T17:47:35.305717+0000","last_active":"2024-04-16T19:36:18.438215+0000","last_peered":"2024-04-16T19:36:18.438215+0000","last_clean":"2024-04-16T19:36:18.438215+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:36:18.438215+0000","last_undegraded":"2024-04-16T19:36:18.438215+0000","last_fullsized":"2024-04-16T19:36:18.438215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438355+0000","last_change":"2024-04-16T17:47:37.326918+0000","last_active":"2024-04-16T19:36:18.438355+0000","last_peered":"2024-04-16T19:36:18.438355+0000","last_clean":"2024-04-16T19:36:18.438355+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:36:18.438355+0000","last_undegraded":"2024-04-16T19:36:18.438355+0000","last_fullsized":"2024-04-16T19:36:18.438355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.203492+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:35:51.203492+0000","last_peered":"2024-04-16T19:35:51.203492+0000","last_clean":"2024-04-16T19:35:51.203492+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:35:51.203492+0000","last_undegraded":"2024-04-16T19:35:51.203492+0000","last_fullsized":"2024-04-16T19:35:51.203492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'56","reported_seq":877,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211394+0000","last_change":"2024-04-16T17:48:34.333949+0000","last_active":"2024-04-16T19:36:06.211394+0000","last_peered":"2024-04-16T19:36:06.211394+0000","last_clean":"2024-04-16T19:36:06.211394+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:36:06.211394+0000","last_undegraded":"2024-04-16T19:36:06.211394+0000","last_fullsized":"2024-04-16T19:36:06.211394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":92,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.671249+0000","last_change":"2024-04-16T19:36:34.671347+0000","last_active":"2024-04-16T19:36:34.671249+0000","last_peered":"2024-04-16T19:36:34.671249+0000","last_clean":"2024-04-16T19:36:34.671249+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:36:34.671249+0000","last_undegraded":"2024-04-16T19:36:34.671249+0000","last_fullsized":"2024-04-16T19:36:34.671249+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.671050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:41.097832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'16","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.439453+0000","last_change":"2024-04-16T17:48:01.239365+0000","last_active":"2024-04-16T19:36:18.439453+0000","last_peered":"2024-04-16T19:36:18.439453+0000","last_clean":"2024-04-16T19:36:18.439453+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:36:18.439453+0000","last_undegraded":"2024-04-16T19:36:18.439453+0000","last_fullsized":"2024-04-16T19:36:18.439453+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193787+0000","last_change":"2024-04-16T17:47:59.192949+0000","last_active":"2024-04-16T19:34:56.193787+0000","last_peered":"2024-04-16T19:34:56.193787+0000","last_clean":"2024-04-16T19:34:56.193787+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:34:56.193787+0000","last_undegraded":"2024-04-16T19:34:56.193787+0000","last_fullsized":"2024-04-16T19:34:56.193787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3625","reported_seq":7229,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.651379+0000","last_change":"2024-04-16T19:36:17.651412+0000","last_active":"2024-04-16T19:36:17.651379+0000","last_peered":"2024-04-16T19:36:17.651379+0000","last_clean":"2024-04-16T19:36:17.651379+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:36:17.651379+0000","last_undegraded":"2024-04-16T19:36:17.651379+0000","last_fullsized":"2024-04-16T19:36:17.651379+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'3625","last_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_deep_scrub":"53'3625","last_deep_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.651328+0000","objects_scrubbed":30,"log_size":1825,"log_dups_size":1800,"ondisk_log_size":1825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:26.898814+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'18","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.465484+0000","last_change":"2024-04-16T19:36:17.465591+0000","last_active":"2024-04-16T19:36:17.465484+0000","last_peered":"2024-04-16T19:36:17.465484+0000","last_clean":"2024-04-16T19:36:17.465484+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:36:17.465484+0000","last_undegraded":"2024-04-16T19:36:17.465484+0000","last_fullsized":"2024-04-16T19:36:17.465484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.465360+0000","objects_scrubbed":9,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:23.892141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.197496+0000","last_change":"2024-04-16T17:47:39.334187+0000","last_active":"2024-04-16T19:35:11.197496+0000","last_peered":"2024-04-16T19:35:11.197496+0000","last_clean":"2024-04-16T19:35:11.197496+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:35:11.197496+0000","last_undegraded":"2024-04-16T19:35:11.197496+0000","last_fullsized":"2024-04-16T19:35:11.197496+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212749+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:36:11.212749+0000","last_peered":"2024-04-16T19:36:11.212749+0000","last_clean":"2024-04-16T19:36:11.212749+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:36:11.212749+0000","last_undegraded":"2024-04-16T19:36:11.212749+0000","last_fullsized":"2024-04-16T19:36:11.212749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194212+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:34:41.194212+0000","last_peered":"2024-04-16T19:34:41.194212+0000","last_clean":"2024-04-16T19:34:41.194212+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:34:41.194212+0000","last_undegraded":"2024-04-16T19:34:41.194212+0000","last_fullsized":"2024-04-16T19:34:41.194212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"48'11","reported_seq":607,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528035+0000","last_change":"2024-04-16T17:47:33.300240+0000","last_active":"2024-04-16T19:36:17.528035+0000","last_peered":"2024-04-16T19:36:17.528035+0000","last_clean":"2024-04-16T19:36:17.528035+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:36:17.528035+0000","last_undegraded":"2024-04-16T19:36:17.528035+0000","last_fullsized":"2024-04-16T19:36:17.528035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'120","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.182439+0000","last_change":"2024-04-16T17:47:35.314675+0000","last_active":"2024-04-16T19:35:26.182439+0000","last_peered":"2024-04-16T19:35:26.182439+0000","last_clean":"2024-04-16T19:35:26.182439+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:35:26.182439+0000","last_undegraded":"2024-04-16T19:35:26.182439+0000","last_fullsized":"2024-04-16T19:35:26.182439+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'12","reported_seq":1334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978616+0000","last_change":"2024-04-16T17:47:37.333045+0000","last_active":"2024-04-16T19:36:17.978616+0000","last_peered":"2024-04-16T19:36:17.978616+0000","last_clean":"2024-04-16T19:36:17.978616+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:36:17.978616+0000","last_undegraded":"2024-04-16T19:36:17.978616+0000","last_fullsized":"2024-04-16T19:36:17.978616+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209444+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:35:51.209444+0000","last_peered":"2024-04-16T19:35:51.209444+0000","last_clean":"2024-04-16T19:35:51.209444+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:35:51.209444+0000","last_undegraded":"2024-04-16T19:35:51.209444+0000","last_fullsized":"2024-04-16T19:35:51.209444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":737,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211512+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:36:06.211512+0000","last_peered":"2024-04-16T19:36:06.211512+0000","last_clean":"2024-04-16T19:36:06.211512+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:36:06.211512+0000","last_undegraded":"2024-04-16T19:36:06.211512+0000","last_fullsized":"2024-04-16T19:36:06.211512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.671404+0000","last_change":"2024-04-16T19:36:35.671455+0000","last_active":"2024-04-16T19:36:35.671404+0000","last_peered":"2024-04-16T19:36:35.671404+0000","last_clean":"2024-04-16T19:36:35.671404+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:36:35.671404+0000","last_undegraded":"2024-04-16T19:36:35.671404+0000","last_fullsized":"2024-04-16T19:36:35.671404+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.671258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:03.249930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'19","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980217+0000","last_change":"2024-04-16T17:48:01.256079+0000","last_active":"2024-04-16T19:36:17.980217+0000","last_peered":"2024-04-16T19:36:17.980217+0000","last_clean":"2024-04-16T19:36:17.980217+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:36:17.980217+0000","last_undegraded":"2024-04-16T19:36:17.980217+0000","last_fullsized":"2024-04-16T19:36:17.980217+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.166880+0000","last_change":"2024-04-16T17:47:59.183129+0000","last_active":"2024-04-16T19:34:21.166880+0000","last_peered":"2024-04-16T19:34:21.166880+0000","last_clean":"2024-04-16T19:34:21.166880+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:34:21.166880+0000","last_undegraded":"2024-04-16T19:34:21.166880+0000","last_fullsized":"2024-04-16T19:34:21.166880+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'408","reported_seq":768,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.522616+0000","last_change":"2024-04-16T19:36:31.522794+0000","last_active":"2024-04-16T19:36:31.522616+0000","last_peered":"2024-04-16T19:36:31.522616+0000","last_clean":"2024-04-16T19:36:31.522616+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:36:31.522616+0000","last_undegraded":"2024-04-16T19:36:31.522616+0000","last_fullsized":"2024-04-16T19:36:31.522616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'408","last_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_deep_scrub":"53'408","last_deep_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.522481+0000","objects_scrubbed":23,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:59.101153+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'12","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.541977+0000","last_change":"2024-04-16T19:36:17.542045+0000","last_active":"2024-04-16T19:36:17.541977+0000","last_peered":"2024-04-16T19:36:17.541977+0000","last_clean":"2024-04-16T19:36:17.541977+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:36:17.541977+0000","last_undegraded":"2024-04-16T19:36:17.541977+0000","last_fullsized":"2024-04-16T19:36:17.541977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.541913+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:24.087701+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.193431+0000","last_change":"2024-04-16T17:47:39.699810+0000","last_active":"2024-04-16T19:35:11.193431+0000","last_peered":"2024-04-16T19:35:11.193431+0000","last_clean":"2024-04-16T19:35:11.193431+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:35:11.193431+0000","last_undegraded":"2024-04-16T19:35:11.193431+0000","last_fullsized":"2024-04-16T19:35:11.193431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212828+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:36:11.212828+0000","last_peered":"2024-04-16T19:36:11.212828+0000","last_clean":"2024-04-16T19:36:11.212828+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:36:11.212828+0000","last_undegraded":"2024-04-16T19:36:11.212828+0000","last_fullsized":"2024-04-16T19:36:11.212828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.188529+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:34:41.188529+0000","last_peered":"2024-04-16T19:34:41.188529+0000","last_clean":"2024-04-16T19:34:41.188529+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:34:41.188529+0000","last_undegraded":"2024-04-16T19:34:41.188529+0000","last_fullsized":"2024-04-16T19:34:41.188529+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978536+0000","last_change":"2024-04-16T17:47:33.305749+0000","last_active":"2024-04-16T19:36:17.978536+0000","last_peered":"2024-04-16T19:36:17.978536+0000","last_clean":"2024-04-16T19:36:17.978536+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:36:17.978536+0000","last_undegraded":"2024-04-16T19:36:17.978536+0000","last_fullsized":"2024-04-16T19:36:17.978536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978433+0000","last_change":"2024-04-16T17:47:35.314832+0000","last_active":"2024-04-16T19:36:17.978433+0000","last_peered":"2024-04-16T19:36:17.978433+0000","last_clean":"2024-04-16T19:36:17.978433+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:36:17.978433+0000","last_undegraded":"2024-04-16T19:36:17.978433+0000","last_fullsized":"2024-04-16T19:36:17.978433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'12","reported_seq":1335,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.203680+0000","last_change":"2024-04-16T17:47:37.334246+0000","last_active":"2024-04-16T19:35:31.203680+0000","last_peered":"2024-04-16T19:35:31.203680+0000","last_clean":"2024-04-16T19:35:31.203680+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:35:31.203680+0000","last_undegraded":"2024-04-16T19:35:31.203680+0000","last_fullsized":"2024-04-16T19:35:31.203680+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.209696+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:36:16.209696+0000","last_peered":"2024-04-16T19:36:16.209696+0000","last_clean":"2024-04-16T19:36:16.209696+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:36:16.209696+0000","last_undegraded":"2024-04-16T19:36:16.209696+0000","last_fullsized":"2024-04-16T19:36:16.209696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'164","reported_seq":1854,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.196738+0000","last_change":"2024-04-16T17:48:34.332673+0000","last_active":"2024-04-16T19:34:51.196738+0000","last_peered":"2024-04-16T19:34:51.196738+0000","last_clean":"2024-04-16T19:34:51.196738+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:34:51.196738+0000","last_undegraded":"2024-04-16T19:34:51.196738+0000","last_fullsized":"2024-04-16T19:34:51.196738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.195559+0000","last_change":"2024-04-16T17:48:27.306930+0000","last_active":"2024-04-16T19:34:46.195559+0000","last_peered":"2024-04-16T19:34:46.195559+0000","last_clean":"2024-04-16T19:34:46.195559+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:34:46.195559+0000","last_undegraded":"2024-04-16T19:34:46.195559+0000","last_fullsized":"2024-04-16T19:34:46.195559+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'10","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.439114+0000","last_change":"2024-04-16T17:48:01.252700+0000","last_active":"2024-04-16T19:36:18.439114+0000","last_peered":"2024-04-16T19:36:18.439114+0000","last_clean":"2024-04-16T19:36:18.439114+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:36:18.439114+0000","last_undegraded":"2024-04-16T19:36:18.439114+0000","last_fullsized":"2024-04-16T19:36:18.439114+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.913468+0000","last_change":"2024-04-16T17:47:59.200630+0000","last_active":"2024-04-16T19:36:18.913468+0000","last_peered":"2024-04-16T19:36:18.913468+0000","last_clean":"2024-04-16T19:36:18.913468+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:36:18.913468+0000","last_undegraded":"2024-04-16T19:36:18.913468+0000","last_fullsized":"2024-04-16T19:36:18.913468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1110","reported_seq":1775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.529609+0000","last_change":"2024-04-16T19:36:31.529728+0000","last_active":"2024-04-16T19:36:31.529609+0000","last_peered":"2024-04-16T19:36:31.529609+0000","last_clean":"2024-04-16T19:36:31.529609+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:36:31.529609+0000","last_undegraded":"2024-04-16T19:36:31.529609+0000","last_fullsized":"2024-04-16T19:36:31.529609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1110","last_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_deep_scrub":"53'1110","last_deep_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.529471+0000","objects_scrubbed":23,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:18.708104+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143914,"num_omap_keys":628,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"53'8","reported_seq":149,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.477509+0000","last_change":"2024-04-16T19:36:20.477590+0000","last_active":"2024-04-16T19:36:20.477509+0000","last_peered":"2024-04-16T19:36:20.477509+0000","last_clean":"2024-04-16T19:36:20.477509+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:36:20.477509+0000","last_undegraded":"2024-04-16T19:36:20.477509+0000","last_fullsized":"2024-04-16T19:36:20.477509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.477384+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:25.916610+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210353+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:35:46.210353+0000","last_peered":"2024-04-16T19:35:46.210353+0000","last_clean":"2024-04-16T19:35:46.210353+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:35:46.210353+0000","last_undegraded":"2024-04-16T19:35:46.210353+0000","last_fullsized":"2024-04-16T19:35:46.210353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206209+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:35:46.206209+0000","last_peered":"2024-04-16T19:35:46.206209+0000","last_clean":"2024-04-16T19:35:46.206209+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:35:46.206209+0000","last_undegraded":"2024-04-16T19:35:46.206209+0000","last_fullsized":"2024-04-16T19:35:46.206209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208452+0000","last_change":"2024-04-16T17:47:33.309390+0000","last_active":"2024-04-16T19:35:56.208452+0000","last_peered":"2024-04-16T19:35:56.208452+0000","last_clean":"2024-04-16T19:35:56.208452+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:35:56.208452+0000","last_undegraded":"2024-04-16T19:35:56.208452+0000","last_fullsized":"2024-04-16T19:35:56.208452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'200","reported_seq":420,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912551+0000","last_change":"2024-04-16T17:47:35.314414+0000","last_active":"2024-04-16T19:36:18.912551+0000","last_peered":"2024-04-16T19:36:18.912551+0000","last_clean":"2024-04-16T19:36:18.912551+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:36:18.912551+0000","last_undegraded":"2024-04-16T19:36:18.912551+0000","last_fullsized":"2024-04-16T19:36:18.912551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206430+0000","last_change":"2024-04-16T17:47:37.335122+0000","last_active":"2024-04-16T19:35:41.206430+0000","last_peered":"2024-04-16T19:35:41.206430+0000","last_clean":"2024-04-16T19:35:41.206430+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:35:41.206430+0000","last_undegraded":"2024-04-16T19:35:41.206430+0000","last_fullsized":"2024-04-16T19:35:41.206430+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209992+0000","last_change":"2024-04-16T17:47:39.339833+0000","last_active":"2024-04-16T19:36:01.209992+0000","last_peered":"2024-04-16T19:36:01.209992+0000","last_clean":"2024-04-16T19:36:01.209992+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:36:01.209992+0000","last_undegraded":"2024-04-16T19:36:01.209992+0000","last_fullsized":"2024-04-16T19:36:01.209992+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194292+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:36:16.194292+0000","last_peered":"2024-04-16T19:36:16.194292+0000","last_clean":"2024-04-16T19:36:16.194292+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:36:16.194292+0000","last_undegraded":"2024-04-16T19:36:16.194292+0000","last_fullsized":"2024-04-16T19:36:16.194292+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192597+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:34:51.192597+0000","last_peered":"2024-04-16T19:34:51.192597+0000","last_clean":"2024-04-16T19:34:51.192597+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:34:51.192597+0000","last_undegraded":"2024-04-16T19:34:51.192597+0000","last_fullsized":"2024-04-16T19:34:51.192597+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.200405+0000","last_change":"2024-04-16T17:48:27.299718+0000","last_active":"2024-04-16T19:35:06.200405+0000","last_peered":"2024-04-16T19:35:06.200405+0000","last_clean":"2024-04-16T19:35:06.200405+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:35:06.200405+0000","last_undegraded":"2024-04-16T19:35:06.200405+0000","last_fullsized":"2024-04-16T19:35:06.200405+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'2","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980048+0000","last_change":"2024-04-16T17:48:01.253979+0000","last_active":"2024-04-16T19:36:17.980048+0000","last_peered":"2024-04-16T19:36:17.980048+0000","last_clean":"2024-04-16T19:36:17.980048+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:36:17.980048+0000","last_undegraded":"2024-04-16T19:36:17.980048+0000","last_fullsized":"2024-04-16T19:36:17.980048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587202+0000","last_change":"2024-04-16T17:47:59.193745+0000","last_active":"2024-04-16T19:36:16.587202+0000","last_peered":"2024-04-16T19:36:16.587202+0000","last_clean":"2024-04-16T19:36:16.587202+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:36:16.587202+0000","last_undegraded":"2024-04-16T19:36:16.587202+0000","last_fullsized":"2024-04-16T19:36:16.587202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1195","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.442330+0000","last_change":"2024-04-16T19:36:32.442434+0000","last_active":"2024-04-16T19:36:32.442330+0000","last_peered":"2024-04-16T19:36:32.442330+0000","last_clean":"2024-04-16T19:36:32.442330+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:36:32.442330+0000","last_undegraded":"2024-04-16T19:36:32.442330+0000","last_fullsized":"2024-04-16T19:36:32.442330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1195","last_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_deep_scrub":"53'1195","last_deep_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.442194+0000","objects_scrubbed":22,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:37.854921+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215932,"num_omap_keys":942,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'17","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.474347+0000","last_change":"2024-04-16T19:36:21.474426+0000","last_active":"2024-04-16T19:36:21.474347+0000","last_peered":"2024-04-16T19:36:21.474347+0000","last_clean":"2024-04-16T19:36:21.474347+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:36:21.474347+0000","last_undegraded":"2024-04-16T19:36:21.474347+0000","last_fullsized":"2024-04-16T19:36:21.474347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'17","last_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_deep_scrub":"53'17","last_deep_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.474217+0000","objects_scrubbed":3,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:40.463538+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202090+0000","last_change":"2024-04-16T17:47:23.266202+0000","last_active":"2024-04-16T19:35:46.202090+0000","last_peered":"2024-04-16T19:35:46.202090+0000","last_clean":"2024-04-16T19:35:46.202090+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:35:46.202090+0000","last_undegraded":"2024-04-16T19:35:46.202090+0000","last_fullsized":"2024-04-16T19:35:46.202090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199383+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:35:21.199383+0000","last_peered":"2024-04-16T19:35:21.199383+0000","last_clean":"2024-04-16T19:35:21.199383+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:35:21.199383+0000","last_undegraded":"2024-04-16T19:35:21.199383+0000","last_fullsized":"2024-04-16T19:35:21.199383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"31'1","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189364+0000","last_change":"2024-04-16T17:47:33.302299+0000","last_active":"2024-04-16T19:35:56.189364+0000","last_peered":"2024-04-16T19:35:56.189364+0000","last_clean":"2024-04-16T19:35:56.189364+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:35:56.189364+0000","last_undegraded":"2024-04-16T19:35:56.189364+0000","last_fullsized":"2024-04-16T19:35:56.189364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'80","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.204985+0000","last_change":"2024-04-16T17:47:35.321698+0000","last_active":"2024-04-16T19:35:36.204985+0000","last_peered":"2024-04-16T19:35:36.204985+0000","last_clean":"2024-04-16T19:35:36.204985+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:35:36.204985+0000","last_undegraded":"2024-04-16T19:35:36.204985+0000","last_fullsized":"2024-04-16T19:35:36.204985+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528378+0000","last_change":"2024-04-16T17:47:37.325461+0000","last_active":"2024-04-16T19:36:17.528378+0000","last_peered":"2024-04-16T19:36:17.528378+0000","last_clean":"2024-04-16T19:36:17.528378+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:36:17.528378+0000","last_undegraded":"2024-04-16T19:36:17.528378+0000","last_fullsized":"2024-04-16T19:36:17.528378+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438680+0000","last_change":"2024-04-16T17:47:39.323337+0000","last_active":"2024-04-16T19:36:18.438680+0000","last_peered":"2024-04-16T19:36:18.438680+0000","last_clean":"2024-04-16T19:36:18.438680+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:36:18.438680+0000","last_undegraded":"2024-04-16T19:36:18.438680+0000","last_fullsized":"2024-04-16T19:36:18.438680+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217900+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:36:16.217900+0000","last_peered":"2024-04-16T19:36:16.217900+0000","last_clean":"2024-04-16T19:36:16.217900+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:36:16.217900+0000","last_undegraded":"2024-04-16T19:36:16.217900+0000","last_fullsized":"2024-04-16T19:36:16.217900+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'70","reported_seq":902,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.177658+0000","last_change":"2024-04-16T17:48:34.345575+0000","last_active":"2024-04-16T19:35:06.177658+0000","last_peered":"2024-04-16T19:35:06.177658+0000","last_clean":"2024-04-16T19:35:06.177658+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:35:06.177658+0000","last_undegraded":"2024-04-16T19:35:06.177658+0000","last_fullsized":"2024-04-16T19:35:06.177658+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.191236+0000","last_change":"2024-04-16T17:48:27.295648+0000","last_active":"2024-04-16T19:34:46.191236+0000","last_peered":"2024-04-16T19:34:46.191236+0000","last_clean":"2024-04-16T19:34:46.191236+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:34:46.191236+0000","last_undegraded":"2024-04-16T19:34:46.191236+0000","last_fullsized":"2024-04-16T19:34:46.191236+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980079+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:36:17.980079+0000","last_peered":"2024-04-16T19:36:17.980079+0000","last_clean":"2024-04-16T19:36:17.980079+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:36:17.980079+0000","last_undegraded":"2024-04-16T19:36:17.980079+0000","last_fullsized":"2024-04-16T19:36:17.980079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.684591+0000","last_change":"2024-04-16T19:36:32.684642+0000","last_active":"2024-04-16T19:36:32.684591+0000","last_peered":"2024-04-16T19:36:32.684591+0000","last_clean":"2024-04-16T19:36:32.684591+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:36:32.684591+0000","last_undegraded":"2024-04-16T19:36:32.684591+0000","last_fullsized":"2024-04-16T19:36:32.684591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.684452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:11.352292+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1071","reported_seq":6142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.466107+0000","last_change":"2024-04-16T19:36:31.466251+0000","last_active":"2024-04-16T19:36:31.466107+0000","last_peered":"2024-04-16T19:36:31.466107+0000","last_clean":"2024-04-16T19:36:31.466107+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:36:31.466107+0000","last_undegraded":"2024-04-16T19:36:31.466107+0000","last_fullsized":"2024-04-16T19:36:31.466107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1071","last_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_deep_scrub":"53'1071","last_deep_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.465959+0000","objects_scrubbed":22,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:36.878686+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71967,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'8","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.607489+0000","last_change":"2024-04-16T19:36:24.607587+0000","last_active":"2024-04-16T19:36:24.607489+0000","last_peered":"2024-04-16T19:36:24.607489+0000","last_clean":"2024-04-16T19:36:24.607489+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:36:24.607489+0000","last_undegraded":"2024-04-16T19:36:24.607489+0000","last_fullsized":"2024-04-16T19:36:24.607489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.607347+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:30.046572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212436+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:36:11.212436+0000","last_peered":"2024-04-16T19:36:11.212436+0000","last_clean":"2024-04-16T19:36:11.212436+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:36:11.212436+0000","last_undegraded":"2024-04-16T19:36:11.212436+0000","last_fullsized":"2024-04-16T19:36:11.212436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204505+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:35:21.204505+0000","last_peered":"2024-04-16T19:35:21.204505+0000","last_clean":"2024-04-16T19:35:21.204505+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:35:21.204505+0000","last_undegraded":"2024-04-16T19:35:21.204505+0000","last_fullsized":"2024-04-16T19:35:21.204505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.184692+0000","last_change":"2024-04-16T17:47:33.302725+0000","last_active":"2024-04-16T19:35:36.184692+0000","last_peered":"2024-04-16T19:35:36.184692+0000","last_clean":"2024-04-16T19:35:36.184692+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:35:36.184692+0000","last_undegraded":"2024-04-16T19:35:36.184692+0000","last_fullsized":"2024-04-16T19:35:36.184692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'152","reported_seq":348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.212875+0000","last_change":"2024-04-16T17:47:35.317742+0000","last_active":"2024-04-16T19:35:56.212875+0000","last_peered":"2024-04-16T19:35:56.212875+0000","last_clean":"2024-04-16T19:35:56.212875+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:35:56.212875+0000","last_undegraded":"2024-04-16T19:35:56.212875+0000","last_fullsized":"2024-04-16T19:35:56.212875+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'12","reported_seq":1339,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209943+0000","last_change":"2024-04-16T17:47:37.318010+0000","last_active":"2024-04-16T19:36:01.209943+0000","last_peered":"2024-04-16T19:36:01.209943+0000","last_clean":"2024-04-16T19:36:01.209943+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:36:01.209943+0000","last_undegraded":"2024-04-16T19:36:01.209943+0000","last_fullsized":"2024-04-16T19:36:01.209943+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210298+0000","last_change":"2024-04-16T17:47:39.328909+0000","last_active":"2024-04-16T19:36:01.210298+0000","last_peered":"2024-04-16T19:36:01.210298+0000","last_clean":"2024-04-16T19:36:01.210298+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:36:01.210298+0000","last_undegraded":"2024-04-16T19:36:01.210298+0000","last_fullsized":"2024-04-16T19:36:01.210298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181074+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:35:21.181074+0000","last_peered":"2024-04-16T19:35:21.181074+0000","last_clean":"2024-04-16T19:35:21.181074+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:35:21.181074+0000","last_undegraded":"2024-04-16T19:35:21.181074+0000","last_fullsized":"2024-04-16T19:35:21.181074+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'74","reported_seq":1219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.193647+0000","last_change":"2024-04-16T17:48:34.327024+0000","last_active":"2024-04-16T19:34:51.193647+0000","last_peered":"2024-04-16T19:34:51.193647+0000","last_clean":"2024-04-16T19:34:51.193647+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:34:51.193647+0000","last_undegraded":"2024-04-16T19:34:51.193647+0000","last_fullsized":"2024-04-16T19:34:51.193647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.189936+0000","last_change":"2024-04-16T17:48:27.301787+0000","last_active":"2024-04-16T19:34:46.189936+0000","last_peered":"2024-04-16T19:34:46.189936+0000","last_clean":"2024-04-16T19:34:46.189936+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:34:46.189936+0000","last_undegraded":"2024-04-16T19:34:46.189936+0000","last_fullsized":"2024-04-16T19:34:46.189936+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'17","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587390+0000","last_change":"2024-04-16T17:48:01.243297+0000","last_active":"2024-04-16T19:36:16.587390+0000","last_peered":"2024-04-16T19:36:16.587390+0000","last_clean":"2024-04-16T19:36:16.587390+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:36:16.587390+0000","last_undegraded":"2024-04-16T19:36:16.587390+0000","last_fullsized":"2024-04-16T19:36:16.587390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.913770+0000","last_change":"2024-04-16T17:47:59.193938+0000","last_active":"2024-04-16T19:36:18.913770+0000","last_peered":"2024-04-16T19:36:18.913770+0000","last_clean":"2024-04-16T19:36:18.913770+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:36:18.913770+0000","last_undegraded":"2024-04-16T19:36:18.913770+0000","last_fullsized":"2024-04-16T19:36:18.913770+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'736","reported_seq":1477,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.630355+0000","last_change":"2024-04-16T19:36:18.630399+0000","last_active":"2024-04-16T19:36:18.630355+0000","last_peered":"2024-04-16T19:36:18.630355+0000","last_clean":"2024-04-16T19:36:18.630355+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:36:18.630355+0000","last_undegraded":"2024-04-16T19:36:18.630355+0000","last_fullsized":"2024-04-16T19:36:18.630355+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'736","last_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_deep_scrub":"53'736","last_deep_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.630302+0000","objects_scrubbed":33,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:45.495590+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":75043,"num_omap_keys":332,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'18","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.577485+0000","last_change":"2024-04-16T19:36:26.577559+0000","last_active":"2024-04-16T19:36:26.577485+0000","last_peered":"2024-04-16T19:36:26.577485+0000","last_clean":"2024-04-16T19:36:26.577485+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:36:26.577485+0000","last_undegraded":"2024-04-16T19:36:26.577485+0000","last_fullsized":"2024-04-16T19:36:26.577485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.577356+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:41.709302+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201519+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:35:26.201519+0000","last_peered":"2024-04-16T19:35:26.201519+0000","last_clean":"2024-04-16T19:35:26.201519+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:35:26.201519+0000","last_undegraded":"2024-04-16T19:35:26.201519+0000","last_fullsized":"2024-04-16T19:35:26.201519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199263+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:35:21.199263+0000","last_peered":"2024-04-16T19:35:21.199263+0000","last_clean":"2024-04-16T19:35:21.199263+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:35:21.199263+0000","last_undegraded":"2024-04-16T19:35:21.199263+0000","last_fullsized":"2024-04-16T19:35:21.199263+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.527954+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:36:17.527954+0000","last_peered":"2024-04-16T19:36:17.527954+0000","last_clean":"2024-04-16T19:36:17.527954+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:36:17.527954+0000","last_undegraded":"2024-04-16T19:36:17.527954+0000","last_fullsized":"2024-04-16T19:36:17.527954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'88","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208721+0000","last_change":"2024-04-16T17:47:35.317589+0000","last_active":"2024-04-16T19:35:56.208721+0000","last_peered":"2024-04-16T19:35:56.208721+0000","last_clean":"2024-04-16T19:35:56.208721+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:35:56.208721+0000","last_undegraded":"2024-04-16T19:35:56.208721+0000","last_fullsized":"2024-04-16T19:35:56.208721+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.190730+0000","last_change":"2024-04-16T17:47:37.330541+0000","last_active":"2024-04-16T19:36:01.190730+0000","last_peered":"2024-04-16T19:36:01.190730+0000","last_clean":"2024-04-16T19:36:01.190730+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:36:01.190730+0000","last_undegraded":"2024-04-16T19:36:01.190730+0000","last_fullsized":"2024-04-16T19:36:01.190730+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.527536+0000","last_change":"2024-04-16T19:36:33.527616+0000","last_active":"2024-04-16T19:36:33.527536+0000","last_peered":"2024-04-16T19:36:33.527536+0000","last_clean":"2024-04-16T19:36:33.527536+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:36:33.527536+0000","last_undegraded":"2024-04-16T19:36:33.527536+0000","last_fullsized":"2024-04-16T19:36:33.527536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.527307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:38.972728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201397+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:35:21.201397+0000","last_peered":"2024-04-16T19:35:21.201397+0000","last_clean":"2024-04-16T19:35:21.201397+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:35:21.201397+0000","last_undegraded":"2024-04-16T19:35:21.201397+0000","last_fullsized":"2024-04-16T19:35:21.201397+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'40","reported_seq":974,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.191458+0000","last_change":"2024-04-16T17:48:34.333313+0000","last_active":"2024-04-16T19:34:46.191458+0000","last_peered":"2024-04-16T19:34:46.191458+0000","last_clean":"2024-04-16T19:34:46.191458+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:34:46.191458+0000","last_undegraded":"2024-04-16T19:34:46.191458+0000","last_fullsized":"2024-04-16T19:34:46.191458+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.196063+0000","last_change":"2024-04-16T17:48:27.304408+0000","last_active":"2024-04-16T19:35:21.196063+0000","last_peered":"2024-04-16T19:35:21.196063+0000","last_clean":"2024-04-16T19:35:21.196063+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:35:21.196063+0000","last_undegraded":"2024-04-16T19:35:21.196063+0000","last_fullsized":"2024-04-16T19:35:21.196063+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'9","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914639+0000","last_change":"2024-04-16T17:48:01.259988+0000","last_active":"2024-04-16T19:36:18.914639+0000","last_peered":"2024-04-16T19:36:18.914639+0000","last_clean":"2024-04-16T19:36:18.914639+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:36:18.914639+0000","last_undegraded":"2024-04-16T19:36:18.914639+0000","last_fullsized":"2024-04-16T19:36:18.914639+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'555","reported_seq":2289,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.979684+0000","last_change":"2024-04-16T17:47:59.181942+0000","last_active":"2024-04-16T19:36:17.979684+0000","last_peered":"2024-04-16T19:36:17.979684+0000","last_clean":"2024-04-16T19:36:17.979684+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:36:17.979684+0000","last_undegraded":"2024-04-16T19:36:17.979684+0000","last_fullsized":"2024-04-16T19:36:17.979684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'925","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.519148+0000","last_change":"2024-04-16T19:36:29.519272+0000","last_active":"2024-04-16T19:36:29.519148+0000","last_peered":"2024-04-16T19:36:29.519148+0000","last_clean":"2024-04-16T19:36:29.519148+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:36:29.519148+0000","last_undegraded":"2024-04-16T19:36:29.519148+0000","last_fullsized":"2024-04-16T19:36:29.519148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.519049+0000","objects_scrubbed":24,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:29.400044+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143864,"num_omap_keys":628,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'19","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.480753+0000","last_change":"2024-04-16T19:36:30.480855+0000","last_active":"2024-04-16T19:36:30.480753+0000","last_peered":"2024-04-16T19:36:30.480753+0000","last_clean":"2024-04-16T19:36:30.480753+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:36:30.480753+0000","last_undegraded":"2024-04-16T19:36:30.480753+0000","last_fullsized":"2024-04-16T19:36:30.480753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.480620+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:17.845024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197270+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:35:26.197270+0000","last_peered":"2024-04-16T19:35:26.197270+0000","last_clean":"2024-04-16T19:35:26.197270+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:35:26.197270+0000","last_undegraded":"2024-04-16T19:35:26.197270+0000","last_fullsized":"2024-04-16T19:35:26.197270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201194+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:35:21.201194+0000","last_peered":"2024-04-16T19:35:21.201194+0000","last_clean":"2024-04-16T19:35:21.201194+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:35:21.201194+0000","last_undegraded":"2024-04-16T19:35:21.201194+0000","last_fullsized":"2024-04-16T19:35:21.201194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":176,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.184791+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:35:36.184791+0000","last_peered":"2024-04-16T19:35:36.184791+0000","last_clean":"2024-04-16T19:35:36.184791+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:35:36.184791+0000","last_undegraded":"2024-04-16T19:35:36.184791+0000","last_fullsized":"2024-04-16T19:35:36.184791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'104","reported_seq":274,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.199943+0000","last_change":"2024-04-16T17:47:35.307146+0000","last_active":"2024-04-16T19:35:36.199943+0000","last_peered":"2024-04-16T19:35:36.199943+0000","last_clean":"2024-04-16T19:35:36.199943+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:35:36.199943+0000","last_undegraded":"2024-04-16T19:35:36.199943+0000","last_fullsized":"2024-04-16T19:35:36.199943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.214073+0000","last_change":"2024-04-16T17:47:37.324514+0000","last_active":"2024-04-16T19:36:16.214073+0000","last_peered":"2024-04-16T19:36:16.214073+0000","last_clean":"2024-04-16T19:36:16.214073+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:36:16.214073+0000","last_undegraded":"2024-04-16T19:36:16.214073+0000","last_fullsized":"2024-04-16T19:36:16.214073+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.457022+0000","last_change":"2024-04-16T19:36:30.457084+0000","last_active":"2024-04-16T19:36:30.457022+0000","last_peered":"2024-04-16T19:36:30.457022+0000","last_clean":"2024-04-16T19:36:30.457022+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:36:30.457022+0000","last_undegraded":"2024-04-16T19:36:30.457022+0000","last_fullsized":"2024-04-16T19:36:30.457022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.456877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:36.883658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200338+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:35:21.200338+0000","last_peered":"2024-04-16T19:35:21.200338+0000","last_clean":"2024-04-16T19:35:21.200338+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:35:21.200338+0000","last_undegraded":"2024-04-16T19:35:21.200338+0000","last_fullsized":"2024-04-16T19:35:21.200338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":883,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.190172+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:34:46.190172+0000","last_peered":"2024-04-16T19:34:46.190172+0000","last_clean":"2024-04-16T19:34:46.190172+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:34:46.190172+0000","last_undegraded":"2024-04-16T19:34:46.190172+0000","last_fullsized":"2024-04-16T19:34:46.190172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.190023+0000","last_change":"2024-04-16T17:48:27.308234+0000","last_active":"2024-04-16T19:34:46.190023+0000","last_peered":"2024-04-16T19:34:46.190023+0000","last_clean":"2024-04-16T19:34:46.190023+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:34:46.190023+0000","last_undegraded":"2024-04-16T19:34:46.190023+0000","last_fullsized":"2024-04-16T19:34:46.190023+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'135","reported_seq":300,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.196890+0000","last_change":"2024-04-16T17:48:01.254153+0000","last_active":"2024-04-16T19:34:51.196890+0000","last_peered":"2024-04-16T19:34:51.196890+0000","last_clean":"2024-04-16T19:34:51.196890+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:34:51.196890+0000","last_undegraded":"2024-04-16T19:34:51.196890+0000","last_fullsized":"2024-04-16T19:34:51.196890+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.514976+0000","last_change":"2024-04-16T19:36:24.515035+0000","last_active":"2024-04-16T19:36:24.514976+0000","last_peered":"2024-04-16T19:36:24.514976+0000","last_clean":"2024-04-16T19:36:24.514976+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:36:24.514976+0000","last_undegraded":"2024-04-16T19:36:24.514976+0000","last_fullsized":"2024-04-16T19:36:24.514976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.514834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:04:02.534977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'702","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.544776+0000","last_change":"2024-04-16T19:36:22.544916+0000","last_active":"2024-04-16T19:36:22.544776+0000","last_peered":"2024-04-16T19:36:22.544776+0000","last_clean":"2024-04-16T19:36:22.544776+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:36:22.544776+0000","last_undegraded":"2024-04-16T19:36:22.544776+0000","last_fullsized":"2024-04-16T19:36:22.544776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'702","last_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_deep_scrub":"53'702","last_deep_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.544630+0000","objects_scrubbed":23,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:27.704961+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71922,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.499314+0000","last_change":"2024-04-16T19:36:24.499314+0000","last_active":"2024-04-16T19:36:24.499314+0000","last_peered":"2024-04-16T19:36:24.499314+0000","last_clean":"2024-04-16T19:36:24.499314+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:36:24.499314+0000","last_undegraded":"2024-04-16T19:36:24.499314+0000","last_fullsized":"2024-04-16T19:36:24.499314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.498983+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:48.052702+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.205683+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:35:26.205683+0000","last_peered":"2024-04-16T19:35:26.205683+0000","last_clean":"2024-04-16T19:35:26.205683+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:35:26.205683+0000","last_undegraded":"2024-04-16T19:35:26.205683+0000","last_fullsized":"2024-04-16T19:35:26.205683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217992+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:36:16.217992+0000","last_peered":"2024-04-16T19:36:16.217992+0000","last_clean":"2024-04-16T19:36:16.217992+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:36:16.217992+0000","last_undegraded":"2024-04-16T19:36:16.217992+0000","last_fullsized":"2024-04-16T19:36:16.217992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"48'13","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.205087+0000","last_change":"2024-04-16T17:47:33.309159+0000","last_active":"2024-04-16T19:35:36.205087+0000","last_peered":"2024-04-16T19:35:36.205087+0000","last_clean":"2024-04-16T19:35:36.205087+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:35:36.205087+0000","last_undegraded":"2024-04-16T19:35:36.205087+0000","last_fullsized":"2024-04-16T19:35:36.205087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'152","reported_seq":346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.192239+0000","last_change":"2024-04-16T17:47:35.305022+0000","last_active":"2024-04-16T19:36:06.192239+0000","last_peered":"2024-04-16T19:36:06.192239+0000","last_clean":"2024-04-16T19:36:06.192239+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:36:06.192239+0000","last_undegraded":"2024-04-16T19:36:06.192239+0000","last_fullsized":"2024-04-16T19:36:06.192239+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'12","reported_seq":1337,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186055+0000","last_change":"2024-04-16T17:47:37.331019+0000","last_active":"2024-04-16T19:35:41.186055+0000","last_peered":"2024-04-16T19:35:41.186055+0000","last_clean":"2024-04-16T19:35:41.186055+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:35:41.186055+0000","last_undegraded":"2024-04-16T19:35:41.186055+0000","last_fullsized":"2024-04-16T19:35:41.186055+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.211873+0000","last_change":"2024-04-16T17:47:39.331395+0000","last_active":"2024-04-16T19:36:01.211873+0000","last_peered":"2024-04-16T19:36:01.211873+0000","last_clean":"2024-04-16T19:36:01.211873+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:36:01.211873+0000","last_undegraded":"2024-04-16T19:36:01.211873+0000","last_fullsized":"2024-04-16T19:36:01.211873+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194434+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:34:41.194434+0000","last_peered":"2024-04-16T19:34:41.194434+0000","last_clean":"2024-04-16T19:34:41.194434+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:34:41.194434+0000","last_undegraded":"2024-04-16T19:34:41.194434+0000","last_fullsized":"2024-04-16T19:34:41.194434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'86","reported_seq":1118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.196810+0000","last_change":"2024-04-16T17:48:34.326754+0000","last_active":"2024-04-16T19:34:51.196810+0000","last_peered":"2024-04-16T19:34:51.196810+0000","last_clean":"2024-04-16T19:34:51.196810+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:34:51.196810+0000","last_undegraded":"2024-04-16T19:34:51.196810+0000","last_fullsized":"2024-04-16T19:34:51.196810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194571+0000","last_change":"2024-04-16T17:48:27.298815+0000","last_active":"2024-04-16T19:34:41.194571+0000","last_peered":"2024-04-16T19:34:41.194571+0000","last_clean":"2024-04-16T19:34:41.194571+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:34:41.194571+0000","last_undegraded":"2024-04-16T19:34:41.194571+0000","last_fullsized":"2024-04-16T19:34:41.194571+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"48'1","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.049876+0000","last_change":"2024-04-16T17:48:01.246604+0000","last_active":"2024-04-16T19:36:17.049876+0000","last_peered":"2024-04-16T19:36:17.049876+0000","last_clean":"2024-04-16T19:36:17.049876+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:36:17.049876+0000","last_undegraded":"2024-04-16T19:36:17.049876+0000","last_fullsized":"2024-04-16T19:36:17.049876+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.565703+0000","last_change":"2024-04-16T19:36:24.565807+0000","last_active":"2024-04-16T19:36:24.565703+0000","last_peered":"2024-04-16T19:36:24.565703+0000","last_clean":"2024-04-16T19:36:24.565703+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:36:24.565703+0000","last_undegraded":"2024-04-16T19:36:24.565703+0000","last_fullsized":"2024-04-16T19:36:24.565703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.565541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:30.252893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1508","reported_seq":5513,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.585799+0000","last_change":"2024-04-16T19:36:27.585867+0000","last_active":"2024-04-16T19:36:27.585799+0000","last_peered":"2024-04-16T19:36:27.585799+0000","last_clean":"2024-04-16T19:36:27.585799+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:36:27.585799+0000","last_undegraded":"2024-04-16T19:36:27.585799+0000","last_fullsized":"2024-04-16T19:36:27.585799+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1508","last_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_deep_scrub":"53'1508","last_deep_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.585661+0000","objects_scrubbed":25,"log_size":1508,"log_dups_size":0,"ondisk_log_size":1508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:18.587606+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250399,"num_omap_keys":1045,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'14","reported_seq":240,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.424045+0000","last_change":"2024-04-16T19:36:27.424219+0000","last_active":"2024-04-16T19:36:27.424045+0000","last_peered":"2024-04-16T19:36:27.424045+0000","last_clean":"2024-04-16T19:36:27.424045+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:36:27.424045+0000","last_undegraded":"2024-04-16T19:36:27.424045+0000","last_fullsized":"2024-04-16T19:36:27.424045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.423818+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:57.529566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197378+0000","last_change":"2024-04-16T17:47:23.266018+0000","last_active":"2024-04-16T19:35:26.197378+0000","last_peered":"2024-04-16T19:35:26.197378+0000","last_clean":"2024-04-16T19:35:26.197378+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:35:26.197378+0000","last_undegraded":"2024-04-16T19:35:26.197378+0000","last_fullsized":"2024-04-16T19:35:26.197378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200918+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:35:41.200918+0000","last_peered":"2024-04-16T19:35:41.200918+0000","last_clean":"2024-04-16T19:35:41.200918+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:35:41.200918+0000","last_undegraded":"2024-04-16T19:35:41.200918+0000","last_fullsized":"2024-04-16T19:35:41.200918+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210154+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:36:01.210154+0000","last_peered":"2024-04-16T19:36:01.210154+0000","last_clean":"2024-04-16T19:36:01.210154+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:36:01.210154+0000","last_undegraded":"2024-04-16T19:36:01.210154+0000","last_fullsized":"2024-04-16T19:36:01.210154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'120","reported_seq":293,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.205111+0000","last_change":"2024-04-16T17:47:35.319180+0000","last_active":"2024-04-16T19:35:36.205111+0000","last_peered":"2024-04-16T19:35:36.205111+0000","last_clean":"2024-04-16T19:35:36.205111+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:35:36.205111+0000","last_undegraded":"2024-04-16T19:35:36.205111+0000","last_fullsized":"2024-04-16T19:35:36.205111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'12","reported_seq":1337,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.201095+0000","last_change":"2024-04-16T17:47:37.326971+0000","last_active":"2024-04-16T19:35:41.201095+0000","last_peered":"2024-04-16T19:35:41.201095+0000","last_clean":"2024-04-16T19:35:41.201095+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:35:41.201095+0000","last_undegraded":"2024-04-16T19:35:41.201095+0000","last_fullsized":"2024-04-16T19:35:41.201095+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.206057+0000","last_change":"2024-04-16T17:47:39.325433+0000","last_active":"2024-04-16T19:36:01.206057+0000","last_peered":"2024-04-16T19:36:01.206057+0000","last_clean":"2024-04-16T19:36:01.206057+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:36:01.206057+0000","last_undegraded":"2024-04-16T19:36:01.206057+0000","last_fullsized":"2024-04-16T19:36:01.206057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215663+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:36:16.215663+0000","last_peered":"2024-04-16T19:36:16.215663+0000","last_clean":"2024-04-16T19:36:16.215663+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:36:16.215663+0000","last_undegraded":"2024-04-16T19:36:16.215663+0000","last_fullsized":"2024-04-16T19:36:16.215663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'90","reported_seq":978,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201673+0000","last_change":"2024-04-16T17:48:34.333850+0000","last_active":"2024-04-16T19:35:26.201673+0000","last_peered":"2024-04-16T19:35:26.201673+0000","last_clean":"2024-04-16T19:35:26.201673+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:35:26.201673+0000","last_undegraded":"2024-04-16T19:35:26.201673+0000","last_fullsized":"2024-04-16T19:35:26.201673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194526+0000","last_change":"2024-04-16T17:48:27.299707+0000","last_active":"2024-04-16T19:34:41.194526+0000","last_peered":"2024-04-16T19:34:41.194526+0000","last_clean":"2024-04-16T19:34:41.194526+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:34:41.194526+0000","last_undegraded":"2024-04-16T19:34:41.194526+0000","last_fullsized":"2024-04-16T19:34:41.194526+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'5","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.049527+0000","last_change":"2024-04-16T17:48:01.258518+0000","last_active":"2024-04-16T19:36:17.049527+0000","last_peered":"2024-04-16T19:36:17.049527+0000","last_clean":"2024-04-16T19:36:17.049527+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:36:17.049527+0000","last_undegraded":"2024-04-16T19:36:17.049527+0000","last_fullsized":"2024-04-16T19:36:17.049527+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'555","reported_seq":1943,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.049531+0000","last_change":"2024-04-16T17:47:59.184752+0000","last_active":"2024-04-16T19:36:17.049531+0000","last_peered":"2024-04-16T19:36:17.049531+0000","last_clean":"2024-04-16T19:36:17.049531+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:36:17.049531+0000","last_undegraded":"2024-04-16T19:36:17.049531+0000","last_fullsized":"2024-04-16T19:36:17.049531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'676","reported_seq":995,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635509+0000","last_change":"2024-04-16T19:36:20.635509+0000","last_active":"2024-04-16T19:36:20.635509+0000","last_peered":"2024-04-16T19:36:20.635509+0000","last_clean":"2024-04-16T19:36:20.635509+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:36:20.635509+0000","last_undegraded":"2024-04-16T19:36:20.635509+0000","last_fullsized":"2024-04-16T19:36:20.635509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'676","last_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_deep_scrub":"53'676","last_deep_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635277+0000","objects_scrubbed":25,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:10:55.447676+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71899,"num_omap_keys":314,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'20","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.500999+0000","last_change":"2024-04-16T19:36:18.501046+0000","last_active":"2024-04-16T19:36:18.500999+0000","last_peered":"2024-04-16T19:36:18.500999+0000","last_clean":"2024-04-16T19:36:18.500999+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:36:18.500999+0000","last_undegraded":"2024-04-16T19:36:18.500999+0000","last_fullsized":"2024-04-16T19:36:18.500999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.500951+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:17.147393+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210872+0000","last_change":"2024-04-16T17:47:23.261436+0000","last_active":"2024-04-16T19:35:56.210872+0000","last_peered":"2024-04-16T19:35:56.210872+0000","last_clean":"2024-04-16T19:35:56.210872+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:35:56.210872+0000","last_undegraded":"2024-04-16T19:35:56.210872+0000","last_fullsized":"2024-04-16T19:35:56.210872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.185743+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:35:41.185743+0000","last_peered":"2024-04-16T19:35:41.185743+0000","last_clean":"2024-04-16T19:35:41.185743+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:35:41.185743+0000","last_undegraded":"2024-04-16T19:35:41.185743+0000","last_fullsized":"2024-04-16T19:35:41.185743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.208172+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:35:36.208172+0000","last_peered":"2024-04-16T19:35:36.208172+0000","last_clean":"2024-04-16T19:35:36.208172+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:35:36.208172+0000","last_undegraded":"2024-04-16T19:35:36.208172+0000","last_fullsized":"2024-04-16T19:35:36.208172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'176","reported_seq":382,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.203507+0000","last_change":"2024-04-16T17:47:35.317949+0000","last_active":"2024-04-16T19:35:36.203507+0000","last_peered":"2024-04-16T19:35:36.203507+0000","last_clean":"2024-04-16T19:35:36.203507+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:35:36.203507+0000","last_undegraded":"2024-04-16T19:35:36.203507+0000","last_fullsized":"2024-04-16T19:35:36.203507+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209341+0000","last_change":"2024-04-16T17:47:37.332636+0000","last_active":"2024-04-16T19:35:41.209341+0000","last_peered":"2024-04-16T19:35:41.209341+0000","last_clean":"2024-04-16T19:35:41.209341+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:35:41.209341+0000","last_undegraded":"2024-04-16T19:35:41.209341+0000","last_fullsized":"2024-04-16T19:35:41.209341+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.569888+0000","last_change":"2024-04-16T19:36:34.569940+0000","last_active":"2024-04-16T19:36:34.569888+0000","last_peered":"2024-04-16T19:36:34.569888+0000","last_clean":"2024-04-16T19:36:34.569888+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:36:34.569888+0000","last_undegraded":"2024-04-16T19:36:34.569888+0000","last_fullsized":"2024-04-16T19:36:34.569888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.569751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:40.008977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528987+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:36:17.528987+0000","last_peered":"2024-04-16T19:36:17.528987+0000","last_clean":"2024-04-16T19:36:17.528987+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:36:17.528987+0000","last_undegraded":"2024-04-16T19:36:17.528987+0000","last_fullsized":"2024-04-16T19:36:17.528987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'118","reported_seq":1503,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.190007+0000","last_change":"2024-04-16T17:48:34.329015+0000","last_active":"2024-04-16T19:34:56.190007+0000","last_peered":"2024-04-16T19:34:56.190007+0000","last_clean":"2024-04-16T19:34:56.190007+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:34:56.190007+0000","last_undegraded":"2024-04-16T19:34:56.190007+0000","last_fullsized":"2024-04-16T19:34:56.190007+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.049965+0000","last_change":"2024-04-16T17:48:27.296987+0000","last_active":"2024-04-16T19:36:17.049965+0000","last_peered":"2024-04-16T19:36:17.049965+0000","last_clean":"2024-04-16T19:36:17.049965+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:36:17.049965+0000","last_undegraded":"2024-04-16T19:36:17.049965+0000","last_fullsized":"2024-04-16T19:36:17.049965+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'8","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.206226+0000","last_change":"2024-04-16T17:48:01.240484+0000","last_active":"2024-04-16T19:36:01.206226+0000","last_peered":"2024-04-16T19:36:01.206226+0000","last_clean":"2024-04-16T19:36:01.206226+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:36:01.206226+0000","last_undegraded":"2024-04-16T19:36:01.206226+0000","last_fullsized":"2024-04-16T19:36:01.206226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":92,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.191409+0000","last_change":"2024-04-16T17:47:59.184121+0000","last_active":"2024-04-16T19:34:41.191409+0000","last_peered":"2024-04-16T19:34:41.191409+0000","last_clean":"2024-04-16T19:34:41.191409+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:34:41.191409+0000","last_undegraded":"2024-04-16T19:34:41.191409+0000","last_fullsized":"2024-04-16T19:34:41.191409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'804","reported_seq":43172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.600535+0000","last_change":"2024-04-16T19:36:25.600624+0000","last_active":"2024-04-16T19:36:25.600535+0000","last_peered":"2024-04-16T19:36:25.600535+0000","last_clean":"2024-04-16T19:36:25.600535+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:36:25.600535+0000","last_undegraded":"2024-04-16T19:36:25.600535+0000","last_fullsized":"2024-04-16T19:36:25.600535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'804","last_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_deep_scrub":"53'804","last_deep_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.600462+0000","objects_scrubbed":21,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:05.088020+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71960,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'9","reported_seq":210,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194164+0000","last_change":"2024-04-16T17:47:55.143458+0000","last_active":"2024-04-16T19:36:16.194164+0000","last_peered":"2024-04-16T19:36:16.194164+0000","last_clean":"2024-04-16T19:36:16.194164+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:36:16.194164+0000","last_undegraded":"2024-04-16T19:36:16.194164+0000","last_fullsized":"2024-04-16T19:36:16.194164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978084+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:36:17.978084+0000","last_peered":"2024-04-16T19:36:17.978084+0000","last_clean":"2024-04-16T19:36:17.978084+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:36:17.978084+0000","last_undegraded":"2024-04-16T19:36:17.978084+0000","last_fullsized":"2024-04-16T19:36:17.978084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.202904+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:35:16.202904+0000","last_peered":"2024-04-16T19:35:16.202904+0000","last_clean":"2024-04-16T19:35:16.202904+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:35:16.202904+0000","last_undegraded":"2024-04-16T19:35:16.202904+0000","last_fullsized":"2024-04-16T19:35:16.202904+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206209+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:35:41.206209+0000","last_peered":"2024-04-16T19:35:41.206209+0000","last_clean":"2024-04-16T19:35:41.206209+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:35:41.206209+0000","last_undegraded":"2024-04-16T19:35:41.206209+0000","last_fullsized":"2024-04-16T19:35:41.206209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204821+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:35:56.204821+0000","last_peered":"2024-04-16T19:35:56.204821+0000","last_clean":"2024-04-16T19:35:56.204821+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:35:56.204821+0000","last_undegraded":"2024-04-16T19:35:56.204821+0000","last_fullsized":"2024-04-16T19:35:56.204821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'72","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208814+0000","last_change":"2024-04-16T17:47:35.310088+0000","last_active":"2024-04-16T19:35:56.208814+0000","last_peered":"2024-04-16T19:35:56.208814+0000","last_clean":"2024-04-16T19:35:56.208814+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:35:56.208814+0000","last_undegraded":"2024-04-16T19:35:56.208814+0000","last_fullsized":"2024-04-16T19:35:56.208814+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210279+0000","last_change":"2024-04-16T17:47:37.327822+0000","last_active":"2024-04-16T19:36:01.210279+0000","last_peered":"2024-04-16T19:36:01.210279+0000","last_clean":"2024-04-16T19:36:01.210279+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:36:01.210279+0000","last_undegraded":"2024-04-16T19:36:01.210279+0000","last_fullsized":"2024-04-16T19:36:01.210279+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209770+0000","last_change":"2024-04-16T17:47:39.341107+0000","last_active":"2024-04-16T19:36:01.209770+0000","last_peered":"2024-04-16T19:36:01.209770+0000","last_clean":"2024-04-16T19:36:01.209770+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:36:01.209770+0000","last_undegraded":"2024-04-16T19:36:01.209770+0000","last_fullsized":"2024-04-16T19:36:01.209770+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:36.193232+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:34:36.193232+0000","last_peered":"2024-04-16T19:34:36.193232+0000","last_clean":"2024-04-16T19:34:36.193232+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:34:36.193232+0000","last_undegraded":"2024-04-16T19:34:36.193232+0000","last_fullsized":"2024-04-16T19:34:36.193232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'100","reported_seq":1747,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.439909+0000","last_change":"2024-04-16T17:48:34.333902+0000","last_active":"2024-04-16T19:36:18.439909+0000","last_peered":"2024-04-16T19:36:18.439909+0000","last_clean":"2024-04-16T19:36:18.439909+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:36:18.439909+0000","last_undegraded":"2024-04-16T19:36:18.439909+0000","last_fullsized":"2024-04-16T19:36:18.439909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.439752+0000","last_change":"2024-04-16T17:48:27.296497+0000","last_active":"2024-04-16T19:36:18.439752+0000","last_peered":"2024-04-16T19:36:18.439752+0000","last_clean":"2024-04-16T19:36:18.439752+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:36:18.439752+0000","last_undegraded":"2024-04-16T19:36:18.439752+0000","last_fullsized":"2024-04-16T19:36:18.439752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'8","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210112+0000","last_change":"2024-04-16T17:48:01.261918+0000","last_active":"2024-04-16T19:36:01.210112+0000","last_peered":"2024-04-16T19:36:01.210112+0000","last_clean":"2024-04-16T19:36:01.210112+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:36:01.210112+0000","last_undegraded":"2024-04-16T19:36:01.210112+0000","last_fullsized":"2024-04-16T19:36:01.210112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.190879+0000","last_change":"2024-04-16T17:47:59.182311+0000","last_active":"2024-04-16T19:36:01.190879+0000","last_peered":"2024-04-16T19:36:01.190879+0000","last_clean":"2024-04-16T19:36:01.190879+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:36:01.190879+0000","last_undegraded":"2024-04-16T19:36:01.190879+0000","last_fullsized":"2024-04-16T19:36:01.190879+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'925","reported_seq":3829,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.639148+0000","last_change":"2024-04-16T19:36:25.639219+0000","last_active":"2024-04-16T19:36:25.639148+0000","last_peered":"2024-04-16T19:36:25.639148+0000","last_clean":"2024-04-16T19:36:25.639148+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:36:25.639148+0000","last_undegraded":"2024-04-16T19:36:25.639148+0000","last_fullsized":"2024-04-16T19:36:25.639148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.639013+0000","objects_scrubbed":28,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:11.472224+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89384,"num_omap_keys":416,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'36","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.510632+0000","last_change":"2024-04-16T19:36:24.510731+0000","last_active":"2024-04-16T19:36:24.510632+0000","last_peered":"2024-04-16T19:36:24.510632+0000","last_clean":"2024-04-16T19:36:24.510632+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:36:24.510632+0000","last_undegraded":"2024-04-16T19:36:24.510632+0000","last_fullsized":"2024-04-16T19:36:24.510632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'36","last_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_deep_scrub":"53'36","last_deep_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.510495+0000","objects_scrubbed":7,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:21.114823+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207927+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:35:46.207927+0000","last_peered":"2024-04-16T19:35:46.207927+0000","last_clean":"2024-04-16T19:35:46.207927+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:35:46.207927+0000","last_undegraded":"2024-04-16T19:35:46.207927+0000","last_fullsized":"2024-04-16T19:35:46.207927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192365+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:34:56.192365+0000","last_peered":"2024-04-16T19:34:56.192365+0000","last_clean":"2024-04-16T19:34:56.192365+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:34:56.192365+0000","last_undegraded":"2024-04-16T19:34:56.192365+0000","last_fullsized":"2024-04-16T19:34:56.192365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200997+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:35:41.200997+0000","last_peered":"2024-04-16T19:35:41.200997+0000","last_clean":"2024-04-16T19:35:41.200997+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:35:41.200997+0000","last_undegraded":"2024-04-16T19:35:41.200997+0000","last_fullsized":"2024-04-16T19:35:41.200997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586344+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:36:16.586344+0000","last_peered":"2024-04-16T19:36:16.586344+0000","last_clean":"2024-04-16T19:36:16.586344+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:36:16.586344+0000","last_undegraded":"2024-04-16T19:36:16.586344+0000","last_fullsized":"2024-04-16T19:36:16.586344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'144","reported_seq":331,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210819+0000","last_change":"2024-04-16T17:47:35.314035+0000","last_active":"2024-04-16T19:35:56.210819+0000","last_peered":"2024-04-16T19:35:56.210819+0000","last_clean":"2024-04-16T19:35:56.210819+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:35:56.210819+0000","last_undegraded":"2024-04-16T19:35:56.210819+0000","last_fullsized":"2024-04-16T19:35:56.210819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.214518+0000","last_change":"2024-04-16T17:47:37.334598+0000","last_active":"2024-04-16T19:36:01.214518+0000","last_peered":"2024-04-16T19:36:01.214518+0000","last_clean":"2024-04-16T19:36:01.214518+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:36:01.214518+0000","last_undegraded":"2024-04-16T19:36:01.214518+0000","last_fullsized":"2024-04-16T19:36:01.214518+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209715+0000","last_change":"2024-04-16T17:47:39.339775+0000","last_active":"2024-04-16T19:36:01.209715+0000","last_peered":"2024-04-16T19:36:01.209715+0000","last_clean":"2024-04-16T19:36:01.209715+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:36:01.209715+0000","last_undegraded":"2024-04-16T19:36:01.209715+0000","last_fullsized":"2024-04-16T19:36:01.209715+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915237+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:36:18.915237+0000","last_peered":"2024-04-16T19:36:18.915237+0000","last_clean":"2024-04-16T19:36:18.915237+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:36:18.915237+0000","last_undegraded":"2024-04-16T19:36:18.915237+0000","last_fullsized":"2024-04-16T19:36:18.915237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914927+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:36:18.914927+0000","last_peered":"2024-04-16T19:36:18.914927+0000","last_clean":"2024-04-16T19:36:18.914927+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:36:18.914927+0000","last_undegraded":"2024-04-16T19:36:18.914927+0000","last_fullsized":"2024-04-16T19:36:18.914927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914754+0000","last_change":"2024-04-16T17:48:27.307048+0000","last_active":"2024-04-16T19:36:18.914754+0000","last_peered":"2024-04-16T19:36:18.914754+0000","last_clean":"2024-04-16T19:36:18.914754+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:36:18.914754+0000","last_undegraded":"2024-04-16T19:36:18.914754+0000","last_fullsized":"2024-04-16T19:36:18.914754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'33","reported_seq":196,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980147+0000","last_change":"2024-04-16T17:48:01.250054+0000","last_active":"2024-04-16T19:36:17.980147+0000","last_peered":"2024-04-16T19:36:17.980147+0000","last_clean":"2024-04-16T19:36:17.980147+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:36:17.980147+0000","last_undegraded":"2024-04-16T19:36:17.980147+0000","last_fullsized":"2024-04-16T19:36:17.980147+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.526931+0000","last_change":"2024-04-16T19:36:34.526983+0000","last_active":"2024-04-16T19:36:34.526931+0000","last_peered":"2024-04-16T19:36:34.526931+0000","last_clean":"2024-04-16T19:36:34.526931+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:36:34.526931+0000","last_undegraded":"2024-04-16T19:36:34.526931+0000","last_fullsized":"2024-04-16T19:36:34.526931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.526798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:20.876919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'282","reported_seq":668,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.418934+0000","last_change":"2024-04-16T19:36:30.419051+0000","last_active":"2024-04-16T19:36:30.418934+0000","last_peered":"2024-04-16T19:36:30.418934+0000","last_clean":"2024-04-16T19:36:30.418934+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:36:30.418934+0000","last_undegraded":"2024-04-16T19:36:30.418934+0000","last_fullsized":"2024-04-16T19:36:30.418934+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'282","last_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_deep_scrub":"53'282","last_deep_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.418793+0000","objects_scrubbed":18,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:10.455320+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'16","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.403615+0000","last_change":"2024-04-16T19:36:29.403701+0000","last_active":"2024-04-16T19:36:29.403615+0000","last_peered":"2024-04-16T19:36:29.403615+0000","last_clean":"2024-04-16T19:36:29.403615+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:36:29.403615+0000","last_undegraded":"2024-04-16T19:36:29.403615+0000","last_fullsized":"2024-04-16T19:36:29.403615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.403484+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:20.878671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.208067+0000","last_change":"2024-04-16T17:47:23.265711+0000","last_active":"2024-04-16T19:35:46.208067+0000","last_peered":"2024-04-16T19:35:46.208067+0000","last_clean":"2024-04-16T19:35:46.208067+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:35:46.208067+0000","last_undegraded":"2024-04-16T19:35:46.208067+0000","last_fullsized":"2024-04-16T19:35:46.208067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189694+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:34:56.189694+0000","last_peered":"2024-04-16T19:34:56.189694+0000","last_clean":"2024-04-16T19:34:56.189694+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:34:56.189694+0000","last_undegraded":"2024-04-16T19:34:56.189694+0000","last_fullsized":"2024-04-16T19:34:56.189694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209117+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:35:51.209117+0000","last_peered":"2024-04-16T19:35:51.209117+0000","last_clean":"2024-04-16T19:35:51.209117+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:35:51.209117+0000","last_undegraded":"2024-04-16T19:35:51.209117+0000","last_fullsized":"2024-04-16T19:35:51.209117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"48'4","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189709+0000","last_change":"2024-04-16T17:47:33.301081+0000","last_active":"2024-04-16T19:35:56.189709+0000","last_peered":"2024-04-16T19:35:56.189709+0000","last_clean":"2024-04-16T19:35:56.189709+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:35:56.189709+0000","last_undegraded":"2024-04-16T19:35:56.189709+0000","last_fullsized":"2024-04-16T19:35:56.189709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'151","reported_seq":1557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912876+0000","last_change":"2024-04-16T17:47:35.316795+0000","last_active":"2024-04-16T19:36:18.912876+0000","last_peered":"2024-04-16T19:36:18.912876+0000","last_clean":"2024-04-16T19:36:18.912876+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:36:18.912876+0000","last_undegraded":"2024-04-16T19:36:18.912876+0000","last_fullsized":"2024-04-16T19:36:18.912876+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210142+0000","last_change":"2024-04-16T17:47:37.326201+0000","last_active":"2024-04-16T19:36:01.210142+0000","last_peered":"2024-04-16T19:36:01.210142+0000","last_clean":"2024-04-16T19:36:01.210142+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:36:01.210142+0000","last_undegraded":"2024-04-16T19:36:01.210142+0000","last_fullsized":"2024-04-16T19:36:01.210142+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205180+0000","last_change":"2024-04-16T17:47:39.332609+0000","last_active":"2024-04-16T19:35:41.205180+0000","last_peered":"2024-04-16T19:35:41.205180+0000","last_clean":"2024-04-16T19:35:41.205180+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:35:41.205180+0000","last_undegraded":"2024-04-16T19:35:41.205180+0000","last_fullsized":"2024-04-16T19:35:41.205180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.440188+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:36:18.440188+0000","last_peered":"2024-04-16T19:36:18.440188+0000","last_clean":"2024-04-16T19:36:18.440188+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:36:18.440188+0000","last_undegraded":"2024-04-16T19:36:18.440188+0000","last_fullsized":"2024-04-16T19:36:18.440188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'76","reported_seq":1160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587954+0000","last_change":"2024-04-16T17:48:34.318880+0000","last_active":"2024-04-16T19:36:16.587954+0000","last_peered":"2024-04-16T19:36:16.587954+0000","last_clean":"2024-04-16T19:36:16.587954+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:36:16.587954+0000","last_undegraded":"2024-04-16T19:36:16.587954+0000","last_fullsized":"2024-04-16T19:36:16.587954+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.188956+0000","last_change":"2024-04-16T17:48:27.299238+0000","last_active":"2024-04-16T19:34:41.188956+0000","last_peered":"2024-04-16T19:34:41.188956+0000","last_clean":"2024-04-16T19:34:41.188956+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:34:41.188956+0000","last_undegraded":"2024-04-16T19:34:41.188956+0000","last_fullsized":"2024-04-16T19:34:41.188956+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'6","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210230+0000","last_change":"2024-04-16T17:48:01.256444+0000","last_active":"2024-04-16T19:36:01.210230+0000","last_peered":"2024-04-16T19:36:01.210230+0000","last_clean":"2024-04-16T19:36:01.210230+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:36:01.210230+0000","last_undegraded":"2024-04-16T19:36:01.210230+0000","last_fullsized":"2024-04-16T19:36:01.210230+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'555","reported_seq":1934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528600+0000","last_change":"2024-04-16T17:47:59.183633+0000","last_active":"2024-04-16T19:36:17.528600+0000","last_peered":"2024-04-16T19:36:17.528600+0000","last_clean":"2024-04-16T19:36:17.528600+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:36:17.528600+0000","last_undegraded":"2024-04-16T19:36:17.528600+0000","last_fullsized":"2024-04-16T19:36:17.528600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'327","reported_seq":654,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.537153+0000","last_change":"2024-04-16T19:36:23.537297+0000","last_active":"2024-04-16T19:36:23.537153+0000","last_peered":"2024-04-16T19:36:23.537153+0000","last_clean":"2024-04-16T19:36:23.537153+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:36:23.537153+0000","last_undegraded":"2024-04-16T19:36:23.537153+0000","last_fullsized":"2024-04-16T19:36:23.537153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'327","last_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_deep_scrub":"53'327","last_deep_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.537014+0000","objects_scrubbed":21,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:14.538959+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'7","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.487632+0000","last_change":"2024-04-16T19:36:18.487675+0000","last_active":"2024-04-16T19:36:18.487632+0000","last_peered":"2024-04-16T19:36:18.487632+0000","last_clean":"2024-04-16T19:36:18.487632+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:36:18.487632+0000","last_undegraded":"2024-04-16T19:36:18.487632+0000","last_fullsized":"2024-04-16T19:36:18.487632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.487557+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:46.066228+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206529+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:35:46.206529+0000","last_peered":"2024-04-16T19:35:46.206529+0000","last_clean":"2024-04-16T19:35:46.206529+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:35:46.206529+0000","last_undegraded":"2024-04-16T19:35:46.206529+0000","last_fullsized":"2024-04-16T19:35:46.206529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.196366+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:35:01.196366+0000","last_peered":"2024-04-16T19:35:01.196366+0000","last_clean":"2024-04-16T19:35:01.196366+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:35:01.196366+0000","last_undegraded":"2024-04-16T19:35:01.196366+0000","last_fullsized":"2024-04-16T19:35:01.196366+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":293,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.202966+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:35:16.202966+0000","last_peered":"2024-04-16T19:35:16.202966+0000","last_clean":"2024-04-16T19:35:16.202966+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:35:16.202966+0000","last_undegraded":"2024-04-16T19:35:16.202966+0000","last_fullsized":"2024-04-16T19:35:16.202966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205065+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:35:41.205065+0000","last_peered":"2024-04-16T19:35:41.205065+0000","last_clean":"2024-04-16T19:35:41.205065+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:35:41.205065+0000","last_undegraded":"2024-04-16T19:35:41.205065+0000","last_fullsized":"2024-04-16T19:35:41.205065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":505,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208846+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:35:56.208846+0000","last_peered":"2024-04-16T19:35:56.208846+0000","last_clean":"2024-04-16T19:35:56.208846+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:35:56.208846+0000","last_undegraded":"2024-04-16T19:35:56.208846+0000","last_fullsized":"2024-04-16T19:35:56.208846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'177","reported_seq":1613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210760+0000","last_change":"2024-04-16T17:47:35.312413+0000","last_active":"2024-04-16T19:35:56.210760+0000","last_peered":"2024-04-16T19:35:56.210760+0000","last_clean":"2024-04-16T19:35:56.210760+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:35:56.210760+0000","last_undegraded":"2024-04-16T19:35:56.210760+0000","last_fullsized":"2024-04-16T19:35:56.210760+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438305+0000","last_change":"2024-04-16T17:47:37.327792+0000","last_active":"2024-04-16T19:36:18.438305+0000","last_peered":"2024-04-16T19:36:18.438305+0000","last_clean":"2024-04-16T19:36:18.438305+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:36:18.438305+0000","last_undegraded":"2024-04-16T19:36:18.438305+0000","last_fullsized":"2024-04-16T19:36:18.438305+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.464452+0000","last_change":"2024-04-16T19:36:28.464452+0000","last_active":"2024-04-16T19:36:28.464452+0000","last_peered":"2024-04-16T19:36:28.464452+0000","last_clean":"2024-04-16T19:36:28.464452+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:36:28.464452+0000","last_undegraded":"2024-04-16T19:36:28.464452+0000","last_fullsized":"2024-04-16T19:36:28.464452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.464263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:07.132103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.188297+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:35:51.188297+0000","last_peered":"2024-04-16T19:35:51.188297+0000","last_clean":"2024-04-16T19:35:51.188297+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:35:51.188297+0000","last_undegraded":"2024-04-16T19:35:51.188297+0000","last_fullsized":"2024-04-16T19:35:51.188297+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1180,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588024+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:36:16.588024+0000","last_peered":"2024-04-16T19:36:16.588024+0000","last_clean":"2024-04-16T19:36:16.588024+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:36:16.588024+0000","last_undegraded":"2024-04-16T19:36:16.588024+0000","last_fullsized":"2024-04-16T19:36:16.588024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.171648+0000","last_change":"2024-04-16T17:48:27.304274+0000","last_active":"2024-04-16T19:34:41.171648+0000","last_peered":"2024-04-16T19:34:41.171648+0000","last_clean":"2024-04-16T19:34:41.171648+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:34:41.171648+0000","last_undegraded":"2024-04-16T19:34:41.171648+0000","last_fullsized":"2024-04-16T19:34:41.171648+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'19","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.529264+0000","last_change":"2024-04-16T19:36:30.529412+0000","last_active":"2024-04-16T19:36:30.529264+0000","last_peered":"2024-04-16T19:36:30.529264+0000","last_clean":"2024-04-16T19:36:30.529264+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:36:30.529264+0000","last_undegraded":"2024-04-16T19:36:30.529264+0000","last_fullsized":"2024-04-16T19:36:30.529264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.529097+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:25.629336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'556","reported_seq":1933,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.659990+0000","last_change":"2024-04-16T19:36:28.660054+0000","last_active":"2024-04-16T19:36:28.659990+0000","last_peered":"2024-04-16T19:36:28.659990+0000","last_clean":"2024-04-16T19:36:28.659990+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:36:28.659990+0000","last_undegraded":"2024-04-16T19:36:28.659990+0000","last_fullsized":"2024-04-16T19:36:28.659990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.659827+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:31.852118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'532","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.416491+0000","last_change":"2024-04-16T19:36:24.416491+0000","last_active":"2024-04-16T19:36:24.416491+0000","last_peered":"2024-04-16T19:36:24.416491+0000","last_clean":"2024-04-16T19:36:24.416491+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:36:24.416491+0000","last_undegraded":"2024-04-16T19:36:24.416491+0000","last_fullsized":"2024-04-16T19:36:24.416491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.416318+0000","objects_scrubbed":18,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:15.418263+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"53'6","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.488767+0000","last_change":"2024-04-16T19:36:23.488844+0000","last_active":"2024-04-16T19:36:23.488767+0000","last_peered":"2024-04-16T19:36:23.488767+0000","last_clean":"2024-04-16T19:36:23.488767+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:36:23.488767+0000","last_undegraded":"2024-04-16T19:36:23.488767+0000","last_fullsized":"2024-04-16T19:36:23.488767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.488637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:34:36.790606+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202127+0000","last_change":"2024-04-16T17:47:23.266373+0000","last_active":"2024-04-16T19:35:46.202127+0000","last_peered":"2024-04-16T19:35:46.202127+0000","last_clean":"2024-04-16T19:35:46.202127+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:35:46.202127+0000","last_undegraded":"2024-04-16T19:35:46.202127+0000","last_fullsized":"2024-04-16T19:35:46.202127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189734+0000","last_change":"2024-04-16T17:47:32.273580+0000","last_active":"2024-04-16T19:35:56.189734+0000","last_peered":"2024-04-16T19:35:56.189734+0000","last_clean":"2024-04-16T19:35:56.189734+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:35:56.189734+0000","last_undegraded":"2024-04-16T19:35:56.189734+0000","last_fullsized":"2024-04-16T19:35:56.189734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.196438+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:35:01.196438+0000","last_peered":"2024-04-16T19:35:01.196438+0000","last_clean":"2024-04-16T19:35:01.196438+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:35:01.196438+0000","last_undegraded":"2024-04-16T19:35:01.196438+0000","last_fullsized":"2024-04-16T19:35:01.196438+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.192014+0000","last_change":"2024-04-16T17:47:33.300039+0000","last_active":"2024-04-16T19:36:06.192014+0000","last_peered":"2024-04-16T19:36:06.192014+0000","last_clean":"2024-04-16T19:36:06.192014+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:36:06.192014+0000","last_undegraded":"2024-04-16T19:36:06.192014+0000","last_fullsized":"2024-04-16T19:36:06.192014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'144","reported_seq":348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204740+0000","last_change":"2024-04-16T17:47:35.305470+0000","last_active":"2024-04-16T19:35:56.204740+0000","last_peered":"2024-04-16T19:35:56.204740+0000","last_clean":"2024-04-16T19:35:56.204740+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:35:56.204740+0000","last_undegraded":"2024-04-16T19:35:56.204740+0000","last_fullsized":"2024-04-16T19:35:56.204740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.190843+0000","last_change":"2024-04-16T17:47:37.334307+0000","last_active":"2024-04-16T19:36:01.190843+0000","last_peered":"2024-04-16T19:36:01.190843+0000","last_clean":"2024-04-16T19:36:01.190843+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:36:01.190843+0000","last_undegraded":"2024-04-16T19:36:01.190843+0000","last_fullsized":"2024-04-16T19:36:01.190843+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.204771+0000","last_change":"2024-04-16T17:47:39.343779+0000","last_active":"2024-04-16T19:35:41.204771+0000","last_peered":"2024-04-16T19:35:41.204771+0000","last_clean":"2024-04-16T19:35:41.204771+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:35:41.204771+0000","last_undegraded":"2024-04-16T19:35:41.204771+0000","last_fullsized":"2024-04-16T19:35:41.204771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528888+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:36:17.528888+0000","last_peered":"2024-04-16T19:36:17.528888+0000","last_clean":"2024-04-16T19:36:17.528888+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:36:17.528888+0000","last_undegraded":"2024-04-16T19:36:17.528888+0000","last_fullsized":"2024-04-16T19:36:17.528888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'122","reported_seq":1764,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204310+0000","last_change":"2024-04-16T17:48:34.337192+0000","last_active":"2024-04-16T19:35:21.204310+0000","last_peered":"2024-04-16T19:35:21.204310+0000","last_clean":"2024-04-16T19:35:21.204310+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:35:21.204310+0000","last_undegraded":"2024-04-16T19:35:21.204310+0000","last_fullsized":"2024-04-16T19:35:21.204310+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.188600+0000","last_change":"2024-04-16T17:48:27.301617+0000","last_active":"2024-04-16T19:34:41.188600+0000","last_peered":"2024-04-16T19:34:41.188600+0000","last_clean":"2024-04-16T19:34:41.188600+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:34:41.188600+0000","last_undegraded":"2024-04-16T19:34:41.188600+0000","last_fullsized":"2024-04-16T19:34:41.188600+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'20","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.530462+0000","last_change":"2024-04-16T19:36:31.530526+0000","last_active":"2024-04-16T19:36:31.530462+0000","last_peered":"2024-04-16T19:36:31.530462+0000","last_clean":"2024-04-16T19:36:31.530462+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:36:31.530462+0000","last_undegraded":"2024-04-16T19:36:31.530462+0000","last_fullsized":"2024-04-16T19:36:31.530462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.530356+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:33:36.382371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.979971+0000","last_change":"2024-04-16T17:47:59.185616+0000","last_active":"2024-04-16T19:36:17.979971+0000","last_peered":"2024-04-16T19:36:17.979971+0000","last_clean":"2024-04-16T19:36:17.979971+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:36:17.979971+0000","last_undegraded":"2024-04-16T19:36:17.979971+0000","last_fullsized":"2024-04-16T19:36:17.979971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1182","reported_seq":1443,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.649769+0000","last_change":"2024-04-16T19:36:23.649928+0000","last_active":"2024-04-16T19:36:23.649769+0000","last_peered":"2024-04-16T19:36:23.649769+0000","last_clean":"2024-04-16T19:36:23.649769+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:36:23.649769+0000","last_undegraded":"2024-04-16T19:36:23.649769+0000","last_fullsized":"2024-04-16T19:36:23.649769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1182","last_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_deep_scrub":"53'1182","last_deep_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.649623+0000","objects_scrubbed":23,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:09.999744+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215927,"num_omap_keys":942,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'28","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.423045+0000","last_change":"2024-04-16T19:36:25.423158+0000","last_active":"2024-04-16T19:36:25.423045+0000","last_peered":"2024-04-16T19:36:25.423045+0000","last_clean":"2024-04-16T19:36:25.423045+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:36:25.423045+0000","last_undegraded":"2024-04-16T19:36:25.423045+0000","last_fullsized":"2024-04-16T19:36:25.423045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'28","last_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_deep_scrub":"53'28","last_deep_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.422911+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:22.027239+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209861+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:36:01.209861+0000","last_peered":"2024-04-16T19:36:01.209861+0000","last_clean":"2024-04-16T19:36:01.209861+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:36:01.209861+0000","last_undegraded":"2024-04-16T19:36:01.209861+0000","last_fullsized":"2024-04-16T19:36:01.209861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186094+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:35:41.186094+0000","last_peered":"2024-04-16T19:35:41.186094+0000","last_clean":"2024-04-16T19:35:41.186094+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:35:41.186094+0000","last_undegraded":"2024-04-16T19:35:41.186094+0000","last_fullsized":"2024-04-16T19:35:41.186094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.199839+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:35:36.199839+0000","last_peered":"2024-04-16T19:35:36.199839+0000","last_clean":"2024-04-16T19:35:36.199839+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:35:36.199839+0000","last_undegraded":"2024-04-16T19:35:36.199839+0000","last_fullsized":"2024-04-16T19:35:36.199839+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213000+0000","last_change":"2024-04-16T17:47:33.300948+0000","last_active":"2024-04-16T19:35:56.213000+0000","last_peered":"2024-04-16T19:35:56.213000+0000","last_clean":"2024-04-16T19:35:56.213000+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:35:56.213000+0000","last_undegraded":"2024-04-16T19:35:56.213000+0000","last_fullsized":"2024-04-16T19:35:56.213000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'120","reported_seq":297,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213049+0000","last_change":"2024-04-16T17:47:35.313794+0000","last_active":"2024-04-16T19:35:56.213049+0000","last_peered":"2024-04-16T19:35:56.213049+0000","last_clean":"2024-04-16T19:35:56.213049+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:35:56.213049+0000","last_undegraded":"2024-04-16T19:35:56.213049+0000","last_fullsized":"2024-04-16T19:35:56.213049+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'12","reported_seq":1341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210065+0000","last_change":"2024-04-16T17:47:37.329509+0000","last_active":"2024-04-16T19:36:01.210065+0000","last_peered":"2024-04-16T19:36:01.210065+0000","last_clean":"2024-04-16T19:36:01.210065+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:36:01.210065+0000","last_undegraded":"2024-04-16T19:36:01.210065+0000","last_fullsized":"2024-04-16T19:36:01.210065+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.534123+0000","last_change":"2024-04-16T19:36:35.534176+0000","last_active":"2024-04-16T19:36:35.534123+0000","last_peered":"2024-04-16T19:36:35.534123+0000","last_clean":"2024-04-16T19:36:35.534123+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:36:35.534123+0000","last_undegraded":"2024-04-16T19:36:35.534123+0000","last_fullsized":"2024-04-16T19:36:35.534123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.533984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:55.743351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181002+0000","last_change":"2024-04-16T18:10:50.537157+0000","last_active":"2024-04-16T19:35:21.181002+0000","last_peered":"2024-04-16T19:35:21.181002+0000","last_clean":"2024-04-16T19:35:21.181002+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:35:21.181002+0000","last_undegraded":"2024-04-16T19:35:21.181002+0000","last_fullsized":"2024-04-16T19:35:21.181002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1283,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.189839+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:34:46.189839+0000","last_peered":"2024-04-16T19:34:46.189839+0000","last_clean":"2024-04-16T19:34:46.189839+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:34:46.189839+0000","last_undegraded":"2024-04-16T19:34:46.189839+0000","last_fullsized":"2024-04-16T19:34:46.189839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.178823+0000","last_change":"2024-04-16T17:48:27.305636+0000","last_active":"2024-04-16T19:35:11.178823+0000","last_peered":"2024-04-16T19:35:11.178823+0000","last_clean":"2024-04-16T19:35:11.178823+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:35:11.178823+0000","last_undegraded":"2024-04-16T19:35:11.178823+0000","last_fullsized":"2024-04-16T19:35:11.178823+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'13","reported_seq":148,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210014+0000","last_change":"2024-04-16T17:48:01.266136+0000","last_active":"2024-04-16T19:36:01.210014+0000","last_peered":"2024-04-16T19:36:01.210014+0000","last_clean":"2024-04-16T19:36:01.210014+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:36:01.210014+0000","last_undegraded":"2024-04-16T19:36:01.210014+0000","last_fullsized":"2024-04-16T19:36:01.210014+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.212043+0000","last_change":"2024-04-16T17:47:59.188856+0000","last_active":"2024-04-16T19:36:01.212043+0000","last_peered":"2024-04-16T19:36:01.212043+0000","last_clean":"2024-04-16T19:36:01.212043+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:36:01.212043+0000","last_undegraded":"2024-04-16T19:36:01.212043+0000","last_fullsized":"2024-04-16T19:36:01.212043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1343","reported_seq":1906,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.619403+0000","last_change":"2024-04-16T19:36:21.619494+0000","last_active":"2024-04-16T19:36:21.619403+0000","last_peered":"2024-04-16T19:36:21.619403+0000","last_clean":"2024-04-16T19:36:21.619403+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:36:21.619403+0000","last_undegraded":"2024-04-16T19:36:21.619403+0000","last_fullsized":"2024-04-16T19:36:21.619403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1343","last_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_deep_scrub":"53'1343","last_deep_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.619266+0000","objects_scrubbed":29,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:34.145789+0000","scrub_duration":71,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":216307,"num_omap_keys":944,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'8","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.444455+0000","last_change":"2024-04-16T19:36:33.444542+0000","last_active":"2024-04-16T19:36:33.444455+0000","last_peered":"2024-04-16T19:36:33.444455+0000","last_clean":"2024-04-16T19:36:33.444455+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:36:33.444455+0000","last_undegraded":"2024-04-16T19:36:33.444455+0000","last_fullsized":"2024-04-16T19:36:33.444455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.444316+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:36.300483+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206348+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:35:46.206348+0000","last_peered":"2024-04-16T19:35:46.206348+0000","last_clean":"2024-04-16T19:35:46.206348+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:35:46.206348+0000","last_undegraded":"2024-04-16T19:35:46.206348+0000","last_fullsized":"2024-04-16T19:35:46.206348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206492+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:35:41.206492+0000","last_peered":"2024-04-16T19:35:41.206492+0000","last_clean":"2024-04-16T19:35:41.206492+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:35:41.206492+0000","last_undegraded":"2024-04-16T19:35:41.206492+0000","last_fullsized":"2024-04-16T19:35:41.206492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.198223+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:34:56.198223+0000","last_peered":"2024-04-16T19:34:56.198223+0000","last_clean":"2024-04-16T19:34:56.198223+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:34:56.198223+0000","last_undegraded":"2024-04-16T19:34:56.198223+0000","last_fullsized":"2024-04-16T19:34:56.198223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208557+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:35:56.208557+0000","last_peered":"2024-04-16T19:35:56.208557+0000","last_clean":"2024-04-16T19:35:56.208557+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:35:56.208557+0000","last_undegraded":"2024-04-16T19:35:56.208557+0000","last_fullsized":"2024-04-16T19:35:56.208557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'168","reported_seq":369,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210580+0000","last_change":"2024-04-16T17:47:35.321995+0000","last_active":"2024-04-16T19:35:56.210580+0000","last_peered":"2024-04-16T19:35:56.210580+0000","last_clean":"2024-04-16T19:35:56.210580+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:35:56.210580+0000","last_undegraded":"2024-04-16T19:35:56.210580+0000","last_fullsized":"2024-04-16T19:35:56.210580+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'12","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211023+0000","last_change":"2024-04-16T17:47:37.323586+0000","last_active":"2024-04-16T19:36:06.211023+0000","last_peered":"2024-04-16T19:36:06.211023+0000","last_clean":"2024-04-16T19:36:06.211023+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:36:06.211023+0000","last_undegraded":"2024-04-16T19:36:06.211023+0000","last_fullsized":"2024-04-16T19:36:06.211023+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.501363+0000","last_change":"2024-04-16T19:36:34.501465+0000","last_active":"2024-04-16T19:36:34.501363+0000","last_peered":"2024-04-16T19:36:34.501363+0000","last_clean":"2024-04-16T19:36:34.501363+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:36:34.501363+0000","last_undegraded":"2024-04-16T19:36:34.501363+0000","last_fullsized":"2024-04-16T19:36:34.501363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.501193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:08.812307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.198858+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:35:21.198858+0000","last_peered":"2024-04-16T19:35:21.198858+0000","last_clean":"2024-04-16T19:35:21.198858+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:35:21.198858+0000","last_undegraded":"2024-04-16T19:35:21.198858+0000","last_fullsized":"2024-04-16T19:35:21.198858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1187,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.191107+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:34:46.191107+0000","last_peered":"2024-04-16T19:34:46.191107+0000","last_clean":"2024-04-16T19:34:46.191107+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:34:46.191107+0000","last_undegraded":"2024-04-16T19:34:46.191107+0000","last_fullsized":"2024-04-16T19:34:46.191107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980422+0000","last_change":"2024-04-16T17:48:27.298996+0000","last_active":"2024-04-16T19:36:17.980422+0000","last_peered":"2024-04-16T19:36:17.980422+0000","last_clean":"2024-04-16T19:36:17.980422+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:36:17.980422+0000","last_undegraded":"2024-04-16T19:36:17.980422+0000","last_fullsized":"2024-04-16T19:36:17.980422+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"53'7","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.547816+0000","last_change":"2024-04-16T19:36:32.547894+0000","last_active":"2024-04-16T19:36:32.547816+0000","last_peered":"2024-04-16T19:36:32.547816+0000","last_clean":"2024-04-16T19:36:32.547816+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:36:32.547816+0000","last_undegraded":"2024-04-16T19:36:32.547816+0000","last_fullsized":"2024-04-16T19:36:32.547816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.547672+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:45.074195+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'556","reported_seq":2004,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.565740+0000","last_change":"2024-04-16T19:36:33.565806+0000","last_active":"2024-04-16T19:36:33.565740+0000","last_peered":"2024-04-16T19:36:33.565740+0000","last_clean":"2024-04-16T19:36:33.565740+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:36:33.565740+0000","last_undegraded":"2024-04-16T19:36:33.565740+0000","last_fullsized":"2024-04-16T19:36:33.565740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.565603+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:32.212045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'575","reported_seq":853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.628047+0000","last_change":"2024-04-16T19:36:22.628176+0000","last_active":"2024-04-16T19:36:22.628047+0000","last_peered":"2024-04-16T19:36:22.628047+0000","last_clean":"2024-04-16T19:36:22.628047+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:36:22.628047+0000","last_undegraded":"2024-04-16T19:36:22.628047+0000","last_fullsized":"2024-04-16T19:36:22.628047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'575","last_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_deep_scrub":"53'575","last_deep_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.627889+0000","objects_scrubbed":22,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:21.274331+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71488,"num_omap_keys":312,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'18","reported_seq":261,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.483626+0000","last_change":"2024-04-16T19:36:29.483703+0000","last_active":"2024-04-16T19:36:29.483626+0000","last_peered":"2024-04-16T19:36:29.483626+0000","last_clean":"2024-04-16T19:36:29.483626+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:36:29.483626+0000","last_undegraded":"2024-04-16T19:36:29.483626+0000","last_fullsized":"2024-04-16T19:36:29.483626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.483500+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:36:53.084475+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207780+0000","last_change":"2024-04-16T17:47:23.265822+0000","last_active":"2024-04-16T19:35:46.207780+0000","last_peered":"2024-04-16T19:35:46.207780+0000","last_clean":"2024-04-16T19:35:46.207780+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:35:46.207780+0000","last_undegraded":"2024-04-16T19:35:46.207780+0000","last_fullsized":"2024-04-16T19:35:46.207780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211258+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:36:06.211258+0000","last_peered":"2024-04-16T19:36:06.211258+0000","last_clean":"2024-04-16T19:36:06.211258+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:36:06.211258+0000","last_undegraded":"2024-04-16T19:36:06.211258+0000","last_fullsized":"2024-04-16T19:36:06.211258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192623+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:34:56.192623+0000","last_peered":"2024-04-16T19:34:56.192623+0000","last_clean":"2024-04-16T19:34:56.192623+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:34:56.192623+0000","last_undegraded":"2024-04-16T19:34:56.192623+0000","last_fullsized":"2024-04-16T19:34:56.192623+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208829+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:35:56.208829+0000","last_peered":"2024-04-16T19:35:56.208829+0000","last_clean":"2024-04-16T19:35:56.208829+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:35:56.208829+0000","last_undegraded":"2024-04-16T19:35:56.208829+0000","last_fullsized":"2024-04-16T19:35:56.208829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'160","reported_seq":369,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212483+0000","last_change":"2024-04-16T17:47:35.312410+0000","last_active":"2024-04-16T19:36:11.212483+0000","last_peered":"2024-04-16T19:36:11.212483+0000","last_clean":"2024-04-16T19:36:11.212483+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:36:11.212483+0000","last_undegraded":"2024-04-16T19:36:11.212483+0000","last_fullsized":"2024-04-16T19:36:11.212483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.205991+0000","last_change":"2024-04-16T17:47:37.322938+0000","last_active":"2024-04-16T19:36:01.205991+0000","last_peered":"2024-04-16T19:36:01.205991+0000","last_clean":"2024-04-16T19:36:01.205991+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:36:01.205991+0000","last_undegraded":"2024-04-16T19:36:01.205991+0000","last_fullsized":"2024-04-16T19:36:01.205991+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.422338+0000","last_change":"2024-04-16T19:36:32.422395+0000","last_active":"2024-04-16T19:36:32.422338+0000","last_peered":"2024-04-16T19:36:32.422338+0000","last_clean":"2024-04-16T19:36:32.422338+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:36:32.422338+0000","last_undegraded":"2024-04-16T19:36:32.422338+0000","last_fullsized":"2024-04-16T19:36:32.422338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.422189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:35.278356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201369+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:35:26.201369+0000","last_peered":"2024-04-16T19:35:26.201369+0000","last_clean":"2024-04-16T19:35:26.201369+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:35:26.201369+0000","last_undegraded":"2024-04-16T19:35:26.201369+0000","last_fullsized":"2024-04-16T19:35:26.201369+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.206683+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:35:31.206683+0000","last_peered":"2024-04-16T19:35:31.206683+0000","last_clean":"2024-04-16T19:35:31.206683+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:35:31.206683+0000","last_undegraded":"2024-04-16T19:35:31.206683+0000","last_fullsized":"2024-04-16T19:35:31.206683+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.211924+0000","last_change":"2024-04-16T17:48:27.307428+0000","last_active":"2024-04-16T19:36:01.211924+0000","last_peered":"2024-04-16T19:36:01.211924+0000","last_clean":"2024-04-16T19:36:01.211924+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:36:01.211924+0000","last_undegraded":"2024-04-16T19:36:01.211924+0000","last_fullsized":"2024-04-16T19:36:01.211924+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'16","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.198514+0000","last_change":"2024-04-16T17:48:01.240690+0000","last_active":"2024-04-16T19:35:31.198514+0000","last_peered":"2024-04-16T19:35:31.198514+0000","last_clean":"2024-04-16T19:35:31.198514+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:35:31.198514+0000","last_undegraded":"2024-04-16T19:35:31.198514+0000","last_fullsized":"2024-04-16T19:35:31.198514+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.501219+0000","last_change":"2024-04-16T19:36:35.501282+0000","last_active":"2024-04-16T19:36:35.501219+0000","last_peered":"2024-04-16T19:36:35.501219+0000","last_clean":"2024-04-16T19:36:35.501219+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:36:35.501219+0000","last_undegraded":"2024-04-16T19:36:35.501219+0000","last_fullsized":"2024-04-16T19:36:35.501219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.501075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:40.913802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'871","reported_seq":1181,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.697221+0000","last_change":"2024-04-16T19:36:18.697375+0000","last_active":"2024-04-16T19:36:18.697221+0000","last_peered":"2024-04-16T19:36:18.697221+0000","last_clean":"2024-04-16T19:36:18.697221+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:36:18.697221+0000","last_undegraded":"2024-04-16T19:36:18.697221+0000","last_fullsized":"2024-04-16T19:36:18.697221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'871","last_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_deep_scrub":"53'871","last_deep_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.697076+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:49.607583+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143413,"num_omap_keys":626,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'16","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.208672+0000","last_change":"2024-04-16T17:49:32.398153+0000","last_active":"2024-04-16T19:36:11.208672+0000","last_peered":"2024-04-16T19:36:11.208672+0000","last_clean":"2024-04-16T19:36:11.208672+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:36:11.208672+0000","last_undegraded":"2024-04-16T19:36:11.208672+0000","last_fullsized":"2024-04-16T19:36:11.208672+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210322+0000","last_change":"2024-04-16T17:47:39.336834+0000","last_active":"2024-04-16T19:36:01.210322+0000","last_peered":"2024-04-16T19:36:01.210322+0000","last_clean":"2024-04-16T19:36:01.210322+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:36:01.210322+0000","last_undegraded":"2024-04-16T19:36:01.210322+0000","last_fullsized":"2024-04-16T19:36:01.210322+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.187216+0000","last_change":"2024-04-16T17:47:23.257996+0000","last_active":"2024-04-16T19:34:46.187216+0000","last_peered":"2024-04-16T19:34:46.187216+0000","last_clean":"2024-04-16T19:34:46.187216+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:34:46.187216+0000","last_undegraded":"2024-04-16T19:34:46.187216+0000","last_fullsized":"2024-04-16T19:34:46.187216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.203208+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:35:16.203208+0000","last_peered":"2024-04-16T19:35:16.203208+0000","last_clean":"2024-04-16T19:35:16.203208+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:35:16.203208+0000","last_undegraded":"2024-04-16T19:35:16.203208+0000","last_fullsized":"2024-04-16T19:35:16.203208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"48'12","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194889+0000","last_change":"2024-04-16T17:47:33.297781+0000","last_active":"2024-04-16T19:34:56.194889+0000","last_peered":"2024-04-16T19:34:56.194889+0000","last_clean":"2024-04-16T19:34:56.194889+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:34:56.194889+0000","last_undegraded":"2024-04-16T19:34:56.194889+0000","last_fullsized":"2024-04-16T19:34:56.194889+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'104","reported_seq":272,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175329+0000","last_change":"2024-04-16T17:47:35.310182+0000","last_active":"2024-04-16T19:34:56.175329+0000","last_peered":"2024-04-16T19:34:56.175329+0000","last_clean":"2024-04-16T19:34:56.175329+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:34:56.175329+0000","last_undegraded":"2024-04-16T19:34:56.175329+0000","last_fullsized":"2024-04-16T19:34:56.175329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.196232+0000","last_change":"2024-04-16T17:47:37.334854+0000","last_active":"2024-04-16T19:35:01.196232+0000","last_peered":"2024-04-16T19:35:01.196232+0000","last_clean":"2024-04-16T19:35:01.196232+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:35:01.196232+0000","last_undegraded":"2024-04-16T19:35:01.196232+0000","last_fullsized":"2024-04-16T19:35:01.196232+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204551+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:35:56.204551+0000","last_peered":"2024-04-16T19:35:56.204551+0000","last_clean":"2024-04-16T19:35:56.204551+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:35:56.204551+0000","last_undegraded":"2024-04-16T19:35:56.204551+0000","last_fullsized":"2024-04-16T19:35:56.204551+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'55","reported_seq":672,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194769+0000","last_change":"2024-04-16T17:48:34.343212+0000","last_active":"2024-04-16T19:34:56.194769+0000","last_peered":"2024-04-16T19:34:56.194769+0000","last_clean":"2024-04-16T19:34:56.194769+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:34:56.194769+0000","last_undegraded":"2024-04-16T19:34:56.194769+0000","last_fullsized":"2024-04-16T19:34:56.194769+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.206098+0000","last_change":"2024-04-16T17:48:27.301479+0000","last_active":"2024-04-16T19:36:01.206098+0000","last_peered":"2024-04-16T19:36:01.206098+0000","last_clean":"2024-04-16T19:36:01.206098+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:36:01.206098+0000","last_undegraded":"2024-04-16T19:36:01.206098+0000","last_fullsized":"2024-04-16T19:36:01.206098+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'9","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.198414+0000","last_change":"2024-04-16T17:48:01.250254+0000","last_active":"2024-04-16T19:35:31.198414+0000","last_peered":"2024-04-16T19:35:31.198414+0000","last_clean":"2024-04-16T19:35:31.198414+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:35:31.198414+0000","last_undegraded":"2024-04-16T19:35:31.198414+0000","last_fullsized":"2024-04-16T19:35:31.198414+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.206907+0000","last_change":"2024-04-16T17:47:59.176369+0000","last_active":"2024-04-16T19:35:31.206907+0000","last_peered":"2024-04-16T19:35:31.206907+0000","last_clean":"2024-04-16T19:35:31.206907+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:35:31.206907+0000","last_undegraded":"2024-04-16T19:35:31.206907+0000","last_fullsized":"2024-04-16T19:35:31.206907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'616","reported_seq":2312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.598175+0000","last_change":"2024-04-16T19:36:23.598300+0000","last_active":"2024-04-16T19:36:23.598175+0000","last_peered":"2024-04-16T19:36:23.598175+0000","last_clean":"2024-04-16T19:36:23.598175+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:36:23.598175+0000","last_undegraded":"2024-04-16T19:36:23.598175+0000","last_fullsized":"2024-04-16T19:36:23.598175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'616","last_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_deep_scrub":"53'616","last_deep_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.598031+0000","objects_scrubbed":24,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:37.526844+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.510530+0000","last_change":"2024-04-16T19:36:23.510613+0000","last_active":"2024-04-16T19:36:23.510530+0000","last_peered":"2024-04-16T19:36:23.510530+0000","last_clean":"2024-04-16T19:36:23.510530+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:36:23.510530+0000","last_undegraded":"2024-04-16T19:36:23.510530+0000","last_fullsized":"2024-04-16T19:36:23.510530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.510390+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:03.621835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.537581+0000","last_change":"2024-04-16T19:36:32.537632+0000","last_active":"2024-04-16T19:36:32.537581+0000","last_peered":"2024-04-16T19:36:32.537581+0000","last_clean":"2024-04-16T19:36:32.537581+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:36:32.537581+0000","last_undegraded":"2024-04-16T19:36:32.537581+0000","last_fullsized":"2024-04-16T19:36:32.537581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.537440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:47.348326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.192595+0000","last_change":"2024-04-16T17:47:23.260667+0000","last_active":"2024-04-16T19:34:46.192595+0000","last_peered":"2024-04-16T19:34:46.192595+0000","last_clean":"2024-04-16T19:34:46.192595+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:34:46.192595+0000","last_undegraded":"2024-04-16T19:34:46.192595+0000","last_fullsized":"2024-04-16T19:34:46.192595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215650+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:36:06.215650+0000","last_peered":"2024-04-16T19:36:06.215650+0000","last_clean":"2024-04-16T19:36:06.215650+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:36:06.215650+0000","last_undegraded":"2024-04-16T19:36:06.215650+0000","last_fullsized":"2024-04-16T19:36:06.215650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"48'1","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194967+0000","last_change":"2024-04-16T17:47:33.297748+0000","last_active":"2024-04-16T19:34:56.194967+0000","last_peered":"2024-04-16T19:34:56.194967+0000","last_clean":"2024-04-16T19:34:56.194967+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:34:56.194967+0000","last_undegraded":"2024-04-16T19:34:56.194967+0000","last_fullsized":"2024-04-16T19:34:56.194967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'50","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193696+0000","last_change":"2024-04-16T17:47:35.314682+0000","last_active":"2024-04-16T19:34:56.193696+0000","last_peered":"2024-04-16T19:34:56.193696+0000","last_clean":"2024-04-16T19:34:56.193696+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:34:56.193696+0000","last_undegraded":"2024-04-16T19:34:56.193696+0000","last_fullsized":"2024-04-16T19:34:56.193696+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.199285+0000","last_change":"2024-04-16T17:47:37.327586+0000","last_active":"2024-04-16T19:35:01.199285+0000","last_peered":"2024-04-16T19:35:01.199285+0000","last_clean":"2024-04-16T19:35:01.199285+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:35:01.199285+0000","last_undegraded":"2024-04-16T19:35:01.199285+0000","last_fullsized":"2024-04-16T19:35:01.199285+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207221+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:35:51.207221+0000","last_peered":"2024-04-16T19:35:51.207221+0000","last_clean":"2024-04-16T19:35:51.207221+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:35:51.207221+0000","last_undegraded":"2024-04-16T19:35:51.207221+0000","last_fullsized":"2024-04-16T19:35:51.207221+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'99","reported_seq":1688,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.213247+0000","last_change":"2024-04-16T17:48:34.338854+0000","last_active":"2024-04-16T19:36:06.213247+0000","last_peered":"2024-04-16T19:36:06.213247+0000","last_clean":"2024-04-16T19:36:06.213247+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:36:06.213247+0000","last_undegraded":"2024-04-16T19:36:06.213247+0000","last_fullsized":"2024-04-16T19:36:06.213247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191009+0000","last_change":"2024-04-16T17:48:27.301694+0000","last_active":"2024-04-16T19:34:51.191009+0000","last_peered":"2024-04-16T19:34:51.191009+0000","last_clean":"2024-04-16T19:34:51.191009+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:34:51.191009+0000","last_undegraded":"2024-04-16T19:34:51.191009+0000","last_fullsized":"2024-04-16T19:34:51.191009+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'13","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193762+0000","last_change":"2024-04-16T17:48:01.248700+0000","last_active":"2024-04-16T19:34:56.193762+0000","last_peered":"2024-04-16T19:34:56.193762+0000","last_clean":"2024-04-16T19:34:56.193762+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:34:56.193762+0000","last_undegraded":"2024-04-16T19:34:56.193762+0000","last_fullsized":"2024-04-16T19:34:56.193762+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'20","reported_seq":1423,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.913791+0000","last_change":"2024-04-16T17:47:59.196995+0000","last_active":"2024-04-16T19:36:18.913791+0000","last_peered":"2024-04-16T19:36:18.913791+0000","last_clean":"2024-04-16T19:36:18.913791+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:36:18.913791+0000","last_undegraded":"2024-04-16T19:36:18.913791+0000","last_fullsized":"2024-04-16T19:36:18.913791+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'6983","reported_seq":10559,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.979889+0000","last_change":"2024-04-16T17:47:57.195602+0000","last_active":"2024-04-16T19:36:17.979889+0000","last_peered":"2024-04-16T19:36:17.979889+0000","last_clean":"2024-04-16T19:36:17.979889+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:36:17.979889+0000","last_undegraded":"2024-04-16T19:36:17.979889+0000","last_fullsized":"2024-04-16T19:36:17.979889+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:56.136446+0000","last_clean_scrub_stamp":"2024-04-16T17:47:56.136446+0000","objects_scrubbed":0,"log_size":1783,"log_dups_size":3000,"ondisk_log_size":1783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'10","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.437802+0000","last_change":"2024-04-16T19:36:27.437901+0000","last_active":"2024-04-16T19:36:27.437802+0000","last_peered":"2024-04-16T19:36:27.437802+0000","last_clean":"2024-04-16T19:36:27.437802+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:36:27.437802+0000","last_undegraded":"2024-04-16T19:36:27.437802+0000","last_fullsized":"2024-04-16T19:36:27.437802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.437656+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:14.616289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.197683+0000","last_change":"2024-04-16T17:47:39.329121+0000","last_active":"2024-04-16T19:35:11.197683+0000","last_peered":"2024-04-16T19:35:11.197683+0000","last_clean":"2024-04-16T19:35:11.197683+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:35:11.197683+0000","last_undegraded":"2024-04-16T19:35:11.197683+0000","last_fullsized":"2024-04-16T19:35:11.197683+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.172925+0000","last_change":"2024-04-16T17:47:23.269620+0000","last_active":"2024-04-16T19:34:46.172925+0000","last_peered":"2024-04-16T19:34:46.172925+0000","last_clean":"2024-04-16T19:34:46.172925+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:34:46.172925+0000","last_undegraded":"2024-04-16T19:34:46.172925+0000","last_fullsized":"2024-04-16T19:34:46.172925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211353+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:36:06.211353+0000","last_peered":"2024-04-16T19:36:06.211353+0000","last_clean":"2024-04-16T19:36:06.211353+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:36:06.211353+0000","last_undegraded":"2024-04-16T19:36:06.211353+0000","last_fullsized":"2024-04-16T19:36:06.211353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189585+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:34:56.189585+0000","last_peered":"2024-04-16T19:34:56.189585+0000","last_clean":"2024-04-16T19:34:56.189585+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:34:56.189585+0000","last_undegraded":"2024-04-16T19:34:56.189585+0000","last_fullsized":"2024-04-16T19:34:56.189585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'104","reported_seq":272,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175489+0000","last_change":"2024-04-16T17:47:35.310781+0000","last_active":"2024-04-16T19:34:56.175489+0000","last_peered":"2024-04-16T19:34:56.175489+0000","last_clean":"2024-04-16T19:34:56.175489+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:34:56.175489+0000","last_undegraded":"2024-04-16T19:34:56.175489+0000","last_fullsized":"2024-04-16T19:34:56.175489+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.193626+0000","last_change":"2024-04-16T17:47:37.322773+0000","last_active":"2024-04-16T19:35:01.193626+0000","last_peered":"2024-04-16T19:35:01.193626+0000","last_clean":"2024-04-16T19:35:01.193626+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:35:01.193626+0000","last_undegraded":"2024-04-16T19:35:01.193626+0000","last_fullsized":"2024-04-16T19:35:01.193626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.211846+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:35:51.211846+0000","last_peered":"2024-04-16T19:35:51.211846+0000","last_clean":"2024-04-16T19:35:51.211846+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:35:51.211846+0000","last_undegraded":"2024-04-16T19:35:51.211846+0000","last_fullsized":"2024-04-16T19:35:51.211846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:36.187344+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:34:36.187344+0000","last_peered":"2024-04-16T19:34:36.187344+0000","last_clean":"2024-04-16T19:34:36.187344+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:34:36.187344+0000","last_undegraded":"2024-04-16T19:34:36.187344+0000","last_fullsized":"2024-04-16T19:34:36.187344+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181098+0000","last_change":"2024-04-16T17:48:27.293279+0000","last_active":"2024-04-16T19:35:21.181098+0000","last_peered":"2024-04-16T19:35:21.181098+0000","last_clean":"2024-04-16T19:35:21.181098+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:35:21.181098+0000","last_undegraded":"2024-04-16T19:35:21.181098+0000","last_fullsized":"2024-04-16T19:35:21.181098+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"48'8","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.186048+0000","last_change":"2024-04-16T17:48:01.237052+0000","last_active":"2024-04-16T19:34:21.186048+0000","last_peered":"2024-04-16T19:34:21.186048+0000","last_clean":"2024-04-16T19:34:21.186048+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:34:21.186048+0000","last_undegraded":"2024-04-16T19:34:21.186048+0000","last_fullsized":"2024-04-16T19:34:21.186048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'556","reported_seq":1967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.516649+0000","last_change":"2024-04-16T19:36:35.516719+0000","last_active":"2024-04-16T19:36:35.516649+0000","last_peered":"2024-04-16T19:36:35.516649+0000","last_clean":"2024-04-16T19:36:35.516649+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:36:35.516649+0000","last_undegraded":"2024-04-16T19:36:35.516649+0000","last_fullsized":"2024-04-16T19:36:35.516649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.516509+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:41.203861+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'563","reported_seq":804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.482688+0000","last_change":"2024-04-16T19:36:29.482688+0000","last_active":"2024-04-16T19:36:29.482688+0000","last_peered":"2024-04-16T19:36:29.482688+0000","last_clean":"2024-04-16T19:36:29.482688+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:36:29.482688+0000","last_undegraded":"2024-04-16T19:36:29.482688+0000","last_fullsized":"2024-04-16T19:36:29.482688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.482404+0000","objects_scrubbed":19,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.518930+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71952,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'8","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.461125+0000","last_change":"2024-04-16T19:36:21.461301+0000","last_active":"2024-04-16T19:36:21.461125+0000","last_peered":"2024-04-16T19:36:21.461125+0000","last_clean":"2024-04-16T19:36:21.461125+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:36:21.461125+0000","last_undegraded":"2024-04-16T19:36:21.461125+0000","last_fullsized":"2024-04-16T19:36:21.461125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.460913+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:00.948470+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.196271+0000","last_change":"2024-04-16T17:47:39.339617+0000","last_active":"2024-04-16T19:35:11.196271+0000","last_peered":"2024-04-16T19:35:11.196271+0000","last_clean":"2024-04-16T19:35:11.196271+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:35:11.196271+0000","last_undegraded":"2024-04-16T19:35:11.196271+0000","last_fullsized":"2024-04-16T19:35:11.196271+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.192486+0000","last_change":"2024-04-16T17:47:23.263370+0000","last_active":"2024-04-16T19:34:46.192486+0000","last_peered":"2024-04-16T19:34:46.192486+0000","last_clean":"2024-04-16T19:34:46.192486+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:34:46.192486+0000","last_undegraded":"2024-04-16T19:34:46.192486+0000","last_fullsized":"2024-04-16T19:34:46.192486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211668+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:36:06.211668+0000","last_peered":"2024-04-16T19:36:06.211668+0000","last_clean":"2024-04-16T19:36:06.211668+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:36:06.211668+0000","last_undegraded":"2024-04-16T19:36:06.211668+0000","last_fullsized":"2024-04-16T19:36:06.211668+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.197964+0000","last_change":"2024-04-16T17:47:33.307304+0000","last_active":"2024-04-16T19:34:56.197964+0000","last_peered":"2024-04-16T19:34:56.197964+0000","last_clean":"2024-04-16T19:34:56.197964+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:34:56.197964+0000","last_undegraded":"2024-04-16T19:34:56.197964+0000","last_fullsized":"2024-04-16T19:34:56.197964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'80","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192429+0000","last_change":"2024-04-16T17:47:35.311451+0000","last_active":"2024-04-16T19:34:56.192429+0000","last_peered":"2024-04-16T19:34:56.192429+0000","last_clean":"2024-04-16T19:34:56.192429+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:34:56.192429+0000","last_undegraded":"2024-04-16T19:34:56.192429+0000","last_fullsized":"2024-04-16T19:34:56.192429+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.195110+0000","last_change":"2024-04-16T17:47:37.326333+0000","last_active":"2024-04-16T19:35:01.195110+0000","last_peered":"2024-04-16T19:35:01.195110+0000","last_clean":"2024-04-16T19:35:01.195110+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:35:01.195110+0000","last_undegraded":"2024-04-16T19:35:01.195110+0000","last_fullsized":"2024-04-16T19:35:01.195110+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204458+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:35:21.204458+0000","last_peered":"2024-04-16T19:35:21.204458+0000","last_clean":"2024-04-16T19:35:21.204458+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:35:21.204458+0000","last_undegraded":"2024-04-16T19:35:21.204458+0000","last_fullsized":"2024-04-16T19:35:21.204458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'89","reported_seq":695,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.198124+0000","last_change":"2024-04-16T17:48:34.328117+0000","last_active":"2024-04-16T19:34:56.198124+0000","last_peered":"2024-04-16T19:34:56.198124+0000","last_clean":"2024-04-16T19:34:56.198124+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:34:56.198124+0000","last_undegraded":"2024-04-16T19:34:56.198124+0000","last_fullsized":"2024-04-16T19:34:56.198124+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192721+0000","last_change":"2024-04-16T17:48:27.295608+0000","last_active":"2024-04-16T19:34:51.192721+0000","last_peered":"2024-04-16T19:34:51.192721+0000","last_clean":"2024-04-16T19:34:51.192721+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:34:51.192721+0000","last_undegraded":"2024-04-16T19:34:51.192721+0000","last_fullsized":"2024-04-16T19:34:51.192721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'12","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.181227+0000","last_change":"2024-04-16T17:48:01.246592+0000","last_active":"2024-04-16T19:34:21.181227+0000","last_peered":"2024-04-16T19:34:21.181227+0000","last_clean":"2024-04-16T19:34:21.181227+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:34:21.181227+0000","last_undegraded":"2024-04-16T19:34:21.181227+0000","last_fullsized":"2024-04-16T19:34:21.181227+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.470863+0000","last_change":"2024-04-16T19:36:33.470929+0000","last_active":"2024-04-16T19:36:33.470863+0000","last_peered":"2024-04-16T19:36:33.470863+0000","last_clean":"2024-04-16T19:36:33.470863+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:36:33.470863+0000","last_undegraded":"2024-04-16T19:36:33.470863+0000","last_fullsized":"2024-04-16T19:36:33.470863+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.470725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:02.342594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'476","reported_seq":4438,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.660175+0000","last_change":"2024-04-16T19:36:19.660343+0000","last_active":"2024-04-16T19:36:19.660175+0000","last_peered":"2024-04-16T19:36:19.660175+0000","last_clean":"2024-04-16T19:36:19.660175+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:36:19.660175+0000","last_undegraded":"2024-04-16T19:36:19.660175+0000","last_fullsized":"2024-04-16T19:36:19.660175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.659960+0000","objects_scrubbed":19,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:52.175458+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'6","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.197536+0000","last_change":"2024-04-16T17:47:55.174874+0000","last_active":"2024-04-16T19:35:16.197536+0000","last_peered":"2024-04-16T19:35:16.197536+0000","last_clean":"2024-04-16T19:35:16.197536+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:35:16.197536+0000","last_undegraded":"2024-04-16T19:35:16.197536+0000","last_fullsized":"2024-04-16T19:35:16.197536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210764+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:35:46.210764+0000","last_peered":"2024-04-16T19:35:46.210764+0000","last_clean":"2024-04-16T19:35:46.210764+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:35:46.210764+0000","last_undegraded":"2024-04-16T19:35:46.210764+0000","last_fullsized":"2024-04-16T19:35:46.210764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.191930+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:36:06.191930+0000","last_peered":"2024-04-16T19:36:06.191930+0000","last_clean":"2024-04-16T19:36:06.191930+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:36:06.191930+0000","last_undegraded":"2024-04-16T19:36:06.191930+0000","last_fullsized":"2024-04-16T19:36:06.191930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212786+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:36:11.212786+0000","last_peered":"2024-04-16T19:36:11.212786+0000","last_clean":"2024-04-16T19:36:11.212786+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:36:11.212786+0000","last_undegraded":"2024-04-16T19:36:11.212786+0000","last_fullsized":"2024-04-16T19:36:11.212786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'88","reported_seq":250,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197698+0000","last_change":"2024-04-16T17:47:35.319036+0000","last_active":"2024-04-16T19:35:06.197698+0000","last_peered":"2024-04-16T19:35:06.197698+0000","last_clean":"2024-04-16T19:35:06.197698+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:35:06.197698+0000","last_undegraded":"2024-04-16T19:35:06.197698+0000","last_fullsized":"2024-04-16T19:35:06.197698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.201845+0000","last_change":"2024-04-16T17:47:37.330847+0000","last_active":"2024-04-16T19:35:11.201845+0000","last_peered":"2024-04-16T19:35:11.201845+0000","last_clean":"2024-04-16T19:35:11.201845+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:35:11.201845+0000","last_undegraded":"2024-04-16T19:35:11.201845+0000","last_fullsized":"2024-04-16T19:35:11.201845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.511170+0000","last_change":"2024-04-16T19:36:25.511222+0000","last_active":"2024-04-16T19:36:25.511170+0000","last_peered":"2024-04-16T19:36:25.511170+0000","last_clean":"2024-04-16T19:36:25.511170+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:36:25.511170+0000","last_undegraded":"2024-04-16T19:36:25.511170+0000","last_fullsized":"2024-04-16T19:36:25.511170+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.511031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:41:57.452906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.200556+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:35:26.200556+0000","last_peered":"2024-04-16T19:35:26.200556+0000","last_clean":"2024-04-16T19:35:26.200556+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:35:26.200556+0000","last_undegraded":"2024-04-16T19:35:26.200556+0000","last_fullsized":"2024-04-16T19:35:26.200556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'71","reported_seq":968,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.440000+0000","last_change":"2024-04-16T17:48:34.313054+0000","last_active":"2024-04-16T19:36:18.440000+0000","last_peered":"2024-04-16T19:36:18.440000+0000","last_clean":"2024-04-16T19:36:18.440000+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:36:18.440000+0000","last_undegraded":"2024-04-16T19:36:18.440000+0000","last_fullsized":"2024-04-16T19:36:18.440000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191392+0000","last_change":"2024-04-16T17:48:27.309014+0000","last_active":"2024-04-16T19:34:51.191392+0000","last_peered":"2024-04-16T19:34:51.191392+0000","last_clean":"2024-04-16T19:34:51.191392+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:34:51.191392+0000","last_undegraded":"2024-04-16T19:34:51.191392+0000","last_fullsized":"2024-04-16T19:34:51.191392+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'4","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.183760+0000","last_change":"2024-04-16T17:48:01.251229+0000","last_active":"2024-04-16T19:34:21.183760+0000","last_peered":"2024-04-16T19:34:21.183760+0000","last_clean":"2024-04-16T19:34:21.183760+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:34:21.183760+0000","last_undegraded":"2024-04-16T19:34:21.183760+0000","last_fullsized":"2024-04-16T19:34:21.183760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.550467+0000","last_change":"2024-04-16T19:36:28.550520+0000","last_active":"2024-04-16T19:36:28.550467+0000","last_peered":"2024-04-16T19:36:28.550467+0000","last_clean":"2024-04-16T19:36:28.550467+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:36:28.550467+0000","last_undegraded":"2024-04-16T19:36:28.550467+0000","last_fullsized":"2024-04-16T19:36:28.550467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.550328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:52.280854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11393","reported_seq":61823,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.677407+0000","last_change":"2024-04-16T19:36:19.677565+0000","last_active":"2024-04-16T19:36:19.677407+0000","last_peered":"2024-04-16T19:36:19.677407+0000","last_clean":"2024-04-16T19:36:19.677407+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:36:19.677407+0000","last_undegraded":"2024-04-16T19:36:19.677407+0000","last_fullsized":"2024-04-16T19:36:19.677407+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11393","last_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_deep_scrub":"53'11393","last_deep_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.677264+0000","objects_scrubbed":23,"log_size":1693,"log_dups_size":3000,"ondisk_log_size":1693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:14.777503+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209607,"num_omap_keys":977,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'23","reported_seq":358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.461538+0000","last_change":"2024-04-16T19:36:18.461701+0000","last_active":"2024-04-16T19:36:18.461538+0000","last_peered":"2024-04-16T19:36:18.461538+0000","last_clean":"2024-04-16T19:36:18.461538+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:36:18.461538+0000","last_undegraded":"2024-04-16T19:36:18.461538+0000","last_fullsized":"2024-04-16T19:36:18.461538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.461393+0000","objects_scrubbed":11,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:24.888175+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210256+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:36:01.210256+0000","last_peered":"2024-04-16T19:36:01.210256+0000","last_clean":"2024-04-16T19:36:01.210256+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:36:01.210256+0000","last_undegraded":"2024-04-16T19:36:01.210256+0000","last_fullsized":"2024-04-16T19:36:01.210256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048372+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:36:17.048372+0000","last_peered":"2024-04-16T19:36:17.048372+0000","last_clean":"2024-04-16T19:36:17.048372+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:36:17.048372+0000","last_undegraded":"2024-04-16T19:36:17.048372+0000","last_fullsized":"2024-04-16T19:36:17.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213328+0000","last_change":"2024-04-16T17:47:33.296653+0000","last_active":"2024-04-16T19:35:56.213328+0000","last_peered":"2024-04-16T19:35:56.213328+0000","last_clean":"2024-04-16T19:35:56.213328+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:35:56.213328+0000","last_undegraded":"2024-04-16T19:35:56.213328+0000","last_fullsized":"2024-04-16T19:35:56.213328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'152","reported_seq":341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.200620+0000","last_change":"2024-04-16T17:47:35.317123+0000","last_active":"2024-04-16T19:35:06.200620+0000","last_peered":"2024-04-16T19:35:06.200620+0000","last_clean":"2024-04-16T19:35:06.200620+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:35:06.200620+0000","last_undegraded":"2024-04-16T19:35:06.200620+0000","last_fullsized":"2024-04-16T19:35:06.200620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.198864+0000","last_change":"2024-04-16T17:47:37.334019+0000","last_active":"2024-04-16T19:35:11.198864+0000","last_peered":"2024-04-16T19:35:11.198864+0000","last_clean":"2024-04-16T19:35:11.198864+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:35:11.198864+0000","last_undegraded":"2024-04-16T19:35:11.198864+0000","last_fullsized":"2024-04-16T19:35:11.198864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.426536+0000","last_change":"2024-04-16T19:36:35.426597+0000","last_active":"2024-04-16T19:36:35.426536+0000","last_peered":"2024-04-16T19:36:35.426536+0000","last_clean":"2024-04-16T19:36:35.426536+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:36:35.426536+0000","last_undegraded":"2024-04-16T19:36:35.426536+0000","last_fullsized":"2024-04-16T19:36:35.426536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.426385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:09.737500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201791+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:35:26.201791+0000","last_peered":"2024-04-16T19:35:26.201791+0000","last_clean":"2024-04-16T19:35:26.201791+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:35:26.201791+0000","last_undegraded":"2024-04-16T19:35:26.201791+0000","last_fullsized":"2024-04-16T19:35:26.201791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'103","reported_seq":1820,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.195147+0000","last_change":"2024-04-16T17:48:34.337561+0000","last_active":"2024-04-16T19:34:56.195147+0000","last_peered":"2024-04-16T19:34:56.195147+0000","last_clean":"2024-04-16T19:34:56.195147+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:34:56.195147+0000","last_undegraded":"2024-04-16T19:34:56.195147+0000","last_fullsized":"2024-04-16T19:34:56.195147+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587780+0000","last_change":"2024-04-16T17:48:27.293905+0000","last_active":"2024-04-16T19:36:16.587780+0000","last_peered":"2024-04-16T19:36:16.587780+0000","last_clean":"2024-04-16T19:36:16.587780+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:36:16.587780+0000","last_undegraded":"2024-04-16T19:36:16.587780+0000","last_fullsized":"2024-04-16T19:36:16.587780+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'8","reported_seq":92,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.183685+0000","last_change":"2024-04-16T17:48:01.248531+0000","last_active":"2024-04-16T19:34:21.183685+0000","last_peered":"2024-04-16T19:34:21.183685+0000","last_clean":"2024-04-16T19:34:21.183685+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:34:21.183685+0000","last_undegraded":"2024-04-16T19:34:21.183685+0000","last_fullsized":"2024-04-16T19:34:21.183685+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204381+0000","last_change":"2024-04-16T17:47:59.187755+0000","last_active":"2024-04-16T19:35:21.204381+0000","last_peered":"2024-04-16T19:35:21.204381+0000","last_clean":"2024-04-16T19:35:21.204381+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:35:21.204381+0000","last_undegraded":"2024-04-16T19:35:21.204381+0000","last_fullsized":"2024-04-16T19:35:21.204381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'206","reported_seq":440,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.574363+0000","last_change":"2024-04-16T19:36:35.574523+0000","last_active":"2024-04-16T19:36:35.574363+0000","last_peered":"2024-04-16T19:36:35.574363+0000","last_clean":"2024-04-16T19:36:35.574363+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:36:35.574363+0000","last_undegraded":"2024-04-16T19:36:35.574363+0000","last_fullsized":"2024-04-16T19:36:35.574363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'206","last_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_deep_scrub":"53'206","last_deep_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.574163+0000","objects_scrubbed":10,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:15.061721+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'20","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.646780+0000","last_change":"2024-04-16T19:36:30.646869+0000","last_active":"2024-04-16T19:36:30.646780+0000","last_peered":"2024-04-16T19:36:30.646780+0000","last_clean":"2024-04-16T19:36:30.646780+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:36:30.646780+0000","last_undegraded":"2024-04-16T19:36:30.646780+0000","last_fullsized":"2024-04-16T19:36:30.646780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.646698+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:37.151471+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194008+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:34:56.194008+0000","last_peered":"2024-04-16T19:34:56.194008+0000","last_clean":"2024-04-16T19:34:56.194008+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:34:56.194008+0000","last_undegraded":"2024-04-16T19:34:56.194008+0000","last_fullsized":"2024-04-16T19:34:56.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188408+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:34:51.188408+0000","last_peered":"2024-04-16T19:34:51.188408+0000","last_clean":"2024-04-16T19:34:51.188408+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:34:51.188408+0000","last_undegraded":"2024-04-16T19:34:51.188408+0000","last_fullsized":"2024-04-16T19:34:51.188408+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"48'5","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192321+0000","last_change":"2024-04-16T17:47:33.306460+0000","last_active":"2024-04-16T19:35:06.192321+0000","last_peered":"2024-04-16T19:35:06.192321+0000","last_clean":"2024-04-16T19:35:06.192321+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:35:06.192321+0000","last_undegraded":"2024-04-16T19:35:06.192321+0000","last_fullsized":"2024-04-16T19:35:06.192321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'104","reported_seq":278,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189800+0000","last_change":"2024-04-16T17:47:35.314274+0000","last_active":"2024-04-16T19:35:56.189800+0000","last_peered":"2024-04-16T19:35:56.189800+0000","last_clean":"2024-04-16T19:35:56.189800+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:35:56.189800+0000","last_undegraded":"2024-04-16T19:35:56.189800+0000","last_fullsized":"2024-04-16T19:35:56.189800+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.214612+0000","last_change":"2024-04-16T17:47:37.331871+0000","last_active":"2024-04-16T19:36:11.214612+0000","last_peered":"2024-04-16T19:36:11.214612+0000","last_clean":"2024-04-16T19:36:11.214612+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:36:11.214612+0000","last_undegraded":"2024-04-16T19:36:11.214612+0000","last_fullsized":"2024-04-16T19:36:11.214612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.199321+0000","last_change":"2024-04-16T17:47:39.323414+0000","last_active":"2024-04-16T19:35:01.199321+0000","last_peered":"2024-04-16T19:35:01.199321+0000","last_clean":"2024-04-16T19:35:01.199321+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:35:01.199321+0000","last_undegraded":"2024-04-16T19:35:01.199321+0000","last_fullsized":"2024-04-16T19:35:01.199321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":46,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.214641+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:36:11.214641+0000","last_peered":"2024-04-16T19:36:11.214641+0000","last_clean":"2024-04-16T19:36:11.214641+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:36:11.214641+0000","last_undegraded":"2024-04-16T19:36:11.214641+0000","last_fullsized":"2024-04-16T19:36:11.214641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'68","reported_seq":883,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.195089+0000","last_change":"2024-04-16T17:48:34.343716+0000","last_active":"2024-04-16T19:34:56.195089+0000","last_peered":"2024-04-16T19:34:56.195089+0000","last_clean":"2024-04-16T19:34:56.195089+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:34:56.195089+0000","last_undegraded":"2024-04-16T19:34:56.195089+0000","last_fullsized":"2024-04-16T19:34:56.195089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188569+0000","last_change":"2024-04-16T17:48:27.305044+0000","last_active":"2024-04-16T19:34:51.188569+0000","last_peered":"2024-04-16T19:34:51.188569+0000","last_clean":"2024-04-16T19:34:51.188569+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:34:51.188569+0000","last_undegraded":"2024-04-16T19:34:51.188569+0000","last_fullsized":"2024-04-16T19:34:51.188569+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"48'1","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.205918+0000","last_change":"2024-04-16T17:48:01.254059+0000","last_active":"2024-04-16T19:35:46.205918+0000","last_peered":"2024-04-16T19:35:46.205918+0000","last_clean":"2024-04-16T19:35:46.205918+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:35:46.205918+0000","last_undegraded":"2024-04-16T19:35:46.205918+0000","last_fullsized":"2024-04-16T19:35:46.205918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199184+0000","last_change":"2024-04-16T17:47:59.195638+0000","last_active":"2024-04-16T19:35:21.199184+0000","last_peered":"2024-04-16T19:35:21.199184+0000","last_clean":"2024-04-16T19:35:21.199184+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:35:21.199184+0000","last_undegraded":"2024-04-16T19:35:21.199184+0000","last_fullsized":"2024-04-16T19:35:21.199184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'509","reported_seq":3967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635824+0000","last_change":"2024-04-16T19:36:20.636035+0000","last_active":"2024-04-16T19:36:20.635824+0000","last_peered":"2024-04-16T19:36:20.635824+0000","last_clean":"2024-04-16T19:36:20.635824+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:36:20.635824+0000","last_undegraded":"2024-04-16T19:36:20.635824+0000","last_fullsized":"2024-04-16T19:36:20.635824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635659+0000","objects_scrubbed":21,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:04.155514+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71921,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'34","reported_seq":243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.437126+0000","last_change":"2024-04-16T19:36:23.437306+0000","last_active":"2024-04-16T19:36:23.437126+0000","last_peered":"2024-04-16T19:36:23.437126+0000","last_clean":"2024-04-16T19:36:23.437126+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:36:23.437126+0000","last_undegraded":"2024-04-16T19:36:23.437126+0000","last_fullsized":"2024-04-16T19:36:23.437126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.436922+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:38.568868+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175524+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:34:56.175524+0000","last_peered":"2024-04-16T19:34:56.175524+0000","last_clean":"2024-04-16T19:34:56.175524+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:34:56.175524+0000","last_undegraded":"2024-04-16T19:34:56.175524+0000","last_fullsized":"2024-04-16T19:34:56.175524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192550+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:34:51.192550+0000","last_peered":"2024-04-16T19:34:51.192550+0000","last_clean":"2024-04-16T19:34:51.192550+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:34:51.192550+0000","last_undegraded":"2024-04-16T19:34:51.192550+0000","last_fullsized":"2024-04-16T19:34:51.192550+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"33'3","reported_seq":803,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197667+0000","last_change":"2024-04-16T17:47:33.297794+0000","last_active":"2024-04-16T19:35:06.197667+0000","last_peered":"2024-04-16T19:35:06.197667+0000","last_clean":"2024-04-16T19:35:06.197667+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:35:06.197667+0000","last_undegraded":"2024-04-16T19:35:06.197667+0000","last_fullsized":"2024-04-16T19:35:06.197667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'120","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.216918+0000","last_change":"2024-04-16T17:47:35.319657+0000","last_active":"2024-04-16T19:36:11.216918+0000","last_peered":"2024-04-16T19:36:11.216918+0000","last_clean":"2024-04-16T19:36:11.216918+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:36:11.216918+0000","last_undegraded":"2024-04-16T19:36:11.216918+0000","last_fullsized":"2024-04-16T19:36:11.216918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'12","reported_seq":1341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210179+0000","last_change":"2024-04-16T17:47:37.331215+0000","last_active":"2024-04-16T19:36:01.210179+0000","last_peered":"2024-04-16T19:36:01.210179+0000","last_clean":"2024-04-16T19:36:01.210179+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:36:01.210179+0000","last_undegraded":"2024-04-16T19:36:01.210179+0000","last_fullsized":"2024-04-16T19:36:01.210179+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.199192+0000","last_change":"2024-04-16T17:47:39.332386+0000","last_active":"2024-04-16T19:35:01.199192+0000","last_peered":"2024-04-16T19:35:01.199192+0000","last_clean":"2024-04-16T19:35:01.199192+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:35:01.199192+0000","last_undegraded":"2024-04-16T19:35:01.199192+0000","last_fullsized":"2024-04-16T19:35:01.199192+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188309+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:34:51.188309+0000","last_peered":"2024-04-16T19:34:51.188309+0000","last_clean":"2024-04-16T19:34:51.188309+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:34:51.188309+0000","last_undegraded":"2024-04-16T19:34:51.188309+0000","last_fullsized":"2024-04-16T19:34:51.188309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'86","reported_seq":1558,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204604+0000","last_change":"2024-04-16T17:48:34.318628+0000","last_active":"2024-04-16T19:35:56.204604+0000","last_peered":"2024-04-16T19:35:56.204604+0000","last_clean":"2024-04-16T19:35:56.204604+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:35:56.204604+0000","last_undegraded":"2024-04-16T19:35:56.204604+0000","last_fullsized":"2024-04-16T19:35:56.204604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207543+0000","last_change":"2024-04-16T17:48:27.299691+0000","last_active":"2024-04-16T19:35:51.207543+0000","last_peered":"2024-04-16T19:35:51.207543+0000","last_clean":"2024-04-16T19:35:51.207543+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:35:51.207543+0000","last_undegraded":"2024-04-16T19:35:51.207543+0000","last_fullsized":"2024-04-16T19:35:51.207543+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'6","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201218+0000","last_change":"2024-04-16T17:48:01.246789+0000","last_active":"2024-04-16T19:35:21.201218+0000","last_peered":"2024-04-16T19:35:21.201218+0000","last_clean":"2024-04-16T19:35:21.201218+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:35:21.201218+0000","last_undegraded":"2024-04-16T19:35:21.201218+0000","last_fullsized":"2024-04-16T19:35:21.201218+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'555","reported_seq":1954,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213122+0000","last_change":"2024-04-16T17:47:59.182365+0000","last_active":"2024-04-16T19:35:56.213122+0000","last_peered":"2024-04-16T19:35:56.213122+0000","last_clean":"2024-04-16T19:35:56.213122+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:35:56.213122+0000","last_undegraded":"2024-04-16T19:35:56.213122+0000","last_fullsized":"2024-04-16T19:35:56.213122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'508","reported_seq":735,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.543404+0000","last_change":"2024-04-16T19:36:25.543534+0000","last_active":"2024-04-16T19:36:25.543404+0000","last_peered":"2024-04-16T19:36:25.543404+0000","last_clean":"2024-04-16T19:36:25.543404+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:36:25.543404+0000","last_undegraded":"2024-04-16T19:36:25.543404+0000","last_fullsized":"2024-04-16T19:36:25.543404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.543268+0000","objects_scrubbed":16,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:59.596711+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71934,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'8","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.642207+0000","last_change":"2024-04-16T19:36:27.642341+0000","last_active":"2024-04-16T19:36:27.642207+0000","last_peered":"2024-04-16T19:36:27.642207+0000","last_clean":"2024-04-16T19:36:27.642207+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:36:27.642207+0000","last_undegraded":"2024-04-16T19:36:27.642207+0000","last_fullsized":"2024-04-16T19:36:27.642207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.642045+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:07.753490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189839+0000","last_change":"2024-04-16T17:47:23.266570+0000","last_active":"2024-04-16T19:34:56.189839+0000","last_peered":"2024-04-16T19:34:56.189839+0000","last_clean":"2024-04-16T19:34:56.189839+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:34:56.189839+0000","last_undegraded":"2024-04-16T19:34:56.189839+0000","last_fullsized":"2024-04-16T19:34:56.189839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191235+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:34:51.191235+0000","last_peered":"2024-04-16T19:34:51.191235+0000","last_clean":"2024-04-16T19:34:51.191235+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:34:51.191235+0000","last_undegraded":"2024-04-16T19:34:51.191235+0000","last_fullsized":"2024-04-16T19:34:51.191235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192192+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:35:06.192192+0000","last_peered":"2024-04-16T19:35:06.192192+0000","last_clean":"2024-04-16T19:35:06.192192+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:35:06.192192+0000","last_undegraded":"2024-04-16T19:35:06.192192+0000","last_fullsized":"2024-04-16T19:35:06.192192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'152","reported_seq":342,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.183496+0000","last_change":"2024-04-16T17:47:35.310214+0000","last_active":"2024-04-16T19:34:21.183496+0000","last_peered":"2024-04-16T19:34:21.183496+0000","last_clean":"2024-04-16T19:34:21.183496+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:34:21.183496+0000","last_undegraded":"2024-04-16T19:34:21.183496+0000","last_fullsized":"2024-04-16T19:34:21.183496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586749+0000","last_change":"2024-04-16T17:47:37.328751+0000","last_active":"2024-04-16T19:36:16.586749+0000","last_peered":"2024-04-16T19:36:16.586749+0000","last_clean":"2024-04-16T19:36:16.586749+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:36:16.586749+0000","last_undegraded":"2024-04-16T19:36:16.586749+0000","last_fullsized":"2024-04-16T19:36:16.586749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.979160+0000","last_change":"2024-04-16T17:47:39.336993+0000","last_active":"2024-04-16T19:36:17.979160+0000","last_peered":"2024-04-16T19:36:17.979160+0000","last_clean":"2024-04-16T19:36:17.979160+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:36:17.979160+0000","last_undegraded":"2024-04-16T19:36:17.979160+0000","last_fullsized":"2024-04-16T19:36:17.979160+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.174203+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:34:51.174203+0000","last_peered":"2024-04-16T19:34:51.174203+0000","last_clean":"2024-04-16T19:34:51.174203+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:34:51.174203+0000","last_undegraded":"2024-04-16T19:34:51.174203+0000","last_fullsized":"2024-04-16T19:34:51.174203+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'99","reported_seq":1432,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210505+0000","last_change":"2024-04-16T17:48:34.341651+0000","last_active":"2024-04-16T19:35:56.210505+0000","last_peered":"2024-04-16T19:35:56.210505+0000","last_clean":"2024-04-16T19:35:56.210505+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:35:56.210505+0000","last_undegraded":"2024-04-16T19:35:56.210505+0000","last_fullsized":"2024-04-16T19:35:56.210505+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207296+0000","last_change":"2024-04-16T17:48:27.306641+0000","last_active":"2024-04-16T19:35:51.207296+0000","last_peered":"2024-04-16T19:35:51.207296+0000","last_clean":"2024-04-16T19:35:51.207296+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:35:51.207296+0000","last_undegraded":"2024-04-16T19:35:51.207296+0000","last_fullsized":"2024-04-16T19:35:51.207296+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'7","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200105+0000","last_change":"2024-04-16T17:48:01.263221+0000","last_active":"2024-04-16T19:35:21.200105+0000","last_peered":"2024-04-16T19:35:21.200105+0000","last_clean":"2024-04-16T19:35:21.200105+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:35:21.200105+0000","last_undegraded":"2024-04-16T19:35:21.200105+0000","last_fullsized":"2024-04-16T19:35:21.200105+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'555","reported_seq":1901,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.167203+0000","last_change":"2024-04-16T17:47:59.183413+0000","last_active":"2024-04-16T19:34:21.167203+0000","last_peered":"2024-04-16T19:34:21.167203+0000","last_clean":"2024-04-16T19:34:21.167203+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:34:21.167203+0000","last_undegraded":"2024-04-16T19:34:21.167203+0000","last_fullsized":"2024-04-16T19:34:21.167203+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'250","reported_seq":489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.484262+0000","last_change":"2024-04-16T19:36:27.484420+0000","last_active":"2024-04-16T19:36:27.484262+0000","last_peered":"2024-04-16T19:36:27.484262+0000","last_clean":"2024-04-16T19:36:27.484262+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:36:27.484262+0000","last_undegraded":"2024-04-16T19:36:27.484262+0000","last_fullsized":"2024-04-16T19:36:27.484262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.484127+0000","objects_scrubbed":20,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:04.899387+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.205027+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:35:56.205027+0000","last_peered":"2024-04-16T19:35:56.205027+0000","last_clean":"2024-04-16T19:35:56.205027+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:35:56.205027+0000","last_undegraded":"2024-04-16T19:35:56.205027+0000","last_fullsized":"2024-04-16T19:35:56.205027+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189793+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:34:56.189793+0000","last_peered":"2024-04-16T19:34:56.189793+0000","last_clean":"2024-04-16T19:34:56.189793+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:34:56.189793+0000","last_undegraded":"2024-04-16T19:34:56.189793+0000","last_fullsized":"2024-04-16T19:34:56.189793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191305+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:34:51.191305+0000","last_peered":"2024-04-16T19:34:51.191305+0000","last_clean":"2024-04-16T19:34:51.191305+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:34:51.191305+0000","last_undegraded":"2024-04-16T19:34:51.191305+0000","last_fullsized":"2024-04-16T19:34:51.191305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.194844+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:35:06.194844+0000","last_peered":"2024-04-16T19:35:06.194844+0000","last_clean":"2024-04-16T19:35:06.194844+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:35:06.194844+0000","last_undegraded":"2024-04-16T19:35:06.194844+0000","last_fullsized":"2024-04-16T19:35:06.194844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586539+0000","last_change":"2024-04-16T17:47:35.303292+0000","last_active":"2024-04-16T19:36:16.586539+0000","last_peered":"2024-04-16T19:36:16.586539+0000","last_clean":"2024-04-16T19:36:16.586539+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:36:16.586539+0000","last_undegraded":"2024-04-16T19:36:16.586539+0000","last_fullsized":"2024-04-16T19:36:16.586539+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:36.192983+0000","last_change":"2024-04-16T17:47:37.331029+0000","last_active":"2024-04-16T19:34:36.192983+0000","last_peered":"2024-04-16T19:34:36.192983+0000","last_clean":"2024-04-16T19:34:36.192983+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:34:36.192983+0000","last_undegraded":"2024-04-16T19:34:36.192983+0000","last_fullsized":"2024-04-16T19:34:36.192983+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.477611+0000","last_change":"2024-04-16T19:36:26.477669+0000","last_active":"2024-04-16T19:36:26.477611+0000","last_peered":"2024-04-16T19:36:26.477611+0000","last_clean":"2024-04-16T19:36:26.477611+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:36:26.477611+0000","last_undegraded":"2024-04-16T19:36:26.477611+0000","last_fullsized":"2024-04-16T19:36:26.477611+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.477464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:27.657085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192426+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:34:51.192426+0000","last_peered":"2024-04-16T19:34:51.192426+0000","last_clean":"2024-04-16T19:34:51.192426+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:34:51.192426+0000","last_undegraded":"2024-04-16T19:34:51.192426+0000","last_fullsized":"2024-04-16T19:34:51.192426+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'96","reported_seq":920,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189586+0000","last_change":"2024-04-16T17:48:34.324041+0000","last_active":"2024-04-16T19:35:56.189586+0000","last_peered":"2024-04-16T19:35:56.189586+0000","last_clean":"2024-04-16T19:35:56.189586+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:35:56.189586+0000","last_undegraded":"2024-04-16T19:35:56.189586+0000","last_fullsized":"2024-04-16T19:35:56.189586+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209067+0000","last_change":"2024-04-16T17:48:27.307648+0000","last_active":"2024-04-16T19:35:51.209067+0000","last_peered":"2024-04-16T19:35:51.209067+0000","last_clean":"2024-04-16T19:35:51.209067+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:35:51.209067+0000","last_undegraded":"2024-04-16T19:35:51.209067+0000","last_fullsized":"2024-04-16T19:35:51.209067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'7","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.198997+0000","last_change":"2024-04-16T17:48:01.248401+0000","last_active":"2024-04-16T19:35:21.198997+0000","last_peered":"2024-04-16T19:35:21.198997+0000","last_clean":"2024-04-16T19:35:21.198997+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:35:21.198997+0000","last_undegraded":"2024-04-16T19:35:21.198997+0000","last_fullsized":"2024-04-16T19:35:21.198997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.183798+0000","last_change":"2024-04-16T17:47:59.199961+0000","last_active":"2024-04-16T19:34:21.183798+0000","last_peered":"2024-04-16T19:34:21.183798+0000","last_clean":"2024-04-16T19:34:21.183798+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:34:21.183798+0000","last_undegraded":"2024-04-16T19:34:21.183798+0000","last_fullsized":"2024-04-16T19:34:21.183798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'141","reported_seq":317,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.454137+0000","last_change":"2024-04-16T19:36:28.454241+0000","last_active":"2024-04-16T19:36:28.454137+0000","last_peered":"2024-04-16T19:36:28.454137+0000","last_clean":"2024-04-16T19:36:28.454137+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:36:28.454137+0000","last_undegraded":"2024-04-16T19:36:28.454137+0000","last_fullsized":"2024-04-16T19:36:28.454137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'141","last_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_deep_scrub":"53'141","last_deep_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.453990+0000","objects_scrubbed":12,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:15.632623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.478104+0000","last_change":"2024-04-16T19:36:22.478173+0000","last_active":"2024-04-16T19:36:22.478104+0000","last_peered":"2024-04-16T19:36:22.478104+0000","last_clean":"2024-04-16T19:36:22.478104+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:36:22.478104+0000","last_undegraded":"2024-04-16T19:36:22.478104+0000","last_fullsized":"2024-04-16T19:36:22.478104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.477939+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:46.331273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.185189+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:34:21.185189+0000","last_peered":"2024-04-16T19:34:21.185189+0000","last_clean":"2024-04-16T19:34:21.185189+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:34:21.185189+0000","last_undegraded":"2024-04-16T19:34:21.185189+0000","last_fullsized":"2024-04-16T19:34:21.185189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.188432+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:35:51.188432+0000","last_peered":"2024-04-16T19:35:51.188432+0000","last_clean":"2024-04-16T19:35:51.188432+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:35:51.188432+0000","last_undegraded":"2024-04-16T19:35:51.188432+0000","last_fullsized":"2024-04-16T19:35:51.188432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438194+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:36:18.438194+0000","last_peered":"2024-04-16T19:36:18.438194+0000","last_clean":"2024-04-16T19:36:18.438194+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:36:18.438194+0000","last_undegraded":"2024-04-16T19:36:18.438194+0000","last_fullsized":"2024-04-16T19:36:18.438194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'32","reported_seq":161,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.196282+0000","last_change":"2024-04-16T17:47:35.317051+0000","last_active":"2024-04-16T19:35:06.196282+0000","last_peered":"2024-04-16T19:35:06.196282+0000","last_clean":"2024-04-16T19:35:06.196282+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:35:06.196282+0000","last_undegraded":"2024-04-16T19:35:06.196282+0000","last_fullsized":"2024-04-16T19:35:06.196282+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.201733+0000","last_change":"2024-04-16T17:47:37.320351+0000","last_active":"2024-04-16T19:35:11.201733+0000","last_peered":"2024-04-16T19:35:11.201733+0000","last_clean":"2024-04-16T19:35:11.201733+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:35:11.201733+0000","last_undegraded":"2024-04-16T19:35:11.201733+0000","last_fullsized":"2024-04-16T19:35:11.201733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.979039+0000","last_change":"2024-04-16T17:47:39.329617+0000","last_active":"2024-04-16T19:36:17.979039+0000","last_peered":"2024-04-16T19:36:17.979039+0000","last_clean":"2024-04-16T19:36:17.979039+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:36:17.979039+0000","last_undegraded":"2024-04-16T19:36:17.979039+0000","last_fullsized":"2024-04-16T19:36:17.979039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.193772+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:34:51.193772+0000","last_peered":"2024-04-16T19:34:51.193772+0000","last_clean":"2024-04-16T19:34:51.193772+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:34:51.193772+0000","last_undegraded":"2024-04-16T19:34:51.193772+0000","last_fullsized":"2024-04-16T19:34:51.193772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'126","reported_seq":1451,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189517+0000","last_change":"2024-04-16T17:48:34.338811+0000","last_active":"2024-04-16T19:35:56.189517+0000","last_peered":"2024-04-16T19:35:56.189517+0000","last_clean":"2024-04-16T19:35:56.189517+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:35:56.189517+0000","last_undegraded":"2024-04-16T19:35:56.189517+0000","last_fullsized":"2024-04-16T19:35:56.189517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209222+0000","last_change":"2024-04-16T17:48:27.288374+0000","last_active":"2024-04-16T19:35:51.209222+0000","last_peered":"2024-04-16T19:35:51.209222+0000","last_clean":"2024-04-16T19:35:51.209222+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:35:51.209222+0000","last_undegraded":"2024-04-16T19:35:51.209222+0000","last_fullsized":"2024-04-16T19:35:51.209222+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'7","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199867+0000","last_change":"2024-04-16T17:48:01.263690+0000","last_active":"2024-04-16T19:35:21.199867+0000","last_peered":"2024-04-16T19:35:21.199867+0000","last_clean":"2024-04-16T19:35:21.199867+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:35:21.199867+0000","last_undegraded":"2024-04-16T19:35:21.199867+0000","last_fullsized":"2024-04-16T19:35:21.199867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.183845+0000","last_change":"2024-04-16T17:47:59.199357+0000","last_active":"2024-04-16T19:34:21.183845+0000","last_peered":"2024-04-16T19:34:21.183845+0000","last_clean":"2024-04-16T19:34:21.183845+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:34:21.183845+0000","last_undegraded":"2024-04-16T19:34:21.183845+0000","last_fullsized":"2024-04-16T19:34:21.183845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11080","reported_seq":18168,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.523465+0000","last_change":"2024-04-16T19:36:22.523528+0000","last_active":"2024-04-16T19:36:22.523465+0000","last_peered":"2024-04-16T19:36:22.523465+0000","last_clean":"2024-04-16T19:36:22.523465+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:36:22.523465+0000","last_undegraded":"2024-04-16T19:36:22.523465+0000","last_fullsized":"2024-04-16T19:36:22.523465+0000","mapping_epoch":34,"log_start":"53'9300","ondisk_log_start":"53'9300","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11080","last_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_deep_scrub":"53'11080","last_deep_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.523321+0000","objects_scrubbed":25,"log_size":1780,"log_dups_size":3000,"ondisk_log_size":1780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:02.010878+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'10","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.675516+0000","last_change":"2024-04-16T19:36:33.675610+0000","last_active":"2024-04-16T19:36:33.675516+0000","last_peered":"2024-04-16T19:36:33.675516+0000","last_clean":"2024-04-16T19:36:33.675516+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:36:33.675516+0000","last_undegraded":"2024-04-16T19:36:33.675516+0000","last_fullsized":"2024-04-16T19:36:33.675516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.675369+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:17.195224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.216640+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:36:11.216640+0000","last_peered":"2024-04-16T19:36:11.216640+0000","last_clean":"2024-04-16T19:36:11.216640+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:36:11.216640+0000","last_undegraded":"2024-04-16T19:36:11.216640+0000","last_fullsized":"2024-04-16T19:36:11.216640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205439+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:35:41.205439+0000","last_peered":"2024-04-16T19:35:41.205439+0000","last_clean":"2024-04-16T19:35:41.205439+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:35:41.205439+0000","last_undegraded":"2024-04-16T19:35:41.205439+0000","last_fullsized":"2024-04-16T19:35:41.205439+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1344,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:26.187684+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:34:26.187684+0000","last_peered":"2024-04-16T19:34:26.187684+0000","last_clean":"2024-04-16T19:34:26.187684+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:34:26.187684+0000","last_undegraded":"2024-04-16T19:34:26.187684+0000","last_fullsized":"2024-04-16T19:34:26.187684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192236+0000","last_change":"2024-04-16T17:47:35.308166+0000","last_active":"2024-04-16T19:35:06.192236+0000","last_peered":"2024-04-16T19:35:06.192236+0000","last_clean":"2024-04-16T19:35:06.192236+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:35:06.192236+0000","last_undegraded":"2024-04-16T19:35:06.192236+0000","last_fullsized":"2024-04-16T19:35:06.192236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.197590+0000","last_change":"2024-04-16T17:47:37.321988+0000","last_active":"2024-04-16T19:35:11.197590+0000","last_peered":"2024-04-16T19:35:11.197590+0000","last_clean":"2024-04-16T19:35:11.197590+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:35:11.197590+0000","last_undegraded":"2024-04-16T19:35:11.197590+0000","last_fullsized":"2024-04-16T19:35:11.197590+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.199759+0000","last_change":"2024-04-16T17:47:39.326823+0000","last_active":"2024-04-16T19:35:36.199759+0000","last_peered":"2024-04-16T19:35:36.199759+0000","last_clean":"2024-04-16T19:35:36.199759+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:35:36.199759+0000","last_undegraded":"2024-04-16T19:35:36.199759+0000","last_fullsized":"2024-04-16T19:35:36.199759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:38.301675+0000","last_clean_scrub_stamp":"2024-04-16T17:47:38.301675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'22","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189565+0000","last_change":"2024-04-16T17:47:55.158595+0000","last_active":"2024-04-16T19:35:56.189565+0000","last_peered":"2024-04-16T19:35:56.189565+0000","last_clean":"2024-04-16T19:35:56.189565+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:35:56.189565+0000","last_undegraded":"2024-04-16T19:35:56.189565+0000","last_fullsized":"2024-04-16T19:35:56.189565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189771+0000","last_change":"2024-04-16T17:47:32.278320+0000","last_active":"2024-04-16T19:35:56.189771+0000","last_peered":"2024-04-16T19:35:56.189771+0000","last_clean":"2024-04-16T19:35:56.189771+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:35:56.189771+0000","last_undegraded":"2024-04-16T19:35:56.189771+0000","last_fullsized":"2024-04-16T19:35:56.189771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200847+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:35:41.200847+0000","last_peered":"2024-04-16T19:35:41.200847+0000","last_clean":"2024-04-16T19:35:41.200847+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:35:41.200847+0000","last_undegraded":"2024-04-16T19:35:41.200847+0000","last_fullsized":"2024-04-16T19:35:41.200847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'6","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.472692+0000","last_change":"2024-04-16T19:36:34.472767+0000","last_active":"2024-04-16T19:36:34.472692+0000","last_peered":"2024-04-16T19:36:34.472692+0000","last_clean":"2024-04-16T19:36:34.472692+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:36:34.472692+0000","last_undegraded":"2024-04-16T19:36:34.472692+0000","last_fullsized":"2024-04-16T19:36:34.472692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.472561+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:21.836965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.208030+0000","last_change":"2024-04-16T17:47:23.260359+0000","last_active":"2024-04-16T19:35:46.208030+0000","last_peered":"2024-04-16T19:35:46.208030+0000","last_clean":"2024-04-16T19:35:46.208030+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:35:46.208030+0000","last_undegraded":"2024-04-16T19:35:46.208030+0000","last_fullsized":"2024-04-16T19:35:46.208030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.185828+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:35:41.185828+0000","last_peered":"2024-04-16T19:35:41.185828+0000","last_clean":"2024-04-16T19:35:41.185828+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:35:41.185828+0000","last_undegraded":"2024-04-16T19:35:41.185828+0000","last_fullsized":"2024-04-16T19:35:41.185828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'10","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.445550+0000","last_change":"2024-04-16T19:36:30.445631+0000","last_active":"2024-04-16T19:36:30.445550+0000","last_peered":"2024-04-16T19:36:30.445550+0000","last_clean":"2024-04-16T19:36:30.445550+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:36:30.445550+0000","last_undegraded":"2024-04-16T19:36:30.445550+0000","last_fullsized":"2024-04-16T19:36:30.445550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.445415+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:00.551163+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206084+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:35:46.206084+0000","last_peered":"2024-04-16T19:35:46.206084+0000","last_clean":"2024-04-16T19:35:46.206084+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:35:46.206084+0000","last_undegraded":"2024-04-16T19:35:46.206084+0000","last_fullsized":"2024-04-16T19:35:46.206084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209219+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:35:41.209219+0000","last_peered":"2024-04-16T19:35:41.209219+0000","last_clean":"2024-04-16T19:35:41.209219+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:35:41.209219+0000","last_undegraded":"2024-04-16T19:35:41.209219+0000","last_fullsized":"2024-04-16T19:35:41.209219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'12","reported_seq":248,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.464290+0000","last_change":"2024-04-16T19:36:34.464387+0000","last_active":"2024-04-16T19:36:34.464290+0000","last_peered":"2024-04-16T19:36:34.464290+0000","last_clean":"2024-04-16T19:36:34.464290+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:36:34.464290+0000","last_undegraded":"2024-04-16T19:36:34.464290+0000","last_fullsized":"2024-04-16T19:36:34.464290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.464153+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:03.336022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207947+0000","last_change":"2024-04-16T17:47:23.261639+0000","last_active":"2024-04-16T19:35:46.207947+0000","last_peered":"2024-04-16T19:35:46.207947+0000","last_clean":"2024-04-16T19:35:46.207947+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:35:46.207947+0000","last_undegraded":"2024-04-16T19:35:46.207947+0000","last_fullsized":"2024-04-16T19:35:46.207947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.213866+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:36:16.213866+0000","last_peered":"2024-04-16T19:36:16.213866+0000","last_clean":"2024-04-16T19:36:16.213866+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:36:16.213866+0000","last_undegraded":"2024-04-16T19:36:16.213866+0000","last_fullsized":"2024-04-16T19:36:16.213866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"53'6","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.528043+0000","last_change":"2024-04-16T19:36:28.528121+0000","last_active":"2024-04-16T19:36:28.528043+0000","last_peered":"2024-04-16T19:36:28.528043+0000","last_clean":"2024-04-16T19:36:28.528043+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:36:28.528043+0000","last_undegraded":"2024-04-16T19:36:28.528043+0000","last_fullsized":"2024-04-16T19:36:28.528043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.527896+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:25.132224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194347+0000","last_change":"2024-04-16T17:47:23.253450+0000","last_active":"2024-04-16T19:36:16.194347+0000","last_peered":"2024-04-16T19:36:16.194347+0000","last_clean":"2024-04-16T19:36:16.194347+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:36:16.194347+0000","last_undegraded":"2024-04-16T19:36:16.194347+0000","last_fullsized":"2024-04-16T19:36:16.194347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205237+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:35:41.205237+0000","last_peered":"2024-04-16T19:35:41.205237+0000","last_clean":"2024-04-16T19:35:41.205237+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:35:41.205237+0000","last_undegraded":"2024-04-16T19:35:41.205237+0000","last_fullsized":"2024-04-16T19:35:41.205237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"53'2","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.582380+0000","last_change":"2024-04-16T19:36:26.582466+0000","last_active":"2024-04-16T19:36:26.582380+0000","last_peered":"2024-04-16T19:36:26.582380+0000","last_clean":"2024-04-16T19:36:26.582380+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:36:26.582380+0000","last_undegraded":"2024-04-16T19:36:26.582380+0000","last_fullsized":"2024-04-16T19:36:26.582380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.582238+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:10.024512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202168+0000","last_change":"2024-04-16T17:47:23.257600+0000","last_active":"2024-04-16T19:35:46.202168+0000","last_peered":"2024-04-16T19:35:46.202168+0000","last_clean":"2024-04-16T19:35:46.202168+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:35:46.202168+0000","last_undegraded":"2024-04-16T19:35:46.202168+0000","last_fullsized":"2024-04-16T19:35:46.202168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206571+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:35:41.206571+0000","last_peered":"2024-04-16T19:35:41.206571+0000","last_clean":"2024-04-16T19:35:41.206571+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:35:41.206571+0000","last_undegraded":"2024-04-16T19:35:41.206571+0000","last_fullsized":"2024-04-16T19:35:41.206571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'46","reported_seq":266,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.466241+0000","last_change":"2024-04-16T19:36:19.466346+0000","last_active":"2024-04-16T19:36:19.466241+0000","last_peered":"2024-04-16T19:36:19.466241+0000","last_clean":"2024-04-16T19:36:19.466241+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:36:19.466241+0000","last_undegraded":"2024-04-16T19:36:19.466241+0000","last_fullsized":"2024-04-16T19:36:19.466241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'46","last_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_deep_scrub":"53'46","last_deep_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.466115+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:05.816236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189483+0000","last_change":"2024-04-16T17:47:32.281080+0000","last_active":"2024-04-16T19:35:56.189483+0000","last_peered":"2024-04-16T19:35:56.189483+0000","last_clean":"2024-04-16T19:35:56.189483+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:35:56.189483+0000","last_undegraded":"2024-04-16T19:35:56.189483+0000","last_fullsized":"2024-04-16T19:35:56.189483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209423+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:35:41.209423+0000","last_peered":"2024-04-16T19:35:41.209423+0000","last_clean":"2024-04-16T19:35:41.209423+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:35:41.209423+0000","last_undegraded":"2024-04-16T19:35:41.209423+0000","last_fullsized":"2024-04-16T19:35:41.209423+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204923+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:35:56.204923+0000","last_peered":"2024-04-16T19:35:56.204923+0000","last_clean":"2024-04-16T19:35:56.204923+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:35:56.204923+0000","last_undegraded":"2024-04-16T19:35:56.204923+0000","last_fullsized":"2024-04-16T19:35:56.204923+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210401+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:35:46.210401+0000","last_peered":"2024-04-16T19:35:46.210401+0000","last_clean":"2024-04-16T19:35:46.210401+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:35:46.210401+0000","last_undegraded":"2024-04-16T19:35:46.210401+0000","last_fullsized":"2024-04-16T19:35:46.210401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186120+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:35:41.186120+0000","last_peered":"2024-04-16T19:35:41.186120+0000","last_clean":"2024-04-16T19:35:41.186120+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:35:41.186120+0000","last_undegraded":"2024-04-16T19:35:41.186120+0000","last_fullsized":"2024-04-16T19:35:41.186120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'21","reported_seq":207,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.453307+0000","last_change":"2024-04-16T19:36:21.453405+0000","last_active":"2024-04-16T19:36:21.453307+0000","last_peered":"2024-04-16T19:36:21.453307+0000","last_clean":"2024-04-16T19:36:21.453307+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:36:21.453307+0000","last_undegraded":"2024-04-16T19:36:21.453307+0000","last_fullsized":"2024-04-16T19:36:21.453307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.453170+0000","objects_scrubbed":6,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:26.892395+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.191534+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:34:46.191534+0000","last_peered":"2024-04-16T19:34:46.191534+0000","last_clean":"2024-04-16T19:34:46.191534+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:34:46.191534+0000","last_undegraded":"2024-04-16T19:34:46.191534+0000","last_fullsized":"2024-04-16T19:34:46.191534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.210981+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:36:06.210981+0000","last_peered":"2024-04-16T19:36:06.210981+0000","last_clean":"2024-04-16T19:36:06.210981+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:36:06.210981+0000","last_undegraded":"2024-04-16T19:36:06.210981+0000","last_fullsized":"2024-04-16T19:36:06.210981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'26","reported_seq":333,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.543796+0000","last_change":"2024-04-16T19:36:21.543887+0000","last_active":"2024-04-16T19:36:21.543796+0000","last_peered":"2024-04-16T19:36:21.543796+0000","last_clean":"2024-04-16T19:36:21.543796+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:36:21.543796+0000","last_undegraded":"2024-04-16T19:36:21.543796+0000","last_fullsized":"2024-04-16T19:36:21.543796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'26","last_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_deep_scrub":"53'26","last_deep_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.543655+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:47.643023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.187361+0000","last_change":"2024-04-16T17:47:23.264252+0000","last_active":"2024-04-16T19:34:46.187361+0000","last_peered":"2024-04-16T19:34:46.187361+0000","last_clean":"2024-04-16T19:34:46.187361+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:34:46.187361+0000","last_undegraded":"2024-04-16T19:34:46.187361+0000","last_fullsized":"2024-04-16T19:34:46.187361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200035+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:35:21.200035+0000","last_peered":"2024-04-16T19:35:21.200035+0000","last_clean":"2024-04-16T19:35:21.200035+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:35:21.200035+0000","last_undegraded":"2024-04-16T19:35:21.200035+0000","last_fullsized":"2024-04-16T19:35:21.200035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'2","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.488252+0000","last_change":"2024-04-16T19:36:22.488332+0000","last_active":"2024-04-16T19:36:22.488252+0000","last_peered":"2024-04-16T19:36:22.488252+0000","last_clean":"2024-04-16T19:36:22.488252+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:36:22.488252+0000","last_undegraded":"2024-04-16T19:36:22.488252+0000","last_fullsized":"2024-04-16T19:36:22.488252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.488109+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:37.620055+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189679+0000","last_change":"2024-04-16T17:47:23.265700+0000","last_active":"2024-04-16T19:35:56.189679+0000","last_peered":"2024-04-16T19:35:56.189679+0000","last_clean":"2024-04-16T19:35:56.189679+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:35:56.189679+0000","last_undegraded":"2024-04-16T19:35:56.189679+0000","last_fullsized":"2024-04-16T19:35:56.189679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199116+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:35:21.199116+0000","last_peered":"2024-04-16T19:35:21.199116+0000","last_clean":"2024-04-16T19:35:21.199116+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:35:21.199116+0000","last_undegraded":"2024-04-16T19:35:21.199116+0000","last_fullsized":"2024-04-16T19:35:21.199116+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'18","reported_seq":245,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194253+0000","last_change":"2024-04-16T17:47:55.147785+0000","last_active":"2024-04-16T19:36:16.194253+0000","last_peered":"2024-04-16T19:36:16.194253+0000","last_clean":"2024-04-16T19:36:16.194253+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:36:16.194253+0000","last_undegraded":"2024-04-16T19:36:16.194253+0000","last_fullsized":"2024-04-16T19:36:16.194253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.205594+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:35:26.205594+0000","last_peered":"2024-04-16T19:35:26.205594+0000","last_clean":"2024-04-16T19:35:26.205594+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:35:26.205594+0000","last_undegraded":"2024-04-16T19:35:26.205594+0000","last_fullsized":"2024-04-16T19:35:26.205594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200185+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:35:21.200185+0000","last_peered":"2024-04-16T19:35:21.200185+0000","last_clean":"2024-04-16T19:35:21.200185+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:35:21.200185+0000","last_undegraded":"2024-04-16T19:35:21.200185+0000","last_fullsized":"2024-04-16T19:35:21.200185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.568771+0000","last_change":"2024-04-16T19:36:22.568861+0000","last_active":"2024-04-16T19:36:22.568771+0000","last_peered":"2024-04-16T19:36:22.568771+0000","last_clean":"2024-04-16T19:36:22.568771+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:36:22.568771+0000","last_undegraded":"2024-04-16T19:36:22.568771+0000","last_fullsized":"2024-04-16T19:36:22.568771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.568637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:27.641841+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197209+0000","last_change":"2024-04-16T17:47:23.265903+0000","last_active":"2024-04-16T19:35:26.197209+0000","last_peered":"2024-04-16T19:35:26.197209+0000","last_clean":"2024-04-16T19:35:26.197209+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:35:26.197209+0000","last_undegraded":"2024-04-16T19:35:26.197209+0000","last_fullsized":"2024-04-16T19:35:26.197209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204436+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:35:21.204436+0000","last_peered":"2024-04-16T19:35:21.204436+0000","last_clean":"2024-04-16T19:35:21.204436+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:35:21.204436+0000","last_undegraded":"2024-04-16T19:35:21.204436+0000","last_fullsized":"2024-04-16T19:35:21.204436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"48'2","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194454+0000","last_change":"2024-04-16T17:47:55.162693+0000","last_active":"2024-04-16T19:36:16.194454+0000","last_peered":"2024-04-16T19:36:16.194454+0000","last_clean":"2024-04-16T19:36:16.194454+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:36:16.194454+0000","last_undegraded":"2024-04-16T19:36:16.194454+0000","last_fullsized":"2024-04-16T19:36:16.194454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201494+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:35:26.201494+0000","last_peered":"2024-04-16T19:35:26.201494+0000","last_clean":"2024-04-16T19:35:26.201494+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:35:26.201494+0000","last_undegraded":"2024-04-16T19:35:26.201494+0000","last_fullsized":"2024-04-16T19:35:26.201494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213227+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:35:56.213227+0000","last_peered":"2024-04-16T19:35:56.213227+0000","last_clean":"2024-04-16T19:35:56.213227+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:35:56.213227+0000","last_undegraded":"2024-04-16T19:35:56.213227+0000","last_fullsized":"2024-04-16T19:35:56.213227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'6","reported_seq":182,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.607311+0000","last_change":"2024-04-16T19:36:19.607390+0000","last_active":"2024-04-16T19:36:19.607311+0000","last_peered":"2024-04-16T19:36:19.607311+0000","last_clean":"2024-04-16T19:36:19.607311+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:36:19.607311+0000","last_undegraded":"2024-04-16T19:36:19.607311+0000","last_fullsized":"2024-04-16T19:36:19.607311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.607181+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:07:42.514098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.202288+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:35:26.202288+0000","last_peered":"2024-04-16T19:35:26.202288+0000","last_clean":"2024-04-16T19:35:26.202288+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:35:26.202288+0000","last_undegraded":"2024-04-16T19:35:26.202288+0000","last_fullsized":"2024-04-16T19:35:26.202288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.171926+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:34:41.171926+0000","last_peered":"2024-04-16T19:34:41.171926+0000","last_clean":"2024-04-16T19:34:41.171926+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:34:41.171926+0000","last_undegraded":"2024-04-16T19:34:41.171926+0000","last_fullsized":"2024-04-16T19:34:41.171926+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'12","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498647+0000","last_change":"2024-04-16T19:36:19.498750+0000","last_active":"2024-04-16T19:36:19.498647+0000","last_peered":"2024-04-16T19:36:19.498647+0000","last_clean":"2024-04-16T19:36:19.498647+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:36:19.498647+0000","last_undegraded":"2024-04-16T19:36:19.498647+0000","last_fullsized":"2024-04-16T19:36:19.498647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498508+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:33.427321+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978420+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:36:17.978420+0000","last_peered":"2024-04-16T19:36:17.978420+0000","last_clean":"2024-04-16T19:36:17.978420+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:36:17.978420+0000","last_undegraded":"2024-04-16T19:36:17.978420+0000","last_fullsized":"2024-04-16T19:36:17.978420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.188922+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:34:41.188922+0000","last_peered":"2024-04-16T19:34:41.188922+0000","last_clean":"2024-04-16T19:34:41.188922+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:34:41.188922+0000","last_undegraded":"2024-04-16T19:34:41.188922+0000","last_fullsized":"2024-04-16T19:34:41.188922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'16","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.449658+0000","last_change":"2024-04-16T19:36:28.449751+0000","last_active":"2024-04-16T19:36:28.449658+0000","last_peered":"2024-04-16T19:36:28.449658+0000","last_clean":"2024-04-16T19:36:28.449658+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:36:28.449658+0000","last_undegraded":"2024-04-16T19:36:28.449658+0000","last_fullsized":"2024-04-16T19:36:28.449658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.449522+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:19.924709+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194112+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:34:41.194112+0000","last_peered":"2024-04-16T19:34:41.194112+0000","last_clean":"2024-04-16T19:34:41.194112+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:34:41.194112+0000","last_undegraded":"2024-04-16T19:34:41.194112+0000","last_fullsized":"2024-04-16T19:34:41.194112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:21.185509+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:34:21.185509+0000","last_peered":"2024-04-16T19:34:21.185509+0000","last_clean":"2024-04-16T19:34:21.185509+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:34:21.185509+0000","last_undegraded":"2024-04-16T19:34:21.185509+0000","last_fullsized":"2024-04-16T19:34:21.185509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'20","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.594397+0000","last_change":"2024-04-16T19:36:27.594512+0000","last_active":"2024-04-16T19:36:27.594397+0000","last_peered":"2024-04-16T19:36:27.594397+0000","last_clean":"2024-04-16T19:36:27.594397+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:36:27.594397+0000","last_undegraded":"2024-04-16T19:36:27.594397+0000","last_fullsized":"2024-04-16T19:36:27.594397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.594261+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:01.851085+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.191347+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:34:41.191347+0000","last_peered":"2024-04-16T19:34:41.191347+0000","last_clean":"2024-04-16T19:34:41.191347+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:34:41.191347+0000","last_undegraded":"2024-04-16T19:34:41.191347+0000","last_fullsized":"2024-04-16T19:34:41.191347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212640+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:36:11.212640+0000","last_peered":"2024-04-16T19:36:11.212640+0000","last_clean":"2024-04-16T19:36:11.212640+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:36:11.212640+0000","last_undegraded":"2024-04-16T19:36:11.212640+0000","last_fullsized":"2024-04-16T19:36:11.212640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2844,"ondisk_log_size":2844,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3706,"ondisk_log_size":3706,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":887,"ondisk_log_size":887,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2422969,"num_omap_keys":10622,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27834,"ondisk_log_size":27834,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4463,"ondisk_log_size":4463,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":449,"ondisk_log_size":449,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641580,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46584,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811016,"statfs":{"total":107374182400,"available":107326480384,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641579,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641579,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":83292,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":104774308,"statfs":{"total":107374182400,"available":107288891392,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641579,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46572,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811028,"statfs":{"total":107374182400,"available":107326492672,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641580,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":55408,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":104802192,"statfs":{"total":107374182400,"available":107317444608,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641580,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":32700,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":104824900,"statfs":{"total":107374182400,"available":107340697600,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:36:39.888 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.888 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.888 INFO:tasks.ceph:pgid 9.34 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.888 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.888 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.889 INFO:tasks.ceph:pgid 9.37 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.889 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.889 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.889 INFO:tasks.ceph:pgid 9.36 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.889 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.889 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.890 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.890 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.890 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.890 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.890 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.891 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.891 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.891 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.891 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.891 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.892 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.892 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.892 INFO:tasks.ceph:pgid 9.3f last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.892 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.892 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.892 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.893 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.893 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.893 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.893 INFO:tasks.ceph:pgid 9.38 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.893 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.893 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.894 INFO:tasks.ceph:pgid 15.1f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.894 INFO:tasks.ceph:pgid 14.1e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.894 INFO:tasks.ceph:pgid 13.1d last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.894 INFO:tasks.ceph:pgid 12.1c last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.894 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.895 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.895 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.895 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.895 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.895 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.895 INFO:tasks.ceph:pgid 15.1e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.896 INFO:tasks.ceph:pgid 14.1f last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.896 INFO:tasks.ceph:pgid 13.1c last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.896 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.896 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.896 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.897 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.897 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.897 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.897 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.897 INFO:tasks.ceph:pgid 15.1d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.897 INFO:tasks.ceph:pgid 14.1c last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.898 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.898 INFO:tasks.ceph:pgid 11.19 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.898 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.898 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.898 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.899 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.899 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.899 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.899 INFO:tasks.ceph:pgid 15.1c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.899 INFO:tasks.ceph:pgid 14.1d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.899 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.900 INFO:tasks.ceph:pgid 11.18 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.900 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.900 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.900 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.900 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.900 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.901 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.901 INFO:tasks.ceph:pgid 15.3 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.901 INFO:tasks.ceph:pgid 14.2 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.901 INFO:tasks.ceph:pgid 13.1 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.901 INFO:tasks.ceph:pgid 12.0 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.902 INFO:tasks.ceph:pgid 11.7 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.902 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.902 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.902 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.902 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.902 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.903 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.903 INFO:tasks.ceph:pgid 15.2 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.903 INFO:tasks.ceph:pgid 14.3 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.903 INFO:tasks.ceph:pgid 13.0 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.903 INFO:tasks.ceph:pgid 12.1 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.903 INFO:tasks.ceph:pgid 11.6 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.904 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.904 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.904 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.904 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.904 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.904 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.905 INFO:tasks.ceph:pgid 15.1 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.905 INFO:tasks.ceph:pgid 14.0 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.905 INFO:tasks.ceph:pgid 13.3 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.905 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.905 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.906 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.906 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.906 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.906 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.906 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.906 INFO:tasks.ceph:pgid 15.0 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.907 INFO:tasks.ceph:pgid 14.1 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.907 INFO:tasks.ceph:pgid 13.2 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.907 INFO:tasks.ceph:pgid 12.3 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.907 INFO:tasks.ceph:pgid 11.4 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.907 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.907 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.908 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.908 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.908 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.908 INFO:tasks.ceph:pgid 15.7 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.908 INFO:tasks.ceph:pgid 14.6 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.909 INFO:tasks.ceph:pgid 13.5 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.909 INFO:tasks.ceph:pgid 12.4 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.909 INFO:tasks.ceph:pgid 11.3 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.909 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.909 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.909 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.910 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.910 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.910 INFO:tasks.ceph:pgid 15.6 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.910 INFO:tasks.ceph:pgid 14.7 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.910 INFO:tasks.ceph:pgid 13.4 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.911 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.911 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.911 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.911 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.911 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.912 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.912 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.912 INFO:tasks.ceph:pgid 15.5 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.912 INFO:tasks.ceph:pgid 14.4 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.912 INFO:tasks.ceph:pgid 13.7 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.912 INFO:tasks.ceph:pgid 12.6 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.913 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.913 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.913 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.913 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.913 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.913 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.914 INFO:tasks.ceph:pgid 15.4 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.914 INFO:tasks.ceph:pgid 14.5 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.914 INFO:tasks.ceph:pgid 13.6 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.914 INFO:tasks.ceph:pgid 12.7 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.914 INFO:tasks.ceph:pgid 11.0 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.914 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.915 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.915 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.915 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.915 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.915 INFO:tasks.ceph:pgid 15.e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.916 INFO:tasks.ceph:pgid 14.f last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.916 INFO:tasks.ceph:pgid 13.c last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.916 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.916 INFO:tasks.ceph:pgid 11.a last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.916 INFO:tasks.ceph:pgid 9.8 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.916 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.917 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.917 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.917 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.917 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.917 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.917 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.918 INFO:tasks.ceph:pgid 15.f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.918 INFO:tasks.ceph:pgid 14.e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.918 INFO:tasks.ceph:pgid 13.d last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.918 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.918 INFO:tasks.ceph:pgid 11.b last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.919 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.919 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.919 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.919 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.919 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.919 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.920 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.920 INFO:tasks.ceph:pgid 15.c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.920 INFO:tasks.ceph:pgid 14.d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.920 INFO:tasks.ceph:pgid 13.e last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.920 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.920 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.921 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.921 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.921 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.921 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.921 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.921 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.922 INFO:tasks.ceph:pgid 15.d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.922 INFO:tasks.ceph:pgid 14.c last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.922 INFO:tasks.ceph:pgid 13.f last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.922 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.922 INFO:tasks.ceph:pgid 11.9 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.923 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.923 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.923 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-16T17:46:37.490454+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=37, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.923 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.923 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.923 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.924 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.924 INFO:tasks.ceph:pgid 15.a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.924 INFO:tasks.ceph:pgid 14.b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.924 INFO:tasks.ceph:pgid 13.8 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.924 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.925 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.925 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.925 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.925 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.925 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.925 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.926 INFO:tasks.ceph:pgid 15.b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.926 INFO:tasks.ceph:pgid 14.a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.926 INFO:tasks.ceph:pgid 13.9 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.926 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.926 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.926 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.927 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.927 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.927 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.927 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.927 INFO:tasks.ceph:pgid 15.8 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.927 INFO:tasks.ceph:pgid 14.9 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.928 INFO:tasks.ceph:pgid 13.a last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.928 INFO:tasks.ceph:pgid 12.b last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.928 INFO:tasks.ceph:pgid 11.c last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.928 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.928 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.928 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.929 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.929 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.929 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.929 INFO:tasks.ceph:pgid 15.9 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.929 INFO:tasks.ceph:pgid 14.8 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.929 INFO:tasks.ceph:pgid 13.b last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.930 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.930 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.930 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.930 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.930 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.931 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.931 INFO:tasks.ceph:pgid 15.18 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.931 INFO:tasks.ceph:pgid 14.19 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.931 INFO:tasks.ceph:pgid 13.1a last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.931 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.932 INFO:tasks.ceph:pgid 9.1e last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.932 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.932 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.932 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.932 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.932 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.933 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.933 INFO:tasks.ceph:pgid 15.19 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.933 INFO:tasks.ceph:pgid 14.18 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.933 INFO:tasks.ceph:pgid 13.1b last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.933 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.933 INFO:tasks.ceph:pgid 11.1d last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.934 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.934 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.934 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.934 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.934 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.934 INFO:tasks.ceph:pgid 15.1a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.935 INFO:tasks.ceph:pgid 14.1b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.935 INFO:tasks.ceph:pgid 13.18 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.935 INFO:tasks.ceph:pgid 12.19 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.935 INFO:tasks.ceph:pgid 11.1e last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.935 INFO:tasks.ceph:pgid 10.1f last_scrub_stamp 2024-04-16T17:47:56.136446+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=56, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.935 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.936 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.936 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.936 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.936 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.936 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.936 INFO:tasks.ceph:pgid 15.1b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.937 INFO:tasks.ceph:pgid 14.1a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.937 INFO:tasks.ceph:pgid 13.19 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.937 INFO:tasks.ceph:pgid 12.18 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.937 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.937 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.937 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.938 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.938 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.938 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.938 INFO:tasks.ceph:pgid 15.14 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.938 INFO:tasks.ceph:pgid 14.15 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.939 INFO:tasks.ceph:pgid 13.16 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.939 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.939 INFO:tasks.ceph:pgid 9.12 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.939 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.939 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.939 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.940 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.940 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.940 INFO:tasks.ceph:pgid 15.15 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.940 INFO:tasks.ceph:pgid 14.14 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.940 INFO:tasks.ceph:pgid 13.17 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.940 INFO:tasks.ceph:pgid 12.16 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.941 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.941 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.941 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.941 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.941 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.942 INFO:tasks.ceph:pgid 15.16 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.942 INFO:tasks.ceph:pgid 14.17 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.942 INFO:tasks.ceph:pgid 13.14 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.942 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.942 INFO:tasks.ceph:pgid 11.12 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.943 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.943 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.943 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.943 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.943 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.943 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.944 INFO:tasks.ceph:pgid 15.17 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.944 INFO:tasks.ceph:pgid 14.16 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.944 INFO:tasks.ceph:pgid 13.15 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.944 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.944 INFO:tasks.ceph:pgid 11.13 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.944 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.945 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.945 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.945 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.945 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.945 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.945 INFO:tasks.ceph:pgid 15.10 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.946 INFO:tasks.ceph:pgid 14.11 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.946 INFO:tasks.ceph:pgid 13.12 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.946 INFO:tasks.ceph:pgid 12.13 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.946 INFO:tasks.ceph:pgid 11.14 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.946 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.946 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.947 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.947 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.947 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.947 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.947 INFO:tasks.ceph:pgid 15.11 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.947 INFO:tasks.ceph:pgid 14.10 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.948 INFO:tasks.ceph:pgid 13.13 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.948 INFO:tasks.ceph:pgid 12.12 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.948 INFO:tasks.ceph:pgid 11.15 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.948 INFO:tasks.ceph:pgid 9.17 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.948 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.949 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.949 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.949 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.949 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.949 INFO:tasks.ceph:pgid 15.12 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.949 INFO:tasks.ceph:pgid 14.13 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.950 INFO:tasks.ceph:pgid 13.10 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.950 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.950 INFO:tasks.ceph:pgid 11.16 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.950 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.950 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.950 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.951 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.951 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.951 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.951 INFO:tasks.ceph:pgid 15.13 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.951 INFO:tasks.ceph:pgid 14.12 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.951 INFO:tasks.ceph:pgid 13.11 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.952 INFO:tasks.ceph:pgid 12.10 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.952 INFO:tasks.ceph:pgid 11.17 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.952 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.952 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.952 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.952 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.953 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.953 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-16T17:47:38.301675+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=38, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.953 INFO:tasks.ceph:pgid 9.2a last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.953 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.953 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.953 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.954 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.954 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.954 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.954 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.954 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.955 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.955 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.955 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.955 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.955 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.955 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.956 INFO:tasks.ceph:pgid 9.2d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.956 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.956 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.956 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.956 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.956 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.957 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.957 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.957 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.957 INFO:tasks.ceph:pgid 9.21 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.957 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.957 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.958 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.958 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.958 INFO:tasks.ceph:pgid 9.27 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.958 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.958 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.959 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.959 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.959 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.959 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.959 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.959 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.960 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.960 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:36:39.960 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-16T19:36:59.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:37:00.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:37:00.371 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:37:00.374 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3357,"stamp":"2024-04-16T19:36:58.352950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2481916,"num_omap_keys":10882,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40924,"ondisk_log_size":40924,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":345712,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":335542,"kb_avail":628799888,"statfs":{"total":644245094400,"available":643891085312,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":343595680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58789,"num_omap_keys":257,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010621"},"pg_stats":[{"pgid":"9.35","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.507786+0000","last_change":"2024-04-16T19:36:20.507786+0000","last_active":"2024-04-16T19:36:20.507786+0000","last_peered":"2024-04-16T19:36:20.507786+0000","last_clean":"2024-04-16T19:36:20.507786+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:36:20.507786+0000","last_undegraded":"2024-04-16T19:36:20.507786+0000","last_fullsized":"2024-04-16T19:36:20.507786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.507587+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:21:12.018020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209498+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:35:41.209498+0000","last_peered":"2024-04-16T19:35:41.209498+0000","last_clean":"2024-04-16T19:35:41.209498+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:35:41.209498+0000","last_undegraded":"2024-04-16T19:35:41.209498+0000","last_fullsized":"2024-04-16T19:35:41.209498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192292+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:34:56.192292+0000","last_peered":"2024-04-16T19:34:56.192292+0000","last_clean":"2024-04-16T19:34:56.192292+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:34:56.192292+0000","last_undegraded":"2024-04-16T19:34:56.192292+0000","last_fullsized":"2024-04-16T19:34:56.192292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'23","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.496453+0000","last_change":"2024-04-16T19:36:37.496453+0000","last_active":"2024-04-16T19:36:37.496453+0000","last_peered":"2024-04-16T19:36:37.496453+0000","last_clean":"2024-04-16T19:36:37.496453+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:36:37.496453+0000","last_undegraded":"2024-04-16T19:36:37.496453+0000","last_fullsized":"2024-04-16T19:36:37.496453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.496061+0000","objects_scrubbed":9,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:45.797648+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206637+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:35:41.206637+0000","last_peered":"2024-04-16T19:35:41.206637+0000","last_clean":"2024-04-16T19:35:41.206637+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:35:41.206637+0000","last_undegraded":"2024-04-16T19:35:41.206637+0000","last_fullsized":"2024-04-16T19:35:41.206637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.197906+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:34:56.197906+0000","last_peered":"2024-04-16T19:34:56.197906+0000","last_clean":"2024-04-16T19:34:56.197906+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:34:56.197906+0000","last_undegraded":"2024-04-16T19:34:56.197906+0000","last_fullsized":"2024-04-16T19:34:56.197906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'25","reported_seq":183,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.596357+0000","last_change":"2024-04-16T19:36:46.596413+0000","last_active":"2024-04-16T19:36:46.596357+0000","last_peered":"2024-04-16T19:36:46.596357+0000","last_clean":"2024-04-16T19:36:46.596357+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:36:46.596357+0000","last_undegraded":"2024-04-16T19:36:46.596357+0000","last_fullsized":"2024-04-16T19:36:46.596357+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'25","last_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_deep_scrub":"53'25","last_deep_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.596297+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:26.632824+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205312+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:35:41.205312+0000","last_peered":"2024-04-16T19:35:41.205312+0000","last_clean":"2024-04-16T19:35:41.205312+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:35:41.205312+0000","last_undegraded":"2024-04-16T19:35:41.205312+0000","last_fullsized":"2024-04-16T19:35:41.205312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189744+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:34:56.189744+0000","last_peered":"2024-04-16T19:34:56.189744+0000","last_clean":"2024-04-16T19:34:56.189744+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:34:56.189744+0000","last_undegraded":"2024-04-16T19:34:56.189744+0000","last_fullsized":"2024-04-16T19:34:56.189744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.049703+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:36:17.049703+0000","last_peered":"2024-04-16T19:36:17.049703+0000","last_clean":"2024-04-16T19:36:17.049703+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:36:17.049703+0000","last_undegraded":"2024-04-16T19:36:17.049703+0000","last_fullsized":"2024-04-16T19:36:17.049703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.205993+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:35:46.205993+0000","last_peered":"2024-04-16T19:35:46.205993+0000","last_clean":"2024-04-16T19:35:46.205993+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:35:46.205993+0000","last_undegraded":"2024-04-16T19:35:46.205993+0000","last_fullsized":"2024-04-16T19:35:46.205993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193915+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:34:56.193915+0000","last_peered":"2024-04-16T19:34:56.193915+0000","last_clean":"2024-04-16T19:34:56.193915+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:34:56.193915+0000","last_undegraded":"2024-04-16T19:34:56.193915+0000","last_fullsized":"2024-04-16T19:34:56.193915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'8","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.663272+0000","last_change":"2024-04-16T19:36:31.663358+0000","last_active":"2024-04-16T19:36:31.663272+0000","last_peered":"2024-04-16T19:36:31.663272+0000","last_clean":"2024-04-16T19:36:31.663272+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:36:31.663272+0000","last_undegraded":"2024-04-16T19:36:31.663272+0000","last_fullsized":"2024-04-16T19:36:31.663272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.663140+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:09.078400+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197555+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:35:06.197555+0000","last_peered":"2024-04-16T19:35:06.197555+0000","last_clean":"2024-04-16T19:35:06.197555+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:35:06.197555+0000","last_undegraded":"2024-04-16T19:35:06.197555+0000","last_fullsized":"2024-04-16T19:35:06.197555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207378+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:35:51.207378+0000","last_peered":"2024-04-16T19:35:51.207378+0000","last_clean":"2024-04-16T19:35:51.207378+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:35:51.207378+0000","last_undegraded":"2024-04-16T19:35:51.207378+0000","last_fullsized":"2024-04-16T19:35:51.207378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'20","reported_seq":327,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.492651+0000","last_change":"2024-04-16T19:36:26.492760+0000","last_active":"2024-04-16T19:36:26.492651+0000","last_peered":"2024-04-16T19:36:26.492651+0000","last_clean":"2024-04-16T19:36:26.492651+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:36:26.492651+0000","last_undegraded":"2024-04-16T19:36:26.492651+0000","last_fullsized":"2024-04-16T19:36:26.492651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.492498+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:56.598246+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191143+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:34:51.191143+0000","last_peered":"2024-04-16T19:34:51.191143+0000","last_clean":"2024-04-16T19:34:51.191143+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:34:51.191143+0000","last_undegraded":"2024-04-16T19:34:51.191143+0000","last_fullsized":"2024-04-16T19:34:51.191143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202288+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:35:46.202288+0000","last_peered":"2024-04-16T19:35:46.202288+0000","last_clean":"2024-04-16T19:35:46.202288+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:35:46.202288+0000","last_undegraded":"2024-04-16T19:35:46.202288+0000","last_fullsized":"2024-04-16T19:35:46.202288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'19","reported_seq":285,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.411649+0000","last_change":"2024-04-16T19:36:31.411649+0000","last_active":"2024-04-16T19:36:31.411649+0000","last_peered":"2024-04-16T19:36:31.411649+0000","last_clean":"2024-04-16T19:36:31.411649+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:36:31.411649+0000","last_undegraded":"2024-04-16T19:36:31.411649+0000","last_fullsized":"2024-04-16T19:36:31.411649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.411418+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:18.775822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.196650+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:34:51.196650+0000","last_peered":"2024-04-16T19:34:51.196650+0000","last_clean":"2024-04-16T19:34:51.196650+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:34:51.196650+0000","last_undegraded":"2024-04-16T19:34:51.196650+0000","last_fullsized":"2024-04-16T19:34:51.196650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206153+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:35:46.206153+0000","last_peered":"2024-04-16T19:35:46.206153+0000","last_clean":"2024-04-16T19:35:46.206153+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:35:46.206153+0000","last_undegraded":"2024-04-16T19:35:46.206153+0000","last_fullsized":"2024-04-16T19:35:46.206153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'14","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.657838+0000","last_change":"2024-04-16T19:36:29.657923+0000","last_active":"2024-04-16T19:36:29.657838+0000","last_peered":"2024-04-16T19:36:29.657838+0000","last_clean":"2024-04-16T19:36:29.657838+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:36:29.657838+0000","last_undegraded":"2024-04-16T19:36:29.657838+0000","last_fullsized":"2024-04-16T19:36:29.657838+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.657764+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:42.577097+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.174105+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:34:51.174105+0000","last_peered":"2024-04-16T19:34:51.174105+0000","last_clean":"2024-04-16T19:34:51.174105+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:34:51.174105+0000","last_undegraded":"2024-04-16T19:34:51.174105+0000","last_fullsized":"2024-04-16T19:34:51.174105+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.187294+0000","last_change":"2024-04-16T17:47:23.254199+0000","last_active":"2024-04-16T19:35:46.187294+0000","last_peered":"2024-04-16T19:35:46.187294+0000","last_clean":"2024-04-16T19:35:46.187294+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:35:46.187294+0000","last_undegraded":"2024-04-16T19:35:46.187294+0000","last_fullsized":"2024-04-16T19:35:46.187294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"53'10","reported_seq":315,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.620843+0000","last_change":"2024-04-16T19:36:21.620914+0000","last_active":"2024-04-16T19:36:21.620843+0000","last_peered":"2024-04-16T19:36:21.620843+0000","last_clean":"2024-04-16T19:36:21.620843+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:36:21.620843+0000","last_undegraded":"2024-04-16T19:36:21.620843+0000","last_fullsized":"2024-04-16T19:36:21.620843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.620737+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:28.047519+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.186990+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:34:46.186990+0000","last_peered":"2024-04-16T19:34:46.186990+0000","last_clean":"2024-04-16T19:34:46.186990+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:34:46.186990+0000","last_undegraded":"2024-04-16T19:34:46.186990+0000","last_fullsized":"2024-04-16T19:34:46.186990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048553+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:36:17.048553+0000","last_peered":"2024-04-16T19:36:17.048553+0000","last_clean":"2024-04-16T19:36:17.048553+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:36:17.048553+0000","last_undegraded":"2024-04-16T19:36:17.048553+0000","last_fullsized":"2024-04-16T19:36:17.048553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'18","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.400030+0000","last_change":"2024-04-16T19:36:26.400030+0000","last_active":"2024-04-16T19:36:26.400030+0000","last_peered":"2024-04-16T19:36:26.400030+0000","last_clean":"2024-04-16T19:36:26.400030+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:36:26.400030+0000","last_undegraded":"2024-04-16T19:36:26.400030+0000","last_fullsized":"2024-04-16T19:36:26.400030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.399732+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:00.453174+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.189884+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:34:46.189884+0000","last_peered":"2024-04-16T19:34:46.189884+0000","last_clean":"2024-04-16T19:34:46.189884+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:34:46.189884+0000","last_undegraded":"2024-04-16T19:34:46.189884+0000","last_fullsized":"2024-04-16T19:34:46.189884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215736+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:36:06.215736+0000","last_peered":"2024-04-16T19:36:06.215736+0000","last_clean":"2024-04-16T19:36:06.215736+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:36:06.215736+0000","last_undegraded":"2024-04-16T19:36:06.215736+0000","last_fullsized":"2024-04-16T19:36:06.215736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'16","reported_seq":288,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.440103+0000","last_change":"2024-04-16T19:36:42.440209+0000","last_active":"2024-04-16T19:36:42.440103+0000","last_peered":"2024-04-16T19:36:42.440103+0000","last_clean":"2024-04-16T19:36:42.440103+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:36:42.440103+0000","last_undegraded":"2024-04-16T19:36:42.440103+0000","last_fullsized":"2024-04-16T19:36:42.440103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.439969+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.619590+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.196138+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:35:11.196138+0000","last_peered":"2024-04-16T19:35:11.196138+0000","last_clean":"2024-04-16T19:35:11.196138+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:35:11.196138+0000","last_undegraded":"2024-04-16T19:35:11.196138+0000","last_fullsized":"2024-04-16T19:35:11.196138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211293+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:36:06.211293+0000","last_peered":"2024-04-16T19:36:06.211293+0000","last_clean":"2024-04-16T19:36:06.211293+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:36:06.211293+0000","last_undegraded":"2024-04-16T19:36:06.211293+0000","last_fullsized":"2024-04-16T19:36:06.211293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"53'9","reported_seq":241,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.533226+0000","last_change":"2024-04-16T19:36:29.533377+0000","last_active":"2024-04-16T19:36:29.533226+0000","last_peered":"2024-04-16T19:36:29.533226+0000","last_clean":"2024-04-16T19:36:29.533226+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:36:29.533226+0000","last_undegraded":"2024-04-16T19:36:29.533226+0000","last_fullsized":"2024-04-16T19:36:29.533226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.533050+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:35:44.595573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.193282+0000","last_change":"2024-04-16T17:47:23.254226+0000","last_active":"2024-04-16T19:36:11.193282+0000","last_peered":"2024-04-16T19:36:11.193282+0000","last_clean":"2024-04-16T19:36:11.193282+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:36:11.193282+0000","last_undegraded":"2024-04-16T19:36:11.193282+0000","last_fullsized":"2024-04-16T19:36:11.193282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.207335+0000","last_change":"2024-04-16T17:47:32.284109+0000","last_active":"2024-04-16T19:36:06.207335+0000","last_peered":"2024-04-16T19:36:06.207335+0000","last_clean":"2024-04-16T19:36:06.207335+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:36:06.207335+0000","last_undegraded":"2024-04-16T19:36:06.207335+0000","last_fullsized":"2024-04-16T19:36:06.207335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'10","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498172+0000","last_change":"2024-04-16T19:36:19.498276+0000","last_active":"2024-04-16T19:36:19.498172+0000","last_peered":"2024-04-16T19:36:19.498172+0000","last_clean":"2024-04-16T19:36:19.498172+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:36:19.498172+0000","last_undegraded":"2024-04-16T19:36:19.498172+0000","last_fullsized":"2024-04-16T19:36:19.498172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498013+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:24.571217+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.193367+0000","last_change":"2024-04-16T17:47:23.266132+0000","last_active":"2024-04-16T19:36:11.193367+0000","last_peered":"2024-04-16T19:36:11.193367+0000","last_clean":"2024-04-16T19:36:11.193367+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:36:11.193367+0000","last_undegraded":"2024-04-16T19:36:11.193367+0000","last_fullsized":"2024-04-16T19:36:11.193367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048603+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:36:17.048603+0000","last_peered":"2024-04-16T19:36:17.048603+0000","last_clean":"2024-04-16T19:36:17.048603+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:36:17.048603+0000","last_undegraded":"2024-04-16T19:36:17.048603+0000","last_fullsized":"2024-04-16T19:36:17.048603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'19","reported_seq":283,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.534237+0000","last_change":"2024-04-16T19:36:45.534337+0000","last_active":"2024-04-16T19:36:45.534237+0000","last_peered":"2024-04-16T19:36:45.534237+0000","last_clean":"2024-04-16T19:36:45.534237+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:36:45.534237+0000","last_undegraded":"2024-04-16T19:36:45.534237+0000","last_fullsized":"2024-04-16T19:36:45.534237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.534098+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:29.053953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.208583+0000","last_change":"2024-04-16T17:47:23.257289+0000","last_active":"2024-04-16T19:36:11.208583+0000","last_peered":"2024-04-16T19:36:11.208583+0000","last_clean":"2024-04-16T19:36:11.208583+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:36:11.208583+0000","last_undegraded":"2024-04-16T19:36:11.208583+0000","last_fullsized":"2024-04-16T19:36:11.208583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.198747+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:35:16.198747+0000","last_peered":"2024-04-16T19:35:16.198747+0000","last_clean":"2024-04-16T19:35:16.198747+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:35:16.198747+0000","last_undegraded":"2024-04-16T19:35:16.198747+0000","last_fullsized":"2024-04-16T19:35:16.198747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215769+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:36:16.215769+0000","last_peered":"2024-04-16T19:36:16.215769+0000","last_clean":"2024-04-16T19:36:16.215769+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:36:16.215769+0000","last_undegraded":"2024-04-16T19:36:16.215769+0000","last_fullsized":"2024-04-16T19:36:16.215769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1351,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.201950+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:35:11.201950+0000","last_peered":"2024-04-16T19:35:11.201950+0000","last_clean":"2024-04-16T19:35:11.201950+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:35:11.201950+0000","last_undegraded":"2024-04-16T19:35:11.201950+0000","last_fullsized":"2024-04-16T19:35:11.201950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.214384+0000","last_change":"2024-04-16T17:48:27.308777+0000","last_active":"2024-04-16T19:36:01.214384+0000","last_peered":"2024-04-16T19:36:01.214384+0000","last_clean":"2024-04-16T19:36:01.214384+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:36:01.214384+0000","last_undegraded":"2024-04-16T19:36:01.214384+0000","last_fullsized":"2024-04-16T19:36:01.214384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.645164+0000","last_change":"2024-04-16T19:36:51.645240+0000","last_active":"2024-04-16T19:36:51.645164+0000","last_peered":"2024-04-16T19:36:51.645164+0000","last_clean":"2024-04-16T19:36:51.645164+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:36:51.645164+0000","last_undegraded":"2024-04-16T19:36:51.645164+0000","last_fullsized":"2024-04-16T19:36:51.645164+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.645030+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:42.615404+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.617380+0000","last_change":"2024-04-16T19:36:26.617439+0000","last_active":"2024-04-16T19:36:26.617380+0000","last_peered":"2024-04-16T19:36:26.617380+0000","last_clean":"2024-04-16T19:36:26.617380+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:36:26.617380+0000","last_undegraded":"2024-04-16T19:36:26.617380+0000","last_fullsized":"2024-04-16T19:36:26.617380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.617211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:50.470545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'563","reported_seq":824,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.471001+0000","last_change":"2024-04-16T19:36:20.471131+0000","last_active":"2024-04-16T19:36:20.471001+0000","last_peered":"2024-04-16T19:36:20.471001+0000","last_clean":"2024-04-16T19:36:20.471001+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:36:20.471001+0000","last_undegraded":"2024-04-16T19:36:20.471001+0000","last_fullsized":"2024-04-16T19:36:20.471001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.470861+0000","objects_scrubbed":16,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:34.399674+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71932,"num_omap_keys":314,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'14","reported_seq":222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.561685+0000","last_change":"2024-04-16T19:36:25.561790+0000","last_active":"2024-04-16T19:36:25.561685+0000","last_peered":"2024-04-16T19:36:25.561685+0000","last_clean":"2024-04-16T19:36:25.561685+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:36:25.561685+0000","last_undegraded":"2024-04-16T19:36:25.561685+0000","last_fullsized":"2024-04-16T19:36:25.561685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.561550+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:02:57.257953+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.527133+0000","last_change":"2024-04-16T19:36:50.527184+0000","last_active":"2024-04-16T19:36:50.527133+0000","last_peered":"2024-04-16T19:36:50.527133+0000","last_clean":"2024-04-16T19:36:50.527133+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:36:50.527133+0000","last_undegraded":"2024-04-16T19:36:50.527133+0000","last_fullsized":"2024-04-16T19:36:50.527133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.527001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:19.398870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.200023+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:35:16.200023+0000","last_peered":"2024-04-16T19:35:16.200023+0000","last_clean":"2024-04-16T19:35:16.200023+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:35:16.200023+0000","last_undegraded":"2024-04-16T19:35:16.200023+0000","last_fullsized":"2024-04-16T19:35:16.200023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194993+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:34:56.194993+0000","last_peered":"2024-04-16T19:34:56.194993+0000","last_clean":"2024-04-16T19:34:56.194993+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:34:56.194993+0000","last_undegraded":"2024-04-16T19:34:56.194993+0000","last_fullsized":"2024-04-16T19:34:56.194993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":770,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912468+0000","last_change":"2024-04-16T17:47:33.303155+0000","last_active":"2024-04-16T19:36:18.912468+0000","last_peered":"2024-04-16T19:36:18.912468+0000","last_clean":"2024-04-16T19:36:18.912468+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:36:18.912468+0000","last_undegraded":"2024-04-16T19:36:18.912468+0000","last_fullsized":"2024-04-16T19:36:18.912468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586459+0000","last_change":"2024-04-16T17:47:35.314855+0000","last_active":"2024-04-16T19:36:16.586459+0000","last_peered":"2024-04-16T19:36:16.586459+0000","last_clean":"2024-04-16T19:36:16.586459+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:36:16.586459+0000","last_undegraded":"2024-04-16T19:36:16.586459+0000","last_fullsized":"2024-04-16T19:36:16.586459+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.400379+0000","last_change":"2024-04-16T19:36:48.400438+0000","last_active":"2024-04-16T19:36:48.400379+0000","last_peered":"2024-04-16T19:36:48.400379+0000","last_clean":"2024-04-16T19:36:48.400379+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:36:48.400379+0000","last_undegraded":"2024-04-16T19:36:48.400379+0000","last_fullsized":"2024-04-16T19:36:48.400379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.400232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:53.461118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181269+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:35:21.181269+0000","last_peered":"2024-04-16T19:35:21.181269+0000","last_clean":"2024-04-16T19:35:21.181269+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:35:21.181269+0000","last_undegraded":"2024-04-16T19:35:21.181269+0000","last_fullsized":"2024-04-16T19:35:21.181269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'83","reported_seq":1422,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588075+0000","last_change":"2024-04-16T17:48:34.319654+0000","last_active":"2024-04-16T19:36:16.588075+0000","last_peered":"2024-04-16T19:36:16.588075+0000","last_clean":"2024-04-16T19:36:16.588075+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:36:16.588075+0000","last_undegraded":"2024-04-16T19:36:16.588075+0000","last_fullsized":"2024-04-16T19:36:16.588075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":76,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209948+0000","last_change":"2024-04-16T17:48:27.303999+0000","last_active":"2024-04-16T19:36:01.209948+0000","last_peered":"2024-04-16T19:36:01.209948+0000","last_clean":"2024-04-16T19:36:01.209948+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:36:01.209948+0000","last_undegraded":"2024-04-16T19:36:01.209948+0000","last_fullsized":"2024-04-16T19:36:01.209948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'9","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.654127+0000","last_change":"2024-04-16T19:36:50.654208+0000","last_active":"2024-04-16T19:36:50.654127+0000","last_peered":"2024-04-16T19:36:50.654127+0000","last_clean":"2024-04-16T19:36:50.654127+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:36:50.654127+0000","last_undegraded":"2024-04-16T19:36:50.654127+0000","last_fullsized":"2024-04-16T19:36:50.654127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.653994+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:30.491092+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.479488+0000","last_change":"2024-04-16T19:36:33.479590+0000","last_active":"2024-04-16T19:36:33.479488+0000","last_peered":"2024-04-16T19:36:33.479488+0000","last_clean":"2024-04-16T19:36:33.479488+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:36:33.479488+0000","last_undegraded":"2024-04-16T19:36:33.479488+0000","last_fullsized":"2024-04-16T19:36:33.479488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.479280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:24.835481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7349","reported_seq":20627,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.602403+0000","last_change":"2024-04-16T19:36:20.602529+0000","last_active":"2024-04-16T19:36:20.602403+0000","last_peered":"2024-04-16T19:36:20.602403+0000","last_clean":"2024-04-16T19:36:20.602403+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:36:20.602403+0000","last_undegraded":"2024-04-16T19:36:20.602403+0000","last_fullsized":"2024-04-16T19:36:20.602403+0000","mapping_epoch":34,"log_start":"53'5600","ondisk_log_start":"53'5600","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7349","last_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_deep_scrub":"53'7349","last_deep_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.602264+0000","objects_scrubbed":23,"log_size":1749,"log_dups_size":3000,"ondisk_log_size":1749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:12.702919+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72188,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'30","reported_seq":375,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.493531+0000","last_change":"2024-04-16T19:36:32.493531+0000","last_active":"2024-04-16T19:36:32.493531+0000","last_peered":"2024-04-16T19:36:32.493531+0000","last_clean":"2024-04-16T19:36:32.493531+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:36:32.493531+0000","last_undegraded":"2024-04-16T19:36:32.493531+0000","last_fullsized":"2024-04-16T19:36:32.493531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'30","last_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_deep_scrub":"53'30","last_deep_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.493294+0000","objects_scrubbed":11,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:58.339768+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.426772+0000","last_change":"2024-04-16T19:36:46.426772+0000","last_active":"2024-04-16T19:36:46.426772+0000","last_peered":"2024-04-16T19:36:46.426772+0000","last_clean":"2024-04-16T19:36:46.426772+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:36:46.426772+0000","last_undegraded":"2024-04-16T19:36:46.426772+0000","last_fullsized":"2024-04-16T19:36:46.426772+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.426574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.708425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212569+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:36:11.212569+0000","last_peered":"2024-04-16T19:36:11.212569+0000","last_clean":"2024-04-16T19:36:11.212569+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:36:11.212569+0000","last_undegraded":"2024-04-16T19:36:11.212569+0000","last_fullsized":"2024-04-16T19:36:11.212569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.190022+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:34:41.190022+0000","last_peered":"2024-04-16T19:34:41.190022+0000","last_clean":"2024-04-16T19:34:41.190022+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:34:41.190022+0000","last_undegraded":"2024-04-16T19:34:41.190022+0000","last_fullsized":"2024-04-16T19:34:41.190022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"48'5","reported_seq":360,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.202876+0000","last_change":"2024-04-16T17:47:33.310538+0000","last_active":"2024-04-16T19:35:31.202876+0000","last_peered":"2024-04-16T19:35:31.202876+0000","last_clean":"2024-04-16T19:35:31.202876+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:35:31.202876+0000","last_undegraded":"2024-04-16T19:35:31.202876+0000","last_fullsized":"2024-04-16T19:35:31.202876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'72","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438215+0000","last_change":"2024-04-16T17:47:35.305717+0000","last_active":"2024-04-16T19:36:18.438215+0000","last_peered":"2024-04-16T19:36:18.438215+0000","last_clean":"2024-04-16T19:36:18.438215+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:36:18.438215+0000","last_undegraded":"2024-04-16T19:36:18.438215+0000","last_fullsized":"2024-04-16T19:36:18.438215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":106,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438355+0000","last_change":"2024-04-16T17:47:37.326918+0000","last_active":"2024-04-16T19:36:18.438355+0000","last_peered":"2024-04-16T19:36:18.438355+0000","last_clean":"2024-04-16T19:36:18.438355+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:36:18.438355+0000","last_undegraded":"2024-04-16T19:36:18.438355+0000","last_fullsized":"2024-04-16T19:36:18.438355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.203492+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:35:51.203492+0000","last_peered":"2024-04-16T19:35:51.203492+0000","last_clean":"2024-04-16T19:35:51.203492+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:35:51.203492+0000","last_undegraded":"2024-04-16T19:35:51.203492+0000","last_fullsized":"2024-04-16T19:35:51.203492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'56","reported_seq":877,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211394+0000","last_change":"2024-04-16T17:48:34.333949+0000","last_active":"2024-04-16T19:36:06.211394+0000","last_peered":"2024-04-16T19:36:06.211394+0000","last_clean":"2024-04-16T19:36:06.211394+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:36:06.211394+0000","last_undegraded":"2024-04-16T19:36:06.211394+0000","last_fullsized":"2024-04-16T19:36:06.211394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.671347+0000","last_change":"2024-04-16T19:36:34.671347+0000","last_active":"2024-04-16T19:36:34.671347+0000","last_peered":"2024-04-16T19:36:34.671347+0000","last_clean":"2024-04-16T19:36:34.671347+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:36:34.671347+0000","last_undegraded":"2024-04-16T19:36:34.671347+0000","last_fullsized":"2024-04-16T19:36:34.671347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.671050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:41.097832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'20","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.631537+0000","last_change":"2024-04-16T19:36:45.631650+0000","last_active":"2024-04-16T19:36:45.631537+0000","last_peered":"2024-04-16T19:36:45.631537+0000","last_clean":"2024-04-16T19:36:45.631537+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:36:45.631537+0000","last_undegraded":"2024-04-16T19:36:45.631537+0000","last_fullsized":"2024-04-16T19:36:45.631537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.631366+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:37.106553+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.586110+0000","last_change":"2024-04-16T19:36:37.586170+0000","last_active":"2024-04-16T19:36:37.586110+0000","last_peered":"2024-04-16T19:36:37.586110+0000","last_clean":"2024-04-16T19:36:37.586110+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:36:37.586110+0000","last_undegraded":"2024-04-16T19:36:37.586110+0000","last_fullsized":"2024-04-16T19:36:37.586110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.585961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:02.979747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3625","reported_seq":7230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.651412+0000","last_change":"2024-04-16T19:36:17.651412+0000","last_active":"2024-04-16T19:36:17.651412+0000","last_peered":"2024-04-16T19:36:17.651412+0000","last_clean":"2024-04-16T19:36:17.651412+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:36:17.651412+0000","last_undegraded":"2024-04-16T19:36:17.651412+0000","last_fullsized":"2024-04-16T19:36:17.651412+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'3625","last_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_deep_scrub":"53'3625","last_deep_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.651328+0000","objects_scrubbed":30,"log_size":1825,"log_dups_size":1800,"ondisk_log_size":1825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:26.898814+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'18","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.465484+0000","last_change":"2024-04-16T19:36:17.465591+0000","last_active":"2024-04-16T19:36:17.465484+0000","last_peered":"2024-04-16T19:36:17.465484+0000","last_clean":"2024-04-16T19:36:17.465484+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:36:17.465484+0000","last_undegraded":"2024-04-16T19:36:17.465484+0000","last_fullsized":"2024-04-16T19:36:17.465484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.465360+0000","objects_scrubbed":9,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:23.892141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.615759+0000","last_change":"2024-04-16T19:36:38.615812+0000","last_active":"2024-04-16T19:36:38.615759+0000","last_peered":"2024-04-16T19:36:38.615759+0000","last_clean":"2024-04-16T19:36:38.615759+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:36:38.615759+0000","last_undegraded":"2024-04-16T19:36:38.615759+0000","last_fullsized":"2024-04-16T19:36:38.615759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.615619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:29.374824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212749+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:36:11.212749+0000","last_peered":"2024-04-16T19:36:11.212749+0000","last_clean":"2024-04-16T19:36:11.212749+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:36:11.212749+0000","last_undegraded":"2024-04-16T19:36:11.212749+0000","last_fullsized":"2024-04-16T19:36:11.212749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194212+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:34:41.194212+0000","last_peered":"2024-04-16T19:34:41.194212+0000","last_clean":"2024-04-16T19:34:41.194212+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:34:41.194212+0000","last_undegraded":"2024-04-16T19:34:41.194212+0000","last_fullsized":"2024-04-16T19:34:41.194212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"48'11","reported_seq":607,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528035+0000","last_change":"2024-04-16T17:47:33.300240+0000","last_active":"2024-04-16T19:36:17.528035+0000","last_peered":"2024-04-16T19:36:17.528035+0000","last_clean":"2024-04-16T19:36:17.528035+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:36:17.528035+0000","last_undegraded":"2024-04-16T19:36:17.528035+0000","last_fullsized":"2024-04-16T19:36:17.528035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'125","reported_seq":304,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.417448+0000","last_change":"2024-04-16T19:36:55.417529+0000","last_active":"2024-04-16T19:36:55.417448+0000","last_peered":"2024-04-16T19:36:55.417448+0000","last_clean":"2024-04-16T19:36:55.417448+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:36:55.417448+0000","last_undegraded":"2024-04-16T19:36:55.417448+0000","last_fullsized":"2024-04-16T19:36:55.417448+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.417311+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:25.523059+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'13","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.351749+0000","last_change":"2024-04-16T19:36:45.351811+0000","last_active":"2024-04-16T19:36:45.351749+0000","last_peered":"2024-04-16T19:36:45.351749+0000","last_clean":"2024-04-16T19:36:45.351749+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:36:45.351749+0000","last_undegraded":"2024-04-16T19:36:45.351749+0000","last_fullsized":"2024-04-16T19:36:45.351749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.351601+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:50.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209444+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:35:51.209444+0000","last_peered":"2024-04-16T19:35:51.209444+0000","last_clean":"2024-04-16T19:35:51.209444+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:35:51.209444+0000","last_undegraded":"2024-04-16T19:35:51.209444+0000","last_fullsized":"2024-04-16T19:35:51.209444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":737,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211512+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:36:06.211512+0000","last_peered":"2024-04-16T19:36:06.211512+0000","last_clean":"2024-04-16T19:36:06.211512+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:36:06.211512+0000","last_undegraded":"2024-04-16T19:36:06.211512+0000","last_fullsized":"2024-04-16T19:36:06.211512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.671404+0000","last_change":"2024-04-16T19:36:35.671455+0000","last_active":"2024-04-16T19:36:35.671404+0000","last_peered":"2024-04-16T19:36:35.671404+0000","last_clean":"2024-04-16T19:36:35.671404+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:36:35.671404+0000","last_undegraded":"2024-04-16T19:36:35.671404+0000","last_fullsized":"2024-04-16T19:36:35.671404+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.671258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:03.249930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'21","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.435794+0000","last_change":"2024-04-16T19:36:50.435867+0000","last_active":"2024-04-16T19:36:50.435794+0000","last_peered":"2024-04-16T19:36:50.435794+0000","last_clean":"2024-04-16T19:36:50.435794+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:36:50.435794+0000","last_undegraded":"2024-04-16T19:36:50.435794+0000","last_fullsized":"2024-04-16T19:36:50.435794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.435653+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:16.012260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.469677+0000","last_change":"2024-04-16T19:36:39.469677+0000","last_active":"2024-04-16T19:36:39.469677+0000","last_peered":"2024-04-16T19:36:39.469677+0000","last_clean":"2024-04-16T19:36:39.469677+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:36:39.469677+0000","last_undegraded":"2024-04-16T19:36:39.469677+0000","last_fullsized":"2024-04-16T19:36:39.469677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.469488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.661779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'408","reported_seq":768,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.522616+0000","last_change":"2024-04-16T19:36:31.522794+0000","last_active":"2024-04-16T19:36:31.522616+0000","last_peered":"2024-04-16T19:36:31.522616+0000","last_clean":"2024-04-16T19:36:31.522616+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:36:31.522616+0000","last_undegraded":"2024-04-16T19:36:31.522616+0000","last_fullsized":"2024-04-16T19:36:31.522616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'408","last_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_deep_scrub":"53'408","last_deep_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.522481+0000","objects_scrubbed":23,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:59.101153+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'12","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.541977+0000","last_change":"2024-04-16T19:36:17.542045+0000","last_active":"2024-04-16T19:36:17.541977+0000","last_peered":"2024-04-16T19:36:17.541977+0000","last_clean":"2024-04-16T19:36:17.541977+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:36:17.541977+0000","last_undegraded":"2024-04-16T19:36:17.541977+0000","last_fullsized":"2024-04-16T19:36:17.541977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.541913+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:24.087701+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.579713+0000","last_change":"2024-04-16T19:36:47.579767+0000","last_active":"2024-04-16T19:36:47.579713+0000","last_peered":"2024-04-16T19:36:47.579713+0000","last_clean":"2024-04-16T19:36:47.579713+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:36:47.579713+0000","last_undegraded":"2024-04-16T19:36:47.579713+0000","last_fullsized":"2024-04-16T19:36:47.579713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.579576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:56.945905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212828+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:36:11.212828+0000","last_peered":"2024-04-16T19:36:11.212828+0000","last_clean":"2024-04-16T19:36:11.212828+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:36:11.212828+0000","last_undegraded":"2024-04-16T19:36:11.212828+0000","last_fullsized":"2024-04-16T19:36:11.212828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.188529+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:34:41.188529+0000","last_peered":"2024-04-16T19:34:41.188529+0000","last_clean":"2024-04-16T19:34:41.188529+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:34:41.188529+0000","last_undegraded":"2024-04-16T19:34:41.188529+0000","last_fullsized":"2024-04-16T19:34:41.188529+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978536+0000","last_change":"2024-04-16T17:47:33.305749+0000","last_active":"2024-04-16T19:36:17.978536+0000","last_peered":"2024-04-16T19:36:17.978536+0000","last_clean":"2024-04-16T19:36:17.978536+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:36:17.978536+0000","last_undegraded":"2024-04-16T19:36:17.978536+0000","last_fullsized":"2024-04-16T19:36:17.978536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978433+0000","last_change":"2024-04-16T17:47:35.314832+0000","last_active":"2024-04-16T19:36:17.978433+0000","last_peered":"2024-04-16T19:36:17.978433+0000","last_clean":"2024-04-16T19:36:17.978433+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:36:17.978433+0000","last_undegraded":"2024-04-16T19:36:17.978433+0000","last_fullsized":"2024-04-16T19:36:17.978433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'13","reported_seq":1344,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.649901+0000","last_change":"2024-04-16T19:36:52.649962+0000","last_active":"2024-04-16T19:36:52.649901+0000","last_peered":"2024-04-16T19:36:52.649901+0000","last_clean":"2024-04-16T19:36:52.649901+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:36:52.649901+0000","last_undegraded":"2024-04-16T19:36:52.649901+0000","last_fullsized":"2024-04-16T19:36:52.649901+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.649763+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.928938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.209696+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:36:16.209696+0000","last_peered":"2024-04-16T19:36:16.209696+0000","last_clean":"2024-04-16T19:36:16.209696+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:36:16.209696+0000","last_undegraded":"2024-04-16T19:36:16.209696+0000","last_fullsized":"2024-04-16T19:36:16.209696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'164","reported_seq":1855,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980419+0000","last_change":"2024-04-16T17:48:34.332673+0000","last_active":"2024-04-16T19:36:17.980419+0000","last_peered":"2024-04-16T19:36:17.980419+0000","last_clean":"2024-04-16T19:36:17.980419+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:36:17.980419+0000","last_undegraded":"2024-04-16T19:36:17.980419+0000","last_fullsized":"2024-04-16T19:36:17.980419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.413351+0000","last_change":"2024-04-16T19:36:52.413405+0000","last_active":"2024-04-16T19:36:52.413351+0000","last_peered":"2024-04-16T19:36:52.413351+0000","last_clean":"2024-04-16T19:36:52.413351+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:36:52.413351+0000","last_undegraded":"2024-04-16T19:36:52.413351+0000","last_fullsized":"2024-04-16T19:36:52.413351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.413208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:42.476038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'13","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.645823+0000","last_change":"2024-04-16T19:36:38.645892+0000","last_active":"2024-04-16T19:36:38.645823+0000","last_peered":"2024-04-16T19:36:38.645823+0000","last_clean":"2024-04-16T19:36:38.645823+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:36:38.645823+0000","last_undegraded":"2024-04-16T19:36:38.645823+0000","last_fullsized":"2024-04-16T19:36:38.645823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.645696+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:29.924307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":158,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.913468+0000","last_change":"2024-04-16T17:47:59.200630+0000","last_active":"2024-04-16T19:36:18.913468+0000","last_peered":"2024-04-16T19:36:18.913468+0000","last_clean":"2024-04-16T19:36:18.913468+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:36:18.913468+0000","last_undegraded":"2024-04-16T19:36:18.913468+0000","last_fullsized":"2024-04-16T19:36:18.913468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1110","reported_seq":1775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.529609+0000","last_change":"2024-04-16T19:36:31.529728+0000","last_active":"2024-04-16T19:36:31.529609+0000","last_peered":"2024-04-16T19:36:31.529609+0000","last_clean":"2024-04-16T19:36:31.529609+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:36:31.529609+0000","last_undegraded":"2024-04-16T19:36:31.529609+0000","last_fullsized":"2024-04-16T19:36:31.529609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1110","last_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_deep_scrub":"53'1110","last_deep_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.529471+0000","objects_scrubbed":23,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:18.708104+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143914,"num_omap_keys":628,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"53'8","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.477590+0000","last_change":"2024-04-16T19:36:20.477590+0000","last_active":"2024-04-16T19:36:20.477590+0000","last_peered":"2024-04-16T19:36:20.477590+0000","last_clean":"2024-04-16T19:36:20.477590+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:36:20.477590+0000","last_undegraded":"2024-04-16T19:36:20.477590+0000","last_fullsized":"2024-04-16T19:36:20.477590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.477384+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:25.916610+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210353+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:35:46.210353+0000","last_peered":"2024-04-16T19:35:46.210353+0000","last_clean":"2024-04-16T19:35:46.210353+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:35:46.210353+0000","last_undegraded":"2024-04-16T19:35:46.210353+0000","last_fullsized":"2024-04-16T19:35:46.210353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206209+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:35:46.206209+0000","last_peered":"2024-04-16T19:35:46.206209+0000","last_clean":"2024-04-16T19:35:46.206209+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:35:46.206209+0000","last_undegraded":"2024-04-16T19:35:46.206209+0000","last_fullsized":"2024-04-16T19:35:46.206209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208452+0000","last_change":"2024-04-16T17:47:33.309390+0000","last_active":"2024-04-16T19:35:56.208452+0000","last_peered":"2024-04-16T19:35:56.208452+0000","last_clean":"2024-04-16T19:35:56.208452+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:35:56.208452+0000","last_undegraded":"2024-04-16T19:35:56.208452+0000","last_fullsized":"2024-04-16T19:35:56.208452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'200","reported_seq":420,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912551+0000","last_change":"2024-04-16T17:47:35.314414+0000","last_active":"2024-04-16T19:36:18.912551+0000","last_peered":"2024-04-16T19:36:18.912551+0000","last_clean":"2024-04-16T19:36:18.912551+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:36:18.912551+0000","last_undegraded":"2024-04-16T19:36:18.912551+0000","last_fullsized":"2024-04-16T19:36:18.912551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.650228+0000","last_change":"2024-04-16T19:36:49.650334+0000","last_active":"2024-04-16T19:36:49.650228+0000","last_peered":"2024-04-16T19:36:49.650228+0000","last_clean":"2024-04-16T19:36:49.650228+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:36:49.650228+0000","last_undegraded":"2024-04-16T19:36:49.650228+0000","last_fullsized":"2024-04-16T19:36:49.650228+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.650064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:03.271488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.443726+0000","last_change":"2024-04-16T19:36:42.443778+0000","last_active":"2024-04-16T19:36:42.443726+0000","last_peered":"2024-04-16T19:36:42.443726+0000","last_clean":"2024-04-16T19:36:42.443726+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:36:42.443726+0000","last_undegraded":"2024-04-16T19:36:42.443726+0000","last_fullsized":"2024-04-16T19:36:42.443726+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.443589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:33.202793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194292+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:36:16.194292+0000","last_peered":"2024-04-16T19:36:16.194292+0000","last_clean":"2024-04-16T19:36:16.194292+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:36:16.194292+0000","last_undegraded":"2024-04-16T19:36:16.194292+0000","last_fullsized":"2024-04-16T19:36:16.194292+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587906+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:36:16.587906+0000","last_peered":"2024-04-16T19:36:16.587906+0000","last_clean":"2024-04-16T19:36:16.587906+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:36:16.587906+0000","last_undegraded":"2024-04-16T19:36:16.587906+0000","last_fullsized":"2024-04-16T19:36:16.587906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.441613+0000","last_change":"2024-04-16T19:36:48.441666+0000","last_active":"2024-04-16T19:36:48.441613+0000","last_peered":"2024-04-16T19:36:48.441613+0000","last_clean":"2024-04-16T19:36:48.441613+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:36:48.441613+0000","last_undegraded":"2024-04-16T19:36:48.441613+0000","last_fullsized":"2024-04-16T19:36:48.441613+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.441481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:54.447789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'4","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.445595+0000","last_change":"2024-04-16T19:36:53.445663+0000","last_active":"2024-04-16T19:36:53.445595+0000","last_peered":"2024-04-16T19:36:53.445595+0000","last_clean":"2024-04-16T19:36:53.445595+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:36:53.445595+0000","last_undegraded":"2024-04-16T19:36:53.445595+0000","last_fullsized":"2024-04-16T19:36:53.445595+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.445453+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:27.787458+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.610902+0000","last_change":"2024-04-16T19:36:39.610922+0000","last_active":"2024-04-16T19:36:39.610902+0000","last_peered":"2024-04-16T19:36:39.610902+0000","last_clean":"2024-04-16T19:36:39.610902+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:36:39.610902+0000","last_undegraded":"2024-04-16T19:36:39.610902+0000","last_fullsized":"2024-04-16T19:36:39.610902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.610843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:16.867566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1195","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.442330+0000","last_change":"2024-04-16T19:36:32.442434+0000","last_active":"2024-04-16T19:36:32.442330+0000","last_peered":"2024-04-16T19:36:32.442330+0000","last_clean":"2024-04-16T19:36:32.442330+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:36:32.442330+0000","last_undegraded":"2024-04-16T19:36:32.442330+0000","last_fullsized":"2024-04-16T19:36:32.442330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1195","last_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_deep_scrub":"53'1195","last_deep_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.442194+0000","objects_scrubbed":22,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:37.854921+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215932,"num_omap_keys":942,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'17","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.474347+0000","last_change":"2024-04-16T19:36:21.474426+0000","last_active":"2024-04-16T19:36:21.474347+0000","last_peered":"2024-04-16T19:36:21.474347+0000","last_clean":"2024-04-16T19:36:21.474347+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:36:21.474347+0000","last_undegraded":"2024-04-16T19:36:21.474347+0000","last_fullsized":"2024-04-16T19:36:21.474347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'17","last_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_deep_scrub":"53'17","last_deep_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.474217+0000","objects_scrubbed":3,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:40.463538+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202090+0000","last_change":"2024-04-16T17:47:23.266202+0000","last_active":"2024-04-16T19:35:46.202090+0000","last_peered":"2024-04-16T19:35:46.202090+0000","last_clean":"2024-04-16T19:35:46.202090+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:35:46.202090+0000","last_undegraded":"2024-04-16T19:35:46.202090+0000","last_fullsized":"2024-04-16T19:35:46.202090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199383+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:35:21.199383+0000","last_peered":"2024-04-16T19:35:21.199383+0000","last_clean":"2024-04-16T19:35:21.199383+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:35:21.199383+0000","last_undegraded":"2024-04-16T19:35:21.199383+0000","last_fullsized":"2024-04-16T19:35:21.199383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"31'1","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189364+0000","last_change":"2024-04-16T17:47:33.302299+0000","last_active":"2024-04-16T19:35:56.189364+0000","last_peered":"2024-04-16T19:35:56.189364+0000","last_clean":"2024-04-16T19:35:56.189364+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:35:56.189364+0000","last_undegraded":"2024-04-16T19:35:56.189364+0000","last_fullsized":"2024-04-16T19:35:56.189364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'80","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.204985+0000","last_change":"2024-04-16T17:47:35.321698+0000","last_active":"2024-04-16T19:35:36.204985+0000","last_peered":"2024-04-16T19:35:36.204985+0000","last_clean":"2024-04-16T19:35:36.204985+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:35:36.204985+0000","last_undegraded":"2024-04-16T19:35:36.204985+0000","last_fullsized":"2024-04-16T19:35:36.204985+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.388357+0000","last_change":"2024-04-16T19:36:54.388412+0000","last_active":"2024-04-16T19:36:54.388357+0000","last_peered":"2024-04-16T19:36:54.388357+0000","last_clean":"2024-04-16T19:36:54.388357+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:36:54.388357+0000","last_undegraded":"2024-04-16T19:36:54.388357+0000","last_fullsized":"2024-04-16T19:36:54.388357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.388213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:28.441656+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.545722+0000","last_change":"2024-04-16T19:36:53.545774+0000","last_active":"2024-04-16T19:36:53.545722+0000","last_peered":"2024-04-16T19:36:53.545722+0000","last_clean":"2024-04-16T19:36:53.545722+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:36:53.545722+0000","last_undegraded":"2024-04-16T19:36:53.545722+0000","last_fullsized":"2024-04-16T19:36:53.545722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.545589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:08.257713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217900+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:36:16.217900+0000","last_peered":"2024-04-16T19:36:16.217900+0000","last_clean":"2024-04-16T19:36:16.217900+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:36:16.217900+0000","last_undegraded":"2024-04-16T19:36:16.217900+0000","last_fullsized":"2024-04-16T19:36:16.217900+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'70","reported_seq":902,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.177658+0000","last_change":"2024-04-16T17:48:34.345575+0000","last_active":"2024-04-16T19:35:06.177658+0000","last_peered":"2024-04-16T19:35:06.177658+0000","last_clean":"2024-04-16T19:35:06.177658+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:35:06.177658+0000","last_undegraded":"2024-04-16T19:35:06.177658+0000","last_fullsized":"2024-04-16T19:35:06.177658+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.686464+0000","last_change":"2024-04-16T19:36:49.686464+0000","last_active":"2024-04-16T19:36:49.686464+0000","last_peered":"2024-04-16T19:36:49.686464+0000","last_clean":"2024-04-16T19:36:49.686464+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:36:49.686464+0000","last_undegraded":"2024-04-16T19:36:49.686464+0000","last_fullsized":"2024-04-16T19:36:49.686464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.686271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.036352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980079+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:36:17.980079+0000","last_peered":"2024-04-16T19:36:17.980079+0000","last_clean":"2024-04-16T19:36:17.980079+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:36:17.980079+0000","last_undegraded":"2024-04-16T19:36:17.980079+0000","last_fullsized":"2024-04-16T19:36:17.980079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.684642+0000","last_change":"2024-04-16T19:36:32.684642+0000","last_active":"2024-04-16T19:36:32.684642+0000","last_peered":"2024-04-16T19:36:32.684642+0000","last_clean":"2024-04-16T19:36:32.684642+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:36:32.684642+0000","last_undegraded":"2024-04-16T19:36:32.684642+0000","last_fullsized":"2024-04-16T19:36:32.684642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.684452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:11.352292+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1071","reported_seq":6142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.466107+0000","last_change":"2024-04-16T19:36:31.466251+0000","last_active":"2024-04-16T19:36:31.466107+0000","last_peered":"2024-04-16T19:36:31.466107+0000","last_clean":"2024-04-16T19:36:31.466107+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:36:31.466107+0000","last_undegraded":"2024-04-16T19:36:31.466107+0000","last_fullsized":"2024-04-16T19:36:31.466107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1071","last_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_deep_scrub":"53'1071","last_deep_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.465959+0000","objects_scrubbed":22,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:36.878686+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71967,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'8","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.607489+0000","last_change":"2024-04-16T19:36:24.607587+0000","last_active":"2024-04-16T19:36:24.607489+0000","last_peered":"2024-04-16T19:36:24.607489+0000","last_clean":"2024-04-16T19:36:24.607489+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:36:24.607489+0000","last_undegraded":"2024-04-16T19:36:24.607489+0000","last_fullsized":"2024-04-16T19:36:24.607489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.607347+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:30.046572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212436+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:36:11.212436+0000","last_peered":"2024-04-16T19:36:11.212436+0000","last_clean":"2024-04-16T19:36:11.212436+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:36:11.212436+0000","last_undegraded":"2024-04-16T19:36:11.212436+0000","last_fullsized":"2024-04-16T19:36:11.212436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204505+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:35:21.204505+0000","last_peered":"2024-04-16T19:35:21.204505+0000","last_clean":"2024-04-16T19:35:21.204505+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:35:21.204505+0000","last_undegraded":"2024-04-16T19:35:21.204505+0000","last_fullsized":"2024-04-16T19:35:21.204505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.184692+0000","last_change":"2024-04-16T17:47:33.302725+0000","last_active":"2024-04-16T19:35:36.184692+0000","last_peered":"2024-04-16T19:35:36.184692+0000","last_clean":"2024-04-16T19:35:36.184692+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:35:36.184692+0000","last_undegraded":"2024-04-16T19:35:36.184692+0000","last_fullsized":"2024-04-16T19:35:36.184692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'152","reported_seq":348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.212875+0000","last_change":"2024-04-16T17:47:35.317742+0000","last_active":"2024-04-16T19:35:56.212875+0000","last_peered":"2024-04-16T19:35:56.212875+0000","last_clean":"2024-04-16T19:35:56.212875+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:35:56.212875+0000","last_undegraded":"2024-04-16T19:35:56.212875+0000","last_fullsized":"2024-04-16T19:35:56.212875+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'12","reported_seq":1339,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209943+0000","last_change":"2024-04-16T17:47:37.318010+0000","last_active":"2024-04-16T19:36:01.209943+0000","last_peered":"2024-04-16T19:36:01.209943+0000","last_clean":"2024-04-16T19:36:01.209943+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:36:01.209943+0000","last_undegraded":"2024-04-16T19:36:01.209943+0000","last_fullsized":"2024-04-16T19:36:01.209943+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.712924+0000","last_change":"2024-04-16T19:36:43.712975+0000","last_active":"2024-04-16T19:36:43.712924+0000","last_peered":"2024-04-16T19:36:43.712924+0000","last_clean":"2024-04-16T19:36:43.712924+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:36:43.712924+0000","last_undegraded":"2024-04-16T19:36:43.712924+0000","last_fullsized":"2024-04-16T19:36:43.712924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.712792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:43.360972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181074+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:35:21.181074+0000","last_peered":"2024-04-16T19:35:21.181074+0000","last_clean":"2024-04-16T19:35:21.181074+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:35:21.181074+0000","last_undegraded":"2024-04-16T19:35:21.181074+0000","last_fullsized":"2024-04-16T19:35:21.181074+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'100","reported_seq":1255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.688914+0000","last_change":"2024-04-16T19:36:47.688914+0000","last_active":"2024-04-16T19:36:47.688914+0000","last_peered":"2024-04-16T19:36:47.688914+0000","last_clean":"2024-04-16T19:36:47.688914+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:36:47.688914+0000","last_undegraded":"2024-04-16T19:36:47.688914+0000","last_fullsized":"2024-04-16T19:36:47.688914+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'100","last_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_deep_scrub":"53'100","last_deep_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.688717+0000","objects_scrubbed":26,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:21.999831+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5240,"num_omap_keys":22,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.472784+0000","last_change":"2024-04-16T19:36:55.472893+0000","last_active":"2024-04-16T19:36:55.472784+0000","last_peered":"2024-04-16T19:36:55.472784+0000","last_clean":"2024-04-16T19:36:55.472784+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:36:55.472784+0000","last_undegraded":"2024-04-16T19:36:55.472784+0000","last_fullsized":"2024-04-16T19:36:55.472784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.472614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:45.535445+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'20","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.637124+0000","last_change":"2024-04-16T19:36:41.637203+0000","last_active":"2024-04-16T19:36:41.637124+0000","last_peered":"2024-04-16T19:36:41.637124+0000","last_clean":"2024-04-16T19:36:41.637124+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:36:41.637124+0000","last_undegraded":"2024-04-16T19:36:41.637124+0000","last_fullsized":"2024-04-16T19:36:41.637124+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.636981+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.258518+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.453915+0000","last_change":"2024-04-16T19:36:39.453971+0000","last_active":"2024-04-16T19:36:39.453915+0000","last_peered":"2024-04-16T19:36:39.453915+0000","last_clean":"2024-04-16T19:36:39.453915+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:36:39.453915+0000","last_undegraded":"2024-04-16T19:36:39.453915+0000","last_fullsized":"2024-04-16T19:36:39.453915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.453770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:59.663137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'736","reported_seq":1478,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.630399+0000","last_change":"2024-04-16T19:36:18.630399+0000","last_active":"2024-04-16T19:36:18.630399+0000","last_peered":"2024-04-16T19:36:18.630399+0000","last_clean":"2024-04-16T19:36:18.630399+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:36:18.630399+0000","last_undegraded":"2024-04-16T19:36:18.630399+0000","last_fullsized":"2024-04-16T19:36:18.630399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'736","last_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_deep_scrub":"53'736","last_deep_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.630302+0000","objects_scrubbed":33,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:45.495590+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":75043,"num_omap_keys":332,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'18","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.577485+0000","last_change":"2024-04-16T19:36:26.577559+0000","last_active":"2024-04-16T19:36:26.577485+0000","last_peered":"2024-04-16T19:36:26.577485+0000","last_clean":"2024-04-16T19:36:26.577485+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:36:26.577485+0000","last_undegraded":"2024-04-16T19:36:26.577485+0000","last_fullsized":"2024-04-16T19:36:26.577485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.577356+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:41.709302+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201519+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:35:26.201519+0000","last_peered":"2024-04-16T19:35:26.201519+0000","last_clean":"2024-04-16T19:35:26.201519+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:35:26.201519+0000","last_undegraded":"2024-04-16T19:35:26.201519+0000","last_fullsized":"2024-04-16T19:35:26.201519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199263+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:35:21.199263+0000","last_peered":"2024-04-16T19:35:21.199263+0000","last_clean":"2024-04-16T19:35:21.199263+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:35:21.199263+0000","last_undegraded":"2024-04-16T19:35:21.199263+0000","last_fullsized":"2024-04-16T19:35:21.199263+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.527954+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:36:17.527954+0000","last_peered":"2024-04-16T19:36:17.527954+0000","last_clean":"2024-04-16T19:36:17.527954+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:36:17.527954+0000","last_undegraded":"2024-04-16T19:36:17.527954+0000","last_fullsized":"2024-04-16T19:36:17.527954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'88","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208721+0000","last_change":"2024-04-16T17:47:35.317589+0000","last_active":"2024-04-16T19:35:56.208721+0000","last_peered":"2024-04-16T19:35:56.208721+0000","last_clean":"2024-04-16T19:35:56.208721+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:35:56.208721+0000","last_undegraded":"2024-04-16T19:35:56.208721+0000","last_fullsized":"2024-04-16T19:35:56.208721+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.415324+0000","last_change":"2024-04-16T19:36:53.415380+0000","last_active":"2024-04-16T19:36:53.415324+0000","last_peered":"2024-04-16T19:36:53.415324+0000","last_clean":"2024-04-16T19:36:53.415324+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:36:53.415324+0000","last_undegraded":"2024-04-16T19:36:53.415324+0000","last_fullsized":"2024-04-16T19:36:53.415324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.415178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:50.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.527536+0000","last_change":"2024-04-16T19:36:33.527616+0000","last_active":"2024-04-16T19:36:33.527536+0000","last_peered":"2024-04-16T19:36:33.527536+0000","last_clean":"2024-04-16T19:36:33.527536+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:36:33.527536+0000","last_undegraded":"2024-04-16T19:36:33.527536+0000","last_fullsized":"2024-04-16T19:36:33.527536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.527307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:38.972728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201397+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:35:21.201397+0000","last_peered":"2024-04-16T19:35:21.201397+0000","last_clean":"2024-04-16T19:35:21.201397+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:35:21.201397+0000","last_undegraded":"2024-04-16T19:35:21.201397+0000","last_fullsized":"2024-04-16T19:35:21.201397+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'40","reported_seq":975,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587893+0000","last_change":"2024-04-16T17:48:34.333313+0000","last_active":"2024-04-16T19:36:16.587893+0000","last_peered":"2024-04-16T19:36:16.587893+0000","last_clean":"2024-04-16T19:36:16.587893+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:36:16.587893+0000","last_undegraded":"2024-04-16T19:36:16.587893+0000","last_fullsized":"2024-04-16T19:36:16.587893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":92,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.684338+0000","last_change":"2024-04-16T19:36:40.684390+0000","last_active":"2024-04-16T19:36:40.684338+0000","last_peered":"2024-04-16T19:36:40.684338+0000","last_clean":"2024-04-16T19:36:40.684338+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:36:40.684338+0000","last_undegraded":"2024-04-16T19:36:40.684338+0000","last_fullsized":"2024-04-16T19:36:40.684338+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:34.096243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'11","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.426664+0000","last_change":"2024-04-16T19:36:52.426789+0000","last_active":"2024-04-16T19:36:52.426664+0000","last_peered":"2024-04-16T19:36:52.426664+0000","last_clean":"2024-04-16T19:36:52.426664+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:36:52.426664+0000","last_undegraded":"2024-04-16T19:36:52.426664+0000","last_fullsized":"2024-04-16T19:36:52.426664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.426496+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:18.642190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'556","reported_seq":2298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.370105+0000","last_change":"2024-04-16T19:36:40.370175+0000","last_active":"2024-04-16T19:36:40.370105+0000","last_peered":"2024-04-16T19:36:40.370105+0000","last_clean":"2024-04-16T19:36:40.370105+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:36:40.370105+0000","last_undegraded":"2024-04-16T19:36:40.370105+0000","last_fullsized":"2024-04-16T19:36:40.370105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.369960+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:17.626683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'925","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.519148+0000","last_change":"2024-04-16T19:36:29.519272+0000","last_active":"2024-04-16T19:36:29.519148+0000","last_peered":"2024-04-16T19:36:29.519148+0000","last_clean":"2024-04-16T19:36:29.519148+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:36:29.519148+0000","last_undegraded":"2024-04-16T19:36:29.519148+0000","last_fullsized":"2024-04-16T19:36:29.519148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.519049+0000","objects_scrubbed":24,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:29.400044+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143864,"num_omap_keys":628,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'19","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.480753+0000","last_change":"2024-04-16T19:36:30.480855+0000","last_active":"2024-04-16T19:36:30.480753+0000","last_peered":"2024-04-16T19:36:30.480753+0000","last_clean":"2024-04-16T19:36:30.480753+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:36:30.480753+0000","last_undegraded":"2024-04-16T19:36:30.480753+0000","last_fullsized":"2024-04-16T19:36:30.480753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.480620+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:17.845024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197270+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:35:26.197270+0000","last_peered":"2024-04-16T19:35:26.197270+0000","last_clean":"2024-04-16T19:35:26.197270+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:35:26.197270+0000","last_undegraded":"2024-04-16T19:35:26.197270+0000","last_fullsized":"2024-04-16T19:35:26.197270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201194+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:35:21.201194+0000","last_peered":"2024-04-16T19:35:21.201194+0000","last_clean":"2024-04-16T19:35:21.201194+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:35:21.201194+0000","last_undegraded":"2024-04-16T19:35:21.201194+0000","last_fullsized":"2024-04-16T19:35:21.201194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":176,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.184791+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:35:36.184791+0000","last_peered":"2024-04-16T19:35:36.184791+0000","last_clean":"2024-04-16T19:35:36.184791+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:35:36.184791+0000","last_undegraded":"2024-04-16T19:35:36.184791+0000","last_fullsized":"2024-04-16T19:35:36.184791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'104","reported_seq":274,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.199943+0000","last_change":"2024-04-16T17:47:35.307146+0000","last_active":"2024-04-16T19:35:36.199943+0000","last_peered":"2024-04-16T19:35:36.199943+0000","last_clean":"2024-04-16T19:35:36.199943+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:35:36.199943+0000","last_undegraded":"2024-04-16T19:35:36.199943+0000","last_fullsized":"2024-04-16T19:35:36.199943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.214073+0000","last_change":"2024-04-16T17:47:37.324514+0000","last_active":"2024-04-16T19:36:16.214073+0000","last_peered":"2024-04-16T19:36:16.214073+0000","last_clean":"2024-04-16T19:36:16.214073+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:36:16.214073+0000","last_undegraded":"2024-04-16T19:36:16.214073+0000","last_fullsized":"2024-04-16T19:36:16.214073+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.457022+0000","last_change":"2024-04-16T19:36:30.457084+0000","last_active":"2024-04-16T19:36:30.457022+0000","last_peered":"2024-04-16T19:36:30.457022+0000","last_clean":"2024-04-16T19:36:30.457022+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:36:30.457022+0000","last_undegraded":"2024-04-16T19:36:30.457022+0000","last_fullsized":"2024-04-16T19:36:30.457022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.456877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:36.883658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200338+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:35:21.200338+0000","last_peered":"2024-04-16T19:35:21.200338+0000","last_clean":"2024-04-16T19:35:21.200338+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:35:21.200338+0000","last_undegraded":"2024-04-16T19:35:21.200338+0000","last_fullsized":"2024-04-16T19:35:21.200338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":884,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914956+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:36:18.914956+0000","last_peered":"2024-04-16T19:36:18.914956+0000","last_clean":"2024-04-16T19:36:18.914956+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:36:18.914956+0000","last_undegraded":"2024-04-16T19:36:18.914956+0000","last_fullsized":"2024-04-16T19:36:18.914956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914995+0000","last_change":"2024-04-16T17:48:27.308234+0000","last_active":"2024-04-16T19:36:18.914995+0000","last_peered":"2024-04-16T19:36:18.914995+0000","last_clean":"2024-04-16T19:36:18.914995+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:36:18.914995+0000","last_undegraded":"2024-04-16T19:36:18.914995+0000","last_fullsized":"2024-04-16T19:36:18.914995+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'142","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.445837+0000","last_change":"2024-04-16T19:36:51.445837+0000","last_active":"2024-04-16T19:36:51.445837+0000","last_peered":"2024-04-16T19:36:51.445837+0000","last_clean":"2024-04-16T19:36:51.445837+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:36:51.445837+0000","last_undegraded":"2024-04-16T19:36:51.445837+0000","last_fullsized":"2024-04-16T19:36:51.445837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'142","last_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_deep_scrub":"53'142","last_deep_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.445664+0000","objects_scrubbed":7,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:24.999267+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9535,"num_omap_keys":55,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.514976+0000","last_change":"2024-04-16T19:36:24.515035+0000","last_active":"2024-04-16T19:36:24.514976+0000","last_peered":"2024-04-16T19:36:24.514976+0000","last_clean":"2024-04-16T19:36:24.514976+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:36:24.514976+0000","last_undegraded":"2024-04-16T19:36:24.514976+0000","last_fullsized":"2024-04-16T19:36:24.514976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.514834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:04:02.534977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'702","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.544776+0000","last_change":"2024-04-16T19:36:22.544916+0000","last_active":"2024-04-16T19:36:22.544776+0000","last_peered":"2024-04-16T19:36:22.544776+0000","last_clean":"2024-04-16T19:36:22.544776+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:36:22.544776+0000","last_undegraded":"2024-04-16T19:36:22.544776+0000","last_fullsized":"2024-04-16T19:36:22.544776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'702","last_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_deep_scrub":"53'702","last_deep_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.544630+0000","objects_scrubbed":23,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:27.704961+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71922,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.499314+0000","last_change":"2024-04-16T19:36:24.499314+0000","last_active":"2024-04-16T19:36:24.499314+0000","last_peered":"2024-04-16T19:36:24.499314+0000","last_clean":"2024-04-16T19:36:24.499314+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:36:24.499314+0000","last_undegraded":"2024-04-16T19:36:24.499314+0000","last_fullsized":"2024-04-16T19:36:24.499314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.498983+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:48.052702+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.205683+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:35:26.205683+0000","last_peered":"2024-04-16T19:35:26.205683+0000","last_clean":"2024-04-16T19:35:26.205683+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:35:26.205683+0000","last_undegraded":"2024-04-16T19:35:26.205683+0000","last_fullsized":"2024-04-16T19:35:26.205683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217992+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:36:16.217992+0000","last_peered":"2024-04-16T19:36:16.217992+0000","last_clean":"2024-04-16T19:36:16.217992+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:36:16.217992+0000","last_undegraded":"2024-04-16T19:36:16.217992+0000","last_fullsized":"2024-04-16T19:36:16.217992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"48'13","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.205087+0000","last_change":"2024-04-16T17:47:33.309159+0000","last_active":"2024-04-16T19:35:36.205087+0000","last_peered":"2024-04-16T19:35:36.205087+0000","last_clean":"2024-04-16T19:35:36.205087+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:35:36.205087+0000","last_undegraded":"2024-04-16T19:35:36.205087+0000","last_fullsized":"2024-04-16T19:35:36.205087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'152","reported_seq":346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.192239+0000","last_change":"2024-04-16T17:47:35.305022+0000","last_active":"2024-04-16T19:36:06.192239+0000","last_peered":"2024-04-16T19:36:06.192239+0000","last_clean":"2024-04-16T19:36:06.192239+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:36:06.192239+0000","last_undegraded":"2024-04-16T19:36:06.192239+0000","last_fullsized":"2024-04-16T19:36:06.192239+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'13","reported_seq":1346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.458428+0000","last_change":"2024-04-16T19:36:51.458492+0000","last_active":"2024-04-16T19:36:51.458428+0000","last_peered":"2024-04-16T19:36:51.458428+0000","last_clean":"2024-04-16T19:36:51.458428+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:36:51.458428+0000","last_undegraded":"2024-04-16T19:36:51.458428+0000","last_fullsized":"2024-04-16T19:36:51.458428+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.458284+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:46:06.590229+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.582250+0000","last_change":"2024-04-16T19:36:36.582358+0000","last_active":"2024-04-16T19:36:36.582250+0000","last_peered":"2024-04-16T19:36:36.582250+0000","last_clean":"2024-04-16T19:36:36.582250+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:36:36.582250+0000","last_undegraded":"2024-04-16T19:36:36.582250+0000","last_fullsized":"2024-04-16T19:36:36.582250+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.582059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:29.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980805+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:36:17.980805+0000","last_peered":"2024-04-16T19:36:17.980805+0000","last_clean":"2024-04-16T19:36:17.980805+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:36:17.980805+0000","last_undegraded":"2024-04-16T19:36:17.980805+0000","last_fullsized":"2024-04-16T19:36:17.980805+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'86","reported_seq":1119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980523+0000","last_change":"2024-04-16T17:48:34.326754+0000","last_active":"2024-04-16T19:36:17.980523+0000","last_peered":"2024-04-16T19:36:17.980523+0000","last_clean":"2024-04-16T19:36:17.980523+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:36:17.980523+0000","last_undegraded":"2024-04-16T19:36:17.980523+0000","last_fullsized":"2024-04-16T19:36:17.980523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.427150+0000","last_change":"2024-04-16T19:36:49.427150+0000","last_active":"2024-04-16T19:36:49.427150+0000","last_peered":"2024-04-16T19:36:49.427150+0000","last_clean":"2024-04-16T19:36:49.427150+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:36:49.427150+0000","last_undegraded":"2024-04-16T19:36:49.427150+0000","last_fullsized":"2024-04-16T19:36:49.427150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.426874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:15.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"53'2","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.645599+0000","last_change":"2024-04-16T19:36:42.645707+0000","last_active":"2024-04-16T19:36:42.645599+0000","last_peered":"2024-04-16T19:36:42.645599+0000","last_clean":"2024-04-16T19:36:42.645599+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:36:42.645599+0000","last_undegraded":"2024-04-16T19:36:42.645599+0000","last_fullsized":"2024-04-16T19:36:42.645599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.645438+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:22.681965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.565703+0000","last_change":"2024-04-16T19:36:24.565807+0000","last_active":"2024-04-16T19:36:24.565703+0000","last_peered":"2024-04-16T19:36:24.565703+0000","last_clean":"2024-04-16T19:36:24.565703+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:36:24.565703+0000","last_undegraded":"2024-04-16T19:36:24.565703+0000","last_fullsized":"2024-04-16T19:36:24.565703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.565541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:30.252893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1508","reported_seq":5513,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.585799+0000","last_change":"2024-04-16T19:36:27.585867+0000","last_active":"2024-04-16T19:36:27.585799+0000","last_peered":"2024-04-16T19:36:27.585799+0000","last_clean":"2024-04-16T19:36:27.585799+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:36:27.585799+0000","last_undegraded":"2024-04-16T19:36:27.585799+0000","last_fullsized":"2024-04-16T19:36:27.585799+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1508","last_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_deep_scrub":"53'1508","last_deep_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.585661+0000","objects_scrubbed":25,"log_size":1508,"log_dups_size":0,"ondisk_log_size":1508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:18.587606+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250399,"num_omap_keys":1045,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'14","reported_seq":240,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.424045+0000","last_change":"2024-04-16T19:36:27.424219+0000","last_active":"2024-04-16T19:36:27.424045+0000","last_peered":"2024-04-16T19:36:27.424045+0000","last_clean":"2024-04-16T19:36:27.424045+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:36:27.424045+0000","last_undegraded":"2024-04-16T19:36:27.424045+0000","last_fullsized":"2024-04-16T19:36:27.424045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.423818+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:57.529566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197378+0000","last_change":"2024-04-16T17:47:23.266018+0000","last_active":"2024-04-16T19:35:26.197378+0000","last_peered":"2024-04-16T19:35:26.197378+0000","last_clean":"2024-04-16T19:35:26.197378+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:35:26.197378+0000","last_undegraded":"2024-04-16T19:35:26.197378+0000","last_fullsized":"2024-04-16T19:35:26.197378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200918+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:35:41.200918+0000","last_peered":"2024-04-16T19:35:41.200918+0000","last_clean":"2024-04-16T19:35:41.200918+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:35:41.200918+0000","last_undegraded":"2024-04-16T19:35:41.200918+0000","last_fullsized":"2024-04-16T19:35:41.200918+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210154+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:36:01.210154+0000","last_peered":"2024-04-16T19:36:01.210154+0000","last_clean":"2024-04-16T19:36:01.210154+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:36:01.210154+0000","last_undegraded":"2024-04-16T19:36:01.210154+0000","last_fullsized":"2024-04-16T19:36:01.210154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'120","reported_seq":293,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.205111+0000","last_change":"2024-04-16T17:47:35.319180+0000","last_active":"2024-04-16T19:35:36.205111+0000","last_peered":"2024-04-16T19:35:36.205111+0000","last_clean":"2024-04-16T19:35:36.205111+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:35:36.205111+0000","last_undegraded":"2024-04-16T19:35:36.205111+0000","last_fullsized":"2024-04-16T19:35:36.205111+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'13","reported_seq":1346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.636594+0000","last_change":"2024-04-16T19:36:55.636714+0000","last_active":"2024-04-16T19:36:55.636594+0000","last_peered":"2024-04-16T19:36:55.636594+0000","last_clean":"2024-04-16T19:36:55.636594+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:36:55.636594+0000","last_undegraded":"2024-04-16T19:36:55.636594+0000","last_fullsized":"2024-04-16T19:36:55.636594+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.636425+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:56:05.862157+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.567557+0000","last_change":"2024-04-16T19:36:51.567609+0000","last_active":"2024-04-16T19:36:51.567557+0000","last_peered":"2024-04-16T19:36:51.567557+0000","last_clean":"2024-04-16T19:36:51.567557+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:36:51.567557+0000","last_undegraded":"2024-04-16T19:36:51.567557+0000","last_fullsized":"2024-04-16T19:36:51.567557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.567419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:32.289248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215663+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:36:16.215663+0000","last_peered":"2024-04-16T19:36:16.215663+0000","last_clean":"2024-04-16T19:36:16.215663+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:36:16.215663+0000","last_undegraded":"2024-04-16T19:36:16.215663+0000","last_fullsized":"2024-04-16T19:36:16.215663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'90","reported_seq":978,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201673+0000","last_change":"2024-04-16T17:48:34.333850+0000","last_active":"2024-04-16T19:35:26.201673+0000","last_peered":"2024-04-16T19:35:26.201673+0000","last_clean":"2024-04-16T19:35:26.201673+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:35:26.201673+0000","last_undegraded":"2024-04-16T19:35:26.201673+0000","last_fullsized":"2024-04-16T19:35:26.201673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.389594+0000","last_change":"2024-04-16T19:36:46.389594+0000","last_active":"2024-04-16T19:36:46.389594+0000","last_peered":"2024-04-16T19:36:46.389594+0000","last_clean":"2024-04-16T19:36:46.389594+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:36:46.389594+0000","last_undegraded":"2024-04-16T19:36:46.389594+0000","last_fullsized":"2024-04-16T19:36:46.389594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.389402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:49.372119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'8","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.681828+0000","last_change":"2024-04-16T19:36:40.681959+0000","last_active":"2024-04-16T19:36:40.681828+0000","last_peered":"2024-04-16T19:36:40.681828+0000","last_clean":"2024-04-16T19:36:40.681828+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:36:40.681828+0000","last_undegraded":"2024-04-16T19:36:40.681828+0000","last_fullsized":"2024-04-16T19:36:40.681828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.681665+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:15.914709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'556","reported_seq":1952,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.630262+0000","last_change":"2024-04-16T19:36:46.630326+0000","last_active":"2024-04-16T19:36:46.630262+0000","last_peered":"2024-04-16T19:36:46.630262+0000","last_clean":"2024-04-16T19:36:46.630262+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:36:46.630262+0000","last_undegraded":"2024-04-16T19:36:46.630262+0000","last_fullsized":"2024-04-16T19:36:46.630262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.630122+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:54.931710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'676","reported_seq":995,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635509+0000","last_change":"2024-04-16T19:36:20.635509+0000","last_active":"2024-04-16T19:36:20.635509+0000","last_peered":"2024-04-16T19:36:20.635509+0000","last_clean":"2024-04-16T19:36:20.635509+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:36:20.635509+0000","last_undegraded":"2024-04-16T19:36:20.635509+0000","last_fullsized":"2024-04-16T19:36:20.635509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'676","last_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_deep_scrub":"53'676","last_deep_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635277+0000","objects_scrubbed":25,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:10:55.447676+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71899,"num_omap_keys":314,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'20","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.500999+0000","last_change":"2024-04-16T19:36:18.501046+0000","last_active":"2024-04-16T19:36:18.500999+0000","last_peered":"2024-04-16T19:36:18.500999+0000","last_clean":"2024-04-16T19:36:18.500999+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:36:18.500999+0000","last_undegraded":"2024-04-16T19:36:18.500999+0000","last_fullsized":"2024-04-16T19:36:18.500999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.500951+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:17.147393+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210872+0000","last_change":"2024-04-16T17:47:23.261436+0000","last_active":"2024-04-16T19:35:56.210872+0000","last_peered":"2024-04-16T19:35:56.210872+0000","last_clean":"2024-04-16T19:35:56.210872+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:35:56.210872+0000","last_undegraded":"2024-04-16T19:35:56.210872+0000","last_fullsized":"2024-04-16T19:35:56.210872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.185743+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:35:41.185743+0000","last_peered":"2024-04-16T19:35:41.185743+0000","last_clean":"2024-04-16T19:35:41.185743+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:35:41.185743+0000","last_undegraded":"2024-04-16T19:35:41.185743+0000","last_fullsized":"2024-04-16T19:35:41.185743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.208172+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:35:36.208172+0000","last_peered":"2024-04-16T19:35:36.208172+0000","last_clean":"2024-04-16T19:35:36.208172+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:35:36.208172+0000","last_undegraded":"2024-04-16T19:35:36.208172+0000","last_fullsized":"2024-04-16T19:35:36.208172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'176","reported_seq":382,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.203507+0000","last_change":"2024-04-16T17:47:35.317949+0000","last_active":"2024-04-16T19:35:36.203507+0000","last_peered":"2024-04-16T19:35:36.203507+0000","last_clean":"2024-04-16T19:35:36.203507+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:35:36.203507+0000","last_undegraded":"2024-04-16T19:35:36.203507+0000","last_fullsized":"2024-04-16T19:35:36.203507+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209341+0000","last_change":"2024-04-16T17:47:37.332636+0000","last_active":"2024-04-16T19:35:41.209341+0000","last_peered":"2024-04-16T19:35:41.209341+0000","last_clean":"2024-04-16T19:35:41.209341+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:35:41.209341+0000","last_undegraded":"2024-04-16T19:35:41.209341+0000","last_fullsized":"2024-04-16T19:35:41.209341+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.569888+0000","last_change":"2024-04-16T19:36:34.569940+0000","last_active":"2024-04-16T19:36:34.569888+0000","last_peered":"2024-04-16T19:36:34.569888+0000","last_clean":"2024-04-16T19:36:34.569888+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:36:34.569888+0000","last_undegraded":"2024-04-16T19:36:34.569888+0000","last_fullsized":"2024-04-16T19:36:34.569888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.569751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:40.008977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528987+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:36:17.528987+0000","last_peered":"2024-04-16T19:36:17.528987+0000","last_clean":"2024-04-16T19:36:17.528987+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:36:17.528987+0000","last_undegraded":"2024-04-16T19:36:17.528987+0000","last_fullsized":"2024-04-16T19:36:17.528987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'118","reported_seq":1503,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.190007+0000","last_change":"2024-04-16T17:48:34.329015+0000","last_active":"2024-04-16T19:34:56.190007+0000","last_peered":"2024-04-16T19:34:56.190007+0000","last_clean":"2024-04-16T19:34:56.190007+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:34:56.190007+0000","last_undegraded":"2024-04-16T19:34:56.190007+0000","last_fullsized":"2024-04-16T19:34:56.190007+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.606337+0000","last_change":"2024-04-16T19:36:37.606393+0000","last_active":"2024-04-16T19:36:37.606337+0000","last_peered":"2024-04-16T19:36:37.606337+0000","last_clean":"2024-04-16T19:36:37.606337+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:36:37.606337+0000","last_undegraded":"2024-04-16T19:36:37.606337+0000","last_fullsized":"2024-04-16T19:36:37.606337+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.606203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:48.287865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'11","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.682939+0000","last_change":"2024-04-16T19:36:43.683003+0000","last_active":"2024-04-16T19:36:43.682939+0000","last_peered":"2024-04-16T19:36:43.682939+0000","last_clean":"2024-04-16T19:36:43.682939+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:36:43.682939+0000","last_undegraded":"2024-04-16T19:36:43.682939+0000","last_fullsized":"2024-04-16T19:36:43.682939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.682806+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:30.180345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.630682+0000","last_change":"2024-04-16T19:36:48.630682+0000","last_active":"2024-04-16T19:36:48.630682+0000","last_peered":"2024-04-16T19:36:48.630682+0000","last_clean":"2024-04-16T19:36:48.630682+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:36:48.630682+0000","last_undegraded":"2024-04-16T19:36:48.630682+0000","last_fullsized":"2024-04-16T19:36:48.630682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.630423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:08.839790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'804","reported_seq":43172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.600535+0000","last_change":"2024-04-16T19:36:25.600624+0000","last_active":"2024-04-16T19:36:25.600535+0000","last_peered":"2024-04-16T19:36:25.600535+0000","last_clean":"2024-04-16T19:36:25.600535+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:36:25.600535+0000","last_undegraded":"2024-04-16T19:36:25.600535+0000","last_fullsized":"2024-04-16T19:36:25.600535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'804","last_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_deep_scrub":"53'804","last_deep_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.600462+0000","objects_scrubbed":21,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:05.088020+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71960,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'12","reported_seq":221,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.489782+0000","last_change":"2024-04-16T19:36:49.489878+0000","last_active":"2024-04-16T19:36:49.489782+0000","last_peered":"2024-04-16T19:36:49.489782+0000","last_clean":"2024-04-16T19:36:49.489782+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:36:49.489782+0000","last_undegraded":"2024-04-16T19:36:49.489782+0000","last_fullsized":"2024-04-16T19:36:49.489782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.489640+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:54.928866+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978084+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:36:17.978084+0000","last_peered":"2024-04-16T19:36:17.978084+0000","last_clean":"2024-04-16T19:36:17.978084+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:36:17.978084+0000","last_undegraded":"2024-04-16T19:36:17.978084+0000","last_fullsized":"2024-04-16T19:36:17.978084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.202904+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:35:16.202904+0000","last_peered":"2024-04-16T19:35:16.202904+0000","last_clean":"2024-04-16T19:35:16.202904+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:35:16.202904+0000","last_undegraded":"2024-04-16T19:35:16.202904+0000","last_fullsized":"2024-04-16T19:35:16.202904+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206209+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:35:41.206209+0000","last_peered":"2024-04-16T19:35:41.206209+0000","last_clean":"2024-04-16T19:35:41.206209+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:35:41.206209+0000","last_undegraded":"2024-04-16T19:35:41.206209+0000","last_fullsized":"2024-04-16T19:35:41.206209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204821+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:35:56.204821+0000","last_peered":"2024-04-16T19:35:56.204821+0000","last_clean":"2024-04-16T19:35:56.204821+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:35:56.204821+0000","last_undegraded":"2024-04-16T19:35:56.204821+0000","last_fullsized":"2024-04-16T19:35:56.204821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'72","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208814+0000","last_change":"2024-04-16T17:47:35.310088+0000","last_active":"2024-04-16T19:35:56.208814+0000","last_peered":"2024-04-16T19:35:56.208814+0000","last_clean":"2024-04-16T19:35:56.208814+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:35:56.208814+0000","last_undegraded":"2024-04-16T19:35:56.208814+0000","last_fullsized":"2024-04-16T19:35:56.208814+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210279+0000","last_change":"2024-04-16T17:47:37.327822+0000","last_active":"2024-04-16T19:36:01.210279+0000","last_peered":"2024-04-16T19:36:01.210279+0000","last_clean":"2024-04-16T19:36:01.210279+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:36:01.210279+0000","last_undegraded":"2024-04-16T19:36:01.210279+0000","last_fullsized":"2024-04-16T19:36:01.210279+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.477027+0000","last_change":"2024-04-16T19:36:40.477078+0000","last_active":"2024-04-16T19:36:40.477027+0000","last_peered":"2024-04-16T19:36:40.477027+0000","last_clean":"2024-04-16T19:36:40.477027+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:36:40.477027+0000","last_undegraded":"2024-04-16T19:36:40.477027+0000","last_fullsized":"2024-04-16T19:36:40.477027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.476892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:55.189016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980695+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:36:17.980695+0000","last_peered":"2024-04-16T19:36:17.980695+0000","last_clean":"2024-04-16T19:36:17.980695+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:36:17.980695+0000","last_undegraded":"2024-04-16T19:36:17.980695+0000","last_fullsized":"2024-04-16T19:36:17.980695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'100","reported_seq":1747,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.439909+0000","last_change":"2024-04-16T17:48:34.333902+0000","last_active":"2024-04-16T19:36:18.439909+0000","last_peered":"2024-04-16T19:36:18.439909+0000","last_clean":"2024-04-16T19:36:18.439909+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:36:18.439909+0000","last_undegraded":"2024-04-16T19:36:18.439909+0000","last_fullsized":"2024-04-16T19:36:18.439909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.670649+0000","last_change":"2024-04-16T19:36:37.670700+0000","last_active":"2024-04-16T19:36:37.670649+0000","last_peered":"2024-04-16T19:36:37.670649+0000","last_clean":"2024-04-16T19:36:37.670649+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:36:37.670649+0000","last_undegraded":"2024-04-16T19:36:37.670649+0000","last_fullsized":"2024-04-16T19:36:37.670649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.670504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:43.115925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'8","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210112+0000","last_change":"2024-04-16T17:48:01.261918+0000","last_active":"2024-04-16T19:36:01.210112+0000","last_peered":"2024-04-16T19:36:01.210112+0000","last_clean":"2024-04-16T19:36:01.210112+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:36:01.210112+0000","last_undegraded":"2024-04-16T19:36:01.210112+0000","last_fullsized":"2024-04-16T19:36:01.210112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.465640+0000","last_change":"2024-04-16T19:36:41.465695+0000","last_active":"2024-04-16T19:36:41.465640+0000","last_peered":"2024-04-16T19:36:41.465640+0000","last_clean":"2024-04-16T19:36:41.465640+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:36:41.465640+0000","last_undegraded":"2024-04-16T19:36:41.465640+0000","last_fullsized":"2024-04-16T19:36:41.465640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.465493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:42:13.407368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'925","reported_seq":3830,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.639219+0000","last_change":"2024-04-16T19:36:25.639219+0000","last_active":"2024-04-16T19:36:25.639219+0000","last_peered":"2024-04-16T19:36:25.639219+0000","last_clean":"2024-04-16T19:36:25.639219+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:36:25.639219+0000","last_undegraded":"2024-04-16T19:36:25.639219+0000","last_fullsized":"2024-04-16T19:36:25.639219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.639013+0000","objects_scrubbed":28,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:11.472224+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89384,"num_omap_keys":416,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'36","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.510632+0000","last_change":"2024-04-16T19:36:24.510731+0000","last_active":"2024-04-16T19:36:24.510632+0000","last_peered":"2024-04-16T19:36:24.510632+0000","last_clean":"2024-04-16T19:36:24.510632+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:36:24.510632+0000","last_undegraded":"2024-04-16T19:36:24.510632+0000","last_fullsized":"2024-04-16T19:36:24.510632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'36","last_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_deep_scrub":"53'36","last_deep_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.510495+0000","objects_scrubbed":7,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:21.114823+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207927+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:35:46.207927+0000","last_peered":"2024-04-16T19:35:46.207927+0000","last_clean":"2024-04-16T19:35:46.207927+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:35:46.207927+0000","last_undegraded":"2024-04-16T19:35:46.207927+0000","last_fullsized":"2024-04-16T19:35:46.207927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192365+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:34:56.192365+0000","last_peered":"2024-04-16T19:34:56.192365+0000","last_clean":"2024-04-16T19:34:56.192365+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:34:56.192365+0000","last_undegraded":"2024-04-16T19:34:56.192365+0000","last_fullsized":"2024-04-16T19:34:56.192365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200997+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:35:41.200997+0000","last_peered":"2024-04-16T19:35:41.200997+0000","last_clean":"2024-04-16T19:35:41.200997+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:35:41.200997+0000","last_undegraded":"2024-04-16T19:35:41.200997+0000","last_fullsized":"2024-04-16T19:35:41.200997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586344+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:36:16.586344+0000","last_peered":"2024-04-16T19:36:16.586344+0000","last_clean":"2024-04-16T19:36:16.586344+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:36:16.586344+0000","last_undegraded":"2024-04-16T19:36:16.586344+0000","last_fullsized":"2024-04-16T19:36:16.586344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'144","reported_seq":331,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210819+0000","last_change":"2024-04-16T17:47:35.314035+0000","last_active":"2024-04-16T19:35:56.210819+0000","last_peered":"2024-04-16T19:35:56.210819+0000","last_clean":"2024-04-16T19:35:56.210819+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:35:56.210819+0000","last_undegraded":"2024-04-16T19:35:56.210819+0000","last_fullsized":"2024-04-16T19:35:56.210819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.356065+0000","last_change":"2024-04-16T19:36:43.356174+0000","last_active":"2024-04-16T19:36:43.356065+0000","last_peered":"2024-04-16T19:36:43.356065+0000","last_clean":"2024-04-16T19:36:43.356065+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:36:43.356065+0000","last_undegraded":"2024-04-16T19:36:43.356065+0000","last_fullsized":"2024-04-16T19:36:43.356065+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.355868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:33.637719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.430570+0000","last_change":"2024-04-16T19:36:44.430621+0000","last_active":"2024-04-16T19:36:44.430570+0000","last_peered":"2024-04-16T19:36:44.430570+0000","last_clean":"2024-04-16T19:36:44.430570+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:36:44.430570+0000","last_undegraded":"2024-04-16T19:36:44.430570+0000","last_fullsized":"2024-04-16T19:36:44.430570+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.430438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:03.942655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915237+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:36:18.915237+0000","last_peered":"2024-04-16T19:36:18.915237+0000","last_clean":"2024-04-16T19:36:18.915237+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:36:18.915237+0000","last_undegraded":"2024-04-16T19:36:18.915237+0000","last_fullsized":"2024-04-16T19:36:18.915237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914927+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:36:18.914927+0000","last_peered":"2024-04-16T19:36:18.914927+0000","last_clean":"2024-04-16T19:36:18.914927+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:36:18.914927+0000","last_undegraded":"2024-04-16T19:36:18.914927+0000","last_fullsized":"2024-04-16T19:36:18.914927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":81,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914754+0000","last_change":"2024-04-16T17:48:27.307048+0000","last_active":"2024-04-16T19:36:18.914754+0000","last_peered":"2024-04-16T19:36:18.914754+0000","last_clean":"2024-04-16T19:36:18.914754+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:36:18.914754+0000","last_undegraded":"2024-04-16T19:36:18.914754+0000","last_fullsized":"2024-04-16T19:36:18.914754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'42","reported_seq":213,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.456089+0000","last_change":"2024-04-16T19:36:54.456164+0000","last_active":"2024-04-16T19:36:54.456089+0000","last_peered":"2024-04-16T19:36:54.456089+0000","last_clean":"2024-04-16T19:36:54.456089+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:36:54.456089+0000","last_undegraded":"2024-04-16T19:36:54.456089+0000","last_fullsized":"2024-04-16T19:36:54.456089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'42","last_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_deep_scrub":"53'42","last_deep_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.455949+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:53.521634+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.526931+0000","last_change":"2024-04-16T19:36:34.526983+0000","last_active":"2024-04-16T19:36:34.526931+0000","last_peered":"2024-04-16T19:36:34.526931+0000","last_clean":"2024-04-16T19:36:34.526931+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:36:34.526931+0000","last_undegraded":"2024-04-16T19:36:34.526931+0000","last_fullsized":"2024-04-16T19:36:34.526931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.526798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:20.876919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'282","reported_seq":668,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.418934+0000","last_change":"2024-04-16T19:36:30.419051+0000","last_active":"2024-04-16T19:36:30.418934+0000","last_peered":"2024-04-16T19:36:30.418934+0000","last_clean":"2024-04-16T19:36:30.418934+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:36:30.418934+0000","last_undegraded":"2024-04-16T19:36:30.418934+0000","last_fullsized":"2024-04-16T19:36:30.418934+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'282","last_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_deep_scrub":"53'282","last_deep_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.418793+0000","objects_scrubbed":18,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:10.455320+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'16","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.403701+0000","last_change":"2024-04-16T19:36:29.403701+0000","last_active":"2024-04-16T19:36:29.403701+0000","last_peered":"2024-04-16T19:36:29.403701+0000","last_clean":"2024-04-16T19:36:29.403701+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:36:29.403701+0000","last_undegraded":"2024-04-16T19:36:29.403701+0000","last_fullsized":"2024-04-16T19:36:29.403701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.403484+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:20.878671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.208067+0000","last_change":"2024-04-16T17:47:23.265711+0000","last_active":"2024-04-16T19:35:46.208067+0000","last_peered":"2024-04-16T19:35:46.208067+0000","last_clean":"2024-04-16T19:35:46.208067+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:35:46.208067+0000","last_undegraded":"2024-04-16T19:35:46.208067+0000","last_fullsized":"2024-04-16T19:35:46.208067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189694+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:34:56.189694+0000","last_peered":"2024-04-16T19:34:56.189694+0000","last_clean":"2024-04-16T19:34:56.189694+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:34:56.189694+0000","last_undegraded":"2024-04-16T19:34:56.189694+0000","last_fullsized":"2024-04-16T19:34:56.189694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209117+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:35:51.209117+0000","last_peered":"2024-04-16T19:35:51.209117+0000","last_clean":"2024-04-16T19:35:51.209117+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:35:51.209117+0000","last_undegraded":"2024-04-16T19:35:51.209117+0000","last_fullsized":"2024-04-16T19:35:51.209117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"48'4","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189709+0000","last_change":"2024-04-16T17:47:33.301081+0000","last_active":"2024-04-16T19:35:56.189709+0000","last_peered":"2024-04-16T19:35:56.189709+0000","last_clean":"2024-04-16T19:35:56.189709+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:35:56.189709+0000","last_undegraded":"2024-04-16T19:35:56.189709+0000","last_fullsized":"2024-04-16T19:35:56.189709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'151","reported_seq":1557,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912876+0000","last_change":"2024-04-16T17:47:35.316795+0000","last_active":"2024-04-16T19:36:18.912876+0000","last_peered":"2024-04-16T19:36:18.912876+0000","last_clean":"2024-04-16T19:36:18.912876+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:36:18.912876+0000","last_undegraded":"2024-04-16T19:36:18.912876+0000","last_fullsized":"2024-04-16T19:36:18.912876+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210142+0000","last_change":"2024-04-16T17:47:37.326201+0000","last_active":"2024-04-16T19:36:01.210142+0000","last_peered":"2024-04-16T19:36:01.210142+0000","last_clean":"2024-04-16T19:36:01.210142+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:36:01.210142+0000","last_undegraded":"2024-04-16T19:36:01.210142+0000","last_fullsized":"2024-04-16T19:36:01.210142+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.593720+0000","last_change":"2024-04-16T19:36:40.593776+0000","last_active":"2024-04-16T19:36:40.593720+0000","last_peered":"2024-04-16T19:36:40.593720+0000","last_clean":"2024-04-16T19:36:40.593720+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:36:40.593720+0000","last_undegraded":"2024-04-16T19:36:40.593720+0000","last_fullsized":"2024-04-16T19:36:40.593720+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.593577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.105795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.440188+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:36:18.440188+0000","last_peered":"2024-04-16T19:36:18.440188+0000","last_clean":"2024-04-16T19:36:18.440188+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:36:18.440188+0000","last_undegraded":"2024-04-16T19:36:18.440188+0000","last_fullsized":"2024-04-16T19:36:18.440188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'76","reported_seq":1160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587954+0000","last_change":"2024-04-16T17:48:34.318880+0000","last_active":"2024-04-16T19:36:16.587954+0000","last_peered":"2024-04-16T19:36:16.587954+0000","last_clean":"2024-04-16T19:36:16.587954+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:36:16.587954+0000","last_undegraded":"2024-04-16T19:36:16.587954+0000","last_fullsized":"2024-04-16T19:36:16.587954+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.396377+0000","last_change":"2024-04-16T19:36:50.396377+0000","last_active":"2024-04-16T19:36:50.396377+0000","last_peered":"2024-04-16T19:36:50.396377+0000","last_clean":"2024-04-16T19:36:50.396377+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:36:50.396377+0000","last_undegraded":"2024-04-16T19:36:50.396377+0000","last_fullsized":"2024-04-16T19:36:50.396377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.396177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:28.233570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'6","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210230+0000","last_change":"2024-04-16T17:48:01.256444+0000","last_active":"2024-04-16T19:36:01.210230+0000","last_peered":"2024-04-16T19:36:01.210230+0000","last_clean":"2024-04-16T19:36:01.210230+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:36:01.210230+0000","last_undegraded":"2024-04-16T19:36:01.210230+0000","last_fullsized":"2024-04-16T19:36:01.210230+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'556","reported_seq":1943,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.469613+0000","last_change":"2024-04-16T19:36:37.469677+0000","last_active":"2024-04-16T19:36:37.469613+0000","last_peered":"2024-04-16T19:36:37.469613+0000","last_clean":"2024-04-16T19:36:37.469613+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:36:37.469613+0000","last_undegraded":"2024-04-16T19:36:37.469613+0000","last_fullsized":"2024-04-16T19:36:37.469613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.469478+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:42.920753+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'327","reported_seq":654,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.537153+0000","last_change":"2024-04-16T19:36:23.537297+0000","last_active":"2024-04-16T19:36:23.537153+0000","last_peered":"2024-04-16T19:36:23.537153+0000","last_clean":"2024-04-16T19:36:23.537153+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:36:23.537153+0000","last_undegraded":"2024-04-16T19:36:23.537153+0000","last_fullsized":"2024-04-16T19:36:23.537153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'327","last_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_deep_scrub":"53'327","last_deep_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.537014+0000","objects_scrubbed":21,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:14.538959+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'7","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.487632+0000","last_change":"2024-04-16T19:36:18.487675+0000","last_active":"2024-04-16T19:36:18.487632+0000","last_peered":"2024-04-16T19:36:18.487632+0000","last_clean":"2024-04-16T19:36:18.487632+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:36:18.487632+0000","last_undegraded":"2024-04-16T19:36:18.487632+0000","last_fullsized":"2024-04-16T19:36:18.487632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.487557+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:46.066228+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206529+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:35:46.206529+0000","last_peered":"2024-04-16T19:35:46.206529+0000","last_clean":"2024-04-16T19:35:46.206529+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:35:46.206529+0000","last_undegraded":"2024-04-16T19:35:46.206529+0000","last_fullsized":"2024-04-16T19:35:46.206529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.196366+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:35:01.196366+0000","last_peered":"2024-04-16T19:35:01.196366+0000","last_clean":"2024-04-16T19:35:01.196366+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:35:01.196366+0000","last_undegraded":"2024-04-16T19:35:01.196366+0000","last_fullsized":"2024-04-16T19:35:01.196366+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":293,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.202966+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:35:16.202966+0000","last_peered":"2024-04-16T19:35:16.202966+0000","last_clean":"2024-04-16T19:35:16.202966+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:35:16.202966+0000","last_undegraded":"2024-04-16T19:35:16.202966+0000","last_fullsized":"2024-04-16T19:35:16.202966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205065+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:35:41.205065+0000","last_peered":"2024-04-16T19:35:41.205065+0000","last_clean":"2024-04-16T19:35:41.205065+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:35:41.205065+0000","last_undegraded":"2024-04-16T19:35:41.205065+0000","last_fullsized":"2024-04-16T19:35:41.205065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":505,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208846+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:35:56.208846+0000","last_peered":"2024-04-16T19:35:56.208846+0000","last_clean":"2024-04-16T19:35:56.208846+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:35:56.208846+0000","last_undegraded":"2024-04-16T19:35:56.208846+0000","last_fullsized":"2024-04-16T19:35:56.208846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'177","reported_seq":1613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210760+0000","last_change":"2024-04-16T17:47:35.312413+0000","last_active":"2024-04-16T19:35:56.210760+0000","last_peered":"2024-04-16T19:35:56.210760+0000","last_clean":"2024-04-16T19:35:56.210760+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:35:56.210760+0000","last_undegraded":"2024-04-16T19:35:56.210760+0000","last_fullsized":"2024-04-16T19:35:56.210760+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.591692+0000","last_change":"2024-04-16T19:36:54.591795+0000","last_active":"2024-04-16T19:36:54.591692+0000","last_peered":"2024-04-16T19:36:54.591692+0000","last_clean":"2024-04-16T19:36:54.591692+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:36:54.591692+0000","last_undegraded":"2024-04-16T19:36:54.591692+0000","last_fullsized":"2024-04-16T19:36:54.591692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.591530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:30.795070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.464452+0000","last_change":"2024-04-16T19:36:28.464452+0000","last_active":"2024-04-16T19:36:28.464452+0000","last_peered":"2024-04-16T19:36:28.464452+0000","last_clean":"2024-04-16T19:36:28.464452+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:36:28.464452+0000","last_undegraded":"2024-04-16T19:36:28.464452+0000","last_fullsized":"2024-04-16T19:36:28.464452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.464263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:07.132103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.188297+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:35:51.188297+0000","last_peered":"2024-04-16T19:35:51.188297+0000","last_clean":"2024-04-16T19:35:51.188297+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:35:51.188297+0000","last_undegraded":"2024-04-16T19:35:51.188297+0000","last_fullsized":"2024-04-16T19:35:51.188297+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1180,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588024+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:36:16.588024+0000","last_peered":"2024-04-16T19:36:16.588024+0000","last_clean":"2024-04-16T19:36:16.588024+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:36:16.588024+0000","last_undegraded":"2024-04-16T19:36:16.588024+0000","last_fullsized":"2024-04-16T19:36:16.588024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.422295+0000","last_change":"2024-04-16T19:36:40.422295+0000","last_active":"2024-04-16T19:36:40.422295+0000","last_peered":"2024-04-16T19:36:40.422295+0000","last_clean":"2024-04-16T19:36:40.422295+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:36:40.422295+0000","last_undegraded":"2024-04-16T19:36:40.422295+0000","last_fullsized":"2024-04-16T19:36:40.422295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.422090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:53.341423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'19","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.529264+0000","last_change":"2024-04-16T19:36:30.529412+0000","last_active":"2024-04-16T19:36:30.529264+0000","last_peered":"2024-04-16T19:36:30.529264+0000","last_clean":"2024-04-16T19:36:30.529264+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:36:30.529264+0000","last_undegraded":"2024-04-16T19:36:30.529264+0000","last_fullsized":"2024-04-16T19:36:30.529264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.529097+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:25.629336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'556","reported_seq":1933,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.659990+0000","last_change":"2024-04-16T19:36:28.660054+0000","last_active":"2024-04-16T19:36:28.659990+0000","last_peered":"2024-04-16T19:36:28.659990+0000","last_clean":"2024-04-16T19:36:28.659990+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:36:28.659990+0000","last_undegraded":"2024-04-16T19:36:28.659990+0000","last_fullsized":"2024-04-16T19:36:28.659990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.659827+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:31.852118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'532","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.416491+0000","last_change":"2024-04-16T19:36:24.416491+0000","last_active":"2024-04-16T19:36:24.416491+0000","last_peered":"2024-04-16T19:36:24.416491+0000","last_clean":"2024-04-16T19:36:24.416491+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:36:24.416491+0000","last_undegraded":"2024-04-16T19:36:24.416491+0000","last_fullsized":"2024-04-16T19:36:24.416491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.416318+0000","objects_scrubbed":18,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:15.418263+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"53'6","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.488767+0000","last_change":"2024-04-16T19:36:23.488844+0000","last_active":"2024-04-16T19:36:23.488767+0000","last_peered":"2024-04-16T19:36:23.488767+0000","last_clean":"2024-04-16T19:36:23.488767+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:36:23.488767+0000","last_undegraded":"2024-04-16T19:36:23.488767+0000","last_fullsized":"2024-04-16T19:36:23.488767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.488637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:34:36.790606+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202127+0000","last_change":"2024-04-16T17:47:23.266373+0000","last_active":"2024-04-16T19:35:46.202127+0000","last_peered":"2024-04-16T19:35:46.202127+0000","last_clean":"2024-04-16T19:35:46.202127+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:35:46.202127+0000","last_undegraded":"2024-04-16T19:35:46.202127+0000","last_fullsized":"2024-04-16T19:35:46.202127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189734+0000","last_change":"2024-04-16T17:47:32.273580+0000","last_active":"2024-04-16T19:35:56.189734+0000","last_peered":"2024-04-16T19:35:56.189734+0000","last_clean":"2024-04-16T19:35:56.189734+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:35:56.189734+0000","last_undegraded":"2024-04-16T19:35:56.189734+0000","last_fullsized":"2024-04-16T19:35:56.189734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.196438+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:35:01.196438+0000","last_peered":"2024-04-16T19:35:01.196438+0000","last_clean":"2024-04-16T19:35:01.196438+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:35:01.196438+0000","last_undegraded":"2024-04-16T19:35:01.196438+0000","last_fullsized":"2024-04-16T19:35:01.196438+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.192014+0000","last_change":"2024-04-16T17:47:33.300039+0000","last_active":"2024-04-16T19:36:06.192014+0000","last_peered":"2024-04-16T19:36:06.192014+0000","last_clean":"2024-04-16T19:36:06.192014+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:36:06.192014+0000","last_undegraded":"2024-04-16T19:36:06.192014+0000","last_fullsized":"2024-04-16T19:36:06.192014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'144","reported_seq":348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204740+0000","last_change":"2024-04-16T17:47:35.305470+0000","last_active":"2024-04-16T19:35:56.204740+0000","last_peered":"2024-04-16T19:35:56.204740+0000","last_clean":"2024-04-16T19:35:56.204740+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:35:56.204740+0000","last_undegraded":"2024-04-16T19:35:56.204740+0000","last_fullsized":"2024-04-16T19:35:56.204740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.468997+0000","last_change":"2024-04-16T19:36:50.469053+0000","last_active":"2024-04-16T19:36:50.468997+0000","last_peered":"2024-04-16T19:36:50.468997+0000","last_clean":"2024-04-16T19:36:50.468997+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:36:50.468997+0000","last_undegraded":"2024-04-16T19:36:50.468997+0000","last_fullsized":"2024-04-16T19:36:50.468997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.468857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:29.956415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.495260+0000","last_change":"2024-04-16T19:36:38.495312+0000","last_active":"2024-04-16T19:36:38.495260+0000","last_peered":"2024-04-16T19:36:38.495260+0000","last_clean":"2024-04-16T19:36:38.495260+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:36:38.495260+0000","last_undegraded":"2024-04-16T19:36:38.495260+0000","last_fullsized":"2024-04-16T19:36:38.495260+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.495125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:12.806239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528888+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:36:17.528888+0000","last_peered":"2024-04-16T19:36:17.528888+0000","last_clean":"2024-04-16T19:36:17.528888+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:36:17.528888+0000","last_undegraded":"2024-04-16T19:36:17.528888+0000","last_fullsized":"2024-04-16T19:36:17.528888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'122","reported_seq":1764,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204310+0000","last_change":"2024-04-16T17:48:34.337192+0000","last_active":"2024-04-16T19:35:21.204310+0000","last_peered":"2024-04-16T19:35:21.204310+0000","last_clean":"2024-04-16T19:35:21.204310+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:35:21.204310+0000","last_undegraded":"2024-04-16T19:35:21.204310+0000","last_fullsized":"2024-04-16T19:35:21.204310+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.400809+0000","last_change":"2024-04-16T19:36:49.400809+0000","last_active":"2024-04-16T19:36:49.400809+0000","last_peered":"2024-04-16T19:36:49.400809+0000","last_clean":"2024-04-16T19:36:49.400809+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:36:49.400809+0000","last_undegraded":"2024-04-16T19:36:49.400809+0000","last_fullsized":"2024-04-16T19:36:49.400809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.400614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:52.383331+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'20","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.530462+0000","last_change":"2024-04-16T19:36:31.530526+0000","last_active":"2024-04-16T19:36:31.530462+0000","last_peered":"2024-04-16T19:36:31.530462+0000","last_clean":"2024-04-16T19:36:31.530462+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:36:31.530462+0000","last_undegraded":"2024-04-16T19:36:31.530462+0000","last_fullsized":"2024-04-16T19:36:31.530462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.530356+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:33:36.382371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.979971+0000","last_change":"2024-04-16T17:47:59.185616+0000","last_active":"2024-04-16T19:36:17.979971+0000","last_peered":"2024-04-16T19:36:17.979971+0000","last_clean":"2024-04-16T19:36:17.979971+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:36:17.979971+0000","last_undegraded":"2024-04-16T19:36:17.979971+0000","last_fullsized":"2024-04-16T19:36:17.979971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1182","reported_seq":1443,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.649769+0000","last_change":"2024-04-16T19:36:23.649928+0000","last_active":"2024-04-16T19:36:23.649769+0000","last_peered":"2024-04-16T19:36:23.649769+0000","last_clean":"2024-04-16T19:36:23.649769+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:36:23.649769+0000","last_undegraded":"2024-04-16T19:36:23.649769+0000","last_fullsized":"2024-04-16T19:36:23.649769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1182","last_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_deep_scrub":"53'1182","last_deep_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.649623+0000","objects_scrubbed":23,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:09.999744+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215927,"num_omap_keys":942,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'28","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.423045+0000","last_change":"2024-04-16T19:36:25.423158+0000","last_active":"2024-04-16T19:36:25.423045+0000","last_peered":"2024-04-16T19:36:25.423045+0000","last_clean":"2024-04-16T19:36:25.423045+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:36:25.423045+0000","last_undegraded":"2024-04-16T19:36:25.423045+0000","last_fullsized":"2024-04-16T19:36:25.423045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'28","last_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_deep_scrub":"53'28","last_deep_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.422911+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:22.027239+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209861+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:36:01.209861+0000","last_peered":"2024-04-16T19:36:01.209861+0000","last_clean":"2024-04-16T19:36:01.209861+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:36:01.209861+0000","last_undegraded":"2024-04-16T19:36:01.209861+0000","last_fullsized":"2024-04-16T19:36:01.209861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186094+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:35:41.186094+0000","last_peered":"2024-04-16T19:35:41.186094+0000","last_clean":"2024-04-16T19:35:41.186094+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:35:41.186094+0000","last_undegraded":"2024-04-16T19:35:41.186094+0000","last_fullsized":"2024-04-16T19:35:41.186094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.199839+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:35:36.199839+0000","last_peered":"2024-04-16T19:35:36.199839+0000","last_clean":"2024-04-16T19:35:36.199839+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:35:36.199839+0000","last_undegraded":"2024-04-16T19:35:36.199839+0000","last_fullsized":"2024-04-16T19:35:36.199839+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213000+0000","last_change":"2024-04-16T17:47:33.300948+0000","last_active":"2024-04-16T19:35:56.213000+0000","last_peered":"2024-04-16T19:35:56.213000+0000","last_clean":"2024-04-16T19:35:56.213000+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:35:56.213000+0000","last_undegraded":"2024-04-16T19:35:56.213000+0000","last_fullsized":"2024-04-16T19:35:56.213000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'120","reported_seq":297,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213049+0000","last_change":"2024-04-16T17:47:35.313794+0000","last_active":"2024-04-16T19:35:56.213049+0000","last_peered":"2024-04-16T19:35:56.213049+0000","last_clean":"2024-04-16T19:35:56.213049+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:35:56.213049+0000","last_undegraded":"2024-04-16T19:35:56.213049+0000","last_fullsized":"2024-04-16T19:35:56.213049+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.688114+0000","last_change":"2024-04-16T19:36:44.688178+0000","last_active":"2024-04-16T19:36:44.688114+0000","last_peered":"2024-04-16T19:36:44.688114+0000","last_clean":"2024-04-16T19:36:44.688114+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:36:44.688114+0000","last_undegraded":"2024-04-16T19:36:44.688114+0000","last_fullsized":"2024-04-16T19:36:44.688114+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.687973+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:44.814523+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.534123+0000","last_change":"2024-04-16T19:36:35.534176+0000","last_active":"2024-04-16T19:36:35.534123+0000","last_peered":"2024-04-16T19:36:35.534123+0000","last_clean":"2024-04-16T19:36:35.534123+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:36:35.534123+0000","last_undegraded":"2024-04-16T19:36:35.534123+0000","last_fullsized":"2024-04-16T19:36:35.534123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.533984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:55.743351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181002+0000","last_change":"2024-04-16T18:10:50.537157+0000","last_active":"2024-04-16T19:35:21.181002+0000","last_peered":"2024-04-16T19:35:21.181002+0000","last_clean":"2024-04-16T19:35:21.181002+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:35:21.181002+0000","last_undegraded":"2024-04-16T19:35:21.181002+0000","last_fullsized":"2024-04-16T19:35:21.181002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1284,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915002+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:36:18.915002+0000","last_peered":"2024-04-16T19:36:18.915002+0000","last_clean":"2024-04-16T19:36:18.915002+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:36:18.915002+0000","last_undegraded":"2024-04-16T19:36:18.915002+0000","last_fullsized":"2024-04-16T19:36:18.915002+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.504746+0000","last_change":"2024-04-16T19:36:44.504798+0000","last_active":"2024-04-16T19:36:44.504746+0000","last_peered":"2024-04-16T19:36:44.504746+0000","last_clean":"2024-04-16T19:36:44.504746+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:36:44.504746+0000","last_undegraded":"2024-04-16T19:36:44.504746+0000","last_fullsized":"2024-04-16T19:36:44.504746+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.504608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:08:17.020106+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'21","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.684658+0000","last_change":"2024-04-16T19:36:52.684736+0000","last_active":"2024-04-16T19:36:52.684658+0000","last_peered":"2024-04-16T19:36:52.684658+0000","last_clean":"2024-04-16T19:36:52.684658+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:36:52.684658+0000","last_undegraded":"2024-04-16T19:36:52.684658+0000","last_fullsized":"2024-04-16T19:36:52.684658+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.684531+0000","objects_scrubbed":8,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:32.649666+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.646374+0000","last_change":"2024-04-16T19:36:45.646430+0000","last_active":"2024-04-16T19:36:45.646374+0000","last_peered":"2024-04-16T19:36:45.646374+0000","last_clean":"2024-04-16T19:36:45.646374+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:36:45.646374+0000","last_undegraded":"2024-04-16T19:36:45.646374+0000","last_fullsized":"2024-04-16T19:36:45.646374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.646241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:48.502408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1343","reported_seq":1906,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.619403+0000","last_change":"2024-04-16T19:36:21.619494+0000","last_active":"2024-04-16T19:36:21.619403+0000","last_peered":"2024-04-16T19:36:21.619403+0000","last_clean":"2024-04-16T19:36:21.619403+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:36:21.619403+0000","last_undegraded":"2024-04-16T19:36:21.619403+0000","last_fullsized":"2024-04-16T19:36:21.619403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1343","last_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_deep_scrub":"53'1343","last_deep_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.619266+0000","objects_scrubbed":29,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:34.145789+0000","scrub_duration":71,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":216307,"num_omap_keys":944,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'8","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.444455+0000","last_change":"2024-04-16T19:36:33.444542+0000","last_active":"2024-04-16T19:36:33.444455+0000","last_peered":"2024-04-16T19:36:33.444455+0000","last_clean":"2024-04-16T19:36:33.444455+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:36:33.444455+0000","last_undegraded":"2024-04-16T19:36:33.444455+0000","last_fullsized":"2024-04-16T19:36:33.444455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.444316+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:36.300483+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206348+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:35:46.206348+0000","last_peered":"2024-04-16T19:35:46.206348+0000","last_clean":"2024-04-16T19:35:46.206348+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:35:46.206348+0000","last_undegraded":"2024-04-16T19:35:46.206348+0000","last_fullsized":"2024-04-16T19:35:46.206348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206492+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:35:41.206492+0000","last_peered":"2024-04-16T19:35:41.206492+0000","last_clean":"2024-04-16T19:35:41.206492+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:35:41.206492+0000","last_undegraded":"2024-04-16T19:35:41.206492+0000","last_fullsized":"2024-04-16T19:35:41.206492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.198223+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:34:56.198223+0000","last_peered":"2024-04-16T19:34:56.198223+0000","last_clean":"2024-04-16T19:34:56.198223+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:34:56.198223+0000","last_undegraded":"2024-04-16T19:34:56.198223+0000","last_fullsized":"2024-04-16T19:34:56.198223+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208557+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:35:56.208557+0000","last_peered":"2024-04-16T19:35:56.208557+0000","last_clean":"2024-04-16T19:35:56.208557+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:35:56.208557+0000","last_undegraded":"2024-04-16T19:35:56.208557+0000","last_fullsized":"2024-04-16T19:35:56.208557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'168","reported_seq":369,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210580+0000","last_change":"2024-04-16T17:47:35.321995+0000","last_active":"2024-04-16T19:35:56.210580+0000","last_peered":"2024-04-16T19:35:56.210580+0000","last_clean":"2024-04-16T19:35:56.210580+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:35:56.210580+0000","last_undegraded":"2024-04-16T19:35:56.210580+0000","last_fullsized":"2024-04-16T19:35:56.210580+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'12","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211023+0000","last_change":"2024-04-16T17:47:37.323586+0000","last_active":"2024-04-16T19:36:06.211023+0000","last_peered":"2024-04-16T19:36:06.211023+0000","last_clean":"2024-04-16T19:36:06.211023+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:36:06.211023+0000","last_undegraded":"2024-04-16T19:36:06.211023+0000","last_fullsized":"2024-04-16T19:36:06.211023+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.501363+0000","last_change":"2024-04-16T19:36:34.501465+0000","last_active":"2024-04-16T19:36:34.501363+0000","last_peered":"2024-04-16T19:36:34.501363+0000","last_clean":"2024-04-16T19:36:34.501363+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:36:34.501363+0000","last_undegraded":"2024-04-16T19:36:34.501363+0000","last_fullsized":"2024-04-16T19:36:34.501363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.501193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:08.812307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.198858+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:35:21.198858+0000","last_peered":"2024-04-16T19:35:21.198858+0000","last_clean":"2024-04-16T19:35:21.198858+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:35:21.198858+0000","last_undegraded":"2024-04-16T19:35:21.198858+0000","last_fullsized":"2024-04-16T19:35:21.198858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587804+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:36:16.587804+0000","last_peered":"2024-04-16T19:36:16.587804+0000","last_clean":"2024-04-16T19:36:16.587804+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:36:16.587804+0000","last_undegraded":"2024-04-16T19:36:16.587804+0000","last_fullsized":"2024-04-16T19:36:16.587804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.397896+0000","last_change":"2024-04-16T19:36:47.398006+0000","last_active":"2024-04-16T19:36:47.397896+0000","last_peered":"2024-04-16T19:36:47.397896+0000","last_clean":"2024-04-16T19:36:47.397896+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:36:47.397896+0000","last_undegraded":"2024-04-16T19:36:47.397896+0000","last_fullsized":"2024-04-16T19:36:47.397896+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.397724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:25.235116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.547894+0000","last_change":"2024-04-16T19:36:32.547894+0000","last_active":"2024-04-16T19:36:32.547894+0000","last_peered":"2024-04-16T19:36:32.547894+0000","last_clean":"2024-04-16T19:36:32.547894+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:36:32.547894+0000","last_undegraded":"2024-04-16T19:36:32.547894+0000","last_fullsized":"2024-04-16T19:36:32.547894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.547672+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:45.074195+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'556","reported_seq":2004,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.565740+0000","last_change":"2024-04-16T19:36:33.565806+0000","last_active":"2024-04-16T19:36:33.565740+0000","last_peered":"2024-04-16T19:36:33.565740+0000","last_clean":"2024-04-16T19:36:33.565740+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:36:33.565740+0000","last_undegraded":"2024-04-16T19:36:33.565740+0000","last_fullsized":"2024-04-16T19:36:33.565740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.565603+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:32.212045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'575","reported_seq":853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.628047+0000","last_change":"2024-04-16T19:36:22.628176+0000","last_active":"2024-04-16T19:36:22.628047+0000","last_peered":"2024-04-16T19:36:22.628047+0000","last_clean":"2024-04-16T19:36:22.628047+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:36:22.628047+0000","last_undegraded":"2024-04-16T19:36:22.628047+0000","last_fullsized":"2024-04-16T19:36:22.628047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'575","last_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_deep_scrub":"53'575","last_deep_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.627889+0000","objects_scrubbed":22,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:21.274331+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71488,"num_omap_keys":312,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'18","reported_seq":261,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.483626+0000","last_change":"2024-04-16T19:36:29.483703+0000","last_active":"2024-04-16T19:36:29.483626+0000","last_peered":"2024-04-16T19:36:29.483626+0000","last_clean":"2024-04-16T19:36:29.483626+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:36:29.483626+0000","last_undegraded":"2024-04-16T19:36:29.483626+0000","last_fullsized":"2024-04-16T19:36:29.483626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.483500+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:36:53.084475+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207780+0000","last_change":"2024-04-16T17:47:23.265822+0000","last_active":"2024-04-16T19:35:46.207780+0000","last_peered":"2024-04-16T19:35:46.207780+0000","last_clean":"2024-04-16T19:35:46.207780+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:35:46.207780+0000","last_undegraded":"2024-04-16T19:35:46.207780+0000","last_fullsized":"2024-04-16T19:35:46.207780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211258+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:36:06.211258+0000","last_peered":"2024-04-16T19:36:06.211258+0000","last_clean":"2024-04-16T19:36:06.211258+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:36:06.211258+0000","last_undegraded":"2024-04-16T19:36:06.211258+0000","last_fullsized":"2024-04-16T19:36:06.211258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192623+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:34:56.192623+0000","last_peered":"2024-04-16T19:34:56.192623+0000","last_clean":"2024-04-16T19:34:56.192623+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:34:56.192623+0000","last_undegraded":"2024-04-16T19:34:56.192623+0000","last_fullsized":"2024-04-16T19:34:56.192623+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208829+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:35:56.208829+0000","last_peered":"2024-04-16T19:35:56.208829+0000","last_clean":"2024-04-16T19:35:56.208829+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:35:56.208829+0000","last_undegraded":"2024-04-16T19:35:56.208829+0000","last_fullsized":"2024-04-16T19:35:56.208829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'160","reported_seq":369,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212483+0000","last_change":"2024-04-16T17:47:35.312410+0000","last_active":"2024-04-16T19:36:11.212483+0000","last_peered":"2024-04-16T19:36:11.212483+0000","last_clean":"2024-04-16T19:36:11.212483+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:36:11.212483+0000","last_undegraded":"2024-04-16T19:36:11.212483+0000","last_fullsized":"2024-04-16T19:36:11.212483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.205991+0000","last_change":"2024-04-16T17:47:37.322938+0000","last_active":"2024-04-16T19:36:01.205991+0000","last_peered":"2024-04-16T19:36:01.205991+0000","last_clean":"2024-04-16T19:36:01.205991+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:36:01.205991+0000","last_undegraded":"2024-04-16T19:36:01.205991+0000","last_fullsized":"2024-04-16T19:36:01.205991+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.422338+0000","last_change":"2024-04-16T19:36:32.422395+0000","last_active":"2024-04-16T19:36:32.422338+0000","last_peered":"2024-04-16T19:36:32.422338+0000","last_clean":"2024-04-16T19:36:32.422338+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:36:32.422338+0000","last_undegraded":"2024-04-16T19:36:32.422338+0000","last_fullsized":"2024-04-16T19:36:32.422338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.422189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:35.278356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201369+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:35:26.201369+0000","last_peered":"2024-04-16T19:35:26.201369+0000","last_clean":"2024-04-16T19:35:26.201369+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:35:26.201369+0000","last_undegraded":"2024-04-16T19:35:26.201369+0000","last_fullsized":"2024-04-16T19:35:26.201369+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.206683+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:35:31.206683+0000","last_peered":"2024-04-16T19:35:31.206683+0000","last_clean":"2024-04-16T19:35:31.206683+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:35:31.206683+0000","last_undegraded":"2024-04-16T19:35:31.206683+0000","last_fullsized":"2024-04-16T19:35:31.206683+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.635196+0000","last_change":"2024-04-16T19:36:39.635263+0000","last_active":"2024-04-16T19:36:39.635196+0000","last_peered":"2024-04-16T19:36:39.635196+0000","last_clean":"2024-04-16T19:36:39.635196+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:36:39.635196+0000","last_undegraded":"2024-04-16T19:36:39.635196+0000","last_fullsized":"2024-04-16T19:36:39.635196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.635056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:26.132595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'20","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.709187+0000","last_change":"2024-04-16T19:36:42.709267+0000","last_active":"2024-04-16T19:36:42.709187+0000","last_peered":"2024-04-16T19:36:42.709187+0000","last_clean":"2024-04-16T19:36:42.709187+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:36:42.709187+0000","last_undegraded":"2024-04-16T19:36:42.709187+0000","last_fullsized":"2024-04-16T19:36:42.709187+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.709051+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:57.264715+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.501219+0000","last_change":"2024-04-16T19:36:35.501282+0000","last_active":"2024-04-16T19:36:35.501219+0000","last_peered":"2024-04-16T19:36:35.501219+0000","last_clean":"2024-04-16T19:36:35.501219+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:36:35.501219+0000","last_undegraded":"2024-04-16T19:36:35.501219+0000","last_fullsized":"2024-04-16T19:36:35.501219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.501075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:40.913802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'871","reported_seq":1181,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.697221+0000","last_change":"2024-04-16T19:36:18.697375+0000","last_active":"2024-04-16T19:36:18.697221+0000","last_peered":"2024-04-16T19:36:18.697221+0000","last_clean":"2024-04-16T19:36:18.697221+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:36:18.697221+0000","last_undegraded":"2024-04-16T19:36:18.697221+0000","last_fullsized":"2024-04-16T19:36:18.697221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'871","last_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_deep_scrub":"53'871","last_deep_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.697076+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:49.607583+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143413,"num_omap_keys":626,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'16","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.208672+0000","last_change":"2024-04-16T17:49:32.398153+0000","last_active":"2024-04-16T19:36:11.208672+0000","last_peered":"2024-04-16T19:36:11.208672+0000","last_clean":"2024-04-16T19:36:11.208672+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:36:11.208672+0000","last_undegraded":"2024-04-16T19:36:11.208672+0000","last_fullsized":"2024-04-16T19:36:11.208672+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.570337+0000","last_change":"2024-04-16T19:36:36.570388+0000","last_active":"2024-04-16T19:36:36.570337+0000","last_peered":"2024-04-16T19:36:36.570337+0000","last_clean":"2024-04-16T19:36:36.570337+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:36:36.570337+0000","last_undegraded":"2024-04-16T19:36:36.570337+0000","last_fullsized":"2024-04-16T19:36:36.570337+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.570198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:51.282322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.187216+0000","last_change":"2024-04-16T17:47:23.257996+0000","last_active":"2024-04-16T19:34:46.187216+0000","last_peered":"2024-04-16T19:34:46.187216+0000","last_clean":"2024-04-16T19:34:46.187216+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:34:46.187216+0000","last_undegraded":"2024-04-16T19:34:46.187216+0000","last_fullsized":"2024-04-16T19:34:46.187216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.203208+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:35:16.203208+0000","last_peered":"2024-04-16T19:35:16.203208+0000","last_clean":"2024-04-16T19:35:16.203208+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:35:16.203208+0000","last_undegraded":"2024-04-16T19:35:16.203208+0000","last_fullsized":"2024-04-16T19:35:16.203208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"48'12","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194889+0000","last_change":"2024-04-16T17:47:33.297781+0000","last_active":"2024-04-16T19:34:56.194889+0000","last_peered":"2024-04-16T19:34:56.194889+0000","last_clean":"2024-04-16T19:34:56.194889+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:34:56.194889+0000","last_undegraded":"2024-04-16T19:34:56.194889+0000","last_fullsized":"2024-04-16T19:34:56.194889+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'104","reported_seq":272,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175329+0000","last_change":"2024-04-16T17:47:35.310182+0000","last_active":"2024-04-16T19:34:56.175329+0000","last_peered":"2024-04-16T19:34:56.175329+0000","last_clean":"2024-04-16T19:34:56.175329+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:34:56.175329+0000","last_undegraded":"2024-04-16T19:34:56.175329+0000","last_fullsized":"2024-04-16T19:34:56.175329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.627219+0000","last_change":"2024-04-16T19:36:50.627271+0000","last_active":"2024-04-16T19:36:50.627219+0000","last_peered":"2024-04-16T19:36:50.627219+0000","last_clean":"2024-04-16T19:36:50.627219+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:36:50.627219+0000","last_undegraded":"2024-04-16T19:36:50.627219+0000","last_fullsized":"2024-04-16T19:36:50.627219+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.627076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:16.020863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204551+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:35:56.204551+0000","last_peered":"2024-04-16T19:35:56.204551+0000","last_clean":"2024-04-16T19:35:56.204551+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:35:56.204551+0000","last_undegraded":"2024-04-16T19:35:56.204551+0000","last_fullsized":"2024-04-16T19:35:56.204551+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'55","reported_seq":672,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194769+0000","last_change":"2024-04-16T17:48:34.343212+0000","last_active":"2024-04-16T19:34:56.194769+0000","last_peered":"2024-04-16T19:34:56.194769+0000","last_clean":"2024-04-16T19:34:56.194769+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:34:56.194769+0000","last_undegraded":"2024-04-16T19:34:56.194769+0000","last_fullsized":"2024-04-16T19:34:56.194769+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.640311+0000","last_change":"2024-04-16T19:36:36.640365+0000","last_active":"2024-04-16T19:36:36.640311+0000","last_peered":"2024-04-16T19:36:36.640311+0000","last_clean":"2024-04-16T19:36:36.640311+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:36:36.640311+0000","last_undegraded":"2024-04-16T19:36:36.640311+0000","last_fullsized":"2024-04-16T19:36:36.640311+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.640174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:51.451059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'15","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.647852+0000","last_change":"2024-04-16T19:36:39.647955+0000","last_active":"2024-04-16T19:36:39.647852+0000","last_peered":"2024-04-16T19:36:39.647852+0000","last_clean":"2024-04-16T19:36:39.647852+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:36:39.647852+0000","last_undegraded":"2024-04-16T19:36:39.647852+0000","last_fullsized":"2024-04-16T19:36:39.647852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.647719+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:50.538220+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.430796+0000","last_change":"2024-04-16T19:36:38.430850+0000","last_active":"2024-04-16T19:36:38.430796+0000","last_peered":"2024-04-16T19:36:38.430796+0000","last_clean":"2024-04-16T19:36:38.430796+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:36:38.430796+0000","last_undegraded":"2024-04-16T19:36:38.430796+0000","last_fullsized":"2024-04-16T19:36:38.430796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.430655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:03.824441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'616","reported_seq":2312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.598175+0000","last_change":"2024-04-16T19:36:23.598300+0000","last_active":"2024-04-16T19:36:23.598175+0000","last_peered":"2024-04-16T19:36:23.598175+0000","last_clean":"2024-04-16T19:36:23.598175+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:36:23.598175+0000","last_undegraded":"2024-04-16T19:36:23.598175+0000","last_fullsized":"2024-04-16T19:36:23.598175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'616","last_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_deep_scrub":"53'616","last_deep_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.598031+0000","objects_scrubbed":24,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:37.526844+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.510530+0000","last_change":"2024-04-16T19:36:23.510613+0000","last_active":"2024-04-16T19:36:23.510530+0000","last_peered":"2024-04-16T19:36:23.510530+0000","last_clean":"2024-04-16T19:36:23.510530+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:36:23.510530+0000","last_undegraded":"2024-04-16T19:36:23.510530+0000","last_fullsized":"2024-04-16T19:36:23.510530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.510390+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:03.621835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.537581+0000","last_change":"2024-04-16T19:36:32.537632+0000","last_active":"2024-04-16T19:36:32.537581+0000","last_peered":"2024-04-16T19:36:32.537581+0000","last_clean":"2024-04-16T19:36:32.537581+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:36:32.537581+0000","last_undegraded":"2024-04-16T19:36:32.537581+0000","last_fullsized":"2024-04-16T19:36:32.537581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.537440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:47.348326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.192595+0000","last_change":"2024-04-16T17:47:23.260667+0000","last_active":"2024-04-16T19:34:46.192595+0000","last_peered":"2024-04-16T19:34:46.192595+0000","last_clean":"2024-04-16T19:34:46.192595+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:34:46.192595+0000","last_undegraded":"2024-04-16T19:34:46.192595+0000","last_fullsized":"2024-04-16T19:34:46.192595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215650+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:36:06.215650+0000","last_peered":"2024-04-16T19:36:06.215650+0000","last_clean":"2024-04-16T19:36:06.215650+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:36:06.215650+0000","last_undegraded":"2024-04-16T19:36:06.215650+0000","last_fullsized":"2024-04-16T19:36:06.215650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"48'1","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194967+0000","last_change":"2024-04-16T17:47:33.297748+0000","last_active":"2024-04-16T19:34:56.194967+0000","last_peered":"2024-04-16T19:34:56.194967+0000","last_clean":"2024-04-16T19:34:56.194967+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:34:56.194967+0000","last_undegraded":"2024-04-16T19:34:56.194967+0000","last_fullsized":"2024-04-16T19:34:56.194967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'50","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193696+0000","last_change":"2024-04-16T17:47:35.314682+0000","last_active":"2024-04-16T19:34:56.193696+0000","last_peered":"2024-04-16T19:34:56.193696+0000","last_clean":"2024-04-16T19:34:56.193696+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:34:56.193696+0000","last_undegraded":"2024-04-16T19:34:56.193696+0000","last_fullsized":"2024-04-16T19:34:56.193696+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.199285+0000","last_change":"2024-04-16T17:47:37.327586+0000","last_active":"2024-04-16T19:35:01.199285+0000","last_peered":"2024-04-16T19:35:01.199285+0000","last_clean":"2024-04-16T19:35:01.199285+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:35:01.199285+0000","last_undegraded":"2024-04-16T19:35:01.199285+0000","last_fullsized":"2024-04-16T19:35:01.199285+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207221+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:35:51.207221+0000","last_peered":"2024-04-16T19:35:51.207221+0000","last_clean":"2024-04-16T19:35:51.207221+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:35:51.207221+0000","last_undegraded":"2024-04-16T19:35:51.207221+0000","last_fullsized":"2024-04-16T19:35:51.207221+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'129","reported_seq":1727,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.778020+0000","last_change":"2024-04-16T19:36:55.778092+0000","last_active":"2024-04-16T19:36:55.778020+0000","last_peered":"2024-04-16T19:36:55.778020+0000","last_clean":"2024-04-16T19:36:55.778020+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:36:55.778020+0000","last_undegraded":"2024-04-16T19:36:55.778020+0000","last_fullsized":"2024-04-16T19:36:55.778020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'129","last_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_deep_scrub":"53'129","last_deep_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.777891+0000","objects_scrubbed":30,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:46.059742+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8321,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.410425+0000","last_change":"2024-04-16T19:36:51.410485+0000","last_active":"2024-04-16T19:36:51.410425+0000","last_peered":"2024-04-16T19:36:51.410425+0000","last_clean":"2024-04-16T19:36:51.410425+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:36:51.410425+0000","last_undegraded":"2024-04-16T19:36:51.410425+0000","last_fullsized":"2024-04-16T19:36:51.410425+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.410273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:57.416580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'18","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.684157+0000","last_change":"2024-04-16T19:36:45.684234+0000","last_active":"2024-04-16T19:36:45.684157+0000","last_peered":"2024-04-16T19:36:45.684157+0000","last_clean":"2024-04-16T19:36:45.684157+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:36:45.684157+0000","last_undegraded":"2024-04-16T19:36:45.684157+0000","last_fullsized":"2024-04-16T19:36:45.684157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.684019+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:54.649522+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'21","reported_seq":1432,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.402575+0000","last_change":"2024-04-16T19:36:43.402646+0000","last_active":"2024-04-16T19:36:43.402575+0000","last_peered":"2024-04-16T19:36:43.402575+0000","last_clean":"2024-04-16T19:36:43.402575+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:36:43.402575+0000","last_undegraded":"2024-04-16T19:36:43.402575+0000","last_fullsized":"2024-04-16T19:36:43.402575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.402444+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:20.659167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'7003","reported_seq":10587,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.479202+0000","last_change":"2024-04-16T19:36:36.479319+0000","last_active":"2024-04-16T19:36:36.479202+0000","last_peered":"2024-04-16T19:36:36.479202+0000","last_clean":"2024-04-16T19:36:36.479202+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:36:36.479202+0000","last_undegraded":"2024-04-16T19:36:36.479202+0000","last_fullsized":"2024-04-16T19:36:36.479202+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7003","last_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_deep_scrub":"53'7003","last_deep_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.479068+0000","objects_scrubbed":20,"log_size":1803,"log_dups_size":3000,"ondisk_log_size":1803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:56.688434+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'10","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.437802+0000","last_change":"2024-04-16T19:36:27.437901+0000","last_active":"2024-04-16T19:36:27.437802+0000","last_peered":"2024-04-16T19:36:27.437802+0000","last_clean":"2024-04-16T19:36:27.437802+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:36:27.437802+0000","last_undegraded":"2024-04-16T19:36:27.437802+0000","last_fullsized":"2024-04-16T19:36:27.437802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.437656+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:14.616289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.669172+0000","last_change":"2024-04-16T19:36:42.669224+0000","last_active":"2024-04-16T19:36:42.669172+0000","last_peered":"2024-04-16T19:36:42.669172+0000","last_clean":"2024-04-16T19:36:42.669172+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:36:42.669172+0000","last_undegraded":"2024-04-16T19:36:42.669172+0000","last_fullsized":"2024-04-16T19:36:42.669172+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.669008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:32.950859+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.172925+0000","last_change":"2024-04-16T17:47:23.269620+0000","last_active":"2024-04-16T19:34:46.172925+0000","last_peered":"2024-04-16T19:34:46.172925+0000","last_clean":"2024-04-16T19:34:46.172925+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:34:46.172925+0000","last_undegraded":"2024-04-16T19:34:46.172925+0000","last_fullsized":"2024-04-16T19:34:46.172925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211353+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:36:06.211353+0000","last_peered":"2024-04-16T19:36:06.211353+0000","last_clean":"2024-04-16T19:36:06.211353+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:36:06.211353+0000","last_undegraded":"2024-04-16T19:36:06.211353+0000","last_fullsized":"2024-04-16T19:36:06.211353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189585+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:34:56.189585+0000","last_peered":"2024-04-16T19:34:56.189585+0000","last_clean":"2024-04-16T19:34:56.189585+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:34:56.189585+0000","last_undegraded":"2024-04-16T19:34:56.189585+0000","last_fullsized":"2024-04-16T19:34:56.189585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'104","reported_seq":272,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175489+0000","last_change":"2024-04-16T17:47:35.310781+0000","last_active":"2024-04-16T19:34:56.175489+0000","last_peered":"2024-04-16T19:34:56.175489+0000","last_clean":"2024-04-16T19:34:56.175489+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:34:56.175489+0000","last_undegraded":"2024-04-16T19:34:56.175489+0000","last_fullsized":"2024-04-16T19:34:56.175489+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.193626+0000","last_change":"2024-04-16T17:47:37.322773+0000","last_active":"2024-04-16T19:35:01.193626+0000","last_peered":"2024-04-16T19:35:01.193626+0000","last_clean":"2024-04-16T19:35:01.193626+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:35:01.193626+0000","last_undegraded":"2024-04-16T19:35:01.193626+0000","last_fullsized":"2024-04-16T19:35:01.193626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.211846+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:35:51.211846+0000","last_peered":"2024-04-16T19:35:51.211846+0000","last_clean":"2024-04-16T19:35:51.211846+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:35:51.211846+0000","last_undegraded":"2024-04-16T19:35:51.211846+0000","last_fullsized":"2024-04-16T19:35:51.211846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:36.187344+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:34:36.187344+0000","last_peered":"2024-04-16T19:34:36.187344+0000","last_clean":"2024-04-16T19:34:36.187344+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:34:36.187344+0000","last_undegraded":"2024-04-16T19:34:36.187344+0000","last_fullsized":"2024-04-16T19:34:36.187344+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.499740+0000","last_change":"2024-04-16T19:36:36.499794+0000","last_active":"2024-04-16T19:36:36.499740+0000","last_peered":"2024-04-16T19:36:36.499740+0000","last_clean":"2024-04-16T19:36:36.499740+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:36:36.499740+0000","last_undegraded":"2024-04-16T19:36:36.499740+0000","last_fullsized":"2024-04-16T19:36:36.499740+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.499596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:22.332807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"53'16","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.619264+0000","last_change":"2024-04-16T19:36:44.619264+0000","last_active":"2024-04-16T19:36:44.619264+0000","last_peered":"2024-04-16T19:36:44.619264+0000","last_clean":"2024-04-16T19:36:44.619264+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:36:44.619264+0000","last_undegraded":"2024-04-16T19:36:44.619264+0000","last_fullsized":"2024-04-16T19:36:44.619264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.619046+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:50.031773+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'556","reported_seq":1967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.516649+0000","last_change":"2024-04-16T19:36:35.516719+0000","last_active":"2024-04-16T19:36:35.516649+0000","last_peered":"2024-04-16T19:36:35.516649+0000","last_clean":"2024-04-16T19:36:35.516649+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:36:35.516649+0000","last_undegraded":"2024-04-16T19:36:35.516649+0000","last_fullsized":"2024-04-16T19:36:35.516649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.516509+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:41.203861+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'563","reported_seq":804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.482688+0000","last_change":"2024-04-16T19:36:29.482688+0000","last_active":"2024-04-16T19:36:29.482688+0000","last_peered":"2024-04-16T19:36:29.482688+0000","last_clean":"2024-04-16T19:36:29.482688+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:36:29.482688+0000","last_undegraded":"2024-04-16T19:36:29.482688+0000","last_fullsized":"2024-04-16T19:36:29.482688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.482404+0000","objects_scrubbed":19,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.518930+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71952,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'8","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.461125+0000","last_change":"2024-04-16T19:36:21.461301+0000","last_active":"2024-04-16T19:36:21.461125+0000","last_peered":"2024-04-16T19:36:21.461125+0000","last_clean":"2024-04-16T19:36:21.461125+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:36:21.461125+0000","last_undegraded":"2024-04-16T19:36:21.461125+0000","last_fullsized":"2024-04-16T19:36:21.461125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.460913+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:00.948470+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.460735+0000","last_change":"2024-04-16T19:36:45.460790+0000","last_active":"2024-04-16T19:36:45.460735+0000","last_peered":"2024-04-16T19:36:45.460735+0000","last_clean":"2024-04-16T19:36:45.460735+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:36:45.460735+0000","last_undegraded":"2024-04-16T19:36:45.460735+0000","last_fullsized":"2024-04-16T19:36:45.460735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.460592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:12.082129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.192486+0000","last_change":"2024-04-16T17:47:23.263370+0000","last_active":"2024-04-16T19:34:46.192486+0000","last_peered":"2024-04-16T19:34:46.192486+0000","last_clean":"2024-04-16T19:34:46.192486+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:34:46.192486+0000","last_undegraded":"2024-04-16T19:34:46.192486+0000","last_fullsized":"2024-04-16T19:34:46.192486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211668+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:36:06.211668+0000","last_peered":"2024-04-16T19:36:06.211668+0000","last_clean":"2024-04-16T19:36:06.211668+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:36:06.211668+0000","last_undegraded":"2024-04-16T19:36:06.211668+0000","last_fullsized":"2024-04-16T19:36:06.211668+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.197964+0000","last_change":"2024-04-16T17:47:33.307304+0000","last_active":"2024-04-16T19:34:56.197964+0000","last_peered":"2024-04-16T19:34:56.197964+0000","last_clean":"2024-04-16T19:34:56.197964+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:34:56.197964+0000","last_undegraded":"2024-04-16T19:34:56.197964+0000","last_fullsized":"2024-04-16T19:34:56.197964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'80","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192429+0000","last_change":"2024-04-16T17:47:35.311451+0000","last_active":"2024-04-16T19:34:56.192429+0000","last_peered":"2024-04-16T19:34:56.192429+0000","last_clean":"2024-04-16T19:34:56.192429+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:34:56.192429+0000","last_undegraded":"2024-04-16T19:34:56.192429+0000","last_fullsized":"2024-04-16T19:34:56.192429+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:01.195110+0000","last_change":"2024-04-16T17:47:37.326333+0000","last_active":"2024-04-16T19:35:01.195110+0000","last_peered":"2024-04-16T19:35:01.195110+0000","last_clean":"2024-04-16T19:35:01.195110+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:35:01.195110+0000","last_undegraded":"2024-04-16T19:35:01.195110+0000","last_fullsized":"2024-04-16T19:35:01.195110+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204458+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:35:21.204458+0000","last_peered":"2024-04-16T19:35:21.204458+0000","last_clean":"2024-04-16T19:35:21.204458+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:35:21.204458+0000","last_undegraded":"2024-04-16T19:35:21.204458+0000","last_fullsized":"2024-04-16T19:35:21.204458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'89","reported_seq":695,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.198124+0000","last_change":"2024-04-16T17:48:34.328117+0000","last_active":"2024-04-16T19:34:56.198124+0000","last_peered":"2024-04-16T19:34:56.198124+0000","last_clean":"2024-04-16T19:34:56.198124+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:34:56.198124+0000","last_undegraded":"2024-04-16T19:34:56.198124+0000","last_fullsized":"2024-04-16T19:34:56.198124+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.678560+0000","last_change":"2024-04-16T19:36:47.678659+0000","last_active":"2024-04-16T19:36:47.678560+0000","last_peered":"2024-04-16T19:36:47.678560+0000","last_clean":"2024-04-16T19:36:47.678560+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:36:47.678560+0000","last_undegraded":"2024-04-16T19:36:47.678560+0000","last_fullsized":"2024-04-16T19:36:47.678560+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.678408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:02:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.735233+0000","last_change":"2024-04-16T19:36:41.735233+0000","last_active":"2024-04-16T19:36:41.735233+0000","last_peered":"2024-04-16T19:36:41.735233+0000","last_clean":"2024-04-16T19:36:41.735233+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:36:41.735233+0000","last_undegraded":"2024-04-16T19:36:41.735233+0000","last_fullsized":"2024-04-16T19:36:41.735233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.735066+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:52.416728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.470863+0000","last_change":"2024-04-16T19:36:33.470929+0000","last_active":"2024-04-16T19:36:33.470863+0000","last_peered":"2024-04-16T19:36:33.470863+0000","last_clean":"2024-04-16T19:36:33.470863+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:36:33.470863+0000","last_undegraded":"2024-04-16T19:36:33.470863+0000","last_fullsized":"2024-04-16T19:36:33.470863+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.470725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:02.342594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'476","reported_seq":4438,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.660175+0000","last_change":"2024-04-16T19:36:19.660343+0000","last_active":"2024-04-16T19:36:19.660175+0000","last_peered":"2024-04-16T19:36:19.660175+0000","last_clean":"2024-04-16T19:36:19.660175+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:36:19.660175+0000","last_undegraded":"2024-04-16T19:36:19.660175+0000","last_fullsized":"2024-04-16T19:36:19.660175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.659960+0000","objects_scrubbed":19,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:52.175458+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'8","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.516188+0000","last_change":"2024-04-16T19:36:36.516257+0000","last_active":"2024-04-16T19:36:36.516188+0000","last_peered":"2024-04-16T19:36:36.516188+0000","last_clean":"2024-04-16T19:36:36.516188+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:36:36.516188+0000","last_undegraded":"2024-04-16T19:36:36.516188+0000","last_fullsized":"2024-04-16T19:36:36.516188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.516049+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:56.739641+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210764+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:35:46.210764+0000","last_peered":"2024-04-16T19:35:46.210764+0000","last_clean":"2024-04-16T19:35:46.210764+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:35:46.210764+0000","last_undegraded":"2024-04-16T19:35:46.210764+0000","last_fullsized":"2024-04-16T19:35:46.210764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.191930+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:36:06.191930+0000","last_peered":"2024-04-16T19:36:06.191930+0000","last_clean":"2024-04-16T19:36:06.191930+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:36:06.191930+0000","last_undegraded":"2024-04-16T19:36:06.191930+0000","last_fullsized":"2024-04-16T19:36:06.191930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212786+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:36:11.212786+0000","last_peered":"2024-04-16T19:36:11.212786+0000","last_clean":"2024-04-16T19:36:11.212786+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:36:11.212786+0000","last_undegraded":"2024-04-16T19:36:11.212786+0000","last_fullsized":"2024-04-16T19:36:11.212786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'88","reported_seq":250,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197698+0000","last_change":"2024-04-16T17:47:35.319036+0000","last_active":"2024-04-16T19:35:06.197698+0000","last_peered":"2024-04-16T19:35:06.197698+0000","last_clean":"2024-04-16T19:35:06.197698+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:35:06.197698+0000","last_undegraded":"2024-04-16T19:35:06.197698+0000","last_fullsized":"2024-04-16T19:35:06.197698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.201845+0000","last_change":"2024-04-16T17:47:37.330847+0000","last_active":"2024-04-16T19:35:11.201845+0000","last_peered":"2024-04-16T19:35:11.201845+0000","last_clean":"2024-04-16T19:35:11.201845+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:35:11.201845+0000","last_undegraded":"2024-04-16T19:35:11.201845+0000","last_fullsized":"2024-04-16T19:35:11.201845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.511170+0000","last_change":"2024-04-16T19:36:25.511222+0000","last_active":"2024-04-16T19:36:25.511170+0000","last_peered":"2024-04-16T19:36:25.511170+0000","last_clean":"2024-04-16T19:36:25.511170+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:36:25.511170+0000","last_undegraded":"2024-04-16T19:36:25.511170+0000","last_fullsized":"2024-04-16T19:36:25.511170+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.511031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:41:57.452906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.200556+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:35:26.200556+0000","last_peered":"2024-04-16T19:35:26.200556+0000","last_clean":"2024-04-16T19:35:26.200556+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:35:26.200556+0000","last_undegraded":"2024-04-16T19:35:26.200556+0000","last_fullsized":"2024-04-16T19:35:26.200556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'102","reported_seq":1008,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.618748+0000","last_change":"2024-04-16T19:36:48.618887+0000","last_active":"2024-04-16T19:36:48.618748+0000","last_peered":"2024-04-16T19:36:48.618748+0000","last_clean":"2024-04-16T19:36:48.618748+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:36:48.618748+0000","last_undegraded":"2024-04-16T19:36:48.618748+0000","last_fullsized":"2024-04-16T19:36:48.618748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'102","last_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_deep_scrub":"53'102","last_deep_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.618582+0000","objects_scrubbed":31,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:54.031308+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7116,"num_omap_keys":28,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191392+0000","last_change":"2024-04-16T17:48:27.309014+0000","last_active":"2024-04-16T19:34:51.191392+0000","last_peered":"2024-04-16T19:34:51.191392+0000","last_clean":"2024-04-16T19:34:51.191392+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:34:51.191392+0000","last_undegraded":"2024-04-16T19:34:51.191392+0000","last_fullsized":"2024-04-16T19:34:51.191392+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'7","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.481640+0000","last_change":"2024-04-16T19:36:54.481710+0000","last_active":"2024-04-16T19:36:54.481640+0000","last_peered":"2024-04-16T19:36:54.481640+0000","last_clean":"2024-04-16T19:36:54.481640+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:36:54.481640+0000","last_undegraded":"2024-04-16T19:36:54.481640+0000","last_fullsized":"2024-04-16T19:36:54.481640+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.481476+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:28.035078+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.550467+0000","last_change":"2024-04-16T19:36:28.550520+0000","last_active":"2024-04-16T19:36:28.550467+0000","last_peered":"2024-04-16T19:36:28.550467+0000","last_clean":"2024-04-16T19:36:28.550467+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:36:28.550467+0000","last_undegraded":"2024-04-16T19:36:28.550467+0000","last_fullsized":"2024-04-16T19:36:28.550467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.550328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:52.280854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11393","reported_seq":61823,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.677407+0000","last_change":"2024-04-16T19:36:19.677565+0000","last_active":"2024-04-16T19:36:19.677407+0000","last_peered":"2024-04-16T19:36:19.677407+0000","last_clean":"2024-04-16T19:36:19.677407+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:36:19.677407+0000","last_undegraded":"2024-04-16T19:36:19.677407+0000","last_fullsized":"2024-04-16T19:36:19.677407+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11393","last_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_deep_scrub":"53'11393","last_deep_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.677264+0000","objects_scrubbed":23,"log_size":1693,"log_dups_size":3000,"ondisk_log_size":1693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:14.777503+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209607,"num_omap_keys":977,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'23","reported_seq":358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.461538+0000","last_change":"2024-04-16T19:36:18.461701+0000","last_active":"2024-04-16T19:36:18.461538+0000","last_peered":"2024-04-16T19:36:18.461538+0000","last_clean":"2024-04-16T19:36:18.461538+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:36:18.461538+0000","last_undegraded":"2024-04-16T19:36:18.461538+0000","last_fullsized":"2024-04-16T19:36:18.461538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.461393+0000","objects_scrubbed":11,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:24.888175+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210256+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:36:01.210256+0000","last_peered":"2024-04-16T19:36:01.210256+0000","last_clean":"2024-04-16T19:36:01.210256+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:36:01.210256+0000","last_undegraded":"2024-04-16T19:36:01.210256+0000","last_fullsized":"2024-04-16T19:36:01.210256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048372+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:36:17.048372+0000","last_peered":"2024-04-16T19:36:17.048372+0000","last_clean":"2024-04-16T19:36:17.048372+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:36:17.048372+0000","last_undegraded":"2024-04-16T19:36:17.048372+0000","last_fullsized":"2024-04-16T19:36:17.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213328+0000","last_change":"2024-04-16T17:47:33.296653+0000","last_active":"2024-04-16T19:35:56.213328+0000","last_peered":"2024-04-16T19:35:56.213328+0000","last_clean":"2024-04-16T19:35:56.213328+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:35:56.213328+0000","last_undegraded":"2024-04-16T19:35:56.213328+0000","last_fullsized":"2024-04-16T19:35:56.213328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'152","reported_seq":341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.200620+0000","last_change":"2024-04-16T17:47:35.317123+0000","last_active":"2024-04-16T19:35:06.200620+0000","last_peered":"2024-04-16T19:35:06.200620+0000","last_clean":"2024-04-16T19:35:06.200620+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:35:06.200620+0000","last_undegraded":"2024-04-16T19:35:06.200620+0000","last_fullsized":"2024-04-16T19:35:06.200620+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.656638+0000","last_change":"2024-04-16T19:36:53.656690+0000","last_active":"2024-04-16T19:36:53.656638+0000","last_peered":"2024-04-16T19:36:53.656638+0000","last_clean":"2024-04-16T19:36:53.656638+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:36:53.656638+0000","last_undegraded":"2024-04-16T19:36:53.656638+0000","last_fullsized":"2024-04-16T19:36:53.656638+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.656501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:12.757030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.426536+0000","last_change":"2024-04-16T19:36:35.426597+0000","last_active":"2024-04-16T19:36:35.426536+0000","last_peered":"2024-04-16T19:36:35.426536+0000","last_clean":"2024-04-16T19:36:35.426536+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:36:35.426536+0000","last_undegraded":"2024-04-16T19:36:35.426536+0000","last_fullsized":"2024-04-16T19:36:35.426536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.426385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:09.737500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201791+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:35:26.201791+0000","last_peered":"2024-04-16T19:35:26.201791+0000","last_clean":"2024-04-16T19:35:26.201791+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:35:26.201791+0000","last_undegraded":"2024-04-16T19:35:26.201791+0000","last_fullsized":"2024-04-16T19:35:26.201791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'143","reported_seq":1869,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.705923+0000","last_change":"2024-04-16T19:36:51.706017+0000","last_active":"2024-04-16T19:36:51.705923+0000","last_peered":"2024-04-16T19:36:51.705923+0000","last_clean":"2024-04-16T19:36:51.705923+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:36:51.705923+0000","last_undegraded":"2024-04-16T19:36:51.705923+0000","last_fullsized":"2024-04-16T19:36:51.705923+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'143","last_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_deep_scrub":"53'143","last_deep_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.705779+0000","objects_scrubbed":40,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:42.464983+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7446,"num_omap_keys":29,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.646565+0000","last_change":"2024-04-16T19:36:48.646617+0000","last_active":"2024-04-16T19:36:48.646565+0000","last_peered":"2024-04-16T19:36:48.646565+0000","last_clean":"2024-04-16T19:36:48.646565+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:36:48.646565+0000","last_undegraded":"2024-04-16T19:36:48.646565+0000","last_fullsized":"2024-04-16T19:36:48.646565+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.646427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:00.794914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'8","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.913793+0000","last_change":"2024-04-16T17:48:01.248531+0000","last_active":"2024-04-16T19:36:18.913793+0000","last_peered":"2024-04-16T19:36:18.913793+0000","last_clean":"2024-04-16T19:36:18.913793+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:36:18.913793+0000","last_undegraded":"2024-04-16T19:36:18.913793+0000","last_fullsized":"2024-04-16T19:36:18.913793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.416813+0000","last_change":"2024-04-16T19:36:55.416865+0000","last_active":"2024-04-16T19:36:55.416813+0000","last_peered":"2024-04-16T19:36:55.416813+0000","last_clean":"2024-04-16T19:36:55.416813+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:36:55.416813+0000","last_undegraded":"2024-04-16T19:36:55.416813+0000","last_fullsized":"2024-04-16T19:36:55.416813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.416624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:21.421511+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'206","reported_seq":440,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.574363+0000","last_change":"2024-04-16T19:36:35.574523+0000","last_active":"2024-04-16T19:36:35.574363+0000","last_peered":"2024-04-16T19:36:35.574363+0000","last_clean":"2024-04-16T19:36:35.574363+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:36:35.574363+0000","last_undegraded":"2024-04-16T19:36:35.574363+0000","last_fullsized":"2024-04-16T19:36:35.574363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'206","last_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_deep_scrub":"53'206","last_deep_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.574163+0000","objects_scrubbed":10,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:15.061721+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'20","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.646780+0000","last_change":"2024-04-16T19:36:30.646869+0000","last_active":"2024-04-16T19:36:30.646780+0000","last_peered":"2024-04-16T19:36:30.646780+0000","last_clean":"2024-04-16T19:36:30.646780+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:36:30.646780+0000","last_undegraded":"2024-04-16T19:36:30.646780+0000","last_fullsized":"2024-04-16T19:36:30.646780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.646698+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:37.151471+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194008+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:34:56.194008+0000","last_peered":"2024-04-16T19:34:56.194008+0000","last_clean":"2024-04-16T19:34:56.194008+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:34:56.194008+0000","last_undegraded":"2024-04-16T19:34:56.194008+0000","last_fullsized":"2024-04-16T19:34:56.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188408+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:34:51.188408+0000","last_peered":"2024-04-16T19:34:51.188408+0000","last_clean":"2024-04-16T19:34:51.188408+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:34:51.188408+0000","last_undegraded":"2024-04-16T19:34:51.188408+0000","last_fullsized":"2024-04-16T19:34:51.188408+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"48'5","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192321+0000","last_change":"2024-04-16T17:47:33.306460+0000","last_active":"2024-04-16T19:35:06.192321+0000","last_peered":"2024-04-16T19:35:06.192321+0000","last_clean":"2024-04-16T19:35:06.192321+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:35:06.192321+0000","last_undegraded":"2024-04-16T19:35:06.192321+0000","last_fullsized":"2024-04-16T19:35:06.192321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'104","reported_seq":278,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189800+0000","last_change":"2024-04-16T17:47:35.314274+0000","last_active":"2024-04-16T19:35:56.189800+0000","last_peered":"2024-04-16T19:35:56.189800+0000","last_clean":"2024-04-16T19:35:56.189800+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:35:56.189800+0000","last_undegraded":"2024-04-16T19:35:56.189800+0000","last_fullsized":"2024-04-16T19:35:56.189800+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.699354+0000","last_change":"2024-04-16T19:36:54.699404+0000","last_active":"2024-04-16T19:36:54.699354+0000","last_peered":"2024-04-16T19:36:54.699354+0000","last_clean":"2024-04-16T19:36:54.699354+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:36:54.699354+0000","last_undegraded":"2024-04-16T19:36:54.699354+0000","last_fullsized":"2024-04-16T19:36:54.699354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.699216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:21.320753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.350473+0000","last_change":"2024-04-16T19:36:42.350584+0000","last_active":"2024-04-16T19:36:42.350473+0000","last_peered":"2024-04-16T19:36:42.350473+0000","last_clean":"2024-04-16T19:36:42.350473+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:36:42.350473+0000","last_undegraded":"2024-04-16T19:36:42.350473+0000","last_fullsized":"2024-04-16T19:36:42.350473+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.350304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.971842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":46,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.214641+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:36:11.214641+0000","last_peered":"2024-04-16T19:36:11.214641+0000","last_clean":"2024-04-16T19:36:11.214641+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:36:11.214641+0000","last_undegraded":"2024-04-16T19:36:11.214641+0000","last_fullsized":"2024-04-16T19:36:11.214641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'68","reported_seq":883,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.195089+0000","last_change":"2024-04-16T17:48:34.343716+0000","last_active":"2024-04-16T19:34:56.195089+0000","last_peered":"2024-04-16T19:34:56.195089+0000","last_clean":"2024-04-16T19:34:56.195089+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:34:56.195089+0000","last_undegraded":"2024-04-16T19:34:56.195089+0000","last_fullsized":"2024-04-16T19:34:56.195089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.668585+0000","last_change":"2024-04-16T19:36:44.668638+0000","last_active":"2024-04-16T19:36:44.668585+0000","last_peered":"2024-04-16T19:36:44.668585+0000","last_clean":"2024-04-16T19:36:44.668585+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:36:44.668585+0000","last_undegraded":"2024-04-16T19:36:44.668585+0000","last_fullsized":"2024-04-16T19:36:44.668585+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.668449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:31.847082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"53'2","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.465959+0000","last_change":"2024-04-16T19:36:53.466069+0000","last_active":"2024-04-16T19:36:53.465959+0000","last_peered":"2024-04-16T19:36:53.465959+0000","last_clean":"2024-04-16T19:36:53.465959+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:36:53.465959+0000","last_undegraded":"2024-04-16T19:36:53.465959+0000","last_fullsized":"2024-04-16T19:36:53.465959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.465746+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:19.042353+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.452816+0000","last_change":"2024-04-16T19:36:41.452876+0000","last_active":"2024-04-16T19:36:41.452816+0000","last_peered":"2024-04-16T19:36:41.452816+0000","last_clean":"2024-04-16T19:36:41.452816+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:36:41.452816+0000","last_undegraded":"2024-04-16T19:36:41.452816+0000","last_fullsized":"2024-04-16T19:36:41.452816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.452676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:06.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'509","reported_seq":3967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635824+0000","last_change":"2024-04-16T19:36:20.636035+0000","last_active":"2024-04-16T19:36:20.635824+0000","last_peered":"2024-04-16T19:36:20.635824+0000","last_clean":"2024-04-16T19:36:20.635824+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:36:20.635824+0000","last_undegraded":"2024-04-16T19:36:20.635824+0000","last_fullsized":"2024-04-16T19:36:20.635824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635659+0000","objects_scrubbed":21,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:04.155514+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71921,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'34","reported_seq":243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.437126+0000","last_change":"2024-04-16T19:36:23.437306+0000","last_active":"2024-04-16T19:36:23.437126+0000","last_peered":"2024-04-16T19:36:23.437126+0000","last_clean":"2024-04-16T19:36:23.437126+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:36:23.437126+0000","last_undegraded":"2024-04-16T19:36:23.437126+0000","last_fullsized":"2024-04-16T19:36:23.437126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.436922+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:38.568868+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175524+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:34:56.175524+0000","last_peered":"2024-04-16T19:34:56.175524+0000","last_clean":"2024-04-16T19:34:56.175524+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:34:56.175524+0000","last_undegraded":"2024-04-16T19:34:56.175524+0000","last_fullsized":"2024-04-16T19:34:56.175524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192550+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:34:51.192550+0000","last_peered":"2024-04-16T19:34:51.192550+0000","last_clean":"2024-04-16T19:34:51.192550+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:34:51.192550+0000","last_undegraded":"2024-04-16T19:34:51.192550+0000","last_fullsized":"2024-04-16T19:34:51.192550+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"33'3","reported_seq":803,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197667+0000","last_change":"2024-04-16T17:47:33.297794+0000","last_active":"2024-04-16T19:35:06.197667+0000","last_peered":"2024-04-16T19:35:06.197667+0000","last_clean":"2024-04-16T19:35:06.197667+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:35:06.197667+0000","last_undegraded":"2024-04-16T19:35:06.197667+0000","last_fullsized":"2024-04-16T19:35:06.197667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'120","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.216918+0000","last_change":"2024-04-16T17:47:35.319657+0000","last_active":"2024-04-16T19:36:11.216918+0000","last_peered":"2024-04-16T19:36:11.216918+0000","last_clean":"2024-04-16T19:36:11.216918+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:36:11.216918+0000","last_undegraded":"2024-04-16T19:36:11.216918+0000","last_fullsized":"2024-04-16T19:36:11.216918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'12","reported_seq":1341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210179+0000","last_change":"2024-04-16T17:47:37.331215+0000","last_active":"2024-04-16T19:36:01.210179+0000","last_peered":"2024-04-16T19:36:01.210179+0000","last_clean":"2024-04-16T19:36:01.210179+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:36:01.210179+0000","last_undegraded":"2024-04-16T19:36:01.210179+0000","last_fullsized":"2024-04-16T19:36:01.210179+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.388068+0000","last_change":"2024-04-16T19:36:39.388173+0000","last_active":"2024-04-16T19:36:39.388068+0000","last_peered":"2024-04-16T19:36:39.388068+0000","last_clean":"2024-04-16T19:36:39.388068+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:36:39.388068+0000","last_undegraded":"2024-04-16T19:36:39.388068+0000","last_fullsized":"2024-04-16T19:36:39.388068+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.387834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:30.147039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188309+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:34:51.188309+0000","last_peered":"2024-04-16T19:34:51.188309+0000","last_clean":"2024-04-16T19:34:51.188309+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:34:51.188309+0000","last_undegraded":"2024-04-16T19:34:51.188309+0000","last_fullsized":"2024-04-16T19:34:51.188309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'123","reported_seq":1604,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.593801+0000","last_change":"2024-04-16T19:36:52.593899+0000","last_active":"2024-04-16T19:36:52.593801+0000","last_peered":"2024-04-16T19:36:52.593801+0000","last_clean":"2024-04-16T19:36:52.593801+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:36:52.593801+0000","last_undegraded":"2024-04-16T19:36:52.593801+0000","last_fullsized":"2024-04-16T19:36:52.593801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'123","last_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_deep_scrub":"53'123","last_deep_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.593639+0000","objects_scrubbed":37,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:52.173838+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5671,"num_omap_keys":23,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.668143+0000","last_change":"2024-04-16T19:36:53.668197+0000","last_active":"2024-04-16T19:36:53.668143+0000","last_peered":"2024-04-16T19:36:53.668143+0000","last_clean":"2024-04-16T19:36:53.668143+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:36:53.668143+0000","last_undegraded":"2024-04-16T19:36:53.668143+0000","last_fullsized":"2024-04-16T19:36:53.668143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.668006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:20:47.522228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'10","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.679540+0000","last_change":"2024-04-16T19:36:41.679612+0000","last_active":"2024-04-16T19:36:41.679540+0000","last_peered":"2024-04-16T19:36:41.679540+0000","last_clean":"2024-04-16T19:36:41.679540+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:36:41.679540+0000","last_undegraded":"2024-04-16T19:36:41.679540+0000","last_fullsized":"2024-04-16T19:36:41.679540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.679332+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:33.154520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'556","reported_seq":1963,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.395959+0000","last_change":"2024-04-16T19:36:44.396025+0000","last_active":"2024-04-16T19:36:44.395959+0000","last_peered":"2024-04-16T19:36:44.395959+0000","last_clean":"2024-04-16T19:36:44.395959+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:36:44.395959+0000","last_undegraded":"2024-04-16T19:36:44.395959+0000","last_fullsized":"2024-04-16T19:36:44.395959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.395808+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:46.739394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'508","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.543534+0000","last_change":"2024-04-16T19:36:25.543534+0000","last_active":"2024-04-16T19:36:25.543534+0000","last_peered":"2024-04-16T19:36:25.543534+0000","last_clean":"2024-04-16T19:36:25.543534+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:36:25.543534+0000","last_undegraded":"2024-04-16T19:36:25.543534+0000","last_fullsized":"2024-04-16T19:36:25.543534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.543268+0000","objects_scrubbed":16,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:59.596711+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71934,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'8","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.642341+0000","last_change":"2024-04-16T19:36:27.642341+0000","last_active":"2024-04-16T19:36:27.642341+0000","last_peered":"2024-04-16T19:36:27.642341+0000","last_clean":"2024-04-16T19:36:27.642341+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:36:27.642341+0000","last_undegraded":"2024-04-16T19:36:27.642341+0000","last_fullsized":"2024-04-16T19:36:27.642341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.642045+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:07.753490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189839+0000","last_change":"2024-04-16T17:47:23.266570+0000","last_active":"2024-04-16T19:34:56.189839+0000","last_peered":"2024-04-16T19:34:56.189839+0000","last_clean":"2024-04-16T19:34:56.189839+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:34:56.189839+0000","last_undegraded":"2024-04-16T19:34:56.189839+0000","last_fullsized":"2024-04-16T19:34:56.189839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191235+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:34:51.191235+0000","last_peered":"2024-04-16T19:34:51.191235+0000","last_clean":"2024-04-16T19:34:51.191235+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:34:51.191235+0000","last_undegraded":"2024-04-16T19:34:51.191235+0000","last_fullsized":"2024-04-16T19:34:51.191235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192192+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:35:06.192192+0000","last_peered":"2024-04-16T19:35:06.192192+0000","last_clean":"2024-04-16T19:35:06.192192+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:35:06.192192+0000","last_undegraded":"2024-04-16T19:35:06.192192+0000","last_fullsized":"2024-04-16T19:35:06.192192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'152","reported_seq":343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912598+0000","last_change":"2024-04-16T17:47:35.310214+0000","last_active":"2024-04-16T19:36:18.912598+0000","last_peered":"2024-04-16T19:36:18.912598+0000","last_clean":"2024-04-16T19:36:18.912598+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:36:18.912598+0000","last_undegraded":"2024-04-16T19:36:18.912598+0000","last_fullsized":"2024-04-16T19:36:18.912598+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.656216+0000","last_change":"2024-04-16T19:36:46.656320+0000","last_active":"2024-04-16T19:36:46.656216+0000","last_peered":"2024-04-16T19:36:46.656216+0000","last_clean":"2024-04-16T19:36:46.656216+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:36:46.656216+0000","last_undegraded":"2024-04-16T19:36:46.656216+0000","last_fullsized":"2024-04-16T19:36:46.656216+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.656048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:11.940546+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.451910+0000","last_change":"2024-04-16T19:36:37.452021+0000","last_active":"2024-04-16T19:36:37.451910+0000","last_peered":"2024-04-16T19:36:37.451910+0000","last_clean":"2024-04-16T19:36:37.451910+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:36:37.451910+0000","last_undegraded":"2024-04-16T19:36:37.451910+0000","last_fullsized":"2024-04-16T19:36:37.451910+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.451765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:52.163889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.174203+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:34:51.174203+0000","last_peered":"2024-04-16T19:34:51.174203+0000","last_clean":"2024-04-16T19:34:51.174203+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:34:51.174203+0000","last_undegraded":"2024-04-16T19:34:51.174203+0000","last_fullsized":"2024-04-16T19:34:51.174203+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'99","reported_seq":1432,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.210505+0000","last_change":"2024-04-16T17:48:34.341651+0000","last_active":"2024-04-16T19:35:56.210505+0000","last_peered":"2024-04-16T19:35:56.210505+0000","last_clean":"2024-04-16T19:35:56.210505+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:35:56.210505+0000","last_undegraded":"2024-04-16T19:35:56.210505+0000","last_fullsized":"2024-04-16T19:35:56.210505+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":79,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207296+0000","last_change":"2024-04-16T17:48:27.306641+0000","last_active":"2024-04-16T19:35:51.207296+0000","last_peered":"2024-04-16T19:35:51.207296+0000","last_clean":"2024-04-16T19:35:51.207296+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:35:51.207296+0000","last_undegraded":"2024-04-16T19:35:51.207296+0000","last_fullsized":"2024-04-16T19:35:51.207296+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:26.252757+0000","last_clean_scrub_stamp":"2024-04-16T17:48:26.252757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'11","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.624192+0000","last_change":"2024-04-16T19:36:55.624271+0000","last_active":"2024-04-16T19:36:55.624192+0000","last_peered":"2024-04-16T19:36:55.624192+0000","last_clean":"2024-04-16T19:36:55.624192+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:36:55.624192+0000","last_undegraded":"2024-04-16T19:36:55.624192+0000","last_fullsized":"2024-04-16T19:36:55.624192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.624051+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:11.910789+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'556","reported_seq":1911,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.421934+0000","last_change":"2024-04-16T19:36:38.421934+0000","last_active":"2024-04-16T19:36:38.421934+0000","last_peered":"2024-04-16T19:36:38.421934+0000","last_clean":"2024-04-16T19:36:38.421934+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:36:38.421934+0000","last_undegraded":"2024-04-16T19:36:38.421934+0000","last_fullsized":"2024-04-16T19:36:38.421934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.421654+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:18.533099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'250","reported_seq":489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.484262+0000","last_change":"2024-04-16T19:36:27.484420+0000","last_active":"2024-04-16T19:36:27.484262+0000","last_peered":"2024-04-16T19:36:27.484262+0000","last_clean":"2024-04-16T19:36:27.484262+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:36:27.484262+0000","last_undegraded":"2024-04-16T19:36:27.484262+0000","last_fullsized":"2024-04-16T19:36:27.484262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.484127+0000","objects_scrubbed":20,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:04.899387+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.205027+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:35:56.205027+0000","last_peered":"2024-04-16T19:35:56.205027+0000","last_clean":"2024-04-16T19:35:56.205027+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:35:56.205027+0000","last_undegraded":"2024-04-16T19:35:56.205027+0000","last_fullsized":"2024-04-16T19:35:56.205027+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189793+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:34:56.189793+0000","last_peered":"2024-04-16T19:34:56.189793+0000","last_clean":"2024-04-16T19:34:56.189793+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:34:56.189793+0000","last_undegraded":"2024-04-16T19:34:56.189793+0000","last_fullsized":"2024-04-16T19:34:56.189793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191305+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:34:51.191305+0000","last_peered":"2024-04-16T19:34:51.191305+0000","last_clean":"2024-04-16T19:34:51.191305+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:34:51.191305+0000","last_undegraded":"2024-04-16T19:34:51.191305+0000","last_fullsized":"2024-04-16T19:34:51.191305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.194844+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:35:06.194844+0000","last_peered":"2024-04-16T19:35:06.194844+0000","last_clean":"2024-04-16T19:35:06.194844+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:35:06.194844+0000","last_undegraded":"2024-04-16T19:35:06.194844+0000","last_fullsized":"2024-04-16T19:35:06.194844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'80","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586539+0000","last_change":"2024-04-16T17:47:35.303292+0000","last_active":"2024-04-16T19:36:16.586539+0000","last_peered":"2024-04-16T19:36:16.586539+0000","last_clean":"2024-04-16T19:36:16.586539+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:36:16.586539+0000","last_undegraded":"2024-04-16T19:36:16.586539+0000","last_fullsized":"2024-04-16T19:36:16.586539+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:36.192983+0000","last_change":"2024-04-16T17:47:37.331029+0000","last_active":"2024-04-16T19:34:36.192983+0000","last_peered":"2024-04-16T19:34:36.192983+0000","last_clean":"2024-04-16T19:34:36.192983+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:34:36.192983+0000","last_undegraded":"2024-04-16T19:34:36.192983+0000","last_fullsized":"2024-04-16T19:34:36.192983+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.477611+0000","last_change":"2024-04-16T19:36:26.477669+0000","last_active":"2024-04-16T19:36:26.477611+0000","last_peered":"2024-04-16T19:36:26.477611+0000","last_clean":"2024-04-16T19:36:26.477611+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:36:26.477611+0000","last_undegraded":"2024-04-16T19:36:26.477611+0000","last_fullsized":"2024-04-16T19:36:26.477611+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.477464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:27.657085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192426+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:34:51.192426+0000","last_peered":"2024-04-16T19:34:51.192426+0000","last_clean":"2024-04-16T19:34:51.192426+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:34:51.192426+0000","last_undegraded":"2024-04-16T19:34:51.192426+0000","last_fullsized":"2024-04-16T19:34:51.192426+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'126","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.542707+0000","last_change":"2024-04-16T19:36:48.542776+0000","last_active":"2024-04-16T19:36:48.542707+0000","last_peered":"2024-04-16T19:36:48.542707+0000","last_clean":"2024-04-16T19:36:48.542707+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:36:48.542707+0000","last_undegraded":"2024-04-16T19:36:48.542707+0000","last_fullsized":"2024-04-16T19:36:48.542707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.542572+0000","objects_scrubbed":30,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:02.471385+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6535,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.626702+0000","last_change":"2024-04-16T19:36:43.626754+0000","last_active":"2024-04-16T19:36:43.626702+0000","last_peered":"2024-04-16T19:36:43.626702+0000","last_clean":"2024-04-16T19:36:43.626702+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:36:43.626702+0000","last_undegraded":"2024-04-16T19:36:43.626702+0000","last_fullsized":"2024-04-16T19:36:43.626702+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.626565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:30.990969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'7","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.198997+0000","last_change":"2024-04-16T17:48:01.248401+0000","last_active":"2024-04-16T19:35:21.198997+0000","last_peered":"2024-04-16T19:35:21.198997+0000","last_clean":"2024-04-16T19:35:21.198997+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:35:21.198997+0000","last_undegraded":"2024-04-16T19:35:21.198997+0000","last_fullsized":"2024-04-16T19:35:21.198997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.913618+0000","last_change":"2024-04-16T17:47:59.199961+0000","last_active":"2024-04-16T19:36:18.913618+0000","last_peered":"2024-04-16T19:36:18.913618+0000","last_clean":"2024-04-16T19:36:18.913618+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:36:18.913618+0000","last_undegraded":"2024-04-16T19:36:18.913618+0000","last_fullsized":"2024-04-16T19:36:18.913618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:58.151336+0000","last_clean_scrub_stamp":"2024-04-16T17:47:58.151336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'141","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.454241+0000","last_change":"2024-04-16T19:36:28.454241+0000","last_active":"2024-04-16T19:36:28.454241+0000","last_peered":"2024-04-16T19:36:28.454241+0000","last_clean":"2024-04-16T19:36:28.454241+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:36:28.454241+0000","last_undegraded":"2024-04-16T19:36:28.454241+0000","last_fullsized":"2024-04-16T19:36:28.454241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'141","last_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_deep_scrub":"53'141","last_deep_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.453990+0000","objects_scrubbed":12,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:15.632623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.478104+0000","last_change":"2024-04-16T19:36:22.478173+0000","last_active":"2024-04-16T19:36:22.478104+0000","last_peered":"2024-04-16T19:36:22.478104+0000","last_clean":"2024-04-16T19:36:22.478104+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:36:22.478104+0000","last_undegraded":"2024-04-16T19:36:22.478104+0000","last_fullsized":"2024-04-16T19:36:22.478104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.477939+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:46.331273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586046+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:36:16.586046+0000","last_peered":"2024-04-16T19:36:16.586046+0000","last_clean":"2024-04-16T19:36:16.586046+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:36:16.586046+0000","last_undegraded":"2024-04-16T19:36:16.586046+0000","last_fullsized":"2024-04-16T19:36:16.586046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.188432+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:35:51.188432+0000","last_peered":"2024-04-16T19:35:51.188432+0000","last_clean":"2024-04-16T19:35:51.188432+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:35:51.188432+0000","last_undegraded":"2024-04-16T19:35:51.188432+0000","last_fullsized":"2024-04-16T19:35:51.188432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438194+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:36:18.438194+0000","last_peered":"2024-04-16T19:36:18.438194+0000","last_clean":"2024-04-16T19:36:18.438194+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:36:18.438194+0000","last_undegraded":"2024-04-16T19:36:18.438194+0000","last_fullsized":"2024-04-16T19:36:18.438194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'32","reported_seq":161,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.196282+0000","last_change":"2024-04-16T17:47:35.317051+0000","last_active":"2024-04-16T19:35:06.196282+0000","last_peered":"2024-04-16T19:35:06.196282+0000","last_clean":"2024-04-16T19:35:06.196282+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:35:06.196282+0000","last_undegraded":"2024-04-16T19:35:06.196282+0000","last_fullsized":"2024-04-16T19:35:06.196282+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.201733+0000","last_change":"2024-04-16T17:47:37.320351+0000","last_active":"2024-04-16T19:35:11.201733+0000","last_peered":"2024-04-16T19:35:11.201733+0000","last_clean":"2024-04-16T19:35:11.201733+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:35:11.201733+0000","last_undegraded":"2024-04-16T19:35:11.201733+0000","last_fullsized":"2024-04-16T19:35:11.201733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.360832+0000","last_change":"2024-04-16T19:36:41.360886+0000","last_active":"2024-04-16T19:36:41.360832+0000","last_peered":"2024-04-16T19:36:41.360832+0000","last_clean":"2024-04-16T19:36:41.360832+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:36:41.360832+0000","last_undegraded":"2024-04-16T19:36:41.360832+0000","last_fullsized":"2024-04-16T19:36:41.360832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.360675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.872893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.193772+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:34:51.193772+0000","last_peered":"2024-04-16T19:34:51.193772+0000","last_clean":"2024-04-16T19:34:51.193772+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:34:51.193772+0000","last_undegraded":"2024-04-16T19:34:51.193772+0000","last_fullsized":"2024-04-16T19:34:51.193772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'155","reported_seq":1489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.495554+0000","last_change":"2024-04-16T19:36:52.495627+0000","last_active":"2024-04-16T19:36:52.495554+0000","last_peered":"2024-04-16T19:36:52.495554+0000","last_clean":"2024-04-16T19:36:52.495554+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:36:52.495554+0000","last_undegraded":"2024-04-16T19:36:52.495554+0000","last_fullsized":"2024-04-16T19:36:52.495554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'155","last_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_deep_scrub":"53'155","last_deep_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.495389+0000","objects_scrubbed":29,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.497334+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8801,"num_omap_keys":37,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.641083+0000","last_change":"2024-04-16T19:36:38.641135+0000","last_active":"2024-04-16T19:36:38.641083+0000","last_peered":"2024-04-16T19:36:38.641083+0000","last_clean":"2024-04-16T19:36:38.641083+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:36:38.641083+0000","last_undegraded":"2024-04-16T19:36:38.641083+0000","last_fullsized":"2024-04-16T19:36:38.641083+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.640946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:53.196611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'11","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.644638+0000","last_change":"2024-04-16T19:36:54.644719+0000","last_active":"2024-04-16T19:36:54.644638+0000","last_peered":"2024-04-16T19:36:54.644638+0000","last_clean":"2024-04-16T19:36:54.644638+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:36:54.644638+0000","last_undegraded":"2024-04-16T19:36:54.644638+0000","last_fullsized":"2024-04-16T19:36:54.644638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.644499+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:20.331700+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.384126+0000","last_change":"2024-04-16T19:36:47.384178+0000","last_active":"2024-04-16T19:36:47.384126+0000","last_peered":"2024-04-16T19:36:47.384126+0000","last_clean":"2024-04-16T19:36:47.384126+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:36:47.384126+0000","last_undegraded":"2024-04-16T19:36:47.384126+0000","last_fullsized":"2024-04-16T19:36:47.384126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.383981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:49.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11080","reported_seq":18169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.523528+0000","last_change":"2024-04-16T19:36:22.523528+0000","last_active":"2024-04-16T19:36:22.523528+0000","last_peered":"2024-04-16T19:36:22.523528+0000","last_clean":"2024-04-16T19:36:22.523528+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:36:22.523528+0000","last_undegraded":"2024-04-16T19:36:22.523528+0000","last_fullsized":"2024-04-16T19:36:22.523528+0000","mapping_epoch":34,"log_start":"53'9300","ondisk_log_start":"53'9300","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11080","last_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_deep_scrub":"53'11080","last_deep_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.523321+0000","objects_scrubbed":25,"log_size":1780,"log_dups_size":3000,"ondisk_log_size":1780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:02.010878+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'10","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.675610+0000","last_change":"2024-04-16T19:36:33.675610+0000","last_active":"2024-04-16T19:36:33.675610+0000","last_peered":"2024-04-16T19:36:33.675610+0000","last_clean":"2024-04-16T19:36:33.675610+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:36:33.675610+0000","last_undegraded":"2024-04-16T19:36:33.675610+0000","last_fullsized":"2024-04-16T19:36:33.675610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.675369+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:17.195224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.216640+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:36:11.216640+0000","last_peered":"2024-04-16T19:36:11.216640+0000","last_clean":"2024-04-16T19:36:11.216640+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:36:11.216640+0000","last_undegraded":"2024-04-16T19:36:11.216640+0000","last_fullsized":"2024-04-16T19:36:11.216640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205439+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:35:41.205439+0000","last_peered":"2024-04-16T19:35:41.205439+0000","last_clean":"2024-04-16T19:35:41.205439+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:35:41.205439+0000","last_undegraded":"2024-04-16T19:35:41.205439+0000","last_fullsized":"2024-04-16T19:35:41.205439+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048951+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:36:17.048951+0000","last_peered":"2024-04-16T19:36:17.048951+0000","last_clean":"2024-04-16T19:36:17.048951+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:36:17.048951+0000","last_undegraded":"2024-04-16T19:36:17.048951+0000","last_fullsized":"2024-04-16T19:36:17.048951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'80","reported_seq":233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192236+0000","last_change":"2024-04-16T17:47:35.308166+0000","last_active":"2024-04-16T19:35:06.192236+0000","last_peered":"2024-04-16T19:35:06.192236+0000","last_clean":"2024-04-16T19:35:06.192236+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:35:06.192236+0000","last_undegraded":"2024-04-16T19:35:06.192236+0000","last_fullsized":"2024-04-16T19:35:06.192236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:34.273829+0000","last_clean_scrub_stamp":"2024-04-16T17:47:34.273829+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":104,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.197590+0000","last_change":"2024-04-16T17:47:37.321988+0000","last_active":"2024-04-16T19:35:11.197590+0000","last_peered":"2024-04-16T19:35:11.197590+0000","last_clean":"2024-04-16T19:35:11.197590+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:35:11.197590+0000","last_undegraded":"2024-04-16T19:35:11.197590+0000","last_fullsized":"2024-04-16T19:35:11.197590+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:36.289910+0000","last_clean_scrub_stamp":"2024-04-16T17:47:36.289910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.560046+0000","last_change":"2024-04-16T19:36:49.560147+0000","last_active":"2024-04-16T19:36:49.560046+0000","last_peered":"2024-04-16T19:36:49.560046+0000","last_clean":"2024-04-16T19:36:49.560046+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:36:49.560046+0000","last_undegraded":"2024-04-16T19:36:49.560046+0000","last_fullsized":"2024-04-16T19:36:49.560046+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.559817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:52.415984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'31","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.493564+0000","last_change":"2024-04-16T19:36:46.493705+0000","last_active":"2024-04-16T19:36:46.493564+0000","last_peered":"2024-04-16T19:36:46.493564+0000","last_clean":"2024-04-16T19:36:46.493564+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:36:46.493564+0000","last_undegraded":"2024-04-16T19:36:46.493564+0000","last_fullsized":"2024-04-16T19:36:46.493564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'31","last_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_deep_scrub":"53'31","last_deep_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.493417+0000","objects_scrubbed":9,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:52.920199+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189771+0000","last_change":"2024-04-16T17:47:32.278320+0000","last_active":"2024-04-16T19:35:56.189771+0000","last_peered":"2024-04-16T19:35:56.189771+0000","last_clean":"2024-04-16T19:35:56.189771+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:35:56.189771+0000","last_undegraded":"2024-04-16T19:35:56.189771+0000","last_fullsized":"2024-04-16T19:35:56.189771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200847+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:35:41.200847+0000","last_peered":"2024-04-16T19:35:41.200847+0000","last_clean":"2024-04-16T19:35:41.200847+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:35:41.200847+0000","last_undegraded":"2024-04-16T19:35:41.200847+0000","last_fullsized":"2024-04-16T19:35:41.200847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'6","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.472692+0000","last_change":"2024-04-16T19:36:34.472767+0000","last_active":"2024-04-16T19:36:34.472692+0000","last_peered":"2024-04-16T19:36:34.472692+0000","last_clean":"2024-04-16T19:36:34.472692+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:36:34.472692+0000","last_undegraded":"2024-04-16T19:36:34.472692+0000","last_fullsized":"2024-04-16T19:36:34.472692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.472561+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:21.836965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.208030+0000","last_change":"2024-04-16T17:47:23.260359+0000","last_active":"2024-04-16T19:35:46.208030+0000","last_peered":"2024-04-16T19:35:46.208030+0000","last_clean":"2024-04-16T19:35:46.208030+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:35:46.208030+0000","last_undegraded":"2024-04-16T19:35:46.208030+0000","last_fullsized":"2024-04-16T19:35:46.208030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.185828+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:35:41.185828+0000","last_peered":"2024-04-16T19:35:41.185828+0000","last_clean":"2024-04-16T19:35:41.185828+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:35:41.185828+0000","last_undegraded":"2024-04-16T19:35:41.185828+0000","last_fullsized":"2024-04-16T19:35:41.185828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'10","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.445631+0000","last_change":"2024-04-16T19:36:30.445631+0000","last_active":"2024-04-16T19:36:30.445631+0000","last_peered":"2024-04-16T19:36:30.445631+0000","last_clean":"2024-04-16T19:36:30.445631+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:36:30.445631+0000","last_undegraded":"2024-04-16T19:36:30.445631+0000","last_fullsized":"2024-04-16T19:36:30.445631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.445415+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:00.551163+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206084+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:35:46.206084+0000","last_peered":"2024-04-16T19:35:46.206084+0000","last_clean":"2024-04-16T19:35:46.206084+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:35:46.206084+0000","last_undegraded":"2024-04-16T19:35:46.206084+0000","last_fullsized":"2024-04-16T19:35:46.206084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209219+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:35:41.209219+0000","last_peered":"2024-04-16T19:35:41.209219+0000","last_clean":"2024-04-16T19:35:41.209219+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:35:41.209219+0000","last_undegraded":"2024-04-16T19:35:41.209219+0000","last_fullsized":"2024-04-16T19:35:41.209219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'12","reported_seq":248,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.464290+0000","last_change":"2024-04-16T19:36:34.464387+0000","last_active":"2024-04-16T19:36:34.464290+0000","last_peered":"2024-04-16T19:36:34.464290+0000","last_clean":"2024-04-16T19:36:34.464290+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:36:34.464290+0000","last_undegraded":"2024-04-16T19:36:34.464290+0000","last_fullsized":"2024-04-16T19:36:34.464290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.464153+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:03.336022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207947+0000","last_change":"2024-04-16T17:47:23.261639+0000","last_active":"2024-04-16T19:35:46.207947+0000","last_peered":"2024-04-16T19:35:46.207947+0000","last_clean":"2024-04-16T19:35:46.207947+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:35:46.207947+0000","last_undegraded":"2024-04-16T19:35:46.207947+0000","last_fullsized":"2024-04-16T19:35:46.207947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.213866+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:36:16.213866+0000","last_peered":"2024-04-16T19:36:16.213866+0000","last_clean":"2024-04-16T19:36:16.213866+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:36:16.213866+0000","last_undegraded":"2024-04-16T19:36:16.213866+0000","last_fullsized":"2024-04-16T19:36:16.213866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"53'6","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.528043+0000","last_change":"2024-04-16T19:36:28.528121+0000","last_active":"2024-04-16T19:36:28.528043+0000","last_peered":"2024-04-16T19:36:28.528043+0000","last_clean":"2024-04-16T19:36:28.528043+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:36:28.528043+0000","last_undegraded":"2024-04-16T19:36:28.528043+0000","last_fullsized":"2024-04-16T19:36:28.528043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.527896+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:25.132224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194347+0000","last_change":"2024-04-16T17:47:23.253450+0000","last_active":"2024-04-16T19:36:16.194347+0000","last_peered":"2024-04-16T19:36:16.194347+0000","last_clean":"2024-04-16T19:36:16.194347+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:36:16.194347+0000","last_undegraded":"2024-04-16T19:36:16.194347+0000","last_fullsized":"2024-04-16T19:36:16.194347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205237+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:35:41.205237+0000","last_peered":"2024-04-16T19:35:41.205237+0000","last_clean":"2024-04-16T19:35:41.205237+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:35:41.205237+0000","last_undegraded":"2024-04-16T19:35:41.205237+0000","last_fullsized":"2024-04-16T19:35:41.205237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"53'2","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.582380+0000","last_change":"2024-04-16T19:36:26.582466+0000","last_active":"2024-04-16T19:36:26.582380+0000","last_peered":"2024-04-16T19:36:26.582380+0000","last_clean":"2024-04-16T19:36:26.582380+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:36:26.582380+0000","last_undegraded":"2024-04-16T19:36:26.582380+0000","last_fullsized":"2024-04-16T19:36:26.582380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.582238+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:10.024512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202168+0000","last_change":"2024-04-16T17:47:23.257600+0000","last_active":"2024-04-16T19:35:46.202168+0000","last_peered":"2024-04-16T19:35:46.202168+0000","last_clean":"2024-04-16T19:35:46.202168+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:35:46.202168+0000","last_undegraded":"2024-04-16T19:35:46.202168+0000","last_fullsized":"2024-04-16T19:35:46.202168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206571+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:35:41.206571+0000","last_peered":"2024-04-16T19:35:41.206571+0000","last_clean":"2024-04-16T19:35:41.206571+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:35:41.206571+0000","last_undegraded":"2024-04-16T19:35:41.206571+0000","last_fullsized":"2024-04-16T19:35:41.206571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'46","reported_seq":266,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.466241+0000","last_change":"2024-04-16T19:36:19.466346+0000","last_active":"2024-04-16T19:36:19.466241+0000","last_peered":"2024-04-16T19:36:19.466241+0000","last_clean":"2024-04-16T19:36:19.466241+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:36:19.466241+0000","last_undegraded":"2024-04-16T19:36:19.466241+0000","last_fullsized":"2024-04-16T19:36:19.466241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'46","last_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_deep_scrub":"53'46","last_deep_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.466115+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:05.816236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189483+0000","last_change":"2024-04-16T17:47:32.281080+0000","last_active":"2024-04-16T19:35:56.189483+0000","last_peered":"2024-04-16T19:35:56.189483+0000","last_clean":"2024-04-16T19:35:56.189483+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:35:56.189483+0000","last_undegraded":"2024-04-16T19:35:56.189483+0000","last_fullsized":"2024-04-16T19:35:56.189483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209423+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:35:41.209423+0000","last_peered":"2024-04-16T19:35:41.209423+0000","last_clean":"2024-04-16T19:35:41.209423+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:35:41.209423+0000","last_undegraded":"2024-04-16T19:35:41.209423+0000","last_fullsized":"2024-04-16T19:35:41.209423+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204923+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:35:56.204923+0000","last_peered":"2024-04-16T19:35:56.204923+0000","last_clean":"2024-04-16T19:35:56.204923+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:35:56.204923+0000","last_undegraded":"2024-04-16T19:35:56.204923+0000","last_fullsized":"2024-04-16T19:35:56.204923+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210401+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:35:46.210401+0000","last_peered":"2024-04-16T19:35:46.210401+0000","last_clean":"2024-04-16T19:35:46.210401+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:35:46.210401+0000","last_undegraded":"2024-04-16T19:35:46.210401+0000","last_fullsized":"2024-04-16T19:35:46.210401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186120+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:35:41.186120+0000","last_peered":"2024-04-16T19:35:41.186120+0000","last_clean":"2024-04-16T19:35:41.186120+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:35:41.186120+0000","last_undegraded":"2024-04-16T19:35:41.186120+0000","last_fullsized":"2024-04-16T19:35:41.186120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'21","reported_seq":207,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.453307+0000","last_change":"2024-04-16T19:36:21.453405+0000","last_active":"2024-04-16T19:36:21.453307+0000","last_peered":"2024-04-16T19:36:21.453307+0000","last_clean":"2024-04-16T19:36:21.453307+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:36:21.453307+0000","last_undegraded":"2024-04-16T19:36:21.453307+0000","last_fullsized":"2024-04-16T19:36:21.453307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.453170+0000","objects_scrubbed":6,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:26.892395+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.191534+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:34:46.191534+0000","last_peered":"2024-04-16T19:34:46.191534+0000","last_clean":"2024-04-16T19:34:46.191534+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:34:46.191534+0000","last_undegraded":"2024-04-16T19:34:46.191534+0000","last_fullsized":"2024-04-16T19:34:46.191534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.210981+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:36:06.210981+0000","last_peered":"2024-04-16T19:36:06.210981+0000","last_clean":"2024-04-16T19:36:06.210981+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:36:06.210981+0000","last_undegraded":"2024-04-16T19:36:06.210981+0000","last_fullsized":"2024-04-16T19:36:06.210981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'26","reported_seq":334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.543887+0000","last_change":"2024-04-16T19:36:21.543887+0000","last_active":"2024-04-16T19:36:21.543887+0000","last_peered":"2024-04-16T19:36:21.543887+0000","last_clean":"2024-04-16T19:36:21.543887+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:36:21.543887+0000","last_undegraded":"2024-04-16T19:36:21.543887+0000","last_fullsized":"2024-04-16T19:36:21.543887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'26","last_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_deep_scrub":"53'26","last_deep_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.543655+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:47.643023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:46.187361+0000","last_change":"2024-04-16T17:47:23.264252+0000","last_active":"2024-04-16T19:34:46.187361+0000","last_peered":"2024-04-16T19:34:46.187361+0000","last_clean":"2024-04-16T19:34:46.187361+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:34:46.187361+0000","last_undegraded":"2024-04-16T19:34:46.187361+0000","last_fullsized":"2024-04-16T19:34:46.187361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200035+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:35:21.200035+0000","last_peered":"2024-04-16T19:35:21.200035+0000","last_clean":"2024-04-16T19:35:21.200035+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:35:21.200035+0000","last_undegraded":"2024-04-16T19:35:21.200035+0000","last_fullsized":"2024-04-16T19:35:21.200035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'2","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.488252+0000","last_change":"2024-04-16T19:36:22.488332+0000","last_active":"2024-04-16T19:36:22.488252+0000","last_peered":"2024-04-16T19:36:22.488252+0000","last_clean":"2024-04-16T19:36:22.488252+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:36:22.488252+0000","last_undegraded":"2024-04-16T19:36:22.488252+0000","last_fullsized":"2024-04-16T19:36:22.488252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.488109+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:37.620055+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.189679+0000","last_change":"2024-04-16T17:47:23.265700+0000","last_active":"2024-04-16T19:35:56.189679+0000","last_peered":"2024-04-16T19:35:56.189679+0000","last_clean":"2024-04-16T19:35:56.189679+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:35:56.189679+0000","last_undegraded":"2024-04-16T19:35:56.189679+0000","last_fullsized":"2024-04-16T19:35:56.189679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199116+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:35:21.199116+0000","last_peered":"2024-04-16T19:35:21.199116+0000","last_clean":"2024-04-16T19:35:21.199116+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:35:21.199116+0000","last_undegraded":"2024-04-16T19:35:21.199116+0000","last_fullsized":"2024-04-16T19:35:21.199116+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'22","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.501837+0000","last_change":"2024-04-16T19:36:47.501923+0000","last_active":"2024-04-16T19:36:47.501837+0000","last_peered":"2024-04-16T19:36:47.501837+0000","last_clean":"2024-04-16T19:36:47.501837+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:36:47.501837+0000","last_undegraded":"2024-04-16T19:36:47.501837+0000","last_fullsized":"2024-04-16T19:36:47.501837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.501703+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:52.574907+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.205594+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:35:26.205594+0000","last_peered":"2024-04-16T19:35:26.205594+0000","last_clean":"2024-04-16T19:35:26.205594+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:35:26.205594+0000","last_undegraded":"2024-04-16T19:35:26.205594+0000","last_fullsized":"2024-04-16T19:35:26.205594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200185+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:35:21.200185+0000","last_peered":"2024-04-16T19:35:21.200185+0000","last_clean":"2024-04-16T19:35:21.200185+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:35:21.200185+0000","last_undegraded":"2024-04-16T19:35:21.200185+0000","last_fullsized":"2024-04-16T19:35:21.200185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.568771+0000","last_change":"2024-04-16T19:36:22.568861+0000","last_active":"2024-04-16T19:36:22.568771+0000","last_peered":"2024-04-16T19:36:22.568771+0000","last_clean":"2024-04-16T19:36:22.568771+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:36:22.568771+0000","last_undegraded":"2024-04-16T19:36:22.568771+0000","last_fullsized":"2024-04-16T19:36:22.568771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.568637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:27.641841+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197209+0000","last_change":"2024-04-16T17:47:23.265903+0000","last_active":"2024-04-16T19:35:26.197209+0000","last_peered":"2024-04-16T19:35:26.197209+0000","last_clean":"2024-04-16T19:35:26.197209+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:35:26.197209+0000","last_undegraded":"2024-04-16T19:35:26.197209+0000","last_fullsized":"2024-04-16T19:35:26.197209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204436+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:35:21.204436+0000","last_peered":"2024-04-16T19:35:21.204436+0000","last_clean":"2024-04-16T19:35:21.204436+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:35:21.204436+0000","last_undegraded":"2024-04-16T19:35:21.204436+0000","last_fullsized":"2024-04-16T19:35:21.204436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"53'4","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.473816+0000","last_change":"2024-04-16T19:36:43.473937+0000","last_active":"2024-04-16T19:36:43.473816+0000","last_peered":"2024-04-16T19:36:43.473816+0000","last_clean":"2024-04-16T19:36:43.473816+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:36:43.473816+0000","last_undegraded":"2024-04-16T19:36:43.473816+0000","last_fullsized":"2024-04-16T19:36:43.473816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.473655+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:59.715799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201494+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:35:26.201494+0000","last_peered":"2024-04-16T19:35:26.201494+0000","last_clean":"2024-04-16T19:35:26.201494+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:35:26.201494+0000","last_undegraded":"2024-04-16T19:35:26.201494+0000","last_fullsized":"2024-04-16T19:35:26.201494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213227+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:35:56.213227+0000","last_peered":"2024-04-16T19:35:56.213227+0000","last_clean":"2024-04-16T19:35:56.213227+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:35:56.213227+0000","last_undegraded":"2024-04-16T19:35:56.213227+0000","last_fullsized":"2024-04-16T19:35:56.213227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'6","reported_seq":182,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.607311+0000","last_change":"2024-04-16T19:36:19.607390+0000","last_active":"2024-04-16T19:36:19.607311+0000","last_peered":"2024-04-16T19:36:19.607311+0000","last_clean":"2024-04-16T19:36:19.607311+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:36:19.607311+0000","last_undegraded":"2024-04-16T19:36:19.607311+0000","last_fullsized":"2024-04-16T19:36:19.607311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.607181+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:07:42.514098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.202288+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:35:26.202288+0000","last_peered":"2024-04-16T19:35:26.202288+0000","last_clean":"2024-04-16T19:35:26.202288+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:35:26.202288+0000","last_undegraded":"2024-04-16T19:35:26.202288+0000","last_fullsized":"2024-04-16T19:35:26.202288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.171926+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:34:41.171926+0000","last_peered":"2024-04-16T19:34:41.171926+0000","last_clean":"2024-04-16T19:34:41.171926+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:34:41.171926+0000","last_undegraded":"2024-04-16T19:34:41.171926+0000","last_fullsized":"2024-04-16T19:34:41.171926+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'12","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498647+0000","last_change":"2024-04-16T19:36:19.498750+0000","last_active":"2024-04-16T19:36:19.498647+0000","last_peered":"2024-04-16T19:36:19.498647+0000","last_clean":"2024-04-16T19:36:19.498647+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:36:19.498647+0000","last_undegraded":"2024-04-16T19:36:19.498647+0000","last_fullsized":"2024-04-16T19:36:19.498647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498508+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:33.427321+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978420+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:36:17.978420+0000","last_peered":"2024-04-16T19:36:17.978420+0000","last_clean":"2024-04-16T19:36:17.978420+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:36:17.978420+0000","last_undegraded":"2024-04-16T19:36:17.978420+0000","last_fullsized":"2024-04-16T19:36:17.978420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.188922+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:34:41.188922+0000","last_peered":"2024-04-16T19:34:41.188922+0000","last_clean":"2024-04-16T19:34:41.188922+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:34:41.188922+0000","last_undegraded":"2024-04-16T19:34:41.188922+0000","last_fullsized":"2024-04-16T19:34:41.188922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'16","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.449658+0000","last_change":"2024-04-16T19:36:28.449751+0000","last_active":"2024-04-16T19:36:28.449658+0000","last_peered":"2024-04-16T19:36:28.449658+0000","last_clean":"2024-04-16T19:36:28.449658+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:36:28.449658+0000","last_undegraded":"2024-04-16T19:36:28.449658+0000","last_fullsized":"2024-04-16T19:36:28.449658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.449522+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:19.924709+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.194112+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:34:41.194112+0000","last_peered":"2024-04-16T19:34:41.194112+0000","last_clean":"2024-04-16T19:34:41.194112+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:34:41.194112+0000","last_undegraded":"2024-04-16T19:34:41.194112+0000","last_fullsized":"2024-04-16T19:34:41.194112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586303+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:36:16.586303+0000","last_peered":"2024-04-16T19:36:16.586303+0000","last_clean":"2024-04-16T19:36:16.586303+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:36:16.586303+0000","last_undegraded":"2024-04-16T19:36:16.586303+0000","last_fullsized":"2024-04-16T19:36:16.586303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'20","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.594397+0000","last_change":"2024-04-16T19:36:27.594512+0000","last_active":"2024-04-16T19:36:27.594397+0000","last_peered":"2024-04-16T19:36:27.594397+0000","last_clean":"2024-04-16T19:36:27.594397+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:36:27.594397+0000","last_undegraded":"2024-04-16T19:36:27.594397+0000","last_fullsized":"2024-04-16T19:36:27.594397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.594261+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:01.851085+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:41.191347+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:34:41.191347+0000","last_peered":"2024-04-16T19:34:41.191347+0000","last_clean":"2024-04-16T19:34:41.191347+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:34:41.191347+0000","last_undegraded":"2024-04-16T19:34:41.191347+0000","last_fullsized":"2024-04-16T19:34:41.191347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212640+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:36:11.212640+0000","last_peered":"2024-04-16T19:36:11.212640+0000","last_clean":"2024-04-16T19:36:11.212640+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:36:11.212640+0000","last_undegraded":"2024-04-16T19:36:11.212640+0000","last_fullsized":"2024-04-16T19:36:11.212640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":49130,"num_omap_keys":198,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3067,"ondisk_log_size":3067,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3711,"ondisk_log_size":3711,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2422969,"num_omap_keys":10622,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27854,"ondisk_log_size":27854,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4469,"ondisk_log_size":4469,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9817,"num_omap_keys":62,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":546,"ondisk_log_size":546,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641584,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46584,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811016,"statfs":{"total":107374182400,"available":107326480384,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641583,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641583,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":83292,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":104774308,"statfs":{"total":107374182400,"available":107288891392,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641583,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":46572,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":104811028,"statfs":{"total":107374182400,"available":107326492672,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641584,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":59568,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":104798032,"statfs":{"total":107374182400,"available":107313184768,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641584,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":32700,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":104824900,"statfs":{"total":107374182400,"available":107340697600,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:37:00.394 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.394 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.395 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.395 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.395 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.395 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.395 INFO:tasks.ceph:pgid 9.36 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.395 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.396 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.396 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.396 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.396 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.396 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.397 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.397 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.397 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.397 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.397 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.397 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.398 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.398 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.398 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.398 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.398 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.399 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.399 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.399 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.399 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.399 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.399 INFO:tasks.ceph:pgid 15.1f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.400 INFO:tasks.ceph:pgid 14.1e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.400 INFO:tasks.ceph:pgid 13.1d last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.400 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.400 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.400 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.400 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.401 INFO:tasks.ceph:pgid 15.1e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.401 INFO:tasks.ceph:pgid 14.1f last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.401 INFO:tasks.ceph:pgid 13.1c last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.401 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.401 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.402 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.402 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.402 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.402 INFO:tasks.ceph:pgid 15.1d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.402 INFO:tasks.ceph:pgid 14.1c last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.403 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.403 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.403 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.403 INFO:tasks.ceph:pgid 15.1c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.403 INFO:tasks.ceph:pgid 14.1d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.403 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.404 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.404 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.404 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.404 INFO:tasks.ceph:pgid 15.3 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.404 INFO:tasks.ceph:pgid 14.2 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.405 INFO:tasks.ceph:pgid 11.7 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.405 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.405 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.405 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.405 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.405 INFO:tasks.ceph:pgid 15.2 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.406 INFO:tasks.ceph:pgid 14.3 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.406 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.406 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.406 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.406 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.407 INFO:tasks.ceph:pgid 15.1 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.407 INFO:tasks.ceph:pgid 14.0 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.407 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.407 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.407 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.408 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.408 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.408 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.408 INFO:tasks.ceph:pgid 15.0 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.408 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.409 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.409 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.409 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.409 INFO:tasks.ceph:pgid 15.7 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.409 INFO:tasks.ceph:pgid 14.6 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.410 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.410 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.410 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.410 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.410 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.410 INFO:tasks.ceph:pgid 15.6 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.411 INFO:tasks.ceph:pgid 14.7 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.411 INFO:tasks.ceph:pgid 13.4 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.411 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.411 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.411 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.411 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.412 INFO:tasks.ceph:pgid 15.5 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.412 INFO:tasks.ceph:pgid 14.4 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.412 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.412 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.412 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.413 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.413 INFO:tasks.ceph:pgid 15.4 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.413 INFO:tasks.ceph:pgid 14.5 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.413 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.413 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.414 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.414 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.414 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.414 INFO:tasks.ceph:pgid 15.e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.414 INFO:tasks.ceph:pgid 14.f last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.414 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.415 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.415 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.415 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.415 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.415 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.416 INFO:tasks.ceph:pgid 15.f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.416 INFO:tasks.ceph:pgid 14.e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.416 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.416 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.416 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.416 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.417 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.417 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.417 INFO:tasks.ceph:pgid 15.c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.417 INFO:tasks.ceph:pgid 14.d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.417 INFO:tasks.ceph:pgid 13.e last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.418 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.418 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.418 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.418 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.418 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.419 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.419 INFO:tasks.ceph:pgid 15.d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.419 INFO:tasks.ceph:pgid 14.c last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.419 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.419 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.420 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.420 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-16T17:46:37.490454+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=37, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.420 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.420 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.420 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.420 INFO:tasks.ceph:pgid 15.a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.421 INFO:tasks.ceph:pgid 14.b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.421 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.421 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.421 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.421 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.422 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.422 INFO:tasks.ceph:pgid 15.b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.422 INFO:tasks.ceph:pgid 14.a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.422 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.422 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.423 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.423 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.423 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.423 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.423 INFO:tasks.ceph:pgid 15.8 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.423 INFO:tasks.ceph:pgid 14.9 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.424 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.424 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.424 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.424 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.424 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.425 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.425 INFO:tasks.ceph:pgid 15.9 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.425 INFO:tasks.ceph:pgid 14.8 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.425 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.425 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.426 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.426 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.426 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.426 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.426 INFO:tasks.ceph:pgid 15.18 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.426 INFO:tasks.ceph:pgid 14.19 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.427 INFO:tasks.ceph:pgid 9.1e last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.427 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.427 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.427 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.427 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.428 INFO:tasks.ceph:pgid 15.19 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.428 INFO:tasks.ceph:pgid 14.18 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.428 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.428 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.428 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.428 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.429 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.429 INFO:tasks.ceph:pgid 15.1a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.429 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.429 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.429 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.430 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.430 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.430 INFO:tasks.ceph:pgid 15.1b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.430 INFO:tasks.ceph:pgid 14.1a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.430 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.430 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.431 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.431 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.431 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.431 INFO:tasks.ceph:pgid 15.14 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.431 INFO:tasks.ceph:pgid 14.15 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.432 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.432 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.432 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.432 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.432 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.432 INFO:tasks.ceph:pgid 15.15 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.433 INFO:tasks.ceph:pgid 13.17 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.433 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.433 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.433 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.433 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.434 INFO:tasks.ceph:pgid 15.16 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.434 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.434 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.434 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.434 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.434 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.435 INFO:tasks.ceph:pgid 15.17 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.435 INFO:tasks.ceph:pgid 14.16 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.435 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.435 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.435 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.436 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.436 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.436 INFO:tasks.ceph:pgid 15.10 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.436 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.436 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.437 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.437 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.437 INFO:tasks.ceph:pgid 15.11 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.437 INFO:tasks.ceph:pgid 14.10 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.437 INFO:tasks.ceph:pgid 13.13 last_scrub_stamp 2024-04-16T17:48:26.252757+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=26, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.437 INFO:tasks.ceph:pgid 9.17 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.438 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.438 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.438 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.438 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.438 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.438 INFO:tasks.ceph:pgid 15.12 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.439 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.439 INFO:tasks.ceph:pgid 11.16 last_scrub_stamp 2024-04-16T17:47:58.151336+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=58, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.439 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.439 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.439 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.440 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.440 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.440 INFO:tasks.ceph:pgid 15.13 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.440 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.440 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.440 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.441 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-16T17:47:34.273829+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=34, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.441 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-16T17:47:36.289910+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.441 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.441 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.441 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.442 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.442 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.442 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.442 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.442 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.442 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.443 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.443 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.443 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.443 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.443 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.444 INFO:tasks.ceph:pgid 9.2d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.444 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.444 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.444 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.444 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.444 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.445 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.445 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.445 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.445 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.445 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.445 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.446 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.446 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.446 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.446 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.446 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.447 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.447 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.447 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.447 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.447 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.447 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:00.448 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-16T19:37:20.449 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:37:20.849 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:37:20.877 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:37:20.880 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3368,"stamp":"2024-04-16T19:37:20.369004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2589937,"num_omap_keys":11324,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41584,"ondisk_log_size":41584,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":358192,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":348022,"kb_avail":628787408,"statfs":{"total":644245094400,"available":643878305792,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":356375200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":42264,"num_omap_keys":176,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008053"},"pg_stats":[{"pgid":"9.35","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.507786+0000","last_change":"2024-04-16T19:36:20.507786+0000","last_active":"2024-04-16T19:36:20.507786+0000","last_peered":"2024-04-16T19:36:20.507786+0000","last_clean":"2024-04-16T19:36:20.507786+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:36:20.507786+0000","last_undegraded":"2024-04-16T19:36:20.507786+0000","last_fullsized":"2024-04-16T19:36:20.507786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.507587+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:21:12.018020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209498+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:35:41.209498+0000","last_peered":"2024-04-16T19:35:41.209498+0000","last_clean":"2024-04-16T19:35:41.209498+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:35:41.209498+0000","last_undegraded":"2024-04-16T19:35:41.209498+0000","last_fullsized":"2024-04-16T19:35:41.209498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.192292+0000","last_change":"2024-04-16T17:47:23.264833+0000","last_active":"2024-04-16T19:34:56.192292+0000","last_peered":"2024-04-16T19:34:56.192292+0000","last_clean":"2024-04-16T19:34:56.192292+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:34:56.192292+0000","last_undegraded":"2024-04-16T19:34:56.192292+0000","last_fullsized":"2024-04-16T19:34:56.192292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'23","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.496453+0000","last_change":"2024-04-16T19:36:37.496453+0000","last_active":"2024-04-16T19:36:37.496453+0000","last_peered":"2024-04-16T19:36:37.496453+0000","last_clean":"2024-04-16T19:36:37.496453+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:36:37.496453+0000","last_undegraded":"2024-04-16T19:36:37.496453+0000","last_fullsized":"2024-04-16T19:36:37.496453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.496061+0000","objects_scrubbed":9,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:45.797648+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206637+0000","last_change":"2024-04-16T17:47:21.236420+0000","last_active":"2024-04-16T19:35:41.206637+0000","last_peered":"2024-04-16T19:35:41.206637+0000","last_clean":"2024-04-16T19:35:41.206637+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:35:41.206637+0000","last_undegraded":"2024-04-16T19:35:41.206637+0000","last_fullsized":"2024-04-16T19:35:41.206637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.197906+0000","last_change":"2024-04-16T17:47:23.273251+0000","last_active":"2024-04-16T19:34:56.197906+0000","last_peered":"2024-04-16T19:34:56.197906+0000","last_clean":"2024-04-16T19:34:56.197906+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:34:56.197906+0000","last_undegraded":"2024-04-16T19:34:56.197906+0000","last_fullsized":"2024-04-16T19:34:56.197906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'25","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.596413+0000","last_change":"2024-04-16T19:36:46.596413+0000","last_active":"2024-04-16T19:36:46.596413+0000","last_peered":"2024-04-16T19:36:46.596413+0000","last_clean":"2024-04-16T19:36:46.596413+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:36:46.596413+0000","last_undegraded":"2024-04-16T19:36:46.596413+0000","last_fullsized":"2024-04-16T19:36:46.596413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'25","last_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_deep_scrub":"53'25","last_deep_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.596297+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:26.632824+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205312+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:35:41.205312+0000","last_peered":"2024-04-16T19:35:41.205312+0000","last_clean":"2024-04-16T19:35:41.205312+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:35:41.205312+0000","last_undegraded":"2024-04-16T19:35:41.205312+0000","last_fullsized":"2024-04-16T19:35:41.205312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189744+0000","last_change":"2024-04-16T17:47:23.254286+0000","last_active":"2024-04-16T19:34:56.189744+0000","last_peered":"2024-04-16T19:34:56.189744+0000","last_clean":"2024-04-16T19:34:56.189744+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:34:56.189744+0000","last_undegraded":"2024-04-16T19:34:56.189744+0000","last_fullsized":"2024-04-16T19:34:56.189744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'9","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.049703+0000","last_change":"2024-04-16T17:49:32.393119+0000","last_active":"2024-04-16T19:36:17.049703+0000","last_peered":"2024-04-16T19:36:17.049703+0000","last_clean":"2024-04-16T19:36:17.049703+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:36:17.049703+0000","last_undegraded":"2024-04-16T19:36:17.049703+0000","last_fullsized":"2024-04-16T19:36:17.049703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.205993+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:35:46.205993+0000","last_peered":"2024-04-16T19:35:46.205993+0000","last_clean":"2024-04-16T19:35:46.205993+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:35:46.205993+0000","last_undegraded":"2024-04-16T19:35:46.205993+0000","last_fullsized":"2024-04-16T19:35:46.205993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.193915+0000","last_change":"2024-04-16T17:47:23.267638+0000","last_active":"2024-04-16T19:34:56.193915+0000","last_peered":"2024-04-16T19:34:56.193915+0000","last_clean":"2024-04-16T19:34:56.193915+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:34:56.193915+0000","last_undegraded":"2024-04-16T19:34:56.193915+0000","last_fullsized":"2024-04-16T19:34:56.193915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'8","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.663272+0000","last_change":"2024-04-16T19:36:31.663358+0000","last_active":"2024-04-16T19:36:31.663272+0000","last_peered":"2024-04-16T19:36:31.663272+0000","last_clean":"2024-04-16T19:36:31.663272+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:36:31.663272+0000","last_undegraded":"2024-04-16T19:36:31.663272+0000","last_fullsized":"2024-04-16T19:36:31.663272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.663140+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:09.078400+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.197555+0000","last_change":"2024-04-16T17:47:32.279222+0000","last_active":"2024-04-16T19:35:06.197555+0000","last_peered":"2024-04-16T19:35:06.197555+0000","last_clean":"2024-04-16T19:35:06.197555+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:35:06.197555+0000","last_undegraded":"2024-04-16T19:35:06.197555+0000","last_fullsized":"2024-04-16T19:35:06.197555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207378+0000","last_change":"2024-04-16T17:47:23.259990+0000","last_active":"2024-04-16T19:35:51.207378+0000","last_peered":"2024-04-16T19:35:51.207378+0000","last_clean":"2024-04-16T19:35:51.207378+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:35:51.207378+0000","last_undegraded":"2024-04-16T19:35:51.207378+0000","last_fullsized":"2024-04-16T19:35:51.207378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'20","reported_seq":327,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.492651+0000","last_change":"2024-04-16T19:36:26.492760+0000","last_active":"2024-04-16T19:36:26.492651+0000","last_peered":"2024-04-16T19:36:26.492651+0000","last_clean":"2024-04-16T19:36:26.492651+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:36:26.492651+0000","last_undegraded":"2024-04-16T19:36:26.492651+0000","last_fullsized":"2024-04-16T19:36:26.492651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.492498+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:56.598246+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191143+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:34:51.191143+0000","last_peered":"2024-04-16T19:34:51.191143+0000","last_clean":"2024-04-16T19:34:51.191143+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:34:51.191143+0000","last_undegraded":"2024-04-16T19:34:51.191143+0000","last_fullsized":"2024-04-16T19:34:51.191143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.202288+0000","last_change":"2024-04-16T17:47:23.250385+0000","last_active":"2024-04-16T19:35:46.202288+0000","last_peered":"2024-04-16T19:35:46.202288+0000","last_clean":"2024-04-16T19:35:46.202288+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:35:46.202288+0000","last_undegraded":"2024-04-16T19:35:46.202288+0000","last_fullsized":"2024-04-16T19:35:46.202288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'19","reported_seq":285,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.411649+0000","last_change":"2024-04-16T19:36:31.411649+0000","last_active":"2024-04-16T19:36:31.411649+0000","last_peered":"2024-04-16T19:36:31.411649+0000","last_clean":"2024-04-16T19:36:31.411649+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:36:31.411649+0000","last_undegraded":"2024-04-16T19:36:31.411649+0000","last_fullsized":"2024-04-16T19:36:31.411649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.411418+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:18.775822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.196650+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:34:51.196650+0000","last_peered":"2024-04-16T19:34:51.196650+0000","last_clean":"2024-04-16T19:34:51.196650+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:34:51.196650+0000","last_undegraded":"2024-04-16T19:34:51.196650+0000","last_fullsized":"2024-04-16T19:34:51.196650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206153+0000","last_change":"2024-04-16T17:47:23.270715+0000","last_active":"2024-04-16T19:35:46.206153+0000","last_peered":"2024-04-16T19:35:46.206153+0000","last_clean":"2024-04-16T19:35:46.206153+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:35:46.206153+0000","last_undegraded":"2024-04-16T19:35:46.206153+0000","last_fullsized":"2024-04-16T19:35:46.206153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'14","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.657838+0000","last_change":"2024-04-16T19:36:29.657923+0000","last_active":"2024-04-16T19:36:29.657838+0000","last_peered":"2024-04-16T19:36:29.657838+0000","last_clean":"2024-04-16T19:36:29.657838+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:36:29.657838+0000","last_undegraded":"2024-04-16T19:36:29.657838+0000","last_fullsized":"2024-04-16T19:36:29.657838+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.657764+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:42.577097+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.174105+0000","last_change":"2024-04-16T17:47:21.234259+0000","last_active":"2024-04-16T19:34:51.174105+0000","last_peered":"2024-04-16T19:34:51.174105+0000","last_clean":"2024-04-16T19:34:51.174105+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:34:51.174105+0000","last_undegraded":"2024-04-16T19:34:51.174105+0000","last_fullsized":"2024-04-16T19:34:51.174105+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.492993+0000","last_change":"2024-04-16T19:37:13.493045+0000","last_active":"2024-04-16T19:37:13.492993+0000","last_peered":"2024-04-16T19:37:13.492993+0000","last_clean":"2024-04-16T19:37:13.492993+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:37:13.492993+0000","last_undegraded":"2024-04-16T19:37:13.492993+0000","last_fullsized":"2024-04-16T19:37:13.492993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.492847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.492847+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.492847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:18.553733+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"53'10","reported_seq":315,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.620843+0000","last_change":"2024-04-16T19:36:21.620914+0000","last_active":"2024-04-16T19:36:21.620843+0000","last_peered":"2024-04-16T19:36:21.620843+0000","last_clean":"2024-04-16T19:36:21.620843+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:36:21.620843+0000","last_undegraded":"2024-04-16T19:36:21.620843+0000","last_fullsized":"2024-04-16T19:36:21.620843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.620737+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:28.047519+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.437819+0000","last_change":"2024-04-16T17:47:23.247187+0000","last_active":"2024-04-16T19:36:18.437819+0000","last_peered":"2024-04-16T19:36:18.437819+0000","last_clean":"2024-04-16T19:36:18.437819+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:36:18.437819+0000","last_undegraded":"2024-04-16T19:36:18.437819+0000","last_fullsized":"2024-04-16T19:36:18.437819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048553+0000","last_change":"2024-04-16T17:47:32.283058+0000","last_active":"2024-04-16T19:36:17.048553+0000","last_peered":"2024-04-16T19:36:17.048553+0000","last_clean":"2024-04-16T19:36:17.048553+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:36:17.048553+0000","last_undegraded":"2024-04-16T19:36:17.048553+0000","last_fullsized":"2024-04-16T19:36:17.048553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'18","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.400030+0000","last_change":"2024-04-16T19:36:26.400030+0000","last_active":"2024-04-16T19:36:26.400030+0000","last_peered":"2024-04-16T19:36:26.400030+0000","last_clean":"2024-04-16T19:36:26.400030+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:36:26.400030+0000","last_undegraded":"2024-04-16T19:36:26.400030+0000","last_fullsized":"2024-04-16T19:36:26.400030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.399732+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:00.453174+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912052+0000","last_change":"2024-04-16T17:47:23.255016+0000","last_active":"2024-04-16T19:36:18.912052+0000","last_peered":"2024-04-16T19:36:18.912052+0000","last_clean":"2024-04-16T19:36:18.912052+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:36:18.912052+0000","last_undegraded":"2024-04-16T19:36:18.912052+0000","last_fullsized":"2024-04-16T19:36:18.912052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215736+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:36:06.215736+0000","last_peered":"2024-04-16T19:36:06.215736+0000","last_clean":"2024-04-16T19:36:06.215736+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:36:06.215736+0000","last_undegraded":"2024-04-16T19:36:06.215736+0000","last_fullsized":"2024-04-16T19:36:06.215736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'16","reported_seq":288,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.440103+0000","last_change":"2024-04-16T19:36:42.440209+0000","last_active":"2024-04-16T19:36:42.440103+0000","last_peered":"2024-04-16T19:36:42.440103+0000","last_clean":"2024-04-16T19:36:42.440103+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:36:42.440103+0000","last_undegraded":"2024-04-16T19:36:42.440103+0000","last_fullsized":"2024-04-16T19:36:42.440103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.439969+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.619590+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:11.196138+0000","last_change":"2024-04-16T17:47:23.266392+0000","last_active":"2024-04-16T19:35:11.196138+0000","last_peered":"2024-04-16T19:35:11.196138+0000","last_clean":"2024-04-16T19:35:11.196138+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:35:11.196138+0000","last_undegraded":"2024-04-16T19:35:11.196138+0000","last_fullsized":"2024-04-16T19:35:11.196138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211293+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:36:06.211293+0000","last_peered":"2024-04-16T19:36:06.211293+0000","last_clean":"2024-04-16T19:36:06.211293+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:36:06.211293+0000","last_undegraded":"2024-04-16T19:36:06.211293+0000","last_fullsized":"2024-04-16T19:36:06.211293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"53'9","reported_seq":241,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.533226+0000","last_change":"2024-04-16T19:36:29.533377+0000","last_active":"2024-04-16T19:36:29.533226+0000","last_peered":"2024-04-16T19:36:29.533226+0000","last_clean":"2024-04-16T19:36:29.533226+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:36:29.533226+0000","last_undegraded":"2024-04-16T19:36:29.533226+0000","last_fullsized":"2024-04-16T19:36:29.533226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.533050+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:35:44.595573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.441994+0000","last_change":"2024-04-16T19:37:11.442105+0000","last_active":"2024-04-16T19:37:11.441994+0000","last_peered":"2024-04-16T19:37:11.441994+0000","last_clean":"2024-04-16T19:37:11.441994+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:37:11.441994+0000","last_undegraded":"2024-04-16T19:37:11.441994+0000","last_fullsized":"2024-04-16T19:37:11.441994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.441824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.441824+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.441824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:43:01.723675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.597121+0000","last_change":"2024-04-16T19:37:14.597172+0000","last_active":"2024-04-16T19:37:14.597121+0000","last_peered":"2024-04-16T19:37:14.597121+0000","last_clean":"2024-04-16T19:37:14.597121+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:37:14.597121+0000","last_undegraded":"2024-04-16T19:37:14.597121+0000","last_fullsized":"2024-04-16T19:37:14.597121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.596983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.596983+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.596983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:21.782197+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'10","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498172+0000","last_change":"2024-04-16T19:36:19.498276+0000","last_active":"2024-04-16T19:36:19.498172+0000","last_peered":"2024-04-16T19:36:19.498172+0000","last_clean":"2024-04-16T19:36:19.498172+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:36:19.498172+0000","last_undegraded":"2024-04-16T19:36:19.498172+0000","last_fullsized":"2024-04-16T19:36:19.498172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498013+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:24.571217+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.430185+0000","last_change":"2024-04-16T19:37:07.430253+0000","last_active":"2024-04-16T19:37:07.430185+0000","last_peered":"2024-04-16T19:37:07.430185+0000","last_clean":"2024-04-16T19:37:07.430185+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:37:07.430185+0000","last_undegraded":"2024-04-16T19:37:07.430185+0000","last_fullsized":"2024-04-16T19:37:07.430185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.430037+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.430037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:58.189241+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048603+0000","last_change":"2024-04-16T17:47:32.284938+0000","last_active":"2024-04-16T19:36:17.048603+0000","last_peered":"2024-04-16T19:36:17.048603+0000","last_clean":"2024-04-16T19:36:17.048603+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:36:17.048603+0000","last_undegraded":"2024-04-16T19:36:17.048603+0000","last_fullsized":"2024-04-16T19:36:17.048603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'19","reported_seq":283,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.534237+0000","last_change":"2024-04-16T19:36:45.534337+0000","last_active":"2024-04-16T19:36:45.534237+0000","last_peered":"2024-04-16T19:36:45.534237+0000","last_clean":"2024-04-16T19:36:45.534237+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:36:45.534237+0000","last_undegraded":"2024-04-16T19:36:45.534237+0000","last_fullsized":"2024-04-16T19:36:45.534237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.534098+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:29.053953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.606734+0000","last_change":"2024-04-16T19:37:15.606785+0000","last_active":"2024-04-16T19:37:15.606734+0000","last_peered":"2024-04-16T19:37:15.606734+0000","last_clean":"2024-04-16T19:37:15.606734+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:37:15.606734+0000","last_undegraded":"2024-04-16T19:37:15.606734+0000","last_fullsized":"2024-04-16T19:37:15.606734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.606601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.606601+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.606601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:41.974274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.198747+0000","last_change":"2024-04-16T17:47:21.232784+0000","last_active":"2024-04-16T19:35:16.198747+0000","last_peered":"2024-04-16T19:35:16.198747+0000","last_clean":"2024-04-16T19:35:16.198747+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:35:16.198747+0000","last_undegraded":"2024-04-16T19:35:16.198747+0000","last_fullsized":"2024-04-16T19:35:16.198747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215769+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:36:16.215769+0000","last_peered":"2024-04-16T19:36:16.215769+0000","last_clean":"2024-04-16T19:36:16.215769+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:36:16.215769+0000","last_undegraded":"2024-04-16T19:36:16.215769+0000","last_fullsized":"2024-04-16T19:36:16.215769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'97","reported_seq":1352,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980675+0000","last_change":"2024-04-16T17:48:34.337243+0000","last_active":"2024-04-16T19:36:17.980675+0000","last_peered":"2024-04-16T19:36:17.980675+0000","last_clean":"2024-04-16T19:36:17.980675+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:36:17.980675+0000","last_undegraded":"2024-04-16T19:36:17.980675+0000","last_fullsized":"2024-04-16T19:36:17.980675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.434783+0000","last_change":"2024-04-16T19:36:56.434907+0000","last_active":"2024-04-16T19:36:56.434783+0000","last_peered":"2024-04-16T19:36:56.434783+0000","last_clean":"2024-04-16T19:36:56.434783+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:36:56.434783+0000","last_undegraded":"2024-04-16T19:36:56.434783+0000","last_fullsized":"2024-04-16T19:36:56.434783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:56.434608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:56.434608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.434608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:12.721346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.645164+0000","last_change":"2024-04-16T19:36:51.645240+0000","last_active":"2024-04-16T19:36:51.645164+0000","last_peered":"2024-04-16T19:36:51.645164+0000","last_clean":"2024-04-16T19:36:51.645164+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:36:51.645164+0000","last_undegraded":"2024-04-16T19:36:51.645164+0000","last_fullsized":"2024-04-16T19:36:51.645164+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.645030+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:42.615404+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.617380+0000","last_change":"2024-04-16T19:36:26.617439+0000","last_active":"2024-04-16T19:36:26.617380+0000","last_peered":"2024-04-16T19:36:26.617380+0000","last_clean":"2024-04-16T19:36:26.617380+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:36:26.617380+0000","last_undegraded":"2024-04-16T19:36:26.617380+0000","last_fullsized":"2024-04-16T19:36:26.617380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.617211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:50.470545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'563","reported_seq":824,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.471001+0000","last_change":"2024-04-16T19:36:20.471131+0000","last_active":"2024-04-16T19:36:20.471001+0000","last_peered":"2024-04-16T19:36:20.471001+0000","last_clean":"2024-04-16T19:36:20.471001+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:36:20.471001+0000","last_undegraded":"2024-04-16T19:36:20.471001+0000","last_fullsized":"2024-04-16T19:36:20.471001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.470861+0000","objects_scrubbed":16,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:34.399674+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71932,"num_omap_keys":314,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'14","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.561790+0000","last_change":"2024-04-16T19:36:25.561790+0000","last_active":"2024-04-16T19:36:25.561790+0000","last_peered":"2024-04-16T19:36:25.561790+0000","last_clean":"2024-04-16T19:36:25.561790+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:36:25.561790+0000","last_undegraded":"2024-04-16T19:36:25.561790+0000","last_fullsized":"2024-04-16T19:36:25.561790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.561550+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:02:57.257953+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.527133+0000","last_change":"2024-04-16T19:36:50.527184+0000","last_active":"2024-04-16T19:36:50.527133+0000","last_peered":"2024-04-16T19:36:50.527133+0000","last_clean":"2024-04-16T19:36:50.527133+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:36:50.527133+0000","last_undegraded":"2024-04-16T19:36:50.527133+0000","last_fullsized":"2024-04-16T19:36:50.527133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.527001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:19.398870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.200023+0000","last_change":"2024-04-16T17:47:23.258040+0000","last_active":"2024-04-16T19:35:16.200023+0000","last_peered":"2024-04-16T19:35:16.200023+0000","last_clean":"2024-04-16T19:35:16.200023+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:35:16.200023+0000","last_undegraded":"2024-04-16T19:35:16.200023+0000","last_fullsized":"2024-04-16T19:35:16.200023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048561+0000","last_change":"2024-04-16T17:47:32.284404+0000","last_active":"2024-04-16T19:36:17.048561+0000","last_peered":"2024-04-16T19:36:17.048561+0000","last_clean":"2024-04-16T19:36:17.048561+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:36:17.048561+0000","last_undegraded":"2024-04-16T19:36:17.048561+0000","last_fullsized":"2024-04-16T19:36:17.048561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"53'4","reported_seq":779,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.342632+0000","last_change":"2024-04-16T19:37:14.342695+0000","last_active":"2024-04-16T19:37:14.342632+0000","last_peered":"2024-04-16T19:37:14.342632+0000","last_clean":"2024-04-16T19:37:14.342632+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:37:14.342632+0000","last_undegraded":"2024-04-16T19:37:14.342632+0000","last_fullsized":"2024-04-16T19:37:14.342632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:14.342493+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:14.342493+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.342493+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:34.674033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.673841+0000","last_change":"2024-04-16T19:37:10.673931+0000","last_active":"2024-04-16T19:37:10.673841+0000","last_peered":"2024-04-16T19:37:10.673841+0000","last_clean":"2024-04-16T19:37:10.673841+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:37:10.673841+0000","last_undegraded":"2024-04-16T19:37:10.673841+0000","last_fullsized":"2024-04-16T19:37:10.673841+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:10.673712+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:10.673712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.673712+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:39.298894+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.400379+0000","last_change":"2024-04-16T19:36:48.400438+0000","last_active":"2024-04-16T19:36:48.400379+0000","last_peered":"2024-04-16T19:36:48.400379+0000","last_clean":"2024-04-16T19:36:48.400379+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:36:48.400379+0000","last_undegraded":"2024-04-16T19:36:48.400379+0000","last_fullsized":"2024-04-16T19:36:48.400379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.400232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:53.461118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181269+0000","last_change":"2024-04-16T18:10:50.555912+0000","last_active":"2024-04-16T19:35:21.181269+0000","last_peered":"2024-04-16T19:35:21.181269+0000","last_clean":"2024-04-16T19:35:21.181269+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:35:21.181269+0000","last_undegraded":"2024-04-16T19:35:21.181269+0000","last_fullsized":"2024-04-16T19:35:21.181269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'113","reported_seq":1461,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.578659+0000","last_change":"2024-04-16T19:37:01.578802+0000","last_active":"2024-04-16T19:37:01.578659+0000","last_peered":"2024-04-16T19:37:01.578659+0000","last_clean":"2024-04-16T19:37:01.578659+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:37:01.578659+0000","last_undegraded":"2024-04-16T19:37:01.578659+0000","last_fullsized":"2024-04-16T19:37:01.578659+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2024-04-16T19:37:01.578449+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2024-04-16T19:37:01.578449+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.578449+0000","objects_scrubbed":30,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:34.021657+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6547,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.437635+0000","last_change":"2024-04-16T19:36:59.437695+0000","last_active":"2024-04-16T19:36:59.437635+0000","last_peered":"2024-04-16T19:36:59.437635+0000","last_clean":"2024-04-16T19:36:59.437635+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:36:59.437635+0000","last_undegraded":"2024-04-16T19:36:59.437635+0000","last_fullsized":"2024-04-16T19:36:59.437635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.437461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.437461+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.437461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:15.724199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'9","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.654127+0000","last_change":"2024-04-16T19:36:50.654208+0000","last_active":"2024-04-16T19:36:50.654127+0000","last_peered":"2024-04-16T19:36:50.654127+0000","last_clean":"2024-04-16T19:36:50.654127+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:36:50.654127+0000","last_undegraded":"2024-04-16T19:36:50.654127+0000","last_fullsized":"2024-04-16T19:36:50.654127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.653994+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:30.491092+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.479488+0000","last_change":"2024-04-16T19:36:33.479590+0000","last_active":"2024-04-16T19:36:33.479488+0000","last_peered":"2024-04-16T19:36:33.479488+0000","last_clean":"2024-04-16T19:36:33.479488+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:36:33.479488+0000","last_undegraded":"2024-04-16T19:36:33.479488+0000","last_fullsized":"2024-04-16T19:36:33.479488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.479280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:24.835481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7349","reported_seq":20627,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.602403+0000","last_change":"2024-04-16T19:36:20.602529+0000","last_active":"2024-04-16T19:36:20.602403+0000","last_peered":"2024-04-16T19:36:20.602403+0000","last_clean":"2024-04-16T19:36:20.602403+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:36:20.602403+0000","last_undegraded":"2024-04-16T19:36:20.602403+0000","last_fullsized":"2024-04-16T19:36:20.602403+0000","mapping_epoch":34,"log_start":"53'5600","ondisk_log_start":"53'5600","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7349","last_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_deep_scrub":"53'7349","last_deep_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.602264+0000","objects_scrubbed":23,"log_size":1749,"log_dups_size":3000,"ondisk_log_size":1749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:12.702919+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72188,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'30","reported_seq":375,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.493531+0000","last_change":"2024-04-16T19:36:32.493531+0000","last_active":"2024-04-16T19:36:32.493531+0000","last_peered":"2024-04-16T19:36:32.493531+0000","last_clean":"2024-04-16T19:36:32.493531+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:36:32.493531+0000","last_undegraded":"2024-04-16T19:36:32.493531+0000","last_fullsized":"2024-04-16T19:36:32.493531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'30","last_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_deep_scrub":"53'30","last_deep_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.493294+0000","objects_scrubbed":11,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:58.339768+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.426772+0000","last_change":"2024-04-16T19:36:46.426772+0000","last_active":"2024-04-16T19:36:46.426772+0000","last_peered":"2024-04-16T19:36:46.426772+0000","last_clean":"2024-04-16T19:36:46.426772+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:36:46.426772+0000","last_undegraded":"2024-04-16T19:36:46.426772+0000","last_fullsized":"2024-04-16T19:36:46.426772+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.426574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.708425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212569+0000","last_change":"2024-04-16T17:47:23.268885+0000","last_active":"2024-04-16T19:36:11.212569+0000","last_peered":"2024-04-16T19:36:11.212569+0000","last_clean":"2024-04-16T19:36:11.212569+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:36:11.212569+0000","last_undegraded":"2024-04-16T19:36:11.212569+0000","last_fullsized":"2024-04-16T19:36:11.212569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.585662+0000","last_change":"2024-04-16T17:47:21.244985+0000","last_active":"2024-04-16T19:36:16.585662+0000","last_peered":"2024-04-16T19:36:16.585662+0000","last_clean":"2024-04-16T19:36:16.585662+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:36:16.585662+0000","last_undegraded":"2024-04-16T19:36:16.585662+0000","last_fullsized":"2024-04-16T19:36:16.585662+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"53'7","reported_seq":370,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.684190+0000","last_change":"2024-04-16T19:37:15.684265+0000","last_active":"2024-04-16T19:37:15.684190+0000","last_peered":"2024-04-16T19:37:15.684190+0000","last_clean":"2024-04-16T19:37:15.684190+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:37:15.684190+0000","last_undegraded":"2024-04-16T19:37:15.684190+0000","last_fullsized":"2024-04-16T19:37:15.684190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:15.684059+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:15.684059+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.684059+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:55.937399+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'75","reported_seq":229,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.595038+0000","last_change":"2024-04-16T19:37:02.595110+0000","last_active":"2024-04-16T19:37:02.595038+0000","last_peered":"2024-04-16T19:37:02.595038+0000","last_clean":"2024-04-16T19:37:02.595038+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:37:02.595038+0000","last_undegraded":"2024-04-16T19:37:02.595038+0000","last_fullsized":"2024-04-16T19:37:02.595038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'75","last_scrub_stamp":"2024-04-16T19:37:02.594894+0000","last_deep_scrub":"53'75","last_deep_scrub_stamp":"2024-04-16T19:37:02.594894+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.594894+0000","objects_scrubbed":3,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:07.655780+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.580869+0000","last_change":"2024-04-16T19:36:57.580927+0000","last_active":"2024-04-16T19:36:57.580869+0000","last_peered":"2024-04-16T19:36:57.580869+0000","last_clean":"2024-04-16T19:36:57.580869+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:36:57.580869+0000","last_undegraded":"2024-04-16T19:36:57.580869+0000","last_fullsized":"2024-04-16T19:36:57.580869+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:57.580729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:57.580729+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.580729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:16.681259+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.203492+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:35:51.203492+0000","last_peered":"2024-04-16T19:35:51.203492+0000","last_clean":"2024-04-16T19:35:51.203492+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:35:51.203492+0000","last_undegraded":"2024-04-16T19:35:51.203492+0000","last_fullsized":"2024-04-16T19:35:51.203492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'84","reported_seq":914,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.739730+0000","last_change":"2024-04-16T19:37:13.739802+0000","last_active":"2024-04-16T19:37:13.739730+0000","last_peered":"2024-04-16T19:37:13.739730+0000","last_clean":"2024-04-16T19:37:13.739730+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:37:13.739730+0000","last_undegraded":"2024-04-16T19:37:13.739730+0000","last_fullsized":"2024-04-16T19:37:13.739730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:13.739588+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:13.739588+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.739588+0000","objects_scrubbed":28,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:54.664404+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4355,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.671347+0000","last_change":"2024-04-16T19:36:34.671347+0000","last_active":"2024-04-16T19:36:34.671347+0000","last_peered":"2024-04-16T19:36:34.671347+0000","last_clean":"2024-04-16T19:36:34.671347+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:36:34.671347+0000","last_undegraded":"2024-04-16T19:36:34.671347+0000","last_fullsized":"2024-04-16T19:36:34.671347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.671050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:41.097832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'20","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.631537+0000","last_change":"2024-04-16T19:36:45.631650+0000","last_active":"2024-04-16T19:36:45.631537+0000","last_peered":"2024-04-16T19:36:45.631537+0000","last_clean":"2024-04-16T19:36:45.631537+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:36:45.631537+0000","last_undegraded":"2024-04-16T19:36:45.631537+0000","last_fullsized":"2024-04-16T19:36:45.631537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.631366+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:37.106553+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.586110+0000","last_change":"2024-04-16T19:36:37.586170+0000","last_active":"2024-04-16T19:36:37.586110+0000","last_peered":"2024-04-16T19:36:37.586110+0000","last_clean":"2024-04-16T19:36:37.586110+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:36:37.586110+0000","last_undegraded":"2024-04-16T19:36:37.586110+0000","last_fullsized":"2024-04-16T19:36:37.586110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.585961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:02.979747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3625","reported_seq":7230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.651412+0000","last_change":"2024-04-16T19:36:17.651412+0000","last_active":"2024-04-16T19:36:17.651412+0000","last_peered":"2024-04-16T19:36:17.651412+0000","last_clean":"2024-04-16T19:36:17.651412+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:36:17.651412+0000","last_undegraded":"2024-04-16T19:36:17.651412+0000","last_fullsized":"2024-04-16T19:36:17.651412+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'3625","last_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_deep_scrub":"53'3625","last_deep_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.651328+0000","objects_scrubbed":30,"log_size":1825,"log_dups_size":1800,"ondisk_log_size":1825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:26.898814+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'18","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.465484+0000","last_change":"2024-04-16T19:36:17.465591+0000","last_active":"2024-04-16T19:36:17.465484+0000","last_peered":"2024-04-16T19:36:17.465484+0000","last_clean":"2024-04-16T19:36:17.465484+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:36:17.465484+0000","last_undegraded":"2024-04-16T19:36:17.465484+0000","last_fullsized":"2024-04-16T19:36:17.465484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.465360+0000","objects_scrubbed":9,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:23.892141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.615759+0000","last_change":"2024-04-16T19:36:38.615812+0000","last_active":"2024-04-16T19:36:38.615759+0000","last_peered":"2024-04-16T19:36:38.615759+0000","last_clean":"2024-04-16T19:36:38.615759+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:36:38.615759+0000","last_undegraded":"2024-04-16T19:36:38.615759+0000","last_fullsized":"2024-04-16T19:36:38.615759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.615619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:29.374824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212749+0000","last_change":"2024-04-16T17:47:23.256650+0000","last_active":"2024-04-16T19:36:11.212749+0000","last_peered":"2024-04-16T19:36:11.212749+0000","last_clean":"2024-04-16T19:36:11.212749+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:36:11.212749+0000","last_undegraded":"2024-04-16T19:36:11.212749+0000","last_fullsized":"2024-04-16T19:36:11.212749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977659+0000","last_change":"2024-04-16T17:47:21.254345+0000","last_active":"2024-04-16T19:36:17.977659+0000","last_peered":"2024-04-16T19:36:17.977659+0000","last_clean":"2024-04-16T19:36:17.977659+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:36:17.977659+0000","last_undegraded":"2024-04-16T19:36:17.977659+0000","last_fullsized":"2024-04-16T19:36:17.977659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"53'13","reported_seq":617,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.385910+0000","last_change":"2024-04-16T19:37:08.386038+0000","last_active":"2024-04-16T19:37:08.385910+0000","last_peered":"2024-04-16T19:37:08.385910+0000","last_clean":"2024-04-16T19:37:08.385910+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:37:08.385910+0000","last_undegraded":"2024-04-16T19:37:08.385910+0000","last_fullsized":"2024-04-16T19:37:08.385910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:08.385737+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:08.385737+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.385737+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:45.642460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'125","reported_seq":304,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.417448+0000","last_change":"2024-04-16T19:36:55.417529+0000","last_active":"2024-04-16T19:36:55.417448+0000","last_peered":"2024-04-16T19:36:55.417448+0000","last_clean":"2024-04-16T19:36:55.417448+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:36:55.417448+0000","last_undegraded":"2024-04-16T19:36:55.417448+0000","last_fullsized":"2024-04-16T19:36:55.417448+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.417311+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:25.523059+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'13","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.351749+0000","last_change":"2024-04-16T19:36:45.351811+0000","last_active":"2024-04-16T19:36:45.351749+0000","last_peered":"2024-04-16T19:36:45.351749+0000","last_clean":"2024-04-16T19:36:45.351749+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:36:45.351749+0000","last_undegraded":"2024-04-16T19:36:45.351749+0000","last_fullsized":"2024-04-16T19:36:45.351749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.351601+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:50.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209444+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:35:51.209444+0000","last_peered":"2024-04-16T19:35:51.209444+0000","last_clean":"2024-04-16T19:35:51.209444+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:35:51.209444+0000","last_undegraded":"2024-04-16T19:35:51.209444+0000","last_fullsized":"2024-04-16T19:35:51.209444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'59","reported_seq":737,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211512+0000","last_change":"2024-04-16T17:48:34.343838+0000","last_active":"2024-04-16T19:36:06.211512+0000","last_peered":"2024-04-16T19:36:06.211512+0000","last_clean":"2024-04-16T19:36:06.211512+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:36:06.211512+0000","last_undegraded":"2024-04-16T19:36:06.211512+0000","last_fullsized":"2024-04-16T19:36:06.211512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.671455+0000","last_change":"2024-04-16T19:36:35.671455+0000","last_active":"2024-04-16T19:36:35.671455+0000","last_peered":"2024-04-16T19:36:35.671455+0000","last_clean":"2024-04-16T19:36:35.671455+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:36:35.671455+0000","last_undegraded":"2024-04-16T19:36:35.671455+0000","last_fullsized":"2024-04-16T19:36:35.671455+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.671258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:03.249930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'21","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.435794+0000","last_change":"2024-04-16T19:36:50.435867+0000","last_active":"2024-04-16T19:36:50.435794+0000","last_peered":"2024-04-16T19:36:50.435794+0000","last_clean":"2024-04-16T19:36:50.435794+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:36:50.435794+0000","last_undegraded":"2024-04-16T19:36:50.435794+0000","last_fullsized":"2024-04-16T19:36:50.435794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.435653+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:16.012260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.469677+0000","last_change":"2024-04-16T19:36:39.469677+0000","last_active":"2024-04-16T19:36:39.469677+0000","last_peered":"2024-04-16T19:36:39.469677+0000","last_clean":"2024-04-16T19:36:39.469677+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:36:39.469677+0000","last_undegraded":"2024-04-16T19:36:39.469677+0000","last_fullsized":"2024-04-16T19:36:39.469677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.469488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.661779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'408","reported_seq":769,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.522794+0000","last_change":"2024-04-16T19:36:31.522794+0000","last_active":"2024-04-16T19:36:31.522794+0000","last_peered":"2024-04-16T19:36:31.522794+0000","last_clean":"2024-04-16T19:36:31.522794+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:36:31.522794+0000","last_undegraded":"2024-04-16T19:36:31.522794+0000","last_fullsized":"2024-04-16T19:36:31.522794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'408","last_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_deep_scrub":"53'408","last_deep_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.522481+0000","objects_scrubbed":23,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:59.101153+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'12","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.541977+0000","last_change":"2024-04-16T19:36:17.542045+0000","last_active":"2024-04-16T19:36:17.541977+0000","last_peered":"2024-04-16T19:36:17.541977+0000","last_clean":"2024-04-16T19:36:17.541977+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:36:17.541977+0000","last_undegraded":"2024-04-16T19:36:17.541977+0000","last_fullsized":"2024-04-16T19:36:17.541977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.541913+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:24.087701+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.579713+0000","last_change":"2024-04-16T19:36:47.579767+0000","last_active":"2024-04-16T19:36:47.579713+0000","last_peered":"2024-04-16T19:36:47.579713+0000","last_clean":"2024-04-16T19:36:47.579713+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:36:47.579713+0000","last_undegraded":"2024-04-16T19:36:47.579713+0000","last_fullsized":"2024-04-16T19:36:47.579713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.579576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:56.945905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212828+0000","last_change":"2024-04-16T17:47:23.254971+0000","last_active":"2024-04-16T19:36:11.212828+0000","last_peered":"2024-04-16T19:36:11.212828+0000","last_clean":"2024-04-16T19:36:11.212828+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:36:11.212828+0000","last_undegraded":"2024-04-16T19:36:11.212828+0000","last_fullsized":"2024-04-16T19:36:11.212828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911701+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:36:18.911701+0000","last_peered":"2024-04-16T19:36:18.911701+0000","last_clean":"2024-04-16T19:36:18.911701+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:36:18.911701+0000","last_undegraded":"2024-04-16T19:36:18.911701+0000","last_fullsized":"2024-04-16T19:36:18.911701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.516275+0000","last_change":"2024-04-16T19:37:13.516327+0000","last_active":"2024-04-16T19:37:13.516275+0000","last_peered":"2024-04-16T19:37:13.516275+0000","last_clean":"2024-04-16T19:37:13.516275+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:37:13.516275+0000","last_undegraded":"2024-04-16T19:37:13.516275+0000","last_fullsized":"2024-04-16T19:37:13.516275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.516141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.516141+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.516141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:59.747526+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.526949+0000","last_change":"2024-04-16T19:37:09.527017+0000","last_active":"2024-04-16T19:37:09.526949+0000","last_peered":"2024-04-16T19:37:09.526949+0000","last_clean":"2024-04-16T19:37:09.526949+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:37:09.526949+0000","last_undegraded":"2024-04-16T19:37:09.526949+0000","last_fullsized":"2024-04-16T19:37:09.526949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:09.526805+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:09.526805+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.526805+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:03.234997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'13","reported_seq":1344,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.649901+0000","last_change":"2024-04-16T19:36:52.649962+0000","last_active":"2024-04-16T19:36:52.649901+0000","last_peered":"2024-04-16T19:36:52.649901+0000","last_clean":"2024-04-16T19:36:52.649901+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:36:52.649901+0000","last_undegraded":"2024-04-16T19:36:52.649901+0000","last_fullsized":"2024-04-16T19:36:52.649901+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.649763+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.928938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.209696+0000","last_change":"2024-04-16T18:10:50.548544+0000","last_active":"2024-04-16T19:36:16.209696+0000","last_peered":"2024-04-16T19:36:16.209696+0000","last_clean":"2024-04-16T19:36:16.209696+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:36:16.209696+0000","last_undegraded":"2024-04-16T19:36:16.209696+0000","last_fullsized":"2024-04-16T19:36:16.209696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'200","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.601906+0000","last_change":"2024-04-16T19:37:08.601960+0000","last_active":"2024-04-16T19:37:08.601906+0000","last_peered":"2024-04-16T19:37:08.601906+0000","last_clean":"2024-04-16T19:37:08.601906+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:37:08.601906+0000","last_undegraded":"2024-04-16T19:37:08.601906+0000","last_fullsized":"2024-04-16T19:37:08.601906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'200","last_scrub_stamp":"2024-04-16T19:37:08.601835+0000","last_deep_scrub":"53'200","last_deep_scrub_stamp":"2024-04-16T19:37:08.601835+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.601835+0000","objects_scrubbed":36,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:25.200215+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19169,"num_omap_keys":76,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.413351+0000","last_change":"2024-04-16T19:36:52.413405+0000","last_active":"2024-04-16T19:36:52.413351+0000","last_peered":"2024-04-16T19:36:52.413351+0000","last_clean":"2024-04-16T19:36:52.413351+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:36:52.413351+0000","last_undegraded":"2024-04-16T19:36:52.413351+0000","last_fullsized":"2024-04-16T19:36:52.413351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.413208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:42.476038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'13","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.645823+0000","last_change":"2024-04-16T19:36:38.645892+0000","last_active":"2024-04-16T19:36:38.645823+0000","last_peered":"2024-04-16T19:36:38.645823+0000","last_clean":"2024-04-16T19:36:38.645823+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:36:38.645823+0000","last_undegraded":"2024-04-16T19:36:38.645823+0000","last_fullsized":"2024-04-16T19:36:38.645823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.645696+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:29.924307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":158,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.474393+0000","last_change":"2024-04-16T19:36:58.474445+0000","last_active":"2024-04-16T19:36:58.474393+0000","last_peered":"2024-04-16T19:36:58.474393+0000","last_clean":"2024-04-16T19:36:58.474393+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:36:58.474393+0000","last_undegraded":"2024-04-16T19:36:58.474393+0000","last_fullsized":"2024-04-16T19:36:58.474393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.474247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.474247+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.474247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:24.479134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1110","reported_seq":1775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.529609+0000","last_change":"2024-04-16T19:36:31.529728+0000","last_active":"2024-04-16T19:36:31.529609+0000","last_peered":"2024-04-16T19:36:31.529609+0000","last_clean":"2024-04-16T19:36:31.529609+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:36:31.529609+0000","last_undegraded":"2024-04-16T19:36:31.529609+0000","last_fullsized":"2024-04-16T19:36:31.529609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1110","last_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_deep_scrub":"53'1110","last_deep_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.529471+0000","objects_scrubbed":23,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:18.708104+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143914,"num_omap_keys":628,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"53'8","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.477590+0000","last_change":"2024-04-16T19:36:20.477590+0000","last_active":"2024-04-16T19:36:20.477590+0000","last_peered":"2024-04-16T19:36:20.477590+0000","last_clean":"2024-04-16T19:36:20.477590+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:36:20.477590+0000","last_undegraded":"2024-04-16T19:36:20.477590+0000","last_fullsized":"2024-04-16T19:36:20.477590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.477384+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:25.916610+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210353+0000","last_change":"2024-04-16T17:47:23.257369+0000","last_active":"2024-04-16T19:35:46.210353+0000","last_peered":"2024-04-16T19:35:46.210353+0000","last_clean":"2024-04-16T19:35:46.210353+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:35:46.210353+0000","last_undegraded":"2024-04-16T19:35:46.210353+0000","last_fullsized":"2024-04-16T19:35:46.210353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206209+0000","last_change":"2024-04-16T17:47:21.257195+0000","last_active":"2024-04-16T19:35:46.206209+0000","last_peered":"2024-04-16T19:35:46.206209+0000","last_clean":"2024-04-16T19:35:46.206209+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:35:46.206209+0000","last_undegraded":"2024-04-16T19:35:46.206209+0000","last_fullsized":"2024-04-16T19:35:46.206209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.372126+0000","last_change":"2024-04-16T19:37:13.372176+0000","last_active":"2024-04-16T19:37:13.372126+0000","last_peered":"2024-04-16T19:37:13.372126+0000","last_clean":"2024-04-16T19:37:13.372126+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:37:13.372126+0000","last_undegraded":"2024-04-16T19:37:13.372126+0000","last_fullsized":"2024-04-16T19:37:13.372126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.371985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.371985+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.371985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:47:39.425403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'209","reported_seq":437,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.363263+0000","last_change":"2024-04-16T19:37:08.363339+0000","last_active":"2024-04-16T19:37:08.363263+0000","last_peered":"2024-04-16T19:37:08.363263+0000","last_clean":"2024-04-16T19:37:08.363263+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:37:08.363263+0000","last_undegraded":"2024-04-16T19:37:08.363263+0000","last_fullsized":"2024-04-16T19:37:08.363263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'209","last_scrub_stamp":"2024-04-16T19:37:08.363125+0000","last_deep_scrub":"53'209","last_deep_scrub_stamp":"2024-04-16T19:37:08.363125+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.363125+0000","objects_scrubbed":9,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:57.353295+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.650228+0000","last_change":"2024-04-16T19:36:49.650334+0000","last_active":"2024-04-16T19:36:49.650228+0000","last_peered":"2024-04-16T19:36:49.650228+0000","last_clean":"2024-04-16T19:36:49.650228+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:36:49.650228+0000","last_undegraded":"2024-04-16T19:36:49.650228+0000","last_fullsized":"2024-04-16T19:36:49.650228+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.650064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:03.271488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.443726+0000","last_change":"2024-04-16T19:36:42.443778+0000","last_active":"2024-04-16T19:36:42.443726+0000","last_peered":"2024-04-16T19:36:42.443726+0000","last_clean":"2024-04-16T19:36:42.443726+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:36:42.443726+0000","last_undegraded":"2024-04-16T19:36:42.443726+0000","last_fullsized":"2024-04-16T19:36:42.443726+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.443589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:33.202793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.194292+0000","last_change":"2024-04-16T18:10:50.554707+0000","last_active":"2024-04-16T19:36:16.194292+0000","last_peered":"2024-04-16T19:36:16.194292+0000","last_clean":"2024-04-16T19:36:16.194292+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:36:16.194292+0000","last_undegraded":"2024-04-16T19:36:16.194292+0000","last_fullsized":"2024-04-16T19:36:16.194292+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587906+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:36:16.587906+0000","last_peered":"2024-04-16T19:36:16.587906+0000","last_clean":"2024-04-16T19:36:16.587906+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:36:16.587906+0000","last_undegraded":"2024-04-16T19:36:16.587906+0000","last_fullsized":"2024-04-16T19:36:16.587906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.441666+0000","last_change":"2024-04-16T19:36:48.441666+0000","last_active":"2024-04-16T19:36:48.441666+0000","last_peered":"2024-04-16T19:36:48.441666+0000","last_clean":"2024-04-16T19:36:48.441666+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:36:48.441666+0000","last_undegraded":"2024-04-16T19:36:48.441666+0000","last_fullsized":"2024-04-16T19:36:48.441666+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.441481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:54.447789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'4","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.445595+0000","last_change":"2024-04-16T19:36:53.445663+0000","last_active":"2024-04-16T19:36:53.445595+0000","last_peered":"2024-04-16T19:36:53.445595+0000","last_clean":"2024-04-16T19:36:53.445595+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:36:53.445595+0000","last_undegraded":"2024-04-16T19:36:53.445595+0000","last_fullsized":"2024-04-16T19:36:53.445595+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.445453+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:27.787458+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.610902+0000","last_change":"2024-04-16T19:36:39.610922+0000","last_active":"2024-04-16T19:36:39.610902+0000","last_peered":"2024-04-16T19:36:39.610902+0000","last_clean":"2024-04-16T19:36:39.610902+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:36:39.610902+0000","last_undegraded":"2024-04-16T19:36:39.610902+0000","last_fullsized":"2024-04-16T19:36:39.610902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.610843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:16.867566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1195","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.442330+0000","last_change":"2024-04-16T19:36:32.442434+0000","last_active":"2024-04-16T19:36:32.442330+0000","last_peered":"2024-04-16T19:36:32.442330+0000","last_clean":"2024-04-16T19:36:32.442330+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:36:32.442330+0000","last_undegraded":"2024-04-16T19:36:32.442330+0000","last_fullsized":"2024-04-16T19:36:32.442330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1195","last_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_deep_scrub":"53'1195","last_deep_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.442194+0000","objects_scrubbed":22,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:37.854921+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215932,"num_omap_keys":942,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'17","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.474347+0000","last_change":"2024-04-16T19:36:21.474426+0000","last_active":"2024-04-16T19:36:21.474347+0000","last_peered":"2024-04-16T19:36:21.474347+0000","last_clean":"2024-04-16T19:36:21.474347+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:36:21.474347+0000","last_undegraded":"2024-04-16T19:36:21.474347+0000","last_fullsized":"2024-04-16T19:36:21.474347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'17","last_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_deep_scrub":"53'17","last_deep_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.474217+0000","objects_scrubbed":3,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:40.463538+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.640159+0000","last_change":"2024-04-16T19:37:08.640210+0000","last_active":"2024-04-16T19:37:08.640159+0000","last_peered":"2024-04-16T19:37:08.640159+0000","last_clean":"2024-04-16T19:37:08.640159+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:37:08.640159+0000","last_undegraded":"2024-04-16T19:37:08.640159+0000","last_fullsized":"2024-04-16T19:37:08.640159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:08.640022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:08.640022+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.640022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:42.193624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199383+0000","last_change":"2024-04-16T17:47:21.257611+0000","last_active":"2024-04-16T19:35:21.199383+0000","last_peered":"2024-04-16T19:35:21.199383+0000","last_clean":"2024-04-16T19:35:21.199383+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:35:21.199383+0000","last_undegraded":"2024-04-16T19:35:21.199383+0000","last_fullsized":"2024-04-16T19:35:21.199383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"53'2","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.485428+0000","last_change":"2024-04-16T19:37:12.485493+0000","last_active":"2024-04-16T19:37:12.485428+0000","last_peered":"2024-04-16T19:37:12.485428+0000","last_clean":"2024-04-16T19:37:12.485428+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:37:12.485428+0000","last_undegraded":"2024-04-16T19:37:12.485428+0000","last_fullsized":"2024-04-16T19:37:12.485428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:12.485279+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:12.485279+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.485279+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.828865+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'84","reported_seq":250,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.729167+0000","last_change":"2024-04-16T19:36:57.729240+0000","last_active":"2024-04-16T19:36:57.729167+0000","last_peered":"2024-04-16T19:36:57.729167+0000","last_clean":"2024-04-16T19:36:57.729167+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:36:57.729167+0000","last_undegraded":"2024-04-16T19:36:57.729167+0000","last_fullsized":"2024-04-16T19:36:57.729167+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:36:57.729025+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:36:57.729025+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.729025+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:26:51.751473+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.388357+0000","last_change":"2024-04-16T19:36:54.388412+0000","last_active":"2024-04-16T19:36:54.388357+0000","last_peered":"2024-04-16T19:36:54.388357+0000","last_clean":"2024-04-16T19:36:54.388357+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:36:54.388357+0000","last_undegraded":"2024-04-16T19:36:54.388357+0000","last_fullsized":"2024-04-16T19:36:54.388357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.388213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:28.441656+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.545722+0000","last_change":"2024-04-16T19:36:53.545774+0000","last_active":"2024-04-16T19:36:53.545722+0000","last_peered":"2024-04-16T19:36:53.545722+0000","last_clean":"2024-04-16T19:36:53.545722+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:36:53.545722+0000","last_undegraded":"2024-04-16T19:36:53.545722+0000","last_fullsized":"2024-04-16T19:36:53.545722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.545589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:08.257713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217900+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:36:16.217900+0000","last_peered":"2024-04-16T19:36:16.217900+0000","last_clean":"2024-04-16T19:36:16.217900+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:36:16.217900+0000","last_undegraded":"2024-04-16T19:36:16.217900+0000","last_fullsized":"2024-04-16T19:36:16.217900+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'90","reported_seq":931,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.425451+0000","last_change":"2024-04-16T19:36:56.425451+0000","last_active":"2024-04-16T19:36:56.425451+0000","last_peered":"2024-04-16T19:36:56.425451+0000","last_clean":"2024-04-16T19:36:56.425451+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:36:56.425451+0000","last_undegraded":"2024-04-16T19:36:56.425451+0000","last_fullsized":"2024-04-16T19:36:56.425451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'90","last_scrub_stamp":"2024-04-16T19:36:56.425221+0000","last_deep_scrub":"53'90","last_deep_scrub_stamp":"2024-04-16T19:36:56.425221+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.425221+0000","objects_scrubbed":20,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:43.603855+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6839,"num_omap_keys":28,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.686464+0000","last_change":"2024-04-16T19:36:49.686464+0000","last_active":"2024-04-16T19:36:49.686464+0000","last_peered":"2024-04-16T19:36:49.686464+0000","last_clean":"2024-04-16T19:36:49.686464+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:36:49.686464+0000","last_undegraded":"2024-04-16T19:36:49.686464+0000","last_fullsized":"2024-04-16T19:36:49.686464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.686271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.036352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980079+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:36:17.980079+0000","last_peered":"2024-04-16T19:36:17.980079+0000","last_clean":"2024-04-16T19:36:17.980079+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:36:17.980079+0000","last_undegraded":"2024-04-16T19:36:17.980079+0000","last_fullsized":"2024-04-16T19:36:17.980079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.684642+0000","last_change":"2024-04-16T19:36:32.684642+0000","last_active":"2024-04-16T19:36:32.684642+0000","last_peered":"2024-04-16T19:36:32.684642+0000","last_clean":"2024-04-16T19:36:32.684642+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:36:32.684642+0000","last_undegraded":"2024-04-16T19:36:32.684642+0000","last_fullsized":"2024-04-16T19:36:32.684642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.684452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:11.352292+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1071","reported_seq":6142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.466107+0000","last_change":"2024-04-16T19:36:31.466251+0000","last_active":"2024-04-16T19:36:31.466107+0000","last_peered":"2024-04-16T19:36:31.466107+0000","last_clean":"2024-04-16T19:36:31.466107+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:36:31.466107+0000","last_undegraded":"2024-04-16T19:36:31.466107+0000","last_fullsized":"2024-04-16T19:36:31.466107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1071","last_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_deep_scrub":"53'1071","last_deep_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.465959+0000","objects_scrubbed":22,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:36.878686+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71967,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'8","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.607489+0000","last_change":"2024-04-16T19:36:24.607587+0000","last_active":"2024-04-16T19:36:24.607489+0000","last_peered":"2024-04-16T19:36:24.607489+0000","last_clean":"2024-04-16T19:36:24.607489+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:36:24.607489+0000","last_undegraded":"2024-04-16T19:36:24.607489+0000","last_fullsized":"2024-04-16T19:36:24.607489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.607347+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:30.046572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212436+0000","last_change":"2024-04-16T17:47:23.267294+0000","last_active":"2024-04-16T19:36:11.212436+0000","last_peered":"2024-04-16T19:36:11.212436+0000","last_clean":"2024-04-16T19:36:11.212436+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:36:11.212436+0000","last_undegraded":"2024-04-16T19:36:11.212436+0000","last_fullsized":"2024-04-16T19:36:11.212436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204505+0000","last_change":"2024-04-16T17:47:21.251571+0000","last_active":"2024-04-16T19:35:21.204505+0000","last_peered":"2024-04-16T19:35:21.204505+0000","last_clean":"2024-04-16T19:35:21.204505+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:35:21.204505+0000","last_undegraded":"2024-04-16T19:35:21.204505+0000","last_fullsized":"2024-04-16T19:35:21.204505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.493197+0000","last_change":"2024-04-16T19:37:14.493250+0000","last_active":"2024-04-16T19:37:14.493197+0000","last_peered":"2024-04-16T19:37:14.493197+0000","last_clean":"2024-04-16T19:37:14.493197+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:37:14.493197+0000","last_undegraded":"2024-04-16T19:37:14.493197+0000","last_fullsized":"2024-04-16T19:37:14.493197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.493058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.493058+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.493058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:17.475775+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'159","reported_seq":363,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.570200+0000","last_change":"2024-04-16T19:37:06.570279+0000","last_active":"2024-04-16T19:37:06.570200+0000","last_peered":"2024-04-16T19:37:06.570200+0000","last_clean":"2024-04-16T19:37:06.570200+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:37:06.570200+0000","last_undegraded":"2024-04-16T19:37:06.570200+0000","last_fullsized":"2024-04-16T19:37:06.570200+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:06.570065+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:06.570065+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.570065+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:27.105572+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'13","reported_seq":1348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.514311+0000","last_change":"2024-04-16T19:37:06.514379+0000","last_active":"2024-04-16T19:37:06.514311+0000","last_peered":"2024-04-16T19:37:06.514311+0000","last_clean":"2024-04-16T19:37:06.514311+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:37:06.514311+0000","last_undegraded":"2024-04-16T19:37:06.514311+0000","last_fullsized":"2024-04-16T19:37:06.514311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:06.514177+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:06.514177+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.514177+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:28.619214+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.712924+0000","last_change":"2024-04-16T19:36:43.712975+0000","last_active":"2024-04-16T19:36:43.712924+0000","last_peered":"2024-04-16T19:36:43.712924+0000","last_clean":"2024-04-16T19:36:43.712924+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:36:43.712924+0000","last_undegraded":"2024-04-16T19:36:43.712924+0000","last_fullsized":"2024-04-16T19:36:43.712924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.712792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:43.360972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.181074+0000","last_change":"2024-04-16T18:10:50.553097+0000","last_active":"2024-04-16T19:35:21.181074+0000","last_peered":"2024-04-16T19:35:21.181074+0000","last_clean":"2024-04-16T19:35:21.181074+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:35:21.181074+0000","last_undegraded":"2024-04-16T19:35:21.181074+0000","last_fullsized":"2024-04-16T19:35:21.181074+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'100","reported_seq":1255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.688914+0000","last_change":"2024-04-16T19:36:47.688914+0000","last_active":"2024-04-16T19:36:47.688914+0000","last_peered":"2024-04-16T19:36:47.688914+0000","last_clean":"2024-04-16T19:36:47.688914+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:36:47.688914+0000","last_undegraded":"2024-04-16T19:36:47.688914+0000","last_fullsized":"2024-04-16T19:36:47.688914+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'100","last_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_deep_scrub":"53'100","last_deep_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.688717+0000","objects_scrubbed":26,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:21.999831+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5240,"num_omap_keys":22,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.472784+0000","last_change":"2024-04-16T19:36:55.472893+0000","last_active":"2024-04-16T19:36:55.472784+0000","last_peered":"2024-04-16T19:36:55.472784+0000","last_clean":"2024-04-16T19:36:55.472784+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:36:55.472784+0000","last_undegraded":"2024-04-16T19:36:55.472784+0000","last_fullsized":"2024-04-16T19:36:55.472784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.472614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:45.535445+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'20","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.637124+0000","last_change":"2024-04-16T19:36:41.637203+0000","last_active":"2024-04-16T19:36:41.637124+0000","last_peered":"2024-04-16T19:36:41.637124+0000","last_clean":"2024-04-16T19:36:41.637124+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:36:41.637124+0000","last_undegraded":"2024-04-16T19:36:41.637124+0000","last_fullsized":"2024-04-16T19:36:41.637124+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.636981+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.258518+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.453915+0000","last_change":"2024-04-16T19:36:39.453971+0000","last_active":"2024-04-16T19:36:39.453915+0000","last_peered":"2024-04-16T19:36:39.453915+0000","last_clean":"2024-04-16T19:36:39.453915+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:36:39.453915+0000","last_undegraded":"2024-04-16T19:36:39.453915+0000","last_fullsized":"2024-04-16T19:36:39.453915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.453770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:59.663137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'736","reported_seq":1478,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.630399+0000","last_change":"2024-04-16T19:36:18.630399+0000","last_active":"2024-04-16T19:36:18.630399+0000","last_peered":"2024-04-16T19:36:18.630399+0000","last_clean":"2024-04-16T19:36:18.630399+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:36:18.630399+0000","last_undegraded":"2024-04-16T19:36:18.630399+0000","last_fullsized":"2024-04-16T19:36:18.630399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'736","last_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_deep_scrub":"53'736","last_deep_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.630302+0000","objects_scrubbed":33,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:45.495590+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":75043,"num_omap_keys":332,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'18","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.577485+0000","last_change":"2024-04-16T19:36:26.577559+0000","last_active":"2024-04-16T19:36:26.577485+0000","last_peered":"2024-04-16T19:36:26.577485+0000","last_clean":"2024-04-16T19:36:26.577485+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:36:26.577485+0000","last_undegraded":"2024-04-16T19:36:26.577485+0000","last_fullsized":"2024-04-16T19:36:26.577485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.577356+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:41.709302+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201519+0000","last_change":"2024-04-16T17:47:23.267655+0000","last_active":"2024-04-16T19:35:26.201519+0000","last_peered":"2024-04-16T19:35:26.201519+0000","last_clean":"2024-04-16T19:35:26.201519+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:35:26.201519+0000","last_undegraded":"2024-04-16T19:35:26.201519+0000","last_fullsized":"2024-04-16T19:35:26.201519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199263+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:35:21.199263+0000","last_peered":"2024-04-16T19:35:21.199263+0000","last_clean":"2024-04-16T19:35:21.199263+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:35:21.199263+0000","last_undegraded":"2024-04-16T19:35:21.199263+0000","last_fullsized":"2024-04-16T19:35:21.199263+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.527954+0000","last_change":"2024-04-16T17:47:33.304659+0000","last_active":"2024-04-16T19:36:17.527954+0000","last_peered":"2024-04-16T19:36:17.527954+0000","last_clean":"2024-04-16T19:36:17.527954+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:36:17.527954+0000","last_undegraded":"2024-04-16T19:36:17.527954+0000","last_fullsized":"2024-04-16T19:36:17.527954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'93","reported_seq":270,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.563674+0000","last_change":"2024-04-16T19:37:07.563764+0000","last_active":"2024-04-16T19:37:07.563674+0000","last_peered":"2024-04-16T19:37:07.563674+0000","last_clean":"2024-04-16T19:37:07.563674+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:37:07.563674+0000","last_undegraded":"2024-04-16T19:37:07.563674+0000","last_fullsized":"2024-04-16T19:37:07.563674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'93","last_scrub_stamp":"2024-04-16T19:37:07.563526+0000","last_deep_scrub":"53'93","last_deep_scrub_stamp":"2024-04-16T19:37:07.563526+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.563526+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:24.161906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.415324+0000","last_change":"2024-04-16T19:36:53.415380+0000","last_active":"2024-04-16T19:36:53.415324+0000","last_peered":"2024-04-16T19:36:53.415324+0000","last_clean":"2024-04-16T19:36:53.415324+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:36:53.415324+0000","last_undegraded":"2024-04-16T19:36:53.415324+0000","last_fullsized":"2024-04-16T19:36:53.415324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.415178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:50.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.527536+0000","last_change":"2024-04-16T19:36:33.527616+0000","last_active":"2024-04-16T19:36:33.527536+0000","last_peered":"2024-04-16T19:36:33.527536+0000","last_clean":"2024-04-16T19:36:33.527536+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:36:33.527536+0000","last_undegraded":"2024-04-16T19:36:33.527536+0000","last_fullsized":"2024-04-16T19:36:33.527536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.527307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:38.972728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201397+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:35:21.201397+0000","last_peered":"2024-04-16T19:35:21.201397+0000","last_clean":"2024-04-16T19:35:21.201397+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:35:21.201397+0000","last_undegraded":"2024-04-16T19:35:21.201397+0000","last_fullsized":"2024-04-16T19:35:21.201397+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'62","reported_seq":1005,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.553275+0000","last_change":"2024-04-16T19:37:05.553378+0000","last_active":"2024-04-16T19:37:05.553275+0000","last_peered":"2024-04-16T19:37:05.553275+0000","last_clean":"2024-04-16T19:37:05.553275+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:37:05.553275+0000","last_undegraded":"2024-04-16T19:37:05.553275+0000","last_fullsized":"2024-04-16T19:37:05.553275+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'62","last_scrub_stamp":"2024-04-16T19:37:05.553135+0000","last_deep_scrub":"53'62","last_deep_scrub_stamp":"2024-04-16T19:37:05.553135+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.553135+0000","objects_scrubbed":22,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:26.088642+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1541,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":92,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.684338+0000","last_change":"2024-04-16T19:36:40.684390+0000","last_active":"2024-04-16T19:36:40.684338+0000","last_peered":"2024-04-16T19:36:40.684338+0000","last_clean":"2024-04-16T19:36:40.684338+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:36:40.684338+0000","last_undegraded":"2024-04-16T19:36:40.684338+0000","last_fullsized":"2024-04-16T19:36:40.684338+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:34.096243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'11","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.426664+0000","last_change":"2024-04-16T19:36:52.426789+0000","last_active":"2024-04-16T19:36:52.426664+0000","last_peered":"2024-04-16T19:36:52.426664+0000","last_clean":"2024-04-16T19:36:52.426664+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:36:52.426664+0000","last_undegraded":"2024-04-16T19:36:52.426664+0000","last_fullsized":"2024-04-16T19:36:52.426664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.426496+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:18.642190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'556","reported_seq":2298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.370105+0000","last_change":"2024-04-16T19:36:40.370175+0000","last_active":"2024-04-16T19:36:40.370105+0000","last_peered":"2024-04-16T19:36:40.370105+0000","last_clean":"2024-04-16T19:36:40.370105+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:36:40.370105+0000","last_undegraded":"2024-04-16T19:36:40.370105+0000","last_fullsized":"2024-04-16T19:36:40.370105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.369960+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:17.626683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'925","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.519148+0000","last_change":"2024-04-16T19:36:29.519272+0000","last_active":"2024-04-16T19:36:29.519148+0000","last_peered":"2024-04-16T19:36:29.519148+0000","last_clean":"2024-04-16T19:36:29.519148+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:36:29.519148+0000","last_undegraded":"2024-04-16T19:36:29.519148+0000","last_fullsized":"2024-04-16T19:36:29.519148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.519049+0000","objects_scrubbed":24,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:29.400044+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143864,"num_omap_keys":628,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'19","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.480753+0000","last_change":"2024-04-16T19:36:30.480855+0000","last_active":"2024-04-16T19:36:30.480753+0000","last_peered":"2024-04-16T19:36:30.480753+0000","last_clean":"2024-04-16T19:36:30.480753+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:36:30.480753+0000","last_undegraded":"2024-04-16T19:36:30.480753+0000","last_fullsized":"2024-04-16T19:36:30.480753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.480620+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:17.845024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.197270+0000","last_change":"2024-04-16T17:47:23.255450+0000","last_active":"2024-04-16T19:35:26.197270+0000","last_peered":"2024-04-16T19:35:26.197270+0000","last_clean":"2024-04-16T19:35:26.197270+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:35:26.197270+0000","last_undegraded":"2024-04-16T19:35:26.197270+0000","last_fullsized":"2024-04-16T19:35:26.197270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.201194+0000","last_change":"2024-04-16T17:47:21.232392+0000","last_active":"2024-04-16T19:35:21.201194+0000","last_peered":"2024-04-16T19:35:21.201194+0000","last_clean":"2024-04-16T19:35:21.201194+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:35:21.201194+0000","last_undegraded":"2024-04-16T19:35:21.201194+0000","last_fullsized":"2024-04-16T19:35:21.201194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"48'3","reported_seq":176,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.184791+0000","last_change":"2024-04-16T17:47:33.310901+0000","last_active":"2024-04-16T19:35:36.184791+0000","last_peered":"2024-04-16T19:35:36.184791+0000","last_clean":"2024-04-16T19:35:36.184791+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:35:36.184791+0000","last_undegraded":"2024-04-16T19:35:36.184791+0000","last_fullsized":"2024-04-16T19:35:36.184791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'109","reported_seq":287,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.642949+0000","last_change":"2024-04-16T19:37:01.642980+0000","last_active":"2024-04-16T19:37:01.642949+0000","last_peered":"2024-04-16T19:37:01.642949+0000","last_clean":"2024-04-16T19:37:01.642949+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:37:01.642949+0000","last_undegraded":"2024-04-16T19:37:01.642949+0000","last_fullsized":"2024-04-16T19:37:01.642949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:37:01.642886+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:37:01.642886+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.642886+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:36:01.291066+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.529837+0000","last_change":"2024-04-16T19:37:03.529944+0000","last_active":"2024-04-16T19:37:03.529837+0000","last_peered":"2024-04-16T19:37:03.529837+0000","last_clean":"2024-04-16T19:37:03.529837+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:37:03.529837+0000","last_undegraded":"2024-04-16T19:37:03.529837+0000","last_fullsized":"2024-04-16T19:37:03.529837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.529668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.529668+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.529668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:01.869011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.457022+0000","last_change":"2024-04-16T19:36:30.457084+0000","last_active":"2024-04-16T19:36:30.457022+0000","last_peered":"2024-04-16T19:36:30.457022+0000","last_clean":"2024-04-16T19:36:30.457022+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:36:30.457022+0000","last_undegraded":"2024-04-16T19:36:30.457022+0000","last_fullsized":"2024-04-16T19:36:30.457022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.456877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:36.883658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200338+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:35:21.200338+0000","last_peered":"2024-04-16T19:35:21.200338+0000","last_clean":"2024-04-16T19:35:21.200338+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:35:21.200338+0000","last_undegraded":"2024-04-16T19:35:21.200338+0000","last_fullsized":"2024-04-16T19:35:21.200338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'82","reported_seq":884,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914956+0000","last_change":"2024-04-16T17:48:34.320431+0000","last_active":"2024-04-16T19:36:18.914956+0000","last_peered":"2024-04-16T19:36:18.914956+0000","last_clean":"2024-04-16T19:36:18.914956+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:36:18.914956+0000","last_undegraded":"2024-04-16T19:36:18.914956+0000","last_fullsized":"2024-04-16T19:36:18.914956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.315787+0000","last_change":"2024-04-16T19:37:11.315847+0000","last_active":"2024-04-16T19:37:11.315787+0000","last_peered":"2024-04-16T19:37:11.315787+0000","last_clean":"2024-04-16T19:37:11.315787+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:37:11.315787+0000","last_undegraded":"2024-04-16T19:37:11.315787+0000","last_fullsized":"2024-04-16T19:37:11.315787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.315613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.315613+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.315613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:39.336603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'142","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.445837+0000","last_change":"2024-04-16T19:36:51.445837+0000","last_active":"2024-04-16T19:36:51.445837+0000","last_peered":"2024-04-16T19:36:51.445837+0000","last_clean":"2024-04-16T19:36:51.445837+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:36:51.445837+0000","last_undegraded":"2024-04-16T19:36:51.445837+0000","last_fullsized":"2024-04-16T19:36:51.445837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'142","last_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_deep_scrub":"53'142","last_deep_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.445664+0000","objects_scrubbed":7,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:24.999267+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9535,"num_omap_keys":55,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.514976+0000","last_change":"2024-04-16T19:36:24.515035+0000","last_active":"2024-04-16T19:36:24.514976+0000","last_peered":"2024-04-16T19:36:24.514976+0000","last_clean":"2024-04-16T19:36:24.514976+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:36:24.514976+0000","last_undegraded":"2024-04-16T19:36:24.514976+0000","last_fullsized":"2024-04-16T19:36:24.514976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.514834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:04:02.534977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'702","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.544776+0000","last_change":"2024-04-16T19:36:22.544916+0000","last_active":"2024-04-16T19:36:22.544776+0000","last_peered":"2024-04-16T19:36:22.544776+0000","last_clean":"2024-04-16T19:36:22.544776+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:36:22.544776+0000","last_undegraded":"2024-04-16T19:36:22.544776+0000","last_fullsized":"2024-04-16T19:36:22.544776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'702","last_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_deep_scrub":"53'702","last_deep_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.544630+0000","objects_scrubbed":23,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:27.704961+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71922,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.499314+0000","last_change":"2024-04-16T19:36:24.499314+0000","last_active":"2024-04-16T19:36:24.499314+0000","last_peered":"2024-04-16T19:36:24.499314+0000","last_clean":"2024-04-16T19:36:24.499314+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:36:24.499314+0000","last_undegraded":"2024-04-16T19:36:24.499314+0000","last_fullsized":"2024-04-16T19:36:24.499314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.498983+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:48.052702+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.205683+0000","last_change":"2024-04-16T17:47:23.258294+0000","last_active":"2024-04-16T19:35:26.205683+0000","last_peered":"2024-04-16T19:35:26.205683+0000","last_clean":"2024-04-16T19:35:26.205683+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:35:26.205683+0000","last_undegraded":"2024-04-16T19:35:26.205683+0000","last_fullsized":"2024-04-16T19:35:26.205683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217992+0000","last_change":"2024-04-16T17:47:21.254091+0000","last_active":"2024-04-16T19:36:16.217992+0000","last_peered":"2024-04-16T19:36:16.217992+0000","last_clean":"2024-04-16T19:36:16.217992+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:36:16.217992+0000","last_undegraded":"2024-04-16T19:36:16.217992+0000","last_fullsized":"2024-04-16T19:36:16.217992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"53'14","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.644762+0000","last_change":"2024-04-16T19:37:04.644828+0000","last_active":"2024-04-16T19:37:04.644762+0000","last_peered":"2024-04-16T19:37:04.644762+0000","last_clean":"2024-04-16T19:37:04.644762+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:37:04.644762+0000","last_undegraded":"2024-04-16T19:37:04.644762+0000","last_fullsized":"2024-04-16T19:37:04.644762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:37:04.644628+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:37:04.644628+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.644628+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:54.707458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'159","reported_seq":361,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.425798+0000","last_change":"2024-04-16T19:37:01.425941+0000","last_active":"2024-04-16T19:37:01.425798+0000","last_peered":"2024-04-16T19:37:01.425798+0000","last_clean":"2024-04-16T19:37:01.425798+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:37:01.425798+0000","last_undegraded":"2024-04-16T19:37:01.425798+0000","last_fullsized":"2024-04-16T19:37:01.425798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:01.425632+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:01.425632+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.425632+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:04.281799+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'13","reported_seq":1346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.458428+0000","last_change":"2024-04-16T19:36:51.458492+0000","last_active":"2024-04-16T19:36:51.458428+0000","last_peered":"2024-04-16T19:36:51.458428+0000","last_clean":"2024-04-16T19:36:51.458428+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:36:51.458428+0000","last_undegraded":"2024-04-16T19:36:51.458428+0000","last_fullsized":"2024-04-16T19:36:51.458428+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.458284+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:46:06.590229+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.582250+0000","last_change":"2024-04-16T19:36:36.582358+0000","last_active":"2024-04-16T19:36:36.582250+0000","last_peered":"2024-04-16T19:36:36.582250+0000","last_clean":"2024-04-16T19:36:36.582250+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:36:36.582250+0000","last_undegraded":"2024-04-16T19:36:36.582250+0000","last_fullsized":"2024-04-16T19:36:36.582250+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.582059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:29.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980805+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:36:17.980805+0000","last_peered":"2024-04-16T19:36:17.980805+0000","last_clean":"2024-04-16T19:36:17.980805+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:36:17.980805+0000","last_undegraded":"2024-04-16T19:36:17.980805+0000","last_fullsized":"2024-04-16T19:36:17.980805+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'118","reported_seq":1160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.521304+0000","last_change":"2024-04-16T19:37:00.521375+0000","last_active":"2024-04-16T19:37:00.521304+0000","last_peered":"2024-04-16T19:37:00.521304+0000","last_clean":"2024-04-16T19:37:00.521304+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:37:00.521304+0000","last_undegraded":"2024-04-16T19:37:00.521304+0000","last_fullsized":"2024-04-16T19:37:00.521304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'118","last_scrub_stamp":"2024-04-16T19:37:00.521172+0000","last_deep_scrub":"53'118","last_deep_scrub_stamp":"2024-04-16T19:37:00.521172+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.521172+0000","objects_scrubbed":32,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:10.815633+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7005,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.427150+0000","last_change":"2024-04-16T19:36:49.427150+0000","last_active":"2024-04-16T19:36:49.427150+0000","last_peered":"2024-04-16T19:36:49.427150+0000","last_clean":"2024-04-16T19:36:49.427150+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:36:49.427150+0000","last_undegraded":"2024-04-16T19:36:49.427150+0000","last_fullsized":"2024-04-16T19:36:49.427150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.426874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:15.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"53'2","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.645599+0000","last_change":"2024-04-16T19:36:42.645707+0000","last_active":"2024-04-16T19:36:42.645599+0000","last_peered":"2024-04-16T19:36:42.645599+0000","last_clean":"2024-04-16T19:36:42.645599+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:36:42.645599+0000","last_undegraded":"2024-04-16T19:36:42.645599+0000","last_fullsized":"2024-04-16T19:36:42.645599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.645438+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:22.681965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.565703+0000","last_change":"2024-04-16T19:36:24.565807+0000","last_active":"2024-04-16T19:36:24.565703+0000","last_peered":"2024-04-16T19:36:24.565703+0000","last_clean":"2024-04-16T19:36:24.565703+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:36:24.565703+0000","last_undegraded":"2024-04-16T19:36:24.565703+0000","last_fullsized":"2024-04-16T19:36:24.565703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.565541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:30.252893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1508","reported_seq":5513,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.585799+0000","last_change":"2024-04-16T19:36:27.585867+0000","last_active":"2024-04-16T19:36:27.585799+0000","last_peered":"2024-04-16T19:36:27.585799+0000","last_clean":"2024-04-16T19:36:27.585799+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:36:27.585799+0000","last_undegraded":"2024-04-16T19:36:27.585799+0000","last_fullsized":"2024-04-16T19:36:27.585799+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1508","last_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_deep_scrub":"53'1508","last_deep_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.585661+0000","objects_scrubbed":25,"log_size":1508,"log_dups_size":0,"ondisk_log_size":1508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:18.587606+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250399,"num_omap_keys":1045,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'14","reported_seq":240,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.424045+0000","last_change":"2024-04-16T19:36:27.424219+0000","last_active":"2024-04-16T19:36:27.424045+0000","last_peered":"2024-04-16T19:36:27.424045+0000","last_clean":"2024-04-16T19:36:27.424045+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:36:27.424045+0000","last_undegraded":"2024-04-16T19:36:27.424045+0000","last_fullsized":"2024-04-16T19:36:27.424045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.423818+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:57.529566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.650311+0000","last_change":"2024-04-16T19:37:06.650366+0000","last_active":"2024-04-16T19:37:06.650311+0000","last_peered":"2024-04-16T19:37:06.650311+0000","last_clean":"2024-04-16T19:37:06.650311+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:37:06.650311+0000","last_undegraded":"2024-04-16T19:37:06.650311+0000","last_fullsized":"2024-04-16T19:37:06.650311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:06.650180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:06.650180+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.650180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:32.865874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200918+0000","last_change":"2024-04-16T17:47:21.223805+0000","last_active":"2024-04-16T19:35:41.200918+0000","last_peered":"2024-04-16T19:35:41.200918+0000","last_clean":"2024-04-16T19:35:41.200918+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:35:41.200918+0000","last_undegraded":"2024-04-16T19:35:41.200918+0000","last_fullsized":"2024-04-16T19:35:41.200918+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210154+0000","last_change":"2024-04-16T17:47:33.301811+0000","last_active":"2024-04-16T19:36:01.210154+0000","last_peered":"2024-04-16T19:36:01.210154+0000","last_clean":"2024-04-16T19:36:01.210154+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:36:01.210154+0000","last_undegraded":"2024-04-16T19:36:01.210154+0000","last_fullsized":"2024-04-16T19:36:01.210154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'125","reported_seq":306,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.693242+0000","last_change":"2024-04-16T19:36:58.693314+0000","last_active":"2024-04-16T19:36:58.693242+0000","last_peered":"2024-04-16T19:36:58.693242+0000","last_clean":"2024-04-16T19:36:58.693242+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:36:58.693242+0000","last_undegraded":"2024-04-16T19:36:58.693242+0000","last_fullsized":"2024-04-16T19:36:58.693242+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:58.693112+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:58.693112+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.693112+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:03.753998+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'13","reported_seq":1346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.636594+0000","last_change":"2024-04-16T19:36:55.636714+0000","last_active":"2024-04-16T19:36:55.636594+0000","last_peered":"2024-04-16T19:36:55.636594+0000","last_clean":"2024-04-16T19:36:55.636594+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:36:55.636594+0000","last_undegraded":"2024-04-16T19:36:55.636594+0000","last_fullsized":"2024-04-16T19:36:55.636594+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.636425+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:56:05.862157+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.567557+0000","last_change":"2024-04-16T19:36:51.567609+0000","last_active":"2024-04-16T19:36:51.567557+0000","last_peered":"2024-04-16T19:36:51.567557+0000","last_clean":"2024-04-16T19:36:51.567557+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:36:51.567557+0000","last_undegraded":"2024-04-16T19:36:51.567557+0000","last_fullsized":"2024-04-16T19:36:51.567557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.567419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:32.289248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215663+0000","last_change":"2024-04-16T18:10:50.539044+0000","last_active":"2024-04-16T19:36:16.215663+0000","last_peered":"2024-04-16T19:36:16.215663+0000","last_clean":"2024-04-16T19:36:16.215663+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:36:16.215663+0000","last_undegraded":"2024-04-16T19:36:16.215663+0000","last_fullsized":"2024-04-16T19:36:16.215663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'117","reported_seq":1014,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.675197+0000","last_change":"2024-04-16T19:37:09.675266+0000","last_active":"2024-04-16T19:37:09.675197+0000","last_peered":"2024-04-16T19:37:09.675197+0000","last_clean":"2024-04-16T19:37:09.675197+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:37:09.675197+0000","last_undegraded":"2024-04-16T19:37:09.675197+0000","last_fullsized":"2024-04-16T19:37:09.675197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'117","last_scrub_stamp":"2024-04-16T19:37:09.675046+0000","last_deep_scrub":"53'117","last_deep_scrub_stamp":"2024-04-16T19:37:09.675046+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.675046+0000","objects_scrubbed":27,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:14.071688+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4717,"num_omap_keys":20,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.389594+0000","last_change":"2024-04-16T19:36:46.389594+0000","last_active":"2024-04-16T19:36:46.389594+0000","last_peered":"2024-04-16T19:36:46.389594+0000","last_clean":"2024-04-16T19:36:46.389594+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:36:46.389594+0000","last_undegraded":"2024-04-16T19:36:46.389594+0000","last_fullsized":"2024-04-16T19:36:46.389594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.389402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:49.372119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'8","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.681828+0000","last_change":"2024-04-16T19:36:40.681959+0000","last_active":"2024-04-16T19:36:40.681828+0000","last_peered":"2024-04-16T19:36:40.681828+0000","last_clean":"2024-04-16T19:36:40.681828+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:36:40.681828+0000","last_undegraded":"2024-04-16T19:36:40.681828+0000","last_fullsized":"2024-04-16T19:36:40.681828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.681665+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:15.914709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'556","reported_seq":1952,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.630262+0000","last_change":"2024-04-16T19:36:46.630326+0000","last_active":"2024-04-16T19:36:46.630262+0000","last_peered":"2024-04-16T19:36:46.630262+0000","last_clean":"2024-04-16T19:36:46.630262+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:36:46.630262+0000","last_undegraded":"2024-04-16T19:36:46.630262+0000","last_fullsized":"2024-04-16T19:36:46.630262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.630122+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:54.931710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'676","reported_seq":995,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635509+0000","last_change":"2024-04-16T19:36:20.635509+0000","last_active":"2024-04-16T19:36:20.635509+0000","last_peered":"2024-04-16T19:36:20.635509+0000","last_clean":"2024-04-16T19:36:20.635509+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:36:20.635509+0000","last_undegraded":"2024-04-16T19:36:20.635509+0000","last_fullsized":"2024-04-16T19:36:20.635509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'676","last_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_deep_scrub":"53'676","last_deep_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635277+0000","objects_scrubbed":25,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:10:55.447676+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71899,"num_omap_keys":314,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'20","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.500999+0000","last_change":"2024-04-16T19:36:18.501046+0000","last_active":"2024-04-16T19:36:18.500999+0000","last_peered":"2024-04-16T19:36:18.500999+0000","last_clean":"2024-04-16T19:36:18.500999+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:36:18.500999+0000","last_undegraded":"2024-04-16T19:36:18.500999+0000","last_fullsized":"2024-04-16T19:36:18.500999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.500951+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:17.147393+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.634453+0000","last_change":"2024-04-16T19:37:10.634505+0000","last_active":"2024-04-16T19:37:10.634453+0000","last_peered":"2024-04-16T19:37:10.634453+0000","last_clean":"2024-04-16T19:37:10.634453+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:37:10.634453+0000","last_undegraded":"2024-04-16T19:37:10.634453+0000","last_fullsized":"2024-04-16T19:37:10.634453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:10.634314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:10.634314+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.634314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:37.001987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.185743+0000","last_change":"2024-04-16T17:47:21.234044+0000","last_active":"2024-04-16T19:35:41.185743+0000","last_peered":"2024-04-16T19:35:41.185743+0000","last_clean":"2024-04-16T19:35:41.185743+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:35:41.185743+0000","last_undegraded":"2024-04-16T19:35:41.185743+0000","last_fullsized":"2024-04-16T19:35:41.185743+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"48'2","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.208172+0000","last_change":"2024-04-16T17:47:33.294785+0000","last_active":"2024-04-16T19:35:36.208172+0000","last_peered":"2024-04-16T19:35:36.208172+0000","last_clean":"2024-04-16T19:35:36.208172+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:35:36.208172+0000","last_undegraded":"2024-04-16T19:35:36.208172+0000","last_fullsized":"2024-04-16T19:35:36.208172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'184","reported_seq":398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.363093+0000","last_change":"2024-04-16T19:37:05.363174+0000","last_active":"2024-04-16T19:37:05.363093+0000","last_peered":"2024-04-16T19:37:05.363093+0000","last_clean":"2024-04-16T19:37:05.363093+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:37:05.363093+0000","last_undegraded":"2024-04-16T19:37:05.363093+0000","last_fullsized":"2024-04-16T19:37:05.363093+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'184","last_scrub_stamp":"2024-04-16T19:37:05.362967+0000","last_deep_scrub":"53'184","last_deep_scrub_stamp":"2024-04-16T19:37:05.362967+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.362967+0000","objects_scrubbed":8,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:30.682003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.486047+0000","last_change":"2024-04-16T19:36:58.486097+0000","last_active":"2024-04-16T19:36:58.486047+0000","last_peered":"2024-04-16T19:36:58.486047+0000","last_clean":"2024-04-16T19:36:58.486047+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:36:58.486047+0000","last_undegraded":"2024-04-16T19:36:58.486047+0000","last_fullsized":"2024-04-16T19:36:58.486047+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.485902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.485902+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.485902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:24.853575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.569888+0000","last_change":"2024-04-16T19:36:34.569940+0000","last_active":"2024-04-16T19:36:34.569888+0000","last_peered":"2024-04-16T19:36:34.569888+0000","last_clean":"2024-04-16T19:36:34.569888+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:36:34.569888+0000","last_undegraded":"2024-04-16T19:36:34.569888+0000","last_fullsized":"2024-04-16T19:36:34.569888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.569751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:40.008977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528987+0000","last_change":"2024-04-16T18:10:50.555565+0000","last_active":"2024-04-16T19:36:17.528987+0000","last_peered":"2024-04-16T19:36:17.528987+0000","last_clean":"2024-04-16T19:36:17.528987+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:36:17.528987+0000","last_undegraded":"2024-04-16T19:36:17.528987+0000","last_fullsized":"2024-04-16T19:36:17.528987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'153","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.638858+0000","last_change":"2024-04-16T19:36:56.638858+0000","last_active":"2024-04-16T19:36:56.638858+0000","last_peered":"2024-04-16T19:36:56.638858+0000","last_clean":"2024-04-16T19:36:56.638858+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:36:56.638858+0000","last_undegraded":"2024-04-16T19:36:56.638858+0000","last_fullsized":"2024-04-16T19:36:56.638858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'153","last_scrub_stamp":"2024-04-16T19:36:56.638728+0000","last_deep_scrub":"53'153","last_deep_scrub_stamp":"2024-04-16T19:36:56.638728+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.638728+0000","objects_scrubbed":35,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:45.633510+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11617,"num_omap_keys":48,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.606337+0000","last_change":"2024-04-16T19:36:37.606393+0000","last_active":"2024-04-16T19:36:37.606337+0000","last_peered":"2024-04-16T19:36:37.606337+0000","last_clean":"2024-04-16T19:36:37.606337+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:36:37.606337+0000","last_undegraded":"2024-04-16T19:36:37.606337+0000","last_fullsized":"2024-04-16T19:36:37.606337+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.606203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:48.287865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'11","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.682939+0000","last_change":"2024-04-16T19:36:43.683003+0000","last_active":"2024-04-16T19:36:43.682939+0000","last_peered":"2024-04-16T19:36:43.682939+0000","last_clean":"2024-04-16T19:36:43.682939+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:36:43.682939+0000","last_undegraded":"2024-04-16T19:36:43.682939+0000","last_fullsized":"2024-04-16T19:36:43.682939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.682806+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:30.180345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.630682+0000","last_change":"2024-04-16T19:36:48.630682+0000","last_active":"2024-04-16T19:36:48.630682+0000","last_peered":"2024-04-16T19:36:48.630682+0000","last_clean":"2024-04-16T19:36:48.630682+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:36:48.630682+0000","last_undegraded":"2024-04-16T19:36:48.630682+0000","last_fullsized":"2024-04-16T19:36:48.630682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.630423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:08.839790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'804","reported_seq":43172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.600535+0000","last_change":"2024-04-16T19:36:25.600624+0000","last_active":"2024-04-16T19:36:25.600535+0000","last_peered":"2024-04-16T19:36:25.600535+0000","last_clean":"2024-04-16T19:36:25.600535+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:36:25.600535+0000","last_undegraded":"2024-04-16T19:36:25.600535+0000","last_fullsized":"2024-04-16T19:36:25.600535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'804","last_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_deep_scrub":"53'804","last_deep_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.600462+0000","objects_scrubbed":21,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:05.088020+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71960,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'12","reported_seq":221,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.489782+0000","last_change":"2024-04-16T19:36:49.489878+0000","last_active":"2024-04-16T19:36:49.489782+0000","last_peered":"2024-04-16T19:36:49.489782+0000","last_clean":"2024-04-16T19:36:49.489782+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:36:49.489782+0000","last_undegraded":"2024-04-16T19:36:49.489782+0000","last_fullsized":"2024-04-16T19:36:49.489782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.489640+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:54.928866+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978084+0000","last_change":"2024-04-16T17:47:23.258413+0000","last_active":"2024-04-16T19:36:17.978084+0000","last_peered":"2024-04-16T19:36:17.978084+0000","last_clean":"2024-04-16T19:36:17.978084+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:36:17.978084+0000","last_undegraded":"2024-04-16T19:36:17.978084+0000","last_fullsized":"2024-04-16T19:36:17.978084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977554+0000","last_change":"2024-04-16T17:46:37.575050+0000","last_active":"2024-04-16T19:36:17.977554+0000","last_peered":"2024-04-16T19:36:17.977554+0000","last_clean":"2024-04-16T19:36:17.977554+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:36:17.977554+0000","last_undegraded":"2024-04-16T19:36:17.977554+0000","last_fullsized":"2024-04-16T19:36:17.977554+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206209+0000","last_change":"2024-04-16T17:47:21.251255+0000","last_active":"2024-04-16T19:35:41.206209+0000","last_peered":"2024-04-16T19:35:41.206209+0000","last_clean":"2024-04-16T19:35:41.206209+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:35:41.206209+0000","last_undegraded":"2024-04-16T19:35:41.206209+0000","last_fullsized":"2024-04-16T19:35:41.206209+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204821+0000","last_change":"2024-04-16T17:47:33.305377+0000","last_active":"2024-04-16T19:35:56.204821+0000","last_peered":"2024-04-16T19:35:56.204821+0000","last_clean":"2024-04-16T19:35:56.204821+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:35:56.204821+0000","last_undegraded":"2024-04-16T19:35:56.204821+0000","last_fullsized":"2024-04-16T19:35:56.204821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'77","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.348075+0000","last_change":"2024-04-16T19:37:12.348141+0000","last_active":"2024-04-16T19:37:12.348075+0000","last_peered":"2024-04-16T19:37:12.348075+0000","last_clean":"2024-04-16T19:37:12.348075+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:37:12.348075+0000","last_undegraded":"2024-04-16T19:37:12.348075+0000","last_fullsized":"2024-04-16T19:37:12.348075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'77","last_scrub_stamp":"2024-04-16T19:37:12.348001+0000","last_deep_scrub":"53'77","last_deep_scrub_stamp":"2024-04-16T19:37:12.348001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.348001+0000","objects_scrubbed":5,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.056193+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.566977+0000","last_change":"2024-04-16T19:36:59.567032+0000","last_active":"2024-04-16T19:36:59.566977+0000","last_peered":"2024-04-16T19:36:59.566977+0000","last_clean":"2024-04-16T19:36:59.566977+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:36:59.566977+0000","last_undegraded":"2024-04-16T19:36:59.566977+0000","last_fullsized":"2024-04-16T19:36:59.566977+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.566839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.566839+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.566839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:45.847122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.477027+0000","last_change":"2024-04-16T19:36:40.477078+0000","last_active":"2024-04-16T19:36:40.477027+0000","last_peered":"2024-04-16T19:36:40.477027+0000","last_clean":"2024-04-16T19:36:40.477027+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:36:40.477027+0000","last_undegraded":"2024-04-16T19:36:40.477027+0000","last_fullsized":"2024-04-16T19:36:40.477027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.476892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:55.189016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980695+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:36:17.980695+0000","last_peered":"2024-04-16T19:36:17.980695+0000","last_clean":"2024-04-16T19:36:17.980695+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:36:17.980695+0000","last_undegraded":"2024-04-16T19:36:17.980695+0000","last_fullsized":"2024-04-16T19:36:17.980695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'140","reported_seq":1796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.660278+0000","last_change":"2024-04-16T19:37:00.660381+0000","last_active":"2024-04-16T19:37:00.660278+0000","last_peered":"2024-04-16T19:37:00.660278+0000","last_clean":"2024-04-16T19:37:00.660278+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:37:00.660278+0000","last_undegraded":"2024-04-16T19:37:00.660278+0000","last_fullsized":"2024-04-16T19:37:00.660278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'140","last_scrub_stamp":"2024-04-16T19:37:00.660142+0000","last_deep_scrub":"53'140","last_deep_scrub_stamp":"2024-04-16T19:37:00.660142+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.660142+0000","objects_scrubbed":40,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:07.318185+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9228,"num_omap_keys":36,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.670649+0000","last_change":"2024-04-16T19:36:37.670700+0000","last_active":"2024-04-16T19:36:37.670649+0000","last_peered":"2024-04-16T19:36:37.670649+0000","last_clean":"2024-04-16T19:36:37.670649+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:36:37.670649+0000","last_undegraded":"2024-04-16T19:36:37.670649+0000","last_fullsized":"2024-04-16T19:36:37.670649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.670504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:43.115925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'14","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.601233+0000","last_change":"2024-04-16T19:36:56.601312+0000","last_active":"2024-04-16T19:36:56.601233+0000","last_peered":"2024-04-16T19:36:56.601233+0000","last_clean":"2024-04-16T19:36:56.601233+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:36:56.601233+0000","last_undegraded":"2024-04-16T19:36:56.601233+0000","last_fullsized":"2024-04-16T19:36:56.601233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:56.601090+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:56.601090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.601090+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:12.535988+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.465640+0000","last_change":"2024-04-16T19:36:41.465695+0000","last_active":"2024-04-16T19:36:41.465640+0000","last_peered":"2024-04-16T19:36:41.465640+0000","last_clean":"2024-04-16T19:36:41.465640+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:36:41.465640+0000","last_undegraded":"2024-04-16T19:36:41.465640+0000","last_fullsized":"2024-04-16T19:36:41.465640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.465493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:42:13.407368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'925","reported_seq":3830,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.639219+0000","last_change":"2024-04-16T19:36:25.639219+0000","last_active":"2024-04-16T19:36:25.639219+0000","last_peered":"2024-04-16T19:36:25.639219+0000","last_clean":"2024-04-16T19:36:25.639219+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:36:25.639219+0000","last_undegraded":"2024-04-16T19:36:25.639219+0000","last_fullsized":"2024-04-16T19:36:25.639219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.639013+0000","objects_scrubbed":28,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:11.472224+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89384,"num_omap_keys":416,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'36","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.510632+0000","last_change":"2024-04-16T19:36:24.510731+0000","last_active":"2024-04-16T19:36:24.510632+0000","last_peered":"2024-04-16T19:36:24.510632+0000","last_clean":"2024-04-16T19:36:24.510632+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:36:24.510632+0000","last_undegraded":"2024-04-16T19:36:24.510632+0000","last_fullsized":"2024-04-16T19:36:24.510632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'36","last_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_deep_scrub":"53'36","last_deep_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.510495+0000","objects_scrubbed":7,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:21.114823+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.207927+0000","last_change":"2024-04-16T17:47:23.260237+0000","last_active":"2024-04-16T19:35:46.207927+0000","last_peered":"2024-04-16T19:35:46.207927+0000","last_clean":"2024-04-16T19:35:46.207927+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:35:46.207927+0000","last_undegraded":"2024-04-16T19:35:46.207927+0000","last_fullsized":"2024-04-16T19:35:46.207927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911512+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:36:18.911512+0000","last_peered":"2024-04-16T19:36:18.911512+0000","last_clean":"2024-04-16T19:36:18.911512+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:36:18.911512+0000","last_undegraded":"2024-04-16T19:36:18.911512+0000","last_fullsized":"2024-04-16T19:36:18.911512+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200997+0000","last_change":"2024-04-16T17:47:21.234640+0000","last_active":"2024-04-16T19:35:41.200997+0000","last_peered":"2024-04-16T19:35:41.200997+0000","last_clean":"2024-04-16T19:35:41.200997+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:35:41.200997+0000","last_undegraded":"2024-04-16T19:35:41.200997+0000","last_fullsized":"2024-04-16T19:35:41.200997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"48'1","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586344+0000","last_change":"2024-04-16T17:47:33.303188+0000","last_active":"2024-04-16T19:36:16.586344+0000","last_peered":"2024-04-16T19:36:16.586344+0000","last_clean":"2024-04-16T19:36:16.586344+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:36:16.586344+0000","last_undegraded":"2024-04-16T19:36:16.586344+0000","last_fullsized":"2024-04-16T19:36:16.586344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'150","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.635861+0000","last_change":"2024-04-16T19:37:01.635936+0000","last_active":"2024-04-16T19:37:01.635861+0000","last_peered":"2024-04-16T19:37:01.635861+0000","last_clean":"2024-04-16T19:37:01.635861+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:37:01.635861+0000","last_undegraded":"2024-04-16T19:37:01.635861+0000","last_fullsized":"2024-04-16T19:37:01.635861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'150","last_scrub_stamp":"2024-04-16T19:37:01.635730+0000","last_deep_scrub":"53'150","last_deep_scrub_stamp":"2024-04-16T19:37:01.635730+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.635730+0000","objects_scrubbed":6,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:47:12.500214+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.356065+0000","last_change":"2024-04-16T19:36:43.356174+0000","last_active":"2024-04-16T19:36:43.356065+0000","last_peered":"2024-04-16T19:36:43.356065+0000","last_clean":"2024-04-16T19:36:43.356065+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:36:43.356065+0000","last_undegraded":"2024-04-16T19:36:43.356065+0000","last_fullsized":"2024-04-16T19:36:43.356065+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.355868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:33.637719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.430570+0000","last_change":"2024-04-16T19:36:44.430621+0000","last_active":"2024-04-16T19:36:44.430570+0000","last_peered":"2024-04-16T19:36:44.430570+0000","last_clean":"2024-04-16T19:36:44.430570+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:36:44.430570+0000","last_undegraded":"2024-04-16T19:36:44.430570+0000","last_fullsized":"2024-04-16T19:36:44.430570+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.430438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:03.942655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915237+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:36:18.915237+0000","last_peered":"2024-04-16T19:36:18.915237+0000","last_clean":"2024-04-16T19:36:18.915237+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:36:18.915237+0000","last_undegraded":"2024-04-16T19:36:18.915237+0000","last_fullsized":"2024-04-16T19:36:18.915237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914927+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:36:18.914927+0000","last_peered":"2024-04-16T19:36:18.914927+0000","last_clean":"2024-04-16T19:36:18.914927+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:36:18.914927+0000","last_undegraded":"2024-04-16T19:36:18.914927+0000","last_fullsized":"2024-04-16T19:36:18.914927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.359735+0000","last_change":"2024-04-16T19:37:07.359795+0000","last_active":"2024-04-16T19:37:07.359735+0000","last_peered":"2024-04-16T19:37:07.359735+0000","last_clean":"2024-04-16T19:37:07.359735+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:37:07.359735+0000","last_undegraded":"2024-04-16T19:37:07.359735+0000","last_fullsized":"2024-04-16T19:37:07.359735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.359591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.359591+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.359591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:05.698934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'42","reported_seq":213,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.456089+0000","last_change":"2024-04-16T19:36:54.456164+0000","last_active":"2024-04-16T19:36:54.456089+0000","last_peered":"2024-04-16T19:36:54.456089+0000","last_clean":"2024-04-16T19:36:54.456089+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:36:54.456089+0000","last_undegraded":"2024-04-16T19:36:54.456089+0000","last_fullsized":"2024-04-16T19:36:54.456089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'42","last_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_deep_scrub":"53'42","last_deep_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.455949+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:53.521634+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.526931+0000","last_change":"2024-04-16T19:36:34.526983+0000","last_active":"2024-04-16T19:36:34.526931+0000","last_peered":"2024-04-16T19:36:34.526931+0000","last_clean":"2024-04-16T19:36:34.526931+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:36:34.526931+0000","last_undegraded":"2024-04-16T19:36:34.526931+0000","last_fullsized":"2024-04-16T19:36:34.526931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.526798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:20.876919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'282","reported_seq":668,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.418934+0000","last_change":"2024-04-16T19:36:30.419051+0000","last_active":"2024-04-16T19:36:30.418934+0000","last_peered":"2024-04-16T19:36:30.418934+0000","last_clean":"2024-04-16T19:36:30.418934+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:36:30.418934+0000","last_undegraded":"2024-04-16T19:36:30.418934+0000","last_fullsized":"2024-04-16T19:36:30.418934+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'282","last_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_deep_scrub":"53'282","last_deep_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.418793+0000","objects_scrubbed":18,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:10.455320+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'16","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.403701+0000","last_change":"2024-04-16T19:36:29.403701+0000","last_active":"2024-04-16T19:36:29.403701+0000","last_peered":"2024-04-16T19:36:29.403701+0000","last_clean":"2024-04-16T19:36:29.403701+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:36:29.403701+0000","last_undegraded":"2024-04-16T19:36:29.403701+0000","last_fullsized":"2024-04-16T19:36:29.403701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.403484+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:20.878671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.632868+0000","last_change":"2024-04-16T19:37:13.632975+0000","last_active":"2024-04-16T19:37:13.632868+0000","last_peered":"2024-04-16T19:37:13.632868+0000","last_clean":"2024-04-16T19:37:13.632868+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:37:13.632868+0000","last_undegraded":"2024-04-16T19:37:13.632868+0000","last_fullsized":"2024-04-16T19:37:13.632868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.632699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.632699+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.632699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:35.211444+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.437342+0000","last_change":"2024-04-16T17:46:37.574813+0000","last_active":"2024-04-16T19:36:18.437342+0000","last_peered":"2024-04-16T19:36:18.437342+0000","last_clean":"2024-04-16T19:36:18.437342+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:36:18.437342+0000","last_undegraded":"2024-04-16T19:36:18.437342+0000","last_fullsized":"2024-04-16T19:36:18.437342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209117+0000","last_change":"2024-04-16T17:47:21.229740+0000","last_active":"2024-04-16T19:35:51.209117+0000","last_peered":"2024-04-16T19:35:51.209117+0000","last_clean":"2024-04-16T19:35:51.209117+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:35:51.209117+0000","last_undegraded":"2024-04-16T19:35:51.209117+0000","last_fullsized":"2024-04-16T19:35:51.209117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"53'6","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.409314+0000","last_change":"2024-04-16T19:37:10.409384+0000","last_active":"2024-04-16T19:37:10.409314+0000","last_peered":"2024-04-16T19:37:10.409314+0000","last_clean":"2024-04-16T19:37:10.409314+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:37:10.409314+0000","last_undegraded":"2024-04-16T19:37:10.409314+0000","last_fullsized":"2024-04-16T19:37:10.409314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:10.409173+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:10.409173+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.409173+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:37.030711+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'159","reported_seq":1573,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.333313+0000","last_change":"2024-04-16T19:37:06.333399+0000","last_active":"2024-04-16T19:37:06.333313+0000","last_peered":"2024-04-16T19:37:06.333313+0000","last_clean":"2024-04-16T19:37:06.333313+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:37:06.333313+0000","last_undegraded":"2024-04-16T19:37:06.333313+0000","last_fullsized":"2024-04-16T19:37:06.333313+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:06.333171+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:06.333171+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.333171+0000","objects_scrubbed":8,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:29.002269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.544890+0000","last_change":"2024-04-16T19:37:02.544990+0000","last_active":"2024-04-16T19:37:02.544890+0000","last_peered":"2024-04-16T19:37:02.544890+0000","last_clean":"2024-04-16T19:37:02.544890+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:37:02.544890+0000","last_undegraded":"2024-04-16T19:37:02.544890+0000","last_fullsized":"2024-04-16T19:37:02.544890+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.544686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.544686+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.544686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:25.213783+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.593720+0000","last_change":"2024-04-16T19:36:40.593776+0000","last_active":"2024-04-16T19:36:40.593720+0000","last_peered":"2024-04-16T19:36:40.593720+0000","last_clean":"2024-04-16T19:36:40.593720+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:36:40.593720+0000","last_undegraded":"2024-04-16T19:36:40.593720+0000","last_fullsized":"2024-04-16T19:36:40.593720+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.593577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.105795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.440188+0000","last_change":"2024-04-16T18:10:50.548694+0000","last_active":"2024-04-16T19:36:18.440188+0000","last_peered":"2024-04-16T19:36:18.440188+0000","last_clean":"2024-04-16T19:36:18.440188+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:36:18.440188+0000","last_undegraded":"2024-04-16T19:36:18.440188+0000","last_fullsized":"2024-04-16T19:36:18.440188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'106","reported_seq":1199,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.610166+0000","last_change":"2024-04-16T19:36:57.610238+0000","last_active":"2024-04-16T19:36:57.610166+0000","last_peered":"2024-04-16T19:36:57.610166+0000","last_clean":"2024-04-16T19:36:57.610166+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:36:57.610166+0000","last_undegraded":"2024-04-16T19:36:57.610166+0000","last_fullsized":"2024-04-16T19:36:57.610166+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'106","last_scrub_stamp":"2024-04-16T19:36:57.610032+0000","last_deep_scrub":"53'106","last_deep_scrub_stamp":"2024-04-16T19:36:57.610032+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.610032+0000","objects_scrubbed":30,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:04.921796+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6018,"num_omap_keys":24,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.396377+0000","last_change":"2024-04-16T19:36:50.396377+0000","last_active":"2024-04-16T19:36:50.396377+0000","last_peered":"2024-04-16T19:36:50.396377+0000","last_clean":"2024-04-16T19:36:50.396377+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:36:50.396377+0000","last_undegraded":"2024-04-16T19:36:50.396377+0000","last_fullsized":"2024-04-16T19:36:50.396377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.396177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:28.233570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.563153+0000","last_change":"2024-04-16T19:36:58.563232+0000","last_active":"2024-04-16T19:36:58.563153+0000","last_peered":"2024-04-16T19:36:58.563153+0000","last_clean":"2024-04-16T19:36:58.563153+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:36:58.563153+0000","last_undegraded":"2024-04-16T19:36:58.563153+0000","last_fullsized":"2024-04-16T19:36:58.563153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:58.563020+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:58.563020+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.563020+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:47:33.896197+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'556","reported_seq":1943,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.469613+0000","last_change":"2024-04-16T19:36:37.469677+0000","last_active":"2024-04-16T19:36:37.469613+0000","last_peered":"2024-04-16T19:36:37.469613+0000","last_clean":"2024-04-16T19:36:37.469613+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:36:37.469613+0000","last_undegraded":"2024-04-16T19:36:37.469613+0000","last_fullsized":"2024-04-16T19:36:37.469613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.469478+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:42.920753+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'327","reported_seq":654,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.537153+0000","last_change":"2024-04-16T19:36:23.537297+0000","last_active":"2024-04-16T19:36:23.537153+0000","last_peered":"2024-04-16T19:36:23.537153+0000","last_clean":"2024-04-16T19:36:23.537153+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:36:23.537153+0000","last_undegraded":"2024-04-16T19:36:23.537153+0000","last_fullsized":"2024-04-16T19:36:23.537153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'327","last_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_deep_scrub":"53'327","last_deep_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.537014+0000","objects_scrubbed":21,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:14.538959+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'7","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.487632+0000","last_change":"2024-04-16T19:36:18.487675+0000","last_active":"2024-04-16T19:36:18.487632+0000","last_peered":"2024-04-16T19:36:18.487632+0000","last_clean":"2024-04-16T19:36:18.487632+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:36:18.487632+0000","last_undegraded":"2024-04-16T19:36:18.487632+0000","last_fullsized":"2024-04-16T19:36:18.487632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.487557+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:46.066228+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206529+0000","last_change":"2024-04-16T17:47:23.267253+0000","last_active":"2024-04-16T19:35:46.206529+0000","last_peered":"2024-04-16T19:35:46.206529+0000","last_clean":"2024-04-16T19:35:46.206529+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:35:46.206529+0000","last_undegraded":"2024-04-16T19:35:46.206529+0000","last_fullsized":"2024-04-16T19:35:46.206529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048360+0000","last_change":"2024-04-16T17:46:37.511342+0000","last_active":"2024-04-16T19:36:17.048360+0000","last_peered":"2024-04-16T19:36:17.048360+0000","last_clean":"2024-04-16T19:36:17.048360+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:36:17.048360+0000","last_undegraded":"2024-04-16T19:36:17.048360+0000","last_fullsized":"2024-04-16T19:36:17.048360+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":294,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977594+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:36:17.977594+0000","last_peered":"2024-04-16T19:36:17.977594+0000","last_clean":"2024-04-16T19:36:17.977594+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:36:17.977594+0000","last_undegraded":"2024-04-16T19:36:17.977594+0000","last_fullsized":"2024-04-16T19:36:17.977594+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205065+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:35:41.205065+0000","last_peered":"2024-04-16T19:35:41.205065+0000","last_clean":"2024-04-16T19:35:41.205065+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:35:41.205065+0000","last_undegraded":"2024-04-16T19:35:41.205065+0000","last_fullsized":"2024-04-16T19:35:41.205065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"48'6","reported_seq":505,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208846+0000","last_change":"2024-04-16T17:47:33.303062+0000","last_active":"2024-04-16T19:35:56.208846+0000","last_peered":"2024-04-16T19:35:56.208846+0000","last_clean":"2024-04-16T19:35:56.208846+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:35:56.208846+0000","last_undegraded":"2024-04-16T19:35:56.208846+0000","last_fullsized":"2024-04-16T19:35:56.208846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'187","reported_seq":1631,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.658006+0000","last_change":"2024-04-16T19:37:02.658104+0000","last_active":"2024-04-16T19:37:02.658006+0000","last_peered":"2024-04-16T19:37:02.658006+0000","last_clean":"2024-04-16T19:37:02.658006+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:37:02.658006+0000","last_undegraded":"2024-04-16T19:37:02.658006+0000","last_fullsized":"2024-04-16T19:37:02.658006+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'187","last_scrub_stamp":"2024-04-16T19:37:02.657862+0000","last_deep_scrub":"53'187","last_deep_scrub_stamp":"2024-04-16T19:37:02.657862+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.657862+0000","objects_scrubbed":10,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:00.007944+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.591692+0000","last_change":"2024-04-16T19:36:54.591795+0000","last_active":"2024-04-16T19:36:54.591692+0000","last_peered":"2024-04-16T19:36:54.591692+0000","last_clean":"2024-04-16T19:36:54.591692+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:36:54.591692+0000","last_undegraded":"2024-04-16T19:36:54.591692+0000","last_fullsized":"2024-04-16T19:36:54.591692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.591530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:30.795070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.464452+0000","last_change":"2024-04-16T19:36:28.464452+0000","last_active":"2024-04-16T19:36:28.464452+0000","last_peered":"2024-04-16T19:36:28.464452+0000","last_clean":"2024-04-16T19:36:28.464452+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:36:28.464452+0000","last_undegraded":"2024-04-16T19:36:28.464452+0000","last_fullsized":"2024-04-16T19:36:28.464452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.464263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:07.132103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.188297+0000","last_change":"2024-04-16T18:10:50.544693+0000","last_active":"2024-04-16T19:35:51.188297+0000","last_peered":"2024-04-16T19:35:51.188297+0000","last_clean":"2024-04-16T19:35:51.188297+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:35:51.188297+0000","last_undegraded":"2024-04-16T19:35:51.188297+0000","last_fullsized":"2024-04-16T19:35:51.188297+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1180,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588024+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:36:16.588024+0000","last_peered":"2024-04-16T19:36:16.588024+0000","last_clean":"2024-04-16T19:36:16.588024+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:36:16.588024+0000","last_undegraded":"2024-04-16T19:36:16.588024+0000","last_fullsized":"2024-04-16T19:36:16.588024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.422295+0000","last_change":"2024-04-16T19:36:40.422295+0000","last_active":"2024-04-16T19:36:40.422295+0000","last_peered":"2024-04-16T19:36:40.422295+0000","last_clean":"2024-04-16T19:36:40.422295+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:36:40.422295+0000","last_undegraded":"2024-04-16T19:36:40.422295+0000","last_fullsized":"2024-04-16T19:36:40.422295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.422090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:53.341423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'19","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.529264+0000","last_change":"2024-04-16T19:36:30.529412+0000","last_active":"2024-04-16T19:36:30.529264+0000","last_peered":"2024-04-16T19:36:30.529264+0000","last_clean":"2024-04-16T19:36:30.529264+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:36:30.529264+0000","last_undegraded":"2024-04-16T19:36:30.529264+0000","last_fullsized":"2024-04-16T19:36:30.529264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.529097+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:25.629336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'556","reported_seq":1933,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.659990+0000","last_change":"2024-04-16T19:36:28.660054+0000","last_active":"2024-04-16T19:36:28.659990+0000","last_peered":"2024-04-16T19:36:28.659990+0000","last_clean":"2024-04-16T19:36:28.659990+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:36:28.659990+0000","last_undegraded":"2024-04-16T19:36:28.659990+0000","last_fullsized":"2024-04-16T19:36:28.659990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.659827+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:31.852118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'532","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.416491+0000","last_change":"2024-04-16T19:36:24.416491+0000","last_active":"2024-04-16T19:36:24.416491+0000","last_peered":"2024-04-16T19:36:24.416491+0000","last_clean":"2024-04-16T19:36:24.416491+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:36:24.416491+0000","last_undegraded":"2024-04-16T19:36:24.416491+0000","last_fullsized":"2024-04-16T19:36:24.416491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.416318+0000","objects_scrubbed":18,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:15.418263+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"53'6","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.488767+0000","last_change":"2024-04-16T19:36:23.488844+0000","last_active":"2024-04-16T19:36:23.488767+0000","last_peered":"2024-04-16T19:36:23.488767+0000","last_clean":"2024-04-16T19:36:23.488767+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:36:23.488767+0000","last_undegraded":"2024-04-16T19:36:23.488767+0000","last_fullsized":"2024-04-16T19:36:23.488767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.488637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:34:36.790606+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.610550+0000","last_change":"2024-04-16T19:37:10.610651+0000","last_active":"2024-04-16T19:37:10.610550+0000","last_peered":"2024-04-16T19:37:10.610550+0000","last_clean":"2024-04-16T19:37:10.610550+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:37:10.610550+0000","last_undegraded":"2024-04-16T19:37:10.610550+0000","last_fullsized":"2024-04-16T19:37:10.610550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:10.610381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:10.610381+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.610381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:44.952386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.387252+0000","last_change":"2024-04-16T19:37:03.387307+0000","last_active":"2024-04-16T19:37:03.387252+0000","last_peered":"2024-04-16T19:37:03.387252+0000","last_clean":"2024-04-16T19:37:03.387252+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:37:03.387252+0000","last_undegraded":"2024-04-16T19:37:03.387252+0000","last_fullsized":"2024-04-16T19:37:03.387252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.387109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.387109+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.387109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:37.698223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048265+0000","last_change":"2024-04-16T17:46:37.511500+0000","last_active":"2024-04-16T19:36:17.048265+0000","last_peered":"2024-04-16T19:36:17.048265+0000","last_clean":"2024-04-16T19:36:17.048265+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:36:17.048265+0000","last_undegraded":"2024-04-16T19:36:17.048265+0000","last_fullsized":"2024-04-16T19:36:17.048265+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.417938+0000","last_change":"2024-04-16T19:37:06.418036+0000","last_active":"2024-04-16T19:37:06.417938+0000","last_peered":"2024-04-16T19:37:06.417938+0000","last_clean":"2024-04-16T19:37:06.417938+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:37:06.417938+0000","last_undegraded":"2024-04-16T19:37:06.417938+0000","last_fullsized":"2024-04-16T19:37:06.417938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:06.417783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:06.417783+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.417783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:31.811569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'152","reported_seq":364,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.627842+0000","last_change":"2024-04-16T19:37:03.627959+0000","last_active":"2024-04-16T19:37:03.627842+0000","last_peered":"2024-04-16T19:37:03.627842+0000","last_clean":"2024-04-16T19:37:03.627842+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:37:03.627842+0000","last_undegraded":"2024-04-16T19:37:03.627842+0000","last_fullsized":"2024-04-16T19:37:03.627842+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'152","last_scrub_stamp":"2024-04-16T19:37:03.627704+0000","last_deep_scrub":"53'152","last_deep_scrub_stamp":"2024-04-16T19:37:03.627704+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.627704+0000","objects_scrubbed":8,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:06.610421+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.468997+0000","last_change":"2024-04-16T19:36:50.469053+0000","last_active":"2024-04-16T19:36:50.468997+0000","last_peered":"2024-04-16T19:36:50.468997+0000","last_clean":"2024-04-16T19:36:50.468997+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:36:50.468997+0000","last_undegraded":"2024-04-16T19:36:50.468997+0000","last_fullsized":"2024-04-16T19:36:50.468997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.468857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:29.956415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.495260+0000","last_change":"2024-04-16T19:36:38.495312+0000","last_active":"2024-04-16T19:36:38.495260+0000","last_peered":"2024-04-16T19:36:38.495260+0000","last_clean":"2024-04-16T19:36:38.495260+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:36:38.495260+0000","last_undegraded":"2024-04-16T19:36:38.495260+0000","last_fullsized":"2024-04-16T19:36:38.495260+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.495125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:12.806239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528888+0000","last_change":"2024-04-16T18:10:50.544394+0000","last_active":"2024-04-16T19:36:17.528888+0000","last_peered":"2024-04-16T19:36:17.528888+0000","last_clean":"2024-04-16T19:36:17.528888+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:36:17.528888+0000","last_undegraded":"2024-04-16T19:36:17.528888+0000","last_fullsized":"2024-04-16T19:36:17.528888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'152","reported_seq":1803,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.565649+0000","last_change":"2024-04-16T19:37:12.565726+0000","last_active":"2024-04-16T19:37:12.565649+0000","last_peered":"2024-04-16T19:37:12.565649+0000","last_clean":"2024-04-16T19:37:12.565649+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:37:12.565649+0000","last_undegraded":"2024-04-16T19:37:12.565649+0000","last_fullsized":"2024-04-16T19:37:12.565649+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'152","last_scrub_stamp":"2024-04-16T19:37:12.565502+0000","last_deep_scrub":"53'152","last_deep_scrub_stamp":"2024-04-16T19:37:12.565502+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.565502+0000","objects_scrubbed":30,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:48.502447+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7790,"num_omap_keys":33,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.400809+0000","last_change":"2024-04-16T19:36:49.400809+0000","last_active":"2024-04-16T19:36:49.400809+0000","last_peered":"2024-04-16T19:36:49.400809+0000","last_clean":"2024-04-16T19:36:49.400809+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:36:49.400809+0000","last_undegraded":"2024-04-16T19:36:49.400809+0000","last_fullsized":"2024-04-16T19:36:49.400809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.400614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:52.383331+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'20","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.530462+0000","last_change":"2024-04-16T19:36:31.530526+0000","last_active":"2024-04-16T19:36:31.530462+0000","last_peered":"2024-04-16T19:36:31.530462+0000","last_clean":"2024-04-16T19:36:31.530462+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:36:31.530462+0000","last_undegraded":"2024-04-16T19:36:31.530462+0000","last_fullsized":"2024-04-16T19:36:31.530462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.530356+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:33:36.382371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.461572+0000","last_change":"2024-04-16T19:36:57.461629+0000","last_active":"2024-04-16T19:36:57.461572+0000","last_peered":"2024-04-16T19:36:57.461572+0000","last_clean":"2024-04-16T19:36:57.461572+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:36:57.461572+0000","last_undegraded":"2024-04-16T19:36:57.461572+0000","last_fullsized":"2024-04-16T19:36:57.461572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:57.461431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:57.461431+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.461431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:04.646645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1182","reported_seq":1443,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.649769+0000","last_change":"2024-04-16T19:36:23.649928+0000","last_active":"2024-04-16T19:36:23.649769+0000","last_peered":"2024-04-16T19:36:23.649769+0000","last_clean":"2024-04-16T19:36:23.649769+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:36:23.649769+0000","last_undegraded":"2024-04-16T19:36:23.649769+0000","last_fullsized":"2024-04-16T19:36:23.649769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1182","last_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_deep_scrub":"53'1182","last_deep_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.649623+0000","objects_scrubbed":23,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:09.999744+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215927,"num_omap_keys":942,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'28","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.423045+0000","last_change":"2024-04-16T19:36:25.423158+0000","last_active":"2024-04-16T19:36:25.423045+0000","last_peered":"2024-04-16T19:36:25.423045+0000","last_clean":"2024-04-16T19:36:25.423045+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:36:25.423045+0000","last_undegraded":"2024-04-16T19:36:25.423045+0000","last_fullsized":"2024-04-16T19:36:25.423045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'28","last_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_deep_scrub":"53'28","last_deep_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.422911+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:22.027239+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.209861+0000","last_change":"2024-04-16T17:47:23.262676+0000","last_active":"2024-04-16T19:36:01.209861+0000","last_peered":"2024-04-16T19:36:01.209861+0000","last_clean":"2024-04-16T19:36:01.209861+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:36:01.209861+0000","last_undegraded":"2024-04-16T19:36:01.209861+0000","last_fullsized":"2024-04-16T19:36:01.209861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186094+0000","last_change":"2024-04-16T17:47:21.232419+0000","last_active":"2024-04-16T19:35:41.186094+0000","last_peered":"2024-04-16T19:35:41.186094+0000","last_clean":"2024-04-16T19:35:41.186094+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:35:41.186094+0000","last_undegraded":"2024-04-16T19:35:41.186094+0000","last_fullsized":"2024-04-16T19:35:41.186094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:36.199839+0000","last_change":"2024-04-16T17:46:38.111986+0000","last_active":"2024-04-16T19:35:36.199839+0000","last_peered":"2024-04-16T19:35:36.199839+0000","last_clean":"2024-04-16T19:35:36.199839+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:35:36.199839+0000","last_undegraded":"2024-04-16T19:35:36.199839+0000","last_fullsized":"2024-04-16T19:35:36.199839+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.543234+0000","last_change":"2024-04-16T19:37:14.543287+0000","last_active":"2024-04-16T19:37:14.543234+0000","last_peered":"2024-04-16T19:37:14.543234+0000","last_clean":"2024-04-16T19:37:14.543234+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:37:14.543234+0000","last_undegraded":"2024-04-16T19:37:14.543234+0000","last_fullsized":"2024-04-16T19:37:14.543234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.543097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.543097+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.543097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:55.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'125","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.486453+0000","last_change":"2024-04-16T19:37:10.486531+0000","last_active":"2024-04-16T19:37:10.486453+0000","last_peered":"2024-04-16T19:37:10.486453+0000","last_clean":"2024-04-16T19:37:10.486453+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:37:10.486453+0000","last_undegraded":"2024-04-16T19:37:10.486453+0000","last_fullsized":"2024-04-16T19:37:10.486453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:37:10.486312+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:37:10.486312+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.486312+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:14.882954+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.688114+0000","last_change":"2024-04-16T19:36:44.688178+0000","last_active":"2024-04-16T19:36:44.688114+0000","last_peered":"2024-04-16T19:36:44.688114+0000","last_clean":"2024-04-16T19:36:44.688114+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:36:44.688114+0000","last_undegraded":"2024-04-16T19:36:44.688114+0000","last_fullsized":"2024-04-16T19:36:44.688114+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.687973+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:44.814523+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.534123+0000","last_change":"2024-04-16T19:36:35.534176+0000","last_active":"2024-04-16T19:36:35.534123+0000","last_peered":"2024-04-16T19:36:35.534123+0000","last_clean":"2024-04-16T19:36:35.534123+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:36:35.534123+0000","last_undegraded":"2024-04-16T19:36:35.534123+0000","last_fullsized":"2024-04-16T19:36:35.534123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.533984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:55.743351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":64,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.406743+0000","last_change":"2024-04-16T19:37:00.406770+0000","last_active":"2024-04-16T19:37:00.406743+0000","last_peered":"2024-04-16T19:37:00.406743+0000","last_clean":"2024-04-16T19:37:00.406743+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:37:00.406743+0000","last_undegraded":"2024-04-16T19:37:00.406743+0000","last_fullsized":"2024-04-16T19:37:00.406743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.406670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.406670+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.406670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:05.819397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1284,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915002+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:36:18.915002+0000","last_peered":"2024-04-16T19:36:18.915002+0000","last_clean":"2024-04-16T19:36:18.915002+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:36:18.915002+0000","last_undegraded":"2024-04-16T19:36:18.915002+0000","last_fullsized":"2024-04-16T19:36:18.915002+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.504746+0000","last_change":"2024-04-16T19:36:44.504798+0000","last_active":"2024-04-16T19:36:44.504746+0000","last_peered":"2024-04-16T19:36:44.504746+0000","last_clean":"2024-04-16T19:36:44.504746+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:36:44.504746+0000","last_undegraded":"2024-04-16T19:36:44.504746+0000","last_fullsized":"2024-04-16T19:36:44.504746+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.504608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:08:17.020106+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'21","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.684658+0000","last_change":"2024-04-16T19:36:52.684736+0000","last_active":"2024-04-16T19:36:52.684658+0000","last_peered":"2024-04-16T19:36:52.684658+0000","last_clean":"2024-04-16T19:36:52.684658+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:36:52.684658+0000","last_undegraded":"2024-04-16T19:36:52.684658+0000","last_fullsized":"2024-04-16T19:36:52.684658+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.684531+0000","objects_scrubbed":8,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:32.649666+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.646374+0000","last_change":"2024-04-16T19:36:45.646430+0000","last_active":"2024-04-16T19:36:45.646374+0000","last_peered":"2024-04-16T19:36:45.646374+0000","last_clean":"2024-04-16T19:36:45.646374+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:36:45.646374+0000","last_undegraded":"2024-04-16T19:36:45.646374+0000","last_fullsized":"2024-04-16T19:36:45.646374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.646241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:48.502408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1343","reported_seq":1906,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.619403+0000","last_change":"2024-04-16T19:36:21.619494+0000","last_active":"2024-04-16T19:36:21.619403+0000","last_peered":"2024-04-16T19:36:21.619403+0000","last_clean":"2024-04-16T19:36:21.619403+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:36:21.619403+0000","last_undegraded":"2024-04-16T19:36:21.619403+0000","last_fullsized":"2024-04-16T19:36:21.619403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1343","last_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_deep_scrub":"53'1343","last_deep_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.619266+0000","objects_scrubbed":29,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:34.145789+0000","scrub_duration":71,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":216307,"num_omap_keys":944,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'8","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.444455+0000","last_change":"2024-04-16T19:36:33.444542+0000","last_active":"2024-04-16T19:36:33.444455+0000","last_peered":"2024-04-16T19:36:33.444455+0000","last_clean":"2024-04-16T19:36:33.444455+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:36:33.444455+0000","last_undegraded":"2024-04-16T19:36:33.444455+0000","last_fullsized":"2024-04-16T19:36:33.444455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.444316+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:36.300483+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206348+0000","last_change":"2024-04-16T17:47:23.259926+0000","last_active":"2024-04-16T19:35:46.206348+0000","last_peered":"2024-04-16T19:35:46.206348+0000","last_clean":"2024-04-16T19:35:46.206348+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:35:46.206348+0000","last_undegraded":"2024-04-16T19:35:46.206348+0000","last_fullsized":"2024-04-16T19:35:46.206348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206492+0000","last_change":"2024-04-16T17:47:21.230826+0000","last_active":"2024-04-16T19:35:41.206492+0000","last_peered":"2024-04-16T19:35:41.206492+0000","last_clean":"2024-04-16T19:35:41.206492+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:35:41.206492+0000","last_undegraded":"2024-04-16T19:35:41.206492+0000","last_fullsized":"2024-04-16T19:35:41.206492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977536+0000","last_change":"2024-04-16T17:46:37.575013+0000","last_active":"2024-04-16T19:36:17.977536+0000","last_peered":"2024-04-16T19:36:17.977536+0000","last_clean":"2024-04-16T19:36:17.977536+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:36:17.977536+0000","last_undegraded":"2024-04-16T19:36:17.977536+0000","last_fullsized":"2024-04-16T19:36:17.977536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208557+0000","last_change":"2024-04-16T17:47:33.300538+0000","last_active":"2024-04-16T19:35:56.208557+0000","last_peered":"2024-04-16T19:35:56.208557+0000","last_clean":"2024-04-16T19:35:56.208557+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:35:56.208557+0000","last_undegraded":"2024-04-16T19:35:56.208557+0000","last_fullsized":"2024-04-16T19:35:56.208557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'175","reported_seq":384,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.762300+0000","last_change":"2024-04-16T19:36:56.762377+0000","last_active":"2024-04-16T19:36:56.762300+0000","last_peered":"2024-04-16T19:36:56.762300+0000","last_clean":"2024-04-16T19:36:56.762300+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:36:56.762300+0000","last_undegraded":"2024-04-16T19:36:56.762300+0000","last_fullsized":"2024-04-16T19:36:56.762300+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'175","last_scrub_stamp":"2024-04-16T19:36:56.762164+0000","last_deep_scrub":"53'175","last_deep_scrub_stamp":"2024-04-16T19:36:56.762164+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.762164+0000","objects_scrubbed":7,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:59.105751+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'13","reported_seq":1352,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.360335+0000","last_change":"2024-04-16T19:37:02.360401+0000","last_active":"2024-04-16T19:37:02.360335+0000","last_peered":"2024-04-16T19:37:02.360335+0000","last_clean":"2024-04-16T19:37:02.360335+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:37:02.360335+0000","last_undegraded":"2024-04-16T19:37:02.360335+0000","last_fullsized":"2024-04-16T19:37:02.360335+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:02.360196+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:02.360196+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.360196+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:23.356000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.501363+0000","last_change":"2024-04-16T19:36:34.501465+0000","last_active":"2024-04-16T19:36:34.501363+0000","last_peered":"2024-04-16T19:36:34.501363+0000","last_clean":"2024-04-16T19:36:34.501363+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:36:34.501363+0000","last_undegraded":"2024-04-16T19:36:34.501363+0000","last_fullsized":"2024-04-16T19:36:34.501363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.501193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:08.812307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.198858+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:35:21.198858+0000","last_peered":"2024-04-16T19:35:21.198858+0000","last_clean":"2024-04-16T19:35:21.198858+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:35:21.198858+0000","last_undegraded":"2024-04-16T19:35:21.198858+0000","last_fullsized":"2024-04-16T19:35:21.198858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587804+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:36:16.587804+0000","last_peered":"2024-04-16T19:36:16.587804+0000","last_clean":"2024-04-16T19:36:16.587804+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:36:16.587804+0000","last_undegraded":"2024-04-16T19:36:16.587804+0000","last_fullsized":"2024-04-16T19:36:16.587804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.397896+0000","last_change":"2024-04-16T19:36:47.398006+0000","last_active":"2024-04-16T19:36:47.397896+0000","last_peered":"2024-04-16T19:36:47.397896+0000","last_clean":"2024-04-16T19:36:47.397896+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:36:47.397896+0000","last_undegraded":"2024-04-16T19:36:47.397896+0000","last_fullsized":"2024-04-16T19:36:47.397896+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.397724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:25.235116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.547894+0000","last_change":"2024-04-16T19:36:32.547894+0000","last_active":"2024-04-16T19:36:32.547894+0000","last_peered":"2024-04-16T19:36:32.547894+0000","last_clean":"2024-04-16T19:36:32.547894+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:36:32.547894+0000","last_undegraded":"2024-04-16T19:36:32.547894+0000","last_fullsized":"2024-04-16T19:36:32.547894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.547672+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:45.074195+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'556","reported_seq":2004,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.565740+0000","last_change":"2024-04-16T19:36:33.565806+0000","last_active":"2024-04-16T19:36:33.565740+0000","last_peered":"2024-04-16T19:36:33.565740+0000","last_clean":"2024-04-16T19:36:33.565740+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:36:33.565740+0000","last_undegraded":"2024-04-16T19:36:33.565740+0000","last_fullsized":"2024-04-16T19:36:33.565740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.565603+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:32.212045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'575","reported_seq":853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.628047+0000","last_change":"2024-04-16T19:36:22.628176+0000","last_active":"2024-04-16T19:36:22.628047+0000","last_peered":"2024-04-16T19:36:22.628047+0000","last_clean":"2024-04-16T19:36:22.628047+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:36:22.628047+0000","last_undegraded":"2024-04-16T19:36:22.628047+0000","last_fullsized":"2024-04-16T19:36:22.628047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'575","last_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_deep_scrub":"53'575","last_deep_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.627889+0000","objects_scrubbed":22,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:21.274331+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71488,"num_omap_keys":312,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'18","reported_seq":261,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.483626+0000","last_change":"2024-04-16T19:36:29.483703+0000","last_active":"2024-04-16T19:36:29.483626+0000","last_peered":"2024-04-16T19:36:29.483626+0000","last_clean":"2024-04-16T19:36:29.483626+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:36:29.483626+0000","last_undegraded":"2024-04-16T19:36:29.483626+0000","last_fullsized":"2024-04-16T19:36:29.483626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.483500+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:36:53.084475+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.621869+0000","last_change":"2024-04-16T19:37:15.621921+0000","last_active":"2024-04-16T19:37:15.621869+0000","last_peered":"2024-04-16T19:37:15.621869+0000","last_clean":"2024-04-16T19:37:15.621869+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:37:15.621869+0000","last_undegraded":"2024-04-16T19:37:15.621869+0000","last_fullsized":"2024-04-16T19:37:15.621869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.621731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.621731+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.621731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:58:34.123975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211258+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:36:06.211258+0000","last_peered":"2024-04-16T19:36:06.211258+0000","last_clean":"2024-04-16T19:36:06.211258+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:36:06.211258+0000","last_undegraded":"2024-04-16T19:36:06.211258+0000","last_fullsized":"2024-04-16T19:36:06.211258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911538+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:36:18.911538+0000","last_peered":"2024-04-16T19:36:18.911538+0000","last_clean":"2024-04-16T19:36:18.911538+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:36:18.911538+0000","last_undegraded":"2024-04-16T19:36:18.911538+0000","last_fullsized":"2024-04-16T19:36:18.911538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.208829+0000","last_change":"2024-04-16T17:47:33.303091+0000","last_active":"2024-04-16T19:35:56.208829+0000","last_peered":"2024-04-16T19:35:56.208829+0000","last_clean":"2024-04-16T19:35:56.208829+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:35:56.208829+0000","last_undegraded":"2024-04-16T19:35:56.208829+0000","last_fullsized":"2024-04-16T19:35:56.208829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'168","reported_seq":385,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.727125+0000","last_change":"2024-04-16T19:37:12.727211+0000","last_active":"2024-04-16T19:37:12.727125+0000","last_peered":"2024-04-16T19:37:12.727125+0000","last_clean":"2024-04-16T19:37:12.727125+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:37:12.727125+0000","last_undegraded":"2024-04-16T19:37:12.727125+0000","last_fullsized":"2024-04-16T19:37:12.727125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'168","last_scrub_stamp":"2024-04-16T19:37:12.726993+0000","last_deep_scrub":"53'168","last_deep_scrub_stamp":"2024-04-16T19:37:12.726993+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.726993+0000","objects_scrubbed":8,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:58.958378+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.606377+0000","last_change":"2024-04-16T19:36:58.606428+0000","last_active":"2024-04-16T19:36:58.606377+0000","last_peered":"2024-04-16T19:36:58.606377+0000","last_clean":"2024-04-16T19:36:58.606377+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:36:58.606377+0000","last_undegraded":"2024-04-16T19:36:58.606377+0000","last_fullsized":"2024-04-16T19:36:58.606377+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.606242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.606242+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.606242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:23.041599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.422338+0000","last_change":"2024-04-16T19:36:32.422395+0000","last_active":"2024-04-16T19:36:32.422338+0000","last_peered":"2024-04-16T19:36:32.422338+0000","last_clean":"2024-04-16T19:36:32.422338+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:36:32.422338+0000","last_undegraded":"2024-04-16T19:36:32.422338+0000","last_fullsized":"2024-04-16T19:36:32.422338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.422189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:35.278356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201369+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:35:26.201369+0000","last_peered":"2024-04-16T19:35:26.201369+0000","last_clean":"2024-04-16T19:35:26.201369+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:35:26.201369+0000","last_undegraded":"2024-04-16T19:35:26.201369+0000","last_fullsized":"2024-04-16T19:35:26.201369+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.206683+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:35:31.206683+0000","last_peered":"2024-04-16T19:35:31.206683+0000","last_clean":"2024-04-16T19:35:31.206683+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:35:31.206683+0000","last_undegraded":"2024-04-16T19:35:31.206683+0000","last_fullsized":"2024-04-16T19:35:31.206683+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.635196+0000","last_change":"2024-04-16T19:36:39.635263+0000","last_active":"2024-04-16T19:36:39.635196+0000","last_peered":"2024-04-16T19:36:39.635196+0000","last_clean":"2024-04-16T19:36:39.635196+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:36:39.635196+0000","last_undegraded":"2024-04-16T19:36:39.635196+0000","last_fullsized":"2024-04-16T19:36:39.635196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.635056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:26.132595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'20","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.709187+0000","last_change":"2024-04-16T19:36:42.709267+0000","last_active":"2024-04-16T19:36:42.709187+0000","last_peered":"2024-04-16T19:36:42.709187+0000","last_clean":"2024-04-16T19:36:42.709187+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:36:42.709187+0000","last_undegraded":"2024-04-16T19:36:42.709187+0000","last_fullsized":"2024-04-16T19:36:42.709187+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.709051+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:57.264715+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.501219+0000","last_change":"2024-04-16T19:36:35.501282+0000","last_active":"2024-04-16T19:36:35.501219+0000","last_peered":"2024-04-16T19:36:35.501219+0000","last_clean":"2024-04-16T19:36:35.501219+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:36:35.501219+0000","last_undegraded":"2024-04-16T19:36:35.501219+0000","last_fullsized":"2024-04-16T19:36:35.501219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.501075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:40.913802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'871","reported_seq":1181,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.697221+0000","last_change":"2024-04-16T19:36:18.697375+0000","last_active":"2024-04-16T19:36:18.697221+0000","last_peered":"2024-04-16T19:36:18.697221+0000","last_clean":"2024-04-16T19:36:18.697221+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:36:18.697221+0000","last_undegraded":"2024-04-16T19:36:18.697221+0000","last_fullsized":"2024-04-16T19:36:18.697221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'871","last_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_deep_scrub":"53'871","last_deep_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.697076+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:49.607583+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143413,"num_omap_keys":626,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'22","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.656560+0000","last_change":"2024-04-16T19:37:04.656657+0000","last_active":"2024-04-16T19:37:04.656560+0000","last_peered":"2024-04-16T19:37:04.656560+0000","last_clean":"2024-04-16T19:37:04.656560+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:37:04.656560+0000","last_undegraded":"2024-04-16T19:37:04.656560+0000","last_fullsized":"2024-04-16T19:37:04.656560+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:37:04.656410+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:37:04.656410+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.656410+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:42.493803+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.570337+0000","last_change":"2024-04-16T19:36:36.570388+0000","last_active":"2024-04-16T19:36:36.570337+0000","last_peered":"2024-04-16T19:36:36.570337+0000","last_clean":"2024-04-16T19:36:36.570337+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:36:36.570337+0000","last_undegraded":"2024-04-16T19:36:36.570337+0000","last_fullsized":"2024-04-16T19:36:36.570337+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.570198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:51.282322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.603234+0000","last_change":"2024-04-16T19:37:11.603285+0000","last_active":"2024-04-16T19:37:11.603234+0000","last_peered":"2024-04-16T19:37:11.603234+0000","last_clean":"2024-04-16T19:37:11.603234+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:37:11.603234+0000","last_undegraded":"2024-04-16T19:37:11.603234+0000","last_fullsized":"2024-04-16T19:37:11.603234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.603093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.603093+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.603093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:10.668778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:16.203208+0000","last_change":"2024-04-16T17:47:21.254124+0000","last_active":"2024-04-16T19:35:16.203208+0000","last_peered":"2024-04-16T19:35:16.203208+0000","last_clean":"2024-04-16T19:35:16.203208+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:35:16.203208+0000","last_undegraded":"2024-04-16T19:35:16.203208+0000","last_fullsized":"2024-04-16T19:35:16.203208+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"53'14","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.665922+0000","last_change":"2024-04-16T19:37:05.665922+0000","last_active":"2024-04-16T19:37:05.665922+0000","last_peered":"2024-04-16T19:37:05.665922+0000","last_clean":"2024-04-16T19:37:05.665922+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:37:05.665922+0000","last_undegraded":"2024-04-16T19:37:05.665922+0000","last_fullsized":"2024-04-16T19:37:05.665922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:37:05.665717+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:37:05.665717+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.665717+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:45.630852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'109","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.370839+0000","last_change":"2024-04-16T19:36:59.370839+0000","last_active":"2024-04-16T19:36:59.370839+0000","last_peered":"2024-04-16T19:36:59.370839+0000","last_clean":"2024-04-16T19:36:59.370839+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:36:59.370839+0000","last_undegraded":"2024-04-16T19:36:59.370839+0000","last_fullsized":"2024-04-16T19:36:59.370839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:59.370626+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:59.370626+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.370626+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:46.735030+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.627219+0000","last_change":"2024-04-16T19:36:50.627271+0000","last_active":"2024-04-16T19:36:50.627219+0000","last_peered":"2024-04-16T19:36:50.627219+0000","last_clean":"2024-04-16T19:36:50.627219+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:36:50.627219+0000","last_undegraded":"2024-04-16T19:36:50.627219+0000","last_fullsized":"2024-04-16T19:36:50.627219+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.627076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:16.020863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204551+0000","last_change":"2024-04-16T18:10:50.555211+0000","last_active":"2024-04-16T19:35:56.204551+0000","last_peered":"2024-04-16T19:35:56.204551+0000","last_clean":"2024-04-16T19:35:56.204551+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:35:56.204551+0000","last_undegraded":"2024-04-16T19:35:56.204551+0000","last_fullsized":"2024-04-16T19:35:56.204551+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'82","reported_seq":708,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.725733+0000","last_change":"2024-04-16T19:37:03.725800+0000","last_active":"2024-04-16T19:37:03.725733+0000","last_peered":"2024-04-16T19:37:03.725733+0000","last_clean":"2024-04-16T19:37:03.725733+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:37:03.725733+0000","last_undegraded":"2024-04-16T19:37:03.725733+0000","last_fullsized":"2024-04-16T19:37:03.725733+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'82","last_scrub_stamp":"2024-04-16T19:37:03.725602+0000","last_deep_scrub":"53'82","last_deep_scrub_stamp":"2024-04-16T19:37:03.725602+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.725602+0000","objects_scrubbed":27,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:37.279204+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3517,"num_omap_keys":14,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.640311+0000","last_change":"2024-04-16T19:36:36.640365+0000","last_active":"2024-04-16T19:36:36.640311+0000","last_peered":"2024-04-16T19:36:36.640311+0000","last_clean":"2024-04-16T19:36:36.640311+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:36:36.640311+0000","last_undegraded":"2024-04-16T19:36:36.640311+0000","last_fullsized":"2024-04-16T19:36:36.640311+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.640174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:51.451059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'15","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.647852+0000","last_change":"2024-04-16T19:36:39.647955+0000","last_active":"2024-04-16T19:36:39.647852+0000","last_peered":"2024-04-16T19:36:39.647852+0000","last_clean":"2024-04-16T19:36:39.647852+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:36:39.647852+0000","last_undegraded":"2024-04-16T19:36:39.647852+0000","last_fullsized":"2024-04-16T19:36:39.647852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.647719+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:50.538220+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.430796+0000","last_change":"2024-04-16T19:36:38.430850+0000","last_active":"2024-04-16T19:36:38.430796+0000","last_peered":"2024-04-16T19:36:38.430796+0000","last_clean":"2024-04-16T19:36:38.430796+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:36:38.430796+0000","last_undegraded":"2024-04-16T19:36:38.430796+0000","last_fullsized":"2024-04-16T19:36:38.430796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.430655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:03.824441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'616","reported_seq":2312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.598175+0000","last_change":"2024-04-16T19:36:23.598300+0000","last_active":"2024-04-16T19:36:23.598175+0000","last_peered":"2024-04-16T19:36:23.598175+0000","last_clean":"2024-04-16T19:36:23.598175+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:36:23.598175+0000","last_undegraded":"2024-04-16T19:36:23.598175+0000","last_fullsized":"2024-04-16T19:36:23.598175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'616","last_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_deep_scrub":"53'616","last_deep_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.598031+0000","objects_scrubbed":24,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:37.526844+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.510530+0000","last_change":"2024-04-16T19:36:23.510613+0000","last_active":"2024-04-16T19:36:23.510530+0000","last_peered":"2024-04-16T19:36:23.510530+0000","last_clean":"2024-04-16T19:36:23.510530+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:36:23.510530+0000","last_undegraded":"2024-04-16T19:36:23.510530+0000","last_fullsized":"2024-04-16T19:36:23.510530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.510390+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:03.621835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.537581+0000","last_change":"2024-04-16T19:36:32.537632+0000","last_active":"2024-04-16T19:36:32.537581+0000","last_peered":"2024-04-16T19:36:32.537581+0000","last_clean":"2024-04-16T19:36:32.537581+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:36:32.537581+0000","last_undegraded":"2024-04-16T19:36:32.537581+0000","last_fullsized":"2024-04-16T19:36:32.537581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.537440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:47.348326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.588585+0000","last_change":"2024-04-16T19:37:12.588636+0000","last_active":"2024-04-16T19:37:12.588585+0000","last_peered":"2024-04-16T19:37:12.588585+0000","last_clean":"2024-04-16T19:37:12.588585+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:37:12.588585+0000","last_undegraded":"2024-04-16T19:37:12.588585+0000","last_fullsized":"2024-04-16T19:37:12.588585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:12.588451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:12.588451+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.588451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:58.868734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215650+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:36:06.215650+0000","last_peered":"2024-04-16T19:36:06.215650+0000","last_clean":"2024-04-16T19:36:06.215650+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:36:06.215650+0000","last_undegraded":"2024-04-16T19:36:06.215650+0000","last_fullsized":"2024-04-16T19:36:06.215650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"53'2","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.615140+0000","last_change":"2024-04-16T19:37:08.615140+0000","last_active":"2024-04-16T19:37:08.615140+0000","last_peered":"2024-04-16T19:37:08.615140+0000","last_clean":"2024-04-16T19:37:08.615140+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:37:08.615140+0000","last_undegraded":"2024-04-16T19:37:08.615140+0000","last_fullsized":"2024-04-16T19:37:08.615140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:08.614950+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:08.614950+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.614950+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:13.456557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'53","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.682005+0000","last_change":"2024-04-16T19:37:11.682005+0000","last_active":"2024-04-16T19:37:11.682005+0000","last_peered":"2024-04-16T19:37:11.682005+0000","last_clean":"2024-04-16T19:37:11.682005+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:37:11.682005+0000","last_undegraded":"2024-04-16T19:37:11.682005+0000","last_fullsized":"2024-04-16T19:37:11.682005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'53","last_scrub_stamp":"2024-04-16T19:37:11.681806+0000","last_deep_scrub":"53'53","last_deep_scrub_stamp":"2024-04-16T19:37:11.681806+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.681806+0000","objects_scrubbed":3,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:47.618751+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.518323+0000","last_change":"2024-04-16T19:37:02.518374+0000","last_active":"2024-04-16T19:37:02.518323+0000","last_peered":"2024-04-16T19:37:02.518323+0000","last_clean":"2024-04-16T19:37:02.518323+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:37:02.518323+0000","last_undegraded":"2024-04-16T19:37:02.518323+0000","last_fullsized":"2024-04-16T19:37:02.518323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.518192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.518192+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.518192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:34.961401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207221+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:35:51.207221+0000","last_peered":"2024-04-16T19:35:51.207221+0000","last_clean":"2024-04-16T19:35:51.207221+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:35:51.207221+0000","last_undegraded":"2024-04-16T19:35:51.207221+0000","last_fullsized":"2024-04-16T19:35:51.207221+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'129","reported_seq":1727,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.778020+0000","last_change":"2024-04-16T19:36:55.778092+0000","last_active":"2024-04-16T19:36:55.778020+0000","last_peered":"2024-04-16T19:36:55.778020+0000","last_clean":"2024-04-16T19:36:55.778020+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:36:55.778020+0000","last_undegraded":"2024-04-16T19:36:55.778020+0000","last_fullsized":"2024-04-16T19:36:55.778020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'129","last_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_deep_scrub":"53'129","last_deep_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.777891+0000","objects_scrubbed":30,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:46.059742+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8321,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.410485+0000","last_change":"2024-04-16T19:36:51.410485+0000","last_active":"2024-04-16T19:36:51.410485+0000","last_peered":"2024-04-16T19:36:51.410485+0000","last_clean":"2024-04-16T19:36:51.410485+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:36:51.410485+0000","last_undegraded":"2024-04-16T19:36:51.410485+0000","last_fullsized":"2024-04-16T19:36:51.410485+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.410273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:57.416580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'18","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.684157+0000","last_change":"2024-04-16T19:36:45.684234+0000","last_active":"2024-04-16T19:36:45.684157+0000","last_peered":"2024-04-16T19:36:45.684157+0000","last_clean":"2024-04-16T19:36:45.684157+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:36:45.684157+0000","last_undegraded":"2024-04-16T19:36:45.684157+0000","last_fullsized":"2024-04-16T19:36:45.684157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.684019+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:54.649522+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'21","reported_seq":1432,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.402575+0000","last_change":"2024-04-16T19:36:43.402646+0000","last_active":"2024-04-16T19:36:43.402575+0000","last_peered":"2024-04-16T19:36:43.402575+0000","last_clean":"2024-04-16T19:36:43.402575+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:36:43.402575+0000","last_undegraded":"2024-04-16T19:36:43.402575+0000","last_fullsized":"2024-04-16T19:36:43.402575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.402444+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:20.659167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'7003","reported_seq":10587,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.479202+0000","last_change":"2024-04-16T19:36:36.479319+0000","last_active":"2024-04-16T19:36:36.479202+0000","last_peered":"2024-04-16T19:36:36.479202+0000","last_clean":"2024-04-16T19:36:36.479202+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:36:36.479202+0000","last_undegraded":"2024-04-16T19:36:36.479202+0000","last_fullsized":"2024-04-16T19:36:36.479202+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7003","last_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_deep_scrub":"53'7003","last_deep_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.479068+0000","objects_scrubbed":20,"log_size":1803,"log_dups_size":3000,"ondisk_log_size":1803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:56.688434+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'10","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.437802+0000","last_change":"2024-04-16T19:36:27.437901+0000","last_active":"2024-04-16T19:36:27.437802+0000","last_peered":"2024-04-16T19:36:27.437802+0000","last_clean":"2024-04-16T19:36:27.437802+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:36:27.437802+0000","last_undegraded":"2024-04-16T19:36:27.437802+0000","last_fullsized":"2024-04-16T19:36:27.437802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.437656+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:14.616289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.669172+0000","last_change":"2024-04-16T19:36:42.669224+0000","last_active":"2024-04-16T19:36:42.669172+0000","last_peered":"2024-04-16T19:36:42.669172+0000","last_clean":"2024-04-16T19:36:42.669172+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:36:42.669172+0000","last_undegraded":"2024-04-16T19:36:42.669172+0000","last_fullsized":"2024-04-16T19:36:42.669172+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.669008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:32.950859+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.392128+0000","last_change":"2024-04-16T19:37:05.392128+0000","last_active":"2024-04-16T19:37:05.392128+0000","last_peered":"2024-04-16T19:37:05.392128+0000","last_clean":"2024-04-16T19:37:05.392128+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:37:05.392128+0000","last_undegraded":"2024-04-16T19:37:05.392128+0000","last_fullsized":"2024-04-16T19:37:05.392128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:05.391932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:05.391932+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.391932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:20.104056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211353+0000","last_change":"2024-04-16T17:47:21.255828+0000","last_active":"2024-04-16T19:36:06.211353+0000","last_peered":"2024-04-16T19:36:06.211353+0000","last_clean":"2024-04-16T19:36:06.211353+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:36:06.211353+0000","last_undegraded":"2024-04-16T19:36:06.211353+0000","last_fullsized":"2024-04-16T19:36:06.211353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438006+0000","last_change":"2024-04-16T17:47:33.297514+0000","last_active":"2024-04-16T19:36:18.438006+0000","last_peered":"2024-04-16T19:36:18.438006+0000","last_clean":"2024-04-16T19:36:18.438006+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:36:18.438006+0000","last_undegraded":"2024-04-16T19:36:18.438006+0000","last_fullsized":"2024-04-16T19:36:18.438006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'109","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.376859+0000","last_change":"2024-04-16T19:36:58.376859+0000","last_active":"2024-04-16T19:36:58.376859+0000","last_peered":"2024-04-16T19:36:58.376859+0000","last_clean":"2024-04-16T19:36:58.376859+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:36:58.376859+0000","last_undegraded":"2024-04-16T19:36:58.376859+0000","last_fullsized":"2024-04-16T19:36:58.376859+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:58.376556+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:58.376556+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.376556+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:38.413082+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.395249+0000","last_change":"2024-04-16T19:37:04.395303+0000","last_active":"2024-04-16T19:37:04.395249+0000","last_peered":"2024-04-16T19:37:04.395249+0000","last_clean":"2024-04-16T19:37:04.395249+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:37:04.395249+0000","last_undegraded":"2024-04-16T19:37:04.395249+0000","last_fullsized":"2024-04-16T19:37:04.395249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.395110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.395110+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.395110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:24:17.565659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.211846+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:35:51.211846+0000","last_peered":"2024-04-16T19:35:51.211846+0000","last_clean":"2024-04-16T19:35:51.211846+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:35:51.211846+0000","last_undegraded":"2024-04-16T19:35:51.211846+0000","last_fullsized":"2024-04-16T19:35:51.211846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1360,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914935+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:36:18.914935+0000","last_peered":"2024-04-16T19:36:18.914935+0000","last_clean":"2024-04-16T19:36:18.914935+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:36:18.914935+0000","last_undegraded":"2024-04-16T19:36:18.914935+0000","last_fullsized":"2024-04-16T19:36:18.914935+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.499740+0000","last_change":"2024-04-16T19:36:36.499794+0000","last_active":"2024-04-16T19:36:36.499740+0000","last_peered":"2024-04-16T19:36:36.499740+0000","last_clean":"2024-04-16T19:36:36.499740+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:36:36.499740+0000","last_undegraded":"2024-04-16T19:36:36.499740+0000","last_fullsized":"2024-04-16T19:36:36.499740+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.499596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:22.332807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"53'16","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.619264+0000","last_change":"2024-04-16T19:36:44.619264+0000","last_active":"2024-04-16T19:36:44.619264+0000","last_peered":"2024-04-16T19:36:44.619264+0000","last_clean":"2024-04-16T19:36:44.619264+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:36:44.619264+0000","last_undegraded":"2024-04-16T19:36:44.619264+0000","last_fullsized":"2024-04-16T19:36:44.619264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.619046+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:50.031773+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'556","reported_seq":1967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.516649+0000","last_change":"2024-04-16T19:36:35.516719+0000","last_active":"2024-04-16T19:36:35.516649+0000","last_peered":"2024-04-16T19:36:35.516649+0000","last_clean":"2024-04-16T19:36:35.516649+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:36:35.516649+0000","last_undegraded":"2024-04-16T19:36:35.516649+0000","last_fullsized":"2024-04-16T19:36:35.516649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.516509+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:41.203861+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'563","reported_seq":804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.482688+0000","last_change":"2024-04-16T19:36:29.482688+0000","last_active":"2024-04-16T19:36:29.482688+0000","last_peered":"2024-04-16T19:36:29.482688+0000","last_clean":"2024-04-16T19:36:29.482688+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:36:29.482688+0000","last_undegraded":"2024-04-16T19:36:29.482688+0000","last_fullsized":"2024-04-16T19:36:29.482688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.482404+0000","objects_scrubbed":19,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.518930+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71952,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'8","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.461125+0000","last_change":"2024-04-16T19:36:21.461301+0000","last_active":"2024-04-16T19:36:21.461125+0000","last_peered":"2024-04-16T19:36:21.461125+0000","last_clean":"2024-04-16T19:36:21.461125+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:36:21.461125+0000","last_undegraded":"2024-04-16T19:36:21.461125+0000","last_fullsized":"2024-04-16T19:36:21.461125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.460913+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:00.948470+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.460735+0000","last_change":"2024-04-16T19:36:45.460790+0000","last_active":"2024-04-16T19:36:45.460735+0000","last_peered":"2024-04-16T19:36:45.460735+0000","last_clean":"2024-04-16T19:36:45.460735+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:36:45.460735+0000","last_undegraded":"2024-04-16T19:36:45.460735+0000","last_fullsized":"2024-04-16T19:36:45.460735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.460592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:12.082129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.596749+0000","last_change":"2024-04-16T19:37:11.596852+0000","last_active":"2024-04-16T19:37:11.596749+0000","last_peered":"2024-04-16T19:37:11.596749+0000","last_clean":"2024-04-16T19:37:11.596749+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:37:11.596749+0000","last_undegraded":"2024-04-16T19:37:11.596749+0000","last_fullsized":"2024-04-16T19:37:11.596749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.596575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.596575+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.596575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:32.592380+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211668+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:36:06.211668+0000","last_peered":"2024-04-16T19:36:06.211668+0000","last_clean":"2024-04-16T19:36:06.211668+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:36:06.211668+0000","last_undegraded":"2024-04-16T19:36:06.211668+0000","last_fullsized":"2024-04-16T19:36:06.211668+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.501299+0000","last_change":"2024-04-16T19:37:11.501299+0000","last_active":"2024-04-16T19:37:11.501299+0000","last_peered":"2024-04-16T19:37:11.501299+0000","last_clean":"2024-04-16T19:37:11.501299+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:37:11.501299+0000","last_undegraded":"2024-04-16T19:37:11.501299+0000","last_fullsized":"2024-04-16T19:37:11.501299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.501103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.501103+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.501103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:40.126284+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'84","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.376165+0000","last_change":"2024-04-16T19:37:09.376165+0000","last_active":"2024-04-16T19:37:09.376165+0000","last_peered":"2024-04-16T19:37:09.376165+0000","last_clean":"2024-04-16T19:37:09.376165+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:37:09.376165+0000","last_undegraded":"2024-04-16T19:37:09.376165+0000","last_fullsized":"2024-04-16T19:37:09.376165+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:09.375852+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:09.375852+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.375852+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:29.911359+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.541710+0000","last_change":"2024-04-16T19:37:00.541762+0000","last_active":"2024-04-16T19:37:00.541710+0000","last_peered":"2024-04-16T19:37:00.541710+0000","last_clean":"2024-04-16T19:37:00.541710+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:37:00.541710+0000","last_undegraded":"2024-04-16T19:37:00.541710+0000","last_fullsized":"2024-04-16T19:37:00.541710+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.541572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.541572+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.541572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:22.120317+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204458+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:35:21.204458+0000","last_peered":"2024-04-16T19:35:21.204458+0000","last_clean":"2024-04-16T19:35:21.204458+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:35:21.204458+0000","last_undegraded":"2024-04-16T19:35:21.204458+0000","last_fullsized":"2024-04-16T19:35:21.204458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'116","reported_seq":731,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.561221+0000","last_change":"2024-04-16T19:37:04.561294+0000","last_active":"2024-04-16T19:37:04.561221+0000","last_peered":"2024-04-16T19:37:04.561221+0000","last_clean":"2024-04-16T19:37:04.561221+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:37:04.561221+0000","last_undegraded":"2024-04-16T19:37:04.561221+0000","last_fullsized":"2024-04-16T19:37:04.561221+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'116","last_scrub_stamp":"2024-04-16T19:37:04.561086+0000","last_deep_scrub":"53'116","last_deep_scrub_stamp":"2024-04-16T19:37:04.561086+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.561086+0000","objects_scrubbed":27,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:02.900429+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5638,"num_omap_keys":24,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.678659+0000","last_change":"2024-04-16T19:36:47.678659+0000","last_active":"2024-04-16T19:36:47.678659+0000","last_peered":"2024-04-16T19:36:47.678659+0000","last_clean":"2024-04-16T19:36:47.678659+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:36:47.678659+0000","last_undegraded":"2024-04-16T19:36:47.678659+0000","last_fullsized":"2024-04-16T19:36:47.678659+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.678408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:02:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.735233+0000","last_change":"2024-04-16T19:36:41.735233+0000","last_active":"2024-04-16T19:36:41.735233+0000","last_peered":"2024-04-16T19:36:41.735233+0000","last_clean":"2024-04-16T19:36:41.735233+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:36:41.735233+0000","last_undegraded":"2024-04-16T19:36:41.735233+0000","last_fullsized":"2024-04-16T19:36:41.735233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.735066+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:52.416728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.470863+0000","last_change":"2024-04-16T19:36:33.470929+0000","last_active":"2024-04-16T19:36:33.470863+0000","last_peered":"2024-04-16T19:36:33.470863+0000","last_clean":"2024-04-16T19:36:33.470863+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:36:33.470863+0000","last_undegraded":"2024-04-16T19:36:33.470863+0000","last_fullsized":"2024-04-16T19:36:33.470863+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.470725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:02.342594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'476","reported_seq":4438,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.660175+0000","last_change":"2024-04-16T19:36:19.660343+0000","last_active":"2024-04-16T19:36:19.660175+0000","last_peered":"2024-04-16T19:36:19.660175+0000","last_clean":"2024-04-16T19:36:19.660175+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:36:19.660175+0000","last_undegraded":"2024-04-16T19:36:19.660175+0000","last_fullsized":"2024-04-16T19:36:19.660175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.659960+0000","objects_scrubbed":19,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:52.175458+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'8","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.516188+0000","last_change":"2024-04-16T19:36:36.516257+0000","last_active":"2024-04-16T19:36:36.516188+0000","last_peered":"2024-04-16T19:36:36.516188+0000","last_clean":"2024-04-16T19:36:36.516188+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:36:36.516188+0000","last_undegraded":"2024-04-16T19:36:36.516188+0000","last_fullsized":"2024-04-16T19:36:36.516188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.516049+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:56.739641+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210764+0000","last_change":"2024-04-16T17:47:23.257867+0000","last_active":"2024-04-16T19:35:46.210764+0000","last_peered":"2024-04-16T19:35:46.210764+0000","last_clean":"2024-04-16T19:35:46.210764+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:35:46.210764+0000","last_undegraded":"2024-04-16T19:35:46.210764+0000","last_fullsized":"2024-04-16T19:35:46.210764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.191930+0000","last_change":"2024-04-16T17:47:21.240002+0000","last_active":"2024-04-16T19:36:06.191930+0000","last_peered":"2024-04-16T19:36:06.191930+0000","last_clean":"2024-04-16T19:36:06.191930+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:36:06.191930+0000","last_undegraded":"2024-04-16T19:36:06.191930+0000","last_fullsized":"2024-04-16T19:36:06.191930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212786+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:36:11.212786+0000","last_peered":"2024-04-16T19:36:11.212786+0000","last_clean":"2024-04-16T19:36:11.212786+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:36:11.212786+0000","last_undegraded":"2024-04-16T19:36:11.212786+0000","last_fullsized":"2024-04-16T19:36:11.212786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'93","reported_seq":263,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.659263+0000","last_change":"2024-04-16T19:37:00.659310+0000","last_active":"2024-04-16T19:37:00.659263+0000","last_peered":"2024-04-16T19:37:00.659263+0000","last_clean":"2024-04-16T19:37:00.659263+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:37:00.659263+0000","last_undegraded":"2024-04-16T19:37:00.659263+0000","last_fullsized":"2024-04-16T19:37:00.659263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'93","last_scrub_stamp":"2024-04-16T19:37:00.659199+0000","last_deep_scrub":"53'93","last_deep_scrub_stamp":"2024-04-16T19:37:00.659199+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.659199+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:06.665506+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.473332+0000","last_change":"2024-04-16T19:37:01.473385+0000","last_active":"2024-04-16T19:37:01.473332+0000","last_peered":"2024-04-16T19:37:01.473332+0000","last_clean":"2024-04-16T19:37:01.473332+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:37:01.473332+0000","last_undegraded":"2024-04-16T19:37:01.473332+0000","last_fullsized":"2024-04-16T19:37:01.473332+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:01.473191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:01.473191+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.473191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:23.051936+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.511170+0000","last_change":"2024-04-16T19:36:25.511222+0000","last_active":"2024-04-16T19:36:25.511170+0000","last_peered":"2024-04-16T19:36:25.511170+0000","last_clean":"2024-04-16T19:36:25.511170+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:36:25.511170+0000","last_undegraded":"2024-04-16T19:36:25.511170+0000","last_fullsized":"2024-04-16T19:36:25.511170+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.511031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:41:57.452906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.200556+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:35:26.200556+0000","last_peered":"2024-04-16T19:35:26.200556+0000","last_clean":"2024-04-16T19:35:26.200556+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:35:26.200556+0000","last_undegraded":"2024-04-16T19:35:26.200556+0000","last_fullsized":"2024-04-16T19:35:26.200556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'102","reported_seq":1008,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.618748+0000","last_change":"2024-04-16T19:36:48.618887+0000","last_active":"2024-04-16T19:36:48.618748+0000","last_peered":"2024-04-16T19:36:48.618748+0000","last_clean":"2024-04-16T19:36:48.618748+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:36:48.618748+0000","last_undegraded":"2024-04-16T19:36:48.618748+0000","last_fullsized":"2024-04-16T19:36:48.618748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'102","last_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_deep_scrub":"53'102","last_deep_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.618582+0000","objects_scrubbed":31,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:54.031308+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7116,"num_omap_keys":28,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.371768+0000","last_change":"2024-04-16T19:37:15.371768+0000","last_active":"2024-04-16T19:37:15.371768+0000","last_peered":"2024-04-16T19:37:15.371768+0000","last_clean":"2024-04-16T19:37:15.371768+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:37:15.371768+0000","last_undegraded":"2024-04-16T19:37:15.371768+0000","last_fullsized":"2024-04-16T19:37:15.371768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.371583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.371583+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.371583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:51.308528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'7","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.481640+0000","last_change":"2024-04-16T19:36:54.481710+0000","last_active":"2024-04-16T19:36:54.481640+0000","last_peered":"2024-04-16T19:36:54.481640+0000","last_clean":"2024-04-16T19:36:54.481640+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:36:54.481640+0000","last_undegraded":"2024-04-16T19:36:54.481640+0000","last_fullsized":"2024-04-16T19:36:54.481640+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.481476+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:28.035078+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.550467+0000","last_change":"2024-04-16T19:36:28.550520+0000","last_active":"2024-04-16T19:36:28.550467+0000","last_peered":"2024-04-16T19:36:28.550467+0000","last_clean":"2024-04-16T19:36:28.550467+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:36:28.550467+0000","last_undegraded":"2024-04-16T19:36:28.550467+0000","last_fullsized":"2024-04-16T19:36:28.550467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.550328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:52.280854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11393","reported_seq":61823,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.677407+0000","last_change":"2024-04-16T19:36:19.677565+0000","last_active":"2024-04-16T19:36:19.677407+0000","last_peered":"2024-04-16T19:36:19.677407+0000","last_clean":"2024-04-16T19:36:19.677407+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:36:19.677407+0000","last_undegraded":"2024-04-16T19:36:19.677407+0000","last_fullsized":"2024-04-16T19:36:19.677407+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11393","last_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_deep_scrub":"53'11393","last_deep_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.677264+0000","objects_scrubbed":23,"log_size":1693,"log_dups_size":3000,"ondisk_log_size":1693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:14.777503+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209607,"num_omap_keys":977,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'23","reported_seq":358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.461538+0000","last_change":"2024-04-16T19:36:18.461701+0000","last_active":"2024-04-16T19:36:18.461538+0000","last_peered":"2024-04-16T19:36:18.461538+0000","last_clean":"2024-04-16T19:36:18.461538+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:36:18.461538+0000","last_undegraded":"2024-04-16T19:36:18.461538+0000","last_fullsized":"2024-04-16T19:36:18.461538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.461393+0000","objects_scrubbed":11,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:24.888175+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:01.210256+0000","last_change":"2024-04-16T17:47:23.262619+0000","last_active":"2024-04-16T19:36:01.210256+0000","last_peered":"2024-04-16T19:36:01.210256+0000","last_clean":"2024-04-16T19:36:01.210256+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:36:01.210256+0000","last_undegraded":"2024-04-16T19:36:01.210256+0000","last_fullsized":"2024-04-16T19:36:01.210256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048372+0000","last_change":"2024-04-16T17:47:21.239750+0000","last_active":"2024-04-16T19:36:17.048372+0000","last_peered":"2024-04-16T19:36:17.048372+0000","last_clean":"2024-04-16T19:36:17.048372+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:36:17.048372+0000","last_undegraded":"2024-04-16T19:36:17.048372+0000","last_fullsized":"2024-04-16T19:36:17.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.570694+0000","last_change":"2024-04-16T19:37:15.570748+0000","last_active":"2024-04-16T19:37:15.570694+0000","last_peered":"2024-04-16T19:37:15.570694+0000","last_clean":"2024-04-16T19:37:15.570694+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:37:15.570694+0000","last_undegraded":"2024-04-16T19:37:15.570694+0000","last_fullsized":"2024-04-16T19:37:15.570694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.570547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.570547+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.570547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:35.464724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'159","reported_seq":356,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.579164+0000","last_change":"2024-04-16T19:37:07.579240+0000","last_active":"2024-04-16T19:37:07.579164+0000","last_peered":"2024-04-16T19:37:07.579164+0000","last_clean":"2024-04-16T19:37:07.579164+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:37:07.579164+0000","last_undegraded":"2024-04-16T19:37:07.579164+0000","last_fullsized":"2024-04-16T19:37:07.579164+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:07.579029+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:07.579029+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.579029+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:29.684066+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.656638+0000","last_change":"2024-04-16T19:36:53.656690+0000","last_active":"2024-04-16T19:36:53.656638+0000","last_peered":"2024-04-16T19:36:53.656638+0000","last_clean":"2024-04-16T19:36:53.656638+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:36:53.656638+0000","last_undegraded":"2024-04-16T19:36:53.656638+0000","last_fullsized":"2024-04-16T19:36:53.656638+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.656501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:12.757030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.426536+0000","last_change":"2024-04-16T19:36:35.426597+0000","last_active":"2024-04-16T19:36:35.426536+0000","last_peered":"2024-04-16T19:36:35.426536+0000","last_clean":"2024-04-16T19:36:35.426536+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:36:35.426536+0000","last_undegraded":"2024-04-16T19:36:35.426536+0000","last_fullsized":"2024-04-16T19:36:35.426536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.426385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:09.737500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201791+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:35:26.201791+0000","last_peered":"2024-04-16T19:35:26.201791+0000","last_clean":"2024-04-16T19:35:26.201791+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:35:26.201791+0000","last_undegraded":"2024-04-16T19:35:26.201791+0000","last_fullsized":"2024-04-16T19:35:26.201791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'143","reported_seq":1869,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.705923+0000","last_change":"2024-04-16T19:36:51.706017+0000","last_active":"2024-04-16T19:36:51.705923+0000","last_peered":"2024-04-16T19:36:51.705923+0000","last_clean":"2024-04-16T19:36:51.705923+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:36:51.705923+0000","last_undegraded":"2024-04-16T19:36:51.705923+0000","last_fullsized":"2024-04-16T19:36:51.705923+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'143","last_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_deep_scrub":"53'143","last_deep_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.705779+0000","objects_scrubbed":40,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:42.464983+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7446,"num_omap_keys":29,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.646565+0000","last_change":"2024-04-16T19:36:48.646617+0000","last_active":"2024-04-16T19:36:48.646565+0000","last_peered":"2024-04-16T19:36:48.646565+0000","last_clean":"2024-04-16T19:36:48.646565+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:36:48.646565+0000","last_undegraded":"2024-04-16T19:36:48.646565+0000","last_fullsized":"2024-04-16T19:36:48.646565+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.646427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:00.794914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'10","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.516567+0000","last_change":"2024-04-16T19:36:56.516633+0000","last_active":"2024-04-16T19:36:56.516567+0000","last_peered":"2024-04-16T19:36:56.516567+0000","last_clean":"2024-04-16T19:36:56.516567+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:36:56.516567+0000","last_undegraded":"2024-04-16T19:36:56.516567+0000","last_fullsized":"2024-04-16T19:36:56.516567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:56.516427+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:56.516427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.516427+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:30.858432+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.416813+0000","last_change":"2024-04-16T19:36:55.416865+0000","last_active":"2024-04-16T19:36:55.416813+0000","last_peered":"2024-04-16T19:36:55.416813+0000","last_clean":"2024-04-16T19:36:55.416813+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:36:55.416813+0000","last_undegraded":"2024-04-16T19:36:55.416813+0000","last_fullsized":"2024-04-16T19:36:55.416813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.416624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:21.421511+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'206","reported_seq":440,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.574363+0000","last_change":"2024-04-16T19:36:35.574523+0000","last_active":"2024-04-16T19:36:35.574363+0000","last_peered":"2024-04-16T19:36:35.574363+0000","last_clean":"2024-04-16T19:36:35.574363+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:36:35.574363+0000","last_undegraded":"2024-04-16T19:36:35.574363+0000","last_fullsized":"2024-04-16T19:36:35.574363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'206","last_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_deep_scrub":"53'206","last_deep_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.574163+0000","objects_scrubbed":10,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:15.061721+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'20","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.646780+0000","last_change":"2024-04-16T19:36:30.646869+0000","last_active":"2024-04-16T19:36:30.646780+0000","last_peered":"2024-04-16T19:36:30.646780+0000","last_clean":"2024-04-16T19:36:30.646780+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:36:30.646780+0000","last_undegraded":"2024-04-16T19:36:30.646780+0000","last_fullsized":"2024-04-16T19:36:30.646780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.646698+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:37.151471+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.194008+0000","last_change":"2024-04-16T17:47:23.258182+0000","last_active":"2024-04-16T19:34:56.194008+0000","last_peered":"2024-04-16T19:34:56.194008+0000","last_clean":"2024-04-16T19:34:56.194008+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:34:56.194008+0000","last_undegraded":"2024-04-16T19:34:56.194008+0000","last_fullsized":"2024-04-16T19:34:56.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.188408+0000","last_change":"2024-04-16T17:47:21.231514+0000","last_active":"2024-04-16T19:34:51.188408+0000","last_peered":"2024-04-16T19:34:51.188408+0000","last_clean":"2024-04-16T19:34:51.188408+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:34:51.188408+0000","last_undegraded":"2024-04-16T19:34:51.188408+0000","last_fullsized":"2024-04-16T19:34:51.188408+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"53'7","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.654377+0000","last_change":"2024-04-16T19:37:05.654446+0000","last_active":"2024-04-16T19:37:05.654377+0000","last_peered":"2024-04-16T19:37:05.654377+0000","last_clean":"2024-04-16T19:37:05.654377+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:37:05.654377+0000","last_undegraded":"2024-04-16T19:37:05.654377+0000","last_fullsized":"2024-04-16T19:37:05.654377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:05.654239+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:05.654239+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.654239+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:11.660547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'109","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.373414+0000","last_change":"2024-04-16T19:36:57.373474+0000","last_active":"2024-04-16T19:36:57.373414+0000","last_peered":"2024-04-16T19:36:57.373414+0000","last_clean":"2024-04-16T19:36:57.373414+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:36:57.373414+0000","last_undegraded":"2024-04-16T19:36:57.373414+0000","last_fullsized":"2024-04-16T19:36:57.373414+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:57.373338+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:57.373338+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.373338+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:48.848526+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.699354+0000","last_change":"2024-04-16T19:36:54.699404+0000","last_active":"2024-04-16T19:36:54.699354+0000","last_peered":"2024-04-16T19:36:54.699354+0000","last_clean":"2024-04-16T19:36:54.699354+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:36:54.699354+0000","last_undegraded":"2024-04-16T19:36:54.699354+0000","last_fullsized":"2024-04-16T19:36:54.699354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.699216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:21.320753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.350473+0000","last_change":"2024-04-16T19:36:42.350584+0000","last_active":"2024-04-16T19:36:42.350473+0000","last_peered":"2024-04-16T19:36:42.350473+0000","last_clean":"2024-04-16T19:36:42.350473+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:36:42.350473+0000","last_undegraded":"2024-04-16T19:36:42.350473+0000","last_fullsized":"2024-04-16T19:36:42.350473+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.350304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.971842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":46,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.214641+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:36:11.214641+0000","last_peered":"2024-04-16T19:36:11.214641+0000","last_clean":"2024-04-16T19:36:11.214641+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:36:11.214641+0000","last_undegraded":"2024-04-16T19:36:11.214641+0000","last_fullsized":"2024-04-16T19:36:11.214641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'90","reported_seq":913,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.682295+0000","last_change":"2024-04-16T19:37:07.682412+0000","last_active":"2024-04-16T19:37:07.682295+0000","last_peered":"2024-04-16T19:37:07.682295+0000","last_clean":"2024-04-16T19:37:07.682295+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:37:07.682295+0000","last_undegraded":"2024-04-16T19:37:07.682295+0000","last_fullsized":"2024-04-16T19:37:07.682295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'90","last_scrub_stamp":"2024-04-16T19:37:07.682156+0000","last_deep_scrub":"53'90","last_deep_scrub_stamp":"2024-04-16T19:37:07.682156+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.682156+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:33.687043+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5939,"num_omap_keys":27,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.668638+0000","last_change":"2024-04-16T19:36:44.668638+0000","last_active":"2024-04-16T19:36:44.668638+0000","last_peered":"2024-04-16T19:36:44.668638+0000","last_clean":"2024-04-16T19:36:44.668638+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:36:44.668638+0000","last_undegraded":"2024-04-16T19:36:44.668638+0000","last_fullsized":"2024-04-16T19:36:44.668638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.668449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:31.847082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"53'2","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.465959+0000","last_change":"2024-04-16T19:36:53.466069+0000","last_active":"2024-04-16T19:36:53.465959+0000","last_peered":"2024-04-16T19:36:53.465959+0000","last_clean":"2024-04-16T19:36:53.465959+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:36:53.465959+0000","last_undegraded":"2024-04-16T19:36:53.465959+0000","last_fullsized":"2024-04-16T19:36:53.465959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.465746+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:19.042353+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.452816+0000","last_change":"2024-04-16T19:36:41.452876+0000","last_active":"2024-04-16T19:36:41.452816+0000","last_peered":"2024-04-16T19:36:41.452816+0000","last_clean":"2024-04-16T19:36:41.452816+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:36:41.452816+0000","last_undegraded":"2024-04-16T19:36:41.452816+0000","last_fullsized":"2024-04-16T19:36:41.452816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.452676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:06.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'509","reported_seq":3967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635824+0000","last_change":"2024-04-16T19:36:20.636035+0000","last_active":"2024-04-16T19:36:20.635824+0000","last_peered":"2024-04-16T19:36:20.635824+0000","last_clean":"2024-04-16T19:36:20.635824+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:36:20.635824+0000","last_undegraded":"2024-04-16T19:36:20.635824+0000","last_fullsized":"2024-04-16T19:36:20.635824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635659+0000","objects_scrubbed":21,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:04.155514+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71921,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'34","reported_seq":243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.437126+0000","last_change":"2024-04-16T19:36:23.437306+0000","last_active":"2024-04-16T19:36:23.437126+0000","last_peered":"2024-04-16T19:36:23.437126+0000","last_clean":"2024-04-16T19:36:23.437126+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:36:23.437126+0000","last_undegraded":"2024-04-16T19:36:23.437126+0000","last_fullsized":"2024-04-16T19:36:23.437126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.436922+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:38.568868+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.175524+0000","last_change":"2024-04-16T17:47:23.250942+0000","last_active":"2024-04-16T19:34:56.175524+0000","last_peered":"2024-04-16T19:34:56.175524+0000","last_clean":"2024-04-16T19:34:56.175524+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:34:56.175524+0000","last_undegraded":"2024-04-16T19:34:56.175524+0000","last_fullsized":"2024-04-16T19:34:56.175524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.192550+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:34:51.192550+0000","last_peered":"2024-04-16T19:34:51.192550+0000","last_clean":"2024-04-16T19:34:51.192550+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:34:51.192550+0000","last_undegraded":"2024-04-16T19:34:51.192550+0000","last_fullsized":"2024-04-16T19:34:51.192550+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"53'5","reported_seq":813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.631361+0000","last_change":"2024-04-16T19:37:06.631427+0000","last_active":"2024-04-16T19:37:06.631361+0000","last_peered":"2024-04-16T19:37:06.631361+0000","last_clean":"2024-04-16T19:37:06.631361+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:37:06.631361+0000","last_undegraded":"2024-04-16T19:37:06.631361+0000","last_fullsized":"2024-04-16T19:37:06.631361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'5","last_scrub_stamp":"2024-04-16T19:37:06.631224+0000","last_deep_scrub":"53'5","last_deep_scrub_stamp":"2024-04-16T19:37:06.631224+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.631224+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:52:08.200734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'125","reported_seq":312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.573654+0000","last_change":"2024-04-16T19:37:05.573733+0000","last_active":"2024-04-16T19:37:05.573654+0000","last_peered":"2024-04-16T19:37:05.573654+0000","last_clean":"2024-04-16T19:37:05.573654+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:37:05.573654+0000","last_undegraded":"2024-04-16T19:37:05.573654+0000","last_fullsized":"2024-04-16T19:37:05.573654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:37:05.573508+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:37:05.573508+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.573508+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:54.563678+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.345458+0000","last_change":"2024-04-16T19:37:01.345526+0000","last_active":"2024-04-16T19:37:01.345458+0000","last_peered":"2024-04-16T19:37:01.345458+0000","last_clean":"2024-04-16T19:37:01.345458+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:37:01.345458+0000","last_undegraded":"2024-04-16T19:37:01.345458+0000","last_fullsized":"2024-04-16T19:37:01.345458+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:01.345307+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:01.345307+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.345307+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:27.712980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.388068+0000","last_change":"2024-04-16T19:36:39.388173+0000","last_active":"2024-04-16T19:36:39.388068+0000","last_peered":"2024-04-16T19:36:39.388068+0000","last_clean":"2024-04-16T19:36:39.388068+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:36:39.388068+0000","last_undegraded":"2024-04-16T19:36:39.388068+0000","last_fullsized":"2024-04-16T19:36:39.388068+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.387834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:30.147039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.440112+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:36:18.440112+0000","last_peered":"2024-04-16T19:36:18.440112+0000","last_clean":"2024-04-16T19:36:18.440112+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:36:18.440112+0000","last_undegraded":"2024-04-16T19:36:18.440112+0000","last_fullsized":"2024-04-16T19:36:18.440112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'123","reported_seq":1604,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.593801+0000","last_change":"2024-04-16T19:36:52.593899+0000","last_active":"2024-04-16T19:36:52.593801+0000","last_peered":"2024-04-16T19:36:52.593801+0000","last_clean":"2024-04-16T19:36:52.593801+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:36:52.593801+0000","last_undegraded":"2024-04-16T19:36:52.593801+0000","last_fullsized":"2024-04-16T19:36:52.593801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'123","last_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_deep_scrub":"53'123","last_deep_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.593639+0000","objects_scrubbed":37,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:52.173838+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5671,"num_omap_keys":23,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.668143+0000","last_change":"2024-04-16T19:36:53.668197+0000","last_active":"2024-04-16T19:36:53.668143+0000","last_peered":"2024-04-16T19:36:53.668143+0000","last_clean":"2024-04-16T19:36:53.668143+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:36:53.668143+0000","last_undegraded":"2024-04-16T19:36:53.668143+0000","last_fullsized":"2024-04-16T19:36:53.668143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.668006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:20:47.522228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'10","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.679540+0000","last_change":"2024-04-16T19:36:41.679612+0000","last_active":"2024-04-16T19:36:41.679540+0000","last_peered":"2024-04-16T19:36:41.679540+0000","last_clean":"2024-04-16T19:36:41.679540+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:36:41.679540+0000","last_undegraded":"2024-04-16T19:36:41.679540+0000","last_fullsized":"2024-04-16T19:36:41.679540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.679332+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:33.154520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'556","reported_seq":1963,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.395959+0000","last_change":"2024-04-16T19:36:44.396025+0000","last_active":"2024-04-16T19:36:44.395959+0000","last_peered":"2024-04-16T19:36:44.395959+0000","last_clean":"2024-04-16T19:36:44.395959+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:36:44.395959+0000","last_undegraded":"2024-04-16T19:36:44.395959+0000","last_fullsized":"2024-04-16T19:36:44.395959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.395808+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:46.739394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'508","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.543534+0000","last_change":"2024-04-16T19:36:25.543534+0000","last_active":"2024-04-16T19:36:25.543534+0000","last_peered":"2024-04-16T19:36:25.543534+0000","last_clean":"2024-04-16T19:36:25.543534+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:36:25.543534+0000","last_undegraded":"2024-04-16T19:36:25.543534+0000","last_fullsized":"2024-04-16T19:36:25.543534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.543268+0000","objects_scrubbed":16,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:59.596711+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71934,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'8","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.642341+0000","last_change":"2024-04-16T19:36:27.642341+0000","last_active":"2024-04-16T19:36:27.642341+0000","last_peered":"2024-04-16T19:36:27.642341+0000","last_clean":"2024-04-16T19:36:27.642341+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:36:27.642341+0000","last_undegraded":"2024-04-16T19:36:27.642341+0000","last_fullsized":"2024-04-16T19:36:27.642341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.642045+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:07.753490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.599983+0000","last_change":"2024-04-16T19:37:12.600045+0000","last_active":"2024-04-16T19:37:12.599983+0000","last_peered":"2024-04-16T19:37:12.599983+0000","last_clean":"2024-04-16T19:37:12.599983+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:37:12.599983+0000","last_undegraded":"2024-04-16T19:37:12.599983+0000","last_fullsized":"2024-04-16T19:37:12.599983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:12.599830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:12.599830+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.599830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:38.604717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191235+0000","last_change":"2024-04-16T17:47:21.258208+0000","last_active":"2024-04-16T19:34:51.191235+0000","last_peered":"2024-04-16T19:34:51.191235+0000","last_clean":"2024-04-16T19:34:51.191235+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:34:51.191235+0000","last_undegraded":"2024-04-16T19:34:51.191235+0000","last_fullsized":"2024-04-16T19:34:51.191235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"48'14","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.192192+0000","last_change":"2024-04-16T17:47:33.303004+0000","last_active":"2024-04-16T19:35:06.192192+0000","last_peered":"2024-04-16T19:35:06.192192+0000","last_clean":"2024-04-16T19:35:06.192192+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:35:06.192192+0000","last_undegraded":"2024-04-16T19:35:06.192192+0000","last_fullsized":"2024-04-16T19:35:06.192192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'159","reported_seq":358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.333939+0000","last_change":"2024-04-16T19:37:10.334092+0000","last_active":"2024-04-16T19:37:10.333939+0000","last_peered":"2024-04-16T19:37:10.333939+0000","last_clean":"2024-04-16T19:37:10.333939+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:37:10.333939+0000","last_undegraded":"2024-04-16T19:37:10.333939+0000","last_fullsized":"2024-04-16T19:37:10.333939+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:10.333771+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:10.333771+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.333771+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:16:33.077948+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.656216+0000","last_change":"2024-04-16T19:36:46.656320+0000","last_active":"2024-04-16T19:36:46.656216+0000","last_peered":"2024-04-16T19:36:46.656216+0000","last_clean":"2024-04-16T19:36:46.656216+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:36:46.656216+0000","last_undegraded":"2024-04-16T19:36:46.656216+0000","last_fullsized":"2024-04-16T19:36:46.656216+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.656048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:11.940546+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.451910+0000","last_change":"2024-04-16T19:36:37.452021+0000","last_active":"2024-04-16T19:36:37.451910+0000","last_peered":"2024-04-16T19:36:37.451910+0000","last_clean":"2024-04-16T19:36:37.451910+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:36:37.451910+0000","last_undegraded":"2024-04-16T19:36:37.451910+0000","last_fullsized":"2024-04-16T19:36:37.451910+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.451765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:52.163889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.528952+0000","last_change":"2024-04-16T18:10:50.554799+0000","last_active":"2024-04-16T19:36:17.528952+0000","last_peered":"2024-04-16T19:36:17.528952+0000","last_clean":"2024-04-16T19:36:17.528952+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:36:17.528952+0000","last_undegraded":"2024-04-16T19:36:17.528952+0000","last_fullsized":"2024-04-16T19:36:17.528952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'134","reported_seq":1476,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.725767+0000","last_change":"2024-04-16T19:36:59.725923+0000","last_active":"2024-04-16T19:36:59.725767+0000","last_peered":"2024-04-16T19:36:59.725767+0000","last_clean":"2024-04-16T19:36:59.725767+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:36:59.725767+0000","last_undegraded":"2024-04-16T19:36:59.725767+0000","last_fullsized":"2024-04-16T19:36:59.725767+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'134","last_scrub_stamp":"2024-04-16T19:36:59.725594+0000","last_deep_scrub":"53'134","last_deep_scrub_stamp":"2024-04-16T19:36:59.725594+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.725594+0000","objects_scrubbed":35,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:25.010093+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7807,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.355648+0000","last_change":"2024-04-16T19:37:03.355699+0000","last_active":"2024-04-16T19:37:03.355648+0000","last_peered":"2024-04-16T19:37:03.355648+0000","last_clean":"2024-04-16T19:37:03.355648+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:37:03.355648+0000","last_undegraded":"2024-04-16T19:37:03.355648+0000","last_fullsized":"2024-04-16T19:37:03.355648+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.355513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.355513+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.355513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:13.649973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'11","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.624192+0000","last_change":"2024-04-16T19:36:55.624271+0000","last_active":"2024-04-16T19:36:55.624192+0000","last_peered":"2024-04-16T19:36:55.624192+0000","last_clean":"2024-04-16T19:36:55.624192+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:36:55.624192+0000","last_undegraded":"2024-04-16T19:36:55.624192+0000","last_fullsized":"2024-04-16T19:36:55.624192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.624051+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:11.910789+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'556","reported_seq":1911,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.421934+0000","last_change":"2024-04-16T19:36:38.421934+0000","last_active":"2024-04-16T19:36:38.421934+0000","last_peered":"2024-04-16T19:36:38.421934+0000","last_clean":"2024-04-16T19:36:38.421934+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:36:38.421934+0000","last_undegraded":"2024-04-16T19:36:38.421934+0000","last_fullsized":"2024-04-16T19:36:38.421934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.421654+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:18.533099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'250","reported_seq":489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.484262+0000","last_change":"2024-04-16T19:36:27.484420+0000","last_active":"2024-04-16T19:36:27.484262+0000","last_peered":"2024-04-16T19:36:27.484262+0000","last_clean":"2024-04-16T19:36:27.484262+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:36:27.484262+0000","last_undegraded":"2024-04-16T19:36:27.484262+0000","last_fullsized":"2024-04-16T19:36:27.484262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.484127+0000","objects_scrubbed":20,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:04.899387+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"48'3","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.205027+0000","last_change":"2024-04-16T17:49:32.402007+0000","last_active":"2024-04-16T19:35:56.205027+0000","last_peered":"2024-04-16T19:35:56.205027+0000","last_clean":"2024-04-16T19:35:56.205027+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:35:56.205027+0000","last_undegraded":"2024-04-16T19:35:56.205027+0000","last_fullsized":"2024-04-16T19:35:56.205027+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:56.189793+0000","last_change":"2024-04-16T17:47:23.247060+0000","last_active":"2024-04-16T19:34:56.189793+0000","last_peered":"2024-04-16T19:34:56.189793+0000","last_clean":"2024-04-16T19:34:56.189793+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:34:56.189793+0000","last_undegraded":"2024-04-16T19:34:56.189793+0000","last_fullsized":"2024-04-16T19:34:56.189793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:34:51.191305+0000","last_change":"2024-04-16T17:47:21.258172+0000","last_active":"2024-04-16T19:34:51.191305+0000","last_peered":"2024-04-16T19:34:51.191305+0000","last_clean":"2024-04-16T19:34:51.191305+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:34:51.191305+0000","last_undegraded":"2024-04-16T19:34:51.191305+0000","last_fullsized":"2024-04-16T19:34:51.191305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"48'16","reported_seq":178,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:06.194844+0000","last_change":"2024-04-16T17:47:33.302189+0000","last_active":"2024-04-16T19:35:06.194844+0000","last_peered":"2024-04-16T19:35:06.194844+0000","last_clean":"2024-04-16T19:35:06.194844+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:35:06.194844+0000","last_undegraded":"2024-04-16T19:35:06.194844+0000","last_fullsized":"2024-04-16T19:35:06.194844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.665782+0000","last_change":"2024-04-16T19:37:14.665856+0000","last_active":"2024-04-16T19:37:14.665782+0000","last_peered":"2024-04-16T19:37:14.665782+0000","last_clean":"2024-04-16T19:37:14.665782+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:37:14.665782+0000","last_undegraded":"2024-04-16T19:37:14.665782+0000","last_fullsized":"2024-04-16T19:37:14.665782+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:14.665644+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:14.665644+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.665644+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:34.559821+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.446778+0000","last_change":"2024-04-16T19:36:59.446778+0000","last_active":"2024-04-16T19:36:59.446778+0000","last_peered":"2024-04-16T19:36:59.446778+0000","last_clean":"2024-04-16T19:36:59.446778+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:36:59.446778+0000","last_undegraded":"2024-04-16T19:36:59.446778+0000","last_fullsized":"2024-04-16T19:36:59.446778+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.446587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.446587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.446587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:20.442391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.477611+0000","last_change":"2024-04-16T19:36:26.477669+0000","last_active":"2024-04-16T19:36:26.477611+0000","last_peered":"2024-04-16T19:36:26.477611+0000","last_clean":"2024-04-16T19:36:26.477611+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:36:26.477611+0000","last_undegraded":"2024-04-16T19:36:26.477611+0000","last_fullsized":"2024-04-16T19:36:26.477611+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.477464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:27.657085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588112+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:36:16.588112+0000","last_peered":"2024-04-16T19:36:16.588112+0000","last_clean":"2024-04-16T19:36:16.588112+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:36:16.588112+0000","last_undegraded":"2024-04-16T19:36:16.588112+0000","last_fullsized":"2024-04-16T19:36:16.588112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'126","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.542707+0000","last_change":"2024-04-16T19:36:48.542776+0000","last_active":"2024-04-16T19:36:48.542707+0000","last_peered":"2024-04-16T19:36:48.542707+0000","last_clean":"2024-04-16T19:36:48.542707+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:36:48.542707+0000","last_undegraded":"2024-04-16T19:36:48.542707+0000","last_fullsized":"2024-04-16T19:36:48.542707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.542572+0000","objects_scrubbed":30,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:02.471385+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6535,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.626702+0000","last_change":"2024-04-16T19:36:43.626754+0000","last_active":"2024-04-16T19:36:43.626702+0000","last_peered":"2024-04-16T19:36:43.626702+0000","last_clean":"2024-04-16T19:36:43.626702+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:36:43.626702+0000","last_undegraded":"2024-04-16T19:36:43.626702+0000","last_fullsized":"2024-04-16T19:36:43.626702+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.626565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:30.990969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'13","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.507190+0000","last_change":"2024-04-16T19:36:57.507267+0000","last_active":"2024-04-16T19:36:57.507190+0000","last_peered":"2024-04-16T19:36:57.507190+0000","last_clean":"2024-04-16T19:36:57.507190+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:36:57.507190+0000","last_undegraded":"2024-04-16T19:36:57.507190+0000","last_fullsized":"2024-04-16T19:36:57.507190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:57.507046+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:57.507046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.507046+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:56.572731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.388632+0000","last_change":"2024-04-16T19:37:00.388683+0000","last_active":"2024-04-16T19:37:00.388632+0000","last_peered":"2024-04-16T19:37:00.388632+0000","last_clean":"2024-04-16T19:37:00.388632+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:37:00.388632+0000","last_undegraded":"2024-04-16T19:37:00.388632+0000","last_fullsized":"2024-04-16T19:37:00.388632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.388491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.388491+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.388491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:07.573705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'141","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.454241+0000","last_change":"2024-04-16T19:36:28.454241+0000","last_active":"2024-04-16T19:36:28.454241+0000","last_peered":"2024-04-16T19:36:28.454241+0000","last_clean":"2024-04-16T19:36:28.454241+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:36:28.454241+0000","last_undegraded":"2024-04-16T19:36:28.454241+0000","last_fullsized":"2024-04-16T19:36:28.454241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'141","last_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_deep_scrub":"53'141","last_deep_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.453990+0000","objects_scrubbed":12,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:15.632623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.478104+0000","last_change":"2024-04-16T19:36:22.478173+0000","last_active":"2024-04-16T19:36:22.478104+0000","last_peered":"2024-04-16T19:36:22.478104+0000","last_clean":"2024-04-16T19:36:22.478104+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:36:22.478104+0000","last_undegraded":"2024-04-16T19:36:22.478104+0000","last_fullsized":"2024-04-16T19:36:22.478104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.477939+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:46.331273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586046+0000","last_change":"2024-04-16T17:47:23.256555+0000","last_active":"2024-04-16T19:36:16.586046+0000","last_peered":"2024-04-16T19:36:16.586046+0000","last_clean":"2024-04-16T19:36:16.586046+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:36:16.586046+0000","last_undegraded":"2024-04-16T19:36:16.586046+0000","last_fullsized":"2024-04-16T19:36:16.586046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.188432+0000","last_change":"2024-04-16T17:47:21.250514+0000","last_active":"2024-04-16T19:35:51.188432+0000","last_peered":"2024-04-16T19:35:51.188432+0000","last_clean":"2024-04-16T19:35:51.188432+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:35:51.188432+0000","last_undegraded":"2024-04-16T19:35:51.188432+0000","last_fullsized":"2024-04-16T19:35:51.188432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.438194+0000","last_change":"2024-04-16T17:47:33.297459+0000","last_active":"2024-04-16T19:36:18.438194+0000","last_peered":"2024-04-16T19:36:18.438194+0000","last_clean":"2024-04-16T19:36:18.438194+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:36:18.438194+0000","last_undegraded":"2024-04-16T19:36:18.438194+0000","last_fullsized":"2024-04-16T19:36:18.438194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'34","reported_seq":171,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.608009+0000","last_change":"2024-04-16T19:37:08.608145+0000","last_active":"2024-04-16T19:37:08.608009+0000","last_peered":"2024-04-16T19:37:08.608009+0000","last_clean":"2024-04-16T19:37:08.608009+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:37:08.608009+0000","last_undegraded":"2024-04-16T19:37:08.608009+0000","last_fullsized":"2024-04-16T19:37:08.608009+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:37:08.607831+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:37:08.607831+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.607831+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:02.316023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.532299+0000","last_change":"2024-04-16T19:37:03.532350+0000","last_active":"2024-04-16T19:37:03.532299+0000","last_peered":"2024-04-16T19:37:03.532299+0000","last_clean":"2024-04-16T19:37:03.532299+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:37:03.532299+0000","last_undegraded":"2024-04-16T19:37:03.532299+0000","last_fullsized":"2024-04-16T19:37:03.532299+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.532166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.532166+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.532166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:26.201264+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.360832+0000","last_change":"2024-04-16T19:36:41.360886+0000","last_active":"2024-04-16T19:36:41.360832+0000","last_peered":"2024-04-16T19:36:41.360832+0000","last_clean":"2024-04-16T19:36:41.360832+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:36:41.360832+0000","last_undegraded":"2024-04-16T19:36:41.360832+0000","last_fullsized":"2024-04-16T19:36:41.360832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.360675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.872893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.050546+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:36:17.050546+0000","last_peered":"2024-04-16T19:36:17.050546+0000","last_clean":"2024-04-16T19:36:17.050546+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:36:17.050546+0000","last_undegraded":"2024-04-16T19:36:17.050546+0000","last_fullsized":"2024-04-16T19:36:17.050546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'155","reported_seq":1489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.495554+0000","last_change":"2024-04-16T19:36:52.495627+0000","last_active":"2024-04-16T19:36:52.495554+0000","last_peered":"2024-04-16T19:36:52.495554+0000","last_clean":"2024-04-16T19:36:52.495554+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:36:52.495554+0000","last_undegraded":"2024-04-16T19:36:52.495554+0000","last_fullsized":"2024-04-16T19:36:52.495554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'155","last_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_deep_scrub":"53'155","last_deep_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.495389+0000","objects_scrubbed":29,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.497334+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8801,"num_omap_keys":37,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.641083+0000","last_change":"2024-04-16T19:36:38.641135+0000","last_active":"2024-04-16T19:36:38.641083+0000","last_peered":"2024-04-16T19:36:38.641083+0000","last_clean":"2024-04-16T19:36:38.641083+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:36:38.641083+0000","last_undegraded":"2024-04-16T19:36:38.641083+0000","last_fullsized":"2024-04-16T19:36:38.641083+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.640946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:53.196611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'11","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.644638+0000","last_change":"2024-04-16T19:36:54.644719+0000","last_active":"2024-04-16T19:36:54.644638+0000","last_peered":"2024-04-16T19:36:54.644638+0000","last_clean":"2024-04-16T19:36:54.644638+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:36:54.644638+0000","last_undegraded":"2024-04-16T19:36:54.644638+0000","last_fullsized":"2024-04-16T19:36:54.644638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.644499+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:20.331700+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.384126+0000","last_change":"2024-04-16T19:36:47.384178+0000","last_active":"2024-04-16T19:36:47.384126+0000","last_peered":"2024-04-16T19:36:47.384126+0000","last_clean":"2024-04-16T19:36:47.384126+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:36:47.384126+0000","last_undegraded":"2024-04-16T19:36:47.384126+0000","last_fullsized":"2024-04-16T19:36:47.384126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.383981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:49.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11080","reported_seq":18169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.523528+0000","last_change":"2024-04-16T19:36:22.523528+0000","last_active":"2024-04-16T19:36:22.523528+0000","last_peered":"2024-04-16T19:36:22.523528+0000","last_clean":"2024-04-16T19:36:22.523528+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:36:22.523528+0000","last_undegraded":"2024-04-16T19:36:22.523528+0000","last_fullsized":"2024-04-16T19:36:22.523528+0000","mapping_epoch":34,"log_start":"53'9300","ondisk_log_start":"53'9300","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11080","last_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_deep_scrub":"53'11080","last_deep_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.523321+0000","objects_scrubbed":25,"log_size":1780,"log_dups_size":3000,"ondisk_log_size":1780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:02.010878+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'10","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.675610+0000","last_change":"2024-04-16T19:36:33.675610+0000","last_active":"2024-04-16T19:36:33.675610+0000","last_peered":"2024-04-16T19:36:33.675610+0000","last_clean":"2024-04-16T19:36:33.675610+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:36:33.675610+0000","last_undegraded":"2024-04-16T19:36:33.675610+0000","last_fullsized":"2024-04-16T19:36:33.675610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.675369+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:17.195224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.216640+0000","last_change":"2024-04-16T17:47:23.260062+0000","last_active":"2024-04-16T19:36:11.216640+0000","last_peered":"2024-04-16T19:36:11.216640+0000","last_clean":"2024-04-16T19:36:11.216640+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:36:11.216640+0000","last_undegraded":"2024-04-16T19:36:11.216640+0000","last_fullsized":"2024-04-16T19:36:11.216640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205439+0000","last_change":"2024-04-16T17:47:21.244757+0000","last_active":"2024-04-16T19:35:41.205439+0000","last_peered":"2024-04-16T19:35:41.205439+0000","last_clean":"2024-04-16T19:35:41.205439+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:35:41.205439+0000","last_undegraded":"2024-04-16T19:35:41.205439+0000","last_fullsized":"2024-04-16T19:35:41.205439+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'4","reported_seq":1345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048951+0000","last_change":"2024-04-16T17:47:33.297024+0000","last_active":"2024-04-16T19:36:17.048951+0000","last_peered":"2024-04-16T19:36:17.048951+0000","last_clean":"2024-04-16T19:36:17.048951+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:36:17.048951+0000","last_undegraded":"2024-04-16T19:36:17.048951+0000","last_fullsized":"2024-04-16T19:36:17.048951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'84","reported_seq":245,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.654052+0000","last_change":"2024-04-16T19:36:59.654126+0000","last_active":"2024-04-16T19:36:59.654052+0000","last_peered":"2024-04-16T19:36:59.654052+0000","last_clean":"2024-04-16T19:36:59.654052+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:36:59.654052+0000","last_undegraded":"2024-04-16T19:36:59.654052+0000","last_fullsized":"2024-04-16T19:36:59.654052+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:36:59.653919+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:36:59.653919+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.653919+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:10.641324+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.508215+0000","last_change":"2024-04-16T19:37:04.508267+0000","last_active":"2024-04-16T19:37:04.508215+0000","last_peered":"2024-04-16T19:37:04.508215+0000","last_clean":"2024-04-16T19:37:04.508215+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:37:04.508215+0000","last_undegraded":"2024-04-16T19:37:04.508215+0000","last_fullsized":"2024-04-16T19:37:04.508215+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.508075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.508075+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.508075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:53.498245+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.560046+0000","last_change":"2024-04-16T19:36:49.560147+0000","last_active":"2024-04-16T19:36:49.560046+0000","last_peered":"2024-04-16T19:36:49.560046+0000","last_clean":"2024-04-16T19:36:49.560046+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:36:49.560046+0000","last_undegraded":"2024-04-16T19:36:49.560046+0000","last_fullsized":"2024-04-16T19:36:49.560046+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.559817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:52.415984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'31","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.493564+0000","last_change":"2024-04-16T19:36:46.493705+0000","last_active":"2024-04-16T19:36:46.493564+0000","last_peered":"2024-04-16T19:36:46.493564+0000","last_clean":"2024-04-16T19:36:46.493564+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:36:46.493564+0000","last_undegraded":"2024-04-16T19:36:46.493564+0000","last_fullsized":"2024-04-16T19:36:46.493564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'31","last_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_deep_scrub":"53'31","last_deep_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.493417+0000","objects_scrubbed":9,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:52.920199+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.429456+0000","last_change":"2024-04-16T19:37:02.429510+0000","last_active":"2024-04-16T19:37:02.429456+0000","last_peered":"2024-04-16T19:37:02.429456+0000","last_clean":"2024-04-16T19:37:02.429456+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:37:02.429456+0000","last_undegraded":"2024-04-16T19:37:02.429456+0000","last_fullsized":"2024-04-16T19:37:02.429456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.429304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.429304+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.429304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:31.301173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.200847+0000","last_change":"2024-04-16T17:47:21.235566+0000","last_active":"2024-04-16T19:35:41.200847+0000","last_peered":"2024-04-16T19:35:41.200847+0000","last_clean":"2024-04-16T19:35:41.200847+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:35:41.200847+0000","last_undegraded":"2024-04-16T19:35:41.200847+0000","last_fullsized":"2024-04-16T19:35:41.200847+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'6","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.472692+0000","last_change":"2024-04-16T19:36:34.472767+0000","last_active":"2024-04-16T19:36:34.472692+0000","last_peered":"2024-04-16T19:36:34.472692+0000","last_clean":"2024-04-16T19:36:34.472692+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:36:34.472692+0000","last_undegraded":"2024-04-16T19:36:34.472692+0000","last_fullsized":"2024-04-16T19:36:34.472692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.472561+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:21.836965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.621011+0000","last_change":"2024-04-16T19:37:14.621063+0000","last_active":"2024-04-16T19:37:14.621011+0000","last_peered":"2024-04-16T19:37:14.621011+0000","last_clean":"2024-04-16T19:37:14.621011+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:37:14.621011+0000","last_undegraded":"2024-04-16T19:37:14.621011+0000","last_fullsized":"2024-04-16T19:37:14.621011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.620868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.620868+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.620868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:39.939904+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.185828+0000","last_change":"2024-04-16T17:47:21.233716+0000","last_active":"2024-04-16T19:35:41.185828+0000","last_peered":"2024-04-16T19:35:41.185828+0000","last_clean":"2024-04-16T19:35:41.185828+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:35:41.185828+0000","last_undegraded":"2024-04-16T19:35:41.185828+0000","last_fullsized":"2024-04-16T19:35:41.185828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'10","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.445631+0000","last_change":"2024-04-16T19:36:30.445631+0000","last_active":"2024-04-16T19:36:30.445631+0000","last_peered":"2024-04-16T19:36:30.445631+0000","last_clean":"2024-04-16T19:36:30.445631+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:36:30.445631+0000","last_undegraded":"2024-04-16T19:36:30.445631+0000","last_fullsized":"2024-04-16T19:36:30.445631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.445415+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:00.551163+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.206084+0000","last_change":"2024-04-16T17:47:23.265624+0000","last_active":"2024-04-16T19:35:46.206084+0000","last_peered":"2024-04-16T19:35:46.206084+0000","last_clean":"2024-04-16T19:35:46.206084+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:35:46.206084+0000","last_undegraded":"2024-04-16T19:35:46.206084+0000","last_fullsized":"2024-04-16T19:35:46.206084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209219+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:35:41.209219+0000","last_peered":"2024-04-16T19:35:41.209219+0000","last_clean":"2024-04-16T19:35:41.209219+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:35:41.209219+0000","last_undegraded":"2024-04-16T19:35:41.209219+0000","last_fullsized":"2024-04-16T19:35:41.209219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'12","reported_seq":248,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.464290+0000","last_change":"2024-04-16T19:36:34.464387+0000","last_active":"2024-04-16T19:36:34.464290+0000","last_peered":"2024-04-16T19:36:34.464290+0000","last_clean":"2024-04-16T19:36:34.464290+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:36:34.464290+0000","last_undegraded":"2024-04-16T19:36:34.464290+0000","last_fullsized":"2024-04-16T19:36:34.464290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.464153+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:03.336022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.621593+0000","last_change":"2024-04-16T19:37:09.621645+0000","last_active":"2024-04-16T19:37:09.621593+0000","last_peered":"2024-04-16T19:37:09.621593+0000","last_clean":"2024-04-16T19:37:09.621593+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:37:09.621593+0000","last_undegraded":"2024-04-16T19:37:09.621593+0000","last_fullsized":"2024-04-16T19:37:09.621593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.621459+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:24:41.966705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.213866+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:36:16.213866+0000","last_peered":"2024-04-16T19:36:16.213866+0000","last_clean":"2024-04-16T19:36:16.213866+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:36:16.213866+0000","last_undegraded":"2024-04-16T19:36:16.213866+0000","last_fullsized":"2024-04-16T19:36:16.213866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"53'6","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.528043+0000","last_change":"2024-04-16T19:36:28.528121+0000","last_active":"2024-04-16T19:36:28.528043+0000","last_peered":"2024-04-16T19:36:28.528043+0000","last_clean":"2024-04-16T19:36:28.528043+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:36:28.528043+0000","last_undegraded":"2024-04-16T19:36:28.528043+0000","last_fullsized":"2024-04-16T19:36:28.528043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.527896+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:25.132224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.475642+0000","last_change":"2024-04-16T19:37:15.475697+0000","last_active":"2024-04-16T19:37:15.475642+0000","last_peered":"2024-04-16T19:37:15.475642+0000","last_clean":"2024-04-16T19:37:15.475642+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:37:15.475642+0000","last_undegraded":"2024-04-16T19:37:15.475642+0000","last_fullsized":"2024-04-16T19:37:15.475642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.475502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.475502+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.475502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:53.312895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205237+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:35:41.205237+0000","last_peered":"2024-04-16T19:35:41.205237+0000","last_clean":"2024-04-16T19:35:41.205237+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:35:41.205237+0000","last_undegraded":"2024-04-16T19:35:41.205237+0000","last_fullsized":"2024-04-16T19:35:41.205237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"53'2","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.582380+0000","last_change":"2024-04-16T19:36:26.582466+0000","last_active":"2024-04-16T19:36:26.582380+0000","last_peered":"2024-04-16T19:36:26.582380+0000","last_clean":"2024-04-16T19:36:26.582380+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:36:26.582380+0000","last_undegraded":"2024-04-16T19:36:26.582380+0000","last_fullsized":"2024-04-16T19:36:26.582380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.582238+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:10.024512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.583135+0000","last_change":"2024-04-16T19:37:13.583188+0000","last_active":"2024-04-16T19:37:13.583135+0000","last_peered":"2024-04-16T19:37:13.583135+0000","last_clean":"2024-04-16T19:37:13.583135+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:37:13.583135+0000","last_undegraded":"2024-04-16T19:37:13.583135+0000","last_fullsized":"2024-04-16T19:37:13.583135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.583001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.583001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.583001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:29.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.206571+0000","last_change":"2024-04-16T17:47:21.229051+0000","last_active":"2024-04-16T19:35:41.206571+0000","last_peered":"2024-04-16T19:35:41.206571+0000","last_clean":"2024-04-16T19:35:41.206571+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:35:41.206571+0000","last_undegraded":"2024-04-16T19:35:41.206571+0000","last_fullsized":"2024-04-16T19:35:41.206571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'46","reported_seq":266,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.466241+0000","last_change":"2024-04-16T19:36:19.466346+0000","last_active":"2024-04-16T19:36:19.466241+0000","last_peered":"2024-04-16T19:36:19.466241+0000","last_clean":"2024-04-16T19:36:19.466241+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:36:19.466241+0000","last_undegraded":"2024-04-16T19:36:19.466241+0000","last_fullsized":"2024-04-16T19:36:19.466241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'46","last_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_deep_scrub":"53'46","last_deep_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.466115+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:05.816236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.436774+0000","last_change":"2024-04-16T19:37:04.436877+0000","last_active":"2024-04-16T19:37:04.436774+0000","last_peered":"2024-04-16T19:37:04.436774+0000","last_clean":"2024-04-16T19:37:04.436774+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:37:04.436774+0000","last_undegraded":"2024-04-16T19:37:04.436774+0000","last_fullsized":"2024-04-16T19:37:04.436774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.436610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.436610+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.436610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:24.645977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209423+0000","last_change":"2024-04-16T17:47:21.253387+0000","last_active":"2024-04-16T19:35:41.209423+0000","last_peered":"2024-04-16T19:35:41.209423+0000","last_clean":"2024-04-16T19:35:41.209423+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:35:41.209423+0000","last_undegraded":"2024-04-16T19:35:41.209423+0000","last_fullsized":"2024-04-16T19:35:41.209423+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'13","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.204923+0000","last_change":"2024-04-16T17:49:32.398311+0000","last_active":"2024-04-16T19:35:56.204923+0000","last_peered":"2024-04-16T19:35:56.204923+0000","last_clean":"2024-04-16T19:35:56.204923+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:35:56.204923+0000","last_undegraded":"2024-04-16T19:35:56.204923+0000","last_fullsized":"2024-04-16T19:35:56.204923+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:54.119057+0000","last_clean_scrub_stamp":"2024-04-16T17:47:54.119057+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.210401+0000","last_change":"2024-04-16T17:47:23.250856+0000","last_active":"2024-04-16T19:35:46.210401+0000","last_peered":"2024-04-16T19:35:46.210401+0000","last_clean":"2024-04-16T19:35:46.210401+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:35:46.210401+0000","last_undegraded":"2024-04-16T19:35:46.210401+0000","last_fullsized":"2024-04-16T19:35:46.210401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.186120+0000","last_change":"2024-04-16T17:47:21.228256+0000","last_active":"2024-04-16T19:35:41.186120+0000","last_peered":"2024-04-16T19:35:41.186120+0000","last_clean":"2024-04-16T19:35:41.186120+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:35:41.186120+0000","last_undegraded":"2024-04-16T19:35:41.186120+0000","last_fullsized":"2024-04-16T19:35:41.186120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'21","reported_seq":207,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.453307+0000","last_change":"2024-04-16T19:36:21.453405+0000","last_active":"2024-04-16T19:36:21.453307+0000","last_peered":"2024-04-16T19:36:21.453307+0000","last_clean":"2024-04-16T19:36:21.453307+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:36:21.453307+0000","last_undegraded":"2024-04-16T19:36:21.453307+0000","last_fullsized":"2024-04-16T19:36:21.453307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.453170+0000","objects_scrubbed":6,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:26.892395+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586277+0000","last_change":"2024-04-16T17:47:23.256663+0000","last_active":"2024-04-16T19:36:16.586277+0000","last_peered":"2024-04-16T19:36:16.586277+0000","last_clean":"2024-04-16T19:36:16.586277+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:36:16.586277+0000","last_undegraded":"2024-04-16T19:36:16.586277+0000","last_fullsized":"2024-04-16T19:36:16.586277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.210981+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:36:06.210981+0000","last_peered":"2024-04-16T19:36:06.210981+0000","last_clean":"2024-04-16T19:36:06.210981+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:36:06.210981+0000","last_undegraded":"2024-04-16T19:36:06.210981+0000","last_fullsized":"2024-04-16T19:36:06.210981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'26","reported_seq":334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.543887+0000","last_change":"2024-04-16T19:36:21.543887+0000","last_active":"2024-04-16T19:36:21.543887+0000","last_peered":"2024-04-16T19:36:21.543887+0000","last_clean":"2024-04-16T19:36:21.543887+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:36:21.543887+0000","last_undegraded":"2024-04-16T19:36:21.543887+0000","last_fullsized":"2024-04-16T19:36:21.543887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'26","last_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_deep_scrub":"53'26","last_deep_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.543655+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:47.643023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.658886+0000","last_change":"2024-04-16T19:37:09.658979+0000","last_active":"2024-04-16T19:37:09.658886+0000","last_peered":"2024-04-16T19:37:09.658886+0000","last_clean":"2024-04-16T19:37:09.658886+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:37:09.658886+0000","last_undegraded":"2024-04-16T19:37:09.658886+0000","last_fullsized":"2024-04-16T19:37:09.658886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.658720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.658720+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.658720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:59.721551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200035+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:35:21.200035+0000","last_peered":"2024-04-16T19:35:21.200035+0000","last_clean":"2024-04-16T19:35:21.200035+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:35:21.200035+0000","last_undegraded":"2024-04-16T19:35:21.200035+0000","last_fullsized":"2024-04-16T19:35:21.200035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'2","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.488252+0000","last_change":"2024-04-16T19:36:22.488332+0000","last_active":"2024-04-16T19:36:22.488252+0000","last_peered":"2024-04-16T19:36:22.488252+0000","last_clean":"2024-04-16T19:36:22.488252+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:36:22.488252+0000","last_undegraded":"2024-04-16T19:36:22.488252+0000","last_fullsized":"2024-04-16T19:36:22.488252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.488109+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:37.620055+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.356894+0000","last_change":"2024-04-16T19:37:09.356950+0000","last_active":"2024-04-16T19:37:09.356894+0000","last_peered":"2024-04-16T19:37:09.356894+0000","last_clean":"2024-04-16T19:37:09.356894+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:37:09.356894+0000","last_undegraded":"2024-04-16T19:37:09.356894+0000","last_fullsized":"2024-04-16T19:37:09.356894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.356747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.356747+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.356747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:28.868964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.199116+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:35:21.199116+0000","last_peered":"2024-04-16T19:35:21.199116+0000","last_clean":"2024-04-16T19:35:21.199116+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:35:21.199116+0000","last_undegraded":"2024-04-16T19:35:21.199116+0000","last_fullsized":"2024-04-16T19:35:21.199116+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'22","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.501837+0000","last_change":"2024-04-16T19:36:47.501923+0000","last_active":"2024-04-16T19:36:47.501837+0000","last_peered":"2024-04-16T19:36:47.501837+0000","last_clean":"2024-04-16T19:36:47.501837+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:36:47.501837+0000","last_undegraded":"2024-04-16T19:36:47.501837+0000","last_fullsized":"2024-04-16T19:36:47.501837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.501703+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:52.574907+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.205594+0000","last_change":"2024-04-16T17:47:23.254509+0000","last_active":"2024-04-16T19:35:26.205594+0000","last_peered":"2024-04-16T19:35:26.205594+0000","last_clean":"2024-04-16T19:35:26.205594+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:35:26.205594+0000","last_undegraded":"2024-04-16T19:35:26.205594+0000","last_fullsized":"2024-04-16T19:35:26.205594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.200185+0000","last_change":"2024-04-16T17:47:21.244693+0000","last_active":"2024-04-16T19:35:21.200185+0000","last_peered":"2024-04-16T19:35:21.200185+0000","last_clean":"2024-04-16T19:35:21.200185+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:35:21.200185+0000","last_undegraded":"2024-04-16T19:35:21.200185+0000","last_fullsized":"2024-04-16T19:35:21.200185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.568771+0000","last_change":"2024-04-16T19:36:22.568861+0000","last_active":"2024-04-16T19:36:22.568771+0000","last_peered":"2024-04-16T19:36:22.568771+0000","last_clean":"2024-04-16T19:36:22.568771+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:36:22.568771+0000","last_undegraded":"2024-04-16T19:36:22.568771+0000","last_fullsized":"2024-04-16T19:36:22.568771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.568637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:27.641841+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.661136+0000","last_change":"2024-04-16T19:37:07.661244+0000","last_active":"2024-04-16T19:37:07.661136+0000","last_peered":"2024-04-16T19:37:07.661136+0000","last_clean":"2024-04-16T19:37:07.661136+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:37:07.661136+0000","last_undegraded":"2024-04-16T19:37:07.661136+0000","last_fullsized":"2024-04-16T19:37:07.661136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.660970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.660970+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.660970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:33.237577+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204436+0000","last_change":"2024-04-16T17:47:21.257702+0000","last_active":"2024-04-16T19:35:21.204436+0000","last_peered":"2024-04-16T19:35:21.204436+0000","last_clean":"2024-04-16T19:35:21.204436+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:35:21.204436+0000","last_undegraded":"2024-04-16T19:35:21.204436+0000","last_fullsized":"2024-04-16T19:35:21.204436+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"53'4","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.473816+0000","last_change":"2024-04-16T19:36:43.473937+0000","last_active":"2024-04-16T19:36:43.473816+0000","last_peered":"2024-04-16T19:36:43.473816+0000","last_clean":"2024-04-16T19:36:43.473816+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:36:43.473816+0000","last_undegraded":"2024-04-16T19:36:43.473816+0000","last_fullsized":"2024-04-16T19:36:43.473816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.473655+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:59.715799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201494+0000","last_change":"2024-04-16T17:47:23.258272+0000","last_active":"2024-04-16T19:35:26.201494+0000","last_peered":"2024-04-16T19:35:26.201494+0000","last_clean":"2024-04-16T19:35:26.201494+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:35:26.201494+0000","last_undegraded":"2024-04-16T19:35:26.201494+0000","last_fullsized":"2024-04-16T19:35:26.201494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:56.213227+0000","last_change":"2024-04-16T17:47:21.250881+0000","last_active":"2024-04-16T19:35:56.213227+0000","last_peered":"2024-04-16T19:35:56.213227+0000","last_clean":"2024-04-16T19:35:56.213227+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:35:56.213227+0000","last_undegraded":"2024-04-16T19:35:56.213227+0000","last_fullsized":"2024-04-16T19:35:56.213227+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'6","reported_seq":182,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.607311+0000","last_change":"2024-04-16T19:36:19.607390+0000","last_active":"2024-04-16T19:36:19.607311+0000","last_peered":"2024-04-16T19:36:19.607311+0000","last_clean":"2024-04-16T19:36:19.607311+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:36:19.607311+0000","last_undegraded":"2024-04-16T19:36:19.607311+0000","last_fullsized":"2024-04-16T19:36:19.607311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.607181+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:07:42.514098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.202288+0000","last_change":"2024-04-16T17:47:23.256278+0000","last_active":"2024-04-16T19:35:26.202288+0000","last_peered":"2024-04-16T19:35:26.202288+0000","last_clean":"2024-04-16T19:35:26.202288+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:35:26.202288+0000","last_undegraded":"2024-04-16T19:35:26.202288+0000","last_fullsized":"2024-04-16T19:35:26.202288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.527662+0000","last_change":"2024-04-16T17:47:21.226455+0000","last_active":"2024-04-16T19:36:17.527662+0000","last_peered":"2024-04-16T19:36:17.527662+0000","last_clean":"2024-04-16T19:36:17.527662+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:36:17.527662+0000","last_undegraded":"2024-04-16T19:36:17.527662+0000","last_fullsized":"2024-04-16T19:36:17.527662+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'12","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498647+0000","last_change":"2024-04-16T19:36:19.498750+0000","last_active":"2024-04-16T19:36:19.498647+0000","last_peered":"2024-04-16T19:36:19.498647+0000","last_clean":"2024-04-16T19:36:19.498647+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:36:19.498647+0000","last_undegraded":"2024-04-16T19:36:19.498647+0000","last_fullsized":"2024-04-16T19:36:19.498647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498508+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:33.427321+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978420+0000","last_change":"2024-04-16T17:47:23.251541+0000","last_active":"2024-04-16T19:36:17.978420+0000","last_peered":"2024-04-16T19:36:17.978420+0000","last_clean":"2024-04-16T19:36:17.978420+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:36:17.978420+0000","last_undegraded":"2024-04-16T19:36:17.978420+0000","last_fullsized":"2024-04-16T19:36:17.978420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.912121+0000","last_change":"2024-04-16T17:47:21.258576+0000","last_active":"2024-04-16T19:36:18.912121+0000","last_peered":"2024-04-16T19:36:18.912121+0000","last_clean":"2024-04-16T19:36:18.912121+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:36:18.912121+0000","last_undegraded":"2024-04-16T19:36:18.912121+0000","last_fullsized":"2024-04-16T19:36:18.912121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'16","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.449658+0000","last_change":"2024-04-16T19:36:28.449751+0000","last_active":"2024-04-16T19:36:28.449658+0000","last_peered":"2024-04-16T19:36:28.449658+0000","last_clean":"2024-04-16T19:36:28.449658+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:36:28.449658+0000","last_undegraded":"2024-04-16T19:36:28.449658+0000","last_fullsized":"2024-04-16T19:36:28.449658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.449522+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:19.924709+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977941+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:36:17.977941+0000","last_peered":"2024-04-16T19:36:17.977941+0000","last_clean":"2024-04-16T19:36:17.977941+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:36:17.977941+0000","last_undegraded":"2024-04-16T19:36:17.977941+0000","last_fullsized":"2024-04-16T19:36:17.977941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.586303+0000","last_change":"2024-04-16T17:47:23.259199+0000","last_active":"2024-04-16T19:36:16.586303+0000","last_peered":"2024-04-16T19:36:16.586303+0000","last_clean":"2024-04-16T19:36:16.586303+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:36:16.586303+0000","last_undegraded":"2024-04-16T19:36:16.586303+0000","last_fullsized":"2024-04-16T19:36:16.586303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'20","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.594397+0000","last_change":"2024-04-16T19:36:27.594512+0000","last_active":"2024-04-16T19:36:27.594397+0000","last_peered":"2024-04-16T19:36:27.594397+0000","last_clean":"2024-04-16T19:36:27.594397+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:36:27.594397+0000","last_undegraded":"2024-04-16T19:36:27.594397+0000","last_fullsized":"2024-04-16T19:36:27.594397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.594261+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:01.851085+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.048425+0000","last_change":"2024-04-16T17:47:21.239499+0000","last_active":"2024-04-16T19:36:17.048425+0000","last_peered":"2024-04-16T19:36:17.048425+0000","last_clean":"2024-04-16T19:36:17.048425+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:36:17.048425+0000","last_undegraded":"2024-04-16T19:36:17.048425+0000","last_fullsized":"2024-04-16T19:36:17.048425+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212640+0000","last_change":"2024-04-16T17:47:23.265257+0000","last_active":"2024-04-16T19:36:11.212640+0000","last_peered":"2024-04-16T19:36:11.212640+0000","last_clean":"2024-04-16T19:36:11.212640+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:36:11.212640+0000","last_undegraded":"2024-04-16T19:36:11.212640+0000","last_fullsized":"2024-04-16T19:36:11.212640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:22.213063+0000","last_clean_scrub_stamp":"2024-04-16T17:47:22.213063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":156857,"num_omap_keys":638,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3508,"ondisk_log_size":3508,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3885,"ondisk_log_size":3885,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2422969,"num_omap_keys":10622,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27854,"ondisk_log_size":27854,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4469,"ondisk_log_size":4469,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9817,"num_omap_keys":62,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":566,"ondisk_log_size":566,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641588,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50744,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806856,"statfs":{"total":107374182400,"available":107322220544,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641587,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":76996,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":104780604,"statfs":{"total":107374182400,"available":107295338496,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641587,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":87452,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":104770148,"statfs":{"total":107374182400,"available":107284631552,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641587,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50732,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806868,"statfs":{"total":107374182400,"available":107322232832,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641588,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":59568,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":104798032,"statfs":{"total":107374182400,"available":107313184768,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641588,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":32700,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":104824900,"statfs":{"total":107374182400,"available":107340697600,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:37:20.899 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.899 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.900 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.900 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.900 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.900 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.900 INFO:tasks.ceph:pgid 9.36 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.901 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.901 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.901 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.901 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.901 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.902 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.902 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.902 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.902 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.903 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.903 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.903 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.903 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.903 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.904 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.904 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.904 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.904 INFO:tasks.ceph:pgid 15.1f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.905 INFO:tasks.ceph:pgid 14.1e last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.905 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.905 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.905 INFO:tasks.ceph:pgid 15.1e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.906 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.906 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.906 INFO:tasks.ceph:pgid 15.1d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.906 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.907 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.907 INFO:tasks.ceph:pgid 15.1c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.907 INFO:tasks.ceph:pgid 14.1d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.907 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.908 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.908 INFO:tasks.ceph:pgid 15.3 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.908 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.908 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.908 INFO:tasks.ceph:pgid 15.2 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.909 INFO:tasks.ceph:pgid 14.3 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.909 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.909 INFO:tasks.ceph:pgid 15.1 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.909 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.910 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.910 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.910 INFO:tasks.ceph:pgid 15.0 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.910 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.911 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.911 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.911 INFO:tasks.ceph:pgid 15.7 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.912 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.912 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.912 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.912 INFO:tasks.ceph:pgid 15.6 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.912 INFO:tasks.ceph:pgid 14.7 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.913 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.913 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.913 INFO:tasks.ceph:pgid 15.5 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.913 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.914 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.914 INFO:tasks.ceph:pgid 15.4 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.914 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.914 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.915 INFO:tasks.ceph:pgid 15.e last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.915 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.915 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.915 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.915 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.916 INFO:tasks.ceph:pgid 15.f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.916 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.916 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.916 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.917 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.917 INFO:tasks.ceph:pgid 15.c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.917 INFO:tasks.ceph:pgid 14.d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.917 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.917 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.918 INFO:tasks.ceph:pgid 15.d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.918 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.918 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.918 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-16T17:46:37.490454+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=37, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.919 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.919 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.919 INFO:tasks.ceph:pgid 15.a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.919 INFO:tasks.ceph:pgid 14.b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.920 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.920 INFO:tasks.ceph:pgid 15.b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.920 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.920 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.920 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.921 INFO:tasks.ceph:pgid 14.9 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.921 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.921 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.921 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.922 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.922 INFO:tasks.ceph:pgid 15.9 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.922 INFO:tasks.ceph:pgid 14.8 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.922 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.922 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.923 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.923 INFO:tasks.ceph:pgid 15.18 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.923 INFO:tasks.ceph:pgid 14.19 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.923 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.924 INFO:tasks.ceph:pgid 15.19 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.924 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.924 INFO:tasks.ceph:pgid 15.1a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.924 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.925 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.925 INFO:tasks.ceph:pgid 15.1b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.925 INFO:tasks.ceph:pgid 14.1a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.925 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.926 INFO:tasks.ceph:pgid 15.14 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.926 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.926 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.926 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.927 INFO:tasks.ceph:pgid 15.15 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.927 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.927 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.927 INFO:tasks.ceph:pgid 15.16 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.927 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.928 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.928 INFO:tasks.ceph:pgid 15.17 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.928 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.928 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.929 INFO:tasks.ceph:pgid 15.10 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.929 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.929 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.929 INFO:tasks.ceph:pgid 15.11 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.930 INFO:tasks.ceph:pgid 9.17 last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.930 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.930 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.930 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.930 INFO:tasks.ceph:pgid 15.12 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.931 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.931 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.931 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.931 INFO:tasks.ceph:pgid 15.13 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.931 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.932 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.932 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.932 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.932 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.933 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.933 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.933 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.933 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.933 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.934 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.934 INFO:tasks.ceph:pgid 9.2d last_scrub_stamp 2024-04-16T17:47:54.119057+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=54, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.934 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.934 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.934 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.935 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.935 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.935 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.935 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.935 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.936 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.936 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.936 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.936 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.936 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.937 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.937 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.937 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.937 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.937 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.938 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-16T17:47:22.213063+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:20.938 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-16T19:37:40.939 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:37:41.322 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:37:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:37:41.366 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3378,"stamp":"2024-04-16T19:37:40.385807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2607626,"num_omap_keys":11398,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41692,"ondisk_log_size":41692,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":380400,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":370230,"kb_avail":628765200,"statfs":{"total":644245094400,"available":643855564800,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":379116192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009254"},"pg_stats":[{"pgid":"9.35","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.507786+0000","last_change":"2024-04-16T19:36:20.507786+0000","last_active":"2024-04-16T19:36:20.507786+0000","last_peered":"2024-04-16T19:36:20.507786+0000","last_clean":"2024-04-16T19:36:20.507786+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:36:20.507786+0000","last_undegraded":"2024-04-16T19:36:20.507786+0000","last_fullsized":"2024-04-16T19:36:20.507786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.507587+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:21:12.018020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209498+0000","last_change":"2024-04-16T17:47:21.239049+0000","last_active":"2024-04-16T19:35:41.209498+0000","last_peered":"2024-04-16T19:35:41.209498+0000","last_clean":"2024-04-16T19:35:41.209498+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:35:41.209498+0000","last_undegraded":"2024-04-16T19:35:41.209498+0000","last_fullsized":"2024-04-16T19:35:41.209498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.506926+0000","last_change":"2024-04-16T19:37:23.506926+0000","last_active":"2024-04-16T19:37:23.506926+0000","last_peered":"2024-04-16T19:37:23.506926+0000","last_clean":"2024-04-16T19:37:23.506926+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:37:23.506926+0000","last_undegraded":"2024-04-16T19:37:23.506926+0000","last_fullsized":"2024-04-16T19:37:23.506926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.506739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.506739+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.506739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:09.177876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'23","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.496453+0000","last_change":"2024-04-16T19:36:37.496453+0000","last_active":"2024-04-16T19:36:37.496453+0000","last_peered":"2024-04-16T19:36:37.496453+0000","last_clean":"2024-04-16T19:36:37.496453+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:36:37.496453+0000","last_undegraded":"2024-04-16T19:36:37.496453+0000","last_fullsized":"2024-04-16T19:36:37.496453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.496061+0000","objects_scrubbed":9,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:45.797648+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.658852+0000","last_change":"2024-04-16T19:37:31.658906+0000","last_active":"2024-04-16T19:37:31.658852+0000","last_peered":"2024-04-16T19:37:31.658852+0000","last_clean":"2024-04-16T19:37:31.658852+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:37:31.658852+0000","last_undegraded":"2024-04-16T19:37:31.658852+0000","last_fullsized":"2024-04-16T19:37:31.658852+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.658712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.658712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.658712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:17.329849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.734066+0000","last_change":"2024-04-16T19:37:32.734117+0000","last_active":"2024-04-16T19:37:32.734066+0000","last_peered":"2024-04-16T19:37:32.734066+0000","last_clean":"2024-04-16T19:37:32.734066+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:37:32.734066+0000","last_undegraded":"2024-04-16T19:37:32.734066+0000","last_fullsized":"2024-04-16T19:37:32.734066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.733931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.733931+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.733931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:18.127607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'25","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.596413+0000","last_change":"2024-04-16T19:36:46.596413+0000","last_active":"2024-04-16T19:36:46.596413+0000","last_peered":"2024-04-16T19:36:46.596413+0000","last_clean":"2024-04-16T19:36:46.596413+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:36:46.596413+0000","last_undegraded":"2024-04-16T19:36:46.596413+0000","last_fullsized":"2024-04-16T19:36:46.596413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'25","last_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_deep_scrub":"53'25","last_deep_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.596297+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:26.632824+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205312+0000","last_change":"2024-04-16T17:47:21.239181+0000","last_active":"2024-04-16T19:35:41.205312+0000","last_peered":"2024-04-16T19:35:41.205312+0000","last_clean":"2024-04-16T19:35:41.205312+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:35:41.205312+0000","last_undegraded":"2024-04-16T19:35:41.205312+0000","last_fullsized":"2024-04-16T19:35:41.205312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.564449+0000","last_change":"2024-04-16T19:37:19.564449+0000","last_active":"2024-04-16T19:37:19.564449+0000","last_peered":"2024-04-16T19:37:19.564449+0000","last_clean":"2024-04-16T19:37:19.564449+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:37:19.564449+0000","last_undegraded":"2024-04-16T19:37:19.564449+0000","last_fullsized":"2024-04-16T19:37:19.564449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.564241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.564241+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.564241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:52.007450+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'15","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.698085+0000","last_change":"2024-04-16T19:37:34.698183+0000","last_active":"2024-04-16T19:37:34.698085+0000","last_peered":"2024-04-16T19:37:34.698085+0000","last_clean":"2024-04-16T19:37:34.698085+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:37:34.698085+0000","last_undegraded":"2024-04-16T19:37:34.698085+0000","last_fullsized":"2024-04-16T19:37:34.698085+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2024-04-16T19:37:34.697956+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2024-04-16T19:37:34.697956+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.697956+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:52.645750+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:46.205993+0000","last_change":"2024-04-16T17:47:21.234705+0000","last_active":"2024-04-16T19:35:46.205993+0000","last_peered":"2024-04-16T19:35:46.205993+0000","last_clean":"2024-04-16T19:35:46.205993+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:35:46.205993+0000","last_undegraded":"2024-04-16T19:35:46.205993+0000","last_fullsized":"2024-04-16T19:35:46.205993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.709971+0000","last_change":"2024-04-16T19:37:31.710023+0000","last_active":"2024-04-16T19:37:31.709971+0000","last_peered":"2024-04-16T19:37:31.709971+0000","last_clean":"2024-04-16T19:37:31.709971+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:37:31.709971+0000","last_undegraded":"2024-04-16T19:37:31.709971+0000","last_fullsized":"2024-04-16T19:37:31.709971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.709834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.709834+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.709834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:17.103509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'8","reported_seq":193,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.663358+0000","last_change":"2024-04-16T19:36:31.663358+0000","last_active":"2024-04-16T19:36:31.663358+0000","last_peered":"2024-04-16T19:36:31.663358+0000","last_clean":"2024-04-16T19:36:31.663358+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:36:31.663358+0000","last_undegraded":"2024-04-16T19:36:31.663358+0000","last_fullsized":"2024-04-16T19:36:31.663358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.663140+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:09.078400+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.620716+0000","last_change":"2024-04-16T19:37:17.620716+0000","last_active":"2024-04-16T19:37:17.620716+0000","last_peered":"2024-04-16T19:37:17.620716+0000","last_clean":"2024-04-16T19:37:17.620716+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:37:17.620716+0000","last_undegraded":"2024-04-16T19:37:17.620716+0000","last_fullsized":"2024-04-16T19:37:17.620716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:17.620394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:17.620394+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.620394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:23:06.022567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.530402+0000","last_change":"2024-04-16T19:37:21.530497+0000","last_active":"2024-04-16T19:37:21.530402+0000","last_peered":"2024-04-16T19:37:21.530402+0000","last_clean":"2024-04-16T19:37:21.530402+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:37:21.530402+0000","last_undegraded":"2024-04-16T19:37:21.530402+0000","last_fullsized":"2024-04-16T19:37:21.530402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.530183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.530183+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.530183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:14:58.265947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'20","reported_seq":328,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.492760+0000","last_change":"2024-04-16T19:36:26.492760+0000","last_active":"2024-04-16T19:36:26.492760+0000","last_peered":"2024-04-16T19:36:26.492760+0000","last_clean":"2024-04-16T19:36:26.492760+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:36:26.492760+0000","last_undegraded":"2024-04-16T19:36:26.492760+0000","last_fullsized":"2024-04-16T19:36:26.492760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.492498+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:56.598246+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911930+0000","last_change":"2024-04-16T17:47:21.230666+0000","last_active":"2024-04-16T19:36:18.911930+0000","last_peered":"2024-04-16T19:36:18.911930+0000","last_clean":"2024-04-16T19:36:18.911930+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:36:18.911930+0000","last_undegraded":"2024-04-16T19:36:18.911930+0000","last_fullsized":"2024-04-16T19:36:18.911930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.508406+0000","last_change":"2024-04-16T19:37:21.508466+0000","last_active":"2024-04-16T19:37:21.508406+0000","last_peered":"2024-04-16T19:37:21.508406+0000","last_clean":"2024-04-16T19:37:21.508406+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:37:21.508406+0000","last_undegraded":"2024-04-16T19:37:21.508406+0000","last_fullsized":"2024-04-16T19:37:21.508406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.508258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.508258+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.508258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:19.847601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'19","reported_seq":285,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.411649+0000","last_change":"2024-04-16T19:36:31.411649+0000","last_active":"2024-04-16T19:36:31.411649+0000","last_peered":"2024-04-16T19:36:31.411649+0000","last_clean":"2024-04-16T19:36:31.411649+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:36:31.411649+0000","last_undegraded":"2024-04-16T19:36:31.411649+0000","last_fullsized":"2024-04-16T19:36:31.411649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.411418+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:18.775822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.978056+0000","last_change":"2024-04-16T17:47:32.282182+0000","last_active":"2024-04-16T19:36:17.978056+0000","last_peered":"2024-04-16T19:36:17.978056+0000","last_clean":"2024-04-16T19:36:17.978056+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:36:17.978056+0000","last_undegraded":"2024-04-16T19:36:17.978056+0000","last_fullsized":"2024-04-16T19:36:17.978056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.465699+0000","last_change":"2024-04-16T19:37:35.465799+0000","last_active":"2024-04-16T19:37:35.465699+0000","last_peered":"2024-04-16T19:37:35.465699+0000","last_clean":"2024-04-16T19:37:35.465699+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:37:35.465699+0000","last_undegraded":"2024-04-16T19:37:35.465699+0000","last_fullsized":"2024-04-16T19:37:35.465699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.465488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.465488+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.465488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:37:45.779394+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'14","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.657838+0000","last_change":"2024-04-16T19:36:29.657923+0000","last_active":"2024-04-16T19:36:29.657838+0000","last_peered":"2024-04-16T19:36:29.657838+0000","last_clean":"2024-04-16T19:36:29.657838+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:36:29.657838+0000","last_undegraded":"2024-04-16T19:36:29.657838+0000","last_fullsized":"2024-04-16T19:36:29.657838+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.657764+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:42.577097+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.468103+0000","last_change":"2024-04-16T19:37:23.468202+0000","last_active":"2024-04-16T19:37:23.468103+0000","last_peered":"2024-04-16T19:37:23.468103+0000","last_clean":"2024-04-16T19:37:23.468103+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:37:23.468103+0000","last_undegraded":"2024-04-16T19:37:23.468103+0000","last_fullsized":"2024-04-16T19:37:23.468103+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.467898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.467898+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.467898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:49.472785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.492993+0000","last_change":"2024-04-16T19:37:13.493045+0000","last_active":"2024-04-16T19:37:13.492993+0000","last_peered":"2024-04-16T19:37:13.492993+0000","last_clean":"2024-04-16T19:37:13.492993+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:37:13.492993+0000","last_undegraded":"2024-04-16T19:37:13.492993+0000","last_fullsized":"2024-04-16T19:37:13.492993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.492847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.492847+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.492847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:18.553733+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"53'10","reported_seq":315,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.620843+0000","last_change":"2024-04-16T19:36:21.620914+0000","last_active":"2024-04-16T19:36:21.620843+0000","last_peered":"2024-04-16T19:36:21.620843+0000","last_clean":"2024-04-16T19:36:21.620843+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:36:21.620843+0000","last_undegraded":"2024-04-16T19:36:21.620843+0000","last_fullsized":"2024-04-16T19:36:21.620843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.620737+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:28.047519+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.545769+0000","last_change":"2024-04-16T19:37:29.545820+0000","last_active":"2024-04-16T19:37:29.545769+0000","last_peered":"2024-04-16T19:37:29.545769+0000","last_clean":"2024-04-16T19:37:29.545769+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:37:29.545769+0000","last_undegraded":"2024-04-16T19:37:29.545769+0000","last_fullsized":"2024-04-16T19:37:29.545769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.545634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.545634+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.545634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:15:05.482579+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.578900+0000","last_change":"2024-04-16T19:37:19.578952+0000","last_active":"2024-04-16T19:37:19.578900+0000","last_peered":"2024-04-16T19:37:19.578900+0000","last_clean":"2024-04-16T19:37:19.578900+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:37:19.578900+0000","last_undegraded":"2024-04-16T19:37:19.578900+0000","last_fullsized":"2024-04-16T19:37:19.578900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.578763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.578763+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.578763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:41.683801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'18","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.400030+0000","last_change":"2024-04-16T19:36:26.400030+0000","last_active":"2024-04-16T19:36:26.400030+0000","last_peered":"2024-04-16T19:36:26.400030+0000","last_clean":"2024-04-16T19:36:26.400030+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:36:26.400030+0000","last_undegraded":"2024-04-16T19:36:26.400030+0000","last_fullsized":"2024-04-16T19:36:26.400030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.399732+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:00.453174+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.498675+0000","last_change":"2024-04-16T19:37:22.498728+0000","last_active":"2024-04-16T19:37:22.498675+0000","last_peered":"2024-04-16T19:37:22.498675+0000","last_clean":"2024-04-16T19:37:22.498675+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:37:22.498675+0000","last_undegraded":"2024-04-16T19:37:22.498675+0000","last_fullsized":"2024-04-16T19:37:22.498675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.498536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.498536+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.498536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:15:49.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215736+0000","last_change":"2024-04-16T17:47:21.250344+0000","last_active":"2024-04-16T19:36:06.215736+0000","last_peered":"2024-04-16T19:36:06.215736+0000","last_clean":"2024-04-16T19:36:06.215736+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:36:06.215736+0000","last_undegraded":"2024-04-16T19:36:06.215736+0000","last_fullsized":"2024-04-16T19:36:06.215736+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'16","reported_seq":288,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.440103+0000","last_change":"2024-04-16T19:36:42.440209+0000","last_active":"2024-04-16T19:36:42.440103+0000","last_peered":"2024-04-16T19:36:42.440103+0000","last_clean":"2024-04-16T19:36:42.440103+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:36:42.440103+0000","last_undegraded":"2024-04-16T19:36:42.440103+0000","last_fullsized":"2024-04-16T19:36:42.440103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.439969+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.619590+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.474066+0000","last_change":"2024-04-16T19:37:29.474066+0000","last_active":"2024-04-16T19:37:29.474066+0000","last_peered":"2024-04-16T19:37:29.474066+0000","last_clean":"2024-04-16T19:37:29.474066+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:37:29.474066+0000","last_undegraded":"2024-04-16T19:37:29.474066+0000","last_fullsized":"2024-04-16T19:37:29.474066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.473871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.473871+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.473871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:59.956479+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211293+0000","last_change":"2024-04-16T17:47:21.251059+0000","last_active":"2024-04-16T19:36:06.211293+0000","last_peered":"2024-04-16T19:36:06.211293+0000","last_clean":"2024-04-16T19:36:06.211293+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:36:06.211293+0000","last_undegraded":"2024-04-16T19:36:06.211293+0000","last_fullsized":"2024-04-16T19:36:06.211293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"53'9","reported_seq":241,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.533226+0000","last_change":"2024-04-16T19:36:29.533377+0000","last_active":"2024-04-16T19:36:29.533226+0000","last_peered":"2024-04-16T19:36:29.533226+0000","last_clean":"2024-04-16T19:36:29.533226+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:36:29.533226+0000","last_undegraded":"2024-04-16T19:36:29.533226+0000","last_fullsized":"2024-04-16T19:36:29.533226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.533050+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:35:44.595573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.441994+0000","last_change":"2024-04-16T19:37:11.442105+0000","last_active":"2024-04-16T19:37:11.441994+0000","last_peered":"2024-04-16T19:37:11.441994+0000","last_clean":"2024-04-16T19:37:11.441994+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:37:11.441994+0000","last_undegraded":"2024-04-16T19:37:11.441994+0000","last_fullsized":"2024-04-16T19:37:11.441994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.441824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.441824+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.441824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:43:01.723675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.597121+0000","last_change":"2024-04-16T19:37:14.597172+0000","last_active":"2024-04-16T19:37:14.597121+0000","last_peered":"2024-04-16T19:37:14.597121+0000","last_clean":"2024-04-16T19:37:14.597121+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:37:14.597121+0000","last_undegraded":"2024-04-16T19:37:14.597121+0000","last_fullsized":"2024-04-16T19:37:14.597121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.596983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.596983+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.596983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:21.782197+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'10","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498172+0000","last_change":"2024-04-16T19:36:19.498276+0000","last_active":"2024-04-16T19:36:19.498172+0000","last_peered":"2024-04-16T19:36:19.498172+0000","last_clean":"2024-04-16T19:36:19.498172+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:36:19.498172+0000","last_undegraded":"2024-04-16T19:36:19.498172+0000","last_fullsized":"2024-04-16T19:36:19.498172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498013+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:24.571217+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.430185+0000","last_change":"2024-04-16T19:37:07.430253+0000","last_active":"2024-04-16T19:37:07.430185+0000","last_peered":"2024-04-16T19:37:07.430185+0000","last_clean":"2024-04-16T19:37:07.430185+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:37:07.430185+0000","last_undegraded":"2024-04-16T19:37:07.430185+0000","last_fullsized":"2024-04-16T19:37:07.430185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.430037+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.430037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:58.189241+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.595934+0000","last_change":"2024-04-16T19:37:23.595986+0000","last_active":"2024-04-16T19:37:23.595934+0000","last_peered":"2024-04-16T19:37:23.595934+0000","last_clean":"2024-04-16T19:37:23.595934+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:37:23.595934+0000","last_undegraded":"2024-04-16T19:37:23.595934+0000","last_fullsized":"2024-04-16T19:37:23.595934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.595797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.595797+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.595797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:16.016867+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'19","reported_seq":283,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.534237+0000","last_change":"2024-04-16T19:36:45.534337+0000","last_active":"2024-04-16T19:36:45.534237+0000","last_peered":"2024-04-16T19:36:45.534237+0000","last_clean":"2024-04-16T19:36:45.534237+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:36:45.534237+0000","last_undegraded":"2024-04-16T19:36:45.534237+0000","last_fullsized":"2024-04-16T19:36:45.534237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.534098+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:29.053953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.606734+0000","last_change":"2024-04-16T19:37:15.606785+0000","last_active":"2024-04-16T19:37:15.606734+0000","last_peered":"2024-04-16T19:37:15.606734+0000","last_clean":"2024-04-16T19:37:15.606734+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:37:15.606734+0000","last_undegraded":"2024-04-16T19:37:15.606734+0000","last_fullsized":"2024-04-16T19:37:15.606734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.606601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.606601+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.606601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:41.974274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.602747+0000","last_change":"2024-04-16T19:37:35.602747+0000","last_active":"2024-04-16T19:37:35.602747+0000","last_peered":"2024-04-16T19:37:35.602747+0000","last_clean":"2024-04-16T19:37:35.602747+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:37:35.602747+0000","last_undegraded":"2024-04-16T19:37:35.602747+0000","last_fullsized":"2024-04-16T19:37:35.602747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.602554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.602554+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.602554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:47:10.329538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.215769+0000","last_change":"2024-04-16T18:10:50.542499+0000","last_active":"2024-04-16T19:36:16.215769+0000","last_peered":"2024-04-16T19:36:16.215769+0000","last_clean":"2024-04-16T19:36:16.215769+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:36:16.215769+0000","last_undegraded":"2024-04-16T19:36:16.215769+0000","last_fullsized":"2024-04-16T19:36:16.215769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'126","reported_seq":1390,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.649445+0000","last_change":"2024-04-16T19:37:16.649511+0000","last_active":"2024-04-16T19:37:16.649445+0000","last_peered":"2024-04-16T19:37:16.649445+0000","last_clean":"2024-04-16T19:37:16.649445+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:37:16.649445+0000","last_undegraded":"2024-04-16T19:37:16.649445+0000","last_fullsized":"2024-04-16T19:37:16.649445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2024-04-16T19:37:16.649314+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2024-04-16T19:37:16.649314+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.649314+0000","objects_scrubbed":29,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:56.902654+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7267,"num_omap_keys":30,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.434783+0000","last_change":"2024-04-16T19:36:56.434907+0000","last_active":"2024-04-16T19:36:56.434783+0000","last_peered":"2024-04-16T19:36:56.434783+0000","last_clean":"2024-04-16T19:36:56.434783+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:36:56.434783+0000","last_undegraded":"2024-04-16T19:36:56.434783+0000","last_fullsized":"2024-04-16T19:36:56.434783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:56.434608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:56.434608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.434608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:12.721346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.645164+0000","last_change":"2024-04-16T19:36:51.645240+0000","last_active":"2024-04-16T19:36:51.645164+0000","last_peered":"2024-04-16T19:36:51.645164+0000","last_clean":"2024-04-16T19:36:51.645164+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:36:51.645164+0000","last_undegraded":"2024-04-16T19:36:51.645164+0000","last_fullsized":"2024-04-16T19:36:51.645164+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.645030+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:42.615404+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.617380+0000","last_change":"2024-04-16T19:36:26.617439+0000","last_active":"2024-04-16T19:36:26.617380+0000","last_peered":"2024-04-16T19:36:26.617380+0000","last_clean":"2024-04-16T19:36:26.617380+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:36:26.617380+0000","last_undegraded":"2024-04-16T19:36:26.617380+0000","last_fullsized":"2024-04-16T19:36:26.617380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.617211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:50.470545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'563","reported_seq":824,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.471001+0000","last_change":"2024-04-16T19:36:20.471131+0000","last_active":"2024-04-16T19:36:20.471001+0000","last_peered":"2024-04-16T19:36:20.471001+0000","last_clean":"2024-04-16T19:36:20.471001+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:36:20.471001+0000","last_undegraded":"2024-04-16T19:36:20.471001+0000","last_fullsized":"2024-04-16T19:36:20.471001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.470861+0000","objects_scrubbed":16,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:34.399674+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71932,"num_omap_keys":314,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'14","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.561790+0000","last_change":"2024-04-16T19:36:25.561790+0000","last_active":"2024-04-16T19:36:25.561790+0000","last_peered":"2024-04-16T19:36:25.561790+0000","last_clean":"2024-04-16T19:36:25.561790+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:36:25.561790+0000","last_undegraded":"2024-04-16T19:36:25.561790+0000","last_fullsized":"2024-04-16T19:36:25.561790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.561550+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:02:57.257953+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.527133+0000","last_change":"2024-04-16T19:36:50.527184+0000","last_active":"2024-04-16T19:36:50.527133+0000","last_peered":"2024-04-16T19:36:50.527133+0000","last_clean":"2024-04-16T19:36:50.527133+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:36:50.527133+0000","last_undegraded":"2024-04-16T19:36:50.527133+0000","last_fullsized":"2024-04-16T19:36:50.527133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.527001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:19.398870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.596329+0000","last_change":"2024-04-16T19:37:18.596329+0000","last_active":"2024-04-16T19:37:18.596329+0000","last_peered":"2024-04-16T19:37:18.596329+0000","last_clean":"2024-04-16T19:37:18.596329+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:37:18.596329+0000","last_undegraded":"2024-04-16T19:37:18.596329+0000","last_fullsized":"2024-04-16T19:37:18.596329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.596131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.596131+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.596131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:47.551456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.595318+0000","last_change":"2024-04-16T19:37:21.595369+0000","last_active":"2024-04-16T19:37:21.595318+0000","last_peered":"2024-04-16T19:37:21.595318+0000","last_clean":"2024-04-16T19:37:21.595318+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:37:21.595318+0000","last_undegraded":"2024-04-16T19:37:21.595318+0000","last_fullsized":"2024-04-16T19:37:21.595318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.595177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.595177+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.595177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:58:43.035164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"53'4","reported_seq":779,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.342632+0000","last_change":"2024-04-16T19:37:14.342695+0000","last_active":"2024-04-16T19:37:14.342632+0000","last_peered":"2024-04-16T19:37:14.342632+0000","last_clean":"2024-04-16T19:37:14.342632+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:37:14.342632+0000","last_undegraded":"2024-04-16T19:37:14.342632+0000","last_fullsized":"2024-04-16T19:37:14.342632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:14.342493+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:14.342493+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.342493+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:34.674033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.673841+0000","last_change":"2024-04-16T19:37:10.673931+0000","last_active":"2024-04-16T19:37:10.673841+0000","last_peered":"2024-04-16T19:37:10.673841+0000","last_clean":"2024-04-16T19:37:10.673841+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:37:10.673841+0000","last_undegraded":"2024-04-16T19:37:10.673841+0000","last_fullsized":"2024-04-16T19:37:10.673841+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:10.673712+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:10.673712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.673712+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:39.298894+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.400379+0000","last_change":"2024-04-16T19:36:48.400438+0000","last_active":"2024-04-16T19:36:48.400379+0000","last_peered":"2024-04-16T19:36:48.400379+0000","last_clean":"2024-04-16T19:36:48.400379+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:36:48.400379+0000","last_undegraded":"2024-04-16T19:36:48.400379+0000","last_fullsized":"2024-04-16T19:36:48.400379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.400232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:53.461118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.561771+0000","last_change":"2024-04-16T19:37:33.561877+0000","last_active":"2024-04-16T19:37:33.561771+0000","last_peered":"2024-04-16T19:37:33.561771+0000","last_clean":"2024-04-16T19:37:33.561771+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:37:33.561771+0000","last_undegraded":"2024-04-16T19:37:33.561771+0000","last_fullsized":"2024-04-16T19:37:33.561771+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.561603+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:22.551773+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'113","reported_seq":1461,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.578659+0000","last_change":"2024-04-16T19:37:01.578802+0000","last_active":"2024-04-16T19:37:01.578659+0000","last_peered":"2024-04-16T19:37:01.578659+0000","last_clean":"2024-04-16T19:37:01.578659+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:37:01.578659+0000","last_undegraded":"2024-04-16T19:37:01.578659+0000","last_fullsized":"2024-04-16T19:37:01.578659+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2024-04-16T19:37:01.578449+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2024-04-16T19:37:01.578449+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.578449+0000","objects_scrubbed":30,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:34.021657+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6547,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.437635+0000","last_change":"2024-04-16T19:36:59.437695+0000","last_active":"2024-04-16T19:36:59.437635+0000","last_peered":"2024-04-16T19:36:59.437635+0000","last_clean":"2024-04-16T19:36:59.437635+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:36:59.437635+0000","last_undegraded":"2024-04-16T19:36:59.437635+0000","last_fullsized":"2024-04-16T19:36:59.437635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.437461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.437461+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.437461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:15.724199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'9","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.654127+0000","last_change":"2024-04-16T19:36:50.654208+0000","last_active":"2024-04-16T19:36:50.654127+0000","last_peered":"2024-04-16T19:36:50.654127+0000","last_clean":"2024-04-16T19:36:50.654127+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:36:50.654127+0000","last_undegraded":"2024-04-16T19:36:50.654127+0000","last_fullsized":"2024-04-16T19:36:50.654127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.653994+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:30.491092+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.479488+0000","last_change":"2024-04-16T19:36:33.479590+0000","last_active":"2024-04-16T19:36:33.479488+0000","last_peered":"2024-04-16T19:36:33.479488+0000","last_clean":"2024-04-16T19:36:33.479488+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:36:33.479488+0000","last_undegraded":"2024-04-16T19:36:33.479488+0000","last_fullsized":"2024-04-16T19:36:33.479488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.479280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:24.835481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7349","reported_seq":20627,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.602403+0000","last_change":"2024-04-16T19:36:20.602529+0000","last_active":"2024-04-16T19:36:20.602403+0000","last_peered":"2024-04-16T19:36:20.602403+0000","last_clean":"2024-04-16T19:36:20.602403+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:36:20.602403+0000","last_undegraded":"2024-04-16T19:36:20.602403+0000","last_fullsized":"2024-04-16T19:36:20.602403+0000","mapping_epoch":34,"log_start":"53'5600","ondisk_log_start":"53'5600","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7349","last_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_deep_scrub":"53'7349","last_deep_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.602264+0000","objects_scrubbed":23,"log_size":1749,"log_dups_size":3000,"ondisk_log_size":1749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:12.702919+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72188,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'30","reported_seq":375,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.493531+0000","last_change":"2024-04-16T19:36:32.493531+0000","last_active":"2024-04-16T19:36:32.493531+0000","last_peered":"2024-04-16T19:36:32.493531+0000","last_clean":"2024-04-16T19:36:32.493531+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:36:32.493531+0000","last_undegraded":"2024-04-16T19:36:32.493531+0000","last_fullsized":"2024-04-16T19:36:32.493531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'30","last_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_deep_scrub":"53'30","last_deep_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.493294+0000","objects_scrubbed":11,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:58.339768+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.426772+0000","last_change":"2024-04-16T19:36:46.426772+0000","last_active":"2024-04-16T19:36:46.426772+0000","last_peered":"2024-04-16T19:36:46.426772+0000","last_clean":"2024-04-16T19:36:46.426772+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:36:46.426772+0000","last_undegraded":"2024-04-16T19:36:46.426772+0000","last_fullsized":"2024-04-16T19:36:46.426772+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.426574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.708425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.500576+0000","last_change":"2024-04-16T19:37:33.500650+0000","last_active":"2024-04-16T19:37:33.500576+0000","last_peered":"2024-04-16T19:37:33.500576+0000","last_clean":"2024-04-16T19:37:33.500576+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:37:33.500576+0000","last_undegraded":"2024-04-16T19:37:33.500576+0000","last_fullsized":"2024-04-16T19:37:33.500576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.500439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.500439+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.500439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:15:01.340149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.728481+0000","last_change":"2024-04-16T19:37:30.728533+0000","last_active":"2024-04-16T19:37:30.728481+0000","last_peered":"2024-04-16T19:37:30.728481+0000","last_clean":"2024-04-16T19:37:30.728481+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:37:30.728481+0000","last_undegraded":"2024-04-16T19:37:30.728481+0000","last_fullsized":"2024-04-16T19:37:30.728481+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.728346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.728346+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.728346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:59:13.778947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"53'7","reported_seq":370,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.684190+0000","last_change":"2024-04-16T19:37:15.684265+0000","last_active":"2024-04-16T19:37:15.684190+0000","last_peered":"2024-04-16T19:37:15.684190+0000","last_clean":"2024-04-16T19:37:15.684190+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:37:15.684190+0000","last_undegraded":"2024-04-16T19:37:15.684190+0000","last_fullsized":"2024-04-16T19:37:15.684190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:15.684059+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:15.684059+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.684059+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:55.937399+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'75","reported_seq":229,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.595038+0000","last_change":"2024-04-16T19:37:02.595110+0000","last_active":"2024-04-16T19:37:02.595038+0000","last_peered":"2024-04-16T19:37:02.595038+0000","last_clean":"2024-04-16T19:37:02.595038+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:37:02.595038+0000","last_undegraded":"2024-04-16T19:37:02.595038+0000","last_fullsized":"2024-04-16T19:37:02.595038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'75","last_scrub_stamp":"2024-04-16T19:37:02.594894+0000","last_deep_scrub":"53'75","last_deep_scrub_stamp":"2024-04-16T19:37:02.594894+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.594894+0000","objects_scrubbed":3,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:07.655780+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.580869+0000","last_change":"2024-04-16T19:36:57.580927+0000","last_active":"2024-04-16T19:36:57.580869+0000","last_peered":"2024-04-16T19:36:57.580869+0000","last_clean":"2024-04-16T19:36:57.580869+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:36:57.580869+0000","last_undegraded":"2024-04-16T19:36:57.580869+0000","last_fullsized":"2024-04-16T19:36:57.580869+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:57.580729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:57.580729+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.580729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:16.681259+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.203492+0000","last_change":"2024-04-16T18:10:50.565329+0000","last_active":"2024-04-16T19:35:51.203492+0000","last_peered":"2024-04-16T19:35:51.203492+0000","last_clean":"2024-04-16T19:35:51.203492+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:35:51.203492+0000","last_undegraded":"2024-04-16T19:35:51.203492+0000","last_fullsized":"2024-04-16T19:35:51.203492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'84","reported_seq":914,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.739730+0000","last_change":"2024-04-16T19:37:13.739802+0000","last_active":"2024-04-16T19:37:13.739730+0000","last_peered":"2024-04-16T19:37:13.739730+0000","last_clean":"2024-04-16T19:37:13.739730+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:37:13.739730+0000","last_undegraded":"2024-04-16T19:37:13.739730+0000","last_fullsized":"2024-04-16T19:37:13.739730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:13.739588+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:13.739588+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.739588+0000","objects_scrubbed":28,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:54.664404+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4355,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.671347+0000","last_change":"2024-04-16T19:36:34.671347+0000","last_active":"2024-04-16T19:36:34.671347+0000","last_peered":"2024-04-16T19:36:34.671347+0000","last_clean":"2024-04-16T19:36:34.671347+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:36:34.671347+0000","last_undegraded":"2024-04-16T19:36:34.671347+0000","last_fullsized":"2024-04-16T19:36:34.671347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.671050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:41.097832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'20","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.631537+0000","last_change":"2024-04-16T19:36:45.631650+0000","last_active":"2024-04-16T19:36:45.631537+0000","last_peered":"2024-04-16T19:36:45.631537+0000","last_clean":"2024-04-16T19:36:45.631537+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:36:45.631537+0000","last_undegraded":"2024-04-16T19:36:45.631537+0000","last_fullsized":"2024-04-16T19:36:45.631537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.631366+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:37.106553+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.586170+0000","last_change":"2024-04-16T19:36:37.586170+0000","last_active":"2024-04-16T19:36:37.586170+0000","last_peered":"2024-04-16T19:36:37.586170+0000","last_clean":"2024-04-16T19:36:37.586170+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:36:37.586170+0000","last_undegraded":"2024-04-16T19:36:37.586170+0000","last_fullsized":"2024-04-16T19:36:37.586170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.585961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:02.979747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3625","reported_seq":7230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.651412+0000","last_change":"2024-04-16T19:36:17.651412+0000","last_active":"2024-04-16T19:36:17.651412+0000","last_peered":"2024-04-16T19:36:17.651412+0000","last_clean":"2024-04-16T19:36:17.651412+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:36:17.651412+0000","last_undegraded":"2024-04-16T19:36:17.651412+0000","last_fullsized":"2024-04-16T19:36:17.651412+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'3625","last_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_deep_scrub":"53'3625","last_deep_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.651328+0000","objects_scrubbed":30,"log_size":1825,"log_dups_size":1800,"ondisk_log_size":1825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:26.898814+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'18","reported_seq":254,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.465484+0000","last_change":"2024-04-16T19:36:17.465591+0000","last_active":"2024-04-16T19:36:17.465484+0000","last_peered":"2024-04-16T19:36:17.465484+0000","last_clean":"2024-04-16T19:36:17.465484+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:36:17.465484+0000","last_undegraded":"2024-04-16T19:36:17.465484+0000","last_fullsized":"2024-04-16T19:36:17.465484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.465360+0000","objects_scrubbed":9,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:23.892141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.615759+0000","last_change":"2024-04-16T19:36:38.615812+0000","last_active":"2024-04-16T19:36:38.615759+0000","last_peered":"2024-04-16T19:36:38.615759+0000","last_clean":"2024-04-16T19:36:38.615759+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:36:38.615759+0000","last_undegraded":"2024-04-16T19:36:38.615759+0000","last_fullsized":"2024-04-16T19:36:38.615759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.615619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:29.374824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.635237+0000","last_change":"2024-04-16T19:37:24.635287+0000","last_active":"2024-04-16T19:37:24.635237+0000","last_peered":"2024-04-16T19:37:24.635237+0000","last_clean":"2024-04-16T19:37:24.635237+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:37:24.635237+0000","last_undegraded":"2024-04-16T19:37:24.635237+0000","last_fullsized":"2024-04-16T19:37:24.635237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.635106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.635106+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.635106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:01.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.693226+0000","last_change":"2024-04-16T19:37:23.693280+0000","last_active":"2024-04-16T19:37:23.693226+0000","last_peered":"2024-04-16T19:37:23.693226+0000","last_clean":"2024-04-16T19:37:23.693226+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:37:23.693226+0000","last_undegraded":"2024-04-16T19:37:23.693226+0000","last_fullsized":"2024-04-16T19:37:23.693226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.693084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.693084+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.693084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:49.613545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"53'13","reported_seq":617,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.385910+0000","last_change":"2024-04-16T19:37:08.386038+0000","last_active":"2024-04-16T19:37:08.385910+0000","last_peered":"2024-04-16T19:37:08.385910+0000","last_clean":"2024-04-16T19:37:08.385910+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:37:08.385910+0000","last_undegraded":"2024-04-16T19:37:08.385910+0000","last_fullsized":"2024-04-16T19:37:08.385910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:08.385737+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:08.385737+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.385737+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:45.642460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'125","reported_seq":304,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.417448+0000","last_change":"2024-04-16T19:36:55.417529+0000","last_active":"2024-04-16T19:36:55.417448+0000","last_peered":"2024-04-16T19:36:55.417448+0000","last_clean":"2024-04-16T19:36:55.417448+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:36:55.417448+0000","last_undegraded":"2024-04-16T19:36:55.417448+0000","last_fullsized":"2024-04-16T19:36:55.417448+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.417311+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:25.523059+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'13","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.351749+0000","last_change":"2024-04-16T19:36:45.351811+0000","last_active":"2024-04-16T19:36:45.351749+0000","last_peered":"2024-04-16T19:36:45.351749+0000","last_clean":"2024-04-16T19:36:45.351749+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:36:45.351749+0000","last_undegraded":"2024-04-16T19:36:45.351749+0000","last_fullsized":"2024-04-16T19:36:45.351749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.351601+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:50.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.209444+0000","last_change":"2024-04-16T18:10:50.558833+0000","last_active":"2024-04-16T19:35:51.209444+0000","last_peered":"2024-04-16T19:35:51.209444+0000","last_clean":"2024-04-16T19:35:51.209444+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:35:51.209444+0000","last_undegraded":"2024-04-16T19:35:51.209444+0000","last_fullsized":"2024-04-16T19:35:51.209444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'84","reported_seq":771,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.627099+0000","last_change":"2024-04-16T19:37:17.627212+0000","last_active":"2024-04-16T19:37:17.627099+0000","last_peered":"2024-04-16T19:37:17.627099+0000","last_clean":"2024-04-16T19:37:17.627099+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:37:17.627099+0000","last_undegraded":"2024-04-16T19:37:17.627099+0000","last_fullsized":"2024-04-16T19:37:17.627099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:17.626928+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:17.626928+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.626928+0000","objects_scrubbed":25,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:42.762558+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3553,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.671455+0000","last_change":"2024-04-16T19:36:35.671455+0000","last_active":"2024-04-16T19:36:35.671455+0000","last_peered":"2024-04-16T19:36:35.671455+0000","last_clean":"2024-04-16T19:36:35.671455+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:36:35.671455+0000","last_undegraded":"2024-04-16T19:36:35.671455+0000","last_fullsized":"2024-04-16T19:36:35.671455+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.671258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:03.249930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'21","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.435794+0000","last_change":"2024-04-16T19:36:50.435867+0000","last_active":"2024-04-16T19:36:50.435794+0000","last_peered":"2024-04-16T19:36:50.435794+0000","last_clean":"2024-04-16T19:36:50.435794+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:36:50.435794+0000","last_undegraded":"2024-04-16T19:36:50.435794+0000","last_fullsized":"2024-04-16T19:36:50.435794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.435653+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:16.012260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.469677+0000","last_change":"2024-04-16T19:36:39.469677+0000","last_active":"2024-04-16T19:36:39.469677+0000","last_peered":"2024-04-16T19:36:39.469677+0000","last_clean":"2024-04-16T19:36:39.469677+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:36:39.469677+0000","last_undegraded":"2024-04-16T19:36:39.469677+0000","last_fullsized":"2024-04-16T19:36:39.469677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.469488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.661779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'408","reported_seq":769,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.522794+0000","last_change":"2024-04-16T19:36:31.522794+0000","last_active":"2024-04-16T19:36:31.522794+0000","last_peered":"2024-04-16T19:36:31.522794+0000","last_clean":"2024-04-16T19:36:31.522794+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:36:31.522794+0000","last_undegraded":"2024-04-16T19:36:31.522794+0000","last_fullsized":"2024-04-16T19:36:31.522794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'408","last_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_deep_scrub":"53'408","last_deep_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.522481+0000","objects_scrubbed":23,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:59.101153+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'12","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.541977+0000","last_change":"2024-04-16T19:36:17.542045+0000","last_active":"2024-04-16T19:36:17.541977+0000","last_peered":"2024-04-16T19:36:17.541977+0000","last_clean":"2024-04-16T19:36:17.541977+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:36:17.541977+0000","last_undegraded":"2024-04-16T19:36:17.541977+0000","last_fullsized":"2024-04-16T19:36:17.541977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.541913+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:24.087701+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.579713+0000","last_change":"2024-04-16T19:36:47.579767+0000","last_active":"2024-04-16T19:36:47.579713+0000","last_peered":"2024-04-16T19:36:47.579713+0000","last_clean":"2024-04-16T19:36:47.579713+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:36:47.579713+0000","last_undegraded":"2024-04-16T19:36:47.579713+0000","last_fullsized":"2024-04-16T19:36:47.579713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.579576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:56.945905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.662189+0000","last_change":"2024-04-16T19:37:28.662247+0000","last_active":"2024-04-16T19:37:28.662189+0000","last_peered":"2024-04-16T19:37:28.662189+0000","last_clean":"2024-04-16T19:37:28.662189+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:37:28.662189+0000","last_undegraded":"2024-04-16T19:37:28.662189+0000","last_fullsized":"2024-04-16T19:37:28.662189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.662050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.662050+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.662050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:01:18.105359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911701+0000","last_change":"2024-04-16T17:47:21.236360+0000","last_active":"2024-04-16T19:36:18.911701+0000","last_peered":"2024-04-16T19:36:18.911701+0000","last_clean":"2024-04-16T19:36:18.911701+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:36:18.911701+0000","last_undegraded":"2024-04-16T19:36:18.911701+0000","last_fullsized":"2024-04-16T19:36:18.911701+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.516275+0000","last_change":"2024-04-16T19:37:13.516327+0000","last_active":"2024-04-16T19:37:13.516275+0000","last_peered":"2024-04-16T19:37:13.516275+0000","last_clean":"2024-04-16T19:37:13.516275+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:37:13.516275+0000","last_undegraded":"2024-04-16T19:37:13.516275+0000","last_fullsized":"2024-04-16T19:37:13.516275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.516141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.516141+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.516141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:59.747526+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.526949+0000","last_change":"2024-04-16T19:37:09.527017+0000","last_active":"2024-04-16T19:37:09.526949+0000","last_peered":"2024-04-16T19:37:09.526949+0000","last_clean":"2024-04-16T19:37:09.526949+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:37:09.526949+0000","last_undegraded":"2024-04-16T19:37:09.526949+0000","last_fullsized":"2024-04-16T19:37:09.526949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:09.526805+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:09.526805+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.526805+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:03.234997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'13","reported_seq":1344,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.649901+0000","last_change":"2024-04-16T19:36:52.649962+0000","last_active":"2024-04-16T19:36:52.649901+0000","last_peered":"2024-04-16T19:36:52.649901+0000","last_clean":"2024-04-16T19:36:52.649901+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:36:52.649901+0000","last_undegraded":"2024-04-16T19:36:52.649901+0000","last_fullsized":"2024-04-16T19:36:52.649901+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.649763+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.928938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.503286+0000","last_change":"2024-04-16T19:37:33.503338+0000","last_active":"2024-04-16T19:37:33.503286+0000","last_peered":"2024-04-16T19:37:33.503286+0000","last_clean":"2024-04-16T19:37:33.503286+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:37:33.503286+0000","last_undegraded":"2024-04-16T19:37:33.503286+0000","last_fullsized":"2024-04-16T19:37:33.503286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.503150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.503150+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.503150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:33:13.756490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'200","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.601906+0000","last_change":"2024-04-16T19:37:08.601960+0000","last_active":"2024-04-16T19:37:08.601906+0000","last_peered":"2024-04-16T19:37:08.601906+0000","last_clean":"2024-04-16T19:37:08.601906+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:37:08.601906+0000","last_undegraded":"2024-04-16T19:37:08.601906+0000","last_fullsized":"2024-04-16T19:37:08.601906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'200","last_scrub_stamp":"2024-04-16T19:37:08.601835+0000","last_deep_scrub":"53'200","last_deep_scrub_stamp":"2024-04-16T19:37:08.601835+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.601835+0000","objects_scrubbed":36,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:25.200215+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19169,"num_omap_keys":76,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.413351+0000","last_change":"2024-04-16T19:36:52.413405+0000","last_active":"2024-04-16T19:36:52.413351+0000","last_peered":"2024-04-16T19:36:52.413351+0000","last_clean":"2024-04-16T19:36:52.413351+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:36:52.413351+0000","last_undegraded":"2024-04-16T19:36:52.413351+0000","last_fullsized":"2024-04-16T19:36:52.413351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.413208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:42.476038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'13","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.645823+0000","last_change":"2024-04-16T19:36:38.645892+0000","last_active":"2024-04-16T19:36:38.645823+0000","last_peered":"2024-04-16T19:36:38.645823+0000","last_clean":"2024-04-16T19:36:38.645823+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:36:38.645823+0000","last_undegraded":"2024-04-16T19:36:38.645823+0000","last_fullsized":"2024-04-16T19:36:38.645823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.645696+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:29.924307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":158,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.474393+0000","last_change":"2024-04-16T19:36:58.474445+0000","last_active":"2024-04-16T19:36:58.474393+0000","last_peered":"2024-04-16T19:36:58.474393+0000","last_clean":"2024-04-16T19:36:58.474393+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:36:58.474393+0000","last_undegraded":"2024-04-16T19:36:58.474393+0000","last_fullsized":"2024-04-16T19:36:58.474393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.474247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.474247+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.474247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:24.479134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1110","reported_seq":1775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.529609+0000","last_change":"2024-04-16T19:36:31.529728+0000","last_active":"2024-04-16T19:36:31.529609+0000","last_peered":"2024-04-16T19:36:31.529609+0000","last_clean":"2024-04-16T19:36:31.529609+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:36:31.529609+0000","last_undegraded":"2024-04-16T19:36:31.529609+0000","last_fullsized":"2024-04-16T19:36:31.529609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1110","last_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_deep_scrub":"53'1110","last_deep_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.529471+0000","objects_scrubbed":23,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:18.708104+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143914,"num_omap_keys":628,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"53'8","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.477590+0000","last_change":"2024-04-16T19:36:20.477590+0000","last_active":"2024-04-16T19:36:20.477590+0000","last_peered":"2024-04-16T19:36:20.477590+0000","last_clean":"2024-04-16T19:36:20.477590+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:36:20.477590+0000","last_undegraded":"2024-04-16T19:36:20.477590+0000","last_fullsized":"2024-04-16T19:36:20.477590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.477384+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:25.916610+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.695046+0000","last_change":"2024-04-16T19:37:22.695098+0000","last_active":"2024-04-16T19:37:22.695046+0000","last_peered":"2024-04-16T19:37:22.695046+0000","last_clean":"2024-04-16T19:37:22.695046+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:37:22.695046+0000","last_undegraded":"2024-04-16T19:37:22.695046+0000","last_fullsized":"2024-04-16T19:37:22.695046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.694908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.694908+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.694908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:18.580946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.469846+0000","last_change":"2024-04-16T19:37:32.469897+0000","last_active":"2024-04-16T19:37:32.469846+0000","last_peered":"2024-04-16T19:37:32.469846+0000","last_clean":"2024-04-16T19:37:32.469846+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:37:32.469846+0000","last_undegraded":"2024-04-16T19:37:32.469846+0000","last_fullsized":"2024-04-16T19:37:32.469846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.469708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.469708+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.469708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:23.814824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.372126+0000","last_change":"2024-04-16T19:37:13.372176+0000","last_active":"2024-04-16T19:37:13.372126+0000","last_peered":"2024-04-16T19:37:13.372126+0000","last_clean":"2024-04-16T19:37:13.372126+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:37:13.372126+0000","last_undegraded":"2024-04-16T19:37:13.372126+0000","last_fullsized":"2024-04-16T19:37:13.372126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.371985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.371985+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.371985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:47:39.425403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'209","reported_seq":437,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.363263+0000","last_change":"2024-04-16T19:37:08.363339+0000","last_active":"2024-04-16T19:37:08.363263+0000","last_peered":"2024-04-16T19:37:08.363263+0000","last_clean":"2024-04-16T19:37:08.363263+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:37:08.363263+0000","last_undegraded":"2024-04-16T19:37:08.363263+0000","last_fullsized":"2024-04-16T19:37:08.363263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'209","last_scrub_stamp":"2024-04-16T19:37:08.363125+0000","last_deep_scrub":"53'209","last_deep_scrub_stamp":"2024-04-16T19:37:08.363125+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.363125+0000","objects_scrubbed":9,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:57.353295+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.650228+0000","last_change":"2024-04-16T19:36:49.650334+0000","last_active":"2024-04-16T19:36:49.650228+0000","last_peered":"2024-04-16T19:36:49.650228+0000","last_clean":"2024-04-16T19:36:49.650228+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:36:49.650228+0000","last_undegraded":"2024-04-16T19:36:49.650228+0000","last_fullsized":"2024-04-16T19:36:49.650228+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.650064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:03.271488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.443726+0000","last_change":"2024-04-16T19:36:42.443778+0000","last_active":"2024-04-16T19:36:42.443726+0000","last_peered":"2024-04-16T19:36:42.443726+0000","last_clean":"2024-04-16T19:36:42.443726+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:36:42.443726+0000","last_undegraded":"2024-04-16T19:36:42.443726+0000","last_fullsized":"2024-04-16T19:36:42.443726+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.443589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:33.202793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.499959+0000","last_change":"2024-04-16T19:37:30.500012+0000","last_active":"2024-04-16T19:37:30.499959+0000","last_peered":"2024-04-16T19:37:30.499959+0000","last_clean":"2024-04-16T19:37:30.499959+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:37:30.499959+0000","last_undegraded":"2024-04-16T19:37:30.499959+0000","last_fullsized":"2024-04-16T19:37:30.499959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.499811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.499811+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.499811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:55.818847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'109","reported_seq":1189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587906+0000","last_change":"2024-04-16T17:48:34.346572+0000","last_active":"2024-04-16T19:36:16.587906+0000","last_peered":"2024-04-16T19:36:16.587906+0000","last_clean":"2024-04-16T19:36:16.587906+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:36:16.587906+0000","last_undegraded":"2024-04-16T19:36:16.587906+0000","last_fullsized":"2024-04-16T19:36:16.587906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.441666+0000","last_change":"2024-04-16T19:36:48.441666+0000","last_active":"2024-04-16T19:36:48.441666+0000","last_peered":"2024-04-16T19:36:48.441666+0000","last_clean":"2024-04-16T19:36:48.441666+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:36:48.441666+0000","last_undegraded":"2024-04-16T19:36:48.441666+0000","last_fullsized":"2024-04-16T19:36:48.441666+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.441481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:54.447789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'4","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.445595+0000","last_change":"2024-04-16T19:36:53.445663+0000","last_active":"2024-04-16T19:36:53.445595+0000","last_peered":"2024-04-16T19:36:53.445595+0000","last_clean":"2024-04-16T19:36:53.445595+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:36:53.445595+0000","last_undegraded":"2024-04-16T19:36:53.445595+0000","last_fullsized":"2024-04-16T19:36:53.445595+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.445453+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:27.787458+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.610902+0000","last_change":"2024-04-16T19:36:39.610922+0000","last_active":"2024-04-16T19:36:39.610902+0000","last_peered":"2024-04-16T19:36:39.610902+0000","last_clean":"2024-04-16T19:36:39.610902+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:36:39.610902+0000","last_undegraded":"2024-04-16T19:36:39.610902+0000","last_fullsized":"2024-04-16T19:36:39.610902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.610843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:16.867566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1195","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.442330+0000","last_change":"2024-04-16T19:36:32.442434+0000","last_active":"2024-04-16T19:36:32.442330+0000","last_peered":"2024-04-16T19:36:32.442330+0000","last_clean":"2024-04-16T19:36:32.442330+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:36:32.442330+0000","last_undegraded":"2024-04-16T19:36:32.442330+0000","last_fullsized":"2024-04-16T19:36:32.442330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1195","last_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_deep_scrub":"53'1195","last_deep_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.442194+0000","objects_scrubbed":22,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:37.854921+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215932,"num_omap_keys":942,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'17","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.474347+0000","last_change":"2024-04-16T19:36:21.474426+0000","last_active":"2024-04-16T19:36:21.474347+0000","last_peered":"2024-04-16T19:36:21.474347+0000","last_clean":"2024-04-16T19:36:21.474347+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:36:21.474347+0000","last_undegraded":"2024-04-16T19:36:21.474347+0000","last_fullsized":"2024-04-16T19:36:21.474347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'17","last_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_deep_scrub":"53'17","last_deep_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.474217+0000","objects_scrubbed":3,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:40.463538+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.640159+0000","last_change":"2024-04-16T19:37:08.640210+0000","last_active":"2024-04-16T19:37:08.640159+0000","last_peered":"2024-04-16T19:37:08.640159+0000","last_clean":"2024-04-16T19:37:08.640159+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:37:08.640159+0000","last_undegraded":"2024-04-16T19:37:08.640159+0000","last_fullsized":"2024-04-16T19:37:08.640159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:08.640022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:08.640022+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.640022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:42.193624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.517042+0000","last_change":"2024-04-16T19:37:30.517042+0000","last_active":"2024-04-16T19:37:30.517042+0000","last_peered":"2024-04-16T19:37:30.517042+0000","last_clean":"2024-04-16T19:37:30.517042+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:37:30.517042+0000","last_undegraded":"2024-04-16T19:37:30.517042+0000","last_fullsized":"2024-04-16T19:37:30.517042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.516852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.516852+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.516852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:47.230865+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"53'2","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.485428+0000","last_change":"2024-04-16T19:37:12.485493+0000","last_active":"2024-04-16T19:37:12.485428+0000","last_peered":"2024-04-16T19:37:12.485428+0000","last_clean":"2024-04-16T19:37:12.485428+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:37:12.485428+0000","last_undegraded":"2024-04-16T19:37:12.485428+0000","last_fullsized":"2024-04-16T19:37:12.485428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:12.485279+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:12.485279+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.485279+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.828865+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'84","reported_seq":250,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.729167+0000","last_change":"2024-04-16T19:36:57.729240+0000","last_active":"2024-04-16T19:36:57.729167+0000","last_peered":"2024-04-16T19:36:57.729167+0000","last_clean":"2024-04-16T19:36:57.729167+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:36:57.729167+0000","last_undegraded":"2024-04-16T19:36:57.729167+0000","last_fullsized":"2024-04-16T19:36:57.729167+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:36:57.729025+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:36:57.729025+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.729025+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:26:51.751473+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.388357+0000","last_change":"2024-04-16T19:36:54.388412+0000","last_active":"2024-04-16T19:36:54.388357+0000","last_peered":"2024-04-16T19:36:54.388357+0000","last_clean":"2024-04-16T19:36:54.388357+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:36:54.388357+0000","last_undegraded":"2024-04-16T19:36:54.388357+0000","last_fullsized":"2024-04-16T19:36:54.388357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.388213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:28.441656+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.545722+0000","last_change":"2024-04-16T19:36:53.545774+0000","last_active":"2024-04-16T19:36:53.545722+0000","last_peered":"2024-04-16T19:36:53.545722+0000","last_clean":"2024-04-16T19:36:53.545722+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:36:53.545722+0000","last_undegraded":"2024-04-16T19:36:53.545722+0000","last_fullsized":"2024-04-16T19:36:53.545722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.545589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:08.257713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.217900+0000","last_change":"2024-04-16T18:10:50.562129+0000","last_active":"2024-04-16T19:36:16.217900+0000","last_peered":"2024-04-16T19:36:16.217900+0000","last_clean":"2024-04-16T19:36:16.217900+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:36:16.217900+0000","last_undegraded":"2024-04-16T19:36:16.217900+0000","last_fullsized":"2024-04-16T19:36:16.217900+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'90","reported_seq":931,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.425451+0000","last_change":"2024-04-16T19:36:56.425451+0000","last_active":"2024-04-16T19:36:56.425451+0000","last_peered":"2024-04-16T19:36:56.425451+0000","last_clean":"2024-04-16T19:36:56.425451+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:36:56.425451+0000","last_undegraded":"2024-04-16T19:36:56.425451+0000","last_fullsized":"2024-04-16T19:36:56.425451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'90","last_scrub_stamp":"2024-04-16T19:36:56.425221+0000","last_deep_scrub":"53'90","last_deep_scrub_stamp":"2024-04-16T19:36:56.425221+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.425221+0000","objects_scrubbed":20,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:43.603855+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6839,"num_omap_keys":28,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.686464+0000","last_change":"2024-04-16T19:36:49.686464+0000","last_active":"2024-04-16T19:36:49.686464+0000","last_peered":"2024-04-16T19:36:49.686464+0000","last_clean":"2024-04-16T19:36:49.686464+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:36:49.686464+0000","last_undegraded":"2024-04-16T19:36:49.686464+0000","last_fullsized":"2024-04-16T19:36:49.686464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.686271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.036352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'9","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980079+0000","last_change":"2024-04-16T17:49:32.412466+0000","last_active":"2024-04-16T19:36:17.980079+0000","last_peered":"2024-04-16T19:36:17.980079+0000","last_clean":"2024-04-16T19:36:17.980079+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:36:17.980079+0000","last_undegraded":"2024-04-16T19:36:17.980079+0000","last_fullsized":"2024-04-16T19:36:17.980079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:00.194486+0000","last_clean_scrub_stamp":"2024-04-16T17:48:00.194486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.684642+0000","last_change":"2024-04-16T19:36:32.684642+0000","last_active":"2024-04-16T19:36:32.684642+0000","last_peered":"2024-04-16T19:36:32.684642+0000","last_clean":"2024-04-16T19:36:32.684642+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:36:32.684642+0000","last_undegraded":"2024-04-16T19:36:32.684642+0000","last_fullsized":"2024-04-16T19:36:32.684642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.684452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:11.352292+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1071","reported_seq":6142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.466107+0000","last_change":"2024-04-16T19:36:31.466251+0000","last_active":"2024-04-16T19:36:31.466107+0000","last_peered":"2024-04-16T19:36:31.466107+0000","last_clean":"2024-04-16T19:36:31.466107+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:36:31.466107+0000","last_undegraded":"2024-04-16T19:36:31.466107+0000","last_fullsized":"2024-04-16T19:36:31.466107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1071","last_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_deep_scrub":"53'1071","last_deep_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.465959+0000","objects_scrubbed":22,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:36.878686+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71967,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'8","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.607489+0000","last_change":"2024-04-16T19:36:24.607587+0000","last_active":"2024-04-16T19:36:24.607489+0000","last_peered":"2024-04-16T19:36:24.607489+0000","last_clean":"2024-04-16T19:36:24.607489+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:36:24.607489+0000","last_undegraded":"2024-04-16T19:36:24.607489+0000","last_fullsized":"2024-04-16T19:36:24.607489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.607347+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:30.046572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.497814+0000","last_change":"2024-04-16T19:37:31.497868+0000","last_active":"2024-04-16T19:37:31.497814+0000","last_peered":"2024-04-16T19:37:31.497814+0000","last_clean":"2024-04-16T19:37:31.497814+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:37:31.497814+0000","last_undegraded":"2024-04-16T19:37:31.497814+0000","last_fullsized":"2024-04-16T19:37:31.497814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.497663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.497663+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.497663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:56.021787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.691898+0000","last_change":"2024-04-16T19:37:31.691898+0000","last_active":"2024-04-16T19:37:31.691898+0000","last_peered":"2024-04-16T19:37:31.691898+0000","last_clean":"2024-04-16T19:37:31.691898+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:37:31.691898+0000","last_undegraded":"2024-04-16T19:37:31.691898+0000","last_fullsized":"2024-04-16T19:37:31.691898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.691712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.691712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.691712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:59:14.742312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.493197+0000","last_change":"2024-04-16T19:37:14.493250+0000","last_active":"2024-04-16T19:37:14.493197+0000","last_peered":"2024-04-16T19:37:14.493197+0000","last_clean":"2024-04-16T19:37:14.493197+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:37:14.493197+0000","last_undegraded":"2024-04-16T19:37:14.493197+0000","last_fullsized":"2024-04-16T19:37:14.493197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.493058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.493058+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.493058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:17.475775+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'159","reported_seq":363,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.570200+0000","last_change":"2024-04-16T19:37:06.570279+0000","last_active":"2024-04-16T19:37:06.570200+0000","last_peered":"2024-04-16T19:37:06.570200+0000","last_clean":"2024-04-16T19:37:06.570200+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:37:06.570200+0000","last_undegraded":"2024-04-16T19:37:06.570200+0000","last_fullsized":"2024-04-16T19:37:06.570200+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:06.570065+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:06.570065+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.570065+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:27.105572+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'13","reported_seq":1348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.514311+0000","last_change":"2024-04-16T19:37:06.514379+0000","last_active":"2024-04-16T19:37:06.514311+0000","last_peered":"2024-04-16T19:37:06.514311+0000","last_clean":"2024-04-16T19:37:06.514311+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:37:06.514311+0000","last_undegraded":"2024-04-16T19:37:06.514311+0000","last_fullsized":"2024-04-16T19:37:06.514311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:06.514177+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:06.514177+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.514177+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:28.619214+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.712924+0000","last_change":"2024-04-16T19:36:43.712975+0000","last_active":"2024-04-16T19:36:43.712924+0000","last_peered":"2024-04-16T19:36:43.712924+0000","last_clean":"2024-04-16T19:36:43.712924+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:36:43.712924+0000","last_undegraded":"2024-04-16T19:36:43.712924+0000","last_fullsized":"2024-04-16T19:36:43.712924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.712792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:43.360972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.502570+0000","last_change":"2024-04-16T19:37:29.502625+0000","last_active":"2024-04-16T19:37:29.502570+0000","last_peered":"2024-04-16T19:37:29.502570+0000","last_clean":"2024-04-16T19:37:29.502570+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:37:29.502570+0000","last_undegraded":"2024-04-16T19:37:29.502570+0000","last_fullsized":"2024-04-16T19:37:29.502570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.502425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.502425+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.502425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:24:42.672974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'100","reported_seq":1255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.688914+0000","last_change":"2024-04-16T19:36:47.688914+0000","last_active":"2024-04-16T19:36:47.688914+0000","last_peered":"2024-04-16T19:36:47.688914+0000","last_clean":"2024-04-16T19:36:47.688914+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:36:47.688914+0000","last_undegraded":"2024-04-16T19:36:47.688914+0000","last_fullsized":"2024-04-16T19:36:47.688914+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'100","last_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_deep_scrub":"53'100","last_deep_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.688717+0000","objects_scrubbed":26,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:21.999831+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5240,"num_omap_keys":22,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.472784+0000","last_change":"2024-04-16T19:36:55.472893+0000","last_active":"2024-04-16T19:36:55.472784+0000","last_peered":"2024-04-16T19:36:55.472784+0000","last_clean":"2024-04-16T19:36:55.472784+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:36:55.472784+0000","last_undegraded":"2024-04-16T19:36:55.472784+0000","last_fullsized":"2024-04-16T19:36:55.472784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.472614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:45.535445+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'20","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.637124+0000","last_change":"2024-04-16T19:36:41.637203+0000","last_active":"2024-04-16T19:36:41.637124+0000","last_peered":"2024-04-16T19:36:41.637124+0000","last_clean":"2024-04-16T19:36:41.637124+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:36:41.637124+0000","last_undegraded":"2024-04-16T19:36:41.637124+0000","last_fullsized":"2024-04-16T19:36:41.637124+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.636981+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.258518+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.453915+0000","last_change":"2024-04-16T19:36:39.453971+0000","last_active":"2024-04-16T19:36:39.453915+0000","last_peered":"2024-04-16T19:36:39.453915+0000","last_clean":"2024-04-16T19:36:39.453915+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:36:39.453915+0000","last_undegraded":"2024-04-16T19:36:39.453915+0000","last_fullsized":"2024-04-16T19:36:39.453915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.453770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:59.663137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'736","reported_seq":1478,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.630399+0000","last_change":"2024-04-16T19:36:18.630399+0000","last_active":"2024-04-16T19:36:18.630399+0000","last_peered":"2024-04-16T19:36:18.630399+0000","last_clean":"2024-04-16T19:36:18.630399+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:36:18.630399+0000","last_undegraded":"2024-04-16T19:36:18.630399+0000","last_fullsized":"2024-04-16T19:36:18.630399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'736","last_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_deep_scrub":"53'736","last_deep_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.630302+0000","objects_scrubbed":33,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:45.495590+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":75043,"num_omap_keys":332,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'18","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.577485+0000","last_change":"2024-04-16T19:36:26.577559+0000","last_active":"2024-04-16T19:36:26.577485+0000","last_peered":"2024-04-16T19:36:26.577485+0000","last_clean":"2024-04-16T19:36:26.577485+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:36:26.577485+0000","last_undegraded":"2024-04-16T19:36:26.577485+0000","last_fullsized":"2024-04-16T19:36:26.577485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.577356+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:41.709302+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.649798+0000","last_change":"2024-04-16T19:37:33.649851+0000","last_active":"2024-04-16T19:37:33.649798+0000","last_peered":"2024-04-16T19:37:33.649798+0000","last_clean":"2024-04-16T19:37:33.649798+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:37:33.649798+0000","last_undegraded":"2024-04-16T19:37:33.649798+0000","last_fullsized":"2024-04-16T19:37:33.649798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.649657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.649657+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.649657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:40.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911568+0000","last_change":"2024-04-16T17:47:21.236822+0000","last_active":"2024-04-16T19:36:18.911568+0000","last_peered":"2024-04-16T19:36:18.911568+0000","last_clean":"2024-04-16T19:36:18.911568+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:36:18.911568+0000","last_undegraded":"2024-04-16T19:36:18.911568+0000","last_fullsized":"2024-04-16T19:36:18.911568+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.491099+0000","last_change":"2024-04-16T19:37:16.491152+0000","last_active":"2024-04-16T19:37:16.491099+0000","last_peered":"2024-04-16T19:37:16.491099+0000","last_clean":"2024-04-16T19:37:16.491099+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:37:16.491099+0000","last_undegraded":"2024-04-16T19:37:16.491099+0000","last_fullsized":"2024-04-16T19:37:16.491099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:16.490961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:16.490961+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.490961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:22.497268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'93","reported_seq":270,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.563674+0000","last_change":"2024-04-16T19:37:07.563764+0000","last_active":"2024-04-16T19:37:07.563674+0000","last_peered":"2024-04-16T19:37:07.563674+0000","last_clean":"2024-04-16T19:37:07.563674+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:37:07.563674+0000","last_undegraded":"2024-04-16T19:37:07.563674+0000","last_fullsized":"2024-04-16T19:37:07.563674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'93","last_scrub_stamp":"2024-04-16T19:37:07.563526+0000","last_deep_scrub":"53'93","last_deep_scrub_stamp":"2024-04-16T19:37:07.563526+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.563526+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:24.161906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.415324+0000","last_change":"2024-04-16T19:36:53.415380+0000","last_active":"2024-04-16T19:36:53.415324+0000","last_peered":"2024-04-16T19:36:53.415324+0000","last_clean":"2024-04-16T19:36:53.415324+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:36:53.415324+0000","last_undegraded":"2024-04-16T19:36:53.415324+0000","last_fullsized":"2024-04-16T19:36:53.415324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.415178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:50.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.527536+0000","last_change":"2024-04-16T19:36:33.527616+0000","last_active":"2024-04-16T19:36:33.527536+0000","last_peered":"2024-04-16T19:36:33.527536+0000","last_clean":"2024-04-16T19:36:33.527536+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:36:33.527536+0000","last_undegraded":"2024-04-16T19:36:33.527536+0000","last_fullsized":"2024-04-16T19:36:33.527536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.527307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:38.972728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.050423+0000","last_change":"2024-04-16T18:10:50.548290+0000","last_active":"2024-04-16T19:36:17.050423+0000","last_peered":"2024-04-16T19:36:17.050423+0000","last_clean":"2024-04-16T19:36:17.050423+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:36:17.050423+0000","last_undegraded":"2024-04-16T19:36:17.050423+0000","last_fullsized":"2024-04-16T19:36:17.050423+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'62","reported_seq":1005,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.553275+0000","last_change":"2024-04-16T19:37:05.553378+0000","last_active":"2024-04-16T19:37:05.553275+0000","last_peered":"2024-04-16T19:37:05.553275+0000","last_clean":"2024-04-16T19:37:05.553275+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:37:05.553275+0000","last_undegraded":"2024-04-16T19:37:05.553275+0000","last_fullsized":"2024-04-16T19:37:05.553275+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'62","last_scrub_stamp":"2024-04-16T19:37:05.553135+0000","last_deep_scrub":"53'62","last_deep_scrub_stamp":"2024-04-16T19:37:05.553135+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.553135+0000","objects_scrubbed":22,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:26.088642+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1541,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.684390+0000","last_change":"2024-04-16T19:36:40.684390+0000","last_active":"2024-04-16T19:36:40.684390+0000","last_peered":"2024-04-16T19:36:40.684390+0000","last_clean":"2024-04-16T19:36:40.684390+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:36:40.684390+0000","last_undegraded":"2024-04-16T19:36:40.684390+0000","last_fullsized":"2024-04-16T19:36:40.684390+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:34.096243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'11","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.426664+0000","last_change":"2024-04-16T19:36:52.426789+0000","last_active":"2024-04-16T19:36:52.426664+0000","last_peered":"2024-04-16T19:36:52.426664+0000","last_clean":"2024-04-16T19:36:52.426664+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:36:52.426664+0000","last_undegraded":"2024-04-16T19:36:52.426664+0000","last_fullsized":"2024-04-16T19:36:52.426664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.426496+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:18.642190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'556","reported_seq":2298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.370105+0000","last_change":"2024-04-16T19:36:40.370175+0000","last_active":"2024-04-16T19:36:40.370105+0000","last_peered":"2024-04-16T19:36:40.370105+0000","last_clean":"2024-04-16T19:36:40.370105+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:36:40.370105+0000","last_undegraded":"2024-04-16T19:36:40.370105+0000","last_fullsized":"2024-04-16T19:36:40.370105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.369960+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:17.626683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'925","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.519148+0000","last_change":"2024-04-16T19:36:29.519272+0000","last_active":"2024-04-16T19:36:29.519148+0000","last_peered":"2024-04-16T19:36:29.519148+0000","last_clean":"2024-04-16T19:36:29.519148+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:36:29.519148+0000","last_undegraded":"2024-04-16T19:36:29.519148+0000","last_fullsized":"2024-04-16T19:36:29.519148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.519049+0000","objects_scrubbed":24,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:29.400044+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143864,"num_omap_keys":628,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'19","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.480753+0000","last_change":"2024-04-16T19:36:30.480855+0000","last_active":"2024-04-16T19:36:30.480753+0000","last_peered":"2024-04-16T19:36:30.480753+0000","last_clean":"2024-04-16T19:36:30.480753+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:36:30.480753+0000","last_undegraded":"2024-04-16T19:36:30.480753+0000","last_fullsized":"2024-04-16T19:36:30.480753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.480620+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:17.845024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.573783+0000","last_change":"2024-04-16T19:37:17.573834+0000","last_active":"2024-04-16T19:37:17.573783+0000","last_peered":"2024-04-16T19:37:17.573783+0000","last_clean":"2024-04-16T19:37:17.573783+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:37:17.573783+0000","last_undegraded":"2024-04-16T19:37:17.573783+0000","last_fullsized":"2024-04-16T19:37:17.573783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:17.573640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:17.573640+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.573640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:27.868101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.637829+0000","last_change":"2024-04-16T19:37:30.637829+0000","last_active":"2024-04-16T19:37:30.637829+0000","last_peered":"2024-04-16T19:37:30.637829+0000","last_clean":"2024-04-16T19:37:30.637829+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:37:30.637829+0000","last_undegraded":"2024-04-16T19:37:30.637829+0000","last_fullsized":"2024-04-16T19:37:30.637829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.637636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.637636+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.637636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:15:57.900942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"53'4","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.387551+0000","last_change":"2024-04-16T19:37:26.387622+0000","last_active":"2024-04-16T19:37:26.387551+0000","last_peered":"2024-04-16T19:37:26.387551+0000","last_clean":"2024-04-16T19:37:26.387551+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:37:26.387551+0000","last_undegraded":"2024-04-16T19:37:26.387551+0000","last_fullsized":"2024-04-16T19:37:26.387551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:26.387392+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:26.387392+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.387392+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:52.755065+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'109","reported_seq":287,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.642949+0000","last_change":"2024-04-16T19:37:01.642980+0000","last_active":"2024-04-16T19:37:01.642949+0000","last_peered":"2024-04-16T19:37:01.642949+0000","last_clean":"2024-04-16T19:37:01.642949+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:37:01.642949+0000","last_undegraded":"2024-04-16T19:37:01.642949+0000","last_fullsized":"2024-04-16T19:37:01.642949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:37:01.642886+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:37:01.642886+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.642886+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:36:01.291066+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.529837+0000","last_change":"2024-04-16T19:37:03.529944+0000","last_active":"2024-04-16T19:37:03.529837+0000","last_peered":"2024-04-16T19:37:03.529837+0000","last_clean":"2024-04-16T19:37:03.529837+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:37:03.529837+0000","last_undegraded":"2024-04-16T19:37:03.529837+0000","last_fullsized":"2024-04-16T19:37:03.529837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.529668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.529668+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.529668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:01.869011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.457022+0000","last_change":"2024-04-16T19:36:30.457084+0000","last_active":"2024-04-16T19:36:30.457022+0000","last_peered":"2024-04-16T19:36:30.457022+0000","last_clean":"2024-04-16T19:36:30.457022+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:36:30.457022+0000","last_undegraded":"2024-04-16T19:36:30.457022+0000","last_fullsized":"2024-04-16T19:36:30.457022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.456877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:36.883658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588157+0000","last_change":"2024-04-16T18:10:50.529899+0000","last_active":"2024-04-16T19:36:16.588157+0000","last_peered":"2024-04-16T19:36:16.588157+0000","last_clean":"2024-04-16T19:36:16.588157+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:36:16.588157+0000","last_undegraded":"2024-04-16T19:36:16.588157+0000","last_fullsized":"2024-04-16T19:36:16.588157+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'106","reported_seq":916,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.482238+0000","last_change":"2024-04-16T19:37:19.482414+0000","last_active":"2024-04-16T19:37:19.482238+0000","last_peered":"2024-04-16T19:37:19.482238+0000","last_clean":"2024-04-16T19:37:19.482238+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:37:19.482238+0000","last_undegraded":"2024-04-16T19:37:19.482238+0000","last_fullsized":"2024-04-16T19:37:19.482238+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'106","last_scrub_stamp":"2024-04-16T19:37:19.482066+0000","last_deep_scrub":"53'106","last_deep_scrub_stamp":"2024-04-16T19:37:19.482066+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.482066+0000","objects_scrubbed":24,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:59.735406+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6869,"num_omap_keys":30,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.315787+0000","last_change":"2024-04-16T19:37:11.315847+0000","last_active":"2024-04-16T19:37:11.315787+0000","last_peered":"2024-04-16T19:37:11.315787+0000","last_clean":"2024-04-16T19:37:11.315787+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:37:11.315787+0000","last_undegraded":"2024-04-16T19:37:11.315787+0000","last_fullsized":"2024-04-16T19:37:11.315787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.315613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.315613+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.315613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:39.336603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'142","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.445837+0000","last_change":"2024-04-16T19:36:51.445837+0000","last_active":"2024-04-16T19:36:51.445837+0000","last_peered":"2024-04-16T19:36:51.445837+0000","last_clean":"2024-04-16T19:36:51.445837+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:36:51.445837+0000","last_undegraded":"2024-04-16T19:36:51.445837+0000","last_fullsized":"2024-04-16T19:36:51.445837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'142","last_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_deep_scrub":"53'142","last_deep_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.445664+0000","objects_scrubbed":7,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:24.999267+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9535,"num_omap_keys":55,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.514976+0000","last_change":"2024-04-16T19:36:24.515035+0000","last_active":"2024-04-16T19:36:24.514976+0000","last_peered":"2024-04-16T19:36:24.514976+0000","last_clean":"2024-04-16T19:36:24.514976+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:36:24.514976+0000","last_undegraded":"2024-04-16T19:36:24.514976+0000","last_fullsized":"2024-04-16T19:36:24.514976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.514834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:04:02.534977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'702","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.544776+0000","last_change":"2024-04-16T19:36:22.544916+0000","last_active":"2024-04-16T19:36:22.544776+0000","last_peered":"2024-04-16T19:36:22.544776+0000","last_clean":"2024-04-16T19:36:22.544776+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:36:22.544776+0000","last_undegraded":"2024-04-16T19:36:22.544776+0000","last_fullsized":"2024-04-16T19:36:22.544776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'702","last_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_deep_scrub":"53'702","last_deep_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.544630+0000","objects_scrubbed":23,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:27.704961+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71922,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.499314+0000","last_change":"2024-04-16T19:36:24.499314+0000","last_active":"2024-04-16T19:36:24.499314+0000","last_peered":"2024-04-16T19:36:24.499314+0000","last_clean":"2024-04-16T19:36:24.499314+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:36:24.499314+0000","last_undegraded":"2024-04-16T19:36:24.499314+0000","last_fullsized":"2024-04-16T19:36:24.499314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.498983+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:48.052702+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.758628+0000","last_change":"2024-04-16T19:37:26.758733+0000","last_active":"2024-04-16T19:37:26.758628+0000","last_peered":"2024-04-16T19:37:26.758628+0000","last_clean":"2024-04-16T19:37:26.758628+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:37:26.758628+0000","last_undegraded":"2024-04-16T19:37:26.758628+0000","last_fullsized":"2024-04-16T19:37:26.758628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.758459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.758459+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.758459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:26:28.700003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.763635+0000","last_change":"2024-04-16T19:37:27.763686+0000","last_active":"2024-04-16T19:37:27.763635+0000","last_peered":"2024-04-16T19:37:27.763635+0000","last_clean":"2024-04-16T19:37:27.763635+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:37:27.763635+0000","last_undegraded":"2024-04-16T19:37:27.763635+0000","last_fullsized":"2024-04-16T19:37:27.763635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.763498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.763498+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.763498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:44.477511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"53'14","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.644762+0000","last_change":"2024-04-16T19:37:04.644828+0000","last_active":"2024-04-16T19:37:04.644762+0000","last_peered":"2024-04-16T19:37:04.644762+0000","last_clean":"2024-04-16T19:37:04.644762+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:37:04.644762+0000","last_undegraded":"2024-04-16T19:37:04.644762+0000","last_fullsized":"2024-04-16T19:37:04.644762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:37:04.644628+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:37:04.644628+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.644628+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:54.707458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'159","reported_seq":361,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.425798+0000","last_change":"2024-04-16T19:37:01.425941+0000","last_active":"2024-04-16T19:37:01.425798+0000","last_peered":"2024-04-16T19:37:01.425798+0000","last_clean":"2024-04-16T19:37:01.425798+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:37:01.425798+0000","last_undegraded":"2024-04-16T19:37:01.425798+0000","last_fullsized":"2024-04-16T19:37:01.425798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:01.425632+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:01.425632+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.425632+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:04.281799+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'13","reported_seq":1346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.458428+0000","last_change":"2024-04-16T19:36:51.458492+0000","last_active":"2024-04-16T19:36:51.458428+0000","last_peered":"2024-04-16T19:36:51.458428+0000","last_clean":"2024-04-16T19:36:51.458428+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:36:51.458428+0000","last_undegraded":"2024-04-16T19:36:51.458428+0000","last_fullsized":"2024-04-16T19:36:51.458428+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.458284+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:46:06.590229+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.582250+0000","last_change":"2024-04-16T19:36:36.582358+0000","last_active":"2024-04-16T19:36:36.582250+0000","last_peered":"2024-04-16T19:36:36.582250+0000","last_clean":"2024-04-16T19:36:36.582250+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:36:36.582250+0000","last_undegraded":"2024-04-16T19:36:36.582250+0000","last_fullsized":"2024-04-16T19:36:36.582250+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.582059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:29.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":58,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980805+0000","last_change":"2024-04-16T18:10:50.541215+0000","last_active":"2024-04-16T19:36:17.980805+0000","last_peered":"2024-04-16T19:36:17.980805+0000","last_clean":"2024-04-16T19:36:17.980805+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:36:17.980805+0000","last_undegraded":"2024-04-16T19:36:17.980805+0000","last_fullsized":"2024-04-16T19:36:17.980805+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'118","reported_seq":1160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.521304+0000","last_change":"2024-04-16T19:37:00.521375+0000","last_active":"2024-04-16T19:37:00.521304+0000","last_peered":"2024-04-16T19:37:00.521304+0000","last_clean":"2024-04-16T19:37:00.521304+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:37:00.521304+0000","last_undegraded":"2024-04-16T19:37:00.521304+0000","last_fullsized":"2024-04-16T19:37:00.521304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'118","last_scrub_stamp":"2024-04-16T19:37:00.521172+0000","last_deep_scrub":"53'118","last_deep_scrub_stamp":"2024-04-16T19:37:00.521172+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.521172+0000","objects_scrubbed":32,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:10.815633+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7005,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.427150+0000","last_change":"2024-04-16T19:36:49.427150+0000","last_active":"2024-04-16T19:36:49.427150+0000","last_peered":"2024-04-16T19:36:49.427150+0000","last_clean":"2024-04-16T19:36:49.427150+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:36:49.427150+0000","last_undegraded":"2024-04-16T19:36:49.427150+0000","last_fullsized":"2024-04-16T19:36:49.427150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.426874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:15.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"53'2","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.645599+0000","last_change":"2024-04-16T19:36:42.645707+0000","last_active":"2024-04-16T19:36:42.645599+0000","last_peered":"2024-04-16T19:36:42.645599+0000","last_clean":"2024-04-16T19:36:42.645599+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:36:42.645599+0000","last_undegraded":"2024-04-16T19:36:42.645599+0000","last_fullsized":"2024-04-16T19:36:42.645599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.645438+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:22.681965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.565703+0000","last_change":"2024-04-16T19:36:24.565807+0000","last_active":"2024-04-16T19:36:24.565703+0000","last_peered":"2024-04-16T19:36:24.565703+0000","last_clean":"2024-04-16T19:36:24.565703+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:36:24.565703+0000","last_undegraded":"2024-04-16T19:36:24.565703+0000","last_fullsized":"2024-04-16T19:36:24.565703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.565541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:30.252893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1508","reported_seq":5513,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.585799+0000","last_change":"2024-04-16T19:36:27.585867+0000","last_active":"2024-04-16T19:36:27.585799+0000","last_peered":"2024-04-16T19:36:27.585799+0000","last_clean":"2024-04-16T19:36:27.585799+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:36:27.585799+0000","last_undegraded":"2024-04-16T19:36:27.585799+0000","last_fullsized":"2024-04-16T19:36:27.585799+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1508","last_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_deep_scrub":"53'1508","last_deep_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.585661+0000","objects_scrubbed":25,"log_size":1508,"log_dups_size":0,"ondisk_log_size":1508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:18.587606+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250399,"num_omap_keys":1045,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'14","reported_seq":240,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.424045+0000","last_change":"2024-04-16T19:36:27.424219+0000","last_active":"2024-04-16T19:36:27.424045+0000","last_peered":"2024-04-16T19:36:27.424045+0000","last_clean":"2024-04-16T19:36:27.424045+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:36:27.424045+0000","last_undegraded":"2024-04-16T19:36:27.424045+0000","last_fullsized":"2024-04-16T19:36:27.424045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.423818+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:57.529566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.650311+0000","last_change":"2024-04-16T19:37:06.650366+0000","last_active":"2024-04-16T19:37:06.650311+0000","last_peered":"2024-04-16T19:37:06.650311+0000","last_clean":"2024-04-16T19:37:06.650311+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:37:06.650311+0000","last_undegraded":"2024-04-16T19:37:06.650311+0000","last_fullsized":"2024-04-16T19:37:06.650311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:06.650180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:06.650180+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.650180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:32.865874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.474344+0000","last_change":"2024-04-16T19:37:24.474396+0000","last_active":"2024-04-16T19:37:24.474344+0000","last_peered":"2024-04-16T19:37:24.474344+0000","last_clean":"2024-04-16T19:37:24.474344+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:37:24.474344+0000","last_undegraded":"2024-04-16T19:37:24.474344+0000","last_fullsized":"2024-04-16T19:37:24.474344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.474210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.474210+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.474210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:46.579247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"53'4","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.499565+0000","last_change":"2024-04-16T19:37:18.499632+0000","last_active":"2024-04-16T19:37:18.499565+0000","last_peered":"2024-04-16T19:37:18.499565+0000","last_clean":"2024-04-16T19:37:18.499565+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:37:18.499565+0000","last_undegraded":"2024-04-16T19:37:18.499565+0000","last_fullsized":"2024-04-16T19:37:18.499565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:18.499434+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:18.499434+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.499434+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:31:26.309566+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'125","reported_seq":306,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.693242+0000","last_change":"2024-04-16T19:36:58.693314+0000","last_active":"2024-04-16T19:36:58.693242+0000","last_peered":"2024-04-16T19:36:58.693242+0000","last_clean":"2024-04-16T19:36:58.693242+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:36:58.693242+0000","last_undegraded":"2024-04-16T19:36:58.693242+0000","last_fullsized":"2024-04-16T19:36:58.693242+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:58.693112+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:58.693112+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.693112+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:03.753998+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'13","reported_seq":1346,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.636594+0000","last_change":"2024-04-16T19:36:55.636714+0000","last_active":"2024-04-16T19:36:55.636594+0000","last_peered":"2024-04-16T19:36:55.636594+0000","last_clean":"2024-04-16T19:36:55.636594+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:36:55.636594+0000","last_undegraded":"2024-04-16T19:36:55.636594+0000","last_fullsized":"2024-04-16T19:36:55.636594+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.636425+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:56:05.862157+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.567557+0000","last_change":"2024-04-16T19:36:51.567609+0000","last_active":"2024-04-16T19:36:51.567557+0000","last_peered":"2024-04-16T19:36:51.567557+0000","last_clean":"2024-04-16T19:36:51.567557+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:36:51.567557+0000","last_undegraded":"2024-04-16T19:36:51.567557+0000","last_fullsized":"2024-04-16T19:36:51.567557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.567419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:32.289248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.653902+0000","last_change":"2024-04-16T19:37:35.653955+0000","last_active":"2024-04-16T19:37:35.653902+0000","last_peered":"2024-04-16T19:37:35.653902+0000","last_clean":"2024-04-16T19:37:35.653902+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:37:35.653902+0000","last_undegraded":"2024-04-16T19:37:35.653902+0000","last_fullsized":"2024-04-16T19:37:35.653902+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.653764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.653764+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.653764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:19.654955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'117","reported_seq":1015,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.675266+0000","last_change":"2024-04-16T19:37:09.675266+0000","last_active":"2024-04-16T19:37:09.675266+0000","last_peered":"2024-04-16T19:37:09.675266+0000","last_clean":"2024-04-16T19:37:09.675266+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:37:09.675266+0000","last_undegraded":"2024-04-16T19:37:09.675266+0000","last_fullsized":"2024-04-16T19:37:09.675266+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'117","last_scrub_stamp":"2024-04-16T19:37:09.675046+0000","last_deep_scrub":"53'117","last_deep_scrub_stamp":"2024-04-16T19:37:09.675046+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.675046+0000","objects_scrubbed":27,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:14.071688+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4717,"num_omap_keys":20,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.389594+0000","last_change":"2024-04-16T19:36:46.389594+0000","last_active":"2024-04-16T19:36:46.389594+0000","last_peered":"2024-04-16T19:36:46.389594+0000","last_clean":"2024-04-16T19:36:46.389594+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:36:46.389594+0000","last_undegraded":"2024-04-16T19:36:46.389594+0000","last_fullsized":"2024-04-16T19:36:46.389594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.389402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:49.372119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'8","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.681828+0000","last_change":"2024-04-16T19:36:40.681959+0000","last_active":"2024-04-16T19:36:40.681828+0000","last_peered":"2024-04-16T19:36:40.681828+0000","last_clean":"2024-04-16T19:36:40.681828+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:36:40.681828+0000","last_undegraded":"2024-04-16T19:36:40.681828+0000","last_fullsized":"2024-04-16T19:36:40.681828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.681665+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:15.914709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'556","reported_seq":1952,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.630262+0000","last_change":"2024-04-16T19:36:46.630326+0000","last_active":"2024-04-16T19:36:46.630262+0000","last_peered":"2024-04-16T19:36:46.630262+0000","last_clean":"2024-04-16T19:36:46.630262+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:36:46.630262+0000","last_undegraded":"2024-04-16T19:36:46.630262+0000","last_fullsized":"2024-04-16T19:36:46.630262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.630122+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:54.931710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'676","reported_seq":995,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635509+0000","last_change":"2024-04-16T19:36:20.635509+0000","last_active":"2024-04-16T19:36:20.635509+0000","last_peered":"2024-04-16T19:36:20.635509+0000","last_clean":"2024-04-16T19:36:20.635509+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:36:20.635509+0000","last_undegraded":"2024-04-16T19:36:20.635509+0000","last_fullsized":"2024-04-16T19:36:20.635509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'676","last_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_deep_scrub":"53'676","last_deep_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635277+0000","objects_scrubbed":25,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:10:55.447676+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71899,"num_omap_keys":314,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'20","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.500999+0000","last_change":"2024-04-16T19:36:18.501046+0000","last_active":"2024-04-16T19:36:18.500999+0000","last_peered":"2024-04-16T19:36:18.500999+0000","last_clean":"2024-04-16T19:36:18.500999+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:36:18.500999+0000","last_undegraded":"2024-04-16T19:36:18.500999+0000","last_fullsized":"2024-04-16T19:36:18.500999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.500951+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:17.147393+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.634453+0000","last_change":"2024-04-16T19:37:10.634505+0000","last_active":"2024-04-16T19:37:10.634453+0000","last_peered":"2024-04-16T19:37:10.634453+0000","last_clean":"2024-04-16T19:37:10.634453+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:37:10.634453+0000","last_undegraded":"2024-04-16T19:37:10.634453+0000","last_fullsized":"2024-04-16T19:37:10.634453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:10.634314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:10.634314+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.634314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:37.001987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.453613+0000","last_change":"2024-04-16T19:37:22.453667+0000","last_active":"2024-04-16T19:37:22.453613+0000","last_peered":"2024-04-16T19:37:22.453613+0000","last_clean":"2024-04-16T19:37:22.453613+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:37:22.453613+0000","last_undegraded":"2024-04-16T19:37:22.453613+0000","last_fullsized":"2024-04-16T19:37:22.453613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.453471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.453471+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.453471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:21.519156+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"53'4","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.621956+0000","last_change":"2024-04-16T19:37:17.622021+0000","last_active":"2024-04-16T19:37:17.621956+0000","last_peered":"2024-04-16T19:37:17.621956+0000","last_clean":"2024-04-16T19:37:17.621956+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:37:17.621956+0000","last_undegraded":"2024-04-16T19:37:17.621956+0000","last_fullsized":"2024-04-16T19:37:17.621956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:17.621822+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:17.621822+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.621822+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:46.534752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'184","reported_seq":398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.363093+0000","last_change":"2024-04-16T19:37:05.363174+0000","last_active":"2024-04-16T19:37:05.363093+0000","last_peered":"2024-04-16T19:37:05.363093+0000","last_clean":"2024-04-16T19:37:05.363093+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:37:05.363093+0000","last_undegraded":"2024-04-16T19:37:05.363093+0000","last_fullsized":"2024-04-16T19:37:05.363093+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'184","last_scrub_stamp":"2024-04-16T19:37:05.362967+0000","last_deep_scrub":"53'184","last_deep_scrub_stamp":"2024-04-16T19:37:05.362967+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.362967+0000","objects_scrubbed":8,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:30.682003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.486047+0000","last_change":"2024-04-16T19:36:58.486097+0000","last_active":"2024-04-16T19:36:58.486047+0000","last_peered":"2024-04-16T19:36:58.486047+0000","last_clean":"2024-04-16T19:36:58.486047+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:36:58.486047+0000","last_undegraded":"2024-04-16T19:36:58.486047+0000","last_fullsized":"2024-04-16T19:36:58.486047+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.485902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.485902+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.485902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:24.853575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.569888+0000","last_change":"2024-04-16T19:36:34.569940+0000","last_active":"2024-04-16T19:36:34.569888+0000","last_peered":"2024-04-16T19:36:34.569888+0000","last_clean":"2024-04-16T19:36:34.569888+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:36:34.569888+0000","last_undegraded":"2024-04-16T19:36:34.569888+0000","last_fullsized":"2024-04-16T19:36:34.569888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.569751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:40.008977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.547899+0000","last_change":"2024-04-16T19:37:32.547957+0000","last_active":"2024-04-16T19:37:32.547899+0000","last_peered":"2024-04-16T19:37:32.547899+0000","last_clean":"2024-04-16T19:37:32.547899+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:37:32.547899+0000","last_undegraded":"2024-04-16T19:37:32.547899+0000","last_fullsized":"2024-04-16T19:37:32.547899+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.547754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.547754+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.547754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:30.887097+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'153","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.638858+0000","last_change":"2024-04-16T19:36:56.638858+0000","last_active":"2024-04-16T19:36:56.638858+0000","last_peered":"2024-04-16T19:36:56.638858+0000","last_clean":"2024-04-16T19:36:56.638858+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:36:56.638858+0000","last_undegraded":"2024-04-16T19:36:56.638858+0000","last_fullsized":"2024-04-16T19:36:56.638858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'153","last_scrub_stamp":"2024-04-16T19:36:56.638728+0000","last_deep_scrub":"53'153","last_deep_scrub_stamp":"2024-04-16T19:36:56.638728+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.638728+0000","objects_scrubbed":35,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:45.633510+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11617,"num_omap_keys":48,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.606337+0000","last_change":"2024-04-16T19:36:37.606393+0000","last_active":"2024-04-16T19:36:37.606337+0000","last_peered":"2024-04-16T19:36:37.606337+0000","last_clean":"2024-04-16T19:36:37.606337+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:36:37.606337+0000","last_undegraded":"2024-04-16T19:36:37.606337+0000","last_fullsized":"2024-04-16T19:36:37.606337+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.606203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:48.287865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'11","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.682939+0000","last_change":"2024-04-16T19:36:43.683003+0000","last_active":"2024-04-16T19:36:43.682939+0000","last_peered":"2024-04-16T19:36:43.682939+0000","last_clean":"2024-04-16T19:36:43.682939+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:36:43.682939+0000","last_undegraded":"2024-04-16T19:36:43.682939+0000","last_fullsized":"2024-04-16T19:36:43.682939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.682806+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:30.180345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.630682+0000","last_change":"2024-04-16T19:36:48.630682+0000","last_active":"2024-04-16T19:36:48.630682+0000","last_peered":"2024-04-16T19:36:48.630682+0000","last_clean":"2024-04-16T19:36:48.630682+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:36:48.630682+0000","last_undegraded":"2024-04-16T19:36:48.630682+0000","last_fullsized":"2024-04-16T19:36:48.630682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.630423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:08.839790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'804","reported_seq":43172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.600535+0000","last_change":"2024-04-16T19:36:25.600624+0000","last_active":"2024-04-16T19:36:25.600535+0000","last_peered":"2024-04-16T19:36:25.600535+0000","last_clean":"2024-04-16T19:36:25.600535+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:36:25.600535+0000","last_undegraded":"2024-04-16T19:36:25.600535+0000","last_fullsized":"2024-04-16T19:36:25.600535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'804","last_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_deep_scrub":"53'804","last_deep_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.600462+0000","objects_scrubbed":21,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:05.088020+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71960,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'12","reported_seq":221,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.489782+0000","last_change":"2024-04-16T19:36:49.489878+0000","last_active":"2024-04-16T19:36:49.489782+0000","last_peered":"2024-04-16T19:36:49.489782+0000","last_clean":"2024-04-16T19:36:49.489782+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:36:49.489782+0000","last_undegraded":"2024-04-16T19:36:49.489782+0000","last_fullsized":"2024-04-16T19:36:49.489782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.489640+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:54.928866+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.715185+0000","last_change":"2024-04-16T19:37:28.715238+0000","last_active":"2024-04-16T19:37:28.715185+0000","last_peered":"2024-04-16T19:37:28.715185+0000","last_clean":"2024-04-16T19:37:28.715185+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:37:28.715185+0000","last_undegraded":"2024-04-16T19:37:28.715185+0000","last_fullsized":"2024-04-16T19:37:28.715185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.715038+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:13.816322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.737864+0000","last_change":"2024-04-16T19:37:35.737965+0000","last_active":"2024-04-16T19:37:35.737864+0000","last_peered":"2024-04-16T19:37:35.737864+0000","last_clean":"2024-04-16T19:37:35.737864+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:37:35.737864+0000","last_undegraded":"2024-04-16T19:37:35.737864+0000","last_fullsized":"2024-04-16T19:37:35.737864+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.737665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.737665+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.737665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:14.578611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.627731+0000","last_change":"2024-04-16T19:37:22.627786+0000","last_active":"2024-04-16T19:37:22.627731+0000","last_peered":"2024-04-16T19:37:22.627731+0000","last_clean":"2024-04-16T19:37:22.627731+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:37:22.627731+0000","last_undegraded":"2024-04-16T19:37:22.627731+0000","last_fullsized":"2024-04-16T19:37:22.627731+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.627590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.627590+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.627590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:51.252771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.517076+0000","last_change":"2024-04-16T19:37:30.517182+0000","last_active":"2024-04-16T19:37:30.517076+0000","last_peered":"2024-04-16T19:37:30.517076+0000","last_clean":"2024-04-16T19:37:30.517076+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:37:30.517076+0000","last_undegraded":"2024-04-16T19:37:30.517076+0000","last_fullsized":"2024-04-16T19:37:30.517076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.516899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.516899+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.516899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:16.748285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'77","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.348075+0000","last_change":"2024-04-16T19:37:12.348141+0000","last_active":"2024-04-16T19:37:12.348075+0000","last_peered":"2024-04-16T19:37:12.348075+0000","last_clean":"2024-04-16T19:37:12.348075+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:37:12.348075+0000","last_undegraded":"2024-04-16T19:37:12.348075+0000","last_fullsized":"2024-04-16T19:37:12.348075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'77","last_scrub_stamp":"2024-04-16T19:37:12.348001+0000","last_deep_scrub":"53'77","last_deep_scrub_stamp":"2024-04-16T19:37:12.348001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.348001+0000","objects_scrubbed":5,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.056193+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.566977+0000","last_change":"2024-04-16T19:36:59.567032+0000","last_active":"2024-04-16T19:36:59.566977+0000","last_peered":"2024-04-16T19:36:59.566977+0000","last_clean":"2024-04-16T19:36:59.566977+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:36:59.566977+0000","last_undegraded":"2024-04-16T19:36:59.566977+0000","last_fullsized":"2024-04-16T19:36:59.566977+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.566839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.566839+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.566839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:45.847122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.477027+0000","last_change":"2024-04-16T19:36:40.477078+0000","last_active":"2024-04-16T19:36:40.477027+0000","last_peered":"2024-04-16T19:36:40.477027+0000","last_clean":"2024-04-16T19:36:40.477027+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:36:40.477027+0000","last_undegraded":"2024-04-16T19:36:40.477027+0000","last_fullsized":"2024-04-16T19:36:40.477027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.476892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:55.189016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.980695+0000","last_change":"2024-04-16T18:10:50.543327+0000","last_active":"2024-04-16T19:36:17.980695+0000","last_peered":"2024-04-16T19:36:17.980695+0000","last_clean":"2024-04-16T19:36:17.980695+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:36:17.980695+0000","last_undegraded":"2024-04-16T19:36:17.980695+0000","last_fullsized":"2024-04-16T19:36:17.980695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'140","reported_seq":1796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.660278+0000","last_change":"2024-04-16T19:37:00.660381+0000","last_active":"2024-04-16T19:37:00.660278+0000","last_peered":"2024-04-16T19:37:00.660278+0000","last_clean":"2024-04-16T19:37:00.660278+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:37:00.660278+0000","last_undegraded":"2024-04-16T19:37:00.660278+0000","last_fullsized":"2024-04-16T19:37:00.660278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'140","last_scrub_stamp":"2024-04-16T19:37:00.660142+0000","last_deep_scrub":"53'140","last_deep_scrub_stamp":"2024-04-16T19:37:00.660142+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.660142+0000","objects_scrubbed":40,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:07.318185+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9228,"num_omap_keys":36,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.670649+0000","last_change":"2024-04-16T19:36:37.670700+0000","last_active":"2024-04-16T19:36:37.670649+0000","last_peered":"2024-04-16T19:36:37.670649+0000","last_clean":"2024-04-16T19:36:37.670649+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:36:37.670649+0000","last_undegraded":"2024-04-16T19:36:37.670649+0000","last_fullsized":"2024-04-16T19:36:37.670649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.670504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:43.115925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'14","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.601233+0000","last_change":"2024-04-16T19:36:56.601312+0000","last_active":"2024-04-16T19:36:56.601233+0000","last_peered":"2024-04-16T19:36:56.601233+0000","last_clean":"2024-04-16T19:36:56.601233+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:36:56.601233+0000","last_undegraded":"2024-04-16T19:36:56.601233+0000","last_fullsized":"2024-04-16T19:36:56.601233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:56.601090+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:56.601090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.601090+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:12.535988+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.465640+0000","last_change":"2024-04-16T19:36:41.465695+0000","last_active":"2024-04-16T19:36:41.465640+0000","last_peered":"2024-04-16T19:36:41.465640+0000","last_clean":"2024-04-16T19:36:41.465640+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:36:41.465640+0000","last_undegraded":"2024-04-16T19:36:41.465640+0000","last_fullsized":"2024-04-16T19:36:41.465640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.465493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:42:13.407368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'925","reported_seq":3830,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.639219+0000","last_change":"2024-04-16T19:36:25.639219+0000","last_active":"2024-04-16T19:36:25.639219+0000","last_peered":"2024-04-16T19:36:25.639219+0000","last_clean":"2024-04-16T19:36:25.639219+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:36:25.639219+0000","last_undegraded":"2024-04-16T19:36:25.639219+0000","last_fullsized":"2024-04-16T19:36:25.639219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.639013+0000","objects_scrubbed":28,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:11.472224+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89384,"num_omap_keys":416,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'36","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.510632+0000","last_change":"2024-04-16T19:36:24.510731+0000","last_active":"2024-04-16T19:36:24.510632+0000","last_peered":"2024-04-16T19:36:24.510632+0000","last_clean":"2024-04-16T19:36:24.510632+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:36:24.510632+0000","last_undegraded":"2024-04-16T19:36:24.510632+0000","last_fullsized":"2024-04-16T19:36:24.510632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'36","last_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_deep_scrub":"53'36","last_deep_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.510495+0000","objects_scrubbed":7,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:21.114823+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.586912+0000","last_change":"2024-04-16T19:37:16.586965+0000","last_active":"2024-04-16T19:37:16.586912+0000","last_peered":"2024-04-16T19:37:16.586912+0000","last_clean":"2024-04-16T19:37:16.586912+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:37:16.586912+0000","last_undegraded":"2024-04-16T19:37:16.586912+0000","last_fullsized":"2024-04-16T19:37:16.586912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:16.586769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:16.586769+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.586769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:42:14.606566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911512+0000","last_change":"2024-04-16T17:46:37.574923+0000","last_active":"2024-04-16T19:36:18.911512+0000","last_peered":"2024-04-16T19:36:18.911512+0000","last_clean":"2024-04-16T19:36:18.911512+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:36:18.911512+0000","last_undegraded":"2024-04-16T19:36:18.911512+0000","last_fullsized":"2024-04-16T19:36:18.911512+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.485772+0000","last_change":"2024-04-16T19:37:26.485823+0000","last_active":"2024-04-16T19:37:26.485772+0000","last_peered":"2024-04-16T19:37:26.485772+0000","last_clean":"2024-04-16T19:37:26.485772+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:37:26.485772+0000","last_undegraded":"2024-04-16T19:37:26.485772+0000","last_fullsized":"2024-04-16T19:37:26.485772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.485632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.485632+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.485632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:20.193824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"53'2","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.638661+0000","last_change":"2024-04-16T19:37:16.638790+0000","last_active":"2024-04-16T19:37:16.638661+0000","last_peered":"2024-04-16T19:37:16.638661+0000","last_clean":"2024-04-16T19:37:16.638661+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:37:16.638661+0000","last_undegraded":"2024-04-16T19:37:16.638661+0000","last_fullsized":"2024-04-16T19:37:16.638661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:16.638491+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:16.638491+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.638491+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:45.551421+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'150","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.635861+0000","last_change":"2024-04-16T19:37:01.635936+0000","last_active":"2024-04-16T19:37:01.635861+0000","last_peered":"2024-04-16T19:37:01.635861+0000","last_clean":"2024-04-16T19:37:01.635861+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:37:01.635861+0000","last_undegraded":"2024-04-16T19:37:01.635861+0000","last_fullsized":"2024-04-16T19:37:01.635861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'150","last_scrub_stamp":"2024-04-16T19:37:01.635730+0000","last_deep_scrub":"53'150","last_deep_scrub_stamp":"2024-04-16T19:37:01.635730+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.635730+0000","objects_scrubbed":6,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:47:12.500214+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.356065+0000","last_change":"2024-04-16T19:36:43.356174+0000","last_active":"2024-04-16T19:36:43.356065+0000","last_peered":"2024-04-16T19:36:43.356065+0000","last_clean":"2024-04-16T19:36:43.356065+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:36:43.356065+0000","last_undegraded":"2024-04-16T19:36:43.356065+0000","last_fullsized":"2024-04-16T19:36:43.356065+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.355868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:33.637719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.430570+0000","last_change":"2024-04-16T19:36:44.430621+0000","last_active":"2024-04-16T19:36:44.430570+0000","last_peered":"2024-04-16T19:36:44.430570+0000","last_clean":"2024-04-16T19:36:44.430570+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:36:44.430570+0000","last_undegraded":"2024-04-16T19:36:44.430570+0000","last_fullsized":"2024-04-16T19:36:44.430570+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.430438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:03.942655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":55,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915237+0000","last_change":"2024-04-16T18:10:50.559365+0000","last_active":"2024-04-16T19:36:18.915237+0000","last_peered":"2024-04-16T19:36:18.915237+0000","last_clean":"2024-04-16T19:36:18.915237+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:36:18.915237+0000","last_undegraded":"2024-04-16T19:36:18.915237+0000","last_fullsized":"2024-04-16T19:36:18.915237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'119","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914927+0000","last_change":"2024-04-16T17:48:34.322060+0000","last_active":"2024-04-16T19:36:18.914927+0000","last_peered":"2024-04-16T19:36:18.914927+0000","last_clean":"2024-04-16T19:36:18.914927+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:36:18.914927+0000","last_undegraded":"2024-04-16T19:36:18.914927+0000","last_fullsized":"2024-04-16T19:36:18.914927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.359735+0000","last_change":"2024-04-16T19:37:07.359795+0000","last_active":"2024-04-16T19:37:07.359735+0000","last_peered":"2024-04-16T19:37:07.359735+0000","last_clean":"2024-04-16T19:37:07.359735+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:37:07.359735+0000","last_undegraded":"2024-04-16T19:37:07.359735+0000","last_fullsized":"2024-04-16T19:37:07.359735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.359591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.359591+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.359591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:05.698934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'42","reported_seq":213,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.456089+0000","last_change":"2024-04-16T19:36:54.456164+0000","last_active":"2024-04-16T19:36:54.456089+0000","last_peered":"2024-04-16T19:36:54.456089+0000","last_clean":"2024-04-16T19:36:54.456089+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:36:54.456089+0000","last_undegraded":"2024-04-16T19:36:54.456089+0000","last_fullsized":"2024-04-16T19:36:54.456089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'42","last_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_deep_scrub":"53'42","last_deep_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.455949+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:53.521634+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.526931+0000","last_change":"2024-04-16T19:36:34.526983+0000","last_active":"2024-04-16T19:36:34.526931+0000","last_peered":"2024-04-16T19:36:34.526931+0000","last_clean":"2024-04-16T19:36:34.526931+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:36:34.526931+0000","last_undegraded":"2024-04-16T19:36:34.526931+0000","last_fullsized":"2024-04-16T19:36:34.526931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.526798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:20.876919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'282","reported_seq":668,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.418934+0000","last_change":"2024-04-16T19:36:30.419051+0000","last_active":"2024-04-16T19:36:30.418934+0000","last_peered":"2024-04-16T19:36:30.418934+0000","last_clean":"2024-04-16T19:36:30.418934+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:36:30.418934+0000","last_undegraded":"2024-04-16T19:36:30.418934+0000","last_fullsized":"2024-04-16T19:36:30.418934+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'282","last_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_deep_scrub":"53'282","last_deep_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.418793+0000","objects_scrubbed":18,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:10.455320+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'16","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.403701+0000","last_change":"2024-04-16T19:36:29.403701+0000","last_active":"2024-04-16T19:36:29.403701+0000","last_peered":"2024-04-16T19:36:29.403701+0000","last_clean":"2024-04-16T19:36:29.403701+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:36:29.403701+0000","last_undegraded":"2024-04-16T19:36:29.403701+0000","last_fullsized":"2024-04-16T19:36:29.403701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.403484+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:20.878671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.632868+0000","last_change":"2024-04-16T19:37:13.632975+0000","last_active":"2024-04-16T19:37:13.632868+0000","last_peered":"2024-04-16T19:37:13.632868+0000","last_clean":"2024-04-16T19:37:13.632868+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:37:13.632868+0000","last_undegraded":"2024-04-16T19:37:13.632868+0000","last_fullsized":"2024-04-16T19:37:13.632868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.632699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.632699+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.632699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:35.211444+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.524869+0000","last_change":"2024-04-16T19:37:22.524969+0000","last_active":"2024-04-16T19:37:22.524869+0000","last_peered":"2024-04-16T19:37:22.524869+0000","last_clean":"2024-04-16T19:37:22.524869+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:37:22.524869+0000","last_undegraded":"2024-04-16T19:37:22.524869+0000","last_fullsized":"2024-04-16T19:37:22.524869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.524669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.524669+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.524669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:11.514839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.624357+0000","last_change":"2024-04-16T19:37:28.624408+0000","last_active":"2024-04-16T19:37:28.624357+0000","last_peered":"2024-04-16T19:37:28.624357+0000","last_clean":"2024-04-16T19:37:28.624357+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:37:28.624357+0000","last_undegraded":"2024-04-16T19:37:28.624357+0000","last_fullsized":"2024-04-16T19:37:28.624357+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.624222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.624222+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.624222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:57.537152+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"53'6","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.409314+0000","last_change":"2024-04-16T19:37:10.409384+0000","last_active":"2024-04-16T19:37:10.409314+0000","last_peered":"2024-04-16T19:37:10.409314+0000","last_clean":"2024-04-16T19:37:10.409314+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:37:10.409314+0000","last_undegraded":"2024-04-16T19:37:10.409314+0000","last_fullsized":"2024-04-16T19:37:10.409314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:10.409173+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:10.409173+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.409173+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:37.030711+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'159","reported_seq":1573,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.333313+0000","last_change":"2024-04-16T19:37:06.333399+0000","last_active":"2024-04-16T19:37:06.333313+0000","last_peered":"2024-04-16T19:37:06.333313+0000","last_clean":"2024-04-16T19:37:06.333313+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:37:06.333313+0000","last_undegraded":"2024-04-16T19:37:06.333313+0000","last_fullsized":"2024-04-16T19:37:06.333313+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:06.333171+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:06.333171+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.333171+0000","objects_scrubbed":8,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:29.002269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.544890+0000","last_change":"2024-04-16T19:37:02.544990+0000","last_active":"2024-04-16T19:37:02.544890+0000","last_peered":"2024-04-16T19:37:02.544890+0000","last_clean":"2024-04-16T19:37:02.544890+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:37:02.544890+0000","last_undegraded":"2024-04-16T19:37:02.544890+0000","last_fullsized":"2024-04-16T19:37:02.544890+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.544686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.544686+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.544686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:25.213783+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.593720+0000","last_change":"2024-04-16T19:36:40.593776+0000","last_active":"2024-04-16T19:36:40.593720+0000","last_peered":"2024-04-16T19:36:40.593720+0000","last_clean":"2024-04-16T19:36:40.593720+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:36:40.593720+0000","last_undegraded":"2024-04-16T19:36:40.593720+0000","last_fullsized":"2024-04-16T19:36:40.593720+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.593577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.105795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.501636+0000","last_change":"2024-04-16T19:37:34.501687+0000","last_active":"2024-04-16T19:37:34.501636+0000","last_peered":"2024-04-16T19:37:34.501636+0000","last_clean":"2024-04-16T19:37:34.501636+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:37:34.501636+0000","last_undegraded":"2024-04-16T19:37:34.501636+0000","last_fullsized":"2024-04-16T19:37:34.501636+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.501500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.501500+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.501500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:03.414430+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'106","reported_seq":1199,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.610166+0000","last_change":"2024-04-16T19:36:57.610238+0000","last_active":"2024-04-16T19:36:57.610166+0000","last_peered":"2024-04-16T19:36:57.610166+0000","last_clean":"2024-04-16T19:36:57.610166+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:36:57.610166+0000","last_undegraded":"2024-04-16T19:36:57.610166+0000","last_fullsized":"2024-04-16T19:36:57.610166+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'106","last_scrub_stamp":"2024-04-16T19:36:57.610032+0000","last_deep_scrub":"53'106","last_deep_scrub_stamp":"2024-04-16T19:36:57.610032+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.610032+0000","objects_scrubbed":30,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:04.921796+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6018,"num_omap_keys":24,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.396377+0000","last_change":"2024-04-16T19:36:50.396377+0000","last_active":"2024-04-16T19:36:50.396377+0000","last_peered":"2024-04-16T19:36:50.396377+0000","last_clean":"2024-04-16T19:36:50.396377+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:36:50.396377+0000","last_undegraded":"2024-04-16T19:36:50.396377+0000","last_fullsized":"2024-04-16T19:36:50.396377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.396177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:28.233570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.563153+0000","last_change":"2024-04-16T19:36:58.563232+0000","last_active":"2024-04-16T19:36:58.563153+0000","last_peered":"2024-04-16T19:36:58.563153+0000","last_clean":"2024-04-16T19:36:58.563153+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:36:58.563153+0000","last_undegraded":"2024-04-16T19:36:58.563153+0000","last_fullsized":"2024-04-16T19:36:58.563153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:58.563020+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:58.563020+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.563020+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:47:33.896197+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'556","reported_seq":1943,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.469613+0000","last_change":"2024-04-16T19:36:37.469677+0000","last_active":"2024-04-16T19:36:37.469613+0000","last_peered":"2024-04-16T19:36:37.469613+0000","last_clean":"2024-04-16T19:36:37.469613+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:36:37.469613+0000","last_undegraded":"2024-04-16T19:36:37.469613+0000","last_fullsized":"2024-04-16T19:36:37.469613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.469478+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:42.920753+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'327","reported_seq":654,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.537153+0000","last_change":"2024-04-16T19:36:23.537297+0000","last_active":"2024-04-16T19:36:23.537153+0000","last_peered":"2024-04-16T19:36:23.537153+0000","last_clean":"2024-04-16T19:36:23.537153+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:36:23.537153+0000","last_undegraded":"2024-04-16T19:36:23.537153+0000","last_fullsized":"2024-04-16T19:36:23.537153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'327","last_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_deep_scrub":"53'327","last_deep_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.537014+0000","objects_scrubbed":21,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:14.538959+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'7","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.487632+0000","last_change":"2024-04-16T19:36:18.487675+0000","last_active":"2024-04-16T19:36:18.487632+0000","last_peered":"2024-04-16T19:36:18.487632+0000","last_clean":"2024-04-16T19:36:18.487632+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:36:18.487632+0000","last_undegraded":"2024-04-16T19:36:18.487632+0000","last_fullsized":"2024-04-16T19:36:18.487632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.487557+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:46.066228+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.677548+0000","last_change":"2024-04-16T19:37:29.677602+0000","last_active":"2024-04-16T19:37:29.677548+0000","last_peered":"2024-04-16T19:37:29.677548+0000","last_clean":"2024-04-16T19:37:29.677548+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:37:29.677548+0000","last_undegraded":"2024-04-16T19:37:29.677548+0000","last_fullsized":"2024-04-16T19:37:29.677548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.677414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.677414+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.677414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:34.454899+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.592078+0000","last_change":"2024-04-16T19:37:25.592131+0000","last_active":"2024-04-16T19:37:25.592078+0000","last_peered":"2024-04-16T19:37:25.592078+0000","last_clean":"2024-04-16T19:37:25.592078+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:37:25.592078+0000","last_undegraded":"2024-04-16T19:37:25.592078+0000","last_fullsized":"2024-04-16T19:37:25.592078+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.591940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.591940+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.591940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:53.199527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"14'117","reported_seq":294,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977594+0000","last_change":"2024-04-16T17:46:38.507284+0000","last_active":"2024-04-16T19:36:17.977594+0000","last_peered":"2024-04-16T19:36:17.977594+0000","last_clean":"2024-04-16T19:36:17.977594+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:36:17.977594+0000","last_undegraded":"2024-04-16T19:36:17.977594+0000","last_fullsized":"2024-04-16T19:36:17.977594+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:37.490454+0000","last_clean_scrub_stamp":"2024-04-16T17:46:37.490454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205065+0000","last_change":"2024-04-16T17:47:21.241020+0000","last_active":"2024-04-16T19:35:41.205065+0000","last_peered":"2024-04-16T19:35:41.205065+0000","last_clean":"2024-04-16T19:35:41.205065+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:35:41.205065+0000","last_undegraded":"2024-04-16T19:35:41.205065+0000","last_fullsized":"2024-04-16T19:35:41.205065+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"53'7","reported_seq":514,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.425097+0000","last_change":"2024-04-16T19:37:16.425217+0000","last_active":"2024-04-16T19:37:16.425097+0000","last_peered":"2024-04-16T19:37:16.425097+0000","last_clean":"2024-04-16T19:37:16.425097+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:37:16.425097+0000","last_undegraded":"2024-04-16T19:37:16.425097+0000","last_fullsized":"2024-04-16T19:37:16.425097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:16.424923+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:16.424923+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.424923+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:44.179170+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'187","reported_seq":1631,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.658006+0000","last_change":"2024-04-16T19:37:02.658104+0000","last_active":"2024-04-16T19:37:02.658006+0000","last_peered":"2024-04-16T19:37:02.658006+0000","last_clean":"2024-04-16T19:37:02.658006+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:37:02.658006+0000","last_undegraded":"2024-04-16T19:37:02.658006+0000","last_fullsized":"2024-04-16T19:37:02.658006+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'187","last_scrub_stamp":"2024-04-16T19:37:02.657862+0000","last_deep_scrub":"53'187","last_deep_scrub_stamp":"2024-04-16T19:37:02.657862+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.657862+0000","objects_scrubbed":10,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:00.007944+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.591692+0000","last_change":"2024-04-16T19:36:54.591795+0000","last_active":"2024-04-16T19:36:54.591692+0000","last_peered":"2024-04-16T19:36:54.591692+0000","last_clean":"2024-04-16T19:36:54.591692+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:36:54.591692+0000","last_undegraded":"2024-04-16T19:36:54.591692+0000","last_fullsized":"2024-04-16T19:36:54.591692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.591530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:30.795070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.464452+0000","last_change":"2024-04-16T19:36:28.464452+0000","last_active":"2024-04-16T19:36:28.464452+0000","last_peered":"2024-04-16T19:36:28.464452+0000","last_clean":"2024-04-16T19:36:28.464452+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:36:28.464452+0000","last_undegraded":"2024-04-16T19:36:28.464452+0000","last_fullsized":"2024-04-16T19:36:28.464452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.464263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:07.132103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":64,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.452959+0000","last_change":"2024-04-16T19:37:28.453064+0000","last_active":"2024-04-16T19:37:28.452959+0000","last_peered":"2024-04-16T19:37:28.452959+0000","last_clean":"2024-04-16T19:37:28.452959+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:37:28.452959+0000","last_undegraded":"2024-04-16T19:37:28.452959+0000","last_fullsized":"2024-04-16T19:37:28.452959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.452788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.452788+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.452788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:38.747249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'64","reported_seq":1180,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588024+0000","last_change":"2024-04-16T17:48:34.344724+0000","last_active":"2024-04-16T19:36:16.588024+0000","last_peered":"2024-04-16T19:36:16.588024+0000","last_clean":"2024-04-16T19:36:16.588024+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:36:16.588024+0000","last_undegraded":"2024-04-16T19:36:16.588024+0000","last_fullsized":"2024-04-16T19:36:16.588024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.422295+0000","last_change":"2024-04-16T19:36:40.422295+0000","last_active":"2024-04-16T19:36:40.422295+0000","last_peered":"2024-04-16T19:36:40.422295+0000","last_clean":"2024-04-16T19:36:40.422295+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:36:40.422295+0000","last_undegraded":"2024-04-16T19:36:40.422295+0000","last_fullsized":"2024-04-16T19:36:40.422295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.422090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:53.341423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'19","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.529264+0000","last_change":"2024-04-16T19:36:30.529412+0000","last_active":"2024-04-16T19:36:30.529264+0000","last_peered":"2024-04-16T19:36:30.529264+0000","last_clean":"2024-04-16T19:36:30.529264+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:36:30.529264+0000","last_undegraded":"2024-04-16T19:36:30.529264+0000","last_fullsized":"2024-04-16T19:36:30.529264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.529097+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:25.629336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'556","reported_seq":1933,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.659990+0000","last_change":"2024-04-16T19:36:28.660054+0000","last_active":"2024-04-16T19:36:28.659990+0000","last_peered":"2024-04-16T19:36:28.659990+0000","last_clean":"2024-04-16T19:36:28.659990+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:36:28.659990+0000","last_undegraded":"2024-04-16T19:36:28.659990+0000","last_fullsized":"2024-04-16T19:36:28.659990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.659827+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:31.852118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'532","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.416491+0000","last_change":"2024-04-16T19:36:24.416491+0000","last_active":"2024-04-16T19:36:24.416491+0000","last_peered":"2024-04-16T19:36:24.416491+0000","last_clean":"2024-04-16T19:36:24.416491+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:36:24.416491+0000","last_undegraded":"2024-04-16T19:36:24.416491+0000","last_fullsized":"2024-04-16T19:36:24.416491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.416318+0000","objects_scrubbed":18,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:15.418263+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"53'6","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.488767+0000","last_change":"2024-04-16T19:36:23.488844+0000","last_active":"2024-04-16T19:36:23.488767+0000","last_peered":"2024-04-16T19:36:23.488767+0000","last_clean":"2024-04-16T19:36:23.488767+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:36:23.488767+0000","last_undegraded":"2024-04-16T19:36:23.488767+0000","last_fullsized":"2024-04-16T19:36:23.488767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.488637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:34:36.790606+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.610550+0000","last_change":"2024-04-16T19:37:10.610651+0000","last_active":"2024-04-16T19:37:10.610550+0000","last_peered":"2024-04-16T19:37:10.610550+0000","last_clean":"2024-04-16T19:37:10.610550+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:37:10.610550+0000","last_undegraded":"2024-04-16T19:37:10.610550+0000","last_fullsized":"2024-04-16T19:37:10.610550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:10.610381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:10.610381+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.610381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:44.952386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.387252+0000","last_change":"2024-04-16T19:37:03.387307+0000","last_active":"2024-04-16T19:37:03.387252+0000","last_peered":"2024-04-16T19:37:03.387252+0000","last_clean":"2024-04-16T19:37:03.387252+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:37:03.387252+0000","last_undegraded":"2024-04-16T19:37:03.387252+0000","last_fullsized":"2024-04-16T19:37:03.387252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.387109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.387109+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.387109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:37.698223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.615042+0000","last_change":"2024-04-16T19:37:26.615094+0000","last_active":"2024-04-16T19:37:26.615042+0000","last_peered":"2024-04-16T19:37:26.615042+0000","last_clean":"2024-04-16T19:37:26.615042+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:37:26.615042+0000","last_undegraded":"2024-04-16T19:37:26.615042+0000","last_fullsized":"2024-04-16T19:37:26.615042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.614905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.614905+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.614905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:31:34.425036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.417938+0000","last_change":"2024-04-16T19:37:06.418036+0000","last_active":"2024-04-16T19:37:06.417938+0000","last_peered":"2024-04-16T19:37:06.417938+0000","last_clean":"2024-04-16T19:37:06.417938+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:37:06.417938+0000","last_undegraded":"2024-04-16T19:37:06.417938+0000","last_fullsized":"2024-04-16T19:37:06.417938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:06.417783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:06.417783+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.417783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:31.811569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'152","reported_seq":364,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.627842+0000","last_change":"2024-04-16T19:37:03.627959+0000","last_active":"2024-04-16T19:37:03.627842+0000","last_peered":"2024-04-16T19:37:03.627842+0000","last_clean":"2024-04-16T19:37:03.627842+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:37:03.627842+0000","last_undegraded":"2024-04-16T19:37:03.627842+0000","last_fullsized":"2024-04-16T19:37:03.627842+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'152","last_scrub_stamp":"2024-04-16T19:37:03.627704+0000","last_deep_scrub":"53'152","last_deep_scrub_stamp":"2024-04-16T19:37:03.627704+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.627704+0000","objects_scrubbed":8,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:06.610421+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.468997+0000","last_change":"2024-04-16T19:36:50.469053+0000","last_active":"2024-04-16T19:36:50.468997+0000","last_peered":"2024-04-16T19:36:50.468997+0000","last_clean":"2024-04-16T19:36:50.468997+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:36:50.468997+0000","last_undegraded":"2024-04-16T19:36:50.468997+0000","last_fullsized":"2024-04-16T19:36:50.468997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.468857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:29.956415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.495260+0000","last_change":"2024-04-16T19:36:38.495312+0000","last_active":"2024-04-16T19:36:38.495260+0000","last_peered":"2024-04-16T19:36:38.495260+0000","last_clean":"2024-04-16T19:36:38.495260+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:36:38.495260+0000","last_undegraded":"2024-04-16T19:36:38.495260+0000","last_fullsized":"2024-04-16T19:36:38.495260+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.495125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:12.806239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.435201+0000","last_change":"2024-04-16T19:37:27.435310+0000","last_active":"2024-04-16T19:37:27.435201+0000","last_peered":"2024-04-16T19:37:27.435201+0000","last_clean":"2024-04-16T19:37:27.435201+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:37:27.435201+0000","last_undegraded":"2024-04-16T19:37:27.435201+0000","last_fullsized":"2024-04-16T19:37:27.435201+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.435035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.435035+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.435035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:48.430839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'152","reported_seq":1804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.565726+0000","last_change":"2024-04-16T19:37:12.565726+0000","last_active":"2024-04-16T19:37:12.565726+0000","last_peered":"2024-04-16T19:37:12.565726+0000","last_clean":"2024-04-16T19:37:12.565726+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:37:12.565726+0000","last_undegraded":"2024-04-16T19:37:12.565726+0000","last_fullsized":"2024-04-16T19:37:12.565726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'152","last_scrub_stamp":"2024-04-16T19:37:12.565502+0000","last_deep_scrub":"53'152","last_deep_scrub_stamp":"2024-04-16T19:37:12.565502+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.565502+0000","objects_scrubbed":30,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:48.502447+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7790,"num_omap_keys":33,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.400809+0000","last_change":"2024-04-16T19:36:49.400809+0000","last_active":"2024-04-16T19:36:49.400809+0000","last_peered":"2024-04-16T19:36:49.400809+0000","last_clean":"2024-04-16T19:36:49.400809+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:36:49.400809+0000","last_undegraded":"2024-04-16T19:36:49.400809+0000","last_fullsized":"2024-04-16T19:36:49.400809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.400614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:52.383331+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'20","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.530462+0000","last_change":"2024-04-16T19:36:31.530526+0000","last_active":"2024-04-16T19:36:31.530462+0000","last_peered":"2024-04-16T19:36:31.530462+0000","last_clean":"2024-04-16T19:36:31.530462+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:36:31.530462+0000","last_undegraded":"2024-04-16T19:36:31.530462+0000","last_fullsized":"2024-04-16T19:36:31.530462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.530356+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:33:36.382371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.461572+0000","last_change":"2024-04-16T19:36:57.461629+0000","last_active":"2024-04-16T19:36:57.461572+0000","last_peered":"2024-04-16T19:36:57.461572+0000","last_clean":"2024-04-16T19:36:57.461572+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:36:57.461572+0000","last_undegraded":"2024-04-16T19:36:57.461572+0000","last_fullsized":"2024-04-16T19:36:57.461572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:57.461431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:57.461431+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.461431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:04.646645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1182","reported_seq":1443,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.649769+0000","last_change":"2024-04-16T19:36:23.649928+0000","last_active":"2024-04-16T19:36:23.649769+0000","last_peered":"2024-04-16T19:36:23.649769+0000","last_clean":"2024-04-16T19:36:23.649769+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:36:23.649769+0000","last_undegraded":"2024-04-16T19:36:23.649769+0000","last_fullsized":"2024-04-16T19:36:23.649769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1182","last_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_deep_scrub":"53'1182","last_deep_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.649623+0000","objects_scrubbed":23,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:09.999744+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215927,"num_omap_keys":942,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'28","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.423045+0000","last_change":"2024-04-16T19:36:25.423158+0000","last_active":"2024-04-16T19:36:25.423045+0000","last_peered":"2024-04-16T19:36:25.423045+0000","last_clean":"2024-04-16T19:36:25.423045+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:36:25.423045+0000","last_undegraded":"2024-04-16T19:36:25.423045+0000","last_fullsized":"2024-04-16T19:36:25.423045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'28","last_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_deep_scrub":"53'28","last_deep_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.422911+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:22.027239+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.639003+0000","last_change":"2024-04-16T19:37:27.639056+0000","last_active":"2024-04-16T19:37:27.639003+0000","last_peered":"2024-04-16T19:37:27.639003+0000","last_clean":"2024-04-16T19:37:27.639003+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:37:27.639003+0000","last_undegraded":"2024-04-16T19:37:27.639003+0000","last_fullsized":"2024-04-16T19:37:27.639003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.638863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.638863+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.638863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:12.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.505863+0000","last_change":"2024-04-16T19:37:20.505891+0000","last_active":"2024-04-16T19:37:20.505863+0000","last_peered":"2024-04-16T19:37:20.505863+0000","last_clean":"2024-04-16T19:37:20.505863+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:37:20.505863+0000","last_undegraded":"2024-04-16T19:37:20.505863+0000","last_fullsized":"2024-04-16T19:37:20.505863+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.505785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.505785+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.505785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:59:10.568615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":151,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.493169+0000","last_change":"2024-04-16T19:37:23.493276+0000","last_active":"2024-04-16T19:37:23.493169+0000","last_peered":"2024-04-16T19:37:23.493169+0000","last_clean":"2024-04-16T19:37:23.493169+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:37:23.493169+0000","last_undegraded":"2024-04-16T19:37:23.493169+0000","last_fullsized":"2024-04-16T19:37:23.493169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.493001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.493001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.493001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:44.028508+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.543234+0000","last_change":"2024-04-16T19:37:14.543287+0000","last_active":"2024-04-16T19:37:14.543234+0000","last_peered":"2024-04-16T19:37:14.543234+0000","last_clean":"2024-04-16T19:37:14.543234+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:37:14.543234+0000","last_undegraded":"2024-04-16T19:37:14.543234+0000","last_fullsized":"2024-04-16T19:37:14.543234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.543097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.543097+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.543097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:55.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'125","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.486453+0000","last_change":"2024-04-16T19:37:10.486531+0000","last_active":"2024-04-16T19:37:10.486453+0000","last_peered":"2024-04-16T19:37:10.486453+0000","last_clean":"2024-04-16T19:37:10.486453+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:37:10.486453+0000","last_undegraded":"2024-04-16T19:37:10.486453+0000","last_fullsized":"2024-04-16T19:37:10.486453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:37:10.486312+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:37:10.486312+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.486312+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:14.882954+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.688114+0000","last_change":"2024-04-16T19:36:44.688178+0000","last_active":"2024-04-16T19:36:44.688114+0000","last_peered":"2024-04-16T19:36:44.688114+0000","last_clean":"2024-04-16T19:36:44.688114+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:36:44.688114+0000","last_undegraded":"2024-04-16T19:36:44.688114+0000","last_fullsized":"2024-04-16T19:36:44.688114+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.687973+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:44.814523+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.534123+0000","last_change":"2024-04-16T19:36:35.534176+0000","last_active":"2024-04-16T19:36:35.534123+0000","last_peered":"2024-04-16T19:36:35.534123+0000","last_clean":"2024-04-16T19:36:35.534123+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:36:35.534123+0000","last_undegraded":"2024-04-16T19:36:35.534123+0000","last_fullsized":"2024-04-16T19:36:35.534123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.533984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:55.743351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":65,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.406770+0000","last_change":"2024-04-16T19:37:00.406770+0000","last_active":"2024-04-16T19:37:00.406770+0000","last_peered":"2024-04-16T19:37:00.406770+0000","last_clean":"2024-04-16T19:37:00.406770+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:37:00.406770+0000","last_undegraded":"2024-04-16T19:37:00.406770+0000","last_fullsized":"2024-04-16T19:37:00.406770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.406670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.406670+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.406670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:05.819397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'91","reported_seq":1284,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915002+0000","last_change":"2024-04-16T17:48:34.321714+0000","last_active":"2024-04-16T19:36:18.915002+0000","last_peered":"2024-04-16T19:36:18.915002+0000","last_clean":"2024-04-16T19:36:18.915002+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:36:18.915002+0000","last_undegraded":"2024-04-16T19:36:18.915002+0000","last_fullsized":"2024-04-16T19:36:18.915002+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.504798+0000","last_change":"2024-04-16T19:36:44.504798+0000","last_active":"2024-04-16T19:36:44.504798+0000","last_peered":"2024-04-16T19:36:44.504798+0000","last_clean":"2024-04-16T19:36:44.504798+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:36:44.504798+0000","last_undegraded":"2024-04-16T19:36:44.504798+0000","last_fullsized":"2024-04-16T19:36:44.504798+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.504608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:08:17.020106+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'21","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.684658+0000","last_change":"2024-04-16T19:36:52.684736+0000","last_active":"2024-04-16T19:36:52.684658+0000","last_peered":"2024-04-16T19:36:52.684658+0000","last_clean":"2024-04-16T19:36:52.684658+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:36:52.684658+0000","last_undegraded":"2024-04-16T19:36:52.684658+0000","last_fullsized":"2024-04-16T19:36:52.684658+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.684531+0000","objects_scrubbed":8,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:32.649666+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.646374+0000","last_change":"2024-04-16T19:36:45.646430+0000","last_active":"2024-04-16T19:36:45.646374+0000","last_peered":"2024-04-16T19:36:45.646374+0000","last_clean":"2024-04-16T19:36:45.646374+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:36:45.646374+0000","last_undegraded":"2024-04-16T19:36:45.646374+0000","last_fullsized":"2024-04-16T19:36:45.646374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.646241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:48.502408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1343","reported_seq":1906,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.619403+0000","last_change":"2024-04-16T19:36:21.619494+0000","last_active":"2024-04-16T19:36:21.619403+0000","last_peered":"2024-04-16T19:36:21.619403+0000","last_clean":"2024-04-16T19:36:21.619403+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:36:21.619403+0000","last_undegraded":"2024-04-16T19:36:21.619403+0000","last_fullsized":"2024-04-16T19:36:21.619403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1343","last_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_deep_scrub":"53'1343","last_deep_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.619266+0000","objects_scrubbed":29,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:34.145789+0000","scrub_duration":71,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":216307,"num_omap_keys":944,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'8","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.444455+0000","last_change":"2024-04-16T19:36:33.444542+0000","last_active":"2024-04-16T19:36:33.444455+0000","last_peered":"2024-04-16T19:36:33.444455+0000","last_clean":"2024-04-16T19:36:33.444455+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:36:33.444455+0000","last_undegraded":"2024-04-16T19:36:33.444455+0000","last_fullsized":"2024-04-16T19:36:33.444455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.444316+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:36.300483+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.653908+0000","last_change":"2024-04-16T19:37:21.653965+0000","last_active":"2024-04-16T19:37:21.653908+0000","last_peered":"2024-04-16T19:37:21.653908+0000","last_clean":"2024-04-16T19:37:21.653908+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:37:21.653908+0000","last_undegraded":"2024-04-16T19:37:21.653908+0000","last_fullsized":"2024-04-16T19:37:21.653908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.653770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.653770+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.653770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:17.539808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.615108+0000","last_change":"2024-04-16T19:37:29.615161+0000","last_active":"2024-04-16T19:37:29.615108+0000","last_peered":"2024-04-16T19:37:29.615108+0000","last_clean":"2024-04-16T19:37:29.615108+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:37:29.615108+0000","last_undegraded":"2024-04-16T19:37:29.615108+0000","last_fullsized":"2024-04-16T19:37:29.615108+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.614969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.614969+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.614969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:15:06.350732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.698876+0000","last_change":"2024-04-16T19:37:34.698992+0000","last_active":"2024-04-16T19:37:34.698876+0000","last_peered":"2024-04-16T19:37:34.698876+0000","last_clean":"2024-04-16T19:37:34.698876+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:37:34.698876+0000","last_undegraded":"2024-04-16T19:37:34.698876+0000","last_fullsized":"2024-04-16T19:37:34.698876+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.698711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.698711+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.698711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:41.231249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.493810+0000","last_change":"2024-04-16T19:37:20.493865+0000","last_active":"2024-04-16T19:37:20.493810+0000","last_peered":"2024-04-16T19:37:20.493810+0000","last_clean":"2024-04-16T19:37:20.493810+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:37:20.493810+0000","last_undegraded":"2024-04-16T19:37:20.493810+0000","last_fullsized":"2024-04-16T19:37:20.493810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.493668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.493668+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.493668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:49.406598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'175","reported_seq":384,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.762300+0000","last_change":"2024-04-16T19:36:56.762377+0000","last_active":"2024-04-16T19:36:56.762300+0000","last_peered":"2024-04-16T19:36:56.762300+0000","last_clean":"2024-04-16T19:36:56.762300+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:36:56.762300+0000","last_undegraded":"2024-04-16T19:36:56.762300+0000","last_fullsized":"2024-04-16T19:36:56.762300+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'175","last_scrub_stamp":"2024-04-16T19:36:56.762164+0000","last_deep_scrub":"53'175","last_deep_scrub_stamp":"2024-04-16T19:36:56.762164+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.762164+0000","objects_scrubbed":7,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:59.105751+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'13","reported_seq":1352,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.360335+0000","last_change":"2024-04-16T19:37:02.360401+0000","last_active":"2024-04-16T19:37:02.360335+0000","last_peered":"2024-04-16T19:37:02.360335+0000","last_clean":"2024-04-16T19:37:02.360335+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:37:02.360335+0000","last_undegraded":"2024-04-16T19:37:02.360335+0000","last_fullsized":"2024-04-16T19:37:02.360335+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:02.360196+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:02.360196+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.360196+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:23.356000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.501363+0000","last_change":"2024-04-16T19:36:34.501465+0000","last_active":"2024-04-16T19:36:34.501363+0000","last_peered":"2024-04-16T19:36:34.501363+0000","last_clean":"2024-04-16T19:36:34.501363+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:36:34.501363+0000","last_undegraded":"2024-04-16T19:36:34.501363+0000","last_fullsized":"2024-04-16T19:36:34.501363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.501193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:08.812307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":57,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.915088+0000","last_change":"2024-04-16T18:10:50.548329+0000","last_active":"2024-04-16T19:36:18.915088+0000","last_peered":"2024-04-16T19:36:18.915088+0000","last_clean":"2024-04-16T19:36:18.915088+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:36:18.915088+0000","last_undegraded":"2024-04-16T19:36:18.915088+0000","last_fullsized":"2024-04-16T19:36:18.915088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'64","reported_seq":1188,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.587804+0000","last_change":"2024-04-16T17:48:34.344424+0000","last_active":"2024-04-16T19:36:16.587804+0000","last_peered":"2024-04-16T19:36:16.587804+0000","last_clean":"2024-04-16T19:36:16.587804+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:36:16.587804+0000","last_undegraded":"2024-04-16T19:36:16.587804+0000","last_fullsized":"2024-04-16T19:36:16.587804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.397896+0000","last_change":"2024-04-16T19:36:47.398006+0000","last_active":"2024-04-16T19:36:47.397896+0000","last_peered":"2024-04-16T19:36:47.397896+0000","last_clean":"2024-04-16T19:36:47.397896+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:36:47.397896+0000","last_undegraded":"2024-04-16T19:36:47.397896+0000","last_fullsized":"2024-04-16T19:36:47.397896+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.397724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:25.235116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.547894+0000","last_change":"2024-04-16T19:36:32.547894+0000","last_active":"2024-04-16T19:36:32.547894+0000","last_peered":"2024-04-16T19:36:32.547894+0000","last_clean":"2024-04-16T19:36:32.547894+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:36:32.547894+0000","last_undegraded":"2024-04-16T19:36:32.547894+0000","last_fullsized":"2024-04-16T19:36:32.547894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.547672+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:45.074195+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'556","reported_seq":2004,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.565740+0000","last_change":"2024-04-16T19:36:33.565806+0000","last_active":"2024-04-16T19:36:33.565740+0000","last_peered":"2024-04-16T19:36:33.565740+0000","last_clean":"2024-04-16T19:36:33.565740+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:36:33.565740+0000","last_undegraded":"2024-04-16T19:36:33.565740+0000","last_fullsized":"2024-04-16T19:36:33.565740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.565603+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:32.212045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'575","reported_seq":853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.628047+0000","last_change":"2024-04-16T19:36:22.628176+0000","last_active":"2024-04-16T19:36:22.628047+0000","last_peered":"2024-04-16T19:36:22.628047+0000","last_clean":"2024-04-16T19:36:22.628047+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:36:22.628047+0000","last_undegraded":"2024-04-16T19:36:22.628047+0000","last_fullsized":"2024-04-16T19:36:22.628047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'575","last_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_deep_scrub":"53'575","last_deep_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.627889+0000","objects_scrubbed":22,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:21.274331+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71488,"num_omap_keys":312,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'18","reported_seq":261,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.483626+0000","last_change":"2024-04-16T19:36:29.483703+0000","last_active":"2024-04-16T19:36:29.483626+0000","last_peered":"2024-04-16T19:36:29.483626+0000","last_clean":"2024-04-16T19:36:29.483626+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:36:29.483626+0000","last_undegraded":"2024-04-16T19:36:29.483626+0000","last_fullsized":"2024-04-16T19:36:29.483626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.483500+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:36:53.084475+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.621869+0000","last_change":"2024-04-16T19:37:15.621921+0000","last_active":"2024-04-16T19:37:15.621869+0000","last_peered":"2024-04-16T19:37:15.621869+0000","last_clean":"2024-04-16T19:37:15.621869+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:37:15.621869+0000","last_undegraded":"2024-04-16T19:37:15.621869+0000","last_fullsized":"2024-04-16T19:37:15.621869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.621731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.621731+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.621731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:58:34.123975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211258+0000","last_change":"2024-04-16T17:47:21.229342+0000","last_active":"2024-04-16T19:36:06.211258+0000","last_peered":"2024-04-16T19:36:06.211258+0000","last_clean":"2024-04-16T19:36:06.211258+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:36:06.211258+0000","last_undegraded":"2024-04-16T19:36:06.211258+0000","last_fullsized":"2024-04-16T19:36:06.211258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911538+0000","last_change":"2024-04-16T17:46:38.112666+0000","last_active":"2024-04-16T19:36:18.911538+0000","last_peered":"2024-04-16T19:36:18.911538+0000","last_clean":"2024-04-16T19:36:18.911538+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:36:18.911538+0000","last_undegraded":"2024-04-16T19:36:18.911538+0000","last_fullsized":"2024-04-16T19:36:18.911538+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:46:36.485975+0000","last_clean_scrub_stamp":"2024-04-16T17:46:36.485975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.594734+0000","last_change":"2024-04-16T19:37:18.594788+0000","last_active":"2024-04-16T19:37:18.594734+0000","last_peered":"2024-04-16T19:37:18.594734+0000","last_clean":"2024-04-16T19:37:18.594734+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:37:18.594734+0000","last_undegraded":"2024-04-16T19:37:18.594734+0000","last_fullsized":"2024-04-16T19:37:18.594734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.594597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.594597+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.594597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:43.732534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'168","reported_seq":385,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.727125+0000","last_change":"2024-04-16T19:37:12.727211+0000","last_active":"2024-04-16T19:37:12.727125+0000","last_peered":"2024-04-16T19:37:12.727125+0000","last_clean":"2024-04-16T19:37:12.727125+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:37:12.727125+0000","last_undegraded":"2024-04-16T19:37:12.727125+0000","last_fullsized":"2024-04-16T19:37:12.727125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'168","last_scrub_stamp":"2024-04-16T19:37:12.726993+0000","last_deep_scrub":"53'168","last_deep_scrub_stamp":"2024-04-16T19:37:12.726993+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.726993+0000","objects_scrubbed":8,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:58.958378+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.606377+0000","last_change":"2024-04-16T19:36:58.606428+0000","last_active":"2024-04-16T19:36:58.606377+0000","last_peered":"2024-04-16T19:36:58.606377+0000","last_clean":"2024-04-16T19:36:58.606377+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:36:58.606377+0000","last_undegraded":"2024-04-16T19:36:58.606377+0000","last_fullsized":"2024-04-16T19:36:58.606377+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.606242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.606242+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.606242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:23.041599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.422338+0000","last_change":"2024-04-16T19:36:32.422395+0000","last_active":"2024-04-16T19:36:32.422338+0000","last_peered":"2024-04-16T19:36:32.422338+0000","last_clean":"2024-04-16T19:36:32.422338+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:36:32.422338+0000","last_undegraded":"2024-04-16T19:36:32.422338+0000","last_fullsized":"2024-04-16T19:36:32.422338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.422189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:35.278356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":75,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201369+0000","last_change":"2024-04-16T18:10:50.536155+0000","last_active":"2024-04-16T19:35:26.201369+0000","last_peered":"2024-04-16T19:35:26.201369+0000","last_clean":"2024-04-16T19:35:26.201369+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:35:26.201369+0000","last_undegraded":"2024-04-16T19:35:26.201369+0000","last_fullsized":"2024-04-16T19:35:26.201369+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'88","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:31.206683+0000","last_change":"2024-04-16T17:49:32.408137+0000","last_active":"2024-04-16T19:35:31.206683+0000","last_peered":"2024-04-16T19:35:31.206683+0000","last_clean":"2024-04-16T19:35:31.206683+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:35:31.206683+0000","last_undegraded":"2024-04-16T19:35:31.206683+0000","last_fullsized":"2024-04-16T19:35:31.206683+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.635196+0000","last_change":"2024-04-16T19:36:39.635263+0000","last_active":"2024-04-16T19:36:39.635196+0000","last_peered":"2024-04-16T19:36:39.635196+0000","last_clean":"2024-04-16T19:36:39.635196+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:36:39.635196+0000","last_undegraded":"2024-04-16T19:36:39.635196+0000","last_fullsized":"2024-04-16T19:36:39.635196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.635056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:26.132595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'20","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.709187+0000","last_change":"2024-04-16T19:36:42.709267+0000","last_active":"2024-04-16T19:36:42.709187+0000","last_peered":"2024-04-16T19:36:42.709187+0000","last_clean":"2024-04-16T19:36:42.709187+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:36:42.709187+0000","last_undegraded":"2024-04-16T19:36:42.709187+0000","last_fullsized":"2024-04-16T19:36:42.709187+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.709051+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:57.264715+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.501219+0000","last_change":"2024-04-16T19:36:35.501282+0000","last_active":"2024-04-16T19:36:35.501219+0000","last_peered":"2024-04-16T19:36:35.501219+0000","last_clean":"2024-04-16T19:36:35.501219+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:36:35.501219+0000","last_undegraded":"2024-04-16T19:36:35.501219+0000","last_fullsized":"2024-04-16T19:36:35.501219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.501075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:40.913802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'871","reported_seq":1181,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.697221+0000","last_change":"2024-04-16T19:36:18.697375+0000","last_active":"2024-04-16T19:36:18.697221+0000","last_peered":"2024-04-16T19:36:18.697221+0000","last_clean":"2024-04-16T19:36:18.697221+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:36:18.697221+0000","last_undegraded":"2024-04-16T19:36:18.697221+0000","last_fullsized":"2024-04-16T19:36:18.697221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'871","last_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_deep_scrub":"53'871","last_deep_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.697076+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:49.607583+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143413,"num_omap_keys":626,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'22","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.656560+0000","last_change":"2024-04-16T19:37:04.656657+0000","last_active":"2024-04-16T19:37:04.656560+0000","last_peered":"2024-04-16T19:37:04.656560+0000","last_clean":"2024-04-16T19:37:04.656560+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:37:04.656560+0000","last_undegraded":"2024-04-16T19:37:04.656560+0000","last_fullsized":"2024-04-16T19:37:04.656560+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:37:04.656410+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:37:04.656410+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.656410+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:42.493803+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.570337+0000","last_change":"2024-04-16T19:36:36.570388+0000","last_active":"2024-04-16T19:36:36.570337+0000","last_peered":"2024-04-16T19:36:36.570337+0000","last_clean":"2024-04-16T19:36:36.570337+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:36:36.570337+0000","last_undegraded":"2024-04-16T19:36:36.570337+0000","last_fullsized":"2024-04-16T19:36:36.570337+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.570198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:51.282322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.603234+0000","last_change":"2024-04-16T19:37:11.603285+0000","last_active":"2024-04-16T19:37:11.603234+0000","last_peered":"2024-04-16T19:37:11.603234+0000","last_clean":"2024-04-16T19:37:11.603234+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:37:11.603234+0000","last_undegraded":"2024-04-16T19:37:11.603234+0000","last_fullsized":"2024-04-16T19:37:11.603234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.603093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.603093+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.603093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:10.668778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.725630+0000","last_change":"2024-04-16T19:37:25.725630+0000","last_active":"2024-04-16T19:37:25.725630+0000","last_peered":"2024-04-16T19:37:25.725630+0000","last_clean":"2024-04-16T19:37:25.725630+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:37:25.725630+0000","last_undegraded":"2024-04-16T19:37:25.725630+0000","last_fullsized":"2024-04-16T19:37:25.725630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.725436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.725436+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.725436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:02.147676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"53'14","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.665922+0000","last_change":"2024-04-16T19:37:05.665922+0000","last_active":"2024-04-16T19:37:05.665922+0000","last_peered":"2024-04-16T19:37:05.665922+0000","last_clean":"2024-04-16T19:37:05.665922+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:37:05.665922+0000","last_undegraded":"2024-04-16T19:37:05.665922+0000","last_fullsized":"2024-04-16T19:37:05.665922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:37:05.665717+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:37:05.665717+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.665717+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:45.630852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'109","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.370839+0000","last_change":"2024-04-16T19:36:59.370839+0000","last_active":"2024-04-16T19:36:59.370839+0000","last_peered":"2024-04-16T19:36:59.370839+0000","last_clean":"2024-04-16T19:36:59.370839+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:36:59.370839+0000","last_undegraded":"2024-04-16T19:36:59.370839+0000","last_fullsized":"2024-04-16T19:36:59.370839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:59.370626+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:59.370626+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.370626+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:46.735030+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.627271+0000","last_change":"2024-04-16T19:36:50.627271+0000","last_active":"2024-04-16T19:36:50.627271+0000","last_peered":"2024-04-16T19:36:50.627271+0000","last_clean":"2024-04-16T19:36:50.627271+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:36:50.627271+0000","last_undegraded":"2024-04-16T19:36:50.627271+0000","last_fullsized":"2024-04-16T19:36:50.627271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.627076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:16.020863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.527983+0000","last_change":"2024-04-16T19:37:35.528091+0000","last_active":"2024-04-16T19:37:35.527983+0000","last_peered":"2024-04-16T19:37:35.527983+0000","last_clean":"2024-04-16T19:37:35.527983+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:37:35.527983+0000","last_undegraded":"2024-04-16T19:37:35.527983+0000","last_fullsized":"2024-04-16T19:37:35.527983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.527793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.527793+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.527793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:00.663423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'82","reported_seq":709,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.725800+0000","last_change":"2024-04-16T19:37:03.725800+0000","last_active":"2024-04-16T19:37:03.725800+0000","last_peered":"2024-04-16T19:37:03.725800+0000","last_clean":"2024-04-16T19:37:03.725800+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:37:03.725800+0000","last_undegraded":"2024-04-16T19:37:03.725800+0000","last_fullsized":"2024-04-16T19:37:03.725800+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'82","last_scrub_stamp":"2024-04-16T19:37:03.725602+0000","last_deep_scrub":"53'82","last_deep_scrub_stamp":"2024-04-16T19:37:03.725602+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.725602+0000","objects_scrubbed":27,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:37.279204+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3517,"num_omap_keys":14,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.640311+0000","last_change":"2024-04-16T19:36:36.640365+0000","last_active":"2024-04-16T19:36:36.640311+0000","last_peered":"2024-04-16T19:36:36.640311+0000","last_clean":"2024-04-16T19:36:36.640311+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:36:36.640311+0000","last_undegraded":"2024-04-16T19:36:36.640311+0000","last_fullsized":"2024-04-16T19:36:36.640311+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.640174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:51.451059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'15","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.647852+0000","last_change":"2024-04-16T19:36:39.647955+0000","last_active":"2024-04-16T19:36:39.647852+0000","last_peered":"2024-04-16T19:36:39.647852+0000","last_clean":"2024-04-16T19:36:39.647852+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:36:39.647852+0000","last_undegraded":"2024-04-16T19:36:39.647852+0000","last_fullsized":"2024-04-16T19:36:39.647852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.647719+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:50.538220+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.430796+0000","last_change":"2024-04-16T19:36:38.430850+0000","last_active":"2024-04-16T19:36:38.430796+0000","last_peered":"2024-04-16T19:36:38.430796+0000","last_clean":"2024-04-16T19:36:38.430796+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:36:38.430796+0000","last_undegraded":"2024-04-16T19:36:38.430796+0000","last_fullsized":"2024-04-16T19:36:38.430796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.430655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:03.824441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'616","reported_seq":2312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.598175+0000","last_change":"2024-04-16T19:36:23.598300+0000","last_active":"2024-04-16T19:36:23.598175+0000","last_peered":"2024-04-16T19:36:23.598175+0000","last_clean":"2024-04-16T19:36:23.598175+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:36:23.598175+0000","last_undegraded":"2024-04-16T19:36:23.598175+0000","last_fullsized":"2024-04-16T19:36:23.598175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'616","last_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_deep_scrub":"53'616","last_deep_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.598031+0000","objects_scrubbed":24,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:37.526844+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.510530+0000","last_change":"2024-04-16T19:36:23.510613+0000","last_active":"2024-04-16T19:36:23.510530+0000","last_peered":"2024-04-16T19:36:23.510530+0000","last_clean":"2024-04-16T19:36:23.510530+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:36:23.510530+0000","last_undegraded":"2024-04-16T19:36:23.510530+0000","last_fullsized":"2024-04-16T19:36:23.510530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.510390+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:03.621835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.537581+0000","last_change":"2024-04-16T19:36:32.537632+0000","last_active":"2024-04-16T19:36:32.537581+0000","last_peered":"2024-04-16T19:36:32.537581+0000","last_clean":"2024-04-16T19:36:32.537581+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:36:32.537581+0000","last_undegraded":"2024-04-16T19:36:32.537581+0000","last_fullsized":"2024-04-16T19:36:32.537581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.537440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:47.348326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.588585+0000","last_change":"2024-04-16T19:37:12.588636+0000","last_active":"2024-04-16T19:37:12.588585+0000","last_peered":"2024-04-16T19:37:12.588585+0000","last_clean":"2024-04-16T19:37:12.588585+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:37:12.588585+0000","last_undegraded":"2024-04-16T19:37:12.588585+0000","last_fullsized":"2024-04-16T19:37:12.588585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:12.588451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:12.588451+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.588451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:58.868734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.215650+0000","last_change":"2024-04-16T17:47:21.241651+0000","last_active":"2024-04-16T19:36:06.215650+0000","last_peered":"2024-04-16T19:36:06.215650+0000","last_clean":"2024-04-16T19:36:06.215650+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:36:06.215650+0000","last_undegraded":"2024-04-16T19:36:06.215650+0000","last_fullsized":"2024-04-16T19:36:06.215650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"53'2","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.615140+0000","last_change":"2024-04-16T19:37:08.615140+0000","last_active":"2024-04-16T19:37:08.615140+0000","last_peered":"2024-04-16T19:37:08.615140+0000","last_clean":"2024-04-16T19:37:08.615140+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:37:08.615140+0000","last_undegraded":"2024-04-16T19:37:08.615140+0000","last_fullsized":"2024-04-16T19:37:08.615140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:08.614950+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:08.614950+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.614950+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:13.456557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'53","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.682005+0000","last_change":"2024-04-16T19:37:11.682005+0000","last_active":"2024-04-16T19:37:11.682005+0000","last_peered":"2024-04-16T19:37:11.682005+0000","last_clean":"2024-04-16T19:37:11.682005+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:37:11.682005+0000","last_undegraded":"2024-04-16T19:37:11.682005+0000","last_fullsized":"2024-04-16T19:37:11.682005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'53","last_scrub_stamp":"2024-04-16T19:37:11.681806+0000","last_deep_scrub":"53'53","last_deep_scrub_stamp":"2024-04-16T19:37:11.681806+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.681806+0000","objects_scrubbed":3,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:47.618751+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.518374+0000","last_change":"2024-04-16T19:37:02.518374+0000","last_active":"2024-04-16T19:37:02.518374+0000","last_peered":"2024-04-16T19:37:02.518374+0000","last_clean":"2024-04-16T19:37:02.518374+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:37:02.518374+0000","last_undegraded":"2024-04-16T19:37:02.518374+0000","last_fullsized":"2024-04-16T19:37:02.518374+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.518192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.518192+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.518192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:34.961401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.207221+0000","last_change":"2024-04-16T18:10:50.558181+0000","last_active":"2024-04-16T19:35:51.207221+0000","last_peered":"2024-04-16T19:35:51.207221+0000","last_clean":"2024-04-16T19:35:51.207221+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:35:51.207221+0000","last_undegraded":"2024-04-16T19:35:51.207221+0000","last_fullsized":"2024-04-16T19:35:51.207221+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'129","reported_seq":1727,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.778020+0000","last_change":"2024-04-16T19:36:55.778092+0000","last_active":"2024-04-16T19:36:55.778020+0000","last_peered":"2024-04-16T19:36:55.778020+0000","last_clean":"2024-04-16T19:36:55.778020+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:36:55.778020+0000","last_undegraded":"2024-04-16T19:36:55.778020+0000","last_fullsized":"2024-04-16T19:36:55.778020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'129","last_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_deep_scrub":"53'129","last_deep_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.777891+0000","objects_scrubbed":30,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:46.059742+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8321,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.410485+0000","last_change":"2024-04-16T19:36:51.410485+0000","last_active":"2024-04-16T19:36:51.410485+0000","last_peered":"2024-04-16T19:36:51.410485+0000","last_clean":"2024-04-16T19:36:51.410485+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:36:51.410485+0000","last_undegraded":"2024-04-16T19:36:51.410485+0000","last_fullsized":"2024-04-16T19:36:51.410485+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.410273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:57.416580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'18","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.684234+0000","last_change":"2024-04-16T19:36:45.684234+0000","last_active":"2024-04-16T19:36:45.684234+0000","last_peered":"2024-04-16T19:36:45.684234+0000","last_clean":"2024-04-16T19:36:45.684234+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:36:45.684234+0000","last_undegraded":"2024-04-16T19:36:45.684234+0000","last_fullsized":"2024-04-16T19:36:45.684234+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.684019+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:54.649522+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'21","reported_seq":1432,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.402575+0000","last_change":"2024-04-16T19:36:43.402646+0000","last_active":"2024-04-16T19:36:43.402575+0000","last_peered":"2024-04-16T19:36:43.402575+0000","last_clean":"2024-04-16T19:36:43.402575+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:36:43.402575+0000","last_undegraded":"2024-04-16T19:36:43.402575+0000","last_fullsized":"2024-04-16T19:36:43.402575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.402444+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:20.659167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'7003","reported_seq":10587,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.479202+0000","last_change":"2024-04-16T19:36:36.479319+0000","last_active":"2024-04-16T19:36:36.479202+0000","last_peered":"2024-04-16T19:36:36.479202+0000","last_clean":"2024-04-16T19:36:36.479202+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:36:36.479202+0000","last_undegraded":"2024-04-16T19:36:36.479202+0000","last_fullsized":"2024-04-16T19:36:36.479202+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7003","last_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_deep_scrub":"53'7003","last_deep_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.479068+0000","objects_scrubbed":20,"log_size":1803,"log_dups_size":3000,"ondisk_log_size":1803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:56.688434+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'10","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.437802+0000","last_change":"2024-04-16T19:36:27.437901+0000","last_active":"2024-04-16T19:36:27.437802+0000","last_peered":"2024-04-16T19:36:27.437802+0000","last_clean":"2024-04-16T19:36:27.437802+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:36:27.437802+0000","last_undegraded":"2024-04-16T19:36:27.437802+0000","last_fullsized":"2024-04-16T19:36:27.437802+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.437656+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:14.616289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.669172+0000","last_change":"2024-04-16T19:36:42.669224+0000","last_active":"2024-04-16T19:36:42.669172+0000","last_peered":"2024-04-16T19:36:42.669172+0000","last_clean":"2024-04-16T19:36:42.669172+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:36:42.669172+0000","last_undegraded":"2024-04-16T19:36:42.669172+0000","last_fullsized":"2024-04-16T19:36:42.669172+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.669008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:32.950859+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.392128+0000","last_change":"2024-04-16T19:37:05.392128+0000","last_active":"2024-04-16T19:37:05.392128+0000","last_peered":"2024-04-16T19:37:05.392128+0000","last_clean":"2024-04-16T19:37:05.392128+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:37:05.392128+0000","last_undegraded":"2024-04-16T19:37:05.392128+0000","last_fullsized":"2024-04-16T19:37:05.392128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:05.391932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:05.391932+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.391932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:20.104056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.480574+0000","last_change":"2024-04-16T19:37:34.480625+0000","last_active":"2024-04-16T19:37:34.480574+0000","last_peered":"2024-04-16T19:37:34.480574+0000","last_clean":"2024-04-16T19:37:34.480574+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:37:34.480574+0000","last_undegraded":"2024-04-16T19:37:34.480574+0000","last_fullsized":"2024-04-16T19:37:34.480574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.480436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.480436+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.480436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:39.055910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"53'4","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.536820+0000","last_change":"2024-04-16T19:37:18.536882+0000","last_active":"2024-04-16T19:37:18.536820+0000","last_peered":"2024-04-16T19:37:18.536820+0000","last_clean":"2024-04-16T19:37:18.536820+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:37:18.536820+0000","last_undegraded":"2024-04-16T19:37:18.536820+0000","last_fullsized":"2024-04-16T19:37:18.536820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:18.536691+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:18.536691+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.536691+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:40.115436+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'109","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.376859+0000","last_change":"2024-04-16T19:36:58.376859+0000","last_active":"2024-04-16T19:36:58.376859+0000","last_peered":"2024-04-16T19:36:58.376859+0000","last_clean":"2024-04-16T19:36:58.376859+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:36:58.376859+0000","last_undegraded":"2024-04-16T19:36:58.376859+0000","last_fullsized":"2024-04-16T19:36:58.376859+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:58.376556+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:58.376556+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.376556+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:38.413082+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.395303+0000","last_change":"2024-04-16T19:37:04.395303+0000","last_active":"2024-04-16T19:37:04.395303+0000","last_peered":"2024-04-16T19:37:04.395303+0000","last_clean":"2024-04-16T19:37:04.395303+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:37:04.395303+0000","last_undegraded":"2024-04-16T19:37:04.395303+0000","last_fullsized":"2024-04-16T19:37:04.395303+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.395110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.395110+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.395110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:24:17.565659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":51,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:51.211846+0000","last_change":"2024-04-16T18:10:50.545548+0000","last_active":"2024-04-16T19:35:51.211846+0000","last_peered":"2024-04-16T19:35:51.211846+0000","last_clean":"2024-04-16T19:35:51.211846+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:35:51.211846+0000","last_undegraded":"2024-04-16T19:35:51.211846+0000","last_fullsized":"2024-04-16T19:35:51.211846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'100","reported_seq":1360,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.914935+0000","last_change":"2024-04-16T17:49:32.417456+0000","last_active":"2024-04-16T19:36:18.914935+0000","last_peered":"2024-04-16T19:36:18.914935+0000","last_clean":"2024-04-16T19:36:18.914935+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:36:18.914935+0000","last_undegraded":"2024-04-16T19:36:18.914935+0000","last_fullsized":"2024-04-16T19:36:18.914935+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:48:33.281090+0000","last_clean_scrub_stamp":"2024-04-16T17:48:33.281090+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.499740+0000","last_change":"2024-04-16T19:36:36.499794+0000","last_active":"2024-04-16T19:36:36.499740+0000","last_peered":"2024-04-16T19:36:36.499740+0000","last_clean":"2024-04-16T19:36:36.499740+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:36:36.499740+0000","last_undegraded":"2024-04-16T19:36:36.499740+0000","last_fullsized":"2024-04-16T19:36:36.499740+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.499596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:22.332807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"53'16","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.619264+0000","last_change":"2024-04-16T19:36:44.619264+0000","last_active":"2024-04-16T19:36:44.619264+0000","last_peered":"2024-04-16T19:36:44.619264+0000","last_clean":"2024-04-16T19:36:44.619264+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:36:44.619264+0000","last_undegraded":"2024-04-16T19:36:44.619264+0000","last_fullsized":"2024-04-16T19:36:44.619264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.619046+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:50.031773+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'556","reported_seq":1967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.516649+0000","last_change":"2024-04-16T19:36:35.516719+0000","last_active":"2024-04-16T19:36:35.516649+0000","last_peered":"2024-04-16T19:36:35.516649+0000","last_clean":"2024-04-16T19:36:35.516649+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:36:35.516649+0000","last_undegraded":"2024-04-16T19:36:35.516649+0000","last_fullsized":"2024-04-16T19:36:35.516649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.516509+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:41.203861+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'563","reported_seq":804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.482688+0000","last_change":"2024-04-16T19:36:29.482688+0000","last_active":"2024-04-16T19:36:29.482688+0000","last_peered":"2024-04-16T19:36:29.482688+0000","last_clean":"2024-04-16T19:36:29.482688+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:36:29.482688+0000","last_undegraded":"2024-04-16T19:36:29.482688+0000","last_fullsized":"2024-04-16T19:36:29.482688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.482404+0000","objects_scrubbed":19,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.518930+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71952,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'8","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.461125+0000","last_change":"2024-04-16T19:36:21.461301+0000","last_active":"2024-04-16T19:36:21.461125+0000","last_peered":"2024-04-16T19:36:21.461125+0000","last_clean":"2024-04-16T19:36:21.461125+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:36:21.461125+0000","last_undegraded":"2024-04-16T19:36:21.461125+0000","last_fullsized":"2024-04-16T19:36:21.461125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.460913+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:00.948470+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.460735+0000","last_change":"2024-04-16T19:36:45.460790+0000","last_active":"2024-04-16T19:36:45.460735+0000","last_peered":"2024-04-16T19:36:45.460735+0000","last_clean":"2024-04-16T19:36:45.460735+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:36:45.460735+0000","last_undegraded":"2024-04-16T19:36:45.460735+0000","last_fullsized":"2024-04-16T19:36:45.460735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.460592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:12.082129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.596749+0000","last_change":"2024-04-16T19:37:11.596852+0000","last_active":"2024-04-16T19:37:11.596749+0000","last_peered":"2024-04-16T19:37:11.596749+0000","last_clean":"2024-04-16T19:37:11.596749+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:37:11.596749+0000","last_undegraded":"2024-04-16T19:37:11.596749+0000","last_fullsized":"2024-04-16T19:37:11.596749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.596575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.596575+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.596575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:32.592380+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.211668+0000","last_change":"2024-04-16T17:47:21.236618+0000","last_active":"2024-04-16T19:36:06.211668+0000","last_peered":"2024-04-16T19:36:06.211668+0000","last_clean":"2024-04-16T19:36:06.211668+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:36:06.211668+0000","last_undegraded":"2024-04-16T19:36:06.211668+0000","last_fullsized":"2024-04-16T19:36:06.211668+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.501299+0000","last_change":"2024-04-16T19:37:11.501299+0000","last_active":"2024-04-16T19:37:11.501299+0000","last_peered":"2024-04-16T19:37:11.501299+0000","last_clean":"2024-04-16T19:37:11.501299+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:37:11.501299+0000","last_undegraded":"2024-04-16T19:37:11.501299+0000","last_fullsized":"2024-04-16T19:37:11.501299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.501103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.501103+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.501103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:40.126284+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'84","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.376165+0000","last_change":"2024-04-16T19:37:09.376165+0000","last_active":"2024-04-16T19:37:09.376165+0000","last_peered":"2024-04-16T19:37:09.376165+0000","last_clean":"2024-04-16T19:37:09.376165+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:37:09.376165+0000","last_undegraded":"2024-04-16T19:37:09.376165+0000","last_fullsized":"2024-04-16T19:37:09.376165+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:09.375852+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:09.375852+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.375852+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:29.911359+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.541762+0000","last_change":"2024-04-16T19:37:00.541762+0000","last_active":"2024-04-16T19:37:00.541762+0000","last_peered":"2024-04-16T19:37:00.541762+0000","last_clean":"2024-04-16T19:37:00.541762+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:37:00.541762+0000","last_undegraded":"2024-04-16T19:37:00.541762+0000","last_fullsized":"2024-04-16T19:37:00.541762+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.541572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.541572+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.541572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:22.120317+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":53,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:21.204458+0000","last_change":"2024-04-16T18:10:50.542401+0000","last_active":"2024-04-16T19:35:21.204458+0000","last_peered":"2024-04-16T19:35:21.204458+0000","last_clean":"2024-04-16T19:35:21.204458+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:35:21.204458+0000","last_undegraded":"2024-04-16T19:35:21.204458+0000","last_fullsized":"2024-04-16T19:35:21.204458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'116","reported_seq":732,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.561294+0000","last_change":"2024-04-16T19:37:04.561294+0000","last_active":"2024-04-16T19:37:04.561294+0000","last_peered":"2024-04-16T19:37:04.561294+0000","last_clean":"2024-04-16T19:37:04.561294+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:37:04.561294+0000","last_undegraded":"2024-04-16T19:37:04.561294+0000","last_fullsized":"2024-04-16T19:37:04.561294+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'116","last_scrub_stamp":"2024-04-16T19:37:04.561086+0000","last_deep_scrub":"53'116","last_deep_scrub_stamp":"2024-04-16T19:37:04.561086+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.561086+0000","objects_scrubbed":27,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:02.900429+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5638,"num_omap_keys":24,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.678659+0000","last_change":"2024-04-16T19:36:47.678659+0000","last_active":"2024-04-16T19:36:47.678659+0000","last_peered":"2024-04-16T19:36:47.678659+0000","last_clean":"2024-04-16T19:36:47.678659+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:36:47.678659+0000","last_undegraded":"2024-04-16T19:36:47.678659+0000","last_fullsized":"2024-04-16T19:36:47.678659+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.678408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:02:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.735233+0000","last_change":"2024-04-16T19:36:41.735233+0000","last_active":"2024-04-16T19:36:41.735233+0000","last_peered":"2024-04-16T19:36:41.735233+0000","last_clean":"2024-04-16T19:36:41.735233+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:36:41.735233+0000","last_undegraded":"2024-04-16T19:36:41.735233+0000","last_fullsized":"2024-04-16T19:36:41.735233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.735066+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:52.416728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.470863+0000","last_change":"2024-04-16T19:36:33.470929+0000","last_active":"2024-04-16T19:36:33.470863+0000","last_peered":"2024-04-16T19:36:33.470863+0000","last_clean":"2024-04-16T19:36:33.470863+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:36:33.470863+0000","last_undegraded":"2024-04-16T19:36:33.470863+0000","last_fullsized":"2024-04-16T19:36:33.470863+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.470725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:02.342594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'476","reported_seq":4438,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.660175+0000","last_change":"2024-04-16T19:36:19.660343+0000","last_active":"2024-04-16T19:36:19.660175+0000","last_peered":"2024-04-16T19:36:19.660175+0000","last_clean":"2024-04-16T19:36:19.660175+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:36:19.660175+0000","last_undegraded":"2024-04-16T19:36:19.660175+0000","last_fullsized":"2024-04-16T19:36:19.660175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.659960+0000","objects_scrubbed":19,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:52.175458+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'8","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.516257+0000","last_change":"2024-04-16T19:36:36.516257+0000","last_active":"2024-04-16T19:36:36.516257+0000","last_peered":"2024-04-16T19:36:36.516257+0000","last_clean":"2024-04-16T19:36:36.516257+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:36:36.516257+0000","last_undegraded":"2024-04-16T19:36:36.516257+0000","last_fullsized":"2024-04-16T19:36:36.516257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.516049+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:56.739641+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.683430+0000","last_change":"2024-04-16T19:37:24.683485+0000","last_active":"2024-04-16T19:37:24.683430+0000","last_peered":"2024-04-16T19:37:24.683430+0000","last_clean":"2024-04-16T19:37:24.683430+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:37:24.683430+0000","last_undegraded":"2024-04-16T19:37:24.683430+0000","last_fullsized":"2024-04-16T19:37:24.683430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.683294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.683294+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.683294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:08.684485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.472813+0000","last_change":"2024-04-16T19:37:24.472868+0000","last_active":"2024-04-16T19:37:24.472813+0000","last_peered":"2024-04-16T19:37:24.472813+0000","last_clean":"2024-04-16T19:37:24.472813+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:37:24.472813+0000","last_undegraded":"2024-04-16T19:37:24.472813+0000","last_fullsized":"2024-04-16T19:37:24.472813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.472644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.472644+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.472644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:40.759382+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.212786+0000","last_change":"2024-04-16T17:47:33.298781+0000","last_active":"2024-04-16T19:36:11.212786+0000","last_peered":"2024-04-16T19:36:11.212786+0000","last_clean":"2024-04-16T19:36:11.212786+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:36:11.212786+0000","last_undegraded":"2024-04-16T19:36:11.212786+0000","last_fullsized":"2024-04-16T19:36:11.212786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:32.253794+0000","last_clean_scrub_stamp":"2024-04-16T17:47:32.253794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'93","reported_seq":264,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.659310+0000","last_change":"2024-04-16T19:37:00.659310+0000","last_active":"2024-04-16T19:37:00.659310+0000","last_peered":"2024-04-16T19:37:00.659310+0000","last_clean":"2024-04-16T19:37:00.659310+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:37:00.659310+0000","last_undegraded":"2024-04-16T19:37:00.659310+0000","last_fullsized":"2024-04-16T19:37:00.659310+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'93","last_scrub_stamp":"2024-04-16T19:37:00.659199+0000","last_deep_scrub":"53'93","last_deep_scrub_stamp":"2024-04-16T19:37:00.659199+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.659199+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:06.665506+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.473332+0000","last_change":"2024-04-16T19:37:01.473385+0000","last_active":"2024-04-16T19:37:01.473332+0000","last_peered":"2024-04-16T19:37:01.473332+0000","last_clean":"2024-04-16T19:37:01.473332+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:37:01.473332+0000","last_undegraded":"2024-04-16T19:37:01.473332+0000","last_fullsized":"2024-04-16T19:37:01.473332+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:01.473191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:01.473191+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.473191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:23.051936+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.511222+0000","last_change":"2024-04-16T19:36:25.511222+0000","last_active":"2024-04-16T19:36:25.511222+0000","last_peered":"2024-04-16T19:36:25.511222+0000","last_clean":"2024-04-16T19:36:25.511222+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:36:25.511222+0000","last_undegraded":"2024-04-16T19:36:25.511222+0000","last_fullsized":"2024-04-16T19:36:25.511222+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.511031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:41:57.452906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.200556+0000","last_change":"2024-04-16T18:10:50.562024+0000","last_active":"2024-04-16T19:35:26.200556+0000","last_peered":"2024-04-16T19:35:26.200556+0000","last_clean":"2024-04-16T19:35:26.200556+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:35:26.200556+0000","last_undegraded":"2024-04-16T19:35:26.200556+0000","last_fullsized":"2024-04-16T19:35:26.200556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'102","reported_seq":1008,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.618748+0000","last_change":"2024-04-16T19:36:48.618887+0000","last_active":"2024-04-16T19:36:48.618748+0000","last_peered":"2024-04-16T19:36:48.618748+0000","last_clean":"2024-04-16T19:36:48.618748+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:36:48.618748+0000","last_undegraded":"2024-04-16T19:36:48.618748+0000","last_fullsized":"2024-04-16T19:36:48.618748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'102","last_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_deep_scrub":"53'102","last_deep_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.618582+0000","objects_scrubbed":31,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:54.031308+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7116,"num_omap_keys":28,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.371768+0000","last_change":"2024-04-16T19:37:15.371768+0000","last_active":"2024-04-16T19:37:15.371768+0000","last_peered":"2024-04-16T19:37:15.371768+0000","last_clean":"2024-04-16T19:37:15.371768+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:37:15.371768+0000","last_undegraded":"2024-04-16T19:37:15.371768+0000","last_fullsized":"2024-04-16T19:37:15.371768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.371583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.371583+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.371583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:51.308528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'7","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.481640+0000","last_change":"2024-04-16T19:36:54.481710+0000","last_active":"2024-04-16T19:36:54.481640+0000","last_peered":"2024-04-16T19:36:54.481640+0000","last_clean":"2024-04-16T19:36:54.481640+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:36:54.481640+0000","last_undegraded":"2024-04-16T19:36:54.481640+0000","last_fullsized":"2024-04-16T19:36:54.481640+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.481476+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:28.035078+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.550467+0000","last_change":"2024-04-16T19:36:28.550520+0000","last_active":"2024-04-16T19:36:28.550467+0000","last_peered":"2024-04-16T19:36:28.550467+0000","last_clean":"2024-04-16T19:36:28.550467+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:36:28.550467+0000","last_undegraded":"2024-04-16T19:36:28.550467+0000","last_fullsized":"2024-04-16T19:36:28.550467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.550328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:52.280854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11393","reported_seq":61823,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.677407+0000","last_change":"2024-04-16T19:36:19.677565+0000","last_active":"2024-04-16T19:36:19.677407+0000","last_peered":"2024-04-16T19:36:19.677407+0000","last_clean":"2024-04-16T19:36:19.677407+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:36:19.677407+0000","last_undegraded":"2024-04-16T19:36:19.677407+0000","last_fullsized":"2024-04-16T19:36:19.677407+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11393","last_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_deep_scrub":"53'11393","last_deep_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.677264+0000","objects_scrubbed":23,"log_size":1693,"log_dups_size":3000,"ondisk_log_size":1693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:14.777503+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209607,"num_omap_keys":977,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'23","reported_seq":359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.461701+0000","last_change":"2024-04-16T19:36:18.461701+0000","last_active":"2024-04-16T19:36:18.461701+0000","last_peered":"2024-04-16T19:36:18.461701+0000","last_clean":"2024-04-16T19:36:18.461701+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:36:18.461701+0000","last_undegraded":"2024-04-16T19:36:18.461701+0000","last_fullsized":"2024-04-16T19:36:18.461701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.461393+0000","objects_scrubbed":11,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:24.888175+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.617521+0000","last_change":"2024-04-16T19:37:25.617573+0000","last_active":"2024-04-16T19:37:25.617521+0000","last_peered":"2024-04-16T19:37:25.617521+0000","last_clean":"2024-04-16T19:37:25.617521+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:37:25.617521+0000","last_undegraded":"2024-04-16T19:37:25.617521+0000","last_fullsized":"2024-04-16T19:37:25.617521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.617386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.617386+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.617386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:26:27.558930+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.633414+0000","last_change":"2024-04-16T19:37:24.633506+0000","last_active":"2024-04-16T19:37:24.633414+0000","last_peered":"2024-04-16T19:37:24.633414+0000","last_clean":"2024-04-16T19:37:24.633414+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:37:24.633414+0000","last_undegraded":"2024-04-16T19:37:24.633414+0000","last_fullsized":"2024-04-16T19:37:24.633414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.633250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.633250+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.633250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:05.960575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.570694+0000","last_change":"2024-04-16T19:37:15.570748+0000","last_active":"2024-04-16T19:37:15.570694+0000","last_peered":"2024-04-16T19:37:15.570694+0000","last_clean":"2024-04-16T19:37:15.570694+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:37:15.570694+0000","last_undegraded":"2024-04-16T19:37:15.570694+0000","last_fullsized":"2024-04-16T19:37:15.570694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.570547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.570547+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.570547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:35.464724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'159","reported_seq":357,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.579240+0000","last_change":"2024-04-16T19:37:07.579240+0000","last_active":"2024-04-16T19:37:07.579240+0000","last_peered":"2024-04-16T19:37:07.579240+0000","last_clean":"2024-04-16T19:37:07.579240+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:37:07.579240+0000","last_undegraded":"2024-04-16T19:37:07.579240+0000","last_fullsized":"2024-04-16T19:37:07.579240+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:07.579029+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:07.579029+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.579029+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:29.684066+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.656638+0000","last_change":"2024-04-16T19:36:53.656690+0000","last_active":"2024-04-16T19:36:53.656638+0000","last_peered":"2024-04-16T19:36:53.656638+0000","last_clean":"2024-04-16T19:36:53.656638+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:36:53.656638+0000","last_undegraded":"2024-04-16T19:36:53.656638+0000","last_fullsized":"2024-04-16T19:36:53.656638+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.656501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:12.757030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.426597+0000","last_change":"2024-04-16T19:36:35.426597+0000","last_active":"2024-04-16T19:36:35.426597+0000","last_peered":"2024-04-16T19:36:35.426597+0000","last_clean":"2024-04-16T19:36:35.426597+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:36:35.426597+0000","last_undegraded":"2024-04-16T19:36:35.426597+0000","last_fullsized":"2024-04-16T19:36:35.426597+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.426385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:09.737500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":52,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:26.201791+0000","last_change":"2024-04-16T18:10:50.552856+0000","last_active":"2024-04-16T19:35:26.201791+0000","last_peered":"2024-04-16T19:35:26.201791+0000","last_clean":"2024-04-16T19:35:26.201791+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:35:26.201791+0000","last_undegraded":"2024-04-16T19:35:26.201791+0000","last_fullsized":"2024-04-16T19:35:26.201791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'143","reported_seq":1870,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.706017+0000","last_change":"2024-04-16T19:36:51.706017+0000","last_active":"2024-04-16T19:36:51.706017+0000","last_peered":"2024-04-16T19:36:51.706017+0000","last_clean":"2024-04-16T19:36:51.706017+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:36:51.706017+0000","last_undegraded":"2024-04-16T19:36:51.706017+0000","last_fullsized":"2024-04-16T19:36:51.706017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'143","last_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_deep_scrub":"53'143","last_deep_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.705779+0000","objects_scrubbed":40,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:42.464983+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7446,"num_omap_keys":29,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.646565+0000","last_change":"2024-04-16T19:36:48.646617+0000","last_active":"2024-04-16T19:36:48.646565+0000","last_peered":"2024-04-16T19:36:48.646565+0000","last_clean":"2024-04-16T19:36:48.646565+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:36:48.646565+0000","last_undegraded":"2024-04-16T19:36:48.646565+0000","last_fullsized":"2024-04-16T19:36:48.646565+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.646427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:00.794914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'10","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.516567+0000","last_change":"2024-04-16T19:36:56.516633+0000","last_active":"2024-04-16T19:36:56.516567+0000","last_peered":"2024-04-16T19:36:56.516567+0000","last_clean":"2024-04-16T19:36:56.516567+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:36:56.516567+0000","last_undegraded":"2024-04-16T19:36:56.516567+0000","last_fullsized":"2024-04-16T19:36:56.516567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:56.516427+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:56.516427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.516427+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:30.858432+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.416813+0000","last_change":"2024-04-16T19:36:55.416865+0000","last_active":"2024-04-16T19:36:55.416813+0000","last_peered":"2024-04-16T19:36:55.416813+0000","last_clean":"2024-04-16T19:36:55.416813+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:36:55.416813+0000","last_undegraded":"2024-04-16T19:36:55.416813+0000","last_fullsized":"2024-04-16T19:36:55.416813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.416624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:21.421511+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'206","reported_seq":440,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.574363+0000","last_change":"2024-04-16T19:36:35.574523+0000","last_active":"2024-04-16T19:36:35.574363+0000","last_peered":"2024-04-16T19:36:35.574363+0000","last_clean":"2024-04-16T19:36:35.574363+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:36:35.574363+0000","last_undegraded":"2024-04-16T19:36:35.574363+0000","last_fullsized":"2024-04-16T19:36:35.574363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'206","last_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_deep_scrub":"53'206","last_deep_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.574163+0000","objects_scrubbed":10,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:15.061721+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'20","reported_seq":224,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.646869+0000","last_change":"2024-04-16T19:36:30.646869+0000","last_active":"2024-04-16T19:36:30.646869+0000","last_peered":"2024-04-16T19:36:30.646869+0000","last_clean":"2024-04-16T19:36:30.646869+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:36:30.646869+0000","last_undegraded":"2024-04-16T19:36:30.646869+0000","last_fullsized":"2024-04-16T19:36:30.646869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.646698+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:37.151471+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.645921+0000","last_change":"2024-04-16T19:37:20.645921+0000","last_active":"2024-04-16T19:37:20.645921+0000","last_peered":"2024-04-16T19:37:20.645921+0000","last_clean":"2024-04-16T19:37:20.645921+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:37:20.645921+0000","last_undegraded":"2024-04-16T19:37:20.645921+0000","last_fullsized":"2024-04-16T19:37:20.645921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.645734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.645734+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.645734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:44.885356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.452849+0000","last_change":"2024-04-16T19:37:25.452902+0000","last_active":"2024-04-16T19:37:25.452849+0000","last_peered":"2024-04-16T19:37:25.452849+0000","last_clean":"2024-04-16T19:37:25.452849+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:37:25.452849+0000","last_undegraded":"2024-04-16T19:37:25.452849+0000","last_fullsized":"2024-04-16T19:37:25.452849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.452712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.452712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.452712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:42.051092+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"53'7","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.654446+0000","last_change":"2024-04-16T19:37:05.654446+0000","last_active":"2024-04-16T19:37:05.654446+0000","last_peered":"2024-04-16T19:37:05.654446+0000","last_clean":"2024-04-16T19:37:05.654446+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:37:05.654446+0000","last_undegraded":"2024-04-16T19:37:05.654446+0000","last_fullsized":"2024-04-16T19:37:05.654446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:05.654239+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:05.654239+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.654239+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:11.660547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'109","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.373414+0000","last_change":"2024-04-16T19:36:57.373474+0000","last_active":"2024-04-16T19:36:57.373414+0000","last_peered":"2024-04-16T19:36:57.373414+0000","last_clean":"2024-04-16T19:36:57.373414+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:36:57.373414+0000","last_undegraded":"2024-04-16T19:36:57.373414+0000","last_fullsized":"2024-04-16T19:36:57.373414+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:57.373338+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:57.373338+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.373338+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:48.848526+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.699354+0000","last_change":"2024-04-16T19:36:54.699404+0000","last_active":"2024-04-16T19:36:54.699354+0000","last_peered":"2024-04-16T19:36:54.699354+0000","last_clean":"2024-04-16T19:36:54.699354+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:36:54.699354+0000","last_undegraded":"2024-04-16T19:36:54.699354+0000","last_fullsized":"2024-04-16T19:36:54.699354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.699216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:21.320753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.350584+0000","last_change":"2024-04-16T19:36:42.350584+0000","last_active":"2024-04-16T19:36:42.350584+0000","last_peered":"2024-04-16T19:36:42.350584+0000","last_clean":"2024-04-16T19:36:42.350584+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:36:42.350584+0000","last_undegraded":"2024-04-16T19:36:42.350584+0000","last_fullsized":"2024-04-16T19:36:42.350584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.350304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.971842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":46,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:11.214641+0000","last_change":"2024-04-16T18:11:33.605964+0000","last_active":"2024-04-16T19:36:11.214641+0000","last_peered":"2024-04-16T19:36:11.214641+0000","last_clean":"2024-04-16T19:36:11.214641+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:36:11.214641+0000","last_undegraded":"2024-04-16T19:36:11.214641+0000","last_fullsized":"2024-04-16T19:36:11.214641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'90","reported_seq":914,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.682412+0000","last_change":"2024-04-16T19:37:07.682412+0000","last_active":"2024-04-16T19:37:07.682412+0000","last_peered":"2024-04-16T19:37:07.682412+0000","last_clean":"2024-04-16T19:37:07.682412+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:37:07.682412+0000","last_undegraded":"2024-04-16T19:37:07.682412+0000","last_fullsized":"2024-04-16T19:37:07.682412+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'90","last_scrub_stamp":"2024-04-16T19:37:07.682156+0000","last_deep_scrub":"53'90","last_deep_scrub_stamp":"2024-04-16T19:37:07.682156+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.682156+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:33.687043+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5939,"num_omap_keys":27,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.668638+0000","last_change":"2024-04-16T19:36:44.668638+0000","last_active":"2024-04-16T19:36:44.668638+0000","last_peered":"2024-04-16T19:36:44.668638+0000","last_clean":"2024-04-16T19:36:44.668638+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:36:44.668638+0000","last_undegraded":"2024-04-16T19:36:44.668638+0000","last_fullsized":"2024-04-16T19:36:44.668638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.668449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:31.847082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"53'2","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.465959+0000","last_change":"2024-04-16T19:36:53.466069+0000","last_active":"2024-04-16T19:36:53.465959+0000","last_peered":"2024-04-16T19:36:53.465959+0000","last_clean":"2024-04-16T19:36:53.465959+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:36:53.465959+0000","last_undegraded":"2024-04-16T19:36:53.465959+0000","last_fullsized":"2024-04-16T19:36:53.465959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.465746+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:19.042353+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.452816+0000","last_change":"2024-04-16T19:36:41.452876+0000","last_active":"2024-04-16T19:36:41.452816+0000","last_peered":"2024-04-16T19:36:41.452816+0000","last_clean":"2024-04-16T19:36:41.452816+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:36:41.452816+0000","last_undegraded":"2024-04-16T19:36:41.452816+0000","last_fullsized":"2024-04-16T19:36:41.452816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.452676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:06.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'509","reported_seq":3967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635824+0000","last_change":"2024-04-16T19:36:20.636035+0000","last_active":"2024-04-16T19:36:20.635824+0000","last_peered":"2024-04-16T19:36:20.635824+0000","last_clean":"2024-04-16T19:36:20.635824+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:36:20.635824+0000","last_undegraded":"2024-04-16T19:36:20.635824+0000","last_fullsized":"2024-04-16T19:36:20.635824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635659+0000","objects_scrubbed":21,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:04.155514+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71921,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'34","reported_seq":244,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.437306+0000","last_change":"2024-04-16T19:36:23.437306+0000","last_active":"2024-04-16T19:36:23.437306+0000","last_peered":"2024-04-16T19:36:23.437306+0000","last_clean":"2024-04-16T19:36:23.437306+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:36:23.437306+0000","last_undegraded":"2024-04-16T19:36:23.437306+0000","last_fullsized":"2024-04-16T19:36:23.437306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.436922+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:38.568868+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.447112+0000","last_change":"2024-04-16T19:37:17.447112+0000","last_active":"2024-04-16T19:37:17.447112+0000","last_peered":"2024-04-16T19:37:17.447112+0000","last_clean":"2024-04-16T19:37:17.447112+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:37:17.447112+0000","last_undegraded":"2024-04-16T19:37:17.447112+0000","last_fullsized":"2024-04-16T19:37:17.447112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:17.446905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:17.446905+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.446905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:43.662599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.585768+0000","last_change":"2024-04-16T17:47:21.243851+0000","last_active":"2024-04-16T19:36:16.585768+0000","last_peered":"2024-04-16T19:36:16.585768+0000","last_clean":"2024-04-16T19:36:16.585768+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:36:16.585768+0000","last_undegraded":"2024-04-16T19:36:16.585768+0000","last_fullsized":"2024-04-16T19:36:16.585768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"53'5","reported_seq":814,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.631427+0000","last_change":"2024-04-16T19:37:06.631427+0000","last_active":"2024-04-16T19:37:06.631427+0000","last_peered":"2024-04-16T19:37:06.631427+0000","last_clean":"2024-04-16T19:37:06.631427+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:37:06.631427+0000","last_undegraded":"2024-04-16T19:37:06.631427+0000","last_fullsized":"2024-04-16T19:37:06.631427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'5","last_scrub_stamp":"2024-04-16T19:37:06.631224+0000","last_deep_scrub":"53'5","last_deep_scrub_stamp":"2024-04-16T19:37:06.631224+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.631224+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:52:08.200734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'125","reported_seq":312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.573654+0000","last_change":"2024-04-16T19:37:05.573733+0000","last_active":"2024-04-16T19:37:05.573654+0000","last_peered":"2024-04-16T19:37:05.573654+0000","last_clean":"2024-04-16T19:37:05.573654+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:37:05.573654+0000","last_undegraded":"2024-04-16T19:37:05.573654+0000","last_fullsized":"2024-04-16T19:37:05.573654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:37:05.573508+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:37:05.573508+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.573508+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:54.563678+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.345458+0000","last_change":"2024-04-16T19:37:01.345526+0000","last_active":"2024-04-16T19:37:01.345458+0000","last_peered":"2024-04-16T19:37:01.345458+0000","last_clean":"2024-04-16T19:37:01.345458+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:37:01.345458+0000","last_undegraded":"2024-04-16T19:37:01.345458+0000","last_fullsized":"2024-04-16T19:37:01.345458+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:01.345307+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:01.345307+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.345307+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:27.712980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.388173+0000","last_change":"2024-04-16T19:36:39.388173+0000","last_active":"2024-04-16T19:36:39.388173+0000","last_peered":"2024-04-16T19:36:39.388173+0000","last_clean":"2024-04-16T19:36:39.388173+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:36:39.388173+0000","last_undegraded":"2024-04-16T19:36:39.388173+0000","last_fullsized":"2024-04-16T19:36:39.388173+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.387834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:30.147039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.440112+0000","last_change":"2024-04-16T18:10:50.555228+0000","last_active":"2024-04-16T19:36:18.440112+0000","last_peered":"2024-04-16T19:36:18.440112+0000","last_clean":"2024-04-16T19:36:18.440112+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:36:18.440112+0000","last_undegraded":"2024-04-16T19:36:18.440112+0000","last_fullsized":"2024-04-16T19:36:18.440112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'123","reported_seq":1604,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.593801+0000","last_change":"2024-04-16T19:36:52.593899+0000","last_active":"2024-04-16T19:36:52.593801+0000","last_peered":"2024-04-16T19:36:52.593801+0000","last_clean":"2024-04-16T19:36:52.593801+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:36:52.593801+0000","last_undegraded":"2024-04-16T19:36:52.593801+0000","last_fullsized":"2024-04-16T19:36:52.593801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'123","last_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_deep_scrub":"53'123","last_deep_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.593639+0000","objects_scrubbed":37,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:52.173838+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5671,"num_omap_keys":23,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.668143+0000","last_change":"2024-04-16T19:36:53.668197+0000","last_active":"2024-04-16T19:36:53.668143+0000","last_peered":"2024-04-16T19:36:53.668143+0000","last_clean":"2024-04-16T19:36:53.668143+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:36:53.668143+0000","last_undegraded":"2024-04-16T19:36:53.668143+0000","last_fullsized":"2024-04-16T19:36:53.668143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.668006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:20:47.522228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'10","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.679540+0000","last_change":"2024-04-16T19:36:41.679612+0000","last_active":"2024-04-16T19:36:41.679540+0000","last_peered":"2024-04-16T19:36:41.679540+0000","last_clean":"2024-04-16T19:36:41.679540+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:36:41.679540+0000","last_undegraded":"2024-04-16T19:36:41.679540+0000","last_fullsized":"2024-04-16T19:36:41.679540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.679332+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:33.154520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'556","reported_seq":1963,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.395959+0000","last_change":"2024-04-16T19:36:44.396025+0000","last_active":"2024-04-16T19:36:44.395959+0000","last_peered":"2024-04-16T19:36:44.395959+0000","last_clean":"2024-04-16T19:36:44.395959+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:36:44.395959+0000","last_undegraded":"2024-04-16T19:36:44.395959+0000","last_fullsized":"2024-04-16T19:36:44.395959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.395808+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:46.739394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'508","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.543534+0000","last_change":"2024-04-16T19:36:25.543534+0000","last_active":"2024-04-16T19:36:25.543534+0000","last_peered":"2024-04-16T19:36:25.543534+0000","last_clean":"2024-04-16T19:36:25.543534+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:36:25.543534+0000","last_undegraded":"2024-04-16T19:36:25.543534+0000","last_fullsized":"2024-04-16T19:36:25.543534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.543268+0000","objects_scrubbed":16,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:59.596711+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71934,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'8","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.642341+0000","last_change":"2024-04-16T19:36:27.642341+0000","last_active":"2024-04-16T19:36:27.642341+0000","last_peered":"2024-04-16T19:36:27.642341+0000","last_clean":"2024-04-16T19:36:27.642341+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:36:27.642341+0000","last_undegraded":"2024-04-16T19:36:27.642341+0000","last_fullsized":"2024-04-16T19:36:27.642341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.642045+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:07.753490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.600045+0000","last_change":"2024-04-16T19:37:12.600045+0000","last_active":"2024-04-16T19:37:12.600045+0000","last_peered":"2024-04-16T19:37:12.600045+0000","last_clean":"2024-04-16T19:37:12.600045+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:37:12.600045+0000","last_undegraded":"2024-04-16T19:37:12.600045+0000","last_fullsized":"2024-04-16T19:37:12.600045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:12.599830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:12.599830+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.599830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:38.604717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.480527+0000","last_change":"2024-04-16T19:37:26.480578+0000","last_active":"2024-04-16T19:37:26.480527+0000","last_peered":"2024-04-16T19:37:26.480527+0000","last_clean":"2024-04-16T19:37:26.480527+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:37:26.480527+0000","last_undegraded":"2024-04-16T19:37:26.480527+0000","last_fullsized":"2024-04-16T19:37:26.480527+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.480382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.480382+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.480382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:44.428175+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"53'16","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.554624+0000","last_change":"2024-04-16T19:37:20.554624+0000","last_active":"2024-04-16T19:37:20.554624+0000","last_peered":"2024-04-16T19:37:20.554624+0000","last_clean":"2024-04-16T19:37:20.554624+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:37:20.554624+0000","last_undegraded":"2024-04-16T19:37:20.554624+0000","last_fullsized":"2024-04-16T19:37:20.554624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:37:20.554417+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:37:20.554417+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.554417+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:43.223515+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'159","reported_seq":358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.333939+0000","last_change":"2024-04-16T19:37:10.334092+0000","last_active":"2024-04-16T19:37:10.333939+0000","last_peered":"2024-04-16T19:37:10.333939+0000","last_clean":"2024-04-16T19:37:10.333939+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:37:10.333939+0000","last_undegraded":"2024-04-16T19:37:10.333939+0000","last_fullsized":"2024-04-16T19:37:10.333939+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:10.333771+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:10.333771+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.333771+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:16:33.077948+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.656216+0000","last_change":"2024-04-16T19:36:46.656320+0000","last_active":"2024-04-16T19:36:46.656216+0000","last_peered":"2024-04-16T19:36:46.656216+0000","last_clean":"2024-04-16T19:36:46.656216+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:36:46.656216+0000","last_undegraded":"2024-04-16T19:36:46.656216+0000","last_fullsized":"2024-04-16T19:36:46.656216+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.656048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:11.940546+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.451910+0000","last_change":"2024-04-16T19:36:37.452021+0000","last_active":"2024-04-16T19:36:37.451910+0000","last_peered":"2024-04-16T19:36:37.451910+0000","last_clean":"2024-04-16T19:36:37.451910+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:36:37.451910+0000","last_undegraded":"2024-04-16T19:36:37.451910+0000","last_fullsized":"2024-04-16T19:36:37.451910+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.451765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:52.163889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.540337+0000","last_change":"2024-04-16T19:37:31.540392+0000","last_active":"2024-04-16T19:37:31.540337+0000","last_peered":"2024-04-16T19:37:31.540337+0000","last_clean":"2024-04-16T19:37:31.540337+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:37:31.540337+0000","last_undegraded":"2024-04-16T19:37:31.540337+0000","last_fullsized":"2024-04-16T19:37:31.540337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.540200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.540200+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.540200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:54.209298+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'134","reported_seq":1476,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.725767+0000","last_change":"2024-04-16T19:36:59.725923+0000","last_active":"2024-04-16T19:36:59.725767+0000","last_peered":"2024-04-16T19:36:59.725767+0000","last_clean":"2024-04-16T19:36:59.725767+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:36:59.725767+0000","last_undegraded":"2024-04-16T19:36:59.725767+0000","last_fullsized":"2024-04-16T19:36:59.725767+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'134","last_scrub_stamp":"2024-04-16T19:36:59.725594+0000","last_deep_scrub":"53'134","last_deep_scrub_stamp":"2024-04-16T19:36:59.725594+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.725594+0000","objects_scrubbed":35,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:25.010093+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7807,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.355648+0000","last_change":"2024-04-16T19:37:03.355699+0000","last_active":"2024-04-16T19:37:03.355648+0000","last_peered":"2024-04-16T19:37:03.355648+0000","last_clean":"2024-04-16T19:37:03.355648+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:37:03.355648+0000","last_undegraded":"2024-04-16T19:37:03.355648+0000","last_fullsized":"2024-04-16T19:37:03.355648+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.355513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.355513+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.355513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:13.649973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'11","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.624192+0000","last_change":"2024-04-16T19:36:55.624271+0000","last_active":"2024-04-16T19:36:55.624192+0000","last_peered":"2024-04-16T19:36:55.624192+0000","last_clean":"2024-04-16T19:36:55.624192+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:36:55.624192+0000","last_undegraded":"2024-04-16T19:36:55.624192+0000","last_fullsized":"2024-04-16T19:36:55.624192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.624051+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:11.910789+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'556","reported_seq":1911,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.421934+0000","last_change":"2024-04-16T19:36:38.421934+0000","last_active":"2024-04-16T19:36:38.421934+0000","last_peered":"2024-04-16T19:36:38.421934+0000","last_clean":"2024-04-16T19:36:38.421934+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:36:38.421934+0000","last_undegraded":"2024-04-16T19:36:38.421934+0000","last_fullsized":"2024-04-16T19:36:38.421934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.421654+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:18.533099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'250","reported_seq":489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.484262+0000","last_change":"2024-04-16T19:36:27.484420+0000","last_active":"2024-04-16T19:36:27.484262+0000","last_peered":"2024-04-16T19:36:27.484262+0000","last_clean":"2024-04-16T19:36:27.484262+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:36:27.484262+0000","last_undegraded":"2024-04-16T19:36:27.484262+0000","last_fullsized":"2024-04-16T19:36:27.484262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.484127+0000","objects_scrubbed":20,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:04.899387+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"53'6","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.473636+0000","last_change":"2024-04-16T19:37:32.473718+0000","last_active":"2024-04-16T19:37:32.473636+0000","last_peered":"2024-04-16T19:37:32.473636+0000","last_clean":"2024-04-16T19:37:32.473636+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:37:32.473636+0000","last_undegraded":"2024-04-16T19:37:32.473636+0000","last_fullsized":"2024-04-16T19:37:32.473636+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:32.473504+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:32.473504+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.473504+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:52.367681+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.545483+0000","last_change":"2024-04-16T19:37:28.545539+0000","last_active":"2024-04-16T19:37:28.545483+0000","last_peered":"2024-04-16T19:37:28.545483+0000","last_clean":"2024-04-16T19:37:28.545483+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:37:28.545483+0000","last_undegraded":"2024-04-16T19:37:28.545483+0000","last_fullsized":"2024-04-16T19:37:28.545483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.545352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.545352+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.545352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:57.170533+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.477321+0000","last_change":"2024-04-16T19:37:28.477372+0000","last_active":"2024-04-16T19:37:28.477321+0000","last_peered":"2024-04-16T19:37:28.477321+0000","last_clean":"2024-04-16T19:37:28.477321+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:37:28.477321+0000","last_undegraded":"2024-04-16T19:37:28.477321+0000","last_fullsized":"2024-04-16T19:37:28.477321+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.477187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.477187+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.477187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:49.091407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"53'20","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.468760+0000","last_change":"2024-04-16T19:37:17.468760+0000","last_active":"2024-04-16T19:37:17.468760+0000","last_peered":"2024-04-16T19:37:17.468760+0000","last_clean":"2024-04-16T19:37:17.468760+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:37:17.468760+0000","last_undegraded":"2024-04-16T19:37:17.468760+0000","last_fullsized":"2024-04-16T19:37:17.468760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:37:17.468418+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:37:17.468418+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.468418+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:45.076005+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.665782+0000","last_change":"2024-04-16T19:37:14.665856+0000","last_active":"2024-04-16T19:37:14.665782+0000","last_peered":"2024-04-16T19:37:14.665782+0000","last_clean":"2024-04-16T19:37:14.665782+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:37:14.665782+0000","last_undegraded":"2024-04-16T19:37:14.665782+0000","last_fullsized":"2024-04-16T19:37:14.665782+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:14.665644+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:14.665644+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.665644+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:34.559821+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.446778+0000","last_change":"2024-04-16T19:36:59.446778+0000","last_active":"2024-04-16T19:36:59.446778+0000","last_peered":"2024-04-16T19:36:59.446778+0000","last_clean":"2024-04-16T19:36:59.446778+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:36:59.446778+0000","last_undegraded":"2024-04-16T19:36:59.446778+0000","last_fullsized":"2024-04-16T19:36:59.446778+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.446587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.446587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.446587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:20.442391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.477611+0000","last_change":"2024-04-16T19:36:26.477669+0000","last_active":"2024-04-16T19:36:26.477611+0000","last_peered":"2024-04-16T19:36:26.477611+0000","last_clean":"2024-04-16T19:36:26.477611+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:36:26.477611+0000","last_undegraded":"2024-04-16T19:36:26.477611+0000","last_fullsized":"2024-04-16T19:36:26.477611+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.477464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:27.657085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.588112+0000","last_change":"2024-04-16T18:10:50.527550+0000","last_active":"2024-04-16T19:36:16.588112+0000","last_peered":"2024-04-16T19:36:16.588112+0000","last_clean":"2024-04-16T19:36:16.588112+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:36:16.588112+0000","last_undegraded":"2024-04-16T19:36:16.588112+0000","last_fullsized":"2024-04-16T19:36:16.588112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'126","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.542707+0000","last_change":"2024-04-16T19:36:48.542776+0000","last_active":"2024-04-16T19:36:48.542707+0000","last_peered":"2024-04-16T19:36:48.542707+0000","last_clean":"2024-04-16T19:36:48.542707+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:36:48.542707+0000","last_undegraded":"2024-04-16T19:36:48.542707+0000","last_fullsized":"2024-04-16T19:36:48.542707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.542572+0000","objects_scrubbed":30,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:02.471385+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6535,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.626702+0000","last_change":"2024-04-16T19:36:43.626754+0000","last_active":"2024-04-16T19:36:43.626702+0000","last_peered":"2024-04-16T19:36:43.626702+0000","last_clean":"2024-04-16T19:36:43.626702+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:36:43.626702+0000","last_undegraded":"2024-04-16T19:36:43.626702+0000","last_fullsized":"2024-04-16T19:36:43.626702+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.626565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:30.990969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'13","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.507190+0000","last_change":"2024-04-16T19:36:57.507267+0000","last_active":"2024-04-16T19:36:57.507190+0000","last_peered":"2024-04-16T19:36:57.507190+0000","last_clean":"2024-04-16T19:36:57.507190+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:36:57.507190+0000","last_undegraded":"2024-04-16T19:36:57.507190+0000","last_fullsized":"2024-04-16T19:36:57.507190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:57.507046+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:57.507046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.507046+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:56.572731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.388632+0000","last_change":"2024-04-16T19:37:00.388683+0000","last_active":"2024-04-16T19:37:00.388632+0000","last_peered":"2024-04-16T19:37:00.388632+0000","last_clean":"2024-04-16T19:37:00.388632+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:37:00.388632+0000","last_undegraded":"2024-04-16T19:37:00.388632+0000","last_fullsized":"2024-04-16T19:37:00.388632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.388491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.388491+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.388491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:07.573705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'141","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.454241+0000","last_change":"2024-04-16T19:36:28.454241+0000","last_active":"2024-04-16T19:36:28.454241+0000","last_peered":"2024-04-16T19:36:28.454241+0000","last_clean":"2024-04-16T19:36:28.454241+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:36:28.454241+0000","last_undegraded":"2024-04-16T19:36:28.454241+0000","last_fullsized":"2024-04-16T19:36:28.454241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'141","last_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_deep_scrub":"53'141","last_deep_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.453990+0000","objects_scrubbed":12,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:15.632623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.478104+0000","last_change":"2024-04-16T19:36:22.478173+0000","last_active":"2024-04-16T19:36:22.478104+0000","last_peered":"2024-04-16T19:36:22.478104+0000","last_clean":"2024-04-16T19:36:22.478104+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:36:22.478104+0000","last_undegraded":"2024-04-16T19:36:22.478104+0000","last_fullsized":"2024-04-16T19:36:22.478104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.477939+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:46.331273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.645392+0000","last_change":"2024-04-16T19:37:26.645447+0000","last_active":"2024-04-16T19:37:26.645392+0000","last_peered":"2024-04-16T19:37:26.645392+0000","last_clean":"2024-04-16T19:37:26.645392+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:37:26.645392+0000","last_undegraded":"2024-04-16T19:37:26.645392+0000","last_fullsized":"2024-04-16T19:37:26.645392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.645255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.645255+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.645255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:43.359269+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.433032+0000","last_change":"2024-04-16T19:37:25.433086+0000","last_active":"2024-04-16T19:37:25.433032+0000","last_peered":"2024-04-16T19:37:25.433032+0000","last_clean":"2024-04-16T19:37:25.433032+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:37:25.433032+0000","last_undegraded":"2024-04-16T19:37:25.433032+0000","last_fullsized":"2024-04-16T19:37:25.433032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.432884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.432884+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.432884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:32.618098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.568435+0000","last_change":"2024-04-16T19:37:16.568539+0000","last_active":"2024-04-16T19:37:16.568435+0000","last_peered":"2024-04-16T19:37:16.568435+0000","last_clean":"2024-04-16T19:37:16.568435+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:37:16.568435+0000","last_undegraded":"2024-04-16T19:37:16.568435+0000","last_fullsized":"2024-04-16T19:37:16.568435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'3","last_scrub_stamp":"2024-04-16T19:37:16.568259+0000","last_deep_scrub":"48'3","last_deep_scrub_stamp":"2024-04-16T19:37:16.568259+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.568259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:37.564063+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'34","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.608145+0000","last_change":"2024-04-16T19:37:08.608145+0000","last_active":"2024-04-16T19:37:08.608145+0000","last_peered":"2024-04-16T19:37:08.608145+0000","last_clean":"2024-04-16T19:37:08.608145+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:37:08.608145+0000","last_undegraded":"2024-04-16T19:37:08.608145+0000","last_fullsized":"2024-04-16T19:37:08.608145+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:37:08.607831+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:37:08.607831+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.607831+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:02.316023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.532299+0000","last_change":"2024-04-16T19:37:03.532350+0000","last_active":"2024-04-16T19:37:03.532299+0000","last_peered":"2024-04-16T19:37:03.532299+0000","last_clean":"2024-04-16T19:37:03.532299+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:37:03.532299+0000","last_undegraded":"2024-04-16T19:37:03.532299+0000","last_fullsized":"2024-04-16T19:37:03.532299+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.532166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.532166+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.532166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:26.201264+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.360832+0000","last_change":"2024-04-16T19:36:41.360886+0000","last_active":"2024-04-16T19:36:41.360832+0000","last_peered":"2024-04-16T19:36:41.360832+0000","last_clean":"2024-04-16T19:36:41.360832+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:36:41.360832+0000","last_undegraded":"2024-04-16T19:36:41.360832+0000","last_fullsized":"2024-04-16T19:36:41.360832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.360675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.872893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.050546+0000","last_change":"2024-04-16T18:10:50.548741+0000","last_active":"2024-04-16T19:36:17.050546+0000","last_peered":"2024-04-16T19:36:17.050546+0000","last_clean":"2024-04-16T19:36:17.050546+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:36:17.050546+0000","last_undegraded":"2024-04-16T19:36:17.050546+0000","last_fullsized":"2024-04-16T19:36:17.050546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T18:10:49.502938+0000","last_clean_scrub_stamp":"2024-04-16T18:10:49.502938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'155","reported_seq":1489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.495554+0000","last_change":"2024-04-16T19:36:52.495627+0000","last_active":"2024-04-16T19:36:52.495554+0000","last_peered":"2024-04-16T19:36:52.495554+0000","last_clean":"2024-04-16T19:36:52.495554+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:36:52.495554+0000","last_undegraded":"2024-04-16T19:36:52.495554+0000","last_fullsized":"2024-04-16T19:36:52.495554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'155","last_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_deep_scrub":"53'155","last_deep_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.495389+0000","objects_scrubbed":29,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.497334+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8801,"num_omap_keys":37,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.641083+0000","last_change":"2024-04-16T19:36:38.641135+0000","last_active":"2024-04-16T19:36:38.641083+0000","last_peered":"2024-04-16T19:36:38.641083+0000","last_clean":"2024-04-16T19:36:38.641083+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:36:38.641083+0000","last_undegraded":"2024-04-16T19:36:38.641083+0000","last_fullsized":"2024-04-16T19:36:38.641083+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.640946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:53.196611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'11","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.644638+0000","last_change":"2024-04-16T19:36:54.644719+0000","last_active":"2024-04-16T19:36:54.644638+0000","last_peered":"2024-04-16T19:36:54.644638+0000","last_clean":"2024-04-16T19:36:54.644638+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:36:54.644638+0000","last_undegraded":"2024-04-16T19:36:54.644638+0000","last_fullsized":"2024-04-16T19:36:54.644638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.644499+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:20.331700+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.384126+0000","last_change":"2024-04-16T19:36:47.384178+0000","last_active":"2024-04-16T19:36:47.384126+0000","last_peered":"2024-04-16T19:36:47.384126+0000","last_clean":"2024-04-16T19:36:47.384126+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:36:47.384126+0000","last_undegraded":"2024-04-16T19:36:47.384126+0000","last_fullsized":"2024-04-16T19:36:47.384126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.383981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:49.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11080","reported_seq":18169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.523528+0000","last_change":"2024-04-16T19:36:22.523528+0000","last_active":"2024-04-16T19:36:22.523528+0000","last_peered":"2024-04-16T19:36:22.523528+0000","last_clean":"2024-04-16T19:36:22.523528+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:36:22.523528+0000","last_undegraded":"2024-04-16T19:36:22.523528+0000","last_fullsized":"2024-04-16T19:36:22.523528+0000","mapping_epoch":34,"log_start":"53'9300","ondisk_log_start":"53'9300","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11080","last_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_deep_scrub":"53'11080","last_deep_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.523321+0000","objects_scrubbed":25,"log_size":1780,"log_dups_size":3000,"ondisk_log_size":1780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:02.010878+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'10","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.675610+0000","last_change":"2024-04-16T19:36:33.675610+0000","last_active":"2024-04-16T19:36:33.675610+0000","last_peered":"2024-04-16T19:36:33.675610+0000","last_clean":"2024-04-16T19:36:33.675610+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:36:33.675610+0000","last_undegraded":"2024-04-16T19:36:33.675610+0000","last_fullsized":"2024-04-16T19:36:33.675610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.675369+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:17.195224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.692475+0000","last_change":"2024-04-16T19:37:30.692526+0000","last_active":"2024-04-16T19:37:30.692475+0000","last_peered":"2024-04-16T19:37:30.692475+0000","last_clean":"2024-04-16T19:37:30.692475+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:37:30.692475+0000","last_undegraded":"2024-04-16T19:37:30.692475+0000","last_fullsized":"2024-04-16T19:37:30.692475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.692343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.692343+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.692343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:35.469828+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.678927+0000","last_change":"2024-04-16T19:37:32.678985+0000","last_active":"2024-04-16T19:37:32.678927+0000","last_peered":"2024-04-16T19:37:32.678927+0000","last_clean":"2024-04-16T19:37:32.678927+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:37:32.678927+0000","last_undegraded":"2024-04-16T19:37:32.678927+0000","last_fullsized":"2024-04-16T19:37:32.678927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.678780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.678780+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.678780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:16:02.392190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'6","reported_seq":1355,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.657921+0000","last_change":"2024-04-16T19:37:33.657991+0000","last_active":"2024-04-16T19:37:33.657921+0000","last_peered":"2024-04-16T19:37:33.657921+0000","last_clean":"2024-04-16T19:37:33.657921+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:37:33.657921+0000","last_undegraded":"2024-04-16T19:37:33.657921+0000","last_fullsized":"2024-04-16T19:37:33.657921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:33.657758+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:33.657758+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.657758+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:38.138252+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'84","reported_seq":246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.654126+0000","last_change":"2024-04-16T19:36:59.654126+0000","last_active":"2024-04-16T19:36:59.654126+0000","last_peered":"2024-04-16T19:36:59.654126+0000","last_clean":"2024-04-16T19:36:59.654126+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:36:59.654126+0000","last_undegraded":"2024-04-16T19:36:59.654126+0000","last_fullsized":"2024-04-16T19:36:59.654126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:36:59.653919+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:36:59.653919+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.653919+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:10.641324+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.508215+0000","last_change":"2024-04-16T19:37:04.508267+0000","last_active":"2024-04-16T19:37:04.508215+0000","last_peered":"2024-04-16T19:37:04.508215+0000","last_clean":"2024-04-16T19:37:04.508215+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:37:04.508215+0000","last_undegraded":"2024-04-16T19:37:04.508215+0000","last_fullsized":"2024-04-16T19:37:04.508215+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.508075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.508075+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.508075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:53.498245+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.560046+0000","last_change":"2024-04-16T19:36:49.560147+0000","last_active":"2024-04-16T19:36:49.560046+0000","last_peered":"2024-04-16T19:36:49.560046+0000","last_clean":"2024-04-16T19:36:49.560046+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:36:49.560046+0000","last_undegraded":"2024-04-16T19:36:49.560046+0000","last_fullsized":"2024-04-16T19:36:49.560046+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.559817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:52.415984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'31","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.493564+0000","last_change":"2024-04-16T19:36:46.493705+0000","last_active":"2024-04-16T19:36:46.493564+0000","last_peered":"2024-04-16T19:36:46.493564+0000","last_clean":"2024-04-16T19:36:46.493564+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:36:46.493564+0000","last_undegraded":"2024-04-16T19:36:46.493564+0000","last_fullsized":"2024-04-16T19:36:46.493564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'31","last_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_deep_scrub":"53'31","last_deep_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.493417+0000","objects_scrubbed":9,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:52.920199+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.429456+0000","last_change":"2024-04-16T19:37:02.429510+0000","last_active":"2024-04-16T19:37:02.429456+0000","last_peered":"2024-04-16T19:37:02.429456+0000","last_clean":"2024-04-16T19:37:02.429456+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:37:02.429456+0000","last_undegraded":"2024-04-16T19:37:02.429456+0000","last_fullsized":"2024-04-16T19:37:02.429456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.429304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.429304+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.429304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:31.301173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.528042+0000","last_change":"2024-04-16T19:37:27.528146+0000","last_active":"2024-04-16T19:37:27.528042+0000","last_peered":"2024-04-16T19:37:27.528042+0000","last_clean":"2024-04-16T19:37:27.528042+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:37:27.528042+0000","last_undegraded":"2024-04-16T19:37:27.528042+0000","last_fullsized":"2024-04-16T19:37:27.528042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.527854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.527854+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.527854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:31.924496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'6","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.472692+0000","last_change":"2024-04-16T19:36:34.472767+0000","last_active":"2024-04-16T19:36:34.472692+0000","last_peered":"2024-04-16T19:36:34.472692+0000","last_clean":"2024-04-16T19:36:34.472692+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:36:34.472692+0000","last_undegraded":"2024-04-16T19:36:34.472692+0000","last_fullsized":"2024-04-16T19:36:34.472692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.472561+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:21.836965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.621011+0000","last_change":"2024-04-16T19:37:14.621063+0000","last_active":"2024-04-16T19:37:14.621011+0000","last_peered":"2024-04-16T19:37:14.621011+0000","last_clean":"2024-04-16T19:37:14.621011+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:37:14.621011+0000","last_undegraded":"2024-04-16T19:37:14.621011+0000","last_fullsized":"2024-04-16T19:37:14.621011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.620868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.620868+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.620868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:39.939904+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.481971+0000","last_change":"2024-04-16T19:37:21.482025+0000","last_active":"2024-04-16T19:37:21.481971+0000","last_peered":"2024-04-16T19:37:21.481971+0000","last_clean":"2024-04-16T19:37:21.481971+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:37:21.481971+0000","last_undegraded":"2024-04-16T19:37:21.481971+0000","last_fullsized":"2024-04-16T19:37:21.481971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.481828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.481828+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.481828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:55.823833+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'10","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.445631+0000","last_change":"2024-04-16T19:36:30.445631+0000","last_active":"2024-04-16T19:36:30.445631+0000","last_peered":"2024-04-16T19:36:30.445631+0000","last_clean":"2024-04-16T19:36:30.445631+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:36:30.445631+0000","last_undegraded":"2024-04-16T19:36:30.445631+0000","last_fullsized":"2024-04-16T19:36:30.445631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.445415+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:00.551163+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.476825+0000","last_change":"2024-04-16T19:37:27.476876+0000","last_active":"2024-04-16T19:37:27.476825+0000","last_peered":"2024-04-16T19:37:27.476825+0000","last_clean":"2024-04-16T19:37:27.476825+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:37:27.476825+0000","last_undegraded":"2024-04-16T19:37:27.476825+0000","last_fullsized":"2024-04-16T19:37:27.476825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.476688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.476688+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.476688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:11.477879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.209219+0000","last_change":"2024-04-16T17:47:21.239800+0000","last_active":"2024-04-16T19:35:41.209219+0000","last_peered":"2024-04-16T19:35:41.209219+0000","last_clean":"2024-04-16T19:35:41.209219+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:35:41.209219+0000","last_undegraded":"2024-04-16T19:35:41.209219+0000","last_fullsized":"2024-04-16T19:35:41.209219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'12","reported_seq":248,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.464290+0000","last_change":"2024-04-16T19:36:34.464387+0000","last_active":"2024-04-16T19:36:34.464290+0000","last_peered":"2024-04-16T19:36:34.464290+0000","last_clean":"2024-04-16T19:36:34.464290+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:36:34.464290+0000","last_undegraded":"2024-04-16T19:36:34.464290+0000","last_fullsized":"2024-04-16T19:36:34.464290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.464153+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:03.336022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.621593+0000","last_change":"2024-04-16T19:37:09.621645+0000","last_active":"2024-04-16T19:37:09.621593+0000","last_peered":"2024-04-16T19:37:09.621593+0000","last_clean":"2024-04-16T19:37:09.621593+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:37:09.621593+0000","last_undegraded":"2024-04-16T19:37:09.621593+0000","last_fullsized":"2024-04-16T19:37:09.621593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.621459+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:24:41.966705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.213866+0000","last_change":"2024-04-16T17:47:21.241187+0000","last_active":"2024-04-16T19:36:16.213866+0000","last_peered":"2024-04-16T19:36:16.213866+0000","last_clean":"2024-04-16T19:36:16.213866+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:36:16.213866+0000","last_undegraded":"2024-04-16T19:36:16.213866+0000","last_fullsized":"2024-04-16T19:36:16.213866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"53'6","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.528121+0000","last_change":"2024-04-16T19:36:28.528121+0000","last_active":"2024-04-16T19:36:28.528121+0000","last_peered":"2024-04-16T19:36:28.528121+0000","last_clean":"2024-04-16T19:36:28.528121+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:36:28.528121+0000","last_undegraded":"2024-04-16T19:36:28.528121+0000","last_fullsized":"2024-04-16T19:36:28.528121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.527896+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:25.132224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.475642+0000","last_change":"2024-04-16T19:37:15.475697+0000","last_active":"2024-04-16T19:37:15.475642+0000","last_peered":"2024-04-16T19:37:15.475642+0000","last_clean":"2024-04-16T19:37:15.475642+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:37:15.475642+0000","last_undegraded":"2024-04-16T19:37:15.475642+0000","last_fullsized":"2024-04-16T19:37:15.475642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.475502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.475502+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.475502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:53.312895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:35:41.205237+0000","last_change":"2024-04-16T17:47:21.243549+0000","last_active":"2024-04-16T19:35:41.205237+0000","last_peered":"2024-04-16T19:35:41.205237+0000","last_clean":"2024-04-16T19:35:41.205237+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:35:41.205237+0000","last_undegraded":"2024-04-16T19:35:41.205237+0000","last_fullsized":"2024-04-16T19:35:41.205237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"53'2","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.582466+0000","last_change":"2024-04-16T19:36:26.582466+0000","last_active":"2024-04-16T19:36:26.582466+0000","last_peered":"2024-04-16T19:36:26.582466+0000","last_clean":"2024-04-16T19:36:26.582466+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:36:26.582466+0000","last_undegraded":"2024-04-16T19:36:26.582466+0000","last_fullsized":"2024-04-16T19:36:26.582466+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.582238+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:10.024512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.583135+0000","last_change":"2024-04-16T19:37:13.583188+0000","last_active":"2024-04-16T19:37:13.583135+0000","last_peered":"2024-04-16T19:37:13.583135+0000","last_clean":"2024-04-16T19:37:13.583135+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:37:13.583135+0000","last_undegraded":"2024-04-16T19:37:13.583135+0000","last_fullsized":"2024-04-16T19:37:13.583135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.583001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.583001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.583001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:29.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.629955+0000","last_change":"2024-04-16T19:37:27.630014+0000","last_active":"2024-04-16T19:37:27.629955+0000","last_peered":"2024-04-16T19:37:27.629955+0000","last_clean":"2024-04-16T19:37:27.629955+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:37:27.629955+0000","last_undegraded":"2024-04-16T19:37:27.629955+0000","last_fullsized":"2024-04-16T19:37:27.629955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.629814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.629814+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.629814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:55:33.873646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'46","reported_seq":266,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.466241+0000","last_change":"2024-04-16T19:36:19.466346+0000","last_active":"2024-04-16T19:36:19.466241+0000","last_peered":"2024-04-16T19:36:19.466241+0000","last_clean":"2024-04-16T19:36:19.466241+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:36:19.466241+0000","last_undegraded":"2024-04-16T19:36:19.466241+0000","last_fullsized":"2024-04-16T19:36:19.466241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'46","last_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_deep_scrub":"53'46","last_deep_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.466115+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:05.816236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.436774+0000","last_change":"2024-04-16T19:37:04.436877+0000","last_active":"2024-04-16T19:37:04.436774+0000","last_peered":"2024-04-16T19:37:04.436774+0000","last_clean":"2024-04-16T19:37:04.436774+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:37:04.436774+0000","last_undegraded":"2024-04-16T19:37:04.436774+0000","last_fullsized":"2024-04-16T19:37:04.436774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.436610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.436610+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.436610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:24.645977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.712899+0000","last_change":"2024-04-16T19:37:29.713003+0000","last_active":"2024-04-16T19:37:29.712899+0000","last_peered":"2024-04-16T19:37:29.712899+0000","last_clean":"2024-04-16T19:37:29.712899+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:37:29.712899+0000","last_undegraded":"2024-04-16T19:37:29.712899+0000","last_fullsized":"2024-04-16T19:37:29.712899+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.712731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.712731+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.712731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:01:19.156040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'18","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.512117+0000","last_change":"2024-04-16T19:37:31.512195+0000","last_active":"2024-04-16T19:37:31.512117+0000","last_peered":"2024-04-16T19:37:31.512117+0000","last_clean":"2024-04-16T19:37:31.512117+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:37:31.512117+0000","last_undegraded":"2024-04-16T19:37:31.512117+0000","last_fullsized":"2024-04-16T19:37:31.512117+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:37:31.512041+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:37:31.512041+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.512041+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:12.436858+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.661881+0000","last_change":"2024-04-16T19:37:19.661934+0000","last_active":"2024-04-16T19:37:19.661881+0000","last_peered":"2024-04-16T19:37:19.661881+0000","last_clean":"2024-04-16T19:37:19.661881+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:37:19.661881+0000","last_undegraded":"2024-04-16T19:37:19.661881+0000","last_fullsized":"2024-04-16T19:37:19.661881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.661744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.661744+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.661744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:44.799680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.505291+0000","last_change":"2024-04-16T19:37:19.505347+0000","last_active":"2024-04-16T19:37:19.505291+0000","last_peered":"2024-04-16T19:37:19.505291+0000","last_clean":"2024-04-16T19:37:19.505291+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:37:19.505291+0000","last_undegraded":"2024-04-16T19:37:19.505291+0000","last_fullsized":"2024-04-16T19:37:19.505291+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.505147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.505147+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.505147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:53.058749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'21","reported_seq":207,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.453307+0000","last_change":"2024-04-16T19:36:21.453405+0000","last_active":"2024-04-16T19:36:21.453307+0000","last_peered":"2024-04-16T19:36:21.453307+0000","last_clean":"2024-04-16T19:36:21.453307+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:36:21.453307+0000","last_undegraded":"2024-04-16T19:36:21.453307+0000","last_fullsized":"2024-04-16T19:36:21.453307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.453170+0000","objects_scrubbed":6,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:26.892395+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.685941+0000","last_change":"2024-04-16T19:37:22.686051+0000","last_active":"2024-04-16T19:37:22.685941+0000","last_peered":"2024-04-16T19:37:22.685941+0000","last_clean":"2024-04-16T19:37:22.685941+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:37:22.685941+0000","last_undegraded":"2024-04-16T19:37:22.685941+0000","last_fullsized":"2024-04-16T19:37:22.685941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.685775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.685775+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.685775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:48.606236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:06.210981+0000","last_change":"2024-04-16T17:47:32.279326+0000","last_active":"2024-04-16T19:36:06.210981+0000","last_peered":"2024-04-16T19:36:06.210981+0000","last_clean":"2024-04-16T19:36:06.210981+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:36:06.210981+0000","last_undegraded":"2024-04-16T19:36:06.210981+0000","last_fullsized":"2024-04-16T19:36:06.210981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'26","reported_seq":334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.543887+0000","last_change":"2024-04-16T19:36:21.543887+0000","last_active":"2024-04-16T19:36:21.543887+0000","last_peered":"2024-04-16T19:36:21.543887+0000","last_clean":"2024-04-16T19:36:21.543887+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:36:21.543887+0000","last_undegraded":"2024-04-16T19:36:21.543887+0000","last_fullsized":"2024-04-16T19:36:21.543887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'26","last_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_deep_scrub":"53'26","last_deep_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.543655+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:47.643023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.658886+0000","last_change":"2024-04-16T19:37:09.658979+0000","last_active":"2024-04-16T19:37:09.658886+0000","last_peered":"2024-04-16T19:37:09.658886+0000","last_clean":"2024-04-16T19:37:09.658886+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:37:09.658886+0000","last_undegraded":"2024-04-16T19:37:09.658886+0000","last_fullsized":"2024-04-16T19:37:09.658886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.658720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.658720+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.658720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:59.721551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:16.585885+0000","last_change":"2024-04-16T17:47:21.243638+0000","last_active":"2024-04-16T19:36:16.585885+0000","last_peered":"2024-04-16T19:36:16.585885+0000","last_clean":"2024-04-16T19:36:16.585885+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:36:16.585885+0000","last_undegraded":"2024-04-16T19:36:16.585885+0000","last_fullsized":"2024-04-16T19:36:16.585885+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'2","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.488252+0000","last_change":"2024-04-16T19:36:22.488332+0000","last_active":"2024-04-16T19:36:22.488252+0000","last_peered":"2024-04-16T19:36:22.488252+0000","last_clean":"2024-04-16T19:36:22.488252+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:36:22.488252+0000","last_undegraded":"2024-04-16T19:36:22.488252+0000","last_fullsized":"2024-04-16T19:36:22.488252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.488109+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:37.620055+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.356894+0000","last_change":"2024-04-16T19:37:09.356950+0000","last_active":"2024-04-16T19:37:09.356894+0000","last_peered":"2024-04-16T19:37:09.356894+0000","last_clean":"2024-04-16T19:37:09.356894+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:37:09.356894+0000","last_undegraded":"2024-04-16T19:37:09.356894+0000","last_fullsized":"2024-04-16T19:37:09.356894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.356747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.356747+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.356747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:28.868964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.911836+0000","last_change":"2024-04-16T17:47:21.253447+0000","last_active":"2024-04-16T19:36:18.911836+0000","last_peered":"2024-04-16T19:36:18.911836+0000","last_clean":"2024-04-16T19:36:18.911836+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:36:18.911836+0000","last_undegraded":"2024-04-16T19:36:18.911836+0000","last_fullsized":"2024-04-16T19:36:18.911836+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'22","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.501837+0000","last_change":"2024-04-16T19:36:47.501923+0000","last_active":"2024-04-16T19:36:47.501837+0000","last_peered":"2024-04-16T19:36:47.501837+0000","last_clean":"2024-04-16T19:36:47.501837+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:36:47.501837+0000","last_undegraded":"2024-04-16T19:36:47.501837+0000","last_fullsized":"2024-04-16T19:36:47.501837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.501703+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:52.574907+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.674066+0000","last_change":"2024-04-16T19:37:20.674118+0000","last_active":"2024-04-16T19:37:20.674066+0000","last_peered":"2024-04-16T19:37:20.674066+0000","last_clean":"2024-04-16T19:37:20.674066+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:37:20.674066+0000","last_undegraded":"2024-04-16T19:37:20.674066+0000","last_fullsized":"2024-04-16T19:37:20.674066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.673928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.673928+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.673928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:08.556042+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.634312+0000","last_change":"2024-04-16T19:37:34.634312+0000","last_active":"2024-04-16T19:37:34.634312+0000","last_peered":"2024-04-16T19:37:34.634312+0000","last_clean":"2024-04-16T19:37:34.634312+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:37:34.634312+0000","last_undegraded":"2024-04-16T19:37:34.634312+0000","last_fullsized":"2024-04-16T19:37:34.634312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.634115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.634115+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.634115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:13.475061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.568771+0000","last_change":"2024-04-16T19:36:22.568861+0000","last_active":"2024-04-16T19:36:22.568771+0000","last_peered":"2024-04-16T19:36:22.568771+0000","last_clean":"2024-04-16T19:36:22.568771+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:36:22.568771+0000","last_undegraded":"2024-04-16T19:36:22.568771+0000","last_fullsized":"2024-04-16T19:36:22.568771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.568637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:27.641841+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.661136+0000","last_change":"2024-04-16T19:37:07.661244+0000","last_active":"2024-04-16T19:37:07.661136+0000","last_peered":"2024-04-16T19:37:07.661136+0000","last_clean":"2024-04-16T19:37:07.661136+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:37:07.661136+0000","last_undegraded":"2024-04-16T19:37:07.661136+0000","last_fullsized":"2024-04-16T19:37:07.661136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.660970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.660970+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.660970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:33.237577+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.721669+0000","last_change":"2024-04-16T19:37:21.721669+0000","last_active":"2024-04-16T19:37:21.721669+0000","last_peered":"2024-04-16T19:37:21.721669+0000","last_clean":"2024-04-16T19:37:21.721669+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:37:21.721669+0000","last_undegraded":"2024-04-16T19:37:21.721669+0000","last_fullsized":"2024-04-16T19:37:21.721669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.721479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.721479+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.721479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:45.961100+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"53'4","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.473816+0000","last_change":"2024-04-16T19:36:43.473937+0000","last_active":"2024-04-16T19:36:43.473816+0000","last_peered":"2024-04-16T19:36:43.473816+0000","last_clean":"2024-04-16T19:36:43.473816+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:36:43.473816+0000","last_undegraded":"2024-04-16T19:36:43.473816+0000","last_fullsized":"2024-04-16T19:36:43.473816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.473655+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:59.715799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.652759+0000","last_change":"2024-04-16T19:37:23.652821+0000","last_active":"2024-04-16T19:37:23.652759+0000","last_peered":"2024-04-16T19:37:23.652759+0000","last_clean":"2024-04-16T19:37:23.652759+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:37:23.652759+0000","last_undegraded":"2024-04-16T19:37:23.652759+0000","last_fullsized":"2024-04-16T19:37:23.652759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.652616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.652616+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.652616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:07.653807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.710257+0000","last_change":"2024-04-16T19:37:33.710309+0000","last_active":"2024-04-16T19:37:33.710257+0000","last_peered":"2024-04-16T19:37:33.710257+0000","last_clean":"2024-04-16T19:37:33.710257+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:37:33.710257+0000","last_undegraded":"2024-04-16T19:37:33.710257+0000","last_fullsized":"2024-04-16T19:37:33.710257+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.710122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.710122+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.710122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:16:03.423532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'6","reported_seq":182,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.607311+0000","last_change":"2024-04-16T19:36:19.607390+0000","last_active":"2024-04-16T19:36:19.607311+0000","last_peered":"2024-04-16T19:36:19.607311+0000","last_clean":"2024-04-16T19:36:19.607311+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:36:19.607311+0000","last_undegraded":"2024-04-16T19:36:19.607311+0000","last_fullsized":"2024-04-16T19:36:19.607311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.607181+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:07:42.514098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.546329+0000","last_change":"2024-04-16T19:37:20.546364+0000","last_active":"2024-04-16T19:37:20.546329+0000","last_peered":"2024-04-16T19:37:20.546329+0000","last_clean":"2024-04-16T19:37:20.546329+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:37:20.546329+0000","last_undegraded":"2024-04-16T19:37:20.546329+0000","last_fullsized":"2024-04-16T19:37:20.546329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.546236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.546236+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.546236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:08.656140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.479097+0000","last_change":"2024-04-16T19:37:18.479150+0000","last_active":"2024-04-16T19:37:18.479097+0000","last_peered":"2024-04-16T19:37:18.479097+0000","last_clean":"2024-04-16T19:37:18.479097+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:37:18.479097+0000","last_undegraded":"2024-04-16T19:37:18.479097+0000","last_fullsized":"2024-04-16T19:37:18.479097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.478956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.478956+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.478956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:44.055564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'12","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498647+0000","last_change":"2024-04-16T19:36:19.498750+0000","last_active":"2024-04-16T19:36:19.498647+0000","last_peered":"2024-04-16T19:36:19.498647+0000","last_clean":"2024-04-16T19:36:19.498647+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:36:19.498647+0000","last_undegraded":"2024-04-16T19:36:19.498647+0000","last_fullsized":"2024-04-16T19:36:19.498647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498508+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:33.427321+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.620126+0000","last_change":"2024-04-16T19:37:18.620180+0000","last_active":"2024-04-16T19:37:18.620126+0000","last_peered":"2024-04-16T19:37:18.620126+0000","last_clean":"2024-04-16T19:37:18.620126+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:37:18.620126+0000","last_undegraded":"2024-04-16T19:37:18.620126+0000","last_fullsized":"2024-04-16T19:37:18.620126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.619988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.619988+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.619988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:43.755618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.541500+0000","last_change":"2024-04-16T19:37:24.541551+0000","last_active":"2024-04-16T19:37:24.541500+0000","last_peered":"2024-04-16T19:37:24.541500+0000","last_clean":"2024-04-16T19:37:24.541500+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:37:24.541500+0000","last_undegraded":"2024-04-16T19:37:24.541500+0000","last_fullsized":"2024-04-16T19:37:24.541500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.541369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.541369+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.541369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:12.423483+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'16","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.449658+0000","last_change":"2024-04-16T19:36:28.449751+0000","last_active":"2024-04-16T19:36:28.449658+0000","last_peered":"2024-04-16T19:36:28.449658+0000","last_clean":"2024-04-16T19:36:28.449658+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:36:28.449658+0000","last_undegraded":"2024-04-16T19:36:28.449658+0000","last_fullsized":"2024-04-16T19:36:28.449658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.449522+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:19.924709+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.977941+0000","last_change":"2024-04-16T17:47:21.246697+0000","last_active":"2024-04-16T19:36:17.977941+0000","last_peered":"2024-04-16T19:36:17.977941+0000","last_clean":"2024-04-16T19:36:17.977941+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:36:17.977941+0000","last_undegraded":"2024-04-16T19:36:17.977941+0000","last_fullsized":"2024-04-16T19:36:17.977941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:47:20.200038+0000","last_clean_scrub_stamp":"2024-04-16T17:47:20.200038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.599119+0000","last_change":"2024-04-16T19:37:19.599170+0000","last_active":"2024-04-16T19:37:19.599119+0000","last_peered":"2024-04-16T19:37:19.599119+0000","last_clean":"2024-04-16T19:37:19.599119+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:37:19.599119+0000","last_undegraded":"2024-04-16T19:37:19.599119+0000","last_fullsized":"2024-04-16T19:37:19.599119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.598990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.598990+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.598990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:07.481104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'20","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.594397+0000","last_change":"2024-04-16T19:36:27.594512+0000","last_active":"2024-04-16T19:36:27.594397+0000","last_peered":"2024-04-16T19:36:27.594397+0000","last_clean":"2024-04-16T19:36:27.594397+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:36:27.594397+0000","last_undegraded":"2024-04-16T19:36:27.594397+0000","last_fullsized":"2024-04-16T19:36:27.594397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.594261+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:01.851085+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.683833+0000","last_change":"2024-04-16T19:37:32.683888+0000","last_active":"2024-04-16T19:37:32.683833+0000","last_peered":"2024-04-16T19:37:32.683833+0000","last_clean":"2024-04-16T19:37:32.683833+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:37:32.683833+0000","last_undegraded":"2024-04-16T19:37:32.683833+0000","last_fullsized":"2024-04-16T19:37:32.683833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.683695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.683695+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.683695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:20.565810+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.495541+0000","last_change":"2024-04-16T19:37:25.495643+0000","last_active":"2024-04-16T19:37:25.495541+0000","last_peered":"2024-04-16T19:37:25.495541+0000","last_clean":"2024-04-16T19:37:25.495541+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:37:25.495541+0000","last_undegraded":"2024-04-16T19:37:25.495541+0000","last_fullsized":"2024-04-16T19:37:25.495541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.495372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.495372+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.495372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:29.975866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":174546,"num_omap_keys":712,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3586,"ondisk_log_size":3586,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3885,"ondisk_log_size":3885,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":958,"ondisk_log_size":958,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2422969,"num_omap_keys":10622,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27854,"ondisk_log_size":27854,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4469,"ondisk_log_size":4469,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9817,"num_omap_keys":62,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":566,"ondisk_log_size":566,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641592,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50744,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806856,"statfs":{"total":107374182400,"available":107322220544,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641591,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":81156,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":104776444,"statfs":{"total":107374182400,"available":107291078656,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641591,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":87452,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":104770148,"statfs":{"total":107374182400,"available":107284631552,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641591,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50732,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806868,"statfs":{"total":107374182400,"available":107322232832,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641592,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":59568,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":104798032,"statfs":{"total":107374182400,"available":107313184768,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641592,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50748,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806852,"statfs":{"total":107374182400,"available":107322216448,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:37:41.386 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.387 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.387 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.387 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.387 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.388 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.388 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.388 INFO:tasks.ceph:pgid 15.1f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.389 INFO:tasks.ceph:pgid 15.1d last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.389 INFO:tasks.ceph:pgid 15.1c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.389 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.390 INFO:tasks.ceph:pgid 14.3 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.390 INFO:tasks.ceph:pgid 15.1 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.390 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2024-04-16T17:48:00.194486+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=0, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.390 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.391 INFO:tasks.ceph:pgid 15.7 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.391 INFO:tasks.ceph:pgid 15.6 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.391 INFO:tasks.ceph:pgid 15.5 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.392 INFO:tasks.ceph:pgid 15.f last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.392 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.392 INFO:tasks.ceph:pgid 15.c last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.393 INFO:tasks.ceph:pgid 14.d last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.393 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-16T17:46:37.490454+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=37, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.393 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.393 INFO:tasks.ceph:pgid 14.b last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.394 INFO:tasks.ceph:pgid 14.9 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.394 INFO:tasks.ceph:pgid 15.9 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.394 INFO:tasks.ceph:pgid 14.8 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.395 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.395 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-16T17:46:36.485975+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=46, tm_sec=36, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.395 INFO:tasks.ceph:pgid 15.18 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.395 INFO:tasks.ceph:pgid 14.19 last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.396 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.396 INFO:tasks.ceph:pgid 15.1a last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.396 INFO:tasks.ceph:pgid 15.1b last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.396 INFO:tasks.ceph:pgid 14.1a last_scrub_stamp 2024-04-16T17:48:33.281090+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=48, tm_sec=33, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.397 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.397 INFO:tasks.ceph:pgid 15.14 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.397 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-16T17:47:32.253794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=32, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.397 INFO:tasks.ceph:pgid 15.15 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.398 INFO:tasks.ceph:pgid 15.16 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.398 INFO:tasks.ceph:pgid 15.17 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.398 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.399 INFO:tasks.ceph:pgid 15.10 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.399 INFO:tasks.ceph:pgid 15.12 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.399 INFO:tasks.ceph:pgid 15.13 last_scrub_stamp 2024-04-16T18:10:49.502938+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=18, tm_min=10, tm_sec=49, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.400 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.400 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.400 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.400 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.401 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.401 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.401 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-16T17:47:20.200038+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=17, tm_min=47, tm_sec=20, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=19, tm_min=36, tm_sec=15, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T19:37:41.402 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-16T19:38:01.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2024-04-16T19:38:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T19:38:01.837 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T19:38:01.839 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":3388,"stamp":"2024-04-16T19:38:00.401487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":793345,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":494329,"num_read_kb":403218,"num_write":75885,"num_write_kb":18262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":369,"num_keys_recovered":2,"num_objects_omap":993,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2657298,"num_omap_keys":11600,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41895,"ondisk_log_size":41895,"up":1042,"acting":1042,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1042,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":629145600,"kb_used":380400,"kb_used_data":10112,"kb_used_omap":9,"kb_used_meta":370230,"kb_avail":628765200,"statfs":{"total":644245094400,"available":643855564800,"internally_reserved":0,"allocated":10354688,"data_stored":3685786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":379116192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31909,"num_omap_keys":132,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008540"},"pg_stats":[{"pgid":"9.35","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.507786+0000","last_change":"2024-04-16T19:36:20.507786+0000","last_active":"2024-04-16T19:36:20.507786+0000","last_peered":"2024-04-16T19:36:20.507786+0000","last_clean":"2024-04-16T19:36:20.507786+0000","last_became_active":"2024-04-16T17:47:55.157839+0000","last_became_peered":"2024-04-16T17:47:55.157839+0000","last_unstale":"2024-04-16T19:36:20.507786+0000","last_undegraded":"2024-04-16T19:36:20.507786+0000","last_fullsized":"2024-04-16T19:36:20.507786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:20.507587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.507587+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:21:12.018020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":72,"num_write":29,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:37.645466+0000","last_change":"2024-04-16T19:37:37.645466+0000","last_active":"2024-04-16T19:37:37.645466+0000","last_peered":"2024-04-16T19:37:37.645466+0000","last_clean":"2024-04-16T19:37:37.645466+0000","last_became_active":"2024-04-16T17:47:21.238353+0000","last_became_peered":"2024-04-16T17:47:21.238353+0000","last_unstale":"2024-04-16T19:37:37.645466+0000","last_undegraded":"2024-04-16T19:37:37.645466+0000","last_fullsized":"2024-04-16T19:37:37.645466+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:37.645273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:37.645273+0000","last_clean_scrub_stamp":"2024-04-16T19:37:37.645273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:20:12.654937+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.506926+0000","last_change":"2024-04-16T19:37:23.506926+0000","last_active":"2024-04-16T19:37:23.506926+0000","last_peered":"2024-04-16T19:37:23.506926+0000","last_clean":"2024-04-16T19:37:23.506926+0000","last_became_active":"2024-04-16T17:47:23.261716+0000","last_became_peered":"2024-04-16T17:47:23.261716+0000","last_unstale":"2024-04-16T19:37:23.506926+0000","last_undegraded":"2024-04-16T19:37:23.506926+0000","last_fullsized":"2024-04-16T19:37:23.506926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.506739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.506739+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.506739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:09.177876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.34","version":"53'23","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.496453+0000","last_change":"2024-04-16T19:36:37.496453+0000","last_active":"2024-04-16T19:36:37.496453+0000","last_peered":"2024-04-16T19:36:37.496453+0000","last_clean":"2024-04-16T19:36:37.496453+0000","last_became_active":"2024-04-16T17:47:55.167580+0000","last_became_peered":"2024-04-16T17:47:55.167580+0000","last_unstale":"2024-04-16T19:36:37.496453+0000","last_undegraded":"2024-04-16T19:36:37.496453+0000","last_fullsized":"2024-04-16T19:36:37.496453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:37.496061+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.496061+0000","objects_scrubbed":9,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:45.797648+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5178,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":461,"num_read_kb":301,"num_write":110,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.658906+0000","last_change":"2024-04-16T19:37:31.658906+0000","last_active":"2024-04-16T19:37:31.658906+0000","last_peered":"2024-04-16T19:37:31.658906+0000","last_clean":"2024-04-16T19:37:31.658906+0000","last_became_active":"2024-04-16T17:47:21.236052+0000","last_became_peered":"2024-04-16T17:47:21.236052+0000","last_unstale":"2024-04-16T19:37:31.658906+0000","last_undegraded":"2024-04-16T19:37:31.658906+0000","last_fullsized":"2024-04-16T19:37:31.658906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.658712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.658712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.658712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:17.329849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.734066+0000","last_change":"2024-04-16T19:37:32.734117+0000","last_active":"2024-04-16T19:37:32.734066+0000","last_peered":"2024-04-16T19:37:32.734066+0000","last_clean":"2024-04-16T19:37:32.734066+0000","last_became_active":"2024-04-16T17:47:23.272729+0000","last_became_peered":"2024-04-16T17:47:23.272729+0000","last_unstale":"2024-04-16T19:37:32.734066+0000","last_undegraded":"2024-04-16T19:37:32.734066+0000","last_fullsized":"2024-04-16T19:37:32.734066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.733931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.733931+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.733931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:18.127607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"53'25","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.596413+0000","last_change":"2024-04-16T19:36:46.596413+0000","last_active":"2024-04-16T19:36:46.596413+0000","last_peered":"2024-04-16T19:36:46.596413+0000","last_clean":"2024-04-16T19:36:46.596413+0000","last_became_active":"2024-04-16T17:47:55.156257+0000","last_became_peered":"2024-04-16T17:47:55.156257+0000","last_unstale":"2024-04-16T19:36:46.596413+0000","last_undegraded":"2024-04-16T19:36:46.596413+0000","last_fullsized":"2024-04-16T19:36:46.596413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'25","last_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_deep_scrub":"53'25","last_deep_scrub_stamp":"2024-04-16T19:36:46.596297+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.596297+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:26.632824+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":115,"num_write":108,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:37.637713+0000","last_change":"2024-04-16T19:37:37.637713+0000","last_active":"2024-04-16T19:37:37.637713+0000","last_peered":"2024-04-16T19:37:37.637713+0000","last_clean":"2024-04-16T19:37:37.637713+0000","last_became_active":"2024-04-16T17:47:21.238430+0000","last_became_peered":"2024-04-16T17:47:21.238430+0000","last_unstale":"2024-04-16T19:37:37.637713+0000","last_undegraded":"2024-04-16T19:37:37.637713+0000","last_fullsized":"2024-04-16T19:37:37.637713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:37.637515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:37.637515+0000","last_clean_scrub_stamp":"2024-04-16T19:37:37.637515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:20.179638+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.564449+0000","last_change":"2024-04-16T19:37:19.564449+0000","last_active":"2024-04-16T19:37:19.564449+0000","last_peered":"2024-04-16T19:37:19.564449+0000","last_clean":"2024-04-16T19:37:19.564449+0000","last_became_active":"2024-04-16T17:47:23.253572+0000","last_became_peered":"2024-04-16T17:47:23.253572+0000","last_unstale":"2024-04-16T19:37:19.564449+0000","last_undegraded":"2024-04-16T19:37:19.564449+0000","last_fullsized":"2024-04-16T19:37:19.564449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.564241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.564241+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.564241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:52.007450+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"53'15","reported_seq":206,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.698085+0000","last_change":"2024-04-16T19:37:34.698183+0000","last_active":"2024-04-16T19:37:34.698085+0000","last_peered":"2024-04-16T19:37:34.698085+0000","last_clean":"2024-04-16T19:37:34.698085+0000","last_became_active":"2024-04-16T17:49:32.392383+0000","last_became_peered":"2024-04-16T17:49:32.392383+0000","last_unstale":"2024-04-16T19:37:34.698085+0000","last_undegraded":"2024-04-16T19:37:34.698085+0000","last_fullsized":"2024-04-16T19:37:34.698085+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2024-04-16T19:37:34.697956+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2024-04-16T19:37:34.697956+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.697956+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:52.645750+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":203,"num_read_kb":149,"num_write":76,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:41.457389+0000","last_change":"2024-04-16T19:37:41.457389+0000","last_active":"2024-04-16T19:37:41.457389+0000","last_peered":"2024-04-16T19:37:41.457389+0000","last_clean":"2024-04-16T19:37:41.457389+0000","last_became_active":"2024-04-16T17:47:21.233913+0000","last_became_peered":"2024-04-16T17:47:21.233913+0000","last_unstale":"2024-04-16T19:37:41.457389+0000","last_undegraded":"2024-04-16T19:37:41.457389+0000","last_fullsized":"2024-04-16T19:37:41.457389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:41.457187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:41.457187+0000","last_clean_scrub_stamp":"2024-04-16T19:37:41.457187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:37.081071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.709971+0000","last_change":"2024-04-16T19:37:31.710023+0000","last_active":"2024-04-16T19:37:31.709971+0000","last_peered":"2024-04-16T19:37:31.709971+0000","last_clean":"2024-04-16T19:37:31.709971+0000","last_became_active":"2024-04-16T17:47:23.267245+0000","last_became_peered":"2024-04-16T17:47:23.267245+0000","last_unstale":"2024-04-16T19:37:31.709971+0000","last_undegraded":"2024-04-16T19:37:31.709971+0000","last_fullsized":"2024-04-16T19:37:31.709971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.709834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.709834+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.709834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:17.103509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"53'8","reported_seq":193,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.663358+0000","last_change":"2024-04-16T19:36:31.663358+0000","last_active":"2024-04-16T19:36:31.663358+0000","last_peered":"2024-04-16T19:36:31.663358+0000","last_clean":"2024-04-16T19:36:31.663358+0000","last_became_active":"2024-04-16T17:47:55.159220+0000","last_became_peered":"2024-04-16T17:47:55.159220+0000","last_unstale":"2024-04-16T19:36:31.663358+0000","last_undegraded":"2024-04-16T19:36:31.663358+0000","last_fullsized":"2024-04-16T19:36:31.663358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:31.663140+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.663140+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:09.078400+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":135,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.620716+0000","last_change":"2024-04-16T19:37:17.620716+0000","last_active":"2024-04-16T19:37:17.620716+0000","last_peered":"2024-04-16T19:37:17.620716+0000","last_clean":"2024-04-16T19:37:17.620716+0000","last_became_active":"2024-04-16T17:47:32.278482+0000","last_became_peered":"2024-04-16T17:47:32.278482+0000","last_unstale":"2024-04-16T19:37:17.620716+0000","last_undegraded":"2024-04-16T19:37:17.620716+0000","last_fullsized":"2024-04-16T19:37:17.620716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:17.620394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:17.620394+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.620394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:23:06.022567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.530497+0000","last_change":"2024-04-16T19:37:21.530497+0000","last_active":"2024-04-16T19:37:21.530497+0000","last_peered":"2024-04-16T19:37:21.530497+0000","last_clean":"2024-04-16T19:37:21.530497+0000","last_became_active":"2024-04-16T17:47:23.259268+0000","last_became_peered":"2024-04-16T17:47:23.259268+0000","last_unstale":"2024-04-16T19:37:21.530497+0000","last_undegraded":"2024-04-16T19:37:21.530497+0000","last_fullsized":"2024-04-16T19:37:21.530497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.530183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.530183+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.530183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:14:58.265947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"53'20","reported_seq":328,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.492760+0000","last_change":"2024-04-16T19:36:26.492760+0000","last_active":"2024-04-16T19:36:26.492760+0000","last_peered":"2024-04-16T19:36:26.492760+0000","last_clean":"2024-04-16T19:36:26.492760+0000","last_became_active":"2024-04-16T17:47:55.163285+0000","last_became_peered":"2024-04-16T17:47:55.163285+0000","last_unstale":"2024-04-16T19:36:26.492760+0000","last_undegraded":"2024-04-16T19:36:26.492760+0000","last_fullsized":"2024-04-16T19:36:26.492760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:26.492498+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.492498+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:56.598246+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":519,"num_read_kb":331,"num_write":109,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:40.430654+0000","last_change":"2024-04-16T19:37:40.430705+0000","last_active":"2024-04-16T19:37:40.430654+0000","last_peered":"2024-04-16T19:37:40.430654+0000","last_clean":"2024-04-16T19:37:40.430654+0000","last_became_active":"2024-04-16T17:47:21.229488+0000","last_became_peered":"2024-04-16T17:47:21.229488+0000","last_unstale":"2024-04-16T19:37:40.430654+0000","last_undegraded":"2024-04-16T19:37:40.430654+0000","last_fullsized":"2024-04-16T19:37:40.430654+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:40.430511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:40.430511+0000","last_clean_scrub_stamp":"2024-04-16T19:37:40.430511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:20:15.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.508466+0000","last_change":"2024-04-16T19:37:21.508466+0000","last_active":"2024-04-16T19:37:21.508466+0000","last_peered":"2024-04-16T19:37:21.508466+0000","last_clean":"2024-04-16T19:37:21.508466+0000","last_became_active":"2024-04-16T17:47:23.249674+0000","last_became_peered":"2024-04-16T17:47:23.249674+0000","last_unstale":"2024-04-16T19:37:21.508466+0000","last_undegraded":"2024-04-16T19:37:21.508466+0000","last_fullsized":"2024-04-16T19:37:21.508466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.508258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.508258+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.508258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:19.847601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.33","version":"53'19","reported_seq":285,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.411649+0000","last_change":"2024-04-16T19:36:31.411649+0000","last_active":"2024-04-16T19:36:31.411649+0000","last_peered":"2024-04-16T19:36:31.411649+0000","last_clean":"2024-04-16T19:36:31.411649+0000","last_became_active":"2024-04-16T17:47:55.152416+0000","last_became_peered":"2024-04-16T17:47:55.152416+0000","last_unstale":"2024-04-16T19:36:31.411649+0000","last_undegraded":"2024-04-16T19:36:31.411649+0000","last_fullsized":"2024-04-16T19:36:31.411649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:31.411418+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.411418+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:18.775822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":415,"num_read_kb":261,"num_write":82,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:42.616845+0000","last_change":"2024-04-16T19:37:42.616948+0000","last_active":"2024-04-16T19:37:42.616845+0000","last_peered":"2024-04-16T19:37:42.616845+0000","last_clean":"2024-04-16T19:37:42.616845+0000","last_became_active":"2024-04-16T17:47:32.281679+0000","last_became_peered":"2024-04-16T17:47:32.281679+0000","last_unstale":"2024-04-16T19:37:42.616845+0000","last_undegraded":"2024-04-16T19:37:42.616845+0000","last_fullsized":"2024-04-16T19:37:42.616845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:42.616677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:42.616677+0000","last_clean_scrub_stamp":"2024-04-16T19:37:42.616677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:36.879525+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.465799+0000","last_change":"2024-04-16T19:37:35.465799+0000","last_active":"2024-04-16T19:37:35.465799+0000","last_peered":"2024-04-16T19:37:35.465799+0000","last_clean":"2024-04-16T19:37:35.465799+0000","last_became_active":"2024-04-16T17:47:23.269934+0000","last_became_peered":"2024-04-16T17:47:23.269934+0000","last_unstale":"2024-04-16T19:37:35.465799+0000","last_undegraded":"2024-04-16T19:37:35.465799+0000","last_fullsized":"2024-04-16T19:37:35.465799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.465488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.465488+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.465488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:37:45.779394+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.32","version":"53'14","reported_seq":227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.657838+0000","last_change":"2024-04-16T19:36:29.657923+0000","last_active":"2024-04-16T19:36:29.657838+0000","last_peered":"2024-04-16T19:36:29.657838+0000","last_clean":"2024-04-16T19:36:29.657838+0000","last_became_active":"2024-04-16T17:47:55.163048+0000","last_became_peered":"2024-04-16T17:47:55.163048+0000","last_unstale":"2024-04-16T19:36:29.657838+0000","last_undegraded":"2024-04-16T19:36:29.657838+0000","last_fullsized":"2024-04-16T19:36:29.657838+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:29.657764+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.657764+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:42.577097+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":280,"num_read_kb":183,"num_write":85,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.468103+0000","last_change":"2024-04-16T19:37:23.468202+0000","last_active":"2024-04-16T19:37:23.468103+0000","last_peered":"2024-04-16T19:37:23.468103+0000","last_clean":"2024-04-16T19:37:23.468103+0000","last_became_active":"2024-04-16T17:47:21.233573+0000","last_became_peered":"2024-04-16T17:47:21.233573+0000","last_unstale":"2024-04-16T19:37:23.468103+0000","last_undegraded":"2024-04-16T19:37:23.468103+0000","last_fullsized":"2024-04-16T19:37:23.468103+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.467898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.467898+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.467898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:49.472785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.493045+0000","last_change":"2024-04-16T19:37:13.493045+0000","last_active":"2024-04-16T19:37:13.493045+0000","last_peered":"2024-04-16T19:37:13.493045+0000","last_clean":"2024-04-16T19:37:13.493045+0000","last_became_active":"2024-04-16T17:47:23.253716+0000","last_became_peered":"2024-04-16T17:47:23.253716+0000","last_unstale":"2024-04-16T19:37:13.493045+0000","last_undegraded":"2024-04-16T19:37:13.493045+0000","last_fullsized":"2024-04-16T19:37:13.493045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.492847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.492847+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.492847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:18.553733+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"53'10","reported_seq":315,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.620843+0000","last_change":"2024-04-16T19:36:21.620914+0000","last_active":"2024-04-16T19:36:21.620843+0000","last_peered":"2024-04-16T19:36:21.620843+0000","last_clean":"2024-04-16T19:36:21.620843+0000","last_became_active":"2024-04-16T17:47:55.183669+0000","last_became_peered":"2024-04-16T17:47:55.183669+0000","last_unstale":"2024-04-16T19:36:21.620843+0000","last_undegraded":"2024-04-16T19:36:21.620843+0000","last_fullsized":"2024-04-16T19:36:21.620843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:21.620737+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.620737+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:28.047519+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":485,"num_read_kb":292,"num_write":55,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.545769+0000","last_change":"2024-04-16T19:37:29.545820+0000","last_active":"2024-04-16T19:37:29.545769+0000","last_peered":"2024-04-16T19:37:29.545769+0000","last_clean":"2024-04-16T19:37:29.545769+0000","last_became_active":"2024-04-16T17:47:23.246602+0000","last_became_peered":"2024-04-16T17:47:23.246602+0000","last_unstale":"2024-04-16T19:37:29.545769+0000","last_undegraded":"2024-04-16T19:37:29.545769+0000","last_fullsized":"2024-04-16T19:37:29.545769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.545634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.545634+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.545634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:15:05.482579+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.578900+0000","last_change":"2024-04-16T19:37:19.578952+0000","last_active":"2024-04-16T19:37:19.578900+0000","last_peered":"2024-04-16T19:37:19.578900+0000","last_clean":"2024-04-16T19:37:19.578900+0000","last_became_active":"2024-04-16T17:47:32.282012+0000","last_became_peered":"2024-04-16T17:47:32.282012+0000","last_unstale":"2024-04-16T19:37:19.578900+0000","last_undegraded":"2024-04-16T19:37:19.578900+0000","last_fullsized":"2024-04-16T19:37:19.578900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.578763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.578763+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.578763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:41.683801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"53'18","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.400030+0000","last_change":"2024-04-16T19:36:26.400030+0000","last_active":"2024-04-16T19:36:26.400030+0000","last_peered":"2024-04-16T19:36:26.400030+0000","last_clean":"2024-04-16T19:36:26.400030+0000","last_became_active":"2024-04-16T17:47:55.160350+0000","last_became_peered":"2024-04-16T17:47:55.160350+0000","last_unstale":"2024-04-16T19:36:26.400030+0000","last_undegraded":"2024-04-16T19:36:26.400030+0000","last_fullsized":"2024-04-16T19:36:26.400030+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.399732+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.399732+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:00.453174+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":91,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.498675+0000","last_change":"2024-04-16T19:37:22.498728+0000","last_active":"2024-04-16T19:37:22.498675+0000","last_peered":"2024-04-16T19:37:22.498675+0000","last_clean":"2024-04-16T19:37:22.498675+0000","last_became_active":"2024-04-16T17:47:23.254341+0000","last_became_peered":"2024-04-16T17:47:23.254341+0000","last_unstale":"2024-04-16T19:37:22.498675+0000","last_undegraded":"2024-04-16T19:37:22.498675+0000","last_fullsized":"2024-04-16T19:37:22.498675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.498536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.498536+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.498536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:15:49.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:41.653196+0000","last_change":"2024-04-16T19:37:41.653307+0000","last_active":"2024-04-16T19:37:41.653196+0000","last_peered":"2024-04-16T19:37:41.653196+0000","last_clean":"2024-04-16T19:37:41.653196+0000","last_became_active":"2024-04-16T17:47:21.247742+0000","last_became_peered":"2024-04-16T17:47:21.247742+0000","last_unstale":"2024-04-16T19:37:41.653196+0000","last_undegraded":"2024-04-16T19:37:41.653196+0000","last_fullsized":"2024-04-16T19:37:41.653196+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:41.653023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:41.653023+0000","last_clean_scrub_stamp":"2024-04-16T19:37:41.653023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:11.391727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"53'16","reported_seq":288,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.440103+0000","last_change":"2024-04-16T19:36:42.440209+0000","last_active":"2024-04-16T19:36:42.440103+0000","last_peered":"2024-04-16T19:36:42.440103+0000","last_clean":"2024-04-16T19:36:42.440103+0000","last_became_active":"2024-04-16T17:47:55.161397+0000","last_became_peered":"2024-04-16T17:47:55.161397+0000","last_unstale":"2024-04-16T19:36:42.440103+0000","last_undegraded":"2024-04-16T19:36:42.440103+0000","last_fullsized":"2024-04-16T19:36:42.440103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:42.439969+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.439969+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.619590+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":410,"num_read_kb":300,"num_write":107,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.474066+0000","last_change":"2024-04-16T19:37:29.474066+0000","last_active":"2024-04-16T19:37:29.474066+0000","last_peered":"2024-04-16T19:37:29.474066+0000","last_clean":"2024-04-16T19:37:29.474066+0000","last_became_active":"2024-04-16T17:47:23.265326+0000","last_became_peered":"2024-04-16T17:47:23.265326+0000","last_unstale":"2024-04-16T19:37:29.474066+0000","last_undegraded":"2024-04-16T19:37:29.474066+0000","last_fullsized":"2024-04-16T19:37:29.474066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.473871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.473871+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.473871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:59.956479+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:44.549709+0000","last_change":"2024-04-16T19:37:44.549760+0000","last_active":"2024-04-16T19:37:44.549709+0000","last_peered":"2024-04-16T19:37:44.549709+0000","last_clean":"2024-04-16T19:37:44.549709+0000","last_became_active":"2024-04-16T17:47:21.250414+0000","last_became_peered":"2024-04-16T17:47:21.250414+0000","last_unstale":"2024-04-16T19:37:44.549709+0000","last_undegraded":"2024-04-16T19:37:44.549709+0000","last_fullsized":"2024-04-16T19:37:44.549709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:44.549563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:44.549563+0000","last_clean_scrub_stamp":"2024-04-16T19:37:44.549563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:29:42.970639+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3e","version":"53'9","reported_seq":241,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.533226+0000","last_change":"2024-04-16T19:36:29.533377+0000","last_active":"2024-04-16T19:36:29.533226+0000","last_peered":"2024-04-16T19:36:29.533226+0000","last_clean":"2024-04-16T19:36:29.533226+0000","last_became_active":"2024-04-16T17:47:55.164219+0000","last_became_peered":"2024-04-16T17:47:55.164219+0000","last_unstale":"2024-04-16T19:36:29.533226+0000","last_undegraded":"2024-04-16T19:36:29.533226+0000","last_fullsized":"2024-04-16T19:36:29.533226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:29.533050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.533050+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:35:44.595573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":191,"num_write":41,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.441994+0000","last_change":"2024-04-16T19:37:11.442105+0000","last_active":"2024-04-16T19:37:11.441994+0000","last_peered":"2024-04-16T19:37:11.441994+0000","last_clean":"2024-04-16T19:37:11.441994+0000","last_became_active":"2024-04-16T17:47:23.253869+0000","last_became_peered":"2024-04-16T17:47:23.253869+0000","last_unstale":"2024-04-16T19:37:11.441994+0000","last_undegraded":"2024-04-16T19:37:11.441994+0000","last_fullsized":"2024-04-16T19:37:11.441994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.441824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.441824+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.441824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:43:01.723675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.597121+0000","last_change":"2024-04-16T19:37:14.597172+0000","last_active":"2024-04-16T19:37:14.597121+0000","last_peered":"2024-04-16T19:37:14.597121+0000","last_clean":"2024-04-16T19:37:14.597121+0000","last_became_active":"2024-04-16T17:47:32.283289+0000","last_became_peered":"2024-04-16T17:47:32.283289+0000","last_unstale":"2024-04-16T19:37:14.597121+0000","last_undegraded":"2024-04-16T19:37:14.597121+0000","last_fullsized":"2024-04-16T19:37:14.597121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.596983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.596983+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.596983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:21.782197+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.39","version":"53'10","reported_seq":203,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498276+0000","last_change":"2024-04-16T19:36:19.498276+0000","last_active":"2024-04-16T19:36:19.498276+0000","last_peered":"2024-04-16T19:36:19.498276+0000","last_clean":"2024-04-16T19:36:19.498276+0000","last_became_active":"2024-04-16T17:47:55.171291+0000","last_became_peered":"2024-04-16T17:47:55.171291+0000","last_unstale":"2024-04-16T19:36:19.498276+0000","last_undegraded":"2024-04-16T19:36:19.498276+0000","last_fullsized":"2024-04-16T19:36:19.498276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:19.498013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498013+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:24.571217+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":229,"num_read_kb":151,"num_write":59,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.430185+0000","last_change":"2024-04-16T19:37:07.430253+0000","last_active":"2024-04-16T19:37:07.430185+0000","last_peered":"2024-04-16T19:37:07.430185+0000","last_clean":"2024-04-16T19:37:07.430185+0000","last_became_active":"2024-04-16T17:47:23.264853+0000","last_became_peered":"2024-04-16T17:47:23.264853+0000","last_unstale":"2024-04-16T19:37:07.430185+0000","last_undegraded":"2024-04-16T19:37:07.430185+0000","last_fullsized":"2024-04-16T19:37:07.430185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.430037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.430037+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.430037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:58.189241+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.595934+0000","last_change":"2024-04-16T19:37:23.595986+0000","last_active":"2024-04-16T19:37:23.595934+0000","last_peered":"2024-04-16T19:37:23.595934+0000","last_clean":"2024-04-16T19:37:23.595934+0000","last_became_active":"2024-04-16T17:47:32.283646+0000","last_became_peered":"2024-04-16T17:47:32.283646+0000","last_unstale":"2024-04-16T19:37:23.595934+0000","last_undegraded":"2024-04-16T19:37:23.595934+0000","last_fullsized":"2024-04-16T19:37:23.595934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.595797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.595797+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.595797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:16.016867+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.38","version":"53'19","reported_seq":284,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.534337+0000","last_change":"2024-04-16T19:36:45.534337+0000","last_active":"2024-04-16T19:36:45.534337+0000","last_peered":"2024-04-16T19:36:45.534337+0000","last_clean":"2024-04-16T19:36:45.534337+0000","last_became_active":"2024-04-16T17:47:55.158479+0000","last_became_peered":"2024-04-16T17:47:55.158479+0000","last_unstale":"2024-04-16T19:36:45.534337+0000","last_undegraded":"2024-04-16T19:36:45.534337+0000","last_fullsized":"2024-04-16T19:36:45.534337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:45.534098+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.534098+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:29.053953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":420,"num_read_kb":258,"num_write":95,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.606734+0000","last_change":"2024-04-16T19:37:15.606785+0000","last_active":"2024-04-16T19:37:15.606734+0000","last_peered":"2024-04-16T19:37:15.606734+0000","last_clean":"2024-04-16T19:37:15.606734+0000","last_became_active":"2024-04-16T17:47:23.255295+0000","last_became_peered":"2024-04-16T17:47:23.255295+0000","last_unstale":"2024-04-16T19:37:15.606734+0000","last_undegraded":"2024-04-16T19:37:15.606734+0000","last_fullsized":"2024-04-16T19:37:15.606734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.606601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.606601+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.606601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:41.974274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.602747+0000","last_change":"2024-04-16T19:37:35.602747+0000","last_active":"2024-04-16T19:37:35.602747+0000","last_peered":"2024-04-16T19:37:35.602747+0000","last_clean":"2024-04-16T19:37:35.602747+0000","last_became_active":"2024-04-16T17:47:21.232375+0000","last_became_peered":"2024-04-16T17:47:21.232375+0000","last_unstale":"2024-04-16T19:37:35.602747+0000","last_undegraded":"2024-04-16T19:37:35.602747+0000","last_fullsized":"2024-04-16T19:37:35.602747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.602554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.602554+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.602554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:47:10.329538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:36.691355+0000","last_change":"2024-04-16T19:37:36.691411+0000","last_active":"2024-04-16T19:37:36.691355+0000","last_peered":"2024-04-16T19:37:36.691355+0000","last_clean":"2024-04-16T19:37:36.691355+0000","last_became_active":"2024-04-16T18:10:50.541805+0000","last_became_peered":"2024-04-16T18:10:50.541805+0000","last_unstale":"2024-04-16T19:37:36.691355+0000","last_undegraded":"2024-04-16T19:37:36.691355+0000","last_fullsized":"2024-04-16T19:37:36.691355+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:36.691197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:36.691197+0000","last_clean_scrub_stamp":"2024-04-16T19:37:36.691197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:57.305417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"53'126","reported_seq":1390,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.649445+0000","last_change":"2024-04-16T19:37:16.649511+0000","last_active":"2024-04-16T19:37:16.649445+0000","last_peered":"2024-04-16T19:37:16.649445+0000","last_clean":"2024-04-16T19:37:16.649445+0000","last_became_active":"2024-04-16T17:48:34.336562+0000","last_became_peered":"2024-04-16T17:48:34.336562+0000","last_unstale":"2024-04-16T19:37:16.649445+0000","last_undegraded":"2024-04-16T19:37:16.649445+0000","last_fullsized":"2024-04-16T19:37:16.649445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2024-04-16T19:37:16.649314+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2024-04-16T19:37:16.649314+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.649314+0000","objects_scrubbed":29,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:56.902654+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2163,"num_read_kb":2130,"num_write":148,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7267,"num_omap_keys":30,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.434783+0000","last_change":"2024-04-16T19:36:56.434907+0000","last_active":"2024-04-16T19:36:56.434783+0000","last_peered":"2024-04-16T19:36:56.434783+0000","last_clean":"2024-04-16T19:36:56.434783+0000","last_became_active":"2024-04-16T17:48:27.308328+0000","last_became_peered":"2024-04-16T17:48:27.308328+0000","last_unstale":"2024-04-16T19:36:56.434783+0000","last_undegraded":"2024-04-16T19:36:56.434783+0000","last_fullsized":"2024-04-16T19:36:56.434783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:56.434608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:56.434608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.434608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:12.721346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.645164+0000","last_change":"2024-04-16T19:36:51.645240+0000","last_active":"2024-04-16T19:36:51.645164+0000","last_peered":"2024-04-16T19:36:51.645164+0000","last_clean":"2024-04-16T19:36:51.645164+0000","last_became_active":"2024-04-16T17:48:01.265649+0000","last_became_peered":"2024-04-16T17:48:01.265649+0000","last_unstale":"2024-04-16T19:36:51.645164+0000","last_undegraded":"2024-04-16T19:36:51.645164+0000","last_fullsized":"2024-04-16T19:36:51.645164+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:51.645030+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.645030+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:42.615404+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":84,"num_write":27,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.617380+0000","last_change":"2024-04-16T19:36:26.617439+0000","last_active":"2024-04-16T19:36:26.617380+0000","last_peered":"2024-04-16T19:36:26.617380+0000","last_clean":"2024-04-16T19:36:26.617380+0000","last_became_active":"2024-04-16T17:47:59.178876+0000","last_became_peered":"2024-04-16T17:47:59.178876+0000","last_unstale":"2024-04-16T19:36:26.617380+0000","last_undegraded":"2024-04-16T19:36:26.617380+0000","last_fullsized":"2024-04-16T19:36:26.617380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.617211+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.617211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:50.470545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"53'563","reported_seq":824,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.471001+0000","last_change":"2024-04-16T19:36:20.471131+0000","last_active":"2024-04-16T19:36:20.471001+0000","last_peered":"2024-04-16T19:36:20.471001+0000","last_clean":"2024-04-16T19:36:20.471001+0000","last_became_active":"2024-04-16T17:47:57.176855+0000","last_became_peered":"2024-04-16T17:47:57.176855+0000","last_unstale":"2024-04-16T19:36:20.471001+0000","last_undegraded":"2024-04-16T19:36:20.471001+0000","last_fullsized":"2024-04-16T19:36:20.471001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:20.470861+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.470861+0000","objects_scrubbed":16,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:34.399674+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":700,"num_read_kb":692,"num_write":954,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71932,"num_omap_keys":314,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"53'14","reported_seq":223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.561790+0000","last_change":"2024-04-16T19:36:25.561790+0000","last_active":"2024-04-16T19:36:25.561790+0000","last_peered":"2024-04-16T19:36:25.561790+0000","last_clean":"2024-04-16T19:36:25.561790+0000","last_became_active":"2024-04-16T17:47:55.165467+0000","last_became_peered":"2024-04-16T17:47:55.165467+0000","last_unstale":"2024-04-16T19:36:25.561790+0000","last_undegraded":"2024-04-16T19:36:25.561790+0000","last_fullsized":"2024-04-16T19:36:25.561790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:25.561550+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.561550+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:02:57.257953+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":269,"num_read_kb":176,"num_write":74,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.527133+0000","last_change":"2024-04-16T19:36:50.527184+0000","last_active":"2024-04-16T19:36:50.527133+0000","last_peered":"2024-04-16T19:36:50.527133+0000","last_clean":"2024-04-16T19:36:50.527133+0000","last_became_active":"2024-04-16T17:47:39.325600+0000","last_became_peered":"2024-04-16T17:47:39.325600+0000","last_unstale":"2024-04-16T19:36:50.527133+0000","last_undegraded":"2024-04-16T19:36:50.527133+0000","last_fullsized":"2024-04-16T19:36:50.527133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.527001+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.527001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:19.398870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.596329+0000","last_change":"2024-04-16T19:37:18.596329+0000","last_active":"2024-04-16T19:37:18.596329+0000","last_peered":"2024-04-16T19:37:18.596329+0000","last_clean":"2024-04-16T19:37:18.596329+0000","last_became_active":"2024-04-16T17:47:23.257711+0000","last_became_peered":"2024-04-16T17:47:23.257711+0000","last_unstale":"2024-04-16T19:37:18.596329+0000","last_undegraded":"2024-04-16T19:37:18.596329+0000","last_fullsized":"2024-04-16T19:37:18.596329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.596131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.596131+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.596131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:47.551456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.595318+0000","last_change":"2024-04-16T19:37:21.595369+0000","last_active":"2024-04-16T19:37:21.595318+0000","last_peered":"2024-04-16T19:37:21.595318+0000","last_clean":"2024-04-16T19:37:21.595318+0000","last_became_active":"2024-04-16T17:47:32.282652+0000","last_became_peered":"2024-04-16T17:47:32.282652+0000","last_unstale":"2024-04-16T19:37:21.595318+0000","last_undegraded":"2024-04-16T19:37:21.595318+0000","last_fullsized":"2024-04-16T19:37:21.595318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.595177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.595177+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.595177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:58:43.035164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"53'4","reported_seq":779,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.342632+0000","last_change":"2024-04-16T19:37:14.342695+0000","last_active":"2024-04-16T19:37:14.342632+0000","last_peered":"2024-04-16T19:37:14.342632+0000","last_clean":"2024-04-16T19:37:14.342632+0000","last_became_active":"2024-04-16T17:47:33.302437+0000","last_became_peered":"2024-04-16T17:47:33.302437+0000","last_unstale":"2024-04-16T19:37:14.342632+0000","last_undegraded":"2024-04-16T19:37:14.342632+0000","last_fullsized":"2024-04-16T19:37:14.342632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:14.342493+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:14.342493+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.342493+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:34.674033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":906,"num_read_kb":904,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.673841+0000","last_change":"2024-04-16T19:37:10.673931+0000","last_active":"2024-04-16T19:37:10.673841+0000","last_peered":"2024-04-16T19:37:10.673841+0000","last_clean":"2024-04-16T19:37:10.673841+0000","last_became_active":"2024-04-16T17:47:35.314194+0000","last_became_peered":"2024-04-16T17:47:35.314194+0000","last_unstale":"2024-04-16T19:37:10.673841+0000","last_undegraded":"2024-04-16T19:37:10.673841+0000","last_fullsized":"2024-04-16T19:37:10.673841+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:10.673712+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:10.673712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.673712+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:39.298894+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.400379+0000","last_change":"2024-04-16T19:36:48.400438+0000","last_active":"2024-04-16T19:36:48.400379+0000","last_peered":"2024-04-16T19:36:48.400379+0000","last_clean":"2024-04-16T19:36:48.400379+0000","last_became_active":"2024-04-16T17:47:37.332244+0000","last_became_peered":"2024-04-16T17:47:37.332244+0000","last_unstale":"2024-04-16T19:36:48.400379+0000","last_undegraded":"2024-04-16T19:36:48.400379+0000","last_fullsized":"2024-04-16T19:36:48.400379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.400232+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.400232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:53.461118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.561771+0000","last_change":"2024-04-16T19:37:33.561877+0000","last_active":"2024-04-16T19:37:33.561771+0000","last_peered":"2024-04-16T19:37:33.561771+0000","last_clean":"2024-04-16T19:37:33.561771+0000","last_became_active":"2024-04-16T18:10:50.554714+0000","last_became_peered":"2024-04-16T18:10:50.554714+0000","last_unstale":"2024-04-16T19:37:33.561771+0000","last_undegraded":"2024-04-16T19:37:33.561771+0000","last_fullsized":"2024-04-16T19:37:33.561771+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.561603+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:22.551773+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"53'113","reported_seq":1461,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.578659+0000","last_change":"2024-04-16T19:37:01.578802+0000","last_active":"2024-04-16T19:37:01.578659+0000","last_peered":"2024-04-16T19:37:01.578659+0000","last_clean":"2024-04-16T19:37:01.578659+0000","last_became_active":"2024-04-16T17:48:34.318608+0000","last_became_peered":"2024-04-16T17:48:34.318608+0000","last_unstale":"2024-04-16T19:37:01.578659+0000","last_undegraded":"2024-04-16T19:37:01.578659+0000","last_fullsized":"2024-04-16T19:37:01.578659+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2024-04-16T19:37:01.578449+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2024-04-16T19:37:01.578449+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.578449+0000","objects_scrubbed":30,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:34.021657+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2240,"num_read_kb":2210,"num_write":124,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6547,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.437635+0000","last_change":"2024-04-16T19:36:59.437695+0000","last_active":"2024-04-16T19:36:59.437635+0000","last_peered":"2024-04-16T19:36:59.437635+0000","last_clean":"2024-04-16T19:36:59.437635+0000","last_became_active":"2024-04-16T17:48:27.303350+0000","last_became_peered":"2024-04-16T17:48:27.303350+0000","last_unstale":"2024-04-16T19:36:59.437635+0000","last_undegraded":"2024-04-16T19:36:59.437635+0000","last_fullsized":"2024-04-16T19:36:59.437635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.437461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.437461+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.437461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:15.724199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1d","version":"53'9","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.654127+0000","last_change":"2024-04-16T19:36:50.654208+0000","last_active":"2024-04-16T19:36:50.654127+0000","last_peered":"2024-04-16T19:36:50.654127+0000","last_clean":"2024-04-16T19:36:50.654127+0000","last_became_active":"2024-04-16T17:48:01.266030+0000","last_became_peered":"2024-04-16T17:48:01.266030+0000","last_unstale":"2024-04-16T19:36:50.654127+0000","last_undegraded":"2024-04-16T19:36:50.654127+0000","last_fullsized":"2024-04-16T19:36:50.654127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'9","last_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_deep_scrub":"53'9","last_deep_scrub_stamp":"2024-04-16T19:36:50.653994+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.653994+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:30.491092+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":13,"num_write":10,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.479488+0000","last_change":"2024-04-16T19:36:33.479590+0000","last_active":"2024-04-16T19:36:33.479488+0000","last_peered":"2024-04-16T19:36:33.479488+0000","last_clean":"2024-04-16T19:36:33.479488+0000","last_became_active":"2024-04-16T17:47:59.189948+0000","last_became_peered":"2024-04-16T17:47:59.189948+0000","last_unstale":"2024-04-16T19:36:33.479488+0000","last_undegraded":"2024-04-16T19:36:33.479488+0000","last_fullsized":"2024-04-16T19:36:33.479488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.479280+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.479280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:24.835481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"53'7349","reported_seq":20627,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.602403+0000","last_change":"2024-04-16T19:36:20.602529+0000","last_active":"2024-04-16T19:36:20.602403+0000","last_peered":"2024-04-16T19:36:20.602403+0000","last_clean":"2024-04-16T19:36:20.602403+0000","last_became_active":"2024-04-16T17:47:57.179121+0000","last_became_peered":"2024-04-16T17:47:57.179121+0000","last_unstale":"2024-04-16T19:36:20.602403+0000","last_undegraded":"2024-04-16T19:36:20.602403+0000","last_fullsized":"2024-04-16T19:36:20.602403+0000","mapping_epoch":34,"log_start":"53'5600","ondisk_log_start":"53'5600","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7349","last_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_deep_scrub":"53'7349","last_deep_scrub_stamp":"2024-04-16T19:36:20.602264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.602264+0000","objects_scrubbed":23,"log_size":1749,"log_dups_size":3000,"ondisk_log_size":1749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:12.702919+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":645,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33553,"num_read_kb":33453,"num_write":7894,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72188,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"53'30","reported_seq":375,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.493531+0000","last_change":"2024-04-16T19:36:32.493531+0000","last_active":"2024-04-16T19:36:32.493531+0000","last_peered":"2024-04-16T19:36:32.493531+0000","last_clean":"2024-04-16T19:36:32.493531+0000","last_became_active":"2024-04-16T17:47:55.177888+0000","last_became_peered":"2024-04-16T17:47:55.177888+0000","last_unstale":"2024-04-16T19:36:32.493531+0000","last_undegraded":"2024-04-16T19:36:32.493531+0000","last_fullsized":"2024-04-16T19:36:32.493531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'30","last_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_deep_scrub":"53'30","last_deep_scrub_stamp":"2024-04-16T19:36:32.493294+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.493294+0000","objects_scrubbed":11,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:58.339768+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":613,"num_read_kb":375,"num_write":159,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.426772+0000","last_change":"2024-04-16T19:36:46.426772+0000","last_active":"2024-04-16T19:36:46.426772+0000","last_peered":"2024-04-16T19:36:46.426772+0000","last_clean":"2024-04-16T19:36:46.426772+0000","last_became_active":"2024-04-16T17:47:39.332420+0000","last_became_peered":"2024-04-16T17:47:39.332420+0000","last_unstale":"2024-04-16T19:36:46.426772+0000","last_undegraded":"2024-04-16T19:36:46.426772+0000","last_fullsized":"2024-04-16T19:36:46.426772+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.426574+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.426574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.708425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.500576+0000","last_change":"2024-04-16T19:37:33.500650+0000","last_active":"2024-04-16T19:37:33.500576+0000","last_peered":"2024-04-16T19:37:33.500576+0000","last_clean":"2024-04-16T19:37:33.500576+0000","last_became_active":"2024-04-16T17:47:23.268318+0000","last_became_peered":"2024-04-16T17:47:23.268318+0000","last_unstale":"2024-04-16T19:37:33.500576+0000","last_undegraded":"2024-04-16T19:37:33.500576+0000","last_fullsized":"2024-04-16T19:37:33.500576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.500439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.500439+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.500439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:15:01.340149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.728481+0000","last_change":"2024-04-16T19:37:30.728533+0000","last_active":"2024-04-16T19:37:30.728481+0000","last_peered":"2024-04-16T19:37:30.728481+0000","last_clean":"2024-04-16T19:37:30.728481+0000","last_became_active":"2024-04-16T17:47:21.244783+0000","last_became_peered":"2024-04-16T17:47:21.244783+0000","last_unstale":"2024-04-16T19:37:30.728481+0000","last_undegraded":"2024-04-16T19:37:30.728481+0000","last_fullsized":"2024-04-16T19:37:30.728481+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.728346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.728346+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.728346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:59:13.778947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"53'7","reported_seq":371,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.684265+0000","last_change":"2024-04-16T19:37:15.684265+0000","last_active":"2024-04-16T19:37:15.684265+0000","last_peered":"2024-04-16T19:37:15.684265+0000","last_clean":"2024-04-16T19:37:15.684265+0000","last_became_active":"2024-04-16T17:47:33.310116+0000","last_became_peered":"2024-04-16T17:47:33.310116+0000","last_unstale":"2024-04-16T19:37:15.684265+0000","last_undegraded":"2024-04-16T19:37:15.684265+0000","last_fullsized":"2024-04-16T19:37:15.684265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:15.684059+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:15.684059+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.684059+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:55.937399+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2768,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":480,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"53'75","reported_seq":229,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.595038+0000","last_change":"2024-04-16T19:37:02.595110+0000","last_active":"2024-04-16T19:37:02.595038+0000","last_peered":"2024-04-16T19:37:02.595038+0000","last_clean":"2024-04-16T19:37:02.595038+0000","last_became_active":"2024-04-16T17:47:35.305069+0000","last_became_peered":"2024-04-16T17:47:35.305069+0000","last_unstale":"2024-04-16T19:37:02.595038+0000","last_undegraded":"2024-04-16T19:37:02.595038+0000","last_fullsized":"2024-04-16T19:37:02.595038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'75","last_scrub_stamp":"2024-04-16T19:37:02.594894+0000","last_deep_scrub":"53'75","last_deep_scrub_stamp":"2024-04-16T19:37:02.594894+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.594894+0000","objects_scrubbed":3,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:07.655780+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":105,"num_write":72,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.580869+0000","last_change":"2024-04-16T19:36:57.580927+0000","last_active":"2024-04-16T19:36:57.580869+0000","last_peered":"2024-04-16T19:36:57.580869+0000","last_clean":"2024-04-16T19:36:57.580869+0000","last_became_active":"2024-04-16T17:47:37.323773+0000","last_became_peered":"2024-04-16T17:47:37.323773+0000","last_unstale":"2024-04-16T19:36:57.580869+0000","last_undegraded":"2024-04-16T19:36:57.580869+0000","last_fullsized":"2024-04-16T19:36:57.580869+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:57.580729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:57.580729+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.580729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:16.681259+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:37.524793+0000","last_change":"2024-04-16T19:37:37.524906+0000","last_active":"2024-04-16T19:37:37.524793+0000","last_peered":"2024-04-16T19:37:37.524793+0000","last_clean":"2024-04-16T19:37:37.524793+0000","last_became_active":"2024-04-16T18:10:50.564585+0000","last_became_peered":"2024-04-16T18:10:50.564585+0000","last_unstale":"2024-04-16T19:37:37.524793+0000","last_undegraded":"2024-04-16T19:37:37.524793+0000","last_fullsized":"2024-04-16T19:37:37.524793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:37.524115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:37.524115+0000","last_clean_scrub_stamp":"2024-04-16T19:37:37.524115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:25.406230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"53'84","reported_seq":914,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.739730+0000","last_change":"2024-04-16T19:37:13.739802+0000","last_active":"2024-04-16T19:37:13.739730+0000","last_peered":"2024-04-16T19:37:13.739730+0000","last_clean":"2024-04-16T19:37:13.739730+0000","last_became_active":"2024-04-16T17:48:34.333320+0000","last_became_peered":"2024-04-16T17:48:34.333320+0000","last_unstale":"2024-04-16T19:37:13.739730+0000","last_undegraded":"2024-04-16T19:37:13.739730+0000","last_fullsized":"2024-04-16T19:37:13.739730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:13.739588+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:13.739588+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.739588+0000","objects_scrubbed":28,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:54.664404+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1351,"num_read_kb":1337,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4355,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.671347+0000","last_change":"2024-04-16T19:36:34.671347+0000","last_active":"2024-04-16T19:36:34.671347+0000","last_peered":"2024-04-16T19:36:34.671347+0000","last_clean":"2024-04-16T19:36:34.671347+0000","last_became_active":"2024-04-16T17:48:27.297705+0000","last_became_peered":"2024-04-16T17:48:27.297705+0000","last_unstale":"2024-04-16T19:36:34.671347+0000","last_undegraded":"2024-04-16T19:36:34.671347+0000","last_fullsized":"2024-04-16T19:36:34.671347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.671050+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.671050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:41.097832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"53'20","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.631537+0000","last_change":"2024-04-16T19:36:45.631650+0000","last_active":"2024-04-16T19:36:45.631537+0000","last_peered":"2024-04-16T19:36:45.631537+0000","last_clean":"2024-04-16T19:36:45.631537+0000","last_became_active":"2024-04-16T17:48:01.237305+0000","last_became_peered":"2024-04-16T17:48:01.237305+0000","last_unstale":"2024-04-16T19:36:45.631537+0000","last_undegraded":"2024-04-16T19:36:45.631537+0000","last_fullsized":"2024-04-16T19:36:45.631537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:45.631366+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.631366+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:37.106553+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":58,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.586170+0000","last_change":"2024-04-16T19:36:37.586170+0000","last_active":"2024-04-16T19:36:37.586170+0000","last_peered":"2024-04-16T19:36:37.586170+0000","last_clean":"2024-04-16T19:36:37.586170+0000","last_became_active":"2024-04-16T17:47:59.192292+0000","last_became_peered":"2024-04-16T17:47:59.192292+0000","last_unstale":"2024-04-16T19:36:37.586170+0000","last_undegraded":"2024-04-16T19:36:37.586170+0000","last_fullsized":"2024-04-16T19:36:37.586170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.585961+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.585961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:02.979747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"53'3625","reported_seq":7230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.651412+0000","last_change":"2024-04-16T19:36:17.651412+0000","last_active":"2024-04-16T19:36:17.651412+0000","last_peered":"2024-04-16T19:36:17.651412+0000","last_clean":"2024-04-16T19:36:17.651412+0000","last_became_active":"2024-04-16T17:47:57.174075+0000","last_became_peered":"2024-04-16T17:47:57.174075+0000","last_unstale":"2024-04-16T19:36:17.651412+0000","last_undegraded":"2024-04-16T19:36:17.651412+0000","last_fullsized":"2024-04-16T19:36:17.651412+0000","mapping_epoch":34,"log_start":"53'1800","ondisk_log_start":"53'1800","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'3625","last_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_deep_scrub":"53'3625","last_deep_scrub_stamp":"2024-04-16T19:36:17.651328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.651328+0000","objects_scrubbed":30,"log_size":1825,"log_dups_size":1800,"ondisk_log_size":1825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:17:26.898814+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":7171,"num_read_kb":7074,"num_write":293,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'18","reported_seq":255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.465591+0000","last_change":"2024-04-16T19:36:17.465591+0000","last_active":"2024-04-16T19:36:17.465591+0000","last_peered":"2024-04-16T19:36:17.465591+0000","last_clean":"2024-04-16T19:36:17.465591+0000","last_became_active":"2024-04-16T17:47:55.176760+0000","last_became_peered":"2024-04-16T17:47:55.176760+0000","last_unstale":"2024-04-16T19:36:17.465591+0000","last_undegraded":"2024-04-16T19:36:17.465591+0000","last_fullsized":"2024-04-16T19:36:17.465591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:17.465360+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.465360+0000","objects_scrubbed":9,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:23.892141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":347,"num_read_kb":225,"num_write":103,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.615812+0000","last_change":"2024-04-16T19:36:38.615812+0000","last_active":"2024-04-16T19:36:38.615812+0000","last_peered":"2024-04-16T19:36:38.615812+0000","last_clean":"2024-04-16T19:36:38.615812+0000","last_became_active":"2024-04-16T17:47:39.333436+0000","last_became_peered":"2024-04-16T17:47:39.333436+0000","last_unstale":"2024-04-16T19:36:38.615812+0000","last_undegraded":"2024-04-16T19:36:38.615812+0000","last_fullsized":"2024-04-16T19:36:38.615812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.615619+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.615619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:29.374824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.635237+0000","last_change":"2024-04-16T19:37:24.635287+0000","last_active":"2024-04-16T19:37:24.635237+0000","last_peered":"2024-04-16T19:37:24.635237+0000","last_clean":"2024-04-16T19:37:24.635237+0000","last_became_active":"2024-04-16T17:47:23.256312+0000","last_became_peered":"2024-04-16T17:47:23.256312+0000","last_unstale":"2024-04-16T19:37:24.635237+0000","last_undegraded":"2024-04-16T19:37:24.635237+0000","last_fullsized":"2024-04-16T19:37:24.635237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.635106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.635106+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.635106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:01.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.693226+0000","last_change":"2024-04-16T19:37:23.693280+0000","last_active":"2024-04-16T19:37:23.693226+0000","last_peered":"2024-04-16T19:37:23.693226+0000","last_clean":"2024-04-16T19:37:23.693226+0000","last_became_active":"2024-04-16T17:47:21.252785+0000","last_became_peered":"2024-04-16T17:47:21.252785+0000","last_unstale":"2024-04-16T19:37:23.693226+0000","last_undegraded":"2024-04-16T19:37:23.693226+0000","last_fullsized":"2024-04-16T19:37:23.693226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.693084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.693084+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.693084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:49.613545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"53'13","reported_seq":617,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.385910+0000","last_change":"2024-04-16T19:37:08.386038+0000","last_active":"2024-04-16T19:37:08.385910+0000","last_peered":"2024-04-16T19:37:08.385910+0000","last_clean":"2024-04-16T19:37:08.385910+0000","last_became_active":"2024-04-16T17:47:33.299533+0000","last_became_peered":"2024-04-16T17:47:33.299533+0000","last_unstale":"2024-04-16T19:37:08.385910+0000","last_undegraded":"2024-04-16T19:37:08.385910+0000","last_fullsized":"2024-04-16T19:37:08.385910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:08.385737+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:08.385737+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.385737+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:45.642460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":589,"num_write":22,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"53'125","reported_seq":305,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.417529+0000","last_change":"2024-04-16T19:36:55.417529+0000","last_active":"2024-04-16T19:36:55.417529+0000","last_peered":"2024-04-16T19:36:55.417529+0000","last_clean":"2024-04-16T19:36:55.417529+0000","last_became_active":"2024-04-16T17:47:35.313910+0000","last_became_peered":"2024-04-16T17:47:35.313910+0000","last_unstale":"2024-04-16T19:36:55.417529+0000","last_undegraded":"2024-04-16T19:36:55.417529+0000","last_fullsized":"2024-04-16T19:36:55.417529+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:55.417311+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.417311+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:25.523059+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"53'13","reported_seq":1343,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.351749+0000","last_change":"2024-04-16T19:36:45.351811+0000","last_active":"2024-04-16T19:36:45.351749+0000","last_peered":"2024-04-16T19:36:45.351749+0000","last_clean":"2024-04-16T19:36:45.351749+0000","last_became_active":"2024-04-16T17:47:37.332366+0000","last_became_peered":"2024-04-16T17:47:37.332366+0000","last_unstale":"2024-04-16T19:36:45.351749+0000","last_undegraded":"2024-04-16T19:36:45.351749+0000","last_fullsized":"2024-04-16T19:36:45.351749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:45.351601+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.351601+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:50:50.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:39.761720+0000","last_change":"2024-04-16T19:37:39.761770+0000","last_active":"2024-04-16T19:37:39.761720+0000","last_peered":"2024-04-16T19:37:39.761720+0000","last_clean":"2024-04-16T19:37:39.761720+0000","last_became_active":"2024-04-16T18:10:50.558204+0000","last_became_peered":"2024-04-16T18:10:50.558204+0000","last_unstale":"2024-04-16T19:37:39.761720+0000","last_undegraded":"2024-04-16T19:37:39.761720+0000","last_fullsized":"2024-04-16T19:37:39.761720+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:39.761584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:39.761584+0000","last_clean_scrub_stamp":"2024-04-16T19:37:39.761584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:24.862868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"53'84","reported_seq":771,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.627099+0000","last_change":"2024-04-16T19:37:17.627212+0000","last_active":"2024-04-16T19:37:17.627099+0000","last_peered":"2024-04-16T19:37:17.627099+0000","last_clean":"2024-04-16T19:37:17.627099+0000","last_became_active":"2024-04-16T17:48:34.343110+0000","last_became_peered":"2024-04-16T17:48:34.343110+0000","last_unstale":"2024-04-16T19:37:17.627099+0000","last_undegraded":"2024-04-16T19:37:17.627099+0000","last_fullsized":"2024-04-16T19:37:17.627099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:17.626928+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:17.626928+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.626928+0000","objects_scrubbed":25,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:42.762558+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1017,"num_read_kb":1056,"num_write":81,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3553,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.671455+0000","last_change":"2024-04-16T19:36:35.671455+0000","last_active":"2024-04-16T19:36:35.671455+0000","last_peered":"2024-04-16T19:36:35.671455+0000","last_clean":"2024-04-16T19:36:35.671455+0000","last_became_active":"2024-04-16T17:48:27.296690+0000","last_became_peered":"2024-04-16T17:48:27.296690+0000","last_unstale":"2024-04-16T19:36:35.671455+0000","last_undegraded":"2024-04-16T19:36:35.671455+0000","last_fullsized":"2024-04-16T19:36:35.671455+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.671258+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.671258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:24:03.249930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"53'21","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.435794+0000","last_change":"2024-04-16T19:36:50.435867+0000","last_active":"2024-04-16T19:36:50.435794+0000","last_peered":"2024-04-16T19:36:50.435794+0000","last_clean":"2024-04-16T19:36:50.435794+0000","last_became_active":"2024-04-16T17:48:01.251499+0000","last_became_peered":"2024-04-16T17:48:01.251499+0000","last_unstale":"2024-04-16T19:36:50.435794+0000","last_undegraded":"2024-04-16T19:36:50.435794+0000","last_fullsized":"2024-04-16T19:36:50.435794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:50.435653+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.435653+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:16.012260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":32,"num_write":74,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.469677+0000","last_change":"2024-04-16T19:36:39.469677+0000","last_active":"2024-04-16T19:36:39.469677+0000","last_peered":"2024-04-16T19:36:39.469677+0000","last_clean":"2024-04-16T19:36:39.469677+0000","last_became_active":"2024-04-16T17:47:59.182439+0000","last_became_peered":"2024-04-16T17:47:59.182439+0000","last_unstale":"2024-04-16T19:36:39.469677+0000","last_undegraded":"2024-04-16T19:36:39.469677+0000","last_fullsized":"2024-04-16T19:36:39.469677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.469488+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.469488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:42.661779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"53'408","reported_seq":769,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.522794+0000","last_change":"2024-04-16T19:36:31.522794+0000","last_active":"2024-04-16T19:36:31.522794+0000","last_peered":"2024-04-16T19:36:31.522794+0000","last_clean":"2024-04-16T19:36:31.522794+0000","last_became_active":"2024-04-16T17:47:57.175007+0000","last_became_peered":"2024-04-16T17:47:57.175007+0000","last_unstale":"2024-04-16T19:36:31.522794+0000","last_undegraded":"2024-04-16T19:36:31.522794+0000","last_fullsized":"2024-04-16T19:36:31.522794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'408","last_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_deep_scrub":"53'408","last_deep_scrub_stamp":"2024-04-16T19:36:31.522481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.522481+0000","objects_scrubbed":23,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:59.101153+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":649,"num_read_kb":638,"num_write":417,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'12","reported_seq":189,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:17.542045+0000","last_change":"2024-04-16T19:36:17.542045+0000","last_active":"2024-04-16T19:36:17.542045+0000","last_peered":"2024-04-16T19:36:17.542045+0000","last_clean":"2024-04-16T19:36:17.542045+0000","last_became_active":"2024-04-16T17:47:55.161237+0000","last_became_peered":"2024-04-16T17:47:55.161237+0000","last_unstale":"2024-04-16T19:36:17.542045+0000","last_undegraded":"2024-04-16T19:36:17.542045+0000","last_fullsized":"2024-04-16T19:36:17.542045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:17.541913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:17.541913+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:08:24.087701+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":147,"num_write":75,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.579767+0000","last_change":"2024-04-16T19:36:47.579767+0000","last_active":"2024-04-16T19:36:47.579767+0000","last_peered":"2024-04-16T19:36:47.579767+0000","last_clean":"2024-04-16T19:36:47.579767+0000","last_became_active":"2024-04-16T17:47:39.699084+0000","last_became_peered":"2024-04-16T17:47:39.699084+0000","last_unstale":"2024-04-16T19:36:47.579767+0000","last_undegraded":"2024-04-16T19:36:47.579767+0000","last_fullsized":"2024-04-16T19:36:47.579767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.579576+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.579576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:56.945905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.662189+0000","last_change":"2024-04-16T19:37:28.662247+0000","last_active":"2024-04-16T19:37:28.662189+0000","last_peered":"2024-04-16T19:37:28.662189+0000","last_clean":"2024-04-16T19:37:28.662189+0000","last_became_active":"2024-04-16T17:47:23.254442+0000","last_became_peered":"2024-04-16T17:47:23.254442+0000","last_unstale":"2024-04-16T19:37:28.662189+0000","last_undegraded":"2024-04-16T19:37:28.662189+0000","last_fullsized":"2024-04-16T19:37:28.662189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.662050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.662050+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.662050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:01:18.105359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:42.501220+0000","last_change":"2024-04-16T19:37:42.501272+0000","last_active":"2024-04-16T19:37:42.501220+0000","last_peered":"2024-04-16T19:37:42.501220+0000","last_clean":"2024-04-16T19:37:42.501220+0000","last_became_active":"2024-04-16T17:47:21.235745+0000","last_became_peered":"2024-04-16T17:47:21.235745+0000","last_unstale":"2024-04-16T19:37:42.501220+0000","last_undegraded":"2024-04-16T19:37:42.501220+0000","last_fullsized":"2024-04-16T19:37:42.501220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:42.501082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:42.501082+0000","last_clean_scrub_stamp":"2024-04-16T19:37:42.501082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:55.525792+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.516275+0000","last_change":"2024-04-16T19:37:13.516327+0000","last_active":"2024-04-16T19:37:13.516275+0000","last_peered":"2024-04-16T19:37:13.516275+0000","last_clean":"2024-04-16T19:37:13.516275+0000","last_became_active":"2024-04-16T17:47:33.304636+0000","last_became_peered":"2024-04-16T17:47:33.304636+0000","last_unstale":"2024-04-16T19:37:13.516275+0000","last_undegraded":"2024-04-16T19:37:13.516275+0000","last_fullsized":"2024-04-16T19:37:13.516275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.516141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.516141+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.516141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:59.747526+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.526949+0000","last_change":"2024-04-16T19:37:09.527017+0000","last_active":"2024-04-16T19:37:09.526949+0000","last_peered":"2024-04-16T19:37:09.526949+0000","last_clean":"2024-04-16T19:37:09.526949+0000","last_became_active":"2024-04-16T17:47:35.314147+0000","last_became_peered":"2024-04-16T17:47:35.314147+0000","last_unstale":"2024-04-16T19:37:09.526949+0000","last_undegraded":"2024-04-16T19:37:09.526949+0000","last_fullsized":"2024-04-16T19:37:09.526949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:09.526805+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:09.526805+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.526805+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:03.234997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"53'13","reported_seq":1345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.649962+0000","last_change":"2024-04-16T19:36:52.649962+0000","last_active":"2024-04-16T19:36:52.649962+0000","last_peered":"2024-04-16T19:36:52.649962+0000","last_clean":"2024-04-16T19:36:52.649962+0000","last_became_active":"2024-04-16T17:47:37.333593+0000","last_became_peered":"2024-04-16T17:47:37.333593+0000","last_unstale":"2024-04-16T19:36:52.649962+0000","last_undegraded":"2024-04-16T19:36:52.649962+0000","last_fullsized":"2024-04-16T19:36:52.649962+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:52.649763+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.649763+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.928938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.503286+0000","last_change":"2024-04-16T19:37:33.503338+0000","last_active":"2024-04-16T19:37:33.503286+0000","last_peered":"2024-04-16T19:37:33.503286+0000","last_clean":"2024-04-16T19:37:33.503286+0000","last_became_active":"2024-04-16T18:10:50.547757+0000","last_became_peered":"2024-04-16T18:10:50.547757+0000","last_unstale":"2024-04-16T19:37:33.503286+0000","last_undegraded":"2024-04-16T19:37:33.503286+0000","last_fullsized":"2024-04-16T19:37:33.503286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.503150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.503150+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.503150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:33:13.756490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"53'200","reported_seq":1900,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.601906+0000","last_change":"2024-04-16T19:37:08.601960+0000","last_active":"2024-04-16T19:37:08.601906+0000","last_peered":"2024-04-16T19:37:08.601906+0000","last_clean":"2024-04-16T19:37:08.601906+0000","last_became_active":"2024-04-16T17:48:34.331932+0000","last_became_peered":"2024-04-16T17:48:34.331932+0000","last_unstale":"2024-04-16T19:37:08.601906+0000","last_undegraded":"2024-04-16T19:37:08.601906+0000","last_fullsized":"2024-04-16T19:37:08.601906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'200","last_scrub_stamp":"2024-04-16T19:37:08.601835+0000","last_deep_scrub":"53'200","last_deep_scrub_stamp":"2024-04-16T19:37:08.601835+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.601835+0000","objects_scrubbed":36,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:25.200215+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3150,"num_read_kb":3384,"num_write":323,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19169,"num_omap_keys":76,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.413351+0000","last_change":"2024-04-16T19:36:52.413405+0000","last_active":"2024-04-16T19:36:52.413351+0000","last_peered":"2024-04-16T19:36:52.413351+0000","last_clean":"2024-04-16T19:36:52.413351+0000","last_became_active":"2024-04-16T17:48:27.306175+0000","last_became_peered":"2024-04-16T17:48:27.306175+0000","last_unstale":"2024-04-16T19:36:52.413351+0000","last_undegraded":"2024-04-16T19:36:52.413351+0000","last_fullsized":"2024-04-16T19:36:52.413351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:52.413208+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.413208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:42.476038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"53'13","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.645823+0000","last_change":"2024-04-16T19:36:38.645892+0000","last_active":"2024-04-16T19:36:38.645823+0000","last_peered":"2024-04-16T19:36:38.645823+0000","last_clean":"2024-04-16T19:36:38.645823+0000","last_became_active":"2024-04-16T17:48:01.252028+0000","last_became_peered":"2024-04-16T17:48:01.252028+0000","last_unstale":"2024-04-16T19:36:38.645823+0000","last_undegraded":"2024-04-16T19:36:38.645823+0000","last_fullsized":"2024-04-16T19:36:38.645823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:38.645696+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.645696+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:29.924307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":33,"num_write":21,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":158,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.474393+0000","last_change":"2024-04-16T19:36:58.474445+0000","last_active":"2024-04-16T19:36:58.474393+0000","last_peered":"2024-04-16T19:36:58.474393+0000","last_clean":"2024-04-16T19:36:58.474393+0000","last_became_active":"2024-04-16T17:47:59.199939+0000","last_became_peered":"2024-04-16T17:47:59.199939+0000","last_unstale":"2024-04-16T19:36:58.474393+0000","last_undegraded":"2024-04-16T19:36:58.474393+0000","last_fullsized":"2024-04-16T19:36:58.474393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.474247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.474247+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.474247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:24.479134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"53'1110","reported_seq":1775,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.529609+0000","last_change":"2024-04-16T19:36:31.529728+0000","last_active":"2024-04-16T19:36:31.529609+0000","last_peered":"2024-04-16T19:36:31.529609+0000","last_clean":"2024-04-16T19:36:31.529609+0000","last_became_active":"2024-04-16T17:47:57.186194+0000","last_became_peered":"2024-04-16T17:47:57.186194+0000","last_unstale":"2024-04-16T19:36:31.529609+0000","last_undegraded":"2024-04-16T19:36:31.529609+0000","last_fullsized":"2024-04-16T19:36:31.529609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1110","last_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_deep_scrub":"53'1110","last_deep_scrub_stamp":"2024-04-16T19:36:31.529471+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.529471+0000","objects_scrubbed":23,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:18.708104+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29791,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2234,"num_read_kb":2044,"num_write":1964,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143914,"num_omap_keys":628,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"53'8","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.477590+0000","last_change":"2024-04-16T19:36:20.477590+0000","last_active":"2024-04-16T19:36:20.477590+0000","last_peered":"2024-04-16T19:36:20.477590+0000","last_clean":"2024-04-16T19:36:20.477590+0000","last_became_active":"2024-04-16T17:47:55.176763+0000","last_became_peered":"2024-04-16T17:47:55.176763+0000","last_unstale":"2024-04-16T19:36:20.477590+0000","last_undegraded":"2024-04-16T19:36:20.477590+0000","last_fullsized":"2024-04-16T19:36:20.477590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:20.477384+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.477384+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:25.916610+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.695098+0000","last_change":"2024-04-16T19:37:22.695098+0000","last_active":"2024-04-16T19:37:22.695098+0000","last_peered":"2024-04-16T19:37:22.695098+0000","last_clean":"2024-04-16T19:37:22.695098+0000","last_became_active":"2024-04-16T17:47:23.256625+0000","last_became_peered":"2024-04-16T17:47:23.256625+0000","last_unstale":"2024-04-16T19:37:22.695098+0000","last_undegraded":"2024-04-16T19:37:22.695098+0000","last_fullsized":"2024-04-16T19:37:22.695098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.694908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.694908+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.694908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:18.580946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.469846+0000","last_change":"2024-04-16T19:37:32.469897+0000","last_active":"2024-04-16T19:37:32.469846+0000","last_peered":"2024-04-16T19:37:32.469846+0000","last_clean":"2024-04-16T19:37:32.469846+0000","last_became_active":"2024-04-16T17:47:21.256530+0000","last_became_peered":"2024-04-16T17:47:21.256530+0000","last_unstale":"2024-04-16T19:37:32.469846+0000","last_undegraded":"2024-04-16T19:37:32.469846+0000","last_fullsized":"2024-04-16T19:37:32.469846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.469708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.469708+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.469708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:23.814824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.372126+0000","last_change":"2024-04-16T19:37:13.372176+0000","last_active":"2024-04-16T19:37:13.372126+0000","last_peered":"2024-04-16T19:37:13.372126+0000","last_clean":"2024-04-16T19:37:13.372126+0000","last_became_active":"2024-04-16T17:47:33.308568+0000","last_became_peered":"2024-04-16T17:47:33.308568+0000","last_unstale":"2024-04-16T19:37:13.372126+0000","last_undegraded":"2024-04-16T19:37:13.372126+0000","last_fullsized":"2024-04-16T19:37:13.372126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.371985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.371985+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.371985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:47:39.425403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"53'209","reported_seq":437,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.363263+0000","last_change":"2024-04-16T19:37:08.363339+0000","last_active":"2024-04-16T19:37:08.363263+0000","last_peered":"2024-04-16T19:37:08.363263+0000","last_clean":"2024-04-16T19:37:08.363263+0000","last_became_active":"2024-04-16T17:47:35.313668+0000","last_became_peered":"2024-04-16T17:47:35.313668+0000","last_unstale":"2024-04-16T19:37:08.363263+0000","last_undegraded":"2024-04-16T19:37:08.363263+0000","last_fullsized":"2024-04-16T19:37:08.363263+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'209","last_scrub_stamp":"2024-04-16T19:37:08.363125+0000","last_deep_scrub":"53'209","last_deep_scrub_stamp":"2024-04-16T19:37:08.363125+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.363125+0000","objects_scrubbed":9,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:57.353295+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":305,"num_read_kb":294,"num_write":200,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.650334+0000","last_change":"2024-04-16T19:36:49.650334+0000","last_active":"2024-04-16T19:36:49.650334+0000","last_peered":"2024-04-16T19:36:49.650334+0000","last_clean":"2024-04-16T19:36:49.650334+0000","last_became_active":"2024-04-16T17:47:37.334676+0000","last_became_peered":"2024-04-16T17:47:37.334676+0000","last_unstale":"2024-04-16T19:36:49.650334+0000","last_undegraded":"2024-04-16T19:36:49.650334+0000","last_fullsized":"2024-04-16T19:36:49.650334+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.650064+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.650064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:03.271488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.443726+0000","last_change":"2024-04-16T19:36:42.443778+0000","last_active":"2024-04-16T19:36:42.443726+0000","last_peered":"2024-04-16T19:36:42.443726+0000","last_clean":"2024-04-16T19:36:42.443726+0000","last_became_active":"2024-04-16T17:47:39.339168+0000","last_became_peered":"2024-04-16T17:47:39.339168+0000","last_unstale":"2024-04-16T19:36:42.443726+0000","last_undegraded":"2024-04-16T19:36:42.443726+0000","last_fullsized":"2024-04-16T19:36:42.443726+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.443589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.443589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:33.202793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.499959+0000","last_change":"2024-04-16T19:37:30.500012+0000","last_active":"2024-04-16T19:37:30.499959+0000","last_peered":"2024-04-16T19:37:30.499959+0000","last_clean":"2024-04-16T19:37:30.499959+0000","last_became_active":"2024-04-16T18:10:50.553923+0000","last_became_peered":"2024-04-16T18:10:50.553923+0000","last_unstale":"2024-04-16T19:37:30.499959+0000","last_undegraded":"2024-04-16T19:37:30.499959+0000","last_fullsized":"2024-04-16T19:37:30.499959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.499811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.499811+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.499811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:55.818847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"53'138","reported_seq":1227,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:46.647276+0000","last_change":"2024-04-16T19:37:46.647343+0000","last_active":"2024-04-16T19:37:46.647276+0000","last_peered":"2024-04-16T19:37:46.647276+0000","last_clean":"2024-04-16T19:37:46.647276+0000","last_became_active":"2024-04-16T17:48:34.345903+0000","last_became_peered":"2024-04-16T17:48:34.345903+0000","last_unstale":"2024-04-16T19:37:46.647276+0000","last_undegraded":"2024-04-16T19:37:46.647276+0000","last_fullsized":"2024-04-16T19:37:46.647276+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'138","last_scrub_stamp":"2024-04-16T19:37:46.647140+0000","last_deep_scrub":"53'138","last_deep_scrub_stamp":"2024-04-16T19:37:46.647140+0000","last_clean_scrub_stamp":"2024-04-16T19:37:46.647140+0000","objects_scrubbed":29,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:50:45.584478+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1756,"num_read_kb":1832,"num_write":170,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9891,"num_omap_keys":41,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.441666+0000","last_change":"2024-04-16T19:36:48.441666+0000","last_active":"2024-04-16T19:36:48.441666+0000","last_peered":"2024-04-16T19:36:48.441666+0000","last_clean":"2024-04-16T19:36:48.441666+0000","last_became_active":"2024-04-16T17:48:27.296831+0000","last_became_peered":"2024-04-16T17:48:27.296831+0000","last_unstale":"2024-04-16T19:36:48.441666+0000","last_undegraded":"2024-04-16T19:36:48.441666+0000","last_fullsized":"2024-04-16T19:36:48.441666+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.441481+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.441481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:54.447789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"53'4","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.445595+0000","last_change":"2024-04-16T19:36:53.445663+0000","last_active":"2024-04-16T19:36:53.445595+0000","last_peered":"2024-04-16T19:36:53.445595+0000","last_clean":"2024-04-16T19:36:53.445595+0000","last_became_active":"2024-04-16T17:48:01.253481+0000","last_became_peered":"2024-04-16T17:48:01.253481+0000","last_unstale":"2024-04-16T19:36:53.445595+0000","last_undegraded":"2024-04-16T19:36:53.445595+0000","last_fullsized":"2024-04-16T19:36:53.445595+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:53.445453+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.445453+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:27.787458+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.610902+0000","last_change":"2024-04-16T19:36:39.610922+0000","last_active":"2024-04-16T19:36:39.610902+0000","last_peered":"2024-04-16T19:36:39.610902+0000","last_clean":"2024-04-16T19:36:39.610902+0000","last_became_active":"2024-04-16T17:47:59.192968+0000","last_became_peered":"2024-04-16T17:47:59.192968+0000","last_unstale":"2024-04-16T19:36:39.610902+0000","last_undegraded":"2024-04-16T19:36:39.610902+0000","last_fullsized":"2024-04-16T19:36:39.610902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.610843+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.610843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:16.867566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'1195","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.442330+0000","last_change":"2024-04-16T19:36:32.442434+0000","last_active":"2024-04-16T19:36:32.442330+0000","last_peered":"2024-04-16T19:36:32.442330+0000","last_clean":"2024-04-16T19:36:32.442330+0000","last_became_active":"2024-04-16T17:47:57.191144+0000","last_became_peered":"2024-04-16T17:47:57.191144+0000","last_unstale":"2024-04-16T19:36:32.442330+0000","last_undegraded":"2024-04-16T19:36:32.442330+0000","last_fullsized":"2024-04-16T19:36:32.442330+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1195","last_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_deep_scrub":"53'1195","last_deep_scrub_stamp":"2024-04-16T19:36:32.442194+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.442194+0000","objects_scrubbed":22,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:37.854921+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1407,"num_read_kb":1393,"num_write":2138,"num_write_kb":956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215932,"num_omap_keys":942,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"53'17","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.474347+0000","last_change":"2024-04-16T19:36:21.474426+0000","last_active":"2024-04-16T19:36:21.474347+0000","last_peered":"2024-04-16T19:36:21.474347+0000","last_clean":"2024-04-16T19:36:21.474347+0000","last_became_active":"2024-04-16T17:47:55.162748+0000","last_became_peered":"2024-04-16T17:47:55.162748+0000","last_unstale":"2024-04-16T19:36:21.474347+0000","last_undegraded":"2024-04-16T19:36:21.474347+0000","last_fullsized":"2024-04-16T19:36:21.474347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'17","last_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_deep_scrub":"53'17","last_deep_scrub_stamp":"2024-04-16T19:36:21.474217+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.474217+0000","objects_scrubbed":3,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:46:40.463538+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":42,"num_write":91,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.640210+0000","last_change":"2024-04-16T19:37:08.640210+0000","last_active":"2024-04-16T19:37:08.640210+0000","last_peered":"2024-04-16T19:37:08.640210+0000","last_clean":"2024-04-16T19:37:08.640210+0000","last_became_active":"2024-04-16T17:47:23.264607+0000","last_became_peered":"2024-04-16T17:47:23.264607+0000","last_unstale":"2024-04-16T19:37:08.640210+0000","last_undegraded":"2024-04-16T19:37:08.640210+0000","last_fullsized":"2024-04-16T19:37:08.640210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:08.640022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:08.640022+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.640022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:42.193624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.517042+0000","last_change":"2024-04-16T19:37:30.517042+0000","last_active":"2024-04-16T19:37:30.517042+0000","last_peered":"2024-04-16T19:37:30.517042+0000","last_clean":"2024-04-16T19:37:30.517042+0000","last_became_active":"2024-04-16T17:47:21.257032+0000","last_became_peered":"2024-04-16T17:47:21.257032+0000","last_unstale":"2024-04-16T19:37:30.517042+0000","last_undegraded":"2024-04-16T19:37:30.517042+0000","last_fullsized":"2024-04-16T19:37:30.517042+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.516852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.516852+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.516852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:47.230865+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"53'2","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.485428+0000","last_change":"2024-04-16T19:37:12.485493+0000","last_active":"2024-04-16T19:37:12.485428+0000","last_peered":"2024-04-16T19:37:12.485428+0000","last_clean":"2024-04-16T19:37:12.485428+0000","last_became_active":"2024-04-16T17:47:33.301623+0000","last_became_peered":"2024-04-16T17:47:33.301623+0000","last_unstale":"2024-04-16T19:37:12.485428+0000","last_undegraded":"2024-04-16T19:37:12.485428+0000","last_fullsized":"2024-04-16T19:37:12.485428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:12.485279+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:12.485279+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.485279+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.828865+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"53'84","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.729240+0000","last_change":"2024-04-16T19:36:57.729240+0000","last_active":"2024-04-16T19:36:57.729240+0000","last_peered":"2024-04-16T19:36:57.729240+0000","last_clean":"2024-04-16T19:36:57.729240+0000","last_became_active":"2024-04-16T17:47:35.320957+0000","last_became_peered":"2024-04-16T17:47:35.320957+0000","last_unstale":"2024-04-16T19:36:57.729240+0000","last_undegraded":"2024-04-16T19:36:57.729240+0000","last_fullsized":"2024-04-16T19:36:57.729240+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:36:57.729025+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:36:57.729025+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.729025+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:26:51.751473+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.388357+0000","last_change":"2024-04-16T19:36:54.388412+0000","last_active":"2024-04-16T19:36:54.388357+0000","last_peered":"2024-04-16T19:36:54.388357+0000","last_clean":"2024-04-16T19:36:54.388357+0000","last_became_active":"2024-04-16T17:47:37.324778+0000","last_became_peered":"2024-04-16T17:47:37.324778+0000","last_unstale":"2024-04-16T19:36:54.388357+0000","last_undegraded":"2024-04-16T19:36:54.388357+0000","last_fullsized":"2024-04-16T19:36:54.388357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.388213+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.388213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:49:28.441656+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.545722+0000","last_change":"2024-04-16T19:36:53.545774+0000","last_active":"2024-04-16T19:36:53.545722+0000","last_peered":"2024-04-16T19:36:53.545722+0000","last_clean":"2024-04-16T19:36:53.545722+0000","last_became_active":"2024-04-16T17:47:39.322661+0000","last_became_peered":"2024-04-16T17:47:39.322661+0000","last_unstale":"2024-04-16T19:36:53.545722+0000","last_undegraded":"2024-04-16T19:36:53.545722+0000","last_fullsized":"2024-04-16T19:36:53.545722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.545589+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.545589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:08.257713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:49.608935+0000","last_change":"2024-04-16T19:37:49.608986+0000","last_active":"2024-04-16T19:37:49.608935+0000","last_peered":"2024-04-16T19:37:49.608935+0000","last_clean":"2024-04-16T19:37:49.608935+0000","last_became_active":"2024-04-16T18:10:50.561156+0000","last_became_peered":"2024-04-16T18:10:50.561156+0000","last_unstale":"2024-04-16T19:37:49.608935+0000","last_undegraded":"2024-04-16T19:37:49.608935+0000","last_fullsized":"2024-04-16T19:37:49.608935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:49.608797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:49.608797+0000","last_clean_scrub_stamp":"2024-04-16T19:37:49.608797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:31:47.966741+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"53'90","reported_seq":931,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.425451+0000","last_change":"2024-04-16T19:36:56.425451+0000","last_active":"2024-04-16T19:36:56.425451+0000","last_peered":"2024-04-16T19:36:56.425451+0000","last_clean":"2024-04-16T19:36:56.425451+0000","last_became_active":"2024-04-16T17:48:34.344982+0000","last_became_peered":"2024-04-16T17:48:34.344982+0000","last_unstale":"2024-04-16T19:36:56.425451+0000","last_undegraded":"2024-04-16T19:36:56.425451+0000","last_fullsized":"2024-04-16T19:36:56.425451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'90","last_scrub_stamp":"2024-04-16T19:36:56.425221+0000","last_deep_scrub":"53'90","last_deep_scrub_stamp":"2024-04-16T19:36:56.425221+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.425221+0000","objects_scrubbed":20,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:43.603855+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1332,"num_read_kb":1396,"num_write":112,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6839,"num_omap_keys":28,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.686464+0000","last_change":"2024-04-16T19:36:49.686464+0000","last_active":"2024-04-16T19:36:49.686464+0000","last_peered":"2024-04-16T19:36:49.686464+0000","last_clean":"2024-04-16T19:36:49.686464+0000","last_became_active":"2024-04-16T17:48:27.294939+0000","last_became_peered":"2024-04-16T17:48:27.294939+0000","last_unstale":"2024-04-16T19:36:49.686464+0000","last_undegraded":"2024-04-16T19:36:49.686464+0000","last_fullsized":"2024-04-16T19:36:49.686464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.686271+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.686271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.036352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:44.646832+0000","last_change":"2024-04-16T19:37:44.646907+0000","last_active":"2024-04-16T19:37:44.646832+0000","last_peered":"2024-04-16T19:37:44.646832+0000","last_clean":"2024-04-16T19:37:44.646832+0000","last_became_active":"2024-04-16T17:49:32.401316+0000","last_became_peered":"2024-04-16T17:49:32.401316+0000","last_unstale":"2024-04-16T19:37:44.646832+0000","last_undegraded":"2024-04-16T19:37:44.646832+0000","last_fullsized":"2024-04-16T19:37:44.646832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:37:44.646700+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:37:44.646700+0000","last_clean_scrub_stamp":"2024-04-16T19:37:44.646700+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:29.044278+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":898,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":31,"num_write":25,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.684642+0000","last_change":"2024-04-16T19:36:32.684642+0000","last_active":"2024-04-16T19:36:32.684642+0000","last_peered":"2024-04-16T19:36:32.684642+0000","last_clean":"2024-04-16T19:36:32.684642+0000","last_became_active":"2024-04-16T17:47:59.195780+0000","last_became_peered":"2024-04-16T17:47:59.195780+0000","last_unstale":"2024-04-16T19:36:32.684642+0000","last_undegraded":"2024-04-16T19:36:32.684642+0000","last_fullsized":"2024-04-16T19:36:32.684642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.684452+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.684452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:11.352292+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"53'1071","reported_seq":6142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.466107+0000","last_change":"2024-04-16T19:36:31.466251+0000","last_active":"2024-04-16T19:36:31.466107+0000","last_peered":"2024-04-16T19:36:31.466107+0000","last_clean":"2024-04-16T19:36:31.466107+0000","last_became_active":"2024-04-16T17:47:57.189032+0000","last_became_peered":"2024-04-16T17:47:57.189032+0000","last_unstale":"2024-04-16T19:36:31.466107+0000","last_undegraded":"2024-04-16T19:36:31.466107+0000","last_fullsized":"2024-04-16T19:36:31.466107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1071","last_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_deep_scrub":"53'1071","last_deep_scrub_stamp":"2024-04-16T19:36:31.465959+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.465959+0000","objects_scrubbed":22,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:36.878686+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7943,"num_read_kb":7928,"num_write":1005,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71967,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'8","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.607489+0000","last_change":"2024-04-16T19:36:24.607587+0000","last_active":"2024-04-16T19:36:24.607489+0000","last_peered":"2024-04-16T19:36:24.607489+0000","last_clean":"2024-04-16T19:36:24.607489+0000","last_became_active":"2024-04-16T17:47:55.182317+0000","last_became_peered":"2024-04-16T17:47:55.182317+0000","last_unstale":"2024-04-16T19:36:24.607489+0000","last_undegraded":"2024-04-16T19:36:24.607489+0000","last_fullsized":"2024-04-16T19:36:24.607489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:24.607347+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.607347+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:30.046572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":185,"num_read_kb":117,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.497814+0000","last_change":"2024-04-16T19:37:31.497868+0000","last_active":"2024-04-16T19:37:31.497814+0000","last_peered":"2024-04-16T19:37:31.497814+0000","last_clean":"2024-04-16T19:37:31.497814+0000","last_became_active":"2024-04-16T17:47:23.265172+0000","last_became_peered":"2024-04-16T17:47:23.265172+0000","last_unstale":"2024-04-16T19:37:31.497814+0000","last_undegraded":"2024-04-16T19:37:31.497814+0000","last_fullsized":"2024-04-16T19:37:31.497814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.497663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.497663+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.497663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:56.021787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.691898+0000","last_change":"2024-04-16T19:37:31.691898+0000","last_active":"2024-04-16T19:37:31.691898+0000","last_peered":"2024-04-16T19:37:31.691898+0000","last_clean":"2024-04-16T19:37:31.691898+0000","last_became_active":"2024-04-16T17:47:21.247304+0000","last_became_peered":"2024-04-16T17:47:21.247304+0000","last_unstale":"2024-04-16T19:37:31.691898+0000","last_undegraded":"2024-04-16T19:37:31.691898+0000","last_fullsized":"2024-04-16T19:37:31.691898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.691712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.691712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.691712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:59:14.742312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.493250+0000","last_change":"2024-04-16T19:37:14.493250+0000","last_active":"2024-04-16T19:37:14.493250+0000","last_peered":"2024-04-16T19:37:14.493250+0000","last_clean":"2024-04-16T19:37:14.493250+0000","last_became_active":"2024-04-16T17:47:33.302023+0000","last_became_peered":"2024-04-16T17:47:33.302023+0000","last_unstale":"2024-04-16T19:37:14.493250+0000","last_undegraded":"2024-04-16T19:37:14.493250+0000","last_fullsized":"2024-04-16T19:37:14.493250+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.493058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.493058+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.493058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:17.475775+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"53'159","reported_seq":363,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.570200+0000","last_change":"2024-04-16T19:37:06.570279+0000","last_active":"2024-04-16T19:37:06.570200+0000","last_peered":"2024-04-16T19:37:06.570200+0000","last_clean":"2024-04-16T19:37:06.570200+0000","last_became_active":"2024-04-16T17:47:35.317096+0000","last_became_peered":"2024-04-16T17:47:35.317096+0000","last_unstale":"2024-04-16T19:37:06.570200+0000","last_undegraded":"2024-04-16T19:37:06.570200+0000","last_fullsized":"2024-04-16T19:37:06.570200+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:06.570065+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:06.570065+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.570065+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:27.105572+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"53'13","reported_seq":1348,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.514311+0000","last_change":"2024-04-16T19:37:06.514379+0000","last_active":"2024-04-16T19:37:06.514311+0000","last_peered":"2024-04-16T19:37:06.514311+0000","last_clean":"2024-04-16T19:37:06.514311+0000","last_became_active":"2024-04-16T17:47:37.317236+0000","last_became_peered":"2024-04-16T17:47:37.317236+0000","last_unstale":"2024-04-16T19:37:06.514311+0000","last_undegraded":"2024-04-16T19:37:06.514311+0000","last_fullsized":"2024-04-16T19:37:06.514311+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:06.514177+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:06.514177+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.514177+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:28.619214+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.712924+0000","last_change":"2024-04-16T19:36:43.712975+0000","last_active":"2024-04-16T19:36:43.712924+0000","last_peered":"2024-04-16T19:36:43.712924+0000","last_clean":"2024-04-16T19:36:43.712924+0000","last_became_active":"2024-04-16T17:47:39.328225+0000","last_became_peered":"2024-04-16T17:47:39.328225+0000","last_unstale":"2024-04-16T19:36:43.712924+0000","last_undegraded":"2024-04-16T19:36:43.712924+0000","last_fullsized":"2024-04-16T19:36:43.712924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.712792+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.712792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:43.360972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.502570+0000","last_change":"2024-04-16T19:37:29.502625+0000","last_active":"2024-04-16T19:37:29.502570+0000","last_peered":"2024-04-16T19:37:29.502570+0000","last_clean":"2024-04-16T19:37:29.502570+0000","last_became_active":"2024-04-16T18:10:50.546228+0000","last_became_peered":"2024-04-16T18:10:50.546228+0000","last_unstale":"2024-04-16T19:37:29.502570+0000","last_undegraded":"2024-04-16T19:37:29.502570+0000","last_fullsized":"2024-04-16T19:37:29.502570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.502425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.502425+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.502425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:24:42.672974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"53'100","reported_seq":1255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.688914+0000","last_change":"2024-04-16T19:36:47.688914+0000","last_active":"2024-04-16T19:36:47.688914+0000","last_peered":"2024-04-16T19:36:47.688914+0000","last_clean":"2024-04-16T19:36:47.688914+0000","last_became_active":"2024-04-16T17:48:34.326283+0000","last_became_peered":"2024-04-16T17:48:34.326283+0000","last_unstale":"2024-04-16T19:36:47.688914+0000","last_undegraded":"2024-04-16T19:36:47.688914+0000","last_fullsized":"2024-04-16T19:36:47.688914+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'100","last_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_deep_scrub":"53'100","last_deep_scrub_stamp":"2024-04-16T19:36:47.688717+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.688717+0000","objects_scrubbed":26,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:21.999831+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1906,"num_read_kb":1886,"num_write":101,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5240,"num_omap_keys":22,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.472784+0000","last_change":"2024-04-16T19:36:55.472893+0000","last_active":"2024-04-16T19:36:55.472784+0000","last_peered":"2024-04-16T19:36:55.472784+0000","last_clean":"2024-04-16T19:36:55.472784+0000","last_became_active":"2024-04-16T17:48:27.301091+0000","last_became_peered":"2024-04-16T17:48:27.301091+0000","last_unstale":"2024-04-16T19:36:55.472784+0000","last_undegraded":"2024-04-16T19:36:55.472784+0000","last_fullsized":"2024-04-16T19:36:55.472784+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.472614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.472614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:45.535445+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"53'20","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.637124+0000","last_change":"2024-04-16T19:36:41.637203+0000","last_active":"2024-04-16T19:36:41.637124+0000","last_peered":"2024-04-16T19:36:41.637124+0000","last_clean":"2024-04-16T19:36:41.637124+0000","last_became_active":"2024-04-16T17:48:01.242534+0000","last_became_peered":"2024-04-16T17:48:01.242534+0000","last_unstale":"2024-04-16T19:36:41.637124+0000","last_undegraded":"2024-04-16T19:36:41.637124+0000","last_fullsized":"2024-04-16T19:36:41.637124+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:41.636981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.636981+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.258518+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1087,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":79,"num_write":66,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.453915+0000","last_change":"2024-04-16T19:36:39.453971+0000","last_active":"2024-04-16T19:36:39.453915+0000","last_peered":"2024-04-16T19:36:39.453915+0000","last_clean":"2024-04-16T19:36:39.453915+0000","last_became_active":"2024-04-16T17:47:59.193315+0000","last_became_peered":"2024-04-16T17:47:59.193315+0000","last_unstale":"2024-04-16T19:36:39.453915+0000","last_undegraded":"2024-04-16T19:36:39.453915+0000","last_fullsized":"2024-04-16T19:36:39.453915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.453770+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.453770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:59.663137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"53'736","reported_seq":1478,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.630399+0000","last_change":"2024-04-16T19:36:18.630399+0000","last_active":"2024-04-16T19:36:18.630399+0000","last_peered":"2024-04-16T19:36:18.630399+0000","last_clean":"2024-04-16T19:36:18.630399+0000","last_became_active":"2024-04-16T17:47:57.172533+0000","last_became_peered":"2024-04-16T17:47:57.172533+0000","last_unstale":"2024-04-16T19:36:18.630399+0000","last_undegraded":"2024-04-16T19:36:18.630399+0000","last_fullsized":"2024-04-16T19:36:18.630399+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'736","last_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_deep_scrub":"53'736","last_deep_scrub_stamp":"2024-04-16T19:36:18.630302+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.630302+0000","objects_scrubbed":33,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:45.495590+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1424,"num_read_kb":1318,"num_write":1137,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":75043,"num_omap_keys":332,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"53'18","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.577485+0000","last_change":"2024-04-16T19:36:26.577559+0000","last_active":"2024-04-16T19:36:26.577485+0000","last_peered":"2024-04-16T19:36:26.577485+0000","last_clean":"2024-04-16T19:36:26.577485+0000","last_became_active":"2024-04-16T17:47:55.180870+0000","last_became_peered":"2024-04-16T17:47:55.180870+0000","last_unstale":"2024-04-16T19:36:26.577485+0000","last_undegraded":"2024-04-16T19:36:26.577485+0000","last_fullsized":"2024-04-16T19:36:26.577485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:26.577356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.577356+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:41.709302+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":161,"num_read_kb":104,"num_write":94,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.649851+0000","last_change":"2024-04-16T19:37:33.649851+0000","last_active":"2024-04-16T19:37:33.649851+0000","last_peered":"2024-04-16T19:37:33.649851+0000","last_clean":"2024-04-16T19:37:33.649851+0000","last_became_active":"2024-04-16T17:47:23.267251+0000","last_became_peered":"2024-04-16T17:47:23.267251+0000","last_unstale":"2024-04-16T19:37:33.649851+0000","last_undegraded":"2024-04-16T19:37:33.649851+0000","last_fullsized":"2024-04-16T19:37:33.649851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.649657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.649657+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.649657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:40.182196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:43.501021+0000","last_change":"2024-04-16T19:37:43.501073+0000","last_active":"2024-04-16T19:37:43.501021+0000","last_peered":"2024-04-16T19:37:43.501021+0000","last_clean":"2024-04-16T19:37:43.501021+0000","last_became_active":"2024-04-16T17:47:21.235939+0000","last_became_peered":"2024-04-16T17:47:21.235939+0000","last_unstale":"2024-04-16T19:37:43.501021+0000","last_undegraded":"2024-04-16T19:37:43.501021+0000","last_fullsized":"2024-04-16T19:37:43.501021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:43.500888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:43.500888+0000","last_clean_scrub_stamp":"2024-04-16T19:37:43.500888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:56.820700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.491099+0000","last_change":"2024-04-16T19:37:16.491152+0000","last_active":"2024-04-16T19:37:16.491099+0000","last_peered":"2024-04-16T19:37:16.491099+0000","last_clean":"2024-04-16T19:37:16.491099+0000","last_became_active":"2024-04-16T17:47:33.303987+0000","last_became_peered":"2024-04-16T17:47:33.303987+0000","last_unstale":"2024-04-16T19:37:16.491099+0000","last_undegraded":"2024-04-16T19:37:16.491099+0000","last_fullsized":"2024-04-16T19:37:16.491099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:16.490961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:16.490961+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.490961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:22.497268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"53'93","reported_seq":270,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.563674+0000","last_change":"2024-04-16T19:37:07.563764+0000","last_active":"2024-04-16T19:37:07.563674+0000","last_peered":"2024-04-16T19:37:07.563674+0000","last_clean":"2024-04-16T19:37:07.563674+0000","last_became_active":"2024-04-16T17:47:35.317205+0000","last_became_peered":"2024-04-16T17:47:35.317205+0000","last_unstale":"2024-04-16T19:37:07.563674+0000","last_undegraded":"2024-04-16T19:37:07.563674+0000","last_fullsized":"2024-04-16T19:37:07.563674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'93","last_scrub_stamp":"2024-04-16T19:37:07.563526+0000","last_deep_scrub":"53'93","last_deep_scrub_stamp":"2024-04-16T19:37:07.563526+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.563526+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:24.161906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.415324+0000","last_change":"2024-04-16T19:36:53.415380+0000","last_active":"2024-04-16T19:36:53.415324+0000","last_peered":"2024-04-16T19:36:53.415324+0000","last_clean":"2024-04-16T19:36:53.415324+0000","last_became_active":"2024-04-16T17:47:37.327808+0000","last_became_peered":"2024-04-16T17:47:37.327808+0000","last_unstale":"2024-04-16T19:36:53.415324+0000","last_undegraded":"2024-04-16T19:36:53.415324+0000","last_fullsized":"2024-04-16T19:36:53.415324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.415178+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.415178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:50.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.527536+0000","last_change":"2024-04-16T19:36:33.527616+0000","last_active":"2024-04-16T19:36:33.527536+0000","last_peered":"2024-04-16T19:36:33.527536+0000","last_clean":"2024-04-16T19:36:33.527536+0000","last_became_active":"2024-04-16T17:47:39.337956+0000","last_became_peered":"2024-04-16T17:47:39.337956+0000","last_unstale":"2024-04-16T19:36:33.527536+0000","last_undegraded":"2024-04-16T19:36:33.527536+0000","last_fullsized":"2024-04-16T19:36:33.527536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.527307+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.527307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:38.972728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":65,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:37.682089+0000","last_change":"2024-04-16T19:37:37.682143+0000","last_active":"2024-04-16T19:37:37.682089+0000","last_peered":"2024-04-16T19:37:37.682089+0000","last_clean":"2024-04-16T19:37:37.682089+0000","last_became_active":"2024-04-16T18:10:50.547591+0000","last_became_peered":"2024-04-16T18:10:50.547591+0000","last_unstale":"2024-04-16T19:37:37.682089+0000","last_undegraded":"2024-04-16T19:37:37.682089+0000","last_fullsized":"2024-04-16T19:37:37.682089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:37.681948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:37.681948+0000","last_clean_scrub_stamp":"2024-04-16T19:37:37.681948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:26:39.623492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"53'62","reported_seq":1005,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.553275+0000","last_change":"2024-04-16T19:37:05.553378+0000","last_active":"2024-04-16T19:37:05.553275+0000","last_peered":"2024-04-16T19:37:05.553275+0000","last_clean":"2024-04-16T19:37:05.553275+0000","last_became_active":"2024-04-16T17:48:34.323898+0000","last_became_peered":"2024-04-16T17:48:34.323898+0000","last_unstale":"2024-04-16T19:37:05.553275+0000","last_undegraded":"2024-04-16T19:37:05.553275+0000","last_fullsized":"2024-04-16T19:37:05.553275+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'62","last_scrub_stamp":"2024-04-16T19:37:05.553135+0000","last_deep_scrub":"53'62","last_deep_scrub_stamp":"2024-04-16T19:37:05.553135+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.553135+0000","objects_scrubbed":22,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:26.088642+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1589,"num_read_kb":1576,"num_write":49,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1541,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.684390+0000","last_change":"2024-04-16T19:36:40.684390+0000","last_active":"2024-04-16T19:36:40.684390+0000","last_peered":"2024-04-16T19:36:40.684390+0000","last_clean":"2024-04-16T19:36:40.684390+0000","last_became_active":"2024-04-16T17:48:27.302702+0000","last_became_peered":"2024-04-16T17:48:27.302702+0000","last_unstale":"2024-04-16T19:36:40.684390+0000","last_undegraded":"2024-04-16T19:36:40.684390+0000","last_fullsized":"2024-04-16T19:36:40.684390+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.684205+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:34.096243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"53'11","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.426664+0000","last_change":"2024-04-16T19:36:52.426789+0000","last_active":"2024-04-16T19:36:52.426664+0000","last_peered":"2024-04-16T19:36:52.426664+0000","last_clean":"2024-04-16T19:36:52.426664+0000","last_became_active":"2024-04-16T17:48:01.259378+0000","last_became_peered":"2024-04-16T17:48:01.259378+0000","last_unstale":"2024-04-16T19:36:52.426664+0000","last_undegraded":"2024-04-16T19:36:52.426664+0000","last_fullsized":"2024-04-16T19:36:52.426664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:52.426496+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.426496+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:18.642190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":28,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3","version":"53'556","reported_seq":2298,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.370105+0000","last_change":"2024-04-16T19:36:40.370175+0000","last_active":"2024-04-16T19:36:40.370105+0000","last_peered":"2024-04-16T19:36:40.370105+0000","last_clean":"2024-04-16T19:36:40.370105+0000","last_became_active":"2024-04-16T17:47:59.181029+0000","last_became_peered":"2024-04-16T17:47:59.181029+0000","last_unstale":"2024-04-16T19:36:40.370105+0000","last_undegraded":"2024-04-16T19:36:40.370105+0000","last_fullsized":"2024-04-16T19:36:40.370105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:40.369960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.369960+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:17.626683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"53'925","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.519148+0000","last_change":"2024-04-16T19:36:29.519272+0000","last_active":"2024-04-16T19:36:29.519148+0000","last_peered":"2024-04-16T19:36:29.519148+0000","last_clean":"2024-04-16T19:36:29.519148+0000","last_became_active":"2024-04-16T17:47:57.174301+0000","last_became_peered":"2024-04-16T17:47:57.174301+0000","last_unstale":"2024-04-16T19:36:29.519148+0000","last_undegraded":"2024-04-16T19:36:29.519148+0000","last_fullsized":"2024-04-16T19:36:29.519148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:29.519049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.519049+0000","objects_scrubbed":24,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:29.400044+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1388,"num_read_kb":1374,"num_write":1587,"num_write_kb":655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143864,"num_omap_keys":628,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"53'19","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.480753+0000","last_change":"2024-04-16T19:36:30.480855+0000","last_active":"2024-04-16T19:36:30.480753+0000","last_peered":"2024-04-16T19:36:30.480753+0000","last_clean":"2024-04-16T19:36:30.480753+0000","last_became_active":"2024-04-16T17:47:55.155040+0000","last_became_peered":"2024-04-16T17:47:55.155040+0000","last_unstale":"2024-04-16T19:36:30.480753+0000","last_undegraded":"2024-04-16T19:36:30.480753+0000","last_fullsized":"2024-04-16T19:36:30.480753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.480620+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.480620+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:17.845024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":75,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.573834+0000","last_change":"2024-04-16T19:37:17.573834+0000","last_active":"2024-04-16T19:37:17.573834+0000","last_peered":"2024-04-16T19:37:17.573834+0000","last_clean":"2024-04-16T19:37:17.573834+0000","last_became_active":"2024-04-16T17:47:23.254768+0000","last_became_peered":"2024-04-16T17:47:23.254768+0000","last_unstale":"2024-04-16T19:37:17.573834+0000","last_undegraded":"2024-04-16T19:37:17.573834+0000","last_fullsized":"2024-04-16T19:37:17.573834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:17.573640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:17.573640+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.573640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:27.868101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.637829+0000","last_change":"2024-04-16T19:37:30.637829+0000","last_active":"2024-04-16T19:37:30.637829+0000","last_peered":"2024-04-16T19:37:30.637829+0000","last_clean":"2024-04-16T19:37:30.637829+0000","last_became_active":"2024-04-16T17:47:21.231616+0000","last_became_peered":"2024-04-16T17:47:21.231616+0000","last_unstale":"2024-04-16T19:37:30.637829+0000","last_undegraded":"2024-04-16T19:37:30.637829+0000","last_fullsized":"2024-04-16T19:37:30.637829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.637636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.637636+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.637636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:15:57.900942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"53'4","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.387622+0000","last_change":"2024-04-16T19:37:26.387622+0000","last_active":"2024-04-16T19:37:26.387622+0000","last_peered":"2024-04-16T19:37:26.387622+0000","last_clean":"2024-04-16T19:37:26.387622+0000","last_became_active":"2024-04-16T17:47:33.310098+0000","last_became_peered":"2024-04-16T17:47:33.310098+0000","last_unstale":"2024-04-16T19:37:26.387622+0000","last_undegraded":"2024-04-16T19:37:26.387622+0000","last_fullsized":"2024-04-16T19:37:26.387622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:26.387392+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:26.387392+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.387392+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:52.755065+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"53'109","reported_seq":288,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.642980+0000","last_change":"2024-04-16T19:37:01.642980+0000","last_active":"2024-04-16T19:37:01.642980+0000","last_peered":"2024-04-16T19:37:01.642980+0000","last_clean":"2024-04-16T19:37:01.642980+0000","last_became_active":"2024-04-16T17:47:35.305540+0000","last_became_peered":"2024-04-16T17:47:35.305540+0000","last_unstale":"2024-04-16T19:37:01.642980+0000","last_undegraded":"2024-04-16T19:37:01.642980+0000","last_fullsized":"2024-04-16T19:37:01.642980+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:37:01.642886+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:37:01.642886+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.642886+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:36:01.291066+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.529837+0000","last_change":"2024-04-16T19:37:03.529944+0000","last_active":"2024-04-16T19:37:03.529837+0000","last_peered":"2024-04-16T19:37:03.529837+0000","last_clean":"2024-04-16T19:37:03.529837+0000","last_became_active":"2024-04-16T17:47:37.323849+0000","last_became_peered":"2024-04-16T17:47:37.323849+0000","last_unstale":"2024-04-16T19:37:03.529837+0000","last_undegraded":"2024-04-16T19:37:03.529837+0000","last_fullsized":"2024-04-16T19:37:03.529837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.529668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.529668+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.529668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:01.869011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.457022+0000","last_change":"2024-04-16T19:36:30.457084+0000","last_active":"2024-04-16T19:36:30.457022+0000","last_peered":"2024-04-16T19:36:30.457022+0000","last_clean":"2024-04-16T19:36:30.457022+0000","last_became_active":"2024-04-16T17:47:39.340301+0000","last_became_peered":"2024-04-16T17:47:39.340301+0000","last_unstale":"2024-04-16T19:36:30.457022+0000","last_undegraded":"2024-04-16T19:36:30.457022+0000","last_fullsized":"2024-04-16T19:36:30.457022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:30.456877+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.456877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:36.883658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":65,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:48.603378+0000","last_change":"2024-04-16T19:37:48.603432+0000","last_active":"2024-04-16T19:37:48.603378+0000","last_peered":"2024-04-16T19:37:48.603378+0000","last_clean":"2024-04-16T19:37:48.603378+0000","last_became_active":"2024-04-16T18:10:50.529235+0000","last_became_peered":"2024-04-16T18:10:50.529235+0000","last_unstale":"2024-04-16T19:37:48.603378+0000","last_undegraded":"2024-04-16T19:37:48.603378+0000","last_fullsized":"2024-04-16T19:37:48.603378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:48.603239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:48.603239+0000","last_clean_scrub_stamp":"2024-04-16T19:37:48.603239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:28.653815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"53'106","reported_seq":916,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.482238+0000","last_change":"2024-04-16T19:37:19.482414+0000","last_active":"2024-04-16T19:37:19.482238+0000","last_peered":"2024-04-16T19:37:19.482238+0000","last_clean":"2024-04-16T19:37:19.482238+0000","last_became_active":"2024-04-16T17:48:34.318417+0000","last_became_peered":"2024-04-16T17:48:34.318417+0000","last_unstale":"2024-04-16T19:37:19.482238+0000","last_undegraded":"2024-04-16T19:37:19.482238+0000","last_fullsized":"2024-04-16T19:37:19.482238+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'106","last_scrub_stamp":"2024-04-16T19:37:19.482066+0000","last_deep_scrub":"53'106","last_deep_scrub_stamp":"2024-04-16T19:37:19.482066+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.482066+0000","objects_scrubbed":24,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:32:59.735406+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1354,"num_read_kb":1316,"num_write":127,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6869,"num_omap_keys":30,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.315787+0000","last_change":"2024-04-16T19:37:11.315847+0000","last_active":"2024-04-16T19:37:11.315787+0000","last_peered":"2024-04-16T19:37:11.315787+0000","last_clean":"2024-04-16T19:37:11.315787+0000","last_became_active":"2024-04-16T17:48:27.307739+0000","last_became_peered":"2024-04-16T17:48:27.307739+0000","last_unstale":"2024-04-16T19:37:11.315787+0000","last_undegraded":"2024-04-16T19:37:11.315787+0000","last_fullsized":"2024-04-16T19:37:11.315787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.315613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.315613+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.315613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:39.336603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"53'142","reported_seq":316,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.445837+0000","last_change":"2024-04-16T19:36:51.445837+0000","last_active":"2024-04-16T19:36:51.445837+0000","last_peered":"2024-04-16T19:36:51.445837+0000","last_clean":"2024-04-16T19:36:51.445837+0000","last_became_active":"2024-04-16T17:48:01.253593+0000","last_became_peered":"2024-04-16T17:48:01.253593+0000","last_unstale":"2024-04-16T19:36:51.445837+0000","last_undegraded":"2024-04-16T19:36:51.445837+0000","last_fullsized":"2024-04-16T19:36:51.445837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'142","last_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_deep_scrub":"53'142","last_deep_scrub_stamp":"2024-04-16T19:36:51.445664+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.445664+0000","objects_scrubbed":7,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:24.999267+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1654,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":339,"num_read_kb":597,"num_write":271,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9535,"num_omap_keys":55,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.514976+0000","last_change":"2024-04-16T19:36:24.515035+0000","last_active":"2024-04-16T19:36:24.514976+0000","last_peered":"2024-04-16T19:36:24.514976+0000","last_clean":"2024-04-16T19:36:24.514976+0000","last_became_active":"2024-04-16T17:47:59.173497+0000","last_became_peered":"2024-04-16T17:47:59.173497+0000","last_unstale":"2024-04-16T19:36:24.514976+0000","last_undegraded":"2024-04-16T19:36:24.514976+0000","last_fullsized":"2024-04-16T19:36:24.514976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.514834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.514834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:04:02.534977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"53'702","reported_seq":1099,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.544776+0000","last_change":"2024-04-16T19:36:22.544916+0000","last_active":"2024-04-16T19:36:22.544776+0000","last_peered":"2024-04-16T19:36:22.544776+0000","last_clean":"2024-04-16T19:36:22.544776+0000","last_became_active":"2024-04-16T17:47:57.181789+0000","last_became_peered":"2024-04-16T17:47:57.181789+0000","last_unstale":"2024-04-16T19:36:22.544776+0000","last_undegraded":"2024-04-16T19:36:22.544776+0000","last_fullsized":"2024-04-16T19:36:22.544776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'702","last_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_deep_scrub":"53'702","last_deep_scrub_stamp":"2024-04-16T19:36:22.544630+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.544630+0000","objects_scrubbed":23,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:27.704961+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":919,"num_read_kb":903,"num_write":1070,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71922,"num_omap_keys":314,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"53'12","reported_seq":218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.499314+0000","last_change":"2024-04-16T19:36:24.499314+0000","last_active":"2024-04-16T19:36:24.499314+0000","last_peered":"2024-04-16T19:36:24.499314+0000","last_clean":"2024-04-16T19:36:24.499314+0000","last_became_active":"2024-04-16T17:47:55.158239+0000","last_became_peered":"2024-04-16T17:47:55.158239+0000","last_unstale":"2024-04-16T19:36:24.499314+0000","last_undegraded":"2024-04-16T19:36:24.499314+0000","last_fullsized":"2024-04-16T19:36:24.499314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:24.498983+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.498983+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:39:48.052702+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":170,"num_write":73,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.758733+0000","last_change":"2024-04-16T19:37:26.758733+0000","last_active":"2024-04-16T19:37:26.758733+0000","last_peered":"2024-04-16T19:37:26.758733+0000","last_clean":"2024-04-16T19:37:26.758733+0000","last_became_active":"2024-04-16T17:47:23.257584+0000","last_became_peered":"2024-04-16T17:47:23.257584+0000","last_unstale":"2024-04-16T19:37:26.758733+0000","last_undegraded":"2024-04-16T19:37:26.758733+0000","last_fullsized":"2024-04-16T19:37:26.758733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.758459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.758459+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.758459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:26:28.700003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.763635+0000","last_change":"2024-04-16T19:37:27.763686+0000","last_active":"2024-04-16T19:37:27.763635+0000","last_peered":"2024-04-16T19:37:27.763635+0000","last_clean":"2024-04-16T19:37:27.763635+0000","last_became_active":"2024-04-16T17:47:21.252712+0000","last_became_peered":"2024-04-16T17:47:21.252712+0000","last_unstale":"2024-04-16T19:37:27.763635+0000","last_undegraded":"2024-04-16T19:37:27.763635+0000","last_fullsized":"2024-04-16T19:37:27.763635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.763498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.763498+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.763498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:44.477511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"53'14","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.644828+0000","last_change":"2024-04-16T19:37:04.644828+0000","last_active":"2024-04-16T19:37:04.644828+0000","last_peered":"2024-04-16T19:37:04.644828+0000","last_clean":"2024-04-16T19:37:04.644828+0000","last_became_active":"2024-04-16T17:47:33.308503+0000","last_became_peered":"2024-04-16T17:47:33.308503+0000","last_unstale":"2024-04-16T19:37:04.644828+0000","last_undegraded":"2024-04-16T19:37:04.644828+0000","last_fullsized":"2024-04-16T19:37:04.644828+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:37:04.644628+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:37:04.644628+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.644628+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:54.707458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":815,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":27,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"53'159","reported_seq":361,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.425798+0000","last_change":"2024-04-16T19:37:01.425941+0000","last_active":"2024-04-16T19:37:01.425798+0000","last_peered":"2024-04-16T19:37:01.425798+0000","last_clean":"2024-04-16T19:37:01.425798+0000","last_became_active":"2024-04-16T17:47:35.303767+0000","last_became_peered":"2024-04-16T17:47:35.303767+0000","last_unstale":"2024-04-16T19:37:01.425798+0000","last_undegraded":"2024-04-16T19:37:01.425798+0000","last_fullsized":"2024-04-16T19:37:01.425798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:01.425632+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:01.425632+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.425632+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:04.281799+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"53'13","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.458492+0000","last_change":"2024-04-16T19:36:51.458492+0000","last_active":"2024-04-16T19:36:51.458492+0000","last_peered":"2024-04-16T19:36:51.458492+0000","last_clean":"2024-04-16T19:36:51.458492+0000","last_became_active":"2024-04-16T17:47:37.328311+0000","last_became_peered":"2024-04-16T17:47:37.328311+0000","last_unstale":"2024-04-16T19:36:51.458492+0000","last_undegraded":"2024-04-16T19:36:51.458492+0000","last_fullsized":"2024-04-16T19:36:51.458492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:51.458284+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.458284+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:46:06.590229+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.582250+0000","last_change":"2024-04-16T19:36:36.582358+0000","last_active":"2024-04-16T19:36:36.582250+0000","last_peered":"2024-04-16T19:36:36.582250+0000","last_clean":"2024-04-16T19:36:36.582250+0000","last_became_active":"2024-04-16T17:47:39.330656+0000","last_became_peered":"2024-04-16T17:47:39.330656+0000","last_unstale":"2024-04-16T19:36:36.582250+0000","last_undegraded":"2024-04-16T19:36:36.582250+0000","last_fullsized":"2024-04-16T19:36:36.582250+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.582059+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.582059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:25:29.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:45.607665+0000","last_change":"2024-04-16T19:37:45.607718+0000","last_active":"2024-04-16T19:37:45.607665+0000","last_peered":"2024-04-16T19:37:45.607665+0000","last_clean":"2024-04-16T19:37:45.607665+0000","last_became_active":"2024-04-16T18:10:50.540514+0000","last_became_peered":"2024-04-16T18:10:50.540514+0000","last_unstale":"2024-04-16T19:37:45.607665+0000","last_undegraded":"2024-04-16T19:37:45.607665+0000","last_fullsized":"2024-04-16T19:37:45.607665+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:45.607527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:45.607527+0000","last_clean_scrub_stamp":"2024-04-16T19:37:45.607527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:34.580578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"53'118","reported_seq":1160,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.521304+0000","last_change":"2024-04-16T19:37:00.521375+0000","last_active":"2024-04-16T19:37:00.521304+0000","last_peered":"2024-04-16T19:37:00.521304+0000","last_clean":"2024-04-16T19:37:00.521304+0000","last_became_active":"2024-04-16T17:48:34.326202+0000","last_became_peered":"2024-04-16T17:48:34.326202+0000","last_unstale":"2024-04-16T19:37:00.521304+0000","last_undegraded":"2024-04-16T19:37:00.521304+0000","last_fullsized":"2024-04-16T19:37:00.521304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'118","last_scrub_stamp":"2024-04-16T19:37:00.521172+0000","last_deep_scrub":"53'118","last_deep_scrub_stamp":"2024-04-16T19:37:00.521172+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.521172+0000","objects_scrubbed":32,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:10.815633+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1729,"num_read_kb":1705,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7005,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.427150+0000","last_change":"2024-04-16T19:36:49.427150+0000","last_active":"2024-04-16T19:36:49.427150+0000","last_peered":"2024-04-16T19:36:49.427150+0000","last_clean":"2024-04-16T19:36:49.427150+0000","last_became_active":"2024-04-16T17:48:27.298065+0000","last_became_peered":"2024-04-16T17:48:27.298065+0000","last_unstale":"2024-04-16T19:36:49.427150+0000","last_undegraded":"2024-04-16T19:36:49.427150+0000","last_fullsized":"2024-04-16T19:36:49.427150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.426874+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.426874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:15.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"53'2","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.645599+0000","last_change":"2024-04-16T19:36:42.645707+0000","last_active":"2024-04-16T19:36:42.645599+0000","last_peered":"2024-04-16T19:36:42.645599+0000","last_clean":"2024-04-16T19:36:42.645599+0000","last_became_active":"2024-04-16T17:48:01.245933+0000","last_became_peered":"2024-04-16T17:48:01.245933+0000","last_unstale":"2024-04-16T19:36:42.645599+0000","last_undegraded":"2024-04-16T19:36:42.645599+0000","last_fullsized":"2024-04-16T19:36:42.645599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:42.645438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.645438+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:22.681965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.565703+0000","last_change":"2024-04-16T19:36:24.565807+0000","last_active":"2024-04-16T19:36:24.565703+0000","last_peered":"2024-04-16T19:36:24.565703+0000","last_clean":"2024-04-16T19:36:24.565703+0000","last_became_active":"2024-04-16T17:47:59.179502+0000","last_became_peered":"2024-04-16T17:47:59.179502+0000","last_unstale":"2024-04-16T19:36:24.565703+0000","last_undegraded":"2024-04-16T19:36:24.565703+0000","last_fullsized":"2024-04-16T19:36:24.565703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:24.565541+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.565541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:30.252893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.0","version":"53'1508","reported_seq":5513,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.585799+0000","last_change":"2024-04-16T19:36:27.585867+0000","last_active":"2024-04-16T19:36:27.585799+0000","last_peered":"2024-04-16T19:36:27.585799+0000","last_clean":"2024-04-16T19:36:27.585799+0000","last_became_active":"2024-04-16T17:47:57.185290+0000","last_became_peered":"2024-04-16T17:47:57.185290+0000","last_unstale":"2024-04-16T19:36:27.585799+0000","last_undegraded":"2024-04-16T19:36:27.585799+0000","last_fullsized":"2024-04-16T19:36:27.585799+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1508","last_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_deep_scrub":"53'1508","last_deep_scrub_stamp":"2024-04-16T19:36:27.585661+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.585661+0000","objects_scrubbed":25,"log_size":1508,"log_dups_size":0,"ondisk_log_size":1508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:18.587606+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9155,"num_read_kb":9010,"num_write":2750,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250399,"num_omap_keys":1045,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"53'14","reported_seq":240,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.424045+0000","last_change":"2024-04-16T19:36:27.424219+0000","last_active":"2024-04-16T19:36:27.424045+0000","last_peered":"2024-04-16T19:36:27.424045+0000","last_clean":"2024-04-16T19:36:27.424045+0000","last_became_active":"2024-04-16T17:47:55.161307+0000","last_became_peered":"2024-04-16T17:47:55.161307+0000","last_unstale":"2024-04-16T19:36:27.424045+0000","last_undegraded":"2024-04-16T19:36:27.424045+0000","last_fullsized":"2024-04-16T19:36:27.424045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:27.423818+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.423818+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:57.529566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":181,"num_write":67,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.650366+0000","last_change":"2024-04-16T19:37:06.650366+0000","last_active":"2024-04-16T19:37:06.650366+0000","last_peered":"2024-04-16T19:37:06.650366+0000","last_clean":"2024-04-16T19:37:06.650366+0000","last_became_active":"2024-04-16T17:47:23.265221+0000","last_became_peered":"2024-04-16T17:47:23.265221+0000","last_unstale":"2024-04-16T19:37:06.650366+0000","last_undegraded":"2024-04-16T19:37:06.650366+0000","last_fullsized":"2024-04-16T19:37:06.650366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:06.650180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:06.650180+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.650180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:32.865874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.474396+0000","last_change":"2024-04-16T19:37:24.474396+0000","last_active":"2024-04-16T19:37:24.474396+0000","last_peered":"2024-04-16T19:37:24.474396+0000","last_clean":"2024-04-16T19:37:24.474396+0000","last_became_active":"2024-04-16T17:47:21.223068+0000","last_became_peered":"2024-04-16T17:47:21.223068+0000","last_unstale":"2024-04-16T19:37:24.474396+0000","last_undegraded":"2024-04-16T19:37:24.474396+0000","last_fullsized":"2024-04-16T19:37:24.474396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.474210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.474210+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.474210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:46.579247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"53'4","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.499565+0000","last_change":"2024-04-16T19:37:18.499632+0000","last_active":"2024-04-16T19:37:18.499565+0000","last_peered":"2024-04-16T19:37:18.499565+0000","last_clean":"2024-04-16T19:37:18.499565+0000","last_became_active":"2024-04-16T17:47:33.301102+0000","last_became_peered":"2024-04-16T17:47:33.301102+0000","last_unstale":"2024-04-16T19:37:18.499565+0000","last_undegraded":"2024-04-16T19:37:18.499565+0000","last_fullsized":"2024-04-16T19:37:18.499565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:18.499434+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:18.499434+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.499434+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:31:26.309566+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"53'125","reported_seq":307,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.693314+0000","last_change":"2024-04-16T19:36:58.693314+0000","last_active":"2024-04-16T19:36:58.693314+0000","last_peered":"2024-04-16T19:36:58.693314+0000","last_clean":"2024-04-16T19:36:58.693314+0000","last_became_active":"2024-04-16T17:47:35.315722+0000","last_became_peered":"2024-04-16T17:47:35.315722+0000","last_unstale":"2024-04-16T19:36:58.693314+0000","last_undegraded":"2024-04-16T19:36:58.693314+0000","last_fullsized":"2024-04-16T19:36:58.693314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:36:58.693112+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:36:58.693112+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.693112+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:03.753998+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"53'13","reported_seq":1347,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.636714+0000","last_change":"2024-04-16T19:36:55.636714+0000","last_active":"2024-04-16T19:36:55.636714+0000","last_peered":"2024-04-16T19:36:55.636714+0000","last_clean":"2024-04-16T19:36:55.636714+0000","last_became_active":"2024-04-16T17:47:37.323886+0000","last_became_peered":"2024-04-16T17:47:37.323886+0000","last_unstale":"2024-04-16T19:36:55.636714+0000","last_undegraded":"2024-04-16T19:36:55.636714+0000","last_fullsized":"2024-04-16T19:36:55.636714+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:55.636425+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.636425+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:56:05.862157+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.567557+0000","last_change":"2024-04-16T19:36:51.567609+0000","last_active":"2024-04-16T19:36:51.567557+0000","last_peered":"2024-04-16T19:36:51.567557+0000","last_clean":"2024-04-16T19:36:51.567557+0000","last_became_active":"2024-04-16T17:47:39.324731+0000","last_became_peered":"2024-04-16T17:47:39.324731+0000","last_unstale":"2024-04-16T19:36:51.567557+0000","last_undegraded":"2024-04-16T19:36:51.567557+0000","last_fullsized":"2024-04-16T19:36:51.567557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.567419+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.567419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:32.289248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.653902+0000","last_change":"2024-04-16T19:37:35.653955+0000","last_active":"2024-04-16T19:37:35.653902+0000","last_peered":"2024-04-16T19:37:35.653902+0000","last_clean":"2024-04-16T19:37:35.653902+0000","last_became_active":"2024-04-16T18:10:50.538329+0000","last_became_peered":"2024-04-16T18:10:50.538329+0000","last_unstale":"2024-04-16T19:37:35.653902+0000","last_undegraded":"2024-04-16T19:37:35.653902+0000","last_fullsized":"2024-04-16T19:37:35.653902+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.653764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.653764+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.653764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:19.654955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"53'117","reported_seq":1015,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.675266+0000","last_change":"2024-04-16T19:37:09.675266+0000","last_active":"2024-04-16T19:37:09.675266+0000","last_peered":"2024-04-16T19:37:09.675266+0000","last_clean":"2024-04-16T19:37:09.675266+0000","last_became_active":"2024-04-16T17:48:34.333189+0000","last_became_peered":"2024-04-16T17:48:34.333189+0000","last_unstale":"2024-04-16T19:37:09.675266+0000","last_undegraded":"2024-04-16T19:37:09.675266+0000","last_fullsized":"2024-04-16T19:37:09.675266+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'117","last_scrub_stamp":"2024-04-16T19:37:09.675046+0000","last_deep_scrub":"53'117","last_deep_scrub_stamp":"2024-04-16T19:37:09.675046+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.675046+0000","objects_scrubbed":27,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:14.071688+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1428,"num_read_kb":1407,"num_write":124,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4717,"num_omap_keys":20,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.389594+0000","last_change":"2024-04-16T19:36:46.389594+0000","last_active":"2024-04-16T19:36:46.389594+0000","last_peered":"2024-04-16T19:36:46.389594+0000","last_clean":"2024-04-16T19:36:46.389594+0000","last_became_active":"2024-04-16T17:48:27.298340+0000","last_became_peered":"2024-04-16T17:48:27.298340+0000","last_unstale":"2024-04-16T19:36:46.389594+0000","last_undegraded":"2024-04-16T19:36:46.389594+0000","last_fullsized":"2024-04-16T19:36:46.389594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.389402+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.389402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:49.372119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"53'8","reported_seq":108,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.681828+0000","last_change":"2024-04-16T19:36:40.681959+0000","last_active":"2024-04-16T19:36:40.681828+0000","last_peered":"2024-04-16T19:36:40.681828+0000","last_clean":"2024-04-16T19:36:40.681828+0000","last_became_active":"2024-04-16T17:48:01.257867+0000","last_became_peered":"2024-04-16T17:48:01.257867+0000","last_unstale":"2024-04-16T19:36:40.681828+0000","last_undegraded":"2024-04-16T19:36:40.681828+0000","last_fullsized":"2024-04-16T19:36:40.681828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:40.681665+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.681665+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:15.914709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"53'556","reported_seq":1952,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.630262+0000","last_change":"2024-04-16T19:36:46.630326+0000","last_active":"2024-04-16T19:36:46.630262+0000","last_peered":"2024-04-16T19:36:46.630262+0000","last_clean":"2024-04-16T19:36:46.630262+0000","last_became_active":"2024-04-16T17:47:59.183515+0000","last_became_peered":"2024-04-16T17:47:59.183515+0000","last_unstale":"2024-04-16T19:36:46.630262+0000","last_undegraded":"2024-04-16T19:36:46.630262+0000","last_fullsized":"2024-04-16T19:36:46.630262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:46.630122+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.630122+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:11:54.931710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"53'676","reported_seq":995,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.635509+0000","last_change":"2024-04-16T19:36:20.635509+0000","last_active":"2024-04-16T19:36:20.635509+0000","last_peered":"2024-04-16T19:36:20.635509+0000","last_clean":"2024-04-16T19:36:20.635509+0000","last_became_active":"2024-04-16T17:47:57.193142+0000","last_became_peered":"2024-04-16T17:47:57.193142+0000","last_unstale":"2024-04-16T19:36:20.635509+0000","last_undegraded":"2024-04-16T19:36:20.635509+0000","last_fullsized":"2024-04-16T19:36:20.635509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'676","last_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_deep_scrub":"53'676","last_deep_scrub_stamp":"2024-04-16T19:36:20.635277+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635277+0000","objects_scrubbed":25,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:10:55.447676+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":861,"num_read_kb":852,"num_write":1082,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71899,"num_omap_keys":314,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"53'20","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.500999+0000","last_change":"2024-04-16T19:36:18.501046+0000","last_active":"2024-04-16T19:36:18.500999+0000","last_peered":"2024-04-16T19:36:18.500999+0000","last_clean":"2024-04-16T19:36:18.500999+0000","last_became_active":"2024-04-16T17:47:55.160786+0000","last_became_peered":"2024-04-16T17:47:55.160786+0000","last_unstale":"2024-04-16T19:36:18.500999+0000","last_undegraded":"2024-04-16T19:36:18.500999+0000","last_fullsized":"2024-04-16T19:36:18.500999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:18.500951+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.500951+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:17.147393+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":148,"num_read_kb":110,"num_write":87,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.634453+0000","last_change":"2024-04-16T19:37:10.634505+0000","last_active":"2024-04-16T19:37:10.634453+0000","last_peered":"2024-04-16T19:37:10.634453+0000","last_clean":"2024-04-16T19:37:10.634453+0000","last_became_active":"2024-04-16T17:47:23.260957+0000","last_became_peered":"2024-04-16T17:47:23.260957+0000","last_unstale":"2024-04-16T19:37:10.634453+0000","last_undegraded":"2024-04-16T19:37:10.634453+0000","last_fullsized":"2024-04-16T19:37:10.634453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:10.634314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:10.634314+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.634314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:37.001987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.453667+0000","last_change":"2024-04-16T19:37:22.453667+0000","last_active":"2024-04-16T19:37:22.453667+0000","last_peered":"2024-04-16T19:37:22.453667+0000","last_clean":"2024-04-16T19:37:22.453667+0000","last_became_active":"2024-04-16T17:47:21.232868+0000","last_became_peered":"2024-04-16T17:47:21.232868+0000","last_unstale":"2024-04-16T19:37:22.453667+0000","last_undegraded":"2024-04-16T19:37:22.453667+0000","last_fullsized":"2024-04-16T19:37:22.453667+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.453471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.453471+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.453471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:21.519156+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"53'4","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.622021+0000","last_change":"2024-04-16T19:37:17.622021+0000","last_active":"2024-04-16T19:37:17.622021+0000","last_peered":"2024-04-16T19:37:17.622021+0000","last_clean":"2024-04-16T19:37:17.622021+0000","last_became_active":"2024-04-16T17:47:33.294095+0000","last_became_peered":"2024-04-16T17:47:33.294095+0000","last_unstale":"2024-04-16T19:37:17.622021+0000","last_undegraded":"2024-04-16T19:37:17.622021+0000","last_fullsized":"2024-04-16T19:37:17.622021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:17.621822+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:17.621822+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.621822+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:46.534752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2509,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":16,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"53'184","reported_seq":399,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.363174+0000","last_change":"2024-04-16T19:37:05.363174+0000","last_active":"2024-04-16T19:37:05.363174+0000","last_peered":"2024-04-16T19:37:05.363174+0000","last_clean":"2024-04-16T19:37:05.363174+0000","last_became_active":"2024-04-16T17:47:35.316480+0000","last_became_peered":"2024-04-16T17:47:35.316480+0000","last_unstale":"2024-04-16T19:37:05.363174+0000","last_undegraded":"2024-04-16T19:37:05.363174+0000","last_fullsized":"2024-04-16T19:37:05.363174+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'184","last_scrub_stamp":"2024-04-16T19:37:05.362967+0000","last_deep_scrub":"53'184","last_deep_scrub_stamp":"2024-04-16T19:37:05.362967+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.362967+0000","objects_scrubbed":8,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:30.682003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":269,"num_read_kb":259,"num_write":176,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.486097+0000","last_change":"2024-04-16T19:36:58.486097+0000","last_active":"2024-04-16T19:36:58.486097+0000","last_peered":"2024-04-16T19:36:58.486097+0000","last_clean":"2024-04-16T19:36:58.486097+0000","last_became_active":"2024-04-16T17:47:37.332046+0000","last_became_peered":"2024-04-16T17:47:37.332046+0000","last_unstale":"2024-04-16T19:36:58.486097+0000","last_undegraded":"2024-04-16T19:36:58.486097+0000","last_fullsized":"2024-04-16T19:36:58.486097+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.485902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.485902+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.485902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:24.853575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.569888+0000","last_change":"2024-04-16T19:36:34.569940+0000","last_active":"2024-04-16T19:36:34.569888+0000","last_peered":"2024-04-16T19:36:34.569888+0000","last_clean":"2024-04-16T19:36:34.569888+0000","last_became_active":"2024-04-16T17:47:39.334592+0000","last_became_peered":"2024-04-16T17:47:39.334592+0000","last_unstale":"2024-04-16T19:36:34.569888+0000","last_undegraded":"2024-04-16T19:36:34.569888+0000","last_fullsized":"2024-04-16T19:36:34.569888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.569751+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.569751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:40.008977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.547899+0000","last_change":"2024-04-16T19:37:32.547957+0000","last_active":"2024-04-16T19:37:32.547899+0000","last_peered":"2024-04-16T19:37:32.547899+0000","last_clean":"2024-04-16T19:37:32.547899+0000","last_became_active":"2024-04-16T18:10:50.553701+0000","last_became_peered":"2024-04-16T18:10:50.553701+0000","last_unstale":"2024-04-16T19:37:32.547899+0000","last_undegraded":"2024-04-16T19:37:32.547899+0000","last_fullsized":"2024-04-16T19:37:32.547899+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.547754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.547754+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.547754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:30.887097+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"53'153","reported_seq":1548,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.638858+0000","last_change":"2024-04-16T19:36:56.638858+0000","last_active":"2024-04-16T19:36:56.638858+0000","last_peered":"2024-04-16T19:36:56.638858+0000","last_clean":"2024-04-16T19:36:56.638858+0000","last_became_active":"2024-04-16T17:48:34.327442+0000","last_became_peered":"2024-04-16T17:48:34.327442+0000","last_unstale":"2024-04-16T19:36:56.638858+0000","last_undegraded":"2024-04-16T19:36:56.638858+0000","last_fullsized":"2024-04-16T19:36:56.638858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'153","last_scrub_stamp":"2024-04-16T19:36:56.638728+0000","last_deep_scrub":"53'153","last_deep_scrub_stamp":"2024-04-16T19:36:56.638728+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.638728+0000","objects_scrubbed":35,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:45.633510+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2338,"num_read_kb":2302,"num_write":183,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11617,"num_omap_keys":48,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.606337+0000","last_change":"2024-04-16T19:36:37.606393+0000","last_active":"2024-04-16T19:36:37.606337+0000","last_peered":"2024-04-16T19:36:37.606337+0000","last_clean":"2024-04-16T19:36:37.606337+0000","last_became_active":"2024-04-16T17:48:27.296179+0000","last_became_peered":"2024-04-16T17:48:27.296179+0000","last_unstale":"2024-04-16T19:36:37.606337+0000","last_undegraded":"2024-04-16T19:36:37.606337+0000","last_fullsized":"2024-04-16T19:36:37.606337+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.606203+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.606203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:48.287865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"53'11","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.682939+0000","last_change":"2024-04-16T19:36:43.683003+0000","last_active":"2024-04-16T19:36:43.682939+0000","last_peered":"2024-04-16T19:36:43.682939+0000","last_clean":"2024-04-16T19:36:43.682939+0000","last_became_active":"2024-04-16T17:48:01.239804+0000","last_became_peered":"2024-04-16T17:48:01.239804+0000","last_unstale":"2024-04-16T19:36:43.682939+0000","last_undegraded":"2024-04-16T19:36:43.682939+0000","last_fullsized":"2024-04-16T19:36:43.682939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:43.682806+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.682806+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:30.180345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.630682+0000","last_change":"2024-04-16T19:36:48.630682+0000","last_active":"2024-04-16T19:36:48.630682+0000","last_peered":"2024-04-16T19:36:48.630682+0000","last_clean":"2024-04-16T19:36:48.630682+0000","last_became_active":"2024-04-16T17:47:59.181331+0000","last_became_peered":"2024-04-16T17:47:59.181331+0000","last_unstale":"2024-04-16T19:36:48.630682+0000","last_undegraded":"2024-04-16T19:36:48.630682+0000","last_fullsized":"2024-04-16T19:36:48.630682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.630423+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.630423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:08.839790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'804","reported_seq":43172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.600535+0000","last_change":"2024-04-16T19:36:25.600624+0000","last_active":"2024-04-16T19:36:25.600535+0000","last_peered":"2024-04-16T19:36:25.600535+0000","last_clean":"2024-04-16T19:36:25.600535+0000","last_became_active":"2024-04-16T17:47:57.185413+0000","last_became_peered":"2024-04-16T17:47:57.185413+0000","last_unstale":"2024-04-16T19:36:25.600535+0000","last_undegraded":"2024-04-16T19:36:25.600535+0000","last_fullsized":"2024-04-16T19:36:25.600535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'804","last_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_deep_scrub":"53'804","last_deep_scrub_stamp":"2024-04-16T19:36:25.600462+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.600462+0000","objects_scrubbed":21,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:05.088020+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":127713,"num_read_kb":85286,"num_write":987,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71960,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"53'12","reported_seq":221,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.489782+0000","last_change":"2024-04-16T19:36:49.489878+0000","last_active":"2024-04-16T19:36:49.489782+0000","last_peered":"2024-04-16T19:36:49.489782+0000","last_clean":"2024-04-16T19:36:49.489782+0000","last_became_active":"2024-04-16T17:47:55.143100+0000","last_became_peered":"2024-04-16T17:47:55.143100+0000","last_unstale":"2024-04-16T19:36:49.489782+0000","last_undegraded":"2024-04-16T19:36:49.489782+0000","last_fullsized":"2024-04-16T19:36:49.489782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:49.489640+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.489640+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:54.928866+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":172,"num_write":62,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.715185+0000","last_change":"2024-04-16T19:37:28.715238+0000","last_active":"2024-04-16T19:37:28.715185+0000","last_peered":"2024-04-16T19:37:28.715185+0000","last_clean":"2024-04-16T19:37:28.715185+0000","last_became_active":"2024-04-16T17:47:23.257634+0000","last_became_peered":"2024-04-16T17:47:23.257634+0000","last_unstale":"2024-04-16T19:37:28.715185+0000","last_undegraded":"2024-04-16T19:37:28.715185+0000","last_fullsized":"2024-04-16T19:37:28.715185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.715038+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:13.816322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.737864+0000","last_change":"2024-04-16T19:37:35.737965+0000","last_active":"2024-04-16T19:37:35.737864+0000","last_peered":"2024-04-16T19:37:35.737864+0000","last_clean":"2024-04-16T19:37:35.737864+0000","last_became_active":"2024-04-16T17:46:37.574126+0000","last_became_peered":"2024-04-16T17:46:37.574126+0000","last_unstale":"2024-04-16T19:37:35.737864+0000","last_undegraded":"2024-04-16T19:37:35.737864+0000","last_fullsized":"2024-04-16T19:37:35.737864+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.737665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.737665+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.737665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:14.578611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.627786+0000","last_change":"2024-04-16T19:37:22.627786+0000","last_active":"2024-04-16T19:37:22.627786+0000","last_peered":"2024-04-16T19:37:22.627786+0000","last_clean":"2024-04-16T19:37:22.627786+0000","last_became_active":"2024-04-16T17:47:21.250874+0000","last_became_peered":"2024-04-16T17:47:21.250874+0000","last_unstale":"2024-04-16T19:37:22.627786+0000","last_undegraded":"2024-04-16T19:37:22.627786+0000","last_fullsized":"2024-04-16T19:37:22.627786+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.627590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.627590+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.627590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:51.252771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.517076+0000","last_change":"2024-04-16T19:37:30.517182+0000","last_active":"2024-04-16T19:37:30.517076+0000","last_peered":"2024-04-16T19:37:30.517076+0000","last_clean":"2024-04-16T19:37:30.517076+0000","last_became_active":"2024-04-16T17:47:33.304529+0000","last_became_peered":"2024-04-16T17:47:33.304529+0000","last_unstale":"2024-04-16T19:37:30.517076+0000","last_undegraded":"2024-04-16T19:37:30.517076+0000","last_fullsized":"2024-04-16T19:37:30.517076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.516899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.516899+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.516899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:16.748285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"53'77","reported_seq":251,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.348075+0000","last_change":"2024-04-16T19:37:12.348141+0000","last_active":"2024-04-16T19:37:12.348075+0000","last_peered":"2024-04-16T19:37:12.348075+0000","last_clean":"2024-04-16T19:37:12.348075+0000","last_became_active":"2024-04-16T17:47:35.309313+0000","last_became_peered":"2024-04-16T17:47:35.309313+0000","last_unstale":"2024-04-16T19:37:12.348075+0000","last_undegraded":"2024-04-16T19:37:12.348075+0000","last_fullsized":"2024-04-16T19:37:12.348075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'77","last_scrub_stamp":"2024-04-16T19:37:12.348001+0000","last_deep_scrub":"53'77","last_deep_scrub_stamp":"2024-04-16T19:37:12.348001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.348001+0000","objects_scrubbed":5,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.056193+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":112,"num_write":72,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.566977+0000","last_change":"2024-04-16T19:36:59.567032+0000","last_active":"2024-04-16T19:36:59.566977+0000","last_peered":"2024-04-16T19:36:59.566977+0000","last_clean":"2024-04-16T19:36:59.566977+0000","last_became_active":"2024-04-16T17:47:37.327081+0000","last_became_peered":"2024-04-16T17:47:37.327081+0000","last_unstale":"2024-04-16T19:36:59.566977+0000","last_undegraded":"2024-04-16T19:36:59.566977+0000","last_fullsized":"2024-04-16T19:36:59.566977+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.566839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.566839+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.566839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:45.847122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.477027+0000","last_change":"2024-04-16T19:36:40.477078+0000","last_active":"2024-04-16T19:36:40.477027+0000","last_peered":"2024-04-16T19:36:40.477027+0000","last_clean":"2024-04-16T19:36:40.477027+0000","last_became_active":"2024-04-16T17:47:39.340486+0000","last_became_peered":"2024-04-16T17:47:39.340486+0000","last_unstale":"2024-04-16T19:36:40.477027+0000","last_undegraded":"2024-04-16T19:36:40.477027+0000","last_fullsized":"2024-04-16T19:36:40.477027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.476892+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.476892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:55.189016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:47.615572+0000","last_change":"2024-04-16T19:37:47.615624+0000","last_active":"2024-04-16T19:37:47.615572+0000","last_peered":"2024-04-16T19:37:47.615572+0000","last_clean":"2024-04-16T19:37:47.615572+0000","last_became_active":"2024-04-16T18:10:50.542621+0000","last_became_peered":"2024-04-16T18:10:50.542621+0000","last_unstale":"2024-04-16T19:37:47.615572+0000","last_undegraded":"2024-04-16T19:37:47.615572+0000","last_fullsized":"2024-04-16T19:37:47.615572+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:47.615431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:47.615431+0000","last_clean_scrub_stamp":"2024-04-16T19:37:47.615431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:50:46.552769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"53'140","reported_seq":1796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.660278+0000","last_change":"2024-04-16T19:37:00.660381+0000","last_active":"2024-04-16T19:37:00.660278+0000","last_peered":"2024-04-16T19:37:00.660278+0000","last_clean":"2024-04-16T19:37:00.660278+0000","last_became_active":"2024-04-16T17:48:34.328841+0000","last_became_peered":"2024-04-16T17:48:34.328841+0000","last_unstale":"2024-04-16T19:37:00.660278+0000","last_undegraded":"2024-04-16T19:37:00.660278+0000","last_fullsized":"2024-04-16T19:37:00.660278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'140","last_scrub_stamp":"2024-04-16T19:37:00.660142+0000","last_deep_scrub":"53'140","last_deep_scrub_stamp":"2024-04-16T19:37:00.660142+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.660142+0000","objects_scrubbed":40,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:07.318185+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2816,"num_read_kb":2781,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9228,"num_omap_keys":36,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.670649+0000","last_change":"2024-04-16T19:36:37.670700+0000","last_active":"2024-04-16T19:36:37.670649+0000","last_peered":"2024-04-16T19:36:37.670649+0000","last_clean":"2024-04-16T19:36:37.670649+0000","last_became_active":"2024-04-16T17:48:27.295782+0000","last_became_peered":"2024-04-16T17:48:27.295782+0000","last_unstale":"2024-04-16T19:36:37.670649+0000","last_undegraded":"2024-04-16T19:36:37.670649+0000","last_fullsized":"2024-04-16T19:36:37.670649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.670504+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.670504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:43.115925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"53'14","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.601233+0000","last_change":"2024-04-16T19:36:56.601312+0000","last_active":"2024-04-16T19:36:56.601233+0000","last_peered":"2024-04-16T19:36:56.601233+0000","last_clean":"2024-04-16T19:36:56.601233+0000","last_became_active":"2024-04-16T17:48:01.256570+0000","last_became_peered":"2024-04-16T17:48:01.256570+0000","last_unstale":"2024-04-16T19:36:56.601233+0000","last_undegraded":"2024-04-16T19:36:56.601233+0000","last_fullsized":"2024-04-16T19:36:56.601233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:36:56.601090+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:36:56.601090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.601090+0000","objects_scrubbed":6,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:12.535988+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":73,"num_write":23,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.465640+0000","last_change":"2024-04-16T19:36:41.465695+0000","last_active":"2024-04-16T19:36:41.465640+0000","last_peered":"2024-04-16T19:36:41.465640+0000","last_clean":"2024-04-16T19:36:41.465640+0000","last_became_active":"2024-04-16T17:47:59.181583+0000","last_became_peered":"2024-04-16T17:47:59.181583+0000","last_unstale":"2024-04-16T19:36:41.465640+0000","last_undegraded":"2024-04-16T19:36:41.465640+0000","last_fullsized":"2024-04-16T19:36:41.465640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.465493+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.465493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:42:13.407368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"53'925","reported_seq":3830,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.639219+0000","last_change":"2024-04-16T19:36:25.639219+0000","last_active":"2024-04-16T19:36:25.639219+0000","last_peered":"2024-04-16T19:36:25.639219+0000","last_clean":"2024-04-16T19:36:25.639219+0000","last_became_active":"2024-04-16T17:47:57.178399+0000","last_became_peered":"2024-04-16T17:47:57.178399+0000","last_unstale":"2024-04-16T19:36:25.639219+0000","last_undegraded":"2024-04-16T19:36:25.639219+0000","last_fullsized":"2024-04-16T19:36:25.639219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'925","last_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_deep_scrub":"53'925","last_deep_scrub_stamp":"2024-04-16T19:36:25.639013+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.639013+0000","objects_scrubbed":28,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:11.472224+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3800,"num_read_kb":3696,"num_write":1305,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89384,"num_omap_keys":416,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.9","version":"53'36","reported_seq":219,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.510632+0000","last_change":"2024-04-16T19:36:24.510731+0000","last_active":"2024-04-16T19:36:24.510632+0000","last_peered":"2024-04-16T19:36:24.510632+0000","last_clean":"2024-04-16T19:36:24.510632+0000","last_became_active":"2024-04-16T17:47:55.165440+0000","last_became_peered":"2024-04-16T17:47:55.165440+0000","last_unstale":"2024-04-16T19:36:24.510632+0000","last_undegraded":"2024-04-16T19:36:24.510632+0000","last_fullsized":"2024-04-16T19:36:24.510632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'36","last_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_deep_scrub":"53'36","last_deep_scrub_stamp":"2024-04-16T19:36:24.510495+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.510495+0000","objects_scrubbed":7,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:21.114823+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":214,"num_read_kb":185,"num_write":191,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.586965+0000","last_change":"2024-04-16T19:37:16.586965+0000","last_active":"2024-04-16T19:37:16.586965+0000","last_peered":"2024-04-16T19:37:16.586965+0000","last_clean":"2024-04-16T19:37:16.586965+0000","last_became_active":"2024-04-16T17:47:23.259904+0000","last_became_peered":"2024-04-16T17:47:23.259904+0000","last_unstale":"2024-04-16T19:37:16.586965+0000","last_undegraded":"2024-04-16T19:37:16.586965+0000","last_fullsized":"2024-04-16T19:37:16.586965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:16.586769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:16.586769+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.586769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:42:14.606566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:37.441580+0000","last_change":"2024-04-16T19:37:37.441678+0000","last_active":"2024-04-16T19:37:37.441580+0000","last_peered":"2024-04-16T19:37:37.441580+0000","last_clean":"2024-04-16T19:37:37.441580+0000","last_became_active":"2024-04-16T17:46:37.574157+0000","last_became_peered":"2024-04-16T17:46:37.574157+0000","last_unstale":"2024-04-16T19:37:37.441580+0000","last_undegraded":"2024-04-16T19:37:37.441580+0000","last_fullsized":"2024-04-16T19:37:37.441580+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:37.441382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:37.441382+0000","last_clean_scrub_stamp":"2024-04-16T19:37:37.441382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:55.036887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.485823+0000","last_change":"2024-04-16T19:37:26.485823+0000","last_active":"2024-04-16T19:37:26.485823+0000","last_peered":"2024-04-16T19:37:26.485823+0000","last_clean":"2024-04-16T19:37:26.485823+0000","last_became_active":"2024-04-16T17:47:21.234031+0000","last_became_peered":"2024-04-16T17:47:21.234031+0000","last_unstale":"2024-04-16T19:37:26.485823+0000","last_undegraded":"2024-04-16T19:37:26.485823+0000","last_fullsized":"2024-04-16T19:37:26.485823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.485632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.485632+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.485632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:20.193824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"53'2","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.638661+0000","last_change":"2024-04-16T19:37:16.638790+0000","last_active":"2024-04-16T19:37:16.638661+0000","last_peered":"2024-04-16T19:37:16.638661+0000","last_clean":"2024-04-16T19:37:16.638661+0000","last_became_active":"2024-04-16T17:47:33.302758+0000","last_became_peered":"2024-04-16T17:47:33.302758+0000","last_unstale":"2024-04-16T19:37:16.638661+0000","last_undegraded":"2024-04-16T19:37:16.638661+0000","last_fullsized":"2024-04-16T19:37:16.638661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:16.638491+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:16.638491+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.638491+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:45.551421+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"53'150","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.635861+0000","last_change":"2024-04-16T19:37:01.635936+0000","last_active":"2024-04-16T19:37:01.635861+0000","last_peered":"2024-04-16T19:37:01.635861+0000","last_clean":"2024-04-16T19:37:01.635861+0000","last_became_active":"2024-04-16T17:47:35.313689+0000","last_became_peered":"2024-04-16T17:47:35.313689+0000","last_unstale":"2024-04-16T19:37:01.635861+0000","last_undegraded":"2024-04-16T19:37:01.635861+0000","last_fullsized":"2024-04-16T19:37:01.635861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'150","last_scrub_stamp":"2024-04-16T19:37:01.635730+0000","last_deep_scrub":"53'150","last_deep_scrub_stamp":"2024-04-16T19:37:01.635730+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.635730+0000","objects_scrubbed":6,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:47:12.500214+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":210,"num_write":144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.356065+0000","last_change":"2024-04-16T19:36:43.356174+0000","last_active":"2024-04-16T19:36:43.356065+0000","last_peered":"2024-04-16T19:36:43.356065+0000","last_clean":"2024-04-16T19:36:43.356065+0000","last_became_active":"2024-04-16T17:47:37.333842+0000","last_became_peered":"2024-04-16T17:47:37.333842+0000","last_unstale":"2024-04-16T19:36:43.356065+0000","last_undegraded":"2024-04-16T19:36:43.356065+0000","last_fullsized":"2024-04-16T19:36:43.356065+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.355868+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.355868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:33.637719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.430570+0000","last_change":"2024-04-16T19:36:44.430621+0000","last_active":"2024-04-16T19:36:44.430570+0000","last_peered":"2024-04-16T19:36:44.430570+0000","last_clean":"2024-04-16T19:36:44.430570+0000","last_became_active":"2024-04-16T17:47:39.339075+0000","last_became_peered":"2024-04-16T17:47:39.339075+0000","last_unstale":"2024-04-16T19:36:44.430570+0000","last_undegraded":"2024-04-16T19:36:44.430570+0000","last_fullsized":"2024-04-16T19:36:44.430570+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.430438+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.430438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:03.942655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:51.570907+0000","last_change":"2024-04-16T19:37:51.570961+0000","last_active":"2024-04-16T19:37:51.570907+0000","last_peered":"2024-04-16T19:37:51.570907+0000","last_clean":"2024-04-16T19:37:51.570907+0000","last_became_active":"2024-04-16T18:10:50.557436+0000","last_became_peered":"2024-04-16T18:10:50.557436+0000","last_unstale":"2024-04-16T19:37:51.570907+0000","last_undegraded":"2024-04-16T19:37:51.570907+0000","last_fullsized":"2024-04-16T19:37:51.570907+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:51.570762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:51.570762+0000","last_clean_scrub_stamp":"2024-04-16T19:37:51.570762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:58:21.280239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"53'150","reported_seq":1258,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:47.577582+0000","last_change":"2024-04-16T19:37:47.577663+0000","last_active":"2024-04-16T19:37:47.577582+0000","last_peered":"2024-04-16T19:37:47.577582+0000","last_clean":"2024-04-16T19:37:47.577582+0000","last_became_active":"2024-04-16T17:48:34.321427+0000","last_became_peered":"2024-04-16T17:48:34.321427+0000","last_unstale":"2024-04-16T19:37:47.577582+0000","last_undegraded":"2024-04-16T19:37:47.577582+0000","last_fullsized":"2024-04-16T19:37:47.577582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'150","last_scrub_stamp":"2024-04-16T19:37:47.577431+0000","last_deep_scrub":"53'150","last_deep_scrub_stamp":"2024-04-16T19:37:47.577431+0000","last_clean_scrub_stamp":"2024-04-16T19:37:47.577431+0000","objects_scrubbed":31,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:29.668324+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1879,"num_read_kb":1895,"num_write":198,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7546,"num_omap_keys":32,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.359735+0000","last_change":"2024-04-16T19:37:07.359795+0000","last_active":"2024-04-16T19:37:07.359735+0000","last_peered":"2024-04-16T19:37:07.359735+0000","last_clean":"2024-04-16T19:37:07.359735+0000","last_became_active":"2024-04-16T17:48:27.306361+0000","last_became_peered":"2024-04-16T17:48:27.306361+0000","last_unstale":"2024-04-16T19:37:07.359735+0000","last_undegraded":"2024-04-16T19:37:07.359735+0000","last_fullsized":"2024-04-16T19:37:07.359735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.359591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.359591+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.359591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:05.698934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"53'42","reported_seq":213,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.456089+0000","last_change":"2024-04-16T19:36:54.456164+0000","last_active":"2024-04-16T19:36:54.456089+0000","last_peered":"2024-04-16T19:36:54.456089+0000","last_clean":"2024-04-16T19:36:54.456089+0000","last_became_active":"2024-04-16T17:48:01.248610+0000","last_became_peered":"2024-04-16T17:48:01.248610+0000","last_unstale":"2024-04-16T19:36:54.456089+0000","last_undegraded":"2024-04-16T19:36:54.456089+0000","last_fullsized":"2024-04-16T19:36:54.456089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'42","last_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_deep_scrub":"53'42","last_deep_scrub_stamp":"2024-04-16T19:36:54.455949+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.455949+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:53.521634+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":178,"num_read_kb":158,"num_write":71,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.526931+0000","last_change":"2024-04-16T19:36:34.526983+0000","last_active":"2024-04-16T19:36:34.526931+0000","last_peered":"2024-04-16T19:36:34.526931+0000","last_clean":"2024-04-16T19:36:34.526931+0000","last_became_active":"2024-04-16T17:47:59.189729+0000","last_became_peered":"2024-04-16T17:47:59.189729+0000","last_unstale":"2024-04-16T19:36:34.526931+0000","last_undegraded":"2024-04-16T19:36:34.526931+0000","last_fullsized":"2024-04-16T19:36:34.526931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.526798+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.526798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:20.876919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"53'282","reported_seq":668,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.418934+0000","last_change":"2024-04-16T19:36:30.419051+0000","last_active":"2024-04-16T19:36:30.418934+0000","last_peered":"2024-04-16T19:36:30.418934+0000","last_clean":"2024-04-16T19:36:30.418934+0000","last_became_active":"2024-04-16T17:47:57.189026+0000","last_became_peered":"2024-04-16T17:47:57.189026+0000","last_unstale":"2024-04-16T19:36:30.418934+0000","last_undegraded":"2024-04-16T19:36:30.418934+0000","last_fullsized":"2024-04-16T19:36:30.418934+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'282","last_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_deep_scrub":"53'282","last_deep_scrub_stamp":"2024-04-16T19:36:30.418793+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.418793+0000","objects_scrubbed":18,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:10.455320+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":568,"num_read_kb":557,"num_write":356,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"53'16","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.403701+0000","last_change":"2024-04-16T19:36:29.403701+0000","last_active":"2024-04-16T19:36:29.403701+0000","last_peered":"2024-04-16T19:36:29.403701+0000","last_clean":"2024-04-16T19:36:29.403701+0000","last_became_active":"2024-04-16T17:47:55.161225+0000","last_became_peered":"2024-04-16T17:47:55.161225+0000","last_unstale":"2024-04-16T19:36:29.403701+0000","last_undegraded":"2024-04-16T19:36:29.403701+0000","last_fullsized":"2024-04-16T19:36:29.403701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:29.403484+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.403484+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:20.878671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":103,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.632975+0000","last_change":"2024-04-16T19:37:13.632975+0000","last_active":"2024-04-16T19:37:13.632975+0000","last_peered":"2024-04-16T19:37:13.632975+0000","last_clean":"2024-04-16T19:37:13.632975+0000","last_became_active":"2024-04-16T17:47:23.265363+0000","last_became_peered":"2024-04-16T17:47:23.265363+0000","last_unstale":"2024-04-16T19:37:13.632975+0000","last_undegraded":"2024-04-16T19:37:13.632975+0000","last_fullsized":"2024-04-16T19:37:13.632975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.632699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.632699+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.632699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:35.211444+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.524869+0000","last_change":"2024-04-16T19:37:22.524969+0000","last_active":"2024-04-16T19:37:22.524869+0000","last_peered":"2024-04-16T19:37:22.524869+0000","last_clean":"2024-04-16T19:37:22.524869+0000","last_became_active":"2024-04-16T17:46:37.574144+0000","last_became_peered":"2024-04-16T17:46:37.574144+0000","last_unstale":"2024-04-16T19:37:22.524869+0000","last_undegraded":"2024-04-16T19:37:22.524869+0000","last_fullsized":"2024-04-16T19:37:22.524869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.524669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.524669+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.524669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:11.514839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.624408+0000","last_change":"2024-04-16T19:37:28.624408+0000","last_active":"2024-04-16T19:37:28.624408+0000","last_peered":"2024-04-16T19:37:28.624408+0000","last_clean":"2024-04-16T19:37:28.624408+0000","last_became_active":"2024-04-16T17:47:21.229279+0000","last_became_peered":"2024-04-16T17:47:21.229279+0000","last_unstale":"2024-04-16T19:37:28.624408+0000","last_undegraded":"2024-04-16T19:37:28.624408+0000","last_fullsized":"2024-04-16T19:37:28.624408+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.624222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.624222+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.624222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:57.537152+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"53'6","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.409314+0000","last_change":"2024-04-16T19:37:10.409384+0000","last_active":"2024-04-16T19:37:10.409314+0000","last_peered":"2024-04-16T19:37:10.409314+0000","last_clean":"2024-04-16T19:37:10.409314+0000","last_became_active":"2024-04-16T17:47:33.300406+0000","last_became_peered":"2024-04-16T17:47:33.300406+0000","last_unstale":"2024-04-16T19:37:10.409314+0000","last_undegraded":"2024-04-16T19:37:10.409314+0000","last_fullsized":"2024-04-16T19:37:10.409314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:10.409173+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:10.409173+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.409173+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:37.030711+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1412,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":11,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"53'159","reported_seq":1573,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.333313+0000","last_change":"2024-04-16T19:37:06.333399+0000","last_active":"2024-04-16T19:37:06.333313+0000","last_peered":"2024-04-16T19:37:06.333313+0000","last_clean":"2024-04-16T19:37:06.333313+0000","last_became_active":"2024-04-16T17:47:35.315876+0000","last_became_peered":"2024-04-16T17:47:35.315876+0000","last_unstale":"2024-04-16T19:37:06.333313+0000","last_undegraded":"2024-04-16T19:37:06.333313+0000","last_fullsized":"2024-04-16T19:37:06.333313+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:06.333171+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:06.333171+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.333171+0000","objects_scrubbed":8,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:29.002269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":220,"num_read_kb":212,"num_write":145,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.544890+0000","last_change":"2024-04-16T19:37:02.544990+0000","last_active":"2024-04-16T19:37:02.544890+0000","last_peered":"2024-04-16T19:37:02.544890+0000","last_clean":"2024-04-16T19:37:02.544890+0000","last_became_active":"2024-04-16T17:47:37.325621+0000","last_became_peered":"2024-04-16T17:47:37.325621+0000","last_unstale":"2024-04-16T19:37:02.544890+0000","last_undegraded":"2024-04-16T19:37:02.544890+0000","last_fullsized":"2024-04-16T19:37:02.544890+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.544686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.544686+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.544686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:25.213783+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.593776+0000","last_change":"2024-04-16T19:36:40.593776+0000","last_active":"2024-04-16T19:36:40.593776+0000","last_peered":"2024-04-16T19:36:40.593776+0000","last_clean":"2024-04-16T19:36:40.593776+0000","last_became_active":"2024-04-16T17:47:39.329907+0000","last_became_peered":"2024-04-16T17:47:39.329907+0000","last_unstale":"2024-04-16T19:36:40.593776+0000","last_undegraded":"2024-04-16T19:36:40.593776+0000","last_fullsized":"2024-04-16T19:36:40.593776+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.593577+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.593577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.105795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.501636+0000","last_change":"2024-04-16T19:37:34.501687+0000","last_active":"2024-04-16T19:37:34.501636+0000","last_peered":"2024-04-16T19:37:34.501636+0000","last_clean":"2024-04-16T19:37:34.501636+0000","last_became_active":"2024-04-16T18:10:50.548135+0000","last_became_peered":"2024-04-16T18:10:50.548135+0000","last_unstale":"2024-04-16T19:37:34.501636+0000","last_undegraded":"2024-04-16T19:37:34.501636+0000","last_fullsized":"2024-04-16T19:37:34.501636+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.501500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.501500+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.501500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:08:03.414430+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.c","version":"53'106","reported_seq":1199,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.610166+0000","last_change":"2024-04-16T19:36:57.610238+0000","last_active":"2024-04-16T19:36:57.610166+0000","last_peered":"2024-04-16T19:36:57.610166+0000","last_clean":"2024-04-16T19:36:57.610166+0000","last_became_active":"2024-04-16T17:48:34.314212+0000","last_became_peered":"2024-04-16T17:48:34.314212+0000","last_unstale":"2024-04-16T19:36:57.610166+0000","last_undegraded":"2024-04-16T19:36:57.610166+0000","last_fullsized":"2024-04-16T19:36:57.610166+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'106","last_scrub_stamp":"2024-04-16T19:36:57.610032+0000","last_deep_scrub":"53'106","last_deep_scrub_stamp":"2024-04-16T19:36:57.610032+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.610032+0000","objects_scrubbed":30,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:04.921796+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1812,"num_read_kb":1784,"num_write":110,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6018,"num_omap_keys":24,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.396377+0000","last_change":"2024-04-16T19:36:50.396377+0000","last_active":"2024-04-16T19:36:50.396377+0000","last_peered":"2024-04-16T19:36:50.396377+0000","last_clean":"2024-04-16T19:36:50.396377+0000","last_became_active":"2024-04-16T17:48:27.298568+0000","last_became_peered":"2024-04-16T17:48:27.298568+0000","last_unstale":"2024-04-16T19:36:50.396377+0000","last_undegraded":"2024-04-16T19:36:50.396377+0000","last_fullsized":"2024-04-16T19:36:50.396377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.396177+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.396177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:28.233570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"53'12","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.563153+0000","last_change":"2024-04-16T19:36:58.563232+0000","last_active":"2024-04-16T19:36:58.563153+0000","last_peered":"2024-04-16T19:36:58.563153+0000","last_clean":"2024-04-16T19:36:58.563153+0000","last_became_active":"2024-04-16T17:48:01.255458+0000","last_became_peered":"2024-04-16T17:48:01.255458+0000","last_unstale":"2024-04-16T19:36:58.563153+0000","last_undegraded":"2024-04-16T19:36:58.563153+0000","last_fullsized":"2024-04-16T19:36:58.563153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:58.563020+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:58.563020+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.563020+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:47:33.896197+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":51,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"53'556","reported_seq":1943,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.469613+0000","last_change":"2024-04-16T19:36:37.469677+0000","last_active":"2024-04-16T19:36:37.469613+0000","last_peered":"2024-04-16T19:36:37.469613+0000","last_clean":"2024-04-16T19:36:37.469613+0000","last_became_active":"2024-04-16T17:47:59.182909+0000","last_became_peered":"2024-04-16T17:47:59.182909+0000","last_unstale":"2024-04-16T19:36:37.469613+0000","last_undegraded":"2024-04-16T19:36:37.469613+0000","last_fullsized":"2024-04-16T19:36:37.469613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:37.469478+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.469478+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:42.920753+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"53'327","reported_seq":654,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.537153+0000","last_change":"2024-04-16T19:36:23.537297+0000","last_active":"2024-04-16T19:36:23.537153+0000","last_peered":"2024-04-16T19:36:23.537153+0000","last_clean":"2024-04-16T19:36:23.537153+0000","last_became_active":"2024-04-16T17:47:57.170382+0000","last_became_peered":"2024-04-16T17:47:57.170382+0000","last_unstale":"2024-04-16T19:36:23.537153+0000","last_undegraded":"2024-04-16T19:36:23.537153+0000","last_fullsized":"2024-04-16T19:36:23.537153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'327","last_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_deep_scrub":"53'327","last_deep_scrub_stamp":"2024-04-16T19:36:23.537014+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.537014+0000","objects_scrubbed":21,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:14.538959+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":539,"num_read_kb":527,"num_write":350,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'7","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.487632+0000","last_change":"2024-04-16T19:36:18.487675+0000","last_active":"2024-04-16T19:36:18.487632+0000","last_peered":"2024-04-16T19:36:18.487632+0000","last_clean":"2024-04-16T19:36:18.487632+0000","last_became_active":"2024-04-16T17:47:55.176685+0000","last_became_peered":"2024-04-16T17:47:55.176685+0000","last_unstale":"2024-04-16T19:36:18.487632+0000","last_undegraded":"2024-04-16T19:36:18.487632+0000","last_fullsized":"2024-04-16T19:36:18.487632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:18.487557+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.487557+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:23:46.066228+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.677602+0000","last_change":"2024-04-16T19:37:29.677602+0000","last_active":"2024-04-16T19:37:29.677602+0000","last_peered":"2024-04-16T19:37:29.677602+0000","last_clean":"2024-04-16T19:37:29.677602+0000","last_became_active":"2024-04-16T17:47:23.266877+0000","last_became_peered":"2024-04-16T17:47:23.266877+0000","last_unstale":"2024-04-16T19:37:29.677602+0000","last_undegraded":"2024-04-16T19:37:29.677602+0000","last_fullsized":"2024-04-16T19:37:29.677602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.677414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.677414+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.677414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:34.454899+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.592078+0000","last_change":"2024-04-16T19:37:25.592131+0000","last_active":"2024-04-16T19:37:25.592078+0000","last_peered":"2024-04-16T19:37:25.592078+0000","last_clean":"2024-04-16T19:37:25.592078+0000","last_became_active":"2024-04-16T17:46:37.510980+0000","last_became_peered":"2024-04-16T17:46:37.510980+0000","last_unstale":"2024-04-16T19:37:25.592078+0000","last_undegraded":"2024-04-16T19:37:25.592078+0000","last_fullsized":"2024-04-16T19:37:25.592078+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.591940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.591940+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.591940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:53.199527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"53'119","reported_seq":304,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:36.695771+0000","last_change":"2024-04-16T19:37:36.695908+0000","last_active":"2024-04-16T19:37:36.695771+0000","last_peered":"2024-04-16T19:37:36.695771+0000","last_clean":"2024-04-16T19:37:36.695771+0000","last_became_active":"2024-04-16T17:46:38.504858+0000","last_became_peered":"2024-04-16T17:46:38.504858+0000","last_unstale":"2024-04-16T19:37:36.695771+0000","last_undegraded":"2024-04-16T19:37:36.695771+0000","last_fullsized":"2024-04-16T19:37:36.695771+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2024-04-16T19:37:36.695567+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2024-04-16T19:37:36.695567+0000","last_clean_scrub_stamp":"2024-04-16T19:37:36.695567+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:47:11.422551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:38.621943+0000","last_change":"2024-04-16T19:37:38.621943+0000","last_active":"2024-04-16T19:37:38.621943+0000","last_peered":"2024-04-16T19:37:38.621943+0000","last_clean":"2024-04-16T19:37:38.621943+0000","last_became_active":"2024-04-16T17:47:21.240580+0000","last_became_peered":"2024-04-16T17:47:21.240580+0000","last_unstale":"2024-04-16T19:37:38.621943+0000","last_undegraded":"2024-04-16T19:37:38.621943+0000","last_fullsized":"2024-04-16T19:37:38.621943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:38.621751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:38.621751+0000","last_clean_scrub_stamp":"2024-04-16T19:37:38.621751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:51.646462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"53'7","reported_seq":514,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.425097+0000","last_change":"2024-04-16T19:37:16.425217+0000","last_active":"2024-04-16T19:37:16.425097+0000","last_peered":"2024-04-16T19:37:16.425097+0000","last_clean":"2024-04-16T19:37:16.425097+0000","last_became_active":"2024-04-16T17:47:33.302543+0000","last_became_peered":"2024-04-16T17:47:33.302543+0000","last_unstale":"2024-04-16T19:37:16.425097+0000","last_undegraded":"2024-04-16T19:37:16.425097+0000","last_fullsized":"2024-04-16T19:37:16.425097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:16.424923+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:16.424923+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.424923+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:44.179170+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":393,"num_read_kb":391,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"53'187","reported_seq":1631,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.658006+0000","last_change":"2024-04-16T19:37:02.658104+0000","last_active":"2024-04-16T19:37:02.658006+0000","last_peered":"2024-04-16T19:37:02.658006+0000","last_clean":"2024-04-16T19:37:02.658006+0000","last_became_active":"2024-04-16T17:47:35.312064+0000","last_became_peered":"2024-04-16T17:47:35.312064+0000","last_unstale":"2024-04-16T19:37:02.658006+0000","last_undegraded":"2024-04-16T19:37:02.658006+0000","last_fullsized":"2024-04-16T19:37:02.658006+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'187","last_scrub_stamp":"2024-04-16T19:37:02.657862+0000","last_deep_scrub":"53'187","last_deep_scrub_stamp":"2024-04-16T19:37:02.657862+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.657862+0000","objects_scrubbed":10,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:00.007944+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":279,"num_read_kb":264,"num_write":170,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.591692+0000","last_change":"2024-04-16T19:36:54.591795+0000","last_active":"2024-04-16T19:36:54.591692+0000","last_peered":"2024-04-16T19:36:54.591692+0000","last_clean":"2024-04-16T19:36:54.591692+0000","last_became_active":"2024-04-16T17:47:37.327099+0000","last_became_peered":"2024-04-16T17:47:37.327099+0000","last_unstale":"2024-04-16T19:36:54.591692+0000","last_undegraded":"2024-04-16T19:36:54.591692+0000","last_fullsized":"2024-04-16T19:36:54.591692+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.591530+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.591530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:30.795070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.464452+0000","last_change":"2024-04-16T19:36:28.464452+0000","last_active":"2024-04-16T19:36:28.464452+0000","last_peered":"2024-04-16T19:36:28.464452+0000","last_clean":"2024-04-16T19:36:28.464452+0000","last_became_active":"2024-04-16T17:47:39.699557+0000","last_became_peered":"2024-04-16T17:47:39.699557+0000","last_unstale":"2024-04-16T19:36:28.464452+0000","last_undegraded":"2024-04-16T19:36:28.464452+0000","last_fullsized":"2024-04-16T19:36:28.464452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.464263+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.464263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:07.132103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":64,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.452959+0000","last_change":"2024-04-16T19:37:28.453064+0000","last_active":"2024-04-16T19:37:28.452959+0000","last_peered":"2024-04-16T19:37:28.452959+0000","last_clean":"2024-04-16T19:37:28.452959+0000","last_became_active":"2024-04-16T18:10:50.544002+0000","last_became_peered":"2024-04-16T18:10:50.544002+0000","last_unstale":"2024-04-16T19:37:28.452959+0000","last_undegraded":"2024-04-16T19:37:28.452959+0000","last_fullsized":"2024-04-16T19:37:28.452959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.452788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.452788+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.452788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:38.747249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"53'91","reported_seq":1216,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:45.648285+0000","last_change":"2024-04-16T19:37:45.648423+0000","last_active":"2024-04-16T19:37:45.648285+0000","last_peered":"2024-04-16T19:37:45.648285+0000","last_clean":"2024-04-16T19:37:45.648285+0000","last_became_active":"2024-04-16T17:48:34.344087+0000","last_became_peered":"2024-04-16T17:48:34.344087+0000","last_unstale":"2024-04-16T19:37:45.648285+0000","last_undegraded":"2024-04-16T19:37:45.648285+0000","last_fullsized":"2024-04-16T19:37:45.648285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'91","last_scrub_stamp":"2024-04-16T19:37:45.648119+0000","last_deep_scrub":"53'91","last_deep_scrub_stamp":"2024-04-16T19:37:45.648119+0000","last_clean_scrub_stamp":"2024-04-16T19:37:45.648119+0000","objects_scrubbed":27,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:04.334580+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1866,"num_read_kb":1844,"num_write":82,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3361,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:40.422295+0000","last_change":"2024-04-16T19:36:40.422295+0000","last_active":"2024-04-16T19:36:40.422295+0000","last_peered":"2024-04-16T19:36:40.422295+0000","last_clean":"2024-04-16T19:36:40.422295+0000","last_became_active":"2024-04-16T17:48:27.303354+0000","last_became_peered":"2024-04-16T17:48:27.303354+0000","last_unstale":"2024-04-16T19:36:40.422295+0000","last_undegraded":"2024-04-16T19:36:40.422295+0000","last_fullsized":"2024-04-16T19:36:40.422295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:40.422090+0000","last_clean_scrub_stamp":"2024-04-16T19:36:40.422090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:00:53.341423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"53'19","reported_seq":154,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.529264+0000","last_change":"2024-04-16T19:36:30.529412+0000","last_active":"2024-04-16T19:36:30.529264+0000","last_peered":"2024-04-16T19:36:30.529264+0000","last_clean":"2024-04-16T19:36:30.529264+0000","last_became_active":"2024-04-16T17:48:01.237436+0000","last_became_peered":"2024-04-16T17:48:01.237436+0000","last_unstale":"2024-04-16T19:36:30.529264+0000","last_undegraded":"2024-04-16T19:36:30.529264+0000","last_fullsized":"2024-04-16T19:36:30.529264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'19","last_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_deep_scrub":"53'19","last_deep_scrub_stamp":"2024-04-16T19:36:30.529097+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.529097+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:25.629336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"53'556","reported_seq":1933,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.659990+0000","last_change":"2024-04-16T19:36:28.660054+0000","last_active":"2024-04-16T19:36:28.659990+0000","last_peered":"2024-04-16T19:36:28.659990+0000","last_clean":"2024-04-16T19:36:28.659990+0000","last_became_active":"2024-04-16T17:47:59.187901+0000","last_became_peered":"2024-04-16T17:47:59.187901+0000","last_unstale":"2024-04-16T19:36:28.659990+0000","last_undegraded":"2024-04-16T19:36:28.659990+0000","last_fullsized":"2024-04-16T19:36:28.659990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:28.659827+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.659827+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:15:31.852118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"53'532","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:24.416491+0000","last_change":"2024-04-16T19:36:24.416491+0000","last_active":"2024-04-16T19:36:24.416491+0000","last_peered":"2024-04-16T19:36:24.416491+0000","last_clean":"2024-04-16T19:36:24.416491+0000","last_became_active":"2024-04-16T17:47:57.187220+0000","last_became_peered":"2024-04-16T17:47:57.187220+0000","last_unstale":"2024-04-16T19:36:24.416491+0000","last_undegraded":"2024-04-16T19:36:24.416491+0000","last_fullsized":"2024-04-16T19:36:24.416491+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2024-04-16T19:36:24.416318+0000","last_clean_scrub_stamp":"2024-04-16T19:36:24.416318+0000","objects_scrubbed":18,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:15.418263+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33836,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1657,"num_read_kb":1441,"num_write":765,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"53'6","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.488767+0000","last_change":"2024-04-16T19:36:23.488844+0000","last_active":"2024-04-16T19:36:23.488767+0000","last_peered":"2024-04-16T19:36:23.488767+0000","last_clean":"2024-04-16T19:36:23.488767+0000","last_became_active":"2024-04-16T17:47:55.172817+0000","last_became_peered":"2024-04-16T17:47:55.172817+0000","last_unstale":"2024-04-16T19:36:23.488767+0000","last_undegraded":"2024-04-16T19:36:23.488767+0000","last_fullsized":"2024-04-16T19:36:23.488767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:23.488637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.488637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:34:36.790606+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.610651+0000","last_change":"2024-04-16T19:37:10.610651+0000","last_active":"2024-04-16T19:37:10.610651+0000","last_peered":"2024-04-16T19:37:10.610651+0000","last_clean":"2024-04-16T19:37:10.610651+0000","last_became_active":"2024-04-16T17:47:23.265540+0000","last_became_peered":"2024-04-16T17:47:23.265540+0000","last_unstale":"2024-04-16T19:37:10.610651+0000","last_undegraded":"2024-04-16T19:37:10.610651+0000","last_fullsized":"2024-04-16T19:37:10.610651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:10.610381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:10.610381+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.610381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:44.952386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.387252+0000","last_change":"2024-04-16T19:37:03.387307+0000","last_active":"2024-04-16T19:37:03.387252+0000","last_peered":"2024-04-16T19:37:03.387252+0000","last_clean":"2024-04-16T19:37:03.387252+0000","last_became_active":"2024-04-16T17:47:32.272144+0000","last_became_peered":"2024-04-16T17:47:32.272144+0000","last_unstale":"2024-04-16T19:37:03.387252+0000","last_undegraded":"2024-04-16T19:37:03.387252+0000","last_fullsized":"2024-04-16T19:37:03.387252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.387109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.387109+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.387109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:37.698223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.615042+0000","last_change":"2024-04-16T19:37:26.615094+0000","last_active":"2024-04-16T19:37:26.615042+0000","last_peered":"2024-04-16T19:37:26.615042+0000","last_clean":"2024-04-16T19:37:26.615042+0000","last_became_active":"2024-04-16T17:46:37.510918+0000","last_became_peered":"2024-04-16T17:46:37.510918+0000","last_unstale":"2024-04-16T19:37:26.615042+0000","last_undegraded":"2024-04-16T19:37:26.615042+0000","last_fullsized":"2024-04-16T19:37:26.615042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.614905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.614905+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.614905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:31:34.425036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.417938+0000","last_change":"2024-04-16T19:37:06.418036+0000","last_active":"2024-04-16T19:37:06.417938+0000","last_peered":"2024-04-16T19:37:06.417938+0000","last_clean":"2024-04-16T19:37:06.417938+0000","last_became_active":"2024-04-16T17:47:33.299181+0000","last_became_peered":"2024-04-16T17:47:33.299181+0000","last_unstale":"2024-04-16T19:37:06.417938+0000","last_undegraded":"2024-04-16T19:37:06.417938+0000","last_fullsized":"2024-04-16T19:37:06.417938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:06.417783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:06.417783+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.417783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:31.811569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"53'152","reported_seq":364,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.627842+0000","last_change":"2024-04-16T19:37:03.627959+0000","last_active":"2024-04-16T19:37:03.627842+0000","last_peered":"2024-04-16T19:37:03.627842+0000","last_clean":"2024-04-16T19:37:03.627842+0000","last_became_active":"2024-04-16T17:47:35.304778+0000","last_became_peered":"2024-04-16T17:47:35.304778+0000","last_unstale":"2024-04-16T19:37:03.627842+0000","last_undegraded":"2024-04-16T19:37:03.627842+0000","last_fullsized":"2024-04-16T19:37:03.627842+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'152","last_scrub_stamp":"2024-04-16T19:37:03.627704+0000","last_deep_scrub":"53'152","last_deep_scrub_stamp":"2024-04-16T19:37:03.627704+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.627704+0000","objects_scrubbed":8,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:28:06.610421+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":231,"num_read_kb":217,"num_write":144,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.468997+0000","last_change":"2024-04-16T19:36:50.469053+0000","last_active":"2024-04-16T19:36:50.468997+0000","last_peered":"2024-04-16T19:36:50.468997+0000","last_clean":"2024-04-16T19:36:50.468997+0000","last_became_active":"2024-04-16T17:47:37.333663+0000","last_became_peered":"2024-04-16T17:47:37.333663+0000","last_unstale":"2024-04-16T19:36:50.468997+0000","last_undegraded":"2024-04-16T19:36:50.468997+0000","last_fullsized":"2024-04-16T19:36:50.468997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.468857+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.468857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:29.956415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.495312+0000","last_change":"2024-04-16T19:36:38.495312+0000","last_active":"2024-04-16T19:36:38.495312+0000","last_peered":"2024-04-16T19:36:38.495312+0000","last_clean":"2024-04-16T19:36:38.495312+0000","last_became_active":"2024-04-16T17:47:39.343237+0000","last_became_peered":"2024-04-16T17:47:39.343237+0000","last_unstale":"2024-04-16T19:36:38.495312+0000","last_undegraded":"2024-04-16T19:36:38.495312+0000","last_fullsized":"2024-04-16T19:36:38.495312+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.495125+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.495125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:12.806239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.435201+0000","last_change":"2024-04-16T19:37:27.435310+0000","last_active":"2024-04-16T19:37:27.435201+0000","last_peered":"2024-04-16T19:37:27.435201+0000","last_clean":"2024-04-16T19:37:27.435201+0000","last_became_active":"2024-04-16T18:10:50.538102+0000","last_became_peered":"2024-04-16T18:10:50.538102+0000","last_unstale":"2024-04-16T19:37:27.435201+0000","last_undegraded":"2024-04-16T19:37:27.435201+0000","last_fullsized":"2024-04-16T19:37:27.435201+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.435035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.435035+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.435035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:48.430839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"53'152","reported_seq":1804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.565726+0000","last_change":"2024-04-16T19:37:12.565726+0000","last_active":"2024-04-16T19:37:12.565726+0000","last_peered":"2024-04-16T19:37:12.565726+0000","last_clean":"2024-04-16T19:37:12.565726+0000","last_became_active":"2024-04-16T17:48:34.336653+0000","last_became_peered":"2024-04-16T17:48:34.336653+0000","last_unstale":"2024-04-16T19:37:12.565726+0000","last_undegraded":"2024-04-16T19:37:12.565726+0000","last_fullsized":"2024-04-16T19:37:12.565726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'152","last_scrub_stamp":"2024-04-16T19:37:12.565502+0000","last_deep_scrub":"53'152","last_deep_scrub_stamp":"2024-04-16T19:37:12.565502+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.565502+0000","objects_scrubbed":30,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:48.502447+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2939,"num_read_kb":2886,"num_write":180,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7790,"num_omap_keys":33,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.400809+0000","last_change":"2024-04-16T19:36:49.400809+0000","last_active":"2024-04-16T19:36:49.400809+0000","last_peered":"2024-04-16T19:36:49.400809+0000","last_clean":"2024-04-16T19:36:49.400809+0000","last_became_active":"2024-04-16T17:48:27.300851+0000","last_became_peered":"2024-04-16T17:48:27.300851+0000","last_unstale":"2024-04-16T19:36:49.400809+0000","last_undegraded":"2024-04-16T19:36:49.400809+0000","last_fullsized":"2024-04-16T19:36:49.400809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.400614+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.400614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:27:52.383331+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"53'20","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:31.530462+0000","last_change":"2024-04-16T19:36:31.530526+0000","last_active":"2024-04-16T19:36:31.530462+0000","last_peered":"2024-04-16T19:36:31.530462+0000","last_clean":"2024-04-16T19:36:31.530462+0000","last_became_active":"2024-04-16T17:48:01.230876+0000","last_became_peered":"2024-04-16T17:48:01.230876+0000","last_unstale":"2024-04-16T19:36:31.530462+0000","last_undegraded":"2024-04-16T19:36:31.530462+0000","last_fullsized":"2024-04-16T19:36:31.530462+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:31.530356+0000","last_clean_scrub_stamp":"2024-04-16T19:36:31.530356+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:33:36.382371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":77,"num_write":57,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.461572+0000","last_change":"2024-04-16T19:36:57.461629+0000","last_active":"2024-04-16T19:36:57.461572+0000","last_peered":"2024-04-16T19:36:57.461572+0000","last_clean":"2024-04-16T19:36:57.461572+0000","last_became_active":"2024-04-16T17:47:59.185180+0000","last_became_peered":"2024-04-16T17:47:59.185180+0000","last_unstale":"2024-04-16T19:36:57.461572+0000","last_undegraded":"2024-04-16T19:36:57.461572+0000","last_fullsized":"2024-04-16T19:36:57.461572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:57.461431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:57.461431+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.461431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:04.646645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"53'1182","reported_seq":1443,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.649769+0000","last_change":"2024-04-16T19:36:23.649928+0000","last_active":"2024-04-16T19:36:23.649769+0000","last_peered":"2024-04-16T19:36:23.649769+0000","last_clean":"2024-04-16T19:36:23.649769+0000","last_became_active":"2024-04-16T17:47:57.187667+0000","last_became_peered":"2024-04-16T17:47:57.187667+0000","last_unstale":"2024-04-16T19:36:23.649769+0000","last_undegraded":"2024-04-16T19:36:23.649769+0000","last_fullsized":"2024-04-16T19:36:23.649769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1182","last_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_deep_scrub":"53'1182","last_deep_scrub_stamp":"2024-04-16T19:36:23.649623+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.649623+0000","objects_scrubbed":23,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:09.999744+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":177,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":1282,"num_write":2135,"num_write_kb":947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":215927,"num_omap_keys":942,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'28","reported_seq":418,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.423045+0000","last_change":"2024-04-16T19:36:25.423158+0000","last_active":"2024-04-16T19:36:25.423045+0000","last_peered":"2024-04-16T19:36:25.423045+0000","last_clean":"2024-04-16T19:36:25.423045+0000","last_became_active":"2024-04-16T17:47:55.163049+0000","last_became_peered":"2024-04-16T17:47:55.163049+0000","last_unstale":"2024-04-16T19:36:25.423045+0000","last_undegraded":"2024-04-16T19:36:25.423045+0000","last_fullsized":"2024-04-16T19:36:25.423045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'28","last_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_deep_scrub":"53'28","last_deep_scrub_stamp":"2024-04-16T19:36:25.422911+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.422911+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:22.027239+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":717,"num_read_kb":429,"num_write":150,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.639003+0000","last_change":"2024-04-16T19:37:27.639056+0000","last_active":"2024-04-16T19:37:27.639003+0000","last_peered":"2024-04-16T19:37:27.639003+0000","last_clean":"2024-04-16T19:37:27.639003+0000","last_became_active":"2024-04-16T17:47:23.262340+0000","last_became_peered":"2024-04-16T17:47:23.262340+0000","last_unstale":"2024-04-16T19:37:27.639003+0000","last_undegraded":"2024-04-16T19:37:27.639003+0000","last_fullsized":"2024-04-16T19:37:27.639003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.638863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.638863+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.638863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:31:12.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.505891+0000","last_change":"2024-04-16T19:37:20.505891+0000","last_active":"2024-04-16T19:37:20.505891+0000","last_peered":"2024-04-16T19:37:20.505891+0000","last_clean":"2024-04-16T19:37:20.505891+0000","last_became_active":"2024-04-16T17:47:21.231899+0000","last_became_peered":"2024-04-16T17:47:21.231899+0000","last_unstale":"2024-04-16T19:37:20.505891+0000","last_undegraded":"2024-04-16T19:37:20.505891+0000","last_fullsized":"2024-04-16T19:37:20.505891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.505785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.505785+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.505785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:59:10.568615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.493276+0000","last_change":"2024-04-16T19:37:23.493276+0000","last_active":"2024-04-16T19:37:23.493276+0000","last_peered":"2024-04-16T19:37:23.493276+0000","last_clean":"2024-04-16T19:37:23.493276+0000","last_became_active":"2024-04-16T17:46:38.111413+0000","last_became_peered":"2024-04-16T17:46:38.111413+0000","last_unstale":"2024-04-16T19:37:23.493276+0000","last_undegraded":"2024-04-16T19:37:23.493276+0000","last_fullsized":"2024-04-16T19:37:23.493276+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.493001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.493001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.493001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:44.028508+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.543234+0000","last_change":"2024-04-16T19:37:14.543287+0000","last_active":"2024-04-16T19:37:14.543234+0000","last_peered":"2024-04-16T19:37:14.543234+0000","last_clean":"2024-04-16T19:37:14.543234+0000","last_became_active":"2024-04-16T17:47:33.300256+0000","last_became_peered":"2024-04-16T17:47:33.300256+0000","last_unstale":"2024-04-16T19:37:14.543234+0000","last_undegraded":"2024-04-16T19:37:14.543234+0000","last_fullsized":"2024-04-16T19:37:14.543234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.543097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.543097+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.543097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:55.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"53'125","reported_seq":310,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.486453+0000","last_change":"2024-04-16T19:37:10.486531+0000","last_active":"2024-04-16T19:37:10.486453+0000","last_peered":"2024-04-16T19:37:10.486453+0000","last_clean":"2024-04-16T19:37:10.486453+0000","last_became_active":"2024-04-16T17:47:35.313079+0000","last_became_peered":"2024-04-16T17:47:35.313079+0000","last_unstale":"2024-04-16T19:37:10.486453+0000","last_undegraded":"2024-04-16T19:37:10.486453+0000","last_fullsized":"2024-04-16T19:37:10.486453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:37:10.486312+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:37:10.486312+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.486312+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:14.882954+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.688114+0000","last_change":"2024-04-16T19:36:44.688178+0000","last_active":"2024-04-16T19:36:44.688114+0000","last_peered":"2024-04-16T19:36:44.688114+0000","last_clean":"2024-04-16T19:36:44.688114+0000","last_became_active":"2024-04-16T17:47:37.328854+0000","last_became_peered":"2024-04-16T17:47:37.328854+0000","last_unstale":"2024-04-16T19:36:44.688114+0000","last_undegraded":"2024-04-16T19:36:44.688114+0000","last_fullsized":"2024-04-16T19:36:44.688114+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:44.687973+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.687973+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:44.814523+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.534176+0000","last_change":"2024-04-16T19:36:35.534176+0000","last_active":"2024-04-16T19:36:35.534176+0000","last_peered":"2024-04-16T19:36:35.534176+0000","last_clean":"2024-04-16T19:36:35.534176+0000","last_became_active":"2024-04-16T17:47:39.697063+0000","last_became_peered":"2024-04-16T17:47:39.697063+0000","last_unstale":"2024-04-16T19:36:35.534176+0000","last_undegraded":"2024-04-16T19:36:35.534176+0000","last_fullsized":"2024-04-16T19:36:35.534176+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.533984+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.533984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:55.743351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":65,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.406770+0000","last_change":"2024-04-16T19:37:00.406770+0000","last_active":"2024-04-16T19:37:00.406770+0000","last_peered":"2024-04-16T19:37:00.406770+0000","last_clean":"2024-04-16T19:37:00.406770+0000","last_became_active":"2024-04-16T18:10:50.536359+0000","last_became_peered":"2024-04-16T18:10:50.536359+0000","last_unstale":"2024-04-16T19:37:00.406770+0000","last_undegraded":"2024-04-16T19:37:00.406770+0000","last_fullsized":"2024-04-16T19:37:00.406770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.406670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.406670+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.406670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:05.819397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"53'120","reported_seq":1322,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:46.555877+0000","last_change":"2024-04-16T19:37:46.555970+0000","last_active":"2024-04-16T19:37:46.555877+0000","last_peered":"2024-04-16T19:37:46.555877+0000","last_clean":"2024-04-16T19:37:46.555877+0000","last_became_active":"2024-04-16T17:48:34.320926+0000","last_became_peered":"2024-04-16T17:48:34.320926+0000","last_unstale":"2024-04-16T19:37:46.555877+0000","last_undegraded":"2024-04-16T19:37:46.555877+0000","last_fullsized":"2024-04-16T19:37:46.555877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'120","last_scrub_stamp":"2024-04-16T19:37:46.555734+0000","last_deep_scrub":"53'120","last_deep_scrub_stamp":"2024-04-16T19:37:46.555734+0000","last_clean_scrub_stamp":"2024-04-16T19:37:46.555734+0000","objects_scrubbed":29,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:49:30.953312+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1911,"num_read_kb":1934,"num_write":131,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7885,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.504798+0000","last_change":"2024-04-16T19:36:44.504798+0000","last_active":"2024-04-16T19:36:44.504798+0000","last_peered":"2024-04-16T19:36:44.504798+0000","last_clean":"2024-04-16T19:36:44.504798+0000","last_became_active":"2024-04-16T17:48:27.305085+0000","last_became_peered":"2024-04-16T17:48:27.305085+0000","last_unstale":"2024-04-16T19:36:44.504798+0000","last_undegraded":"2024-04-16T19:36:44.504798+0000","last_fullsized":"2024-04-16T19:36:44.504798+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.504608+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.504608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:08:17.020106+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"53'21","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.684658+0000","last_change":"2024-04-16T19:36:52.684736+0000","last_active":"2024-04-16T19:36:52.684658+0000","last_peered":"2024-04-16T19:36:52.684658+0000","last_clean":"2024-04-16T19:36:52.684658+0000","last_became_active":"2024-04-16T17:48:01.262898+0000","last_became_peered":"2024-04-16T17:48:01.262898+0000","last_unstale":"2024-04-16T19:36:52.684658+0000","last_undegraded":"2024-04-16T19:36:52.684658+0000","last_fullsized":"2024-04-16T19:36:52.684658+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:52.684531+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.684531+0000","objects_scrubbed":8,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:32.649666+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":163,"num_read_kb":132,"num_write":39,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.646374+0000","last_change":"2024-04-16T19:36:45.646430+0000","last_active":"2024-04-16T19:36:45.646374+0000","last_peered":"2024-04-16T19:36:45.646374+0000","last_clean":"2024-04-16T19:36:45.646374+0000","last_became_active":"2024-04-16T17:47:59.188147+0000","last_became_peered":"2024-04-16T17:47:59.188147+0000","last_unstale":"2024-04-16T19:36:45.646374+0000","last_undegraded":"2024-04-16T19:36:45.646374+0000","last_fullsized":"2024-04-16T19:36:45.646374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.646241+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.646241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:48.502408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1343","reported_seq":1906,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.619403+0000","last_change":"2024-04-16T19:36:21.619494+0000","last_active":"2024-04-16T19:36:21.619403+0000","last_peered":"2024-04-16T19:36:21.619403+0000","last_clean":"2024-04-16T19:36:21.619403+0000","last_became_active":"2024-04-16T17:47:57.191674+0000","last_became_peered":"2024-04-16T17:47:57.191674+0000","last_unstale":"2024-04-16T19:36:21.619403+0000","last_undegraded":"2024-04-16T19:36:21.619403+0000","last_fullsized":"2024-04-16T19:36:21.619403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1343","last_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_deep_scrub":"53'1343","last_deep_scrub_stamp":"2024-04-16T19:36:21.619266+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.619266+0000","objects_scrubbed":29,"log_size":1343,"log_dups_size":0,"ondisk_log_size":1343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:34.145789+0000","scrub_duration":71,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1744,"num_read_kb":1730,"num_write":2367,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":216307,"num_omap_keys":944,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"53'8","reported_seq":184,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.444455+0000","last_change":"2024-04-16T19:36:33.444542+0000","last_active":"2024-04-16T19:36:33.444455+0000","last_peered":"2024-04-16T19:36:33.444455+0000","last_clean":"2024-04-16T19:36:33.444455+0000","last_became_active":"2024-04-16T17:47:55.156249+0000","last_became_peered":"2024-04-16T17:47:55.156249+0000","last_unstale":"2024-04-16T19:36:33.444455+0000","last_undegraded":"2024-04-16T19:36:33.444455+0000","last_fullsized":"2024-04-16T19:36:33.444455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:33.444316+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.444316+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:36.300483+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":121,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.653965+0000","last_change":"2024-04-16T19:37:21.653965+0000","last_active":"2024-04-16T19:37:21.653965+0000","last_peered":"2024-04-16T19:37:21.653965+0000","last_clean":"2024-04-16T19:37:21.653965+0000","last_became_active":"2024-04-16T17:47:23.259573+0000","last_became_peered":"2024-04-16T17:47:23.259573+0000","last_unstale":"2024-04-16T19:37:21.653965+0000","last_undegraded":"2024-04-16T19:37:21.653965+0000","last_fullsized":"2024-04-16T19:37:21.653965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.653770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.653770+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.653770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:17.539808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.615161+0000","last_change":"2024-04-16T19:37:29.615161+0000","last_active":"2024-04-16T19:37:29.615161+0000","last_peered":"2024-04-16T19:37:29.615161+0000","last_clean":"2024-04-16T19:37:29.615161+0000","last_became_active":"2024-04-16T17:47:21.230471+0000","last_became_peered":"2024-04-16T17:47:21.230471+0000","last_unstale":"2024-04-16T19:37:29.615161+0000","last_undegraded":"2024-04-16T19:37:29.615161+0000","last_fullsized":"2024-04-16T19:37:29.615161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.614969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.614969+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.614969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:15:06.350732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.698876+0000","last_change":"2024-04-16T19:37:34.698992+0000","last_active":"2024-04-16T19:37:34.698876+0000","last_peered":"2024-04-16T19:37:34.698876+0000","last_clean":"2024-04-16T19:37:34.698876+0000","last_became_active":"2024-04-16T17:46:37.574257+0000","last_became_peered":"2024-04-16T17:46:37.574257+0000","last_unstale":"2024-04-16T19:37:34.698876+0000","last_undegraded":"2024-04-16T19:37:34.698876+0000","last_fullsized":"2024-04-16T19:37:34.698876+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.698711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.698711+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.698711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:41.231249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.493810+0000","last_change":"2024-04-16T19:37:20.493865+0000","last_active":"2024-04-16T19:37:20.493810+0000","last_peered":"2024-04-16T19:37:20.493810+0000","last_clean":"2024-04-16T19:37:20.493810+0000","last_became_active":"2024-04-16T17:47:33.299632+0000","last_became_peered":"2024-04-16T17:47:33.299632+0000","last_unstale":"2024-04-16T19:37:20.493810+0000","last_undegraded":"2024-04-16T19:37:20.493810+0000","last_fullsized":"2024-04-16T19:37:20.493810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.493668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.493668+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.493668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:49.406598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"53'175","reported_seq":384,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.762300+0000","last_change":"2024-04-16T19:36:56.762377+0000","last_active":"2024-04-16T19:36:56.762300+0000","last_peered":"2024-04-16T19:36:56.762300+0000","last_clean":"2024-04-16T19:36:56.762300+0000","last_became_active":"2024-04-16T17:47:35.321294+0000","last_became_peered":"2024-04-16T17:47:35.321294+0000","last_unstale":"2024-04-16T19:36:56.762300+0000","last_undegraded":"2024-04-16T19:36:56.762300+0000","last_fullsized":"2024-04-16T19:36:56.762300+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'175","last_scrub_stamp":"2024-04-16T19:36:56.762164+0000","last_deep_scrub":"53'175","last_deep_scrub_stamp":"2024-04-16T19:36:56.762164+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.762164+0000","objects_scrubbed":7,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:59.105751+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":252,"num_read_kb":245,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"53'13","reported_seq":1352,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.360335+0000","last_change":"2024-04-16T19:37:02.360401+0000","last_active":"2024-04-16T19:37:02.360335+0000","last_peered":"2024-04-16T19:37:02.360335+0000","last_clean":"2024-04-16T19:37:02.360335+0000","last_became_active":"2024-04-16T17:47:37.322987+0000","last_became_peered":"2024-04-16T17:47:37.322987+0000","last_unstale":"2024-04-16T19:37:02.360335+0000","last_undegraded":"2024-04-16T19:37:02.360335+0000","last_fullsized":"2024-04-16T19:37:02.360335+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:02.360196+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:02.360196+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.360196+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:23.356000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.501363+0000","last_change":"2024-04-16T19:36:34.501465+0000","last_active":"2024-04-16T19:36:34.501363+0000","last_peered":"2024-04-16T19:36:34.501363+0000","last_clean":"2024-04-16T19:36:34.501363+0000","last_became_active":"2024-04-16T17:47:39.697732+0000","last_became_peered":"2024-04-16T17:47:39.697732+0000","last_unstale":"2024-04-16T19:36:34.501363+0000","last_undegraded":"2024-04-16T19:36:34.501363+0000","last_fullsized":"2024-04-16T19:36:34.501363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:34.501193+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.501193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:08.812307+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":65,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:49.549921+0000","last_change":"2024-04-16T19:37:49.549973+0000","last_active":"2024-04-16T19:37:49.549921+0000","last_peered":"2024-04-16T19:37:49.549921+0000","last_clean":"2024-04-16T19:37:49.549921+0000","last_became_active":"2024-04-16T18:10:50.547540+0000","last_became_peered":"2024-04-16T18:10:50.547540+0000","last_unstale":"2024-04-16T19:37:49.549921+0000","last_undegraded":"2024-04-16T19:37:49.549921+0000","last_fullsized":"2024-04-16T19:37:49.549921+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:49.549782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:49.549782+0000","last_clean_scrub_stamp":"2024-04-16T19:37:49.549782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:08.236243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"53'86","reported_seq":1218,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:44.651805+0000","last_change":"2024-04-16T19:37:44.651904+0000","last_active":"2024-04-16T19:37:44.651805+0000","last_peered":"2024-04-16T19:37:44.651805+0000","last_clean":"2024-04-16T19:37:44.651805+0000","last_became_active":"2024-04-16T17:48:34.342932+0000","last_became_peered":"2024-04-16T17:48:34.342932+0000","last_unstale":"2024-04-16T19:37:44.651805+0000","last_undegraded":"2024-04-16T19:37:44.651805+0000","last_fullsized":"2024-04-16T19:37:44.651805+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'86","last_scrub_stamp":"2024-04-16T19:37:44.651667+0000","last_deep_scrub":"53'86","last_deep_scrub_stamp":"2024-04-16T19:37:44.651667+0000","last_clean_scrub_stamp":"2024-04-16T19:37:44.651667+0000","objects_scrubbed":22,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:29:40.069540+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1874,"num_read_kb":1850,"num_write":100,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7031,"num_omap_keys":27,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.397896+0000","last_change":"2024-04-16T19:36:47.398006+0000","last_active":"2024-04-16T19:36:47.397896+0000","last_peered":"2024-04-16T19:36:47.397896+0000","last_clean":"2024-04-16T19:36:47.397896+0000","last_became_active":"2024-04-16T17:48:27.298250+0000","last_became_peered":"2024-04-16T17:48:27.298250+0000","last_unstale":"2024-04-16T19:36:47.397896+0000","last_undegraded":"2024-04-16T19:36:47.397896+0000","last_fullsized":"2024-04-16T19:36:47.397896+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.397724+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.397724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:25.235116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"53'7","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.547894+0000","last_change":"2024-04-16T19:36:32.547894+0000","last_active":"2024-04-16T19:36:32.547894+0000","last_peered":"2024-04-16T19:36:32.547894+0000","last_clean":"2024-04-16T19:36:32.547894+0000","last_became_active":"2024-04-16T17:48:01.251358+0000","last_became_peered":"2024-04-16T17:48:01.251358+0000","last_unstale":"2024-04-16T19:36:32.547894+0000","last_undegraded":"2024-04-16T19:36:32.547894+0000","last_fullsized":"2024-04-16T19:36:32.547894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:32.547672+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.547672+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:45.074195+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":715,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"53'556","reported_seq":2004,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.565740+0000","last_change":"2024-04-16T19:36:33.565806+0000","last_active":"2024-04-16T19:36:33.565740+0000","last_peered":"2024-04-16T19:36:33.565740+0000","last_clean":"2024-04-16T19:36:33.565740+0000","last_became_active":"2024-04-16T17:47:59.196585+0000","last_became_peered":"2024-04-16T17:47:59.196585+0000","last_unstale":"2024-04-16T19:36:33.565740+0000","last_undegraded":"2024-04-16T19:36:33.565740+0000","last_fullsized":"2024-04-16T19:36:33.565740+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:33.565603+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.565603+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:32.212045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"53'575","reported_seq":853,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.628047+0000","last_change":"2024-04-16T19:36:22.628176+0000","last_active":"2024-04-16T19:36:22.628047+0000","last_peered":"2024-04-16T19:36:22.628047+0000","last_clean":"2024-04-16T19:36:22.628047+0000","last_became_active":"2024-04-16T17:47:57.190658+0000","last_became_peered":"2024-04-16T17:47:57.190658+0000","last_unstale":"2024-04-16T19:36:22.628047+0000","last_undegraded":"2024-04-16T19:36:22.628047+0000","last_fullsized":"2024-04-16T19:36:22.628047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'575","last_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_deep_scrub":"53'575","last_deep_scrub_stamp":"2024-04-16T19:36:22.627889+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.627889+0000","objects_scrubbed":22,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:49:21.274331+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":746,"num_read_kb":738,"num_write":935,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71488,"num_omap_keys":312,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.f","version":"53'18","reported_seq":261,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.483626+0000","last_change":"2024-04-16T19:36:29.483703+0000","last_active":"2024-04-16T19:36:29.483626+0000","last_peered":"2024-04-16T19:36:29.483626+0000","last_clean":"2024-04-16T19:36:29.483626+0000","last_became_active":"2024-04-16T17:47:55.178028+0000","last_became_peered":"2024-04-16T17:47:55.178028+0000","last_unstale":"2024-04-16T19:36:29.483626+0000","last_undegraded":"2024-04-16T19:36:29.483626+0000","last_fullsized":"2024-04-16T19:36:29.483626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:29.483500+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.483500+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:36:53.084475+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":350,"num_read_kb":224,"num_write":95,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.621921+0000","last_change":"2024-04-16T19:37:15.621921+0000","last_active":"2024-04-16T19:37:15.621921+0000","last_peered":"2024-04-16T19:37:15.621921+0000","last_clean":"2024-04-16T19:37:15.621921+0000","last_became_active":"2024-04-16T17:47:23.265316+0000","last_became_peered":"2024-04-16T17:47:23.265316+0000","last_unstale":"2024-04-16T19:37:15.621921+0000","last_undegraded":"2024-04-16T19:37:15.621921+0000","last_fullsized":"2024-04-16T19:37:15.621921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.621731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.621731+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.621731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:58:34.123975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:39.404704+0000","last_change":"2024-04-16T19:37:39.404756+0000","last_active":"2024-04-16T19:37:39.404704+0000","last_peered":"2024-04-16T19:37:39.404704+0000","last_clean":"2024-04-16T19:37:39.404704+0000","last_became_active":"2024-04-16T17:47:21.228827+0000","last_became_peered":"2024-04-16T17:47:21.228827+0000","last_unstale":"2024-04-16T19:37:39.404704+0000","last_undegraded":"2024-04-16T19:37:39.404704+0000","last_fullsized":"2024-04-16T19:37:39.404704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:39.404568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:39.404568+0000","last_clean_scrub_stamp":"2024-04-16T19:37:39.404568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:28.124481+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":150,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:38.401899+0000","last_change":"2024-04-16T19:37:38.402002+0000","last_active":"2024-04-16T19:37:38.401899+0000","last_peered":"2024-04-16T19:37:38.401899+0000","last_clean":"2024-04-16T19:37:38.401899+0000","last_became_active":"2024-04-16T17:46:38.111816+0000","last_became_peered":"2024-04-16T17:46:38.111816+0000","last_unstale":"2024-04-16T19:37:38.401899+0000","last_undegraded":"2024-04-16T19:37:38.401899+0000","last_fullsized":"2024-04-16T19:37:38.401899+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:38.401724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:38.401724+0000","last_clean_scrub_stamp":"2024-04-16T19:37:38.401724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:00.477703+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.594734+0000","last_change":"2024-04-16T19:37:18.594788+0000","last_active":"2024-04-16T19:37:18.594734+0000","last_peered":"2024-04-16T19:37:18.594734+0000","last_clean":"2024-04-16T19:37:18.594734+0000","last_became_active":"2024-04-16T17:47:33.302462+0000","last_became_peered":"2024-04-16T17:47:33.302462+0000","last_unstale":"2024-04-16T19:37:18.594734+0000","last_undegraded":"2024-04-16T19:37:18.594734+0000","last_fullsized":"2024-04-16T19:37:18.594734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.594597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.594597+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.594597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:43.732534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"53'168","reported_seq":385,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.727125+0000","last_change":"2024-04-16T19:37:12.727211+0000","last_active":"2024-04-16T19:37:12.727125+0000","last_peered":"2024-04-16T19:37:12.727125+0000","last_clean":"2024-04-16T19:37:12.727125+0000","last_became_active":"2024-04-16T17:47:35.309865+0000","last_became_peered":"2024-04-16T17:47:35.309865+0000","last_unstale":"2024-04-16T19:37:12.727125+0000","last_undegraded":"2024-04-16T19:37:12.727125+0000","last_fullsized":"2024-04-16T19:37:12.727125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'168","last_scrub_stamp":"2024-04-16T19:37:12.726993+0000","last_deep_scrub":"53'168","last_deep_scrub_stamp":"2024-04-16T19:37:12.726993+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.726993+0000","objects_scrubbed":8,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:58.958378+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":238,"num_write":160,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.606377+0000","last_change":"2024-04-16T19:36:58.606428+0000","last_active":"2024-04-16T19:36:58.606377+0000","last_peered":"2024-04-16T19:36:58.606377+0000","last_clean":"2024-04-16T19:36:58.606377+0000","last_became_active":"2024-04-16T17:47:37.322273+0000","last_became_peered":"2024-04-16T17:47:37.322273+0000","last_unstale":"2024-04-16T19:36:58.606377+0000","last_undegraded":"2024-04-16T19:36:58.606377+0000","last_fullsized":"2024-04-16T19:36:58.606377+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:58.606242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:58.606242+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.606242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:10:23.041599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.422395+0000","last_change":"2024-04-16T19:36:32.422395+0000","last_active":"2024-04-16T19:36:32.422395+0000","last_peered":"2024-04-16T19:36:32.422395+0000","last_clean":"2024-04-16T19:36:32.422395+0000","last_became_active":"2024-04-16T17:47:39.699017+0000","last_became_peered":"2024-04-16T17:47:39.699017+0000","last_unstale":"2024-04-16T19:36:32.422395+0000","last_undegraded":"2024-04-16T19:36:32.422395+0000","last_fullsized":"2024-04-16T19:36:32.422395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.422189+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.422189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:35.278356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"50'9","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:49.582402+0000","last_change":"2024-04-16T19:37:49.582402+0000","last_active":"2024-04-16T19:37:49.582402+0000","last_peered":"2024-04-16T19:37:49.582402+0000","last_clean":"2024-04-16T19:37:49.582402+0000","last_became_active":"2024-04-16T18:10:50.534160+0000","last_became_peered":"2024-04-16T18:10:50.534160+0000","last_unstale":"2024-04-16T19:37:49.582402+0000","last_undegraded":"2024-04-16T19:37:49.582402+0000","last_fullsized":"2024-04-16T19:37:49.582402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'9","last_scrub_stamp":"2024-04-16T19:37:49.582029+0000","last_deep_scrub":"50'9","last_deep_scrub_stamp":"2024-04-16T19:37:49.582029+0000","last_clean_scrub_stamp":"2024-04-16T19:37:49.582029+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:35.936413+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"53'117","reported_seq":1138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:43.647272+0000","last_change":"2024-04-16T19:37:43.647272+0000","last_active":"2024-04-16T19:37:43.647272+0000","last_peered":"2024-04-16T19:37:43.647272+0000","last_clean":"2024-04-16T19:37:43.647272+0000","last_became_active":"2024-04-16T17:49:32.394682+0000","last_became_peered":"2024-04-16T17:49:32.394682+0000","last_unstale":"2024-04-16T19:37:43.647272+0000","last_undegraded":"2024-04-16T19:37:43.647272+0000","last_fullsized":"2024-04-16T19:37:43.647272+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'117","last_scrub_stamp":"2024-04-16T19:37:43.647066+0000","last_deep_scrub":"53'117","last_deep_scrub_stamp":"2024-04-16T19:37:43.647066+0000","last_clean_scrub_stamp":"2024-04-16T19:37:43.647066+0000","objects_scrubbed":29,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:41:33.413258+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1657,"num_read_kb":1708,"num_write":128,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7371,"num_omap_keys":29,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.635196+0000","last_change":"2024-04-16T19:36:39.635263+0000","last_active":"2024-04-16T19:36:39.635196+0000","last_peered":"2024-04-16T19:36:39.635196+0000","last_clean":"2024-04-16T19:36:39.635196+0000","last_became_active":"2024-04-16T17:48:27.306978+0000","last_became_peered":"2024-04-16T17:48:27.306978+0000","last_unstale":"2024-04-16T19:36:39.635196+0000","last_undegraded":"2024-04-16T19:36:39.635196+0000","last_fullsized":"2024-04-16T19:36:39.635196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.635056+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.635056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:51:26.132595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"53'20","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.709187+0000","last_change":"2024-04-16T19:36:42.709267+0000","last_active":"2024-04-16T19:36:42.709187+0000","last_peered":"2024-04-16T19:36:42.709187+0000","last_clean":"2024-04-16T19:36:42.709187+0000","last_became_active":"2024-04-16T17:48:01.239964+0000","last_became_peered":"2024-04-16T17:48:01.239964+0000","last_unstale":"2024-04-16T19:36:42.709187+0000","last_undegraded":"2024-04-16T19:36:42.709187+0000","last_fullsized":"2024-04-16T19:36:42.709187+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:42.709051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.709051+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:57.264715+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1312,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":106,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.501219+0000","last_change":"2024-04-16T19:36:35.501282+0000","last_active":"2024-04-16T19:36:35.501219+0000","last_peered":"2024-04-16T19:36:35.501219+0000","last_clean":"2024-04-16T19:36:35.501219+0000","last_became_active":"2024-04-16T17:47:59.191883+0000","last_became_peered":"2024-04-16T17:47:59.191883+0000","last_unstale":"2024-04-16T19:36:35.501219+0000","last_undegraded":"2024-04-16T19:36:35.501219+0000","last_fullsized":"2024-04-16T19:36:35.501219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.501075+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.501075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:40.913802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1d","version":"53'871","reported_seq":1181,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.697221+0000","last_change":"2024-04-16T19:36:18.697375+0000","last_active":"2024-04-16T19:36:18.697221+0000","last_peered":"2024-04-16T19:36:18.697221+0000","last_clean":"2024-04-16T19:36:18.697221+0000","last_became_active":"2024-04-16T17:47:57.192988+0000","last_became_peered":"2024-04-16T17:47:57.192988+0000","last_unstale":"2024-04-16T19:36:18.697221+0000","last_undegraded":"2024-04-16T19:36:18.697221+0000","last_fullsized":"2024-04-16T19:36:18.697221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'871","last_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_deep_scrub":"53'871","last_deep_scrub_stamp":"2024-04-16T19:36:18.697076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.697076+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:49.607583+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1065,"num_read_kb":1054,"num_write":1498,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":143413,"num_omap_keys":626,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'22","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.656560+0000","last_change":"2024-04-16T19:37:04.656657+0000","last_active":"2024-04-16T19:37:04.656560+0000","last_peered":"2024-04-16T19:37:04.656560+0000","last_clean":"2024-04-16T19:37:04.656560+0000","last_became_active":"2024-04-16T17:49:32.396855+0000","last_became_peered":"2024-04-16T17:49:32.396855+0000","last_unstale":"2024-04-16T19:37:04.656560+0000","last_undegraded":"2024-04-16T19:37:04.656560+0000","last_fullsized":"2024-04-16T19:37:04.656560+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:37:04.656410+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:37:04.656410+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.656410+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:42.493803+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":384,"num_read_kb":251,"num_write":106,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":117,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.570337+0000","last_change":"2024-04-16T19:36:36.570388+0000","last_active":"2024-04-16T19:36:36.570337+0000","last_peered":"2024-04-16T19:36:36.570337+0000","last_clean":"2024-04-16T19:36:36.570337+0000","last_became_active":"2024-04-16T17:47:39.335578+0000","last_became_peered":"2024-04-16T17:47:39.335578+0000","last_unstale":"2024-04-16T19:36:36.570337+0000","last_undegraded":"2024-04-16T19:36:36.570337+0000","last_fullsized":"2024-04-16T19:36:36.570337+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.570198+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.570198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:51.282322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.603234+0000","last_change":"2024-04-16T19:37:11.603285+0000","last_active":"2024-04-16T19:37:11.603234+0000","last_peered":"2024-04-16T19:37:11.603234+0000","last_clean":"2024-04-16T19:37:11.603234+0000","last_became_active":"2024-04-16T17:47:23.257311+0000","last_became_peered":"2024-04-16T17:47:23.257311+0000","last_unstale":"2024-04-16T19:37:11.603234+0000","last_undegraded":"2024-04-16T19:37:11.603234+0000","last_fullsized":"2024-04-16T19:37:11.603234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.603093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.603093+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.603093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:10.668778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.725630+0000","last_change":"2024-04-16T19:37:25.725630+0000","last_active":"2024-04-16T19:37:25.725630+0000","last_peered":"2024-04-16T19:37:25.725630+0000","last_clean":"2024-04-16T19:37:25.725630+0000","last_became_active":"2024-04-16T17:47:21.252807+0000","last_became_peered":"2024-04-16T17:47:21.252807+0000","last_unstale":"2024-04-16T19:37:25.725630+0000","last_undegraded":"2024-04-16T19:37:25.725630+0000","last_fullsized":"2024-04-16T19:37:25.725630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.725436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.725436+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.725436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:02.147676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"53'14","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.665922+0000","last_change":"2024-04-16T19:37:05.665922+0000","last_active":"2024-04-16T19:37:05.665922+0000","last_peered":"2024-04-16T19:37:05.665922+0000","last_clean":"2024-04-16T19:37:05.665922+0000","last_became_active":"2024-04-16T17:47:33.296556+0000","last_became_peered":"2024-04-16T17:47:33.296556+0000","last_unstale":"2024-04-16T19:37:05.665922+0000","last_undegraded":"2024-04-16T19:37:05.665922+0000","last_fullsized":"2024-04-16T19:37:05.665922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'14","last_scrub_stamp":"2024-04-16T19:37:05.665717+0000","last_deep_scrub":"53'14","last_deep_scrub_stamp":"2024-04-16T19:37:05.665717+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.665717+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:39:45.630852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"53'109","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.370839+0000","last_change":"2024-04-16T19:36:59.370839+0000","last_active":"2024-04-16T19:36:59.370839+0000","last_peered":"2024-04-16T19:36:59.370839+0000","last_clean":"2024-04-16T19:36:59.370839+0000","last_became_active":"2024-04-16T17:47:35.309720+0000","last_became_peered":"2024-04-16T17:47:35.309720+0000","last_unstale":"2024-04-16T19:36:59.370839+0000","last_undegraded":"2024-04-16T19:36:59.370839+0000","last_fullsized":"2024-04-16T19:36:59.370839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:59.370626+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:59.370626+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.370626+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:46.735030+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:50.627271+0000","last_change":"2024-04-16T19:36:50.627271+0000","last_active":"2024-04-16T19:36:50.627271+0000","last_peered":"2024-04-16T19:36:50.627271+0000","last_clean":"2024-04-16T19:36:50.627271+0000","last_became_active":"2024-04-16T17:47:37.333723+0000","last_became_peered":"2024-04-16T17:47:37.333723+0000","last_unstale":"2024-04-16T19:36:50.627271+0000","last_undegraded":"2024-04-16T19:36:50.627271+0000","last_fullsized":"2024-04-16T19:36:50.627271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:50.627076+0000","last_clean_scrub_stamp":"2024-04-16T19:36:50.627076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:16.020863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:35.527983+0000","last_change":"2024-04-16T19:37:35.528091+0000","last_active":"2024-04-16T19:37:35.527983+0000","last_peered":"2024-04-16T19:37:35.527983+0000","last_clean":"2024-04-16T19:37:35.527983+0000","last_became_active":"2024-04-16T18:10:50.554568+0000","last_became_peered":"2024-04-16T18:10:50.554568+0000","last_unstale":"2024-04-16T19:37:35.527983+0000","last_undegraded":"2024-04-16T19:37:35.527983+0000","last_fullsized":"2024-04-16T19:37:35.527983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:35.527793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:35.527793+0000","last_clean_scrub_stamp":"2024-04-16T19:37:35.527793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:00.663423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"53'82","reported_seq":709,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.725800+0000","last_change":"2024-04-16T19:37:03.725800+0000","last_active":"2024-04-16T19:37:03.725800+0000","last_peered":"2024-04-16T19:37:03.725800+0000","last_clean":"2024-04-16T19:37:03.725800+0000","last_became_active":"2024-04-16T17:48:34.342456+0000","last_became_peered":"2024-04-16T17:48:34.342456+0000","last_unstale":"2024-04-16T19:37:03.725800+0000","last_undegraded":"2024-04-16T19:37:03.725800+0000","last_fullsized":"2024-04-16T19:37:03.725800+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'82","last_scrub_stamp":"2024-04-16T19:37:03.725602+0000","last_deep_scrub":"53'82","last_deep_scrub_stamp":"2024-04-16T19:37:03.725602+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.725602+0000","objects_scrubbed":27,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:37.279204+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":946,"num_read_kb":931,"num_write":73,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3517,"num_omap_keys":14,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.640311+0000","last_change":"2024-04-16T19:36:36.640365+0000","last_active":"2024-04-16T19:36:36.640311+0000","last_peered":"2024-04-16T19:36:36.640311+0000","last_clean":"2024-04-16T19:36:36.640311+0000","last_became_active":"2024-04-16T17:48:27.300160+0000","last_became_peered":"2024-04-16T17:48:27.300160+0000","last_unstale":"2024-04-16T19:36:36.640311+0000","last_undegraded":"2024-04-16T19:36:36.640311+0000","last_fullsized":"2024-04-16T19:36:36.640311+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.640174+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.640174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:51.451059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"53'15","reported_seq":127,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.647852+0000","last_change":"2024-04-16T19:36:39.647955+0000","last_active":"2024-04-16T19:36:39.647852+0000","last_peered":"2024-04-16T19:36:39.647852+0000","last_clean":"2024-04-16T19:36:39.647852+0000","last_became_active":"2024-04-16T17:48:01.244894+0000","last_became_peered":"2024-04-16T17:48:01.244894+0000","last_unstale":"2024-04-16T19:36:39.647852+0000","last_undegraded":"2024-04-16T19:36:39.647852+0000","last_fullsized":"2024-04-16T19:36:39.647852+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'15","last_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_deep_scrub":"53'15","last_deep_scrub_stamp":"2024-04-16T19:36:39.647719+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.647719+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:50.538220+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1803,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":28,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.430796+0000","last_change":"2024-04-16T19:36:38.430850+0000","last_active":"2024-04-16T19:36:38.430796+0000","last_peered":"2024-04-16T19:36:38.430796+0000","last_clean":"2024-04-16T19:36:38.430796+0000","last_became_active":"2024-04-16T17:47:59.175682+0000","last_became_peered":"2024-04-16T17:47:59.175682+0000","last_unstale":"2024-04-16T19:36:38.430796+0000","last_undegraded":"2024-04-16T19:36:38.430796+0000","last_fullsized":"2024-04-16T19:36:38.430796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.430655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.430655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:03.824441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"53'616","reported_seq":2312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.598175+0000","last_change":"2024-04-16T19:36:23.598300+0000","last_active":"2024-04-16T19:36:23.598175+0000","last_peered":"2024-04-16T19:36:23.598175+0000","last_clean":"2024-04-16T19:36:23.598175+0000","last_became_active":"2024-04-16T17:47:57.185237+0000","last_became_peered":"2024-04-16T17:47:57.185237+0000","last_unstale":"2024-04-16T19:36:23.598175+0000","last_undegraded":"2024-04-16T19:36:23.598175+0000","last_fullsized":"2024-04-16T19:36:23.598175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'616","last_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_deep_scrub":"53'616","last_deep_scrub_stamp":"2024-04-16T19:36:23.598031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.598031+0000","objects_scrubbed":24,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:37.526844+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":277,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1043,"num_read_kb":1030,"num_write":687,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.510530+0000","last_change":"2024-04-16T19:36:23.510613+0000","last_active":"2024-04-16T19:36:23.510530+0000","last_peered":"2024-04-16T19:36:23.510530+0000","last_clean":"2024-04-16T19:36:23.510530+0000","last_became_active":"2024-04-16T17:47:55.162932+0000","last_became_peered":"2024-04-16T17:47:55.162932+0000","last_unstale":"2024-04-16T19:36:23.510530+0000","last_undegraded":"2024-04-16T19:36:23.510530+0000","last_fullsized":"2024-04-16T19:36:23.510530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:23.510390+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.510390+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:03.621835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":25,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:32.537581+0000","last_change":"2024-04-16T19:36:32.537632+0000","last_active":"2024-04-16T19:36:32.537581+0000","last_peered":"2024-04-16T19:36:32.537581+0000","last_clean":"2024-04-16T19:36:32.537581+0000","last_became_active":"2024-04-16T17:47:39.338081+0000","last_became_peered":"2024-04-16T17:47:39.338081+0000","last_unstale":"2024-04-16T19:36:32.537581+0000","last_undegraded":"2024-04-16T19:36:32.537581+0000","last_fullsized":"2024-04-16T19:36:32.537581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:32.537440+0000","last_clean_scrub_stamp":"2024-04-16T19:36:32.537440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:00:47.348326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.588585+0000","last_change":"2024-04-16T19:37:12.588636+0000","last_active":"2024-04-16T19:37:12.588585+0000","last_peered":"2024-04-16T19:37:12.588585+0000","last_clean":"2024-04-16T19:37:12.588585+0000","last_became_active":"2024-04-16T17:47:23.260315+0000","last_became_peered":"2024-04-16T17:47:23.260315+0000","last_unstale":"2024-04-16T19:37:12.588585+0000","last_undegraded":"2024-04-16T19:37:12.588585+0000","last_fullsized":"2024-04-16T19:37:12.588585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:12.588451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:12.588451+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.588451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:58.868734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:39.575481+0000","last_change":"2024-04-16T19:37:39.575574+0000","last_active":"2024-04-16T19:37:39.575481+0000","last_peered":"2024-04-16T19:37:39.575481+0000","last_clean":"2024-04-16T19:37:39.575481+0000","last_became_active":"2024-04-16T17:47:21.240848+0000","last_became_peered":"2024-04-16T17:47:21.240848+0000","last_unstale":"2024-04-16T19:37:39.575481+0000","last_undegraded":"2024-04-16T19:37:39.575481+0000","last_fullsized":"2024-04-16T19:37:39.575481+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:39.575271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:39.575271+0000","last_clean_scrub_stamp":"2024-04-16T19:37:39.575271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:48.718595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"53'2","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.615140+0000","last_change":"2024-04-16T19:37:08.615140+0000","last_active":"2024-04-16T19:37:08.615140+0000","last_peered":"2024-04-16T19:37:08.615140+0000","last_clean":"2024-04-16T19:37:08.615140+0000","last_became_active":"2024-04-16T17:47:33.296983+0000","last_became_peered":"2024-04-16T17:47:33.296983+0000","last_unstale":"2024-04-16T19:37:08.615140+0000","last_undegraded":"2024-04-16T19:37:08.615140+0000","last_fullsized":"2024-04-16T19:37:08.615140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:37:08.614950+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:37:08.614950+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.614950+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:35:13.456557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"53'53","reported_seq":197,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.682005+0000","last_change":"2024-04-16T19:37:11.682005+0000","last_active":"2024-04-16T19:37:11.682005+0000","last_peered":"2024-04-16T19:37:11.682005+0000","last_clean":"2024-04-16T19:37:11.682005+0000","last_became_active":"2024-04-16T17:47:35.313736+0000","last_became_peered":"2024-04-16T17:47:35.313736+0000","last_unstale":"2024-04-16T19:37:11.682005+0000","last_undegraded":"2024-04-16T19:37:11.682005+0000","last_fullsized":"2024-04-16T19:37:11.682005+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'53","last_scrub_stamp":"2024-04-16T19:37:11.681806+0000","last_deep_scrub":"53'53","last_deep_scrub_stamp":"2024-04-16T19:37:11.681806+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.681806+0000","objects_scrubbed":3,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:47.618751+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":72,"num_write":52,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.518374+0000","last_change":"2024-04-16T19:37:02.518374+0000","last_active":"2024-04-16T19:37:02.518374+0000","last_peered":"2024-04-16T19:37:02.518374+0000","last_clean":"2024-04-16T19:37:02.518374+0000","last_became_active":"2024-04-16T17:47:37.326902+0000","last_became_peered":"2024-04-16T17:47:37.326902+0000","last_unstale":"2024-04-16T19:37:02.518374+0000","last_undegraded":"2024-04-16T19:37:02.518374+0000","last_fullsized":"2024-04-16T19:37:02.518374+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.518192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.518192+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.518192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:34.961401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:50.550463+0000","last_change":"2024-04-16T19:37:50.550517+0000","last_active":"2024-04-16T19:37:50.550463+0000","last_peered":"2024-04-16T19:37:50.550463+0000","last_clean":"2024-04-16T19:37:50.550463+0000","last_became_active":"2024-04-16T18:10:50.557307+0000","last_became_peered":"2024-04-16T18:10:50.557307+0000","last_unstale":"2024-04-16T19:37:50.550463+0000","last_undegraded":"2024-04-16T19:37:50.550463+0000","last_fullsized":"2024-04-16T19:37:50.550463+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:50.550318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:50.550318+0000","last_clean_scrub_stamp":"2024-04-16T19:37:50.550318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:50:49.487656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"53'129","reported_seq":1727,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.778020+0000","last_change":"2024-04-16T19:36:55.778092+0000","last_active":"2024-04-16T19:36:55.778020+0000","last_peered":"2024-04-16T19:36:55.778020+0000","last_clean":"2024-04-16T19:36:55.778020+0000","last_became_active":"2024-04-16T17:48:34.338224+0000","last_became_peered":"2024-04-16T17:48:34.338224+0000","last_unstale":"2024-04-16T19:36:55.778020+0000","last_undegraded":"2024-04-16T19:36:55.778020+0000","last_fullsized":"2024-04-16T19:36:55.778020+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'129","last_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_deep_scrub":"53'129","last_deep_scrub_stamp":"2024-04-16T19:36:55.777891+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.777891+0000","objects_scrubbed":30,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:46.059742+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2729,"num_read_kb":2749,"num_write":151,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8321,"num_omap_keys":34,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.410485+0000","last_change":"2024-04-16T19:36:51.410485+0000","last_active":"2024-04-16T19:36:51.410485+0000","last_peered":"2024-04-16T19:36:51.410485+0000","last_clean":"2024-04-16T19:36:51.410485+0000","last_became_active":"2024-04-16T17:48:27.300957+0000","last_became_peered":"2024-04-16T17:48:27.300957+0000","last_unstale":"2024-04-16T19:36:51.410485+0000","last_undegraded":"2024-04-16T19:36:51.410485+0000","last_fullsized":"2024-04-16T19:36:51.410485+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:51.410273+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.410273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:45:57.416580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"53'18","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.684234+0000","last_change":"2024-04-16T19:36:45.684234+0000","last_active":"2024-04-16T19:36:45.684234+0000","last_peered":"2024-04-16T19:36:45.684234+0000","last_clean":"2024-04-16T19:36:45.684234+0000","last_became_active":"2024-04-16T17:48:01.243932+0000","last_became_peered":"2024-04-16T17:48:01.243932+0000","last_unstale":"2024-04-16T19:36:45.684234+0000","last_undegraded":"2024-04-16T19:36:45.684234+0000","last_fullsized":"2024-04-16T19:36:45.684234+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:36:45.684019+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.684019+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:09:54.649522+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1628,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":76,"num_read_kb":58,"num_write":32,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"53'21","reported_seq":1432,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.402575+0000","last_change":"2024-04-16T19:36:43.402646+0000","last_active":"2024-04-16T19:36:43.402575+0000","last_peered":"2024-04-16T19:36:43.402575+0000","last_clean":"2024-04-16T19:36:43.402575+0000","last_became_active":"2024-04-16T17:47:59.196101+0000","last_became_peered":"2024-04-16T17:47:59.196101+0000","last_unstale":"2024-04-16T19:36:43.402575+0000","last_undegraded":"2024-04-16T19:36:43.402575+0000","last_fullsized":"2024-04-16T19:36:43.402575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:43.402444+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.402444+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:20.659167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1f","version":"53'7003","reported_seq":10587,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.479202+0000","last_change":"2024-04-16T19:36:36.479319+0000","last_active":"2024-04-16T19:36:36.479202+0000","last_peered":"2024-04-16T19:36:36.479202+0000","last_clean":"2024-04-16T19:36:36.479202+0000","last_became_active":"2024-04-16T17:47:57.194760+0000","last_became_peered":"2024-04-16T17:47:57.194760+0000","last_unstale":"2024-04-16T19:36:36.479202+0000","last_undegraded":"2024-04-16T19:36:36.479202+0000","last_fullsized":"2024-04-16T19:36:36.479202+0000","mapping_epoch":34,"log_start":"53'5200","ondisk_log_start":"53'5200","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7003","last_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_deep_scrub":"53'7003","last_deep_scrub_stamp":"2024-04-16T19:36:36.479068+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.479068+0000","objects_scrubbed":20,"log_size":1803,"log_dups_size":3000,"ondisk_log_size":1803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:38:56.688434+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":13756,"num_read_kb":13744,"num_write":7116,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"53'10","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.437901+0000","last_change":"2024-04-16T19:36:27.437901+0000","last_active":"2024-04-16T19:36:27.437901+0000","last_peered":"2024-04-16T19:36:27.437901+0000","last_clean":"2024-04-16T19:36:27.437901+0000","last_became_active":"2024-04-16T17:47:55.161778+0000","last_became_peered":"2024-04-16T17:47:55.161778+0000","last_unstale":"2024-04-16T19:36:27.437901+0000","last_undegraded":"2024-04-16T19:36:27.437901+0000","last_fullsized":"2024-04-16T19:36:27.437901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:27.437656+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.437656+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:14.616289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5150,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":65,"num_write":51,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.669224+0000","last_change":"2024-04-16T19:36:42.669224+0000","last_active":"2024-04-16T19:36:42.669224+0000","last_peered":"2024-04-16T19:36:42.669224+0000","last_clean":"2024-04-16T19:36:42.669224+0000","last_became_active":"2024-04-16T17:47:39.328314+0000","last_became_peered":"2024-04-16T17:47:39.328314+0000","last_unstale":"2024-04-16T19:36:42.669224+0000","last_undegraded":"2024-04-16T19:36:42.669224+0000","last_fullsized":"2024-04-16T19:36:42.669224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.669008+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.669008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:32.950859+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.392128+0000","last_change":"2024-04-16T19:37:05.392128+0000","last_active":"2024-04-16T19:37:05.392128+0000","last_peered":"2024-04-16T19:37:05.392128+0000","last_clean":"2024-04-16T19:37:05.392128+0000","last_became_active":"2024-04-16T17:47:23.269208+0000","last_became_peered":"2024-04-16T17:47:23.269208+0000","last_unstale":"2024-04-16T19:37:05.392128+0000","last_undegraded":"2024-04-16T19:37:05.392128+0000","last_fullsized":"2024-04-16T19:37:05.392128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:05.391932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:05.391932+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.391932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:20.104056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.480574+0000","last_change":"2024-04-16T19:37:34.480625+0000","last_active":"2024-04-16T19:37:34.480574+0000","last_peered":"2024-04-16T19:37:34.480574+0000","last_clean":"2024-04-16T19:37:34.480574+0000","last_became_active":"2024-04-16T17:47:21.254805+0000","last_became_peered":"2024-04-16T17:47:21.254805+0000","last_unstale":"2024-04-16T19:37:34.480574+0000","last_undegraded":"2024-04-16T19:37:34.480574+0000","last_fullsized":"2024-04-16T19:37:34.480574+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.480436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.480436+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.480436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:52:39.055910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"53'4","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.536820+0000","last_change":"2024-04-16T19:37:18.536882+0000","last_active":"2024-04-16T19:37:18.536820+0000","last_peered":"2024-04-16T19:37:18.536820+0000","last_clean":"2024-04-16T19:37:18.536820+0000","last_became_active":"2024-04-16T17:47:33.296584+0000","last_became_peered":"2024-04-16T17:47:33.296584+0000","last_unstale":"2024-04-16T19:37:18.536820+0000","last_undegraded":"2024-04-16T19:37:18.536820+0000","last_fullsized":"2024-04-16T19:37:18.536820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:37:18.536691+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:37:18.536691+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.536691+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:40.115436+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1009,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":22,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"53'109","reported_seq":286,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:58.376859+0000","last_change":"2024-04-16T19:36:58.376859+0000","last_active":"2024-04-16T19:36:58.376859+0000","last_peered":"2024-04-16T19:36:58.376859+0000","last_clean":"2024-04-16T19:36:58.376859+0000","last_became_active":"2024-04-16T17:47:35.310319+0000","last_became_peered":"2024-04-16T17:47:35.310319+0000","last_unstale":"2024-04-16T19:36:58.376859+0000","last_undegraded":"2024-04-16T19:36:58.376859+0000","last_fullsized":"2024-04-16T19:36:58.376859+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:58.376556+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:58.376556+0000","last_clean_scrub_stamp":"2024-04-16T19:36:58.376556+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:38.413082+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.395303+0000","last_change":"2024-04-16T19:37:04.395303+0000","last_active":"2024-04-16T19:37:04.395303+0000","last_peered":"2024-04-16T19:37:04.395303+0000","last_clean":"2024-04-16T19:37:04.395303+0000","last_became_active":"2024-04-16T17:47:37.321909+0000","last_became_peered":"2024-04-16T17:47:37.321909+0000","last_unstale":"2024-04-16T19:37:04.395303+0000","last_undegraded":"2024-04-16T19:37:04.395303+0000","last_fullsized":"2024-04-16T19:37:04.395303+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.395110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.395110+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.395110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:24:17.565659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":59,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:48.570533+0000","last_change":"2024-04-16T19:37:48.570586+0000","last_active":"2024-04-16T19:37:48.570533+0000","last_peered":"2024-04-16T19:37:48.570533+0000","last_clean":"2024-04-16T19:37:48.570533+0000","last_became_active":"2024-04-16T18:10:50.542419+0000","last_became_peered":"2024-04-16T18:10:50.542419+0000","last_unstale":"2024-04-16T19:37:48.570533+0000","last_undegraded":"2024-04-16T19:37:48.570533+0000","last_fullsized":"2024-04-16T19:37:48.570533+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:48.570394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:48.570394+0000","last_clean_scrub_stamp":"2024-04-16T19:37:48.570394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:53.894006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"53'131","reported_seq":1400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:48.571767+0000","last_change":"2024-04-16T19:37:48.571845+0000","last_active":"2024-04-16T19:37:48.571767+0000","last_peered":"2024-04-16T19:37:48.571767+0000","last_clean":"2024-04-16T19:37:48.571767+0000","last_became_active":"2024-04-16T17:49:32.396110+0000","last_became_peered":"2024-04-16T17:49:32.396110+0000","last_unstale":"2024-04-16T19:37:48.571767+0000","last_undegraded":"2024-04-16T19:37:48.571767+0000","last_fullsized":"2024-04-16T19:37:48.571767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'131","last_scrub_stamp":"2024-04-16T19:37:48.571630+0000","last_deep_scrub":"53'131","last_deep_scrub_stamp":"2024-04-16T19:37:48.571630+0000","last_clean_scrub_stamp":"2024-04-16T19:37:48.571630+0000","objects_scrubbed":31,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:29:43.989503+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2348,"num_read_kb":2316,"num_write":147,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6587,"num_omap_keys":28,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.499794+0000","last_change":"2024-04-16T19:36:36.499794+0000","last_active":"2024-04-16T19:36:36.499794+0000","last_peered":"2024-04-16T19:36:36.499794+0000","last_clean":"2024-04-16T19:36:36.499794+0000","last_became_active":"2024-04-16T17:48:27.292565+0000","last_became_peered":"2024-04-16T17:48:27.292565+0000","last_unstale":"2024-04-16T19:36:36.499794+0000","last_undegraded":"2024-04-16T19:36:36.499794+0000","last_fullsized":"2024-04-16T19:36:36.499794+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:36.499596+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.499596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:09:22.332807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"53'16","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.619264+0000","last_change":"2024-04-16T19:36:44.619264+0000","last_active":"2024-04-16T19:36:44.619264+0000","last_peered":"2024-04-16T19:36:44.619264+0000","last_clean":"2024-04-16T19:36:44.619264+0000","last_became_active":"2024-04-16T17:48:01.236333+0000","last_became_peered":"2024-04-16T17:48:01.236333+0000","last_unstale":"2024-04-16T19:36:44.619264+0000","last_undegraded":"2024-04-16T19:36:44.619264+0000","last_fullsized":"2024-04-16T19:36:44.619264+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:44.619046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.619046+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:50.031773+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1607,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":48,"num_read_kb":40,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"53'556","reported_seq":1967,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.516649+0000","last_change":"2024-04-16T19:36:35.516719+0000","last_active":"2024-04-16T19:36:35.516649+0000","last_peered":"2024-04-16T19:36:35.516649+0000","last_clean":"2024-04-16T19:36:35.516649+0000","last_became_active":"2024-04-16T17:47:59.186706+0000","last_became_peered":"2024-04-16T17:47:59.186706+0000","last_unstale":"2024-04-16T19:36:35.516649+0000","last_undegraded":"2024-04-16T19:36:35.516649+0000","last_fullsized":"2024-04-16T19:36:35.516649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:35.516509+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.516509+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:41.203861+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'563","reported_seq":804,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:29.482688+0000","last_change":"2024-04-16T19:36:29.482688+0000","last_active":"2024-04-16T19:36:29.482688+0000","last_peered":"2024-04-16T19:36:29.482688+0000","last_clean":"2024-04-16T19:36:29.482688+0000","last_became_active":"2024-04-16T17:47:57.180927+0000","last_became_peered":"2024-04-16T17:47:57.180927+0000","last_unstale":"2024-04-16T19:36:29.482688+0000","last_undegraded":"2024-04-16T19:36:29.482688+0000","last_fullsized":"2024-04-16T19:36:29.482688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'563","last_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_deep_scrub":"53'563","last_deep_scrub_stamp":"2024-04-16T19:36:29.482404+0000","last_clean_scrub_stamp":"2024-04-16T19:36:29.482404+0000","objects_scrubbed":19,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.518930+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":678,"num_read_kb":672,"num_write":907,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71952,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'8","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.461301+0000","last_change":"2024-04-16T19:36:21.461301+0000","last_active":"2024-04-16T19:36:21.461301+0000","last_peered":"2024-04-16T19:36:21.461301+0000","last_clean":"2024-04-16T19:36:21.461301+0000","last_became_active":"2024-04-16T17:47:55.183056+0000","last_became_peered":"2024-04-16T17:47:55.183056+0000","last_unstale":"2024-04-16T19:36:21.461301+0000","last_undegraded":"2024-04-16T19:36:21.461301+0000","last_fullsized":"2024-04-16T19:36:21.461301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:21.460913+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.460913+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:00.948470+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":183,"num_write":45,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":109,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:45.460790+0000","last_change":"2024-04-16T19:36:45.460790+0000","last_active":"2024-04-16T19:36:45.460790+0000","last_peered":"2024-04-16T19:36:45.460790+0000","last_clean":"2024-04-16T19:36:45.460790+0000","last_became_active":"2024-04-16T17:47:39.339109+0000","last_became_peered":"2024-04-16T17:47:39.339109+0000","last_unstale":"2024-04-16T19:36:45.460790+0000","last_undegraded":"2024-04-16T19:36:45.460790+0000","last_fullsized":"2024-04-16T19:36:45.460790+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:45.460592+0000","last_clean_scrub_stamp":"2024-04-16T19:36:45.460592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:12.082129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.596749+0000","last_change":"2024-04-16T19:37:11.596852+0000","last_active":"2024-04-16T19:37:11.596749+0000","last_peered":"2024-04-16T19:37:11.596749+0000","last_clean":"2024-04-16T19:37:11.596749+0000","last_became_active":"2024-04-16T17:47:23.263096+0000","last_became_peered":"2024-04-16T17:47:23.263096+0000","last_unstale":"2024-04-16T19:37:11.596749+0000","last_undegraded":"2024-04-16T19:37:11.596749+0000","last_fullsized":"2024-04-16T19:37:11.596749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.596575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.596575+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.596575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:32.592380+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:36.608519+0000","last_change":"2024-04-16T19:37:36.608573+0000","last_active":"2024-04-16T19:37:36.608519+0000","last_peered":"2024-04-16T19:37:36.608519+0000","last_clean":"2024-04-16T19:37:36.608519+0000","last_became_active":"2024-04-16T17:47:21.235319+0000","last_became_peered":"2024-04-16T17:47:21.235319+0000","last_unstale":"2024-04-16T19:37:36.608519+0000","last_undegraded":"2024-04-16T19:37:36.608519+0000","last_fullsized":"2024-04-16T19:37:36.608519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:36.608380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:36.608380+0000","last_clean_scrub_stamp":"2024-04-16T19:37:36.608380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:32:24.604840+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:11.501299+0000","last_change":"2024-04-16T19:37:11.501299+0000","last_active":"2024-04-16T19:37:11.501299+0000","last_peered":"2024-04-16T19:37:11.501299+0000","last_clean":"2024-04-16T19:37:11.501299+0000","last_became_active":"2024-04-16T17:47:33.306512+0000","last_became_peered":"2024-04-16T17:47:33.306512+0000","last_unstale":"2024-04-16T19:37:11.501299+0000","last_undegraded":"2024-04-16T19:37:11.501299+0000","last_fullsized":"2024-04-16T19:37:11.501299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:11.501103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:11.501103+0000","last_clean_scrub_stamp":"2024-04-16T19:37:11.501103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:40.126284+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"53'84","reported_seq":249,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.376165+0000","last_change":"2024-04-16T19:37:09.376165+0000","last_active":"2024-04-16T19:37:09.376165+0000","last_peered":"2024-04-16T19:37:09.376165+0000","last_clean":"2024-04-16T19:37:09.376165+0000","last_became_active":"2024-04-16T17:47:35.310588+0000","last_became_peered":"2024-04-16T17:47:35.310588+0000","last_unstale":"2024-04-16T19:37:09.376165+0000","last_undegraded":"2024-04-16T19:37:09.376165+0000","last_fullsized":"2024-04-16T19:37:09.376165+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:09.375852+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:09.375852+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.375852+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:20:29.911359+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":116,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.541762+0000","last_change":"2024-04-16T19:37:00.541762+0000","last_active":"2024-04-16T19:37:00.541762+0000","last_peered":"2024-04-16T19:37:00.541762+0000","last_clean":"2024-04-16T19:37:00.541762+0000","last_became_active":"2024-04-16T17:47:37.325757+0000","last_became_peered":"2024-04-16T17:47:37.325757+0000","last_unstale":"2024-04-16T19:37:00.541762+0000","last_undegraded":"2024-04-16T19:37:00.541762+0000","last_fullsized":"2024-04-16T19:37:00.541762+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.541572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.541572+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.541572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:22.120317+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:46.564726+0000","last_change":"2024-04-16T19:37:46.564778+0000","last_active":"2024-04-16T19:37:46.564726+0000","last_peered":"2024-04-16T19:37:46.564726+0000","last_clean":"2024-04-16T19:37:46.564726+0000","last_became_active":"2024-04-16T18:10:50.541690+0000","last_became_peered":"2024-04-16T18:10:50.541690+0000","last_unstale":"2024-04-16T19:37:46.564726+0000","last_undegraded":"2024-04-16T19:37:46.564726+0000","last_fullsized":"2024-04-16T19:37:46.564726+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:46.564590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:46.564590+0000","last_clean_scrub_stamp":"2024-04-16T19:37:46.564590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:31:38.969410+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"53'116","reported_seq":732,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.561294+0000","last_change":"2024-04-16T19:37:04.561294+0000","last_active":"2024-04-16T19:37:04.561294+0000","last_peered":"2024-04-16T19:37:04.561294+0000","last_clean":"2024-04-16T19:37:04.561294+0000","last_became_active":"2024-04-16T17:48:34.326690+0000","last_became_peered":"2024-04-16T17:48:34.326690+0000","last_unstale":"2024-04-16T19:37:04.561294+0000","last_undegraded":"2024-04-16T19:37:04.561294+0000","last_fullsized":"2024-04-16T19:37:04.561294+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'116","last_scrub_stamp":"2024-04-16T19:37:04.561086+0000","last_deep_scrub":"53'116","last_deep_scrub_stamp":"2024-04-16T19:37:04.561086+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.561086+0000","objects_scrubbed":27,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:02.900429+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":989,"num_read_kb":961,"num_write":150,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5638,"num_omap_keys":24,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.678659+0000","last_change":"2024-04-16T19:36:47.678659+0000","last_active":"2024-04-16T19:36:47.678659+0000","last_peered":"2024-04-16T19:36:47.678659+0000","last_clean":"2024-04-16T19:36:47.678659+0000","last_became_active":"2024-04-16T17:48:27.294910+0000","last_became_peered":"2024-04-16T17:48:27.294910+0000","last_unstale":"2024-04-16T19:36:47.678659+0000","last_undegraded":"2024-04-16T19:36:47.678659+0000","last_fullsized":"2024-04-16T19:36:47.678659+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.678408+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.678408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:02:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"53'16","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.735233+0000","last_change":"2024-04-16T19:36:41.735233+0000","last_active":"2024-04-16T19:36:41.735233+0000","last_peered":"2024-04-16T19:36:41.735233+0000","last_clean":"2024-04-16T19:36:41.735233+0000","last_became_active":"2024-04-16T17:48:01.245905+0000","last_became_peered":"2024-04-16T17:48:01.245905+0000","last_unstale":"2024-04-16T19:36:41.735233+0000","last_undegraded":"2024-04-16T19:36:41.735233+0000","last_fullsized":"2024-04-16T19:36:41.735233+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:41.735066+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.735066+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:12:52.416728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":98,"num_read_kb":74,"num_write":32,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.470929+0000","last_change":"2024-04-16T19:36:33.470929+0000","last_active":"2024-04-16T19:36:33.470929+0000","last_peered":"2024-04-16T19:36:33.470929+0000","last_clean":"2024-04-16T19:36:33.470929+0000","last_became_active":"2024-04-16T17:47:59.177814+0000","last_became_peered":"2024-04-16T17:47:59.177814+0000","last_unstale":"2024-04-16T19:36:33.470929+0000","last_undegraded":"2024-04-16T19:36:33.470929+0000","last_fullsized":"2024-04-16T19:36:33.470929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:33.470725+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.470725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:02.342594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"53'476","reported_seq":4439,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.660343+0000","last_change":"2024-04-16T19:36:19.660343+0000","last_active":"2024-04-16T19:36:19.660343+0000","last_peered":"2024-04-16T19:36:19.660343+0000","last_clean":"2024-04-16T19:36:19.660343+0000","last_became_active":"2024-04-16T17:47:57.180952+0000","last_became_peered":"2024-04-16T17:47:57.180952+0000","last_unstale":"2024-04-16T19:36:19.660343+0000","last_undegraded":"2024-04-16T19:36:19.660343+0000","last_fullsized":"2024-04-16T19:36:19.660343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'476","last_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_deep_scrub":"53'476","last_deep_scrub_stamp":"2024-04-16T19:36:19.659960+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.659960+0000","objects_scrubbed":19,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:52.175458+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4840,"num_read_kb":4656,"num_write":644,"num_write_kb":395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"53'8","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:36.516257+0000","last_change":"2024-04-16T19:36:36.516257+0000","last_active":"2024-04-16T19:36:36.516257+0000","last_peered":"2024-04-16T19:36:36.516257+0000","last_clean":"2024-04-16T19:36:36.516257+0000","last_became_active":"2024-04-16T17:47:55.168699+0000","last_became_peered":"2024-04-16T17:47:55.168699+0000","last_unstale":"2024-04-16T19:36:36.516257+0000","last_undegraded":"2024-04-16T19:36:36.516257+0000","last_fullsized":"2024-04-16T19:36:36.516257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:36.516049+0000","last_clean_scrub_stamp":"2024-04-16T19:36:36.516049+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:15:56.739641+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":41,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.683485+0000","last_change":"2024-04-16T19:37:24.683485+0000","last_active":"2024-04-16T19:37:24.683485+0000","last_peered":"2024-04-16T19:37:24.683485+0000","last_clean":"2024-04-16T19:37:24.683485+0000","last_became_active":"2024-04-16T17:47:23.257092+0000","last_became_peered":"2024-04-16T17:47:23.257092+0000","last_unstale":"2024-04-16T19:37:24.683485+0000","last_undegraded":"2024-04-16T19:37:24.683485+0000","last_fullsized":"2024-04-16T19:37:24.683485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.683294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.683294+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.683294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:08.684485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.472813+0000","last_change":"2024-04-16T19:37:24.472868+0000","last_active":"2024-04-16T19:37:24.472813+0000","last_peered":"2024-04-16T19:37:24.472813+0000","last_clean":"2024-04-16T19:37:24.472813+0000","last_became_active":"2024-04-16T17:47:21.239645+0000","last_became_peered":"2024-04-16T17:47:21.239645+0000","last_unstale":"2024-04-16T19:37:24.472813+0000","last_undegraded":"2024-04-16T19:37:24.472813+0000","last_fullsized":"2024-04-16T19:37:24.472813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.472644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.472644+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.472644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:40.759382+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:43.660590+0000","last_change":"2024-04-16T19:37:43.660645+0000","last_active":"2024-04-16T19:37:43.660590+0000","last_peered":"2024-04-16T19:37:43.660590+0000","last_clean":"2024-04-16T19:37:43.660590+0000","last_became_active":"2024-04-16T17:47:33.298218+0000","last_became_peered":"2024-04-16T17:47:33.298218+0000","last_unstale":"2024-04-16T19:37:43.660590+0000","last_undegraded":"2024-04-16T19:37:43.660590+0000","last_fullsized":"2024-04-16T19:37:43.660590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:43.660448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:43.660448+0000","last_clean_scrub_stamp":"2024-04-16T19:37:43.660448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:25.751341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"53'93","reported_seq":264,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.659310+0000","last_change":"2024-04-16T19:37:00.659310+0000","last_active":"2024-04-16T19:37:00.659310+0000","last_peered":"2024-04-16T19:37:00.659310+0000","last_clean":"2024-04-16T19:37:00.659310+0000","last_became_active":"2024-04-16T17:47:35.315910+0000","last_became_peered":"2024-04-16T17:47:35.315910+0000","last_unstale":"2024-04-16T19:37:00.659310+0000","last_undegraded":"2024-04-16T19:37:00.659310+0000","last_fullsized":"2024-04-16T19:37:00.659310+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'93","last_scrub_stamp":"2024-04-16T19:37:00.659199+0000","last_deep_scrub":"53'93","last_deep_scrub_stamp":"2024-04-16T19:37:00.659199+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.659199+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:06.665506+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":133,"num_write":88,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.473385+0000","last_change":"2024-04-16T19:37:01.473385+0000","last_active":"2024-04-16T19:37:01.473385+0000","last_peered":"2024-04-16T19:37:01.473385+0000","last_clean":"2024-04-16T19:37:01.473385+0000","last_became_active":"2024-04-16T17:47:37.330086+0000","last_became_peered":"2024-04-16T17:47:37.330086+0000","last_unstale":"2024-04-16T19:37:01.473385+0000","last_undegraded":"2024-04-16T19:37:01.473385+0000","last_fullsized":"2024-04-16T19:37:01.473385+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:01.473191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:01.473191+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.473191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:28:23.051936+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.511222+0000","last_change":"2024-04-16T19:36:25.511222+0000","last_active":"2024-04-16T19:36:25.511222+0000","last_peered":"2024-04-16T19:36:25.511222+0000","last_clean":"2024-04-16T19:36:25.511222+0000","last_became_active":"2024-04-16T17:47:39.700952+0000","last_became_peered":"2024-04-16T17:47:39.700952+0000","last_unstale":"2024-04-16T19:36:25.511222+0000","last_undegraded":"2024-04-16T19:36:25.511222+0000","last_fullsized":"2024-04-16T19:36:25.511222+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:25.511031+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.511031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:41:57.452906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":61,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:52.550559+0000","last_change":"2024-04-16T19:37:52.550614+0000","last_active":"2024-04-16T19:37:52.550559+0000","last_peered":"2024-04-16T19:37:52.550559+0000","last_clean":"2024-04-16T19:37:52.550559+0000","last_became_active":"2024-04-16T18:10:50.561326+0000","last_became_peered":"2024-04-16T18:10:50.561326+0000","last_unstale":"2024-04-16T19:37:52.550559+0000","last_undegraded":"2024-04-16T19:37:52.550559+0000","last_fullsized":"2024-04-16T19:37:52.550559+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:52.550416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:52.550416+0000","last_clean_scrub_stamp":"2024-04-16T19:37:52.550416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:26:38.904801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.14","version":"53'102","reported_seq":1008,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.618748+0000","last_change":"2024-04-16T19:36:48.618887+0000","last_active":"2024-04-16T19:36:48.618748+0000","last_peered":"2024-04-16T19:36:48.618748+0000","last_clean":"2024-04-16T19:36:48.618748+0000","last_became_active":"2024-04-16T17:48:34.312284+0000","last_became_peered":"2024-04-16T17:48:34.312284+0000","last_unstale":"2024-04-16T19:36:48.618748+0000","last_undegraded":"2024-04-16T19:36:48.618748+0000","last_fullsized":"2024-04-16T19:36:48.618748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'102","last_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_deep_scrub":"53'102","last_deep_scrub_stamp":"2024-04-16T19:36:48.618582+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.618582+0000","objects_scrubbed":31,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:51:54.031308+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1364,"num_read_kb":1369,"num_write":107,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7116,"num_omap_keys":28,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.371768+0000","last_change":"2024-04-16T19:37:15.371768+0000","last_active":"2024-04-16T19:37:15.371768+0000","last_peered":"2024-04-16T19:37:15.371768+0000","last_clean":"2024-04-16T19:37:15.371768+0000","last_became_active":"2024-04-16T17:48:27.308267+0000","last_became_peered":"2024-04-16T17:48:27.308267+0000","last_unstale":"2024-04-16T19:37:15.371768+0000","last_undegraded":"2024-04-16T19:37:15.371768+0000","last_fullsized":"2024-04-16T19:37:15.371768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.371583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.371583+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.371583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:51.308528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"53'7","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.481640+0000","last_change":"2024-04-16T19:36:54.481710+0000","last_active":"2024-04-16T19:36:54.481640+0000","last_peered":"2024-04-16T19:36:54.481640+0000","last_clean":"2024-04-16T19:36:54.481640+0000","last_became_active":"2024-04-16T17:48:01.249978+0000","last_became_peered":"2024-04-16T17:48:01.249978+0000","last_unstale":"2024-04-16T19:36:54.481640+0000","last_undegraded":"2024-04-16T19:36:54.481640+0000","last_fullsized":"2024-04-16T19:36:54.481640+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:36:54.481476+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.481476+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:28.035078+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":586,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.550520+0000","last_change":"2024-04-16T19:36:28.550520+0000","last_active":"2024-04-16T19:36:28.550520+0000","last_peered":"2024-04-16T19:36:28.550520+0000","last_clean":"2024-04-16T19:36:28.550520+0000","last_became_active":"2024-04-16T17:47:59.173570+0000","last_became_peered":"2024-04-16T17:47:59.173570+0000","last_unstale":"2024-04-16T19:36:28.550520+0000","last_undegraded":"2024-04-16T19:36:28.550520+0000","last_fullsized":"2024-04-16T19:36:28.550520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:28.550328+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.550328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:52.280854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'11393","reported_seq":61824,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.677565+0000","last_change":"2024-04-16T19:36:19.677565+0000","last_active":"2024-04-16T19:36:19.677565+0000","last_peered":"2024-04-16T19:36:19.677565+0000","last_clean":"2024-04-16T19:36:19.677565+0000","last_became_active":"2024-04-16T17:47:57.190809+0000","last_became_peered":"2024-04-16T17:47:57.190809+0000","last_unstale":"2024-04-16T19:36:19.677565+0000","last_undegraded":"2024-04-16T19:36:19.677565+0000","last_fullsized":"2024-04-16T19:36:19.677565+0000","mapping_epoch":34,"log_start":"53'9700","ondisk_log_start":"53'9700","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11393","last_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_deep_scrub":"53'11393","last_deep_scrub_stamp":"2024-04-16T19:36:19.677264+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.677264+0000","objects_scrubbed":23,"log_size":1693,"log_dups_size":3000,"ondisk_log_size":1693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:06:14.777503+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":149698,"num_read_kb":107305,"num_write":9007,"num_write_kb":1062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209607,"num_omap_keys":977,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'23","reported_seq":359,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:18.461701+0000","last_change":"2024-04-16T19:36:18.461701+0000","last_active":"2024-04-16T19:36:18.461701+0000","last_peered":"2024-04-16T19:36:18.461701+0000","last_clean":"2024-04-16T19:36:18.461701+0000","last_became_active":"2024-04-16T17:47:55.176943+0000","last_became_peered":"2024-04-16T17:47:55.176943+0000","last_unstale":"2024-04-16T19:36:18.461701+0000","last_undegraded":"2024-04-16T19:36:18.461701+0000","last_fullsized":"2024-04-16T19:36:18.461701+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'23","last_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_deep_scrub":"53'23","last_deep_scrub_stamp":"2024-04-16T19:36:18.461393+0000","last_clean_scrub_stamp":"2024-04-16T19:36:18.461393+0000","objects_scrubbed":11,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:24.888175+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":587,"num_read_kb":380,"num_write":133,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.617521+0000","last_change":"2024-04-16T19:37:25.617573+0000","last_active":"2024-04-16T19:37:25.617521+0000","last_peered":"2024-04-16T19:37:25.617521+0000","last_clean":"2024-04-16T19:37:25.617521+0000","last_became_active":"2024-04-16T17:47:23.262310+0000","last_became_peered":"2024-04-16T17:47:23.262310+0000","last_unstale":"2024-04-16T19:37:25.617521+0000","last_undegraded":"2024-04-16T19:37:25.617521+0000","last_fullsized":"2024-04-16T19:37:25.617521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.617386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.617386+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.617386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:26:27.558930+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.633414+0000","last_change":"2024-04-16T19:37:24.633506+0000","last_active":"2024-04-16T19:37:24.633414+0000","last_peered":"2024-04-16T19:37:24.633414+0000","last_clean":"2024-04-16T19:37:24.633414+0000","last_became_active":"2024-04-16T17:47:21.239384+0000","last_became_peered":"2024-04-16T17:47:21.239384+0000","last_unstale":"2024-04-16T19:37:24.633414+0000","last_undegraded":"2024-04-16T19:37:24.633414+0000","last_fullsized":"2024-04-16T19:37:24.633414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.633250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.633250+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.633250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:05.960575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.570694+0000","last_change":"2024-04-16T19:37:15.570748+0000","last_active":"2024-04-16T19:37:15.570694+0000","last_peered":"2024-04-16T19:37:15.570694+0000","last_clean":"2024-04-16T19:37:15.570694+0000","last_became_active":"2024-04-16T17:47:33.295908+0000","last_became_peered":"2024-04-16T17:47:33.295908+0000","last_unstale":"2024-04-16T19:37:15.570694+0000","last_undegraded":"2024-04-16T19:37:15.570694+0000","last_fullsized":"2024-04-16T19:37:15.570694+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.570547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.570547+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.570547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:35.464724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"53'159","reported_seq":357,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.579240+0000","last_change":"2024-04-16T19:37:07.579240+0000","last_active":"2024-04-16T19:37:07.579240+0000","last_peered":"2024-04-16T19:37:07.579240+0000","last_clean":"2024-04-16T19:37:07.579240+0000","last_became_active":"2024-04-16T17:47:35.316352+0000","last_became_peered":"2024-04-16T17:47:35.316352+0000","last_unstale":"2024-04-16T19:37:07.579240+0000","last_undegraded":"2024-04-16T19:37:07.579240+0000","last_fullsized":"2024-04-16T19:37:07.579240+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:07.579029+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:07.579029+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.579029+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:35:29.684066+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.656690+0000","last_change":"2024-04-16T19:36:53.656690+0000","last_active":"2024-04-16T19:36:53.656690+0000","last_peered":"2024-04-16T19:36:53.656690+0000","last_clean":"2024-04-16T19:36:53.656690+0000","last_became_active":"2024-04-16T17:47:37.332883+0000","last_became_peered":"2024-04-16T17:47:37.332883+0000","last_unstale":"2024-04-16T19:36:53.656690+0000","last_undegraded":"2024-04-16T19:36:53.656690+0000","last_fullsized":"2024-04-16T19:36:53.656690+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.656501+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.656501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:12.757030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.426597+0000","last_change":"2024-04-16T19:36:35.426597+0000","last_active":"2024-04-16T19:36:35.426597+0000","last_peered":"2024-04-16T19:36:35.426597+0000","last_clean":"2024-04-16T19:36:35.426597+0000","last_became_active":"2024-04-16T17:47:39.698907+0000","last_became_peered":"2024-04-16T17:47:39.698907+0000","last_unstale":"2024-04-16T19:36:35.426597+0000","last_undegraded":"2024-04-16T19:36:35.426597+0000","last_fullsized":"2024-04-16T19:36:35.426597+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:35.426385+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.426385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:45:09.737500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":61,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:50.586901+0000","last_change":"2024-04-16T19:37:50.586901+0000","last_active":"2024-04-16T19:37:50.586901+0000","last_peered":"2024-04-16T19:37:50.586901+0000","last_clean":"2024-04-16T19:37:50.586901+0000","last_became_active":"2024-04-16T18:10:50.552076+0000","last_became_peered":"2024-04-16T18:10:50.552076+0000","last_unstale":"2024-04-16T19:37:50.586901+0000","last_undegraded":"2024-04-16T19:37:50.586901+0000","last_fullsized":"2024-04-16T19:37:50.586901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:50.586698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:50.586698+0000","last_clean_scrub_stamp":"2024-04-16T19:37:50.586698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:12:19.483205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"53'143","reported_seq":1870,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:51.706017+0000","last_change":"2024-04-16T19:36:51.706017+0000","last_active":"2024-04-16T19:36:51.706017+0000","last_peered":"2024-04-16T19:36:51.706017+0000","last_clean":"2024-04-16T19:36:51.706017+0000","last_became_active":"2024-04-16T17:48:34.336895+0000","last_became_peered":"2024-04-16T17:48:34.336895+0000","last_unstale":"2024-04-16T19:36:51.706017+0000","last_undegraded":"2024-04-16T19:36:51.706017+0000","last_fullsized":"2024-04-16T19:36:51.706017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'143","last_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_deep_scrub":"53'143","last_deep_scrub_stamp":"2024-04-16T19:36:51.705779+0000","last_clean_scrub_stamp":"2024-04-16T19:36:51.705779+0000","objects_scrubbed":40,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:42.464983+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2905,"num_read_kb":2872,"num_write":143,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7446,"num_omap_keys":29,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.646565+0000","last_change":"2024-04-16T19:36:48.646617+0000","last_active":"2024-04-16T19:36:48.646565+0000","last_peered":"2024-04-16T19:36:48.646565+0000","last_clean":"2024-04-16T19:36:48.646565+0000","last_became_active":"2024-04-16T17:48:27.293227+0000","last_became_peered":"2024-04-16T17:48:27.293227+0000","last_unstale":"2024-04-16T19:36:48.646565+0000","last_undegraded":"2024-04-16T19:36:48.646565+0000","last_fullsized":"2024-04-16T19:36:48.646565+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:48.646427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.646427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:22:00.794914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"53'10","reported_seq":103,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:56.516567+0000","last_change":"2024-04-16T19:36:56.516633+0000","last_active":"2024-04-16T19:36:56.516567+0000","last_peered":"2024-04-16T19:36:56.516567+0000","last_clean":"2024-04-16T19:36:56.516567+0000","last_became_active":"2024-04-16T17:48:01.247729+0000","last_became_peered":"2024-04-16T17:48:01.247729+0000","last_unstale":"2024-04-16T19:36:56.516567+0000","last_undegraded":"2024-04-16T19:36:56.516567+0000","last_fullsized":"2024-04-16T19:36:56.516567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:56.516427+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:56.516427+0000","last_clean_scrub_stamp":"2024-04-16T19:36:56.516427+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:30.858432+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.416865+0000","last_change":"2024-04-16T19:36:55.416865+0000","last_active":"2024-04-16T19:36:55.416865+0000","last_peered":"2024-04-16T19:36:55.416865+0000","last_clean":"2024-04-16T19:36:55.416865+0000","last_became_active":"2024-04-16T17:47:59.187047+0000","last_became_peered":"2024-04-16T17:47:59.187047+0000","last_unstale":"2024-04-16T19:36:55.416865+0000","last_undegraded":"2024-04-16T19:36:55.416865+0000","last_fullsized":"2024-04-16T19:36:55.416865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:55.416624+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.416624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:21.421511+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"53'206","reported_seq":441,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:35.574523+0000","last_change":"2024-04-16T19:36:35.574523+0000","last_active":"2024-04-16T19:36:35.574523+0000","last_peered":"2024-04-16T19:36:35.574523+0000","last_clean":"2024-04-16T19:36:35.574523+0000","last_became_active":"2024-04-16T17:47:57.194776+0000","last_became_peered":"2024-04-16T17:47:57.194776+0000","last_unstale":"2024-04-16T19:36:35.574523+0000","last_undegraded":"2024-04-16T19:36:35.574523+0000","last_fullsized":"2024-04-16T19:36:35.574523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'206","last_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_deep_scrub":"53'206","last_deep_scrub_stamp":"2024-04-16T19:36:35.574163+0000","last_clean_scrub_stamp":"2024-04-16T19:36:35.574163+0000","objects_scrubbed":10,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:15.061721+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":309,"num_write":198,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'20","reported_seq":224,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.646869+0000","last_change":"2024-04-16T19:36:30.646869+0000","last_active":"2024-04-16T19:36:30.646869+0000","last_peered":"2024-04-16T19:36:30.646869+0000","last_clean":"2024-04-16T19:36:30.646869+0000","last_became_active":"2024-04-16T17:47:55.159458+0000","last_became_peered":"2024-04-16T17:47:55.159458+0000","last_unstale":"2024-04-16T19:36:30.646869+0000","last_undegraded":"2024-04-16T19:36:30.646869+0000","last_fullsized":"2024-04-16T19:36:30.646869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:30.646698+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.646698+0000","objects_scrubbed":8,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:37.151471+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":252,"num_read_kb":165,"num_write":123,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.645921+0000","last_change":"2024-04-16T19:37:20.645921+0000","last_active":"2024-04-16T19:37:20.645921+0000","last_peered":"2024-04-16T19:37:20.645921+0000","last_clean":"2024-04-16T19:37:20.645921+0000","last_became_active":"2024-04-16T17:47:23.257878+0000","last_became_peered":"2024-04-16T17:47:23.257878+0000","last_unstale":"2024-04-16T19:37:20.645921+0000","last_undegraded":"2024-04-16T19:37:20.645921+0000","last_fullsized":"2024-04-16T19:37:20.645921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.645734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.645734+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.645734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:44.885356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.452849+0000","last_change":"2024-04-16T19:37:25.452902+0000","last_active":"2024-04-16T19:37:25.452849+0000","last_peered":"2024-04-16T19:37:25.452849+0000","last_clean":"2024-04-16T19:37:25.452849+0000","last_became_active":"2024-04-16T17:47:21.229728+0000","last_became_peered":"2024-04-16T17:47:21.229728+0000","last_unstale":"2024-04-16T19:37:25.452849+0000","last_undegraded":"2024-04-16T19:37:25.452849+0000","last_fullsized":"2024-04-16T19:37:25.452849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.452712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.452712+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.452712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:42.051092+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"53'7","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.654446+0000","last_change":"2024-04-16T19:37:05.654446+0000","last_active":"2024-04-16T19:37:05.654446+0000","last_peered":"2024-04-16T19:37:05.654446+0000","last_clean":"2024-04-16T19:37:05.654446+0000","last_became_active":"2024-04-16T17:47:33.305449+0000","last_became_peered":"2024-04-16T17:47:33.305449+0000","last_unstale":"2024-04-16T19:37:05.654446+0000","last_undegraded":"2024-04-16T19:37:05.654446+0000","last_fullsized":"2024-04-16T19:37:05.654446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2024-04-16T19:37:05.654239+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2024-04-16T19:37:05.654239+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.654239+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:46:11.660547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":22,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"53'109","reported_seq":291,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.373414+0000","last_change":"2024-04-16T19:36:57.373474+0000","last_active":"2024-04-16T19:36:57.373414+0000","last_peered":"2024-04-16T19:36:57.373414+0000","last_clean":"2024-04-16T19:36:57.373414+0000","last_became_active":"2024-04-16T17:47:35.313907+0000","last_became_peered":"2024-04-16T17:47:35.313907+0000","last_unstale":"2024-04-16T19:36:57.373414+0000","last_undegraded":"2024-04-16T19:36:57.373414+0000","last_fullsized":"2024-04-16T19:36:57.373414+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'109","last_scrub_stamp":"2024-04-16T19:36:57.373338+0000","last_deep_scrub":"53'109","last_deep_scrub_stamp":"2024-04-16T19:36:57.373338+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.373338+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:48.848526+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":104,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.699354+0000","last_change":"2024-04-16T19:36:54.699404+0000","last_active":"2024-04-16T19:36:54.699354+0000","last_peered":"2024-04-16T19:36:54.699354+0000","last_clean":"2024-04-16T19:36:54.699354+0000","last_became_active":"2024-04-16T17:47:37.324049+0000","last_became_peered":"2024-04-16T17:47:37.324049+0000","last_unstale":"2024-04-16T19:36:54.699354+0000","last_undegraded":"2024-04-16T19:36:54.699354+0000","last_fullsized":"2024-04-16T19:36:54.699354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:54.699216+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.699216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:21.320753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:42.350584+0000","last_change":"2024-04-16T19:36:42.350584+0000","last_active":"2024-04-16T19:36:42.350584+0000","last_peered":"2024-04-16T19:36:42.350584+0000","last_clean":"2024-04-16T19:36:42.350584+0000","last_became_active":"2024-04-16T17:47:39.322741+0000","last_became_peered":"2024-04-16T17:47:39.322741+0000","last_unstale":"2024-04-16T19:36:42.350584+0000","last_undegraded":"2024-04-16T19:36:42.350584+0000","last_fullsized":"2024-04-16T19:36:42.350584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:42.350304+0000","last_clean_scrub_stamp":"2024-04-16T19:36:42.350304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:41:08.971842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":54,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:40.802322+0000","last_change":"2024-04-16T19:37:40.802375+0000","last_active":"2024-04-16T19:37:40.802322+0000","last_peered":"2024-04-16T19:37:40.802322+0000","last_clean":"2024-04-16T19:37:40.802322+0000","last_became_active":"2024-04-16T18:11:33.605431+0000","last_became_peered":"2024-04-16T18:11:33.605431+0000","last_unstale":"2024-04-16T19:37:40.802322+0000","last_undegraded":"2024-04-16T19:37:40.802322+0000","last_fullsized":"2024-04-16T19:37:40.802322+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:40.802184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:40.802184+0000","last_clean_scrub_stamp":"2024-04-16T19:37:40.802184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:01:30.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.16","version":"53'90","reported_seq":914,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.682412+0000","last_change":"2024-04-16T19:37:07.682412+0000","last_active":"2024-04-16T19:37:07.682412+0000","last_peered":"2024-04-16T19:37:07.682412+0000","last_clean":"2024-04-16T19:37:07.682412+0000","last_became_active":"2024-04-16T17:48:34.342517+0000","last_became_peered":"2024-04-16T17:48:34.342517+0000","last_unstale":"2024-04-16T19:37:07.682412+0000","last_undegraded":"2024-04-16T19:37:07.682412+0000","last_fullsized":"2024-04-16T19:37:07.682412+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'90","last_scrub_stamp":"2024-04-16T19:37:07.682156+0000","last_deep_scrub":"53'90","last_deep_scrub_stamp":"2024-04-16T19:37:07.682156+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.682156+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:33.687043+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1369,"num_read_kb":1356,"num_write":105,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5939,"num_omap_keys":27,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.668638+0000","last_change":"2024-04-16T19:36:44.668638+0000","last_active":"2024-04-16T19:36:44.668638+0000","last_peered":"2024-04-16T19:36:44.668638+0000","last_clean":"2024-04-16T19:36:44.668638+0000","last_became_active":"2024-04-16T17:48:27.303580+0000","last_became_peered":"2024-04-16T17:48:27.303580+0000","last_unstale":"2024-04-16T19:36:44.668638+0000","last_undegraded":"2024-04-16T19:36:44.668638+0000","last_fullsized":"2024-04-16T19:36:44.668638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:44.668449+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.668449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:31.847082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"53'2","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.465959+0000","last_change":"2024-04-16T19:36:53.466069+0000","last_active":"2024-04-16T19:36:53.465959+0000","last_peered":"2024-04-16T19:36:53.465959+0000","last_clean":"2024-04-16T19:36:53.465959+0000","last_became_active":"2024-04-16T17:48:01.253385+0000","last_became_peered":"2024-04-16T17:48:01.253385+0000","last_unstale":"2024-04-16T19:36:53.465959+0000","last_undegraded":"2024-04-16T19:36:53.465959+0000","last_fullsized":"2024-04-16T19:36:53.465959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:53.465746+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.465746+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:19.042353+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.452876+0000","last_change":"2024-04-16T19:36:41.452876+0000","last_active":"2024-04-16T19:36:41.452876+0000","last_peered":"2024-04-16T19:36:41.452876+0000","last_clean":"2024-04-16T19:36:41.452876+0000","last_became_active":"2024-04-16T17:47:59.194940+0000","last_became_peered":"2024-04-16T17:47:59.194940+0000","last_unstale":"2024-04-16T19:36:41.452876+0000","last_undegraded":"2024-04-16T19:36:41.452876+0000","last_fullsized":"2024-04-16T19:36:41.452876+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.452676+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.452676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:06:06.846462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"53'509","reported_seq":3968,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:20.636035+0000","last_change":"2024-04-16T19:36:20.636035+0000","last_active":"2024-04-16T19:36:20.636035+0000","last_peered":"2024-04-16T19:36:20.636035+0000","last_clean":"2024-04-16T19:36:20.636035+0000","last_became_active":"2024-04-16T17:47:57.173715+0000","last_became_peered":"2024-04-16T17:47:57.173715+0000","last_unstale":"2024-04-16T19:36:20.636035+0000","last_undegraded":"2024-04-16T19:36:20.636035+0000","last_fullsized":"2024-04-16T19:36:20.636035+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2024-04-16T19:36:20.635659+0000","last_clean_scrub_stamp":"2024-04-16T19:36:20.635659+0000","objects_scrubbed":21,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:04.155514+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7051,"num_read_kb":7045,"num_write":835,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71921,"num_omap_keys":314,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"53'34","reported_seq":244,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:23.437306+0000","last_change":"2024-04-16T19:36:23.437306+0000","last_active":"2024-04-16T19:36:23.437306+0000","last_peered":"2024-04-16T19:36:23.437306+0000","last_clean":"2024-04-16T19:36:23.437306+0000","last_became_active":"2024-04-16T17:47:55.164116+0000","last_became_peered":"2024-04-16T17:47:55.164116+0000","last_unstale":"2024-04-16T19:36:23.437306+0000","last_undegraded":"2024-04-16T19:36:23.437306+0000","last_fullsized":"2024-04-16T19:36:23.437306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:36:23.436922+0000","last_clean_scrub_stamp":"2024-04-16T19:36:23.436922+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:38.568868+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":293,"num_read_kb":218,"num_write":186,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.447112+0000","last_change":"2024-04-16T19:37:17.447112+0000","last_active":"2024-04-16T19:37:17.447112+0000","last_peered":"2024-04-16T19:37:17.447112+0000","last_clean":"2024-04-16T19:37:17.447112+0000","last_became_active":"2024-04-16T17:47:23.249834+0000","last_became_peered":"2024-04-16T17:47:23.249834+0000","last_unstale":"2024-04-16T19:37:17.447112+0000","last_undegraded":"2024-04-16T19:37:17.447112+0000","last_fullsized":"2024-04-16T19:37:17.447112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:17.446905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:17.446905+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.446905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:48:43.662599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:42.630041+0000","last_change":"2024-04-16T19:37:42.630098+0000","last_active":"2024-04-16T19:37:42.630041+0000","last_peered":"2024-04-16T19:37:42.630041+0000","last_clean":"2024-04-16T19:37:42.630041+0000","last_became_active":"2024-04-16T17:47:21.243511+0000","last_became_peered":"2024-04-16T17:47:21.243511+0000","last_unstale":"2024-04-16T19:37:42.630041+0000","last_undegraded":"2024-04-16T19:37:42.630041+0000","last_fullsized":"2024-04-16T19:37:42.630041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:42.629899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:42.629899+0000","last_clean_scrub_stamp":"2024-04-16T19:37:42.629899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:09:41.670212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"53'5","reported_seq":814,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:06.631427+0000","last_change":"2024-04-16T19:37:06.631427+0000","last_active":"2024-04-16T19:37:06.631427+0000","last_peered":"2024-04-16T19:37:06.631427+0000","last_clean":"2024-04-16T19:37:06.631427+0000","last_became_active":"2024-04-16T17:47:33.297139+0000","last_became_peered":"2024-04-16T17:47:33.297139+0000","last_unstale":"2024-04-16T19:37:06.631427+0000","last_undegraded":"2024-04-16T19:37:06.631427+0000","last_fullsized":"2024-04-16T19:37:06.631427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'5","last_scrub_stamp":"2024-04-16T19:37:06.631224+0000","last_deep_scrub":"53'5","last_deep_scrub_stamp":"2024-04-16T19:37:06.631224+0000","last_clean_scrub_stamp":"2024-04-16T19:37:06.631224+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:52:08.200734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1211,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":878,"num_read_kb":1249,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"53'125","reported_seq":312,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:05.573654+0000","last_change":"2024-04-16T19:37:05.573733+0000","last_active":"2024-04-16T19:37:05.573654+0000","last_peered":"2024-04-16T19:37:05.573654+0000","last_clean":"2024-04-16T19:37:05.573654+0000","last_became_active":"2024-04-16T17:47:35.318938+0000","last_became_peered":"2024-04-16T17:47:35.318938+0000","last_unstale":"2024-04-16T19:37:05.573654+0000","last_undegraded":"2024-04-16T19:37:05.573654+0000","last_fullsized":"2024-04-16T19:37:05.573654+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'125","last_scrub_stamp":"2024-04-16T19:37:05.573508+0000","last_deep_scrub":"53'125","last_deep_scrub_stamp":"2024-04-16T19:37:05.573508+0000","last_clean_scrub_stamp":"2024-04-16T19:37:05.573508+0000","objects_scrubbed":5,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:54.563678+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":175,"num_write":120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'13","reported_seq":1350,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:01.345458+0000","last_change":"2024-04-16T19:37:01.345526+0000","last_active":"2024-04-16T19:37:01.345458+0000","last_peered":"2024-04-16T19:37:01.345458+0000","last_clean":"2024-04-16T19:37:01.345458+0000","last_became_active":"2024-04-16T17:47:37.329013+0000","last_became_peered":"2024-04-16T17:47:37.329013+0000","last_unstale":"2024-04-16T19:37:01.345458+0000","last_undegraded":"2024-04-16T19:37:01.345458+0000","last_fullsized":"2024-04-16T19:37:01.345458+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:37:01.345307+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:37:01.345307+0000","last_clean_scrub_stamp":"2024-04-16T19:37:01.345307+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:14:27.712980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:39.388173+0000","last_change":"2024-04-16T19:36:39.388173+0000","last_active":"2024-04-16T19:36:39.388173+0000","last_peered":"2024-04-16T19:36:39.388173+0000","last_clean":"2024-04-16T19:36:39.388173+0000","last_became_active":"2024-04-16T17:47:39.331705+0000","last_became_peered":"2024-04-16T17:47:39.331705+0000","last_unstale":"2024-04-16T19:36:39.388173+0000","last_undegraded":"2024-04-16T19:36:39.388173+0000","last_fullsized":"2024-04-16T19:36:39.388173+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:39.387834+0000","last_clean_scrub_stamp":"2024-04-16T19:36:39.387834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:30.147039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:36.511003+0000","last_change":"2024-04-16T19:37:36.511057+0000","last_active":"2024-04-16T19:37:36.511003+0000","last_peered":"2024-04-16T19:37:36.511003+0000","last_clean":"2024-04-16T19:37:36.511003+0000","last_became_active":"2024-04-16T18:10:50.554521+0000","last_became_peered":"2024-04-16T18:10:50.554521+0000","last_unstale":"2024-04-16T19:37:36.511003+0000","last_undegraded":"2024-04-16T19:37:36.511003+0000","last_fullsized":"2024-04-16T19:37:36.511003+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:36.510868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:36.510868+0000","last_clean_scrub_stamp":"2024-04-16T19:37:36.510868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:01:01.648805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"53'123","reported_seq":1604,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.593801+0000","last_change":"2024-04-16T19:36:52.593899+0000","last_active":"2024-04-16T19:36:52.593801+0000","last_peered":"2024-04-16T19:36:52.593801+0000","last_clean":"2024-04-16T19:36:52.593801+0000","last_became_active":"2024-04-16T17:48:34.317907+0000","last_became_peered":"2024-04-16T17:48:34.317907+0000","last_unstale":"2024-04-16T19:36:52.593801+0000","last_undegraded":"2024-04-16T19:36:52.593801+0000","last_fullsized":"2024-04-16T19:36:52.593801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'123","last_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_deep_scrub":"53'123","last_deep_scrub_stamp":"2024-04-16T19:36:52.593639+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.593639+0000","objects_scrubbed":37,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:52.173838+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":2495,"num_read_kb":2467,"num_write":114,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5671,"num_omap_keys":23,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:53.668143+0000","last_change":"2024-04-16T19:36:53.668197+0000","last_active":"2024-04-16T19:36:53.668143+0000","last_peered":"2024-04-16T19:36:53.668143+0000","last_clean":"2024-04-16T19:36:53.668143+0000","last_became_active":"2024-04-16T17:48:27.298983+0000","last_became_peered":"2024-04-16T17:48:27.298983+0000","last_unstale":"2024-04-16T19:36:53.668143+0000","last_undegraded":"2024-04-16T19:36:53.668143+0000","last_fullsized":"2024-04-16T19:36:53.668143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:53.668006+0000","last_clean_scrub_stamp":"2024-04-16T19:36:53.668006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:20:47.522228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"53'10","reported_seq":122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.679612+0000","last_change":"2024-04-16T19:36:41.679612+0000","last_active":"2024-04-16T19:36:41.679612+0000","last_peered":"2024-04-16T19:36:41.679612+0000","last_clean":"2024-04-16T19:36:41.679612+0000","last_became_active":"2024-04-16T17:48:01.246090+0000","last_became_peered":"2024-04-16T17:48:01.246090+0000","last_unstale":"2024-04-16T19:36:41.679612+0000","last_undegraded":"2024-04-16T19:36:41.679612+0000","last_fullsized":"2024-04-16T19:36:41.679612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:41.679332+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.679332+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:33.154520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"53'556","reported_seq":1963,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:44.395959+0000","last_change":"2024-04-16T19:36:44.396025+0000","last_active":"2024-04-16T19:36:44.395959+0000","last_peered":"2024-04-16T19:36:44.395959+0000","last_clean":"2024-04-16T19:36:44.395959+0000","last_became_active":"2024-04-16T17:47:59.181681+0000","last_became_peered":"2024-04-16T17:47:59.181681+0000","last_unstale":"2024-04-16T19:36:44.395959+0000","last_undegraded":"2024-04-16T19:36:44.395959+0000","last_fullsized":"2024-04-16T19:36:44.395959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:44.395808+0000","last_clean_scrub_stamp":"2024-04-16T19:36:44.395808+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:46.739394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"53'508","reported_seq":736,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:25.543534+0000","last_change":"2024-04-16T19:36:25.543534+0000","last_active":"2024-04-16T19:36:25.543534+0000","last_peered":"2024-04-16T19:36:25.543534+0000","last_clean":"2024-04-16T19:36:25.543534+0000","last_became_active":"2024-04-16T17:47:57.179378+0000","last_became_peered":"2024-04-16T17:47:57.179378+0000","last_unstale":"2024-04-16T19:36:25.543534+0000","last_undegraded":"2024-04-16T19:36:25.543534+0000","last_fullsized":"2024-04-16T19:36:25.543534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'508","last_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_deep_scrub":"53'508","last_deep_scrub_stamp":"2024-04-16T19:36:25.543268+0000","last_clean_scrub_stamp":"2024-04-16T19:36:25.543268+0000","objects_scrubbed":16,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:48:59.596711+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":608,"num_read_kb":601,"num_write":832,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71934,"num_omap_keys":314,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"53'8","reported_seq":155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.642341+0000","last_change":"2024-04-16T19:36:27.642341+0000","last_active":"2024-04-16T19:36:27.642341+0000","last_peered":"2024-04-16T19:36:27.642341+0000","last_clean":"2024-04-16T19:36:27.642341+0000","last_became_active":"2024-04-16T17:47:55.169235+0000","last_became_peered":"2024-04-16T17:47:55.169235+0000","last_unstale":"2024-04-16T19:36:27.642341+0000","last_undegraded":"2024-04-16T19:36:27.642341+0000","last_fullsized":"2024-04-16T19:36:27.642341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'8","last_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_deep_scrub":"53'8","last_deep_scrub_stamp":"2024-04-16T19:36:27.642045+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.642045+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:07.753490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":90,"num_write":49,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:12.600045+0000","last_change":"2024-04-16T19:37:12.600045+0000","last_active":"2024-04-16T19:37:12.600045+0000","last_peered":"2024-04-16T19:37:12.600045+0000","last_clean":"2024-04-16T19:37:12.600045+0000","last_became_active":"2024-04-16T17:47:23.264258+0000","last_became_peered":"2024-04-16T17:47:23.264258+0000","last_unstale":"2024-04-16T19:37:12.600045+0000","last_undegraded":"2024-04-16T19:37:12.600045+0000","last_fullsized":"2024-04-16T19:37:12.600045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:12.599830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:12.599830+0000","last_clean_scrub_stamp":"2024-04-16T19:37:12.599830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:25:38.604717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.480527+0000","last_change":"2024-04-16T19:37:26.480578+0000","last_active":"2024-04-16T19:37:26.480527+0000","last_peered":"2024-04-16T19:37:26.480527+0000","last_clean":"2024-04-16T19:37:26.480527+0000","last_became_active":"2024-04-16T17:47:21.257472+0000","last_became_peered":"2024-04-16T17:47:21.257472+0000","last_unstale":"2024-04-16T19:37:26.480527+0000","last_undegraded":"2024-04-16T19:37:26.480527+0000","last_fullsized":"2024-04-16T19:37:26.480527+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.480382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.480382+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.480382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:48:44.428175+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"53'16","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.554624+0000","last_change":"2024-04-16T19:37:20.554624+0000","last_active":"2024-04-16T19:37:20.554624+0000","last_peered":"2024-04-16T19:37:20.554624+0000","last_clean":"2024-04-16T19:37:20.554624+0000","last_became_active":"2024-04-16T17:47:33.302267+0000","last_became_peered":"2024-04-16T17:47:33.302267+0000","last_unstale":"2024-04-16T19:37:20.554624+0000","last_undegraded":"2024-04-16T19:37:20.554624+0000","last_fullsized":"2024-04-16T19:37:20.554624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:37:20.554417+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:37:20.554417+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.554417+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:43.223515+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"53'159","reported_seq":358,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:10.333939+0000","last_change":"2024-04-16T19:37:10.334092+0000","last_active":"2024-04-16T19:37:10.333939+0000","last_peered":"2024-04-16T19:37:10.333939+0000","last_clean":"2024-04-16T19:37:10.333939+0000","last_became_active":"2024-04-16T17:47:35.309406+0000","last_became_peered":"2024-04-16T17:47:35.309406+0000","last_unstale":"2024-04-16T19:37:10.333939+0000","last_undegraded":"2024-04-16T19:37:10.333939+0000","last_fullsized":"2024-04-16T19:37:10.333939+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'159","last_scrub_stamp":"2024-04-16T19:37:10.333771+0000","last_deep_scrub":"53'159","last_deep_scrub_stamp":"2024-04-16T19:37:10.333771+0000","last_clean_scrub_stamp":"2024-04-16T19:37:10.333771+0000","objects_scrubbed":7,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:16:33.077948+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":233,"num_read_kb":224,"num_write":152,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.656216+0000","last_change":"2024-04-16T19:36:46.656320+0000","last_active":"2024-04-16T19:36:46.656216+0000","last_peered":"2024-04-16T19:36:46.656216+0000","last_clean":"2024-04-16T19:36:46.656216+0000","last_became_active":"2024-04-16T17:47:37.328044+0000","last_became_peered":"2024-04-16T17:47:37.328044+0000","last_unstale":"2024-04-16T19:36:46.656216+0000","last_undegraded":"2024-04-16T19:36:46.656216+0000","last_fullsized":"2024-04-16T19:36:46.656216+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:46.656048+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.656048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:11.940546+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:37.451910+0000","last_change":"2024-04-16T19:36:37.452021+0000","last_active":"2024-04-16T19:36:37.451910+0000","last_peered":"2024-04-16T19:36:37.451910+0000","last_clean":"2024-04-16T19:36:37.451910+0000","last_became_active":"2024-04-16T17:47:39.336313+0000","last_became_peered":"2024-04-16T17:47:39.336313+0000","last_unstale":"2024-04-16T19:36:37.451910+0000","last_undegraded":"2024-04-16T19:36:37.451910+0000","last_fullsized":"2024-04-16T19:36:37.451910+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:37.451765+0000","last_clean_scrub_stamp":"2024-04-16T19:36:37.451765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:29:52.163889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.540337+0000","last_change":"2024-04-16T19:37:31.540392+0000","last_active":"2024-04-16T19:37:31.540337+0000","last_peered":"2024-04-16T19:37:31.540337+0000","last_clean":"2024-04-16T19:37:31.540337+0000","last_became_active":"2024-04-16T18:10:50.554050+0000","last_became_peered":"2024-04-16T18:10:50.554050+0000","last_unstale":"2024-04-16T19:37:31.540337+0000","last_undegraded":"2024-04-16T19:37:31.540337+0000","last_fullsized":"2024-04-16T19:37:31.540337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:31.540200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:31.540200+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.540200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:54.209298+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"53'134","reported_seq":1476,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.725767+0000","last_change":"2024-04-16T19:36:59.725923+0000","last_active":"2024-04-16T19:36:59.725767+0000","last_peered":"2024-04-16T19:36:59.725767+0000","last_clean":"2024-04-16T19:36:59.725767+0000","last_became_active":"2024-04-16T17:48:34.338318+0000","last_became_peered":"2024-04-16T17:48:34.338318+0000","last_unstale":"2024-04-16T19:36:59.725767+0000","last_undegraded":"2024-04-16T19:36:59.725767+0000","last_fullsized":"2024-04-16T19:36:59.725767+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'134","last_scrub_stamp":"2024-04-16T19:36:59.725594+0000","last_deep_scrub":"53'134","last_deep_scrub_stamp":"2024-04-16T19:36:59.725594+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.725594+0000","objects_scrubbed":35,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:25.010093+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2250,"num_read_kb":2270,"num_write":157,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7807,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.355648+0000","last_change":"2024-04-16T19:37:03.355699+0000","last_active":"2024-04-16T19:37:03.355648+0000","last_peered":"2024-04-16T19:37:03.355648+0000","last_clean":"2024-04-16T19:37:03.355648+0000","last_became_active":"2024-04-16T17:48:27.305965+0000","last_became_peered":"2024-04-16T17:48:27.305965+0000","last_unstale":"2024-04-16T19:37:03.355648+0000","last_undegraded":"2024-04-16T19:37:03.355648+0000","last_fullsized":"2024-04-16T19:37:03.355648+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.355513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.355513+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.355513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:02:13.649973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"53'11","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:55.624271+0000","last_change":"2024-04-16T19:36:55.624271+0000","last_active":"2024-04-16T19:36:55.624271+0000","last_peered":"2024-04-16T19:36:55.624271+0000","last_clean":"2024-04-16T19:36:55.624271+0000","last_became_active":"2024-04-16T17:48:01.248762+0000","last_became_peered":"2024-04-16T17:48:01.248762+0000","last_unstale":"2024-04-16T19:36:55.624271+0000","last_undegraded":"2024-04-16T19:36:55.624271+0000","last_fullsized":"2024-04-16T19:36:55.624271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:55.624051+0000","last_clean_scrub_stamp":"2024-04-16T19:36:55.624051+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:11.910789+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":83,"num_read_kb":62,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"53'556","reported_seq":1911,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.421934+0000","last_change":"2024-04-16T19:36:38.421934+0000","last_active":"2024-04-16T19:36:38.421934+0000","last_peered":"2024-04-16T19:36:38.421934+0000","last_clean":"2024-04-16T19:36:38.421934+0000","last_became_active":"2024-04-16T17:47:59.182712+0000","last_became_peered":"2024-04-16T17:47:59.182712+0000","last_unstale":"2024-04-16T19:36:38.421934+0000","last_undegraded":"2024-04-16T19:36:38.421934+0000","last_fullsized":"2024-04-16T19:36:38.421934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'556","last_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_deep_scrub":"53'556","last_deep_scrub_stamp":"2024-04-16T19:36:38.421654+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.421654+0000","objects_scrubbed":1,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:56:18.533099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'250","reported_seq":489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.484262+0000","last_change":"2024-04-16T19:36:27.484420+0000","last_active":"2024-04-16T19:36:27.484262+0000","last_peered":"2024-04-16T19:36:27.484262+0000","last_clean":"2024-04-16T19:36:27.484262+0000","last_became_active":"2024-04-16T17:47:57.171538+0000","last_became_peered":"2024-04-16T17:47:57.171538+0000","last_unstale":"2024-04-16T19:36:27.484262+0000","last_undegraded":"2024-04-16T19:36:27.484262+0000","last_fullsized":"2024-04-16T19:36:27.484262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2024-04-16T19:36:27.484127+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.484127+0000","objects_scrubbed":20,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:04.899387+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":374,"num_read_kb":368,"num_write":290,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"53'6","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.473636+0000","last_change":"2024-04-16T19:37:32.473718+0000","last_active":"2024-04-16T19:37:32.473636+0000","last_peered":"2024-04-16T19:37:32.473636+0000","last_clean":"2024-04-16T19:37:32.473636+0000","last_became_active":"2024-04-16T17:49:32.390397+0000","last_became_peered":"2024-04-16T17:49:32.390397+0000","last_unstale":"2024-04-16T19:37:32.473636+0000","last_undegraded":"2024-04-16T19:37:32.473636+0000","last_fullsized":"2024-04-16T19:37:32.473636+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:32.473504+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:32.473504+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.473504+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:52.367681+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":91,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":132,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.545483+0000","last_change":"2024-04-16T19:37:28.545539+0000","last_active":"2024-04-16T19:37:28.545483+0000","last_peered":"2024-04-16T19:37:28.545483+0000","last_clean":"2024-04-16T19:37:28.545483+0000","last_became_active":"2024-04-16T17:47:23.246411+0000","last_became_peered":"2024-04-16T17:47:23.246411+0000","last_unstale":"2024-04-16T19:37:28.545483+0000","last_undegraded":"2024-04-16T19:37:28.545483+0000","last_fullsized":"2024-04-16T19:37:28.545483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.545352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.545352+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.545352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:57.170533+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:28.477321+0000","last_change":"2024-04-16T19:37:28.477372+0000","last_active":"2024-04-16T19:37:28.477321+0000","last_peered":"2024-04-16T19:37:28.477321+0000","last_clean":"2024-04-16T19:37:28.477321+0000","last_became_active":"2024-04-16T17:47:21.257509+0000","last_became_peered":"2024-04-16T17:47:21.257509+0000","last_unstale":"2024-04-16T19:37:28.477321+0000","last_undegraded":"2024-04-16T19:37:28.477321+0000","last_fullsized":"2024-04-16T19:37:28.477321+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:28.477187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:28.477187+0000","last_clean_scrub_stamp":"2024-04-16T19:37:28.477187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:49.091407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"53'20","reported_seq":191,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:17.468760+0000","last_change":"2024-04-16T19:37:17.468760+0000","last_active":"2024-04-16T19:37:17.468760+0000","last_peered":"2024-04-16T19:37:17.468760+0000","last_clean":"2024-04-16T19:37:17.468760+0000","last_became_active":"2024-04-16T17:47:33.301476+0000","last_became_peered":"2024-04-16T17:47:33.301476+0000","last_unstale":"2024-04-16T19:37:17.468760+0000","last_undegraded":"2024-04-16T19:37:17.468760+0000","last_fullsized":"2024-04-16T19:37:17.468760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:37:17.468418+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:37:17.468418+0000","last_clean_scrub_stamp":"2024-04-16T19:37:17.468418+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:45.076005+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":69,"num_write":32,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"53'84","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.665782+0000","last_change":"2024-04-16T19:37:14.665856+0000","last_active":"2024-04-16T19:37:14.665782+0000","last_peered":"2024-04-16T19:37:14.665782+0000","last_clean":"2024-04-16T19:37:14.665782+0000","last_became_active":"2024-04-16T17:47:35.302813+0000","last_became_peered":"2024-04-16T17:47:35.302813+0000","last_unstale":"2024-04-16T19:37:14.665782+0000","last_undegraded":"2024-04-16T19:37:14.665782+0000","last_fullsized":"2024-04-16T19:37:14.665782+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:37:14.665644+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:37:14.665644+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.665644+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:29:34.559821+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.446778+0000","last_change":"2024-04-16T19:36:59.446778+0000","last_active":"2024-04-16T19:36:59.446778+0000","last_peered":"2024-04-16T19:36:59.446778+0000","last_clean":"2024-04-16T19:36:59.446778+0000","last_became_active":"2024-04-16T17:47:37.330219+0000","last_became_peered":"2024-04-16T17:47:37.330219+0000","last_unstale":"2024-04-16T19:36:59.446778+0000","last_undegraded":"2024-04-16T19:36:59.446778+0000","last_fullsized":"2024-04-16T19:36:59.446778+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:59.446587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:59.446587+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.446587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:20.442391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.477611+0000","last_change":"2024-04-16T19:36:26.477669+0000","last_active":"2024-04-16T19:36:26.477611+0000","last_peered":"2024-04-16T19:36:26.477611+0000","last_clean":"2024-04-16T19:36:26.477611+0000","last_became_active":"2024-04-16T17:47:39.699955+0000","last_became_peered":"2024-04-16T17:47:39.699955+0000","last_unstale":"2024-04-16T19:36:26.477611+0000","last_undegraded":"2024-04-16T19:36:26.477611+0000","last_fullsized":"2024-04-16T19:36:26.477611+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:26.477464+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.477464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:27.657085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:47.642066+0000","last_change":"2024-04-16T19:37:47.642120+0000","last_active":"2024-04-16T19:37:47.642066+0000","last_peered":"2024-04-16T19:37:47.642066+0000","last_clean":"2024-04-16T19:37:47.642066+0000","last_became_active":"2024-04-16T18:10:50.526870+0000","last_became_peered":"2024-04-16T18:10:50.526870+0000","last_unstale":"2024-04-16T19:37:47.642066+0000","last_undegraded":"2024-04-16T19:37:47.642066+0000","last_fullsized":"2024-04-16T19:37:47.642066+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:47.641925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:47.641925+0000","last_clean_scrub_stamp":"2024-04-16T19:37:47.641925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:52.965538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"53'126","reported_seq":959,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:48.542707+0000","last_change":"2024-04-16T19:36:48.542776+0000","last_active":"2024-04-16T19:36:48.542707+0000","last_peered":"2024-04-16T19:36:48.542707+0000","last_clean":"2024-04-16T19:36:48.542707+0000","last_became_active":"2024-04-16T17:48:34.317722+0000","last_became_peered":"2024-04-16T17:48:34.317722+0000","last_unstale":"2024-04-16T19:36:48.542707+0000","last_undegraded":"2024-04-16T19:36:48.542707+0000","last_fullsized":"2024-04-16T19:36:48.542707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2024-04-16T19:36:48.542572+0000","last_clean_scrub_stamp":"2024-04-16T19:36:48.542572+0000","objects_scrubbed":30,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:02.471385+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1302,"num_read_kb":1271,"num_write":137,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6535,"num_omap_keys":25,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.626702+0000","last_change":"2024-04-16T19:36:43.626754+0000","last_active":"2024-04-16T19:36:43.626702+0000","last_peered":"2024-04-16T19:36:43.626702+0000","last_clean":"2024-04-16T19:36:43.626702+0000","last_became_active":"2024-04-16T17:48:27.307091+0000","last_became_peered":"2024-04-16T17:48:27.307091+0000","last_unstale":"2024-04-16T19:36:43.626702+0000","last_undegraded":"2024-04-16T19:36:43.626702+0000","last_fullsized":"2024-04-16T19:36:43.626702+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:43.626565+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.626565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:30.990969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"53'13","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:57.507267+0000","last_change":"2024-04-16T19:36:57.507267+0000","last_active":"2024-04-16T19:36:57.507267+0000","last_peered":"2024-04-16T19:36:57.507267+0000","last_clean":"2024-04-16T19:36:57.507267+0000","last_became_active":"2024-04-16T17:48:01.244973+0000","last_became_peered":"2024-04-16T17:48:01.244973+0000","last_unstale":"2024-04-16T19:36:57.507267+0000","last_undegraded":"2024-04-16T19:36:57.507267+0000","last_fullsized":"2024-04-16T19:36:57.507267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'13","last_scrub_stamp":"2024-04-16T19:36:57.507046+0000","last_deep_scrub":"53'13","last_deep_scrub_stamp":"2024-04-16T19:36:57.507046+0000","last_clean_scrub_stamp":"2024-04-16T19:36:57.507046+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:56.572731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":35,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:00.388632+0000","last_change":"2024-04-16T19:37:00.388683+0000","last_active":"2024-04-16T19:37:00.388632+0000","last_peered":"2024-04-16T19:37:00.388632+0000","last_clean":"2024-04-16T19:37:00.388632+0000","last_became_active":"2024-04-16T17:47:59.198808+0000","last_became_peered":"2024-04-16T17:47:59.198808+0000","last_unstale":"2024-04-16T19:37:00.388632+0000","last_undegraded":"2024-04-16T19:37:00.388632+0000","last_fullsized":"2024-04-16T19:37:00.388632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:00.388491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:00.388491+0000","last_clean_scrub_stamp":"2024-04-16T19:37:00.388491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:07.573705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"53'141","reported_seq":318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.454241+0000","last_change":"2024-04-16T19:36:28.454241+0000","last_active":"2024-04-16T19:36:28.454241+0000","last_peered":"2024-04-16T19:36:28.454241+0000","last_clean":"2024-04-16T19:36:28.454241+0000","last_became_active":"2024-04-16T17:47:57.188106+0000","last_became_peered":"2024-04-16T17:47:57.188106+0000","last_unstale":"2024-04-16T19:36:28.454241+0000","last_undegraded":"2024-04-16T19:36:28.454241+0000","last_fullsized":"2024-04-16T19:36:28.454241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'141","last_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_deep_scrub":"53'141","last_deep_scrub_stamp":"2024-04-16T19:36:28.453990+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.453990+0000","objects_scrubbed":12,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:45:15.632623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":200,"num_read_kb":197,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.478104+0000","last_change":"2024-04-16T19:36:22.478173+0000","last_active":"2024-04-16T19:36:22.478104+0000","last_peered":"2024-04-16T19:36:22.478104+0000","last_clean":"2024-04-16T19:36:22.478104+0000","last_became_active":"2024-04-16T17:47:55.157950+0000","last_became_peered":"2024-04-16T17:47:55.157950+0000","last_unstale":"2024-04-16T19:36:22.478104+0000","last_undegraded":"2024-04-16T19:36:22.478104+0000","last_fullsized":"2024-04-16T19:36:22.478104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.477939+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.477939+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:46.331273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":39,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:26.645392+0000","last_change":"2024-04-16T19:37:26.645447+0000","last_active":"2024-04-16T19:37:26.645392+0000","last_peered":"2024-04-16T19:37:26.645392+0000","last_clean":"2024-04-16T19:37:26.645392+0000","last_became_active":"2024-04-16T17:47:23.254698+0000","last_became_peered":"2024-04-16T17:47:23.254698+0000","last_unstale":"2024-04-16T19:37:26.645392+0000","last_undegraded":"2024-04-16T19:37:26.645392+0000","last_fullsized":"2024-04-16T19:37:26.645392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:26.645255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:26.645255+0000","last_clean_scrub_stamp":"2024-04-16T19:37:26.645255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:43.359269+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":146,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.433086+0000","last_change":"2024-04-16T19:37:25.433086+0000","last_active":"2024-04-16T19:37:25.433086+0000","last_peered":"2024-04-16T19:37:25.433086+0000","last_clean":"2024-04-16T19:37:25.433086+0000","last_became_active":"2024-04-16T17:47:21.249971+0000","last_became_peered":"2024-04-16T17:47:21.249971+0000","last_unstale":"2024-04-16T19:37:25.433086+0000","last_undegraded":"2024-04-16T19:37:25.433086+0000","last_fullsized":"2024-04-16T19:37:25.433086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.432884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.432884+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.432884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:19:32.618098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"48'3","reported_seq":130,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:16.568435+0000","last_change":"2024-04-16T19:37:16.568539+0000","last_active":"2024-04-16T19:37:16.568435+0000","last_peered":"2024-04-16T19:37:16.568435+0000","last_clean":"2024-04-16T19:37:16.568435+0000","last_became_active":"2024-04-16T17:47:33.296719+0000","last_became_peered":"2024-04-16T17:47:33.296719+0000","last_unstale":"2024-04-16T19:37:16.568435+0000","last_undegraded":"2024-04-16T19:37:16.568435+0000","last_fullsized":"2024-04-16T19:37:16.568435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'3","last_scrub_stamp":"2024-04-16T19:37:16.568259+0000","last_deep_scrub":"48'3","last_deep_scrub_stamp":"2024-04-16T19:37:16.568259+0000","last_clean_scrub_stamp":"2024-04-16T19:37:16.568259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:44:37.564063+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"53'34","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:08.608145+0000","last_change":"2024-04-16T19:37:08.608145+0000","last_active":"2024-04-16T19:37:08.608145+0000","last_peered":"2024-04-16T19:37:08.608145+0000","last_clean":"2024-04-16T19:37:08.608145+0000","last_became_active":"2024-04-16T17:47:35.316307+0000","last_became_peered":"2024-04-16T17:47:35.316307+0000","last_unstale":"2024-04-16T19:37:08.608145+0000","last_undegraded":"2024-04-16T19:37:08.608145+0000","last_fullsized":"2024-04-16T19:37:08.608145+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'34","last_scrub_stamp":"2024-04-16T19:37:08.607831+0000","last_deep_scrub":"53'34","last_deep_scrub_stamp":"2024-04-16T19:37:08.607831+0000","last_clean_scrub_stamp":"2024-04-16T19:37:08.607831+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:02.316023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:03.532350+0000","last_change":"2024-04-16T19:37:03.532350+0000","last_active":"2024-04-16T19:37:03.532350+0000","last_peered":"2024-04-16T19:37:03.532350+0000","last_clean":"2024-04-16T19:37:03.532350+0000","last_became_active":"2024-04-16T17:47:37.319681+0000","last_became_peered":"2024-04-16T17:47:37.319681+0000","last_unstale":"2024-04-16T19:37:03.532350+0000","last_undegraded":"2024-04-16T19:37:03.532350+0000","last_fullsized":"2024-04-16T19:37:03.532350+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:03.532166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:03.532166+0000","last_clean_scrub_stamp":"2024-04-16T19:37:03.532166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:49:26.201264+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":110,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:41.360832+0000","last_change":"2024-04-16T19:36:41.360886+0000","last_active":"2024-04-16T19:36:41.360832+0000","last_peered":"2024-04-16T19:36:41.360832+0000","last_clean":"2024-04-16T19:36:41.360832+0000","last_became_active":"2024-04-16T17:47:39.328911+0000","last_became_peered":"2024-04-16T17:47:39.328911+0000","last_unstale":"2024-04-16T19:36:41.360832+0000","last_undegraded":"2024-04-16T19:36:41.360832+0000","last_fullsized":"2024-04-16T19:36:41.360832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:41.360675+0000","last_clean_scrub_stamp":"2024-04-16T19:36:41.360675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:00.872893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":62,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:38.731601+0000","last_change":"2024-04-16T19:37:38.731654+0000","last_active":"2024-04-16T19:37:38.731601+0000","last_peered":"2024-04-16T19:37:38.731601+0000","last_clean":"2024-04-16T19:37:38.731601+0000","last_became_active":"2024-04-16T18:10:50.548124+0000","last_became_peered":"2024-04-16T18:10:50.548124+0000","last_unstale":"2024-04-16T19:37:38.731601+0000","last_undegraded":"2024-04-16T19:37:38.731601+0000","last_fullsized":"2024-04-16T19:37:38.731601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:38.731423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:38.731423+0000","last_clean_scrub_stamp":"2024-04-16T19:37:38.731423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:55.445437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"53'155","reported_seq":1489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:52.495554+0000","last_change":"2024-04-16T19:36:52.495627+0000","last_active":"2024-04-16T19:36:52.495554+0000","last_peered":"2024-04-16T19:36:52.495554+0000","last_clean":"2024-04-16T19:36:52.495554+0000","last_became_active":"2024-04-16T17:48:34.338105+0000","last_became_peered":"2024-04-16T17:48:34.338105+0000","last_unstale":"2024-04-16T19:36:52.495554+0000","last_undegraded":"2024-04-16T19:36:52.495554+0000","last_fullsized":"2024-04-16T19:36:52.495554+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'155","last_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_deep_scrub":"53'155","last_deep_scrub_stamp":"2024-04-16T19:36:52.495389+0000","last_clean_scrub_stamp":"2024-04-16T19:36:52.495389+0000","objects_scrubbed":29,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:42:43.497334+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2303,"num_read_kb":2303,"num_write":197,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8801,"num_omap_keys":37,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:38.641083+0000","last_change":"2024-04-16T19:36:38.641135+0000","last_active":"2024-04-16T19:36:38.641083+0000","last_peered":"2024-04-16T19:36:38.641083+0000","last_clean":"2024-04-16T19:36:38.641083+0000","last_became_active":"2024-04-16T17:48:27.287416+0000","last_became_peered":"2024-04-16T17:48:27.287416+0000","last_unstale":"2024-04-16T19:36:38.641083+0000","last_undegraded":"2024-04-16T19:36:38.641083+0000","last_fullsized":"2024-04-16T19:36:38.641083+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:38.640946+0000","last_clean_scrub_stamp":"2024-04-16T19:36:38.640946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:53.196611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"53'11","reported_seq":128,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:54.644719+0000","last_change":"2024-04-16T19:36:54.644719+0000","last_active":"2024-04-16T19:36:54.644719+0000","last_peered":"2024-04-16T19:36:54.644719+0000","last_clean":"2024-04-16T19:36:54.644719+0000","last_became_active":"2024-04-16T17:48:01.256685+0000","last_became_peered":"2024-04-16T17:48:01.256685+0000","last_unstale":"2024-04-16T19:36:54.644719+0000","last_undegraded":"2024-04-16T19:36:54.644719+0000","last_fullsized":"2024-04-16T19:36:54.644719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11","last_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_deep_scrub":"53'11","last_deep_scrub_stamp":"2024-04-16T19:36:54.644499+0000","last_clean_scrub_stamp":"2024-04-16T19:36:54.644499+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:20.331700+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":903,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":68,"num_read_kb":55,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.384126+0000","last_change":"2024-04-16T19:36:47.384178+0000","last_active":"2024-04-16T19:36:47.384126+0000","last_peered":"2024-04-16T19:36:47.384126+0000","last_clean":"2024-04-16T19:36:47.384126+0000","last_became_active":"2024-04-16T17:47:59.198645+0000","last_became_peered":"2024-04-16T17:47:59.198645+0000","last_unstale":"2024-04-16T19:36:47.384126+0000","last_undegraded":"2024-04-16T19:36:47.384126+0000","last_fullsized":"2024-04-16T19:36:47.384126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:47.383981+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.383981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:49.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"53'11080","reported_seq":18169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.523528+0000","last_change":"2024-04-16T19:36:22.523528+0000","last_active":"2024-04-16T19:36:22.523528+0000","last_peered":"2024-04-16T19:36:22.523528+0000","last_clean":"2024-04-16T19:36:22.523528+0000","last_became_active":"2024-04-16T17:47:57.183381+0000","last_became_peered":"2024-04-16T17:47:57.183381+0000","last_unstale":"2024-04-16T19:36:22.523528+0000","last_undegraded":"2024-04-16T19:36:22.523528+0000","last_fullsized":"2024-04-16T19:36:22.523528+0000","mapping_epoch":34,"log_start":"53'9300","ondisk_log_start":"53'9300","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'11080","last_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_deep_scrub":"53'11080","last_deep_scrub_stamp":"2024-04-16T19:36:22.523321+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.523321+0000","objects_scrubbed":25,"log_size":1780,"log_dups_size":3000,"ondisk_log_size":1780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:02.010878+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":21377,"num_read_kb":21369,"num_write":7807,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"53'10","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:33.675610+0000","last_change":"2024-04-16T19:36:33.675610+0000","last_active":"2024-04-16T19:36:33.675610+0000","last_peered":"2024-04-16T19:36:33.675610+0000","last_clean":"2024-04-16T19:36:33.675610+0000","last_became_active":"2024-04-16T17:47:55.158735+0000","last_became_peered":"2024-04-16T17:47:55.158735+0000","last_unstale":"2024-04-16T19:36:33.675610+0000","last_undegraded":"2024-04-16T19:36:33.675610+0000","last_fullsized":"2024-04-16T19:36:33.675610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:33.675369+0000","last_clean_scrub_stamp":"2024-04-16T19:36:33.675369+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:57:17.195224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":33,"num_write":58,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:30.692475+0000","last_change":"2024-04-16T19:37:30.692526+0000","last_active":"2024-04-16T19:37:30.692475+0000","last_peered":"2024-04-16T19:37:30.692475+0000","last_clean":"2024-04-16T19:37:30.692475+0000","last_became_active":"2024-04-16T17:47:23.256799+0000","last_became_peered":"2024-04-16T17:47:23.256799+0000","last_unstale":"2024-04-16T19:37:30.692475+0000","last_undegraded":"2024-04-16T19:37:30.692475+0000","last_fullsized":"2024-04-16T19:37:30.692475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:30.692343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:30.692343+0000","last_clean_scrub_stamp":"2024-04-16T19:37:30.692343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:35.469828+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.678985+0000","last_change":"2024-04-16T19:37:32.678985+0000","last_active":"2024-04-16T19:37:32.678985+0000","last_peered":"2024-04-16T19:37:32.678985+0000","last_clean":"2024-04-16T19:37:32.678985+0000","last_became_active":"2024-04-16T17:47:21.243955+0000","last_became_peered":"2024-04-16T17:47:21.243955+0000","last_unstale":"2024-04-16T19:37:32.678985+0000","last_undegraded":"2024-04-16T19:37:32.678985+0000","last_fullsized":"2024-04-16T19:37:32.678985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.678780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.678780+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.678780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:16:02.392190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"53'6","reported_seq":1355,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.657921+0000","last_change":"2024-04-16T19:37:33.657991+0000","last_active":"2024-04-16T19:37:33.657921+0000","last_peered":"2024-04-16T19:37:33.657921+0000","last_clean":"2024-04-16T19:37:33.657921+0000","last_became_active":"2024-04-16T17:47:33.296549+0000","last_became_peered":"2024-04-16T17:47:33.296549+0000","last_unstale":"2024-04-16T19:37:33.657921+0000","last_undegraded":"2024-04-16T19:37:33.657921+0000","last_fullsized":"2024-04-16T19:37:33.657921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:37:33.657758+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:37:33.657758+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.657758+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:38.138252+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"53'84","reported_seq":246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:59.654126+0000","last_change":"2024-04-16T19:36:59.654126+0000","last_active":"2024-04-16T19:36:59.654126+0000","last_peered":"2024-04-16T19:36:59.654126+0000","last_clean":"2024-04-16T19:36:59.654126+0000","last_became_active":"2024-04-16T17:47:35.307435+0000","last_became_peered":"2024-04-16T17:47:35.307435+0000","last_unstale":"2024-04-16T19:36:59.654126+0000","last_undegraded":"2024-04-16T19:36:59.654126+0000","last_fullsized":"2024-04-16T19:36:59.654126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'84","last_scrub_stamp":"2024-04-16T19:36:59.653919+0000","last_deep_scrub":"53'84","last_deep_scrub_stamp":"2024-04-16T19:36:59.653919+0000","last_clean_scrub_stamp":"2024-04-16T19:36:59.653919+0000","objects_scrubbed":4,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:10.641324+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":80,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.508267+0000","last_change":"2024-04-16T19:37:04.508267+0000","last_active":"2024-04-16T19:37:04.508267+0000","last_peered":"2024-04-16T19:37:04.508267+0000","last_clean":"2024-04-16T19:37:04.508267+0000","last_became_active":"2024-04-16T17:47:37.321307+0000","last_became_peered":"2024-04-16T17:47:37.321307+0000","last_unstale":"2024-04-16T19:37:04.508267+0000","last_undegraded":"2024-04-16T19:37:04.508267+0000","last_fullsized":"2024-04-16T19:37:04.508267+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.508075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.508075+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.508075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:14:53.498245+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:49.560147+0000","last_change":"2024-04-16T19:36:49.560147+0000","last_active":"2024-04-16T19:36:49.560147+0000","last_peered":"2024-04-16T19:36:49.560147+0000","last_clean":"2024-04-16T19:36:49.560147+0000","last_became_active":"2024-04-16T17:47:39.326210+0000","last_became_peered":"2024-04-16T17:47:39.326210+0000","last_unstale":"2024-04-16T19:36:49.560147+0000","last_undegraded":"2024-04-16T19:36:49.560147+0000","last_fullsized":"2024-04-16T19:36:49.560147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:36:49.559817+0000","last_clean_scrub_stamp":"2024-04-16T19:36:49.559817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:35:52.415984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2a","version":"53'31","reported_seq":215,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:46.493564+0000","last_change":"2024-04-16T19:36:46.493705+0000","last_active":"2024-04-16T19:36:46.493564+0000","last_peered":"2024-04-16T19:36:46.493564+0000","last_clean":"2024-04-16T19:36:46.493564+0000","last_became_active":"2024-04-16T17:47:55.158363+0000","last_became_peered":"2024-04-16T17:47:55.158363+0000","last_unstale":"2024-04-16T19:36:46.493564+0000","last_undegraded":"2024-04-16T19:36:46.493564+0000","last_fullsized":"2024-04-16T19:36:46.493564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'31","last_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_deep_scrub":"53'31","last_deep_scrub_stamp":"2024-04-16T19:36:46.493417+0000","last_clean_scrub_stamp":"2024-04-16T19:36:46.493417+0000","objects_scrubbed":9,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:14:52.920199+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":213,"num_read_kb":145,"num_write":150,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:02.429456+0000","last_change":"2024-04-16T19:37:02.429510+0000","last_active":"2024-04-16T19:37:02.429456+0000","last_peered":"2024-04-16T19:37:02.429456+0000","last_clean":"2024-04-16T19:37:02.429456+0000","last_became_active":"2024-04-16T17:47:32.277072+0000","last_became_peered":"2024-04-16T17:47:32.277072+0000","last_unstale":"2024-04-16T19:37:02.429456+0000","last_undegraded":"2024-04-16T19:37:02.429456+0000","last_fullsized":"2024-04-16T19:37:02.429456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:02.429304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:02.429304+0000","last_clean_scrub_stamp":"2024-04-16T19:37:02.429304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:31.301173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.528146+0000","last_change":"2024-04-16T19:37:27.528146+0000","last_active":"2024-04-16T19:37:27.528146+0000","last_peered":"2024-04-16T19:37:27.528146+0000","last_clean":"2024-04-16T19:37:27.528146+0000","last_became_active":"2024-04-16T17:47:21.235007+0000","last_became_peered":"2024-04-16T17:47:21.235007+0000","last_unstale":"2024-04-16T19:37:27.528146+0000","last_undegraded":"2024-04-16T19:37:27.528146+0000","last_fullsized":"2024-04-16T19:37:27.528146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.527854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.527854+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.527854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:18:31.924496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"53'6","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.472692+0000","last_change":"2024-04-16T19:36:34.472767+0000","last_active":"2024-04-16T19:36:34.472692+0000","last_peered":"2024-04-16T19:36:34.472692+0000","last_clean":"2024-04-16T19:36:34.472692+0000","last_became_active":"2024-04-16T17:47:55.175121+0000","last_became_peered":"2024-04-16T17:47:55.175121+0000","last_unstale":"2024-04-16T19:36:34.472692+0000","last_undegraded":"2024-04-16T19:36:34.472692+0000","last_fullsized":"2024-04-16T19:36:34.472692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:34.472561+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.472561+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:32:21.836965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":265,"num_read_kb":172,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:14.621063+0000","last_change":"2024-04-16T19:37:14.621063+0000","last_active":"2024-04-16T19:37:14.621063+0000","last_peered":"2024-04-16T19:37:14.621063+0000","last_clean":"2024-04-16T19:37:14.621063+0000","last_became_active":"2024-04-16T17:47:23.259891+0000","last_became_peered":"2024-04-16T17:47:23.259891+0000","last_unstale":"2024-04-16T19:37:14.621063+0000","last_undegraded":"2024-04-16T19:37:14.621063+0000","last_fullsized":"2024-04-16T19:37:14.621063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:14.620868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:14.620868+0000","last_clean_scrub_stamp":"2024-04-16T19:37:14.620868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:09:39.939904+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.482025+0000","last_change":"2024-04-16T19:37:21.482025+0000","last_active":"2024-04-16T19:37:21.482025+0000","last_peered":"2024-04-16T19:37:21.482025+0000","last_clean":"2024-04-16T19:37:21.482025+0000","last_became_active":"2024-04-16T17:47:21.233064+0000","last_became_peered":"2024-04-16T17:47:21.233064+0000","last_unstale":"2024-04-16T19:37:21.482025+0000","last_undegraded":"2024-04-16T19:37:21.482025+0000","last_fullsized":"2024-04-16T19:37:21.482025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.481828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.481828+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.481828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:30:55.823833+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"53'10","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:30.445631+0000","last_change":"2024-04-16T19:36:30.445631+0000","last_active":"2024-04-16T19:36:30.445631+0000","last_peered":"2024-04-16T19:36:30.445631+0000","last_clean":"2024-04-16T19:36:30.445631+0000","last_became_active":"2024-04-16T17:47:55.177920+0000","last_became_peered":"2024-04-16T17:47:55.177920+0000","last_unstale":"2024-04-16T19:36:30.445631+0000","last_undegraded":"2024-04-16T19:36:30.445631+0000","last_fullsized":"2024-04-16T19:36:30.445631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'10","last_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_deep_scrub":"53'10","last_deep_scrub_stamp":"2024-04-16T19:36:30.445415+0000","last_clean_scrub_stamp":"2024-04-16T19:36:30.445415+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:53:00.551163+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":51,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.476876+0000","last_change":"2024-04-16T19:37:27.476876+0000","last_active":"2024-04-16T19:37:27.476876+0000","last_peered":"2024-04-16T19:37:27.476876+0000","last_clean":"2024-04-16T19:37:27.476876+0000","last_became_active":"2024-04-16T17:47:23.264583+0000","last_became_peered":"2024-04-16T17:47:23.264583+0000","last_unstale":"2024-04-16T19:37:27.476876+0000","last_undegraded":"2024-04-16T19:37:27.476876+0000","last_fullsized":"2024-04-16T19:37:27.476876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.476688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.476688+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.476688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:11.477879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":145,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:38.596378+0000","last_change":"2024-04-16T19:37:38.596378+0000","last_active":"2024-04-16T19:37:38.596378+0000","last_peered":"2024-04-16T19:37:38.596378+0000","last_clean":"2024-04-16T19:37:38.596378+0000","last_became_active":"2024-04-16T17:47:21.238770+0000","last_became_peered":"2024-04-16T17:47:21.238770+0000","last_unstale":"2024-04-16T19:37:38.596378+0000","last_undegraded":"2024-04-16T19:37:38.596378+0000","last_fullsized":"2024-04-16T19:37:38.596378+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:38.596190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:38.596190+0000","last_clean_scrub_stamp":"2024-04-16T19:37:38.596190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:21.138313+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"53'12","reported_seq":248,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:34.464290+0000","last_change":"2024-04-16T19:36:34.464387+0000","last_active":"2024-04-16T19:36:34.464290+0000","last_peered":"2024-04-16T19:36:34.464290+0000","last_clean":"2024-04-16T19:36:34.464290+0000","last_became_active":"2024-04-16T17:47:55.154649+0000","last_became_peered":"2024-04-16T17:47:55.154649+0000","last_unstale":"2024-04-16T19:36:34.464290+0000","last_undegraded":"2024-04-16T19:36:34.464290+0000","last_fullsized":"2024-04-16T19:36:34.464290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:34.464153+0000","last_clean_scrub_stamp":"2024-04-16T19:36:34.464153+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:40:03.336022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":198,"num_write":70,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.621645+0000","last_change":"2024-04-16T19:37:09.621645+0000","last_active":"2024-04-16T19:37:09.621645+0000","last_peered":"2024-04-16T19:37:09.621645+0000","last_clean":"2024-04-16T19:37:09.621645+0000","last_became_active":"2024-04-16T17:47:23.261242+0000","last_became_peered":"2024-04-16T17:47:23.261242+0000","last_unstale":"2024-04-16T19:37:09.621645+0000","last_undegraded":"2024-04-16T19:37:09.621645+0000","last_fullsized":"2024-04-16T19:37:09.621645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.621459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.621459+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.621459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:24:41.966705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:39.630884+0000","last_change":"2024-04-16T19:37:39.630993+0000","last_active":"2024-04-16T19:37:39.630884+0000","last_peered":"2024-04-16T19:37:39.630884+0000","last_clean":"2024-04-16T19:37:39.630884+0000","last_became_active":"2024-04-16T17:47:21.240641+0000","last_became_peered":"2024-04-16T17:47:21.240641+0000","last_unstale":"2024-04-16T19:37:39.630884+0000","last_undegraded":"2024-04-16T19:37:39.630884+0000","last_fullsized":"2024-04-16T19:37:39.630884+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:39.630706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:39.630706+0000","last_clean_scrub_stamp":"2024-04-16T19:37:39.630706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:35:52.950518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"53'6","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.528121+0000","last_change":"2024-04-16T19:36:28.528121+0000","last_active":"2024-04-16T19:36:28.528121+0000","last_peered":"2024-04-16T19:36:28.528121+0000","last_clean":"2024-04-16T19:36:28.528121+0000","last_became_active":"2024-04-16T17:47:55.180153+0000","last_became_peered":"2024-04-16T17:47:55.180153+0000","last_unstale":"2024-04-16T19:36:28.528121+0000","last_undegraded":"2024-04-16T19:36:28.528121+0000","last_fullsized":"2024-04-16T19:36:28.528121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:28.527896+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.527896+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:25.132224+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":115,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:15.475642+0000","last_change":"2024-04-16T19:37:15.475697+0000","last_active":"2024-04-16T19:37:15.475642+0000","last_peered":"2024-04-16T19:37:15.475642+0000","last_clean":"2024-04-16T19:37:15.475642+0000","last_became_active":"2024-04-16T17:47:23.251684+0000","last_became_peered":"2024-04-16T17:47:23.251684+0000","last_unstale":"2024-04-16T19:37:15.475642+0000","last_undegraded":"2024-04-16T19:37:15.475642+0000","last_fullsized":"2024-04-16T19:37:15.475642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:15.475502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:15.475502+0000","last_clean_scrub_stamp":"2024-04-16T19:37:15.475502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:13:53.312895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:40.616477+0000","last_change":"2024-04-16T19:37:40.616477+0000","last_active":"2024-04-16T19:37:40.616477+0000","last_peered":"2024-04-16T19:37:40.616477+0000","last_clean":"2024-04-16T19:37:40.616477+0000","last_became_active":"2024-04-16T17:47:21.240621+0000","last_became_peered":"2024-04-16T17:47:21.240621+0000","last_unstale":"2024-04-16T19:37:40.616477+0000","last_undegraded":"2024-04-16T19:37:40.616477+0000","last_fullsized":"2024-04-16T19:37:40.616477+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:40.616282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:40.616282+0000","last_clean_scrub_stamp":"2024-04-16T19:37:40.616282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:07.730821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"53'2","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:26.582466+0000","last_change":"2024-04-16T19:36:26.582466+0000","last_active":"2024-04-16T19:36:26.582466+0000","last_peered":"2024-04-16T19:36:26.582466+0000","last_clean":"2024-04-16T19:36:26.582466+0000","last_became_active":"2024-04-16T17:47:55.164859+0000","last_became_peered":"2024-04-16T17:47:55.164859+0000","last_unstale":"2024-04-16T19:36:26.582466+0000","last_undegraded":"2024-04-16T19:36:26.582466+0000","last_fullsized":"2024-04-16T19:36:26.582466+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:26.582238+0000","last_clean_scrub_stamp":"2024-04-16T19:36:26.582238+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:10.024512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:13.583188+0000","last_change":"2024-04-16T19:37:13.583188+0000","last_active":"2024-04-16T19:37:13.583188+0000","last_peered":"2024-04-16T19:37:13.583188+0000","last_clean":"2024-04-16T19:37:13.583188+0000","last_became_active":"2024-04-16T17:47:23.256430+0000","last_became_peered":"2024-04-16T17:47:23.256430+0000","last_unstale":"2024-04-16T19:37:13.583188+0000","last_undegraded":"2024-04-16T19:37:13.583188+0000","last_fullsized":"2024-04-16T19:37:13.583188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:13.583001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:13.583001+0000","last_clean_scrub_stamp":"2024-04-16T19:37:13.583001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:31:29.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:27.630014+0000","last_change":"2024-04-16T19:37:27.630014+0000","last_active":"2024-04-16T19:37:27.630014+0000","last_peered":"2024-04-16T19:37:27.630014+0000","last_clean":"2024-04-16T19:37:27.630014+0000","last_became_active":"2024-04-16T17:47:21.228704+0000","last_became_peered":"2024-04-16T17:47:21.228704+0000","last_unstale":"2024-04-16T19:37:27.630014+0000","last_undegraded":"2024-04-16T19:37:27.630014+0000","last_fullsized":"2024-04-16T19:37:27.630014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:27.629814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:27.629814+0000","last_clean_scrub_stamp":"2024-04-16T19:37:27.629814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:55:33.873646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"53'46","reported_seq":266,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.466241+0000","last_change":"2024-04-16T19:36:19.466346+0000","last_active":"2024-04-16T19:36:19.466241+0000","last_peered":"2024-04-16T19:36:19.466241+0000","last_clean":"2024-04-16T19:36:19.466241+0000","last_became_active":"2024-04-16T17:47:55.160731+0000","last_became_peered":"2024-04-16T17:47:55.160731+0000","last_unstale":"2024-04-16T19:36:19.466241+0000","last_undegraded":"2024-04-16T19:36:19.466241+0000","last_fullsized":"2024-04-16T19:36:19.466241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'46","last_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_deep_scrub":"53'46","last_deep_scrub_stamp":"2024-04-16T19:36:19.466115+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.466115+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:09:05.816236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":338,"num_read_kb":220,"num_write":141,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":131,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:04.436774+0000","last_change":"2024-04-16T19:37:04.436877+0000","last_active":"2024-04-16T19:37:04.436774+0000","last_peered":"2024-04-16T19:37:04.436774+0000","last_clean":"2024-04-16T19:37:04.436774+0000","last_became_active":"2024-04-16T17:47:32.279737+0000","last_became_peered":"2024-04-16T17:47:32.279737+0000","last_unstale":"2024-04-16T19:37:04.436774+0000","last_undegraded":"2024-04-16T19:37:04.436774+0000","last_fullsized":"2024-04-16T19:37:04.436774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:04.436610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:04.436610+0000","last_clean_scrub_stamp":"2024-04-16T19:37:04.436610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:24.645977+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":143,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:29.713003+0000","last_change":"2024-04-16T19:37:29.713003+0000","last_active":"2024-04-16T19:37:29.713003+0000","last_peered":"2024-04-16T19:37:29.713003+0000","last_clean":"2024-04-16T19:37:29.713003+0000","last_became_active":"2024-04-16T17:47:21.252522+0000","last_became_peered":"2024-04-16T17:47:21.252522+0000","last_unstale":"2024-04-16T19:37:29.713003+0000","last_undegraded":"2024-04-16T19:37:29.713003+0000","last_fullsized":"2024-04-16T19:37:29.713003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:29.712731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:29.712731+0000","last_clean_scrub_stamp":"2024-04-16T19:37:29.712731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:01:19.156040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2d","version":"53'18","reported_seq":236,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:31.512117+0000","last_change":"2024-04-16T19:37:31.512195+0000","last_active":"2024-04-16T19:37:31.512117+0000","last_peered":"2024-04-16T19:37:31.512117+0000","last_clean":"2024-04-16T19:37:31.512117+0000","last_became_active":"2024-04-16T17:49:32.396056+0000","last_became_peered":"2024-04-16T17:49:32.396056+0000","last_unstale":"2024-04-16T19:37:31.512117+0000","last_undegraded":"2024-04-16T19:37:31.512117+0000","last_fullsized":"2024-04-16T19:37:31.512117+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2024-04-16T19:37:31.512041+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2024-04-16T19:37:31.512041+0000","last_clean_scrub_stamp":"2024-04-16T19:37:31.512041+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:12.436858+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":361,"num_read_kb":220,"num_write":102,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.661934+0000","last_change":"2024-04-16T19:37:19.661934+0000","last_active":"2024-04-16T19:37:19.661934+0000","last_peered":"2024-04-16T19:37:19.661934+0000","last_clean":"2024-04-16T19:37:19.661934+0000","last_became_active":"2024-04-16T17:47:23.249757+0000","last_became_peered":"2024-04-16T17:47:23.249757+0000","last_unstale":"2024-04-16T19:37:19.661934+0000","last_undegraded":"2024-04-16T19:37:19.661934+0000","last_fullsized":"2024-04-16T19:37:19.661934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.661744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.661744+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.661744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:44.799680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":142,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.505347+0000","last_change":"2024-04-16T19:37:19.505347+0000","last_active":"2024-04-16T19:37:19.505347+0000","last_peered":"2024-04-16T19:37:19.505347+0000","last_clean":"2024-04-16T19:37:19.505347+0000","last_became_active":"2024-04-16T17:47:21.227939+0000","last_became_peered":"2024-04-16T17:47:21.227939+0000","last_unstale":"2024-04-16T19:37:19.505347+0000","last_undegraded":"2024-04-16T19:37:19.505347+0000","last_fullsized":"2024-04-16T19:37:19.505347+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.505147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.505147+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.505147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:53.058749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"53'21","reported_seq":207,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.453307+0000","last_change":"2024-04-16T19:36:21.453405+0000","last_active":"2024-04-16T19:36:21.453307+0000","last_peered":"2024-04-16T19:36:21.453307+0000","last_clean":"2024-04-16T19:36:21.453307+0000","last_became_active":"2024-04-16T17:47:55.169591+0000","last_became_peered":"2024-04-16T17:47:55.169591+0000","last_unstale":"2024-04-16T19:36:21.453307+0000","last_undegraded":"2024-04-16T19:36:21.453307+0000","last_fullsized":"2024-04-16T19:36:21.453307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'21","last_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_deep_scrub":"53'21","last_deep_scrub_stamp":"2024-04-16T19:36:21.453170+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.453170+0000","objects_scrubbed":6,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:26.892395+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":146,"num_write":110,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:22.685941+0000","last_change":"2024-04-16T19:37:22.686051+0000","last_active":"2024-04-16T19:37:22.685941+0000","last_peered":"2024-04-16T19:37:22.685941+0000","last_clean":"2024-04-16T19:37:22.685941+0000","last_became_active":"2024-04-16T17:47:23.256253+0000","last_became_peered":"2024-04-16T17:47:23.256253+0000","last_unstale":"2024-04-16T19:37:22.685941+0000","last_undegraded":"2024-04-16T19:37:22.685941+0000","last_fullsized":"2024-04-16T19:37:22.685941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:22.685775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:22.685775+0000","last_clean_scrub_stamp":"2024-04-16T19:37:22.685775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:48.606236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:45.554819+0000","last_change":"2024-04-16T19:37:45.554872+0000","last_active":"2024-04-16T19:37:45.554819+0000","last_peered":"2024-04-16T19:37:45.554819+0000","last_clean":"2024-04-16T19:37:45.554819+0000","last_became_active":"2024-04-16T17:47:32.278755+0000","last_became_peered":"2024-04-16T17:47:32.278755+0000","last_unstale":"2024-04-16T19:37:45.554819+0000","last_undegraded":"2024-04-16T19:37:45.554819+0000","last_fullsized":"2024-04-16T19:37:45.554819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:45.554669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:45.554669+0000","last_clean_scrub_stamp":"2024-04-16T19:37:45.554669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:41:35.320860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"53'26","reported_seq":334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:21.543887+0000","last_change":"2024-04-16T19:36:21.543887+0000","last_active":"2024-04-16T19:36:21.543887+0000","last_peered":"2024-04-16T19:36:21.543887+0000","last_clean":"2024-04-16T19:36:21.543887+0000","last_became_active":"2024-04-16T17:47:55.177001+0000","last_became_peered":"2024-04-16T17:47:55.177001+0000","last_unstale":"2024-04-16T19:36:21.543887+0000","last_undegraded":"2024-04-16T19:36:21.543887+0000","last_fullsized":"2024-04-16T19:36:21.543887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'26","last_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_deep_scrub":"53'26","last_deep_scrub_stamp":"2024-04-16T19:36:21.543655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:21.543655+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:47.643023+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":504,"num_read_kb":304,"num_write":131,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":135,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.658886+0000","last_change":"2024-04-16T19:37:09.658979+0000","last_active":"2024-04-16T19:37:09.658886+0000","last_peered":"2024-04-16T19:37:09.658886+0000","last_clean":"2024-04-16T19:37:09.658886+0000","last_became_active":"2024-04-16T17:47:23.258949+0000","last_became_peered":"2024-04-16T17:47:23.258949+0000","last_unstale":"2024-04-16T19:37:09.658886+0000","last_undegraded":"2024-04-16T19:37:09.658886+0000","last_fullsized":"2024-04-16T19:37:09.658886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.658720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.658720+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.658720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:58:59.721551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:41.597854+0000","last_change":"2024-04-16T19:37:41.597907+0000","last_active":"2024-04-16T19:37:41.597854+0000","last_peered":"2024-04-16T19:37:41.597854+0000","last_clean":"2024-04-16T19:37:41.597854+0000","last_became_active":"2024-04-16T17:47:21.240741+0000","last_became_peered":"2024-04-16T17:47:21.240741+0000","last_unstale":"2024-04-16T19:37:41.597854+0000","last_undegraded":"2024-04-16T19:37:41.597854+0000","last_fullsized":"2024-04-16T19:37:41.597854+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:41.597713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:41.597713+0000","last_clean_scrub_stamp":"2024-04-16T19:37:41.597713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:11:58.155561+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"53'2","reported_seq":119,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.488252+0000","last_change":"2024-04-16T19:36:22.488332+0000","last_active":"2024-04-16T19:36:22.488252+0000","last_peered":"2024-04-16T19:36:22.488252+0000","last_clean":"2024-04-16T19:36:22.488252+0000","last_became_active":"2024-04-16T17:47:55.167827+0000","last_became_peered":"2024-04-16T17:47:55.167827+0000","last_unstale":"2024-04-16T19:36:22.488252+0000","last_undegraded":"2024-04-16T19:36:22.488252+0000","last_fullsized":"2024-04-16T19:36:22.488252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'2","last_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_deep_scrub":"53'2","last_deep_scrub_stamp":"2024-04-16T19:36:22.488109+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.488109+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:37.620055+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":134,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:09.356894+0000","last_change":"2024-04-16T19:37:09.356950+0000","last_active":"2024-04-16T19:37:09.356894+0000","last_peered":"2024-04-16T19:37:09.356894+0000","last_clean":"2024-04-16T19:37:09.356894+0000","last_became_active":"2024-04-16T17:47:23.264169+0000","last_became_peered":"2024-04-16T17:47:23.264169+0000","last_unstale":"2024-04-16T19:37:09.356894+0000","last_undegraded":"2024-04-16T19:37:09.356894+0000","last_fullsized":"2024-04-16T19:37:09.356894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:09.356747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:09.356747+0000","last_clean_scrub_stamp":"2024-04-16T19:37:09.356747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:28.868964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:36.437233+0000","last_change":"2024-04-16T19:37:36.437286+0000","last_active":"2024-04-16T19:37:36.437233+0000","last_peered":"2024-04-16T19:37:36.437233+0000","last_clean":"2024-04-16T19:37:36.437233+0000","last_became_active":"2024-04-16T17:47:21.252701+0000","last_became_peered":"2024-04-16T17:47:21.252701+0000","last_unstale":"2024-04-16T19:37:36.437233+0000","last_undegraded":"2024-04-16T19:37:36.437233+0000","last_fullsized":"2024-04-16T19:37:36.437233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:36.437085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:36.437085+0000","last_clean_scrub_stamp":"2024-04-16T19:37:36.437085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:50:07.501878+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"53'22","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:47.501837+0000","last_change":"2024-04-16T19:36:47.501923+0000","last_active":"2024-04-16T19:36:47.501837+0000","last_peered":"2024-04-16T19:36:47.501837+0000","last_clean":"2024-04-16T19:36:47.501837+0000","last_became_active":"2024-04-16T17:47:55.147267+0000","last_became_peered":"2024-04-16T17:47:55.147267+0000","last_unstale":"2024-04-16T19:36:47.501837+0000","last_undegraded":"2024-04-16T19:36:47.501837+0000","last_fullsized":"2024-04-16T19:36:47.501837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'22","last_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_deep_scrub":"53'22","last_deep_scrub_stamp":"2024-04-16T19:36:47.501703+0000","last_clean_scrub_stamp":"2024-04-16T19:36:47.501703+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:52.574907+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":330,"num_read_kb":221,"num_write":117,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.674118+0000","last_change":"2024-04-16T19:37:20.674118+0000","last_active":"2024-04-16T19:37:20.674118+0000","last_peered":"2024-04-16T19:37:20.674118+0000","last_clean":"2024-04-16T19:37:20.674118+0000","last_became_active":"2024-04-16T17:47:23.253861+0000","last_became_peered":"2024-04-16T17:47:23.253861+0000","last_unstale":"2024-04-16T19:37:20.674118+0000","last_undegraded":"2024-04-16T19:37:20.674118+0000","last_fullsized":"2024-04-16T19:37:20.674118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.673928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.673928+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.673928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:08.556042+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:34.634312+0000","last_change":"2024-04-16T19:37:34.634312+0000","last_active":"2024-04-16T19:37:34.634312+0000","last_peered":"2024-04-16T19:37:34.634312+0000","last_clean":"2024-04-16T19:37:34.634312+0000","last_became_active":"2024-04-16T17:47:21.244374+0000","last_became_peered":"2024-04-16T17:47:21.244374+0000","last_unstale":"2024-04-16T19:37:34.634312+0000","last_undegraded":"2024-04-16T19:37:34.634312+0000","last_fullsized":"2024-04-16T19:37:34.634312+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:34.634115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:34.634115+0000","last_clean_scrub_stamp":"2024-04-16T19:37:34.634115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:55:13.475061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"53'6","reported_seq":152,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:22.568771+0000","last_change":"2024-04-16T19:36:22.568861+0000","last_active":"2024-04-16T19:36:22.568771+0000","last_peered":"2024-04-16T19:36:22.568771+0000","last_clean":"2024-04-16T19:36:22.568771+0000","last_became_active":"2024-04-16T17:47:55.182564+0000","last_became_peered":"2024-04-16T17:47:55.182564+0000","last_unstale":"2024-04-16T19:36:22.568771+0000","last_undegraded":"2024-04-16T19:36:22.568771+0000","last_fullsized":"2024-04-16T19:36:22.568771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:22.568637+0000","last_clean_scrub_stamp":"2024-04-16T19:36:22.568637+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:27:27.641841+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":62,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:07.661244+0000","last_change":"2024-04-16T19:37:07.661244+0000","last_active":"2024-04-16T19:37:07.661244+0000","last_peered":"2024-04-16T19:37:07.661244+0000","last_clean":"2024-04-16T19:37:07.661244+0000","last_became_active":"2024-04-16T17:47:23.265195+0000","last_became_peered":"2024-04-16T17:47:23.265195+0000","last_unstale":"2024-04-16T19:37:07.661244+0000","last_undegraded":"2024-04-16T19:37:07.661244+0000","last_fullsized":"2024-04-16T19:37:07.661244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:07.660970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:07.660970+0000","last_clean_scrub_stamp":"2024-04-16T19:37:07.660970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:33.237577+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:21.721669+0000","last_change":"2024-04-16T19:37:21.721669+0000","last_active":"2024-04-16T19:37:21.721669+0000","last_peered":"2024-04-16T19:37:21.721669+0000","last_clean":"2024-04-16T19:37:21.721669+0000","last_became_active":"2024-04-16T17:47:21.257033+0000","last_became_peered":"2024-04-16T17:47:21.257033+0000","last_unstale":"2024-04-16T19:37:21.721669+0000","last_undegraded":"2024-04-16T19:37:21.721669+0000","last_fullsized":"2024-04-16T19:37:21.721669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:21.721479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:21.721479+0000","last_clean_scrub_stamp":"2024-04-16T19:37:21.721479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:25:45.961100+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"53'4","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:43.473816+0000","last_change":"2024-04-16T19:36:43.473937+0000","last_active":"2024-04-16T19:36:43.473816+0000","last_peered":"2024-04-16T19:36:43.473816+0000","last_clean":"2024-04-16T19:36:43.473816+0000","last_became_active":"2024-04-16T17:47:55.162042+0000","last_became_peered":"2024-04-16T17:47:55.162042+0000","last_unstale":"2024-04-16T19:36:43.473816+0000","last_undegraded":"2024-04-16T19:36:43.473816+0000","last_fullsized":"2024-04-16T19:36:43.473816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'4","last_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_deep_scrub":"53'4","last_deep_scrub_stamp":"2024-04-16T19:36:43.473655+0000","last_clean_scrub_stamp":"2024-04-16T19:36:43.473655+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:41:59.715799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":16,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:23.652821+0000","last_change":"2024-04-16T19:37:23.652821+0000","last_active":"2024-04-16T19:37:23.652821+0000","last_peered":"2024-04-16T19:37:23.652821+0000","last_clean":"2024-04-16T19:37:23.652821+0000","last_became_active":"2024-04-16T17:47:23.257816+0000","last_became_peered":"2024-04-16T17:47:23.257816+0000","last_unstale":"2024-04-16T19:37:23.652821+0000","last_undegraded":"2024-04-16T19:37:23.652821+0000","last_fullsized":"2024-04-16T19:37:23.652821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:23.652616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:23.652616+0000","last_clean_scrub_stamp":"2024-04-16T19:37:23.652616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:59:07.653807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:33.710257+0000","last_change":"2024-04-16T19:37:33.710309+0000","last_active":"2024-04-16T19:37:33.710257+0000","last_peered":"2024-04-16T19:37:33.710257+0000","last_clean":"2024-04-16T19:37:33.710257+0000","last_became_active":"2024-04-16T17:47:21.250163+0000","last_became_peered":"2024-04-16T17:47:21.250163+0000","last_unstale":"2024-04-16T19:37:33.710257+0000","last_undegraded":"2024-04-16T19:37:33.710257+0000","last_fullsized":"2024-04-16T19:37:33.710257+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:33.710122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:33.710122+0000","last_clean_scrub_stamp":"2024-04-16T19:37:33.710122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:16:03.423532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.24","version":"53'6","reported_seq":182,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.607311+0000","last_change":"2024-04-16T19:36:19.607390+0000","last_active":"2024-04-16T19:36:19.607311+0000","last_peered":"2024-04-16T19:36:19.607311+0000","last_clean":"2024-04-16T19:36:19.607311+0000","last_became_active":"2024-04-16T17:47:55.176795+0000","last_became_peered":"2024-04-16T17:47:55.176795+0000","last_unstale":"2024-04-16T19:36:19.607311+0000","last_undegraded":"2024-04-16T19:36:19.607311+0000","last_fullsized":"2024-04-16T19:36:19.607311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'6","last_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_deep_scrub":"53'6","last_deep_scrub_stamp":"2024-04-16T19:36:19.607181+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.607181+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:07:42.514098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":102,"num_write":35,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:20.546364+0000","last_change":"2024-04-16T19:37:20.546364+0000","last_active":"2024-04-16T19:37:20.546364+0000","last_peered":"2024-04-16T19:37:20.546364+0000","last_clean":"2024-04-16T19:37:20.546364+0000","last_became_active":"2024-04-16T17:47:23.254387+0000","last_became_peered":"2024-04-16T17:47:23.254387+0000","last_unstale":"2024-04-16T19:37:20.546364+0000","last_undegraded":"2024-04-16T19:37:20.546364+0000","last_fullsized":"2024-04-16T19:37:20.546364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:20.546236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:20.546236+0000","last_clean_scrub_stamp":"2024-04-16T19:37:20.546236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:24:08.656140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.479097+0000","last_change":"2024-04-16T19:37:18.479150+0000","last_active":"2024-04-16T19:37:18.479097+0000","last_peered":"2024-04-16T19:37:18.479097+0000","last_clean":"2024-04-16T19:37:18.479097+0000","last_became_active":"2024-04-16T17:47:21.225662+0000","last_became_peered":"2024-04-16T17:47:21.225662+0000","last_unstale":"2024-04-16T19:37:18.479097+0000","last_undegraded":"2024-04-16T19:37:18.479097+0000","last_fullsized":"2024-04-16T19:37:18.479097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.478956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.478956+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.478956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:40:44.055564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.25","version":"53'12","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:19.498647+0000","last_change":"2024-04-16T19:36:19.498750+0000","last_active":"2024-04-16T19:36:19.498647+0000","last_peered":"2024-04-16T19:36:19.498647+0000","last_clean":"2024-04-16T19:36:19.498647+0000","last_became_active":"2024-04-16T17:47:55.177535+0000","last_became_peered":"2024-04-16T17:47:55.177535+0000","last_unstale":"2024-04-16T19:36:19.498647+0000","last_undegraded":"2024-04-16T19:36:19.498647+0000","last_fullsized":"2024-04-16T19:36:19.498647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'12","last_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_deep_scrub":"53'12","last_deep_scrub_stamp":"2024-04-16T19:36:19.498508+0000","last_clean_scrub_stamp":"2024-04-16T19:36:19.498508+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:33.427321+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":194,"num_write":68,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":138,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:18.620126+0000","last_change":"2024-04-16T19:37:18.620180+0000","last_active":"2024-04-16T19:37:18.620126+0000","last_peered":"2024-04-16T19:37:18.620126+0000","last_clean":"2024-04-16T19:37:18.620126+0000","last_became_active":"2024-04-16T17:47:23.250736+0000","last_became_peered":"2024-04-16T17:47:23.250736+0000","last_unstale":"2024-04-16T19:37:18.620126+0000","last_undegraded":"2024-04-16T19:37:18.620126+0000","last_fullsized":"2024-04-16T19:37:18.620126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:18.619988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:18.619988+0000","last_clean_scrub_stamp":"2024-04-16T19:37:18.619988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:00:43.755618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:24.541500+0000","last_change":"2024-04-16T19:37:24.541551+0000","last_active":"2024-04-16T19:37:24.541500+0000","last_peered":"2024-04-16T19:37:24.541500+0000","last_clean":"2024-04-16T19:37:24.541500+0000","last_became_active":"2024-04-16T17:47:21.257904+0000","last_became_peered":"2024-04-16T17:47:21.257904+0000","last_unstale":"2024-04-16T19:37:24.541500+0000","last_undegraded":"2024-04-16T19:37:24.541500+0000","last_fullsized":"2024-04-16T19:37:24.541500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:24.541369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:24.541369+0000","last_clean_scrub_stamp":"2024-04-16T19:37:24.541369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:12.423483+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3a","version":"53'16","reported_seq":213,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:28.449751+0000","last_change":"2024-04-16T19:36:28.449751+0000","last_active":"2024-04-16T19:36:28.449751+0000","last_peered":"2024-04-16T19:36:28.449751+0000","last_clean":"2024-04-16T19:36:28.449751+0000","last_became_active":"2024-04-16T17:47:55.155930+0000","last_became_peered":"2024-04-16T17:47:55.155930+0000","last_unstale":"2024-04-16T19:36:28.449751+0000","last_undegraded":"2024-04-16T19:36:28.449751+0000","last_fullsized":"2024-04-16T19:36:28.449751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2024-04-16T19:36:28.449522+0000","last_clean_scrub_stamp":"2024-04-16T19:36:28.449522+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:00:19.924709+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":153,"num_write":90,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:40.604032+0000","last_change":"2024-04-16T19:37:40.604085+0000","last_active":"2024-04-16T19:37:40.604032+0000","last_peered":"2024-04-16T19:37:40.604032+0000","last_clean":"2024-04-16T19:37:40.604032+0000","last_became_active":"2024-04-16T17:47:21.245605+0000","last_became_peered":"2024-04-16T17:47:21.245605+0000","last_unstale":"2024-04-16T19:37:40.604032+0000","last_undegraded":"2024-04-16T19:37:40.604032+0000","last_fullsized":"2024-04-16T19:37:40.604032+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:40.603872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:40.603872+0000","last_clean_scrub_stamp":"2024-04-16T19:37:40.603872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:16:38.169280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:19.599119+0000","last_change":"2024-04-16T19:37:19.599170+0000","last_active":"2024-04-16T19:37:19.599119+0000","last_peered":"2024-04-16T19:37:19.599119+0000","last_clean":"2024-04-16T19:37:19.599119+0000","last_became_active":"2024-04-16T17:47:23.258435+0000","last_became_peered":"2024-04-16T17:47:23.258435+0000","last_unstale":"2024-04-16T19:37:19.599119+0000","last_undegraded":"2024-04-16T19:37:19.599119+0000","last_fullsized":"2024-04-16T19:37:19.599119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:19.598990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:19.598990+0000","last_clean_scrub_stamp":"2024-04-16T19:37:19.598990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:07.481104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"53'20","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:36:27.594512+0000","last_change":"2024-04-16T19:36:27.594512+0000","last_active":"2024-04-16T19:36:27.594512+0000","last_peered":"2024-04-16T19:36:27.594512+0000","last_clean":"2024-04-16T19:36:27.594512+0000","last_became_active":"2024-04-16T17:47:55.164679+0000","last_became_peered":"2024-04-16T17:47:55.164679+0000","last_unstale":"2024-04-16T19:36:27.594512+0000","last_undegraded":"2024-04-16T19:36:27.594512+0000","last_fullsized":"2024-04-16T19:36:27.594512+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'20","last_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_deep_scrub":"53'20","last_deep_scrub_stamp":"2024-04-16T19:36:27.594261+0000","last_clean_scrub_stamp":"2024-04-16T19:36:27.594261+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:42:01.851085+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":125,"num_write":105,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:32.683833+0000","last_change":"2024-04-16T19:37:32.683888+0000","last_active":"2024-04-16T19:37:32.683833+0000","last_peered":"2024-04-16T19:37:32.683833+0000","last_clean":"2024-04-16T19:37:32.683833+0000","last_became_active":"2024-04-16T17:47:21.239174+0000","last_became_peered":"2024-04-16T17:47:21.239174+0000","last_unstale":"2024-04-16T19:37:32.683833+0000","last_undegraded":"2024-04-16T19:37:32.683833+0000","last_fullsized":"2024-04-16T19:37:32.683833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:32.683695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:32.683695+0000","last_clean_scrub_stamp":"2024-04-16T19:37:32.683695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:40:20.565810+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":133,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-16T19:37:25.495541+0000","last_change":"2024-04-16T19:37:25.495643+0000","last_active":"2024-04-16T19:37:25.495541+0000","last_peered":"2024-04-16T19:37:25.495541+0000","last_clean":"2024-04-16T19:37:25.495541+0000","last_became_active":"2024-04-16T17:47:23.264554+0000","last_became_peered":"2024-04-16T17:47:23.264554+0000","last_unstale":"2024-04-16T19:37:25.495541+0000","last_undegraded":"2024-04-16T19:37:25.495541+0000","last_fullsized":"2024-04-16T19:37:25.495541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T19:37:25.495372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T19:37:25.495372+0000","last_clean_scrub_stamp":"2024-04-16T19:37:25.495372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:29.975866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":940,"num_object_clones":0,"num_object_copies":1880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":61107,"num_read_kb":61084,"num_write":4295,"num_write_kb":1971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":940,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":224218,"num_omap_keys":914,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3784,"ondisk_log_size":3784,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":20248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3263,"num_read_kb":4000,"num_write":177,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":40496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":64,"acting":64,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":5710,"num_read_kb":5469,"num_write":3695,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3885,"ondisk_log_size":3885,"up":64,"acting":64,"num_store_stats":6},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":64,"acting":64,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":24863,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":15173,"num_read_kb":9955,"num_write":4963,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":9,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2613248,"data_stored":49726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":958,"ondisk_log_size":958,"up":128,"acting":128,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":120879,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":406468,"num_read_kb":320286,"num_write":61468,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2422969,"num_omap_keys":10622,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":238430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27854,"ondisk_log_size":27854,"up":64,"acting":64,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4469,"ondisk_log_size":4469,"up":64,"acting":64,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":33285,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":2451,"num_read_kb":2293,"num_write":1115,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":360,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9817,"num_omap_keys":62,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007616,"data_stored":66570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":569,"ondisk_log_size":569,"up":64,"acting":64,"num_store_stats":6},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244641596,"num_pgs":163,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50744,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806856,"statfs":{"total":107374182400,"available":107322220544,"internally_reserved":0,"allocated":1556480,"data_stored":457512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244641595,"num_pgs":183,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":81156,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":104776444,"statfs":{"total":107374182400,"available":107291078656,"internally_reserved":0,"allocated":1437696,"data_stored":377462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244641595,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":87452,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":104770148,"statfs":{"total":107374182400,"available":107284631552,"internally_reserved":0,"allocated":2183168,"data_stored":995977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244641595,"num_pgs":173,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50732,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806868,"statfs":{"total":107374182400,"available":107322232832,"internally_reserved":0,"allocated":1544192,"data_stored":409273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244641596,"num_pgs":174,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":59568,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":104798032,"statfs":{"total":107374182400,"available":107313184768,"internally_reserved":0,"allocated":2072576,"data_stored":1000134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244641596,"num_pgs":180,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":50748,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":104806852,"statfs":{"total":107374182400,"available":107322216448,"internally_reserved":0,"allocated":1560576,"data_stored":445428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":12081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":8675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":6231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":17353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":6329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":6987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":1447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":11379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":59203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":23590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":36208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":36523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":2648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":80258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":16629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":10209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":8370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T19:38:01.866 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster c1 config set global mon_health_to_clog false 2024-04-16T19:38:02.196 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 556, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 364, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 269, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' 2024-04-16T19:38:02.212 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-16T19:38:02.212 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-16T19:38:02.212 DEBUG:tasks.ceph.c1.osd.0:waiting for process to exit 2024-04-16T19:38:02.212 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.280 INFO:tasks.ceph.c1.osd.0:Stopped 2024-04-16T19:38:02.281 DEBUG:tasks.ceph.c1.osd.1:waiting for process to exit 2024-04-16T19:38:02.281 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.376 INFO:tasks.ceph.c1.osd.1:Stopped 2024-04-16T19:38:02.376 DEBUG:tasks.ceph.c1.osd.2:waiting for process to exit 2024-04-16T19:38:02.376 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.467 INFO:tasks.ceph.c1.osd.2:Stopped 2024-04-16T19:38:02.467 DEBUG:tasks.ceph.c1.osd.3:waiting for process to exit 2024-04-16T19:38:02.467 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.552 INFO:tasks.ceph.c1.osd.3:Stopped 2024-04-16T19:38:02.553 DEBUG:tasks.ceph.c1.osd.4:waiting for process to exit 2024-04-16T19:38:02.553 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.645 INFO:tasks.ceph.c1.osd.4:Stopped 2024-04-16T19:38:02.646 DEBUG:tasks.ceph.c1.osd.5:waiting for process to exit 2024-04-16T19:38:02.646 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.747 INFO:tasks.ceph.c1.osd.5:Stopped 2024-04-16T19:38:02.747 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-16T19:38:02.748 DEBUG:tasks.ceph.c1.mgr.x:waiting for process to exit 2024-04-16T19:38:02.748 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.779 INFO:tasks.ceph.c1.mgr.x:Stopped 2024-04-16T19:38:02.779 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-16T19:38:02.780 DEBUG:tasks.ceph.c1.mon.a:waiting for process to exit 2024-04-16T19:38:02.780 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T19:38:02.814 INFO:tasks.ceph.c1.mon.a:Stopped 2024-04-16T19:38:02.814 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-16T19:38:02.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-04-16T19:38:02.871 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c1-0 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T19:38:02.871 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/c1-0 2024-04-16T19:38:03.197 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c1-1 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T19:38:03.198 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/c1-1 2024-04-16T19:38:03.410 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c1-2 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T19:38:03.410 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/c1-2 2024-04-16T19:38:03.576 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c1-3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T19:38:03.577 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/c1-3 2024-04-16T19:38:04.005 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c1-4 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T19:38:04.006 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/c1-4 2024-04-16T19:38:04.207 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/c1-5 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T19:38:04.208 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/c1-5 2024-04-16T19:38:04.364 INFO:tasks.ceph:Archiving mon data... 2024-04-16T19:38:04.365 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/c1-a to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/data/c1.mon.a.tgz 2024-04-16T19:38:04.365 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-04-16T19:38:04.372 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.bJlCiqE6G6 2024-04-16T19:38:04.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/c1-a -- . > /tmp/tmp.bJlCiqE6G6 2024-04-16T19:38:04.525 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.bJlCiqE6G6 2024-04-16T19:38:04.593 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.bJlCiqE6G6 is 407KB 2024-04-16T19:38:04.654 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.bJlCiqE6G6 2024-04-16T19:38:04.659 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-16T19:38:04.660 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/c1.conf /etc/ceph/c1.keyring /home/ubuntu/cephtest/c1.data /home/ubuntu/cephtest/c1.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:38:04.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/c1.conf /etc/ceph/c1.keyring /home/ubuntu/cephtest/c1.data /home/ubuntu/cephtest/c1.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:38:04.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/c1.conf /etc/ceph/c1.keyring /home/ubuntu/cephtest/c1.data /home/ubuntu/cephtest/c1.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:38:04.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/c1.conf /etc/ceph/c1.keyring /home/ubuntu/cephtest/c1.data /home/ubuntu/cephtest/c1.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T19:38:04.764 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T19:38:04.770 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T19:38:04.777 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T19:38:04.782 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T19:38:04.782 INFO:tasks.ceph:Archiving crash dumps... 2024-04-16T19:38:04.785 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/crash to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi071/crash 2024-04-16T19:38:04.786 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T19:38:04.821 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi083/crash 2024-04-16T19:38:04.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T19:38:04.836 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi122/crash 2024-04-16T19:38:04.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T19:38:04.888 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi188/crash 2024-04-16T19:38:04.889 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T19:38:04.902 INFO:tasks.ceph:Compressing logs... 2024-04-16T19:38:04.902 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T19:38:04.905 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T19:38:04.908 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T19:38:04.914 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T19:38:04.929 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.1.28368.log 2024-04-16T19:38:04.929 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-mgr.x.log 2024-04-16T19:38:04.932 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.1.28368.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22718.log 2024-04-16T19:38:04.932 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-mgr.x.log: 92.9% -- replaced with /var/log/ceph/c1-client.1.28368.log.gz 2024-04-16T19:38:04.932 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.1.26407.log 2024-04-16T19:38:04.932 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.22718.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26336.log 2024-04-16T19:38:04.932 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22718.log.gz 2024-04-16T19:38:04.932 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.1.26407.log: gzip -5 --verbose -- /var/log/ceph/rgw.c1.client.1.log 2024-04-16T19:38:04.933 INFO:teuthology.orchestra.run.smithi083.stderr: 82.9% -- replaced with /var/log/ceph/c1-client.1.26407.log.gz 2024-04-16T19:38:04.933 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.26336.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26384.log 2024-04-16T19:38:04.933 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26336.log.gz 2024-04-16T19:38:04.933 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/rgw.c1.client.1.log: gzip -5 --verbose -- /var/log/ceph/ops-log-c1-client.1.log 2024-04-16T19:38:04.942 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-osd.3.log 2024-04-16T19:38:04.942 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.26384.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26384.log.gz 2024-04-16T19:38:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ops-log-c1-client.1.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22766.log 2024-04-16T19:38:04.943 INFO:teuthology.orchestra.run.smithi083.stderr: 35.5% -- replaced with /var/log/ceph/ops-log-c1-client.1.log.gz 2024-04-16T19:38:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26312.log 2024-04-16T19:38:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.22766.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22766.log.gz 2024-04-16T19:38:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26360.log 2024-04-16T19:38:04.943 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.1.26997.log 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.26312.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26312.log.gz 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.26360.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22670.log 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26360.log.gz 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.1.26997.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22742.log 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr: 43.5% -- replaced with /var/log/ceph/c1-client.1.26997.log.gz 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.22670.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22670.log.gz 2024-04-16T19:38:04.944 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-osd.5.log 2024-04-16T19:38:04.945 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22694.log 2024-04-16T19:38:04.945 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.22742.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22742.log.gz 2024-04-16T19:38:04.945 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c1-osd.4.log 2024-04-16T19:38:04.945 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22646.log 2024-04-16T19:38:04.945 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-client.admin.22694.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22694.log.gz 2024-04-16T19:38:04.945 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/c1-osd.4.log: /var/log/ceph/c1-client.admin.22646.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22646.log.gz 2024-04-16T19:38:04.946 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/rgw.c1.client.1.log.gz 2024-04-16T19:38:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.25953.log 2024-04-16T19:38:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.70333.log 2024-04-16T19:38:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.53683.log 2024-04-16T19:38:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.25953.log: /var/log/ceph/c1-client.0.70333.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.70333.log.gz 2024-04-16T19:38:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.82756.log 2024-04-16T19:38:05.057 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.25953.log.gz 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.53683.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.77429.log 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.53683.log.gz 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.82756.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.82756.log.gz 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.112927.log 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.77429.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.89855.log 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr: 81.6% -- replaced with /var/log/ceph/c1-client.0.77429.log.gz 2024-04-16T19:38:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.112927.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.112927.log.gz 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.68449.log 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.89855.log: gzip 0.0% -5 -- replaced with /var/log/ceph/c1-client.0.89855.log.gz --verbose 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/c1-client.0.61373.log 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.68449.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.119427.log 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.68449.log.gz 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.61373.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.61373.log.gz 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.129490.log 2024-04-16T19:38:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.119427.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.119427.log.gz 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.27392.log 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.129490.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.104050.log 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr: 78.4% -- replaced with /var/log/ceph/c1-client.0.129490.log.gz 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.27392.log: gzip -5 --verbose -- 43.9% -- replaced with /var/log/ceph/c1-client.0.27392.log.gz /var/log/ceph/c1-client.admin.22705.log 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.104050.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.59603.log 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.104050.log.gz 2024-04-16T19:38:05.060 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.22705.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.79207.log 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22705.log.gz 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.59603.log: gzip 0.0% -- replaced with /var/log/ceph/c1-client.0.59603.log.gz -5 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/c1-client.0.102870.log 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.79207.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.79207.log.gz 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.35872.log 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.102870.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.33509.log 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.102870.log.gz 2024-04-16T19:38:05.061 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.35872.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.50726.log 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.35872.log.gz 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.33509.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.33509.log.gz 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.124752.log 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.50726.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.50726.log.gz 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.86901.log 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.124752.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.133054.log 2024-04-16T19:38:05.062 INFO:teuthology.orchestra.run.smithi071.stderr: 73.7% -- replaced with /var/log/ceph/c1-client.0.124752.log.gz 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.86901.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26695.log 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.86901.log.gz 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.133054.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.133656.log 0.0% 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/c1-client.0.133054.log.gz 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26695.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26695.log.gz 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140191.log 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/c1-client.0.133656.log: -5 --verbose -- /var/log/ceph/c1-client.0.128900.log 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.133656.log.gz 2024-04-16T19:38:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140191.log: gzip -5 --verbose -- 0.0% /var/log/ceph/c1-client.0.54865.log -- replaced with /var/log/ceph/c1-client.admin.140191.log.gz 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.128900.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.128900.log.gz 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.93999.log 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.54865.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.43002.log 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr: 45.9% -- replaced with /var/log/ceph/c1-client.0.54865.log.gz 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.93999.log: gzip -5 --verbose -- 0.0% /var/log/ceph/c1-osd.2.log -- replaced with /var/log/ceph/c1-client.0.93999.log.gz 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.43002.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.90447.log 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr: 52.4% -- replaced with /var/log/ceph/c1-client.0.43002.log.gz 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.59010.log 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.90447.log: gzip 0.0% -5 -- replaced with /var/log/ceph/c1-client.0.90447.log.gz --verbose 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/c1-client.0.99920.log 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.59010.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.118247.log 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.59010.log.gz 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.99920.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.99920.log.gz 2024-04-16T19:38:05.065 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.121197.log 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.78025.log 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.118247.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.118247.log.gz 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.121197.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.121197.log.gz 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26129.log 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.78025.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139905.log 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr: 57.0% -- replaced with /var/log/ceph/c1-client.0.78025.log.gz 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.105826.log 2024-04-16T19:38:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26129.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26129.log.gz 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139905.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140241.log 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139905.log.gz 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.105826.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26120.log 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.105826.log.gz 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.137261.log 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140241.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140241.log.gz 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26120.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26120.log.gz 2024-04-16T19:38:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.32240.log 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.137261.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.48357.log 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.137261.log.gz 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.32240.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/c1-client.0.32240.log.gz 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/c1-client.admin.140217.log 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.83346.log 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.48357.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.48357.log.gz 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140217.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140217.log.gz 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.42407.log 2024-04-16T19:38:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.83346.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.108189.log 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.83346.log.gz 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.42407.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.92813.log 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr: 50.1% -- replaced with /var/log/ceph/c1-client.0.42407.log.gz 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26280.log 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.108189.log: 70.0% -- replaced with /var/log/ceph/c1-client.0.108189.log.gz 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.92813.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.111151.log 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.92813.log.gz 2024-04-16T19:38:05.069 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26280.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26280.log.gz 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.81576.log 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140087.log 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.111151.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.111151.log.gz 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.81576.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.81576.log.gz 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.65975.log 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140087.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.112337.log 2024-04-16T19:38:05.070 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140087.log.gz 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.65975.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.135461.log 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr: 7.2% -- replaced with /var/log/ceph/c1-client.0.65975.log.gz 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/c1-client.0.112337.log: -- /var/log/ceph/c1-client.0.73884.log 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.112337.log.gz 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.135461.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.72103.log 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.135461.log.gz 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.73884.log: gzip -5 58.4% --verbose -- replaced with /var/log/ceph/c1-client.0.73884.log.gz -- 2024-04-16T19:38:05.071 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/c1-client.0.48947.log 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139931.log 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.72103.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.72103.log.gz 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.48947.log: 77.8% -- replaced with /var/log/ceph/c1-client.0.48947.log.gz 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.26794.log 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139931.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.105236.log 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139931.log.gz 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.26794.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.34099.log 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr: 84.0% -- replaced with /var/log/ceph/c1-client.0.26794.log.gz 2024-04-16T19:38:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.105236.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.62049.log 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.105236.log.gz 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.34099.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.88083.log 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.34099.log.gz 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.62049.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.62049.log.gz 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.49543.log 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.45395.log 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.88083.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.88083.log.gz 2024-04-16T19:38:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.49543.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.49543.log.gz 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.80392.log 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.45395.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26271.log 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.45395.log.gz 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.80392.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.120607.log 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr: 57.6% -- replaced with /var/log/ceph/c1-client.0.80392.log.gz 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.107006.log 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26271.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26271.log.gz 2024-04-16T19:38:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.120607.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.80984.log 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.120607.log.gz 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.107006.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.107006.log.gz 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139697.log 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20231.log 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.80984.log: /var/log/ceph/c1-client.admin.139697.log: 57.5% -- replaced with /var/log/ceph/c1-client.0.80984.log.gz 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139697.log.gz 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.76248.log 2024-04-16T19:38:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.tmp-client.admin.20231.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.139069.log 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20231.log.gz 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.136059.log 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.76248.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.76248.log.gz 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.139069.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.120017.log 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr: 92.9% -- replaced with /var/log/ceph/c1-client.0.139069.log.gz 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.136059.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.121787.log 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.136059.log.gz 2024-04-16T19:38:05.076 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.36462.log 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.120017.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.120017.log.gz 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.121787.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.121787.log.gz 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26478.log 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.36462.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139723.log 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.36462.log.gz 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26478.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.111741.log 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26478.log.gz 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139723.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.128307.log 2024-04-16T19:38:05.077 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139723.log.gz 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/rgw.c1.client.0.log 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.111741.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.111741.log.gz 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.128307.log: 77.7% -- replaced with /var/log/ceph/c1-client.0.128307.log.gz 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139801.log 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/rgw.c1.client.0.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.67233.log 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139801.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139801.log.gz 2024-04-16T19:38:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.38827.log 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.86304.log 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.67233.log: 41.9% -- replaced with /var/log/ceph/c1-client.0.67233.log.gz 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.38827.log: 63.2% -- replaced with /var/log/ceph/c1-client.0.38827.log.gz 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.58420.log 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.86304.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.76838.log 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.86304.log.gz 2024-04-16T19:38:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26477.log 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.58420.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.58420.log.gz 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.76838.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.76838.log.gz 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.78617.log 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.108782.log 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26477.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26477.log.gz 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.78617.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.78617.log.gz 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.72694.log 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.30366.log 2024-04-16T19:38:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.108782.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.108782.log.gz 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.72694.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.72694.log.gz 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.95179.log 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.46578.log 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.30366.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.30366.log.gz 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.95179.log: 57.1% -- replaced with /var/log/ceph/c1-client.0.95179.log.gz 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.99329.log 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.46578.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.114707.log 2024-04-16T19:38:05.081 INFO:teuthology.orchestra.run.smithi071.stderr: 63.3% -- replaced with /var/log/ceph/c1-client.0.46578.log.gz 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.99329.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.99329.log.gz 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26004.log 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.114707.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.88673.log 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.114707.log.gz 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26004.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.69740.log 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26004.log.gz 2024-04-16T19:38:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.88673.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.88673.log.gz 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.91630.log 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.69740.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.69065.log 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.69740.log.gz 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.91630.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.91630.log.gz 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26290.log 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.25978.log 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.69065.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.69065.log.gz 2024-04-16T19:38:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26290.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/c1-client.admin.26290.log.gz 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/c1-client.admin.26482.log 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.25978.log: gzip -5 --verbose -- 0.0% /var/log/ceph/c1-client.0.131858.log -- replaced with /var/log/ceph/c1-client.admin.25978.log.gz 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26482.log: gzip -5 --verbose -- /var/log/ceph/c1-osd.0.log 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26482.log.gz 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.131858.log: 67.4% -- replaced with /var/log/ceph/c1-client.0.131858.log.gz 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1.log 2024-04-16T19:38:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.85122.log 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-osd.0.log: /var/log/ceph/c1.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.75658.log 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.85122.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.37647.log 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr: 69.9% -- replaced with /var/log/ceph/c1-client.0.85122.log.gz 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.75658.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22788.log 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.75658.log.gz 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.37647.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.96953.log 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.37647.log.gz 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.22788.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22788.log.gz 2024-04-16T19:38:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.41817.log 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.96953.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.134257.log 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.96953.log.gz 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.41817.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.130083.log 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.41817.log.gz 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.134257.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.117657.log 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.134257.log.gz 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.130083.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140139.log 2024-04-16T19:38:05.086 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.130083.log.gz 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.117657.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.53091.log 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.117657.log.gz 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140139.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.66639.log 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140139.log.gz 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.53091.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140061.log 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr: 46.3% -- replaced with /var/log/ceph/c1-client.0.53091.log.gz 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.66639.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.138465.log 2024-04-16T19:38:05.087 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.66639.log.gz 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140061.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.118837.log 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140061.log.gz 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.138465.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140165.log 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.138465.log.gz 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139957.log 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.118837.log: /var/log/ceph/c1-client.admin.140165.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.28613.log 2024-04-16T19:38:05.088 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140165.log.gz 0.0% -- replaced with /var/log/ceph/c1-client.0.118837.log.gz 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139957.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.73290.log 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139957.log.gz 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.28613.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.39419.log 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.28613.log.gz 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.73290.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26745.log 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.73290.log.gz 2024-04-16T19:38:05.089 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.39419.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.89263.log 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.39419.log.gz 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26745.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.63548.log 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26745.log.gz 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.89263.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.113518.log 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr: 56.4% -- replaced with /var/log/ceph/c1-client.0.89263.log.gz 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.63548.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.126528.log 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.63548.log.gz 2024-04-16T19:38:05.090 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.113518.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.125345.log 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.113518.log.gz 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.126528.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.64797.log 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.126528.log.gz 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr: 89.6% -- replaced with /var/log/ceph/c1.log.gz 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.125345.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.116477.log 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.125345.log.gz 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.64797.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.47172.log 2024-04-16T19:38:05.091 INFO:teuthology.orchestra.run.smithi071.stderr: 15.2% -- replaced with /var/log/ceph/c1-client.0.64797.log.gz 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.116477.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.54273.log 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.116477.log.gz 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.47172.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26303.log 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.47172.log.gz 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.54273.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.30392.log 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr: 50.1% -- replaced with /var/log/ceph/c1-client.0.54273.log.gz 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/c1-client.admin.26303.log: -- /var/log/ceph/c1-client.0.96361.log 2024-04-16T19:38:05.092 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26303.log.gz 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.30392.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.127122.log 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.30392.log.gz 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.96361.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.107596.log 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr: 50.0% -- replaced with /var/log/ceph/c1-client.0.96361.log.gz 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.127122.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26128.log 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr: 76.6% -- replaced with /var/log/ceph/c1-client.0.127122.log.gz 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.107596.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.94589.log 2024-04-16T19:38:05.093 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.107596.log.gz 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26128.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26128.log.gz 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.60193.log 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.115887.log 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.94589.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.94589.log.gz 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.60193.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.101100.log 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.60193.log.gz 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.115887.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.115887.log.gz 2024-04-16T19:38:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.29781.log 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.104640.log 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.101100.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.101100.log.gz 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.29781.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.52499.log 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.29781.log.gz 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.104640.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.130673.log 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr: 63.4% -- replaced with /var/log/ceph/c1-client.0.104640.log.gz 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.37052.log 2024-04-16T19:38:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.52499.log: 49.6% -- replaced with /var/log/ceph/c1-client.0.52499.log.gz 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.130673.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.57230.log 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr: 78.6% -- replaced with /var/log/ceph/c1-client.0.130673.log.gz 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.37052.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.34689.log 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.37052.log.gz 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.109962.log 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.57230.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.57230.log.gz 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.34689.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139671.log 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.34689.log.gz 2024-04-16T19:38:05.096 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.109962.log: gzip -5 --verbose -- /var/log/ceph/c1-osd.1.log 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr: 58.7% -- replaced with /var/log/ceph/c1-client.0.109962.log.gz 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139671.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.65391.log 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139671.log.gz 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.127715.log 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.65391.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.40009.log 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.65391.log.gz 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.127715.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139749.log 2024-04-16T19:38:05.097 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.127715.log.gz 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.40009.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.51316.log 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr: 47.0% -- replaced with /var/log/ceph/c1-client.0.40009.log.gz 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139749.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26771.log 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139749.log.gz 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.51316.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.122970.log 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr: 60.5% -- replaced with /var/log/ceph/c1-client.0.51316.log.gz 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26771.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.101690.log 2024-04-16T19:38:05.098 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26771.log.gz 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.122970.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.83936.log 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.122970.log.gz 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.101690.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26721.log 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.101690.log.gz 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.83936.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.44803.log 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.83936.log.gz 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26721.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.123563.log 2024-04-16T19:38:05.099 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26721.log.gz 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.44803.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.84530.log 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr: 52.4% -- replaced with /var/log/ceph/c1-client.0.44803.log.gz 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.123563.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26058.log 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr: 55.1% -- replaced with /var/log/ceph/c1-client.0.123563.log.gz 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.84530.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.32919.log 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr: 69.1% -- replaced with /var/log/ceph/c1-client.0.84530.log.gz 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26058.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.74476.log 2024-04-16T19:38:05.100 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26058.log.gz 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.32919.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.91037.log 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.32919.log.gz 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.64181.log 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.74476.log: 57.3% -- replaced with /var/log/ceph/c1-client.0.74476.log.gz 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.91037.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.71513.log 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr: 57.3% -- replaced with /var/log/ceph/c1-client.0.91037.log.gz 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.64181.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.114113.log 2024-04-16T19:38:05.101 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.64181.log.gz 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.71513.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.55457.log 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.71513.log.gz 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.114113.log: gzip -5 --verbose -- /var/log/ceph/ops-log-c1-client.0.log 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr: 79.1% -- replaced with /var/log/ceph/c1-client.0.114113.log.gz 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.55457.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.125935.log 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.55457.log.gz 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ops-log-c1-client.0.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.132450.log 2024-04-16T19:38:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.125935.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.79802.log 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr: 75.8% -- replaced with /var/log/ceph/c1-client.0.125935.log.gz 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.132450.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.30999.log 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.132450.log.gz 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139983.log 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.79802.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.79802.log.gz 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.30999.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.22859.log 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr: 19.1% -- replaced with /var/log/ceph/c1-client.0.30999.log.gz 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.93403.log 2024-04-16T19:38:05.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139983.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139983.log.gz/var/log/ceph/c1-client.admin.22859.log: 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26127.log 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.22859.log.gz 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.93403.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.93403.log.gz 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140113.log 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26127.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.100510.log 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26127.log.gz 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140113.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.110554.log 2024-04-16T19:38:05.104 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140113.log.gz 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.56640.log 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.100510.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.100510.log.gz 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.110554.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.44196.log 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.110554.log.gz 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.60783.log 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.56640.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.56640.log.gz 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.44196.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.115297.log 2024-04-16T19:38:05.105 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/c1-client.0.44196.log.gz 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.70923.log 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.60783.log: /var/log/ceph/c1-client.0.115297.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.131268.log 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.60783.log.gz 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.115297.log.gz 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.70923.log: 0.0%gzip -5 --verbose -- /var/log/ceph/c1-client.0.136661.log 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/c1-client.0.70923.log.gz 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.131268.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26628.log 2024-04-16T19:38:05.106 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.131268.log.gz 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.35282.log 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.136661.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.136661.log.gz 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26628.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.75068.log 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26628.log.gz 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26446.log 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.35282.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.35282.log.gz 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.75068.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.134859.log 2024-04-16T19:38:05.107 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.75068.log.gz 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26446.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26446.log.gz 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.29197.log 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.134859.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.57825.log 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.134859.log.gz 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.29197.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.98147.log 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.29197.log.gz 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.57825.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.102280.log 2024-04-16T19:38:05.108 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.57825.log.gz 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.98147.log: 58.7% -- replaced with /var/log/ceph/c1-client.0.98147.log.gz 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26032.log 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26602.log 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.102280.log: /var/log/ceph/c1-client.admin.26032.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26119.log 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.102280.log.gz 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26032.log.gz 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26602.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26602.log.gz 2024-04-16T19:38:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.43594.log 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.30415.log 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26119.log: /var/log/ceph/c1-client.0.43594.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.87491.log 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% 66.0% -- replaced with /var/log/ceph/c1-client.0.43594.log.gz 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/c1-client.admin.26119.log.gz 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.30415.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.30415.log.gz 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.45986.log 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139827.log 2024-04-16T19:38:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.87491.log: /var/log/ceph/c1-client.0.45986.log: gzip -5 --verbose -- /var/log/ceph/c1-mon.a.log 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr: 49.9% -- replaced with /var/log/ceph/c1-client.0.87491.log.gz 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr: 64.5% -- replaced with /var/log/ceph/c1-client.0.45986.log.gz/var/log/ceph/c1-client.admin.139827.log: 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139827.log.gz 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139775.log 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.51909.log 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-mon.a.log: /var/log/ceph/c1-client.admin.139775.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.40601.log 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139775.log.gz 2024-04-16T19:38:05.111 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.51909.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.51909.log.gz 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.106416.log 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.40601.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.47762.log 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr: 71.2% -- replaced with /var/log/ceph/c1-client.0.40601.log.gz 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.106416.log: 0.0%gzip -5 --verbose -- /var/log/ceph/c1-client.0.98739.log 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/c1-client.0.106416.log.gz 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.47762.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.47762.log.gz 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26130.log 2024-04-16T19:38:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.139879.log 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.98739.log: /var/log/ceph/c1-client.admin.26130.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.98739.log.gz 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26130.log.gz 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1.audit.log 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139879.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139879.log.gz 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.67817.log 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1.audit.log: gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140035.log 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.67817.log: 3.8% -- replaced with /var/log/ceph/c1-client.0.67817.log.gz 2024-04-16T19:38:05.113 INFO:teuthology.orchestra.run.smithi071.stderr: 89.6% -- replaced with /var/log/ceph/c1.audit.log.gzgzip -5 --verbose -- /var/log/ceph/c1-client.0.38237.log 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.140035.log: gzip -5 --verbose -- /var/log/ceph/c1-client.0.50133.log 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0%/var/log/ceph/c1-client.0.38237.log: -- replaced with /var/log/ceph/c1-client.admin.140035.log.gz 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.38237.log.gz 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26263.log 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26507.log 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.50133.log: /var/log/ceph/c1-client.admin.26263.log: 60.5% -- replaced with /var/log/ceph/c1-client.0.50133.log.gz 2024-04-16T19:38:05.114 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26263.log.gz 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.137866.log 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.92223.log 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26507.log: /var/log/ceph/c1-client.0.137866.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26507.log.gz 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.137866.log.gz 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26291.log 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.56048.log 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.92223.log: /var/log/ceph/c1-client.admin.26291.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.92223.log.gz 2024-04-16T19:38:05.115 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26291.log.gz 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.140009.log 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26654.log 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.56048.log: /var/log/ceph/c1-client.admin.140009.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.56048.log.gz 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.admin.140009.log.gz 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26450.log 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.109372.log 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26654.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26654.log.gz 2024-04-16T19:38:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26450.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26450.log.gzgzip -5 --verbose -- /var/log/ceph/c1-client.admin.139853.log 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.109372.log: 0.0%gzip -5 --verbose -- /var/log/ceph/c1-client.0.103460.log 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/c1-client.0.109372.log.gz 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.139853.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.139853.log.gzgzip 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/c1-client.0.41202.log 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.103460.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.103460.log.gz 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.117067.log 2024-04-16T19:38:05.117 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.admin.26084.log 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.41202.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.41202.log.gz 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.117067.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.117067.log.gzgzip -5 --verbose -- /var/log/ceph/c1-client.0.62954.log 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.admin.26084.log: 0.0% -- replaced with /var/log/ceph/c1-client.admin.26084.log.gz 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.124161.log 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.97544.log 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.62954.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.62954.log.gz 2024-04-16T19:38:05.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.124161.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.124161.log.gzgzip -5 --verbose -- /var/log/ceph/c1-client.0.85714.log 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.97544.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.97544.log.gz 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.95771.log 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.82166.log 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.85714.log: /var/log/ceph/c1-client.0.95771.log: 0.0% -- replaced with /var/log/ceph/c1-client.0.85714.log.gz 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/c1-client.0.95771.log.gz 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/c1-client.0.122377.log 2024-04-16T19:38:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/c1-client.0.82166.log: 0.0%/var/log/ceph/c1-client.0.122377.log: -- replaced with /var/log/ceph/c1-client.0.82166.log.gz 2024-04-16T19:38:05.120 INFO:teuthology.orchestra.run.smithi071.stderr: 78.1% -- replaced with /var/log/ceph/c1-client.0.122377.log.gz 2024-04-16T19:38:05.120 INFO:teuthology.orchestra.run.smithi071.stderr: 95.1% -- replaced with /var/log/ceph/ops-log-c1-client.0.log.gz 2024-04-16T19:38:05.133 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.1.26223.log 2024-04-16T19:38:05.133 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26152.log 2024-04-16T19:38:05.134 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.1.26223.log: gzip -5 --verbose -- /var/log/ceph/c2-client.1.26813.log 2024-04-16T19:38:05.134 INFO:teuthology.orchestra.run.smithi188.stderr: 83.1% -- replaced with /var/log/ceph/c2-client.1.26223.log.gz 2024-04-16T19:38:05.134 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.26152.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26200.log 2024-04-16T19:38:05.134 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26152.log.gz 2024-04-16T19:38:05.134 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.1.26813.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22732.log 2024-04-16T19:38:05.134 INFO:teuthology.orchestra.run.smithi188.stderr: 43.9% -- replaced with /var/log/ceph/c2-client.1.26813.log.gz 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.26200.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26200.log.gz 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-c2-client.1.log 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.22732.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22732.log.gz 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22780.log 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ops-log-c2-client.1.log: gzip -5 --verbose -- /var/log/ceph/c2-osd.5.log 35.4% 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr: -- replaced with /var/log/ceph/ops-log-c2-client.1.log.gz 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.22780.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22780.log.gz 2024-04-16T19:38:05.135 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22684.log 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-osd.5.log: gzip -5 --verbose -- /var/log/ceph/rgw.c2.client.1.log 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.22684.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22684.log.gz 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26128.log 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/rgw.c2.client.1.log: gzip -5 --verbose -- /var/log/ceph/c2-mgr.x.log 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.26128.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26128.log.gz 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22708.log 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22756.log 2024-04-16T19:38:05.136 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-mgr.x.log: /var/log/ceph/c2-client.admin.22708.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22708.log.gz 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-osd.4.log 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.22756.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22660.log 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22756.log.gz 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c2-osd.3.log 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.22660.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22660.log.gz 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26176.log 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-osd.3.log: 93.4% -- replaced with /var/log/ceph/rgw.c2.client.1.log.gz 2024-04-16T19:38:05.137 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/c2-client.admin.26176.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26176.log.gz 2024-04-16T19:38:05.208 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.61668.log 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.135972.log 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.61668.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.30183.log 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.61668.log.gz 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.135972.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.135972.log.gz 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.75929.log 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.178044.log 2024-04-16T19:38:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.30183.log: /var/log/ceph/c2-client.0.75929.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.30183.log.gz 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.98379.log 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.75929.log.gz 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.178044.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.200208.log 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.178044.log.gz 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.98379.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22901.log 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.98379.log.gz 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200208.log: 8.2%gzip -- replaced with /var/log/ceph/c2-client.0.200208.log.gz 2024-04-16T19:38:05.210 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/c2-client.0.46275.log 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.22901.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.91859.log 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22901.log.gz 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.46275.log: 52.7% -- replaced with /var/log/ceph/c2-client.0.46275.log.gzgzip 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/c2-client.0.107838.log 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.91859.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.33204.log 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.91859.log.gz 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.107838.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.94824.log 2024-04-16T19:38:05.211 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.107838.log.gz 2024-04-16T19:38:05.212 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.33204.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.156172.log 2024-04-16T19:38:05.212 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.33204.log.gz 2024-04-16T19:38:05.212 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.94824.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.135379.log 2024-04-16T19:38:05.212 INFO:teuthology.orchestra.run.smithi122.stderr: 44.2% -- replaced with /var/log/ceph/c2-client.0.94824.log.gz 2024-04-16T19:38:05.212 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.156172.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.114936.log 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.156172.log.gz 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.135379.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.81406.log 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.135379.log.gz 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.114936.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.162629.log 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr: 56.5% -- replaced with /var/log/ceph/c2-client.0.114936.log.gz 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.81406.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.200219.log 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.81406.log.gz 2024-04-16T19:38:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.162629.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26491.log 2024-04-16T19:38:05.214 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.162629.log.gz 2024-04-16T19:38:05.214 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200219.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.72314.log 2024-04-16T19:38:05.214 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200219.log.gz 2024-04-16T19:38:05.214 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26491.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.199515.log 2024-04-16T19:38:05.214 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26491.log.gz 2024-04-16T19:38:05.217 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.72314.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.72314.log.gz 2024-04-16T19:38:05.217 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.190620.log 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.199515.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.183978.log 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr: 44.2% -- replaced with /var/log/ceph/c2-client.0.199515.log.gz 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.190620.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.54561.log 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.190620.log.gz 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.183978.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.64634.log 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.183978.log.gz 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.54561.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26452.log 2024-04-16T19:38:05.218 INFO:teuthology.orchestra.run.smithi122.stderr: 46.3% -- replaced with /var/log/ceph/c2-client.0.54561.log.gz 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.64634.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.82660.log 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.64634.log.gz 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26452.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.90673.log 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26452.log.gz 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.82660.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.129954.log 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.82660.log.gz 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.90673.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.94231.log 2024-04-16T19:38:05.219 INFO:teuthology.orchestra.run.smithi122.stderr: 48.7% -- replaced with /var/log/ceph/c2-client.0.90673.log.gz 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.129954.log: gzip -5 --verbose 47.4% -- -- replaced with /var/log/ceph/c2-client.0.129954.log.gz /var/log/ceph/c2-client.0.122768.log 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.94231.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.183385.log 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.94231.log.gz 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.122768.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26400.log 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.122768.log.gz 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.183385.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.169086.log 2024-04-16T19:38:05.220 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.183385.log.gz 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26400.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/c2-client.admin.26400.log.gz /var/log/ceph/c2-client.0.172032.log 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.169086.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.28964.log 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.169086.log.gz 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.172032.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.165564.log 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.172032.log.gz 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.28964.log: gzip 0.0% -- replaced with /var/log/ceph/c2-client.0.28964.log.gz -5 2024-04-16T19:38:05.221 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.153235.log 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.165564.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.191213.log 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.165564.log.gz 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.153235.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.97199.log 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.153235.log.gz 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.191213.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.38543.log 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.191213.log.gz 2024-04-16T19:38:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.97199.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26818.log 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.97199.log.gz 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.38543.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.33799.log 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr: 70.2% -- replaced with /var/log/ceph/c2-client.0.38543.log.gz 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26818.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26648.log 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26818.log.gz 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.33799.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22827.log 2024-04-16T19:38:05.223 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.33799.log.gz 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26648.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.74150.log 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26648.log.gz 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.22827.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26497.log 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22827.log.gz 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.74150.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.76522.log 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.74150.log.gz 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26497.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.93638.log 2024-04-16T19:38:05.224 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26497.log.gz 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.76522.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.194178.log 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.76522.log.gz 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.93638.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.164977.log 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.93638.log.gz 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.194178.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.116737.log 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.194178.log.gz 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.164977.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.175079.log 2024-04-16T19:38:05.225 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.164977.log.gz 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.116737.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.52782.log 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.116737.log.gz 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.175079.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.140719.log 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.175079.log.gz 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.52782.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.131145.log 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr: 81.0% -- replaced with /var/log/ceph/c2-client.0.52782.log.gz 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.140719.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.178637.log 2024-04-16T19:38:05.226 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.140719.log.gz 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.131145.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.127512.log 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.178637.log: gzip -5 --verbose -- /var/log/ceph/c2-osd.1.log 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.178637.log.gz 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.127512.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.151474.log 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.127512.log.gz 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.95417.log 2024-04-16T19:38:05.227 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.151474.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.151474.log.gz 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.182792.log 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.95417.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.143684.log 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.95417.log.gz 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.182792.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.182792.log.gz 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.198329.log 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.143684.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.67602.log 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.143684.log.gz 2024-04-16T19:38:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.198329.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.198329.log.gz 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.45090.log 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.67602.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.51006.log 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.67602.log.gz 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.45090.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26346.log 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr: 53.3% -- replaced with /var/log/ceph/c2-client.0.45090.log.gz 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.51006.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.58701.log 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.51006.log.gz 2024-04-16T19:38:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26346.log: gzip -5 --verbose -- /var/log/ceph/rgw.c2.client.0.log 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26346.log.gz 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.58701.log: gzip -5 --verbose -- 45.5% /var/log/ceph/c2-client.0.101930.log 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.58701.log.gz 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr: 96.3% -- replaced with /var/log/ceph/c2-client.0.131145.log.gz 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/rgw.c2.client.0.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.166738.log 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.101930.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.83837.log 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr: 57.7% -- replaced with /var/log/ceph/c2-client.0.101930.log.gz 2024-04-16T19:38:05.230 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.166738.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.182199.log 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.166738.log.gz 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.83837.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.147909.log 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr: 17.8% -- replaced with /var/log/ceph/c2-client.0.83837.log.gz 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.182199.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.88894.log 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.182199.log.gz 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.147909.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.60482.log 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr: 91.0% -- replaced with /var/log/ceph/c2-client.0.147909.log.gz 2024-04-16T19:38:05.231 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.88894.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26486.log 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.88894.log.gz 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.60482.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.99563.log 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.60482.log.gz 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26486.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.160868.log 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26486.log.gz 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.99563.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.99563.log.gz 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.49824.log 2024-04-16T19:38:05.232 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.97792.log 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.160868.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.160868.log.gz 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.49824.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.96013.log 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.49824.log.gz 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.97792.log: 0.0%gzip -- replaced with /var/log/ceph/c2-client.0.97792.log.gz 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/c2-client.0.109612.log 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.96013.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.96013.log.gz 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.71727.log 2024-04-16T19:38:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.109612.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.197143.log 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.109612.log.gz 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.71727.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.185757.log 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.71727.log.gz 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.197143.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.197143.log.gz 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26498.log 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.185757.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.34985.log 2024-04-16T19:38:05.234 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.185757.log.gz 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26498.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26498.log.gz 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.79479.log 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.34985.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.156759.log 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.34985.log.gz 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.79479.log: gzip 0.0% -- replaced with /var/log/ceph/c2-client.0.79479.log.gz -5 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.126919.log 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.156759.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.75336.log 2024-04-16T19:38:05.235 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.156759.log.gz 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.126919.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26970.log 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.126919.log.gz 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.75336.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.63447.log 47.4% 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.75336.log.gz 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26970.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26970.log.gz 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.200246.log 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.59889.log 2024-04-16T19:38:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.63447.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.63447.log.gz 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200246.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.192992.log 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr: 9.2% -- replaced with /var/log/ceph/c2-client.0.200246.log.gz 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.59889.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.85336.log 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.59889.log.gz 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.192992.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.101337.log 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.192992.log.gz 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.85336.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.85336.log.gz 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.90080.log 2024-04-16T19:38:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.101337.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.103699.log 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.101337.log.gz 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.90080.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.167912.log 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr: 46.1% -- replaced with /var/log/ceph/c2-client.0.90080.log.gz 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.103699.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.115525.log 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr: 90.0% -- replaced with /var/log/ceph/c2-client.0.103699.log.gz 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.167912.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.167912.log.gz 2024-04-16T19:38:05.238 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.72901.log 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.115525.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26488.log 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.115525.log.gz 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.72901.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.72901.log.gz 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.100155.log 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26488.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.185164.log 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26488.log.gz 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.100155.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26874.log 2024-04-16T19:38:05.239 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.100155.log.gz 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.185164.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.30767.log 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.185164.log.gz 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26874.log: gzip 0.0% -- replaced with /var/log/ceph/c2-client.admin.26874.log.gz -5 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.120989.log 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.30767.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.118506.log 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr: 18.5% -- replaced with /var/log/ceph/c2-client.0.30767.log.gz 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.120989.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.61075.log 2024-04-16T19:38:05.240 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.120989.log.gz 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.118506.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.129344.log 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/c2-client.0.118506.log.gz 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.61075.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.177451.log 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.61075.log.gz 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.129344.log: 42.5% -- replaced with /var/log/ceph/c2-client.0.129344.log.gz 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.200142.log 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.177451.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.143091.log 2024-04-16T19:38:05.241 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.177451.log.gz 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200142.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20273.log 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200142.log.gz 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.143091.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.27046.log 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.143091.log.gz 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.tmp-client.admin.20273.log: gzip -5 --verbose -- 0.0% /var/log/ceph/c2-client.0.114343.log -- replaced with /var/log/ceph/ceph.tmp-client.admin.20273.log.gz 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.27046.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.150300.log 2024-04-16T19:38:05.242 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.27046.log.gz 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.114343.log: gzip -5 --verbose -- 45.5% /var/log/ceph/c2-client.0.170260.log -- replaced with /var/log/ceph/c2-client.0.114343.log.gz 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.150300.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.150300.log.gz 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.110204.log 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.170260.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.195957.log 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.170260.log.gz 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.110204.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.124547.log 2024-04-16T19:38:05.243 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.110204.log.gz 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.195957.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.83253.log 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.195957.log.gz 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.124547.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.176858.log 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.124547.log.gz 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.83253.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.27122.log 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.83253.log.gz 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.176858.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.98971.log 2024-04-16T19:38:05.244 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.176858.log.gz 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.27122.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.27122.log.gz 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.57515.log 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.98971.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.117330.log 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.98971.log.gz 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.57515.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.170847.log 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.57515.log.gz 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.117330.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.77709.log 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr: 57.0% -- replaced with /var/log/ceph/c2-client.0.117330.log.gz 2024-04-16T19:38:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.170847.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.92452.log 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.170847.log.gz 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.77709.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.174486.log 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.77709.log.gz 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.92452.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.155583.log 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr: 44.2% -- replaced with /var/log/ceph/c2-client.0.92452.log.gz 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.174486.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26372.log 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr: 44.2% -- replaced with /var/log/ceph/c2-client.0.174486.log.gz 2024-04-16T19:38:05.246 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.155583.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.35578.log 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.155583.log.gz 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26372.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.122175.log 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26372.log.gz 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.35578.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.62854.log 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.35578.log.gz 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.122175.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.69376.log 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.122175.log.gz 2024-04-16T19:38:05.247 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.62854.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.91266.log 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.62854.log.gz 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.69376.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.187632.log 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.69376.log.gz 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.91266.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.152061.log 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.91266.log.gz 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.187632.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.137158.log 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/c2-client.0.187632.log.gz 2024-04-16T19:38:05.248 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.152061.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.116118.log 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.152061.log.gz 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.137158.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.42094.log 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr: 48.7% -- replaced with /var/log/ceph/c2-client.0.137158.log.gz 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.116118.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26669.log 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr: 72.6% -- replaced with /var/log/ceph/c2-client.0.116118.log.gz 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.42094.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.200120.log 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr: 92.6% -- replaced with /var/log/ceph/c2-client.0.42094.log.gz 2024-04-16T19:38:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26669.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.189434.log 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26669.log.gz 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200120.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.184571.log 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200120.log.gz 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.189434.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.105474.log 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.189434.log.gz 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.184571.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.103108.log 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.184571.log.gz 2024-04-16T19:38:05.250 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.105474.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.71140.log 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr: 48.7% -- replaced with /var/log/ceph/c2-client.0.105474.log.gz 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.103108.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.39132.log 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr: 75.4% -- replaced with /var/log/ceph/c2-client.0.103108.log.gz 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.71140.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.40318.log 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.71140.log.gz 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.39132.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.168499.log 2024-04-16T19:38:05.251 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.39132.log.gz 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.40318.log: gzip -5 --verbose -- /var/log/ceph/c2.log 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr: 47.0% -- replaced with /var/log/ceph/c2-client.0.40318.log.gz 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.168499.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.144277.log 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.168499.log.gz 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.136565.log 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.144277.log: 45.5%gzip -- replaced with /var/log/ceph/c2-client.0.144277.log.gz 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/c2-client.admin.26495.log 2024-04-16T19:38:05.252 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.117919.log 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.136565.log: 46.1% -- replaced with /var/log/ceph/c2-client.0.136565.log.gz 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26495.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.113750.log 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26495.log.gz 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.117919.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.125733.log 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.117919.log.gz 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.113750.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.86522.log 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.113750.log.gz 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.125733.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.89487.log 2024-04-16T19:38:05.253 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.125733.log.gz 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.96606.log 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.86522.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.86522.log.gz 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.192399.log 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.89487.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.89487.log.gz 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.96606.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.46866.log 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.96606.log.gz 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.120383.log 2024-04-16T19:38:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.192399.log: 44.8% -- replaced with /var/log/ceph/c2-client.0.192399.log.gz 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.46866.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.130552.log 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.46866.log.gz 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.120383.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.152648.log 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/c2-client.0.120383.log.gz 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.56333.log 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.130552.log: 67.9% -- replaced with /var/log/ceph/c2-client.0.130552.log.gz 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.152648.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.110793.log 2024-04-16T19:38:05.255 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.152648.log.gz 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.56333.log: 89.1% -- replaced with /var/log/ceph/c2.log.gz 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.112568.log 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr: 46.0% -- replaced with /var/log/ceph/c2-client.0.56333.log.gz 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.110793.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.154409.log 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.110793.log.gz 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.112568.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26996.log 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.112568.log.gz 2024-04-16T19:38:05.256 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.154409.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.171436.log 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.154409.log.gz 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26996.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.70553.log 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26996.log.gz 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.171436.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.39725.log 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.70553.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.70553.log.gz 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.78302.log 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.128105.log 2024-04-16T19:38:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.39725.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.39725.log.gz 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.78302.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.109023.log 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.78302.log.gz 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.128105.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.52189.log 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr: 56.2% -- replaced with /var/log/ceph/c2-client.0.128105.log.gz 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/c2-client.0.109023.log: -5 --verbose -- /var/log/ceph/c2-client.0.150887.log 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr: 69.2% -- replaced with /var/log/ceph/c2-client.0.109023.log.gz 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.52189.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.175672.log 2024-04-16T19:38:05.258 INFO:teuthology.orchestra.run.smithi122.stderr: 46.8% -- replaced with /var/log/ceph/c2-client.0.52189.log.gz 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.150887.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.32018.log 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.150887.log.gz 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.175672.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.131739.log 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.175672.log.gz 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.32018.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.141905.log 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.32018.log.gz 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.131739.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.74743.log 2024-04-16T19:38:05.259 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.131739.log.gz 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr: 97.1% -- replaced with /var/log/ceph/c2-client.0.171436.log.gz 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.36171.log 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.141905.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.141905.log.gz 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.74743.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.149126.log 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.74743.log.gz 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.36171.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.133011.log 2024-04-16T19:38:05.260 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.36171.log.gz 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.149126.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.59294.log 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.149126.log.gz 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.133011.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.200175.log 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr: 82.5% -- replaced with /var/log/ceph/c2-client.0.133011.log.gz 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.59294.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26824.log 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.59294.log.gz 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200175.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.65227.log 2024-04-16T19:38:05.261 INFO:teuthology.orchestra.run.smithi122.stderr: 7.1% -- replaced with /var/log/ceph/c2-client.0.200175.log.gz 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26824.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26620.log 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26824.log.gz 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.65227.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.145465.log 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.65227.log.gz 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26620.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.36764.log 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26620.log.gz 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.145465.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.139533.log 2024-04-16T19:38:05.262 INFO:teuthology.orchestra.run.smithi122.stderr: 94.3% -- replaced with /var/log/ceph/c2-client.0.145465.log.gz 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.36764.log: 47.4%gzip -- replaced with /var/log/ceph/c2-client.0.36764.log.gz -5 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.137752.log 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.139533.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.147302.log 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.139533.log.gz 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.137752.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.195364.log 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.137752.log.gz 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.147302.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.154996.log 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr: 92.7% -- replaced with /var/log/ceph/c2-client.0.147302.log.gz 2024-04-16T19:38:05.263 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.195364.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.179827.log 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.195364.log.gz 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.154996.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.169673.log 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.154996.log.gz 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.179827.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.85929.log 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.179827.log.gz 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.169673.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.45681.log 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.169673.log.gz 2024-04-16T19:38:05.264 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.85929.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.140126.log 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.85929.log.gz 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.45681.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.47459.log 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr: 75.2% -- replaced with /var/log/ceph/c2-client.0.45681.log.gz 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.140126.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.68787.log 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.140126.log.gz 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.47459.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.65820.log 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr: 50.4% -- replaced with /var/log/ceph/c2-client.0.47459.log.gz 2024-04-16T19:38:05.265 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.68787.log: gzip -5 --verbose -- /var/log/ceph/ops-log-c2-client.0.log 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.68787.log.gz 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.65820.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.87115.log 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.65820.log.gz 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ops-log-c2-client.0.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.84484.log 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.87115.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.119105.log 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.87115.log.gz 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.200186.log 2024-04-16T19:38:05.266 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.84484.log: 11.4% -- replaced with /var/log/ceph/c2-client.0.84484.log.gz 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.119105.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.173893.log 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.119105.log.gz 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200186.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.198922.log 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200186.log.gz 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26826.log 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.173893.log: 44.8% -- replaced with /var/log/ceph/c2-client.0.173893.log.gz 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.198922.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.121582.log 2024-04-16T19:38:05.267 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.198922.log.gz 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26826.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.200153.log 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26826.log.gz 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.22747.log 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.121582.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.121582.log.gz 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200153.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.104295.log 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200153.log.gz 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.22747.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.144870.log 2024-04-16T19:38:05.268 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.22747.log.gz 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.66414.log 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.104295.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.104295.log.gz 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.144870.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.167325.log 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr: 55.2% -- replaced with /var/log/ceph/c2-client.0.144870.log.gz 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.66414.log: 44.8% -- replaced with /var/log/ceph/c2-client.0.66414.log.gzgzip 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/c2-client.0.133600.log 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.88301.log 2024-04-16T19:38:05.269 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.167325.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.167325.log.gz 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.133600.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.108431.log 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr: 46.1% -- replaced with /var/log/ceph/c2-client.0.133600.log.gz 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.88301.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.37357.log 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.88301.log.gz 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2.audit.log 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.108431.log: 68.7% -- replaced with /var/log/ceph/c2-client.0.108431.log.gz 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.37357.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26814.log 2024-04-16T19:38:05.270 INFO:teuthology.orchestra.run.smithi122.stderr: 46.1% -- replaced with /var/log/ceph/c2-client.0.37357.log.gz 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2.audit.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.181013.log 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26814.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.176265.log 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26814.log.gz 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/c2-client.0.181013.log: -5 --verbose -- /var/log/ceph/c2-client.0.200257.log 89.6% 47.4% -- replaced with /var/log/ceph/c2.audit.log.gz 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.181013.log.gz 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.176265.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.176265.log.gz 2024-04-16T19:38:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.43307.log 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200257.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.111975.log 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.128694.log 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.111386.log 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.43307.log: /var/log/ceph/c2-client.0.111975.log: 7.1%/var/log/ceph/c2-client.0.128694.log: -- replaced with /var/log/ceph/c2-client.0.200257.log.gz 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.111975.log.gz 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.43307.log.gz 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.27022.log 2024-04-16T19:38:05.272 INFO:teuthology.orchestra.run.smithi122.stderr: 90.3% -- replaced with /var/log/ceph/c2-client.0.128694.log.gz 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.111386.log: 50.4% -- replaced with /var/log/ceph/c2-client.0.111386.log.gz 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.102519.log 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.113161.log 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.27022.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.27022.log.gz 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.102519.log: 57.1% -- replaced with /var/log/ceph/c2-client.0.102519.log.gz 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.41501.log 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.197736.log 2024-04-16T19:38:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.113161.log: 50.3% -- replaced with /var/log/ceph/c2-client.0.113161.log.gz 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.41501.log: gzip -5 --verbose -- /var/log/ceph/c2-mon.a.log 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.41501.log.gz 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.197736.log: gzip 46.1% -- replaced with /var/log/ceph/c2-client.0.197736.log.gz -5 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.55743.log 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.159694.log 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-mon.a.log: /var/log/ceph/c2-client.0.55743.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.56922.log 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr: 50.4% -- replaced with /var/log/ceph/c2-client.0.55743.log.gz 2024-04-16T19:38:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.159694.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.73485.log 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.159694.log.gz 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.56922.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.162042.log 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.56922.log.gz 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.73485.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.138940.log 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.73485.log.gz 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.162042.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.162042.log.gz 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.138347.log 2024-04-16T19:38:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.157346.log 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.138940.log: 44.8% -- replaced with /var/log/ceph/c2-client.0.138940.log.gz 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.138347.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.27096.log 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.138347.log.gz 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.157346.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.153822.log 0.0% 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.157346.log.gz 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.27145.log 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.27096.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.27096.log.gz 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.153822.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.50417.log 2024-04-16T19:38:05.276 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.153822.log.gz 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.27145.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.93045.log 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr: 84.1% -- replaced with /var/log/ceph/c2-client.0.27145.log.gz 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.50417.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.157933.log 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr: 49.9% -- replaced with /var/log/ceph/c2-client.0.50417.log.gz 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.93045.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.37950.log 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.93045.log.gz 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.157933.log: gzip 0.0% -5 -- replaced with /var/log/ceph/c2-client.0.157933.log.gz --verbose 2024-04-16T19:38:05.277 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/c2-client.0.80728.log 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.161455.log 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.37950.log: 43.5% -- replaced with /var/log/ceph/c2-client.0.37950.log.gz 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.80728.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.43900.log 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.80728.log.gz 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.161455.log: 0.0%gzip -- replaced with /var/log/ceph/c2-client.0.161455.log.gz -5 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.69963.log 2024-04-16T19:38:05.278 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.193585.log 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.43900.log: 50.7% -- replaced with /var/log/ceph/c2-client.0.43900.log.gz 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.69963.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.179230.log 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.69963.log.gz 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.193585.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.149713.log 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.193585.log.gz 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.67008.log 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.179230.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.179230.log.gz 2024-04-16T19:38:05.279 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.149713.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.126326.log 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.149713.log.gz 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.67008.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.48645.log 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr: 46.1% -- replaced with /var/log/ceph/c2-client.0.67008.log.gz 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.164390.log 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.126326.log: 44.8% -- replaced with /var/log/ceph/c2-client.0.126326.log.gz 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.48645.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.141312.log 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.48645.log.gz 2024-04-16T19:38:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.164390.log: gzip 0.0% -- replaced with /var/log/ceph/c2-client.0.164390.log.gz -5 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/c2-client.0.42714.log 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.77115.log 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.141312.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.141312.log.gz 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.42714.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.166151.log 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.42714.log.gz 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.77115.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.27072.log 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.77115.log.gz 2024-04-16T19:38:05.281 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.166151.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.34392.log 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.166151.log.gz 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.27072.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.27072.log.gz 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.181606.log 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.34392.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.27743.log 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.34392.log.gz 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.181606.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.32611.log 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.181606.log.gz 2024-04-16T19:38:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.27743.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.163216.log 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr: 43.9% -- replaced with /var/log/ceph/c2-client.0.27743.log.gz 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.32611.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.200301.log 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.32611.log.gz 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.163216.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.188248.log 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.163216.log.gz 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.200301.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.58108.log 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.200301.log.gz 2024-04-16T19:38:05.283 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.188248.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26655.log 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.188248.log.gz 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.58108.log: gzip -5 --verbose -- /var/log/ceph/c2-osd.2.log 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.58108.log.gz 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26655.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.55150.log 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26655.log.gz 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.146686.log 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.55150.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.68195.log 2024-04-16T19:38:05.284 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.55150.log.gz 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.146686.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.191806.log 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr: 94.2% -- replaced with /var/log/ceph/c2-client.0.146686.log.gz 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.134786.log 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.68195.log: /var/log/ceph/c2-client.0.191806.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.51599.log 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.68195.log.gz 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.191806.log.gz 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.134786.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.200286.log 2024-04-16T19:38:05.285 INFO:teuthology.orchestra.run.smithi122.stderr: 48.7% -- replaced with /var/log/ceph/c2-client.0.134786.log.gz 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.51599.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.29549.log 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr: 60.7% -- replaced with /var/log/ceph/c2-client.0.51599.log.gz 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200286.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.100749.log 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200286.log.gz 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.29549.log: 0.0%gzip -5 --verbose -- /var/log/ceph/c2-client.0.158520.log 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.29549.log.gz 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.100749.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.180420.log 2024-04-16T19:38:05.286 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.100749.log.gz 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26668.log 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.158520.log: /var/log/ceph/c2-client.0.180420.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.148513.log 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.158520.log.gz 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr: 44.8% -- replaced with /var/log/ceph/c2-client.0.180420.log.gz/var/log/ceph/c2-client.admin.26668.log: 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/c2-client.0.200108.log 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.admin.26668.log.gz 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.148513.log: 94.5% -- replaced with /var/log/ceph/c2-client.0.148513.log.gz 2024-04-16T19:38:05.287 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.48048.log 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.200108.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.106656.log 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr: 8.2% -- replaced with /var/log/ceph/c2-client.0.200108.log.gz 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.48048.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26850.log 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr: 78.3% -- replaced with /var/log/ceph/c2-client.0.48048.log.gz 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.106656.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.53971.log 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr: 56.8% -- replaced with /var/log/ceph/c2-client.0.106656.log.gz 2024-04-16T19:38:05.288 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.44496.log 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26850.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26850.log.gz 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.53971.log: gzip -5 --verbose -- /var/log/ceph/c2-osd.0.log 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr: 63.7% -- replaced with /var/log/ceph/c2-client.0.53971.log.gzgzip -5 --verbose -- /var/log/ceph/c2-client.0.53378.log 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.44496.log: /var/log/ceph/c2-osd.0.log: 73.3% -- replaced with /var/log/ceph/c2-client.0.44496.log.gz 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.196550.log 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.53378.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.173304.log 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.53378.log.gz 2024-04-16T19:38:05.289 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.196550.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.196550.log.gz 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.123361.log 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.81990.log 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.173304.log: 66.3% -- replaced with /var/log/ceph/c2-client.0.173304.log.gz 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.123361.log: 45.5%gzip -5 --verbose -- /var/log/ceph/c2-client.0.190027.log 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.123361.log.gz 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.81990.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.81990.log.gz 2024-04-16T19:38:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.134193.log 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.125140.log 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.190027.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.190027.log.gz 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.134193.log: 46.1%gzip -5 --verbose -- /var/log/ceph/c2-client.0.188841.log 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.134193.log.gz 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.125140.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.125140.log.gz 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.186350.log 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.40907.log 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.188841.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.188841.log.gz 2024-04-16T19:38:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.186350.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.159107.log 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr: 40.9% -- replaced with /var/log/ceph/c2-client.0.186350.log.gz 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.40907.log: 70.8% -- replaced with /var/log/ceph/c2-client.0.40907.log.gz 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.123954.log 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.106067.log 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.159107.log: /var/log/ceph/c2-client.0.123954.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.159107.log.gz 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr: 47.4% -- replaced with /var/log/ceph/c2-client.0.123954.log.gz 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.104887.log 2024-04-16T19:38:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.194771.log 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.106067.log: /var/log/ceph/c2-client.0.104887.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.104887.log.gz 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr: 57.4% -- replaced with /var/log/ceph/c2-client.0.106067.log.gz 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26642.log 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.49237.log 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.194771.log: 47.4% -- replaced with /var/log/ceph/c2-client.0.194771.log.gz 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26642.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26642.log.gzgzip -5 --verbose -- /var/log/ceph/c2-client.0.146078.log 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.49237.log: 95.2% -- replaced with /var/log/ceph/ops-log-c2-client.0.log.gz 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.49237.log.gz 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.78895.log 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.146078.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26426.log 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr: 90.1% -- replaced with /var/log/ceph/c2-client.0.146078.log.gz 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.78895.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.78895.log.gz 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.62261.log 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.107245.log 2024-04-16T19:38:05.294 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26426.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26426.log.gz 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.62261.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.142498.log 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr: 45.5% -- replaced with /var/log/ceph/c2-client.0.62261.log.gz 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.87708.log 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.107245.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.26321.log 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.142498.log: 47.4% 47.4% -- replaced with /var/log/ceph/c2-client.0.107245.log.gz 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/c2-client.0.142498.log.gz 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.87708.log: 45.5% -- replaced with /var/log/ceph/c2-client.0.87708.log.gz 2024-04-16T19:38:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.0.80144.log 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.26321.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.64040.log 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.26321.log.gz 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.80144.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.160281.log 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.80144.log.gz 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.64040.log: 48.7% -- replaced with /var/log/ceph/c2-client.0.64040.log.gz 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/c2-client.admin.30160.log 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.160281.log: gzip -5 --verbose -- /var/log/ceph/c2-client.0.163803.log 2024-04-16T19:38:05.296 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.0.160281.log.gz 2024-04-16T19:38:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.30160.log: gzip -5 --verbose -- /var/log/ceph/c2-client.admin.30134.log 2024-04-16T19:38:05.297 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/c2-client.admin.30160.log.gz 2024-04-16T19:38:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.0.163803.log: 0.0% -- replaced with /var/log/ceph/c2-client.0.163803.log.gz 2024-04-16T19:38:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/c2-client.admin.30134.log: 0.0% -- replaced with /var/log/ceph/c2-client.admin.30134.log.gz 2024-04-16T19:38:05.368 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/c2-mgr.x.log.gz 2024-04-16T19:38:05.515 INFO:teuthology.orchestra.run.smithi083.stderr: 91.7% -- replaced with /var/log/ceph/c1-mgr.x.log.gz 2024-04-16T19:38:06.260 INFO:teuthology.orchestra.run.smithi122.stderr: 91.6% -- replaced with /var/log/ceph/c2-mon.a.log.gz 2024-04-16T19:38:07.750 INFO:teuthology.orchestra.run.smithi122.stderr: 93.9% -- replaced with /var/log/ceph/rgw.c2.client.0.log.gz 2024-04-16T19:38:08.295 INFO:teuthology.orchestra.run.smithi071.stderr: 91.6% -- replaced with /var/log/ceph/c1-mon.a.log.gz 2024-04-16T19:38:11.239 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/rgw.c1.client.0.log.gz 2024-04-16T19:38:12.040 INFO:teuthology.orchestra.run.smithi122.stderr: 93.4% -- replaced with /var/log/ceph/c2-osd.1.log.gz 2024-04-16T19:38:12.079 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/c2-osd.2.log.gz 2024-04-16T19:38:12.524 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/c2-osd.0.log.gz 2024-04-16T19:38:12.527 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T19:38:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m7.611s 2024-04-16T19:38:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m24.692s 2024-04-16T19:38:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.973s 2024-04-16T19:38:13.568 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/c2-osd.5.log.gz 2024-04-16T19:38:13.698 INFO:teuthology.orchestra.run.smithi188.stderr: 93.5% -- replaced with /var/log/ceph/c2-osd.3.log.gz 2024-04-16T19:38:13.815 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/c2-osd.4.log.gz 2024-04-16T19:38:13.817 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-04-16T19:38:13.817 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m8.864s 2024-04-16T19:38:13.817 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m25.315s 2024-04-16T19:38:13.817 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.916s 2024-04-16T19:38:16.042 INFO:teuthology.orchestra.run.smithi071.stderr: 93.3% -- replaced with /var/log/ceph/c1-osd.0.log.gz 2024-04-16T19:38:16.363 INFO:teuthology.orchestra.run.smithi071.stderr: 93.2% -- replaced with /var/log/ceph/c1-osd.1.log.gz 2024-04-16T19:38:20.167 INFO:teuthology.orchestra.run.smithi071.stderr: 93.5% -- replaced with /var/log/ceph/c1-osd.2.log.gz 2024-04-16T19:38:20.168 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T19:38:20.168 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m15.261s 2024-04-16T19:38:20.169 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m45.921s 2024-04-16T19:38:20.169 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m1.612s 2024-04-16T19:38:21.687 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/c1-osd.5.log.gz 2024-04-16T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr: 93.3% -- replaced with /var/log/ceph/c1-osd.3.log.gz 2024-04-16T19:38:23.962 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/c1-osd.4.log.gz 2024-04-16T19:38:23.963 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-16T19:38:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m19.053s 2024-04-16T19:38:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m51.650s 2024-04-16T19:38:23.964 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m1.738s 2024-04-16T19:38:23.965 INFO:tasks.ceph:Archiving logs... 2024-04-16T19:38:23.965 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi071/log 2024-04-16T19:38:23.967 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T19:38:29.582 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi083/log 2024-04-16T19:38:29.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T19:38:35.985 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi122/log 2024-04-16T19:38:35.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T19:38:39.409 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi188/log 2024-04-16T19:38:39.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T19:38:42.547 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T19:38:42.557 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 556, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 364, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 269, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' 2024-04-16T19:38:42.560 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T19:38:42.561 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T19:38:42.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T19:38:42.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T19:38:42.568 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T19:38:42.680 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T19:38:42.689 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T19:38:42.690 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T19:38:42.692 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T19:38:42.695 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T19:38:42.697 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T19:38:42.708 INFO:teuthology.orchestra.run.smithi071.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T19:38:42.708 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================== 2024-04-16T19:38:42.708 INFO:teuthology.orchestra.run.smithi071.stdout:+hv01.front.sepi 67.205.162.81 3 u 113 128 377 0.110 -2.347 0.241 2024-04-16T19:38:42.709 INFO:teuthology.orchestra.run.smithi071.stdout:+hv02.front.sepi 63.231.80.2 3 u 77 128 377 0.104 +1.627 0.166 2024-04-16T19:38:42.709 INFO:teuthology.orchestra.run.smithi071.stdout:*hv03.front.sepi 74.6.168.72 3 u 75 128 377 0.123 +0.388 0.092 2024-04-16T19:38:42.709 INFO:teuthology.orchestra.run.smithi071.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-16T19:38:42.711 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T19:38:42.712 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-04-16T19:38:42.712 INFO:teuthology.orchestra.run.smithi083.stdout:+hv01.front.sepi 67.205.162.81 3 u 6 256 377 0.121 -3.257 0.141 2024-04-16T19:38:42.712 INFO:teuthology.orchestra.run.smithi083.stdout:*hv02.front.sepi 63.231.80.2 3 u 2 256 377 0.097 +0.611 0.238 2024-04-16T19:38:42.712 INFO:teuthology.orchestra.run.smithi083.stdout:+hv03.front.sepi 74.6.168.72 3 u 5 256 377 0.114 -0.135 0.127 2024-04-16T19:38:42.712 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-16T19:38:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T19:38:42.713 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-04-16T19:38:42.713 INFO:teuthology.orchestra.run.smithi122.stdout:+hv01.front.sepi 67.205.162.81 3 u 22 256 377 0.182 -3.646 0.276 2024-04-16T19:38:42.713 INFO:teuthology.orchestra.run.smithi122.stdout:+hv02.front.sepi 63.231.80.2 3 u 201 256 377 0.116 +0.327 0.225 2024-04-16T19:38:42.713 INFO:teuthology.orchestra.run.smithi122.stdout:*hv03.front.sepi 74.6.168.72 3 u 203 256 377 0.142 -1.078 0.172 2024-04-16T19:38:42.713 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-16T19:38:42.715 INFO:teuthology.orchestra.run.smithi188.stdout: remote refid st t when poll reach delay offset jitter 2024-04-16T19:38:42.715 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================== 2024-04-16T19:38:42.715 INFO:teuthology.orchestra.run.smithi188.stdout:*hv01.front.sepi 67.205.162.81 3 u 42 256 377 0.235 -5.025 0.299 2024-04-16T19:38:42.715 INFO:teuthology.orchestra.run.smithi188.stdout:+hv02.front.sepi 63.231.80.2 3 u 263 256 377 0.136 -0.828 0.171 2024-04-16T19:38:42.716 INFO:teuthology.orchestra.run.smithi188.stdout:+hv03.front.sepi 74.6.168.72 3 u 239 256 377 0.197 -1.800 0.511 2024-04-16T19:38:42.716 INFO:teuthology.orchestra.run.smithi188.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-16T19:38:42.717 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T19:38:42.728 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T19:38:42.728 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T19:38:42.738 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T19:38:42.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T19:38:42.764 INFO:teuthology.task.internal:Duration was 7315.520590 seconds 2024-04-16T19:38:42.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T19:38:42.774 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T19:38:42.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T19:38:42.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T19:38:42.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T19:38:42.781 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T19:38:42.819 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T19:38:42.819 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T19:38:42.819 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T19:38:42.868 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-16T19:38:42.869 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T19:38:42.878 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T19:38:42.879 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T19:38:42.888 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-04-16T19:38:42.889 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T19:38:42.898 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T19:38:42.898 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T19:38:42.913 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T19:38:42.920 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T19:38:42.933 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T19:38:42.966 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T19:38:42.966 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T19:38:42.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T19:38:42.988 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T19:38:43.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T19:38:43.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T19:38:43.072 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T19:38:43.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T19:38:43.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T19:38:43.099 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T19:38:43.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T19:38:43.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T19:38:43.136 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.145 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.148 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-04-16T19:38:43.154 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-16T19:38:43.157 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.167 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-04-16T19:38:43.180 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-04-16T19:38:43.468 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:38:43.474 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:38:43.480 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:38:43.486 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T19:38:43.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:38:43.514 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T19:38:43.525 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T19:38:43.526 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi071 2024-04-16T19:38:43.526 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T19:38:43.551 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi083 2024-04-16T19:38:43.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T19:38:43.574 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi122 2024-04-16T19:38:43.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T19:38:43.597 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456/remote/smithi188 2024-04-16T19:38:43.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T19:38:43.619 INFO:teuthology.task.internal:Removing archive directory... 2024-04-16T19:38:43.620 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T19:38:43.622 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T19:38:43.625 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T19:38:43.638 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T19:38:43.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T19:38:43.678 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T19:38:43.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T19:38:43.689 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T19:38:43.689 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T19:38:43.692 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T19:38:43.694 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T19:38:43.696 INFO:teuthology.orchestra.run.smithi071.stdout: 658464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 19:38 /home/ubuntu/cephtest 2024-04-16T19:38:43.697 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T19:38:43.698 INFO:teuthology.orchestra.run.smithi083.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 19:38 /home/ubuntu/cephtest 2024-04-16T19:38:43.701 INFO:teuthology.orchestra.run.smithi122.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 19:38 /home/ubuntu/cephtest 2024-04-16T19:38:43.714 INFO:teuthology.orchestra.run.smithi188.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 19:38 /home/ubuntu/cephtest 2024-04-16T19:38:43.714 INFO:teuthology.orchestra.run.smithi188.stdout: 674942 4 -rw-r--r-- 1 ceph root 42 Apr 16 17:47 /home/ubuntu/cephtest/url_file 2024-04-16T19:38:43.715 INFO:teuthology.orchestra.run.smithi188.stdout: 674945 0 srwxr-xr-x 1 root root 0 Apr 16 17:47 /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock 2024-04-16T19:38:43.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T19:38:43.716 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-16T19:38:43.716 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 556, in task yield File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 364, in create_pools yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/rgw.py", line 269, in start_rgw rgwadmin(ctx, client, cmd=['gc', 'process', '--include-all'], check_status=True) File "/home/teuthworker/src/github.com_smanjara_ceph_9e861fd85e2f3f05f3eb90540b6c1e9d55d66a7b/qa/tasks/util/rgw.py", line 34, in rgwadmin proc = remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi188 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-16T19:38:43.730 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T19:38:43.741 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T19:38:43.753 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi122 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 gc process --include-all' 2024-04-16T19:38:43.753 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T19:38:43.783 DEBUG:teuthology.nuke:shortname: smithi071 2024-04-16T19:38:43.784 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T19:38:43.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.261805', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9vtxKrRme6/isUwEX2lhl8Ybr0T6XOkffrdu3KUaVOAXLP5Y37Tmkgxpc7kst406V1WRz+VK6xeEGL3E8XHEI='} 2024-04-16T19:38:43.816 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-16T19:38:43.816 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T19:38:43.841 DEBUG:teuthology.nuke:shortname: smithi122 2024-04-16T19:38:43.841 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T19:38:43.848 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.258949', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6BfUv5py4V6x14q9+tGyc4AQrymyRKyAz6y6sP+C7i7TH6fqhkepIeAS1YXpmNgRSN28Z6rVAE8v6vXjqqcYQ='} 2024-04-16T19:38:43.866 DEBUG:teuthology.nuke:shortname: smithi188 2024-04-16T19:38:43.867 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T19:38:43.877 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.260354', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN7+QuOyR1DYPk/GC5c9CS7zlOEbjs3gZpYoCAodd4KN3mbnad1OmBxBuc9eY060s6zeyO+Ul/4sy25LGzWxW6s='} 2024-04-16T19:38:43.885 INFO:teuthology.orchestra.console.smithi071:Power off 2024-04-16T19:38:43.886 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T19:38:43.903 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/smanjara-2024-04-16_17:28:09-rgw:multisite-wip-shilpa-multisite-tests-distro-default-smithi/7658456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 17:28:17.257390', 'locked_by': 'scheduled_smanjara@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUJ9KsHPOwr6bVTpKPWxuVw2M8nCWXz8Wb3iPZ0uBK78ORQ5r/LsER0oB/sFQ+EsO4buhZ2vscDOaYUzUJAiJM='} 2024-04-16T19:38:43.915 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-04-16T19:38:43.915 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:43.939 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-16T19:38:43.941 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T19:38:43.956 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-16T19:38:43.966 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-16T19:38:43.966 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:43.986 INFO:teuthology.orchestra.console.smithi122:Power off 2024-04-16T19:38:43.986 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T19:38:44.004 INFO:teuthology.orchestra.console.smithi188:Power off 2024-04-16T19:38:44.005 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T19:38:44.016 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-16T19:38:44.016 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-04-16T19:38:44.017 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:44.030 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-04-16T19:38:44.031 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:44.043 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-16T19:38:44.056 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-04-16T19:38:47.957 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:48.018 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:48.044 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:48.057 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:48.076 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-16T19:38:48.131 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-16T19:38:48.159 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-16T19:38:48.171 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-04-16T19:38:52.079 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:52.132 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:52.161 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:52.172 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:52.193 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-04-16T19:38:52.243 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-16T19:38:52.270 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-16T19:38:52.282 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-04-16T19:38:52.293 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-04-16T19:38:52.383 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-04-16T19:38:56.247 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:56.272 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T19:38:56.364 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-16T19:38:56.388 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-16T19:38:56.466 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-16T19:38:56.489 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-04-16T19:38:56.594 INFO:teuthology.run:Summary data: description: rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi} duration: 7315.520589828491 failure_reason: rgw multisite test failures flavor: default owner: scheduled_smanjara@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=80b86bedc2c34da6a0f9464df81b3fa2 status: fail success: false 2024-04-16T19:38:56.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T19:38:56.696 INFO:teuthology.run:FAIL